Interactions between the dorsal and the ventral pathways ... · 56 KOSHINO, CARPENTER, KELLER, AND JUST preted as reflecting the functional connectivity between cortical regions

Copyright 2005 Psychonomic Society Inc 54

Cognitive Affective amp Behavioral Neuroscience2005 5 (1) 54-66

The focus of the present study was to investigate a re-lationship between the dorsal and the ventral streams inhigh-level visual cognition We used fMRI to examinehow large-scale networks of cortical regions associatedwith spatial rotation and perceptual encoding are modu-lated by two variables the degree of mental rotation andthe complexity of the figure being rotated The maincomponents of the network to be examined include theleft and right intraparietal regions (part of the dorsalstream or the so-called where system) the left and rightinferior extrastriate (IES) and inferior temporal regions(part of the ventral or what stream see eg MishkinUngerleider amp Macko 1983 Ungerleider amp Haxby 1994)and the frontal system which has been implicated instudies of visual and spatial working memories

The theoretical perspective is based on the hypothesisthat fMRI-measured activation is a correlate of resourceconsumption in a resource-based computational archi-tecture and that more difficult tasks tend to require the

consumption of more resources (Just Carpenter amp Varma1999) In this perspective fMRI activation provides ameasure of cognitive workload As support for these hy-potheses several studies have shown that fMRI-measuredactivation systematically varies with manipulations oftask difficulty (cognitive workload) in domains as di-verse as sentence comprehension (eg Just CarpenterKeller Eddy amp Thulborn 1996 Keller Carpenter ampJust 2001) n-back tasks (eg Braver Cohen JonidesSmith amp Noll 1997 Grasby et al 1994) and mental ro-tation (eg Carpenter Just Keller Eddy amp Thulborn1999 Cohen et al 1996 Tagaris et al 1997) The generalargument is that the neural implementation of a processrequires physiological resources such as the neuronalcirculatory and glial processes as well as structural con-nectivity that ensures the coordinated communication ofthe networks that subserve the visuospatial experiencesThe fMRI activation measure is interpreted as assessingone facet of the involvement of the large-scale neuralnetworks (Mesulam 1990 1998) that correlate with cog-nitive computations

The study focuses on three cortical areas associatedwith visuospatial processing The first the where systemis a processing stream that projects from the occipitalcortex to the parietal region and that participates in thecomputation of extrapersonal and personal spatial local-ization (eg Mishkin et al 1983 Ungerleider amp Haxby1994) This system is thought to be involved in mentalrotation because the task requires the computation of thespatial coordinates of the object being mentally rotated

A portion of this study was reported at the 2001 Annual Meeting ofthe Psychonomic Society Orlando FL This research was supported byOffice of Naval Research Grant N00014-02-1-0037 and National Institutefor Neurological Diseases and Stroke Grant P01NS35949 The authorsthank Vladimir Cherkassky Robert Mason and Sharlene Newman forhelpful comments on the manuscript and Victor A Stenger for assistancewith the spiral fMRI pulse sequence Correspondence concerning thisarticle should be addressed to H Koshino Department of PsychologyCalifornia State University 5500 University Parkway San BernardinoCA 92407 (e-mail hkoshinocsusbedu)

Interactions between the dorsal and the ventralpathways in mental rotation An fMRI study

HIDEYA KOSHINOCarnegie Mellon University Pittsburgh Pennsylvania

and California State University San Bernardino California

and

PATRICIA A CARPENTER TIMOTHY A KELLER and MARCEL ADAM JUSTCarnegie Mellon University Pittsburgh Pennsylvania

In this fMRI study we examined the relationship between activations in the inferotemporal region(ventral pathway) and the parietal region (dorsal pathway) as well as in the prefrontal cortex (associ-ated with working memory) in a modified mental rotation task We manipulated figural complexity(simple vs complex) to affect the figure recognition process (associated with the ventral pathway)and the amount of rotation (0ordm vs 90ordm) typically associated with the dorsal pathway The pattern of ac-tivation not only showed that both streams are affected by both manipulations but also showed anoveradditive interaction The effect of figural complexity was greater for 90ordm rotation than for 0ordm in mul-tiple regions including the ventral dorsal and prefrontal regions In addition functional connectivityanalyses on the correlations across the time courses of activation between regions of interest showedincreased synchronization among multiple brain areas as task demand increased The results indicatethat both the dorsal and the ventral pathways show interactive effects of object and spatial processingand they suggest that multiple regions interact to perform mental rotation

INTERACTIONS BETWEEN DORSAL AND VENTRAL PATHWAYS 55

and its comparison with the represented coordinates ofthe target object A graded fMRI study of mental rota-tion in which the Shepard and Metzler (1971) task wasused showed that increases in the amount of rotationfrom 0ordm to 120ordm are associated with linearly increasingamounts of activation in the left and right intraparietalsulcus (IPS Carpenter et al 1999) This result was ob-tained even when the self-paced trial times were equatedto the duration of the 0ordm condition indicating that the re-sult is not simply due to time on task Also a controlcondition suggested that eye movements per se contributedminimally to the result the control involving scanningbetween two grids so that the magnitude and number ofeye movements exceeded those in the highest rotationcondition Finally activation around the IPS occurs dur-ing mental rotation even when the input is auditory so itis not dependent on the processing of visual input (JustCarpenter Maguire Diwadkar amp McMains 2001)

Other neuroimaging and event-related potential (ERP)studies also implicate this region In a study involving theShepardndashMetzler rotation task fMRI-measured activationwas found in the left and right parietal regions (Brodmannareas [BAs] 7a and 7b and sometimes BA 40) when arotation condition was compared with a 0ordm rotation condi-tion (eg Cohen et al 1996 Tagaris et al 1997) A PETstudy showed activation in the left posterior-superiorparietal cortex when activation during the mental rota-tion of letters was compared with a condition requiringa simple discrimination of normal and mirror-image up-right letters (Alivisatos amp Petrides 1997) Parietal in-volvement has also been found in three ERP studies of asimpler rotation task in which participants judged whethera letter was normal or mirror-imaged (Desrocher Smithamp Taylor 1995 Peronnet amp Farah 1989 Wijers OttenFeenstra Mulder amp Mulder 1989) As the angular dis-parity increased there was increasing negativity in theERP waveforms in the latency range of 350ndash800 msecparticularly in the parietal and occipital leads which wasinterpreted as reflecting rotation Thus manipulating themental rotation workload should affect the activation inparietal regions

In addition to mental rotation the effect of figuralcomplexity which is hypothesized to affect figural en-coding was examined in this study The so-called whatsystem is associated with a stream that includes the oc-cipital region the IES regions and the inferior temporalgyrus (Ungerleider amp Mishkin 1982) Numerous func-tional neuroimaging studies of visual object recognitionhave shown activation in the inferior temporal cortex(eg Kanwisher Chun McDermott amp Ledden 1996Malach et al 1995) Also lesions in this area can cor-relate with impaired shape-based visual recognition orvisual agnosia (Farah 1990) Microelectrode studieswith macaque monkeys have also shown activation as-sociated in the homologous regions in shape-based vi-sual recognition studies (Logothetis Pauls amp Poggio1995) None of these studies suggests that these patternsof activity occur independently of activity in other corti-cal regions Indeed there is evidence that both the infe-

rior temporal and the IPS regions show increased activa-tion if the recognition of objects from line drawings ismade more difficult by deleting line contours (Diwad-kar Carpenter amp Just 2000)

A third cortical area of interest includes the frontal re-gions that are associated with maintaining and operatingon spatial and visual information over short time periodsoften referred to as spatial working memory (eg Levy ampGoldman-Rakic 2000 Miller Erikson amp Desimone1996 Owen et al 1998 Petrides 1994) Several brain-imaging studies have investigated the hypothesis that theprefrontal cortex is organized according to types of in-formation such as spatial versus verbal information (egSmith Jonides amp Koeppe 1996) and spatial versus ob-ject information (DrsquoEsposito Aguirre Zarahn amp Ballard1998 Levy amp Goldman-Rakic 2000) Other studies havebeen done to examine whether the prefrontal cortex canbe divided into functional subcomponents on the basis oftypes of operations such as maintenance versus manipu-lation (eg DrsquoEsposito Postle Ballard amp Lease 1999Petrides 1994) But none of the proposed dichotomiesgives completely dissociated activation (Carpenter Justamp Reichle 2000 DrsquoEsposito et al 1999) In the presentstudy we examined how prefrontal activation might showsensitivity to variables that impact on two types of visuo-spatial processes figural encoding and rotation

Previous theorizing about the relationship betweenfigural complexity and mental rotation has been basedlargely on behavioral studies in which it has been hy-pothesized that the figural encoding and the rotation pro-cesses are relatively independent of each other since ithad been found that rate of rotation is not affected by fig-ural complexity (Cooper 1975) We considered two pos-sible hypotheses in examining these issues with respectto brain activation One was the independent pathwayhypothesis based on Cooperrsquos conclusion If the amountof rotation and figural complexity affect processes thatare independent of each other the manipulation of stim-ulus complexity should primarily affect the activation ofthe regions in the ventral pathway and the rotation ma-nipulation should primarily affect the parietal regions inthe dorsal pathway On the other hand previous studiesin our laboratory had demonstrated that logically sepa-rable processes can show an overadditive interactionwhen combined For example in language comprehen-sion Keller et al (2001) examined the effects of lexicalfrequency and syntactic complexity processes that werehypothesized to operate on different levels which re-sulted in an overadditive interaction in numerous corti-cal regions suggesting that the two types of processesdraw on a shared infrastructure of resources Thereforethe relationship between the dorsal and the ventral path-ways may be interactive (interactive pathway hypothe-sis) and both the complexity and the rotation manipula-tions may affect both the IES and the parietal regions

We were also interested in examining the temporal as-pects of processing by measuring correlations of the ac-tivation time series between regions of interest (ROIs)within various conditions Such analyses can be inter-

56 KOSHINO CARPENTER KELLER AND JUST

preted as reflecting the functional connectivity betweencortical regions The rationale behind this type of analy-sis is that regions that work together should have similaractivation time series profiles and therefore a correla-tion coefficient between these regionsrsquo activation acrossthe time course should be high (eg Friston 1994 Fris-ton Frith Liddle amp Frackowiak 1993 McIntosh ampGonzalez-Lima 1994) Functional connectivity was in-vestigated with exploratory factor analyses (eg Peter-son et al 1999) The basic idea behind factor analyseswas that if some regions have higher correlations acrossthe time course they should be extracted as factors Inthis case the factors should represent large-scale corti-cal networks (Mesulam 1990 1998) We specificallyexamined whether the functional connectivity betweenpairs of activated regions increased as the task increasedin difficulty A previous study (Diwadkar et al 2000)showed that the functional connectivity between frontaland parietal areas in a spatial working memory task in-creased with the working memory load If functionalconnectivity increases as task difficulty increases as wasshown in Diwadkar et al one might predict that factorstructures grow larger as task difficulty increases Inother words many more brain areas might show corre-lated activity as cognitive workload increases

METHOD

ParticipantsThe participants were 11 (5 females) right-handed volunteer stu-

dents at Carnegie Mellon University whose mean age was 210years (SD 20) Each participant gave signed informed consent(approved by the University of Pittsburgh and the Carnegie MellonInstitutional Review Boards) The participants were familiarizedwith the scanner the fMRI procedure and the mental rotation taskbefore the study commenced

StimuliThe stimulus figures were the random polygons originally de-

veloped by Attneave and Arnoult (1956) and modified by Cooper(1975 Cooper amp Podgorny 1976) with examples shown in Fig-ure 1 The present stimuli were either simple (6 and 8 points) orcomplex (16 and 24 points) for two levels of complexity Variationsin the original standard stimuli had been scaled by Cooper and Pod-gorny to generate six evenly spaced deviations from a standard figure(s) indicated as d1 to d6 The levels of similarity among stimuliwere controlled by utilizing d1 d3 and d5 for the 6- and 16-pointfigures and s d2 and d4 for the 8- and 24-point figures for a totalof six different stimuli

ProcedureThe design consisted of two within-subjects factors the amount

of rotation (0ordm and 90ordm) and figural complexity (simple and com-plex figures) In each trial two stimulus figures were presented sideby side simultaneously The paradigm involved a blocked designwith five trials of a single condition per block resulting in 45-secblocks There were two repetitions of each condition in separateblocks during the first and second halves of a single scanning runlasting about 15 min The order of the conditions (including twononsimultaneous conditions that are not reported here) was ran-domized within each half of the paradigm this same order was usedfor all the participants Across the two blocks of each conditionhalf of the correct responses were same and half were differentBlocks of experimental conditions were separated by a 6-sec rest in-terval In addition there were four blocks of a 24-sec fixation con-dition during which an X was presented at the center of the displayand the participant was asked to fixate the X without doing any-thing else This condition occurred at the beginning of the experi-ment was repeated after every four blocks of experimental trialsand was used to measure baseline brain activation

Each trial began with presentation of an X to be fixated for 1 secfollowed by two stimulus figures presented side by side to the rightand left of the fixation point The stimuli remained on for 9 sec re-gardless of the participantrsquos response The participant was asked todecide as quickly as possible whether the two figures were the sameor different and was told to press the right-hand button if they werethe same and the left-hand button if they were different

The stimuli were projected onto a viewing screen attached withinthe bore of the scanner and were viewed at a distance of approxi-mately 20 cm from the participantsrsquo eyes through two mirrors po-sitioned on top of the head coil At this distance each of the figuressubtended a visual angle of approximately 10ordm Two fiber optic but-ton boxes were used by the participants to signal their responsesStimulus presentation and behavioral data collection were con-trolled with custom-written experimental presentation softwareusing Windows Workstation NT The participants were given eightblocks of five trials on the day before the scan and three practicetrials in the scanner at the beginning of the session

Imaging ParametersScanning was done in a 30T GE Medical Systems scanner (Thul-

born et al 1996) at the University of Pittsburgh Magnetic ResonanceResearch Center A T2-weighted single-shot spiral pulse sequencewith blood oxygen level dependent (BOLD see Kwong et al 1992Ogawa Lee Kay amp Tank 1990) contrast was used with the fol-lowing acquisition parameters TR 1000 msec TE 18 msecflip angle 70ordm FOV 20 20 cm matrix size 64 64 axial-oblique plane with 16 slices and a voxel size of 3125 3125 5 mm with a 1-mm gap High-resolution T1-weighted structural im-ages were acquired with a 3D SPGR volume scan with the follow-ing parameters TR 25 TE 4 flip angle 40ordm FOV 24 18 cm 124 slices resulting in voxel dimensions of 09375 09375 15 mm thick taken axially

Figure 1 Stimulus figures that were used in the mental rotation task


fMRI Data AnalysisImage preprocessing (including baseline correction mean cor-

rection motion correction and trend correction) was performedusing FIASCO (Eddy Fitzgerald Genovese Mockus amp Noll1996 Lazar Eddy Genovese amp Welling 2001 further descriptionand tools are available at wwwstatcmuedu~fiasco) The maxi-mum in-plane displacement estimate between individual imagesand a reference image did not exceed 03 voxels Data from the6-sec rest interval and from the first 6 sec of each block of trialswere discarded to accommodate the rise and fall of the hemodynamicresponse (Bandettini Wong Hinks Tikofsky amp Hyde 1992) Vox-elwise t tests were computed to compare a voxelrsquos mean signal in-tensity in each experimental condition with that in the fixation con-dition Detection of activation was accomplished by thresholdingthe resulting t maps at t 50 which is slightly more conservativethan the Bonferroni-corrected p 05 level for the approximately10000 voxels in all the ROIs we examined The main dependentmeasure was the number of voxels activated above this thresholdWe also computed the average percentage of difference in signal in-tensity (dsi) across activated voxels over the average intensity forthe baseline condition which is reported in the Appendix

To compare the volume of activation across the experimentalconditions anatomical ROIs were defined individually for each par-ticipant by adapting the parcellation scheme of Rademacher and hiscolleagues (Caviness Meyer Makris amp Kennedy 1996 RademacherGalaburda Kennedy Filipek amp Caviness 1992) as is shown inFigure 2 This method uses limiting sulci and coronal planes de-fined by anatomical landmarks to segment cortical regions Be-cause each individualrsquos cortical anatomy is different the ROIs weredrawn on the structural images of each participant to target theanatomical ROI This was done by first computing the mean func-tional image for each of the functional slices These mean images werethen registered in parallel alignment with the anterior-commissurendashposterior-commissure line to a structural volume scan of each par-ticipant The limiting sulci and other anatomical landmarks werethen located by viewing the structural images simultaneously in the

three orthogonal planes and the ROIs were defined by manuallytracing the regions onto the axial image of each structural slice andthen transferring them to the same locations in the coregisteredfunctional slices The ROIs were edited with reference to the func-tional mean image in order to correct for differences in suscepti-bility artifacts between the structural and the functional images Be-cause these distortions were severe in the inferior slices of thefrontal and temporal lobes frontal and temporal ROIs were omit-ted from the most inferior two to four slices on an individual basis

The interrater reliability of this ROI-defining procedure betweentwo trained staff members was previously evaluated for four ROIsin 2 participants in another study (Just et al 2001) The reliabilitymeasure was obtained by dividing the size of the set of voxels thatoverlapped between the two raters by the mean of their two set sizesThe resulting eight reliability measures were in the 78ndash91range with a mean of 84 as high as the reliability reported by thedevelopers of the parcellation scheme This method allows us tomeasure the modulation of the activation by the independent vari-ables in regions that are specified a priori and requires no morph-ing for definition

The ROIs were as follows the dorsolateral prefrontal cortex(DLPFC) the frontal eye field (FEF) the inferior frontal gyrus (IFG)the posterior precentral sulcus (PPREC) the superior medial frontalparacingulate (SMFP) the superior parietal lobule (SPL) the IPSthe inferior parietal lobule (IPL) the supramarginal gyrus (SGA) theinferior temporal lobule (IT) the superior extrastriate (SES) the IESthe calcarine fissure (CALC) and the occipital pole (OP)

To statistically analyze the number of activated voxels the datawere submitted to a 2 (hemisphere) 2 (amount of rotation) 2(stimulus complexity) within-subjects analysis of variance (ANOVA)for each ROI that has separate left- and right-hemisphere regionsFor each ROI that does not have the left and the right hemispheresrepresented separately (the anterior cingulate the supplementarymotor area the SMFP the CALC and the OP) the data were sub-mitted to a 2 (amount of rotation) 2 (stimulus complexity) within-subjects ANOVA

Posterior Precentral Sulcus

Superior Parietal Lobule

Intraparietal Sulcus

Inferior ParietalLobule

SuperiorExtrastriate

OccipitalPole

InferiorExtrastriate

FEF

DLPFC

Temporal

Inferior Temporal

CerebellumPars Opercularis

Pars Triangularis

Supramarginal GyrusA K D

Figure 2 A schematic diagram indicating the regions of interest (ROIs) that wereindividually parcellated for each participant in the present study The cortical par-cellation scheme was adapted from Caviness Meyer Makris and Kennedy (1996) butcollapses a number of parcellation units into single broad ROIs and identifies the re-gion in and around the intraparietal sulcus as a single ROI DLPFC dorsolateral pre-frontal cortex FEF frontal eye field


Functional ConnectivityWe computed a measure of functional connectivity the comodu-

lation or synchronization between the time courses of signal inten-sity for the activated voxels in various ROIs Briefly the processeddata were Fourier interpolated in time to correct for the interleavedslice acquisition sequence A mean signal intensity of the activatedvoxels in each ROI was then computed across the time course foreach of the experimental conditions This analysis was based onlyon observations associated with the task and did not include obser-vations associated with the rest condition The analysis was doneseparately for each ROI with three or more activated voxels for eachparticipant (see Figure 3 for an example of how the activation lev-els in two areas tend to track each other and how the magnitudes ofcorrelation coefficients do not necessarily depend on the levels ofsignal intensity) Then a correlation coefficient of signal intensityacross the time course was computed between two ROIs after thedata from the fixation condition were removed The mean correla-tions (averaged across participants) between the ROIs were com-puted after Fisherrsquos z transformation and a correlation matrix wascreated for each condition An ROI pair was included in the corre-lation matrix if both of the two ROIs for the ROI pair had three ormore mean activated voxels for the most difficult condition whichwas the complex 90ordm rotation condition Therefore the size of theoriginal correlation matrix was controlled for the four experimen-tal conditions in the beginning Then an exploratory factor analysiswas performed (eg Peterson et al 1999) for each condition toexamine which ROIs were grouped with each other Our assump-tion behind the factor analyses was that each factor represents alarge-scale network among brain regions that are correlated or de-scribed as corresponding to some functions (eg Mesulam 19901998) Factors were extracted with the principal component analy-sis and were rotated with the varimax method Factors that hadeigenvalues of 10 or above were retained (an eigenvalue corre-sponds to the equivalent number of ROIs that the factor represents)and ROIs that had factor loadings of 4 or greater were taken into

consideration in interpretation (factor loadings represent the degreeto which each of the ROIs correlates with each of the factors) Ac-cording to this rule the original factor analysis extracted eight fac-tors for the simple 0ordm four factors for the complex 0ordm four factorsfor the simple 90ordm and four factors for the complex 90ordm conditionsWe further excluded factors consisting of two ROIs because wewere interested in large-scale networks This resulted in three fac-tors for the simple 0ordm three factors for the complex 0ordm and threefactors for the complex 90ordm conditions The number of factors forthe simple 90ordm condition remained four

RESULTS

Behavioral DataMean reaction times (RTs) and error rates are shown

in Table 1 RTs were longer for 90ordm of rotation than for0ordm [F(110) 7948 p 001] and longer for the complexfigures than for the simple figures [F(110) 4921 p 001] In addition the effect of stimulus complexity wasgreater for the 90ordm of rotation condition than for 0ordm re-sulting in a significant interaction between the amount ofrotation and complexity [F(110) 56 p 05] Theparticipants also made more errors for 90ordm of rotationthan for 0ordm [F(110) 2012 p 01] However therewas no effect of stimulus complexity [F(110) 003ns] and no interaction [F(110) 001 ns] for theerror rates

fMRI DataThe analysis focused on whether the amount of activa-

tion is additive or interactive The results showed overad-ditive interactions between object complexity and mental

Figure 3 An example of a correlated activation time course in two regions of interest (ROIs) from 1 participant In this fig-ure the percentage of signal increase (dsi) of the right inferior extrastriate (RIES) and the right inferior temporal (RIT) for(A) the complex 0ordm rotation condition [r(96) 029 p 05] and (B) the complex 90ordm rotation condition [r(96) 060 p 001]were plotted against image number Even though the mean dsi is the same for the two conditions (mean dsi is 197 for RIES and179 for RIT for complex 0ordm rotation and mean dsi is 198 for RIES and 180 for RIT for complex 90ordm rotation) the correlationcoefficient was much higher for the complex 90ordm rotation condition These results indicate that the correlation does not neces-sarily depend on the amount of activation The vertical line marks the end of the first block The images corresponding to thefixation condition were excluded from the computation of the correlation coefficient


rotation in many ROIs The 90ordm mental rotation producedmore brain activation than the 0ordm condition did and thiseffect of rotation was greater for the complex figuresthan for the simple figures The number of activated vox-els in each ROI is shown in Table 2 with the Talairach co-ordinates (Talairach amp Tournoux 1988) of the mean cen-troid averaged across the four experimental conditionsand the results of the ANOVA are given in Table 3 Ex-amples of ROIs showing the overadditive interaction aregiven in Figures 4 and 5 As is shown in Figure 5 theoveradditive interaction was significant [F(110) 1494p 01 for the DLPFC F(110) 782 p 05 for theIPS and F(110) 1147 p 01 for the IES]

A second important aspect of the result is that thisoveradditive interaction was present in a majority of theROIs It was present in all the regions of primary inter-est the IPS the inferior temporal and occipital regionsand the frontal areas (see Table 3 for statistics) Howeverthe patterns of the overadditive interactions were not thesame across these ROIs As Table 2 shows for the frontaltemporal and occipital regions the effect of the amountof rotation was observed only for the complex stimulibut not for the simple stimuli However for the parietalregions activation increased from 0ordm to 90ordm of rotation

even for the simple stimuli but increased much more forthe complex stimuli As can be seen in Table 3 there wasno three-way interaction Very similar results were foundon the second dependent measure the average percent-age of dsi shown in the Appendix

Although most ROIs showed bilateral activation forsome manipulations there was more activation in onehemisphere often in the right hemisphere The effect ofthe amount of rotation was greater for the right hemi-sphere than for the left in the DLPFC [F(110) 999p 05] and the IFG [F(110) 986 p 05] The ef-fect of figural complexity was greater for the right hemi-sphere than for the left in the IFG [F(110) 60 p 05]The brain activation was bilateral in the parietal regionsIn the occipital region the complexity effect was greaterfor the left hemisphere than for the right in the SES[F(110) 884 p 05] There was no significant three-way interaction

One might wonder however whether the results areconfounded with duty cycle issues in other words couldthe higher activation for the more difficult condition bedue to the longer RT To examine these issues we ana-lyzed the pattern of activation using only the first 3 secof each trial with the same threshold and ANOVA designas in the original analysis The results of the analysisshow substantially the same patterns of significant re-sults in the major ROIs such as the DLPFC the IPS andthe IES as those reported for the main analysis There-fore it does not seem that the duty cycle issue had mucheffect on the results

The pattern of results was also robust across a rangeof thresholds We analyzed the data with both a lowerthreshold (t 45) and a higher threshold (t 55) todetermine whether the pattern of activation is specific to

Table 1Mean Reaction Times (RTs in Milliseconds)

and Error Rates (in Percentages)

Simple Complex

Amount of RT Error Rate RT Error Rate

Rotation M SD M SD M SD M SD

0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186

Table 2Mean Activated Voxels for Conditions and Their Mean Centroids for Each Region of Interest

Mean Activated Voxels Mean Centroids

Left Hemisphere Right Hemisphere Left Hemisphere Right Hemisphere

Region S0ordm C0ordm S90ordm C90ordm S0ordm C0ordm S90ordm C90ordm x y z x y z

FrontalDorsolateral prefrontal cortex 58 43 40 86 78 60 71 176 34 35 29 34 28 36Frontal eye field 12 12 16 42 37 50 54 79 44 1 42 43 2 40Inferior frontal gyrus 16 13 13 26 50 46 62 102 44 17 20 43 12 24Posterior precentral sulcus 32 33 52 85 25 26 29 72 38 6 47 38 7 51Supplementary motor area 07 05 06 15 3 12 65Superior medial frontal paracingulate 35 28 22 66 5 15 56

Parietal Inferior parietal lobule 35 44 56 96 36 37 56 76 40 53 42 40 58 36Intraparietal sulcus 123 157 220 329 161 212 292 418 26 62 44 31 62 42Superior parietal lobule 65 87 132 207 74 108 126 195 16 64 54 23 62 51

TemporalInferior temporal lobule 61 55 91 132 137 136 161 226 43 64 1 42 64 3

OccipitalInferior extrastriate 127 107 151 246 140 170 169 263 28 75 3 25 76 4Superior extrastriate 50 61 65 136 71 75 90 126 22 83 25 24 81 23Calcarine fissure 116 87 95 132 5 77 8Occipital pole 161 166 188 236 2 92 8

NotemdashMean centroids were computed across four conditions The supplementary motor area the superior medial frontal paracingulate the calcarinefissure and the occipital pole are medial ROIs and are not separated between the left and the right hemispheres S simple C complex


Table 3Results of Analyses of Variance on the Mean Activated Voxels (Rotation Amount [2] Complexity [2] Hemisphere [2])

Region Amount Complexity Hemisphere AC AH CH ACH

FrontalDorsolateral prefrontal cortex 1474 348 260 1494 999 394 210Frontal eye field 602 383 573 741 034 087 071Inferior frontal gyrus 341 679 311 859 986 600 426Posterior precentral sulcus 516 459dagger 053 703 162 050 027Supplementary motor area 212 158 NA 238 NA NA NASuperior medial frontal paracingulate 266 486dagger NA 550 NA NA NA

ParietalInferior parietal lobule 723 893 033 471dagger 087 138 027Intraparietal sulcus 2518 1261 376 782 327 255 000Superior parietal lobule 1457 821 001 434 162 010 126

TemporalInferior temporal lobule 1403 1026 838 672 007 148 030

OccipitalInferior extrastriate 1225 1091 112 1147 101 168 142Superior extrastriate 692 587 152 1159 099 884 081Calcarine fissure 073 005 NA 872 NA NA NAOccipital pole 867 252 NA 121 NA NA NA

Notemdashdf (110) for all comparisons dagger05 p 07 p 05 p 01 p 001

Figure 4 T-maps that were transformed to a standardized space (Talairach amp Tournoux 1988) and aver-aged across participants using MCW-AFNI software (Cox 1996) for the four experimental conditions com-pared with the resting baseline For the dorsolateral prefrontal cortex (DLPFC x 0 y 35 z 28) and theinferior extrastriate area (IES x 0 y 35 z 5) the activation levels were about the same for the simple0ordm the complex 0ordm and the simple 90ordm rotation conditions but much greater for the complex 90ordm conditionshowing overadditive interaction For the intraparietal sulcus (IPS x 0 y 35 z 43) the overadditive inter-action is apparent in addition to an effect of rotation (greater activation for 90ordm rotation than for 0ordm)

Dorsolateral Prefrontal Cortex (DLPFC)

Intraparietal Sulcus (IPS)

Inferior Extrastriate (IES)

Simple 0ordm Complex 0ordm Simple 90ordm Complex 90ordm


the particular threshold we used (t 50) Using thesame ANOVA design we found that exactly the samemain effects and interactions were statistically signifi-cant as in the original analysis for the main ROIs suchas the DLPFC the IPS and the IES Thus the pattern ofresults was not specific to the particular threshold wechose Also as is shown in the Appendix the averagepercentage of dsi showed a very similar pattern of results

Functional ConnectivityThe functional connectivity analysis examined which

regions showed similar temporal patterns of activity Weperformed a separate factor analysis for each conditionand the results showed two systematic patterns (see Fig-ure 6) First there were three main common factors (orclusters of regions) across conditionsmdashnamely factorscorresponding to executive processing spatial process-ing and lower level visual processing For example thefactors in the simple 0ordm condition are shown in the firstthree columns of Table 4 labeled F1 F2 and F3 The firstnetwork corresponding to F1 in this condition consistsprimarily of frontal areas and we refer to it as a centralexecutive network The second network correspondingto F2 in this condition is composed basically of the pari-etal areas and we refer to it as a spatial network Thethird network corresponding to F3 in this condition in-cludes mainly the occipital areas and the inferior tempo-ral area and we refer to this network as a lower visualnetwork Similar networks emerge in the remaining con-ditions even though their communality values are dif-ferent across the conditions The significance of this out-come is that it shows that similar networks of ROIs thatare typically associated with a particular facet of pro-cessing are extracted across conditions even with rota-tional indeterminacy of factor analyses This seems toprovide a way of tentatively identifying a cortical net-work on the basis of fMRI functional connectivities

The second important point that emerged from thefactor analysis is that the number of ROIs for each fac-

tor increased forming a larger network as task difficultyincreased For example in the simple 0ordm condition thesethree networks are fairly distinct from each other withoutmuch overlap between the factors On the other hand inthe complex 90ordm rotation condition the first factor is thelargest of all including the frontal parietal temporal andoccipital areas The third factor also includes the parietaland occipital ROIs covering the dorsal and ventral path-ways The integration is also supported by an increasingnumber of ROIs from the simple 0ordm condition in which themean number of ROIs per factor is 73 to the complex 90ordmcondition in which the mean number of ROIs is 117

DISCUSSION

One of the main goals of this study was to investigatethe effect of figural complexity and amount of rotationon activation in the dorsal and ventral pathways in themental rotation task If object processing and mental ro-tation are independent of each other the manipulation offigural complexity should have primarily affected theventral what system and the manipulation of amount ofrotation should have primarily affected the dorsal wheresystem without an interaction between the two factorsHowever the results showed an overadditive interactionin which the effect of rotation was greater for complexfigures than for simple figures Moreover this overaddi-tive interaction was found across the brain from the frontalto the occipital lobe Therefore our results indicate thatboth the dorsal and the ventral systems respond to themanipulations of figural complexity and mental rotationdisconfirming the independent pathway hypothesis Theresults might reflect that both regions are involved inboth types of processing or that an increase in task de-mand of one type of processing also increases the re-source demand of the other type of processing

The results also suggest that some regions are moreinvolved in certain processes than are others reflectingthe relative specialization of cortical regions As Table 2

Figure 5 Mean activated voxels plotted against the amount of rotation for each hemisphere for selected regions of in-terest DLPFC dorsolateral prefrontal cortex IPS intraparietal sulcus IES inferior extrastriate


Figure 6 The results of factor analyses The factors are rearranged to show the correspondence across the con-ditions instead of the order of communality values which are shown at the bottom of each factor in Table 4 Forthe simple 0ordm complex 0ordm and complex 90ordm rotation conditions Factor 1 is the central executive network Factor2 corresponds to the spatial processing or spatial working memory network and Factor 3 reflects the lower vi-sual network For the simple 90ordm condition the factor structure was slightly different because four factors wereextracted In this condition Factor 1 is the central executive network Factor 2 corresponds to spatial workingmemory Factor 3 reflects the spatial processing network and Factor 4 corresponds to the lower visual network

DLPFC DLPFC

FEF FEFInferior InferiorFrontal Gyrus Frontal Gyrus

Posterior PosteriorPrecentral Sulcus Precentral Sulcus

Superior MedialSuperior Parietal Frontal Superior Parietal

Paracingulate

Intraparietal Sulcus Intraparietal Sulcus

Inferior Parietal Inferior Parietal

Inferior InferiorTemporal Temporal

Superior SuperiorExtrastriate Extrastriate

Inferior InferiorExtrastriate Calcarine Extrastriate

OccipitalPole

DLPFC DLPFC




ParacingulateIntraparietal Sulcus Intraparietal Sulcus





OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF

Inferior InferiorFrontal Gyrus Frontal Gyrus








OccipitalPole

Simple 0ordm Complex 0ordm


L R L R

L R L R

Factor 1 Central executive networkFactor 2 Spatial networkFactor 3 Lower visual network

Factor 1 Central executive networkFactor 2 Spatial and object working memory networkFactor 3 Lower visual network

Factor 1 Central executive networkFactor 2 Spatial working memory networkFactor 3 Spatial processing networkFactor 4 Lower visual network

Factor 1 Central executive networkFactor 2 Integrated working memory networkFactor 3 Visual and spatial network

shows for the frontal temporal and occipital regionsthe effect of the amount of rotation was observed onlyfor the complex stimuli but not for the simple stimuli

However for the parietal regions activation increasedfrom 0ordm to 90ordm of rotation even for the simple stimuli al-though much more so for the complex stimuli


As was described in the introduction similar overad-ditive interactions in brain activation have been found inseveral other domains For example when both lexicaland syntactic processing in the comprehension of writ-ten sentences was made more demanding there was a re-liable overadditive interaction in multiple cortical loci(Keller et al 2001) Similarly when both sentence struc-ture and language (native vs second language) weremore difficult in the comprehension of spoken sentencesthere was a reliable overadditive interaction in multiplecortical loci (Hasegawa Carpenter amp Just 2002) There-fore this type of overadditive interaction seems to be arather general property of the brain function

A difference between Cooperrsquos (1975) results and ourfindings points out the useful role of resource consider-ations Cooper found that figural complexity did not in-teract with (or even affect) the rate of rotation whereasin our study we found an overadditive interaction in the

RT data One possible source of this difference was thedifference in the participantsrsquo familiarity with the stim-ulus figures Each of Cooperrsquos participants was practicedfor two 1-h sessions and then was tested in four 1-h ses-sions On the other hand our participants were given apractice session of approximately 6 min on the day be-fore the scan and then were tested in the scanner in a15-min testing session The extended amount of practicethat Cooperrsquos participants enjoyed probably made figuralencoding and mental rotation more automatic and hencemuch less resource demanding as indicated by severalmeasures The error rates were lower than those in ourstudy the RTs were shorter the slopes of the RTs plot-ted against the amount of rotation were shallower andthere was no effect of figural complexity Since both fig-ural encoding and mental rotation were less resource de-manding when these processes were combined even fora large rotation angle the overall resource requirement

Table 4Results of Factor Analysis


Region F1 F2 F3 F1 F2 F3 F1 F2 F3 F4 F1 F2 F3

FrontalDLPFC (L) 75 62 53 81DLPFC (R) 74 50 49 30 61 41 72FEF (L) 88 57 55FEF (R) 46 48 50 54 60 54 57 54IFG (R) 58 69 30 54 45 31PPREC (L) 34 78 84 36 58 38PPREC (R) 84 40 78 49 63 32 48 55 34SMFP 77 30 73 37 82 32 35 42

ParietalSPL (L) 65 83 38 59 40 37 56 49SPL (R) 55 40 73 51 43 50 37 41 30 60IPS (L) 31 54 30 81 31 36 57 56 37 56 50IPS (R) 30 60 32 73 33 53 61 38 56 57IPL (L) 46 49 75 62 50 55 38IPL (R) 37 57 33 63 31 45 46SGA (L) 77 67 36 55 51 64SGA (R) 33 83 78

TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60

OccipitalSES (L) 64 48 51 50 51 33 65SES (R) 33 66 52 54 46 61 30 73IES (L) 55 34 60 30 66 38 66IES (R) 70 36 67 35 61 34 71CALC 79 79 85 79OP 78 31 80 71 77

Communality 357 345 424 268 641 416 349 497 407 391 393 506 596

NotemdashBoldface type indicates the factor loading values that are greater than or equal to 40 and are included in interpretation of the factor andRoman type indicates the factor loading values that are greater than or equal to 30 but are not included in the factor We reorganized the order offactors in the table to make the comparison across conditions easier therefore the factor order does not correspond to the order of communalityFor the simple 0ordm condition Factor 1 (F1) was the central executive network Factor 2 (F2) was the spatial network and Factor 3 (F3) was the lowervisual network For the complex 0ordm condition F1 was the central executive network F2 was the spatial and object working memory network andF3 was the lower visual network For the simple 90ordm condition F1 was the central executive network F2 was the spatial working memory networkF3 was the spatial processing network and F4 was the lower visual network For the complex 90ordm condition F1 was the central executive networkF2 was the integrated working memory network and F3 was the visual and spatial network DLPFC dorsolateral prefrontal cortex FEF frontaleye field IFG inferior frontal gyrus PPREC posterior precentral sulcus SMFP superior medial frontal paracingulate SPL superior parietal lobeIPS intraparietal sulcus IPL inferior parietal lobule SGA supramarginal gyrus IT inferior temporal lobule SES superior extrastriate IES in-ferior extrastriate CALC calcarine fissure OP occipital pole L left R right


may still have been below some capacity limitation Onthe other hand in our study the error rates were higher theRTs were longer and the slopes were steeper indicatinga higher level of resource demand The overadditive re-lation between figural complexity and mental rotationmight arise because the combination of these processesexceeded the capacity limitation at the higher level of de-mand (namely the complex figures requiring 90ordm rota-tion) There might be some adaptive mechanisms fordealing with such a shortage of resources such as trad-ing away processing time for total resource consumption(consuming resources at the maximal rate and extendingthe consumption over a longer period to compensate forthe maximumrsquos not being sufficient) or segmenting thetask processing into sequential components that wouldotherwise be executed in parallel Such mechanismsmight be manifested in the brain activation as a higherlevel of activation for the complex figures rotated 90ordmbecause of extended cumulated resource consumptionandor additional processing steps

Another contribution of the present study is a factoranalysis of functional connectivity Three common fac-tor structures or large-scale cortical networks (Mesu-lam 1990 1998) tended to appear in all the conditions(1) an executive control network consisting mainly ofthe frontal areas (2) a spatial information processingnetwork consisting primarily of the parietal regions and(3) a lower level visual network for object recognitionconsisting mainly of the occipital regions and sometimesthe inferior temporal regions For the easier conditionssuch as the simple 0ordm rotation condition the networkstended to be smaller and the three networks tended to bedistinct from one another However for the more diffi-cult conditions such as the complex 90ordm rotation thesenetworks became larger sharing more brain regions Forexample the spatial and the lower visual networks wererelatively separated in the simple 0ordm complex 0ordm andsimple 90ordm rotation conditions whereas in the complex90ordm rotation condition they were grouped together andformed a single network (F3) This indicates that syn-chronization between the dorsal and the ventral systemsincreased as task difficulty increased suggesting a pos-sible interaction between the two systems

The second point of this analysis was that as task dif-ficulty increased functional connectivity increased Aswas mentioned in the Results section the number ofROIs per factor increased as task difficulty increased in-dicating that the activities of more regions of the brainwere synchronized In other words in the more demand-ing conditions many areas across the whole brain aresynchronized to form the larger scale networks

In summary our data provide evidence against the in-dependent pathways hypothesis Both ventral and dorsalpathways were activated in response to both figural en-coding and rotation although the parietal regions showedmore sensitivity to rotation This suggests either thatboth regions are involved in both types of processing orthat increases in task demand of one type of processing

also increases the resource demand of the other type ofprocessing As cognitive load increases not only doesactivation of the corresponding brain regions increasebut also functional connectivity increases among the re-gions that form the networks Our results add to a grow-ing literature that shows interactive relationships of dif-ferent types of processing among multiple brain areasand correlations between areas in activation that maysuggest that the neural substrates associated with vari-ous cognitive processes often function interactively

REFERENCES

Alivisatos B amp Petrides M (1997) Functional activation of thehuman brain during mental rotation Neuropsychologia 35 111-118

Attneave F amp Arnoult M D (1956) The quantitative study ofshape and pattern perception Psychological Bulletin 53 452-471

Bandettini P A Wong E C Hinks R S Tikofsky R S ampHyde J S (1992) Time course EPI of human brain function duringtask activation Magnetic Resonance in Medicine 25 390-397

Braver T S Cohen J D Jonides J Smith E E amp Noll D C(1997) A parametric study of prefrontal cortex involvement in humanworking memory NeuroImage 5 49-62

Carpenter P A Just M A Keller T A Eddy W amp Thulborn K(1999) Graded functional activation in the visuospatial system withthe amount of task demand Journal of Cognitive Neuroscience 119-24

Carpenter P A Just M A amp Reichle E D (2000) Workingmemory and executive function Evidence from neuroimaging Cur-rent Opinion in Neurobiology 10 195-199

Caviness V S Jr Meyer J Makris N amp Kennedy D N (1996)MRI-based topographic parcellation of human neocortex An anatom-ically specified method with estimate of reliability Journal of Cog-nitive Neuroscience 8 566-587

Cohen M S Kosslyn S M Breiter H C DiGirolamo G JThompson W L Anderson A K Bookheimer S Y RosenB R amp Belliveau J W (1996) Changes in cortical activity dur-ing mental rotation A mapping study using functional MRI Brain119 89-100

Cooper L A (1975) Mental rotation of random two-dimensionalshapes Cognitive Psychology 7 20-43

Cooper L A amp Podgorny P (1976) Mental transformations and vi-sual comparison processes Effects of complexity and similarityJournal of Experimental Psychology Human Perception amp Perfor-mance 2 503-514

Cox R W (1996) AFNI Software for visualization and analysis offunctional magnetic resonance neuroimages Computers amp Biomed-ical Research 29 162-173

DrsquoEsposito M Aguirre G K Zarahn E amp Ballard D (1998)Functional MRI studies of spatial and non-spatial working memoryCognitive Brain Research 7 1-13

DrsquoEsposito M Postle B Ballard D amp Lease J (1999) Main-tenance versus manipulation of information held in working mem-ory An event-related fMRI study Brain amp Cognition 41 66-86

Desrocher M E Smith M L amp Taylor M J (1995) Stimulusand sex differences in performance of mental rotation Evidence fromevent-related potentials Brain amp Cognition 28 14-38

Diwadkar V A Carpenter P A amp Just M A (2000) Collabora-tive activity between parietal and dorso-lateral prefrontal cortex indynamic spatial working memory revealed by fMRI NeuroImage12 85-99

Eddy W F Fitzgerald M Genovese C R Mockus A amp NollD C (1996) Functional imaging analysis softwaremdashcomputationalolio In A Prat (Ed) Proceedings in computational statistics (pp 39-49) Heidelberg Physica-Verlag

Farah M J (1990) Visual agnosia Cambridge MA MIT PressFriston K J (1994) Functional and effective connectivity A synthesis

Human Brain Mapping 2 56-78


Friston K J Frith C D Liddle P F amp Frackowiak R S J(1993) Functional connectivity The principal-component analysisof large PET data sets Journal of Cerebral Blood Flow amp Metabo-lism 13 5-14

Grasby P M Frith C D Friston K J Simpson J Fletcher P CFrackowiak R S J amp Dolan R J (1994) A graded task approachto the functional mapping of brain areas implicated in auditoryndashverbal memory Brain 117 1271-1282

Hasegawa M Carpenter P A amp Just M A (2002) An fMRIstudy of bilingual sentence comprehension and workload Neuro-Image 15 647-660

Just M A Carpenter P A Keller T A Eddy W F amp Thul-born K R (1996) Brain activation modulated by sentence compre-hension Science 274 114-116

Just M A Carpenter P A Maguire M Diwadkar V amp Mc-Mains S (2001) Mental rotation of objects retrieved from memoryA functional MRI study of spatial processing Journal of Experi-mental Psychology General 130 493-504

Just M A Carpenter P A amp Varma S (1999) Computationalmodeling of high-level cognition and brain function Human BrainMapping 8 128-136

Kanwisher N Chun M M McDermott J amp Ledden P J (1996)Functional imaging of human visual recognition Cognitive Brain Re-search 5 55-67

Keller T A Carpenter P A amp Just M A (2001) The neuralbases of sentence comprehension A fMRI examination of syntacticand lexical processing Cerebral Cortex 11 223-237

Kwong K K Belliveau J W Chesler D A Goldberg I EWeisskoff R M Poncelet B P Kennedy D N Hoppel B ECohen M S Turner R Cheng H-M Brady T J amp RosenB R (1992) Dynamic magnetic resonance imaging of human brainactivity during primary sensory stimulation Proceedings of the Na-tional Academy of Sciences 89 5675-5679

Lazar N A Eddy W F Genovese C R amp Welling J S (2001)Statistical issues in fMRI for brain imaging International StatisticalReview 69 105-127

Levy R amp Goldman-Rakic P S (2000) Segregation of workingmemory functions within the dorsolateral prefrontal cortex Experi-mental Brain Research 133 23-32

Logothetis N K Pauls J amp Poggio T (1995) Shape representa-tion in the inferior temporal cortex of monkeys Current Biology 5552-563

Malach R Reppas J B Benson R R Kwong K K Jiang HKennedy W A Ledden P J Brady T J Rosen B R amp TootellR B H (1995) Object-related activity revealed by functional mag-netic resonance imaging in human occipital cortex Proceedings ofthe National Academy of Sciences 92 8135-8139

McIntosh A R amp Gonzalez-Lima F (1994) Structural equationmodeling and its application to network analysis in functional brainimaging Human Brain Mapping 2 2-22

Mesulam M-M (1990) Large-scale neurocognitive networks and dis-tributed processing for attention language and memory Annals ofNeurology 28 597-613

Mesulam M-M (1998) From sensation to cognition Brain 1211013-1052

Miller E K Erikson C A amp Desimone R (1996) Neural mech-

anisms of visual working memory in prefrontal cortex of the macaqueJournal of Neuroscience 16 5154-5167

Mishkin M Ungerleider L G amp Macko K A (1983) Object vi-sion and spatial vision Two cortical pathways Trends in Neuro-sciences 6 414-417

Ogawa S Lee T M Kay A R amp Tank D W (1990) Brain mag-netic resonance imaging with contrast dependent on blood oxygena-tion Proceedings of the National Academy of Sciences 87 9868-9872

Owen A M Stern C E Look R B Tracey I Rosen B R ampPetrides M (1998) Functional organization of spatial and nonspa-tial working memory processing within the human lateral frontal cor-tex Proceedings of the National Academy of Sciences 95 7721-7726

Peronnet F amp Farah M J (1989) Mental rotation An event-related potential study with a validated mental rotation task Brain ampCognition 9 279-288

Peterson B S Skudlarski P Gatenby J C Zhang H AndersonA W amp Gore J C (1999) An fMRI study of Stroop wordndashcolorinterference Evidence for cingulate subregions subserving multipledistributed attentional systems Biological Psychiatry 45 1237-1258

Petrides M (1994) Frontal lobes and working memory Evidencefrom investigations of the effects of cortical excisions in nonhumanprimates In F Boller amp J Grafman (Eds) Handbook of neuro-psychology (Vol 9 pp 59-82) Amsterdam Elsevier

Rademacher J Galaburda A M Kennedy D N Filipek P Aamp Caviness V S Jr (1992) Human cerebral cortex Localizationparcellation and morphometry with magnetic resonance imagingJournal of Cognitive Neuroscience 4 352-374

Shepard R N amp Metzler J (1971) Mental rotation of three-dimensional objects Science 171 701-703

Smith E E Jonides J amp Koeppe R A (1996) Dissociating verbaland spatial working memory using PET Cerebral Cortex 6 11-20

Tagaris G A Kim S-G Strupp J P Andersen P Ugurbil Kamp Georgopoulos A P (1997) Mental rotation studied by func-tional magnetic resonance imaging at high field (4 tesla) Perfor-mance and cortical activation Journal of Cognitive Neuroscience 9419-432

Talairach J amp Tournoux P (1988) Co-planar stereotaxic atlas ofthe human brain New York Thieme

Thulborn K R Voyvodic J McCurtain B Gillen J Chang SJust M A Carpenter P A amp Sweeney J A (1996) High fieldfunctional MRI in humans Applications to cognitive function In P PPavone amp P Rossi (Eds) Proceedings of the European Seminars onDiagnostic and Interventional Imaging (Syllabus 4 pp 91-96) NewYork Springer-Verlag

Ungerleider L G amp Haxby J V (1994) ldquoWhatrdquo and ldquowhererdquo in thehuman brain Current Opinion in Neurobiology 4 157-165

Ungerleider L G amp Mishkin M (1982) Two cortical visual sys-tems In D J Ingle M A Goodale amp R J W Mansfield (Eds) Theanalysis of visual behavior (pp 549-586) Cambridge MA MITPress

Wijers A A Otten L J Feenstra S Mulder G amp MulderL J M (1989) Brain potentials during selective attention memorysearch and mental rotation Psychophysiology 26 452-467

(Continued on next page)


APPENDIXResults of the Analysis With the Percentage of Difference in Signal Intensity (dsi)

Manipulation dsi 2 (Amount) 2 (Complexity) 2 (Hemisphere)Interactive relationship between object recognition and mental rotation The data are shown in

Table A1 and the ANOVA results are shown in Table A2 Many ROIs showed the same tendency toward the over-additive interaction between amount of rotation and complexity as the voxel count data (eg the DLPFC theFEF the IFG the PPREC the IPS the IT and the SES) although they did not reach statistical significance ex-cept for the SES [F(110) 930 p 05] The CALC showed an interaction in which the activation was higherfor simple stimuli than for complex stimuli for 0ordm of rotation whereas this relationship was reversed for 90ordm

Hemispheric difference Most ROIs showed bilateral activation although some ROIs showed hemisphericdifferences (the FEF the IFG and the IES out of 10 ROIs) The right hemisphere showed greater percentageof signal intensity change for the FEF [F(17) 1052 p 01] and the IFG [F(19) 1042 p 01] Aninteraction was found for IFG in which the effect of degree of rotation was greater for the right hemispherethan for the left hemisphere [F(19) 550 p 05]

The IES showed a significant three-way interaction in which the complexity effect was greater for 90ordm ofrotation than for 0ordm for the left hemisphere whereas the complexity effect remained constant across the twolevels of rotation for the right hemisphere

Table A1Mean Difference in Signal Intensity for Conditions for Each Region of Interest

Left Hemisphere Right Hemisphere

Region S0ordm C0ordm S90ordm C90ordm S0ordm C0ordm S90ordm C90ordm

FrontalDorsolateral prefrontal cortex 10 09 10 13 11 11 13 17Frontal eye field 11 12 14 17 17 18 18 22Inferior frontal gyrus 09 09 10 11 11 13 14 18Posterior precentral sulcus 09 10 14 16 12 13 16 19Supplementary motor area 05 11 13 16Superior medial frontal paracingulate 09 11 11 14

ParietalInferior parietal lobule 09 11 13 14 10 11 12 14Intraparietal sulcus 11 13 15 19 11 13 15 19Superior parietal lobule 09 13 16 20 10 12 15 18

TemporalInferior temporal lobule 11 09 12 16 11 13 14 17

OccipitalInferior extrastriate 15 13 15 20 13 16 16 19Superior extrastriate 15 14 16 21 15 15 15 18Calcarine fissure 18 17 17 18Occipital pole 16 18 18 21

Table A2F Values of Analyses of Variance on the Percentage of Signal Intensity

(dsi Amount [2] Complexity [2] Hemisphere [2])

Region df Amount Complexity Hemisphere AC AH CH ACH

FrontalDorsolateral prefrontal cortex 110 1381 185 383 216 258 202 004Frontal eye field 17 365 89 1052 274 000 248 182Inferior frontal gyrus 19 653 465dagger 1042 060 55 278 005Posterior precentral sulcus 16 662 1321 146 198 007 114 010Supplementary motor area 16 1421 1778 NA 092 NA NA NASuperior medial frontal paracingulate 19 398 63 NA 004 NA NA NA



OccipitalInferior extrastriate 110 72 797 003 639 001 357 54

Superior extrastriate 110 2337 64 010 930 322 191 040Calcarine fissure 110 030 003 NA 1420 NA NA NAOccipital pole 19 616 743 NA 014 NA NA NA

dagger05 p 07 p 05 p 01 p 001

(Manuscript received February 4 2004revision accepted for publication September 21 2004)


and its comparison with the represented coordinates ofthe target object A graded fMRI study of mental rota-tion in which the Shepard and Metzler (1971) task wasused showed that increases in the amount of rotationfrom 0ordm to 120ordm are associated with linearly increasingamounts of activation in the left and right intraparietalsulcus (IPS Carpenter et al 1999) This result was ob-tained even when the self-paced trial times were equatedto the duration of the 0ordm condition indicating that the re-sult is not simply due to time on task Also a controlcondition suggested that eye movements per se contributedminimally to the result the control involving scanningbetween two grids so that the magnitude and number ofeye movements exceeded those in the highest rotationcondition Finally activation around the IPS occurs dur-ing mental rotation even when the input is auditory so itis not dependent on the processing of visual input (JustCarpenter Maguire Diwadkar amp McMains 2001)

Other neuroimaging and event-related potential (ERP)studies also implicate this region In a study involving theShepardndashMetzler rotation task fMRI-measured activationwas found in the left and right parietal regions (Brodmannareas [BAs] 7a and 7b and sometimes BA 40) when arotation condition was compared with a 0ordm rotation condi-tion (eg Cohen et al 1996 Tagaris et al 1997) A PETstudy showed activation in the left posterior-superiorparietal cortex when activation during the mental rota-tion of letters was compared with a condition requiringa simple discrimination of normal and mirror-image up-right letters (Alivisatos amp Petrides 1997) Parietal in-volvement has also been found in three ERP studies of asimpler rotation task in which participants judged whethera letter was normal or mirror-imaged (Desrocher Smithamp Taylor 1995 Peronnet amp Farah 1989 Wijers OttenFeenstra Mulder amp Mulder 1989) As the angular dis-parity increased there was increasing negativity in theERP waveforms in the latency range of 350ndash800 msecparticularly in the parietal and occipital leads which wasinterpreted as reflecting rotation Thus manipulating themental rotation workload should affect the activation inparietal regions

In addition to mental rotation the effect of figuralcomplexity which is hypothesized to affect figural en-coding was examined in this study The so-called whatsystem is associated with a stream that includes the oc-cipital region the IES regions and the inferior temporalgyrus (Ungerleider amp Mishkin 1982) Numerous func-tional neuroimaging studies of visual object recognitionhave shown activation in the inferior temporal cortex(eg Kanwisher Chun McDermott amp Ledden 1996Malach et al 1995) Also lesions in this area can cor-relate with impaired shape-based visual recognition orvisual agnosia (Farah 1990) Microelectrode studieswith macaque monkeys have also shown activation as-sociated in the homologous regions in shape-based vi-sual recognition studies (Logothetis Pauls amp Poggio1995) None of these studies suggests that these patternsof activity occur independently of activity in other corti-cal regions Indeed there is evidence that both the infe-

rior temporal and the IPS regions show increased activa-tion if the recognition of objects from line drawings ismade more difficult by deleting line contours (Diwad-kar Carpenter amp Just 2000)

A third cortical area of interest includes the frontal re-gions that are associated with maintaining and operatingon spatial and visual information over short time periodsoften referred to as spatial working memory (eg Levy ampGoldman-Rakic 2000 Miller Erikson amp Desimone1996 Owen et al 1998 Petrides 1994) Several brain-imaging studies have investigated the hypothesis that theprefrontal cortex is organized according to types of in-formation such as spatial versus verbal information (egSmith Jonides amp Koeppe 1996) and spatial versus ob-ject information (DrsquoEsposito Aguirre Zarahn amp Ballard1998 Levy amp Goldman-Rakic 2000) Other studies havebeen done to examine whether the prefrontal cortex canbe divided into functional subcomponents on the basis oftypes of operations such as maintenance versus manipu-lation (eg DrsquoEsposito Postle Ballard amp Lease 1999Petrides 1994) But none of the proposed dichotomiesgives completely dissociated activation (Carpenter Justamp Reichle 2000 DrsquoEsposito et al 1999) In the presentstudy we examined how prefrontal activation might showsensitivity to variables that impact on two types of visuo-spatial processes figural encoding and rotation

Previous theorizing about the relationship betweenfigural complexity and mental rotation has been basedlargely on behavioral studies in which it has been hy-pothesized that the figural encoding and the rotation pro-cesses are relatively independent of each other since ithad been found that rate of rotation is not affected by fig-ural complexity (Cooper 1975) We considered two pos-sible hypotheses in examining these issues with respectto brain activation One was the independent pathwayhypothesis based on Cooperrsquos conclusion If the amountof rotation and figural complexity affect processes thatare independent of each other the manipulation of stim-ulus complexity should primarily affect the activation ofthe regions in the ventral pathway and the rotation ma-nipulation should primarily affect the parietal regions inthe dorsal pathway On the other hand previous studiesin our laboratory had demonstrated that logically sepa-rable processes can show an overadditive interactionwhen combined For example in language comprehen-sion Keller et al (2001) examined the effects of lexicalfrequency and syntactic complexity processes that werehypothesized to operate on different levels which re-sulted in an overadditive interaction in numerous corti-cal regions suggesting that the two types of processesdraw on a shared infrastructure of resources Thereforethe relationship between the dorsal and the ventral path-ways may be interactive (interactive pathway hypothe-sis) and both the complexity and the rotation manipula-tions may affect both the IES and the parietal regions

We were also interested in examining the temporal as-pects of processing by measuring correlations of the ac-tivation time series between regions of interest (ROIs)within various conditions Such analyses can be inter-



METHOD

























OccipitalPole


FEF

DLPFC

Temporal

Inferior Temporal


Pars Triangularis







RESULTS















Simple Complex



0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186





























DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001




METHOD

























OccipitalPole


FEF

DLPFC

Temporal

Inferior Temporal


Pars Triangularis







RESULTS















Simple Complex



0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186





























DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001















OccipitalPole


FEF

DLPFC

Temporal

Inferior Temporal


Pars Triangularis







RESULTS















Simple Complex



0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186





























DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001






RESULTS















Simple Complex



0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186





























DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001











Simple Complex



0ordm 2221 7216 73 101 2569 6581 73 9190ordm 2851 8152 245 113 3669 9482 255 186





























DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001





















DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001








DISCUSSION






DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001




DLPFC DLPFC




Paracingulate






OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC









OccipitalPole

DLPFC DLPFC

FEF FEF









OccipitalPole



L R L R

L R L R
















TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001











TemporalIT (L) 39 42 45 30 57 37 59 47IT (R) 34 51 57 46 30 41 45 56 51 60


Communality 357 345 424 268 641 416 349 497 407 391 393 506 596








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001








REFERENCES











































































dagger05 p 07 p 05 p 01 p 001


























































dagger05 p 07 p 05 p 01 p 001























dagger05 p 07 p 05 p 01 p 001


Documents

Interactions between the dorsal and the ventral pathways ... · 56 KOSHINO, CARPENTER, KELLER, AND JUST preted as reflecting the functional connectivity between cortical regions