Psychonomic Bulletin amp Review2002 9 (4) 625-636

There is a movement afoot in cognitive science to grantthe body a central role in shaping the mind Proponents ofembodied cognition take as their theoretical starting pointnot a mind working on abstract problems but a body thatrequires a mind to make it function These opening linesby Clark (1998) are typical ldquoBiological brains are first andforemost the control systems for biological bodies Biolog-ical bodies move and act in rich real-world surroundingsrdquo(p 506)

Traditionally the various branches of cognitive sciencehave viewed the mind as an abstract information proces-sor whose connections to the outside world were of littletheoretical importance Perceptual and motor systemsthough reasonable objects of inquiry in their own rightwere not considered relevant to understanding ldquocentralrdquocognitive processes Instead they were thought to servemerely as peripheral input and output devices This stancewas evident in the early decades of cognitive psychologywhen most theories of human thinking dealt in proposi-tional forms of knowledge During the same time periodartificial intelligence was dominated by computer modelsof abstract symbol processing Philosophy of mind toomade its contribution to this zeitgeist most notably inFodorrsquos (1983) modularity hypothesis According to Fodorcentral cognition is not modular but its connections to theworld are Perceptual and motor processing are done byinformationally encapsulated plug-ins providing sharplylimited forms of input and output

However there is a radically different stance that also hasroots in diverse branches of cognitive science This stance

has emphasized sensory and motor functions as well as theirimportance for successful interaction with the environmentEarly sources include the view of 19th century psychologiststhat there was no such thing as ldquoimageless thoughtrdquo (Good-win 1999) motor theories of perception such as those sug-gested by William James and others (see Prinz 1987 for areview) the developmental psychology of Jean Piagetwhich emphasized the emergence of cognitive abilities outof a groundwork of sensorimotor abilities and the ecologi-cal psychology of J J Gibson which viewed perception interms of affordancesmdash potential interactions with the envi-ronment In the 1980s linguists began exploring how ab-stract concepts may be based on metaphors for bodily phys-ical concepts (eg Lakoff amp Johnson 1980) At the sametime within the field of artificial intelligence behavior-based robotics began to emphasize routines for interactingwith the environment rather than internal representationsused for abstract thought (see eg Brooks 1986)

This kind of approach has recently attained high visi-bility under the banner of embodied cognition There is agrowing commitment to the idea that the mind must be un-derstood in the context of its relationship to a physicalbody that interacts with the world It is argued that we haveevolved from creatures whose neural resources were de-voted primarily to perceptual and motoric processing andwhose cognitive activity consisted largely of immediateon-line interaction with the environment Hence human cog-nition rather than being centralized abstract and sharplydistinct from peripheral input and output modules may in-stead have deep roots in sensorimotor processing

Although this general approach is enjoying increasinglybroad support there is in fact a great deal of diversity inthe claims involved and the degree of controversy they at-tract If the term embodied cognition is to retain meaning-

625 Copyright 2002 Psychonomic Society Inc

Correspondence should be addressed to M Wilson Department ofPsychology University of California Santa Cruz CA 95064 (e-mailmlwilsoncatsucscedu)

THEORETICAL AND REVIEW ARTICLES

Six views of embodied cognition

MARGARET WILSONUniversity of California Santa Cruz California

The emerging viewpoint of embodied cognition holds that cognitive processes are deeply rooted inthe bodyrsquos interactions with the world This position actually houses a number of distinct claims someof which are more controversial than others This paper distinguishes and evaluates the following sixclaims (1) cognition is situated (2) cognition is time-pressured (3) we off-load cognitive work ontothe environment (4) the environment is part of the cognitive system (5) cognition is for action (6) off-line cognition is body based Of these the first three and the fifth appear to be at least partially trueand their usefulness is best evaluated in terms of the range of their applicability The fourth claim I argueis deeply problematic The sixth claim has received the least attention in the literature on embodiedcognition but it may in fact be the best documented and most powerful of the six claims

626 WILSON

ful use we need to disentangle and evaluate these diverseclaims Among the most prominent are the following

1 Cognition is situated Cognitive activity takes placein the context of a real-world environment and it inher-ently involves perception and action

2 Cognition is time pressured We are ldquomind on thehoofrdquo (Clark 1997) and cognition must be understood interms of how it functions under the pressures of real-timeinteraction with the environment

3 We off-load cognitive work onto the environmentBecause of limits on our information-processing abilities(eg limits on attention and working memory) we exploitthe environment to reduce the cognitive workload We makethe environment hold or even manipulate information forus and we harvest that information only on a need-to-know basis

4 The environment is part of the cognitive systemThe information flow between mind and world is so denseand continuous that for scientists studying the nature ofcognitive activity the mind alone is not a meaningful unitof analysis

5 Cognition is for action The function of the mind isto guide action and cognitive mechanisms such as per-ception and memory must be understood in terms of theirultimate contribution to situation-appropriate behavior

6 Off-line cognition is body based Even when de-coupled from the environment the activity of the mind isgrounded in mechanisms that evolved for interaction withthe environmentmdashthat is mechanisms of sensory pro-cessing and motor control

Frequently in the literature on embodied cognition sev-eral or all of these claims are presented together as if theyrepresented a single point of view This strategy may haveits uses as for example in helping to draw a compellingpicture of what embodied cognition might be and why itmight be important This may have been particularly ap-propriate at the time that attention first was drawn to thisset of ideas when audiences were as yet unfamiliar withthis way of conceptualizing cognition The time has comethough to take a more careful look at each of these claimson its own merits

Claim 1 Cognition Is SituatedA cornerstone of the embodied cognition literature is

the claim that cognition is a situated activity (eg Chiel ampBeer 1997 Clark 1997 Pfeifer amp Scheier 1999 Steelsamp Brooks 1995 a commitment to situated cognition canalso be found in the literature on dynamical systemsmdasheg Beer 2000 Port amp van Gelder 1995 Thelen amp Smith1994 Wiles amp Dartnall 1999) Some authors go so far asto complain that the phrase ldquosituated cognitionrdquo impliesfalsely that there also exists cognition that is not situated(Greeno amp Moore 1993 p 50) It is important then thatwe be clear on what exactly it means for cognition to besituated

Simply put situated cognition is cognition that takesplace in the context of task-relevant inputs and outputsThat is while a cognitive process is being carried out per-

ceptual information continues to come in that affects pro-cessing and motor activity is executed that affects the environment in task-relevant ways Driving holding aconversation and moving around a room while trying toimagine where the furniture should go are all cognitive ac-tivities that are situated in this sense

Even with this basic definition of what it means for cog-nition to be situated we can note that large portions ofhuman cognitive processing are excluded Any cognitiveactivity that takes place ldquooff-linerdquo in the absence of task-relevant input and output is by definition not situated Ex-amples include planning remembering and day-dreamingin contexts not directly relevant to the content of plansmemories or day-dreams

This observation is not new (see eg Clark amp Grush1999 Grush 1997) but given the rhetoric currently to befound in the situated cognition literature the point isworth emphasizing By definition situated cognition in-volves interaction with the things that the cognitive activ-ity is about Yet one of the hallmarks of human cognitionis that it can take place decoupled from any immediate in-teraction with the environment We can lay plans for thefuture and think over what has happened in the past Wecan entertain counterfactuals to consider what might havehappened if circumstances had been different We can con-struct mental representations of situations we have neverexperienced based purely on linguistic input from othersIn short our ability to form mental representations aboutthings that are remote in time and space which is arguablythe sine qua non of human thought in principle cannotyield to a situated cognition analysis

An argument might be made though that situated cog-nition is nevertheless the bedrock of human cognition dueto in our evolutionary history Indeed it is popular to tryto drive intuitions about situated cognition by invoking apicture of our ancestors relying almost entirely on situatedskills Before we got civilized the argument goes the sur-vival value of our mental abilities depended on whetherthey helped us to act in direct response to immediate situ-ations such as obtaining food from the environment oravoiding predators Thus situated cognition may repre-sent our fundamental cognitive architecture even if this isnot always reflected in the artificial activities of our mod-ern world

This view of early humans though most likely exag-gerates the role of these survival-related on-line activitiesin the daily lives of early humans With respect to obtain-ing food meat eating was a late addition to the humanrepertoire and even after the onset of hunting the largemajority of calories were probably still obtained fromgathering Evidence for this claim comes from both thefossil record and the dietary patterns of huntergathererstoday (Leaky 1994) as well as from the dietary patternsof our nearest relatives the chimpanzees and bonobos(de Waal 2001) It might be more appropriate then toconsider gathering when trying to construct a picture ofour cognitive past But gathering lends itself much lesswell to a picture of human cognition as situated cognition

SIX VIEWS OF EMBODIED COGNITION 627

Successful gathering might be expected to benefit a greatdeal from human skills of reflective thoughtmdashremember-ing the terrain coordinating with onersquos fellow gatherersconsidering the probable impact of last weekrsquos rain and soon During the actual act of gathering though it is notclear what situated cognitive skills humans would bring tobear beyond those possessed by any foraging animal (Putin this light we can see that even hunting early humanstyle probably involved considerable nonsituated mentalactivity as well)

In addition to chasing food though being chased bypredators is also supposed to have been a major shapingforce according to this picture of the early human as a sit-uated cognizer Yet while avoiding predators obviously hasa great deal of survival value the situated skills of fight-or-flight are surely ancient shared with many other speciesAgain it is not clear how much mileage can be gotten outof trying to explain human intelligence in these terms In-stead the cognitive abilities that contributed to uniquelyhuman strategies for avoiding predation were probably ofquite a different sort As early humans became increas-ingly sophisticated in their social abilities avoiding pre-dation almost certainly involved increasing use of off-linepreventative and communicative measures

Finally we should consider the mental activities that areknown to have characterized the emerging human popu-lation and that set them apart from earlier hominid speciesThese included increasingly sophisticated tool-makingparticularly the shaping of tools to match a mental tem-plate language allowing communication about hypothet-icals past events and other nonimmediate situations anddepictive art showing the ability to mentally represent whatis not present and to engage in representation for repre-sentationrsquos sake rather than for any situated functionality(see Leakey 1994 for further details) All of these abilitiesreflect the increasingly off-line nature of early humanthought To focus on situated cognition as the fundamen-tal principle of our cognitive architecture is thus to neglectthese species-defining features of human cognition

A few counterarguments to this can be found in the lit-erature Barsalou (1999a) for example suggests that lan-guage was used by early humans primarily for immediatesituated indexical purposes These situated uses of lan-guage were intended to influence the behavior of othersduring activities such as hunting gathering and simplemanufacturing However some of the examples that Barsa-lou gives of situated uses of language appear to be in factoff-line uses where the referent is distant in time or spacemdashas for example in describing distant terrain to people whohave never seen it One can easily think of further nonsitu-ated uses of language that would serve adaptive functionsfor early humans absorbing parental edicts about avoidingdangerous behaviors holding in mind instructions for whatmaterials to go fetch when helping with tool manufactur-ing deciding whether to join in a planned activity such asgoing to the river to cool off and comprehending gossipabout members of the social hierarchy who are not presentIt seems plausible then that language served off-line func-

tions from early on Indeed once the representational ca-pacity of language emerged it is unclear why its full ca-pacity in this respect would not be used

Along different lines Brooks (1999 p 81) argues thatbecause nonsituated cognitive abilities emerged late in thehistory of animal life on this planet after extremely longperiods in which no such innovations appeared these weretherefore the easy problems for evolution to solve (andhence by implication not of much theoretical interest) Infact exactly the opposite can be inferred Easy evolution-ary solutions tend to arise again and again a process knownas convergent evolution In contrast the late emergenceand solitary status of an animal with abilities such as man-ufacturing to a mental template language and artistic de-piction attests to a radical and complex innovation in evo-lutionary engineering

In short an argument for the centrality of situated cog-nition based on the demands of human survival in the wildis not strongly persuasive Furthermore overstating thecase for situated cognition may ultimately impede our un-derstanding of the aspects of cognition that in fact are sit-uated As will be discussed in the next two sections thereis much to be learned about the ways in which we engagein cognitive activity that is tightly connected with our on-going interaction with the environment Spatial cognitionin particular tends to be situated Trying to fit a piece intoa jigsaw puzzle for example may owe more to continuousreevaluating of spatial relationships that are being contin-uously manipulated than it does to any kind of disembod-ied pattern matching (cf Kirsh amp Maglio 1994) For cer-tain kinds of tasks in fact humans may actively choose tosituate themselves (see Section 3)

Claim 2 Cognition is Time PressuredThe previous section considered situated cognition sim-

ply to mean cognition that is situation bound There ap-pears to be more though that is often meant by ldquosituatedcognitionrdquo It is frequently stated that situated agents mustdeal with the constraints of ldquoreal timerdquo or ldquoruntimerdquo (seeeg Brooks 1991b Pfeifer amp Scheier 1999 chap 3 vanGelder amp Port 1995) These phrases are used to highlighta weakness of traditional artificial intelligence modelswhich are generally allowed to build up and manipulateinternal representations of a situation at their leisure Areal creature in a real environment it is pointed out has nosuch leisure It must cope with predators prey stationaryobjects and terrain as fast as the situation dishes them outThe observation that situated cognition takes place ldquoin realtimerdquo is at bottom an observation that situated cognitionmust cope with time pressure

A belief in the importance of time pressure as a shap-ing force in cognitive architecture underlies much of thesituated cognition literature For example in the field ofbehavior-based robotics ldquoautonomous agentsrdquo have beenbuilt to perform tasks such as walking on an uneven sur-face with six legs (Quinn amp Espenschied 1993) brachi-ating or swinging ldquobranch to branchrdquo like an ape (Saito ampFukuda 1994) and navigating around a cluttered envi-

628 WILSON

ronment looking for soda cans without bumping into any-thing (Mataric 1991) Each of these activities requiresreal-time responsiveness to feedback from the environ-ment And although these activities are not especially ldquoin-telligentrdquo in and of themselves it is claimed that greatercognitive complexity can be built up from successive lay-ers of procedures for real-time interaction with the envi-ronment (for reviews see Brooks 1999 Clark 1997Pfeifer amp Scheier 1999)

A similar emphasis on time pressure as a principle thatshapes cognition can be seen as well in human behavioralresearch on situated cognition For example Kirsh andMaglio (1994) have studied the procedures that people usein making time-pressured spatial decisions while playingthe video game Tetris (discussed in more detail in Section 3)This research is conducted with the assumption that situ-ations such as Tetris playing are a microcosm that can elu-cidate general principles of human cognition

One reason that time pressure is thought to matter is thatit creates what has been called a ldquorepresentational bottle-neckrdquo When situations demand fast and continuouslyevolving responses there may simply not be time to buildup a full-blown mental model of the environment fromwhich to derive a plan of action Instead it is argued beinga situated cognizer requires the use of cheap and efficienttricks for generating situation-appropriate action on thefly (In fact a debate has raged over whether a situatedcognizer would make use of internal representations at allsee Agre 1993 Beer 2000 Brooks 1991a Markman ampDietrich 2000 Vera amp Simon 1993) Thus taking real-timesituated action as the starting point for cognitive activityis argued to have far-reaching consequences for cognitivearchitecture

The force of this argument though depends upon theassumption that actual cognizers (humans for example)are indeed engineered so as to circumvent this represen-tational bottleneck and are capable of functioning well andldquonormallyrdquo in time-pressured situations But although onemight wish an ideal cognitive system to have solved theproblem the assumption that we have solved it is dis-putable Confronted with novel cognitive or perceptuo-motor problems humans predictably fall apart under timepressure That is we very often do not successfully copewith the representational bottleneck Lift the demands oftime pressure though and some of the true power ofhuman cognition becomes evident Given the opportunitywe often behave in a decidedly off-line way stepping backobserving assessing planning and only then taking actionIt is far from clear then that the human cognitive systemhas evolved an effective engineering solution for the real-time constraints of the representational bottleneck

Furthermore many of the activities in which we engagein daily life even many that are clearly situated do not in-herently involve time pressure Cases include mundaneactivities such as making sandwiches and paying bills aswell as more demanding cognitive tasks such as doingcrossword puzzles and reading scientific papers In eachof these cases input from and output to the environment

are necessary but they are at the leisure of the cognizer(Of course any task can be performed in a hurry andmany often are But the state of ldquobeing in a hurryrdquo is onethat is cognitively self-imposed and such tasks are gener-ally performed only as fast as they can be even if thismeans being late) Situations in which time pressure is in-herently part of the task such as playing video games or changing lanes in heavy traffic may actually be theexception

This is not to say though that an understanding of real-time interaction with the environment has nothing to con-tribute to our understanding of human cognition A num-ber of important domains may indeed be illuminated byconsidering them from this standpoint The most obviousof these is perceptuomotor coordination of any kind Evensuch basic activities as walking require continuous recip-rocal influence between perceptual flow and motor com-mands Skilled hand movement particularly the manipu-lation of objects in the environment is another persuasiveexample of a time-locked perceptuomotor activity Moresophisticated forms of real-time situated cognition can beseen in any activity that involves continuous updating ofplans in response to rapidly changing conditions Suchchanging conditions often involve the activity of anotherhuman or animal that must be reckoned with Examplesinclude playing a sport driving in traffic and roughhous-ing with a dog As interesting as the principles governingthese cases may be in their own right though the argu-ment that they can be scaled up to provide the governingprinciples of human cognition in general appears to be un-persuasive

Claim 3 We Off-Load Cognitive Work Onto theEnvironment

Despite the fact that we frequently choose to run ourcognitive processes off line it is still true that in some sit-uations we are forced to function on line In those situa-tions what do we do about our cognitive limitations Oneresponse as we have seen is to fall apart However hu-mans are not entirely helpless when confronting the rep-resentational bottleneck and two types of strategies ap-pear to be available when one is confronting on-line taskdemands The first is to rely on preloaded representationsacquired through prior learning (discussed further in Sec-tion 6) What about novel stimuli and tasks though Inthese cases there is a second option which is to reduce thecognitive workload by making use of the environment it-self in strategic waysmdashleaving information out there inthe world to be accessed as needed rather than taking timeto fully encode it and using epistemic actions (Kirsh ampMaglio 1994) to alter the environment in order to reducethe cognitive work remaining to be done

(The environment can also be used as a long-term archiveas in the use of reference books appointment calendarsand computer files This can be thought of as off-loadingto avoid memorizing which is subtly but importantly dif-ferent from off-loading to avoid encoding or holding ac-tive in short-term memory what is present in the immedi-

SIX VIEWS OF EMBODIED COGNITION 629

ate environment It is the latter case that is usually discussedin the literature on off-loading Although the archival casecertainly constitutes off-loading it appears to be of lesstheoretical interest The observation that we use such astrategy does not seem to challenge or shed light on exist-ing theories of cognition The present discussion will there-fore be restricted to what we may call the situated exam-ples of off-loading which are the focus of the literature)

Some investigators have begun to examine how off-loading work onto the environment may be used as a cog-nitive strategy Kirsh and Maglio (1994) as noted earlierhave reported a study involving the game Tetris in whichfalling block shapes must be rotated and horizontallytranslated to fit as compactly as possible with the shapesthat have already fallen The decision of how to orient andplace each block must be made before the block falls toofar to allow the necessary movements The data suggestthat players use actual rotation and translation movementsto simplify the problem to be solved rather than mentallycomputing a solution and then executing it A second ex-ample comes from Ballard Hayhoe Pook and Rao (1997)who asked subjects to reproduce patterns of colored blocksunder time pressure by dragging randomly scattered blockson a computer screen into a work area and arranging themthere Recorded eye movements showed repeated refer-encing of the blocks in the model pattern and these eyemovements occurred at strategic momentsmdashfor exampleto gather information first about a blockrsquos color and thenlater about its precise location within the pattern The au-thors argue that this is a ldquominimal memory strategyrdquo andthey show that it is the strategy most commonly used bysubjects

A few momentsrsquo thought can yield similar examplesfrom daily life Not all of them involve time pressure butother cognitive limitations such as those of attention andworking memory can drive us to a similar kind of off-loading strategy One example used earlier is that ofphysically moving around a room in order to generate so-lutions for where to put furniture Other examples includelaying out the pieces of something that requires assemblyin roughly the order and spatial relationships that they willhave in the finished product or giving directions for howto get somewhere by first turning onersquos self and onersquos lis-tener in the appropriate direction Glenberg and Robertson(1999) have experimentally studied one such exampleshowing that in a compass-and-map task subjects whowere allowed to indexically link written instructions to ob-jects in the environment during a learning phase per-formed better during a test phase than subjects who werenot both on comprehension of new written instructionsand on performance of the actual task

As noted earlier this kind of strategy seems to apply mostusefully to spatial tasks in particular But is off-loadingstrictly limited to cases in which we manipulate spatial in-formation Spatial tasks are only one arena of humanthought If off-loading is useful only for tasks that arethemselves spatial in nature its range of applicability as acognitive strategy is limited

In fact though potential uses of off-loading may be farbroader than this Consider for example such activitiesas counting on onersquos fingers drawing Venn diagrams anddoing math with pencil and paper Many of these activitiesare both situated and spatial in the sense that they involvethe manipulation of spatial relationships among elementsin the environment The advantage is that by doing actualphysical manipulation rather than computing a solution inour heads we save cognitive work However unlike theprevious examples there is also a sense in which these ac-tivities are not situated They are performed in the serviceof cognitive activity about something else something notpresent in the immediate environment

Typically the literature on off-loading has focused oncases in which the world is being used as ldquoits own bestmodelrdquo (Brooks 1991a p 139) Rather than attempt tomentally store and manipulate all the relevant details abouta situation we physically store and manipulate those de-tails out in the world in the very situation itself In theTetris case for example the elements being manipulateddo not serve as tokens for anything but themselves andtheir manipulation does not so much yield informationabout a solution as produce the goal state itself through trialand error In contrast actions such as diagramming repre-sent a quite different use of the environment Here thecognitive system is exploiting external resources to achievea solution or a piece of knowledge whose actual applica-tion will occur at some later time and place if at all

Notice what this buys us This form of off-loadingmdashwhat we might call symbolic off-loadingmdashmay in fact beapplied to spatial tasks as in the case of arranging tokensfor armies on a map but it may also be applied to non-spatial tasks as in the case of using Venn diagrams to de-termine logical relations among categories When the pur-pose of the activity is no longer directly linked to thesituation it also need not be directly linked to spatial prob-lems physical tokens and even their spatial relationshipscan be used to represent abstract nonspatial domains ofthought The history of mathematics attests to the powerbehind this decoupling strategy It should be noted toothat symbolic off-loading need not be deliberate and for-malized but can be seen in such universal and automaticbehaviors as gesturing while speaking It has been found thatgesturing is not epiphenomenal nor even strictly commu-nicative but seems to serve a cognitive function for thespeaker helping to grease the wheels of the thought processthat the speaker is trying to express (see eg Iverson ampGoldin-Meadow 1998 Krauss 1998) As we shall see inSection 6 the use of bodily resources for cognitive pur-poses not directly linked to the situation has potentially farreaching consequences for our understanding of cognitionin general

Claim 4 The Environment Is Partof the Cognitive System

The insight that the body and the environment play a rolein assisting cognitive activity has led some authors to asserta stronger claim that cognition is not an activity of the mind

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

626 WILSON

ful use we need to disentangle and evaluate these diverseclaims Among the most prominent are the following

1 Cognition is situated Cognitive activity takes placein the context of a real-world environment and it inher-ently involves perception and action

2 Cognition is time pressured We are ldquomind on thehoofrdquo (Clark 1997) and cognition must be understood interms of how it functions under the pressures of real-timeinteraction with the environment

3 We off-load cognitive work onto the environmentBecause of limits on our information-processing abilities(eg limits on attention and working memory) we exploitthe environment to reduce the cognitive workload We makethe environment hold or even manipulate information forus and we harvest that information only on a need-to-know basis

4 The environment is part of the cognitive systemThe information flow between mind and world is so denseand continuous that for scientists studying the nature ofcognitive activity the mind alone is not a meaningful unitof analysis

5 Cognition is for action The function of the mind isto guide action and cognitive mechanisms such as per-ception and memory must be understood in terms of theirultimate contribution to situation-appropriate behavior

6 Off-line cognition is body based Even when de-coupled from the environment the activity of the mind isgrounded in mechanisms that evolved for interaction withthe environmentmdashthat is mechanisms of sensory pro-cessing and motor control

Frequently in the literature on embodied cognition sev-eral or all of these claims are presented together as if theyrepresented a single point of view This strategy may haveits uses as for example in helping to draw a compellingpicture of what embodied cognition might be and why itmight be important This may have been particularly ap-propriate at the time that attention first was drawn to thisset of ideas when audiences were as yet unfamiliar withthis way of conceptualizing cognition The time has comethough to take a more careful look at each of these claimson its own merits

Claim 1 Cognition Is SituatedA cornerstone of the embodied cognition literature is

the claim that cognition is a situated activity (eg Chiel ampBeer 1997 Clark 1997 Pfeifer amp Scheier 1999 Steelsamp Brooks 1995 a commitment to situated cognition canalso be found in the literature on dynamical systemsmdasheg Beer 2000 Port amp van Gelder 1995 Thelen amp Smith1994 Wiles amp Dartnall 1999) Some authors go so far asto complain that the phrase ldquosituated cognitionrdquo impliesfalsely that there also exists cognition that is not situated(Greeno amp Moore 1993 p 50) It is important then thatwe be clear on what exactly it means for cognition to besituated

Simply put situated cognition is cognition that takesplace in the context of task-relevant inputs and outputsThat is while a cognitive process is being carried out per-

ceptual information continues to come in that affects pro-cessing and motor activity is executed that affects the environment in task-relevant ways Driving holding aconversation and moving around a room while trying toimagine where the furniture should go are all cognitive ac-tivities that are situated in this sense

Even with this basic definition of what it means for cog-nition to be situated we can note that large portions ofhuman cognitive processing are excluded Any cognitiveactivity that takes place ldquooff-linerdquo in the absence of task-relevant input and output is by definition not situated Ex-amples include planning remembering and day-dreamingin contexts not directly relevant to the content of plansmemories or day-dreams

This observation is not new (see eg Clark amp Grush1999 Grush 1997) but given the rhetoric currently to befound in the situated cognition literature the point isworth emphasizing By definition situated cognition in-volves interaction with the things that the cognitive activ-ity is about Yet one of the hallmarks of human cognitionis that it can take place decoupled from any immediate in-teraction with the environment We can lay plans for thefuture and think over what has happened in the past Wecan entertain counterfactuals to consider what might havehappened if circumstances had been different We can con-struct mental representations of situations we have neverexperienced based purely on linguistic input from othersIn short our ability to form mental representations aboutthings that are remote in time and space which is arguablythe sine qua non of human thought in principle cannotyield to a situated cognition analysis

An argument might be made though that situated cog-nition is nevertheless the bedrock of human cognition dueto in our evolutionary history Indeed it is popular to tryto drive intuitions about situated cognition by invoking apicture of our ancestors relying almost entirely on situatedskills Before we got civilized the argument goes the sur-vival value of our mental abilities depended on whetherthey helped us to act in direct response to immediate situ-ations such as obtaining food from the environment oravoiding predators Thus situated cognition may repre-sent our fundamental cognitive architecture even if this isnot always reflected in the artificial activities of our mod-ern world

This view of early humans though most likely exag-gerates the role of these survival-related on-line activitiesin the daily lives of early humans With respect to obtain-ing food meat eating was a late addition to the humanrepertoire and even after the onset of hunting the largemajority of calories were probably still obtained fromgathering Evidence for this claim comes from both thefossil record and the dietary patterns of huntergathererstoday (Leaky 1994) as well as from the dietary patternsof our nearest relatives the chimpanzees and bonobos(de Waal 2001) It might be more appropriate then toconsider gathering when trying to construct a picture ofour cognitive past But gathering lends itself much lesswell to a picture of human cognition as situated cognition

SIX VIEWS OF EMBODIED COGNITION 627

Successful gathering might be expected to benefit a greatdeal from human skills of reflective thoughtmdashremember-ing the terrain coordinating with onersquos fellow gatherersconsidering the probable impact of last weekrsquos rain and soon During the actual act of gathering though it is notclear what situated cognitive skills humans would bring tobear beyond those possessed by any foraging animal (Putin this light we can see that even hunting early humanstyle probably involved considerable nonsituated mentalactivity as well)

In addition to chasing food though being chased bypredators is also supposed to have been a major shapingforce according to this picture of the early human as a sit-uated cognizer Yet while avoiding predators obviously hasa great deal of survival value the situated skills of fight-or-flight are surely ancient shared with many other speciesAgain it is not clear how much mileage can be gotten outof trying to explain human intelligence in these terms In-stead the cognitive abilities that contributed to uniquelyhuman strategies for avoiding predation were probably ofquite a different sort As early humans became increas-ingly sophisticated in their social abilities avoiding pre-dation almost certainly involved increasing use of off-linepreventative and communicative measures

Finally we should consider the mental activities that areknown to have characterized the emerging human popu-lation and that set them apart from earlier hominid speciesThese included increasingly sophisticated tool-makingparticularly the shaping of tools to match a mental tem-plate language allowing communication about hypothet-icals past events and other nonimmediate situations anddepictive art showing the ability to mentally represent whatis not present and to engage in representation for repre-sentationrsquos sake rather than for any situated functionality(see Leakey 1994 for further details) All of these abilitiesreflect the increasingly off-line nature of early humanthought To focus on situated cognition as the fundamen-tal principle of our cognitive architecture is thus to neglectthese species-defining features of human cognition

A few counterarguments to this can be found in the lit-erature Barsalou (1999a) for example suggests that lan-guage was used by early humans primarily for immediatesituated indexical purposes These situated uses of lan-guage were intended to influence the behavior of othersduring activities such as hunting gathering and simplemanufacturing However some of the examples that Barsa-lou gives of situated uses of language appear to be in factoff-line uses where the referent is distant in time or spacemdashas for example in describing distant terrain to people whohave never seen it One can easily think of further nonsitu-ated uses of language that would serve adaptive functionsfor early humans absorbing parental edicts about avoidingdangerous behaviors holding in mind instructions for whatmaterials to go fetch when helping with tool manufactur-ing deciding whether to join in a planned activity such asgoing to the river to cool off and comprehending gossipabout members of the social hierarchy who are not presentIt seems plausible then that language served off-line func-

tions from early on Indeed once the representational ca-pacity of language emerged it is unclear why its full ca-pacity in this respect would not be used

Along different lines Brooks (1999 p 81) argues thatbecause nonsituated cognitive abilities emerged late in thehistory of animal life on this planet after extremely longperiods in which no such innovations appeared these weretherefore the easy problems for evolution to solve (andhence by implication not of much theoretical interest) Infact exactly the opposite can be inferred Easy evolution-ary solutions tend to arise again and again a process knownas convergent evolution In contrast the late emergenceand solitary status of an animal with abilities such as man-ufacturing to a mental template language and artistic de-piction attests to a radical and complex innovation in evo-lutionary engineering

In short an argument for the centrality of situated cog-nition based on the demands of human survival in the wildis not strongly persuasive Furthermore overstating thecase for situated cognition may ultimately impede our un-derstanding of the aspects of cognition that in fact are sit-uated As will be discussed in the next two sections thereis much to be learned about the ways in which we engagein cognitive activity that is tightly connected with our on-going interaction with the environment Spatial cognitionin particular tends to be situated Trying to fit a piece intoa jigsaw puzzle for example may owe more to continuousreevaluating of spatial relationships that are being contin-uously manipulated than it does to any kind of disembod-ied pattern matching (cf Kirsh amp Maglio 1994) For cer-tain kinds of tasks in fact humans may actively choose tosituate themselves (see Section 3)

Claim 2 Cognition is Time PressuredThe previous section considered situated cognition sim-

ply to mean cognition that is situation bound There ap-pears to be more though that is often meant by ldquosituatedcognitionrdquo It is frequently stated that situated agents mustdeal with the constraints of ldquoreal timerdquo or ldquoruntimerdquo (seeeg Brooks 1991b Pfeifer amp Scheier 1999 chap 3 vanGelder amp Port 1995) These phrases are used to highlighta weakness of traditional artificial intelligence modelswhich are generally allowed to build up and manipulateinternal representations of a situation at their leisure Areal creature in a real environment it is pointed out has nosuch leisure It must cope with predators prey stationaryobjects and terrain as fast as the situation dishes them outThe observation that situated cognition takes place ldquoin realtimerdquo is at bottom an observation that situated cognitionmust cope with time pressure

A belief in the importance of time pressure as a shap-ing force in cognitive architecture underlies much of thesituated cognition literature For example in the field ofbehavior-based robotics ldquoautonomous agentsrdquo have beenbuilt to perform tasks such as walking on an uneven sur-face with six legs (Quinn amp Espenschied 1993) brachi-ating or swinging ldquobranch to branchrdquo like an ape (Saito ampFukuda 1994) and navigating around a cluttered envi-

628 WILSON

ronment looking for soda cans without bumping into any-thing (Mataric 1991) Each of these activities requiresreal-time responsiveness to feedback from the environ-ment And although these activities are not especially ldquoin-telligentrdquo in and of themselves it is claimed that greatercognitive complexity can be built up from successive lay-ers of procedures for real-time interaction with the envi-ronment (for reviews see Brooks 1999 Clark 1997Pfeifer amp Scheier 1999)

A similar emphasis on time pressure as a principle thatshapes cognition can be seen as well in human behavioralresearch on situated cognition For example Kirsh andMaglio (1994) have studied the procedures that people usein making time-pressured spatial decisions while playingthe video game Tetris (discussed in more detail in Section 3)This research is conducted with the assumption that situ-ations such as Tetris playing are a microcosm that can elu-cidate general principles of human cognition

One reason that time pressure is thought to matter is thatit creates what has been called a ldquorepresentational bottle-neckrdquo When situations demand fast and continuouslyevolving responses there may simply not be time to buildup a full-blown mental model of the environment fromwhich to derive a plan of action Instead it is argued beinga situated cognizer requires the use of cheap and efficienttricks for generating situation-appropriate action on thefly (In fact a debate has raged over whether a situatedcognizer would make use of internal representations at allsee Agre 1993 Beer 2000 Brooks 1991a Markman ampDietrich 2000 Vera amp Simon 1993) Thus taking real-timesituated action as the starting point for cognitive activityis argued to have far-reaching consequences for cognitivearchitecture

The force of this argument though depends upon theassumption that actual cognizers (humans for example)are indeed engineered so as to circumvent this represen-tational bottleneck and are capable of functioning well andldquonormallyrdquo in time-pressured situations But although onemight wish an ideal cognitive system to have solved theproblem the assumption that we have solved it is dis-putable Confronted with novel cognitive or perceptuo-motor problems humans predictably fall apart under timepressure That is we very often do not successfully copewith the representational bottleneck Lift the demands oftime pressure though and some of the true power ofhuman cognition becomes evident Given the opportunitywe often behave in a decidedly off-line way stepping backobserving assessing planning and only then taking actionIt is far from clear then that the human cognitive systemhas evolved an effective engineering solution for the real-time constraints of the representational bottleneck

Furthermore many of the activities in which we engagein daily life even many that are clearly situated do not in-herently involve time pressure Cases include mundaneactivities such as making sandwiches and paying bills aswell as more demanding cognitive tasks such as doingcrossword puzzles and reading scientific papers In eachof these cases input from and output to the environment

are necessary but they are at the leisure of the cognizer(Of course any task can be performed in a hurry andmany often are But the state of ldquobeing in a hurryrdquo is onethat is cognitively self-imposed and such tasks are gener-ally performed only as fast as they can be even if thismeans being late) Situations in which time pressure is in-herently part of the task such as playing video games or changing lanes in heavy traffic may actually be theexception

This is not to say though that an understanding of real-time interaction with the environment has nothing to con-tribute to our understanding of human cognition A num-ber of important domains may indeed be illuminated byconsidering them from this standpoint The most obviousof these is perceptuomotor coordination of any kind Evensuch basic activities as walking require continuous recip-rocal influence between perceptual flow and motor com-mands Skilled hand movement particularly the manipu-lation of objects in the environment is another persuasiveexample of a time-locked perceptuomotor activity Moresophisticated forms of real-time situated cognition can beseen in any activity that involves continuous updating ofplans in response to rapidly changing conditions Suchchanging conditions often involve the activity of anotherhuman or animal that must be reckoned with Examplesinclude playing a sport driving in traffic and roughhous-ing with a dog As interesting as the principles governingthese cases may be in their own right though the argu-ment that they can be scaled up to provide the governingprinciples of human cognition in general appears to be un-persuasive

Claim 3 We Off-Load Cognitive Work Onto theEnvironment

Despite the fact that we frequently choose to run ourcognitive processes off line it is still true that in some sit-uations we are forced to function on line In those situa-tions what do we do about our cognitive limitations Oneresponse as we have seen is to fall apart However hu-mans are not entirely helpless when confronting the rep-resentational bottleneck and two types of strategies ap-pear to be available when one is confronting on-line taskdemands The first is to rely on preloaded representationsacquired through prior learning (discussed further in Sec-tion 6) What about novel stimuli and tasks though Inthese cases there is a second option which is to reduce thecognitive workload by making use of the environment it-self in strategic waysmdashleaving information out there inthe world to be accessed as needed rather than taking timeto fully encode it and using epistemic actions (Kirsh ampMaglio 1994) to alter the environment in order to reducethe cognitive work remaining to be done

(The environment can also be used as a long-term archiveas in the use of reference books appointment calendarsand computer files This can be thought of as off-loadingto avoid memorizing which is subtly but importantly dif-ferent from off-loading to avoid encoding or holding ac-tive in short-term memory what is present in the immedi-

SIX VIEWS OF EMBODIED COGNITION 629

ate environment It is the latter case that is usually discussedin the literature on off-loading Although the archival casecertainly constitutes off-loading it appears to be of lesstheoretical interest The observation that we use such astrategy does not seem to challenge or shed light on exist-ing theories of cognition The present discussion will there-fore be restricted to what we may call the situated exam-ples of off-loading which are the focus of the literature)

Some investigators have begun to examine how off-loading work onto the environment may be used as a cog-nitive strategy Kirsh and Maglio (1994) as noted earlierhave reported a study involving the game Tetris in whichfalling block shapes must be rotated and horizontallytranslated to fit as compactly as possible with the shapesthat have already fallen The decision of how to orient andplace each block must be made before the block falls toofar to allow the necessary movements The data suggestthat players use actual rotation and translation movementsto simplify the problem to be solved rather than mentallycomputing a solution and then executing it A second ex-ample comes from Ballard Hayhoe Pook and Rao (1997)who asked subjects to reproduce patterns of colored blocksunder time pressure by dragging randomly scattered blockson a computer screen into a work area and arranging themthere Recorded eye movements showed repeated refer-encing of the blocks in the model pattern and these eyemovements occurred at strategic momentsmdashfor exampleto gather information first about a blockrsquos color and thenlater about its precise location within the pattern The au-thors argue that this is a ldquominimal memory strategyrdquo andthey show that it is the strategy most commonly used bysubjects

A few momentsrsquo thought can yield similar examplesfrom daily life Not all of them involve time pressure butother cognitive limitations such as those of attention andworking memory can drive us to a similar kind of off-loading strategy One example used earlier is that ofphysically moving around a room in order to generate so-lutions for where to put furniture Other examples includelaying out the pieces of something that requires assemblyin roughly the order and spatial relationships that they willhave in the finished product or giving directions for howto get somewhere by first turning onersquos self and onersquos lis-tener in the appropriate direction Glenberg and Robertson(1999) have experimentally studied one such exampleshowing that in a compass-and-map task subjects whowere allowed to indexically link written instructions to ob-jects in the environment during a learning phase per-formed better during a test phase than subjects who werenot both on comprehension of new written instructionsand on performance of the actual task

As noted earlier this kind of strategy seems to apply mostusefully to spatial tasks in particular But is off-loadingstrictly limited to cases in which we manipulate spatial in-formation Spatial tasks are only one arena of humanthought If off-loading is useful only for tasks that arethemselves spatial in nature its range of applicability as acognitive strategy is limited

In fact though potential uses of off-loading may be farbroader than this Consider for example such activitiesas counting on onersquos fingers drawing Venn diagrams anddoing math with pencil and paper Many of these activitiesare both situated and spatial in the sense that they involvethe manipulation of spatial relationships among elementsin the environment The advantage is that by doing actualphysical manipulation rather than computing a solution inour heads we save cognitive work However unlike theprevious examples there is also a sense in which these ac-tivities are not situated They are performed in the serviceof cognitive activity about something else something notpresent in the immediate environment

Typically the literature on off-loading has focused oncases in which the world is being used as ldquoits own bestmodelrdquo (Brooks 1991a p 139) Rather than attempt tomentally store and manipulate all the relevant details abouta situation we physically store and manipulate those de-tails out in the world in the very situation itself In theTetris case for example the elements being manipulateddo not serve as tokens for anything but themselves andtheir manipulation does not so much yield informationabout a solution as produce the goal state itself through trialand error In contrast actions such as diagramming repre-sent a quite different use of the environment Here thecognitive system is exploiting external resources to achievea solution or a piece of knowledge whose actual applica-tion will occur at some later time and place if at all

Notice what this buys us This form of off-loadingmdashwhat we might call symbolic off-loadingmdashmay in fact beapplied to spatial tasks as in the case of arranging tokensfor armies on a map but it may also be applied to non-spatial tasks as in the case of using Venn diagrams to de-termine logical relations among categories When the pur-pose of the activity is no longer directly linked to thesituation it also need not be directly linked to spatial prob-lems physical tokens and even their spatial relationshipscan be used to represent abstract nonspatial domains ofthought The history of mathematics attests to the powerbehind this decoupling strategy It should be noted toothat symbolic off-loading need not be deliberate and for-malized but can be seen in such universal and automaticbehaviors as gesturing while speaking It has been found thatgesturing is not epiphenomenal nor even strictly commu-nicative but seems to serve a cognitive function for thespeaker helping to grease the wheels of the thought processthat the speaker is trying to express (see eg Iverson ampGoldin-Meadow 1998 Krauss 1998) As we shall see inSection 6 the use of bodily resources for cognitive pur-poses not directly linked to the situation has potentially farreaching consequences for our understanding of cognitionin general

Claim 4 The Environment Is Partof the Cognitive System

The insight that the body and the environment play a rolein assisting cognitive activity has led some authors to asserta stronger claim that cognition is not an activity of the mind

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

SIX VIEWS OF EMBODIED COGNITION 627

Successful gathering might be expected to benefit a greatdeal from human skills of reflective thoughtmdashremember-ing the terrain coordinating with onersquos fellow gatherersconsidering the probable impact of last weekrsquos rain and soon During the actual act of gathering though it is notclear what situated cognitive skills humans would bring tobear beyond those possessed by any foraging animal (Putin this light we can see that even hunting early humanstyle probably involved considerable nonsituated mentalactivity as well)

In addition to chasing food though being chased bypredators is also supposed to have been a major shapingforce according to this picture of the early human as a sit-uated cognizer Yet while avoiding predators obviously hasa great deal of survival value the situated skills of fight-or-flight are surely ancient shared with many other speciesAgain it is not clear how much mileage can be gotten outof trying to explain human intelligence in these terms In-stead the cognitive abilities that contributed to uniquelyhuman strategies for avoiding predation were probably ofquite a different sort As early humans became increas-ingly sophisticated in their social abilities avoiding pre-dation almost certainly involved increasing use of off-linepreventative and communicative measures

Finally we should consider the mental activities that areknown to have characterized the emerging human popu-lation and that set them apart from earlier hominid speciesThese included increasingly sophisticated tool-makingparticularly the shaping of tools to match a mental tem-plate language allowing communication about hypothet-icals past events and other nonimmediate situations anddepictive art showing the ability to mentally represent whatis not present and to engage in representation for repre-sentationrsquos sake rather than for any situated functionality(see Leakey 1994 for further details) All of these abilitiesreflect the increasingly off-line nature of early humanthought To focus on situated cognition as the fundamen-tal principle of our cognitive architecture is thus to neglectthese species-defining features of human cognition

A few counterarguments to this can be found in the lit-erature Barsalou (1999a) for example suggests that lan-guage was used by early humans primarily for immediatesituated indexical purposes These situated uses of lan-guage were intended to influence the behavior of othersduring activities such as hunting gathering and simplemanufacturing However some of the examples that Barsa-lou gives of situated uses of language appear to be in factoff-line uses where the referent is distant in time or spacemdashas for example in describing distant terrain to people whohave never seen it One can easily think of further nonsitu-ated uses of language that would serve adaptive functionsfor early humans absorbing parental edicts about avoidingdangerous behaviors holding in mind instructions for whatmaterials to go fetch when helping with tool manufactur-ing deciding whether to join in a planned activity such asgoing to the river to cool off and comprehending gossipabout members of the social hierarchy who are not presentIt seems plausible then that language served off-line func-

tions from early on Indeed once the representational ca-pacity of language emerged it is unclear why its full ca-pacity in this respect would not be used

Along different lines Brooks (1999 p 81) argues thatbecause nonsituated cognitive abilities emerged late in thehistory of animal life on this planet after extremely longperiods in which no such innovations appeared these weretherefore the easy problems for evolution to solve (andhence by implication not of much theoretical interest) Infact exactly the opposite can be inferred Easy evolution-ary solutions tend to arise again and again a process knownas convergent evolution In contrast the late emergenceand solitary status of an animal with abilities such as man-ufacturing to a mental template language and artistic de-piction attests to a radical and complex innovation in evo-lutionary engineering

In short an argument for the centrality of situated cog-nition based on the demands of human survival in the wildis not strongly persuasive Furthermore overstating thecase for situated cognition may ultimately impede our un-derstanding of the aspects of cognition that in fact are sit-uated As will be discussed in the next two sections thereis much to be learned about the ways in which we engagein cognitive activity that is tightly connected with our on-going interaction with the environment Spatial cognitionin particular tends to be situated Trying to fit a piece intoa jigsaw puzzle for example may owe more to continuousreevaluating of spatial relationships that are being contin-uously manipulated than it does to any kind of disembod-ied pattern matching (cf Kirsh amp Maglio 1994) For cer-tain kinds of tasks in fact humans may actively choose tosituate themselves (see Section 3)

Claim 2 Cognition is Time PressuredThe previous section considered situated cognition sim-

ply to mean cognition that is situation bound There ap-pears to be more though that is often meant by ldquosituatedcognitionrdquo It is frequently stated that situated agents mustdeal with the constraints of ldquoreal timerdquo or ldquoruntimerdquo (seeeg Brooks 1991b Pfeifer amp Scheier 1999 chap 3 vanGelder amp Port 1995) These phrases are used to highlighta weakness of traditional artificial intelligence modelswhich are generally allowed to build up and manipulateinternal representations of a situation at their leisure Areal creature in a real environment it is pointed out has nosuch leisure It must cope with predators prey stationaryobjects and terrain as fast as the situation dishes them outThe observation that situated cognition takes place ldquoin realtimerdquo is at bottom an observation that situated cognitionmust cope with time pressure

A belief in the importance of time pressure as a shap-ing force in cognitive architecture underlies much of thesituated cognition literature For example in the field ofbehavior-based robotics ldquoautonomous agentsrdquo have beenbuilt to perform tasks such as walking on an uneven sur-face with six legs (Quinn amp Espenschied 1993) brachi-ating or swinging ldquobranch to branchrdquo like an ape (Saito ampFukuda 1994) and navigating around a cluttered envi-

628 WILSON

ronment looking for soda cans without bumping into any-thing (Mataric 1991) Each of these activities requiresreal-time responsiveness to feedback from the environ-ment And although these activities are not especially ldquoin-telligentrdquo in and of themselves it is claimed that greatercognitive complexity can be built up from successive lay-ers of procedures for real-time interaction with the envi-ronment (for reviews see Brooks 1999 Clark 1997Pfeifer amp Scheier 1999)

A similar emphasis on time pressure as a principle thatshapes cognition can be seen as well in human behavioralresearch on situated cognition For example Kirsh andMaglio (1994) have studied the procedures that people usein making time-pressured spatial decisions while playingthe video game Tetris (discussed in more detail in Section 3)This research is conducted with the assumption that situ-ations such as Tetris playing are a microcosm that can elu-cidate general principles of human cognition

One reason that time pressure is thought to matter is thatit creates what has been called a ldquorepresentational bottle-neckrdquo When situations demand fast and continuouslyevolving responses there may simply not be time to buildup a full-blown mental model of the environment fromwhich to derive a plan of action Instead it is argued beinga situated cognizer requires the use of cheap and efficienttricks for generating situation-appropriate action on thefly (In fact a debate has raged over whether a situatedcognizer would make use of internal representations at allsee Agre 1993 Beer 2000 Brooks 1991a Markman ampDietrich 2000 Vera amp Simon 1993) Thus taking real-timesituated action as the starting point for cognitive activityis argued to have far-reaching consequences for cognitivearchitecture

The force of this argument though depends upon theassumption that actual cognizers (humans for example)are indeed engineered so as to circumvent this represen-tational bottleneck and are capable of functioning well andldquonormallyrdquo in time-pressured situations But although onemight wish an ideal cognitive system to have solved theproblem the assumption that we have solved it is dis-putable Confronted with novel cognitive or perceptuo-motor problems humans predictably fall apart under timepressure That is we very often do not successfully copewith the representational bottleneck Lift the demands oftime pressure though and some of the true power ofhuman cognition becomes evident Given the opportunitywe often behave in a decidedly off-line way stepping backobserving assessing planning and only then taking actionIt is far from clear then that the human cognitive systemhas evolved an effective engineering solution for the real-time constraints of the representational bottleneck

Furthermore many of the activities in which we engagein daily life even many that are clearly situated do not in-herently involve time pressure Cases include mundaneactivities such as making sandwiches and paying bills aswell as more demanding cognitive tasks such as doingcrossword puzzles and reading scientific papers In eachof these cases input from and output to the environment

are necessary but they are at the leisure of the cognizer(Of course any task can be performed in a hurry andmany often are But the state of ldquobeing in a hurryrdquo is onethat is cognitively self-imposed and such tasks are gener-ally performed only as fast as they can be even if thismeans being late) Situations in which time pressure is in-herently part of the task such as playing video games or changing lanes in heavy traffic may actually be theexception

This is not to say though that an understanding of real-time interaction with the environment has nothing to con-tribute to our understanding of human cognition A num-ber of important domains may indeed be illuminated byconsidering them from this standpoint The most obviousof these is perceptuomotor coordination of any kind Evensuch basic activities as walking require continuous recip-rocal influence between perceptual flow and motor com-mands Skilled hand movement particularly the manipu-lation of objects in the environment is another persuasiveexample of a time-locked perceptuomotor activity Moresophisticated forms of real-time situated cognition can beseen in any activity that involves continuous updating ofplans in response to rapidly changing conditions Suchchanging conditions often involve the activity of anotherhuman or animal that must be reckoned with Examplesinclude playing a sport driving in traffic and roughhous-ing with a dog As interesting as the principles governingthese cases may be in their own right though the argu-ment that they can be scaled up to provide the governingprinciples of human cognition in general appears to be un-persuasive

Claim 3 We Off-Load Cognitive Work Onto theEnvironment

Despite the fact that we frequently choose to run ourcognitive processes off line it is still true that in some sit-uations we are forced to function on line In those situa-tions what do we do about our cognitive limitations Oneresponse as we have seen is to fall apart However hu-mans are not entirely helpless when confronting the rep-resentational bottleneck and two types of strategies ap-pear to be available when one is confronting on-line taskdemands The first is to rely on preloaded representationsacquired through prior learning (discussed further in Sec-tion 6) What about novel stimuli and tasks though Inthese cases there is a second option which is to reduce thecognitive workload by making use of the environment it-self in strategic waysmdashleaving information out there inthe world to be accessed as needed rather than taking timeto fully encode it and using epistemic actions (Kirsh ampMaglio 1994) to alter the environment in order to reducethe cognitive work remaining to be done

(The environment can also be used as a long-term archiveas in the use of reference books appointment calendarsand computer files This can be thought of as off-loadingto avoid memorizing which is subtly but importantly dif-ferent from off-loading to avoid encoding or holding ac-tive in short-term memory what is present in the immedi-

SIX VIEWS OF EMBODIED COGNITION 629

ate environment It is the latter case that is usually discussedin the literature on off-loading Although the archival casecertainly constitutes off-loading it appears to be of lesstheoretical interest The observation that we use such astrategy does not seem to challenge or shed light on exist-ing theories of cognition The present discussion will there-fore be restricted to what we may call the situated exam-ples of off-loading which are the focus of the literature)

Some investigators have begun to examine how off-loading work onto the environment may be used as a cog-nitive strategy Kirsh and Maglio (1994) as noted earlierhave reported a study involving the game Tetris in whichfalling block shapes must be rotated and horizontallytranslated to fit as compactly as possible with the shapesthat have already fallen The decision of how to orient andplace each block must be made before the block falls toofar to allow the necessary movements The data suggestthat players use actual rotation and translation movementsto simplify the problem to be solved rather than mentallycomputing a solution and then executing it A second ex-ample comes from Ballard Hayhoe Pook and Rao (1997)who asked subjects to reproduce patterns of colored blocksunder time pressure by dragging randomly scattered blockson a computer screen into a work area and arranging themthere Recorded eye movements showed repeated refer-encing of the blocks in the model pattern and these eyemovements occurred at strategic momentsmdashfor exampleto gather information first about a blockrsquos color and thenlater about its precise location within the pattern The au-thors argue that this is a ldquominimal memory strategyrdquo andthey show that it is the strategy most commonly used bysubjects

A few momentsrsquo thought can yield similar examplesfrom daily life Not all of them involve time pressure butother cognitive limitations such as those of attention andworking memory can drive us to a similar kind of off-loading strategy One example used earlier is that ofphysically moving around a room in order to generate so-lutions for where to put furniture Other examples includelaying out the pieces of something that requires assemblyin roughly the order and spatial relationships that they willhave in the finished product or giving directions for howto get somewhere by first turning onersquos self and onersquos lis-tener in the appropriate direction Glenberg and Robertson(1999) have experimentally studied one such exampleshowing that in a compass-and-map task subjects whowere allowed to indexically link written instructions to ob-jects in the environment during a learning phase per-formed better during a test phase than subjects who werenot both on comprehension of new written instructionsand on performance of the actual task

As noted earlier this kind of strategy seems to apply mostusefully to spatial tasks in particular But is off-loadingstrictly limited to cases in which we manipulate spatial in-formation Spatial tasks are only one arena of humanthought If off-loading is useful only for tasks that arethemselves spatial in nature its range of applicability as acognitive strategy is limited

In fact though potential uses of off-loading may be farbroader than this Consider for example such activitiesas counting on onersquos fingers drawing Venn diagrams anddoing math with pencil and paper Many of these activitiesare both situated and spatial in the sense that they involvethe manipulation of spatial relationships among elementsin the environment The advantage is that by doing actualphysical manipulation rather than computing a solution inour heads we save cognitive work However unlike theprevious examples there is also a sense in which these ac-tivities are not situated They are performed in the serviceof cognitive activity about something else something notpresent in the immediate environment

Typically the literature on off-loading has focused oncases in which the world is being used as ldquoits own bestmodelrdquo (Brooks 1991a p 139) Rather than attempt tomentally store and manipulate all the relevant details abouta situation we physically store and manipulate those de-tails out in the world in the very situation itself In theTetris case for example the elements being manipulateddo not serve as tokens for anything but themselves andtheir manipulation does not so much yield informationabout a solution as produce the goal state itself through trialand error In contrast actions such as diagramming repre-sent a quite different use of the environment Here thecognitive system is exploiting external resources to achievea solution or a piece of knowledge whose actual applica-tion will occur at some later time and place if at all

Notice what this buys us This form of off-loadingmdashwhat we might call symbolic off-loadingmdashmay in fact beapplied to spatial tasks as in the case of arranging tokensfor armies on a map but it may also be applied to non-spatial tasks as in the case of using Venn diagrams to de-termine logical relations among categories When the pur-pose of the activity is no longer directly linked to thesituation it also need not be directly linked to spatial prob-lems physical tokens and even their spatial relationshipscan be used to represent abstract nonspatial domains ofthought The history of mathematics attests to the powerbehind this decoupling strategy It should be noted toothat symbolic off-loading need not be deliberate and for-malized but can be seen in such universal and automaticbehaviors as gesturing while speaking It has been found thatgesturing is not epiphenomenal nor even strictly commu-nicative but seems to serve a cognitive function for thespeaker helping to grease the wheels of the thought processthat the speaker is trying to express (see eg Iverson ampGoldin-Meadow 1998 Krauss 1998) As we shall see inSection 6 the use of bodily resources for cognitive pur-poses not directly linked to the situation has potentially farreaching consequences for our understanding of cognitionin general

Claim 4 The Environment Is Partof the Cognitive System

The insight that the body and the environment play a rolein assisting cognitive activity has led some authors to asserta stronger claim that cognition is not an activity of the mind

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

628 WILSON

ronment looking for soda cans without bumping into any-thing (Mataric 1991) Each of these activities requiresreal-time responsiveness to feedback from the environ-ment And although these activities are not especially ldquoin-telligentrdquo in and of themselves it is claimed that greatercognitive complexity can be built up from successive lay-ers of procedures for real-time interaction with the envi-ronment (for reviews see Brooks 1999 Clark 1997Pfeifer amp Scheier 1999)

A similar emphasis on time pressure as a principle thatshapes cognition can be seen as well in human behavioralresearch on situated cognition For example Kirsh andMaglio (1994) have studied the procedures that people usein making time-pressured spatial decisions while playingthe video game Tetris (discussed in more detail in Section 3)This research is conducted with the assumption that situ-ations such as Tetris playing are a microcosm that can elu-cidate general principles of human cognition

One reason that time pressure is thought to matter is thatit creates what has been called a ldquorepresentational bottle-neckrdquo When situations demand fast and continuouslyevolving responses there may simply not be time to buildup a full-blown mental model of the environment fromwhich to derive a plan of action Instead it is argued beinga situated cognizer requires the use of cheap and efficienttricks for generating situation-appropriate action on thefly (In fact a debate has raged over whether a situatedcognizer would make use of internal representations at allsee Agre 1993 Beer 2000 Brooks 1991a Markman ampDietrich 2000 Vera amp Simon 1993) Thus taking real-timesituated action as the starting point for cognitive activityis argued to have far-reaching consequences for cognitivearchitecture

The force of this argument though depends upon theassumption that actual cognizers (humans for example)are indeed engineered so as to circumvent this represen-tational bottleneck and are capable of functioning well andldquonormallyrdquo in time-pressured situations But although onemight wish an ideal cognitive system to have solved theproblem the assumption that we have solved it is dis-putable Confronted with novel cognitive or perceptuo-motor problems humans predictably fall apart under timepressure That is we very often do not successfully copewith the representational bottleneck Lift the demands oftime pressure though and some of the true power ofhuman cognition becomes evident Given the opportunitywe often behave in a decidedly off-line way stepping backobserving assessing planning and only then taking actionIt is far from clear then that the human cognitive systemhas evolved an effective engineering solution for the real-time constraints of the representational bottleneck

Furthermore many of the activities in which we engagein daily life even many that are clearly situated do not in-herently involve time pressure Cases include mundaneactivities such as making sandwiches and paying bills aswell as more demanding cognitive tasks such as doingcrossword puzzles and reading scientific papers In eachof these cases input from and output to the environment

are necessary but they are at the leisure of the cognizer(Of course any task can be performed in a hurry andmany often are But the state of ldquobeing in a hurryrdquo is onethat is cognitively self-imposed and such tasks are gener-ally performed only as fast as they can be even if thismeans being late) Situations in which time pressure is in-herently part of the task such as playing video games or changing lanes in heavy traffic may actually be theexception

This is not to say though that an understanding of real-time interaction with the environment has nothing to con-tribute to our understanding of human cognition A num-ber of important domains may indeed be illuminated byconsidering them from this standpoint The most obviousof these is perceptuomotor coordination of any kind Evensuch basic activities as walking require continuous recip-rocal influence between perceptual flow and motor com-mands Skilled hand movement particularly the manipu-lation of objects in the environment is another persuasiveexample of a time-locked perceptuomotor activity Moresophisticated forms of real-time situated cognition can beseen in any activity that involves continuous updating ofplans in response to rapidly changing conditions Suchchanging conditions often involve the activity of anotherhuman or animal that must be reckoned with Examplesinclude playing a sport driving in traffic and roughhous-ing with a dog As interesting as the principles governingthese cases may be in their own right though the argu-ment that they can be scaled up to provide the governingprinciples of human cognition in general appears to be un-persuasive

Claim 3 We Off-Load Cognitive Work Onto theEnvironment

Despite the fact that we frequently choose to run ourcognitive processes off line it is still true that in some sit-uations we are forced to function on line In those situa-tions what do we do about our cognitive limitations Oneresponse as we have seen is to fall apart However hu-mans are not entirely helpless when confronting the rep-resentational bottleneck and two types of strategies ap-pear to be available when one is confronting on-line taskdemands The first is to rely on preloaded representationsacquired through prior learning (discussed further in Sec-tion 6) What about novel stimuli and tasks though Inthese cases there is a second option which is to reduce thecognitive workload by making use of the environment it-self in strategic waysmdashleaving information out there inthe world to be accessed as needed rather than taking timeto fully encode it and using epistemic actions (Kirsh ampMaglio 1994) to alter the environment in order to reducethe cognitive work remaining to be done

(The environment can also be used as a long-term archiveas in the use of reference books appointment calendarsand computer files This can be thought of as off-loadingto avoid memorizing which is subtly but importantly dif-ferent from off-loading to avoid encoding or holding ac-tive in short-term memory what is present in the immedi-

SIX VIEWS OF EMBODIED COGNITION 629

ate environment It is the latter case that is usually discussedin the literature on off-loading Although the archival casecertainly constitutes off-loading it appears to be of lesstheoretical interest The observation that we use such astrategy does not seem to challenge or shed light on exist-ing theories of cognition The present discussion will there-fore be restricted to what we may call the situated exam-ples of off-loading which are the focus of the literature)

Some investigators have begun to examine how off-loading work onto the environment may be used as a cog-nitive strategy Kirsh and Maglio (1994) as noted earlierhave reported a study involving the game Tetris in whichfalling block shapes must be rotated and horizontallytranslated to fit as compactly as possible with the shapesthat have already fallen The decision of how to orient andplace each block must be made before the block falls toofar to allow the necessary movements The data suggestthat players use actual rotation and translation movementsto simplify the problem to be solved rather than mentallycomputing a solution and then executing it A second ex-ample comes from Ballard Hayhoe Pook and Rao (1997)who asked subjects to reproduce patterns of colored blocksunder time pressure by dragging randomly scattered blockson a computer screen into a work area and arranging themthere Recorded eye movements showed repeated refer-encing of the blocks in the model pattern and these eyemovements occurred at strategic momentsmdashfor exampleto gather information first about a blockrsquos color and thenlater about its precise location within the pattern The au-thors argue that this is a ldquominimal memory strategyrdquo andthey show that it is the strategy most commonly used bysubjects

A few momentsrsquo thought can yield similar examplesfrom daily life Not all of them involve time pressure butother cognitive limitations such as those of attention andworking memory can drive us to a similar kind of off-loading strategy One example used earlier is that ofphysically moving around a room in order to generate so-lutions for where to put furniture Other examples includelaying out the pieces of something that requires assemblyin roughly the order and spatial relationships that they willhave in the finished product or giving directions for howto get somewhere by first turning onersquos self and onersquos lis-tener in the appropriate direction Glenberg and Robertson(1999) have experimentally studied one such exampleshowing that in a compass-and-map task subjects whowere allowed to indexically link written instructions to ob-jects in the environment during a learning phase per-formed better during a test phase than subjects who werenot both on comprehension of new written instructionsand on performance of the actual task

As noted earlier this kind of strategy seems to apply mostusefully to spatial tasks in particular But is off-loadingstrictly limited to cases in which we manipulate spatial in-formation Spatial tasks are only one arena of humanthought If off-loading is useful only for tasks that arethemselves spatial in nature its range of applicability as acognitive strategy is limited

In fact though potential uses of off-loading may be farbroader than this Consider for example such activitiesas counting on onersquos fingers drawing Venn diagrams anddoing math with pencil and paper Many of these activitiesare both situated and spatial in the sense that they involvethe manipulation of spatial relationships among elementsin the environment The advantage is that by doing actualphysical manipulation rather than computing a solution inour heads we save cognitive work However unlike theprevious examples there is also a sense in which these ac-tivities are not situated They are performed in the serviceof cognitive activity about something else something notpresent in the immediate environment

Typically the literature on off-loading has focused oncases in which the world is being used as ldquoits own bestmodelrdquo (Brooks 1991a p 139) Rather than attempt tomentally store and manipulate all the relevant details abouta situation we physically store and manipulate those de-tails out in the world in the very situation itself In theTetris case for example the elements being manipulateddo not serve as tokens for anything but themselves andtheir manipulation does not so much yield informationabout a solution as produce the goal state itself through trialand error In contrast actions such as diagramming repre-sent a quite different use of the environment Here thecognitive system is exploiting external resources to achievea solution or a piece of knowledge whose actual applica-tion will occur at some later time and place if at all

Notice what this buys us This form of off-loadingmdashwhat we might call symbolic off-loadingmdashmay in fact beapplied to spatial tasks as in the case of arranging tokensfor armies on a map but it may also be applied to non-spatial tasks as in the case of using Venn diagrams to de-termine logical relations among categories When the pur-pose of the activity is no longer directly linked to thesituation it also need not be directly linked to spatial prob-lems physical tokens and even their spatial relationshipscan be used to represent abstract nonspatial domains ofthought The history of mathematics attests to the powerbehind this decoupling strategy It should be noted toothat symbolic off-loading need not be deliberate and for-malized but can be seen in such universal and automaticbehaviors as gesturing while speaking It has been found thatgesturing is not epiphenomenal nor even strictly commu-nicative but seems to serve a cognitive function for thespeaker helping to grease the wheels of the thought processthat the speaker is trying to express (see eg Iverson ampGoldin-Meadow 1998 Krauss 1998) As we shall see inSection 6 the use of bodily resources for cognitive pur-poses not directly linked to the situation has potentially farreaching consequences for our understanding of cognitionin general

Claim 4 The Environment Is Partof the Cognitive System

The insight that the body and the environment play a rolein assisting cognitive activity has led some authors to asserta stronger claim that cognition is not an activity of the mind

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

SIX VIEWS OF EMBODIED COGNITION 629

ate environment It is the latter case that is usually discussedin the literature on off-loading Although the archival casecertainly constitutes off-loading it appears to be of lesstheoretical interest The observation that we use such astrategy does not seem to challenge or shed light on exist-ing theories of cognition The present discussion will there-fore be restricted to what we may call the situated exam-ples of off-loading which are the focus of the literature)

Some investigators have begun to examine how off-loading work onto the environment may be used as a cog-nitive strategy Kirsh and Maglio (1994) as noted earlierhave reported a study involving the game Tetris in whichfalling block shapes must be rotated and horizontallytranslated to fit as compactly as possible with the shapesthat have already fallen The decision of how to orient andplace each block must be made before the block falls toofar to allow the necessary movements The data suggestthat players use actual rotation and translation movementsto simplify the problem to be solved rather than mentallycomputing a solution and then executing it A second ex-ample comes from Ballard Hayhoe Pook and Rao (1997)who asked subjects to reproduce patterns of colored blocksunder time pressure by dragging randomly scattered blockson a computer screen into a work area and arranging themthere Recorded eye movements showed repeated refer-encing of the blocks in the model pattern and these eyemovements occurred at strategic momentsmdashfor exampleto gather information first about a blockrsquos color and thenlater about its precise location within the pattern The au-thors argue that this is a ldquominimal memory strategyrdquo andthey show that it is the strategy most commonly used bysubjects

A few momentsrsquo thought can yield similar examplesfrom daily life Not all of them involve time pressure butother cognitive limitations such as those of attention andworking memory can drive us to a similar kind of off-loading strategy One example used earlier is that ofphysically moving around a room in order to generate so-lutions for where to put furniture Other examples includelaying out the pieces of something that requires assemblyin roughly the order and spatial relationships that they willhave in the finished product or giving directions for howto get somewhere by first turning onersquos self and onersquos lis-tener in the appropriate direction Glenberg and Robertson(1999) have experimentally studied one such exampleshowing that in a compass-and-map task subjects whowere allowed to indexically link written instructions to ob-jects in the environment during a learning phase per-formed better during a test phase than subjects who werenot both on comprehension of new written instructionsand on performance of the actual task

As noted earlier this kind of strategy seems to apply mostusefully to spatial tasks in particular But is off-loadingstrictly limited to cases in which we manipulate spatial in-formation Spatial tasks are only one arena of humanthought If off-loading is useful only for tasks that arethemselves spatial in nature its range of applicability as acognitive strategy is limited

In fact though potential uses of off-loading may be farbroader than this Consider for example such activitiesas counting on onersquos fingers drawing Venn diagrams anddoing math with pencil and paper Many of these activitiesare both situated and spatial in the sense that they involvethe manipulation of spatial relationships among elementsin the environment The advantage is that by doing actualphysical manipulation rather than computing a solution inour heads we save cognitive work However unlike theprevious examples there is also a sense in which these ac-tivities are not situated They are performed in the serviceof cognitive activity about something else something notpresent in the immediate environment

Typically the literature on off-loading has focused oncases in which the world is being used as ldquoits own bestmodelrdquo (Brooks 1991a p 139) Rather than attempt tomentally store and manipulate all the relevant details abouta situation we physically store and manipulate those de-tails out in the world in the very situation itself In theTetris case for example the elements being manipulateddo not serve as tokens for anything but themselves andtheir manipulation does not so much yield informationabout a solution as produce the goal state itself through trialand error In contrast actions such as diagramming repre-sent a quite different use of the environment Here thecognitive system is exploiting external resources to achievea solution or a piece of knowledge whose actual applica-tion will occur at some later time and place if at all

Notice what this buys us This form of off-loadingmdashwhat we might call symbolic off-loadingmdashmay in fact beapplied to spatial tasks as in the case of arranging tokensfor armies on a map but it may also be applied to non-spatial tasks as in the case of using Venn diagrams to de-termine logical relations among categories When the pur-pose of the activity is no longer directly linked to thesituation it also need not be directly linked to spatial prob-lems physical tokens and even their spatial relationshipscan be used to represent abstract nonspatial domains ofthought The history of mathematics attests to the powerbehind this decoupling strategy It should be noted toothat symbolic off-loading need not be deliberate and for-malized but can be seen in such universal and automaticbehaviors as gesturing while speaking It has been found thatgesturing is not epiphenomenal nor even strictly commu-nicative but seems to serve a cognitive function for thespeaker helping to grease the wheels of the thought processthat the speaker is trying to express (see eg Iverson ampGoldin-Meadow 1998 Krauss 1998) As we shall see inSection 6 the use of bodily resources for cognitive pur-poses not directly linked to the situation has potentially farreaching consequences for our understanding of cognitionin general

Claim 4 The Environment Is Partof the Cognitive System

The insight that the body and the environment play a rolein assisting cognitive activity has led some authors to asserta stronger claim that cognition is not an activity of the mind

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

630 WILSON

alone but is instead distributed across the entire interact-ing situation including mind body and environment (seeeg Beer 1995 pp 182ndash183 Greeno amp Moore 1993p 49 Thelen amp Smith 1994 p 17 Wertsch 1998 p 518see also Clark 1998 pp 513ndash516 for discussion) In factrelatively few theorists appear to hold consistently to thisposition in its strong form Nevertheless an attraction tosomething like this claim permeates the literatures on em-bodied and situated cognition It is therefore worth it to bringthe core idea into focus and consider it in some detail

The claim is this The forces that drive cognitive activ-ity do not reside solely inside the head of the individualbut instead are distributed across the individual and thesituation as they interact Therefore to understand cogni-tion we must study the situation and the situated cognizertogether as a single unified system

The first part of this claim is trivially true Causes ofbehavior (and also causes of covert cognitive events suchas thoughts) are surely distributed across the mind plus en-vironment More problematic is the reasoning that con-nects the first part of the claim with the second part Thefact that causal control is distributed across the situationis not sufficient justification for the claim that we muststudy a distributed system Science is not ultimately aboutexplaining the causality of any particular event Instead itis about understanding fundamental principles of organi-zation and function

Consider for example the goal of understanding hydro-gen Before 1900 hydrogen had been observed by scientistsin a large number of contexts and much was known aboutits behavior when it interacted with other chemicals Butnone of this behavior was really understood until the dis-covery in the 20th century of the structure of the atom in-cluding the protons neutrons and electrons that are itscomponents and the discrete orbits that electrons inhabitOnce this was known not only did all the previous obser-vations of hydrogen make sense but the behavior of hy-drogen could be predicted in interactions with elementsnever yet observed The causes of the behavior of hydrogenare always a combination of the nature of hydrogen plusthe specifics of its surrounding context yet explanatorysatisfaction came from understanding the workings of thenarrowly defined system that is the hydrogen atom To haveinsisted that we focus on the study of contextualized be-havior would probably not have led to a theoretical under-standing with anything like this kind of explanatory force

Distributed causality then is not sufficient to drive anargument for distributed cognition Instead we must askwhat kind of system we are interested in studying To an-swer this we must consider the meaning of the word sys-tem as it is being used here For this purpose the contri-butions of systems theorists will be of help (For a lucidsummary of the issues discussed below see Juarrero 1999chap 7)

For a set of things to be considered a system in the for-mal sense these things must be not merely an aggregatea collection of elements that stand in some relation to oneanother (spatial temporal or any other relation) The ele-

ments must in addition have properties that are affectedby their participation in the system Thus the various partsof an automobile can be considered as a system becausethe action of the spark plugs affects the behavior of thepistons the pistons affect the drive shaft and so on

But must all things that have an impact on the elementsof a system themselves be considered part of the systemNo Many systems are open systems existing within thecontext of an environment that can affect and be affectedby the system (No system short of the entire universe istruly closed although some can be considered closed forpractical purposes) Thus for example an ecological regionon earth can be considered a system in that the organismsin that region are integrally dependent on one another butthe sun need not be considered part of the system nor therivers that flow in from elsewhere even though their inputis vital to the ecological system Instead the ecologicalsystem can be considered an open system receiving inputfrom something outside itself The fact that open systemsare open is not generally considered a problem for theiranalysis even when mutual influence with external forcesis continuous

From this description though it should be clear that howone defines the boundaries of a system is partly a matterof judgment and depends on the particular purposes ofonersquos analysis Thus the sun may not be part of the systemwhen one considers the earth in biological terms but it ismost definitely part of the system when one considers theearth in terms of planetary movement The issue for anygiven scientific enterprise is how best to carve nature atits joints

Where does this leave us with respect to defining a cog-nitive system Is it most natural most scientifically pro-ductive to consider the system to be the mind or the mindthe body and certain relevant elements in the immediatephysical environment all taken together To help us an-swer this question it will be useful to introduce a few addi-tional concepts regarding systems and how they function

First a system is defined by its organizationmdashthat isthe functional relations among its elements These rela-tions cannot be changed without changing the identity ofthe system Next systems can be described as either fac-ultativeor obligateFacultative systems are temporary or-ganized for a particular occasion and disbanded readilyObligate systems on the other hand are more or less per-manent at least relative to the lifetime of their parts

We are now in a position to make a few observationsabout a ldquocognitive systemrdquo that is distributed across thesituation The organization of such a systemmdashthe functionalrelations among its elements and indeed the constitutiveelements themselvesmdashwould change every time the per-son moves to a new location or begins interacting with adifferent set of objects That is the system would retain itsidentity only so long as the situation and the personrsquos taskorientation toward that situation did not change Such asystem would clearly be a facultative system and faculta-tive systems like this would arise and disband rapidly andcontinuously during the daily life of the individual person

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

SIX VIEWS OF EMBODIED COGNITION 631

The distributed view of cognition thus trades off the ob-ligate nature of the system in order to buy a system that ismore or less closed

If on the other hand we restrict the system to includeonly the cognitive architecture of the individual mind orbrain we are dealing with a single persisting obligate sys-tem The various components of the systemrsquos organizationmdashperceptual mechanisms attentional filters working mem-ory stores and so onmdashretain their functional roles withinthat system across time The system is undeniably openwith respect to its environment continuously receivinginput that affects the systemrsquos functioning and producingoutput that has consequences for the environmentrsquos fur-ther impact on the system itself But as in the case of hy-drogen or an ecosystem this characteristic of opennessdoes not compromise the systemrsquos status as a systemGiven this analysis it seems clear that a strong view of dis-tributed cognitionmdashthat a cognitive system cannot in prin-ciple be taken to comprise only an individual mindmdashwillnot hold up

Of course we can reject this strong version of distrib-uted cognition and still accept a weaker version in whichstudying the mind-plus-situation is considered to be apromising supplementary avenue of investigation in ad-dition to studying the mind per se Two points should benoted though First taken in this spirit the idea of dis-tributed cognition loses much of its radical cachet Thisview does not seek to revolutionize the field of cognitivescience but simply adds to the list of phenomena that thefield studies Likewise chaos theory did not revolutionizeor overturn our understanding of physics but simply pro-vided an additional tool that helped to broaden the rangeof phenomena that physics could characterize success-fully (Indeed some examples of research on distributedtopics appear to stretch the bounds of what we would rec-ognize as cognition at all The study of the organized be-havior of groups is one such example see eg Hutchins1995)

Second it remains to be seen whether in the long runa distributed approach can provide deep and satisfying in-sights into the nature of cognition If we recall that the goalof science is to find underlying principles and regularitiesrather than to explain specific events then the facultativenature of distributed cognition becomes a problem Whetherthis problem can be overcome to arrive at theoretical in-sights with explanatory power is an issue that awaits proof

Claim 5 Cognition Is for ActionMore broadly than the stringent criteria for situated

cognition the embodied cognition approach leads us toconsider cognitive mechanisms in terms of their functionin serving adaptive activity (see eg Franklin 1995chap 16) The claim that cognition is for action has gainedmomentum from work in perception and memory in par-ticular ldquoVisionrdquo according to Churchland Ramachandranand Sejnowski (1994) ldquohas its evolutionary rationale rootedin improved motor controlrdquo (p 25 see also Ballard 1996OrsquoRegan 1992 Pessoa Thompson amp Noeuml 1998) ldquoMem-

oryrdquo as Glenberg (1997) similarly argues ldquoevolved in ser-vice of perception and action in a three-dimensional envi-ronmentrdquo (p 1)

First let us consider the case of visual perception Thetraditional assumption has been that the purpose of the vi-sual system is to build up an internal representation of theperceived world What is to be done with this representa-tion is then the job of ldquohigherrdquo cognitive areas In keepingwith this approach the ventral and dorsal visual pathwaysin the brain have been thought of as the ldquowhatrdquo and ldquowhererdquopathways generating representations of object structureand spatial relationships respectively In the past decadethough it has been argued that the dorsal stream is moreproperly thought of as a ldquohowrdquo pathway The proposedfunction of this pathway is to serve visually guided actionssuch as reaching and grasping (for reviews see Goodale ampMilner 1992 Jeannerod 1997)

In support of this it has been found that certain kindsof visual input can actually prime motor activity For ex-ample seeing a rectangle of a particular orientation facil-itates performance on a subsequent grasping task pro-vided that the object to be grasped shares that orientation(Craighero Fadiga Umiltagrave amp Rizzolatti 1996) This prim-ing occurs even when the orientation of the rectangle doesnot reliably predict the orientation of the object to begrasped A striking corollary is that visual input can acti-vate covert motor representations in the absence of anytask demands Certain motor neurons in monkeys that areinvolved in controlling tool use also respond to seen toolswithout any motor response on the part of the subject(Grafton Fadiga Arbib amp Rizzolatti 1997 Murata et al1997) Behavioral data reported by Tucker and Ellis (1998)tell a similar story When subjects indicate whether commonobjects (eg a teapot a frying pan) are upright or invertedresponse times are fastest when the response hand is thesame as the hand that would be used to grasp the depictedobject (eg the left hand if the teapotrsquos handle is on the left)

A similar proposal has been advanced for the nature ofmemory storage Glenberg (1997) argues that the traditionalapproach to memory as ldquofor memorizingrdquo needs to be re-placed by a view of memory as ldquothe encoding of patternsof possible physical interaction with a three-dimensionalworldrdquo (p 1) Glenberg seeks to explain a variety of mem-ory phenomena in terms of such perceptuomotor patternsShort-term memory for example is seen not as a distinctmemory ldquosystemrdquo but as the deployment of particular ac-tion skills such as those involved in verbal rehearsal Se-mantic memory and the formation of concepts are simi-larly explained in terms of embodied memory patternsdiffering from episodic memory only in frequency of thepatternrsquos use across many situations

This approach to memory helps make sense of a vari-ety of observations formal and informal that we concep-tualize objects and situations in terms of their functionalrelevance to us rather than neutrally or ldquoas they reallyarerdquo These observations range from laboratory experi-ments on encoding specificity and functional fixedness tothe quip attributed to Maslow that when all you have is a

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

632 WILSON

hammer everything looks like a nail to the fancifulUmwelt drawings of Uexkuumlll (1934 reprints can be foundin Clark 1997) showing what the environment might looklike to creatures with different cognitive agendas Our un-derstanding of the ldquohowrdquo system of vision suggests howthis type of embodied memory might work As we haveseen from the work on priming of motor activity the vi-sual system can engage motor functions without resultingin immediate overt action This is precisely the kind ofmechanism that would be needed to create the perceptuo-motor patterning that Glenberg argues comprises the con-tents of memory

The question we must ask though is how far this viewof perception memory and cognition in general can takeus Can we dispense entirely with representation for rep-resentationrsquos sake neutral with respect to a specific pur-pose or action We need not look far for evidence suggest-ing that we cannot To begin with although the ldquohowrdquosystemof perceptual processing appears to be for action the veryexistence of the ldquowhatrdquo system suggests that not all infor-mation encoding works this way The ventral stream of vi-sual processing does not appear to have the same kinds ofdirect links to the motor system that the dorsal streamdoes Instead the ventral stream goes about identifyingpatterns and objects apparently engaging in perceptionfor perceptionrsquos sake This point is driven home if we con-sider some of the things that this system is asked to en-code First there are visual events such as sunsets thatare always perceived at a distance and do not offer any op-portunity for physical interaction (cf Slater 1997) Sec-ond there are objects whose recognition depends on holis-tic visual appearance rather than on aspects of physicalstructure that offer opportunities for perceptuomotor inter-action Human faces are the showcase example here al-though the same point can be make for recognizing indi-viduals of other categories such as dogs or houses Thirdthere is the case of reading where sheer visual patternrecognition is paramount and opportunities for physicalinteraction with those patterns are virtually nil Thus per-ceptual encoding cannot be accounted for entirely in termsof direct perception-for-action processing channels

The problems get worse when we look beyond percep-tual processing to some of the broader functions of mem-ory Mental concepts for example do not always or evenusually follow physical concrete properties that lendthemselves to action but instead often involve intangibleproperties based on folk-scientific theories or knowledgeof causal history (see eg Keil 1989 Putnam 1970 Rips1989) A classic example is that a mutilated dollar bill isstill a dollar bill but a counterfeit dollar bill is not Simi-larly cheddar cheese is understood to be a dairy productbut soy milk which more closely resembles milk in itsperceptual qualities and action affordances is not

In an ultimate sense it must be true that cognition is foraction Adaptive behavior that promotes survival clearlymust have driven the evolution of our cognitive architec-ture The question though is the following In what wayor ways does our cognitive architecture subserve action

The answer being critiqued here is that the connections toaction are quite direct Individual percepts concepts andmemories are ldquoforrdquo (or are based on) particular action pat-terns The evidence discussed above though suggests thatthis is unlikely to hold true across the board An alterna-tive view is that cognition often subserves action via amore indirect flexible and sophisticated strategy in whichinformation about the nature of the external world isstored for future use without strong commitments on whatthat future use might be

In support of this we can note that our mental conceptsoften contain rich information about the properties of ob-jects information that can be drawn on for a variety ofuses that almost certainly were not originally encoded forWe are in fact capable of breaking out of functional fixed-ness and do so regularly Thus I can notice a piano in anunfamiliar room and being a nonmusician I might thinkof it only as having a bench I can sit on and flat surfaces Ican set my drink on But I can also later call up my knowl-edge of the piano in a variety of unforeseen circumstancesif I need to make a loud noise to get everyonersquos attentionif the door needs to be barricaded against intruders or ifwe are caught in a blizzard without power and need tosmash up some furniture for fuel Notice that these noveluses can be derived from a stored representation of thepiano They need not be triggered by direct observation ofthe piano and its affordances while one is entertaining anew action-based goal

It is true that our mental representations are oftensketchy and incomplete particularly for things that wehave encountered only once and briefly The literature onchange blindness which shows that people can entirelymiss major changes to a scene across very brief time lagsmakes this point forcefully (see Simons amp Levin 1997for a review) But the fact that we are limited in how muchwe can attend to and absorb in a single brief encounterdoes not alter the fact that we can and do build up robustdetailed representations with repeated exposure Further-more it is unclear that the sketchiness of a representationwould prevent it from being a ldquorepresentation for repre-sentationrsquos sakerdquo Our mental representations whethernovel and sketchy or familiar and detailed appear to be toa large extent purpose-neutral or at least to contain infor-mation beyond that needed for the originally conceivedpurpose And this is arguably an adaptive cognitive strat-egy A creature that encodes the world using more or lessveridical mental models has an enormous advantage inproblem-solving flexibility over a creature that encodespurely in terms of presently foreseeable activities

Claim 6 Off-Line Cognition Is Body BasedLet us return now to the kinds of externalized cognitive

activities described in Section 3 in which we manipulatethe environment to help us think about a problem Con-sider the example of counting on onersquos fingers In its fullestform this can be a set of crisp and large movements un-ambiguously setting forth the different fingers as coun-ters But it can also be done more subtly differentiating

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

SIX VIEWS OF EMBODIED COGNITION 633

the positions of the fingers only enough to allow the ownerof the fingers to keep track To the observer this mightlook like mere twitching Imagine then that we push theactivity inward still further allowing only the priming ofmotor programs but no overt movement If this kind ofmental activity can be employed successfully to assist atask such as counting a new vista of cognitive strategiesopens up

Many centralized allegedly abstract cognitive activitiesmay in fact make use of sensorimotor functions in exactlythis kind of covert way Mental structures that originallyevolved for perception or action appear to be co-opted andrun ldquooff-linerdquo decoupled from the physical inputs andoutputs that were their original purpose to assist in think-ing and knowing (Several authors have proposed mecha-nisms by which this decoupling might take place Dennett1995 chap 13 Glenberg 1997 Grush 1996 1998 Stein1994) In general the function of these sensorimotor re-sources is to run a simulation of some aspect of the physi-cal world as a means of representing information or draw-ing inferences

Although this off-line aspect of embodied cognition hasgenerated less attention than situated cognition evidencein its favor has been mounting quietly for many yearsSensorimotor simulations of external situations are in factwidely implicated in human cognition

Mental imagery Imagery including not only the well-studied case of visual imagery but also those of auditoryimagery (Reisberg 1992) and kinesthetic imagery (Par-sons et al 1995) is an obvious example of mentally sim-ulating external events It is a commentary on the histori-cal strength of the nonembodied viewpoint then thatduring the 1980s the study of imagery was dominated bya debate over whether images were in fact image-like inany meaningful sense An elaborate defense had to bemounted to show that imagery involves analogue repre-sentations that functionally preserve spatial and otherproperties of the external world rather than consisting ofbundles of propositions (see Kosslyn 1994 for a review)Today this issue has been firmly resolved in favor of theanalogue nature of images and evidence continues tomount for a close connection between imagery whichtakes place in the absence of relevant external stimulationand the machinery of ordinary perception (see eg Farah1995 Kosslyn Pascual-Leone Felician amp Camposano1999)

Working memory A second example of simulatingphysical events through the off-line use of sensorimotorresources is short-term memory Early models referredabstractly to ldquoitemsrdquo maintained temporarily in memoryBaddeley and Hitch (1974 Baddeley 1986) howeverbuilt a persuasive case for a multicomponent workingmemory system that had separate storage components forverbal and for visuospatial information each of whichwas coded and maintained in something resembling itssurface form The particulars of the Baddeley model havebeen challenged on a variety of grounds but as I have ar-gued elsewhere some version of a sensorimotor model

appears to be the only viable way to account for the largebody of data on working memory (Wilson 2001a) Earlyevidence for the sensorimotor nature of working memoryincluded effects of phonological similarity (worse memoryfor words that sound alike) word length (worse memory forlong words) and articulatory suppression (worse memorywhen the relevant articulatory muscles are kept busy withanother activity such as repeating a nonsense word) Morerecently a similar set of effects but in a different sensori-motor modality has been found for working memory forsign language in deaf subjects Performance drops whento-be-remembered signs have similar hand shapes or aretemporally long or when subjects are required to perform arepetitive movement with their hands (Wilson amp Emmorey1997 1998) Furthermore research on patient popula-tions and brain imaging of normals indicates the involve-ment of speech perception and speech production areas ofthe brain in working memory rehearsal (see Wilson2001a for a review) Thus working memory appears to bean example of a kind of symbolic off-loading similar inspirit to that discussed in Section 3 However instead ofoff-loading all the way out into the environment workingmemory off-loads information onto perceptual and motorcontrol systems in the brain

Episodic memory Long-term memory too is tied incertain ways to our bodiesrsquo experiences with the worldThe point is most obvious in the case of episodic memoryWhether or not one posits a separate episodic memory sys-tem episodic memories are a class of memories defined bytheir contentmdashthey consist of records of spatiotemporallylocalized events as experienced by the rememberer Phe-nomenologically recalling an episodic memory has aquality of ldquorelivingrdquo with all the attendant visual kines-thetic and spatial impressions This is especially truewhen memories are fresh before they have become crys-tallized by retelling into something more resembling se-mantic memories

Implicit memory Implicit memory also appears to bean embodied form of knowledge consisting of a kind ofperceptual andor procedural fluency (see eg CohenEichenbaum Deacedo amp Corkin 1985 Johnston Darkamp Jacoby 1985) Implicit memory is the means by whichwe learn skills automatizing what was formerly effortfulViewed in this light implicit memory can be seen as a wayof taking off line some of the problems that confront thesituated cognizer I noted earlier that when humans are con-fronted with novel complex tasks under time pressure therepresentational bottleneck comes into play and perfor-mance suffers With practice though new skills becomeautomatized reducing cognitive load and circumventingthe representational bottleneck (See Epelboim 1997 forevidence that automatizing a task reduces the need for off-loading work onto the environment) In effect prior expe-rience allows whatever representations are necessary fortask performance to be built up before the fact This strat-egy involves exploiting predictability in the task situationbeing automatizedmdashhence the fact that tasks with consis-tent mapping between stimulus and response can be au-

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

634 WILSON

tomatized but tasks with varied mapping cannot (Schnei-der amp Shiffrin 1977)

Viewing automaticity as a way of tackling the represen-tational bottleneck ahead of time can help explain one ofthe apparent paradoxes of automaticity Traditionallyautomatic processing has been considered the polar oppositeof controlled processing (Schneider amp Shiffrin 1977Shiffrin amp Schneider 1977) yet highly automatized tasksappear to allow greater opportunity for fine-tuned controlof action as well as more robust and stable internal repre-sentations of the situation (cf Uleman amp Bargh 1989)Compare for example a novice driver and an expert drivermaking a left turn or a novice juggler and an expert jug-gler trying to keep three balls in the air In each case thedegree of control over the details of the behavior is quitepoor for the novice and the phenomenological experienceof the situation may be close to chaos For the expert incontrast there is a sense of leisure and clarity as well as ahigh degree of behavioral control These aspects of auto-matic behavior become less mysterious if we consider theprocess of automatizing as one of building up internal rep-resentations of a situation that contains certain regulari-ties thus circumventing the representational bottleneck

Reasoning and problem-solving There is considerableevidence that reasoning and problem-solving make heavyuse of sensorimotor simulation Mental models partic-ularly spatial ones generally improve problem-solvingrelative to abstract approaches A classic example is theBuddhist monk problem prove that a monk climbing amountain from sunrise to sunset one day and descendingthe next day must be at some particular point on the pathat exactly the same time on both days The problem be-comes trivial if one imagines the two days superimposedon one another One instantly ldquoseesrdquo that the ascendingmonk and the descending monk must pass one anothersomewhere Other examples of spatial models assistingreasoning and problem-solving abound in undergraduatecognitive psychology textbooks Furthermore recentwork by Glenberg and colleagues explores how the con-struction of mental models may occur routinely outsidethe context of formal problem-solving in tasks such astext comprehension (Glenberg amp Robertson 1999 2000Kaschak amp Glenberg 2000 see also commentaries onGlenberg amp Robertson 1999 Barsalou 1999a Ohlsson1999 Zwaan 1999)

The domains of cognition listed above are all well estab-lished and noncontroversial examples of off-line embodi-ment Collectively they suggest that there are a wide vari-ety of ways in which sensory and motoric resources may beused for off-line cognitive activity In accord with this thereare also a number of current areas of research exploring fur-ther ways in which off-line cognition may be embodied

For example the field of cognitive linguistics is reexam-ining linguistic processing in terms of broader principlesof cognitive and sensorimotor processing This approachin radical contrast to the formal and abstract syntacticstructures of traditional theories posits that syntax isdeeply tied to semantics (eg Langacker 1987 1991

Talmy 2000 see Tomasello 1998 for a review) Of par-ticular interest for the present purpose this linkage be-tween syntax and semantics rests in part on image schemasrepresenting embodied knowledge of the physical worldThese image schemas make use of perceptual principlessuch as attentional focus and figureground segregation inorder to encode grammatical relations between itemswithin the image schema

A second example is an embodied approach to explain-ing mental concepts We saw earlier that there are prob-lems with trying to explain concepts as direct sensori-motor patterns Nevertheless it is possible that mentalconcepts may be built up out of cognitive primitives thatare themselves sensorimotor in nature Along these linesBarsalou (1999b) has proposed that perceptual symbolsystems are used to build up concepts out of simpler com-ponents that are symbolic and yet at the same time modalFor example the concept chair rather than comprisingabstract arbitrary representations of the components of achair (back legs seat) may instead comprise modal rep-resentations of each of these components and their mutualrelations preserving analogue properties of the thing beingrepresented Whereas this example is quite concrete theinclusion of introspection as one of the modalities helpssupport the modal representation of concepts that wemight think of as more abstract such as feelings (eghungry) and mental activities (eg compare)

A slightly different approach to abstract concepts istaken by Lakoff and Johnson and others who argue thatmental concepts are deeply metaphorical based on a kindof second-order modeling of the physical world and rely-ing on analogies between abstract domains and more con-crete ones (eg Gibbs Bogdanovich Sykes amp Barr1997 Lakoff amp Johnson 1980 1999) As one exampleconsider the concept communication The internal struc-ture of this concept is deeply parallel to our physical un-derstanding of how material can be transferred from onecontainer to another The parallels include metaphoricalmovement of thoughts across space from one personrsquoshead to another metaphorical barriers preventing suc-cessful transfer (as when someone is being ldquothick-headedrdquo)and so on According to this view our mental representa-tion of communication is grounded in our knowledge ofhow the transfer of physical stuff works Thus even highlyabstract mental concepts may be rooted albeit in an indi-rect way in sensory and motoric knowledge

A third example is the role that motoric simulation mayplay in representing and understanding the behavior ofconspecifics Consider the special case of mentally simu-lating something that is imitatiblemdashthat can be mappedisomorphically onto onersquos own body Such stimuli in factprimarily consist of our fellow humans There are good rea-sons to believe that this isomorphism provides a specialfoothold for robust and noneffortful modeling of the be-havior of other people (see Wilson 2001b for review)Given that we are a highly social species the importance ofsuch modeling for purposes of imitating predicting or un-derstanding othersrsquo behavior is potentially quite profound

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

SIX VIEWS OF EMBODIED COGNITION 635

We need not commit ourselves to all of these proposalsin their present form in order to note that there is a generaltrend in progress Areas of human cognition previouslythought to be highly abstract now appear to be yielding toan embodied cognition approach With such a range ofarenas where mental simulation of external events mayplay a role it appears that off-line embodied cognition isa widespread phenomenon in the human mind The timemay have come when we must consider these not as iso-lated pieces of theoretical advancement but as reflectinga very general underlying principle of cognition

ConclusionsRather than continue to treat embodied cognition as a

single viewpoint we need to treat the specific claims thathave been advanced each according to its own meritsOne benefit of greater specificity is the ability to distin-guish on-line aspects of embodied cognition from off-lineaspects The former include the arenas of cognitive activ-ity that are embedded in a task-relevant external situationincluding cases that may involve time pressure and mayinvolve off-loading information or cognitive work ontothe environment In these cases the mind can be seen asoperating to serve the needs of a body interacting with areal-world situation There is much to be learned aboutthese traditionally neglected domains but we should becautious about claims that these principles can be scaledup to explain all of cognition

Off-line aspects of embodied cognition in contrast in-clude any cognitive activities in which sensory and motorresources are brought to bear on mental tasks whose ref-erents are distant in time and space or are altogether imag-inary These include symbolic off-loading where externalresources are used to assist in the mental representationand manipulation of things that are not present as well aspurely internal uses of sensorimotor representations inthe form of mental simulations In these cases rather thanthe mind operating to serve the body we find the body (orits control systems) serving the mind This takeover by themind and the concomitant ability to mentally representwhat is distant in time or space may have been one of thedriving forces behind the runaway train of human intelli-gence that separated us from other hominids

REFERENCES

Agre P E (1993) The symbolic worldview Reply to Vera and SimonCognitive Science 17 61-69

Baddeley A (1986) Working memory Oxford Oxford UniversityPress

Baddeley A amp Hitch G (1974) Working memory In G H Bower(Ed) The psychology of learning and motivation (Vol 18 pp 647-667) Hillsdale NJ Erlbaum

Ballard D H (1996) On the function of visual representation In K A Akins (Ed) Perception (pp 111-131) Oxford Oxford Uni-versity Press

Ballard D H Hayhoe M M Pook P K amp Rao R P N (1997)Deictic codes for the embodiment of cognition Behavioral amp BrainSciences 20 723-767

Barsalou L W (1999a) Language comprehension Archival memoryor preparation for situated action Discourse Processes 28 61-80

Barsalou L W (1999b) Perceptual symbol systems Behavioral ampBrain Sciences 22 577-660

Beer R D (1995) A dynamical systems perspective on agentndashenvironment interaction Artificial Intelligence 72 173-215

Beer R D (2000) Dynamical approaches to cognitive science Trendsin Cognitive Sciences 4 91-99

Brooks R (1986) A robust layered control system for a mobile robotJournal of Robotics amp Automation 2 14-23

Brooks R (1991a) Intelligence without representation Artificial In-telligence Journal 47 139-160

Brooks R (1991b) New approaches to robotics Science 253 1227-1232

Brooks R (1999) Cambrian intelligence The early history of the newAI Cambridge MA MIT Press

Chiel H amp Beer R (1997) The brain has a body Adaptive behavioremerges from interactions of nervous system body and environmentTrends in Neurosciences 20 553-557

Churchland P S Ramachandran V S amp Sjenowski T J (1994)A critique of pure vision In C Koch amp J L Davis (Eds) Large-scaleneuronal theories of the brain (pp 23-60) Cambridge MA MIT Press

Clark A (1997) Being there Putting brain body and world togetheragain Cambridge MA MIT Press

Clark A (1998) Embodied situated and distributed cognition In W Bechtel amp G Graham (Eds) A companion to cognitive science(pp 506-517) Malden MA Blackwell

Clark A amp Grush R (1999) Towards a cognitive robotics AdaptiveBehavior 7 5-16

Cohen N J Eichenbaum H Deacedo B S amp Corkin S (1985)Different memory systems underlying acquisition of procedural anddeclarative knowledge In D S Olton E Gamzu amp S Corkin (Eds)Memory dysfunctions An integration of animal and human researchfrom preclinical and clinical perspectives (Annals of the New YorkAcademy of Sciences Vol 444 pp 54-71) New York New York Acad-emy of Sciences

Craighero L Fadiga L Umiltagrave C A amp Rizzolatti G (1996) Ev-idence for visuomotor priming effect NeuroReport 8 347-349

Dennett D (1995) Darwinrsquos dangerous idea New York Simon ampSchuster

de Waal F B M (2001) The ape and the sushi master Cultural reflec-tions by a primatologist New York Basic Books

Epelboim J (1997) Deictic codes embodiment of cognition and thereal world Behavioral amp Brain Sciences 20 746

Farah M J (1995) The neural bases of mental imagery In M S Gaz-zaniga (Ed) The cognitive neurosciences (pp 963-975) CambridgeMA MIT Press

Fodor J A (1983) The modularity of mind Cambridge MA MITPress

Franklin S (1995) Artificial minds Cambridge MA MIT PressGibbs R W Bogdanovich J M Sykes J R amp Barr D J (1997)

Metaphor in idiom comprehension Journal of Memory amp Language37 141-154

Glenberg A M (1997) What memory is for Behavioral amp Brain Sci-ences 20 1-55

Glenberg A M amp Robertson D A (1999) Indexical understandingof instructions Discourse Processes 28 1-26

Glenberg A M amp Robertson D A (2000) Symbol grounding andmeaning A comparison of high-dimensional and embodied theoriesof meaning Journal of Memory amp Language 43 379-401

Goodale M A amp Milner A D (1992) Separate visual pathways forperception and action Trends in Neurosciences 15 20-25

Goodwin C J (1999) A history of modern psychology New YorkWiley

Grafton S T Fadiga L Arbib M A amp Rizzolatti G (1997)Premotor cortex activation during observation and naming of famil-iar tools NeuroImage 6 231-236

Greeno J G amp Moore J L (1993) Situativity and symbols Re-sponse to Vera and Simon Cognitive Science 17 49-59

Grush R (1996) Emulation and cognition Unpublished doctoral dis-sertation University of California San Diego

Grush R (1997) Yet another design for a brain Review of Port and

636 WILSON

van Gelder (Eds) Mind as motion Philosophical Psychology 10233-242

Grush R (1998) Perception imagery and the sensorimotor loopwwwpittedu~grushpapers21papershtml English translation ofWahrnehmung Vorstellung und die sensomotorische Schleife InF Esken amp H-D Heckmann (Eds) Bewuszligtsein und RepraumlsentationPaderborn Germany Ferdinand Schoumlningh

Hutchins E (1995) Cognition in the wild Cambridge MA MIT PressIverson J M amp Goldin-Meadow S (1998) Why people gesture

when they speak Nature 396 228Jeannerod M (1997) The cognitive neuroscience of action Cambridge

MA BlackwellJohnston W A Dark V J amp Jacoby L L (1985) Perceptual flu-

ency and recognition judgments Journal of Experimental Psychol-ogy Learning Memory amp Cognition 11 3-11

Juarrero A (1999) Dynamics in action Intentional behavior as acomplex system Cambridge MA MIT Press

Kaschak M P amp Glenberg A M (2000) Constructing meaningThe role of affordances and grammatical constructions in sentencecomprehension Journal of Memory amp Language 43 508-529

Keil F C (1989) Concepts kinds and cognitive development Cam-bridge MA MIT Press

Kirsh D amp Maglio P (1994) On distinguishing epistemic from prag-matic action Cognitive Science 18 513-549

Kosslyn S M (1994) Image and brain The resolution of the imagerydebate Cambridge MA MIT Press

Kosslyn S M Pascual-Leone A Felician O amp Camposano S(1999) The role of area 17 in visual imagery Convergent evidencefrom PET and rTMS Science 284 167-170

Krauss R M (1998) Why do we gesture when we speak Current Di-rections in Psychological Science 7 54-60

Lakoff G amp Johnson M (1980) Metaphors we live by Chicago Uni-versity of Chicago Press

Lakoff G amp Johnson M (1999) Philosophy in the flesh The em-bodied mind and its challenge to western thought New York BasicBooks

Langacker R (1987 1991) Foundations of cognitive grammar (2 vols)Stanford Stanford University Press

Leakey R (1994) The origin of humankind New York Basic BooksMarkman A B amp Dietrich E (2000) In defense of representation

Cognitive Psychology 40 138-171Mataric M (1991) Navigating with a rat brain A neurobiologically

inspired model for robot spatial representation In J-A Meyer amp S Wilson (Eds) From animals to animats (pp169-175) CambridgeMA MIT Press

Murata A Fadiga L Fogassi L Gallese V Raos V amp Rizzo-latti G (1997) Object representation in the ventral premotor cortex(area F5) of the monkey Journal of Neurophysiology 78 2226-2230

Ohlsson S (1999) Anchoring language in reality Observations on ref-erence and representation Discourse Processes 28 93-105

OrsquoRegan J K (1992) Solving the ldquorealrdquo mysteries of visual percep-tion The world as an outside memory Canadian Journal of Psychol-ogy 46 461-488

Parsons L M Fox P T Downs J H Glass T Hirsch T BMartin C C Jerabek P A amp Lancaster J L (1995) Use ofimplicit motor imagery for visual shape discrimination as revealed byPET Nature 375 54-58

Pessoa L Thompson E amp Noeuml A (1998) Finding out about f illing-in A guide to perceptual completion for visual science and the phi-losophy of perception Behavioral amp Brain Sciences 21 723-802

Pfeifer R amp Scheier C (1999) Understanding intelligence Cam-bridge MA MIT Press

Port R F amp van Gelder T (1995) Mind as motion Explorations inthe dynamics of cognition Cambridge MA MIT Press

Prinz W (1987) Ideo-motor action In H Heuer amp A F Sanders(Eds) Perspectives on perception and action (pp 47-76) HillsdaleNJ Erlbaum

Putnam H (1970) Is semantics possible In H E Kiefer amp M K Munitz(Eds) Language belief and metaphysics (pp50-63) Albany NY StateUniversity of New York Press

Quinn R amp Espenschied K (1993) Control of a hexapod robot usinga biologically inspired neural network In R Beer R Ritzman amp T McKenna (Eds) Biological neural networks in invertebrate neuro-ethology and robotics (pp 365-381) San Diego Academic Press

Reisberg D (Ed) (1992) Auditory imagery Hillsdale NJ Erlbaum Rips L (1989) Similarity typicality and categorization In S Vosniadou

amp A Ortony (Eds) Similarity and analogical reasoning (pp 21-59)Cambridge Cambridge University Press

Saito F amp Fukuda T (1994) Two link robot brachiation with connec-tionist Q-learning In D Cliff (Ed) From animals to animats 3 (pp309-314) Cambridge MA MIT Press

Schneider W amp Shiffrin R M (1977) Controlled and automatichuman information processing I Detection search and attentionPsychological Review 84 1-66

Shiffrin R M amp Schneider W (1977) Controlled and automatichuman information processing II Perceptual learning automatic at-tending and a general theory Psychological Review 84 127-190

Simons D J amp Levin D T (1997) Change blindness Trends in Cog-nitive Sciences 1 261-267

Slater C (1997) Conceptualizing a sunset sup1 using a sunset as a dis-criminative stimulus Behavioral amp Brain Sciences 20 37-38

Steels L amp Brooks R (1995) The artificial life route to artificial in-telligence Building embodied situated agents Hillsdale NJ Erl-baum

Stein L (1994) Imagination and situated cognition Journal of Exper-imental Theoretical Artificial Intelligence 6 393-407

Talmy L (2000) Toward a cognitive semantics Vol I Conceptualstructuring systems Cambridge MA MIT Press

Thelen E amp Smith L B (1994) A dynamic systems approach to thedevelopment of cognition and action Cambridge MA MIT Press

Tomasello M (1998) Cognitive linguistics In W Bechtel amp G Graham(Eds) A companion to cognitive science (pp 477-487) Malden MABlackwell

Tucker M amp Ellis R (1998) On the relations between seen objectsand components of potential actions Journal of Experimental Psy-chology Human Perception amp Performance 24 830-846

Uexkuumlll J von (1934) A stroll through the worlds of animals and menIn C H Schiller (Ed) Instinctive behavior The development of mod-ern concept (pp 5-80) New York International Universities Press

Uleman J amp Bargh J (Eds) (1989) Unintended thought New YorkGuilford

van Gelder T amp Port R (1995) Itrsquos about time An overview of thedynamical approach to cognition In R Port amp T van Gelder (Eds)Mind as motion Explorations in the dynamics of cognition (pp 1-43)Cambridge MA MIT Press

Vera A H amp Simon H A (1993) Situated action A symbolic inter-pretation Cognitive Science 17 7-48

Wertsch J V (1998) Mediated action In W Bechtel amp G Graham(Eds) A companion to cognitive science (pp 518-525) Malden MABlackwell

Wiles J amp Dartnall T (1999) Perspectives on cognitive scienceTheories experiments and foundations Stamford CT Ablex

Wilson M (2001a) The case for sensorimotor coding in working mem-ory Psychonomic Bulletin amp Review 8 44-57

Wilson M (2001b) Perceiving imitatible stimuli Consequences ofisomorphism between input and output Psychological Bulletin 127543-553

Wilson M amp Emmorey K (1997) A visuospatial ldquophonologicallooprdquo in working memory Evidence from American Sign LanguageMemory amp Cognition 25 313-320

Wilson M amp Emmorey K (1998) A ldquoword length effectrdquo for signlanguage Further evidence for the role of language in structuringworking memory Memory amp Cognition 26 584-590

Zwaan R A (1999) Embodied cognition perceptual symbols and sit-uation models Discourse Processes 28 81-88

(Manuscript received September 12 2000 revision accepted for publication October 5 2001)