Embodied Cognition and the Simulation of Action to ......simulation of other people’s behavior within our own nervous system. This review fo-cuses on aspects of action understanding

THE YEAR IN COGNITIVE NEUROSCIENCE 2009

Embodied Cognition and the Simulationof Action to Understand Others

Scott T. Grafton

University of California, Santa Barbara

Understanding the goals or intentions of other people requires a broad range of eval-uative processes including the decoding of biological motion, knowing about objectproperties, and abilities for recognizing task space requirements and social contexts.It is becoming increasingly evident that some of this decoding is based in part on thesimulation of other people’s behavior within our own nervous system. This review fo-cuses on aspects of action understanding that rely on embodied cognition, that is, theknowledge of the body and how it interacts with the world. This form of cognition pro-vides an essential knowledge base from which action simulation can be used to decodeat least some actions performed by others. Recent functional imaging studies or actionunderstanding are interpreted with a goal of defining conditions when simulation op-erations occur and how this relates with other constructs, including top-down versusbottom-up processing and the functional distinctions between action observation andsocial networks. From this it is argued that action understanding emerges from theengagement of highly flexible computational hierarchies driven by simulation, objectproperties, social context, and kinematic constraints and where the hierarchy is drivenby task structure rather than functional or strict anatomic rules.

Key words: action understanding; simulation; embodied cognition; mirror neuron sys-tem; social-network; parietal cortex

Introduction

Decoding action performed by other peo-ple, from the simplest eye or hand movementsto the most complicated gymnastic movement,is likely to involve a multitude of cognitive pro-cesses that are supported by distinct underlyingneural substrates. This review focuses on thosedecoding processes that are most likely to be re-lated to embodied cognition. Emphasis is givento recent empirical studies that demonstrate therole of simulation as a mechanism for action de-coding. An increasing number of studies are be-ginning to define the level of detail where an ac-tion can be simulated, the neural substrates thatsupport this simulation, and most importantly,learning-dependent changes within these sys-

Address for correspondence: Scott T. Grafton, M.D., UCSB BrainImaging Center, The Sage Center for the Study of Mind, UC SantaBarbara, Santa Barbara, CA 93105. Voice: +805-975-5272; fax: +805-893-4303. [email protected]

tems as subjects acquire physical competencywith new behaviors. These findings fit within alocal framework based on embodied cognitionthat merges into a larger framework that alsoincorporates nonembodied processes for actionunderstanding.

Embodied Cognition: What Can WeDo with Ourselves?

Embodied cognition carries many definitions,but the one used here is the existence of amemory system that encodes knowledge of aperson’s physical competencies and a person iscapable of interacting with the physical world.This knowledge can be used in many ways. Thenotion of embodied cognition has motivated arange of psychological theories, including ex-planations on the origins of language (Fischer& Zwaan 2008), as a roadmap for understand-ing cognitive or motor development (Thelen

The Year in Cognitive Neuroscience 2009: Ann. N.Y. Acad. Sci. 1156: 97–117 (2009).doi: 10.1111/j.1749-6632.2009.04425.x C© 2009 New York Academy of Sciences.

97

98 Annals of the New York Academy of Sciences

et al. 2001), and as a means for explainingthe coupling between physical cues and emo-tional inference in people (Niedenthal 2007).In the current review embodied cognition isused to focus on the interpretation experimentswhere people relate knowledge of their ownbody to understand other people’s body move-ments. This focus harkens back to the orig-inal formulation of embodied cognition thatemerged from the artificial intelligence com-munity two decades ago (Brooks 1991). Themain point from this work was that to performintelligent actions, an individual’s knowledge—in this case a computer’s—of what the bodycan do matters. While we can readily use dis-embodied computers to manipulate symbolicinformation, intelligence requires interactionwith the world. Once an artificial system is en-abled with sensory and motor attributes, thevery nature of what one considers intelligentchanges. Intelligence can then be understoodby observing what an agent perceives and howit acts, not by what it computes. While the focusin robotics is often on overcoming the limita-tions that a physical plant can impose, physicalsystems can provide new capacities that createintelligence. In this view embodied cognition isnot a description of the limitations in cognitionconstrained by a physical plant. Rather, it em-phasizes what is enabled cognitively by having abody. This can include the gains that come fromrelatively low-level physical attributes, such asthe spring-like quality of our musculoskeletalsystem to facilitate walking, or the softness ofour fingertips to aid in grasping a stiff object.Or it can emerge when knowledge about theskills needed for one object or context can begeneralized to new conditions. Ironically, thisknowledge turns out to be exceedingly difficultfor robots to acquire by unsupervised learn-ing and relatively trivial for humans. It is fromthis human perspective that we can considercases where embodied cognition leads to gainsof cognitive function.

There is growing evidence from studies inathletes that physical experience can improveperceptual ability (Tenenbaum et al. 1994). For

example, professional basketball players canconsistently predict if a free throw shot will besuccessful sooner than the general population.Basketball coaches, with moderate physical ex-perience and extensive visual familiarity, fall inthe middle for this predictive ability. The play-ers draw on subtle physical cues such as theflexion of the wrist and ankle more than thenonexpert does (Aglioti et al. 2008).

Historically, embodied cognition of actionhas been investigated with experiments demon-strating interference effects on self-generatedactions that are influenced by the observation ofanother person’s limb postures or actions; alter-natively, this cognition has been studied whenthe decoding of another person’s action is in-fluenced by the simultaneous performance of arelated action (Craighero et al. 2002; Craigheroet al. 1999; Hamilton et al. 2004). Interferenceeffects such as these provide indirect evidencethat some knowledge about action is embodiedand, furthermore, that there may be some formof simulation taking place during action obser-vation. However, as pointed out by Jacob andJeannerod, either mental or motor simulationcould be a means for decoding action under-standing and this might occur at more thanone level of abstraction (2005). Defining thelevel of simulation becomes a central issue forinterpreting both behavioral and physiologicalstudies.

Only recently have action interference stud-ies attempted to isolate the level of abstractionin simulation. As an extreme case, simulationcould occur based on very abstract representa-tions requiring pure mentalizing. One personcould formulate a mental impression of whatanother person might be thinking. It has re-cently been shown that this is more likely to oc-cur for novel situations (Brass et al. 2007). At theopposite end of the spectrum, simulation mightoccur by matching perceived actions with in-ternal models of the same actions. The strongversion of this is direct matching, where kine-matically identical movements are simulated inthe observer as part of the action decodingprocess. In the real world there is much less

Grafton: Embodied Cognition and Stimulation of Action 99

opportunity for direct matching and so if sim-ulation is to occur, a weaker form is needed.The weak form requires a comparison of anal-ogous movements that are not necessarily iden-tical between the observer and observed motorsystems. For example, observing another’s ac-tions from the first- or third-person perspectivecan be simulated at a similar level of abstrac-tion (Anquetil & Jeannerod 2007). And in aweight judgment task, subjects rely on certainvisual heuristics such as the lifting movementof the actor’s hand to determine an object’sweight and largely ignore other kinematic fea-tures such as how the object is moved around,body posture, or how an object is put down(Hamilton et al. 2007).

To make stronger inferences about the kindof simulation that takes place in action obser-vation many groups have begun to incorporateelectrophysiological methods aimed at definingthe functional anatomy of embodied cognitionand the circumstances where there is strongoverlap between action, perception, and under-standing. One electrophysiological approachfor demonstrating simulation within the motorsystem is based on studies that identify modi-fications of motor corticospinal excitability. Intranscranial magnetic stimulation (TMS) stud-ies the relative sensitivity of the corticospinaltract is assessed by stimulating over motor cor-tex and measuring changes in the thresholdneeded to evoke responses in the hand (Aziz-Zadeh et al. 2002; Baldissera, Cavallari et al.2001; Fadiga et al. 1995). This excitability in-creases when subjects observe actions in others.The mere observation of static photographs ofhands positioned with a pincer grip causes anincrease in corticospinal excitability comparedto observation of hand positioned at rest. Thisincreased excitability was specific for the musclethat would be activated during actual executionof the observed action (Urgesi et al. 2006). Ac-tion observation can also amplify learning- orexperience-dependent changes of corticospinalexcitability (Aglioti et al. 2008; Celnik et al.2006). These studies are consistent with simu-lation at the level of the motor cortex and are

helpful in demonstrating the specificity of ac-tions that might lead to simulation. However,they do not localize simulation circuitry outsideof the primary motor cortex.

The current review considers recent evi-dence from cognitive neuroscience that contin-ues to build a conceptual framework for howaction understanding is achieved through sim-ulation within a framework of embodied cog-nition. Five issues that arise from this new re-search are considered:

• What are the neural underpinnings of ac-tion observation and simulation?

• Does physical expertise alter the neuralsubstrates of action understanding?

• Is there hierarchical structure in the de-coding of observed actions?

• What simulation is applicable to the socialworld?

• What is the relationship between em-bodied cognition and the mirror neuronsystem?

The Neural Underpinnings ofAction Understanding

The Action Observation Network

There is now overwhelming evidence fromfunctional magnetic resonance imaging (fMRI)of blood oxygen level-dependent (BOLD) sig-nals that observation by one person of anotherwho is in motion will engage in the observer awidespread, bilateral network of cortical brainregions in a highly reproducible manner (Buc-cino et al. 2001; Frey & Gerry 2006; Graftonet al. 1996; Grezes & Decety 2001; Hari et al.1998; Johnson-Frey et al. 2003; Rizzolatti et al.1996; Saygin et al. 2004; Ulloa & Pineda 2007;Zentgraf et al. 2005). We refer to this as theaction observation network (AON), shown inFig. 1. It includes the bilateral posterior su-perior temporal sulcus (STS), inferior parietallobule (IPL), inferior frontal gyrus (IFG), dorsalpremotor cortex, and ventral premotor cortex.What is striking about this network is that a


Figure 1. An example of the action observa-tion network. Colored areas indicate cortical regionsthat are activated when subjects make an inferenceabout why a person being observed is grasping anobject in a typical context where that object mightbe found. Comparing blood oxygen level-dependent(BOLD) activity during these images relative to ablank screen captures visual evoked responses be-ginning with early visual processing as well as thedecoding of the possible meaning of the action.

perceptual stimulus can lead to such a large-scale recruitment of the brain with a completedisregard of textbook divisions between sen-sory and motor portions of the cortex. Under-standing or predicting the temporal and spatialstructure of a stimulus is just as likely to re-cruit a premotor area (Schubotz & von Cramon2001, 2004). This distributed network likelysupports many subtasks, including the trans-formation of perceptions to action, the simula-tion of observed movements in relationship toknown movements, and the storage of physicalknowledge (both of self and objects) that canbe used for simulation. What is the evidencethat all or part of the AON might support theseprocesses and, in particular, simulation?

An early argument in support of simulationwas motivated by the fact that some of the ar-eas of the AON overlap strongly with areasassociated with movement execution, particu-larly within the IFG and IPL (see Grezes et al.2001 for review). This overlapping cortex, withshared substrates for execution and observa-tion, is referred to as the mirror neuron system(MNS) (Rizzolatti & Craighero 2004). Becausesome of these core areas are also used for ac-tion execution it has been proposed many timesthat there exist underlying mirror neurons thatcould serve as the critical functional substratefor simulation (Gallese et al. 2004). It should benoted however that invoking a mirror neuronsystem is an anatomic rather than a functional

explanation for most imaging experiments andit does no more than constrain the number oflocations where simulation might occur, and itdoes so for only a subset of tasks where sim-ulation might occur. Given this limitation anumber of new and innovative experimentalapproaches can be interpreted as evidence ofsimulation without the need to invoke the MNSheuristic to explain the data.

Simulation and Somatotopy

One fMRI approach to study simulation isbased on the idea that observation of actionsperformed by different body parts should re-cruit different parietal and premotor areas, in aroughly somatotopic distribution. This generalfinding has now been observed in multiple stud-ies (Buccino et al. 2001; Buccino et al. 2004;Gazzola et al. 2006; Goldenberg & Karnath2006). The logic is motivated by the fact thatmotor execution areas have a roughly somato-topic representation, so simulation within thesecircuits should also be somatotopic. It should benoted, however, that differences in activation asa function of somatotopy might be confoundedby other task variables such as familiarity orcomplexity. It is also important to note thatnot all components of the AON demonstratea somatotopic arrangement. While there aredistinct localizations of fMRI-BOLD responseswhen a subject is observing face versus handmotions within the IPL, the same is not true forthe STS. This is possible evidence for modu-larity within the AON. Areas such as STS mayprovide lower level input to IPL and other partsof the AON involved with simulating humanactions at a higher level of analysis (Thompsonet al. 2007).

Simulation and Conspecifics

Another neuroimaging paradigm in sup-port of simulation is based on mapping fMRI-BOLD responses of an observer within theAON during the viewing of conspecifics ver-sus other species as they perform an action


(Buccino, Lui et al. 2004). The key finding isthat when observing a nonconspecific such asa dog performing actions that the observer canperform, there is more activation of the AONcompared to what is seen in the observer for ac-tions not commonly performed (barking). Simi-larly, observation of a humanoid like action by arobot can recruit the AON network more thannonhumanoid actions can. This also suggeststhat simulation might be driven more by the fa-miliarity and feasibility of an action rather thandirect matching to an image of a human body(Demiris & Simmons 2006).

Simulation and Objects

It has long been known that the mere ob-servation of a graspable object will activatecomponents of the action observation networkincluding dorsal premotor cortex and the an-terior intraparietal sulcus (Grafton et al. 1997),and the influence of the object on grasp plan-ning is within the AON (Grezes et al. 2003).Having a graspable object in view also mod-ulates visual attention, even when the objectis irrelevant, and these effects are mediatedthrough dorsal parietal and premotor circuits(Handy et al. 2003). Experiments in which sub-jects vary in their knowledge of tools or ob-jects could potentially be used as paradigms totest for simulation (Frey & Gerry 2006). How-ever, tool knowledge is not necessarily matchedto simulation capacity because the familiarityfor tools might be in terms of object proper-ties rather than hand actions for manipulatingthe object. In addition, the semantic knowledgeand pragmatic knowledge of a tool introducesadditional complexities about what subjects areusing to understand actions are topics that willnot be broached in this review (Beauchampet al. 2003; Chao & Martin 2000; Weisberget al. 2007).

Simulation and Action Familiarity

Another inroad to understand simulation isbased on action familiarity. If an action itself

(rather than the dog or robot performing theaction) is more familiar, then there should begreater activity in a simulation circuit whenthis action is observed. The challenge in de-signing experiments that tap into familiarity ofhuman action is to find behaviors with moreor less novelty. One approach for manipulat-ing familiarity that avoids object-centered ac-tion has been to measure brain activity duringobservation of dancers. When dancers watchsomeone else perform a familiar dance there isgreater activation within the AON comparedto watching a dancer perform an unfamiliarform (Calvo-Merino et al. 2005). This suggeststhat the AON, and in particular the premo-tor and parietal cortex, demonstrate strongersimulation for movements that are physicallyfamiliar. There are two concerns that under-mine the specificity of this interpretation. Thefirst concern is that in this experiment the fa-miliar dances were both physically and visuallyfamiliar. The second is that the familiar dancemovements had names and the unfamiliar didnot. Subverbal naming might be occurring inthis task and thus interact with visual or phys-ical familiarity. These concerns were directlyaddressed in an elegant follow-up study (Calvo-Merino et al. 2006). Men and women of a pro-fessional ballet company observed movies ofballet dance movements that were all stronglyfamiliar in terms of visual recognition naming.The critical manipulation was watching move-ments performed by men or women. In bal-let many of the movements are gender-specific.There was stronger activation within the AONwhen watching dance movements by same sexdancers compared to the opposite sex, presum-ably because these were the movements withwhich the observer had more physical experi-ence. These familiarity effects were most robustin inferior parietal and premotor areas.

Simulation and Action Competence

If activity in the AON is stronger during sim-ulation of movements that are physically famil-iar then there should be learning-dependent


changes in the network as a function of phys-ical training. This idea was tested in a groupof modern dancers, who learned to performa new dance piece. They were scanned onmultiple occasions over several months as theygained physical competency. During scanningthey watched short video clips of the piece theywere learning and another piece they did notpractice. In addition, for each clip they wereinstructed to imagine performing the move-ments. They also rated how well they thoughtthey could perform each segment themselves(Cross et al. 2006). There was stronger activitywithin the AON when watching dance move-ments that had been learned after training. Inaddition, activity in the AON correspondedto self-rated competency on a movement-by-movement basis.

The previous studies suggest there is a formof motor resonance that develops in the AONwhen people observe others performing phys-ically familiar actions. In an interesting exten-sion of this idea, this finding was inverted totest if motor resonance within the AON couldserve as a substrate for learning new movementsbased on observation alone. To do this, subjectslearned dance steps in a computer game similarto “Dance Dance Revolution.” Both symboliccues and a physical model were used to learnspecific dance steps that were accompanied byunique music (Cross et al. 2008). Some dancepieces were learned with actual physical prac-tice. Others were learned only by observationof the model and symbols, without any physi-cal movements. On follow-up testing, subjectscould perform those dances for which they hadphysical practice or observational learning bet-ter than novel dances. As in the previous studies,there was an experience-dependent change ofactivity within the AON for dance steps thathad been physically practiced and, to a lesserdegree, for observational learning. This findingsuggests that the simulation operations withinthe AON may be used both for decoding ob-served actions and also to model new move-ments in the observer. These learning-relatedchanges were limited to parietal and premo-

tor cortex. The extrastriate body area and STSareas demonstrated no change with learningand were only sensitive to the presence or ab-sence of a physical model. This finding providesadditional evidence for modularity within theAON.

In a follow-up analysis the same data setwas tested to determine if the presence ofthe physical model was necessary for learning-dependent changes to take place in the AON.The key finding was that symbolic cuesalone were sufficient to induce learning-relatedchange in the AON. Furthermore, once a dancewas learned, the symbols and music aloneserved as potent stimuli for activating the AON.In other words, an external physical model isnot required for motor simulation to occur oncea particular body movement of the observerhas been associated with symbols and music.(Emily Cross, personal communication) It hasbeen argued this rhythmic coupling of exter-nal cues and induced body motion is a coresubstrate for the pleasures of dancing to music(Janata & Grafton 2003).

Hierarchy and ActionUnderstanding

The experiments described so far are a rep-resentative rather than exhaustive review ofattempts to use functional imaging to cap-ture simulation processes during action ob-servation. Taken in aggregate they providerelatively strong evidence that some form ofsimulation is occurring within the AON duringaction observation. They also begin to demon-strate granularity in the functional specializa-tion within the AON proper, with some areasspecifically linked to physical models (STS) andothers more sensitive to learning-dependent ex-perience (inferior parietal lobule). Two inter-related issues emerge from this granularity. Isspecialization within the AON related to thekind of simulation that is taking place (e.g., di-rect matching, goal-oriented, or abstract) or toother processes such as decoding of biological


motion or contextual information? If there isspecialization related to the level of simulation,is there evidence for any functional hierarchy?

Lesions of the AON

One way to identify specialization within theAON is to test for behavioral deficits in the faceof real or virtual focal lesions to the brain. Inpatients with ischemic stroke of the cerebralcortex a deficit in biologic motion processingcan be present with lesions of the STS. This isconsistent with classic models of visual process-ing where the human homologue of area MTand the extrastriate body area are essential forthe perceptual decoding of body-specific move-ment. What is striking is that a lesion in premo-tor cortex of the AON will also cause a deficitin the processing of biological motion (Saygin2007). While this could be due to a low level dis-ruption in the decoding of motion information,a more likely interpretation is that the premo-tor segment of the AON is capable of formingmore abstract relations between motions andthe meaning of the motion. In support of thislatter interpretation, deficits in the nonlinguis-tic matching of pictures of actions (such as lick-ing something) with appropriate objects (an icecream cone) are also associated with inferiorfrontal cortex lesions (Saygin et al. 2004). Stud-ies of patients with limb apraxia have identifiedgesture comprehension deficits, suggesting thatthere is an anatomic overlap for the neural sub-strates of both production and comprehension(Ferro et al. 1983; Heilman et al. 1982; Rothiet al. 1985). Precise lesion localization has onlyrecently emerged to show that the lesions areconsistently left-lateralized and independentof language comprehension (Buxbaum et al.2005). Precise localization based on lesion over-lap analysis shows that gesture comprehensiondeficits correlate with damage to the opercularand triangularis portions of the inferior frontalgyrus, two regions of the AON. Remarkably, nosuch relationship was observed with lesions cen-tered on the inferior parietal cortex, an area ro-bustly activated by action observation and asso-

ciated strongly with apraxia. The findings sug-gest that left inferior frontal cortex, involving re-gions that are also involved in planning and per-forming actions, is causatively associated withdeficits in the recognition of the correct exe-cution of meaningful gestures (Pazzaglia et al.2008). A continuing concern with gesture com-prehension experiments is whether the behav-ioral tasks are tapping into semantic compre-hension or are due to deficits of embodiedcognition irrespective of semantics.

Innovative studies in normal subjects helpto build a case for an action specific networkinvolving inferior frontal cortex. Transient dis-ruption of inferior premotor cortex by TMS im-pairs judgments in the weight of an object an-other person is lifting and also leads to increasedresponse times in a delayed match to sampletask when the matching is for body actions (Po-brinc & Hamilton 2006; Urgesi et al. 2007).In contrast STS disruption slowed matching ofbody form. These real and virtual lesion datasuggest that the anterior lesions of the AONmay be particularly important for the stor-age or retrieval of action vocabularies, that is,the relations between known body movementsand their probable meaning in terms of actionconsequences.

Simulation and Context

How a body moves in the world or interactswith an object is constrained by the featuresof the environment or object. This interactionbetween the body and world gives some insightinto what a person is doing, but not necessar-ily why they are performing an action. A fun-damental question is how contextual informa-tion is integrated with simulation to generate animproved estimate of another person’s actions.One approach to study this is to add contex-tual information that alters the meaning of anobserved action (Iacoboni et al. 2005). For ex-ample, an object such as a coffee cup can begrasped in different contexts that imply differ-ent intentions of the actor. If it is the start of ameal, the intention is to drink, if the dishes are


a mess, the intention is to clean up. Note thatin this experiment the intentionality is centeredon the hand−object interaction, that is, Whyis a person grasping a cup in a particular way?Adding this contextual information amplifiesthe decoding of why an object is manipulated.This further recruits parietal and premotor ar-eas within the AON. Whether this activationrepresents a simulation or supports a role forembodied cognition is unclear.

Hierarchical Processing of ActionObservation: Evidence

from Repetition Suppression

What is the evidence for hierarchical pro-cessing within the AON? The answer dependsin part on what is meant by “hierarchy.” Theorigin of the word hierarchy can be found insermons from the 1380s that codified the dif-ferent angels into three levels, the Seraphin,Cherubyn, and Trones. The notion of succes-sive orders or grades, one above the other,was eventually generalized to rulers, species,and ultimately any persons, things, or con-cepts. When we watch someone perform anaction, we group the movements into coher-ent subunits or parts with some of them subor-dinate to others. In the making of coffee theindividual task requirements (scooping, boil-ing water, filtering) are subordinate to the goalof brewing (Cooper & Shallice 2006). Thiswhole−part structure can be readily identifiedin imitation studies that show a bias towardsgoal rather than parts, and the same studiesalso show emergence of this bias with develop-ment (Bekkering et al. 2000). Existing computa-tional models that use hierarchy to understandstructured behavior generally assume that thepart−whole organization of an action or itsrecognition is mirrored by the neural repre-sentations that underlie the hierarchical orga-nization (Botvinick 2008; Cooper & Shallice2006). A major challenge is to determine ifputative hierarchies as defined by experimen-tal manipulation and fMRI are consistent withthis assumption. This would require successive

ordering or levels of functional anatomy thatdirectly map to a part−whole structure. Alter-natively, the functional anatomy could simplyreflect differences in other aspects of task struc-ture, such as inherent complexity, task main-tenance, scheduling, or other nonhierarchicalprocesses such as increasing temporal delay.In other words, demonstrating that differentparts of the brain are activated as a functionof task complexity or abstraction by itself is in-sufficient toward establishing that the underly-ing functional anatomy is hierarchical (Badre &D’Esposito 2007). It is also necessary to demon-strate that there is a nesting of part−wholeprocesses, with higher levels dependent onlower levels. This nesting can be very chal-lenging to establish with conventional cogni-tive subtraction paradigms. By using alternativefMRI techniques such as repetition suppressionthe whole−part structure and nesting of in-terrelated processes can be addressed moredirectly.

We used repetition suppression (RS) meth-ods to distinguish different levels of ac-tion understanding. RS has been exten-sively used in studies of visual processing(Grill-Spector & Malach 2001; Kourtzi &Kanwisher 2000), where it is also referred toas “fMRI-adaptation.” It is based on the trial-by-trial reduction of a physiologic response torepeated stimuli. RS is not unique to fMRI andcan be observed with electroencephalographyand even at the level of single neurons. It canbe associated with changes of behavior, suchas reaction time priming (Maccotta & Buckner2004; Wig et al. 2005); however, this behavioraleffect is not required. Most importantly, RS canidentify changes within a class of stimuli or alevel within a potential representational hierar-chy rather than between classes of stimuli. Forexample, areas that recognize the identity ofa particular face will show RS effects that areview-, context-, or hairstyle independent.

Suppression occurs when two successivestimuli are represented in the same neural pop-ulation, and release from suppression occurswhen two successive stimuli are represented


in different populations. Thus, the method as-sumes the existence of population coding withinbrain regions, for which there is extensiveevidence in many parts of the cortex (Brit-ten et al. 1993; Georgopoulos et al. 1986). RSalso requires the population response to changewhen the same stimulus feature is repeated.The details of this change are not known. Itcould be due to a reduction of neuronal firing,a decrease in firing duration, or a sharpeningof neuronal tuning curves (Grill-Spector et al.2006; Krekelberg et al. 2006). Irrespective ofthese different mechanisms for RS, at the pop-ulation level their effects on the BOLD-fMRIsignal are all the same.

To identify possible topologies in the AONcorresponding to a possible hierarchy, a libraryof stimulus sets designed to induce RS basedon different features of simple hand−objectactions were developed. In the initial experi-ment, subjects observed movies of a hand−armreaching and grasping either of two ob-jects from a first person perspective (de C.Hamilton & Grafton 2006). The actor graspedone of the objects, then lifted it and transportedit to the midline and the trial ended. The po-sition of the two objects and which object wasgrasped were independently manipulated. Inthis way repetition of trajectory and repetitionof hand−object grasp could be separated. Thetwo objects in each trial were closely paired interms of size and shape, such as a cookie ora computer diskette, so that grip configurationwas similar. Subjects performed an incidentalmonitoring task, and new objects were intro-duced frequently to avoid the problem of long-term adaptation to a single pair of objects.

The main finding was a strong RS effectin the left anterior intraparietal sulcus (aIPS)when a given object was grasped repeatedly.aIPS was not sensitive to repetition of trajec-tory. RS effects for trajectory were observed inleft lateral occipital sulcus and right superiorprecentral sulcus. If a single network decodedall of these trajectories and hand−object fea-tures similarly then RS would look the samefor the two features. The fact that differences

emerged provides evidence for dissociable sys-tems for decoding limb trajectory informationand hand−object interactions.

Grasping an object, irrespective of how orwhy it is being grasped, is an action goal inand of itself. The RS results suggest that object-centered action goals are localized to aIPS. Thisis a generalization of the finding that this areamatches visual or tactile features of an objectto an appropriate grip. fMRI studies compar-ing reach and grasp identify greater activity inaIPS for grasp (Binkofski et al. 1998; Culhamet al. 2003; Frey et al. 2005). In addition, iso-lated lesions to this region disrupt grasp controlbut not reach kinematics (Binkofski et al. 1998;Frak et al. 2006). This evidence leads to theconclusion that the area is a homologue to areaAIP identified in nonhuman primates includ-ing macaque monkeys (Borra et al. 2008). Atraditional view of AIP function is that it is arepository of grip apertures generated from ob-ject features. Object shape, size, and orientationhave been found to be encoded not only in earlyvisual areas, but also by neurons in monkey areaAIP (Murata et al. 2000). Neurons represent-ing three-dimensional shape have been foundin the caudal intraparietal sulcus (area CIP)(Sakata et al. 2005; Tsutsui et al. 2005) as wellas in an anterior section of the lateral bank ofthe intraparietal sulcus, area AIP (Sakata et al.2005) of monkeys. Object-specific firing occurswith or without vision of the grasping hand(Murata et al. 1996). However, in human stud-ies, repetition suppression of objects does notmodulate activity in aIPS, whereas the graspon an object does (Shmuelof & Zohary 2005),and these grasp RS effects occur irrespective ofthe location of the object in space (Shmuelof &Zohary 2006).

In the RS studies described so far it is as-sumed that the trajectory information carrieslower level information that is subordinate tothe more interesting and behaviorally relevantinformation about what is being grasped. Thus,it can be inferred that these distinctions sup-port a relative functional hierarchy during ac-tion observation that is based on distinctions


between reach kinematics and the goal of theaction defined by what object is grasped. In thiscase, area aIPS is supraordinate in function tothe areas decoding an object identify, limb tra-jectory, or where an object is located in space.Note that unlike the strict hierarchy of angelsdescribed above, the functional anatomic hi-erarchy implied in this action understandingexperiment is relative. Under different circum-stances the trajectory information might bemore important for action understanding thanwhat is being grasped.

To further test for potential hierarchies in theAON, a second RS experiment was performed(Hamilton & Grafton 2007). Subjects observedmovies of an actor reaching and grasping eithera wine bottle or a dumbbell placed on end. Thetrial ended after the object was lifted and placedin a new location. In this study the viewpointwas from the side rather than first person per-spective. As in the first experiment there couldbe repeated observation of grasping for one ob-ject or the other, thus allowing an independentreplication of the goal−object RS effect deter-mined in the first experiment. Independent ofthis, there could be repetition for whether thegrasp was performed with a power grip alongthe thick part of the object or a pincer grip alongthe thin part of the object. In this case the typeof grasp did not imply any specific cues as tothe actor’s intentions because the same action, atransport of the object, was always performed.If aIPS is driven by the specific details of howa hand interacts with an object irrespective ofthe intention of the grasp then there should beRS when the grip type was repeated. On theother hand, if there was no RS in aIPS for griptype, then the effect observed in the first ex-periment should be related to a higher orderrepresentation of goal−object interactions.

The RS effect for what object was graspedwas again localized to left aIPS (extending intothe adjacent IPL) and to a lesser degree withinright aIPS. These findings provide a strongreplication of the first study. Using a statisti-cal threshold appropriate for an exploratoryanalysis, RS for goal−object was also found in

another part of the AON, the left IFG. The RSeffect of how the object was grasped identifiedthree clusters in the supplementary motor ar-eas, middle frontal gyrus, mid intraparietal sul-cus, and inferior and middle occipital regions.Critically, there was no RS for grasp-type in theanterior IPS or in IPL in either the left or righthemisphere.

As in the first study, it is assumed that theneural substrates for decoding what was beinggrasped in this task are supraordinate to thosedecoding how it is being grasped. In this exper-iment the nature of the grasp had no predictivevalue on a subsequent action. This implies afunctional/anatomic hierarchy where the de-coding of grasp, irrespective of object identify,occurs across lateral occipital regions that con-tribute to a visual analysis of hand−object kine-matics for both how the hand approaches anobject, the specific grip on the object and thesubsequent movement of the object. In con-trast, both experiments support a role of aIPSfor representing an object as a goal for the handrather than as a node dedicated to the senso-rimotor transformation of visual features to aparticular hand shape.

Although grasping an object can be consid-ered an action goal, for most activities this is butan interim step toward object manipulation toachieve more complex goals, as in tool use (Frey2008). Thus, the two RS experiments describedabove are not adequate for assessing intention-ality at a level of why an actor is performinga particular action with an object. This wasstudied in a third RS experiment that consid-ered physical outcomes of actions (Hamilton &Grafton 2008). To localize action outcome ef-fects with RS, participants observed actors ma-nipulating objects or tools in a way that led to aspecific outcome. For example, in one trial theymight see the actor reach and grasp the slidingtop of a wooden box and either push or pull thelid. Depending on the starting position of thelid, the outcome of the movement was to eitheropen or close the box. Using RS, the outcome(open or close the box) was independently ma-nipulated relative to the means to accomplish


the outcome (push or pull the lid). Over thecourse of imaging, the subjects watched a bat-tery of movies capturing many familiar behav-iors including turning a stove on or off, switch-ing a light on or off, tying or untying a string,drawing or erasing with a pencil, and hammer-ing a nail or a nut by two different means.

An RS effect for outcomes was found in thebilateral IPL and the IFG. That is, the responseto an outcome was suppressed when the sameoutcome was repeated on a second trial, regard-less of the means used to generate the outcome.Analysis of the responses to each of the individ-ual sets of movies indicated that the RS effectsfor outcome in parietal and frontal areas werenot driven by a single action or outcome, butgeneralized across a wide variety of actions.

In the first two experiments the grasping ofa particular object could be considered a typeof low level outcome in the sense that the fi-nal goal−object interaction was the purposefulmovement. If so, then there might also be ev-idence for an RS effect in aIPS for the morecomplex action outcomes in the third experi-ment. To test this prediction a region-of-interestanalysis was performed in the aIPS. Within thisregion-of-interest, a weak but significant sup-pression for repeated outcomes was also de-tected, supporting a model in which aIPS isinvolved in action outcomes for tasks spanninga range of complexity. It has recently been pro-posed that left aIPS carries specific informa-tion about the interaction between a grasp andthe specific requirements of how to use a tool,which would also be consistent with these find-ings (Valyear et al. 2007).

An analysis of RS effects for means, compar-ing repeated movements and novel movements,identified weak effects in left middle intrapari-etal sulcus, left lateral occipital cortex, and leftSTS. These results show that visual areas sup-port the general analysis of movement features.Again, if it is assumed that understanding themeans to an action are functionally subordinateto understanding the outcome, then the resultscan be interpreted as supporting a relative hi-erarchy within the AON.

Hierarchy for Imagined Actions

Relative hierarchy between kinematics andgoals has recently been defined using a com-pletely different yet complementary approachto the passive observation RS methods de-scribed in the previous section. Instead of pas-sive watching, subjects imagined imitating in-transitive gestures made of point light videosof a human actor (Lestou et al. 2008). Subjectsobserved the human forms performing actionssuch as kicking, waving, knocking, and throw-ing. Direct task contrasts of the videos with mo-tion relative to static frames identified the ex-trastriate body area and areas of STS associatedwith body movement. Consistent with manyprior studies of imagined movement, the con-trast of imagined movement relative to passiveobservation activated bilateral PMv and IPL(Filimon et al. 2007; Grafton et al. 1996; Lotzeet al. 1999; Stephan et al. 1995). Using thesetwo contrasts as localizers, the authors testedfor RS effects in all of these areas as a func-tion of repetition for different goals or move-ments used to generate these goals. In this ex-periment the authors used intransitive actions.Thus, goal of the action was also the “meaning”of the action, such as waving hello, rather than ahand−object interaction coupled to a particu-lar object or use as studied in the other RS stud-ies described above. To manipulate the kind ofmovement made, the authors played moviesbackwards or changed the temporal propertiesof a movie without impairing the observer’sability to detect the meaning of the action.All areas of the AON generated by the local-izer tasks demonstrated robust RS effects wheneither a goal or movement to achieve a goalwas repeated. By examining the RS effects incloser detail it was found that parietal areas andthe STS were particularly sensitive to differ-ences between movements with different mean-ing, whereas premotor areas were more sen-sitive to differences between movements evenwhen they shared similar meanings. The re-sults are very interesting in light of the pre-vious experiments showing goal and outcome


associated RS effects in parietal and premotorareas. However, the imagined imitation addscomplexity because it is unclear what cogni-tive strategy was used to represent the goal.The use of imagined imitation of intransitivegestures introduces some ambiguity because inthis case the action goal, that is, the mean-ing of the action, can be represented eitherby a semantic route or via direct kinematicmatching (Rumiati et al. 2005). In addition,the use of imagined action, while typically con-strained to activations associated within theAON, introduces top-down strategies to rep-resent the particular action and maintain itin working memory. Despite these potentialproblems the results are remarkably consis-tent with the simpler, object-centered studiesbased on passive observation. Both found acloser interaction between task goal and pari-etal activity and task kinematics and premo-tor activity. What is new in this experiment isthe sensitivity within the STS to repetition ofaction meaning. It is possible that for imita-tion of intransitive movements the body move-ment features are the key and possibly exclu-sive source of insight into what the actor isdoing. In terms of a relative hierarchy, the STSthen serves a supraordinate or equivalent roleto parietal cortex in action understanding.

Potential Pitfalls of the RS Method

There are three important issues that mightaffect interpretation of RS experiments. Thefirst is the incorrect assumption that the differ-ent levels of behavior that are examined for RSeffects must be independent of each other forthis approach to be informative. This assump-tion is clearly not true at a behavioral level, asthere is much evidence that the kinematics ofperformed hand actions are altered by the goalof the action (Ansuini et al. 2006; Gentilucciet al. 1997). Thus, in RS imaging experimentsthere is no requirement that the different lev-els must be independent parameters of interest.The RS experiments simply identify brain areasmost sensitive to one or more levels of control,

and this experimental method is biased towardthe detection of modularity across functionallevels. The results of the RS experiments donot exclude the existence of shared processingacross action levels. Obviously, these relativelevels of control all interact with each other.

Given the tight coupling between kinematicsand goals, a second issue is whether RS effectsfor outcome are actually due to more subtle dif-ferences of kinematics rather than the outcomeitself. However, if this were the case, we wouldexpect to see similar RS for both the kinematiceffects for different outcomes and also for kine-matic differences irrespective of outcomes. Thiswas not the case.

The third issue that arises from these ex-periments is the possibility of an attentionalconfound. Parietal cortex, in particular on theright, has been associated with spatial attention(Corbetta et al. 2002), so it becomes importantto consider whether the RS effects we observedcould be due to manipulation of attention. Totest this consideration, participant’s attentionwas manipulated to different task features dur-ing action observation (Hamilton & Grafton2007). The authors did not find any differencesin the RS as a function of guided attention.Furthermore, if attention alone were responsi-ble for the RS we identified, we would expectto see RS in the same “attentional” brain re-gion for every contrast. The fact that a distinctset of brain regions for grasps, goals, and out-comes was found is evidence that the RS effectsare truly reflecting neuronal population codingin different brain areas, rather than a result ofattention.

The Social Milieu

The previous section makes a case for spe-cialization and hierarchy within the AON fora number of simple limb movements based ongestures or hand−object manipulation. In thereal world, actions are far more complex, social,and contextualized. A simple change in anotherperson’s eye gaze, shift of their posture, or handgesture, all within rich contextual cues, can


convey meaning, agency, and intention. Theneed to understand in this more complex en-vironment is paramount. A false interpretationof a potential enemy could lead to a faux pas oreven death. A fundamental challenge in socialneuroscience is to elucidate the mechanismsthat allow this process to unfold automaticallyacross a broad range of contexts and contin-gencies. Recent evidence suggests that actionunderstanding in these more complex scenar-ios is achieved by more than one solution, andthese solutions involve a complex circuitry thatextends beyond the AON described so far.

The Social-Network

Evidence for a distinct social-network (S-N) in the neocortex is motivated by studies inwhich subjects perform any one of a variety ofsocial tasks demonstrating selective activationof the posterior STS, insula, medial prefrontalcortex (mPFC), posterior cingulate, and amyg-dala (Adolphs 2001; Frith & Frith 1999). Theseareas are sensitive to biological motion, ani-macy, the experience and recognition of affectand theory of mind attribution (Allison et al.2000; Anderson & Phelps 2001; Mitchell et al.2002; Pelphrey et al. 2004; Saxe et al. 2004;Wheatley et al. 2007). Observation of limb ges-tures with emotive content (feeling angry, notcaring, etc.) is more likely to activate compo-nents of the S-N, whereas observing instrumen-tal gestures (come here, look over there, etc.) ismore likely to activate left hemisphere parietaland premotor areas of the AON (Gallagher &Frith 2004). The same network is thought tocarry more semantic content related to socialthan mechanical knowledge (Martin & Weis-berg 2003). The S-N construct integrates thesedifferent components within this circuitry toachieve a unified representation of an animatebeing.

Explicit Intention Understandingin the Social-Network

In some cases the decoding of intentional-ity of an animate being can take place within

the S-N based on inferential processes ratherthan direct mapping or simulation. In an im-portant review, Saxe provided a detailed analy-sis of behavioral errors in intentionality decod-ing experiments to build a case that inferentialreasoning can in fact occur without simulation(Saxe 2005). This is a strong argument againstthe conclusion that simulation, and in particu-lar simulation as supported exclusively withinthe mirror neuron system, is sufficient to coverthe range of decoding operations performedin the real world.

There is strong evidence that theory of mindtasks, in which subjects are required to makeexplicit inferential judgments about the inten-tions of other people’s behavior, will activatecomponents of the S-N (Frith & Frith 2006),particularly in areas not typically associatedwith the AON. For example, Pelphrey showedgreater activation in right STS within the S-Nfor reach-to-grasp actions when the observedaction was unsuccessful compared to actionswhen the hand reached a target (Pelphrey et al.2004). While the authors argued that this wasdue to incidental processing of the action, it canbe argued that observing unsuccessful actionswill evoke explicit intention inference on thepart of the observer as he tries to understandwhy the subject is making an error.

This has been reiterated to a certain degreein a recent fMRI study where subjects wereinstructed to observe an actor performing ac-tions with an object such as a coffee cup (deLange et al. 2008). The actor made ordinarymovements, unusual movements to accomplishordinary intention such as sipping, or ordinarymovements that resulted in atypical intentionssuch as placing the coffee cup against the fore-head. Subjects made a judgment about whetherthe means were ordinary or not on some blocks,and whether the intentions were ordinary ornot on other blocks of trials. Thus, the judgmenttask used explicit “top-down” manipulation ofattention to identify differences of means andintentions. The main finding was greater ac-tivity in components of the S-N including rightSTS, posterior cingulate, and mPFC during the


Figure 2. Relationship of action simulation (green) with underlying functional anatomy. The AON (pink)is particularly sensitive to the decoding of kinematics and interim goals, whereas the S-N (yellow) is sensitiveto higher level inference about intentions, outcomes, or mental attribution. Action simulation is more closelyassociated with the AON, but also requires the S-N. The AON is more strongly associated with bottom-upprocesses and the S-N with top-down processes, but these are not exclusive inputs.

judgment of intentions. Note that the cingu-late and prefrontal areas are rarely observed instudies of action observation without explicitinstructions to judge a task. When judging themeans of an action there was greater activity inthe extrastriate body area for unusual actionsand greater activity in the bilateral IFG for or-dinary actions. What this shows is (1) functionaldistinctions between the AON and S-N, and (2)the sensitivity of the S-N to explicit top-downcontrol.

With this experimental insight, a hierarchi-cal model of action understanding, motivatedin part by Csibra, can be associated with the un-derlying functional anatomy (Csibra & Gergely2007). As shown schematically in Fig. 2, ac-tion understanding can be achieved throughbottom-up processes, what Csibra has termed“resonance,” or through top-down “emula-tion.” Action simulation emerges most com-monly at the interim goal level of represen-tation, although it can be found also at thedirect kinematic level of analysis. Not all out-comes, goals, or kinematics are simulated. Atfirst glance the studies reviewed so far in thischapter would support a functional anatomicmodel where the AON, on the left side of Fig. 2,is driven entirely by bottom-up processes, and

the S-N, on the right, is driven only by top-down emulation.

Bottom-Up Recruitment of theSocial-Network?

Is inference supported by the S-N alwaysdriven by top-down processes? One way totest this is to demonstrate automatic bottom-up activation within the S-N without manipu-lating a subject’s attention or their mental stateduring explicit reasoning. Two recent studiesprovide strong, complementary evidence thatintentions can be decoded in the S-N bybottom-up processing. Both studies relied oncontext effects. In the first, the authors manip-ulated the plausibility of an action that on itsown was quite unusual, such as turning a lightswitch on with the knee rather than the hand(Brass et al. 2007). This makes no sense undernormal circumstances and perfect sense if thehands are carrying a large object. Subjects ob-served these two sorts of actions as well as athird, implausible condition such as perform-ing the same action with the hands loaded withonly a small object that could otherwise be heldin one hand. Parametric modulation of BOLDsignal activity across these three conditions


was present in the STS and posterior STS aswell as the mPFC components of the S-N. Thesubjects were instructed to monitor for inter-ruptions in the movies so that task related mod-ulations of BOLD responses were likely to beincidental.

In the second study, a moving object ratherthan human form was used as a stimulus(Wheatley et al. 2007). Context in this case in-fluenced whether the object was perceived asan inanimate object, such as a spinning topmoving amongst toys, or an ice skater movingaround on an ice pond. When context induceda perception of animacy there was robust acti-vation within components of the S-N, includingthe right STS, mPFC and posterior cingulate.The perception of inanimate motion induceddeactivation within these same areas.

There are many other examples of highlyprosocial actions driven by bottom-up pro-cesses that are increasingly associated with ac-tivity in the S-N. Examples include the auto-matic, imitative behavior people adopt withothers (Chartrand & Bargh 1999), the emer-gence of joint attention by multiple observers(Pelphrey et al. 2003), and the emergence ofshared effort by multiple actors working oncommon goals (Sebanz et al. 2006). The rel-evance of these examples to implicit social pro-cessing has been recently reviewed in detailelsewhere (Frith & Frith 2008).

Top-Down Modulation in the AON

The flip side of the previous argument iswhether more abstract interpretive processescan influence simulation by direct matching.To test this, Liepelt and colleagues used a be-havioral task and showed that motor primingand influenced the attribution of intention ofanother persons while they were performing asimple action (Liepelt et al. 2008). Their datasupport a model in which direct matching canbe modulated top-down by the observer’s in-terpretation of whether the observed move-ment is intended or not. In other words, itis likely that no functional anatomic restric-

tion of the AON to bottom-up processes, orthe S-N to top-down processes, exists. Further-more, while simulation is typically induced bybottom-up resonance, it can also come throughemulation.

Action Observation and the MirrorNeuron System

What additional explanatory power is gainedby invoking a mirror neuron system descrip-tion for action observation and simulation? Thecase is clear that some but not all of the areasof the AON overlap with brain areas associatedwith motor execution. Knowing this anatomicspecificity, can we make a stronger model ofaction understanding? Before answering thisit is important to address the two main as-sumptions inherent in the MNS explanation:(1) there are mirror neurons in humans, and (2)the functional overlaps observed on imagingbetween execution and observation are drivenby populations of mirror neurons. Knowing thetruth of these assumptions allows us to claimthat the mirror neuron is a putative substratefor simulation processes. To date the first as-sumption has not been confirmed. The secondhas mostly been evaluated through experimentswhere there might be mirror neurons and re-searchers employ reverse inference. For exam-ple, many fMRI studies have used imitationtasks to define the MNS using either finger orfacial movements (Heiser et al. 2003; Iacoboniet al. 1999; Koski et al. 2003; Koski et al.2002). Both parietal and inferior frontal areasare active during imitative execution or obser-vation. However, imitation requires additionaltask demands including executive control work-ing memory and semantic as well as motoricroutes to action (Rumiati et al. 2005; Tessari &Rumiati 2004). Furthermore, the presence ofactivity in the MNS during action observationis insufficient evidence toward a conclusion thatthe same area would be active during executionin the same subject. This is a form of reverseinference that, from a statistical perspective, is


insufficient if one is to conclude that there arespecific neural substrates for action and per-ception based on observation alone (Poldrack2006, 2008). Ideally, it would be desirable tocompare actions decoded by observation andgenerated by execution in the same subjects andthen to experiment to further this argument(Dinstein et al. 2007); however, the confinesof fMRI constrain the types of movementsthat can exist. A strong test of the secondassumption would be to demonstrate repe-tition suppression within mirror neuron ar-eas for observing and then to execute thesame action, but not a different action.However, no study has found evidence forthis cross agent RS in the putative MNS(Dinstein et al. 2007; Dinstein et al. 2008).

If we put these criticisms aside and allowingfor the existence of a human MNS, the ex-istence of a human mirror neuron becomes auseful but not necessarily essential neural mech-anism that could be used to explain how simu-lation might occur. These cells carry informa-tion about an action that is agent-independent(Gallese et al. 1996). In this sense they are oneform of canonical neuron that represents a classof information at a higher level of abstraction.The key point is that there are other types ofneurons capable of higher level representationsspanning goals and kinematics (Shen & Alexan-der 1997), intentions (Cisek & Kalaska 2004), orcategories and concepts (Miller et al. 2002), andthese other types of neurons could also supportsimulation. Furthermore, the growing numberof human imaging studies showing activationduring action observation and simulation in thecortex outside of the putative human MNS sug-gests that even if there is an MNS system, itsexistence is an insufficient explanation for allfacets of action understanding in humans.

Conclusion

We navigate through a complex world andchoose our behavior based on insights derivedfrom another’s movements, goals, intentions,

and emotions. The review does not make theclaim that embodied cognition is the only wayto understand another’s behavior. It is temptingto conclude that simulation for the decoding ofaction is restricted to the AON and the theory ofmind attribution within a S-N. These latter the-ories are sometimes portrayed as oppositionalin that intention is established when either onenetwork or the other is engaged. While thereis strong evidence that both of these functionalsystems are engaged in action understanding,the posing of the functional anatomy as an ei-ther/or problem undermines the complexityof the problem at hand. This oppositional con-struct is likely to be a result of experimentaldesign rather than a physiological or anatomicmechanism. I have tried to make the case thatthese are compatible, possibly synergistic sys-tems. On both computational and conceptualgrounds, the decoding of others’ intentions re-quires the resolution of a cascade of operationsincluding the evaluation of physical actions, theclarification of who the actor is, establishing thecontext in which an action is performed, andgenerating possible aims of the actor. The stud-ies on the S-N and AON reviewed above showthat they are not mutually exclusive, not limitedto top-down or bottom-up processes, and notstrictly hierarchical with respect to each other.Each can be supraordinate in terms of a de-coding hierarchy, depending on the nature ofthe action and context. These relationships areshown in Fig. 2.

More broadly, growing evidence from cog-nitive neuroscience supports the case for theexistence of embodied cognition as a distinctknowledge domain localized within the AONand S-N. Much as we think of episodic memoryas a special knowledge system with dedicatedencoding and retrieval processes, the same ar-gument can be made for embodied cognition.What is particularly interesting about embod-ied cognition is that it involves a direct interac-tion between what we do and what we see. Itis based on our skills in the organization of ac-tion and the complementary ability to perceiveaction. As in other knowledge domains, there


is capacity for generalization, abstraction, andknowledge-sharing irrespective of language.

Acknowledgments

The author thanks Antonia Hamilton, EmilyCross, Stephanie Ortigue, and the staffs at theDartmouth and UCSB Brain Imaging Centers.Supported in part by PHS grants NS33504 andNS44393, the Dana Foundation, and a grantDAAD19-03-D-0004 to the Institute for Col-laborative Biotechnologies from the U.S. ArmyResearch Office.

Conflicts of Interest

The author declares no conflicts of interest.

References

Adolphs, R. (2001). The neurobiology of social cognition.Curr. Opin. Neurobiol., 11, 231–239.

Aglioti, S. M., Cesari, P., Romani, M., et al. (2008). Actionanticipation and motor resonance in elite basketballplayers. Nat. Neurosci., 11, 1109–1116.

Allison, T., Puce, A., & McCarthy, G. (2000). Social per-ception from visual cues: Role of the STS region.Trends Cogn. Sci., 4, 267–278.

Anderson, A. K., & Phelps, E. A. (2001). Lesions of the hu-man amygdala impair enhanced perception of emo-tionally salient events. Nature, 411, 5.

Anquetil, T., & Jeannerod, M. (2007). Simulated actionsin the first and in the third person perspectives sharecommon representations. Brain Res., 1130, 125–129.

Ansuini, C., Santello, M., Massaccesi, S., et al. (2006).Effects of end-goal on hand shaping. J. Neurophysiol.,95, 2456–2465.

Aziz-Zadeh, L., Maeda, F., Zaidel, E., et al. (2002). Later-alization in motor facilitation during action observa-tion: A TMS study. Experimental Brain Research, 144,127–131.

Badre, D., & D’Esposito, M. (2007). Functional magneticresonance imaging evidence for a hierarchical orga-nization of the prefrontal cortex. J. Cogn. Neurosci.,19, 2082–2099.

Baldissera, F., Cavallari, P., Craighero, L., et al. (2001).Modulation of spinal excitability during observationof hand actions in humans. Eur. J. Neurosci., 13, 190–194.

Beauchamp, M. S., Lee, K. E., Haxby, J. V., et al. (2003).fMRI responses to video and point-light displays ofmoving humans and manipulable objects. J. Cogn.

Neurosci., 15, 991–1001.Bekkering, H., Wohlschlager, A., & Gattis, M. (2000).

Imitation of gestures in children is goal-directed. Q.

J. Exp. Psychol. A, 53, 153–164.Binkofski, F., Dohle, C., Posse, S., et al. (1998). Hu-

man anterior intraparietal area subserves prehen-sion: A combined lesion and functional MRI activa-tion study. Neurology, 50, 1253–1259.

Borra, E., Belmalih, A., Calzavara, R., et al. (2008). Corti-cal connections of the macaque anterior intraparietal(AIP) area. Cereb Cortex, 18, 1094–1111.

Botvinick, M. M. (2008). Hierarchical models of behaviorand prefrontal function. Trends Cogn. Sci. (Regul. Ed.),12, 201–208.

Brass, M., Schmitt, R. M., Spengler, S., et al. (2007).Investigating action understanding: Inferential pro-cesses versus action simulation. Curr. Biol., 17, 2117–2121.

Britten, K. H., Shadlen, M. N., Newsome, W. T., et al.(1993). Responses of neurons in macaque MT tostochastic motion signals. Vis. Neurosci., 10, 1157–1169.

Brooks, R. (April 1991). Intelligence without reason. Pa-per presented at the Proceedings of the International Joint

Conference on Artificial Intelligence, San Mateo, CA.Buccino, G., Binkofski, F., Fink, G. R., et al. (2001). Action

observation activates premotor and parietal areasin a somatotopic manner: An fMRI study. Eur. J.

Neurosci., 13, 400–404.Buccino, G., Binkofski, F., & Riggio, L. (2004). The mirror

neuron system and action recognition. Brain Lang.,89, 370–376.

Buccino, G., Lui, F., Canessa, N., et al. (2004). Neuralcircuits involved in the recognition of actions per-formed by nonconspecifics: An fMRI study. J. Cogn.

Neurosci., 16 , 114–126.Buxbaum, L. J., Kyle, K. M., & Menon, R. (2005). On be-

yond mirror neurons: Internal representations sub-serving imitation and recognition of skilled object-related actions in humans. Brain Res. Cogn. Brain Res.,25, 226–239.

Calvo-Merino, B., Glaser, D. E., Grezes, J., et al. (2005).Action observation and acquired motor skills: AnfMRI study with expert dancers. Cerebral Cortex, 15,1243–1249.

Calvo-Merino, B., Grezes, J., Glaser, D. E., et al. (2006).Seeing or doing? Influence of visual and motor fa-miliarity in action observation. Curr. Biol., 16 , 1905–1910.

Celnik, P., Stefan, K., Hummel, F., et al. (2006). Encod-ing a motor memory in the older adult by actionobservation. NeuroImage, 29, 677–684.


Chao, L. L., & Martin, A. (2000). Representation of ma-nipulable man-made objects in the dorsal stream.NeuroImage, 12, 478–484.

Chartrand, T. L., & Bargh, J. A. (1999). Thechameleon effect: The perception-behavior link andsocial interaction. J. Pers. Soc. Psychol., 76 , 893–910.

Cisek, P., & Kalaska, J. F. (2004). Neural correlates ofmental rehearsal in dorsal premotor cortex. Nature,431, 993–996.

Cooper, R., & Shallice, T. (2006). Hierarchical schemasand goals in the control of sequential behavior. Psy-

chol. Rev., 113, 887–916; discussion 917–831.Corbetta, M., Kincade, J. M., & Shulman, G. L. (2002).

Neural systems for visual orienting and their relation-ships to spatial working memory. J. Cogn. Neurosci., 14,508–523.

Craighero, L., Bello, A., Fadiga, L., et al. (2002). Handaction preparation influences the responses to handpictures. Neuropsychologia, 40, 492–502.

Craighero, L., Fadiga, L., Rizzolatti, G., et al. (1999). Ac-tion for perception: A motor-visual attentional effect.J. Exp. Psychol. Hum. Percept. Perform., 25, 1673–1692.

Cross, E., Kraemer, D., Hamilton, A., et al. (2008). Sen-sitivity of the action observation network to physicaland observational learning. Cerebral Cortex, 12.

Cross, E. S., Hamilton, A. F., & Grafton, S. T. (2006).Building a motor simulation de novo: Observation ofdance by dancers. NeuroImage, 31, 1257–1267.

Csibra, G., & Gergely, G. (2007). ‘Obsessed with goals’:Functions and mechanisms of teleological interpre-tation of actions in humans. Acta. Psychologica, 124,60–78.

Culham, J. C., Danckert, S. L., DeSouza, J. F., et al. (2003).Visually guided grasping produces fMRI activationin dorsal but not ventral stream brain areas. Exp.

Brain Res., 153, 180–189.de C. Hamilton, A. F., & Grafton, S. T. (2007). The motor

hierarchy: From kinematics to goals and intentions.In P. Haggard, Y. Rossetti & M. Kawato (Eds.), Sen-

sorimotor Foundations of Higher Cognition (Vol. Attentionand Performance XXII, pp. 381–408). Oxford: Ox-ford University Press.

de Lange, F. P., Spronk, M., Willems, R. M., et al. (2008).Complementary systems for understanding actionintentions. Curr. Biol., 18, 454–457.

Demiris, Y., & Simmons, G. (2006). Perceiving the un-usual: Temporal properties of hierarchical motorrepresentations for action perception. Neural Networks:

Official J. Inter. Neural Network Soc., 19, 272–284.Dinstein, I., Hasson, U., Rubin, N., et al. (2007). Brain

areas selective for both observed and executed move-ments. J. Neurophysiol., 98, 1415–1427.

Dinstein, I., Thomas, C., Behrmann, M., et al. (2008). Amirror up to nature. Curr. Biol., 18, R13–18.

Fadiga, L., Fogassi, L., Pavesi, G., et al. (1995). Motorfacilitation during action observation: A magneticstimulation study. J. Neurophysiol., 73, 2608–2611.

Ferro, J. M., Martins, I. P., Mariano, G., et al. (1983).CT scan correlates of gesture recognition. J. Neurol.

Neurosurg. Psych., 46 , 943–952.Filimon, F., Nelson, J. D., Hagler, D. J., et al. (2007).

Human cortical representations for reaching: Mir-ror neurons for execution, observation, and imagery.NeuroImage, 37, 1315–1328.

Fischer, M. H., & Zwaan, R. A. (2008). Embodied lan-guage: A review of the role of the motor system inlanguage comprehension. Quart. J. Exp. Psych. (2006),61, 825–850.

Frak, V., Paulignan, Y., Jeannerod, M., et al. (2006). Pre-hension movements in a patient (AC) with posteriorparietal cortex damage and posterior callosal section.Brain Cogn., 60, 43–48.

Frey, S. H. (2008). Tool use, communicative gesture andcerebral asymmetries in the modern human brain.Philos. Trans. R. Soc. Lond., B., Biol. Sci., 363, 1951–1957.

Frey, S. H., & Gerry, V. E. (2006). Modulation of neuralactivity during observational learning of actions andtheir sequential orders. J. Neurosci., 26 , 13194–13201.

Frey, S. H., Vinton, D., Newman-Norlund, R., et al.(2005). Cortical topography of human anterior intra-parietal cortex active during visually-guided grasp-ing. Cogn. Brain Res., 23, 397–405.

Frith, C. D., & Frith, U. (1999). Interacting minds—Abiological basis [see comments]. Science, 286 , 1692–1695.

Frith, C. D., & Frith, U. (2006). How we predict what otherpeople are going to do. Brain Res., 1079, 36–46.

Frith, C. D., & Frith, U. (2008). Implicit and explicit pro-cesses in social cognition. Neuron, 60, 503–510.

Gallagher, H. L., & Frith, C. D. (2004). Dissociable neuralpathways for the perception and recognition of ex-pressive and instrumental gestures. Neuropsychologia,42, 1725–1736.

Gallese, V., Fadiga, L., Fogassi, L., et al. (1996). Actionrecognition in the premotor cortex. Brain, 119, 593–609.

Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifyingview of the basis of social cognition. Trends Cogn. Sci.,8, 396–403.

Gazzola, V., Aziz-Zadeh, L., & Keysers, C. (2006). Em-pathy and the somatotopic auditory mirror systemin humans. Curr. Biol., 16 , 1824–1829.

Gentilucci, M., Negrotti, A., & Gangitano, M. (1997).Planning an action. Exp. Brain Res., 115, 116–128.

Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E.(1986). Neuronal population coding of movementdirection. Science, 233, 1416–1419.


Goldenberg, G., & Karnath, H. O. (2006). The neuralbasis of imitation is body part specific. J. Neurosci.,26 , 6282–6287.

Grafton, S. T., Arbib, M. A., Fadiga, L., et al. (1996).Localization of grasp representations in humans bypositron emission tomography. 2. Observation com-pared with imagination. Exp. Brain Res., 112, 103–111.

Grafton, S. T., Fadiga, L., Arbib, M. A., et al. (1997).Premotor cortex activation during observation andnaming of familiar tools. NeuroImage, 6 , 231–236.

Grezes, J., & Decety, J. (2001). Functional anatomy ofexecution, mental simulation, observation, and verbgeneration of actions: A meta-analysis. Hum. Brain

Mapp., 12, 1–19.Grezes, J., Tucker, M., Armony, J., et al. (2003). Ob-

jects automatically potentiate action: An fMRI studyof implicit processing. Eur. J. Neurosci., 17, 2735–2740.

Grill-Spector, K., Henson, R., & Martin, A. (2006). Rep-etition and the brain: Neural models of stimulus-specific effects. Trends Cogn. Sci. (Regul. Ed.), 10, 14–23.

Grill-Spector, K., & Malach, R. (2001). fMR-adaptation:A tool for studying the functional properties of hu-man cortical neurons. Acta. Psychol. (Amst.), 107, 293–321.

Hamilton, A., Wolpert, D., & Frith, U. (2004). Your ownaction influences how you perceive another person’saction. Curr. Biol., 14, 493–498.

Hamilton, A. F., & Grafton, S. T. (2006). Goal repre-sentation in human anterior intraparietal sulcus. J.

Neurosci., 26 , 1133–1137.Hamilton, A. F., Joyce, D. W., Flanagan, J. R., et al. (2007).

Kinematic cues in perceptual weight judgment andtheir origins in box lifting. Psychol. Res., 71, 13–21.

Hamilton, A. F. d. C., & Grafton, S. T. (2008). Actionoutcomes are represented in human inferior fron-toparietal cortex. Cereb. Cortex, 18, 1160–1168.

Handy, T. C., Grafton, S. T., Shroff, N., et al. (2003).Graspable objects grab attention when the potentialfor action is recognized. Nat. Neurosci., 6 , 421–427.

Hari, R., Forss, N., Avikainen, S., et al. (1998). Activa-tion of human primary motor cortex during actionobservation: A neuromagnetic study. Proc. Natl. Acad.

Sci. USA, 95, 15061–15065.Heilman, K. M., Rothi, L. J., & Valenstein, E. (1982). Two

forms of ideomotor apraxia. Neurology, 32, 342–346.Heiser, M., Iacoboni, M., Maeda, F., et al. (2003). The

essential role of Broca’s area in imitation. Eur. J. Neu-

rosci., 17, 1123–1128.Iacoboni, M., Molnar-Szakacs, I., Gallese, V., et al. (2005).

Grasping the intentions of others with one’s ownmirror neuron system. PLoS Biol., 3, e79.

Iacoboni, M., Woods, R. P., Brass, M., et al. (1999). Cor-

tical mechanisms of human imitation. Science, 286 ,2526–2528.

Jacob, P., & Jeannerod, M. (2005). The motor theory ofsocial cognition: A critique. Trends Cogn. Sci., 9, 21–25.

Janata, P., & Grafton, S. T. (2003). Swinging in the brain:Shared neural substrates for behaviors related to se-quencing and music. Nat. Neurosci., 6 , 682–687.

Johnson-Frey, S. H., Maloof, F. R., Newman-Norlund,R., et al. (2003). Actions or hand-object interactions?Human inferior frontal cortex and action observa-tion. Neuron, 39, 1053–1058.

Koski, L., Iacoboni, M., Dubeau, M. C., et al. (2003).Modulation of cortical activity during different imi-tative behaviors. J. Neurophysiol., 89, 460–471.

Koski, L., Wohlschlager, A., Bekkering, H., et al. (2002).Modulation of motor and premotor activity duringimitation of target-directed actions. Cereb. Cortex, 12,847–855.

Kourtzi, Z., & Kanwisher, N. (2000). Cortical regionsinvolved in perceiving object shape. J. Neurosci., 20,3310–3318.

Krekelberg, B., Boynton, G. M., & van Wezel, R. J. (2006).Adaptation: From single cells to BOLD signals. Trends

Neurosci., 29, 250–256.Lestou, V., Pollick, F. E., & Kourtzi, Z. (2008). Neural

substrates for action understanding at different de-scription levels in the human brain. J. Cogn. Neurosci.,20, 324–341.

Liepelt, R., Cramon, D. Y., & Brass, M. (2008). Whatis matched in direct matching? Intention attributionmodulates motor priming. J. Exp. Psych. Hum. Perc.

Perf., 34, 578–591.Lotze, M., Montoya, P., Erb, M., et al. (1999). Activa-

tion of cortical and cerebellar motor areas duringexecuted and imagined hand movements: An fMRIstudy. J. Cogn. Neurosci., 11, 491–501.

Maccotta, L., & Buckner, R. L. (2004). Evidence forneural effects of repetition that directly correlatewith behavioral priming. J. Cogn. Neurosci., 16 , 1625–1632.

Martin, A., & Weisberg, J. (2003). Neural foundations forunderstanding social and mechanical concepts. Cogn.

Neuropsych., 20, 575–587.Miller, E. K., Freedman, D. J., & Wallis, J. D. (2002).

The prefrontal cortex: Categories, concepts andcognition. Philos. Trans. R. Soc. Lond., B., Biol. Sci.,357, 1123–1136.

Mitchell, J. P., Heatherton, T. F., & Macrae, C. N. (2002).Distinct neural systems subserve person and objectknowledge. Proc. Natl. Acad. Sci. USA, 99, 15238–15243.

Murata, A., Gallese, V., Kaseda, M., et al. (1996). Parietalneurons related to memory-guided hand manipula-tion. J. Neurophysiol., 75, 2180–2186.


Murata, A., Gallese, V., Luppino, G., et al. (2000). Selec-tivity for the shape, size, and orientation of objectsfor grasping in neurons of monkey parietal area AIP.J. Neurophysiol., 83, 2580–2601.

Niedenthal, P. M. (2007). Embodying emotion. Science,316 , 1002–1005.

Pazzaglia, M., Smania, N., Corato, E., et al. (2008).Neural underpinnings of gesture discrimination inpatients with limb apraxia. J. Neurosci., 28, 3030–3041.

Pelphrey, K. A., Morris, J. P., & McCarthy, G. (2004).Grasping the intentions of others: The perceived in-tentionality of an action influences activity in thesuperior temporal sulcus during social perception. J.

Cogn. Neurosci., 16 , 1706–1716.Pelphrey, K. A., Singerman, J. D., Allison, T., et al. (2003).

Brain activation evoked by perception of gaze shifts:The influence of context. Neuropsychologia, 41, 156–170.

Poldrack, R. A. (2006). Can cognitive processes be in-ferred from neuroimaging data? Trends Cogn. Sci.

(Regul. Ed.), 10, 59–63.Poldrack, R. A. (2008). The role of fMRI in cognitive neu-

roscience: Where do we stand? Curr. Opin. Neurobiol.,18, 223–227.

Rizzolatti, G., & Craighero, L. (2004). The mirror-neuronsystem. Annu. Rev. Neurosci., 27, 169–192.

Rizzolatti, G., Fadiga, L., Matelli, M., et al. (1996). Local-ization of grasp representations in humans by PET:1. Observation versus execution. Exp. Brain Res., 111,246–252.

Rothi, L. J., Heilman, K. M., & Watson, R. T. (1985).Pantomime comprehension and ideomotor apraxia.J. Neurol. Neurosurg. Psych., 48, 207–210.

Rumiati, R. I., Weiss, P. H., Tessari, A., et al. (2005). Com-mon and differential neural mechanisms supportingimitation of meaningful and meaningless actions. J.

Cogn. Neurosci., 17, 1420–1431.Sakata, H., Tsutsui, K., & Taira, M. (2005). Toward an

understanding of the neural processing for 3D shapeperception. Neuropsychologia, 43, 151–161.

Saxe, R. (2005). Against simulation: the argument fromerror. Trends Cogn. Sci. (Regul. Ed.), 9, 174–179.

Saxe, R., Xiao, D. K., Kovacs, G., et al. (2004). A regionof right posterior superior temporal sulcus respondsto observed intentional actions. Neuropsychologia, 42,1435–1446.

Saygin, A. P. (2007). Superior temporal and premotorbrain areas necessary for biological motion percep-tion. Brain, 130, 2452–2461.

Saygin, A. P., Wilson, S. M., Dronkers, N. F., et al. (2004).Action comprehension in aphasia: Linguistic andnon-linguistic deficits and their lesion correlates. Neu-

ropsychologia, 42, 1788–1804.Saygin, A. P., Wilson, S. M., Hagler, D. J., et al. (2004).

Point-light biological motion perception activateshuman premotor cortex. J. Neurosci., 24, 6181–6188.

Schubotz, R. I., & von Cramon, D. Y. (2001). Functionalorganization of the lateral premotor cortex: fMRIreveals different regions activated by anticipation ofobject properties, location and speed. Brain Res. Cogn.

Brain Res., 11, 97–112.Schubotz, R. I., & von Cramon, D. Y. (2004). Sequences

of abstract nonbiological stimuli share ventral pre-motor cortex with action observation and imagery.J. Neurosci., 24, 5467–5474.

Sebanz, N., Bekkering, H., & Knoblich, G. (2006). Jointaction: Bodies and minds moving together. Trends

Cogn. Sci., 10, 70–76.Shen, L., & Alexander, G. E. (1997). Preferential repre-

sentation of instructed target location versus limbtrajectory in dorsal premotor area. J. Neurophysiol.,77, 1195–1212.

Shmuelof, L., & Zohary, E. (2005). Dissociation betweenventral and dorsal fMRI activation during object andaction recognition. Neuron, 47, 457–470.

Shmuelof, L., & Zohary, E. (2006). A mirror representa-tion of others’ actions in the human anterior parietalcortex. J. Neurosci., 26 , 9736–9742.

Stephan, K. M., Fink, G. R., Passingham, R. E., et al.(1995). Functional anatomy of the mental represen-tation of upper extremity movements in healthy sub-jects. J. Neurophysiol., 73, 373–386.

Tenenbaum, G., Levy-Kolker, N., Bar-Eli, M., et al.(1994). Information recall of younger and olderskilled athletes: The role of display complexity, at-tentional resources and visual exposure duration. J.

Sports Sci., 12, 529–534.Tessari, A., & Rumiati, R. I. (2004). The strategic

control of multiple routes in imitation of actions.J. Exp. Psychol. Hum. Percept Perform, 30, 1107–1116.

Thelen, E., Schoner, G., Scheier, C., et al. (2001). Thedynamics of embodiment: A field theory of infantperseverative reaching. Behav. Brain Sci., 24, 1–34;discussion 34–86.

Thompson, J. C., Hardee, J. E., Panayiotou, A., et al.(2007). Common and distinct brain activation toviewing dynamic sequences of face and hand move-ments. NeuroImage, 37, 966–973.

Tsutsui, K., Taira, M., & Sakata, H. (2005). Neural mech-anisms of three-dimensional vision. Neurosci. Res., 51,221–229.

Ulloa, E. R., & Pineda, J. A. (2007). Recognition of point-light biological motion: Mu rhythms and mirror neu-ron activity. Behav. Brain Res., 183, 188–194.

Urgesi, C., Candidi, M., Ionta, S., et al. (2007). Represen-tation of body identity and body actions in extras-triate body area and ventral premotor cortex. Nat.

Neurosci., 10, 30–31.


Urgesi, C., Moro, V., Candidi, M., et al. (2006). Mappingimplied body actions in the human motor system. J.

Neurosci., 26 , 7942–7949.Valyear, K. F., Cavina-Pratesi, C., Stiglick, A. J., et al.

(2007). Does tool-related fMRI activity within theintraparietal sulcus reflect the plan to grasp? Neu-

roImage, 36 (Suppl 2), T94–T108.Weisberg, J., van Turennout, M., & Martin, A. (2007).

A neural system for learning about object function.Cereb. Cortex, 17, 513–521.

Wheatley, T., Milleville, S. C., & Martin, A. (2007). Un-

derstanding animate agents: distinct roles for the so-cial network and mirror system. Psych. Sci.: A J. Amer.

Psych. Soc./APS, 18, 469–474.Wig, G. S., Grafton, S. T., Demos, K. E., et al. (2005). Re-

ductions in neural activity underlie behavioral com-ponents of repetition priming. Nat. Neurosci., 8, 1228–1233.

Zentgraf, K., Stark, R., Reiser, M., et al. (2005). Differen-tial activation of pre-SMA and SMA proper duringaction observation: Effects of instructions. NeuroIm-

age, 26 , 662–672.

Documents

Embodied Cognition and the Simulation of Action to ......simulation of other people’s behavior within our own nervous system. This review fo-cuses on aspects of action understanding