Lecture Overview

• Regier System: Limitations• Image Schemas: Recap• Force Dynamic Schemas: Recap• Sensory-Motor Schemas

– Evidence in Primates– Evidence in Humans

• Do motor schemas play a role in language?• A Computational Model of Motor Schemas• Learning Hand Action terms (Bailey)• Cultural Schemas and frames

Limitations

• Scale

• Uniqueness/Plausibility

• Grammar

• Abstract Concepts

• Inference

• Representation

Force Dynamics, modals and causatives

• A gust of wind made the pages of my book turn.

• The appearance of the headmaster made the pupils calm down.

• The breaking of the dam let the water flow from the storage lake.

• The abating of the wind let the sailboat slow down.

Schematic Representation(Talmy)

FD Patterns

• A gust of wind made the pages of my book turn.

• The appearance of the headmaster made the pupils calm down.

• The breaking of the dam let the water flow from the storage lake.

• The abating of the wind let the sailboat slow down.

Semantic field Force-dynamics representation

Physical The ball kept rolling along the green

Physical/psychological 

John can't go out of the house

Intra-psychological He refrained from closing the door

Intra-psychological (lexicalized)

She's civil to him 

Socio-psychological  She gets to go to the park

Closed Class vs. Open Class terms

• Image Schematic and Force Dynamic Patterns are expressed by closed class terms in language– Prepositions (in, on, into, out)

– Modals and causatives (make, let, might, prevent)

• How about open class terms?– Verbs and Event descriptions –

• Motor Schemas - Embodied

– Is there evidence for motor schemas and if so are they used in language?

• Frames – Composed from Image and motor schemas -Cultural

Coordination

• PATTERN GENERATORS, separate neural networks that control each limb, can interact in different ways to produce various gaits.

– In ambling (top) the animal must move the fore and hind leg of one flank in parallel.

– Trotting (middle) requires movement of diagonal limbs (front right and back left, or front left and back right) in unison.

– Galloping (bottom) involves the forelegs, and then the hind legs, acting together

Sensory-Motor Schemas

•A sensory (perceptual) schema determines whether a given situation is present in the environment. – Object Detection– Spatial relation recognition

• Execution of current plans is made up of motor schemas which are akin to control systems but distinguished by the fact that they can be combined to form coordinated control programs

• Sensory and Motor Schemas are closely coupled circuits sensory-motor schemas.

The neural theory

Human concepts are embodied. Many concepts make direct use of the sensory-motor capacities of our body-brain system.

• Many of these capacities are also present in non-human primates.

• Let us look at concepts that make use of our sensory-motor capacities, ex. Grasp.

Area F5

General Purpose Neurons:General Grasping

General Holding

General Manipulating

AA Grasping with the mouth

BB Grasping with the cl. hand

CC Grasping with the ipsil. hand

General Purpose Neurons in Area F5General Purpose Neurons in Area F5

(Rizzolatti et al. 1988)

General Purpose Neurons Achieve

Partial Universality: Their firing correlates with a goal-oriented action of a general type, regardless ofeffector or manner.

Area F5cArea F5c

Convexity region of F5:

Mirror neurons

F5c-PFF5c-PF

Rizzolatti et al. 1998

(Rizzolatti et al. Cog Brain Res 1996)

Observed Action

Executed Action

Executed Action

Strictly congruent mirror neurons (~30%)

Category Loosening in Mirror Neurons (~60%)(Gallese et al. Brain 1996)

A [C] is Observe (Execute) Precision Grip (Prototype) B [D] is Observe (Execute) Whole Hand Pre-hension

The F5c-PF circuit

Links premotor area F5c and parietal area PF (or 7b).

Contains mirror neurons.

Mirror neurons discharge when:

Subject (a monkey) performs various types of goal-related hand actions

and when:

Subject observes another individual performing similar kinds of actions

Phases

Area F5 contains clusters of neurons that control distinctphases of grasping: opening fingers, closing fingers.

Jeannerod, et al., 1995; Rizzolatti, et al., 2001.

Mirror Neurons Achieve

Partial Universality, since they code an action regardless of agent, patient,modality (action/observation/hearing),manner, location.

Partial Role Structure, since they codean agent role and a purpose role.

The Agent Role: In acting, the Subject is an agent of that action.In observing, the Subject identifies the agent ofthe action as having the same role as he haswhen he is acting – namely, the agent role.

The Purpose Role: Mirror neurons fire only forpurposeful actions.

Mirror Neurons Achieve

Category tightening and loosening

The F4-VIP circuit

The F4-VIP Circuit

Links premotor area F4 and parietal area VIP.

Transforms the spatial position of objects in peri-personal space

into motor programs for interacting with those objects.

Examples:

Reaching for the objects, or moving away from them

with various parts of your body such as the arm or head.

Area F4Area F4

Arm reaching

Head turning

Somato-Centered Bimodal RFs in area F4Somato-Centered Bimodal RFs in area F4

(Fogassi et al. 1996)

Somato-Centered Bimodal RFs in area VIPSomato-Centered Bimodal RFs in area VIP

(Colby and Goldberg 1999)

AIP and F5 (Grasping) in Monkey

F5 - grasp commands inpremotor cortexGiacomo Rizzolatti

AIP - grasp affordancesin parietal cortexHideo Sakata

Size Specificity in a Single AIP Cell

•This cell is selective toward small objects, somewhat independent of object type ( Hideo Sakata)

•Note: Some cells show size specificity; others do not.

Summary of Fronto-Parietal Circuits

Motor-Premotor/Parietal Circuits PMv (F5ab) – AIP Circuit

“grasp” neurons – fire in relation to movements of hand prehension necessary to grasp object

F4 (PMC) (behind arcuate) – VIP Circuit transforming peri-personal space coordinates so can move toward objects

PMv (F5c) – PF Circuit F5c different mirror circuits for grasping, placing or manipulating object

Together suggest cognitive representation of the grasp, active in action imitation and action recognition

MULTI-MODAL INTEGRATION

The premotor and parietal areas, rather than havingseparate and independent functions, are neurally integratednot only to control action, but also to serve the function ofconstructing an integrated representation of:

(a) Actions, together with (b) objects acted on, and (c) locations toward which actions are directed.

In these circuits sensory inputs are transformed in order toaccomplish not only motor but also cognitive tasks, such asspace perception and action understanding.

Modeling Motor Schemas

• Relevant requirements (Stromberg, Latash, Kandel, Arbib, Jeannerod, Rizzolatti)

– Should model coordinated, distributed, parameterized control programs required for motor action and perception.

– Should be an active structure.

– Should be able to model concurrent actions and interrupts.

– Should model hierarchical control (higher level motor centers to muscle extensor/flexors.

• Computational model called x-schemas (http://www.icsi.berkeley.edu/NTL)