Lecture 13. The FARS model of Control of Reaching and Grasping 2 Reading Assignments:

1Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 13. FARS 2

Michael Arbib: CS564 - Brain Theory and Artificial Intelligence

University of Southern California, Fall 2001

Lecture 13. The FARS model of Control of Reaching and Grasping 2

Reading Assignments:FARS Model:Fagg, A. H., and Arbib, M. A., 1998, Modeling Parietal-Premotor Interactions in Primate Control of Grasping, Neural Networks, 11:1277-1303.

The class also reviewed material on serial order and basal ganglia contained in the slides for Lecture 9. FARS Model 1


Coarse Coding/Population Coding

To code some variable x lying in an interval [a,b) we could take n cells, with cell i (i = 0, …, n-1) firing if and only if the current value of x lies in the ith subinterval

In coarse coding, we achieve much greater discrimination by taking into account the continuously varying firing level fi of each cell, and then we can decode values of x

actually varying across each interval, using some such formula as

Note: In the Georgopoulos study we saw “negative votes” for firing below the neuron’s resting discharge rate.

n

iaba

n

iaba

)1)((,

)(

1

0i

1

0i

f

1f

n

i

n

i nab

a


The “Visual Front End” of the FARS Model

Visual Cortex

Parietal Cortex

VIP

PIPAIP

F4

How (dorsal)

IT

What (ventral)

(position)

(shape, size, orientation)

(object/grasp transform)

F5

(grasp type)

(arm goal position)


cIPS* IT connections are hard-wired

for a simple set of objectsCylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Cylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Cylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Note the use of coarse coding* In the paper we spoke of PIP where we now say cIPS.


cIPS AIP connections are hard-wired

for a simple set of affordancesCylinder

general

diameter

length

narrow wide

short long

precisionprecision aperture = 20mm

precision precision aperture = 60mm

PIP

AIP

A B C D


IT AIP

The mapping from object identity in IT to maps directly to both the grasp type and the aperture of grasp in AIP when the nature of the object implies such data:

E.g., in the case of AT, the projection from IT can provide the necessary grasp type and parameters for a lipstick but not for a cylinder.

precisionprecision aperture = 20mm

precision precision aperture = 60mm

IT

AIP

A B C D

narrow cylinder (bottle cap) wide cylinder (jar top)

A bottle cap activates a precision grasp with a narrow aperture.

A jar top maps to a precision grasp with a wide aperture.


cIPS IT connections are hard-wired

for a simple set of objectsCylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Cylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Cylinder Box Sphere

PIP

general

diameter

length

general

width

length

general

diameter

height

IT

Note the use of coarse coding


F5 activity during execution of a precision graspThe top two traces show the position of the thumb and index finger.

Left: The next five traces represent the average firing rate of five F5 neurons (set-, extension-, flexion-, hold-, and release-related). The remaining five traces represent the various external (Ready, Go, Go2) and internal (SII) triggering signals.

Right: Illustrating the temporally distributed coding of F5 cells.


Positioning F2, F6 and Areas 46 and SII in Monkey


Prefrontal Influences on F5

F4

Inferior Premotor

Cortex

F5(grasp type)

F6

46

(arm goal position)

F2 (abstract stimuli)

pre-SMA

Dorsal premotor cortex

Fro

nta

l Corte

x


Grasp Selection in F5

Within F5, the active grasps compete through a winner-take-all

Area 46, working in conjunction with F6, supplies task-dependent biases for grasp selection in F5, based upon

task requirements (such as what is going to be done after the grasp), or

a working memory of a recently executed grasp.

The biasing can be on the class of grasp (e.g. power versus precision), or include the parameters of the grasp (e.g. width of aperture) mechanism which incorporates any biases that might be received from area 46.


Supplementary Motor Area (SMA) in Monkey

a unilateral lesion of the SMA disrupts the monkey's ability to allocate his hands to different subtasks of a bimanual task (Brinkman, 1984)

SMA is involved in the temporal organization of complex movements (Tanji, 1994).

Luppino, Matelli, Camarda, & Rizzolatti subdivide SMA:

SMA-proper (F3; the caudal region) has heavy projections to the limb regions of F1 and related portions of the spinal cord.

F6 (pre-SMA) does not project to the spinal cord, and has only moderate projections to areas F3 and F2 (the dorsal premotor cortex), but has a very heavy projection to area F5.


Pre-SMA

F6 (pre-SMA) has a very heavy projection to area F5. Inputs into area F6 include VIP, and area 46.F6 contains neurons that become active when an object that the monkey is about to grasp moves from being out of reach into the peripersonal space of the monkey

Interpretation: this class of pre-SMA (F6) neuron is responsible for generating a go signal when it is appropriate for the monkey to begin a reaching movement.

F4

Inferior Premotor

Cortex

F5(grasp type)

F6

46

(arm goal position)


FrontalCortex

pre-SMA



Area 46

Area 46 has been implicated as a working memory in tasks requiring information to be held during a delay period (Quintana & Fuster, 1993).

This memory has been posited to participate in learning tasks involving complex sequences of movements (Dominey, 1995).

Area 46 projects to F6, and also exchanges connections with area F5 (Luppino, et al., 1990; Matelli, 1994).

When a human is asked to imagine herself grasping an object, activated areas (PET or fMRI) involved include:

Area 46 (Decety, Perani, Jeannerod, Bettinardi, Tadary, Woods, 1994)

Area 44 (a possible F5 homologue)A site along the intra-parietal sulcus (Grafton, et al., 1996).


Role of Area 46 for Grasping in the Dark

During the performance of the task in the light, area 46 maintains a memory of those F5 cells that participate in the grasp.

To simulate performance in the dark, PIP and IT are then cleared.If a new trial is initiated soon enough, area 46 provides positive support to those F5 cells that were active during the first trial.

The area 46 working memory provides a static description of the grasp that was recently executed, in the sense that the temporal aspects of the grasp are not stored - only a memory of those units that were active at some time during the execution.

[Area 46 is involved in human imagination of grasp execution.]

F4

Inferior Premotor

Cortex

F5(grasp type)

F6

46

(arm goal position)


FrontalCortex

pre-SMA



Conditional Tasks and Area F2

Dorsal premotor cortex (F2) is thought to be responsible for the association of arbitrary stimuli (an IS) with the preparation of motor programs (Evarts, et al., 1984; Kurata & Wise, 1988; Mitz, Godshalk, & Wise, 1991; Wise & Mauritz, 1985).

In a task in which a monkey must respond to the display of a pattern with a particular movement of a joystick:

some F2 neurons respond to the sensory-specific qualities of the input. However, many F2 units respond in a way that is more related to the motor set that must be prepared in response to the stimulus.

When a muscimol lesion in this region is induced, the monkey loses the ability to correctly make the arbitrary association.

F4

F5(grasp type)

F6

46

(arm goal position)


pre-SMA


In addition to simulating the Sakata task, we simulate

conditional tasks.


F5 activity during execution of a precision graspThe top two traces show the position of the thumb and index finger.

Left: The next five traces represent the average firing rate of five F5 neurons (set-, extension-, flexion-, hold-, and release-related). The remaining five traces represent the various external (Ready, Go, Go2) and internal (SII) triggering signals.

Right: Illustrating the temporally distributed coding of F5 cells.


The Complete FARS Model

Visual Cortex

Parietal Cortex

VIP

PIPAIP

F4

How (dorsal)

IT

What (ventral)

Inferior Premotor Cortex

(position)

(shape, size, orientation)

(object/grasp transform)

F5(grasp type)

MI

hand

(muscle assemblies)

(elementary sensory features)

F6

46

SI

SII

expectation

motor commands

sensory info

(arm goal position)

(sensory hyperfeatures)


A7(internal model)


Thumb and index finger temporal behavior as a function of cylinder size


F5 cell responses during precision grasps of seven different apertures

This particular cell is active only for narrow precision pinches


F5 movement-related cell (A) and a hold-related (B) cell during the perturbation experiment

20mm/30mm traces correspond to presentation and grasping of a 20mm and a 30mm cylinder, respectively; traces labeled 2030 and 30 20 indicate perturbation trials, in which a 20mm cylinder is switched for a 30mm cylinder, and a 30mm cylinder for a 20mm one, respectively.


Two objects that map to the identical grasp


Comparison of population responses towards two different objects (but identical grasps).

Lighted movement task, AIP (A) and F5 (B) cells; and

AIP populations during fixation (C) and dark movement (D) tasks.


F5 Feedback to AIP

Visual-related AIP receive object-specific inputs; motor-related cells receive recurrent inputs from F5, which do not demonstrate object-specific activity.


A single object mapping to two possible grasps

Before execution, one grasp must be selected based upon the current context (e.g., based upon an Instruction Stimulus).


Two F5 units (A, B) in response to the four conditions: (c,pr), (c,pw), (nc,pr), and (nc,pw). c = conditional; nc = non-conditional; pr = precision pinch; pw = power grasp.


Four boxes of different dimensions

Grasping is performed along the horizontal axis. The two blocks in the left column are grasped using a precision pinch of a 10mm aperture; the blocks on the right require a 20mm aperture.


Comparison of AIP visual responses for objects of the same (A) and different (B) widths; and AIP motor-related responses (dark movement condition) for objects of same (C) and different (D) widths.

Documents

Lecture 13. The FARS model of Control of Reaching and Grasping 2 Reading Assignments: