N94- 30559

AIAA-94-1208-CP

BUILDING BRAINS FOR BODIES

Rodney Allen Brooks and Lynn Andrea Stein

Artificial Intelligence Laboratory

Massachusetts Institute of Technology

Cambridge, MA 02139 f- ¢/

Abstract

We describe a project to capitalize on newly avail-able levels of computational resources in order tounderstand human cognition. We will build an in-tegrated physical system including vision, soundinput and output, and dextrous manipulation, all

controlled by a continuously operating large scaleparallel MIMD computer. The resulting systemwill learn to "think" by building on its bodilyexperiences to accomplish progressively more ab- --stract tasks. Past experience suggests that in at- i

tempting to build such an integrated system we

will have to fundamentally change the way artifi-cial intelligence, cognitive science, linguistics, andphilosophy think about the organization of intel-ligence. We expect to be able to better reconcilethe theories that will be developed with currentwork in neuroscience.

Project Overview

We propose to build an integrated physical humanoidrobot including active vision, sound input and out-put, dextrous manipulation, and the beginnings of lan-guage, all controlled by a continuously operating largescale parallel MIMD computer. This project will cap-italize on newly available levels of computational re-sources in order to meet two goals: an engineering goalof building a prototype general purpose flexible anddextrous autonomous robot and a scientific goal of un-derstanding human cognition. While there have beenprevious attempts at building kinematically humanoidrobots, none have attempted the embodied construc-tion of an autonomous intelligent robot; the requisitecomputational power simply has not previously beenavailable.

The robot will be coupled into the physical worldwith high bandwidth sensing and fast servo-controlledactuators, allowing it to interact with the world on ahuman time scale. A shared time scale will open up

new possibilities for how humans use robots as assis-tants, as well as allowing us to design the robot tolearn new behaviors under human feedback such as

human manual guidance and vocal approval. One ofour engineering goals is to determine tile architecturalrequirements sufficient for an enterprise of this type.Based on our earlier work on mobile robots, our ex-pectation is that the constraints may be different thanthose that are often assumed for large scale parallel

computers. If ratified, such a conclusion could haveimportant impacts on the design of future sub-familiesof large machines.

Recent trends in artificial intelligence, cognitive sci-

ence, neuroseience, psychology, linguistics, and sociol-ogy are converging on an anti-objectivist, body-basedapproach to abstract cognition. Where traditional ap-proaches in these fields advocate an objectively speci-fiable reality--brain-in-a-box, independent of bodilyconstraints--these newer approaches insist that intel-

ligence cannot be separated from the subjective expe-rience of a body. The humanoid robot provides thenecessary substrate for a serious exploration of thesubjectivist--body-based--hypotheses.

There are numerous specific cognitive hypothesesthat could be implemented in one or more of the hu-manoids that will be built during the five-year project.For example, we can vary the extent to which the robotis programmed with an attentional preference for someimages or sounds, and the extent to which the robotis programmed to learn to selectively attend to envi-ronmental input as a by-product of goal attainment(e.g., successful manipulation of objects) or reward byhumans. We can compare the behavioral result of con-structing a humanoid around different hypotheses ofcortical representation, such as coincidence detectionversus interpolating memory versus sequence seekingin counter streams versus time-locked multi-regionalretroactivation. In the later years of the project wecan connect with theories of consciousness by demon-

strating that humanoids designed to continuously acton immediate sensory data (as suggested by Dennett'smultiple drafts model) show more human-like behaviorthan robots designed to construct an elaborate worldmodel.

The act of building and programming behavior-based robots will force us to face not only issues of

Copyright © 1993 by Rodney Allen Brooks and Lynn Andrea Stein. Published by the American Institute ofAeronautics and Astronautics, Inc. with permission.

253

https://ntrs.nasa.gov/search.jsp?R=19940026054 2020-05-23T13:07:49+00:00Z

interfacesbetweentraditionallyassumedmodularities,but eventheideaof modularityitself. By reachingacrosstraditionalboundariesandtyingtogethermanysensingandactingmodalities,wewill quicklyillumi-nateshortcomingsin the standardmodels,sheddinglighton formerlyunrealizedsociologicallyshared,butincorrect,assmnptions.

Background: the power of enabling

technology

An enabling technology--such as the brain that we willbuild--has the ability to revolutionize science. A re-cent example of the far-reaching effects of such techno-logical advances is the field of mobile robotics. Just asthe advent of cheap and accessible mobile robotics dra-matically altered our conceptions of intelligence in thelast decade, we believe that current high-performancecomputing technology makes the present an opportunetime for the construction of a similarly significant in-tegrated intelligent system.

Over the last eight years there has been a renewedinterest in building experimental mobile robot systemsthat operate in unadorned and unmodified natural andunstructured environments. The enabling technologyfor this was the single chip micro-computer. This madeit possible for relatively small groups to build service-able robots largely with graduate student power, ratherthan the legion of engineers that had characterized ear-lier efforts along these lines in the late sixties. Theaccessibility of this technology inspired academic re-searchers to take seriously the idea of building systemsthat would work in the real world.

The act of building and programming behavior-based robots fundamentally changed our understand-ing of what is difficult and what is easy. The effectsof this work on traditional artificial intelligence canbe seen in innumerable areas. Planning research hasundergone a major shift from static planning to dealwith "reactive planning." The emphasis in computervision has moved from recovery from single images orcanned sequences of images to active--or animate--vision, where tile observer is a participant in the worldcontrolling the imaging process in order to simplify theprocessing requirements. Generally, the focus withinAI has shifted from centralized systems to distributedsystems. Further, the work on behavior-based mobile

robots has also had a substantial effect on many otherfields (e.g., on the design of planetary science missions,on silicon micro-machining, on artificial life, and oncognitive science). There has also been considerableinterest from neuroscience circles, and we are just nowstarting to see some bi-directional feedback there.

The grand challenge that we wish to take up is tomake the quantum leap from experimenting with mo-bile robot systems to an almost humanoid integratedhead system with saccading foveated vision, facilitiesfor sound processing and sound production, and a com-pliant, dextrous manipulator. The enabling technology

is massively parallel computing; our brain will havelarge numbers of processors dedicated to particularsub-functions, and interconnected by a fixed topologynetwork.

Scientific Questions

Building an android, an autonomous robot with hu-manoid form, has been a recurring theme in sciencefiction from the inception of the genre with Franken-stein, through the moral dilemmas infesting positronicbrains, the human but not really human C3PO and theever present desire for real humanness as exemplifiedby Commander Data. Their bodies have ranged fromthat of a recycled actual human body through variousdegrees of mechanical sophistication to ones that areindistinguishable (in the stories) from real ones. Andperhaps the most human of all the imagined robots,HAL-9000, did not even have a body.

While various engineering enterprises have modeledtheir artifacts after humans to one degree or another(e.g., WABOT-II at Waseda University and the spacestation tele-robotic servicer of Martin-Marietta) noone has seriously tried to couple human like cognitiveprocesses to these systems. There has been an im-plicit, and sometimes explicit, assumption, even fromthe days of Turing (see Turing (1970) °) that the ul-timate goal of artificial intelligence research was tobuild an android. There have been many studies relat-

ing brain models to computers (Berkeley 1949), cyber-netics (Ashby 1956), and artificial intelligence (Arbib1964), and along the way there have always been semi-popular scientific books discussing the possibilities ofactually building real 'live' androids (Caudill (1992) isperhaps the most recent).

This proposal concerns a plan to build a series ofrobots that are both humanoid in form, humanoid infunction, and to some extent humanoid in computa-tional organization. While one cannot deny the ro-mance of such an enterprise we are realistic enough toknow that we can but scratch the surface of just a fewof the scientific and technological problems involved inbuilding the ultimate humanoid given the time scaleand scope of our proposal, and given the current stateof our knowledge.

The reason that we should try to do this at all isthat for the first time there is plausibly enough com-putation available. High performance parallel compu-tation gives us a new tool that those before us havenot had available and that our contemporaries havechosen not to use in such a grand attempt. Our previ-ous experience in attempting to emulate much simplerorganisms than humans suggests that in attemptingto build such systems we will have to fundamentallychange the way artificial intelligence, cognitive science,psychology, and linguistics think about the organiza-

*Different sources cite 1947 and 1948 as tile time of writ-

ing, but it was not published until long after his death.

254

tionofintelligence.As a result, some new theories willhave to be developed. We expect to be better able toreconcile the new theories with current work in neuro-science. The primary benefits from this work will bein the striving, rather than in the constructed artifact.

Brains

Our goal is to take advantage of the new availability ofmassively parallel computation in dedicated machines.We need parallelism because of the vast amounts of

processing that must be done in order to make senseof a continuous and rich stream of perceptual data.We need parallelism to coordinate the many actuation

systems that need to work in synchrony (e.g., the oc-ular system and the neck must move in a coordinatedfashion at time to maintain image stability) and whichneed to be servoed at high rates. We need parallelismin order to have a continuously operating system that

can be upgraded without having to recompile, reload,and restart all of the software that runs the stable lower

level aspects of the humanoid. And finally we need par-allelism for the cognitive aspects of the system as weare attempting to build a "brain" with more capabilitythan can fit on any existing single processor.

But in real-time embedded systems there is yet an-

other necessary reason for parallelism. It is the factthat there are many things to be attended to, hap-

pening in the world continuously, independent of theagent. From this comes the notion of an agent be-ing situated in the world. Not only must the agentdevote attention to perhaps hundreds of different sen-sors many times per second, but it must also devoteattention "down stream" in the processing chain in

many different places at many times per second as theprocessed sensor data flows through the system. Theactual anaounts of computation needed to be done byeach of these individual processes is in fact quite small,so small that originally we formalized them as aug-mented finite state machines (Brooks 1986), althoughmore recently we have thought of them as real-time

rules (Brooks 1990a). They are too small to have acomplete processor devoted to them in any machinebeyond a CM-2, and even there the processors wouldbe mostly idle. A better approach is to simulate par-allelism in a single conventional processor with its own

local memory.For instance, Ferrell (1993) built a software system

to control a 19 actuator six legged robot using about 60of its sensors. She implemented it as more than 1500parallel processes running on a single Phillips 68070.(It communicated with 7 peripheral processors whichhandled sensor data collection and 100Hz motor ser-

voing.) Most of these parallel processes ran at ratesvarying between 10 and 25 ttertz. Each time each pro-cess ran, it took at most a few dozen instructions beforeblocking, waiting either for the passage of time or forsome other process to send it a message. Clearly, lowcost context switching was important.

The underlying computational model used on thatrobot--and with many tens of other autonomousmobile robots we have built--consisted of networks

of message-passing augmented finite state machines.Each of these AFSMs was a separate process. The

messages were sent over predefined 'wires' from a spe-cific transmitting to a specific receiving AFSM. Themessages were simple numbers (typically 8 bits) whosemeaning depended on the designs of both the transmit-ter and the receiver. An AFSM had additional registerswhich held the most recent incoming message on any

particular wire. This gives a very simple model of par-allelism, even simpler than that of CSP (Hoare 1985).

The registers could have their values fed into a localcombinatoriM circuit to produce new values for regis-ters or to provide an output message. The network ofAFSMs was totally asynchronous, but individual AF-SMs could have fixed duration monostables which pro-vided for dealing with the flow of time in the outsideworld. The behavioral competence of the system was

improved by adding more behavior-specific network tothe existing network. This process was called layering.This was a simplistic and crude analogy to evolution-ary development. As with evolution, at every stage ofthe development the systems were tested. Each of thelayers was a behavior-producing piece of network inits own right, although it might implicitly rely on thepresence of earlier pieces of network. For instance, anexplore layer did not need to explicitly avoid obstacles,as the designer knew that a previous avoid layer wouldtake care of it. A fixed priority arbitration scheme wasused to handle conflicts.

On top of the AFSM substrate we used anotherabstraction known as the Behavior Language, or BL

(Brooks 1990a), which was much easier for the userto program with. The output of the BL compilerwas a standard set of augmented finite state machines;

by maintaining this compatibility all existing softwarecould be retained. When programming in BL the userhas complete access to full Common Lisp as a meta-

language by way of a macro mechanism. Thus the usercould easily develop abstractions on top of BL, whilestill writing programs which compiled down to net-works of AFSMs. In a sense, AFSMs played the role of

assembly language in normal high level computer lan-

guages. But the structure of the AFSM networks en-forced a programming style which naturally compiledinto very efficient small processes. The structure of theBehavior Language enforced a modularity where datasharing was restricted to smallish sets of AFSMs, andwhose only interfaces were essentially asynchronous 1-

deep buffers.

In the humanoid project we believe much of the com-

putation, especially for the lower levels of the system,will naturally be of a similar nature. We expect to

perform different experiments where in some cases thehigher level computations are of the same nature andin other cases tile higher levels will be much more sym-

255

bolicin nature, although the symbolic bindings will berestricted to within individual processors. We need touse software and hardware environments which givesupport to these requirements without sacrificing thehigh levels of performance of which we wish to makeuse.

Software

For the software environment we have a number ofre-

quirements:

• There should be a good softwaredevelopment envi-ronment.

• The system should be completely portable over

many hardware environments, so that we can up-grade to new parallel machines over the lifetime ofthis project.

• The system should provide efficient code for percep-tual processing such as vision.

• The system should let us write high level symbolicprograms when desired.

• The system language should be a standardized lan-guage that is widely known and understood.

In summary our software environment should let us

gain easy access to high performance parallel compu-tation.

SVe have chosen to use Common Lisp (Steele Jr.1990) as the substrate for all software development.This gives us good programming environments includ-ing type checked debugging, rapid prototyping, sym-bolic computation, easy ways of writing embedded lan-guage abstractions, and automatic storage manage-ment. We believe that Common Lisp is superior toC (the other major contender) in all of these aspects.

The problem then is how to use Lisp in a massivelyparallel machine where each node may not have thevast amounts of memory that we have become accus-tomed to feeding Common Lisp implementations onstandard Unix boxes.

We have a long history of building high performance

Lisp compilers (Brooks, Gabriel & Steele Jr. 1982),including one of the two most common commercial Lispcompilers on the market; Lucid Lisp--Brooks, Posner,McDonald, White, Benson & Gabriel (1986).

Recently we have developed L (Brooks 1993), a re-targetable small efficient Lisp which is a downwardlycompatible subset of Common Lisp. When compiledfor a 68000 based machine the load image (withoutthe compiler) is only 140K bytes, but includes mul-tiple values, strings, characters, arrays, a simplifiedbut compatible package system, all the "ordinary" as-pects of format, backquote and comma, sotf etc.,full Common Lisp lambda lists including optionals andkeyword arguments, macros, an inspector, a debug-ger, de:fstruct (integrated with the inspector), block,catch, and throw, etc., full dynamic closures, a full

lexical interpreter, floating point, fast garbage collec-tion, and so on. The compiler runs in time linear inthe size of an input expression, except in the presenceof lexical closures. It nevertheless produces highly op-timized code in most cases. L is missing flet andlabels, generic arithmetic, bignums, rationals, com-plex numbers, the library of sequence functions (whichcan be written within L) and esoteric parts of formatand packages.

The L system is an intellectual descendent of the

dynamically retargetable Lucid Lisp compiler (Brookset al. 1986) and the dynamically retargetable Behav-ior Language compiler (Brooks 1990a). The system istotally written in L with machine dependent backendsfor retargetting. The first backend is for the Motorola68020 (and upwards) family, but it is easily retargetedto new architectures. The process consists of writing asimple machine description, providing code templatesfor about 100 primitive procedures (e.g., fixed preci-sion integer +, *, =, etc., string indexing CrIAR andother accessors, CAR, CDR, etc.), code macro expansionfor about 20 pseudo instructions (e.g, procedure call,procedure exit, checking correct number of arguments,linking CATCH frames, etc.) and two corresponding setsof assembler routines which are too big to be expandedas code templates every time, but are so critical inspeed that they need to be written in machine lan-guage, without the overhead of a procedure call, ratherthan in Lisp (e.g., C0/IS, spreading of multiple values

on the stack, etc.). There is a version of the I/O systemwhich operates by calling C routines (e.g., fgetchal-,etc.; this is how the Macintosh version of L runs) soit is rather simple to port the system to any hardwareplatform we might choose to use in the future.

Note carefully the intention here: L is to be the de-livery vehicle running on the brain hardware of thehumanoid, potentially on hundreds or thousands of

small processors. Since it is fully downward compat-ible with Common Lisp however, we can carry outcode development and debugging on standard work

stations with full programming environments (e.g., inMacintosh Common Lisp, or Lucid Common Lisp withEmacs 19 on a Unix box, or in the Harlequin program-ming environment on a Unix box). Wc can then dy-namically link code into the running system on our

parallel processors.There are two remaining problems: (1) how to main-

tain super critical real-time performance when using aLisp system without hard ephemeral garbage collec-tion, and (2) how to get the level of within-processorparallelism described earlier.

The structure of L's implementation is such thatmultiple independent heaps can be maintained withina single address space, sharing all the code and datasegments of the Lisp proper. In this way super-criticalportions of a system can be placed in a heap where noconsing is occurring, and hence there is no possibilitythat they will be blocked by garbage collection.

256

TheBehaviorLanguage(Brooks1990a)isanexam-pleof a compilerwhichbuildsspecial purpose staticschedulers for low overhead parallelism. Each processran until blocked and the syntax of the language forced

there to always be a blocking condition, so there wasno need for pre-emptive scheduling. Additionally the

syntax and semantics of the language guaranteed thatthere would be zero stack context needed to be savedwhen a blocking condition was reached. We will needto build a new scheduling system with L to addresssimilar issues in this project. To fit in with the phi-

losophy of the rest of the system it must be a dy-namic scheduler so that new processes can be addedand deleted as a user types to the Lisp listener of a

particular processor. Reasonably straightforward datastructures can keep these costs to manageable levels.It is rather straightforward to build a phase into the Lcompiler which can recognize the situations describedabove. Thus it is straightforward to implement a setof macros which will provide a language abstraction

on top of Lisp which will provide all the functionalityof the Behavior Language and which will additionallylet us have dynamic scheduling. Almost certainly a

pre-emptive scheduler will be needed in addition, asit would be difficult to enforce a computation time

limit syntactically when Common Lisp will essentiallybe available to the programmer--at the very least thecase of the pre-emptive scheduler having to strike downa process will be useful as a safety device, and will alsoact as a debugging tool for the user to identify timecritical computations which are stressing the boundedcomputation style of writing. In other cases static anal-ysis will be able to determine maximum stack require-ments for a particular process, and so heap allocatedstacks will be usable, t

The software system so far described will be usedto implement crude forms of 'brain models', wherecomputations will be organized in ways inspired bythe sorts of anatomical divisions we see occurring inanimal brains. Note that we are not saying we will

build a model of a particular brain, but rather therewill be a modularity inspired by such components asvisual cortex, auditory cortex, etc., and within andacross those components there will be further modu-

larity, e.g., a particular subsystem to implement thevestibulo-ocular response (VOR).

Thus besides on-processor parallelism we will need to

provide a modularity tool that packages processes intogroups and limits data sharing between them. Eachpackage will reside on a single processor, but often pro-cessors will host many such packages. A package thatcommunicates with another package should be insu-lated at the syntax level from knowing whether the

other package is on the same or a different processor.The communication medium between such packages

tThe problem with heap allocated stacks in the generalcase is that there will be no overflow protection into the

rest of heap.

will again be 1-deep buffers without queuing or receiptacknowledgment--any such acknowledgment will needto be implemented as a backward channel, much as wesee throughout the cortex (Churchland & Sejnowski

1992). This packaging system can be implemented inCommon Lisp as a macro package.

We expect all such system level software develop-ment to be completed in the first twelve months of the

project.

Computational Hardware

The computational model presented in the previoussection is somewhat different from that usually as-

sumed in high performance parallel computer appli-cations. Typically (Cypher, Ho, Konstantinidou

Messina 1993) there is a strong bias on system re-quirements from the sort of benchmarks that are usedto evaluate performance. The standard benchmarksfor modern high performance computation seem tobe Fortran codes for hydrodynamics, molecular sim-

ulations, or graphics rendering. We are proposing avery different application with very different require-ments; in particular we require real-time response toa wide variety of external and internal events, we re-quire good symbolic computation performance, we re-quire only integer rather than high performance float-ing point operations, _ we require delivery of messagesonly to specific sites determined at program designtime, rather than at run-time, and we require the abil-ity to do very fast context switches because of the largenumber of parallel processes that we intend to run oneach individual processor.

The fact that we will not need to support pointer ref-erences across the computational substrate will meanthat we can rely on much simpler, and therefore

higher performance, parallel computers than manyother researchers--we will not have to worry about

a consistent global memory, cache coherence, or ar-bitrary message routing. Since these are different re-quirements than those that are normally considered,we have to make some measurements with actual pro-

grams before we can we can make an intelligent off theshelf choice of computer hardware.

In order to answer some of these questions we are

currently building a zero-th generation parallel com-puter. It is being built on a very low budget with offthe shelf components wherever possible (a few fairlysimple printed circuit boards need to be fabricated).The processors are 16Mhz Motorola 68332s on a stan-dard board built by Vesta Technology. These plug 16

to a backplane. The backplane provides each processorwith six communications ports (using the integratedtiming processor unit to generate the required signals

iConsider the dynamic range possible in single signalchannels in the human brain and it soon becomes apparentthat all that we wish to do is certainly achievable withneither span of 600 orders of magnitude, or 47 significantbinary digits.

257

alongwith special chip select and standard addressand data lines) and a peripheral processor port. Thecommunications ports will be hand-wired with patchcables, building a fixed topology network. (The ca-bles incorporate a single dual ported RAM (8K by 16bits) that itself includes hardware semaphores writable

and readable by the two processors being connected.)Background processes running on the 68332 operatingsystem provide sustained rate transfers of 60Hz pack-ets of 4K bytes on each port, with higher peak rates ifdesired. These sustained rates do consume processingcycles from the 68332. On non-vision processors weexpect much lower rates will be needed, and even onvision processors we can probably reduce the packetfrequency to around 15Hz. Each processor has an op-erating system, L, and the dynamic scheduler residingin 1M of EPROM. There is 1M of RAM for program,stack and heap space. Up to 256 processors can beconnected together.

Up to 16 backplanes can be connected to a singlefront end processor (FEP) via a shared 500K baud se-rial line to a SCSI emulator. A large network of 68332scan span many FEPs if we choose to extend the con-

struction of this zero-th prototype. Initially we will usea Macintosh as a FEP. Software written in Macintosh

Common Lisp on the FEP will provide disk I/O ser-vices to the 68332's, monitor status and health packetsfrom them, and provide the user with a Lisp listenerto any processor they might choose.

The zero-th version uses the standard Motorola SPI

(serial peripheral interface) to communicate with upto 16 Motorola 6811 processors per 68332. These area single chip processor with onboard EEPROM (2Kbytes) and RAM (256 bytes), including a timer system,all SPI interface, and 8 channels of analog to digitalconversion. We are building a small custom board forthis processor that includes opto-isolated motor driversand some standard analog support for sensors§.

We expect our first backplane to be operational byAugust 1st, 1993 so that we can commence experi-ments with our first prototype body. We will collectstatistics on inter-processor communication through-put, effects of latency, and other measures so that we

call better choose a larger scale parallel processor formore serious versions of the humanoid.

In the meautime, however, there are certain devel-opments on the horizon within the MIT Artificial In-

telligence Lab which we expect to capitalize upon inorder to dramatically upgrade our computational sys-tems for early vision, and hence the resolution at whichwe can afford to process images in real time. The

_We currently have 28 operational robots in our labseach with between 3 and 5 of these 6811 processors, andseveral dozen other robots with at least 1 such processoron board. We have great experience in writing compilerbackends for these processors (including BL) and great ex-perience in using them for all sorts ofservoing, sensor mon-itoring, and communications tasks.

first of these, expected in the fall will be a some-what similar distributed processing system based onthe much higher performance Texas Instrument C40,which comes with built in support for fixed topologymessage passing. We expect these systems to be avail-able in the Fall '93 timeframe. In October '94 we ex-

pect to be able to make use of the Abacus system, abit level reconfigurable vision front-end processor beingbuilt under ARPA sponsorship which promises Tera-opperformance on 16 bit fixed precision operands. Boththese systems will be simply integrable with our zero-thorder parallel processor via the standard dual-portedRAM protocol that we are using.

Bodies

As with the computational hardware, we are also cur-

rently engaged in building a zero-th generation bodyfor early experimentation and design refinement to-wards more serious constructions within the scope ofthis proposal. We are presently limited by budgetaryconstraints to building an immobile, armless, deaf,torso with only black and white vision.

In the following subsections we outline the con-straints and requirements on a full scale humanoidbody and also include where relevant details of ourzero-th level prototype.

Eyes

There has been quite a lot of recent work on animate

vision using saccading stereo cameras, most notablyat Rochester (Ballard 1989), (Coombs 1992), but alsomore recently at many other institutions, such as Ox-ford University.

The humanoid needs a head with high mechanicalperformance eyeballs and foveated vision if it is to be

able to participate in the world with people in a natu-ral way. Even our earliest heads will include two eyes,with foveated vision, able to pan and tilt as a unit, andwith independent saccading ability (three saccades persecond) and vergence control of the eyes. Fundamentalvision based behaviors will include a visually calibratedvestibular-ocular reflex, smooth pursuit, visually cal-ibrated saccades, and object centered foveal relativedepth stereo. Independent visual systems will provideperipheral and foveal motion cues, color discrimina-

tion, human face pop-outs, and eventually face recogni-tion. Over the course of the project, object recognitionbased based on "representations" from body schemasand manipulation interactions will be developed. Thisis completely different from any conventional objectrecognition schemes, and can not be attempted with-out an integrated vision and manipulation environmentas we propose.

The eyeballs need to be able to saccade up to aboutthree times per second, stabilizing for 250ms at eachstop. Additionally the yaw axes should be control-

lable for vergence to a common point and drivable in

258

amannerappropriateforsmoothpursuitandfor im-agestabilizationaspartofa vestibulo-ocular response(VOR) to head movement. The eyeballs do not needto be force or torque controlled but they do need good

fast position and velocity control. We have previouslybuilt a single eyeball, A-eye, on which we implementeda model of VOR, ocular-kinetic response (OKR) andsaccades, all of which used dynamic visually based cal-

ibration (Viola 1990).Other active vision systems have had both eyeballs

mounted on a single tilt axis. We will begin experi-ments with separate tilt axes but if we find that rela-tive tilt motion is not very useful we will back off from

this requirement in later versions of the head.The cameras need to cover a wide field of view,

preferably close to 180 degrees, while also giving afoveated central region. Ideally the images should be

RGB (rather than the very poor color signal of stan-dard NTSC). A resolution of 512 by 512 at both thecoarse and fine scale is desirable.

Our zero-th version of the cameras are black and

white only. Each eyeball consists of two smalllightweight cameras mounted with parallel axes. Onegives a 115 degree field of view and the other gives a20 degree foveated region. In order to handle the im-ages in real time in our zero-th parallel processor wewill subsample the images to be much smaller than theideal.

Later versions of the head will have full RGB color

cameras, wider angles for the peripheral vision, muchfiner grain sampling of the images, and perhaps a col-inear optics set up using optical fiber cables and beamsplitters. With more sophisticated high speed process-

ing available we will also be able to do experimentswith log-polar image representations.

Ears, Voice

Almost no work has been done on sound understand-

ing, as distinct from speech understanding. Thisproject will start on sound understanding to providea much more solid processing base for later work onspeech input. Early behavior layers will spatially cor-relate noises with visual events, and spatial registra-tion will be continuously self calibrating. Efforts willconcentrate on using this physical cross-correlation asa basis for reliably pulling out interesting events from

background noise, and mimicking the cocktail party ef-fect of being able to focus attention on particular soundsources. Visual correlation with face pop-outs, etc.,will then be used to be able to extract human sound

streams. Work will proceed on using these soundsstreams to mimic infant's abilities to ignore language

dependent irrelevances. By the time we get to elemen-tary speech we will therefore have a system able towork in noisy environments and accustomed to multi-

ple speakers with varying accents.Sound perception will consist of three high quality

microphones. (Although the human head uses only

two auditory inputs, it relies heavily on the shape ofthe external ear in determining the vertical componentof directional sound source.) Sound generation will be

accomplished using a speaker.Sound is critical for several aspects of the robot's

activity. First, sound provides immediate feedback formotor manipulation and positioning. Babies learn tofind and use their hands by batting at and manipulat-

ing toys that jingle and rattle. Adults use such cuesas contact noises--the sound of an object hitting the

table--to provide feedback to motor systems. Second,sound aids in socialization even before the emergence

of language. Patterns such as turn-taking and mimicryare critical parts of children's development, and adultsuse guttural gestures to express attitudes and otherconversational cues. Certain signal tones indicate en-

couragement or disapproval to all ages and stages of de-velopment. Finally, even pre-verbal children use soundeffectively to convey intent; until our robots developtrue language, other sounds will necessarily be a ma-jor source of communication.

Torsos

In order for the humanoid to be able to participate inthe same sorts of body metaphors as are used by hu-

mans, it needs to have a symmetric human-like torso.It needs to be able to experience imbalance, feel sym-metry, learn to coordinate head and body motion forstable vision, and be able to experience relief when itrelaxes its body. Additionally the torso must be ableto support the head, the arms, and any objects they

grasp.The torsos we build will initially have a three degree

of freedom hip, with the axes passing through a com-mon point, capable of leaning and twisting to any po-sition in about three seconds--somewhat slower thana human. The neck will also have three degrees of free-

dom, with the axes passing through a common pointwhich will also lie along the spinal axis of the body.The head will be capable of yawing at 90 degrees persecond--less than peak human speed, but well withinthe range of natural human motions. As we build laterversions we expect to increase these performance fig-ures to more closely match the abilities of a human.

Apart from the normal sorts of kinematic sensors,the torso needs a number of additional sensors specifi-

cally aimed at providing input fodder for the develop-ment of bodily metaphors. In particular, strain gaugeson the spine can give the system a feel for its postureand the symmetry of a particular configuration, plus alittle information about any additional load the torso

might bear when an arm picks up something heavy.Heat sensors on the motors and the motor drivers will

give feedback as to how much work has been done bythe body recently, and current sensors on the motorswill give an indication of how hard tile system is work-

ing instantaneously.Our zero-th level torso is roughly 18 inches from the

259

baseof thespineto the base of the neck. This corre-sponds to a smallish adult. It uses DC motors withbuilt in gearboxes. The main concern we have is how

quiet it will be, as we do not want the sound perceptionsystem to be overwhelmed by body noise.

Later versions of the torsos will have touch sensors

integrated around the body, will have more compliantmotion, will be quieter, and will need to provide better

cabling ducts so that the cables can all feed out througha lower body outlet.

Arms

The eventual manipulator system will be a compliantnmlti-degree of freedom arm with a rather simple hand.(A better hand would be nice, but hand research is not

yet at a point where we can get an interesting, easy-touse, off-the-shelf hand.) The arm will be safe enoughthat humans can interact with it, handing it thingsand taking things from it. The arm will be compliantenough that the system will be able to explore its ownbody--for instance, by touching its head system--sothat it will be able to develop its own body metaphors.The full design of the even the first pair of arms is not

yet completely worked out, and current funding doesnot permit the inclusion of arms on the zero-th levelhumanoid. In this section, we describe our desideratafor the arms and hands.

We want the arms to be very compliant yet still ableto lift weights of a few pounds so that they can interactwith human artifacts in interesting ways. Addition-ally we want the arms to have redundant degrees offreedom (rather than the six seen in a standard com-mercial robot arm), so that in many circumstances wecan 'burn' some of those degrees of freedom in orderto align a single joint so that the joint coordinates andtask coordinates very nearly match. This will greatlysimplify control of manipulation. It is the sort of thingpeople do all the time: for example, when bracing anelbow or the base of the palm (or even their middleand last two fingers) on a table to stabilize the hand

during some delicate (or not so delicate) manipulation.The hands in the first instances will be quite simple;

devices that can grasp from above relying heavily onmechanical compliance--they may have as few as onedegree of control freedom.

More sophisticated, however, will be the sensing onthe arms and hands. We will use forms of conduc-tive rubber to get a sense of touch over the surface of

the arm, so that it can detect (compliant) collisions itmight participate in. As with the torso there will beliberal use of strain gauges, heat sensors and currentsensors so that the system can have a 'feel' for how its

arms are being used and how they are performing.We also expect to move towards a more sophisticated

type of hand in later years of this project. Initially,unfortunately, we will be forced to use motions of the

upper joints of the arm for fine manipulation tasks.More sophisticated hands will allow us to use finger

motions, with much lower inertias, to carry out thesetasks.

Development Plan

We plan on modeling the brain at a level above the neu-

ral level, but below what would normally be thoughtof as the cognitive level.

We understand abstraction well enough to know howto engineer a system that has similar properties andconnections to the human brain without having tomodel its detailed local wiring. At the same time it

is clear from the literature that there is no agreementon how things are really organized computationally athigher or modular levels, or indeed whether it even

makes sense to talk about modules of the brain (e.g.,short term memory, and long term memory) as gener-ative structures.

Nevertheless, we expect to be guided, or one mightsay inspired, by what is known about the high levelconnectivity within the human brain (although ad-mittedly much of our knowledge actually comes from

macaques and other primates and is only extrapolatedto be true of humans, a problem of concern to some

brain scientists (Crick 8z Jones 1993)). Thus for in-stance we expect to have identifiable clusters of pro-cessors which we will be able to point to and say theyare performing a role similar to that of the cerebel-lum (e.g., refining gross motor commands into coordi-

nated smooth motions), or the cortex (e.g., some as-pects of searching generalization/specialization hierar-chies in object recognition (Ullman 1991)).

At another level we will directly model human sys-tems where they are known in some detail. For in-stance there is quite a lot known about the control

of eye movements in humans (again mostly extrapo-lated from work with monkeys) and we will build in avestibulo-ocular response (VOR), OKR, smooth pur-suit, and saccades using the best evidence available on

how this is organized in humans (Lisberger 1988).A third level of modeling or inspiration that we will

use is at the developmental level. For instance once wehave some sound understanding developed, we will usemodels of what happens in child language developmentto explore ways of connecting physical actions in the

world to a ground of language and the developmentof symbols (Bates 1979), (Bates, Bretherton & Sny-der 1988), including indexical (Lempert & Kinsbourne1985) and turn-taking behavior, interpretation of toneand facial expressions and the early use of memorizedphrases.

Since we will have a number of faculty, post-doctoralfellows, and graduate students working on concurrentresearch projects, and since we will have a number

of concurrently active humanoid robots, not all piecesthat are developed will be intended to fit together ex-actly. Some will be incompatible experiments in al-

ternate ways of building subsystems, or putting themtogether. Some will be pushing on particular issues in

260

System software (0th)v

System software (com==mercial processor)v

Periperhal MotionSaccades

_3R

Smooth pursuitv

Head/eye coo/_

Vergencebased stereo ,,._

Face pop-outsv

Head/bodyle_lcoord

Bring handsmidline

Body stability,

leaning, resting

Gesture recognitionv

Own hand tracking..=r

Hand Grasping,

linking & transfer

Batting static

objects

Body+arm reachin,,_

Ullman-esquevisual routines

v

Physical schema

based obj. reco 9.

Face rememberi_ng Face recognitionv

Facial gesture rec_g. Body motion recog,L

Specific obj. reco_. Generic object recog.

Body-based metaphors

DOF reduction DOF reduction

(specific coords) (generic coords)

Body mimicrLy

Manipulation turn takin_g

Sound localization Sound-based mani_. Voice/face assoc,=

Sound/motion correl Human voice extraction Proto language

Tone identification Voice turn takina

Visual imagery ,,,= Symbolization

Mental rehearsal Imaginationv

Multiple-drafts emergence

A ASept 1 Sept 1 Sept 1 Sept 1 Sept 11993 1994

1995 1996 1997

Figure 1

261

language,say, that may not be very related to someparticular other issues, e.g., saccades. Also, quiteclearly, at this stage we can not have a developmentplan fully worked out for five years, as many of theearly results will change the way we think about theproblems and what should be the next steps.

In figure 1, we summarize our current plans for de-veloping software systems on board our series of hu-manoids. In many cases there will be earlier workoff-board the robots, but to keep clutter down in the

diagram we have omitted that work here.

Acknowledgements

This paper has benefitted from conversations with andcomments by Catherine Harris, Dan Dennett, MarcelKinsbourne. We are also indebted to the members of

our research groups, individually and collectively, whohave shared their thoughts and enthusiasm.

References

Agre, P. E. & Chapman, D. (1987), Pengi: An Implementation of a Theoryof Activity, in 'Proceedings of the Sixth N_tion&l Conference on ArtificialIntelligence', Morgan Kaufmann, Seattle, Washington, pp. 198-201.

Allen, J., Hendler, J. & Tats, A., eds (1990), Readings in Planning, Mor-

gan Kaufmann, Los Altos, California.

Angle, C. M. & Brooks, R. A. (1990), Small Planetary Rovers, in'IEE]g/I_SJ International Workshop on Intelligent Robots and Systems p,

Ikabara, Japan, pp. 383-388.

Arbib, M. A. (1964), Brains, Machines and Mathematics, McGraw.Hill,New York, New York.

Ashby, W. R. (1958), An Introduction to Cybernetics, Chapman and Hall,

London, United Kingdom.

Ballard, D. H. (1989), Reference Frames for Active Vision, in 'Proceedingsof the International Joint Conference on Artificial Intelligence'_ Detroit,

Michigan, pp. 1635-1641.

Bates, E. (1979), The Emergence of Symbols, Academic Press, New York,New York.

Bates, E., Bretherton, I. & Snyder, L. (1988), From First Words to Gram-mar, Cambridge University Press, Cambridge, United Kingdom.

Beer, R. D. (1990), Intelligence as Adaptive Behavior, Academic Press,Cambridge, Massachusetts.

Berkeley, E. C. (1949), Giant Brains or Machines that Think, John Wiley& Sons, New York, New York.

Bickhard, M. H. (n.d.), How to Build a Machine with Emergent Represen-tational Content, Unpublished manuscript, University of Texas, Austin.

Brachman, R. J. & Levesque, H. J., eds (1985), Readings in KnowledgeRepresentation, Morgan Ksufmann, Los Altos, California.

Braddlck, O., Atklson, J., Hood, B., Harkness, W. & an Faraneh VarghaoKhadem, G. J. (1992), 'Possible blindslght in infants lacking one cerebralhemisphere', Nature 860, 481-483.

Brooks, R. A. (1988), 'A Robust Layered Control System for a MobileRobot', IEEE Journal of Robotics and Automation RA.II, 14-23.

Brooks, R. A. (1989), 'A Robot That Walks: Emergent Behavior from a

Carefully Evolved Network', Neural Computation 1(2), 253"262.

Brooks, R. A. (1990a), The Behavior Language User's Guide, Memo 1227,

Massachusetts Institute of TechnologyArtificlal Intelligence Lab_ Cam-

bridge, Massachusetts.

Brooks, R. A. (1990b), Elephants Don't Play Chess, in P. Mass, ed., 'De-signing Autonomous Agents: Theory and Practice from Biology to En-gineering and Back', MIT Press, Cambridge, Massachusetts, pp. 3-15.

Brooks, R. A. (1991a), Intelligence Without Reason_ in 'Proceedings of thei991 International Joint Conference on Artificial Intelligence', pp. 569-595.

Brooks, R. A. (1991b), 'New Approaches to Robotics', Science 263, 1227-1232.

Brooke, R. A. (1993), L: A Subset of Common Lisp, Technical report,Massachusetts Institute of TechnologyArtificial Intelligence Lab.

Brooks, R. A., Gabriel, R, P. & Steele Jr., G. L. (1982), An Optimiz-

ing Compiler for Lexically Scoped Lisp, in 'Proceedings of the 1982

Symposium on Compiler Construction. ACM SIGPLAN', Boston, Mas-sachusetts, pp. 261-275. Publised as ACM SIGPLAN Notices 17, 6 (June1982).

Brooks, R. A., Poener, D. B., McDonald, J. L., White, a. L., Benson, E.& Gabriel, R. P. (1988), Design of An Optimizing Dynamically Retar-getable Compiler for Common Lisp, in 'Proceedings of the 1986 ACMSymposium on Lisp and Functional Programming', Cambridge, Mas-sachusetts, pp. 67-85.

Caudill, M. (1992), In Our Own Image: Building An Artificial Person,Oxford University Press, New York, New York.

Churchland, P. S. & Seinowski, T. J. (1992), The Computational Brain,MIT Press, Cambridge, Massachusetts.

Connell, J. H. (1987), Creature Building with the Subsumption Architec-

ture, in 'Proceedings of the International Joint Conference on ArtificialIntelligence', Milan, Italy, pp. 1124-1126.

Connell, J. H. (1990), Minlmalist Mobile Robotics: A Colony-style Archi-tecture for a Mobile Robot, Academic Press, Cambridge, Massachusetts.also MIT TR-II51,

Coombs, D. J. (1992), Real.time Gaze Ho]dlng in Binocular Robot Vision,PhD thesis, University of Rochester, Department of CS, Rochester, NewYork.

Crick, F. & Jones, g. (1993), 'Backwardness of human neuroanatomy',

Nature 801, 109-110.

Cypher, R., Ho, A., gonstantlnidou, S. & Messinet, P. (1993), Architec-

tural Requirement _ of Parallel Scientific Applications with Explicit Com-munication, in 'IEEE Proceedings of the 20th International Symposium

on Computer Architecture', San Diego, California, pp. 2-13.

Damasio, H. & Damasio, A. R. (1989), Lesion Analysis in Neuropsychol-

ogy, Oxford University Press, New York, New York.

Dennett, D. C. (1991), Consciousness Explained, Little, Brown, Boston,Massachusetts.

Dennett, D. C. & Kinsbourne, M. (1992), 'Time and the Observer: TheWhere and When of Consciousness in the Brain', Brain and Behavioral

Sciences 15, 183-247.

Drescher, G. L. (1991), Made-Up Minds: A Constrectivist Approach toArtificial Intelligence, MIT Press, Cambridge, Massachusetts.

gdelman, G. M. (1987), Neural Darwlnslm: The Theory of lqeuronal

Group Selection, Basic Books, New York, New York.

Edelman, G. M. (1989), The Remembered Present: A Biological Theoryof Consciousness, Basic Books, New York, New York.

Edelman, G. M. (1902), Bright Air, Brilliant Fire: On the Matter of Mind,Basic Books, New York, New York.

Fendrich, R., Wesslnger, C. M. & Gazzanlga, M. S. (1992), 'Residual Vi.

sion in a Scotoma: Implications for Blindsight', Science 258, 1489-1491.

Ferrel], C. (1993), Robust Agent Control of an Autonomous Robot withMany Sensors and Actuators, Master's thesis, MIT, Department ofEECS, Cambridge, Massachusetts.

Fodor, J. A. (1983), The Modularity of Mind, Bradford Books, MIT Press,Cambridge, Massachusetts.

Harris, C. L. (1991), Parallel Distributed Processing Models andMetaphors for Language and Development, PhD thesis, University ofCalifornia, Department of Cognitive Science, San Diego, California.

Haugeland, J. (1985), Artificial Intelligence: The Very Idea, MIT Press,Cambridge, Massachusets.

Hoare, C. A. R. (1985), Communicating Sequential Processes, Prentice.Hall, gnglewood Cliffs, New Jersey.

Hobbs, J, & Moore, R., eds (1985), Formal Theories of the Commonsense

World, Ablex Publishing Co., Norwood, New Jersey.

Horewill, I. D. (1993), Specialization of Perceptual Processes, PhD thesis,MIT, Department of EECS, Cambridge, Massachusetts.

Horswill, I. D_ & Brooks, It. A. (1988), Situated Vision in a DynamicWorld: Chasing Objects, in 'Proceedings of the Seventh Annual Meetingof the American Association for Artificial Intelligence', St. Paul, Min.

nesota, pp. 796-800.

Johnson, M. (1987), The Body In The Mind, University of Chicago Press,Chicago, Blinois.

Kinsbourne, M. (198T), Mechanisms of unilateral neglect, in M. Jeannerod,ed., 'Neurophysiological and Neuropsychologlcal Aspects of Spatial Ne-glect', Elsevier, North Holland.

Kinsbourne, M. (1988), Integrated field theory of consciousness, in A. Mar-cel & E. Bisiach, eds, 'The Concept of Consciousness in Contemporary

Science', Oxford University Press, London, England.

Kosslyn, S. (1993), Image and brain: The resolution of the imagery debate,Harvard University Press, Cambridge, Massachusetts.

Kulpers, B. & Byun, Y.-T. (1991), 'A robot exploration and map-

ping strategy based on a semantic hierarchy of spatial representations',Robotics and Autonomous Systems 8, 47-63.

Lakoff, G. (1987), Women, Fire, and Dangerous Things, University of

Chicago Press, Chicago, Illinois.

Lakoff, G. & Johnson, M. (1980), Metaphors We Live By, University ofChicago Press, Chicago, Illinois.

Langacker, R. W. (1987), Foundations of cognitive grammar, Volume 1,Stanford University Press, Palo Alto, California.

262

Lempert, H. & Kinsbourne, M. (1985), 'Possible origin of speech in selec-

tive orienting', Psychologic_l Bulletin 9Y, 62-73.

Lisberger, S. G. (1988), 'The neural basis for motor learning in the

vestlbulo-ocu]ar reflex in monkeys', Trends in Neuroscience II, 147-152.

Mart, D. (1982), Vision, W. H. Freeman, San Francisco, California.

Mataric, M. J. (1992a), Designing Emergent Behaviors: From Local In.

teractions to Collective Intelligence, in 'Proceedings of the Second In-

ternational Conference on Simulation of Adaptive Behavior', MIT Press,

Cambridge, Massachusetts, pp. 432-441.

Matsric, M. 5. (1992b), 'Integration of Representation Into Goal.Driven

Behavior-Based Robots', IEEE Journal of Robotics and Automation

8(3), 304-312,

McCarthy, R. A. & Warrington, E. K. (1988), 'Evidence for Modality-

Specific Systems in the Brain', Nature 334, 428-430.

McCarthy, R. A. & Warrington, E. K. (1990), Cognitive Neuropsychology,

Academic Press, San Diego, California.

Minsky, M. (1986), The Society of Mind, Simon and Schuster, New York,

New York.

Minsky, M. & Papert, S. (1969), Perceptrons, MIT Press, Cambridge,

Massachusetts.

Newcombe, F. & Ratcliff, G. (1989), Disorders of Visupspatial Analysis,

in _Handbook of Neuropsychology, Volume '>', Elsevier, New York, New

Yo rk,

Newell, A. _ Simon, H. A. (1981), Computer Science as Empirical Inquiry'.

Symbols and Search, in J. Haugeland, ed., 'Mind Design', MIT Press,

Cambridge, Massachusetts, chapter 1, pp. 35-66.

Penrose, R. (1989), The Emporer's New Mind, Oxford University Press,

Oxford, United Kingdom.

Philip Teite]baum, V. C. P. & Pellis, S. M. (1990), Can Allied Reflexes

Promote the Integration of a Robot's Behavior, in 'Proceedings of the

First International Conference on Simulation of Adaptive Behavior', MIT

Press, Cambridge, Massachusetts, pp. 97-104.

Pomerleau, D. A. (1991), 'Efficient Training of Artificial Neural Networks

for Autonomous Navigation', Neural Computation.

Rosenblatt, F. (1962), Principles of Neurodyn&mics, Spartan, New York,

New York.

Kosenschein, S. J. & Kaelbling, L. P. (1986), The Synthesis of Digital Ma-

chines with Provable Epestemic Properties, in J. Y. Halpern, ed., 'Pro-

ceedings of the Conference on Theoretical Aspects of Reasoning about

Knowledge', Morgan Kaufmann, Monterey, California, pp. 83-98.

Rumelhart, D. E. _ McClelland, J. L., eds (1986), Parallel Distributed

Processing, MIT Press, Cambridge, Massachusetts.

Searle, J. R. (1992), The Rediscovery of the Mind, MIT Press, Cambridge,

Massachusetts.

Simon, H. A. (1969), The Sciences of the Artificial, MIT Press, Cambridge,

Massachusetts.

Springer, S, P. & Deutsch, G. (1981), Left Brain, Right Brain, W.H. Free.

man and Company, New York.

Steele Jr., G. L. (1990), Common Lisp, The Language, second edn, Digital

Press.

Stein, L. A. (to appear), 'Imaglnat[on and Situated Cognition', Journal of

Experimental and Theoretical Artificial Intelligence.

Turing, A. M. (1970), Intelligent Machinery, in B. Meltzer & D. Michie,

eds, 'Machine Intelligence 5', American Elsevier Pub]ishlng, New York,

New York, pp. 3-23.

Ullman, S. (1991), Sequence-Seeklng and Counter Streams: A Mode] for

Information Processing in the Cortex, Memo 1311, M_ss&chusetts Insti-

tute of TechnologyArt[fici&l Intelligence Lab, Cambridge, Massachusetts.

Viola, P. A. (1990), Adaptive Gaze Control, Master's thesis, MIT, Depart-

ment of EECS, Cambridge, Massachusetts.

Weiskrantz, L. (1986), Bl[ndslght, Oxford University Press, Oxford,

United Kingdom.

YahOO, H. & Stein, L. A. (1993), An Adaptive Communicat[on Protocol

for Cooperating Mobile Robots, in J.-A. Meyer, H. Roitblat & S. Wilson,

eds, 'From Animals to Animals: Proceedings of the Second Conference

on the Simulation of Adaptive Behavior', MIT Press, Cambridge, Mas-

sachusetts, pp. 478-485.

263