The Next Generation of Robots?

MIT Artificial Intelligence Laboratory — Research Directions

The Next Generation of Robots?Rodney Brooks

and

Una-May O’Reilly


Our Objectives

• How can biology inform robotic competence?

• How can aspects of human development and social behavior inform robotic competence?


Our Approach

• Exploit the advantages of the robot’s physical embodiment

• Integrate multiple sensory and motor systems to provide robust and stable behavioral constraints

• Capitalize on social cues from an instructor

• Build adaptive systems with a developmental progression to limit complexity


Cog and Kismet

Our Humanoid Platforms


Biological Inspiration for Cog

• Cog has simulated musculature in its arms

• Cog has an implementation of a human model of visual search and attention

• Cog employs context-based attention and internal situations influence action

• Cog uses a naïve model of physics to distinguish animate from inanimate


Social Inspiration for Cog

• A theory of mind

• A theory of body

• Mimicry


Neck pan

Neck tilt

Neck lean

Eye tilt

Left eye panRight eye pan

Camera with wide field of viewCamera with narrow field of viewAxis of rotation

Human <—> Robot

Microphones

Cameras

Speech synthesizer

Facial features

Gaze direction

Head orientation


Levels of Control

robot respondsto human

human respondsto robot

Social Level

Primitives Level

Skills Level

Behavior Level

current goal

current primitive(s)coordination

between motormodalities

perceptualfeedback


Kismet’s Competencies

• Direct Visual Attention

• Recognize Socially Communicated Reinforcement

• Communicate Internal State to Human

• Regulation of Social Interaction


No One in Charge

• 11 400-500 MHz PCs– QNX (vision)

– Linux (speech recognition)

– NT (speech synthesis & vocal affect recognition

NTspeech synthesisaffect recognition

LinuxSpeech

recognition

Face Control

EmotiveResponse

Percept& Motor

Drives & Behavior

L

Tracker

Attnsystem

Dist.to

target

Motionfilter

Eyefinder

Motorctrl

audiospeechcomms

Skinfilter

Colorfilter

QNX

CORBA

CORBA

CORBA

dual-portRAM

CamerasEye, neck, jaw motors

Ear, eyebrow, eyelid,lip motors

Microphone

Speakers

• 4 Motorola 68332 micro-controllers

• L, multi-threaded lisp

• higher-level perception, motivation, behavior, motor skill integration & face control


Visual Attention

inh

ibit

res

et

Frame Grabber

Eye Motor Control

Top down,task-driveninfluences

w w w w

skin tone habituationmotioncolor

attention


Visual Search


Social Constraints

Comfortable interaction speed

Too fast – irritation

response

Too fast,Too close –

threat response

Comfortable interaction distance

Too close – withdrawal response

Too far – calling

behavior

Person draws closer

Person backs off

Beyond sensor range


Cross Cultural Affect

Evidence for 4 contours inKismet-directed speech

time (ms)

pitc

h, f

(kH

z)o

approval

That’s a good bo-o-y! No no baby.

time (ms)pi

tch,

f (

kHz)

o

prohibition

Can youget it?

Can youget it?

time (ms)

pitc

h, f

(kH

z)o

attention

time (ms)

pitc

h, f

(kH

z)o

MMMM Oh, honey.

comfort


Affect Recognizer

prohibition attention & approval

soothing & low-energy neutral

neutral

Soothing & Low-Intensity

neutralvs

Everything Else

Soothingvs

Low-Intensity Neutral

Approval & Attentionvs

Prohibitionvs

High Intensity Neutral

approvalvs

attention

soothing

prohibition

neutral

approval

attention

ener

gy v

aria

nce

pitch mean


Naive Subjects

• 5 female subjects• 4 naive subjects• 1 caregiver

• Four contours and neutral speech

• praise, prohibition, attention, soothing

• Multiple languages• French, German, Indonesian,

English, Russian

• Driven by Human


Facial Expressions

arousal

sleep

displeasure pleasureneutral

excitement

depression

stress

calm

afraidangry

frustrated

relaxed

content

elated

bored

sad

fatigued

happy

surprise

sleepy


Facial Postures in Affect Space

Negativevalence

Open stance

surprise

unhappy

tired

anger

fearLow

arousal

Higharousal

Positivevalence

Closed stance

disgust

accepting

stern

content


Face, Voice, Posture


Turn-Taking / Proto-Dialog

•Naïve subjects•Told to “talk to the robot”•Engage in turn taking•No understanding (on either side) of content


Implemented Model of Visual Search and Attention

Activation Map

Color w

Skin w

Motion w MotorSystem

Motivation System

Habituation w



Hardware – Cog’s Arms

• 6 DOF in each arm

• Series elastic actuator

• Force control

• Spring law

Motor andGearbox

Encoder

Spring

Bearing

Actuatoroutput

xbkxxF )(


Hardware – Cog’s Head

• 7 degrees of freedom

• Human speed and range of motion


Visual and Inertial Sensors

Peripheral View

Foveal View

Peripheral View

Foveal View

3-axis inertial sensor


Computational System

• Designed for real-time responses

• Network of 24 PC’s ranging from 200-800 MHz

• QNX real-time operating system

• Implementation shown today consists of – ~26 QNX processes

– ~75 QNX threads


Documents

The Next Generation of Robots?