T r e n d s & C o n t r o v e r s i e s

Humanoid and Android ScienceHiroshi Ishiguro, Osaka UniversityMinoru Asada, JST ERATO Asada Synergistic IntelligenceProject and Osaka University

Why are we attracted to humanoids and androids? Theanswer is simple: we tend to anthropomorphize nonhuman

things. Humans always anthropomorphize targets ofcommunication and interaction, so we expect muchfrom humanoids. In other words, we find a human in thehumanoid. Recently, robotics researchers have begunshifting from traditional studies on navigation and manip-ulation to studying interaction with robots.

The study of human-robot interaction has been neglect-ing an issue—appearance and behavior. The interactiverobots that have been developed thus far are nonandroidtypes, so the researchers who developed them didn’t focuson the robots’ appearances. Evidently, a robot’s appear-ance influences subjects’ impressions, and it’s an impor-tant factor in evaluating the interaction. Although manytechnical reports compare robots with different behaviors,they haven’t focused on the robots’ appearances. There aremany empirical discussions on simplified robots, such asdolls. However, designing a robot’s appearance, particu-larly to make it appear humanoid, has always been theindustrial designers’ role rather than that of researchersand engineers. This is a serious problem for developingand evaluating interactive robots. Appearance and behav-ior are tightly coupled.

Bridging science and engineeringOne way to tackle the issue is to use a humanlike

robot—an android—to study human-robot interaction.Figure 1a is a humanoid developed by Mitsubishi HeavyIndustry, and figure 1b shows an android that HiroshiIshiguro and his colleagues developed in cooperation withKOKORO. The android has 42 air actuators for the uppertorso, excluding fingers. During development, we deter-mined the actuators’ positions by analyzing a human’smovements using a precise 3D motion-capture system.The actuators can represent unconscious movements suchas chest movements due to breathing, in addition to large,conscious movements of the head and arms. Furthermore,the android can generate facial expressions that are impor-tant for interaction with humans. When we publicized theandroid through the media, we were anxious about ordinaryJapanese people’s reactions. However, it wasn’t uncannyfor them; they just praised the quality and technology.

Developing androids requires contributions from bothrobotics and cognitive science. To realize a more human-

Human-Inspired RobotsSilvia Coradeschi, Orebro University

Robots are just now becoming part of our everyday life and beingused by ordinary people. Future robots will work in hospitals, eldercare centers, schools, and homes. Similarity with humans can facilitateinteraction with a variety of users who don’t have robotics expertise,so it makes sense to take inspiration from humans when developingrobots. However, humanlike appearance can also be deceiving,convincing users that robots can understand and do much morethan they actually can. Developing a humanlike appearance mustgo hand in hand with increasing robots’ cognitive, social, and per-ceptive capabilities.

In this installment of Trends & Controversies, Hiroshi Ishiguro andMinoru Asada present the ultimate imitation of a human’s appear-ance: an android. Androids are robots that not only look like humansbut also move like us and have artificial skin that feels like ours.

Humanlike robots must communicate clearly using natural lan-guage. Stuart Shapiro outlines natural language’s importance bothas a practical way to communicate and a means of encouragingsocial interaction.

Cognitive abilities are important for robots interacting with humansand acting in complex environments. Michael Thielscher presentsthe field of cognitive robotics, which investigates how robots canachieve greater autonomy and flexibility in such environments.

Social competence is also essential in natural human-robot com-munication. Cynthia Breazeal has been a pioneer in developingsocially interactive robots, and she presents the challenges andprogress in this field.

The growing field of socially assistive robotics is becoming animportant testbed for human-inspired robots. Maja J. Mataric out-lines the challenges of aiding people by means of social interactionrather than through physical contact alone.

Finally, Hiroshi Ishida presents olfaction as a new sense for robots.The ability to recognize smells will bring robots closer to humans.This sensor offers great new possibilities but also new challenges forrobotics. —Silvia Coradeschi

74 1541-1672/06/$20.00 © 2006 IEEE IEEE INTELLIGENT SYSTEMSPublished by the IEEE Computer Society

like android, knowledge from human sci-ence (that is, from studying humans) is nec-essary. This new interdisciplinary frame-work between engineering and cognitivescience is called android science.1

In the past, robotics research used knowl-edge from cognitive science, and cognitivescience research utilized robots to verifyhypotheses for understanding humans.However, robotics’ contribution to cogni-tive science has been inadequate. Appear-ance and behavior couldn’t be handled sep-arately, and nonandroid type robots weren’tsufficient as cognitive science tools. Weexpect that using androids with a human-like appearance can solve this problem.Robotics research based on cues from cog-nitive science faces a similar problem, be-cause it’s difficult to recognize whether thecues pertain solely to a robot’s behavior,isolated from its appearance, or to the com-bination of its appearance and behavior. Inthe framework of android science, androidsenable us to directly share knowledge be-tween the development of androids in engi-neering and the understanding of humansin cognitive science.

So, android science has several majorresearch issues. The issues in robotics areto develop a humanlike appearance usingsilicon, humanlike movement, and human-like perception by integrating ubiquitoussensor systems. In cognitive science, theissue is conscious and subconscious recog-nition. When we observe objects, variousmodules are activated in our brain. Each ofthese matches the input sensory data withour brains’ models of human faces, voices,and so on, which affect our reactions. So,even if we recognize that a robot is an an-droid, we react to it as if it were human.Android science’s goal is to find the essen-tial factors of humanlikeness and realize ahumanlike robot.

How can we define humanlikeness?Furthermore, how do we perceive it? Thathumans have both conscious and uncon-scious recognition is fundamental for boththe engineering and scientific approaches. Itwill be an evaluation criterion in androids’development, and it provides us cues forunderstanding the human brain mechanismof recognition.

From android science tohumanoid science

The history of intelligent robotics startedwith Shakey, a robot developed at SRI in

1965. Shakey provided robotics researcherswith several important research questions.We focused on the fundamental issues formaking Shakey more intelligent, such asAI, computer vision, and language recogni-tion. Since Shakey’s development, we’vebeen using these technologies to create newhumanoids and androids, which provide uswith important new research questions asShakey did. These questions are in interdis-ciplinary areas among robotics, cognitivescience, neuroscience, and social science.

Android science is also an interdiscipli-nary framework, but it’s rather limited. Inaddition to humanlike appearance andmovement, we must consider the internalmechanisms, such as humanlike dynamicand adaptive mechanisms and complicatedsensorimotor mechanisms, for more tightlycoupling engineering and science. As wementioned earlier, what we wish to know iswhat a human is. We can understand this bydeveloping humanoids comprising human-like hardware and software. We call thisextended framework humanoid science.

Our project, JST ERATO (Japan Science

and Technology Agency, Exploratory Re-search for Advanced Technology) AsadaSynergistic Intelligence (www.jeap.org), isbased on the humanoid-science framework.Synergistic intelligence means intelligentbehaviors that emerge through interactionwith the environment, including humans.Synergistic effects are expected in brainscience, neuroscience, cognitive science,and developmental psychology. Synergisticintelligence provides a new way of under-standing ourselves and a new design theoryof humanoids through mutual feedbackbetween the design of humanlike robotsand human-related science.

Synergistic intelligence adopts cognitivedevelopmental robotics,2 a methodologythat comprises the design of self-develop-ing structures inside the robot’s brain andenvironmental design (how to set up theenvironment so that the robots embeddedtherein can gradually adapt to more complextasks in more dynamic situations). Here, oneof the most formidable issues is nature ver-sus nurture. To what extent should we embedthe self-developing structure, and to what

JULY/AUGUST 2006 www.computer.org/intelligent 75

Figure 1. Humanoid and android robots. (a) Humanoid Eveliee P1 is based onWAKAMURU, developed by Mitsubishi Heavy Industry. (b) Android Repliee Q2 was developed in cooperation with KOKORO (www.kokoro-dreams.co.jp).

(a) (b)

extent should we expect the environment totrigger development? We’re approachingthis issue using the kinds of topics in figure 2.

References

1. H. Ishiguro, “Interactive Humanoids andAndroids as Ideal Interfaces for Humans,”Proc. 11th Int’l Conf. Intelligent User Inter-faces, ACM Press, 2006, pp. 2–9; www.ed.ams.eng.osaka-u.ac.jp/research/Android/android_eng.htm.

2. M. Asada et al., “Cognitive DevelopmentalRobotics as a New Paradigm for the Design ofHumanoid Robots,” Robotics and AutonomousSystems, vol. 37, 2001, pp. 185–193.

Natural-Language-CompetentRobotsStuart C. Shapiro, State University of NewYork at Buffalo

Why use natural language to communi-cate with computer systems? I’ve alwaysthought the answer was obvious. BecauseNL is the most natural method of commu-nication for people,1 computer systems wouldbe easiest to use if they understood and spokeNL. I often get resistance to this opinion, butthe resistance almost always takes the form ofcomparing current NL interfaces to currentGUIs. Current NL interfaces don’t work as wellas current GUIs because computers don’t

fully understand NL. But that just makes NLunderstanding an important research topic.

Indeed, not only is NLU in general anactive research topic, so is the specific topicof NL interfaces to robots, as evidenced bythe recent ACM Conference on Human-Robot Interaction (HRI 2006, www.hri2006.org). Moreover, not only are people whodescribe themselves as NLU researchersworking to make NL understandable to com-puters, the field of programming languagescan be seen as a bottom-up attack on thesame problem. Many programming lan-guage advances, as far back as the develop-ment of assembly languages, have been pre-sented as letting programmers expressproblems in a high-level language that’s nat-ural for humans.

Definitions and motivationsBy NL, we could be referring to text or

speech. Many NL researchers are workingon text-based systems because they’reinterested in processing information fromthe Web or other documents. Others areusing text as an expedient until speechrecognition becomes easier to use. (I’m inthis group.) So, by an NL-competent robot,I generally mean one that understands andgenerates speech.

What counts as a robot? By robot, Imean an embodied computer system thathas sensor and effector organs. I’m notlimiting the term to hardware-implemented

robots that operate in the real world. I alsoinclude robots with simulated bodies andsensor and effector organs that operate insimulated or virtual reality worlds. I alsoinclude teleoperated as well as autonomousrobots. A teleoperated robot is controlledby a console, joystick, or other device. Idon’t, however, want to argue that users ofprosthetic limbs or master-slave manipula-tors, such as those that manipulate objectsin toxic environments or perform micro-surgery, would necessarily benefit from NLcommunication.

R&D of NL-competent robots has at leasttwo motivations. One is science. Because NLis the natural communication medium forpeople, researchers attempting to achieve a computational understanding of humanintelligent behavior and create devices thatexhibit such behavior must include NL-competent robots in their research agendas(although maybe not in their personal agen-das). The other motivation is pragmatic—the belief that robots will be more useful ifthey are NL-competent. Because I take thescientific motivation to be inarguable, I’llconcentrate on the pragmatics.

Speech as a commandlanguage

It might seem that controllers of teleoper-ated robots wouldn’t need to use NL, giventheir more direct control devices. Experi-ments have shown, however, that using single-word speech commands is benefi-cial.2 This is because of both the naturalnessof speech and the ease of using it when theoperator’s hands and eyes are occupied withthe other controls. However, Manuel Ferreand his colleagues didn’t find a more com-plete speech-based sentence-understandingsystem to be beneficial because it requiredthe operator to pause between words, causinga considerable loss in fluency.2 They foundthat operators don’t work comfortably if thesystem’s recognition rate is under 90 percent.This is another indication that when NL inter-faces don’t understand fluent speech, theydon’t compete well with other interfaces.

If speech is useful for robot controllerswhose hands and eyes are occupied, it’seven more useful for people who can’t usetheir hands or eyes because of long-termdisability or short-term injury. The devel-opment of speech interfaces for assistivedevices is therefore an active research area,and several researchers presented papers onthe topic at HRI 2006.

76 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Understandingemergence of

human intelligence

Design forsymbiotic

humanoids

Synergisticintelligence

Sl-mechanismVerification of computational

models by brain functional imaging andhuman/animal experiments

Perso-SlSynthetic understanding

Cognitive development byscience of imitation

Physio-SIMotor emergence

through dynamic interactionwith environment based on

artificial muscles

Socio-SIUnderstanding and

realization of communicationwith androids

Figure 2. The JST ERATO Asada Synergistic Intelligence project is based on a humanoid-science framework.

NL as a programminglanguage

Although using speech as a commandlanguage is useful, using NL as a robotprogramming language is even more use-ful. Any robot will have a repertoire ofactions and recognizable domain featuresdesigned or programmed into it. A robustunderstanding system that lets a humancontroller express these actions and fea-tures in NL would provide a useful speechcommand language. But consider situa-tions in which a human wants to ask therobot to perform a sequence of actions, todo one of several actions depending onwhat it perceives in its world, or to repeatsome action until it has accomplished someeffect. As we know, providing NL constructsto express these control structures will makeNL a full-fledged programming language.Some of these more complex proceduresmight be ad hoc procedures the controllerwants done only once, but others might bemore common. If the human can conceptu-alize these possibly parameterized proce-dures and express them in a way acceptableto the NL understanding system, and if theNLU system provides a way for the humanto say something like “The way to do X is todo Y,” then the NLU robot command lan-guage will have all the constructs of a proce-dural programming language. StanislaoLauria and his colleagues achieved this tosome extent in the context of a hardwarerobot that can take directions for navigatingin a small model city.3

To use NL to instruct and commandrobots, the NLU system must translate NLinputs into some kind of robot acting pro-gram. That is, the meaning-representationlanguage that the NLU system uses mustbe an autonomous-agent acting language.Unfortunately, researchers designing agentacting languages and those designing rep-resentation languages for the (contextuallygrounded) semantics of NL mostly operateindependently of each other.

A promising approach to building NL-competent robots is to combine these re-searchers’ efforts. For example, the repre-sentation the robot uses to understand averb phrase could be a construct in the ro-bot’s acting language that it can perform. Inthis way, the robot performs according toan imperative sentence by executing thestructure representing its understanding ofthe command. By appropriately appendinga representation of itself and of the current

time to the representation of an action ithas just performed, the robot believes it hasperformed the action. Giving that belief toits NL generation routine produces an NLreport of what it has just done. My col-leagues, students, and I have been pursuingthis approach.4

Given an NL programming language,having an NL development environmentwould also be important, and it would bemost convenient if the robot participated in the development process. For example,Lauria and his colleagues’ navigation robotuses NL generation to ask for explanationsof directions it doesn’t understand.3

Clifford Nass and Scott Brave found thatpeople can’t help but interpret speech in asocial context and attribute human charac-teristics to the speaker, even if they know thespeaker is a machine.1 So, taking the social

aspects of speech into account is importantso that “users will not simply talk at andlisten to computers, nor will computers sim-ply talk at and listen to users. Instead, peo-ple and computers will cooperatively speakwith one another”1 [italics in original].

Future research directionsNL-competent robots aren’t here yet, but

we’re making progress, and speech inter-faces are becoming common at call centersand in automobiles.5 Robotics researchersshould be encouraged to include NL com-petence among their robots’ abilities, andNL researchers should be encouraged toconsider robots as platforms for their work.

References1. C. Nass and S. Brave, Wired for Speech: How

Voice Activates and Advances the Human-Computer Relationship, MIT Press, 2005.

2. M. Ferre et al., “Voice Command Generationfor Teleoperated Robot Systems,” Proc. 7thIEEE Int’l Workshop Robot and Human Com-munication (ROMAN 98), vol. 2, IEEE Press,1998, pp. 679–685.

3. S. Lauria et al., “Training Personal RobotsUsing Natural Language Instruction,” IEEEIntelligent Systems, vol. 16, no. 5, 2001, pp.38–45.

4. S.C. Shapiro and H.O. Ismail, “Anchoring ina Grounded Layered Architecture with Inte-grated Reasoning,” Robotics and AutonomousSystems, vol. 43, nos. 2–3, 2003, pp. 97–108.

5. E. Millard, “Talking Computers Just aroundthe Corner,” Newsfactor Magazine, 14 Mar.2006, www.newsfactor.com/story.xhtml?story_id=03200000ST9C.

Cognitive RoboticsMichael Thielscher, Dresden University ofTechnology

Cognitive robotics aims at modelinghuman-level cognitive faculties in robots.Thinking in abstract terms is humans’ mostdistinct cognitive capability. Specifically, webenefit from this ability when we mentallyentertain the effects of different courses ofaction and then choose the one that willlikely satisfy our needs and achieve ourgoals. This requires us to reflect on the ac-tions we could take and to know whateffects they normally have. In other words,coming up with a plan that’s suitable underthe circumstances results from our ability todraw the right conclusions from our knowl-edge of how the world around us functions.This ability lets us exhibit intelligent behav-ior in situations we haven’t encountered be-fore and that no preprogrammed routinedeals with adequately.

In this regard, robots should greatlyprofit from taking inspiration from our cog-nitive ability to reflect on our own acts.This doesn’t mean that cognitive roboticsmust necessarily take inspiration from howhuman brains implement these functions,considering that brains are networks ofneurons. Instead, the focus is on the high-level cognitive functions themselves—theabilities to represent and reason about knowl-edge to make rational decisions, devise com-plex plans, react sensibly to unforeseen cir-cumstances and problems, and adapt tonew environments. All these capabilitiesare characteristics of human-level intelli-gence. Today’s robots, however, can’t

JULY/AUGUST 2006 www.computer.org/intelligent 77

If speech is useful for robot

controllers whose hands and eyes

are occupied, it’s even more

useful for people who can’t use

their hands or eyes because of

disability or injury.

reflect on what they can do or decide ontheir own whether a preprogrammed behav-ior is sensible under given circumstances.Most AI researchers expect that future cogni-tive robots will be much more autonomousand flexible than those of today.

Formalisms and applicationsCognitive-robotics research goes back to

AI’s beginning. One of the earliest knowl-edge representation formalisms, developedin the early 1960s, was designed specifi-cally to provide systems with knowledge ofactions. This was accompanied by a mech-anism for solving tasks by fully automatedreasoning. In this way, an autonomous sys-tem would be able to compose a suitableplan entirely on its own on the basis of pre-dicted outcomes of actions. However, thefirst implementations of this knowledgerepresentation formalism—known as thesituation calculus—didn’t scale up beyonddomains with a small state space and fewactions. That is, they suffered from theframe problem, which turned out to be oneof the most interesting and challengingformal problems in logic-based AI. Whilesimple to understand, it proved difficult tosolve and over the years has led to an abun-dance of insights and results. The frameproblem was, for instance, the driving forcebehind the emergence of nonmonotonicreasoning, now a vibrant, active AI sub-field in itself.

The first satisfactory solution to the frameproblem didn’t appear until the early ’90s,still using the classical situation calculus asthe underlying knowledge representationlanguage.1 This marked the beginning ofa thriving era for the theory of cognitiverobotics. Other solutions to the frame prob-lem emerged in quick succession usingdifferent representation techniques, such asthe event calculus2 or the fluent calculus.3

In recent years, each of these basic solutionshas been generalized in view of the additionalchallenges that autonomous systems face inreal-world environments. Examples are hy-brid environments with both discrete andcontinuous change or environments whereactions frequently fail because of unknownor uncontrollable circumstances. Thesedevelopments show that using logic-basedrepresentation formalisms can let us dealwith large state spaces, thanks to a high levelof abstraction, and can provide simple,computationally efficient ways to deal withincomplete knowledge and nondeterminism.

In addition to the emergence of increasinglysophisticated representation formalisms,theoretical approaches to cognitive roboticshave evolved into actual programming lan-guages and implementations that let youwrite high-level control programs for auto-nomous systems. Examples of such gen-eral languages are GOLOG (Algol in Logic),based on the situation calculus; FLUX (Flu-ent Calculus Executor), based on the fluentcalculus; and a reasoning system for cogni-tive agents known as SNePS (Semantic Net-work Processing System). Thanks to a highlevel of abstraction paired with efficient rea-soning mechanisms, these languages andtheir implementations let you write and runcontrol programs for complex tasks in envi-ronments that give rise to large state spaces.

Cognitive robotics systems have provedto be competitive in a variety of areas. For

example, planning itself has turned into animportant AI subfield with a variety of ap-plications and its own conferences andorganized competitions for implementedsystems. Symbolic reasoning is also anintegral part of software agents that mustinteract and collaborate in heterogeneousmultiagent systems. A third example of anapplication of logical reasoning about ac-tions is general game playing, which isconcerned with the development of sys-tems that can understand descriptions ofarbitrary games and play them well withouthuman intervention.

More than 40 years of research have ledto significant progress, and cognitive robot-ics is now a vibrant, active field that plays amajor role at all large AI conferences. Sur-prisingly (and disappointingly) enough, theprogress has been mostly restricted totheory and simulation; little progress has

occurred with regard to endowing actualrobots in real-world environments withcognitive functions. One exception is thecontrol program of a robotic museumguide, which employs GOLOG to composehigh-level tour plans according to visitors’interests.4 Another is a high-level controlmodule for unmanned airborne vehicles(helicopters), which observes traffic sce-narios on the ground, predicts what trafficwill do in the near future, and plans thehelicopters’ actions.5

However, these real-world systems useonly a limited form of symbolic reasoningspecifically targeted to the application athand. Cognitive-robotics research hasn’tachieved its main goal, which is to signifi-cantly enhance robots’ autonomy and flexi-bility with the help of high-level reasoningcapabilities. Why not?

Challenges and future workIn comparison to devising software agents

that live in pure virtual environments, con-trolling actual robots that move in the physi-cal world poses two major additional chal-lenges: First, the symbols that form thebasis for the cognitive processes must begrounded in the real world. Second, anautonomous physical system’s actions arefar less reliable than those of a program ina simulated world.

The symbol-grounding problem arisesbecause high-level reasoning is based onabstract terms and concepts, but a robotprocesses nothing but raw sensory data.How to map these data onto a single sym-bol that expresses as complex an entity as,say, “Sandra’s coffee mug” is a largelyunsolved problem. Lacking a sufficientlygeneral solution, the relationships betweensymbols and sensory data must be pre-defined in every detail, causing a signifi-cant loss in the flexibility and autonomy ofhigh-level reasoning.

With regard to the problem of unreliableactuators and sensors in real-world envi-ronments, established solutions are allbased on probabilistic approaches. Specifi-cally, robust algorithms for the basic tasksof self-localization and navigation forautonomous robots require a robot to main-tain probability distributions as its internalworld model, which is updated accordingto probabilistic knowledge of its actions’effects. Contemporary theories of cognitiverobotics, however, still lack a tight integra-tion of logical and probabilistic reasoning.

78 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Theoretical approaches to

cognitive robotics have evolved

into programming languages and

implementations that let you write

high-level control programs for

autonomous systems.

JULY/AUGUST 2006 www.computer.org/intelligent 79

Recently, researchers have looked intocombining probability theory with knowl-edge representation languages such as thesituation or fluent calculus. But so far, nocompetitive implementation combines theexactness of probabilistic calculations withthe power of logical reasoning when itcomes to coping with large state spacescontaining diverse information.

Significant advances toward cognitiverobotics’ actual goal will therefore requirea new generation of theories and imple-mentations, which will emerge from suffi-ciently general solutions to the symbol-grounding problem and novel combinationsof logical and probabilistic reasoning. Thiswould pave the way toward a new genera-tion of autonomous robots. High-level rea-soning is a key aspect of truly intelligentbehavior, which emerges from the ability tomake rational decisions and to devise plansfor complex tasks, as well as to exhibit agreat deal of flexibility dealing with unfore-seeable circumstances. Probabilistic reason-ing alone won’t be enough to reach this levelof intelligence because it doesn’t allow forthe high degree of abstraction needed to copewith a large body of diverse information.

References

1. R. Reiter, Knowledge in Action, MIT Press,2001.

2. M. Shanahan, Solving the Frame Problem: AMathematical Investigation of the CommonSense Law of Inertia, MIT Press, 1997.

3. M. Thielscher, Reasoning Robots: The Artand Science of Programming Robotic Agents,Springer, 2005.

4. W. Burgard et al., “Experiences with an Inter-active Museum Tour-Guide Robot,” ArtificialIntelligence, vol. 114, nos. 1–2, 1999, pp.3–55.

5. P. Doherty et al., “The WITAS Unmanned Aer-ial Vehicle Project,” Proc. 14th EuropeanConf. Artificial Intelligence, IOS Press, 2000,pp. 747–755.

Human-Robot PartnershipCynthia Breazeal, MIT Media Lab

What kind of future do you envision withrobots? The science fiction stories andmovies that fascinated me as a child shapemy dream. There are the mechanical droidsR2-D2 and C-3PO from the movie Star

Wars. There are many wonderful examplesin the short stories of Isaac Asimov, such asRobbie. And more recently, there’s Teddyfrom the movie Artificial Intelligence.

These sci-fi robots differ dramaticallyfrom today’s autonomous mobile robots.We generally view modern autonomousrobots as tools that human specialists use toperform hazardous tasks in remote environ-ments (such as sweeping minefields, inspect-ing oil wells, mapping mines, and exploringother planets). In contrast, my favorite sci-ence fiction robots aren’t tools that do thingsfor us. Rather, they’re socially savvy partnersthat do things with us.

Recent commercial applications aremotivating research into building robotsthat play a beneficial role in ordinary peo-ple’s daily lives (see Maja J. Mataric’s fol-lowing essay on socially assistive robots).This new breed of social (or sociable) robotmust be natural and intuitive enough for theaverage consumer to interact and commu-nicate with, work with as partners, andteach new capabilities.

One of the best known social (or socia-ble) robots is Kismet—the first autonomousrobot explicitly designed to explore face-to-face interactions with people.1 Kismet isconsidered humanoid because of its expres-sive anthropomorphic face, even though itdoesn’t have a humanoid body (see figure3). The research with Kismet focused onexploring the origins of social interactionand communication in people, namely that

which occurs between caregiver and infant,through extensive computational modelingguided by insights from psychology andethology.

In particular, Kismet was one of the earli-est works to explore socio-emotive interac-tion between humans and robots. Inspiredby our earliest infant-caregiver exchanges,which are heavily grounded in the regula-tion of emotion and its expression, Kismetused its emotion system and correspondingexpressive modalities to communicate inter-nal affective states reflecting the degreeto which its drives and goals were beingmet. Internally, Kismet’s emotion modelsinteracted intimately with its cognitive sys-tems to influence behavior and goal arbitra-tion. In effect, Kismet socially negotiatedits interaction with people via its emotiveresponses so that humans would help itachieve its goals and satiate its drives. Thescientific literature on emotion theory, socialcognition, and its development throughinfancy and childhood has inspired andguided the development of several sociallyinteractive robots.2

Challenges and progress insocial robotics

As with any emerging area of inquiry,the fundamental scientific questions thatsocial robotics pursues distinguish it fromother areas.1,2 A few challenges of socialrobotics that I’m particularly interested ininclude these three:

Figure 3. Cynthia Breazeal and her collaborators developed Kismet at the MIT ArtificialIntelligence Lab in the 1990s.

80 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

• Robots with social-cognitive skills.Whereas research with modern autono-mous robots has largely focused on theirability to interact with inanimate objects,socially intelligent robots must under-stand and interact with people. They mustunderstand people in social-psychologicalterms to appreciate the goals, beliefs,feelings, motives, and other mental statesunderlying our behavior.

• Robots that collaborate with people asfull-fledged partners. Socially intelligentrobots work alongside humans as peers,communicate with us in human-centricterms, and participate in successful rela-tionships with people that provide bene-fits over an extended period of time.

• Robots as social learners that learn frompeople. Socially intelligent robots engagein rich forms of social learning with peo-ple, such as imitation and tutelage, tolearn new skills, knowledge, and taskswhile on the job.

Promisingly, there have been initial andongoing strides in all of these areas.2–4

This article briefly surveys my owngroup’s research at the MIT Media Lab andour ongoing progress on these three chal-lenges (see http://robotic.media.mit.edu).Leonardo is the expressive full-bodyhumanoid robot that serves as our experi-mental platform (see figure 4). We studyand evaluate the robot’s performance from atechnical perspective to assess its ability tocommunicate with, collaborate with, and

learn from people in situated task scenarios.Additionally, we conduct human subject stud-ies to assess whether the robot’s performanceis cognitively and behaviorally synergisticfor people and to confirm that the human-robot team is more effective as a whole.

Robots with social-cognitiveskills

Our computational models are deeplyinformed by scientific theories of humanand animal social cognition to endow ourrobots with useful social-cognitive skillsthat are also human compatible. In particu-lar, recent experimental data from neuro-science and psychology provide empiricalevidence that human social cognition isdeeply embodied.5 Simulation theory, forinstance, posits that we use our own bodyand cognitive and affective processes tosimulate the mental processes of others—to take their mental perspective to infertheir intents, beliefs, affect, motives, and soon.6 Inspired by this work, we’ve beendeveloping a simulation theoretic frame-work to endow our robot with mental per-spective-taking skills. For instance, we’vedemonstrated the robot’s ability to learnaffective appraisals of novel objects viasocial referencing or to reason about andinfer its human partner’s goals, attention,beliefs, and affect from their observablebehavior in various initial tests, such as thefalse-belief task. (The false-belief task,which determines whether a person canappreciate that others can have beliefs that

differ from his or her own, is used as adiagnostic tool in developmental psychol-ogy.) These skills play an important role inrobots that collaborate with humans asteammates and learn from human teachers.

Robots and people as full-fledged partners

Collaborative discourse theory has pro-vided a rich theoretical framework to un-derstand and model human-style collabora-tion for human-robot teams. Researchershave developed sophisticated discoursemodels to support human-computer collab-oration where the human and computerestablish and maintain shared beliefs, goals,and plans to coordinate joint action throughnatural language. Human-robot teamworkintroduces an important nonverbal dimen-sion where physical actions communicatemental states (for example, gaze directioncommunicates attention, facial expressioncommunicates emotion, and actions com-municate intentions). These mental andbody states must be coordinated betweenhuman and robot to successfully collaborateon a physical task in a shared space.

We’ve demonstrated our robot’s abilityto apply its social-cognitive skills to com-pare and reason about how its humanpartner’s goal and belief states (as com-municated through verbal and nonverbalbehavior) relate to its own “mental” statesto provide the person with informationaland instrumental support. For example, inthe case of informational support, the robotcan relate its own beliefs about the state ofthe shared workspace to those of the humanon the basis of the visual perspective ofeach. If a visual occlusion prevents the hu-man from knowing important informationabout that region of the workspace, therobot knows to direct the human’s attentionto bring that information into commonground. On the basis of the principles ofthe joint intention and speech act theories,the robot uses a versatile range of nonver-bal behaviors to coordinate effective, effi-cient teamwork. In the case of instrumentalsupport (that is, performing an action tohelp someone achieve a task-oriented goal),the robot can infer the human’s intent (thatis, a desired effect on the workspace) fromobserving the person’s behavior. If the hu-man fails to achieve his or her goal, therobot can reason about how it might helpthe human by either performing a directaction that achieves that goal or by execut-

Figure 4. Leonardo, (a) with and (b) without cosmetic finishing, is being developed atthe MIT Media Lab (aesthetic design by Stan Winston Studio). (images courtesy of SamOgden Photography)

(a) (b)

ing a complementary action that enables thehuman to achieve the goal (for example, therobot might disengage the lock on a deviceto help the human activate that device).

Robots as social learnersThe ability to learn new tasks and skills

quickly and effectively from real-time in-teraction with everyday people is criticalfor socially situated robots. I believe thisentails understanding how people naturallywant to teach robots and then using theseconstraints to design learning algorithmsand architectures to support the teaching-learning interaction.

We’ve carried out human subject studiesto explicitly study how people teach roboticagents. Our results empirically support thepopular assumption that people want robotsto engage in social forms of learning (such asby demonstration, imitation, verbal instruc-tion, or animal training). In particular, peoplewant to guide exploration, motivate thelearner, and adapt their instruction to beappropriate for the learner.

Given our findings, we’re developing alearning framework called socially guidedmachine learning. In SG-ML, the robot isintrinsically motivated to learn and exploreon its own and is socially motivated to lever-age its interactions with people to learn newskills, goals, and tasks. Importantly, we viewthe teaching-learning interaction as a collab-oration in which the human teacher guidesthe robot’s exploration process and the robotcommunicates its internal state (such as whatit understands and where it’s confused) backto the teacher to help guide teaching. Bymaking its learning process transparent tothe human, the robot actively improves itsown learning environment by helping thehuman better tune their instruction for therobot. We’ve demonstrated this bidirectionalprocess whereby the quality of the human’sguidance improves and the robot becomes amore efficient and effective learner.

Furthermore, we’ve found that the robot’steamwork and perspective-taking skills alsoplay an important role in making teaching-learning interactions more robust to the mis-communications or misunderstandings thatinevitably arise even in human-human tute-lage. As the robot observes the human’sdemonstrations, it internally simulates “whatmight I be trying to achieve were I perform-ing these demonstrations in the human’scontext?” The robot therefore interprets andhypothesizes the intended concept being

taught not only from its own perspectivebut also from the human teacher’s visualperspective. Through this process, therobot successfully identifies ambiguous orconfusing demonstrations given by thehuman instructor and clarifies the human’sintent. After disambiguating these prob-lematic demonstrations, the robot correctlylearns the intended task.

My goalUnlike AI’s original goal—to create a

technological system with human-equiva-lent intelligence—my goal is to createrobots that are synergistic and compatiblewith humans. Specifically, they bring valueto us and are valued by us because they’redifferent from us in ways that enhance andcomplement our strengths and abilities.The goal of creating robots that can engage

us as full-fledged partners is as challengingand deep as AI’s original goal because itrequires scientists of the natural and artifi-cial to deeply understand human intelli-gence, behavior, and nature across multipledimensions (that is, cognitive, affective,physical, and social). It also requires scien-tists to understand the dynamics of the hu-man-with-robot system. This fascinatingarea of technological and multidisciplinaryscientific inquiry has the potential to transi-tion the robots of today into our sci-fi robotsidekicks of tomorrow.

References

1. C. Breazeal, Designing Sociable Robots, MITPress, 2002.

2. T. Fong, I. Nourbakshsh, and K. Dautenhahn,“A Survey of Socially Interactive Robots,”Robotics and Autonomous Systems, vol. 42,

nos. 3–4, 2003, pp. 143–166.

3. R.W. Picard, Affective Computing, MIT Press,1997.

4. S. Schaal, “Is Imitation Learning the Routeto Humanoid Robots?” Trends in CognitiveScience, vol. 3, no. 6, 1999, pp. 233–242.

5. L.W. Barsalou et al., “Social Embodiment,”The Psychology of Learning and Motivation,B. Ross, ed., Academic Press, vol. 43, 2003,pp. 43–92.

6. A. Meltzoff and J. Decety, “What ImitationTells Us about Social Cognition: A Rap-prochement between Developmental Psy-chology and Cognitive Neuroscience,” Neu-roscience of Social Interactions: Decoding,Influencing, and Imitating the Actions of Oth-ers, D. Wolpert and C.D. Frith, eds., OxfordUniv. Press, 2003, pp. 491–500.

Socially Assistive RoboticsMaja J. Mataric, University of SouthernCalifornia

Five decades into its development, ro-botics is finally on the brink of enteringour daily lives. Having transitioned fromsuccess in factory automation (auto assem-bly) and high-precision laboratory work(aiding DNA sequencing and medicationdelivery), it is now poised to move intomuch messier, less structured, and moreuncertain human-inhabited environments.While simple robot vacuum cleaners arealready at work in over a million homes,robotics researchers are eagerly develop-ing much more complex and intelligentrobots that will work with and for peoplein hospitals, elder care centers, schools,and homes.

What happens when intelligent robotsand people share an environment and evengoals? This question is at the heart ofhuman-robot interaction, an inherentlyinterdisciplinary research endeavor. HRIbrings together engineering and social sci-ence, cognitive science, neuroscience,ethics, and the disciplines directly relatedto the application domains the robot tech-nology would operate in, such as health-care and education. HRI is as much a studyof people as it is of robots; it requires us togain insight into a new and dynamicallychanging relationship between humans andintelligent machines.

Within the larger context of HRI, the

JULY/AUGUST 2006 www.computer.org/intelligent 81

This fascinating area of

technological and

multidisciplinary scientific inquiry

has the potential to transition the

robots of today into our sci-fi

robot sidekicks of tomorrow.

research in my Interaction Lab focuses onsocially assistive robotics, which aims toaid people by means of social interactionrather than physical contact. SAR bringstogether HRI and assistive robotics, whichhas demonstrated successes in hands-onrehabilitation through the use of high-pre-cision but low-autonomy and low-interac-tivity systems. Consequently, SAR presentsan entirely new set of research challenges,as we strive to understand how robots caninteract with people to effectively and mea-surably help in the difficult processes ofconvalescence, rehabilitation, socialization,training, and education.

Using physical robots for social assistancemight seem counterintuitive. After all, whynot use software agents or devices such asPDAs endowed with intelligent interfaces?The answer rests on the fundamentallyhuman tendency to socially engage withand attribute lifelike properties to machines,especially embodied ones such as robots,which exhibit sufficient (often quite simple)attributes of biological-like movement orappearance. Researchers have shown thehuman projection of intentions, goals, andemotions to such embodied devices to beinescapable, culture independent (althoughfeaturing culture-specific responses), unin-hibited by sophistication of knowledge andfamiliarity with the details of the technol-ogy, and resistant to change over time orwith repeated encounters. This means thatpeople cannot help but respond to engagingphysical machines.

Our research builds on this human char-acteristic to develop a novel robot-assistedresearch paradigm with two distinct butinterrelated goals:

• to study human social and performance-related behavior to better understandthose phenomena and their underlyingmechanisms and

• to elicit sustained productive goal-drivenbehavior as part of the diagnostic, thera-peutic, or educational process.

We’re motivated by the vision of using assis-tive technology as a tool for both scientificdiscovery and societally relevant applicationof assistive robots.

From the basic-science perspective,SAR presents a unique new tool for scien-tific inquiry. Machines have long beenused in research, but usually as passive (ifsophisticated) tools. Through socially

aware robots, we can introduce a funda-mentally new type of machine into theresearch realm: one that can detect, moni-tor, and respond to user behavior in aphysically shared context. As tools of sci-entific inquiry, robots can have both theirform and behavior manipulated in a fullycontrollable manner. We can carefullydesign the information that we share withparticipants about the robot to attempt tocontrol their beliefs and biases. Using thatexperimental paradigm, we can measureparticipants’ responses, both voluntaryand involuntary, ranging from observablebehavior (such as gaze, posture, move-ment patterns, and linguistic interactions)to physiological responses (such as heartrate, body temperature, and galvanic skinresponse). We can also combine robotexperiments with functional brain imag-

ing, aiming toward structural insightsabout relevant social and learning mecha-nisms. As a result, we can ask scientificquestions that we could not previously askor experimentally answer. By perfectlytuning how lifelike, humanlike, believ-able, socially engaging, and intelligentrobots are, we can probe into human so-cial behavior and use the insights to de-sign social interaction toward measurablyimproved therapeutic, educational, andtraining outcomes.

The robot’s physical embodiment is acritical means of eliciting the richness ofthe user’s response, both in the context ofscientific discovery and in real-world ap-plications. Properties and implications ofembodiment present rich avenues for re-search into both human nature and assistivehuman-robot interaction. Such propertiesinclude

• how the robot looks (Is humanlike oranimal-like appearance important anddesirable? Should the robot match the userin approximate size and, if humanoid,apparent age?),

• how the robot behaves (How expressiveshould the robot be relative to its user?What means of expression are mosteffective? How far do physical embodi-ment and embodied cues go in convey-ing information, and what is the appro-priate balance between nonlinguistic andlinguistic interaction? Should the robothave a detectable personality? If so, whatshould it be?), and

• how the robot relates to its niche (Howdoes the hospital environment differfrom school or home? How should therobot handle different types of users inan environment, such as doctors, nurses,therapists, patients, and family at thehospital?).

Situated at the intersection of scienceand technology, SAR research aims to helpsolve major societal challenges, focusingspecifically on the growing aging popula-tion, rising costs of medical care and edu-cation, and shortages of social programsfor education and training of persons withdisabilities. To aid people in need, robotswill have to perform truly challenging tasksin both structured environments (hospitals,elder care facilities, and schools) andunstructured settings (homes, cities, road-ways, and rubble piles).

An outstanding challenge for SAR is therobots’ need to be both useful and engag-ing. The most popular human nurses arenot known to be the most effective; it ishard indeed to compel a person to do some-thing painful, difficult, or dull in a sustainedmanner, such as exercising six hours dailypost-stroke, breathing into a spirometer 10times an hour after cardiac surgery, or per-forming months of vocabulary and pronun-ciation drills. The socially assistive robotmust remain appealing as well as effectiveover a long period, whether months instroke rehabilitation, years in special educa-tion, or, potentially, a lifetime. This presentsa novel, compelling challenge for both ro-botics and AI. Effective human teachers,coaches, nurses, and therapists are all too rareyet too important to do without. Those short-ages serve as motivators for SAR research,whose goal is not to replace humans but tofill in large gaps where attentive and individ-

82 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Through socially aware robots, we

can introduce a fundamentally

new type of machine: one that can

detect, monitor, and respond to

user behavior in a physically

shared context.

JULY/AUGUST 2006 www.computer.org/intelligent 83

ualized human care is diminishing or entirelyunavailable.

Embarking on assistive robotics involvesbeing not only curiosity-driven but alsoproblem-driven, and facing complex scien-tific and technological challenges in messy,noisy, and sensitive real-world domains thatinvolve interactions with vulnerable users.We have placed simple robots with strokepatients (see figure 5), cardiac patients, andchildren (see figure 6) in special educationclassrooms. Our research has demonstratedpositive and promising results,1–5 as well aspointed toward a plethora of fascinatingresearch questions. SAR’s rich potential forgaining novel insights into human cognitionand social behavior, and for improvinghuman quality of life for populations thatmost need it, represents one of the mostexciting and uniquely compelling topics inmodern robotics.

References1. J. Eriksson, M.J. Mataric, and C. Winstein,

“Hands-Off Assistive Robotics for Post-Stroke Arm Rehabilitation,” Proc. IEEE Int’lConf. Rehabilitation Robotics (ICORR 05),IEEE Press, 2005, pp. 21–24.

2. D. Feil-Seifer and M.J. Mataric, “SociallyAssistive Robotics,” Proc. IEEE Int’l Conf.Rehabilitation Robotics (ICORR 05), IEEEPress, 2005, pp. 465–468.

3. R. Gockley and M.J. Mataric, “EncouragingPhysical Therapy Compliance with a Hands-Off Mobile Robot,” Proc. 1st Ann. Conf.Human-Robot Interaction, (HRI 06), ACMPress, 2006 pp. 150–155.

4. K.I. Kang et al., “A Hands-Off Physical Ther-

apy Assistance Robot for Cardiac Patients,”Proc. IEEE Int’l Conf. Rehabilitation Robot-ics (ICORR 05), IEEE Press, 2005, pp.337–340.

5. A. Tapus and M.J. Mataric, “Introvert vs.Extrovert Hands-Off Robot for Post-StrokeRehabilitation Therapy,” to be published inProc. 10th Int’l Symp. Experimental Robot-ics (ISER 06), 2006.

Do Androids Dream of theSweet Smell of Roses?Hiroshi Ishida, Tokyo University of Agri-culture and Technology

Robotics and intelligent systems re-search have historically aimed to make

machines that can replace human work-ers, that can do jobs that humans can’tdo, or both. Industrial robots, for exam-ple, are designed to more quickly andaccurately accomplish tasks that humanworkers formerly did. In this context,robots must do their jobs autonomouslywith minimal interaction with and super-vision from human users. However,human-machine interaction has becomesignificantly more important as the appli-cation areas of robots and intelligent sys-tems have expanded. Home care robotsfor the elderly, for example, must be ableto understand users’ demands and to tellthem what actions the robots are about to take.

This is one reason that robotics research-ers are eager to implement olfactorysensing capabilities in robotic and intelli-gent systems. It’s desirable for robotsworking with humans to share sensoryinformation with human users. If the robotcan see the world and hear sound exactly asthe human user does, it can perceive thesituation similarly, and working coopera-tively with the human will become mucheasier. However, this won’t be enough.Although most people don’t appreciate thevalue of olfaction in our daily lives, ournoses play essential roles in various scenar-ios, such as enjoying meals and detectingdanger. A robot cook that can’t discrimi-nate old milk with a nasty smell, a securityrobot that can’t detect the smell of burning,or a nanny android that can’t smell that it’stime to change diapers all lack an impor-tant functionality.

Figure 5. A stroke patient interacting with a socially assistive robot from Maja Mataric’s lab that is monitoring, coaching, and motivating prescribed exercise therapy.

Figure 6. A child-sized humanoid robot,which Maja J. Mataric and her colleaguesdeveloped, interacts with a child. Therobot works from a desktop or on amobile platform and is an example of theresearch toward robots that can help inthe education and socialization ofchildren with special needs.

Taking inspiration fromhumans or other animals

For robots that work closely with humans,the ultimate goal would be an olfactorycapability that reproduces human olfac-tion. We sense odors when chemical com-pounds stimulate the olfactory receptorcells at the nasal cavity’s roof.1 What makesolfaction a distinctive sensing modality isthe diversity in the stimuli. Several hun-dreds of thousands of different chemicalsubstances are known to elicit odor sensa-tion. Moreover, what we perceive as a sin-gle smell is often the result of a mixture ofa number of chemical substances. Thevapor emanating from a cup of coffeeincludes more than 400 chemicals thathave odors.

Such complexity and diversity is theresult of the olfactory system’s huge num-ber of sensors. The human nose has 10million olfactory receptor cells. Genetic-engineering advances have revealed thatthose cells are classified by their receptorproteins into approximately 400 types.The levels of interactions between odorantmolecules and the receptor proteins deter-mine the cells’ response levels. Each classof cells responds to a wide spectrum ofchemical substances, but in a slightly dif-ferent way from the cells of the otherclasses. So, exposing the cells to a singlepuff of a smell produces a response pat-tern unique to that smell. We discriminateodors by recognizing the cells’ responsepatterns.

Human olfaction isn’t the only model forbuilding artificial olfactory sensing systems.Many animals have keener olfaction thanhumans. We might use such “supersenses”as a model to enable our robots to do thingsthat humans can’t do. Dogs are famous fortheir ability to track trace scents. It’s saidthat trained dogs outperform metal detectorsin landmine detection. Dog noses are bothmore sensitive and more selective thanhuman noses. However, the basic olfactionmechanism—that is, recognition of thereceptor cells’ response patterns—seems tobe common not only to all mammals butalso to a wide variety of animal species.The main difference between human anddog noses is that a dog nose has more olfac-tory receptor cells with a greater variety ofreceptor proteins.

Another famous example of super olfac-tion is male moths that track airborne sex-ual pheromones released by conspecific

females over long distances. To achievesuch feats, the olfactory sensors must beable not only to detect extremely dilutedtarget odors but also to discriminate targetodors from any background odors. Mothshave solved this problem by developing,through evolution, receptor cells more sensi-tive than humans’. A male moth’s pheromonereceptor cell responds even to a single mole-cule of the sexual pheromone, but shows littleresponse to any other chemical substances.Despite their efforts, researchers don’t yetfully understand how the pheromone receptorcell differs from the mammalian olfactoryreceptor cell.

Electronic noses and theirapplications in robotics

The first research on an artificial sensingsystem able to discriminate different odorsusing mammalian olfaction as a model was

published in 1982.2 Since then, researchershave done extensive research on developingelectronic noses. Typically, an electronic noseconsists of an array of chemical sensors withpartial specificity and a pattern-recognitionsystem just like its model’s. Unfortunately,no engineered replica of olfactory receptorcells exists. The most popular sensor inelectronic noses is a metal-oxide semicon-ductor gas sensor. The sensing element isgenerally made of sintered tin dioxide par-ticles, which show variation in conductivityon exposure to reducing gases. Althoughoriginally made for flammable gas leakdetectors, the sensor responds to a widerange of volatile organic compounds. Thesensitivity to different gases can be tunedto some extent in several ways, such aschanging the operating temperature anddoping the sensor with small amounts ofcatalytic materials. Sensors that use organicmaterials are expected to be more compati-

ble with human olfactory receptor cells.Adsorption of gas molecules onto organicmaterials is detectable—for example, bymeasuring the mass increase or swelling ofthe polymer matrix. Typically, an electronic-nose system has only eight to 32 sensorsbecause fabricating numerous sensors withdifferent selectivities into a single array isdifficult.

Despite current sensor technologies’limitations, electronic noses have beentested in a variety of contexts. A typicalelectronic nose setup stores sensor responsepatterns for a repertoire of odors into thecomputer memory in the learning phase.The electronic nose recognizes odors byfinding the best match between the mea-sured and the stored response patterns. Forexample, it can estimate the freshness of apiece of fish by comparing its smell withthose of meat samples with different levelsof rottenness. Researchers have success-fully demonstrated electronic noses’ odor-discrimination capabilities in applicationsranging from quality control of cosmeticproducts to food process monitoring andmedical diagnosis.1 Some electronic-noseinstruments are commercially available.

For these applications, electronic nosesare designed to have odor discriminationcapabilities similar to human olfaction.However, artificial olfaction can have dif-ferent selectivities to odors by using differ-ent sensors, just as in animals’ supersenses.To date, supersensitive gas sensors havebeen developed without a specific animalas a model because the mechanisms under-lying animal olfaction are still open ques-tions. One key to realizing highly sensitivesensors is a built-in signal-amplificationmechanism. For example, Nomadics’ land-mine detector has sensitivity comparableto dog noses in detecting vapor emanatingfrom explosives.1 The key element is afluorescent conjugated polymer film thathas a high affinity for trinitrotoluene.Adsorption of a single TNT moleculequenches the whole polymer chain’s fluo-rescence, causing significant variation inthe sensor output.

In robotic applications, detecting and dis-criminating a specific odor might be just apart of the whole task. If a security robot withartificial olfaction detects a flammable gasleaking in a factory, an appropriate actionwould be to find the leak by tracking the gasplume to its source. This task isn’t as easy asit seems. Because molecular diffusion is slow,

84 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

A security robot that can’t detect

the smell of burning or a nanny

android that can’t smell that it’s

time to change diapers lacks an

important functionality.

even a slight airflow greatly affects gas dis-persal. The flows we encounter in the realworld are almost always turbulent. Becausethe gas plume becomes patchy and mean-dering, tracking the gas-concentration gra-dient often fails to locate the gas source. Tohelp solve this problem, researchers havetried to develop mobile robots that mimicmoths’ instinctive behavior.3 Male mothspredictably steer upwind when they come incontact with a plume of sexual pheromone.The basic feature of plume-tracking behav-ior was implemented into a robot equippedwith gas sensors and airflow sensors. Suc-cessful demonstrations have been providedon tracking gas plumes formed in indoorenvironments.4

OutlookThe major problem in the development

of artificial olfaction is that no sensor asversatile as odor receptor cells exists. Eachsensor has certain advantages and disad-vantages, so users must select a set of sen-sors appropriate for their applications.Because the methodology for systematicsensor selection hasn’t yet been established,selection often involves much trial anderror. Moreover, all gas sensors show driftin their response over time, and frequentrecalibration is laborious. Human olfactioninvolves both sensors of many differenttypes and many sensors of the same type.The former makes our noses versatile; thelatter appears to help alleviate drift. Olfac-tory receptor cells regenerate in four toeight weeks. Although at least some sen-sors are always being replaced, we can stillrecognize the same odors as before thesensor replacement.

Recently, research has been initiated toexploit olfactory receptor proteins them-selves as the ultimate sensing materials.5

Challenges include how to keep receptoractivity on a sensor chip, how to preparereceptor proteins for mass production, andhow to measure with high sensitivity theadsorption of odor molecules onto thereceptors. Once these problems are solved,we’ll obtain a large family of artificial sen-sors with selectivities closely matched tohuman olfaction. If we could somehow addthe self-generating function to those sensors,we should be able to equip our androidswith versatile, drift-free artificial olfaction.

On the software side of artificial olfac-tion, an interesting issue is how to dealwith unknown odors. So far, electronic

noses merely report which smell in theirlearned repertoire elicits the sensor responsepattern that best matches the one being mea-sured, even if the repertoire doesn’t includethe measured odor and the match is far fromperfect. Some electronic noses can recog-nize an unlearned odor as “unlearned,” butthis might not be enough. A robot thatworks closely with humans should be ableto tell human users what smell it’s detect-ing and, if the smell is unfamiliar, how itsmells. However, when you try to explainan unfamiliar smell to someone else, yourealize the difficulty of describing smells.Our language system seems to be built todescribe what we see. Logically describinga perceived smell’s features isn’t alwayspossible. This is partly because odor per-ception involves not only conscious odorrecognition but also subconscious evoca-tion of memories and emotions. We don’tknow yet what features in the responsepattern of our olfactory receptor cells makeus feel sweet when we smell roses. Someresearchers are attempting to make robotsexplain the smells they detect in terms ofabstracted descriptions, such as herbal,oily, or fragrant.6 In theory, if we succeedin providing an array of sensors compati-ble with human noses, we should be ableto let our robots appreciate the sweet smellof roses.

References

1. T.C. Pearce et al., eds. Handbook of MachineOlfaction: Electronic Nose Technology, JohnWiley & Sons, 2003.

2. K. Persaud and G. Dodd, “Analysis of Dis-crimination Mechanisms in the MammalianOlfactory System Using a Model Nose,”Nature, vol. 299, no. 5881, 1982, pp. 352–355.

3. R.A. Russell, Odour Detection by MobileRobots, World Scientific, 1999.

4. H. Ishida et al., “Controlling a Gas/OdorPlume-Tracking Robot Based on TransientResponses of Gas Sensors,” IEEE Sensors J.,vol. 5, no. 3, 2005, pp. 537–545.

5. G. Gomila et al., “Artificial Nose IntegratingBiological Olfactory Receptors and NEMS,”Proc. 11th Int’l Symp. Olfaction and Elec-tronic Nose, 2005, pp. 14–18.

6. A. Loutfi and S. Coradeschi, “Forming OdourCategories Using an Electronic Nose,” Proc.European Conf. Artificial Intelligence (ECAI04), 2004, IOS Press, pp. 119–124.

JULY/AUGUST 2006 www.computer.org/intelligent 85

Hiroshi Ishiguro isa professor at OsakaUniversity. Contacthim at ishiguro@ams.eng.osaka-u.ac.jp.

Minoru Asada is aresearch director atthe JST ERATO AsadaSynergistic Intelli-gence Project and aprofessor at OsakaUniversity. Contacthim at asada@ams.eng.osaka-u.ac.jp.

Stuart C. Shapirois a professor in theDepartment of Com-puter Science andEngineering anddirector of the Centerfor Cognitive Scienceat the State Universityof New York at Buf-

falo. Contact him at shapiro@cse.buffalo.edu.

Michael Thielscheris a professor atDresden Universityof Technology. Con-tact him at mit@inf.tu-dresden.de.

Cynthia Breazeal isan associate profes-sor at the MIT MediaLab. Contact her atcynthiab@media.mit.edu.

Maja J. Mataric is aprofessor of computerscience and neuro-science, founding di-rector of the Centerfor Robotics andEmbedded Systems,and senior associatedean for research in

the Viterbi School of Engineering at the Uni-versity of Southern California. Contact her atmataric@usc.edu; http://robotics.usc.edu/~maja.

Hiroshi Ishida is anassociate professor atthe Tokyo Universityof Agriculture andTechnology. Contacthim at h_ishida@cc.tuat.ac.jp.