KISMET AN

APPLICATION OFHUMANOID ROBOTICS

1

FLOW OF PRESENTATION

• INTRODUCTION• OBJECTIVE• KISMET-THE ROBOT• OVERVIEW OF HARWARE COMPONENTS• FRAMEWORK• WORKING OF PERCEPTS• IMPLICATION OF NEURAL NETWORK• NEURAL MODEL

2

INTRODUCTION• A humanoid robot is an autonomous robot because

it can adapt to changes in its environment or itself and continue to reach its goal.

• Sharing a similar morphology, they can communicate in a manner that supports the natural communication modalities of humans.

• Examples include facial expression, body posture, gesture, gaze direction, and voice.

3

OBJECTIVE A GENERAL FRAMEWORK THAT INTEGRATES

PERCEPTION ,ATTENTION , DRIVES ,EMOTIONS ,BEHAVIOUR SELECTION AND MOTOR ACTS.

OUTCOME

THE ROBOT RESPONDS WITH EXPRESSIVE DISPALYS WHICH REFLECT AN EVER CHANGING STATE

4

KISMET- THE ROBOT

• • Kismet is an expressive robotic creature with • perceptual and motor modalities that imitates human• natural behaviour .

• Input : Video and Auditory sensors.

• Output : vocalizations, facial expressions, and motor• capabilities to adjust the gaze direction of the eyes • and the orientation of the head 5

KISMET'S EXPRESSIONS

CalmCalm Interest Interest Angry Angry

HappyHappy Sad Sad Surprise Surprise

6

OVERVIEW OF HARDWARE COMPONENTS

• VISION SYSTEM

• AUDITORY SYSTEM

• EXPRESSION MOTOR SYSTEM

• VOCALIZATION SYSTEM

7

FRAMEWORK

•

• Kismet is an autonomous robot designed for social interactions with humans.

• APPROACHAPPROACH

• The way infants learn to communicate with adults.

8

FRAMEWORK VIEW

9

COMPONENTS

• System architecture consists of six subsystems :System architecture consists of six subsystems :

• Low-level feature extraction system.• High-level perception system.• Attention system.• Motivation system.• Behavior system.• Motor system.

10

The Low-Level Feature Extraction System

Responsible for processing the raw sensory information into quantities that have behavioral significance for the robot.

For instance, visual and auditory cues such as detecting eyes and the recognition of vocal affect are important for infants.

11

WORKING OF PERCEPTUAL SYSTEM

MOTION DETECTION PROCESS RECEIVES A DIGITIZED 128 x 128 IMAGE FROM THE CAMERA.

INCOMING IMAGES ARE STORED IN A RING OF 3 BUFFERS. ONE HOLDS THE CURRENT IMAGE Io , ONE BUFFER HOLDS

THE PREVIOUS IMAGE I1 , AND THIRD RECEIVES NEW INPUT. ABSOLUTE VALUE OF DIFFERENCE IS THRESHOLDED TO GIVE

RAW IMAGE AS : I I RAW RAW = T( | Io – I= T( | Io – I11 | ). | ). USES RATIO TEMPLATE ALGORITHM.

12

FACE DETECTION

DETERMINE POTENTIAL FACE LOCATIONS WITH RATIO DETERMINE POTENTIAL FACE LOCATIONS WITH RATIO TEMPLATE ALGORITHMTEMPLATE ALGORITHM

13

The Attention System

The low-level visual percepts are sent to the attention system.

This system picks out low-level perceptual stimuli and direct the robot's attention and gaze toward them.

14

The Perceptual System Here low-level features corresponding to the

target stimuli of the attention system are encapsulated into behaviorally relevant percepts.

Each behavior and emotive response has an associated releaserreleaser.

A releaser can be viewed as a collection of feature detectors that are minimally necessary to identify a particular object or event of behavioral significance.

15

The Motivation System It consists of the robot's basic “drives” and

“emotions”. The “drives” represent the basic “needs” of the

robot and are modeled as simple homeostatic regulation mechanisms .

The “emotions” are modeled from a functional perspective .

Currently, six basic emotions are modeled that give the robot synthetic analogs of anger, disgust, fear, joy, sorrow, and surprise .

16

The Behavior System

Organizes the robot's task-based behaviors into a coherent structure.

Each behavior is viewed as a self-interested, goal-directed entity that competes with other behaviors to establish the current task.

An arbitration mechanism is required to determine which behavior(s) to activate and for how long, given that the robot has several motivations that it must tend to and different behaviors that it can use to achieve them.

The main responsibility of the behavior system is to carry out this arbitration.

17

The Motor System

It arbitrates the robot's motor skills and expressions.

It determines how to move the robot so as to carry out that course of action

Coordinates body posture, gaze direction, vocalizations, and facial expressions

18

IMPLICATION OF NEURAL NETWORKS

• Work more like human brain

• Can handle ambiguity better than rule-based • systems.

• Feature of "learning" - a helpful tool to add more • human-like behaviors.

19

NEURAL NETWORK AS ROBOT BRAIN

•INPUT INPUT : SENSORS : SENSORS•OUTPUTOUTPUT : MOTORS : MOTORS

20

MODEL WORKING OF NEURAL BRAIN

The relationship between the sensors and the motors can be described in the following table (where +1 means on and -1 means off).

Sensor1 sensor2 motor1 motor2 +1 +1 -1 -1 +1 -1 +1 -1 -1 +1 -1 +1 -1 -1 +1 +1 21

CONTD...

SENSORS PRE PROCESSING : Sensor data IS modified to fit

into an input vector. INPUT VECTOR : This is a list of inputs for the

neural network. From a math point of view neural nets are matrixes, thus the inputs are called vectors.

22

CONTD... NEURAL BRAIN : This is where the real knowledge is stored. It

consists of neural cells which are interconnected. Each cell is connected to the other cells. A (mathematical) matrix is used to describe these connections.

OUTPUT VECTOR : This is a list of outputs from the neural network, the result of the multiplication of the input vector with the brain matrix.

ACTION FUNCTION : Usually the neural network is not capable to yield results which are directly usable, so a separate function is required to translate the output vector to a specific reaction.

MOTORS :

23

CHALLENGES This research not only aims at trying to build an

open ended learning system, but also to build a system that humans can interact with and train in a natural, instinctive manner. Humans are highly social creatures and use a variety of cues and modalities to communicate with each other. Building systems that can exploit and understand similar social cues could make machines easier for people to use, and enable humans to communicate with machines in richer ways.

24

THANK YOUTHANK YOU

25