View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Artificial Emotion
Characters that display emotion are critical to a rich and believable simulated environment.Emotion is the essential ingredient that creates the difference between
Robotic behaviorLifelike engaging behavior
Traditionally, animators painstakingly created these behaviors for pre-rendered animationsBut, truly interactive characters must generate their behavior autonomously through techniques based upon artificial emotions
Must possess their own personalities and moods
Emotion AI Engine
Ian Wilsonwww.artificial-emotion.com
Engine simulates personality, age, gender and the low level emotional behavior that drives most of our actions.
From this simulation gestures and low level actions are generated. Behaviors can be mapped to application specific elements, i.e. characters, robots, cell phone agents, etc
Behavior of real users can be simulated to predict what type of action that user might take preferenceshow a unique individuals mood changes while using your producthow to adjust that behavior to ideally reward each individual
Engine Features» Simulates millions of unique personalitites » Simulates ages from 5 to 105 » Simulates gender from very femenine to very masculine » Simulates core brain systems responsible for emotion level processing » Generates facial gestures using muscle simulation, MPEG4 FAPs and FACS action units » Generates eye saccade, eye movement speed/range/frequency control, blink » Generates head position movement, pitch and yaw » Generates upper body gestures (spine, shoulders, neck) with walk cycle due to follow soon » Generates low level actions such as movement speed / range / frequency, search patterns, approach, avoid » Input can be as simple as a single integer to drive the whole system » Output is a continuous stream of integer values
» Output is not tied to a specific application for maximum flexibility
Facial Action Coding System (FACS)
Ekman, P., Friesen, W.V., & Hager, J.C. (2002). An anatomically oriented coding system, based on the definition of "action units" (AUs) of a face that cause facial movements. Each AU may correspond to several muscles that together generate a certain facial action. As some muscles give rise to more than one action unit, correspondence between action units and muscle units is only approximate. 46 AUs were considered responsible for expression control and 12 for gaze direction and orientation. The FACS model has been used to synthesize images of facial expressions; exploration of its use in analysis problems has been a topic of continuous research.
MPEG-4
MPEG-4 is the latest compression standard developed by the Moving Picture Experts Group (MPEG) of the ISO, the same group that brought us MPEG-1 and MPEG-2. MPEG-4 builds on the proven success of three fields:
Digital Television Interactive graphics applications (synthetic content); Interactive multimedia (World Wide Web, distribution of and access to content)
MPEG-4 Facial Animation and Definition Paramters
Inspired by FACSFacial Definition Parameter set (FDP) and the Facial Animation Parameter set (FAP) were designed to allow the definition of a facial shape and texture, as well as the animation of faces reproducing expressions, emotions and speech pronunciation.
Facial Animation Parameters
The FAPs are based on the study of minimal facial actions and are closely related to muscle actions. They represent a complete set of basic facial actions, such as squeeze or raise eyebrows, open or close eyelids, and therefore allow the representation of most natural facial expressions. All FAPs involving translational movement are expressed in terms of the Facial Animation Parameter Units (FAPU). FAPUs aim at allowing interpretation of FAPs on any facial model in a consistent way, producing reasonable results in terms of expression and speech pronunciation.
Facial Animation Parameters
JOY a la FAP
open_jaw (F3), lower_t_midlip (F4), raise_b_midlip (F5), stretch_l_cornerlip (F6),stretch_r_cornerlip (F7), raise_l_cornerlip (F12), raise_r_cornerlip (F13), close_t_l_eyelid (F19),close_t_r_eyelid (F20) , close_b_l_eyelid (F21), close_b_r_eyelid (F22), raise_l_m_eyebrow (F33),raise_r_m_eyebrow (F34), lift_l_cheek (F41), lift_r_cheek (F42), stretch_l_cornerlip_o (F53),stretch_r_cornerlip_o (F54)
Facial Definition Parameters
FDPs on the other hand are used to customise a given face model to a particular face. The FDP set contains a 3D mesh (with texture coordinates if texture is used), 3D feature points, and optionally texture and other characteristics such as hair, glasses, age, gender.
Facial Definition Parameters
FAP Engines
IBM Java Toolkit for MPEG-4 (currently active)http://www.alphaworks.ibm.com/tech/tk4mpeg4
Facial Animation Engine (FAE) (University of Genova, Digital and Signal Processing Lab.)
http://www.dsp.dist.unige.it/index.phpMiraface: MPEG-4 FAP Player (MIRALab, University of Geneva)
http://www.accelerationstudios.com/site/contents.acc?doc=9http://miralabwww.unige.ch/
XFace (Cognitive and Communicative Technologies, Instituto Trentino di Cultura)
Open source project, currently active.http://tcc.itc.it/http://xface.itc.it/
Visage Technologies (currently active)Free academic license for most of their products.C++ Software Development Kit “visage|SDK”http://www.visagetechnologies.com/index.html
AE Approach
Emotions comprise three layers of behaviorTop level: momentary emotions
Behaviors that we display briefly in reaction to events
Next level: moodsProlonged emotional states caused by the cumulative effect of momentary emotions
Underlying level: personalityBehavior that we generally display when no momentary emotion or mood overrides
Levels have an order of priorityMomentary emotions over moodMood over personality
Layer Prominence
Momentary emotions are brief reactions to events that assume the highest priority when we select out behavior
Behaviors are short lived and decay quicklyMoods are produced by momentary emotions
Usually by the cumulative affects of a series of momentary emotionsCan gradually increase in prominence
Even after the momentary emotions have subsidedDevelopment depends on whether momentary emotions are positive or negative
If a character were to receive a stream of negative momentary emotions, then the mood would obviously be bad and would decay slowly
Personality level is always present and has a consistent level of prominence
Behavior Selection
The behavior that a character displays depends upon each emotional layer’s prominenceThe more prominent the layer, the higher the probability of that behavior being selected
Uses of AE
Autonomous AE of any depth is rarely seen in commercial interactive entertainment
Some exceptions are P.F. Magic’s Catz and Dogz series, Fujitsu’s fin fin, and Cyberlife’s Creatures series
Uses of AE
IE is currently dominated by genres that require the user to either conquer and/or kill everything in his or her path
Little emotion is required
Emotion primarily serves a social function in IE.Emotional responses are used to make characters believable and engaging
If we walk into a virtual bar and the characters had distinct personalities, the scene would be an immersive and believable simulation
If characters show no emotionOur suspension of disbelief would be immediately brokenWe would be immediately reminded that we were in a computer generated simulation
Uses of AE
Using human characters necessarily implies that their behavior is deep and complexUnfortunately, we are most attuned to recognizing human emotion
And therefore recognizing flawed human emotion.Which could easily break the illusion of an otherwise well constructed simulated environment
One way to attack this problem is to use nonhuman charactersCats, dogs, and Norns all show engaging levels of interactive emotional behavior that maintains the illusion of life
Without having or needing the complexity of human emotional response.
AE Output
AE produces two fundamental components as outputActions
General category dependent on the context of the situation in which the character existsA simulation’s movement system uses AE to select and/or modify an actionAE indicates what actions are appropriate to the character’s personality and current mood
A timid character is unlikely to do anything aggressiveAn outgoing, extroverted character might perform an action enthusiastically
which would probably not be the case for an extreme introvert
AE Output
GesturesHand, body and facialWay to communicate our emotions to the outside worldAE-driven gestures are tied directly to our characters’ personalities and moods and follow definite patterns.
E.g., a sad looking fellow, shoulders hunched over, arms hanging limply and walking slowly as he makes his way through our environmentMight compel a player to ask “Why does he look so sad. What is his story? Should I go and ask him?The kinds of questions that occur to the viewer of a truly interactive experience would be irrelevant without AE.
What is Personality?
The genetic and environmental differences in the brain structures of individuals.As a species, our brains are almost identical
This gives rise to our common sets of behaviors.
But, we are all genetically and environmentally unique.It is the differences that give us our unique behavioral variations of common behavior patterns
i.e., our personality
Personality Model
An area in a 3-D space.The axes of the space are Extroversion, Fear and Agression (EFA Space)Personality traits are represented by points within this space
Positioned according to the amount with which they are correlated with the each axis.
E.g., the trait of anxiety is positioned at (E -30, F +70, A -10)Associated -30% with Extroversion, 70% with Fear, and -10% with Aggression
EFA Space
EFA Space
The position of the center of the personality area (P) represents by how much each of those axes (central traits) define the overall personalityP is the center of a sphere
Contains the set of personality traits that make up the aggregate personalityThe set of traits available to the character
Three Dimensions of Personality
Based on the idea that the brain might have 3 central systems that mediate behavior
Approach SystemAssociated with Extroversion
Behavioral Inhibition SystemAssociated with Fear
Flight/Fight SystemAssociated with Aggression
Three Dimensions of Personality
The behavior that we display at any time is controlled by these 3 systems
And the genetic prominence of each system
E.g., for an anxious personalityBehavioral Inhibition System is very prominentPersonality would have a very high Fear componentPerson generally fearful and cautious
Mood
Determined by perceived signal of punishments and rewards
Inputs for most elements of the systemModulated by the character’s personalityThe position of the personality in EFA space affects the
Range of positive moodsMaximum level of positive moods increases with E
Range of negative moodsMaximum level of negative moods increases with F
Rate of change of moodsSpeed at which moods build up and decay increases with A
Mood Example
Personality with high E, high F and high AMoods would have large negative and large positive valuesMoods would build up and decay rapidlyCharacter would be very moody with large mood swings
Engine Architecture
Nine modulesSix of these represent conceptual neural systems
Emotional reactionsPersonalityPunishment and rewardMoodMemoryMotivations
Two of these are the engine interfaces (API)Input Output
The Self State module is the central data repository for the engine
Represents the characters general emotional state at any time
Engine Architecture
Punishment and Reward
Basic types of inputTakes incoming signals of raw, sensed punishment and reward (p/r) and translates into perceived signals of p/rPerceived p/r depends on
Characters previous history of received p/rPersonality
Punishment and Reward
Use of habituationThe more the character receives, in succession, a signal of one type
The lower effect the signal has
Use of noveltyThe longer the character goes without receiving one type of signal
The greater the effect of that signal when it is received
Punishment and Reward
Character’s personality determines how susceptible it is to punishment or rewardE.g., a psychopath is
Highly susceptible to rewardHighly unsusceptible to punishmentMakes them go after thrills without regard for the consequences
Motiviations
Arranged into a 4-layer hierarchy of needs
Physiological layer is always bottomRelative positions of the remaining 3 layers in the hierarchy are determined by personality
SafetyAffiliationEsteem
E.g., for the psychopath, Esteem is prioritized higher than Affiliation (friendship and kinship)
Memory
Used to facilitate non-cognitive social processing
Used in conjunction with the affiliation and esteem needs to provide enhanced social behaviorKeeps track of how much a particular character is liked based on several factors
Including social preferences of our family and friends
Used by the reactive emotion module
Reactive Emotions
Innate emotional reactions that have not involved any deep cognitive processingRepresent stimulus response reactions of the kind hard wired into our neural circuits
JoySadnessFearAngerSurpriseDisgust
Reactions are modulated by personalityOur psychopath would show very little fear in his reactionsBut may show a great deal of anger
Triggered by p/r signals when they exceed thresholds derived from levels of motivational needs
API Input
All input is in the form of integer streamsp/r signals for each of the physiological and safety needs (8 in total)Input representing signals received by the five senses for emotional reactionsSome needs remain constant if not changed
Assume we are sheltered unless we are informed otherwise
Some needs change unless information is received to the contrary
Hunger will increase unless we receive a reward signal for the hunger need
The engine is updated with input at each developer-determined time step
3 API Output Formats
Positional information required to produce emotional body and facial gestures
For a character whose body joints are being driven by an inverse kenmatics system
Output is the joint deviations from a plain ”vanilla” movement to make the movement emotionalFor a timid character, the spine might be curved backwards, the shoulders hunched forwards, and the head down
Determines characters movement speed, style, and smoothness
A neurotic character would move quickly in short bursts and in a very ”staccato”, jerky fashion
3 API Output Formats
Semantic action planDetermined by the characters motivational needsDetermines both what to do and how to do itHow is taken from current mood and selected personality trait
Character has been out in the cold rain a whileWarmth need receiving a constant stream of punishmentResults in making the mood level highly negative and the warmth need a high priorityFor a character with a timid personality the resulting semantic output would be
Increase warmth very anxiously”increase warmth” comes from motivational need for warmth”very” comes from importance of need and mood level”anxiously” is a trait that this particular character possesses and is appropriate to a negative mood
3 API Output Formats
Raw emotional stateGives the developer the flexibility to use the emotional state in ways not handled by the first two modes
Example Scenario
Jane is a 38-year old mother of two playing an office simulationShe is the boss and she has to manage the well being of her office co-workersIn this game, the tasks that the team has to perform are secondary to their interactions.Different team members respond differently to the same situationShe has to make them all work well together and keep them happy (goal of the game)
Example Scenario
Jane begins by taking a personality test and passes the result to the EngineEngine displays behaviors similar to hersShe then decides the personalities of her 5 coworkers
ExtrovertIntrovertNeuroticPsychopathA character with a personality similar to hers
Example Scenario
Day One: Jane’s alter ego arrives at workHer character is greeted warmly by the extrovertThe introvert sits at his desk, continues typing hardly acknowledging her presenceThe neurotic looks mildly panicked, she does not like change, her shoulders slump forward, she curls up looking submissiveThe psychopath sticks out his chest, shoulders back and fixes Jane’s character with a steely gaze as he marches, quickly and firmly, over to her to make his presence knownThe character similar to Janes’s looks unsure. He is unable to decide if the new boss is good or bad for him. Jane, knowing his personality, can empathize with him so she makes the first move to smile and greet him.