Upload
lamthien
View
221
Download
2
Embed Size (px)
Citation preview
Agents and Avatars
Ruth Aylett
Overview
Agents and Avatars Believability v naturalism Building the body H-anim Moving and sensing
IVAS
Intelligent Virtual Agents (IVAs)– Also:
• Synthetic characters• Embodied conversational characters (ECAs)• Virtual humans• BUT do not have to be humanoid…
– Embodied and autonomous• Require a control architecture - or ‘agent mind’• Insides v outsides: combining AI with graphics
– ‘Inhabit’ a virtual environment
Why IVAs?
Adding life to a VE– Animals, birds, insects– Crowds
• E.g. students in the virtual campus– Increase sense of presence
As a guide or teacher– Front-end to embedded knowledge in a VE
As a character in a story– Computer games
Interface agents– On web pages to make them more human– Personal representative– Sales rep
Why IVAs? - 2
As virtual actors– Instead of extras– Immersive Education Media Stage
As part of a simulation– Hostage release training– Battlefield medical training
Scientific investigation– Buildings evacuation– Testing for disability friendliness– Ecology and animal behaviour
Avatars
Hindu Sanskrit Term - “Representation of aDeity in visible form”– Snow Crash (Neil Stephenson 1992) - “A
graphical representation of yourself”
VE representation of the user– Embodiment need not be humanoid
Driven by the user– So NOT autonomous– A mapping rather than a control problem
Video avatars
“magic video mirror”with back-projection
video image mixedwith computergeneratedoverlay
Creating a video avatar (ALIVE, MIT)
•unencumbered interactionbetween human visitor andvirtual character based onposition, postures, and gestures
•main focus on “virtual presence”
http://www.ai.univie.ac.at/oefai/agents/
Believability
Term introduced by Joe Bates of the OZgroup at CMU in the 1990s– Combined art and technology
Very hard to define– A willing suspension of disbelief?– Seem like ‘real’ characters?– The ‘illusion of life’?– Willingness to attribute an internal state
• Ascribe intentionality
Believability and Naturalism
Are they the same thing? Graphics people seem to think so
– Is Mickey a real mouse?– Is he a believable character?
The uncanny valley
The ‘uncanny valley’– Work by a Japanese
researcher, Mori– Acceptability v
naturalism
Goes -ve asbecomes ‘nearlyhuman’
SEE: http://en.wikipedia.org/wiki/Uncanny_Valley
The problem of expectations
Humans have hard-wired expectations– Used to interpret inter-personal behaviour– Fundamental social skill
Need to invoke this very carefully– Acceptability of movement
• Lip sync a key problem
– Interactive responsiveness• Memory of interaction a key issue• Games create a real problem with instant rewind
Building a body
Use of 3D modelling package– 3ds Character Studio; Poser etc
Creation of skeleton– Required for animation later
Polygonal body– But overall count must be low for real-time
interaction, low 000s or less if possible
Texture to cover body– One body, many textures?
TextureImages
X 4
SilhouetteImages
X 4
Stage 1: Image Capture
•Generic Avatar
•Polygonal Structure
•1500 Polygons
Change Data:Deforms the Mesh
Stage 2: Deform the Mesh
Apply Texturesto Deformed
Mesh
Stage 3: Apply the Textures
The touch up process
What sort of motion?
Walking around– Jumping, running, swimming– Human and non-human
Picking things up– Agent-agent interaction
Gesture Talking heads
– Facial muscles, lip synch
Group Movement– Crowds, flocks, herds
Moving the body
Animation– Drawing on cartoon animation
Motion capture– Drawing on gait analysis and then film
PBM– Drawing on robotics
• Analytically calculated• Learned
Animation
Time consuming for good results– Requires artistic skills
The main character, Woody, in ToyStory had:– 700 degrees of freedom (200 for face and
fifty for the mouth)– 150 people (at Pixar) would generate 3
minutes of animation a week Can it be standardised?
Self-animation
Poses new problems:– Parametrisable animation very desirable
• E.g a walk that could be reused with differentstride length or foot height
– Melding and combining of animations ‘onthe fly’• Not just a morphing problem• ‘starting position’?
– Extra actions inserted by character
Standardising animation?
Animation depends on the structure ofskeleton that is being animated
Can we standardise a humanoidskeleton?– H-anim just such an attempt– Originally in the context of VRML– See www.h-anim.org
H-anim
A set of standard components– Humanoid: root of a figure– Joint: attached using transform specifying
current state of articulation plus geometryassociated with attached body part
– Segment: specifies attributes of physical linksbetween joints
– Site: where can add semantics– Displacer:range of movement allowed for object
in which embedded
H-anim
And in X3d..
<HAnimJoint DEF='hanim_l_hip' center='0.0961 0.9124 -0.0001'name='l_hip'>
<HAnimSegment DEF='hanim_l_thigh' name='l_thigh'/> <HAnimJoint DEF='hanim_l_knee' center='0.1040 0.4867 0.0308' name='l_knee'> <HAnimSegment DEF='hanim_l_calf' name='l_calf'/> <HAnimJoint DEF='hanim_l_ankle' center='0.1101 0.0656 - 0.0736' name='l_ankle'> <HAnimSegment DEF='hanim_l_hindfoot’ name='l_hindfoot'/> </HAnimJoint> </HAnimJoint></HAnimJoint>
Motion capture
Electro-magnetic or by camera Produces the most ‘natural’ results
– Extensively used in film for graphical extras– Can be used on avatars very successfully to transmit user
movement to their graphical representation
Even more problems for self-animation– Harder to parametrise– Worse combining/melding problems
Mocap process
Calibration Capture 3D Position Reconstruction Fitting to the Skeleton Post Processing
Calibration
Triangulate camera position and set origin
Wand– Calibrate and triangulate
cameras and capturevolume
L-frame– Define ground plane, up
vector and origin
θ1
θ2
Calibration
3D Position Reconstruction(Utopia)
3D Position Reconstruction(Reality)
Multiple Hypothesis Tracking
For small number of markers: Size Occlusions are a problem Multiple Hypothesis Tracking
Ringer, et al., 2002
C1
C2
Fitting to the Skeleton
Utopian approach– 10 – 20% length
changes
Markers on bothsides
Joint Displacement Use Rotation Angles
Only
Post Processing
Motion Editing– Cut, Copy, Paste
Motion Warping– Speed up or Slow Down– Rotate, Scale or Translate
Motion Signal Processing– Smoother Motions
Physically-based modelling
Extending robot motor control Forward kinematics
– If you move joints so much– Where does the end effector go?
Inverse kinematics– If you want the end effector at x,y,z…– How much should which joints move?
Computationally demanding and often not verynaturalistic– But the most flexible option– Animation blending often added…
Ragdoll models
Multiple rigid body simulation– Each tied to bone in skeleton– Constraints for joint movement
Extended into procedural animation ofwhole body– Natural Motion: Euphoria
Fish example - spring-massmodel
Learning to move
Use of AI learning algorithms– Move– Evaluate and score– Keep movements which work well
Karl Sims ‘Blockies’– Early 1990s
Terzoplolous - Fish Learning
http://www.csri.utoronto.ca/~dt/
Virtual sensors
Local v global interaction– Global - read from world data-structures– Local - ‘sense’ the environment
Global approach very common– In most computer games– Efficient, easy to test
Advantages of local sensing
Scales well– Not affected by global size of environment
Agent has independence of environment– Up to a point
Makes for believability– Agent perceives what it should– Can’t see you round corners– Emergent complexity
TeleTubby Sensors
Forward ray-tracing sensor isseven meters and sweeps 45o,five times/sec
One vector sensor directed vertically downwards. Intersectionwith the ground is continually beingdetected
All these sensors are attached tothe geometry of the agent
Sensing by message passing
Requires an architecture that distributesevents as messages– Concept of locale
• What is local for local sensing?
– Semantically determined: e.g a room
Scenegraphs not set up for this style ofmessage passing– Think of routing in VRML for example
Credits
Edward Tse, University of Calgary