VRlab - KOCCA · 2010. 5. 1. · – Walking, gait – Automatic Grasping – Models from Motion Capture – Modeling of complex tasks ... for simulation/analysis of crowd dynamics

1

Daniel ThalmannEPFL - VRlab

Dongguk University, Seoul, May 2006

Real-Time AutonomousCrowds

VRlab

2

• Director: Prof. Daniel Thalmann

• About 25 people

• 5 EU projects

• Swiss National Research Foundation

• NCCR CO-ME

Areas of research

• High-level Motion Control Models– Walking, gait

– Automatic Grasping

– Models from Motion Capture

– Modeling of complex tasks

• Standards: MPEG-4, HANIM

3

• Behavioral animation, autonomous agents

• Groups and crowds

• Smart Objects

Areas of research

Areas of research

• Body deformations

• Biomechanics models(medical)

• Health Emergency

4

• Networked VR

• VR interaction

• Haptics

• Augmented reality

Areas of research

sense of presence• Believable interfaces

– Graphics rendering

– Haptic rendering

– Acoustic rendering• Study of human factors

– Personal interfaces (Emotional content)

– Address full range of population: from tech gurus and youngsters to elders and handicapped

5

How to model groups and crowds ?

What is a crowd ?

“A large group of individuals in the same physical environment sharing a common goal and may act

in a different way than when they are alone”(Benesh, 86 and Roloff, 81)

Motivation

• Why do we need virtual crowds?– Real worlds: crowds are ubiquitous– Non-real time applications: (feature

films, commercials, cut-scenes of games) crowds used more and more, usually to increase epic dimensions

– Real-time applications: (games, training simulations) crowds are still rare, most interactive virtual worlds are “ghost towns“

6

Introduction

• Existing application areas for crowd simulations:– Entertainment industry (animation production,

computer games)– Training of police & military (demonstrations, riots

handling)– Architecture (planning of buildings, towns,

visualization)– Safety science (evacuation of buildings, ships,

airplanes)– Sociology (crowd behavior)– Physics (crowd dynamics)

State-of-the-Art

•Procedural animation of flocks (Girard, 90) Film Eurythmy

Maintain proper position and orientation:

~Avoid collisions~Match velocities of neighbours~Move towards the centre of flock

•Flocking systems (Boids) (Reynolds, 1987)

7

Crowd Evacuation Simulators

• inner areas of buildings(Simulex, Thompson & Marchant 95),

• subways (Hareesh et al. 2000),

• ships (Klüpfel et al. 2000)• airplanes (Owen et al. 98).

Model movement of large number of people in usually closed, well-defined spaces like:

LEGION

• Early prototype of LegionTM

developed by Still (2000) for simulation/analysis of crowd dynamics (stadiums).

• Legion v.1.7,: every person treated as individual.

• Each individual scans local environment and chooses action to minimise effort.

8

Crowd Management Training Systems

• CACTUS (Williams 1995) • to assist in planning and training for public order incidents. • crowd groups and police units placed on digitized map and

have probabilistic rules for their interactive behavior (DAG).

•simulator for training US Marines •crowds move within simulated urban

environment along instructor-predefined pathways

•crowd profiles –fanaticism, arousal state, prior experience with non-lethal munitions

•Small Unit Leader Non-Lethal Training System (Varner 98)

State-of-the-Art: 3D systems

Exodus• Bouvier (1996-97)• avoid obstacles using attraction

and repulsion forces analogous to physical electric forces.

• Higher level behavior modeled by transition networks

9

Significant Physics (Brogan and Hodgins, 1997)

State-of-the-Art

Two steps: 1) perception model determines

creatures and obstacles visible to each individual

2) placement algorithm determines desired position for each individual.

• Simulated systems: groups of legged robots, bicycle riders ...

State-of-the-Art: Crowd rendering

• Tecchia and Chrysantou(2001)

• Medieval Total War

• Fast rendering (Image Based – billboarding)

• - Lot of pre-processing• - Static animations

10

• Digital cloning of people using 3D characters (The Borrowers and Titanic, 1998)

State-of-the-Art: Movies

• Fluid mechanics (Stam, 99; Witting, 99)• Hybrid systems: physical forces (flow fluids, goals) and

procedural rules (flocking behaviours). Antz and Bugs Life (1999)


11

Industrial Light and Magic

Star Wars: A Jovial Crowd of Geonosians


PDI/DreamWorks Shrek 2


12

• Lord of the Rings• The Mummy• The Lion King


Games: Act Of War

13

ViCrowd

GB

Script control

(Scripted behaviors

)

Crowd Information

Crowd Behaviour

GIB

External control(Guided

behaviors)

Events and Reactions

Individual Information and behaviours

GIB

GIB

Script Language (Before simulation)

Generated by ViCrowdUser intervention during the simulation

S. Raupp Musse, D.Thalmann, Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE Transactions on Visualization and Computer Graphics, 2001, Vol.7, No2, pp.152-164.

Crowdformed bygroups

• completely programmed or evolvesaccording to behavioural rules

• contains information to be distributedamong groups

• size variable, can change during simulation• possible leader• can be programmed, guided or autonomous• communication, memory, react to events, etc.

Groupformed by agents

• cannot be explicitly controlled• apply the behaviour of the group• possible leader• can change of group• can apply innate behaviours : • goal seeking, follow groups specification,• walk and apply actions, etc.

Agents

14

Emergency Situation

• Crowds

Real-Time Crowds

Concepts

Daniel ThalmannPablo de Heras Ciechomski

Jonathan MaïmJulien Pettré

Sébastien SchertenleibBarbara Yersin

15

Requirements & constraints

• Different from single agent simulations• Conceptual differences – need for variety:

– Variety of agents’ visualizations– Variety of individual trajectories– Variety of behavior– Variety of animations– Variety of reactions to the same situations

• Assets Managements• Data-Driven architecture

Crowd simulation sub-areas

• Crowd behavior generation– How should a virtual crowd behave?

• Crowd motion control– How should virtual entities move around and

avoid collisions?• Integration of crowds in virtual environments

– Which aspects of environment need to be modeled?

• Interaction– How can an end-user interact with the crowd in

real-time

16

Our system

• Crowd simulation for interactive virtual environments:– Goal to reproduce real world scenarios with

large number of virtual humans evolving in real-time

– History: 2 generations of simulation systemsYear 2000- SGI Onyx2 mainframe- Up to several dozens of agents- Few frames per second

Year 2005:- Consumer grade PC- Up to several thousands of agents- Shaders Rendering for Crowds- Greater variety in appearance andanimation

Our system

• Features:– Multi-agent system– Data-Driven– Individual based model– Behavior modeled by Hierarchical Finite State Machine– More complex behaviors described with Python Scripts.– Interation and Feedback about the agents’s status– 3D Shaders Rendering engine– Multi-threaded Environment

17

System Architecture

• Crowd simulation system:– Computation Layers

• Prioritized Tasks• Fast Locality Query• Behaviors Transition

– Visualization layer • Representing Behaviors• Crowd Rendering Engine

– Authoring Tools• Creating and interacting with the crowds• Feedback information about the simulations

• Crowd Component:– Part of the VHD++ real-time framework – vhdCrowdService– Simulated crowd of virtual humans able to move and interact

within virtual environment

A Virtual Human-based VR/AR Platform: VHD++

• Developed by VRlab (EPFL) and MIRALab (Univ. Geneva)

• Is a multithreaded component-based system development– Extendible through Plug-Ins (Crowds, AI, Sound…)

• Comes with a set of standard plug-ins for handling:– Scenegraph (using OpenSceneGraph,

http://www.openscenegraph.org)– Scripting (Python, Lua)– Configuration Files (XML, Collada) for system and data management – Multi-platform (Linux, Windows)

• Used in different domain like– VR, AR, Edutainment,…

18

VHD++: spectrum of applications

VHD++VHD++

VRVRARAR

gamesgamesedutainmentedutainment

infotainmentinfotainment

Social PhobiaSocial Phobia vhd vhd JUSTJUSTimmersive VR healthimmersive VR healthemergency personnel emergency personnel trainingtraining

vhd LIFEPLUSvhd LIFEPLUSAR edutainmentAR edutainmentreconstruction of ancientreconstruction of ancientlife in life in PompeiPompei (Italy)(Italy)

vhd STARvhd STARAR training of hardwareAR training of hardwaremaintenance professionalsmaintenance professionals

vhd ERATOvhd ERATOReconstructing of crowds in Reconstructing of crowds in ancient ancient theaterstheaters

– skeletal and billboard rendering– LODs generated with OGRE3D– fully dynamic animation -> IK possible– greater variety in appearance– Shader Integration

Crowd rendering engine

19

Rendering Management:

• Skeletal deformed characters rendered first, then billboard models (front-to-back)

• Manager stores template humans and keeps record of all existing instances.

• Template humans: set of animations, textures and shaders.

Benefits:

• Benefits of our dual approach are:

– Fully dynamic animation

– We have facial animation detail levels

– Characters look fine from all camera directions and zooms, as opposed to a pure billboard approach

– Memory usage lower than with billboards

20

Crowd rendering engine

• Full integration into OpenGL triangle processing pipeline

Colored spotlight

Specular highlights shading

Cartoon shading

Crowd generation

• 3ds max exporting– Pipeline for converting

character and animation data to format usable by crowd rendering and animation engine

– Exported data:• Mesh• Texture• UV coordinates• Skeleton hierarchy• Deformations bindings• Animations

21

Crowd generation

• Mesh design: around 1’000 polygons each• Textures design: same texture mapping for different meshes;

different characters by varying colors

Designing Texture Variety (1/3)

- Random colors for each body part is not goodenough

- Realistic results require constrained randomness

22


• HSB Maps– HSB space constrained to a

3D color space.– Allow to deal with localized

constraints.– Eyes example :

- Hue from 20 to 250; - Saturation from 30 to 80; - Brightness from 40 to 100.


• Alpha Map Layer– Allow the artist to define the

different body parts.– Allow the application to

modulate the color variety.

• Example of 1 mesh template with 4 different textures.– HSB maps give the illusion of

many different meshes and textures.

23

AI processing• Goal:

– Every agent has its own behavior.

• Agent Management

– Chained Hierarchical State Machine

– State Machine Transitions described within Lua Metatables

– Agent’s tasks are prioritized in order to avoid unnecessary checks. (E.g. updating every state at each frame)

• Scenario Authoring

– Designers can create their own behaviors using scripting interfaces

Behavior engine• Virtual human agent

– Visual• 3D Mesh, Skeletal Animation (compatible with

H-ANIM standard)

– Motor Actions• Steering, Walking, Gestual Animations (Look-

at, Face Animations…)

– Instinct Behaviors• Scenario specific states defining psychological

and physical attributes.

– High-Level Behaviors• Motivation Based Behaviors….

– Authoring• Real-time interaction with the virtual human

allowing to prototype and test the system.

24

Crowds Navigation (1/4)

• Objectives– Crowds

• One to thousands of people • Common or individual goals

– Variety• Spread people• Individual solutions

– Interactivity– Environment

• Uneven surfaces• Multi-floored structure

– Automatic • Unprocessed environment meshes• No user’s intervention


• Principles– Based on a pre-computed Navigation

Graph

– Capture the environment Topology

– Delimit Navigable Areas (vertices)• Out of obstacle

• Limited slopes

• Cylindrical bounding shape

• Local elevation map

– Connect areas with Passages (edges)• Bounding shapes intersections

– Deterministic graph building method (based on Voronoïdiagrams)

25


• Path Planning– Iterative path search

• Provides multiple solutions (edge sequences) leading to the goal

• Captures several homotopyclasses of solutions (i.e. separated by obstacles)

• From the shortest to the longest

– Path declination • Transforms previous paths into

way-point sequences• Exploits passage widths to

produce variety• Individual parameter determines

exactly where each passage is crossed

init

goal


• Results– Way-points following is scalable

(smooth, linear or estimated trajectories)

– Data structure is compact, rich and memory-light

– Navigation graph computed deterministically in minutes

– Robust and versatile

– Inter-collisions are to be considered locally only

26

Crowd Motion control

• Displacement system– Motion of agents between

waypoints– Activate Motor Actions

(walking, running skeletal-based animations)

• Variety– Walking attribute

(velocity, speed…)– Non-static waypoints

through collision free corridors.

– Adaptive walking style using Motion Capture Data

Characters motion using the navigation graph

The promising approach: generate models from motion capture

e.g. Walking model

• Our walking engine: driven by two high-level parameters: speed and human size.

1. interpolation and extrapolation achieved in PCA spaces of each subject to generate new motion according to speed value.

2. time warping method allows to handle the human size parameter.

27

Creation of PCA space• PCA technique transforms set of variables into

smaller set of its linear combination, so as to account for most of variance of original dataset.

• PCA space computed with motion matrix M composed of various walking motion vectors θfrom specific subject v.

θ0: average vector of all n motion vectors of M. α=(α1, α2, …, αm) coefficient vectors e=(e1, e2, …, em) Basic vectors: m first PC's of M,

• walking motion θ:

Characteristics of the model

• We can extrapolate and interpolate realistic and correct walking motion from 0 km/h up to 10 km/h

• Beyond 10 km/h, undesired behavior occurs at skeleton level esp. legs) - Double support phase no longer ensured.

• Scaling size of human directly influences cycle frequency in order to preserve motion properties.

• For 2 humans with different size, smaller one walks with larger frequency for identical motion speed.

28

Crowd authoring

• No matter how good rendering or behavior, there should also be a good way of producing content for simulation

• More individuals more difficult to create unique and varied content– Create and modify characters one by one is too

laborious– Procedural generation can create artifacts (“army-

like” appearance)– Random distributions are hard to control

• Need human-in-the-loop to make creative decisions

Scenario authoring

• CrowdBrush– Goal is to give full

creative power to designers using metaphors of artistic tools

– Works in WYSIWYG (What You See Is What You Get) mode, with a real-time view of the authored scene

29

Scenario authoring

• Brushes

– Tools with visual representation on the screen

– Affect crowd members in different manners: • Create new individuals in the scene

• Change their appearance or behavior

Creation brush Deletion brush “Happy” brush “Sad” brush

Crowds in VR training

• Applications– Training system for urban emergency

situations– Health emergency decision training

• Goal– To reproduce training scenarios based on

real world situations

30

Agent-based Case: Riot in the city

Crowds in virtual heritage

• CAHRISMA

– Reconstructions of mosques

– Around 50 characters

• ERATO

– Reconstruction of theaters and odeons

– Around 1000 characters

31

Future• We will see more and more people

– Faster CPUs & GPUs

– Multi-threaded Architecture (Multi-core CPUs)

– Better algorithms

– Content creation streamlining

• Convergence between non real-time and real-time domains

Documents

VRlab - KOCCA · 2010. 5. 1. · – Walking, gait – Automatic Grasping – Models from Motion Capture – Modeling of complex tasks ... for simulation/analysis of crowd dynamics