108
26 July 2007 1 Virtual Reality Applications based on Physical and Behavioral Simulation Osório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007 Virtual Reality Applications based on Virtual Reality Applications based on Physical and Behavioral Simulation Physical and Behavioral Simulation Prof. Dr. Prof. Dr. Fernando Fernando Osório Osório - - Applied Computing / Applied Computing / Unisinos Unisinos Profa Profa . . Dra Dra . . Soraia Soraia Musse Musse - - Computing Science / PUC Computing Science / PUC - - RS RS Prof. Prof. M.Sc M.Sc . . Farlei Farlei Heinen Heinen - - Computing Eng. / Computing Eng. / Unisinos Unisinos M.Sc M.Sc . Milton Roberto . Milton Roberto Heinen Heinen - - Ph.D. Student at UFRGS Ph.D. Student at UFRGS Prof. Dr. Christian Prof. Dr. Christian Kelber Kelber - - Electrical Eng. / Electrical Eng. / Unisinos Unisinos Gustavo Gustavo Pessin Pessin - - M.Sc M.Sc . Student at . Student at Unisinos Unisinos Applied Computing Research Post-grad Program - PIPCA UNISINOS University - Brazil GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-RS) GPVA - Autonomous Vehicles Research Group (Unisinos) GIA - Artificial Intelligence Research Group (Unisinos) RBV - Rede Brasileira de Visualização [FINEP/Brazil] GRAPHIT GRAPHIT GPVA GPVA GIA GIA RBV RBV - - FINEP FINEP IEEE / DVP - Distinguished Visitors Program Latin America

IEEE / DVP - Distinguished Visitors Program Latin America ...osorio.wait4.org/oldsite/palestras/vr-pbsim.pdf · GIA - Artificial Intelligence Research Group ... \PPT -Demos \Labirinto

  • Upload
    buianh

  • View
    219

  • Download
    2

Embed Size (px)

Citation preview

26 July 2007

1

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Virtual Reality Applications based on Virtual Reality Applications based on Physical and Behavioral SimulationPhysical and Behavioral Simulation

Prof. Dr. Prof. Dr. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . . DraDra. . SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Dr. Christian Prof. Dr. Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. Student at . Student at UnisinosUnisinos

Applied Computing Research Post-grad Program - PIPCAUNISINOS University - Brazil

GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-RS)GPVA - Autonomous Vehicles Research Group (Unisinos)

GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]

GRAPHITGRAPHIT GPVAGPVA GIAGIA

RBV RBV -- FINEPFINEP

IEEE / DVP - Distinguished Visitors Program Latin America

26 July 2007

2

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Virtual Reality Applications based on Virtual Reality Applications based on Physical and Behavioral SimulationPhysical and Behavioral Simulation

Prof. Dr. Prof. Dr. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . . DraDra. . SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Dr. Christian Prof. Dr. Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. Student at . Student at UnisinosUnisinos

GRAPHITGRAPHIT GPVAGPVA GIAGIA

RBV RBV -- FINEPFINEP

IEEE / DVP - Distinguished Visitors Program Latin America

Applied Computing Research Post-grad Program - PIPCAUNISINOS University - Brazil

GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-RS)GPVA - Autonomous Vehicles Research Group (Unisinos)

GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]

26 July 2007

3

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

IEEE / DVP - Distinguished Visitors Program Latin America

Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE DVP Program

Created in 1969by JesuitsNow with: - 30.000 Students- 900 Professors- 16 PPGs (post-grad programs)

26 July 2007

4

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

IEEE / DVP - Distinguished Visitors Program Latin America

Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE DVP Program

IT Resources:

"Polo de Informática"- Technological Park- Hi-Tech BusinessIncubator

Undergraduate courses (4 years+):• Computer Science (CS)• Information Technology and Systems (IT)• Computer Engineering (CE)• Electrical Engineering (EE)Technological courses (3 years):• Digital Games and Entertainment• IT Security• Software Dev. and Quality Management

26 July 2007

5

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

VI Brazilian Symposium on Computer Games and Digital Entertainment

Organized byUnisinosPUC-RS

Conference ChairsSoraia MusseFernando OsórioChristian HofsetzJoão Ricardo BittencourtLuiz Gonzaga Jr.

26 July 2007

6

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

26 July 2007

7

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Introduction VR - Virtual Reality

[Paul Milgram et al. 95]

Virtual Reality

26 July 2007

8

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Introduction VR - Virtual Reality

[Paul Milgram et al. 95]

From REAL to VIRTUAL3D + Immersion + Interaction

Virtual Reality

26 July 2007

9

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

3D Visualization

3D World Recreated:

3D Coordinate System - Axes X, Y, Z

• Create 3D objects- Position, Scale, Orientation- Color, Texture, Light- Mesh of polygons

(faces = polygons)

• "Virtual camera"

Introduction VR - Virtual Reality

\\PPTPPT--DemosDemos\\objobj--3d3d

Virtual Reality

26 July 2007

10

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

3D Visualization

3D World Recreated:

3D Coordinate System - Axes X, Y, Z

• Create 3D objects- Position, Scale, Orientation- Color, Texture, Light- Mesh of polygons

(faces = polygons)

• "Virtual camera"

Introduction VR - Virtual Reality

\\PPTPPT--DemosDemos\\LabirintoLabirinto\\PPTPPT--DemosDemos\\CastleCastle

Virtual Reality

26 July 2007

11

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

Virtual RealityVirtual Reality...

* VRML - 3D Worlds (Geometry)

* QTVR - Panorama 3D (Images)

Introduction VR - Virtual Reality

\\VRMLVRML

Virtual Reality

26 July 2007

12

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Introduction VR - Virtual Reality

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

Virtual RealityVirtual Reality...

* VRML - 3D Worlds (Geometry)

* QTVR - Panorama 3D (Images)

\\QTVRQTVR

Virtual Reality

26 July 2007

13

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2222

Real World Integrated with Virtual Objects

ARToolARTool KitKit

http://http://www.hitl.washington.edu/artoolkitwww.hitl.washington.edu/artoolkit//

Augmented Reality

Augmented Reality: ARToolkit - Positioning 3D Objects using references obtained with a camera (webcam)

26 July 2007

14

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2323

ARToolARTool KitKit

\\ARToolKit2.65vrmlARToolKit2.65vrml\\binbinsimpleVRML.exesimpleVRML.exesimple.exesimple.exe

Augmented Reality

Real World Integrated with Virtual Objects

26 July 2007

15

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2424

IRISA / INRIA IRISA / INRIA -- FranceFrancehttp://www.irisa.fr/lagadic/demo/demohttp://www.irisa.fr/lagadic/demo/demo--ar3/demoar3/demo--ar3ar3--eng.htmleng.html

Real World Integrated with Virtual Objects

Augmented Reality

26 July 2007

16

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction

* Immersion

* Realism

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

26 July 2007

17

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction => Virtual Reality Devices

* Immersion => Virtual Reality Devices

* Realism => Graphical Realism (photo-realism)

Movements

Interaction Real x Virtual

"Physics Realism"

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

26 July 2007

18

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction => Virtual Reality Devices

* Immersion => Virtual Reality Devices

* Realism => Graphical Realism (photo-realism)

Movements

Interaction Real x Virtual

"Physics Realism"

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

How to do it?

26 July 2007

19

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

20

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

21

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

22

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

23

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

24

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

PhantomPhantom

HaptionHaption

CyberForceCyberForce

OmegaOmega

26 July 2007

25

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

[Ari Chapiro - Dance]

[CromosLab]

Agents

26 July 2007

26

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

26 July 2007

27

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Knowledge

AutonomousBehaviour

26 July 2007

28

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. VR and Simulation

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Virtual World

xx

Real World

26 July 2007

29

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

26 July 2007

30

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Virtual World

xx

Real World

Increasing Reality in VR Applications: Physical and Behavioral Simulation

2. VR and Simulation

26 July 2007

31

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

RealisticVR

Increasing Reality in VR Applications: Physical and Behavioral Simulation

Virtual World

xx

Real World

26 July 2007

32

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. Introduction

RealisticVR

Increasing Reality in VR Applications: Physical and Behavioral Simulation

Virtual World

xx

Real World

Real WorldReal WorldSimulationSimulation

26 July 2007

33

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Simulation Tools:

1. OpenSteer

2. ODE - Open Dynamics Engine

3. PhysX AGEIA

4. Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...

3. Physics Simulation Tools

26 July 2007

34

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Simulation Tools:

1. OpenSteer

2. ODE - Open Dynamics Engine

3. PhysX AGEIA

4. Deformable Objets and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...

3. Physics Simulation Tools

Physics:

Physical structure: resistance, mass, density, elasticity;Position and orientation in the 3D space; Kinematics and Dynamics;Linear and angular velocities; Motion (w/ forces and torques), trajectories; Acceleration, deceleration;Attraction and repulsion;Gravity, friction, inertia; Kinetic and potential energy; Laws of energy conservation, linear and angular momentum;Collisions and reaction to collisions;Steering models (wheeled cars, aircrafts, projectiles, boats and ships); Articulated Rigid Bodies Simulation (skeleton, robotic arm);Dynamic Simulation of Deformable Objects: elastic objects; Fluid simulation and Particle Systems (fire, smoke, clouds and liquids).

26 July 2007

35

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. OpenSteer [Reynolds]

Simple steering behaviours

3. Physics Simulation Tools

Geometric: SimpleKinematic: SimplePhysical: SimpleBehavioural: Simple

Steering BehaviourGroup Behaviour

26 July 2007

36

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

1. OpenSteer / Boids

Simple steering behaviours

3. Physics Simulation Tools

26 July 2007

37

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

Simulation of Articulated Rigid Body Dynamics

Open Source Library (C/C++ API)

Used with OSG, Ogre3D, CrystalSpace, ...

3. Physics Simulation Tools

Slider jointSlider jointHinge jointHinge jointBall and socket jointBall and socket joint

Universal jointUniversal joint

26 July 2007

38

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

Simulation of Articulated Rigid Body Dynamics

3. Physics Simulation Tools

Physics Simulation:

- Gravity, friction, acceleration, deceleration; - Generation of motion: applying forces and torques (motors);- Collision avoidance and treatment (reaction, object bounce);- Kinematics models and rigid body dynamics simulation; - Different types of joints with actuators (motors)

Webbots uses ODE [Cyberbotics]

26 July 2007

39

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

26 July 2007

40

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

26 July 2007

41

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

26 July 2007

42

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

26 July 2007

43

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

WebotsCyberbotics

26 July 2007

44

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

2. ODE - Open Dynamics Engine

3. Physics Simulation Tools

Juice[Nate W.]

26 July 2007

45

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

3. PhysX AGEIAHardware Accelerated Physics SimulationPPU - Physics Processing Unit / GPU - Graphics Proc. Unit

3. Physics Simulation Tools

- Complex rigid body object physics system: dynamics and collision detection- Joints and springs. Characters with complex, jointed geometries

for more lifelike motion and interaction- Volumetric fluid creation and simulation- Cloth that drapes and tears the way you would expect it to- Smart particles. Dense smoke and fog that billow around objects in motion.- Explosions that cause dust and collateral debris

26 July 2007

46

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

3. PhysX AGEIA - PPU

3. Physics Simulation Tools

26 July 2007

47

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Simulation Tools:

1. OpenSteer

2. ODE - Open Dynamics Engine

3. PhysX AGEIA

4. Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...

3. Physics Simulation Tools

26 July 2007

48

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

4. Deformable Objects and Fluids

- Finite Elements Methods

- Spring-Mass Systems

- CFD (Computational Fluid Dynamics)

- Level Set Methods

3. Physics Simulation Tools

26 July 2007

49

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

4. Deformable Objects and Fluids- CFD

Computational Fluid Dynamics

- Level Set Methods

3. Physics Simulation Tools

[Fedkiw 2006]

26 July 2007

50

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

3. Physics Simulation Tools

[Fedkiw]

26 July 2007

51

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Simulation Tools:

1. OpenSteer

2. ODE - Open Dynamics Engine

3. PhysX AGEIA

4. Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...REAL TIME SIMULATION !

3. Physics Simulation Tools

26 July 2007

52

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning

4. Intelligent Behaviour

Comportamento

26 July 2007

53

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge Integration

4. Intelligent Behaviour

Reactive-Deliberative Architecture

Control Architectures: Reactive, Deliberative, Hierarchical, Hybrid

Hybrid Architecture

Architecture BDI (Beliefs-Desires-Intentions)

F. Osório et al.[Virtual Concept 2005]

26 July 2007

54

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning

4. Intelligent Behaviour

Robotic [GPVA] Crowds [CromosLab]Ontology [GIA]

26 July 2007

55

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: VR Simulation Tools

6. Conclusions and New Trends

26 July 2007

56

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots SimulatorSEVA 3D - Autonomous Vehicle ParkingLEGGEN - Legged (articulated) Robots Simulator

2. Knowledge and Reasoning in VR Environments

UEM - Urban Environment ModelCrowds Simulation in Normal Life Situations

Robombeiros - Fire Fighting

26 July 2007

57

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SimRob3D

- Our Simulation Tools: SimRob2D (Khepera)SimRob3D Seva2D

Autonomous Robots in VR Environments

26 July 2007

58

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SimRob3D Simulator

> Sensors: Infrared, Sonar, Bumper

> Actuators / Kinematics: Differential, Ackerman

> Realistic Simulation Model: 3D World + noise / error (imprecise sensors and actuators)

Autonomous Robots in VR Environments

26 July 2007

59

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SimRob3D Simulator

> Vehicle Simulation xx Vehicle Control

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

HyCAR

Hybrid Control ofAutonomous Robots

Autonomous Robots in VR Environments

26 July 2007

60

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SimRob3D

Simulator

Autonomous Robots in VR Environments

26 July 2007

61

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SEVA 3D - "Sistema de Estacionamento de Veículos Autônomos"

Sources of Inspiration:

- Baja Buggy remotely controlled by a cell phoneC. Kelber - UNISINOS, Brazil

Published at:IEEE WCCIIJCNN 2006

Autonomous Robots in VR Environments

26 July 2007

62

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

SEVA 3D

26 July 2007

63

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA 3D Simulator

> Vehicle Simulation xx Vehicle Control

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

Autonomous Robots in VR Environments

5. Applications: SEVA 3D

26 July 2007

64

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: FSA - Finite State Automaton

Inputs:

- Sonar Sensors: Stochastic ray-casting / 3D cone)

Outputs:

- Steering Wheel Angle- Gas pedal (car speed + direction: fwd, back)

States:

Autonomous Robots in VR Environments

5. Applications: SEVA 3D

26 July 2007

65

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: Searching Parking Space

5. Applications: SEVA 3D

26 July 2007

66

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: Entering

5. Applications: SEVA 3D

26 July 2007

67

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: Positioning Inside

5. Applications: SEVA 3D

26 July 2007

68

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: Aligning

5. Applications: SEVA 3D

26 July 2007

69

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA: NEURAL FSA - Learning the FSA...

Si(t) + Si(t-1) + ...Temporal Window

Adapted Jordan-Net using RProp Learning

5. Applications: SEVA 3D

26 July 2007

70

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

SEVA3D - Autonomous Vehicle Parking Simulator

3D

SensorsActuatorsKinematicsFSA CtrlANN Ctrl

5. Applications: SEVA 3D

26 July 2007

71

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots SimulatorSEVA 3D - Autonomous Vehicle ParkingLEGGEN - Legged (articulated) Robots Simulator

2. Knowledge and Reasoning in VR Environments

UEM - Urban Environment ModelCrowds Simulation in Normal Life Situations

26 July 2007

72

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Legged Robots Evolution and Walking Control

Sources of Inspiration:

Autonomous Robots in VR Environments

[EPFL]

26 July 2007

73

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Legged Robots Evolution and Walking Control

Sources of Inspiration: LEGGEN - Published at: IEEE WCCI CEC 2006SBIA 2006

33

Autonomous Robots in VR Environments

26 July 2007

74

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

26 July 2007

75

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of Robots: 3D Realistic Virtual Environments- Sensors: infrared, sonar, bumpers, gyro (accelerometers),

GPS, compass, light and vision sensors, etc. - Actuators: legs and arms with angular motors (joints) - Physics: collision, kinematics, rigid body dynamics

Simulation of Legged Autonomous Robots:- Robot ControlArchitectures Implementation

Genetic Evolved Control of Articulated Robots (w/legs)

26 July 2007

76

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of 3D Realistic Virtual Legged Robots LEGGEN Simulator - Tools:

1. OSG- Open Scene Graph (OpenGL + Extensions)[ http://www.openscenegraph.org/ ]

2. ODE - Open Dynamics EngineRigid Body Physics Simulation(gravity, inertia, friction, collision, joints, etc)[ http://www.ode.org/ ]

3. GALib - Genetic Algorithms Simulation[ http://www.lancet.mit.edu/ga/ ]

4. Robot Control FSM: Finite State Machine = Sense + Act

26 July 2007

77

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation main goals:

- Evaluate different Robot Models(hardware configurations)IEEE WCCI / CEC 2006 - Vancouver, Canadá

- Evaluate different Fitness FunctionsIBERAMIA / SBIA - Ribeirão Preto, SP Robot Models

Boston Dynamics

Evaluate different robot models in order to select a better hardware configuration

26 July 2007

78

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

LEGGEN SIMULATOR

Simulation Results:

26 July 2007

79

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

LEGGEN SIMULATOR

Simulation RESULTS:

Tetrapod Video - Distance, Gyro

26 July 2007

80

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

LEGGEN SIMULATOR

Simulation RESULTS: Tetrapod Video - 2 a 2

26 July 2007

81

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

LEGGEN SIMULATOR

Simulation RESULTS: Tetrapod Video - "bloopers"

26 July 2007

82

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots SimulatorSEVA 3D - Autonomous Vehicle ParkingLEGGEN - Legged (articulated) Robots Simulator

2. Knowledge and Reasoning in VR Environments

UEM - Urban Environment ModelCrowds Simulation in Normal Life Situations

Robombeiros - Fire Fighting

26 July 2007

83

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment Model

Sources of Inspiration: CromosLab

Knowledge and Reasoning in VR Environments

26 July 2007

84

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment Model

Sources of Inspiration: CromosLab

Knowledge and Reasoning in VR Environments

Normal Life - Agents:Children going to the schoolAdults going to work at usual times...

Environment:School, Stores, ...Flammable Liquids...

Ontology!

26 July 2007

85

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment ModelKnowledge and Reasoning in VR Environments

- Agents are created using an ontology;

- Ontology includes information of population profiles;

- Ontology includes information about the urban environment;

- People (virtual agents) created based on statistical data or fictitious information;

- Agents move and behave in the urban lifeaccording their usual activities (time),

as described in the ontologies;

- People move during “normal life” in a more realistic way,

without a “random aspect”, which is common in other (not so realistic) works;

- Able to manage crowds in a macrocospic point of view;

- Easy to define, easy to implement, easy to control!

- Knowledge about the general model of the VE can be used to the sKnowledge about the general model of the VE can be used to the simulationimulation;

Structured and semantic environment

26 July 2007

86

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment ModelKnowledge and Reasoning in VR Environments

UEM - Global Ontology

26 July 2007

87

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment ModelKnowledge and Reasoning in VR Environments

UEM - Global Ontology

Has

Components ofEnvironment

Is

26 July 2007

88

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment ModelKnowledge and Reasoning in VR Environments

UEM - Global Ontology

26 July 2007

89

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM - Urban Environment ModelKnowledge and Reasoning in VR Environments

26 July 2007

90

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM Urban Environment Model

Knowledge and Reasoning in VR Environments

At 7:00 AM people are at home

26 July 2007

91

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM Urban Environment Model

Knowledge and Reasoning in VR Environments

At 11:29 AM:Students and employed adults are in school and workWe can observe some other people on the street

26 July 2007

92

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM Urban Environment Model

Knowledge and Reasoning in VR Environments

At 11:29 AM:Students and employed adults are in school and workWe can observe some other people on the streetAt 12:05 PM:Students leave school

26 July 2007

93

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

UEM Urban Environment Model

Knowledge and Reasoning in VR Environments

26 July 2007

94

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots SimulatorSEVA 3D - Autonomous Vehicle ParkingLEGGEN - Legged (articulated) Robots Simulator

2. Knowledge and Reasoning in VR Environments

UEM - Urban Environment ModelCrowds Simulation in Normal Life Situations

Robombeiros - Fire Fighting

26 July 2007

95

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:* 2D and 3D Simulation* Simulation of fire propagation* Autonomous fire-fighting team* Define: Strategy, Mission, Execution

Fire PropagationSimulation:

- Direction and Speedof wind

- Vegetation type andcoverture density

(speed of propagation)- Terrain

Figure: 2D Simulation using SDL library => http://pessin.googlepages.com/robombeiros

Published at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

26 July 2007

96

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:

http://pessin.googlepages.com/robombeirosPublished at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

3D Visualization:- Vegetation, Fire- Autonomous mobile Robots- Stereo 3D - Tools: OSG, ODE, Demeter

26 July 2007

97

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:

3D Simulation:- Fire propagation- Physics- Robot Control

http://pessin.googlepages.com/robombeiros

26 July 2007

98

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: VR Simulation Tools

6. Conclusions and New Trends

26 July 2007

99

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

New Trends

Claytronics - Nanotech[Published at IROS2006]

26 July 2007

100

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) InteractionAgentsSimulation

26 July 2007

101

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) => OpenGL, DirectX, VRML, QTVR, OSGInteraction => Augmented Reality, Haptic Devices, SensorsAgents => Behaviour (Perceive, Act, Interact), ControlSimulation => Models, Physics, ODE

26 July 2007

102

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) => OpenGL, DirectX, VRML, QTVR, OSGInteraction => Augmented Reality, Haptic Devices, SensorsAgents => Behaviour (Perceive, Act, Interact), ControlSimulation => Models, Physics, ODE

* Interaction based on PhysicsPerception Physics => Rigid Body DynamicsAction Soft Body - Deformable, ParticlesKinematics Steering modelsDynamics

26 July 2007

103

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) => OpenGL, DirectX, VRML, QTVR, OSGInteraction => Augmented Reality, Haptic Devices, SensorsAgents => Behaviour (Perceive, Act, Interact), ControlSimulation => Models, Physics, ODE

* Interaction based on PhysicsPerception Physics => Rigid Body DynamicsAction Soft Body - Deformable, ParticlesKinematics Steering modelsDynamics

* Behavioural Simulation and Virtual Autonomous Agents (AI)Behavioural control (e.g. boids)Control Architectures: Deliberative, Reactive, Hierarchical, HybridCognitive...

26 July 2007

104

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) => OpenGL, DirectX, VRML, QTVR, OSGInteraction => Augmented Reality, Haptic Devices, SensorsAgents => Behaviour (Perceive, Act, Interact), ControlSimulation => Models, Physics, ODE

* Behavioural Simulation and Virtual Autonomous Agents (AI)Behavioural control (e.g. boids)Control Architectures: Deliberative, Reactive, Hierarchical, HybridCognitive:- Knowledge- Emotional states- Personality- Personal profile- Special places- Functioning rules (ontology)- Place profile

Agents

Environment

26 July 2007

105

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions => Review

VR... From Real to VirtualVisualization (3D) => OpenGL, DirectX, VRML, QTVR, OSGInteraction => Augmented Reality, Haptic Devices, SensorsAgents => Behaviour (Perceive, Act, Interact), ControlSimulation => Models, Physics, ODE

* Behavioural Simulation and Virtual Autonomous Agents (AI)Behavioural control (e.g. boids)Control Architectures: Deliberative, Reactive, Hierarchical, HybridCognitive:- Knowledge- Emotional states- Personality- Personal profile- Special places- Functioning rules (ontology)- Place profile

Agents

Environment

Autonomy IVREReality VR PBSim

VR++

VeryVery interestinginterestingapplicationsapplications!!

26 July 2007

106

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions and New Trends

Virtual Reality Environments:

Geometric + Kinematic + Physical + Behavioural + Cognitive=

Realistic RV Environments

New Trends:

VR PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces

26 July 2007

107

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

Conclusions and New Trends

Virtual Reality Environments:

Geometric + Kinematic + Physical + Behavioural + Cognitive=

Realistic RV Environments

New Trends:

VR PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces

FullImmersion

26 July 2007

108

Virtual Reality Applications based on Physical and Behavioral SimulationOsório, Musse, Heinen, Kelber, Pessin IEEE Computer Chapter - Colombia Section / Medellin, July 2007

CONTACT INFORMATION

UNISINOS University UNISINOS University -- BrazilBrazil

Applied Computing Research PostApplied Computing Research Post --grad Programgrad Program -- PIPCAPIPCAAutonomous Vehicles Research GroupAutonomous Vehicles Research Group -- GPVAGPVA

WebWeb: : GoogleGoogle -- veiculosveiculos autonomosautonomos

GPVA GPVA WebWeb PagePage: : http://www.eletrica.unisinos.br/~autonomhttp://www.eletrica.unisinos.br/~autonom

ContactContact -- WebWeb PagesPages : : http://inf.unisinos.br/~osorio/http://inf.unisinos.br/~osorio/http://www.inf.pucrs.br/~smusse/http://www.inf.pucrs.br/~smusse/http://ncg.unisinos.br/robotica/http://ncg.unisinos.br/robotica/

Contact: Prof. Dr. Fernando OsórioE-Mail: [email protected]. Dra. Soraia MusseE-Mail: [email protected]. M.Sc. Milton HeinenE-mail: [email protected]