Mental Vision

Achille Peternier

A Computer Graphics Platform for Virtual Reality, Science and Education

Thesis director:Prof. Daniel Thalmann

Plan

• Introduction– Context– Goals

• Our solution– Architecture and components– MVisio graphics engine– Pedagogical modules– Low-cost CAVE and wearable 3D system

• Results– Benchmark– Case studies

• Conclusion

Introduction

Introduction - context

Introduction - context

Introduction - context

Introduction - context

handheld devices portable PCs desktop workstations VR systems

Unique framework for 3D graphics

education wearables research immersive graphics

Introduction - goalsConcept

Result

SoftwareDevice

Project

Introduction - goalsConcept

Result

Software

Device

Project

Introduction - goals

Introduction - goals

Introduction - goals

Our solution

Our solution - architecture

MVisio 3D graphics engine

Pedagogical modules

Corollary tools (low cost CAVE and wearable system)

Our solution - mvisio• User-friendly API, based on an C++ class-oriented architecture• Multi-device rendering on PC, PDA and CAVE (a same source-code compiles everywhere)• Multi-platform (Windows, Linux)• Full OpenGL and OpenGL|ES support• Dynamic scene graph management• Dynamic lighting• Dynamic soft shadows• HDR and bloom lighting• Depth of field• Vertex and pixel shaders• Skinning and animations• Particle emitters• Terrain engine• Video2texture from MPEG1/2 files• Loading of scenes directly exported from 3D Studio MAX through a specific plug-in• 2D GUI system with event handling• Loading of complex 2D interfaces from XML files• Object picking• Support for Head-Mounted Displays (HMD), even on PDA• New customizable plug-in objects

Our solution - mvisio

Our solution - mvisio

MVisio API

User application (.exe)

OpenGL (1.1→1.5)

OpenGL|ES (1.0 CM)

PDA engine

Models (.mve)

Animations (.mva)

PC engine

CAVE engine

client/server architecture

Wall Wall

Wall Floor

Videos (.mpg)

Textures/shaders

Our solution - mvisio#include <mvisio.h>

int main(int argc, char *argv[])

{

MVSETTINGS config;

config.load("configForPC.txt");

MVISIO::init(&config);

MVNODE *scene = MVISIO::load("bar.mve");

MVISIO::clear(true, true);

MVISIO::begin3D(NULL);

scene->pass();

MVISIO::end3D();

MVISIO::swap();

MVISIO::free();

return 0;

}

Initialize MVisio with custom settings

Load a scene (with meshes, lights, textures, etc.)

Clear buffers, start a 3D rendering, tell MVisio to render the scene, execute the rendering, swap back to front buffer

Free resources

Our solution – modules

11 pedagogical demonstrators

Our solution - modules

Lesson slide

Click & drag controls

Simplified interface

PowerPoint-like style

Real-time WYSIWYG

display

Our solution - modules

Classes Practicals Projects

Our solution - cave

Our solution - cave

Shutters and shutter glasses Red & blue glasses

Our solution - cave

Our solution – cave

Before calibration After calibration

Our solution - cave#define MV_CAVE

#include <mvisio.h>

int main(int argc, char *argv[])

{

MVSETTINGS config;

config.load("configForCAVE.txt");

MVISIO::init(&config);

MVNODE *bunny = MVISIO::load("bunny.mve");

MVCLIENT::putUser(1.2, 1.6, 1.2);

MVISIO::clear(true, true, true);

MVISIO::begin3D(NULL);

bunny->pass();

MVISIO::end3D();

MVISIO::swap();

_sleep(5000);

MVISIO::free();

return 0;

}

Specify user’s head position in CAVE relative coordinates

Tell MVisio you want a CAVE rendering

Load CAVE settings from a file (client IP addresses and positions)

Our solution - wearableSee-through

HMD

PDA

VGA signal adapter

Battery packWeight ≈ 0.5 kg

Our solution - wearable#define MV_PDA

#include <mvisio.h>

int main(int argc, char *argv[])

{

MVSETTINGS config;

config.load("configForPDA.txt");

MVISIO::init(&config);

MVNODE *bunny = MVISIO::load("bunny.mve");

MVISIO::clear(true, true, true);

MVISIO::begin3D(NULL);

bunny->pass();

MVISIO::end3D();

MVISIO::swap();

_sleep(5000);

MVISIO::free();

return 0;

}

Tell MVisio you want a PDA rendering

Load PDA settings from a file (portrait/landscape orientation)

Our solution - other3D Studio Max

MVisio application

Our exporter

.MVE

(MVisio Entity file format)

Results

Results - benchmark• Simple cross device application tracking fps and using three different models:• classic static Stanford bunny• a building model (using many separated entities and transparencies)• a 86 bones skinned, animated, textured virtual human

• Basic GUI (some text, a couple of buttons)• We want to evaluate speed issues, code differences and visual consistency among different platforms

Results - benchmarkPDA software rendering(OGL|ES Rasteroid 1.0 CL)

Screen size: 320x240

Bunny: ~5.9 fps Building: ~9.3 fpsV. human: ~6.7 fps

Results - benchmarkPDA hardware rendering(OGL|ES MBX-lite 1.0 CL)

Screen size: 640x480

Bunny: ~23 fps Building: ~34 fpsV. human: ~14 fps

Results - benchmarkPC hardware rendering(Nvidia GForce 8800 GT, Core2 Quad @ 2.4 GHz)

Screen size: 640x480

Bunny: >1400 fps Building: >1000 fpsV. human: ~560 fps

Results - benchmark

2D GUI

3D rendering4 side CAVE rendering(1 server PC, 4 clients)Screen size: 1024x768 (each)Nvidia 9800 GTX

Bunny: >270 fps Building: >220 fpsV. human: ~135 fps

Results – student projects

Results – research projects

Results - publicationsA. Peternier, F. Vexo, D. Thalmann, The Mental Vision framework: a platform

for teaching, practicing and researching with Computer Graphics and Virtual Reality, LNCS Transaction on Edutainment, 2008

A. Peternier, S. Cardin, F. Vexo, D. Thalmann, Practical Design and Implementation of a CAVE system, GRAPP 2007

A. Peternier, F. Vexo, D. Thalmann, Wearable Mixed Reality system in less than 1 pound, EG Symposium on VR, 2006

A. Peternier, D. Thalmann, F. Vexo, Mental Vision: a Computer Graphics teaching platform, Edutainment, LNCS, 2006

Results - disseminationportable V-Cave demos for visitors

VIP events

Conclusion

Conclusion

“from your pocket to your CAVE”

Conclusion

VS

• Optimized for speed• Specific contexts• Complex to use• Expensive• Limited to experts

• Optimized for robustness• Multi purposes• Intuitive• Cost efficient• Ideal for students too

Conclusion

Thanks!

From http://www.mechdyne.com/

inline void setPosition(MVVECTOR3 v) {

#ifdef MV_CAVEMVSETTINGS::sendVector3(MV_SETPOSITION3D, id, v.x, v.y, v.z);

#endifposition = v;

}

inline void setRotation(MVVECTOR3 v) {

#ifdef MV_CAVEMVSETTINGS::sendVector3(MV_SETROTATION3D, id, v.x, v.y, v.z);

#endifrotation = v;

}

bool pass(MVNODE *node, bool recurse){

#ifdef MV_CAVEif (recurse) MVSETTINGS::sendCommand(MV_PASS, node->getId());#endif[…]

}

High-level commands are

forwarded to the connected clients

High-level commands are

forwarded to the connected clients

case MV_SETPOSITION3D: {

MVID id;memcpy(&id, data+1, sizeof(MVID)); MVNODE *_node = MVNODE::findById(id); if (_node){

MVVECTOR3 vector;memcpy(&vector, data+5, sizeof(float)*3);_node->setPosition(vector);

}}break;

case MV_PASS: {

MVID id;memcpy(&id, data+1, sizeof(MVID)); MVNODE *_node = MVNODE::findById(id); if (_node){

_node->pass(); }

}break;

…and parsed client-side, then applied to

the same entity

…and parsed client-side, then applied to

the same entity