Upload
sebastien-kuntz
View
1.224
Download
3
Tags:
Embed Size (px)
DESCRIPTION
MiddleVR is a generic VR plugin. SEARIS is the "Software engineering and architecture for realtime interactive systems" workshop for IEEEVR.
Citation preview
Sébastien ‘Cb’ Kuntz @Cb_VRGeek
Philosophy and architectureSEARIS 2014
Plan
• Introduction
• Origin of MiddleVR
• Goals
• Demo
• Philosophy & Architecture
• Conclusion
(French railways)
“Creating Virtual Reality Games:
The Fundamentals”
Founder & President
Sébastien ‘Cb’ Kuntz
Zone hunter (Virtuality-1994)
FRENCH RAILWAYS - SNCF
• Polhemus magnetic tracker, treadmill, 5DT Glove• C++ / OpenSceneGraph / VRPN / OpenAL
FRENCH RAILWAYS - SNCF
VIRTOOLS
VIRTOOLS VR PACK
• VR tools to simplify VR
Virtual reality
Presence in a virtual world
RidiculousIf it’s real VR, you probably
look ridiculous !
• Build hardware
To create and maintain presence,you need to ...
• Build software
• Build user experience
The challenges of creating a VR software
“VR is mainly a software problem”
Jaron Lanier
Device management
Physical devicePhysical device DriverDriver ApplicationApplication
Device management
Tracker 1Tracker 1Driver Tracker Driver Tracker
11
ApplicationApplication
Tracker 2Tracker 2 Driver Driver Tracker 2Tracker 2
Correct perspective
(c) Johnny Lee
Correct perspective
Stereoscopy
Warping & Blending
Warping & Blending
Graphics cards limitations
• Number of outputs
• Processing power
multiple graphics cards
• Pro
• More outputs
• More processing power
• Con
• 3D engines are not optimized for multi-pipe
Cluster
• Pro
• More outputs
• More processing power
• Con
• 3D engines don’t support clustering
• No seams across displays require 3 layers of sync
• 3D objects synchro (Scenelock)
• New images display (Swaplock)
• Left/Right eyes (Genlock)
Cluster
Interactions
• Many different 3D user interactions
• Navigation
• Selection
• Manipulation
• Draw
• System control
• etc.
Interactions are dependent on hardware
Menus
Haptics(force feedback)
Deployment
• Modify application to
• Change drivers
• Change displays
• Change interactions
• Platform for creating interactive 3d applications
• Widely used
• 2.5 million registered developers
• High-quality rendering
• Deferred renderer
• Occlusion culling
• Realtime shadows
• Ambient occlusion
• Light mapping
Unity 3D Features• Programming
• C#, Javascript
• Profiler
• Physics
• NVidia PhysX
• Cloth
• Soft/rigid bodies
• Ragdolls
• Cars
3D formats
FBX, Collada, 3DS, OBJ…
Audio engine
Networking
Terrain
Editor
• Simple to use
• Create your own tools inside Unity
Asset store
• Buy / sell assets
• 3D models
• Textures
• Shaders
• Scripts
• Editor plugins
DesktopDesktop
• Build hardware
To create and maintain presence,you, the author, need to ...
• Build software
• Build user experience
Existing VR software
• VRPN
• OpenTracker
• VRJuggler
• Virtools
• Worldviz
• …
VRPN
VR Juggler
VIRTOOLS VR PACK
Worldviz Vizard
• Based on OpenSceneGraph
• Python
Worldviz Vizard
The need for betterVR software tools
• Hardware getting cheaper
• But engineers are not getting cheaper
• Need to be more efficient and spend less time creating apps
• Democratization
• VR developers are not all engineers anymore
Goals
• Simplify VR
• Simplify creation of VR applications
• Simplify deployment of VR applications
• Create new VR tools
• Adapting existing 3D applications
What is ?
• Generic VR plugin (C++ SDK)
• Functionalities○Manage input devices
•3D trackers, keyboard, mouse, joystick○Manage display
•Viewports, cameras, stereoscopy, cluster○High-level building-blocks
○Interactions○Menu
Architecture
MiddleVR (C++)MiddleVR (C++)
DriversDrivers OpenGLOpenGL DirectDirectXX
NetworNetworkk
C# WrapperC# Wrapper
GUIGUI UnityUnity
Lua Lua WrapperWrapper
Lua Lua WrapperWrapper
Python Python WrapperWrapperPython Python
WrapperWrapper
MiddleVR Features
Supported Devices• Via VRPN :
• A.R.T• Vicon• Optitrack• Wiimote
• Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers,
• Ascension Flock of Bird
• Intersense IS600, IS900
• 3dTech HiBall-3000 Wide Area Tracker,
• Advanced Realtime Tracking Gmbh DTrack Client,
• World Viz Precision Position Tracker PPT 1.2,
• Natural Point Optitrack Rigid Body Toolkit,
• Phasespace optical tracking OWL,
• ...
• Native drivers• Oculus Rift• Leap Motion• Kinect (Microsoft SDK)• PNI SpacePoint Fusion• GameTrak• Razer Hydra• SpaceMouse• Motion Analysis (beta)
Haptics(force feedback)
VR Camera• MiddleVR computes all the cameras’
parameters automatically
• Correct perspective
• Scale 1
Stereoscopy
• Active stereoscopy (OpenGL Quad-Buffer)
• Which Unity can’t natively handle
• Passive stereoscopy
Viewports management
Simply configure your system
Navigations
Homography
Cluster & Multi-pipe
• Scenelock
• Swaplock
• Genlock
• Multi-pipe (soon)
Supported displays
• Virtually any HMD• VR-Wall, Powerwall• Workbench, Holobench• HoloStage• CAVEs• 3D TVs• zSpace• Igloo Vision• …
Goals
• Simplify creation of VR applications
• Simplicity of adding VR into application
• Simplicity of API
• High-level tools
• Interactions: Selection, manipulation, navigation, menus, GUIs
Example of device management
Driver 1
Driver 2
Device management
MiddleVR
Class reference
The value of Simplicity
The value of simplicity
“Perfection is achieved, not when there is nothing more to add, but
when there is nothing left to take away”
Antoine de St Exupery
The value of simplicity
• For each feature you add
• Test
• Document
• Support / maintain
• Each feature costs !!
The value of simplicity• Huge value in simplicity !
• User side
• Faster to learn, less to remember
• Developer side
• Faster to code
• Easier maintain
• Less bugs
• Business side
• Need less coders
• More interesting features
• Easy download / setup > Quickly validate on user’s hardware !
• Easier support
API in simple english• Long discussions for naming of functions, classes, variables
• Consistent api
• SetNumButtons, SetButtonsNum, SetNbButtons > SetButtonsNb
• French not good at english
• Analog ? Digital ? Proxy ?
• Use natural terms depending on device
• Joystick->GetAxisValue()
• Joystick->IsButtonPressed()
• Keyboard->IsKeyPressed()
• Why “Register” ? Why not “Add” ?
• Why so complex, want to sound very clever ?
• AddDevice, AddCamera, …
• API in degrees, because more intuitive / simple to work with
Simplified API
• Simple but complete API
• Like legos
• Rotate
• Node3D->Rotate( Quat )
• Node3D->Rotate( Euler )
• Node3D->Rotate( Axis, Angle)
• Quat( quat ), Quat( Euler ), Quat( Axis, Angle)
The value of simplicity
“Simplicity is the ultimate sophistication”
DaVinci
The value of simplicity• Design user interface for the 80%
• Don’t ruin the experience for the 20%
• Simple things should be simple
• Complex things can be complex
• Put in an “advanced menu”
• Don’t let beauty of design ruin user experience
• Like real life architects who want their design to be beautiful but totally unusable on a day to day
• Example of Lego
• Simple but complete
Optimizations
• Engineers have a strong tendency to over-optimise
• Complexifies code
• Code must be above all readable
• To avoid bugs
• To maintain maintenability
• If you want to optimise
• First profile your code
• Optimize what is *really* taking time
• not what you *think* takes time
Empathy
“The ability to understand and share the feelings of another.”
Test with users
Hardware SDK are mostly too complicated
Drivers• Why not just a few simple functions with
sensible defaults ?
• Init()
• Update()
• GetValue()
• Destroy()
• And add more functions for more advanced
Documentation
• User Guide
• Class references
Goals
• Simplify deployment of VR applications
• GUI to configure VR systems
• Dynamic reconfiguration of application
• Ship with standard configurations
• Abstractions
Deployment
• Abstractions
• Devices
• 3D Nodes (User)
• Interactions (soon)
Device management
Driver Tracker Driver Tracker 11
ApplicationApplication
Driver Driver Tracker 2Tracker 2
Device abstraction
Virtual Virtual DeviceDevice
ApplicationApplication
Driver Tracker Driver Tracker 11
Driver Driver Tracker 2Tracker 2
Device management
DriverDriver
Virtual Virtual DeviceDevice
ApplicationApplication
SimulatioSimulationn
ReplayReplay
Display
• Cameras & viewport created dynamically
User• Human-centric rather than techno-
centric
Virtual Device Virtual Device 11
HandHand(3D Node)(3D Node) ApplicationApplication
Virtual Device Virtual Device 22
Inverse Inverse KinematicsKinematics
User abstraction
Interactions• MiddleVR includes high-level
interactions
• Navigation
• Selection
• Manipulation
• … More to come
• Future: Interactions abstractions
• Dynamically change interactions based on hardware
Interactions abstraction?
Grab-airGrab-air
NavigationNavigation
ApplicationApplication
PointPoint
SteeringSteering
SelectionSelection
ManipulationManipulation
RaycastinRaycastingg
Go-goGo-go
HandHand
HomerHomer
HandHand
WIMWIM
• How to create portable interactions ?
• Navigations can be portable
• Except collisions
• Generalise what is generalisable
• + Custom part for engine dependent
• Collisions
• Highlights
• Gizmos
Interactions
Menus• Ability to create menus and GUIs in HTML5
(soon)
Create immersive HTML5 GUI
C++ Reflexion
• Also for configuration file
• Cluster synchronisation
• Accessing drivers specific information
Working with multiple 3D engines
• Different coordinate systems
• vrCoordinateSystem( Front, Up, Right )
• +/- X,Y,Z
• OpenGL:
• Front= -Z
• Up = +Y
• Right = +X
• DirectX:
• Front=+Z
• Up=+Y
• Right=+X
Working with multiple 3D engines
• C++
• Compiling
• STL, Boost
• Strings (Wide strings)
• Link issues
• Different compilers (VC2005/2008/2010/2012)
• Different languages
• C wrapper
• Lua wrapper
• Simplicity• Creation• Deployment
•Adaptability•Hardware• Software
Our future
• Games/apps development:
ConclusionConclusion