Upload
sebastien-kuntz
View
1.191
Download
10
Tags:
Embed Size (px)
DESCRIPTION
Unity3D is a great authoring tool to create 3D applications. Regarding immersive virtual reality, its capabilities are limited. We will discuss the challenges and solutions to create great VR applications in Unity!
Citation preview
!
• Founder & President!
• 12 years in VR!
!
• - French railways!
!
•
Sébastien Kuntz
• Simplify VR!
• Creation!
• Deployment!
• So YOU can spend more time creating applications
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Rendering speed!
• MiddleVR for Unity!• Conclusion
Plan
Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
• Platform for creating interactive 3d applications!
• Widely used!
• 2 million registered developers!
• Certified by the US Army and Air Force!
• Clients : Bigpoint, Cartoon Network, Coca-Cola, Disney, Electronic Arts, LEGO, Microsoft, NASA, Nickelodeon, Ubisoft, US Army, Warner Bros.
• High-quality rendering!
• Deferred renderer!
• Occlusion culling!
• Realtime shadows!
• Ambient occlusion!
• Light mapping
Unity 3D Features• Programming!
• C#, Javascript!
• Profiler!
• Physics!
• NVidia PhysX!
• Cloth!
• Soft/rigid bodies!
• Ragdolls!
• Cars
3D formats !
FBX, Collada, 3DS, OBJ…!
Audio engine!
Networking!
Terrain
Editor
• Simple to use!
• Create your own tools inside Unity
Asset store
• Buy / sell assets!
• 3D models!
• Textures!
• Shaders!
• Scripts!
• Editor plugins
Desktop
Plan• Unity!
Challenges!• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Plan• Unity!• Challenges!
Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Virtual reality
Presence in a virtual world
(c) eMagin(c) Barco
Experience of presenceis in your brain !
Cognitive presencePerceptive presence
• Build hardware
To create and maintain presence, you need to ...
• Build software
• Build user experience
Plan• Unity!• Challenges!
• Presence!
Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Device management
Physical device!!
Driver Application
Device management
Tracker 1! Driver Tracker 1
Application
Tracker 2!!
Driver Tracker 2
Plan• Unity!• Challenges!
• Presence!• Device management!
Display management!• Cluster management!• Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Correct perspective
(c) Johnny Lee
Correct perspective
Stereoscopy
Warping & Blending
Graphics cards limitations
• Number of outputs!
• Processing power
multiple graphics cards
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines are not optimized for multi-pipe
Cluster
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines don’t support clustering
• No seams across displays require 3 layers of sync!
• 3D objects synchro (Scenelock)!
• New images display (Swaplock)!
• Left/Right eyes (Genlock)
Cluster
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!
Interactions!• Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Interactions
• Many different 3D user interactions!
• Navigation!
• Selection!
• Manipulation!
• Draw!
• System control!
• etc.
Interactions are dependent on hardware
Menus
Haptics (force feedback)
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!
Deployment!• Speed!
• MiddleVR for Unity!• Conclusion
Deployment
• Modify application to!
• Change drivers!
• Change displays!
• Change interactions
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!
Rendering speed!• MiddleVR for Unity!• Conclusion
Rendering speed
Plan• Unity!
Challenges!• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Rendering Speed!
• MiddleVR for Unity!• Conclusion
Application
Plan• Unity!• Challenges!
• Presence!• Device management!• Display management!• Cluster management!• Interactions!• Deployment!• Speed!
MiddleVR for Unity!• Conclusion
• Build hardware
To create and maintain presence, you, the author, need to ...
• Build software
• Build user experience
“VR is mainly a software problem”!Jaron Lanier
What is MiddleVR ?
• VR plugin !
• Functionalities ○Manage input devices
•3D trackers, keyboard, mouse, joystick !
○Manage display •Viewports, cameras, stereoscopy, cluster
Desktop
http://www.youtube.com/watch?v=EEKH-1EilXs
MiddleVR tutorial
Device management
Device management
Supported Devices• Via VRPN :
• A.R.T • Vicon • Optitrack • Wiimote !
• Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers,
• Ascension Flock of Bird • Intersense IS600, IS900 • 3dTech HiBall-3000 Wide Area Tracker, • Advanced Realtime Tracking Gmbh DTrack
Client, • World Viz Precision Position Tracker PPT
1.2, • Natural Point Optitrack Rigid Body Toolkit, • Phasespace optical tracking OWL, • ...
• Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint
Fusion • GameTrak • Razer Hydra • Motion Analysis (beta)
First integration of Haption’s haptic devices
http://www.youtube.com/watch?v=lnoE_VclnOI
Stereoscopy
• Active stereoscopy (OpenGL Quad-Buffer)!
• Which Unity can’t natively handle!
• Passive stereoscopy
Cluster & Multi-pipe
• Scenelock!
• Swaplock!
• Genlock!
• Multi-pipe (soon)
Supported displays
• Virtually any HMD • VR-Wall, Powerwall • Workbench, Holobench • HoloStage • CAVEs • 3D TVs • zSpace • Igloo Vision • …
Deployment
• Abstractions!
• Devices!
• 3D Nodes (User)!
• Interactions (soon)
Device management
Driver Tracker 1
Application
Driver Tracker 2
Device abstraction
Virtual Device
Application
Driver Tracker 1
Driver Tracker 2
Device management
Driver
Virtual Device Application
Simulation
Replay
Virtual Device 1
Hand (3D Node) Application
Virtual Device 2
Inverse Kinematics
User abstraction
Interactions• MiddleVR includes high-level interactions!
• Navigation!
• Selection!
• Manipulation!
• … More to come!
• Future: Interactions abstractions!
• Dynamically change interactions based on hardware
Interactions abstraction?Grab-air
Navigation
Application
Point
Steering
Selection
Manipulation
Raycasting
Go-go
Hand
Homer
Hand
WIM
Menus
• Ability to create menus and GUIs in HTML5 (soon)
Speed• Unity!
• Many options for optimizations!
• Occlusion culling!
• Static batching!
• Lightmap baking!
• etc.!
• 32-bit editor only!
• But player is 32-bit & 64-bit !!
• Subdivide parts in editor!
• Compositing!
• Cloud
Compositing Stereoscopy
Compositing Image decomposition
VR in the Cloud
• Temporary increase in power!
• Combine with compositing
Perceptive illusions
• Playing with limitations of human perception!
• Sensitivity of senses!
• Selectivity of attention!
• Useful to lower cost/complexity of VR systems
Perceptive illusions
(c) Franck Steinicke - Wurzburg University
Redirected walking
Change blindness :
(c) Evan Suma - USC/ICT
Perceptive illusionsChange blindness