Immersive Virtual Reality

Immersive Virtual RealitySébastien Kuntzhttp://cb.nowan.net/[email protected]

Last updateNovember 2012

Friday, November 16, 12

http://cb.nowan.net/blog


mailto:[email protected]?subject=Slideshare%20contact

mailto:[email protected]?subject=Slideshare%20contact

License

You are free:

• to Share — to copy, distribute and transmit the work

• to Remix — to adapt the work

• to make commercial use of the work

Under the following conditions:• Attribution — You must attribute the work in the

manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

• You must attribute “Immersive VR introduction” to (with this link):• Sébastien Kuntz - http://cb.nowan.net/blog

”Immersive Virtual Reality Introduction” by Sebastien Kuntz is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/

• With the understanding that:• Waiver — Any of the above conditions can be

waived if you get permission from the copyright holder.

• Public Domain — Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.

• Other Rights — In no way are any of the following rights affected by the license:◦ Your fair dealing or fair use rights, or other

applicable copyright exceptions and limitations;◦ The author's moral rights;◦ Rights other persons may have either in the

work itself or in how the work is used, such as publicity or privacy rights.

• Notice — For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page.




http://creativecommons.org/licenses/by-sa/3.0/




http://creativecommons.org/licenses/by/3.0/

http://creativecommons.org/licenses/by/3.0/

http://creativecommons.org/licenses/by/3.0/#

http://creativecommons.org/licenses/by/3.0/#

http://wiki.creativecommons.org/Public_domain

http://wiki.creativecommons.org/Public_domain

http://wiki.creativecommons.org/Frequently_Asked_Questions#Do_Creative_Commons_licenses_affect_fair_use.2C_fair_dealing_or_other_exceptions_to_copyright.3F

http://wiki.creativecommons.org/Frequently_Asked_Questions#Do_Creative_Commons_licenses_affect_fair_use.2C_fair_dealing_or_other_exceptions_to_copyright.3F

http://wiki.creativecommons.org/Frequently_Asked_Questions#I_don.E2.80.99t_like_the_way_a_person_has_used_my_work_in_a_derivative_work_or_included_it_in_a_collective_work.3B_what_can_I_do.3F

http://wiki.creativecommons.org/Frequently_Asked_Questions#I_don.E2.80.99t_like_the_way_a_person_has_used_my_work_in_a_derivative_work_or_included_it_in_a_collective_work.3B_what_can_I_do.3F

http://wiki.creativecommons.org/Frequently_Asked_Questions#When_are_publicity_rights_relevant.3F

http://wiki.creativecommons.org/Frequently_Asked_Questions#When_are_publicity_rights_relevant.3F

Plan

Definitions

VR Systems

Applications

Techniques


WHO AM I ?• Founder editor of

• http://www.imin-vr.com

• AFRV - French VR association: Board member - http://www.af-rv.fr

• VR Geeks association: President - http://www.vrgeeks.org

• Blog: http://cb.nowan.net/blog

• Twitter : @Cb_VRGeek

• 10 years in VR

• Virtools / Dassault Systèmes

• SNCF - French railwaysFriday, November 16, 12

http://www.imin

http://www.imin

http://www.afrv.fr

http://www.afrv.fr

http://www.vrgeeks.org

http://www.vrgeeks.org



What is VR for you ?


Immersive Virtual Reality

Presence in a virtual world

(c) eMagin(c) Barco


Immersive VR = iVR

iVR is the science and technology required for a user to feel present, via perceptive, cognitive

and functional immersion and interaction, in a (computer) generated environment.


Presence

Cognitive presence

Perceptive presence

Results

Natural interactions

Natural reactions


What is VR ?

Realtime 3D != VR

Stereoscopy != VR

3D Tracking != VR

Haptics != VR

3D Sound != VR

3D Interactions != VR

But everything combined can be VR


RidiculousIf it’s real VR, you probably

look ridiculous


Already changing world in industry and research

Democratization

Are we in the future ? No ! We’re still in prehistory !

WE can make a difference, invent systems, applications, businesses, interactions

Why now ?


Measure presence ?


Measure presence ?

Can’t objectively measure presence (qualia)

What you can measure is if people respond to this Virtual Environment as if it is real, which is a result of presence

Compare real-life behavior vs virtual

If avatar: brain zone activates when user accepts the avatar as real


Immersion parametersGraphics framerate

Extent of tracking

Tracking latency (measure?)

Quality of the images

Field of view

Plausibility of behaviors

Range of sensory modalities

... Friday, November 16, 12

Paul Milgram


VR Systems


Projector based VR Systems


CAVE

http://www.jamstec.go.jp/esc/research/Perception/vr.en.htmlFriday, November 16, 12

http://www.jamstec.go.jp/esc/research/Perception/vr.en.html

http://www.jamstec.go.jp/esc/research/Perception/vr.en.html

Cave

(c) Sébastien Kuntz


http://www.youtube.com/watch?v=Hb29bLHEjUgClarte - SAS Cube


http://www.youtube.com/watch?v=Hb29bLHEjUg



Immersia 3

(c) Sébastien KuntzFriday, November 16, 12

Immersia 3 - IRISA Rennes

http://www.youtube.com/watch?v=cTd5DLe8KD0




Calit2 - StarCave

http://www.scivee.tv/node/3648http://www.calit2.net/newsroom/release.php?id=1383


http://www.scivee.tv/node/3648

http://www.scivee.tv/node/3648

Projector basedReality center - HoloStage

(c) Arnaud MutheletFriday, November 16, 12

Projector based - Workbench

http://liris.cnrs.fr/~sbrandel/en/research/VR/LSIIT-IGG


http://liris.cnrs.fr/~sbrandel/en/research/VR/




Projector based

Workbench © PSA Peugeot Citroën Mediatheque.


http://mediatheque.psa.fr/

http://mediatheque.psa.fr/

Clarte R-Screen

http://www.youtube.com/watch?v=AcV5ma5nEVgFriday, November 16, 12

http://www.youtube.com/watch?v=AcV5ma5nEVg

http://www.youtube.com/watch?v=AcV5ma5nEVg

Panoscope 360°

(c) MatsushitaFriday, November 16, 12

La GéodeFriday, November 16, 12

jDome

http://www.youtube.com/watch?v=rNsFSVhZi9wFriday, November 16, 12

http://www.youtube.com/watch?v=rNsFSVhZi9w

http://www.youtube.com/watch?v=rNsFSVhZi9w

ToobImmersive dome

http://youtu.be/N6LiLaNOHHU




Barco - RP-360

• 13 projectors

• Diamètre: 3.4m


Barco - RP-360

http://www.youtube.com/watch?v=3EbrQwpOx24




6 users: stereo + tracking

http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.html

Weimar University (DE)Friday, November 16, 12





6 usersstereoscopy + tracking

• video





Screen based

Full-scale NexCAVE developed and deployed by Calit2 for Saudi Arabia's King Abdullah University of Science and Technology - http://www.calit2.net/newsroom/release.php?id=1743


http://www.calit2.net/newsroom/release.php?id=1743

http://www.calit2.net/newsroom/release.php?id=1743

Ultra-Short throw projectors

• Sanyo PDG-DWL2500

• 1280x800

• 2500 lumens

• Canon LV-8235 UST


Ultra-Short throw projectors

http://www.youtube.com/watch?v=VSblSrTRJh0




TV-3D - 154”

(c) Sébastien Kuntz


Projector based VR Systems(-)

(Very) expensive !

Lots of space ! ( front and behind, AC... )

Not 360°

Cluster hard to program !

(+)

+/- Collaborative

Not much to wear

High resolution, big FOV


HMDHead mounted displays


Vuzix•640x480•32° FOV

•852x480•32° FOV


eMagin Z800

800x600

40° FOV


Sony

HMZ-T1

1280x1024

45°


Sensics

piSight

Multiple screens

zSight

1280x1024

70° FOV


NVIS

nVisor SX60

1280x1024

60° FOV

nVisor SX111

1280x1024

111° FOV


Fakespace Labs

Wide5

150° horizontal FOV

1600x1200

Progressive


• Developer kit

• 90° horizontal FOV

• 1280x800

• 640x800 per eye

OCULUS RIFT


HMD(-)

Low resolution, low FOV

Must wear, adjust

No peripheral vision

(+)

Lightweight

Compact

Easier to program

360° tracking


SummaryProjector/screen based

Cave, holostage, workbench

Collaborative, high FOV

Expensive, Complex

HMD

360°

“Cheap”

Low Fov/resolution


Depth perception


Depth perception

How do you perceive depth ?


Stereoscopy ?

http://www.youtube.com/watch?v=Jd3-eiid-UwJohnny Lee - Head Tracking with a wiimote


http://www.youtube.com/watch?v=Jd3-eiid-Uw




Stereoscopy ?

http://www.youtube.com/watch?v=nRYybo2WNwU

Esiea Virtual ShowcaseFriday, November 16, 12





Stereoscopy

You perceived depth without stereoscopy !


Depth perceptionStereoscopic cues :

eye accomodation

eye convergence angle

disparity between left and right images

Monoscopic cues :

perspective

atmospheric artifacts (fog)

relative sizes

occlusion

parallax

shadows

Parallax can be more important for depth perception!Stereoscopy is important for size and distance

evaluationFriday, November 16, 12

Stereoscopy



Stereoscopy

• Useful for

• Relative distance perception

• Detection of objects

• Discrimination between two objects

• Localisation

• But...

• Fusing problems

• Headaches

• Visual fatigue

• Accommodation / Vergence dissociation


Different stereoscopic separation schemes

Active shutter glasses

Polarized glasses

Anaglyph

Infitec

Autostereoscopic

Holograms


Active stereoscopy


Passive Stereoscopy

Circular PolarizationLinear Polarization


Anaglyph


Infitec

- Eye separation by color spectrum -


Light Efficiency

http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdfFriday, November 16, 12

http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdf

http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdf

Autostereoscopic


Holograms

Stores ALL light interactions through an entire “window”


Holograms


Stereoscopy for multiple users ?


Stereoscopy for multiple users ?

Active + polarized

Sequential


6 users: stereo + tracking







6 usersstereoscopy + tracking

• video





Multi-display


What system can we create with only one graphics card?


What can we do with one graphics card?


Nvidia SLI - ATI CrossfireUse two graphics card for output on only one graphics cards

Alternate frame rendering

Split frame rendering


Scalable graphicsImage decomposition ( ~= SLI )Useful if application is fill-rate limited

frame rate decreases with increasing number of pixels (i.e. when the image resolution increases)


Scalable graphicsDatabase decompositionUseful if application is CPU limited

frame rate decreases with increasing size of 3D scenes


Scalable graphicsStereoscopy


How many displays with one computer ?


Why multi-display ?

Multiple display surfaces

Cave

Increase display size

Increase brightness

CPU/GPU Bottlenecks

Cluster


How ?Matrox DualHead2Go

Matrox TripleHead2Go

Multiple graphics card in a single PC

Often max 2 graphics cards

Cyviz


Multiple graphics cards in one PC

Multi-pipe ~= Multi-GPU

One rendering context per GPU

Problems of resources sharing, threading

Need specific graphics engine adaptations

But best performances


NVidia Quadro Plex•Up to :•4 GPU•4 outputs•16gb memory

•Can combine two Quadro Plex


Quadro Mosaic ModeMosaic mode hides multi-gpu

Only shows one gpu to system

Can have 4 outputs with 1 plex without needing multi-pipe adaptation

Good performances


ATI EyeFinity


Increase size or brightnessMersive autocalibration


What if need more displays ?

96 projectors ?


Cluster !


Cluster Synchronisation

Scene synchronisation (Scenelock)

Swap-Buffer synchronisation (Swaplock)

Eyes synchronisation (Genlock)


Cluster

+

Scalable

Each node can be cheap

Easy node replacement

-

Complex to manage/program


Tracking


TrackingMechanical

Fakespace’s Boom

+

Precise

Fast

Help Lift weight

-

Big, discontinued


TrackingMagnetic

Polhemus

Ascension

+

Precise, fast

-

Wires

Sensitive to metal


TrackingInertial

SpacePoint Fusion

XSens Mocap

Accelerometers, Gyroscopes, magnetometers

+

Very fast

Light

-

Drift

Orientation onlyFriday, November 16, 12

TrackingOptical

ART

ioTracker

OptiTrack

TrackIR

+

Wireless

No electronics

-

Install cameras

Line of sightFriday, November 16, 12

Leap motion

https://www.youtube.com/watch?v=N6hCwjwzUHgFriday, November 16, 12

https://www.youtube.com/watch?v=N6hCwjwzUHg

https://www.youtube.com/watch?v=N6hCwjwzUHg

TrackingMixing Optical

Intersense IS-900

Inertial + Ultrasound

IS-1200

Inertial + AR Patterns

Wiimote

Playstation Move


Optical Body Tracking

4D Views

http://www.4dviews.com/


http://www.4dviews.com

http://www.4dviews.com


Kinect fusion

http://www.youtube.com/watch?v=quGhaggn3cQ




Gloves


Haptics


HapticsForce feedback






TN Games - Gaming Vest


Locomotion


Locomotion

CirculaFloor

• http://www.youtube.com/watch?v=rYsvB2y2Ero

Virtusphere

• http://www.youtube.com/watch?v=qTnnJR-hS7k

OmniDirectionnal Treadmill

• http://www.youtube.com/watch?v=BQw1tsgrJOsFriday, November 16, 12

http://www.youtube.com/watch?v=rYsvB2y2Ero

http://www.youtube.com/watch?v=rYsvB2y2Ero

http://www.youtube.com/watch?v=qTnnJR-hS7k

http://www.youtube.com/watch?v=qTnnJR-hS7k

http://www.youtube.com/watch?v=BQw1tsgrJOs

http://www.youtube.com/watch?v=BQw1tsgrJOs

Kuka robot

http://www.cyberneum.de/f1/Friday, November 16, 12

http://www.cyberneum.de/f1/

http://www.cyberneum.de/f1/

Force Dynamics

http://www.force-dynamics.com/Friday, November 16, 12

http://www.force-dynamics.com

http://www.force-dynamics.com

Wizdish

http://www.wizdish.com




Applications


IVR Applications

Design, prototyping

Marketing

Training

Medical

Research

Geoscience, finance, archeologie, urbanism, sport, art...


iVR ApplicationsDesign, prototyping

Car, aircraft, boat, tractors, house appliances

Collaborative design review

Experience from user’s point of view

Easy and fast to prototype

More iterations, better products

(c) Peugeot PSA

(c) MieleFriday, November 16, 12

PSA Peugeot-Citroen - http://www.youtube.com/watch?v=WhMh191RltQ


http://www.youtube.com/watch?v=WhMh191RltQ

http://www.youtube.com/watch?v=WhMh191RltQ

FORD

http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/Friday, November 16, 12

http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/

http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/

VR4D

http://vr4d.ingenierium.com/Friday, November 16, 12

http://vr4d.ingenierium.com

http://vr4d.ingenierium.com

Ergo-Wide

http://www.clarte.asso.fr/realite-virtuelle.php/en/ERGOWIDE%C2%B3/https://www.youtube.com/watch?v=aczLiNtVyDI


https://www.youtube.com/watch?v=aczLiNtVyDI




Lockheed Martin

http://www.youtube.com/watch?v=itOxIpj68MwFriday, November 16, 12

http://www.youtube.com/watch?v=itOxIpj68Mw

http://www.youtube.com/watch?v=itOxIpj68Mw

Mockup-builder

http://www.youtube.com/watch?v=ZxJD9DXDB1EFriday, November 16, 12

http://www.youtube.com/watch?v=ZxJD9DXDB1E

http://www.youtube.com/watch?v=ZxJD9DXDB1E

iVR ApplicationsMarketing

Product experience

Shopping experience

Procter&Gamble

Kimberley Clarke

(c) University of Gronigen

(c) Virtools


iVR ApplicationsTraining

Repeatability

Controlled environment

Can practice rare / dangerous situations

Can analyze / quantify performance

No health risk for patients

Easy accessibility

Reduced costs

Adaptable difficulty and assistance

Multiple users trained at same time

Attractive

(c) SNCF

(c) University of Florida


SNCF - Fiacre


SNCF - Simurat


http://www.youtube.com/watch?v=FEnalzHVmdI

Saint-Gobain & Clarte

Use less material

Environment friendly

No water, nothing to clean

Faster to setup




RE-lion

http://vimeo.com/25032061Friday, November 16, 12

http://vimeo.com/25032061


ESCIN - Sivic - Crime scene investigation

http://www.youtube.com/watch?v=FC0gHv-Snmw


iVR ApplicationsResearch

Heal

Exposure therapy

PTSD, Phobias

Rehabilitation : motor, addictions

Create Empathy

Pain Distractionhttp://www.youtube.com/watch?v=SinlgK5Q4v4

Crescent - Acrophobia


Burn pain - Snow World

http://www.youtube.com/watch?v=jNIqyyypojgFriday, November 16, 12

Fear of flight

Arte - “A chacun sa réalité”


Summary

IVR strengths

Controlled / adaptable / safe environment

Natural interactions / reactions

Analyze


iVR Games ?

Cave Quake 3 (c) Paul Rajlich

Cave Unreal Tournament(c) Jeffrey Jacobson


https://www.youtube.com/watch?v=r580mzSxSp8





Immersive rail shooter




VR Angry-bot

http://www.youtube.com/watch?v=QSXxjr1SnqAFriday, November 16, 12

http://www.youtube.com/watch?v=QSXxjr1SnqA

http://www.youtube.com/watch?v=QSXxjr1SnqA

Battlefield 3 simulator

www.youtube.com/watch?v=nQR49JGySTM


http://www.youtube.com/watch?v=nQR49JGySTM

http://www.youtube.com/watch?v=nQR49JGySTM

3D User Interactions


VR Techniques3D User Interactions

Bowman classification for generic interactionsSelection

Specifying one or more objects from a setManipulation

Modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.)Navigation

TravelWayfinding

System ControlMenusIssuing commandChanging modeChoosing tool

Symbolic InputOther interactions

Measuring… ? http://www.3dui.org


http://www.3dui.org

http://www.3dui.org

Navidget

http://www.youtube.com/watch?v=wpaKjLZll80Friday, November 16, 12

http://www.youtube.com/watch?v=wpaKjLZll80

http://www.youtube.com/watch?v=wpaKjLZll80


http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/










World builder

http://www.youtube.com/watch?v=VzFpg271sm8Friday, November 16, 12

http://www.youtube.com/watch?v=VzFpg271sm8

http://www.youtube.com/watch?v=VzFpg271sm8

Perceptive illusions


Body representation in VR

Proteus effect : virtual appearance can transform our behaviors towards other people ( Jeremy Bailenson )

It seems possible to transfer our body in virtual bodies


Rubber hand illusion

http://www.youtube.com/watch?v=TCQbygjG0RUFriday, November 16, 12

http://www.youtube.com/watch?v=TCQbygjG0RU

http://www.youtube.com/watch?v=TCQbygjG0RU

Body transfer

http://www.youtube.com/watch?v=8oF8sQvnTlMFriday, November 16, 12

http://www.youtube.com/watch?v=8oF8sQvnTlM

http://www.youtube.com/watch?v=8oF8sQvnTlM

Barbie doll illusion

http://www.youtube.com/watch?v=_z8pSTMfGSoFriday, November 16, 12

http://www.youtube.com/watch?v=_z8pSTMfGSo

http://www.youtube.com/watch?v=_z8pSTMfGSo

First person VR illusion

http://www.youtube.com/watch?v=DlbSVnVRLTUFriday, November 16, 12

http://www.youtube.com/watch?v=DlbSVnVRLTU

http://www.youtube.com/watch?v=DlbSVnVRLTU

Perceptive IlusionsPseudo haptics

http://www.irisa.fr/tactiles

http://kaji-lab.jp/ja/index.php?research






Perceptive IlusionsPseudo haptics


Perceptive IllusionsPlay with perception’s limits

Our perception is dominated by visual sense

Balance

Redirected Walking :

(c) Franck SteinickeFriday, November 16, 12


Change blindness :

(c) Evan Suma - USC/ICTFriday, November 16, 12

https://www.youtube.com/watch?v=FTWVjJ4um4A

Institute of creative technologies - change blindess





Perceptive IllusionsPassive haptics

(c) Luv KohliFriday, November 16, 12

Passive haptics

Ford PVM



Learn about human perception through its limitations

Improve VR experience by taking advantage of those limitations


Conclusion

It is possible to transfer your body in VR

Reality is not what you think

VR is just another reality


http://www.af-rv.frFriday, November 16, 12

http://www.af-rv.fr

http://www.af-rv.fr

AFRV

http://www.youtube.com/watch?v=lHZltjDZBJo&feature=plcp




www.VRGeeks.orgFriday, November 16, 12

http://www.VRGeeks.org






http://www.vrgeeks.org/vr-experiences/vrescapeFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/vrescape

http://www.vrgeeks.org/vr-experiences/vrescape

http://www.vrgeeks.org/vr-experiences/vrsadisticFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/vrsadistic

http://www.vrgeeks.org/vr-experiences/vrsadistic

http://www.vrgeeks.org/vr-experiences/insnaketionFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/insnaketion

http://www.vrgeeks.org/vr-experiences/insnaketion


VR ChallengesWhat we must solve

VR is mainly a SOFTWARE problem

“3DUI is a hard and unexplored software engineering (and ergonomic) problem”

Markerless tracking

Minimum latency

HMDs

Lightweight

Wide FOV

Wireless

Full body haptics

Multi-sensorial experience

Locomotion

Avatars

VR For AllFriday, November 16, 12

VR “Experiences”

Your goal :

Create presence

And don’t break it !!!

Verdun 1916-Time machinehttp://www.youtube.com/watch?v=ZBfYIefenp0


http://www.youtube.com/watch?v=ZBfYIefenp0

http://www.youtube.com/watch?v=ZBfYIefenp0

Conclusion


Thank you !Questions ?

[email protected]


mailto:[email protected]

mailto:[email protected]

Documents

Immersive Virtual Reality