COMP 4010 - Lecture4 VR Technology - Visual and Haptic Displays

LECTURE 4: VR TECHNOLOGY – VISUAL AND HAPTIC DISPLAYS

COMP 4010 – Virtual Reality Semester 5 - 2016

Mark Billinghurst, Bruce Thomas University of South Australia

August 16th 2016

Virtual Reality Technology

Using Technology to Stimulate Senses • Simulate output

•  E.g. simulate real scene

• Map output to devices •  Graphics to HMD

• Use devices to stimulate the senses •  HMD stimulates eyes

Visual Simulation

3D Graphics

HMD

Vision System

Brain

Example: Visual Simulation

Human-Machine Interface

Key Technologies for VR Systems • Visual Display

• Stimulate visual sense

• Audio/Tactile Display • Stimulate hearing/touch

• Tracking • Changing viewpoint • User input

•  Input Devices • Supporting user interaction

Mapping Between Input and Output

Input

Output

VISUAL DISPLAY

Creating an Immersive Experience

• Head Mounted Display • Immerse the eyes

• Projection/Large Screen • Immerse the head/body

• Future Technologies • Neural implants • Contact lens displays, etc

HMD Basic Principles

• Use display with optics to create illusion of virtual screen

Key Properties of HMDs • Lens

•  Focal length, Field of View • Occularity, Interpupillary distance • Eye relief, Eye box

• Display • Resolution, contrast • Power, brightness • Refresh rate

• Ergonomics • Size, weight • Wearability

Simple Magnifier HMD Design

p

q

Eyepiece (one or more lenses) Display

(Image Source)

Eye f Image

1/p + 1/q = 1/f where p = object distance (distance from image source to eyepiece) q = image distance (distance of image from the lens) f = focal length of the lens

Virtual Image

Lens Display

Virtual Image

Focal Length and Diopter

Focal Length - The distance from the surface of a lens at which rays of light converge.

Diopter - The power of a lens. Equal to 1/(focal length of the lens measured in meters)

Field of View

Monocular FOV is the angular subtense (usually expressed in degrees) of the displayed image as measured from the pupil of one eye. Total FOV is the total angular size of the displayed image visible to both eyes.

Binocular(or stereoscopic) FOV refers to the part of the displayed image visible to both eyes.

FOV may be measured horizontally, vertically or diagonally.

Ocularity • Monocular - HMD image goes to only one eye.

• Bioccular - Same HMD image to both eyes.

• Binocular (stereoscopic) - Different but matched images to each eye.

Interpupillary Distance (IPD)

! IPD is the horizontal distance between a user's eyes.

! IPD is the distance between the two optical axes in a binocular view system.

Vignetting and Eye Relief

Vignetting • The blocking or redirecting of light rays as they pass through the optical system.

Eye Relief Distance • Distance from the last optical surface in the HMD optical system to the front surface of the eye.

LEEP Optics

• Large Expanse Extra Perspective

• Very wide FOV for stereoscopic images • Higher resolution in the middle of FOV

•  Lower resolution on the periphery

• Pincushion distortion

LEEP Optics

• Wide field of view optical design

Fresnel Lens

• A lens consisting of a concentric series of simple lens sections

• Result is a thin lens with a short focal length and large diameter

• More even resolution distribution

•  Less distortion •  from lanternroom.com

Relationship bet. angle and screen distance

0.0010.0020.0030.0040.0050.0060.0070.0080.00

R2.006.0010.00

14.00

18.00

22.00

26.00

30.00

34.00

Angle in Radians

Dist

ance

in m

m

LeepFresnel

Distortion in Lens Optics

A rectangle Maps to this

Example Distortion

Oculus Rift DK2 HTC Vive

To Correct for Distortion

• Must predistort image • This is a pixel-based distortion

• Graphics rendering uses linear interpolation!

• Too slow on most systems • Use shader programming

HMD Design Trade-offs

• Resolution vs field of view • As FOV increases, resolution decreases for fixed pixels

• Eye box vs field of view •  Larger eye box limits field of view

• Size, Weight and Power vs everything else

vs.

The Perfect HMD •  “Oakley look” . i.e., thin & small optics •  Low cost & small image generators (OLED, LCOS, …) • Wide field of view 30oto 110o full diagonal field •  Large eye box ~10 mm diameter, for eye ball movement •  Large eye relief > 20 mm, for lash clearance and glasses • High resolution ~ SXGA (1280 x 1024) or higher •  Low distortion < 2% • Bright hundreds of Cd/m2 • Artifact free; no “dirty windows” ; no raster; no sunlight scattering •  Low weight • Other: eye tracking; battery life; connectivity….

Oculus Rift • Cost: $599 USD •  FOV: 110o Horizontal • Refresh rate: 90 Hz • Resolution 1080x1200/eye •  3 DOF orientation tracking •  3 axis positional tracking

Inside an Oculus Rift

HTC Vive

29

Sony Playstation VR

30

https://www.youtube.com/watch?v=Lulzi9LmNSM

Comparison Between HMDs

Computer Based vs. Mobile VR Displays

•  dsfsaf

Google Cardboard

• Released 2014 (Google 20% project) • >5 million shipped/given away • Easy to use developer tools

+ =

Version 1.0 vs Version 2.0

• Version 1.0 – Android focused, magnetic switch, small phone • Version 2.0 – Touch input, iOS/Android, fits many phones

Many Different Cardboard Viewers

Edible Google Cardboard

37

https://www.youtube.com/watch?v=beSfGrM7OXY

Gear VR

Multiple Mobile VR Viewers Available

•  zxcvz

Projection/Large Display Technologies • Room Scale Projection

• CAVE, multi-wall environment • Dome projection

• Hemisphere/spherical display • Head/body inside

• Vehicle Simulator • Simulated visual display in windows

CAVE

• Developed in 1992, EVL University of Illinois Chicago • Multi-walled stereo projection environment

•  Head tracked active stereo

Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., & Hart, J. C. (1992). The CAVE: audio visual experience automatic virtual environment. Communications of the ACM, 35(6), 64-73.

Typical CAVE Setup

•  4 sides, rear projected stereo images

Demo Video – Wisconsin CAVE

https://www.youtube.com/watch?v=mBs-OGDoPDY

CAVE Variations

Stereo Projection • Active Stereo

• Active shutter glasses •  Time synced signal • Brighter images • More expensive

• Passive Stereo • Polarized images •  Two projectors (one/eye) • Cheap glasses (powerless) •  Lower resolution/dimmer •  Less expensive

Caterpillar Demo

https://www.youtube.com/watch?v=r9N1w8PmD1E

Multi-User CAVEs

•  Limitation of CAVEs •  Stereo projection from only one user’s viewpoint

• Solution •  Higher frequency projectors and time slicing

Kulik, A., Kunert, A., Beck, S., Reichel, R., Blach, R., Zink, A., & Froehlich, B. (2011). C1x6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. ACM Transactions on Graphics (TOG), 30(6), 188.

Multiuser Demo

https://www.uni-weimar.de/de/medien/professuren/vr/research/multi-user-virtual-reality/c1x6-a-stereoscopic-six-user-display/

Walt Disney Imagineering’s Digital Immersive Showroom (DISH)

Technology " Large working volume " 10.5m x 7.5m x 4.0m

" 360 Surround Projection " Front projection " Five 4K @ 120 Hz 3D projectors " One 2K @ 120 Hz 3D projectors

" Complex screen geometry " Rounded corners " Overhanging ceiling

" Tech Viz, Unreal, Panda3D

DISH Demo

https://www.youtube.com/watch?v=70hCn9PguI4

Allosphere • Univ. California Santa Barbara

•  One of a kind facility

•  Immersive Spherical display •  10 m diameter •  Inside 3 story anechoic cube •  Passive stereoscopic projection •  26 projectors •  Visual tracking system for input

•  See http://www.allosphere.ucsb.edu/

Kuchera-Morin, J., Wright, M., Wakefield, G., Roberts, C., Adderton, D., Sajadi, B., ... & Majumder, A. (2014). Immersive full-surround multi-user system design. Computers & Graphics, 40, 10-21.

Allosphere Demo

https://www.youtube.com/watch?v=25Ch8eE0vJg

Allosphere Research • Multi-disciplinary research

•  Science, art, engineering

•  Typical research projects •  Brain imaging

•  fMRI imaging data •  Atomic bonding

•  Bond simulation models •  Nano medicine

•  Simulate chemotherapy •  Graph browser

•  Mathematical visualization •  Etc

Brain Imaging

Hydrogen bond

Nano Medicine

Vehicle Simulators • Combine VR displays with vehicle

•  Visual displays on windows •  Motion base for haptic feedback •  Audio feedback

• Physical vehicle controls •  Steering wheel, flight stick, etc

•  Full vehicle simulation •  Emergencies, normal operation, etc •  Weapon operation •  Training scenarios

Lexus Driving Simulator

Tank Turret Simulator

Demo: Boeing 787 Simulator

https://www.youtube.com/watch?v=3iah-blsw_U

HAPTIC/TACTILE DISPLAYS

Haptic Feedback •  Greatly improves realism •  When is it needed?

•  Other cues occluded/obstructed •  Required for task performance •  High bandwidth!

•  Hands and wrist are most important •  High density of touch receptors

•  Two kinds of feedback •  Touch Feedback

•  information on texture, temperature, etc. •  Does not resist user contact

•  Force Feedback •  information on weight, and inertia. •  Actively resists contact motion

Haptic Devices • Pin arrays for the finger(s) •  Force-feedback "arms" •  "Pager" motors • Particle brakes • Passive haptics • Many devices are application specific

•  Like surgical devices

Active Haptics

• Actively resists contact motion • Dimensions?

•  Force resistance •  Frequency Response • Degrees of Freedom •  Latency •  Intrusiveness • Safety • Comfort • Portability

Force Feedback Joysticks

• WingMan Force 3D •  Inexpensive ($60) • Actuators that can move the joystick given system commands

• Max 3.3 N of force • Force feedback driving wheel

Sensable Phantom

• Combined stylus input/haptic output •  6 DOF haptic feedback

Phantom Omni Demo

https://www.youtube.com/watch?v=REA97hRX0WQ

Immersion Cybergrasp

• Haptic feedback on Glove • Combined with glove input

CyberMotion (MPI Tübingen)

68

https://www.youtube.com/watch?v=GhnIK1PILig

CableRobot (MPI Tübingen)

69

https://www.youtube.com/watch?v=cJCsomGwdk0

Haptic Feedback in VR • Virtual contact

•  What should we do when we know that contact has been made with a virtual object?

•  The output of collision detection is the input to virtual contact •  Cues for understanding the nature of contact with objects are

typically over-simplified (e.g., sound) •  Training aids

•  Can we convey additional information using the haptic channel?

Passive Haptics

• Not controlled by system • Pros

• Cheap •  Large scale • Accurate

• Cons • Not dynamic •  Limited use

UNC Being There Project

The Void- https://thevoid.com/

• Passive haptic environment • Warehouse scale VR • Wearable VR systems • Graphics overlaid on real props

The Void Demo

https://www.youtube.com/watch?v=cML814JD09g

Passive Haptic Paddle

• Using physical props to provide haptic feedback •  http://www.cs.wpi.edu/~gogo/hive/

Tactile Feedback Interfaces • Goal: Stimulate skin tactile receptors • How?

•  Air bellows •  Jets •  Actuators (commercial) •  Micropin arrays •  Electrical (research) •  Neuromuscular stimulations (research)

Vibrotactile Cueing Devices • Vibrotactile feedback has been incorporated into many

devices • Can we use this technology to provide scalable, wearable

touch cues?

Tactile Mouse

• Logitch iFeel Mouse • Electrical Actuator

• Shakes up and down (do not disturb XY motion)

• Mouse over buttons • Haptic Bump • Rumble Pack

Vibrotactile Feedback Projects

Navy TSAS Project

TactaBoard and TactaVest

CyberTouch Glove •  Immersion Corporation

• Expensive - $15000 • Six Vibrotactile actuators

• Back of finger • Palm

• Off-centered actuator motor • Rotation speed=frequency of

vibration (0-125 Hz) • When tracked virtual hand intersects with virtual object, send signal to glove to vibrate

www.empathiccomputing.org

@marknb00

mark.billinghurst@unisa.edu.au

bruce.thomas@unisa.edu.au