Virtual Reality II

Visual Imaging in the Electronic Age

Donald P. GreenbergNovember 22, 2016

Lecture #26

Oculus Rift DK2 2014

Oculus Rift 2014

Oculus Rift Distorted Image

Steam VR 2015

Crescent Bay 2015

Google Daydream 2016

HTC and Valve’s SteamVR Vive 2016

Gear VR (Oculus) 2016

FOVE 2016

• WQHD OLED (2560 X 1440)

• Frame rate: 70fps

• Field of view: Up to 100 degrees

• Infrared eye tracking system x 2

• Tracking accuracy: less than 1

degree

• Frame rate: 120fps

Necessary Improvements

• Not currently possible to manufacture required pixel densities

• Higher resolution is required in the foveal region

• Reduction in bandwidth (High resolution at the periphery is not necessary)

• Handling data explosion

Increasing Densities (ppi) of OLED Displays

Need for Foveal Tracking

• Not currently possible to manufacture required pixel densities

• Higher resolution is required in the foveal region

• Reduction in bandwidth (High resolution at the periphery is not necessary)

• Handling data explosion

Retina Cells

Foveal Eye Tracking Constraints 2016

• Speed- needs to be fast enough to meet update requirements

(currently 11 milliseconds, 90 Hz)

• Accuracy- Gaze direction is < 0.5 degree

• Foveal direction accuracy can be ~ 1.0 arc minutes (1/60 of a degree)

• Non-invasive measurements- still need to see entire visual field

Modern: Videooculography

● Background:

○ Varying densities of cornea, lens, air,

fluid within eye

○ Thus, there are different refractive

indices between them

○ Cause four unique reflections

■ Refracted reflections

■ Called Purkinje images,

○ Each is more offset than the previous

Sensing Methods: Retinal Tracking

● Hard problem with current technologies○ Extremely difficult to illuminate

■ Must bounce light off of retina

■ Light comes back through iris

○ Light must be extremely bright

○ Too much exposure will damage retina

● Typically done in ophthalmological

setting

● Presently can only detect faint images of

blood vessels, companies working on it

● Very high angular resolution, but would

presently require occlusion of vision

Modern: Videooculography

● Method:

○ Vector between the pupil and any given

reflection can be measured

○ From vector, arbitrary transformation

■ Screen space for 2D screens

■ Angular space for Gaze

○ Multiple Purkinje images can be used

■ Doubles the measurement

frequency per camera

■ Accuracy & precision improved

Purkinje Reflections

Cornsweet and Crane 1973

Purkinje Reflections

1st and 4th Purkinje Reflections

Bandwidth Reduction

• Not currently possible to manufacture required pixel densities

• Higher resolution is required in the foveal region

• Reduction in bandwidth (High resolution at the periphery is not necessary)

• Handling data explosion

Research on Foveated Displays

No Foveated Rendering

Roadtovr.com

Roadtovr.com

Contemporary ‘blur’ foveated rendering

Roadtovr.com

NVIDIA’s ‘contrast preserving’ rendering

Bandwidth Reduction

• Not currently possible to manufacture required pixel densities

• Higher resolution is required in the foveal region

• Reduction in bandwidth (High resolution at the periphery is not necessary)

• Handling data explosion

ReMake Autodesk 2016

Yang

LeeGibson

Google Earth November 2016

Google Earth November 2016

“Crossing the Chasm”

• Cheap Devices and Mass Consumers

• Taking Advantage of No Information Transfer (Latency)

• Increase in Processing Power

• New Methods of Storytelling

Google Cardboard

Google Cardboard

Pokemon Go

Pokémon Go

“Crossing the Chasm”

• Cheap Devices and Mass Consumers

• Taking Advantage of No Information Transfer (Latency)

• Increase in Processing Power

• New Methods of Storytelling

The eye jumps, comes to rest momentarily (producing a small dot on

the record), then jumps to a new locus of interest.

- David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

Saccadic Motion

Saccadic Motion Early Yarbus Experiments

Wikipedia

“Crossing the Chasm”

• Cheap Devices and Mass Consumers

• Taking Advantage of No Information Transfer (Latency)

• Increase in Processing Power

• New Methods of Storytelling

Offline Rendering vs. Online Rendering

Toy Story 3

John Lehrer. “How It’s Done,” Wired 18.06. http://www.wired.com/magazine/18-06

“Chip density doubles every 18 months.”

Processing Power (P) in 15 years:

𝑃 = 𝑃𝑡𝑜𝑑𝑎𝑦 215𝑦𝑒𝑎𝑟𝑠18𝑚𝑜𝑛𝑡ℎ𝑠 = 𝑃𝑡 2

151.5

= 𝑃𝑡 210 = 1000𝑃𝑡

Moore’s Law

Processing Power Compared

• 2015: iPhone 5 > 1985 Cray-2

(2.7x)

“Crossing the Chasm”

• Cheap Devices and Mass Consumers

• Taking Advantage of No Information Transfer

• Increased Processing Power

• New Methods of Storytelling

Saliency

Spotlight Stories

Google Spotlight, Pearl

https://www.youtube.com/watch?v=WqCH4DNQBUA

1972: Cornell in Perspective

History of Computer Graphics

1967: General Electric

1974: Scientific American Cover

2014: Kitchen Scene

50 Years

History of Virtual Reality

1968: Ivan Sutherland

2013: Google Glass

1990s: HMDs, Henry Fuchs

2016: HTC & Valve’s SteamVR Vive

25 Years

End