Upload
dinhxuyen
View
214
Download
2
Embed Size (px)
Citation preview
Virtual Reality ll
Visual Imaging in the Electronic Age
Donald P. GreenbergNovember 16, 2017
Lecture #22
Fundamentals of Human Perception• Retina, Rods & Cones, Physiology
• Receptive Fields
• Field of View
• Visual Acuity of Resolution
• Opponent Color Theory
• Compression
• Bandwidth Limitations
• Saccades
The Optomotor Cycle
Extraocular Muscles
“Foundations of Sensation and Perception.” Mather, George. 2009.
Saccade Control• Saccade control is the ability of the eye(s) to move quickly from one fixation
point to another (100-300 ms)
• To obtain a complete picture, normal adults perform 3-5 saccades (“snapshots”) per second
• Fixation “restops” are ≈ 50-100 ms
The eye jumps, comes to rest momentarily (producing a small dot on the record), then jumps to a new locus of interest.
- David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.
Saccadic Motion
Saccadic Masking• Visual saccadic suppression
• The brain selectively blocks visual processing during eye movements
• Neither the motion of the eye or subsequent motion blur of the image nor the time gap in visual perception is noticeable to the viewer
Saccadic Masking• There are two major types of saccadic masking or suppression
• Flash suppression is the inability of the light to see a flash of light during a saccade
• Suppression of image displacement is characterized by the inability to perceive whether a target has moved during a saccade.
Peak Angular Velocity
Wikipedia
Human Depth Perception
Depth Perception
Oculomotor
Binocular
Convergence
Monocular
Accommodation
Visual
Binocular
Stereopsis
Monocular
Static Cues
PerspectiveFamiliarity,
Relative Size
Motion, Position Occlusion Texture
GradientShading, Shadows, Highlights
Atmospheric Blur
Motion Parallax
Monocular Human Depth Perception
Depth Perception
Oculomotor
Binocular
Convergence
Monocular
Accommodation
Visual
Binocular
Stereopsis
Monocular
Static Cues
PerspectiveFamiliarity,
Relative Size
Motion, Position Occlusion Texture
GradientShading, Shadows, Highlights
Atmospheric Blur
Motion Parallax
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and Specular Highlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and SpecularHighlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and SpecularHighlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and SpecularHighlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shading, Shadows, and Specular Highlights
• Atmospheric Blur Viewpoint A Viewpoint B Viewpoint C
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shading, Shadows, and Specular Highlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shading, Shadows, and Specular Highlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and SpecularHighlights
• Atmospheric Blur
Monoscopic Depth Cues• Perspective
• Depth from Motion, Relative Size, Position, Familiarity
• Occlusion
• Texture Gradient
• Parallax from Motion
• Shadows and SpecularHighlights
• Atmospheric Blur
• AccommodationNote change in lens shape
Accommodation
• This is the process by which the vertebrate eye changes optical power to maintain a clear image or focus on an object as its distance varies.
Accommodation
The reflex can be controlled but cannot be ‘felt’Accommodation amplitude declines with age
Vergence
• The simultaneous movement of the pupils of the eyes toward or away from one another during focusing.
• This measure of the convergence or divergence of a pair of light rays is defined as vergence.
Vergence Accommodation Conflict
Human Depth Perception
Depth Perception
Oculomotor
Binocular
Convergence
Monocular
Accommodation
Visual
Binocular
Stereopsis
Monocular
Static Cues
PerspectiveFamiliarity,
Relative Size
Motion, Position Occlusion Texture
GradientShading, Shadows, Highlights
Atmospheric Blur
Motion Parallax
Binocular Vision
• Binocular Vision, which is the basis for stereopsis is important for depth perception and covers 114 degrees(horizontally) of the human visual field.
• The remaining sixty to seventy degrees have no binocular vision (because only one eye can see those portions of thevisual field)
Stereoscopic Depth Cues
• Stereopsis
– Horizontal Parallax– Occlusion Revelation
• Shape Change
• Convergence
Stereoscopic Depth Cues
• Stereopsis
• Shape Change
– Standard Stereo– HypoStereo (Gigantism)– HyperStereo (Dwarfism)
• Convergence
Standard
HypoStereo
HyperStereo
Stereoscopic Depth Cues
• Stereopsis
• Shape Change
• Convergence
– Maintain single binocular vision
– Fusion
“Vision and Visual Disabilities – An Introduction,” by Gerd Waloszek, SAP User Experience
Moore’s Law
“Chip density doubles every 18 months.”
Processing Power (P) in 15 years:
Exponential Laws of Computing Growth
War for the Planet of the Apes
Off-line/On-line 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑
10 ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜10𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑜𝑜𝑚𝑚𝑚𝑚𝑜𝑜𝑚𝑚𝑚𝑚𝑜𝑜
= (10 ℎ𝑜𝑜𝑜𝑜.)(60𝑚𝑚𝑚𝑚𝑚𝑚
ℎ𝑟𝑟 )(60 𝑠𝑠𝑠𝑠𝑠𝑠𝑚𝑚𝑚𝑚𝑚𝑚)(1,000𝑚𝑚𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠)
10𝑚𝑚𝑜𝑜
= 3.6 × 106/eye
= 7.2 × 106
Potential Improvements
• Eye tracking and foveal rendering
• Multi resolution displays
• Monoscopic vs. stereoscopic level of detail
• Asynchronous spacewarp
Increasing Densities (ppi) of OLED Displays
Field of View of the Human Eye
Wikipedia
Foveal Eye Tracking Constraints 2016
• Speed- needs to be fast enough to meet update requirements (currently 11 milliseconds, 90 Hz)
• Accuracy- Gaze direction is < 0.5 degree• Foveal direction accuracy can be ~ 1.0 arc minutes (1/60 of a degree)
• Non-invasive measurements- still need to see entire visual field
Eye Tracking
Sensing Methods: Retinal Tracking
● Hard problem with current technologies○ Extremely difficult to illuminate
■ Must bounce light off of retina■ Light comes back through iris
○ Light must be extremely bright○ Too much exposure will damage retina
● Typically done in ophthalmological setting
● Presently can only detect faint images ofblood vessels, companies working on it
● Very high angular resolution, but would presently require occlusion of vision
Purkinje Reflections
Purkinje Reflections
Cornsweet and Crane 1973
Purkinje Reflections
1st and 4th Purkinje Reflections
No Foveated Rendering
Roadtovr.com
Roadtovr.com
Contemporary ‘blur’ foveated rendering
Roadtovr.com
NVIDIA’s ‘contrast preserving’ rendering
Research on Foveated Displays
Alaskan Moose Diorama
Dall Sheep Restoration
Alaska Brown Bear Diorama
Plan of Typical Diorama
LOD Image Based Primitives
Layered Depth Images
Space & Time Warping
• When frame rates are not met, there are several types of solutions, but all have their deficiencies
• Visual artifacts appear because of loss of accuracy– e.g. Imperfect extrapolation, Object Disocclusion trails
Asynchronous Spacewarp
Asynchronous Spacewarp
Asynchronous Spacewarp
Asynchronous Spacewarp
Asynchronous Spacewarp
End