Seeing 3D From 2D Imagesasdad

8/13/2019 Seeing 3D From 2D Imagesasdad

1/47

Seeing 3D from 2D

ImagesWilliam and Craig 115 - 164


2/47

How to make a 2D image appear as3D!

Output is typically 2D Images

Yet we want to show a 3D world!

How can we do this? We can include cues in the image that give our

brain 3D information about the scene

These cues are visual depth cues


3/47

Visual Depth Cues

Monoscopic Depth Cues (single 2D image)

Stereoscopic Depth Cues (two 2D images)

Motion Depth Cues (series of 2D images)Physiological Depth Cues (body cues)


4/47

Monoscopic Depth Cues

Interposition An object that occludes another is closer

Shading Shape info. Shadows are included here

Size Usually, the larger object is closer

Linear Perspective parallel lines converge at a single point

Surface Texture Gradient more detail for closer objects

Height in the visual field

Higher the object is (vertically), thefurther it is

Atmospheric effects further away objects are blurrier

Brightness further away objects are dimmer


5/47

Stereoscopic Display Issues

Stereopsis

Stereoscopic Display Technology

Computing Stereoscopic ImagesStereoscopic Display and HTDs.

Works for objects < 5m. Why?


6/47

Stereopsis

The result of the two slightly different views of theexternal world that our laterally-displaced eyes

receive.


7/47

Retinal Disparity

f1f2

Left Eye Right Eye

Retinal disparity =

If both eyes are fixated on apoint, f1, in space, then animage of f1 if focused atcorresponding points in the

center of the fovea of eacheye. Another point, f2, at adifferent spatial location wouldbe imaged at points in eacheye that may not be the same

distance from the fovea. Thisdifference in distance is theretinal disparity.


8/47

Disparity

If an object is closer than the fixation point, theretinal disparity will be a negative value. Thisis known as crossed disparitybecause the twoeyes must cross to fixate the closer object.

If an object is farther than the fixation point,the retinal disparity will be a positive value.This is known as uncrossed disparitybecausethe two eyes must uncross to fixate the fartherobject.

An object located at the fixation point or whoseimage falls on corresponding points in the tworetinae has a zero disparity.


9/47

Convergence Angles

i

f2

f1

D1

D2a b

c d

1

a+a+c+b+d = 180

b+c+d = 180

a-b= a+(-b) = 1+2

= Retinal Disparity

2


10/47

Miscellaneous Eye Facts

Stereoacuity- the smallest depth that canbe detected based on retinal disparity.

Visual Direction- Perceived spatiallocation of an object relative to an observer.


11/47

Horopters

Corresponding points onthe two retinae are definedas being the same verticaland horizontal distancefrom the center of the

fovea in each eye. Horopter - the locus of

points in space that fall oncorresponding points inthe two retinae when the

two eyes binocularly fixateon a given point in space(zero disparity).

Points on the horopterappear at the same depth

as the fixation point.

f1

f2

Vieth-Mueller

Circle


12/47

Stereoscopic Display

Stereoscopic images are easy to do badly,hard to do well, and impossible to do

correctly.


13/47

Stereoscopic Displays

Stereoscopic display systems create a three-dimensional image (versus a perspectiveimage) by presenting each eye with a

slightly different view of a scene.

Time-parallel

Time-multiplexed


14/47

Time Parallel StereoscopicDisplay

Two Screens

Each eye sees adifferent screen

Optical system directseach eye to the correctview.

HMD stereo is donethis way.

Single Screen

Two different images

projected on the samescreen

Images are polarizedat right angles to each

other.

User wears polarizedglasses (passive

glasses).


15/47

Passive Polarized Projection Issues

Linear Polarization

Ghosting increases when you tilt head

Reduces brightness of image by about

Potential Problems with Multiple Screens (nextslide)

Circular Polarization

Reduces ghosting but also reduces brightnessand crispness of image even more


16/47

Problem with Linear Polarization

With linear polarization,the separation of the leftand right eye images isdependent on the

orientation of the glasseswith respect to theprojected image.

The floor image cannot be

aligned with both the sidescreens and the frontscreens at the same time.


17/47

Time Multiplexed Display

Left and right-eye views of an image arecomputed and alternately displayed on thescreen.

A shuttering system occludes the right eyewhen the left-eye image is being displayedand occludes the left-eye when the right-

eye image is being displayed.


18/47

Stereographics Shutter Glasses


19/47

Screen Parallax

P

Left eye

position

Right eyeposition

Pleft

Pright

Pright

Pleft

P

Display

Screen

Object withpositive

parallax

Object with

negative parallax

The screen parallax is the distance between the projected locationof P on the screen, Pleft, seen by the left eye and the projected

location, Pright, seen by the right eye (different from retinal disparity).


20/47

Screen Parallax (cont.)

f1

p

i

d

Left

eyepoint

Right

eyepoint

Projection

Plane

D

p = i(D-d)/D

where p is the amount of screenparallax for a point, f1, whenprojected onto a plane adistance d from the plane

containing two eyepoints.i is the interocular distance

between eyepoints and

D is the distance from f1 to thenearest point on the plane

containing the two eyepointsd is the distance from the

eyepoint to the nearest pointon the screen


21/47

Screen Parallax

-65.00

-55.00

-45.00

-35.00

-25.00

-15.00

-5.00

5.00

0 50 100 150 200 250 300 350

Distance from Eye

Screen

Parallax

Zero parallax at screen, max positive parallax

is i, negative parallax is equal to I halfway

between eye and screen


22/47

Stereoscopic Voxels

Left EyePoint

Right EyePoint

1

1

1

1

1

1

1

1

1

2

2

2

2

2

2

2

3

3

3

3

3

3

3

4

4

4

4

4

4

5

5

5

5

5

2

A

B


23/47

Screen Parallax and ConvergenceAngles

f1f2 f3

ProjectionPlane

a

Screen parallax dependson closest distance toscreen.

Different convergenceangles can all have thesame screen parallax.

Also depends onassumed eyeseparation.


24/47

How to create correct left- andright-eye views

To specific a single view in almost allgraphics software or hardware you mustspecify:

Eyepoint

Look-at Point

Field-of-View or location of Projection Plane

View Up Direction


25/47

Basic Perspective Projection SetUp from Viewing Paramenters

Y

Z

X

Projection Plane is orthogonal to one of the major axes

(usually Z). That axis is along the vector defined by the

eyepoint and the look-at point.


26/47

What doesnt work

Each view has a different

projection plane

Each view will be presented

(usually) on the same plane


27/47

What Does Work

i i


28/47

Setting Up Projection Geometry

Look at pointEyeLocations

Look at points

Eye

Locations

No

Yes


29/47

Screen Size

The size of the window doesnot affect the retinal disparity

for a real window.

Once computed, the screen parallaxis affected by the size of the displayscreen


30/47

Visual Angle Subtended

Screen parallax is measured in terms of visual angle. This is a screenindependent measure. Studies have shown that the maximum anglethat a non-trained person can usually fuse into a 3D image is about

1.6 degrees. This is about 1/2 the maximum amount of retinal disparityyou would get for a real scene.


31/47

Accommodation/ Convergence

Display Screen


32/47

Position Dependence(without head-tracking)

l d


33/47

Interocular Dependance

F

Modeled Point

PerceivedPoint

Projection Plane


34/47

Obvious Things to Do

Head tracking

Measure Users Interocular Distance


35/47

Another Problem

Many people can not fuse stereoscopicimages if you compute the images withproper eye separation!

Rule of Thumb: Compute with about thereal eye separation.

Works fine with HMDs but causes image

stability problems with HTDs (why?)


36/47

Two View Points with Head-Tracking

Projection Plane

Modeled Point

Perceived Points

Modeled Eyes

True Eyes


37/47

Maximum Depth Plane

Maximum Depth PlaneModeled Eyes

True Eyes

EF

Modeled

Point

PerceivedPoint

ProjectionPlane


38/47

Can we fix this?

Zachary Wartell, "Stereoscopic Head-Tracked Displays: Analysis andDevelopment of Display Algorithms," Ph.D. Dissertation, GeorgiaInstitute of Technology, August 2001.

Zachary Wartell, Larry F. Hodges, William Ribarsky. "An AnalyticComparison of Alpha-False Eye Separation, Image Scaling and Image

Shifting in Stereoscopic Displays," IEEE Transactions on Visualizationand Computer Graphics,April-June 2002, Volume 8, Number 2, pp.129-143. (related tech report is GVU Tech Report 00-09 (Abstract ,PDF, Postscript.)

Zachary Wartell, Larry F. Hodges, William Ribarsky. "BalancingFusion, Image Depth, and Distortion in Stereoscopic Head-Tracked

Displays." SIGGRAPH 99 Conference Proceedings, Annual ConferenceSeries. ACM SIGGRAPH, Addison Wesley, August 1999, p351-357.(Paper:Abstract, PDF, Postscript; SIGGRAPH CD-ROM Supplement,supplement.zip,supplement.tar.Z).
http://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.pdfftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.ps.Zhttp://www.cc.gatech.edu/gvu/reports/1999/abstracts/99-32.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.pdfftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.ps.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.ziphttp://www.cc.gatech.edu/people/home/wartell/supplement.tar.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.tar.Zhttp://www.cc.gatech.edu/people/home/wartell/supplement.zipftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.ps.Zftp://ftp.cc.gatech.edu/pub/gvu/tr/1999/99-32.pdfhttp://www.cc.gatech.edu/gvu/reports/1999/abstracts/99-32.htmlftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.ps.Zftp://ftp.cc.gatech.edu/pub/gvu/tr/2000/00-09.pdfhttp://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.htmlhttp://www.gvu.gatech.edu/gvu/reports/2000/abstracts/00-09.html


39/47

Point of fixation

Distance in centimeters from eye plane

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

10

20

30

40

50

60

70

80

90

10

0

11

0

12

0

13

0

14

0

15

0

Symmetric convergence

Convergence 20 centime ters to the left of the lef t eye

Change in eyepoint separation with change in point of fixation.Centers of rotation of the eyes are assumed to be 6.4 centimeters apart.


40/47

Ghosting

Affected by the amount of light transmittedby the LC shutter in its off state.

Phosphor persistence

Vertical screen position of the image.


41/47

Ghosting (cont.)

Extinction Ratio =Luminance of the correct eye image

------------------------------------------------------------Luminance of the opposite eye ghost image

Image Position Red White

Top 61.3/1 17/1

Middle 50.8/1 14.4/1

Bottom 41.1/1 11/1


42/47


43/47

Time-parallel stereoscopic images

Image quality may also be affected by

Right and left-eye images do not match in color,size, vertical alignment.

Distortion caused by the optical system

Resolution

HMDs interocular settings

Computational model does not match viewinggeometry.


44/47

Motion Depth Cues

Parallax createdby relative headposition and

object beingviewed.

Objects nearer tothe eye move a

greater distance


45/47

Pulfrich Effect

Neat trick

Different levels of illumination requireadditional time (your frame rates differ base

of amount of light)What if we darken one image, and brighten

another?

http://dogfeathers.com/java/pulfrich.htmlwww.cise.ufl.edu/~lok/multimedia/videos/p

ulfrich.avi
http://dogfeathers.com/java/pulfrich.htmlhttp://dogfeathers.com/java/pulfrich.html


46/47

Physiological Depth Cues

Accommodationfocusing adjustmentmade by the eye to change the shapeofthe lens. (up to 3 m)

Convergencemovement of the eyes tobring in the an object into the same locationon the retina of each eye.


47/47

Summary

MonoscopicInterposition is strongest.

Stereopsis is very strong.

Relative Motion is also very strong (orstronger).

Physiological is weakest (we dont even usethem in VR!)

Add as needed

ex. shadows and cartoons

Documents

Seeing 3D From 2D Imagesasdad