The Epipolar Occlusion Camera - cs.purdue.edu fileThe Epipolar Occlusion Camera Paul Rosen Voicu...

The Epipolar

Occlusion Camera

Paul Rosen

Voicu Popescu

Department of Computer SciencePurdue University

2

Depth images

� Expressive

� Inexpensive to

render

� Provide good LOD

control

3

Depth image limitation

� Sample only what

visible from

reference viewpoint

4

Depth image limitation

� Sample only what

visible from

reference viewpoint

� Missing samples

cause gaps, or

“disocclusion errors”

� Even small viewpoint translations cause substantial artifacts

5

Disocclusion errors

� More depth images

� Difficult to pick

viewpoints

� Many duplicate

samples

� Expensive

6

A palliative solution

7

A solution: the EOC

� Render depth image with epipolar occlusion camera� EOC: non-pinhole that

gathers all samples needed as the viewpoint translates

7

8

Outline

� Prior work

� EOC Construction

� EOC Rendering

� Results, discussion and future work

9

Occlusion cameras

� A family of cameras whose rays reach around

occluders to gather samples that are barely

hidden from the reference viewpoint thus

needed to avoid disocclusion errors when the

viewpoint translates

10

Prior work SPOC [Mei ‘05]

11

Prior work DDOC [Popescu ’06, Mo ‘07]

12

DDOC limitations

� Limited disocclusion capability for complex scenes

� Depth discontinuities compete for same image region

� Set of novel viewpoints for which DDOC image

has sufficient samples is defined empirically

EOC Reconstruction

� Viewsegment as opposed to viewpoint

� Makes additional image space as needed to

reveal occluded samples

EOC Depth Image

13

Epipolar occlusion camera

14

Construction

� Given

� A 3-D scene

� Two adjacent viewpoints L and R

� Construct camera model (i.e. set of rays) that is

� Comprehensive—samples surfaces visible from

viewpoints on segment LR

� Non-redundant—rays do not intersect so a 3-D point

has a unique projection

� Efficient—the scene is rendered interactively

15

Epipolar constraints

� As viewpoint translates between L and R, samples “move” on epipolar lines

� Epipolar lines allow partitioning the disocclusion events in disjoint 1-D events

� The EOC model is constructed one epipolar line at the

time

16

Construction

RL

17

Construction

RL

18

Construction

RL

19

Construction

RL

20

Construction

RL

21

Construction

RL

22

Construction

RL

23

Construction

RL

24

Construction

RL

25

Construction

RL

26

Construction

RL

27

Construction

RL

28

Construction

RL

29

Construction

RL

30

Construction

RL

31

Construction

RL

32

Construction

RL

33

Construction

RL

34

Construction

RL

35

Construction

RL

36

Construction

RL

37

Construction

RL

38

Construction

RL

39

Construction

RL

40

Construction

RL

41

Construction

RL

42

Construction

RL

43

Construction

Thin Occluder

RL

44

Construction

Thin Occluder

RL

45

Construction

Thin Occluder

RL

46

Construction

Thin Occluder

RL

47

Construction

Thin Occluder

RL

48

RL

Construction

Thin Occluder

49

RL

Construction

Thin Occluder

50

RL

Construction

Thin Occluder

51

RL

Construction

Thin Occluder

52

Construction examples

RL

53

Construction examples

RL

Rendering

� Given a camera model, one option is ray tracing

� Trace rays back into 3-D scene

� Compute intersections with geometric primitives

� Feed-forward EOC rendering algorithm on GPU

� For each scene triangle

� Project vertices (vertex shader)

� Split into epipolar spans (geometry shader)

� For each span

� Project span endpoints (geometry shader)

� Rasterize projected span (pixel shader)

54

Rendering

RL

55

Rendering

RL

56

Rendering

RL

57

Rendering

RL

58

Rendering

RL

59

Rendering

RL

60

Picket Fence

61

Picket Fence

62

Picket Fence

63

Using EOC Images

� Recovering 3-D positions is trivial� Rays are calculated during construction

� Depth is calculated during rendering

� Scene Reconstruction� Point based rendering is trivial

� Triangulation is more challenging

64

Scene Reconstruction

65

PHC Projection Radial Projection

Radial Epipolar Motion

Sphere

66

Radial Epipolar Motion

Sphere

67

Radial Epipolar Motion

Armadillo

68

4 Depth Images 2 EOC Images

Multiple ImagesCrossing Segments of Projection

69

Multiple ImagesCrossing Segments of Projection

70

4 Depth Images 2 EOC Images

Unity scene

71

Performance

72

Scene TrianglesConstruction

Time

Rendering

Time

Teapot 1K ~100 ms 19 ms

Picket Fence 1K ~100 ms 16 ms

Sphere 1K ~100 ms 36 ms

Armadillo 346K ~100 ms 1,130 ms

Unity 110K ~100 ms 1,793 ms

Bunny

1K

4K

16K

69K

~100 ms15 ms

20 ms

25 ms

68 ms

Conclusions

� The EOC is a robust solution to disocclusion

errors in complex scenes

� Not conservative, as only the first layer depth

discontinuities are examined

� Epipolar constraints are leveraged to separate

disocclusion errors in the image plane into a

manageable set of 1-D disocclusion errors

73

Future work

� The EOC is an infrastructure-level innovation� Produces a “better” depth image� Many potential applications

� Impostors, simplification, LOD

� Visibility, occlusion culling

� 3-D environment maps for reflections & refractions

� Compression, 3-D display bandwidth reduction

� The EOC generalizes the viewpoint (0-D) to a viewsegment (1-D)� Generalizations to a viewtriangle (2-D) or a viewtetrahedron (3-

D)

� Constructing occlusion camera images from photographs?� Might be possible without an intermediate 3-D reconstruction

74

Future work

� Apprehension about removing pinhole constraint not justified

� Non-pinhole images can be rendered quickly in feed-forward fashion

� Fast projection is not reserved for pinholes

� Effective and efficient non-pinholes open the door to

camera model design

� Abandon conventional simplicity and rigidity of camera model

� Design and dynamically optimize camera model for application and data set at hand

75

Acknowledgments

� NSF grant CNS-0417458

� Members of the Computer Graphics and

Visualization Lab @ Purdue

76

Questions

77