Upload
hoangthien
View
219
Download
0
Embed Size (px)
Citation preview
The Epipolar
Occlusion Camera
Paul Rosen
Voicu Popescu
Department of Computer SciencePurdue University
2
Depth images
� Expressive
� Inexpensive to
render
� Provide good LOD
control
3
Depth image limitation
� Sample only what
visible from
reference viewpoint
4
Depth image limitation
� Sample only what
visible from
reference viewpoint
� Missing samples
cause gaps, or
“disocclusion errors”
� Even small viewpoint translations cause substantial artifacts
5
Disocclusion errors
� More depth images
� Difficult to pick
viewpoints
� Many duplicate
samples
� Expensive
6
A palliative solution
7
A solution: the EOC
� Render depth image with epipolar occlusion camera� EOC: non-pinhole that
gathers all samples needed as the viewpoint translates
7
8
Outline
� Prior work
� EOC Construction
� EOC Rendering
� Results, discussion and future work
9
Occlusion cameras
� A family of cameras whose rays reach around
occluders to gather samples that are barely
hidden from the reference viewpoint thus
needed to avoid disocclusion errors when the
viewpoint translates
10
Prior work SPOC [Mei ‘05]
11
Prior work DDOC [Popescu ’06, Mo ‘07]
12
DDOC limitations
� Limited disocclusion capability for complex scenes
� Depth discontinuities compete for same image region
� Set of novel viewpoints for which DDOC image
has sufficient samples is defined empirically
EOC Reconstruction
� Viewsegment as opposed to viewpoint
� Makes additional image space as needed to
reveal occluded samples
EOC Depth Image
13
Epipolar occlusion camera
14
Construction
� Given
� A 3-D scene
� Two adjacent viewpoints L and R
� Construct camera model (i.e. set of rays) that is
� Comprehensive—samples surfaces visible from
viewpoints on segment LR
� Non-redundant—rays do not intersect so a 3-D point
has a unique projection
� Efficient—the scene is rendered interactively
15
Epipolar constraints
� As viewpoint translates between L and R, samples “move” on epipolar lines
� Epipolar lines allow partitioning the disocclusion events in disjoint 1-D events
� The EOC model is constructed one epipolar line at the
time
16
Construction
RL
17
Construction
RL
18
Construction
RL
19
Construction
RL
20
Construction
RL
21
Construction
RL
22
Construction
RL
23
Construction
RL
24
Construction
RL
25
Construction
RL
26
Construction
RL
27
Construction
RL
28
Construction
RL
29
Construction
RL
30
Construction
RL
31
Construction
RL
32
Construction
RL
33
Construction
RL
34
Construction
RL
35
Construction
RL
36
Construction
RL
37
Construction
RL
38
Construction
RL
39
Construction
RL
40
Construction
RL
41
Construction
RL
42
Construction
RL
43
Construction
Thin Occluder
RL
44
Construction
Thin Occluder
RL
45
Construction
Thin Occluder
RL
46
Construction
Thin Occluder
RL
47
Construction
Thin Occluder
RL
48
RL
Construction
Thin Occluder
49
RL
Construction
Thin Occluder
50
RL
Construction
Thin Occluder
51
RL
Construction
Thin Occluder
52
Construction examples
RL
53
Construction examples
RL
Rendering
� Given a camera model, one option is ray tracing
� Trace rays back into 3-D scene
� Compute intersections with geometric primitives
� Feed-forward EOC rendering algorithm on GPU
� For each scene triangle
� Project vertices (vertex shader)
� Split into epipolar spans (geometry shader)
� For each span
� Project span endpoints (geometry shader)
� Rasterize projected span (pixel shader)
54
Rendering
RL
55
Rendering
RL
56
Rendering
RL
57
Rendering
RL
58
Rendering
RL
59
Rendering
RL
60
Picket Fence
61
Picket Fence
62
Picket Fence
63
Using EOC Images
� Recovering 3-D positions is trivial� Rays are calculated during construction
� Depth is calculated during rendering
� Scene Reconstruction� Point based rendering is trivial
� Triangulation is more challenging
64
Scene Reconstruction
65
PHC Projection Radial Projection
Radial Epipolar Motion
Sphere
66
Radial Epipolar Motion
Sphere
67
Radial Epipolar Motion
Armadillo
68
4 Depth Images 2 EOC Images
Multiple ImagesCrossing Segments of Projection
69
Multiple ImagesCrossing Segments of Projection
70
4 Depth Images 2 EOC Images
Unity scene
71
Performance
72
Scene TrianglesConstruction
Time
Rendering
Time
Teapot 1K ~100 ms 19 ms
Picket Fence 1K ~100 ms 16 ms
Sphere 1K ~100 ms 36 ms
Armadillo 346K ~100 ms 1,130 ms
Unity 110K ~100 ms 1,793 ms
Bunny
1K
4K
16K
69K
~100 ms15 ms
20 ms
25 ms
68 ms
Conclusions
� The EOC is a robust solution to disocclusion
errors in complex scenes
� Not conservative, as only the first layer depth
discontinuities are examined
� Epipolar constraints are leveraged to separate
disocclusion errors in the image plane into a
manageable set of 1-D disocclusion errors
73
Future work
� The EOC is an infrastructure-level innovation� Produces a “better” depth image� Many potential applications
� Impostors, simplification, LOD
� Visibility, occlusion culling
� 3-D environment maps for reflections & refractions
� Compression, 3-D display bandwidth reduction
� The EOC generalizes the viewpoint (0-D) to a viewsegment (1-D)� Generalizations to a viewtriangle (2-D) or a viewtetrahedron (3-
D)
� Constructing occlusion camera images from photographs?� Might be possible without an intermediate 3-D reconstruction
74
Future work
� Apprehension about removing pinhole constraint not justified
� Non-pinhole images can be rendered quickly in feed-forward fashion
� Fast projection is not reserved for pinholes
� Effective and efficient non-pinholes open the door to
camera model design
� Abandon conventional simplicity and rigidity of camera model
� Design and dynamically optimize camera model for application and data set at hand
75
Acknowledgments
� NSF grant CNS-0417458
� Members of the Computer Graphics and
Visualization Lab @ Purdue
76
Questions
77