Copyright Philipp Slusallek
Image-Based Rendering
Geometry-based Image-based
Panorama / Environment Map
Sprite / Imposter
Imposter /Sprite with Depth
LDIGeometry
View-dependentTexture Mapping
Lightfield / Lumigraph
Copyright Philipp Slusallek
View Interpolation
1. Interpolating between range images
- Chen and Williams, View Interpolation, 1993
- Shade et al., Layered Depth Images, 1998
2. Correspondences and epipolar analysis
- Laveau and Faugeras, Collection of Images, 1994
- McMillan and Bishop, Plenoptic Modeling, 1995
3. Combination of view synthesis and morphing
- Chen and Williams, 1993
- Seitz and Dyer, 1995
Require depths or correspondences
Copyright Philipp Slusallek
Drawing RGBZ Images
Various techniques:
1. Priority sorted splatting (Gauss, small rectangles, or points)
- Chen and Williams
2. Priority sorted height field
- McMillan and Bishop
3. Micropolygon mesh
- Marks et al.
4. Inverse ray-tracing of height field
- McMillan
5. Decimate and draw as a polygon mesh
Copyright Philipp Slusallek
View Interpolation
Issues:
Visibility/Occlusion
Holes
(Anti-)aliasing
Shading (diffuse)
Copyright Philipp Slusallek
Directly from correspondences
Laveau and Faugeras Algorithm
Existing View1 Existing View2
New View 3
p
p1
p2
l13 l23
p3
Copyright Philipp Slusallek
Merging RGBZ Images
Problems:
Overlaps
- Oversampling
- Occlusion
Gaps or holes
- Undersampling
- Expose unseen areas
Chen and Williams1993
Copyright Philipp Slusallek
View Interpolation [Chen/Williams’93]
Preprocessing:
Correct correspondences for pixel (morph map)
Division into blocks (hierarchical)
Obtain priorities form Z filtering
Display:
Linear interpolation of morph map
Back-to-front rendering of blocks
Fill holes from adjacent pixels
Copyright Philipp Slusallek
Sprites / Imposters
Purpose:
Cache complex rendered geometry
Decouple rendering updates from image updates
Drawing primitives
Issues:
Placement of polygon
Warping of texture (perspective, affine)
Validity of cache
Hierarchy
Continuity with environment
Copyright Philipp Slusallek
Sprites/Imposters with Depth
Better image warping:
Wider range of reuse
Backward mapping only with homograph
New mapping:
Stored depth map
Forward map depth map(approximate geometry)
Backward mapping of colorusing depth information
dd’
d’
Copyright Philipp Slusallek
Mapping with Depth
Solution:
Forward map depth
Reconstruct approximate geometry
Backward map color
I1 (I2) d2 I2
Copyright Philipp Slusallek
Layered Depth Images
Idea:
Handle disocclusion
Store invisible geometry in depth images
Data structure:
Per pixel list of depth samples
Per depth sample: RGBA
Z
Encoded: Normal direction, distance
Pack into cache lines
Copyright Philipp Slusallek
Layered Depth Images
Computation:
Incremental warping computation
Implicit ordering information Process in up to four quadrant
Splat size computation Table lookup
Fixed splat templates
Clipping of LDIs
Copyright Philipp Slusallek
Micro-Polygon rendering
Goal:
Fill Holes by assuming smooth geometry
Approach:
Connect neighboring pixels with polygons
Delete polygons for very large steps
Render micro-polygons
Heavy load on graphics system
Copyright Philipp Slusallek
Some More Applications
Holography
Generating new views of models
Decoupling of rendering and image updates
Z is available
Avoid artifacts
Motion Capture
Matching of model to image data
Morphing for flexible models
Video Rewrite / Talking Head