Copyright Philipp Slusallek IBR: View Interpolation Philipp Slusallek

Copyright Philipp Slusallek

IBR: View Interpolation

Philipp Slusallek


Image-Based Rendering

Geometry-based Image-based

Panorama / Environment Map

Sprite / Imposter

Imposter /Sprite with Depth

LDIGeometry

View-dependentTexture Mapping

Lightfield / Lumigraph


View Interpolation

1. Interpolating between range images

- Chen and Williams, View Interpolation, 1993

- Shade et al., Layered Depth Images, 1998

2. Correspondences and epipolar analysis

- Laveau and Faugeras, Collection of Images, 1994

- McMillan and Bishop, Plenoptic Modeling, 1995

3. Combination of view synthesis and morphing

- Chen and Williams, 1993

- Seitz and Dyer, 1995

Require depths or correspondences


Drawing RGBZ Images

Various techniques:

1. Priority sorted splatting (Gauss, small rectangles, or points)

- Chen and Williams

2. Priority sorted height field

- McMillan and Bishop

3. Micropolygon mesh

- Marks et al.

4. Inverse ray-tracing of height field

- McMillan

5. Decimate and draw as a polygon mesh


View Interpolation

Issues:

Visibility/Occlusion

Holes

(Anti-)aliasing

Shading (diffuse)


Directly from correspondences

Laveau and Faugeras Algorithm

Existing View1 Existing View2

New View 3

p

p1

p2

l13 l23

p3


Merging RGBZ Images

Problems:

Overlaps

- Oversampling

- Occlusion

Gaps or holes

- Undersampling

- Expose unseen areas

Chen and Williams1993


View Interpolation [Chen/Williams’93]

Preprocessing:

Correct correspondences for pixel (morph map)

Division into blocks (hierarchical)

Obtain priorities form Z filtering

Display:

Linear interpolation of morph map

Back-to-front rendering of blocks

Fill holes from adjacent pixels


Sprites / Imposters

Purpose:

Cache complex rendered geometry

Decouple rendering updates from image updates

Drawing primitives

Issues:

Placement of polygon

Warping of texture (perspective, affine)

Validity of cache

Hierarchy

Continuity with environment


Sprites / Imposters


Imposters


Sprites/Imposters with Depth

Better image warping:

Wider range of reuse

Backward mapping only with homograph

New mapping:

Stored depth map

Forward map depth map(approximate geometry)

Backward mapping of colorusing depth information

dd’

d’


Mapping with Depth

Forward Mapping:

Holes and aliasing

I1 d1 (I2 ) I2


Mapping with Depth

Backward Mapping:

What is d?

I1 (I2) d2 I2


Mapping with Depth

Solution:

Forward map depth

Reconstruct approximate geometry

Backward map color

I1 (I2) d2 I2


Layered Depth Images

Idea:

Handle disocclusion

Store invisible geometry in depth images

Data structure:

Per pixel list of depth samples

Per depth sample: RGBA

Z

Encoded: Normal direction, distance

Pack into cache lines



Computation:

Incremental warping computation

Implicit ordering information Process in up to four quadrant

Splat size computation Table lookup

Fixed splat templates

Clipping of LDIs




Micro-Polygon rendering

Goal:

Fill Holes by assuming smooth geometry

Approach:

Connect neighboring pixels with polygons

Delete polygons for very large steps

Render micro-polygons

Heavy load on graphics system


Some more Application: Stereo

Example courtesyL. McMillan, MIT


Some More Applications

Holography

Generating new views of models

Decoupling of rendering and image updates

Z is available

Avoid artifacts

Motion Capture

Matching of model to image data

Morphing for flexible models

Video Rewrite / Talking Head


From Michael Cohen


From Michael Cohen


From Michael Cohen


Image-Based Rendering

Limitations

Static scene geometry

Fixed lighting

Constrained look-from or look-at point

Possibilities:

Complete modeling and rendering system?

Integration of Computer Vision and Computer Graphics?

Documents

Copyright Philipp Slusallek IBR: View Interpolation Philipp Slusallek