Image-Based Visual Hulls

Wojciech Matusik

Chris Buehler

Ramesh Raskar

Steven Gortler

Leonard McMillan

Presentation by: Kenton McHenry

Visual Hull

• Project rays through silhouette of various views to carve out regions containing the object. The intersection of all such cones is the visual hull.

• Not the true object shape, concavities can not be captured.

Visual Hull

Previous Work

• CSG (Constructive Solid Geometry)– Boolean operators over polygonal silohuettes– 3D intersections

• Volume Carving– Large memory requirements– Aliasing

Contribution

• Real time rendering– Perform intersection tests in 2D image space– No explicit hull

Epipolar Geometry

• Epipole: projection of 2nd cameras optical center in our view

• Baseline: line connecting optical centers

• Epipolar line: Projection of ray in our view, can restrict search to this line.

• Epipolar plane: formed by the baseline and the epipolar lines in both images.

Epipolar Geometry

Elminiate Polyhedra Intersections

… is equivelent to …

Elminiate Polyhedron-Line Intersections

• Absolute cross-section (defined by the silouhette) remains fixed (scaled).

• Can project our 3D ray into any plane along the cone, find intersections, and unproject.

Visual Hull Computation

• Given a desired view

• For each of the n2 pixels create a ray from the optical center through the pixel and project it into each of the reference views (epipolar lines)

• Calculate intervals with silouhette

• Lift the intervals back into 3D space

Sampling the hull

Speed up Search over Edges

• Observation: The pixels of a scanline in the desired image scan out a pencil of line segments in the reference image whose slope varies monotonically.

Incremental Computation

Binning

Scanning

Complexity

• n = width/height of images

• k = number of reference images

• l = average number of intersections

• O(lkn2)

Partial Results

Shading

• Use reference images as textures

• View Dependent Texturing

• Assign pixel color based on reference view whose angle is most simular to desired view.

Visibility• We need to watch out for reference views

that have an occluded view of the point.

• All visibility interactions occur within epipolar planes.

2D Visibility

O(nl)

Discrete 3D Visibility• Due to discretization the pixels of an

epipolar line in one image may be on different lines in the other image.

O(lkn2)

Results

Results

Silhouettes

• Fixed cameras => Background substraction

• 2D version of marching cubes to recover edges of silhouette

The Fundamental Matrix

• xrefT F xdes = 0

• 8-point algorithm

• Epipolar line corresponding to xdes in reference image = F xdes = (a b c)T such that ax + by + c = 0