Upload
augustine-floyd
View
220
Download
2
Embed Size (px)
Citation preview
Image-Based Visual HullsImage-Based Visual Hulls
Wojciech Wojciech MatusikMatusik
Chris Buehler Chris Buehler Leonard Leonard McMillanMcMillan
Massachusetts Institute of Technology
Laboratory for Computer Science
Ramesh RaskarRamesh RaskarUniversity of North Carolina
at Chapel Hill
Steven J. GortlerSteven J. Gortler
Harvard University
Motivation Motivation
Real-time acquisition and rendering of dynamic Real-time acquisition and rendering of dynamic
scenesscenes
Previous WorkPrevious Work
•Virtualized RealityVirtualized Reality (Rander’97, Kanade’97, Narayanan’98)(Rander’97, Kanade’97, Narayanan’98)
•Visual HullVisual Hull(Laurentini’94)(Laurentini’94)
•Volume CarvingVolume Carving (Potmesil’87, Szeliski’93, Seitz’97)(Potmesil’87, Szeliski’93, Seitz’97)
•CSG RenderingCSG Rendering(Goldfeather’86, Rappoport’97)(Goldfeather’86, Rappoport’97)
•Image-Based RenderingImage-Based Rendering(McMillan’95, Debevec’96, Debevec’98)(McMillan’95, Debevec’96, Debevec’98)
ContributionsContributions
•View-dependent image-based View-dependent image-based visual hull representation visual hull representation
•Efficient algorithm for rendering Efficient algorithm for rendering the visual hull the visual hull
•Efficient algorithm computing Efficient algorithm computing visibilityvisibility
•A real-time systemA real-time system
What is a Visual Hull?What is a Visual Hull?
Why use a Visual Hull?Why use a Visual Hull?
•Can be computed robustlyCan be computed robustly
•Can be computed efficientlyCan be computed efficiently
- =
background background + +
foregroundforeground
backgroundbackground foreground foreground
Rendering Visual HullsRendering Visual Hulls
Reference 1Reference 2
Desired
Build then SampleBuild then Sample
Reference 1Reference 2
Desired
Build then SampleBuild then Sample
Reference 1Reference 2
Desired
Build then SampleBuild then Sample
Reference 1Reference 2
Desired
Build then SampleBuild then Sample
Reference 1Reference 2
Desired
Build then SampleBuild then Sample
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Sample DirectlySample Directly
Reference 1Reference 2
Desired
Image-Based ComputationImage-Based Computation
Reference 1
Reference 2
Desired
ObservationObservation
•Incremental computation along Incremental computation along scanlinesscanlines
Desired
Reference
BinningBinning
Epipole
•Sort silhouette edges into binsSort silhouette edges into bins
BinningBinning
Epipole
•Sort silhouette edges into binsSort silhouette edges into bins
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
Epipole
Bin 1
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
EpipoleBin 2
Bin 1
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
Epipole
Bin 3
Bin 1
Bin 2
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
Epipole
Bin 4
Bin 1
Bin 2
Bin 3
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
Epipole
Bin 5
Bin 1
Bin 2
Bin 3
Bin 4
BinningBinning
•Sort silhouette edges into binsSort silhouette edges into bins
Epipole
Bin 5
Bin 1
Bin 2
Bin 3
Bin 4
ScanningScanning
Epipole
Bin 1
EpipoleBin 2
ScanningScanning
EpipoleBin 2
ScanningScanning
EpipoleBin 2
ScanningScanning
Epipole
Bin 4
ScanningScanning
Epipole
Bin 5
ScanningScanning
Coarse-to-Fine SamplingCoarse-to-Fine Sampling
IBVH ResultsIBVH Results
•Approximately constant Approximately constant computation per pixel per cameracomputation per pixel per camera
•ParallelizesParallelizes
•Consistent with Consistent with input silhouettesinput silhouettes
Video of IBVHVideo of IBVH
Shading AlgorithmShading Algorithm
•A view-dependent strategyA view-dependent strategy
Visibility AlgorithmVisibility Algorithm
Visibility in 2DVisibility in 2D
Desired view
Reference view
Visibility in 2DVisibility in 2D
Desired view
Reference view
Front-most Points
Visibility in 2DVisibility in 2D
Desired view
Reference view
Visible
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visible
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visible
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
VisibleNot
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visible
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
Visibility in 2DVisibility in 2D
Desired view
Reference view
Coverage Mask
VisibleNot
Shaded Visual HullsShaded Visual Hulls
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
Trigger Signal
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
Compressed video
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
Intersection
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
Visibility
SystemSystem
Server(4x 500 Mhz)
Camera Client
Camera Client
Camera Client
Camera Client
Shading
More IBVH ResultsMore IBVH Results
Future WorkFuture Work
•3D teleconferencing3D teleconferencing
•Distributed computationDistributed computation
•Virtual setsVirtual sets
•Post-production camera effectsPost-production camera effects
•Mixed realityMixed reality
SummarySummary
•Visual hulls with texture can Visual hulls with texture can provide a compelling real-time provide a compelling real-time visualizationsvisualizations
•Visual hulls can be computed Visual hulls can be computed accurately and efficiently in image accurately and efficiently in image spacespace
•View dependent shading with View dependent shading with conservative visibilityconservative visibility
AcknowledgementsAcknowledgements
•DARPA ITO Grant F30602-971-0283DARPA ITO Grant F30602-971-0283
•A generous grant from Intel A generous grant from Intel CorporationCorporation
•NSF Career Awards 9875859 & NSF Career Awards 9875859 & 97033999703399
•Tom Buehler & Kari Anne KjølassTom Buehler & Kari Anne Kjølass
• Thanks to all members of the MIT Computer Graphics Group