Click here to load reader

High Resolution Surface Reconstruction from Overlapping Multiple-Views

  • View
    215

  • Download
    0

Embed Size (px)

DESCRIPTION

Motivation Urban scenes are hard to model Complex geometry and topology Severe occlusions  We target automatic surface reconstruction from stereo vision data http://cvlab.epfl.ch/~strecha/multiview/denseMVS.html

Text of High Resolution Surface Reconstruction from Overlapping Multiple-Views

High Resolution Surface Reconstruction from Overlapping Multiple-Views
Nader Salman Mariette Yvinec Geometrica Team-Project JGA January 29th, 2009 Motivation Urban scenes are hard to model
Complex geometry and topology Severe occlusions We target automatic surface reconstruction from stereo vision data Related Work Image-based methods Model-based methods
Reconstructing 3D models of objects and scenes from N-views Image-based methods Image mosaics [Szeliski et al. 97] Lightfield [Levoy et al. 96] and Lumigraph [Cohen et al. 96] Model-based methods Active sensing approach [Turk et al. 94], [Laurendeau et al. 95], [Levoy et al. 96] Passive sensing approach Silhouette based [Niem et al. 97] Structure from motion Simple planar scenes [Zisserman et al. 96, 98] Semi-automatic reconstruction [Debevec et al. 96] Automatic reconstruction [Boissonnat et al. 88], [Taylor et al. 03], [Hilton 05], [Pons et al. 07] Image based methods : no explicit 3D model of the scene, new views are synthesized by resampling directly the captured images Image mosaics : multiple overlapping images are matched and integrated into a single panoramic image Lightfield & Lumigraph : Large number of calibrated images are captured and resampled to generate new views. Model based methods : Reconstruction of 3D surface model that can subsequently be rendered to generate new views Active sensing : project structured light pattern into the scene to recover dense 3D measurements of points on the scene surface (Range images : 2D images + depth per pixel => dense data) Passive sensing : reconstruct 3D estimates of observed scene features from multiple camera views (Sparse data) Silhouette based: Use foreground/background separation to reconstruct object shape from occluded contours. Structure from motion: Reconstruct 3D estimate of the scene structure from passive video image sequences. Planar surface fitting (Simple planar scenes) : planar surfaces can be fitted to unstructured cloud of3D points. Semi-automatic algorithm : reconstruct scene model from sparse 3D data # volumetric primitives placed according to the 3D features of the scene and constrained optimization reconstructs a best-fit model. Automatic algorithm : reconstruct automatically scene geometry from sparse 3D data set. System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction 5 Image Capture and 3D recovery (1)
Video sequences captured images ( images) ~10k points Matchmoving (camera calibration & point cloud generation) [MatchMover,Voodoo/VooCAT, or Boujou] Image Capture and 3D recovery (2)
High resolution localizedstill pictures captured images (10-50 images) Features extraction [ENPC/Certis CSTB] ~300k points Frames of tracked video [MatchMover, Voodoo/VooCAT, or Boujou] Hardware [Gyroviz] Image Capture and 3D recovery (2)
High resolution localizedstill pictures captured images (10-50 images) Features extraction [ENPC/Certis CSTB] ~300k points Frames of tracked video [MatchMover, Voodoo/VooCAT, or Boujou] Hardware [Gyroviz] 8 Image Capture and 3D recovery (2)
Matchmoving [Voodoo] Output is data dependent - Movement while shooting video - Lighting conditions Sparse point clouds Outliers ~300k points ~10k points Features extraction [ENPC/Certis CSTB] Dense point clouds (10-20x more) Requires calibrated images Many outliers Slow 9 System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction 10 Input Preprocessing (1)
Outliers removal wrt average K-nearest neighbors squared distance Detected outliers Detected outliers Input Preprocessing (2)
Outliers removal wrt point error (number of cameras, cameras cone angle) Detected outliers Detected outliers Input Preprocessing (3)
3D Points merging Point cloud smoothing (jet-fitting + reprojection) [Pouget-Cazals07] System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction 14 Reconstruction Algorithm
1 Extract 3D triangles that lie on the surface of the scene. Extract a surface out of these triangles. 2 Triangle soup extraction (1)
Idea: Extract 3D triangles whose corresponding 2D does not intersect the image contour N times N images Step 3 2D constrained Tessellation Backprojected into 3D space Step 4 Step 1 Filter triangle soup Contour extraction Constraints retrieval Step 2 Triangle soup Triangle soup extraction (2)
Idea: Extract 3D triangles whose corresponding 2D does not intersect the image contour Step 1: Contour extraction Grayscale Effective luminance of a pixel Filter Gaussian smoothing mask Extract contours Frei-chen convolution masks Survey Triangle soup extraction (3)
Idea: Extract 3D triangles whose corresponding 2D does not intersect the image contour Step 2: Constraints retrieval Build complete graph of feature points keep edges that respect the contour The edge weight depends on number of white pixels Adapt Needleman-Wunsch [NW1970] Filter collinear and intersecting edges Give higher priority to short edges Triangle soup extraction (4)
Idea: Extract 3D triangles whose corresponding 2D does not intersect the image contour Step 3: 2D constrained tessellation and backprojection Add constraints in 2D Delaunay tessellation Backproject into 3D space using corresponding 3D points N times N images Triangle Soup System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction 20 Triangle soup filtering (1)
Problem: Triangle soup contains lots of incorrect triangles Filter triangles using size heuristic Compute bounding box of triangle soup Each triangle longest edge is compared to the half diagonal of the bounding box Filter triangles using visibility constraint [Taylor03] Empty ray in space between the camera location and 3D point position. Triangle soup filtering (1)
Before size heuristic After size heuristic 22 Triangle soup filtering (2)
Problem: Triangle soup contains lots of incorrect triangles Filter triangles using size heuristic Compute bounding box of triangle soup Each triangle longest edge is compared to the half diagonal of the bounding box Filter triangles using visibility constraint [Taylor03] Empty ray in space between the camera location and 3D point position. 23 Triangle soup filtering (2)
Before visibility constraint After visibility constraint 24 System Overview Image Capture 2D Constrained Triangulation
3D Triangle Soup Triangle soup Filtering Structure from Motion Input Preprocessing Reconstruction 25 Reconstruction (1) Restricted Delaunay
How do we extract a surface from this polygon soup? Restricted Delaunay Compute 3D Delaunay triangulation of triangle soup vertices Compute restricted Delaunay to the union of triangles Restricted Delaunay reconstruction
Show results Reconstruction (2) OR Restricted Delaunay Extensions:
How do we extract a surface from this polygon soup? Restricted Delaunay Triangles respect empty circle condition Non watertight surface (holes) Extensions: Remesh using restricted Delaunay OR Scatter a large number of points continuously across the surface of each triangle Create an implicit surface that approximates the triangle soup Some results LA_Aerial ~8k points 29 Some results Lion ~20k points 30 Results Ours [Furokawa et al. 2007] [Strecha et al. 2006]
[Pons et al. 2009] Ours 31 Future work More data acquisition and testing for automatic parameter
Improve and complete existing algorithms Triangles soup algorithm Step 1 : Contour extraction Segmentation techniques [Lvy06] Histogram for automatic threshold [Asano08] Step 4 : Triangle soup filtering Some big triangles are correct Use circumscribing circle to differentiate between detected triangles Project triangles on respective images Use cross correlation to detect incorrect triangles Sharp features detection and reconstruction Thank you Nader Salman