Upload
letuong
View
225
Download
3
Embed Size (px)
Citation preview
1
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Title Slide
� Nice Picture of 3D Models
� Nice Picture of SegwayAutomatic 3D Reconstruction
and Modeling
2
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Spot the difference!
3
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
3D Model inspection
4
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Goal and Motivation
� Growing demand for digitizing indoor and outdoor environments
� Applications
� 3D content generation for
� mapping service applications, e.g. google maps, bing maps
� video games based on real locations
� interactive museum exhibits
� Pre-visualization and virtual sets for the film industry
� Virtual Reality Systems
� Realistic simulators for robotic algorithms (e.g. ROS/gazebo)
� Goal: autonomous system for efficient creation of photo
realistic 3D models of indoor and outdoor environments
5
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Overview
Data Acquisition
Registration
SurfaceReconstruction
TextureReconstruction
6
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
3D Object Representations
� Raw Data
� Point Cloud
� Rang Image
� Polygon Soup
� Surfaces
� Meshes
� Parametric
� Implicit
� Solids
� Voxels
� BSP Trees
� CSG
� High Level Structures
� Scene Graphs
� Skeleton
� etc.
7
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Data Acquisition (1)
� Robot hardware including range and imaging sensors
� Frontier based exploration for active environment exploration
� 2D mapping and localization (Grisetti et. al, 2005)
prev. 3D scans
next scan
trajectory
frontier
Current location
ScanningLidar SLR Camera
Pan-tilt unit
ExplorationLidar
Bosch’s 3D Mapping Robot Willow Garage’s PR2
5MP Camera
Tilting Lidar
Base Lidar
Stereo Cameras
8
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Data Acquisition (2)
9
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Data Acquisition (3)
10
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Pairwise Registration
� Iterative Closest Point (Besl & McKay ’92)
� Optimization loop
� Finding point correspondences
(distance, normal, color)
� Minimizing error metric
� Apply transform and iterate
� Fast convergence
� Easy to implement
� Errors accumulate for multiple scans
11
� SLAM: estimating the robot’s pose and
a map at the same time.
� Full SLAM posterior:
� Finding the most probable solution:
� Maximum a-posteriori estimation!
Measurement ModelMotion ModelMap Prior
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Global Registration
A vector of features representing the environment.
The robot’s position and orientation.
A feature observation.
The control vector applied a time t-1.
Motion links (wheel odometry)
Measurement links (scan matching)
12
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Global Registration – Example
2D Floorplan 3D Map
13
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Global Registration with Map Prior
� Full SLAM posterior:
� Use a local surface model to
approximate global map prior:
Motion Model Sensor ModelMap Prior
∏ ⋅⋅⋅⋅= −
i
iiiiiimxzpuxxpmpxpzuxmp ),|(),|()()(),|,(
10η
� Intuition: observation of surfaces belong to smooth manifolds
� Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010)
14
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Global Registration with Map Prior – Example
Registration without map priors Registration with map priors
15
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Surface Reconstruction
16
� Gradient of the smoothed indicator function is equal to the
smoothed surface normal field (Kazhdan et al., 2006)
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Volumetric Surface Reconstruction
χoriented points indicator function iso-surface
χ∇indicator gradient
� Surface reconstruction using a indicator function
� Formulation as a Poisson problem:
� Bilateral filter as smoothing function.
Point Cloud
Surface Model
17
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Appearance Reconstruction
� Reconstruction of material properties (e.g. color, specularity, temperature)
� Inference of material types (e.g. wood, plastic, vegetation)
� Enhancing visual impression by adding more realism
Surface Model Textured Surface Model
18
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Color Reconstruction (1)
1. Surface partitioning
� Surface segmentation into nearly planar regions
� Merging of similar regions using adjacency graphs
2. Unfolding
� Mapping of surface into texture domain
� Finding a conformal mapping which
preserves the angular distortion
� Minimizing the complex conformal
energy (Levy et al., 2002):
3. Color Reconstruction
� Unprojection of 3D points into camera images
19
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Color Reconstruction (2)
� Global optimization of texture color to remove
discontinuities.
� Multi-view blending algorithm:
� Poisson formulation with Dirichlet boundary
conditions (Perez et al., 2003)
before blending
averaging
our algorithm
20
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
21
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Handling Large 3D Models (1)
� Office model (50m x 140m)
� 28M Vertices, 55M Faces
� 772 MByte Geometry
� 600 MByte Texture
� Generation of scene graphs by
iteratively dividing 3D model
� Model simplification of scene graph
nodes to limit number of elements
per node
� View-dependent culling and loading
22
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
23
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
24
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
25
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Failure cases:
Future work:
� Overcome limitation to static environments
� Online integration of new data vs. batch processing
Reconstruction of fine structures (plants, furniture, etc.)
Texture reconstruction failures due to registration errors
26
Automatic 3D Reconstruction and Modeling
Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.
Research and Technology Center North America
Thank you for your attention!
http://bosch-ros-pkg.sourceforge.net/