Automatic 3D Reconstruction and Modeling - ROS.org ... · Automatic 3D Reconstruction and Modeling Benjamin Pitzer ... Title Slide Nice Picture of 3D ... Finding a conformal mapping

1

Automatic 3D Reconstruction and Modeling

Benjamin Pitzer | 10/18/2010 | © 2010 Robert Bosch LLC and affiliates. All rights reserved.

Research and Technology Center North America

Title Slide

� Nice Picture of 3D Models

� Nice Picture of SegwayAutomatic 3D Reconstruction

and Modeling

2




Spot the difference!

3




3D Model inspection

4




Goal and Motivation

� Growing demand for digitizing indoor and outdoor environments

� Applications

� 3D content generation for

� mapping service applications, e.g. google maps, bing maps

� video games based on real locations

� interactive museum exhibits

� Pre-visualization and virtual sets for the film industry

� Virtual Reality Systems

� Realistic simulators for robotic algorithms (e.g. ROS/gazebo)

� Goal: autonomous system for efficient creation of photo

realistic 3D models of indoor and outdoor environments

5




Overview

Data Acquisition

Registration

SurfaceReconstruction

TextureReconstruction

6




3D Object Representations

� Raw Data

� Point Cloud

� Rang Image

� Polygon Soup

� Surfaces

� Meshes

� Parametric

� Implicit

� Solids

� Voxels

� BSP Trees

� CSG

� High Level Structures

� Scene Graphs

� Skeleton

� etc.

7




Data Acquisition (1)

� Robot hardware including range and imaging sensors

� Frontier based exploration for active environment exploration

� 2D mapping and localization (Grisetti et. al, 2005)

prev. 3D scans

next scan

trajectory

frontier

Current location

ScanningLidar SLR Camera

Pan-tilt unit

ExplorationLidar

Bosch’s 3D Mapping Robot Willow Garage’s PR2

5MP Camera

Tilting Lidar

Base Lidar

Stereo Cameras

8





9





10




Pairwise Registration

� Iterative Closest Point (Besl & McKay ’92)

� Optimization loop

� Finding point correspondences

(distance, normal, color)

� Minimizing error metric

� Apply transform and iterate

� Fast convergence

� Easy to implement

� Errors accumulate for multiple scans

11

� SLAM: estimating the robot’s pose and

a map at the same time.

� Full SLAM posterior:

� Finding the most probable solution:

� Maximum a-posteriori estimation!

Measurement ModelMotion ModelMap Prior




Global Registration

A vector of features representing the environment.

The robot’s position and orientation.

A feature observation.

The control vector applied a time t-1.

Motion links (wheel odometry)

Measurement links (scan matching)

12




Global Registration – Example

2D Floorplan 3D Map

13




Global Registration with Map Prior

� Full SLAM posterior:

� Use a local surface model to

approximate global map prior:

Motion Model Sensor ModelMap Prior

∏ ⋅⋅⋅⋅= −

i

iiiiiimxzpuxxpmpxpzuxmp ),|(),|()()(),|,(

10η

� Intuition: observation of surfaces belong to smooth manifolds

� Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010)

14




Global Registration with Map Prior – Example

Registration without map priors Registration with map priors

15




Surface Reconstruction

16

� Gradient of the smoothed indicator function is equal to the

smoothed surface normal field (Kazhdan et al., 2006)




Volumetric Surface Reconstruction

χoriented points indicator function iso-surface

χ∇indicator gradient

� Surface reconstruction using a indicator function

� Formulation as a Poisson problem:

� Bilateral filter as smoothing function.

Point Cloud

Surface Model

17




Appearance Reconstruction

� Reconstruction of material properties (e.g. color, specularity, temperature)

� Inference of material types (e.g. wood, plastic, vegetation)

� Enhancing visual impression by adding more realism

Surface Model Textured Surface Model

18




Color Reconstruction (1)

1. Surface partitioning

� Surface segmentation into nearly planar regions

� Merging of similar regions using adjacency graphs

2. Unfolding

� Mapping of surface into texture domain

� Finding a conformal mapping which

preserves the angular distortion

� Minimizing the complex conformal

energy (Levy et al., 2002):

3. Color Reconstruction

� Unprojection of 3D points into camera images

19




Color Reconstruction (2)

� Global optimization of texture color to remove

discontinuities.

� Multi-view blending algorithm:

� Poisson formulation with Dirichlet boundary

conditions (Perez et al., 2003)

before blending

averaging

our algorithm

20




21




Handling Large 3D Models (1)

� Office model (50m x 140m)

� 28M Vertices, 55M Faces

� 772 MByte Geometry

� 600 MByte Texture

� Generation of scene graphs by

iteratively dividing 3D model

� Model simplification of scene graph

nodes to limit number of elements

per node

� View-dependent culling and loading

22




23




24




25




Failure cases:

Future work:

� Overcome limitation to static environments

� Online integration of new data vs. batch processing

Reconstruction of fine structures (plants, furniture, etc.)

Texture reconstruction failures due to registration errors

26




Thank you for your attention!

http://bosch-ros-pkg.sourceforge.net/

Documents

Automatic 3D Reconstruction and Modeling - ROS.org ... · Automatic 3D Reconstruction and Modeling Benjamin Pitzer ... Title Slide Nice Picture of 3D ... Finding a conformal mapping