Automatic 3D Reconstruction and Modeling

Research and Technology Center North America

Title Slide

� Nice Picture of 3D Models

� Nice Picture of SegwayAutomatic 3D Reconstruction

and Modeling

Spot the difference!

3D Model inspection

Goal and Motivation

� Growing demand for digitizing indoor and outdoor environments

� Applications

� 3D content generation for

� mapping service applications, e.g. google maps, bing maps

� video games based on real locations

� interactive museum exhibits

� Pre-visualization and virtual sets for the film industry

� Virtual Reality Systems

� Realistic simulators for robotic algorithms (e.g. ROS/gazebo)

� Goal: autonomous system for efficient creation of photo

realistic 3D models of indoor and outdoor environments

Overview

Data Acquisition

Registration

SurfaceReconstruction

TextureReconstruction

3D Object Representations

� Raw Data

� Point Cloud

� Rang Image

� Polygon Soup

� Surfaces

� Meshes

� Parametric

� Implicit

� Solids

� Voxels

� BSP Trees

� CSG

� High Level Structures

� Scene Graphs

� Skeleton

� etc.

Data Acquisition (1)

� Robot hardware including range and imaging sensors

� Frontier based exploration for active environment exploration

� 2D mapping and localization (Grisetti et. al, 2005)

prev. 3D scans

next scan

trajectory

frontier

Current location

ScanningLidar SLR Camera

Pan-tilt unit

ExplorationLidar

Bosch’s 3D Mapping Robot Willow Garage’s PR2

5MP Camera

Tilting Lidar

Base Lidar

Stereo Cameras

Pairwise Registration

� Iterative Closest Point (Besl & McKay ’92)

� Optimization loop

� Finding point correspondences

(distance, normal, color)

� Minimizing error metric

� Apply transform and iterate

� Fast convergence

� Easy to implement

� Errors accumulate for multiple scans

� SLAM: estimating the robot’s pose and

a map at the same time.

� Full SLAM posterior:

� Finding the most probable solution:

� Maximum a-posteriori estimation!

Measurement ModelMotion ModelMap Prior

Global Registration

A vector of features representing the environment.

The robot’s position and orientation.

A feature observation.

The control vector applied a time t-1.

Motion links (wheel odometry)

Measurement links (scan matching)

Global Registration – Example

2D Floorplan 3D Map

Global Registration with Map Prior

� Full SLAM posterior:

� Use a local surface model to

approximate global map prior:

Motion Model Sensor ModelMap Prior

∏ ⋅⋅⋅⋅= −

iiiiiimxzpuxxpmpxpzuxmp ),|(),|()()(),|,(

� Intuition: observation of surfaces belong to smooth manifolds

� Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010)

Global Registration with Map Prior – Example

Registration without map priors Registration with map priors

Surface Reconstruction

� Gradient of the smoothed indicator function is equal to the

smoothed surface normal field (Kazhdan et al., 2006)

Volumetric Surface Reconstruction

χoriented points indicator function iso-surface

χ∇indicator gradient

� Surface reconstruction using a indicator function

� Formulation as a Poisson problem:

� Bilateral filter as smoothing function.

Point Cloud

Surface Model

Appearance Reconstruction

� Reconstruction of material properties (e.g. color, specularity, temperature)

� Inference of material types (e.g. wood, plastic, vegetation)

� Enhancing visual impression by adding more realism

Surface Model Textured Surface Model

Color Reconstruction (1)

1. Surface partitioning

� Surface segmentation into nearly planar regions

� Merging of similar regions using adjacency graphs

2. Unfolding

� Mapping of surface into texture domain

� Finding a conformal mapping which

preserves the angular distortion

� Minimizing the complex conformal

energy (Levy et al., 2002):

3. Color Reconstruction

� Unprojection of 3D points into camera images

Color Reconstruction (2)

� Global optimization of texture color to remove

discontinuities.

� Multi-view blending algorithm:

� Poisson formulation with Dirichlet boundary

conditions (Perez et al., 2003)

before blending

averaging

our algorithm

Handling Large 3D Models (1)

� Office model (50m x 140m)

� 28M Vertices, 55M Faces

� 772 MByte Geometry

� 600 MByte Texture

� Generation of scene graphs by

iteratively dividing 3D model

� Model simplification of scene graph

nodes to limit number of elements

per node

� View-dependent culling and loading

Failure cases:

Future work:

� Overcome limitation to static environments

� Online integration of new data vs. batch processing

Reconstruction of fine structures (plants, furniture, etc.)

Texture reconstruction failures due to registration errors

Thank you for your attention!

http://bosch-ros-pkg.sourceforge.net/

Automatic 3D Reconstruction and Modeling - ROS.org ... · Automatic 3D Reconstruction and Modeling...

Documents

3D FRONSAC with PSF reconstruction

EE290T : 3D Reconstruction and Recognition

3D Reconstruction History

Single-view 3D Reconstruction

Tutorial 3D Reconstruction - Sunu Wibirama

Morphometric or 3D reconstruction

Image enhancement, 3D reconstruction & metrology software ...€¦ · Image enhancement, 3D reconstruction & metrology software for Scanning Electron Microscopes 3D reconstruction

Bayesian Reconstruction of 3D Human Motion from …papers.nips.cc/paper/1698-bayesian-reconstruction-of-3d...Bayesian Reconstruction of 3D Human Motion from Single-Camera Video 821

3D Reconstruction & Scanning

Unconstrained 3D Face Reconstruction

3D Building Reconstruction

LNCS 8740 - 3D Reconstruction of Saltanat Gate in ... · 3D Reconstruction of Saltanat Gate in Dolmabahce Palace 457 Fig. 3. 3D Reconstruction Stages and Final 3D model 2 3D Reconstruction

Deformable 3D Reconstruction with an Object Database · PDF fileDeformable 3D Reconstruction with an Object Database Deformable 3D Reconstruction with an Object Database Pablo F. Alcantarilla

In Vivo 3D ACL Reconstruction

3D Reconstruction & Scanning - Dongseo

AUTOMATED 3D ARCHITECTURE RECONSTRUCTION FROM

3D Human Body Reconstruction

Unconstrained 3D Face Reconstruction - CVF Open Accessopenaccess.thecvf.com/.../Roth_Unconstrained_3D_Face_2015_CVPR… · Unconstrained 3D Face Reconstruction Joseph Roth, Yiying

3D reconstruction Slides

3D Reconstruction