Object Recognition in 3D scans - ROS.org · Bastian Steder - Object Recognition in 3D scans - intern presentation 22 Final scene. Bastian Steder - Object Recognition in 3D scans -

Object Recognition in 3D scans

Willow Garage Intern Presentation 09/23/2010

Bastian StederUniversity of Freiburg, Germany

Bastian Steder - Object Recognition in 3D scans - intern presentation 2

Object Recognition in 3D scans

Goal: Object detection andGoal: Object detection andlocalization in point cloudslocalization in point clouds


Working on Range images

Main advantage:Main advantage:Encodes negative informationEncodes negative information


• Match features between model and scene

• Estimate object candidate poses

• Score and filter candidates

General Approach


Feature extraction

Mainly two steps:

• Interest point detection:

Where should features be extracted

• Calculation of a descriptor:

Describe the area around the interest point in a way that makes it easy to compare


Desired properties of our Features

• Use positions where the surface is stable,

but with significant changes in surrounding (I)

• Encode the surface structure (I+D)

• Encode outer shape (I+D)

• Provide local 6DOF transformation

=> NARF: Normal aligned radial features


Border information

• Three kinds of border points

– Obstacle borders

– Shadow borders

– Veil points


How to detect borders?

• Possibilities:

– Impact angles of the sensor beams

– Changes in normals

– …

• What actually works well:

– Changes in distances toneighboring points


Border extraction video


Interest points extraction

• Calculate a score for each image point

– How much do the dominant directions of the surface changes in the vicinity vary?

(border information & principal curvature)

– How stable is the surface on the point itself?

• Find maximums above a threshold


NARF Interest point extraction video


NARF descriptor video


Results

ROC curve


Example – Step 1


Spatial validation – Validation points

Validation points on the model views for a fast spatial validationValidation points on the model views for a fast spatial validation


Example – Step 2


Overlapping poses + ICP

• Removal of multiple instances of the same object in the same position, coming from different feature matches

• Refinement of the object poses using ICP


Example – Step 3


Spatial validation – View Simulation

Comparison of what was actually seen and Comparison of what was actually seen and what it should look like according to the modelwhat it should look like according to the model


Example – Step 4


Video


Final scene


Software and Documentation

• Currently some parts in PCL and some in point_cloud_perception_experimental

– Range image class, border extraction, NARF interest point extraction, NARF descriptor, model class, spatial verification, visualization tools …

• Some Tutorials already exists – there will be more

• Paper submitted to ICRA11


Thank you!

• Willow Garage in general for making this possible

• Radu for always being available for (sometimes stupid) questions and discussions

• Kurt for giving valuable input

• The other interns and normal employees for making it a great time

• The people in the kitchen for protecting me from months of Fast Food


Questions?

Documents

Object Recognition in 3D scans - ROS.org · Bastian Steder - Object Recognition in 3D scans - intern presentation 22 Final scene. Bastian Steder - Object Recognition in 3D scans -