Alessandro Franchi, Federico Tombari, Luigi Di Stefano
BOLD features to detecttexture-less objects
Motivations• Object detection is among the most widely studied topics in
computer vision.
2
The established paradigm for textured object detection relies on matching descriptors of local features•e.g. SIFT, SURF, BRIEF, ORB…•do not work well with texture-less objects
Texture-less object detection is still an open issue in computer vision literature.State-of-the-art approaches operate by means of edge-based template matching•do not scale well w.r.t. the size of the model library and the pose space to be explored
BOLD(Bunch Of Lines Descriptor)
• Objective: envision new features aimed at the detection of texture-less objects, to be injected in a classic descriptor-based object detection pipeline
• Descriptor for line segments
• Aggregates geometric primitives computedover pairs of neighboring segments
• Characteristics:– Invariant to rotation, translation and scale– Robust to noise and blur– Efficient to compute
3
Pose EstimationPose EstimationPose EstimationPose EstimationGeneralized
Hough Transform
Generalized Hough
Transform
Generalized Hough
Transform
Generalized Hough
Transform
Descriptor-based Object Detection
Keypoint DetectionKeypoint Detection
KeypointDescription
KeypointDescription
Descriptor MatchingDescriptor Matching
Keypoint DetectionKeypoint Detection
KeypointDescription
KeypointDescription
Descriptor MatchingDescriptor Matching
BOLD - Detection
5
Input imageInput image Extracted line segments
Extracted line segments• polygonal approximation
of the output of an edge detector
• specific line detection algorithms
• …
Canonical Orientation assignment
Canonical Orientation assignment
BOLD - Description (1)Pairwise geometric primitives
• Different geometric primitives have been evaluated– Normalized segment length– Relative midpoint distance– Relative angles– …
• Selected primitives: α,β angles– encode simultaneously
• relative orientations• relative segment displacements• contrast polarity
6
sj
si
mj
mi
g(mj)
g(mi)
ej1
ej2
ei1
ei2
β
αt
BOLD - Description (2)Bunch of segments construction
• A “bunch” of a segment is defined as the set of its k nearest neighboring (kNN) segments
k = 6k = 6 k = 6k = 6
αβ
45
90
135
180
225
270
315
0
360
36031545 90 135 180 225 270
s2
s1
BOLD - Description (3)Aggregating pairwise primitives
• For each segment pair formed by a segment si and one of the k segments in its bunch we accumulate α and β angles in a 2D joint histogram
α1
β1
α2
β2
EvaluationDatasets
9
“CVLab texture-less dataset” with clutter and occlusions9 models – 55 scenes
“CVLab texture-less dataset” with clutter and occlusions9 models – 55 scenes
“Caltech Covers” synthetic textured dataset with clutter and max 90% occlusions
80 models – 50 scenes
“Caltech Covers” synthetic textured dataset with clutter and max 90% occlusions
80 models – 50 scenes
EvaluationTested methods
Name Type Reference
BOLD Descriptor-based ‐‐‐
SIFT Descriptor-based D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” IJCV, 2004
SURF Descriptor-based H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded up robust features,” ECCV, 2006
ORB Descriptor-based E. Rublee, V. Rabaud, K. Konolige, G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” ICCV, 2011
Line2D Edge-based Template matching
S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit, "Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes," ICCV, 2011
S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit, "Gradient Response Maps for Real-Time Detection of Texture-Less Objects," PAMI, 2011
HALCON Edge-based Template matching http://www.mvtec.com/halcon/
10
BOLD detection examples
Texture-less object detection evaluation"CVLab textureless“ dataset
Detection results over a textured dataset with 9 models and 55 scenes.Detection time include
matching, validation and pose estimation
Detection results over a textured dataset with 9 models and 55 scenes.Detection time include
matching, validation and pose estimation
Textured object detection evaluation"Caltech Covers“ dataset
Detection results over a textured dataset with 80 models and 50 scenes.Detection time include
matching, validation and pose estimation
Detection results over a textured dataset with 80 models and 50 scenes.Detection time include
matching, validation and pose estimation
Scalability w.r.t. number of models"CVLab textureless“ dataset
15
Conclusions• BOLD features allows for leveraging on a standard descriptor-
based pipeline to detect effectively also texture-less objects– robustness to clutter and occlusion– high scalability w.r.t. the size of the model library
• Limitations– Curvilinear shapes (e.g. round objects)
• Linear approximation of highly curved contours is imprecise– Very simple shapes
• Too few segments, limited informativecontent of associated BOLDs
• Possible future developments– Include description of circular and elliptical arcs– Deploying BOLD to attain 3D object detection
based on a multi-view approach– Left-Right bunches
Right bunchRight bunch Left bunchLeft bunch
Datalogic AutomationVia Lavino 26540050 Monte San PietroBologna – ItalyTel. +39 051 3147011 – Fax +39 051 6759324www.datalogic.automation.com
This presentation contains statements that are neither reported financial results nor other historical information. These statements are forward-looking
statements. These forward-looking statements rely on a number of assumptions and are subject to a number of risks and uncertainties, many of which are
outside the control of Datalogic S.p.A., that could cause actual results to differ materially from those expressed in or implied by such statements, such as
future market conditions, currency fluctuations, the behavior of other market participants and the actions of governmental and state regulators
© 2011 Datalogic S.p.A. - All rights reserved. • Protected to the fullest extent under U.S. and international laws. • Copying, or altering of this document is
prohibited without express written consent from Datalogic S.p.A. Datalogic and the Datalogic logo are registered trademarks of Datalogic S.p.A. in many
countries, including the U.S.A. and the E.U. All other brand and product names may be trademarks of their respective owners.
Thank You!