A Test Environment for Feature-Based Range Data Registration

A Test Environment for Feature-Based Range Data Registration

HENDRIK THAMER, DANIEL WEIMERBIBA - Intelligent Production and Logistics Systems

Hochschulring 2028359 Bremen

[email protected]

BERND SCHOLZ-REITERUniversity of Bremen

Bibliothekstrasse 128359 Bremen

[email protected]

Abstract: The analysis of range data is an important step in many automation applications such as robotic manip-ulation, autonomous navigation or human machine interaction systems. Therefore, range sensors are usually usedfor acquiring geometric information about the relevant environment of interest. In order to cover the entire envi-ronment of interest, using one single sensor is often not sufficient. Hence, sensor systems consisting of multiplesensors are used or the object of interest is rotated relative to the sensor system. For integrating the single sensordata into a common reference system, a registration process must be performed in order to determine the rigidtransformations between the single point clouds. This paper presents a test environment for registering multiplepoint cloud data into a common reference frame. The environment focuses on the case when no initial informationregarding the transformation is available and allows efficient testing and evaluation of different parameter setupsin particular steps within a common registration pipeline.

Key–Words: Registration, Range Data, Sensor Data Fusion, Computer Vision, Automation

1 IntroductionThe analysis of range data in order to recognize andreconstruct real world objects is an important task incomputer vision. Due to (self-) occlusions and lim-ited field of views, using single range sensor devicessuch as Time-of-Flight (TOF) cameras, Laser scanneror stereo vision systems results in sensor data cover-ing only a partial view of the scanned objects. How-ever, in most applications a complete 3D representa-tion of single objects or the entire environment is re-quired. The process of aligning several point cloudsin a common global reference frame is called reg-istration. For registration of point clouds, variousapproaches have been proposed for determining thetransformation between the single clouds. Registra-tion algorithms can be categorized in coarse and fineregistration [1]. Coarse registration algorithms esti-mate a initial transformation without prior knowledge.On the contrary, fine registration algorithms optimizea given initial transformation. Without an initial es-timate, algorithms from both categories are usuallyperformed sequentially. The transformation are re-alized by searching and identifying correspondencesbetween two single point clouds. These correspon-dences can be established between points, surfaces orcurves. Thereby, a line of research of this area is fo-cusing on establishing point-to-point correspondencesbetween point clouds.This paper presents a test environment for the registra-

tion of 3D point cloud data. Finding the best parame-ter configurations and single methods within the regis-tration process can be a time-consuming step . There-fore, the environment gives the opportunity to evalu-ate different configurations of the registration pipelinepresented in the Point Cloud Library (PCL) [2] and inRusu et al.[3]. The pipeline consists of different pro-cessing steps. The process starts with applying pre-processing algorithms such as downsampling or noiseremoval. Then, keypoints are identified using key-point detectors. Afterwards, the identified keypointsare described by using local feature description tech-niques. By comparing the features in both clouds, cor-responding points that have a similar feature descrip-tor are determined. Due to the fact that not all of thedetermined correspondences might be correct, wrongcorrespondences are removed by applying correspon-dence rejection algorithms. Then, the remaining cor-respondences are used for estimating the coarse trans-formation between the point clouds. Finally, a fineregistration algorithm such as the Iterative-Closest-Point (ICP) algorithm [4] can be used for optimiz-ing the transformation. Due to the modular softwarestructure of the presented test environment, differentmethods or techniques in each processing step of theregistration pipeline can be configured and evaluated.For instance, the environment gives the opportunity ofselecting different keypoint detectors or local featuredescription techniques. Therefore, optimal configura-

Recent Advances in Computer Science and Applications

ISBN: 978-960-474-317-9 126

tions for specific applications can be easily defined.The remainder of the paper is structured as follows.In the next section, techniques and algorithms are pre-sented concerning the registration of range data. Af-terwards, the test environment is presented in detailfollowing the aforementioned registration pipeline.Here, the configuration possibilities for each process-ing step are described. In the next section, we demon-strate the environment with an application scenariofrom the field of logistics. Finally, we close the pa-per with a short conclusion and an outlook for furtherresearch activities.

2 Registration of Range DataThe objective of registration is the determination ofthe euclidean transformation between a set of rangeimages of an object of interest taken from differentviewpoints, in order to transform them into a com-mon reference frame. Salvi et al. [1] made a compre-hensive survey of the most common techniques andhave classified various approaches and techniques forregistration based on strategy, robustness, motion es-timation and kind of correspondence [1]. The follow-ing explanations are based on this survey. In general,registration methods can be divided into coarse andfine registration. Coarse registration methods com-pute an initial estimate of the transformation betweentwo point clouds. They usually try to find correspon-dences between the range data sets. These correspon-dences can be established between points, surfacesand curves. Generally, the most commonly used cor-respondence method is point-to-point [1]. Therefore,feature description techniques such as Point Signa-tures [5] or Spin Images [6] are used for describingpoints in both range data sets and compared with eachother. Then, the coarse registration estimation is de-fined using the best point-to-point correspondences.Examples for registration techniques using surface-correspondences are Principal Component Analysis[7] and algebraic surface models [8].In contrast to coarse registration techniques, fine reg-istration methods optimize an initial transformationin order to find a more accurate solution. The reg-istration strategy can be distinguished between regis-tering the range data pair-wise or registering the sin-gle data at the same time which is called multi-viewregistration. Fine registration methods are often com-putational expensive. In order to optimize the com-putation time some methods use techniques such ask-d trees for fast nearest-neighbor search. Thereby,the distance to be minimized can be between point-correspondences or between points and a correspond-ing tangent-plane. The most famous fine registration

method is the ICP presented by Besl and McKay [4].Here, the distance between point-correspondences isminimized known as closest points. The algorithmrequires a good initial transformation in order to pre-vent converging in a local optimum. After the applica-tion of the initial transformation estimation, a searchfor closest points between each point in the first pointcloud and in the second cloud is performed in orderto minimize the distance between these correspon-dences. A main requirement for the application of theICP is a significant overlap of the clouds.This paper focuses on coarse registration with iden-tifying point-to-point correspondences in two pointclouds for gaining a complete representation of an ob-ject. Therefore, the feature-based registration pipelinewithin PCL is followed. Usually, the configurationand the method selection of the single steps dependson the properties of the sensor data. Hence, the de-termination of the optimal configuration is a time-consuming step. In order to tackle this issue, we in-troduce a test environment for experimental determi-nation of methods and parameter configurations forfeature-based registration.

Figure 1: Registration pipeline based on [2][3]


ISBN: 978-960-474-317-9 127

3 Test EnvironmentThe system consists of five processing steps represent-ing the whole registration pipeline from coarse to fineregistration. The system has a modular architectureand each module is arbitrarily configurable. The se-lected combination and configuration of modules canbe stored in order to reproduce experimental results.Additionally, new keypoint detectors or feature de-scription techniques can be easily included. Figure 1shows the presented system structure using two pointclouds of a surface containing of a sack from a logis-tic scenario. The clouds represent a partial view of thesack object. A screen shot of the main graphical userinterface is visualized in figure 2. The system is im-plemented in the programming language C++ and Qtfor the graphical user interface. It uses PCL [2] andOpenCV [9] as main libraries for point cloud process-ing and computer vision algorithms. In the followingsection the single steps are described in more detailand methods that are integrated in the test environ-ment are presented.

3.1 Preprocessing

The preprocessing module contains algorithms for en-hancing the quality or the size of the data. Improvingthe quality of the range data has a particular impor-tance regarding noisy sensor data. Due to the workingprinciple of local feature description techniques, mea-surement noise has a strong influence on the resultingfeature descriptor. Therefore, different filters are im-plemented or integrated in the test environment whichcan be applied for reducing the influence of measure-ment noise such as median filtering or outlier removal.For some applications, the descriptiveness of a surfacedescription is also sufficient with a reduced numberof points. Downsampling of point clouds can signif-icantly improves the computation time and is also in-tegrated as module in the preprocessing step.

3.2 Keypoint Detection

In order to prevent computing feature descriptions foreach point in both clouds, keypoints are identified byapplying keypoint detectors. A key point is a pointwithin a point cloud that has a specific property likecorner or other geometric significant properties of anobject in the scene. A famous keypoint from 2D com-puter vision is the SIFT-keypoint (Scale Invariant Fea-ture Transform) proposed by David G. Lowe [10].The detected features are highly distinctive and are in-variant to image scale and rotation. There exists manyresearch work transferring the concept to the three-dimensional domain such as [11][12]. Within the test

environment the version from PCL is used as modulefor key point detection.Another famous keypoint detector from 2D domainis the Harris corner detector [13]. The adaption topoint clouds uses surface normals instead of imagegradients for detecting corners [14]. The last keypointdetector integrated in the test environment is calledNARF (Normal Aligned Radial Feature) [15]. Themethod analyzes object borders and extract featuresin regions where the investigated surface is stable buthas large changes in the neighborhood. The final key-points are chosen by using non-maximum suppres-sion. Within the framework, keypoints based on lo-cal principal curvature values and the Intrinsic ShapeSignatures (ISS)[16] keypoint detector are also inte-grated.

3.3 Feature Description

Local feature description techniques describe pointsaccording to a specific neighborhood. Surface nor-mals or curvatures are not distinctive enough to beused as a robust local feature descriptor. There-fore, multi-dimensional feature spaces are commonlyused in order to classify points lying on the samesurface to the same shape class. Feature descrip-tion can be distinguished by using signatures orhistogram-based feature descriptors. The presentedtest environment focuses on generating and compar-ing histogram-based feature description techniques.The integrated feature descriptors are Point-FeatureHistograms (PFH)[17], Fast Point Feature Histograms(FPFH)[3], and Spin Images [6]. The main workingprinciple of these techniques are to determine a localreference frame and describing geometric propertiesbetween source and neighboring points. Finally, thedescription is stored in an histogram.

3.4 Correspondence Estimation and Rejec-tion

After finding keypoints and computing the corre-sponding feature descriptor, point correspondences inboth point clouds have to be computed. Thereby, thebest correspondences should be used for estimatingthe coarse registration between the clouds. There-fore, wrong correspondences must be automaticallyrejected since they can negatively effect the estima-tion. Within the test environment, this is realizedby applying median rejection and the RANSAC al-gorithm [18]. The median rejection step rejects allcorrespondences with correspondence values k-timeshigher than the median of the complete data.After applying median rejection the RANSAC al-gorithm is applied in order to find the best sub-


ISBN: 978-960-474-317-9 128

Figure 2: Screenshot of the main GUI

set of correspondences and rejecting wrong matches.RANSAC is an iterative method that produces resultswith a certain probability which is strongly affectedby the number of iterations.

3.5 Fine Registration

The resulting coarse transformation can be refined byapplying the ICP-algorithm. The convergence crite-ria of the algorithm are the maximum correspondencedistance and the maximum number of iterations. Bothparameters can be specified within the test environ-ment. In addition to the fine registration matrix, theICP delivers also a value indicating the accuracy ofthe estimation.

4 Example ApplicationThe functionality and benefits of the test environmentare demonstrated using two examples point clouds ofa sack object. The two point clouds are rotated by20 degrees towards each other. Therefore, a sufficientamount of overlap is guaranteed. The clouds are ac-quired by using the Microsoft Kinect Sensor. Nat-urally, the sensor data acquisition could be realizedwith all possible range sensors. Nevertheless, for ap-

plication of the SIFT keypoint detection method coloror intensity information must be also available.Finding the optimal combination and configuration ofthe single modules of the registration pipeline leads tooptimize several parameters depending on the selectedconfiguration. Here, the selection strongly dependson the properties of the sensor data such as influenceof measurement noise or point density and has to bespecified according to the investigated application. Inorder to reduce the influence of measurement noise orreducing the amount of data, the environment offers aset of filters for improving data quality or downsam-pling the cloud to a specific size. The first criticalchoice in the registration pipeline is the selection ofthe keypoint detection method. Figure 3b-d shows theresults of the SIFT, HARRIS, and keypoints based onprinciple curvatures6 for one of the sack objects. Theresults show that each detector delivers different key-points. After localizing keypoints in both clouds, fea-ture description for each detected keypoint are gener-ated based on the local neighborhood. As mentionedin section 3, the test environment offers various localfeature description techniques. Additionally, differentdistance metrics for histogram comparison can be se-lected. Keypoints with a similar feature descriptionare classified as a possible correspondence. Thereby,wrong correspondences can negatively effect the reg-


ISBN: 978-960-474-317-9 129

Figure 3: a) Point Cloud of a sack object b) HARRISc) SIFT d) Principal Curvature

istration results. Therefore, a RANSAC module canbe configured an used for wrong correspondences re-jection. Figure 4 shows the resulting correspondencesfor the SIFT and HARRIS keypoints with FPFH fea-ture descriptions after applying RANSAC for reject-ing wrong correspondences.

Figure 4: a) SIFT keypoints b) HARRIS keypoints

After rejecting wrong correspondences, the trans-formation matrix is based on the remaining correspon-dences. For fine registration, the ICP algorithm can beapplied on the coarse registration transformation in or-der to optimize the registration result. Figure 5 showsthe two clouds and the final merged cloud after appli-cation of the ICP algorithm.

For the presented application scenario, usingSIFT keypoints with FPFH has shown the best re-

Figure 5: a) Two clouds of the sack object b) Mergedcloud

sults concerning registration accuracy. Nevertheless,the selection and configuration of the single modulesstrongly depends on the investigated application sce-nario.

5 ConclusionRange data registration methods can be distinguishedin coarse and fine registration methods. Coarse reg-istration methods estimate a transformation withoutany knowledge or initial estimate about the transfor-mation. Fine registration methods optimize an initialtransformation in order to find the most accurate trans-formation between the clouds. The coarse registrationof range data is often performed by identifying point-to-point correspondences in the point clouds. There-fore, local feature-based methods applied on key-points in the point cloud data have a high potentialfor finding correct point-correspondences due to theirhigh descriptiveness. Nevertheless, selecting the rightmethods and techniques and determining the best pa-rameter can be a time-consuming step and often needsexpert knowledge.In this paper we presented a test environment for afeature-based range data registration pipeline. By us-ing the test environment, the best configuration depen-dent on a specific application can be easily selectedand evaluated. Additionally, single configurations canbe stored and automatically applied to other experi-ment data. In further research, we will use our testenvironment for specific applications from the field ofproduction and logistics. The production scenario isthe combining partial views of micro parts to a com-plete 3D representation in order to perform quality in-spection tasks. The partial range data is acquired by aconfocal laser microscope. Within the logistic appli-cation scenario, the registration of partial views fromlogistic goods such as boxes, barrels or sacks will beperformed. Here, the objective is also a full 3D repre-sentation in order to enable an automatic handling by


ISBN: 978-960-474-317-9 130

robotic systems.

Acknowledgements: The authors gratefully ac-knowledge the financial support by DFG (GermanResearch Foundation) for Subproject B5 ”SichereProzesse” within the SFB 747 (Collaborative Re-search Center) Mikrokaltumformen - Prozesse,Charakterisierung, Optimierung”.

References:

[1] Joaquim Salvi, Carles Matabosch, David Fofi,and Josep Forest. A review of recent range im-age registration methods with accuracy evalu-ation. Image Vision Comput., 25(5):578–596,2007.

[2] R.B. Rusu and S. Cousins. 3d is here: Pointcloud library (pcl). In Robotics and Automa-tion (ICRA), 2011 IEEE International Confer-ence on, pages 1–4, 2011.

[3] Radu Bogdan Rusu, Nico Blodow, and MichaelBeetz. Fast point feature histograms (fpfh) for3d registration. In Robotics and Automation,2009. ICRA ’09. IEEE International Conferenceon, 2009.

[4] P.J. Besl and Neil D. McKay. A method forregistration of 3-d shapes. Pattern Analysisand Machine Intelligence, IEEE Transactionson, 14(2):239–256, 1992.

[5] ChinSeng Chua and Ray Jarvis. Point signa-tures: A new representation for 3d object recog-nition. International Journal of Computer Vi-sion, 25(1):63–85, 1997.

[6] Andrew E. Johnson and Martial Hebert. Usingspin images for efficient object recognition incluttered 3d scenes. IEEE Transactions on Pat-tern Analysis and Machine Intelligence, pages433–449, 1999.

[7] Do Hyun Chung, Il Dong Yun, and Sang Uk Lee.Registration of multiple-range views using thereverse-calibration technique. Pattern Recogni-tion, 31(4):457 – 464, 1998.

[8] J. P. Tarel, H. Civi, and D. B. Cooper. PoseEstimation of Free-Form 3D Objects withoutPoint Matching Using Algebraic Surface Mod-els. Conference, pages 13–21, 1998.

[9] G. Bradski. The OpenCV Library. Dr. Dobb’sJournal of Software Tools, 2000.

[10] David G. Lowe. Distinctive image features fromscale-invariant keypoints. Int. J. Comput. Vision,60(2):91–110, 2004.

[11] Alex Flint, Anthony Dick, and Anton van denHengel. Thrift: Local 3d structure recognition.In Digital Image Computing Techniques and Ap-plications, 9th Biennial Conference of the Aus-tralian Pattern Recognition Society on, pages182 –188, dec. 2007.

[12] Paul Scovanner, Saad Ali, and Mubarak Shah. A3-dimensional sift descriptor and its applicationto action recognition. In Proceedings of the 15thinternational conference on Multimedia, MUL-TIMEDIA ’07, pages 357–360, 2007.

[13] C. Harris and M. J. Stephens. A Combined Cor-ner and Edge Detector. In In Proc. of FourthAlvey Vision Conference, pages 147–152, 1988.

[14] Ivan Sipiran and Benjamin Bustos. A robust 3dinterest points detector based on harris opera-tor. In Proceedings of the 3rd Eurographics con-ference on 3D Object Retrieval, EG 3DOR’10,pages 7–14, 2010.

[15] B. Steder, R. B. Rusu, K. Konolige, and W. Bur-gard. NARF: 3D range image features for objectrecognition. In Workshop on Defining and Solv-ing Realistic Perception Problems in PersonalRobotics at the IEEE/RSJ Int. Conf. on Intelli-gent Robots and Systems (IROS), Taipei, Taiwan,2010.

[16] Yu Zhong. Intrinsic shape signatures: Ashape descriptor for 3d object recognition. InComputer Vision Workshops (ICCV Workshops),2009 IEEE 12th International Conference on,pages 689–696, 2009.

[17] R.B. Rusu, N. Blodow, Z.C. Marton, andM. Beetz. Aligning point cloud views using per-sistent feature histograms. In Intelligent Robotsand Systems, 2008. IROS 2008. IEEE/RSJ In-ternational Conference on, pages 3384 –3391,2008.

[18] Martin A. Fischler and Robert C. Bolles.Random sample consensus: a paradigm formodel fitting with applications to image analy-sis and automated cartography. Commun. ACM,24(6):381–395, 1981.


ISBN: 978-960-474-317-9 131

Documents

A Test Environment for Feature-Based Range Data Registration