3D data segmentation using a non-parametric density

Eurographics Italian Chapter Conference (2006)G. Gallo and S. Battiato and F. Stanco (Editors)

3D data segmentation using a non-parametric densityestimation approach

U. Castellani, M. Cristani, V. Murino

Dipartimento di Informatica, Universiy of VeronaStrada le Grazie 15, 37134 Verona, Italy

AbstractIn this paper, a new segmentation approach for sets of 3D unorganized points is proposed. The method is based ona clustering procedure that separates the modes of a non-parametric multimodal density, following the mean-shiftparadigm. The main idea consists in projecting the source3D data into a set of independent sub-spaces, forminga joint multidimensional space. Each sub-space describes a geometric aspect of the data set, such as the normalsand principal curvatures, so as a dense region in a particular sub-space indicates a set of 3D points sharinga similar value of that feature. A non-parametric clustering method is appliedin this joint space by using amultidimensional kernel. This kernel smoothly takes into account for all thesubspaces, moving towards highdensity regions in the joint space, separating them and providing “natural”clusters of 3D points. The algorithmcan be implemented very easily and only few parameters need to be freely tuned. Experiments are reported, bothon synthetic and real data, assessing the quality of the proposed approach and promoting further developments.

Categories and Subject Descriptors (according to ACM CCS): I.4.6 {Image Processing and Computer Vi-sion}{Segmentation}[Partitioning]

1. Introduction

Segmentation is a vast and complex domain, both in termsof problem formulation and resolution techniques. It con-sists in formally translating the delicate visual notions of ho-mogeneity and similarity, and defining criteria which allowtheir efficient implementation [Pet02]. The goal is to parti-tion the source data into meaningful pieces, i.e. those partscorresponding to the different entities, in the physical and se-mantical sense of the application envisioned. In the realm of3D data, the segmentation comprises numerous applications,for example texture atlas generation [LPRM02, SWG∗03],shape simplification [CSAD04, KT96], shape recognitionand matching [AA93, CF01], and shape modelling and re-trieval [FKS∗04,KT03,FK04]. The literature is vast and sev-eral surveys report interesting approaches for different datarepresentations such as unorganized points, range image,or 3D polygonal meshes [Pet02,HJBJ∗96,AA93]. Roughlyspeaking, the segmentation methods can be categorized intotwo main classes:edge-basedandregion-based[Pet02]. Inthe former, features corresponding to part boundaries arefirst detected and then regions are built, each one formed by

sets of points delimited by the same boundary. In the latter,points sharing the same similarity property are grouped to-gether. In particular, three are the most popular approaches toregion-based segmentation:split-and-mergemethods, iden-tified by a top-down paradigm;region-growingmethods, thatadopt a bottom-up paradigm, andclustering-basedmethods,based on the projection of the points onto a higher dimen-sional space where the clusters (i.e., segments) are recoveredby defining some particular distance functions [JMF99]. Inthis paper a newclustering-basedmethod for 3D unorga-nized points segmentation is proposed. The segmentationis obtained from the source data by introducing a non-parametric density estimation approach based on the meanshift paradigm [CM].The mean shift (MS) clustering operates by shifting a fixedsize estimation window from each data point towards the di-rection of maximal density, and converging into a basin ofattraction, that represents a local mode. The points converg-ing to the same centroid belong to the same region.

Although the mean shift has shown to be a powerful tech-nique for several fields of research such as image and video

c© The Eurographics Association 2006.

U. Castellani & M. Cristani & V. Murino / 3D data segmentationusing a non-parametric density estimation approach

segmentation [CM,WTXC04], tracking [Col03], clustering,and data mining [GSM03], very few works have been ad-dressed to it within the context of three-dimensional datasegmentation. In [HS03], the mean shift has been appliedon the histogram of the principal curvatures in order torecover geometric primitives from range images. In sum-mary, instead of classifying the surface patches on the ba-sis of the mean and the gaussian curvatures values com-binations [AA93], the authors have proposed to detect thetype of primitives by analyzing the clusters obtained ontothe principal curvatures space. In [Sha04], the 3D polygo-nal mesh is projected onto a 2D image by preserving thegeodesic distances among the points and by coloring eachpixel according to the curvature value of its corresponding3D point. Therefore, the segmentation is obtained by adopt-ing the standard mean shift approach for 2D image segmen-tation [CM]. In [YLL ∗05], the 3D polygonal mesh is di-rectly analyzed and the authors proposed to apply the meanshift algorithm on the surface normals space (i.e., the featurespace) as a pre-processing stage. The segmentation is thencompleted by adopting a classical region-growing approach.

In this paper, the source data are unorganized 3D points.For each point, both the normal (such as in [YLL ∗05]) andprincipal curvatures (such as in [Sha04]) are extracted andare joined onto the feature space.

The contribution of the paper is twofold: first, we as-sess the effectiveness of a non-parametric clustering para-digm as 3D segmentation approach for sets of unorganizeddata points; second, we show that in such a framework, theenrichment of the set of features makes the segmentationmore accurate, without the necessity of special pre- or post-processing.

The rest of the paper is organized as follow. After describ-ing an overview of the mean shift procedure in Section2, thedetails of the proposed method are reported in Section3. Re-sults of the algorithm on synthetic and real images are shownin Section4, and finally, in Section5 conclusions are drawnand future perspectives are envisaged.

2. Mean Shift

The mean shift procedure is an old non-parametric densityestimation technique [Fuk90]. The main underlying idea isthat the data feature space is regarded as an empirical prob-ability density function to estimate: therefore, a big concen-tration of points that fall near the locationx indicates a bigdensity nearx.The theoretical framework of the mean shift arises from theparzen windows [DHS01] basic expression, i.e. the kerneldensity estimator, that is

f̂ (x) =1n

n

∑i=1

KH(x−xi) (1)

where f̂ (x) represents the approximated density calculatedin thed-dimensional locationx, n is the number of available

points and

KH(x) = |H|−1/2K(H−1/2x). (2)

Here above,KH can be imagined as a weighted window usedto estimate the density, dependent on the kernelK and thesymmetric positive definited×d bandwidth matrixH. ThefunctionK is a bounded function with compact support (forfull details, see [CM]); the bandwidth matrix codifies the un-certainty associated to the whole feature space.

In the case of particular radial symmetric kernels (see[CM]), K can be specified using only a 1-dimensional func-tion, theprofile k(·), the same for each dimension. Moreover,if we assume independence among the feature dimensionsand equal uncertainty over them, the bandwidth matrix canbe rewritten as proportional to the identity matrixH = h2I.Under such hypotheses, Eq.2 can be rewritten as

f̂h,K(x) =ck,d

nhd

n

∑i=1

k

(

∣

∣

∣

∣

∣

∣

x−xi

h

∣

∣

∣

∣

∣

∣

2)

(3)

whereck,d is a normalizing constant. The kernel profilek(·)models how strongly the points{xi} are taken into accountfor the estimation, in dependence with their distance tox.

Mean shift extends this “static” expression, differentiating(3) and obtaining the density gradient estimator

∇̂ fh,K(x)

=2ck,d

nhd

[

n

∑i=1

g

(

∣

∣

∣

∣

∣

∣

xi −xh

∣

∣

∣

∣

∣

∣

2)

]

∑ni=1 xig

(

∣

∣

∣

∣

xi−xh

∣

∣

∣

∣

2)

∑ni=1 g

(

∣

∣

∣

∣

xi−xh

∣

∣

∣

∣

2) −x

(4)where g(x) = k′(x); This quantity is composed by threeterms; the second one isproportionalto the normalized den-sity gradient obtained with the kernelK.The third one is themean shiftvector, that is guaranteed topoint towards the direction of maximum increase in the den-sity (see [CM]). Therefore, starting from a pointxi in thefeature space, the mean shift produces iteratively a trajec-tory that converges in a stationary pointyi , representing amode of the whole feature space.

3. The proposed method

Our segmentation method can be thought as a cluster-ing process, derived from the approach proposed in [CM].Briefly speaking, the first step of such process is made byapplying the mean shift procedure to all the points{xi},producing several convergency points{yi}. A consistentnumber of close convergency locations,{yi}l , indicates amodeµl . The labeling consists in marking the correspond-ing points{xi}l that produces the set{yi}l with the labell .This happens for all the convergency locationl = 1,2, . . . ,L.

In this paper, we consider each point of the cloud as ad-dimensional entity, living in ajoint domain. In specific, eachxi is formed by thex1,x2,x3 spatial coordinates relative to the



x,y,z axes (thespatial sub-domain), then1,n2,n3 normal co-ordinates (thenormal sub-domain) and thec curvature (thecurvature sub-domain). In particular, the curvature is mod-elled by the so calledcurvedness-index[Pet02]:

CurvnInd(k1,k2) =2π

ln

√

k21 +k2

22

(5)

wherek1 and k2 are the principal curvatures [AA93]. Foreach sub-domain we assume Euclidian metric.In order to explore the joint domain, a multivariate kernel isused [CM,WTXC04], that has the form

Khs,hn,hc(xi)

=C

h3s,h3

n,hck

(

∣

∣

∣

∣

∣

∣

∣

∣

xi,s

hs

∣

∣

∣

∣

∣

∣

∣

∣

2)

k

(

∣

∣

∣

∣

∣

∣

∣

∣

xi,n

hn

∣

∣

∣

∣

∣

∣

∣

∣

2)

k

(

∣

∣

∣

∣

∣

∣

∣

∣

xi,c

hc

∣

∣

∣

∣

∣

∣

∣

∣

2)

(6)

wherexi,s indicates the spatial coordinates of thei−th pointand so on forxi,n and xi,c; C is a normalization constant,andhs,hn,hc are the kernel bandwidths for each sub-domain.These values give to each feature domain the intuitive con-cept of “importance”: strictly speaking, the bigger is thethe related kernel bandwidth, the less important is that fea-ture. In other words, a big amplitude of the kernel tends toagglomerate points in few convergence locations, while asmall kernel highlights better local modes, encouraging clus-ter separations.In this paper we use the Epanechnikov kernel [CM], that canbe described by the profile

k(x) =

{

1−x if 0 ≤ x≤ 10 otherwise

(7)

that differentiated leads to the uniform kernel, i.e. ad-dimensional unit sphere.

4. Experiments

The proposed method has been tested first on synthetic data,in order to give deep and clear insight to how it works.Gaussian noise has been added to each synthetic test in orderto improve the realism of the experiments. Then, the methodhas been applied on real noisy data acquired with different3D sensors. In all the experiments, we perform segmentationon clouds of 7-dimensional points, as explained in Sec.3, in-dividuating 3 separated sub-domains (i.e., the spatial coor-dinates, the normals and the curvatures sub-domains). Thenormals and the principal curvatures are computed by usingclassical quadric fitting estimation [Pet02] and the curved-ness index is recovered from eq. (5). From the source data,points having a degenerated value in computing the quadricfitting have been discarded. The kernel bandwidth valueshave been chosen easily, by following the principle ex-plained above in Sec.3. The current implementation of theproposed method is working under the Matlab environment.

4.1. Synthetic data

The first test has been run on two intersecting planes, eachone opportunely sampled with 200 equally distanced points.Figure1.a shows the image before the segmentation. Since

0

10

20

30

05

1015

2025−5

0

5

10

15

20

25

0

10

20

30

05

1015

2025−5

0

5

10

15

20

25

(a) (b)

Figure 1: Planes: sampled points (a) and segmentation ob-tained using our method (b)

the two planes are intersecting each other, they form a sin-gle connected component, therefore the spatial domain is notenough for segmenting correctly the scene. Thus, by intro-ducing the information about the points directions (i.e., thenormals) the joined space reveals two well separated clus-ters and the correct segmentation is obtained (Figure1.b). Itis worth noting that, for this experiment, the curvature infor-mation is not useful since all the points have the same flatcurvature value.

The second experiment is performed on a plane trespassedby a gauge (Figure2.a). For this experiment the normals arenot enough for separating the two objects since they are notdiscriminative for the identification of the gauge. In fact, thegauge is composed of points having normals towards all thedirections. Therefore, by introducing the information on thecurvature the two objects are better distinguishable. In Fig-ure2.b the final result is proposed.

−505101520−20

24

0

0.5

1

1.5

2

2.5

3

3.5

4

−505101520−20

24

0

0.5

1

1.5

2

2.5

3

3.5

4

(a) (b)

Figure 2: Plane and gauge: sampled points (a) and segmen-tation obtained using our method (b)

4.2. Real data

The proposed method has been tested on two sets of rangeimages acquired with different kind of sensors. The first set



is from theMinolta database [CF98]. The scene consists ofthree spheres on a plane. Even if the scanner is a high res-olution sensor, the image is quite noisy, especially onto theareas among the spheres (Figure3.a). After the segmentationthe result is reasonable since the three spheres are correctlyseparated (Figure3.b). It worth noting that the plane on thebackground has been split into two parts. This is correctsince the points on the upper central parts are attracted to-ward the points on their side. Furthermore, the sphere in themiddle has been split into some small disconnected pieces.This is because in the proposed method no information aboutthe points connectivities have been take into account. We areaddressing this issue for the future works by using the con-nectivities information coming from either the range imageor the polygonal mesh.

−80 −60 −40 −20 0 20 40 60 80 100

−80

−60

−40

−20

0

20

40

60

(a)

−80 −60 −40 −20 0 20 40 60 80 100−50

−40

−30

−20

−10

0

10

20

30

−1200

−1150

−1100

(b)

Figure 3: Spheres: sampled points (a) and segmentation ob-tained using our method (b)

The second set is from theEchoScope[HA96] which isa 3Dacousticcamera for underwater scene acquisition. Thescene consists of a big pillar on the left, the seabottom, andtwo pipes on the right (Figure4). The data are very noisy andthe objects on the scene are very little recognizable. Afterthe segmentation the result is fully convincing since the fourobjects are correctly separated (Figure5).

It is worth noting that for all the proposed experiments,the parameters for the kernel definition have been manu-ally tuned by using trials and errors. Since each sub-space iswell bounded, the estimation of such parameters is not reallytricky. Future works will address the automatic estimation ofthe kernel dimensions by exploiting some machine learningtechniques, as suggested in [GSM03].

−500−400−300−200−1000100200300

−600

−500

−400

−300

−200

−100

0

100

200

300

400

−2000−1000

0 −500−400−300−200−1000100200300 −600−400−2000200400

−2000

−1800

−1600

−1400

−1200

−1000

−800

−600

−400

−200

0

(a) (b)

Figure 4: Underwater acoustic scenario: front view (a) andtop view(b) of the sampled points

−600−500−400−300−200−1000100200300400

−400

−300

−200

−100

0

100

200

300

−2000

−1000 −600−500−400−300−200−1000100200300400−400−2000200

−2000

−1800

−1600

−1400

−1200

−1000

−800

−600

−400

−200

(a) (b)

Figure 5: Underwater acoustic scenario: front view (a) andtop view (b) of the segmentation obtained using our method

5. Conclusions

In this paper, we introduce a 3D segmentation technique de-rived by the mean shift procedure. The geometric attributesof the source data are extracted and modelled onto a set ofsubspaces that define the global feature space, on which theclusters are recovered. The main aim is to show that the non-parametric paradigm derived from the MS strategy well be-haves with 3D segmentation issues. Moreover, we show thatin such a framework, the enrichment of the set of featuresmakes the segmentation more accurate, without the neces-sity of special pre- or post-processing. The proposed methodconsiders as a distinct object the cloud of points exploitingjointly high similarity in the different feature spaces, wherethe importance of each feature can be easily modulated bythe user. This permits to avoid the fitting of rigid paramet-ric models with the data; actually, such an operation needs



heavy tuning steps, that our approach in fact disregards. Theresults are promising, both on the synthetic and real datacases, characterized by different levels of noise. Further re-search is currently under study, specially devoted to makeautomatic the phase of kernel selection. Furthermore, a gen-eral framework for the extension of the proposed approachto polygonal meshes and range images is in progress.

References

[AA93] A RMAN F., AGGARWAL J. K.: Model-based ob-ject recognition in dense-range images: a review.ACMComput. Surv. 25, 1 (1993), 5–43.

[CF98] CAMPBELL R., FLYNN P.: A www-accessible3d image and model database for computer vision re-search. InEmpirical Evaluation Methods in ComputerVision(1998), Bowyer K., Phillips P., (Eds.), IEEE Com-puter Society Press, pp. 148–154.

[CF01] CAMPBELL R. J., FLYNN P. J.: A survey of free-form object representation and recognition techniques.Comput. Vis. Image Underst. 81, 2 (2001), 166–210.

[CM] COMANICIU D., MEER P.: Mean Shift: A RobustApproach Toward Feature Space Analysis.

[Col03] COLLINS R.: Mean-shift blob tracking throughscale space. InCVPR (2)(2003), pp. 234–240.

[CSAD04] COHEN-STEINER D., ALLIEZ P., DESBRUN

M.: Variational shape approximation.ACM Trans. Graph.23, 3 (2004), 905–914.

[DHS01] DUDA R., HART P., STORK D.: Pattern Classi-fication, second ed. John Wiley and Sons, 2001.

[FK04] FUNKHOUSER T., KAZHDAN M.: Shape-basedretrieval and analysis of 3d models. InGRAPH ’04: Pro-ceedings of the conference on SIGGRAPH 2004 coursenotes(New York, NY, USA, 2004), ACM Press, p. 16.

[FKS∗04] FUNKHOUSER T., KAZHDAN M., SHILANE

P., MIN P., KIEFER W., TAL A., RUSINKIEWICZ S.,DOBKIN D.: Modeling by example.ACM Trans. Graph.23, 3 (2004), 652–663.

[Fuk90] FUKUNAGA K.: Statistical Pattern Recognition,second ed. Academic Press, 1990.

[GSM03] GEORGESCU B., SHIMSHONI I., MEER P.:Mean shift based clustering in high dimensions: A textureclassification example. InICCV ’03: Proceedings of theNinth IEEE International Conference on Computer Vision(2003), pp. 456–463.

[HA96] HANSEN R. K., ANDERSEN P. A.: A 3-D un-derwater acoustic camera - properties and applications. InAcoustical Imaging, P.Tortoli, L.Masotti, (Eds.). PlenumPress, 1996, pp. 607–611.

[HJBJ∗96] HOOVER A., JEAN-BAPTISTE G., JIANG X.,

FLYNN P., BUNKE H., GOLDGOF D., BOWYER K., EG-GERT D., FITZGIBBON A., FISHER R.: An experimen-tal comparison of range segmentation algorithms.IEEETransactions on Pattern Analysis and Machine Intelli-gence 7, 18 (1996), 673–689.

[HS03] HAMEIRI E., SHIMSHONI I.: Estimating the prin-cipal curvatures and the darboux frame from real 3-drange data.IEEE Transactions on Systems, Man and Cy-bernetics 33, 4 (2003), 626–637.

[JMF99] JAIN A. K., M URTY M. N., FLYNN P. J.: Dataclustering: a review.ACM Comput. Surv. 31, 3 (1999),264–323.

[KT96] K ALVIN A. D., TAYLOR R. H.: Superfaces:Polygonal mesh simplification with bounded error.IEEEComputer Graphics and Applications 16, 3 (1996), 64–77.

[KT03] K ATZ S., TAL A.: Hierarchical mesh decomposi-tion using fuzzy clustering and cuts.ACM Trans. Graph.22, 3 (2003), 954–961.

[LPRM02] LÉVY B., PETITJEAN S., RAY N., MAILLOT

J.: Least squares conformal maps for automatic textureatlas generation. InSIGGRAPH ’02: Proceedings of the29th annual conference on Computer graphics and in-teractive techniques(New York, NY, USA, 2002), ACMPress, pp. 362–371.

[Pet02] PETITJEAN S.: A survey of methods for recover-ing quadrics in triangle meshes.ACM Comput. Surv. 34,2 (2002), 211–262.

[Sha04] SHAMIR A.: Geodesic mean shift. InProceedingsof the 5th Korea Israel conference on Geometric Modelingand Computer Graphics,(2004), pp. 51–56,.

[SWG∗03] SANDER P. V., WOOD Z. J., GORTLER S. J.,SNYDER J., HOPPE H.: Multi-chart geometry images.In SGP ’03: Proceedings of the 2003 Eurographics/ACMSIGGRAPH symposium on Geometry processing(Aire-la-Ville, Switzerland, Switzerland, 2003), Eurographics As-sociation, pp. 146–155.

[WTXC04] WANG J., THIESSONB., XU Y., COHEN M.:Image and video segmentation by anisotropic kernel meanshift. In ECCV (2)(2004), pp. 238–249.

[YLL ∗05] YAMAUCHI H., LEE S., LEE Y., OHTAKE Y.,BELYAEV A., SEIDEL H.: Feature sensitive mesh seg-mentation with mean shift. InShape Modeling Interna-tional 2005(2005), pp. 236–243.


Documents

3D data segmentation using a non-parametric density