Neurocomputing 120 (2013) 336–345

Contents lists available at ScienceDirect

Neurocomputing

0925-23http://d

n CorrE-m

journal homepage: www.elsevier.com/locate/neucom

Multi-scale gray level co-occurrence matrices for texture description

Fernando Roberti de Siqueira a, William Robson Schwartz b,n, Helio Pedrini a

a Institute of Computing, University of Campinas, Campinas, SP 13083-852, Brazilb Department of Computer Science, Universidade Federal de Minas Gerais, 31270-010, Brazil

a r t i c l e i n f o

Article history:Received 24 December 2011Received in revised form8 August 2012Accepted 10 September 2012Available online 2 April 2013

Keywords:Multi-scale feature descriptorGray level co-occurrence matrixGLCMTexture descriptionImage analysis

12/$ - see front matter & 2013 Elsevier B.V. Ax.doi.org/10.1016/j.neucom.2012.09.042

esponding author. Tel.: +55 31 3409 5847.ail address: william@dcc.ufmg.br (W. Robson

a b s t r a c t

Texture information plays an important role in image analysis. Although several descriptors have beenproposed to extract and analyze texture, the development of automatic systems for image interpretationand object recognition is a difficult task due to the complex aspects of texture. Scale is an importantinformation in texture analysis, since a same texture can be perceived as different texture patterns atdistinct scales. Gray level co-occurrence matrices (GLCM) have been proved to be an effective texturedescriptor. This paper presents a novel strategy for extending the GLCM to multiple scales through twodifferent approaches, a Gaussian scale-space representation, which is constructed by smoothing theimage with larger and larger low-pass filters producing a set of smoothed versions of the original image,and an image pyramid, which is defined by sampling the image both in space and scale. The performanceof the proposed approach is evaluated by applying the multi-scale descriptor on five benchmark texturedata sets and the results are compared to other well-known texture operators, including the originalGLCM, that even though faster than the proposed method, is significantly outperformed in accuracy.

& 2013 Elsevier B.V. All rights reserved.

1. Introduction

Texture can be characterized by regular or random patternsthat repeat over a region [1]. As one of the most importantfeatures for image analysis, texture provides information regardingstructural arrangement of surfaces or changes in intensity or colorbrightness.

Despite the accuracy of the visual human system to recognizetextures, it is a complex task to define a set of textural descriptorsfor image analysis on different domains of knowledge. The largenumber of definitions and descriptors found in the literaturereflects such difficulty [2–12].

Although there is no a unique categorization of the mainrelevant methods for texture description, they can be classifiedas statistical approaches, signal-processing based approaches, geo-metrical approaches, and parametric-model based approaches [13].

Among the statistical approaches, gray level co-occurrencematrices (GLCM) have been proved to be a very powerful texturedescriptor used in image analysis. However, a drawback of theoriginal GLCM is its limited capability of capturing texture infor-mation at multiple scales.

In many descriptors, it is assumed that texture information isfixed at a specific image resolution. The discriminative power of

ll rights reserved.

Schwartz).

texture descriptors can be significantly improved if different scalesare considered among the images during the descriptor extraction.

This work presents a novel scheme for extending the GLCM tobe more robust under scale variation. Two different multi-scalerepresentations are used in the extension of the descriptor. Theperformance of the proposed approach is evaluated by applyingthe multi-scale descriptor on five benchmark texture data sets andthe results are compared to other powerful texture operators.

The paper is organized as follows. Section 2 presents somerelevant concepts and work related to texture descriptors. Section3 describes the proposed multi-scale texture descriptor. Experi-mental results are shown in Section 4. Finally, Section 5 concludesthe paper with final remarks.

2. Related work

The development and analysis of low-level feature descriptorshave been widely considered in the past years. Among the vastlyemployed methods are the scale-invariant feature transform (SIFT)[14], speeded up robust feature (SURF) [15], histogram of orientedgradients (HOG) [16], gradient location and orientation histogram(GLOH) [17], region covariance matrix (RCM) [18], edgelet [19],gray level co-occurrence matrix (GLCM) [20], local binary patterns(LBP) [21], color correlogram (CCG) [22], color coherence vectors(CCV) [23], color indexing [24], steerable filters [25], Gabor filters[26], and shape context [27]. Furthermore, several works

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345 337

comparing different feature descriptors can be found in theliterature [17,28,29].

Some descriptors have taken scale changes into account. Forinstance, extensions of LBP have been proposed to make it a moreeffective texture descriptor. One extension is the multi-resolutionLBP (MLBP) [30], which uses pixel neighborhood of different sizes.Another extension represents LBP descriptors in Gabor transformdomain (LGBP) [31]. A third variation extends LBP to pyramidtransform domain (PLBP) [32]. PHOG [33] represents an imagewith histograms of orientation gradients over spatial pyramids.HWVP [34] represents texture information of an image through ahierarchical wavelet packet transform. GIST [35] is a high dimen-sional descriptor that represents texture information with filteringbased on oriented multiple scale.

Low-level descriptors are designed to focus in visual character-istics such as texture, shape and color. Since this paper proposes amulti-scale extension of the texture-based method GLCM, wereview approaches to extracting textural information. Section 2.1describes a set of texture-based feature descriptors. Then, Section2.2 reviews the GLCM method and its extensions.

2.1. Texture descriptors

Many approaches to extracting textural information have beenproposed in the literature, including gray level co-occurrencematrices [20], gray level run length matrices [36], wavelet trans-forms [37], Gabor filters [38], texture unit [39], local binarypatterns [21], texture feature coding method [40], coordinatedclusters representation [41], granulometry [42], Markov randomfields [43], and simultaneous autoregressive models [44]. Thissection reviews those approaches that have achieved accurateresults in texture classification [29,45–47]. These descriptors arethen compared to the proposed method during the experimentalvalidation (Section 4).

Granulometry. The term granulometry is used in the field ofmaterials science to characterize the granularity of materials bypassing them through sieves of different sizes while measuringtheir mass retained by each sieve.

This principle can be transposed to the field of image proces-sing [42,48,49], where an operator consists in analyzing theamount of image detail removed by applying morphologicalopenings γλ of increasing size λ.

The mass is represented by the sum of the pixel values, knownas image volume (Vol). The volumes of the opened images areplotted against λ, producing a granulometric curve. The normal-ized version of the operator for an image f can be written as

Gðf Þ ¼ Volðγλðf ÞÞVolðf Þ ð1Þ

Negative values of λ can be interpreted as a morphologicalclosing operator with a structuring element of size λ.

Local binary patterns. The local binary patterns (LBP) descriptorcharacterizes the spatial structure of a texture and presents thecharacteristics of being invariant to monotonic transformations ofthe gray-levels [21]. On its standard version, a pixel c withintensity g(c) is labeled as defined by Eq. (2), where pixels pbelong to a 3�3 neighborhood with gray levels gp ðp¼0;1;2;…;7Þ:

Sðgp−gcÞ ¼1; gp≥gc0; gpogc

(ð2Þ

Then, the pattern of the pixel neighborhood for an image with 256intensity values is computed by summing the corresponding

thresholded values Sðgp−gcÞ weighted by a binomial factor of 2k as

LBP¼ ∑7

p ¼ 0Sðgp−gcÞ2p ð3Þ

After computing the labeling for each pixel of the image, a 256-binhistogram of the resulting labels is used as a feature descriptor forthe texture.

Several variations of LBP have been proposed, including theimproved local binary pattern (ILBP) [50]. Different from the LBP,the ILBP uses the average of the 3�3 neighborhood as thethreshold and also considers the central pixel to estimate its label,with values in the interval ½0;510�. The ILBP is defined as in thefollowing equation:

ILBP¼ ∑7

p ¼ 0Sðgp−mÞ2p þ Sðgc−mÞ28 ð4Þ

wherem denotes the average of the 3�3 neighborhood and S(x) isdefined as

SðxÞ ¼1; x400; x≤0

(ð5Þ

m¼ 19

∑7

p ¼ 0gp þ gc ð6Þ

Similar to LBP, once the labels have been computed for everypixel, a histogram is computed; in this case, a 511-bin histogramwill be used as feature vector.

Gabor filters. Gabor filters capture visual properties such asspatial localization, spatial frequency and orientation of thestructures present in the image. Widely employed to objectrecognition, Gabor filters present illumination invariance sincethey detect invariant spatial frequency [38]. The most commonform of the Gabor filters is shown in Eq. (7), where μ and ν denotethe orientation and scale of the Gabor kernels, z¼ ðx; yÞ, ∥ � ∥ is thenorm operator, and kμ;ν ¼ kνðcos ϕμ; sin ϕμÞ, in which kν ¼ kmax=f

ν

and ϕμ ¼ πμ=8 where kmax it the maximum frequency and f denotesthe spacing factor between kernels in the frequency domain:

ψμ;νðzÞ ¼∥kμ;ν∥2

s2eð−∥kμ;ν∥

2∥z∥2=2s2Þ½eikμνz−e−s2=2� ð7Þ

The feature vector extracted using the Gabor filters is obtainedwith the convolution of the gray-scale image with the filters.Let Iðx; yÞ be the image, its convolution with a Gabor filter isdefined according to Eq. (8), where n denotes the convolutionoperator:

Gψ Iðx; y; μ; νÞ ¼ Iðx; yÞnψμνðzÞ ð8Þ

In this work, we consider five scales μ∈f0;…;4g and eightorientations ν∈f0;…;7g, which result in 40 Gabor filters. All filtersare convolved and the resulting magnitudes are used as descrip-tors. After the convolution, the feature vector is composed by theconcatenation of the mean and standard deviation of theconvolved image.

2.2. Gray level co-occurrence matrix

An approach to extracting textural information regarding graylevel transition between two pixels uses a co-occurrence matrix.Given a spatial relationship defined among pixels in a texture,such matrix represents the joint distribution of gray-level pairs ofneighboring pixels. Therefore, matrices providing different infor-mation are obtained by modifying the spatial relationship (differ-ent orientation or distance between pixels). Descriptors areextracted from these matrices.

Fig. 1. Multi-scale approaches for GLCM. (a) Pyramid decomposition; (b) Gaussian smoothing.

Fig. 2. Examples of 25 texture samples extracted from UMD data set [63].

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345338

Fig. 3. Examples of 25 texture samples extracted from UIUC data set [62].

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345 339

The number of rows and columns of the co-occurrence matrixdepends only on the gray levels in the texture and not on theimage size. The element Pðm;nÞ of a co-occurrence matrix indi-cates the number of transitions between the gray level m and nthat take place in the texture according to a given spatial relation-ship. It is customary to quantize the number of gray levelintensities to reduce the size of the co-occurrence matrix. Thenumber of bins usually varies from 8 to 256.

Before computing the co-occurrence matrix, it is necessary todefine relations among pixels, that is, the arrangement of pixelsfrom which the transitions will be considered. A set S is built.Each element in this set is a pair of coordinates of each pixelinvolved in the relationship. Once S is defined, Eq. (9) is used tocount the number of transitions between each pair of gray levels inthe texture. In this equation, f ðx; yÞ indicates the gray level of apixel located at (x,y) in the image:

Pðm;nÞ ¼ fðði; jÞ; ðk; lÞÞ∈Sjf ði; jÞ ¼m and f ðk; lÞ ¼ ng ð9ÞOnce the frequency of each gray level transition is computed,

Pðm;nÞ is placed at the m-th row and n-th column of the matrix.Then, feature descriptors are extracted after a normalization basedon Eq. (10), where Hg denotes the largest gray level:

pm;n ¼Pðm;nÞ

∑Hg

i ¼ 0∑Hg

j ¼ 0Pði; jÞ; m;n¼ 0;…;Hg ð10Þ

According to Eq. (9), the co-occurrence matrix depends on thegray level transitions between pairs of pixels in set S. This way, it ispossible to arbitrarily specify the distance and the angle betweenthe pairs. Haralick et al. [20] defined specifically which transitionsshould be considered to compute co-occurrence matrices. Twoadditional parameters are included, d and θ. These parameters

define the displacement and angle between pixels in S, respec-tively. Therefore, several matrices can be obtained with smallchanges in these parameters.

To describe the properties contained in the co-occurrencematrices, Haralick et al. proposed 14 statistical measures that arecomputed from the matrices: angular second moment, contrast,correlation, sum of squares, inverse difference moment, sumaverage, sum variance, sum entropy, entropy, difference variance,difference entropy, two information measures of correlation, andmaximal correlation coefficient.

Besides the original version of the GLCM, several variationshave been proposed. Focusing on optimization, Clausi and Jernigan[51] employ linked lists exploiting the sparsity of the co-occurrence matrices to reduce the computation time; Tahir et al.[52] propose the use of field programmable gate arrays (FPGA) toaccelerate the calculation of GLCM and Haralick texture features,achieving speed-up of 9 times.

Extensions of the GLCM have been proposed. To increase thediscriminability of the descriptors, Gelzinis et al. [53] extractdescriptors considering simultaneously different values for para-meter d. Walker et al. [54] have proposed to form co-occurrencematrix-based features by weighted summation of GLCM elementsfrom areas presenting high discrimination. Furthermore, additionof color information has also been considered for co-occurrencematrices [55].

Multi-scale analysis has also been performed using the GLCM.Hu [56] and Pacifici et al. [57] consider multiple scales by changingthe window size from which the GLCM descriptors are extracted.Rakwatin et al. [58] propose that the image be rescaled to differentsizes, extracting co-occurrence descriptors from each size.Nguyen-Duc et al. [59] have obtained improved results on

Fig. 4. Examples of texture samples extracted from OuTex data set [67].

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345340

content-based image retrieval employing a combination of con-tourlet transform [60] and GLCM. First, the contourlet transform isperformed for four subbands of the image, then the GLCM featuresare extracted from each one.

Differently from previous works that have addressed multiplescales for GLCM, this paper exploits two approaches to performingthe multi-scale analysis on the images: pyramid decompositionand Gaussian smoothing. Furthermore, five strategies to combinethe descriptors extracted from each scale are considered andevaluated.

3. Multi-scale gray level co-occurrence matrix

Two common multi-resolution representations are Gaussiansmoothing and pyramid decomposition [61]. Gaussian smoothingis a space-scale representation, where a sequence of images atdifferent levels of space-scale are built with variable kernel sizes.In pyramid representation, the original image is progressivelyreduced at each level of the pyramid. Pyramid images can begenerated by applying the Gaussian smooth filtering, Laplacianoperator, low-pass filters of wavelet transform, among otherschemes.

The use of multiple scales can improve the discriminativepower of texture descriptors, since they are able to extractinformation that could not be present in certain scales. This workproposes an extension of the gray level co-occurrence matrix(GLCM) to multiple scales using the Gaussian smoothing andpyramid decomposition to generate a number of image represen-tations of the original image.

The co-occurrence matrix is created at each level of resolution,resulting in several scales. Features extracted from GLCM are thencombined into a single feature descriptor. All features areextracted from images in grayscale, that is, color data sets areconverted into grayscale before their extraction. Fig. 1 illustratesthe main steps of the multi-scale approaches.

Once the features are extracted from each scale, it is necessaryto combine them in a manner to take advantage of the multiplescale information. To achieve that, five different combinationstrategies are proposed to merge the features.

The first one, referred to as concatenation, is a simple union of thescales, that is, features extracted from each scale are concatenated ina single feature vector that contains all the information.

In the second method, referred to as local, a normalization isapplied to each feature descriptor extracted from a certain scale,then all normalized features are joined together through a simpleconcatenation.

The third scheme, (L+G), is achieved by applying a normal-ization to the feature descriptors extracted from each scale,followed by a global normalization of all features united in asingle feature vector.

The fourth strategy, referred to as corresponding, normalizescorresponding elements of the scales in the descriptor, that is, thefirst feature extracted from the first scale is normalized in relationto the first features of every other scale, then the second feature isnormalized according to the second feature of each scale, and soforth. This combination is adequate when features have verydiscrepant value intervals, even in a single scale.

Finally, since all the previous multi-scale combination strategiesincrease the size of the final feature descriptor, a fifth method, referred

Fig. 5. Examples of texture samples extracted from VisTex data set [66].

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345 341

to asweighted, is proposed to maintain the size of the descriptor equalto the number of features extracted from a single scale. Instead of anormalization, as used in the fourth method, a weighted mean is usedto combine the values of corresponding elements of each scale. Eachscale will be assigned with decreasing weight of value 1=2k, where k isthe number of the current scale.

4. Experimental results

This section describes and discusses the results obtainedthrough the evaluation of the feature descriptors applied totexture classification. All experiments are conducted on an IntelCore i7-2630QM processor, 2.2 GHz with 8 GB of RAM running64-bit Windows operating system. The method was implementedusing C++ programming.

We consider five texture data sets: UIUC [62], UMD [63],Brodatz [64], OuTex [65] and VisTex [66]. All data sets were usedto estimate and evaluate the parameters employed to assess the

effectiveness of the proposed multi-scale feature descriptor fortexture classification.

Section 4.1 presents a brief explanation of each data set used inour experiments and the classification protocols employed. InSection 4.2, we describe the parameter values used for eachmethod. Then, evaluations of the proposed approach and compar-isons to other feature extraction methods are presented anddiscussed in Section 4.3.

4.1. Data sets

This section describes the main characteristics of the five datasets used in our experiments.

4.1.1. UMD data setThe UMD high-resolution data set [63] contains images of

1280�960 pixels, with 1000 images split into 25 classes, givinga total of 40 samples per class. This data set includes images of

Fig. 6. Examples of texture samples extracted from Brodatz data set [64].

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345342

floor textures, plants, fruits, among others. A mosaic containingexamples of all classes can be seen in Fig. 2.

4.1.2. UIUC data setThe UIUC data set [62] is a texture data set composed of 1000

images of 640�480 pixels, distributed in 25 classes (variations ofwood, gravel, fur, carpet, brick, among others) with 40 sampleseach. Fig. 3 shows examples of images for each texture.

4.1.3. OuTex data setOuTex [67] is a framework for evaluation of texture classifica-

tion and segmentation. It contains several images and protocolsfor texture classification. Different textures were acquired underseveral illuminations and different orientations.

Several test suites for texture classification have been proposed.Out of all these, TC_00005 is used in this work since its results arenot saturated. This test suite contains 8832 sample images of32�32 pixels, belonging to 24 classes of textures. The classifica-tion protocol for this suite is composed of 100 different combina-tions for training and test. We execute for all of them and report

the average classification rates. Fig. 4 shows examples of textureimages.

4.1.4. VisTex data setVisTex [66] is a collection of texture images that are repre-

sentative of real world conditions. Our experiments included54 images of resolution with 512�512 pixels split into 16samples of 128�128 pixels, according to work described byArvis et al. [68]. Such images are available at the OuTex site[67] as test suite Contrib_TC_00006. According to this suite, foreach texture class, half of the samples are used in the training setand the other half are used as test data. Fig. 5 shows examples oftexture images.

4.1.5. Brodatz data setThe Brodatz [64] photo album is a widely used texture data set,

often treated as a benchmark for texture classification. It contains111 different texture classes with 512�512 pixels, which aresubdivided to compose several classes. The test suite Con-trib_TC_00004 is used in this work (available at the OuTex site

Table 2Classification rates (%) achieved when the pyramid decomposition is considered forthe GLCM and multiple approaches for feature descriptor combination areemployed, as described in Section 3 (due to limited space, we used the term L+G, which means local followed by global normalization).

Data set Scales Combination approach

Local L+G Corresponding Weighted Concatenation

UMD 2 90.84 88.27 92.81 91.67 93.033 88.26 88.14 95.28 91.74 94.074 89.88 89.88 94.65 91.77 94.52

UIUC 2 71.82 68.91 68.35 70.60 75.383 72.18 72.18 71.93 70.99 79.024 73.19 73.99 71.07 72.12 80.37

Brodatz 2 68.27 68.27 67.45 67.80 68.033 65.77 65.77 64.10 66.72 61.814 58.13 58.13 52.08 60.56 53.18

VisTex 2 90.28 90.28 86.11 93.98 94.683 90.05 90.05 80.79 93.06 94.914 87.04 87.04 68.98 93.75 92.59

OuTex 2 84.61 84.61 73.79 83.26 84.293 82.28 82.28 70.94 82.22 82.444 76.66 76.66 67.47 82.01 74.33

Table 3Classification rates (%) achieved when the Gaussian smoothing is considered for theGLCM and multiple approaches for feature descriptor combination are employed,as described in Section 3 (due to limited space, we used the term L+G, whichmeans local followed by global normalization).

Data set Scales Combination approach

Local L+G Corresponding Weighted Concatenation

UMD 2 92.40 88.33 90.59 91.99 93.253 88.25 88.25 93.89 92.29 94.284 88.46 88.46 95.28 92.42 94.94

UIUC 2 80.08 80.08 81.88 81.74 81.023 80.13 80.13 81.79 81.88 81.854 81.88 81.88 81.74 81.88 81.79

Brodatz 2 80.82 80.82 86.37 73.50 81.823 81.21 81.21 87.07 76.45 83.484 81.29 81.29 87.19 77.71 83.05

VisTex 2 94.44 91.20 94.21 94.21 96.063 91.90 91.90 93.98 93.52 95.144 91.90 91.90 96.06 93.06 95.37

OuTex 2 88.71 88.71 84.98 84.56 88.303 89.50 89.50 87.09 84.94 88.754 89.33 89.31 87.36 84.65 88.38

Table 1Parameter estimation for all data sets considering single scale GLCM (20 trainingsamples were considered for UMD and UIUC data sets). The results showclassification rates, in percentages.

Data set d Bins

8 16 32 64 128 256

UMD 1 90.07 90.86 91.60 91.62 91.29 91.562 91.49 92.36 92.66 92.44 92.17 92.013 91.90 92.60 92.68 92.52 92.59 92.78

UIUC 1 65.15 67.23 68.23 68.21 68.83 71.382 71.44 72.47 71.55 72.26 72.27 73.013 73.70 75.67 75.56 75.06 74.16 75.53

Brodatz 1 67.06 70.46 72.27 74.24 75.63 76.282 73.37 76.69 78.30 78.58 77.97 77.803 65.30 70.31 72.68 76.13 74.59 73.83

VisTex 1 93.75 93.98 93.52 93.75 93.98 94.442 93.98 93.52 92.82 92.82 93.52 93.293 90.74 90.97 90.51 90.97 91.20 91.20

OuTex 1 77.56 78.39 81.18 80.98 82.22 83.022 78.56 78.13 79.83 79.92 80.75 82.193 77.08 75.61 77.07 76.86 78.35 80.00

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345 343

[67]). It is composed of 2048 images of 64�64 pixels dividedequally among 32 classes. Ten combinations for training and testare considered for this suite. Fig. 6 shows a mosaic containingsamples of Brodatz data set.

4.2. Experimental setup

For all experiments, the nearest neighbor classifier is appliedafter all variables are normalized to present zero mean and unitvariance. In addition, principal component analysis (PCA) isapplied to avoid dimensionality issues—the estimation of the bestdimensionality is performed by cross-validation and it is allowedto vary between 5 and 40 dimensions. The dimensionality whichachieved the best classification rate for the data set was deter-mined and chosen, whose values are presented in Table 4.

A subset of 12 descriptors from the original 14 descriptorsdescribed in Section 2.2 is considered for the GLCM: angularsecond moment, contrast, correlation, sum of squares, inversedifference moment, sum average, sum variance, sum entropy,

entropy, difference variance, difference entropy and maximalcorrelation coefficient. Co-occurrence matrices for four orienta-tions are computed (01, 451, 901 and 1351) and used to composethe feature vector with 48 dimensions (12 per orientation).

The values for parameter d and the number of bins for theGLCM are experimentally estimated to be 1 and 256, respectively(these values are used throughout the experiments). Table 1 showsthe parameter estimation. According to the table, the higher thenumber of bins, the better are the results for most data sets.However, the displacement parameter d has not presented thesame behavior. For data sets with larger samples such as UMD andUIUC, larger values of the displacement provide better results,while the opposite behavior takes place in data sets with sampleswith small size (VisTex and OuTex).

Regarding the remaining feature descriptors that will beconsidered in the comparison, for the granulometry we usedkernel size equal to 25 and step equal to 2 in UIUC and UMD datasets, and kernel size equal to 5 and step equal to 1 for theremaining data sets, which leads to feature vectors with 104 and40 dimensions, respectively. In Gabor filters, we used 8 orienta-tions and 5 scales, generating 40 filters for convolution. For eachfiltered image, two measures were extracted (mean and standarddeviation), which lead to 80 descriptors. The LBP and ILBP wereused with their standard configuration and the resulting featurevectors have 256 and 512 dimensions, respectively.

4.3. Evaluations and comparisons

We have evaluated the number of decomposition levels,approaches used to decompose the images into multiple scales,and strategies for combining the feature descriptors extractedfrom different scales. The results are shown in Tables 2 and 3.According to them, the scale decomposition based on Gaussiansmoothing has provided higher classification rates for all data setsand the most significant difference occurs with the OuTex data set(84.61% with Pyramid decomposition and 89.50% with Gaussiansmoothing). This is because the small sample size for this data setassociated with the pyramid decomposition results in the compu-tation of the GLCM for very small images, resulting, therefore,on sparse matrices. The Gaussian smoothing decomposition isconsidered during the comparison to other methods.

Tables 2 and 3 also evaluate five approaches to combine thedescriptors extracted from each scale. According to the results,

Table 4Texture classification results achieved by multiple texture descriptors for differentdata sets. The second column indicates the number of samples used during trainingfor the UMD and UIUC data sets. The results show classification rates onpercentage.

Data set Samples Feature extraction method

LBP ILBP Gabor Granulometry GLCM

Original Multi-scale

UMD 5 72.5 73.3 75.7 75.4 83.1 85.210 81.5 81.3 82.3 83.8 88.7 91.215 85.4 85.0 84.9 88.0 91.1 93.520 88.1 87.4 88.1 90.6 92.7 95.2

UIUC 5 41.7 44.8 51.0 48.3 58.9 65.110 44.6 51.7 59.4 58.9 68.5 74.415 49.2 54.7 65.1 64.5 72.7 78.920 53.4 61.3 69.9 68.6 75.6 81.7

Brodatz – 87.1 89.4 89.4 88.8 78.3 87.2VisTex – 96.5 97.9 92.6 94.4 94.4 96.0OuTex – 89.0 82.0 84.9 89.2 83.0 89.4

Table 5Computational time (in seconds) required to perform the feature extraction.

Data set Feature extraction method

LBP ILBP Gabor Granulometry GLCM

Original Multi-scale

UMD 94.86 105.85 4568.99 9979.91 135.84 633.43UIUC 24.95 27.21 1371.59 2536.75 39.35 183.10Brodatz 0.78 0.88 110.12 68.51 12.30 56.05VisTex 1.95 2.14 95.17 116.60 8.05 32.16OuTex 1.89 3.96 297.94 79.20 41.66 180.49

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345344

three approaches have achieved higher classification rates: localnormalization, normalization based on corresponding descriptors,and simple concatenation without normalization. The combina-tion strategy, based on reducing the weight of the descriptorsextracted from smaller scales, showed a weak performance (mean-ing that the importance of all scales should be equally considered).The combination strategy that achieved the best results for eachdata set is used in the comparisons to other methods.

Finally, comparisons among the proposed multi-scale featuredescriptor with other well-known methods for texture analysis areshown in Table 4. For the UMD and UIUC data sets, we show theresults when a different number of samples are used for training(as indicated in the second column). The remaining data setsfollow the protocols described in Section 4.1.

There are important observations that can be made accordingto the results shown in Table 4. First, significant improvementshave been achieved for all data sets when the multi-scale GLCM isemployed, compared to the original approach. Second, consideringthe other methods in the literature, the proposed approachachieved the best classification rates on three out of the five testeddata sets. Furthermore, it is also important to point out that themost significant improvements were achieved on data sets pre-senting samples with large sizes (UMD and UIUC), in which moreinformation can be captured by a multi-scale approach. Finally,although the Gabor filters also perform multi-scale analysis, theclassification rates are not as high as the ones achieved by theproposed approach.

Table 5 shows the computational time required for each featurein all tested data sets. It can be observed that multi-scale GLCMtakes more time than its single-scale version, which is expected

since that the GLCM has to be computed in every level of thepyramid in the proposed method.

5. Conclusions

This paper describes an extension of the GLCM texture descrip-tor to multiple scales based on a Gaussian smoothing approachand a pyramid decomposition. Features extracted from GLCM atdifferent scales are combined and evaluated for each multi-resolution representation.

The performance of the proposed approach is evaluated byapplying the multi-scale descriptor on five benchmark texturedata sets and the results are compared to other well-knowndescriptors for texture analysis. The proposed multi-scale GLCMachieved significant improvements for all tested data sets.

Acknowledgments

The authors are grateful to FAPESP, FAPEMIG, CNPq, and CAPESfor the financial support.

References

[1] W. Schwartz, H. Pedrini, Textured image segmentation based on spatialdependence using a Markov random field model, in: 2006 IEEE InternationalConference on Image Processing, Atlanta, GA, USA, 2006, pp. 2449–2452.

[2] R.M. Haralick, Statistical and structural approaches to texture, Proc. IEEE 67 (5)(1979) 786–804.

[3] T. Reed, J.M.H. Dubuf, A review of recent texture segmentation and featureextraction techniques, CVGIP: Image Understanding 57 (3) (1993) 359–372.

[4] Y. Xu, S.B. Huang, H. Ji, Integrating local feature and global statistics for textureanalysis, in: 16th IEEE International Conference on Image Processing, 2009,pp. 1377–1380.

[5] H. Tamura, M. Shunji, Y. Yamawaki, Textural features corresponding to visualperception, IEEE Trans. Syst. Man Cybern. 8 (1978) 237–247.

[6] N.E. Lasmar, Y. Stitou, Y. Berthoumieu, Multiscale skewed heavy tailed modelfor texture analysis, in: 16th IEEE International Conference on Image Proces-sing, 2009, pp. 2281–2284.

[7] Z. Tao, H. Wenxue, W. Jinjia, A novel texture analysis method based on graphspectral theory, in: Fifth International Conference on Intelligent InformationHiding and Multimedia Signal Processing, 2009, pp. 467–470.

[8] Y. Pang, X. Li, Y. Yuan, Robust tensor analysis with L1-norm, IEEE Trans. CircuitsSyst. Video Technol. 20 (2) (2010) 172–178.

[9] Y. Pang, X. Li, Y. Yuan, D. Tao, J. Pan, Fast Haar transform based featureextraction for face representation and recognition, IEEE Trans. Inf. ForensicsSecurity 4 (3) (2009) 441–450.

[10] Y. Pang, Y. Yuan, X. Li, Iterative subspace analysis based on feature linedistance, IEEE Trans. Image Process. 18 (4) (2009) 903–907.

[11] Y. Pang, W. Li, Y. Yuan, J. Pan, Fully affine invariant SURF for image matching,Neurocomputing 85 (2012) 6–10.

[12] Y. Pang, H. Yan, Y. Yuan, K. Wang, Robust CoHOG feature extraction in humancentered image/video management system, IEEE Trans. Syst. Man Cybern. PartB: Cybern. 42 (2) (2012) 458–468.

[13] M. Tuceryan, A.K. Jain, Texture analysis, in: C.H. Chen, L.F. Pau, P. Wang (Eds.),The Handbook of Pattern Recognition and Computer Vision, World ScientificPublishing Co., 1998, pp. 207–248.

[14] D.G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J.Comput. Vision 60 (2004) 91–110.

[15] H. Bay, T. Tuytelaars, L. Van Gool, SURF: speeded-up robust features, in:European Conference on Computer Vision, 2006, pp. 346–359.

[16] N. Dalai, B. Triggs, I. Rhone-Alps, F. Montbonnot, Histograms of orientedgradients for human detection, in: IEEE Conference on Computer Vision andPattern Recognition, 2005, pp. 886–893.

[17] K. Mikolajczyk, C. Schmid, A performance evaluation of local descriptors, IEEETrans. Pattern Anal. Mach. Intell. 27 (10) (2005) 1615–1630.

[18] O. Tuzel, F. Porikli, P. Meer, Region covariance: a fast descriptor for detectionand classification, in: Proceedings of the 9th European Conference onComputer Vision, 2006, pp. 589–600.

[19] B. Wu, R. Nevatia, Detection of multiple, partially occluded humans in a singleimage by Bayesian combination of edgelet part detectors, in: IEEE Interna-tional Conference on Computer Vision, 2005, pp. 90–97.

[20] R.M. Haralick, K. Shanmugam, I. Dinstein, Textural features for image classi-fication, IEEE Trans. Syst. Man Cybern. 3 (6) (1973) 610–621.

[21] T. Ojala, M. Pietikäinen, D. Harwood, A comparative study of texture measureswith classification based on featured distributions, Pattern Recognition 29 (1)(1996) 51–59.

F. Roberti de Siqueira et al. / Neurocomputing 120 (2013) 336–345 345

[22] J. Huang, S.R. Kumar, M. Mitra, W.J. Zhu, R. Zabih, Image indexing using colorcorrelograms, in: Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, 1997, pp. 762–768.

[23] G. Pass, R. Zabih, J. Miller, Comparing images using color coherence vectors,in: Proceedings of the ACM International Conference onMultimedia, 1996, pp. 65–73.

[24] M.J. Swain, D.H. Ballard, Color indexing, Int. J. Comput. Vision 7 (1991) 11–32.[25] W.T. Freeman, E.H. Adelson, The design and use of steerable filters, IEEE Trans.

Pattern Anal. Mach. Intell. 13 (1991) 891–906.[26] A.K. Jain, F. Farrokhnia, Unsupervised texture segmentation using Gabor filters,

Pattern Recognition 24 (1991) 1167–1186.[27] S. Belongie, J. Malik, J. Puzicha, Shape matching and object recognition using

shape contexts, IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002) 509–522.[28] J. Zhang, S. Lazebnik, C. Schmid, Local features and kernels for classification of

texture and object categories: a comprehensive study, Int. J. Comput. Vision 73(2) (2007) 213–238.

[29] T. Randen, J.H. Husoy, Filtering for texture classification: a comparative study,IEEE Trans. Pattern Anal. Mach. Intell. 21 (4) (1999) 291–310.

[30] T. Ojala, M. Pietikäinen, T. Mäenpää, Multiresolution gray-scale and rotationinvariant texture classification with local binary patterns, IEEE Trans. PatternAnal. Mach. Intell. 24 (2002) 971–987.

[31] W. Zhang, S. Shan, W. Gao, X. Chen, H. Zhang, Local gabor binary patternhistogram sequence (LGBPHS): a novel non-statistical model for face repre-sentation and recognition, in: 10th IEEE International Conference on Compu-ter Vision, vol. 1, 2005, pp. 786–791.

[32] X. Qian, X.S. Hua, P. Chen, L. Ke, PLBP: an effective local binary patterns texturedescriptor with pyramid representation, Pattern Recognition 44 (2011)2502–2515.

[33] A. Bosch, A. Zisserman, X. Munoz, Representing shape with a spatial pyramidkernel, in: 6th ACM International Conference on Image and Video Retrieval,The Netherlands, Amsterdam, 2007, pp. 401–408.

[34] X. Qian, G. Liu, D. Guo, Z. Li, Z. Wang, H. Wang, Object categorization usinghierarchical wavelet packet texture descriptors, in: 11th IEEE InternationalSymposium on Multimedia, 2009, pp. 44–51.

[35] A. Torralba, K. Murphy, W. Freeman, M. Rubin, Context-based vision system forplace and object recognition, in: Ninth IEEE International Conference onComputer Vision, 2003, pp. 273–280.

[36] M.M. Galloway, Texture analysis using gray level run lengths, Comput.Graphics Image Process. 4 (1975) 172–179.

[37] M. Unser, Texture classification and segmentation using wavelet frames, IEEETrans. Image Process. 4 (11) (1995) 1549–1560.

[38] J.G. Daugman, Uncertainty relation for resolution in space, spatial frequency,and orientation optimized by two-dimensional visual cortical filters, J. Opt.Soc. Am. A 2 (1985) 1160–1169.

[39] D.C. He, L. Wang, Texture unit, texture spectrum, and texture analysis, IEEETrans. Geosci. Remote Sensing 28 (4) (1990) 509–512.

[40] M.H. Horng, Y.N. Sunb, X.Z. Lin, Texture feature coding method for classifica-tion of liver sonography, Comput. Med. Imaging Graphics 26 (1) (2002) 33–42.

[41] E. Kurmyshev, M. Cervantes, A quasi-statistical approach to digital imagerepresentation, Rev. Mex. Fís. 42 (1) (1996) 104–116.

[42] A. Hanbury, U. Kandaswamy, D.A. Adjeroh, Illumination-invariant morpholo-gical texture classification, in: Christian Ronse, et al. (Eds.), MathematicalMorphology: 40 Years On, Springer, The Netherlands, 2005, pp. 377–386.

[43] G.R. Cross, A.K. Jain, Markov random field texture models, IEEE Trans. PatternAnal. Mach. Intell. 5 (1) (1983) 25–39.

[44] J. Mao, A.K. Jain, Texture classification and segmentation using multiresolutionsimultaneous autoregressive models, Pattern Recognition 25 (1992) 173–188.

[45] M. Singh, S. Singh, Spatial texture analysis: a comparative study, in: Proceedingsof the International Conference on Pattern Recognition, 2002, pp. 676–679.

[46] J. Zhang, Brief review of invariant texture analysis methods, Pattern Recogni-tion 35 (3) (2002) 735–747.

[47] E.R. Davies, Introduction to texture analysis, in: J.S. Majid Mirmehdi XianghuaXie (Ed.), Handbook of Texture Analysis, Imperial College Press, 2008,pp. 1–31.

[48] J. Serra, Image Analysis and Mathematical Morphology, Academic PressNew York, NY, USA, 1982.

[49] H.J.A.M. Heijmans, Morphological Image Operators, Academic Press, New York,NY, USA, 1994.

[50] H. Jin, Q. Liu, H. Lu, X. Tong, Face detection using improved LBP under Bayesianframework, in: Proceedings of the Third International Conference on Imageand Graphics, 2004, pp. 306–309.

[51] D. Clausi, M. Jernigan, A fast method to determine co-occurrence texturefeatures, IEEE Trans. Geosci. Remote Sensing 36 (1) (1998) 298–300.

[52] M. Tahir, A. Bouridane, F. Kurugollu, An FPGA based coprocessor for GLCM andHaralick texture features and their application in prostate cancer classification,Analog Integrated Circuits Signal Process. 43 (2) (2005) 205–215.

[53] A. Gelzinis, A. Verikas, M. Bacauskiene, Increasing the discrimination power ofthe co-occurrence matrix-based features, Pattern Recognition 40 (9) (2007)2367–2372.

[54] R. Walker, P. Jackway, D. Longstaff, Genetic algorithm optimization of adaptivemulti-scale GLCM features, Int. J. Pattern Recognition Artif. Intell. 17 (1) (2003)17–39.

[55] M. Benco, R. Hudec, Novel method for color textures features extraction basedon GLCM, Radioengineering 4 (16) (2007) 64–67.

[56] Y. Hu, Unsupervised texture classification by combining multi-scale featuresand K-means classifier, in: Chinese Conference on Pattern Recognition, 2009,pp. 1–5.

[57] F. Pacifici, M. Chini, Urban land-use multi-scale textural analysis, in: IEEEInternational Geoscience and Remote Sensing Symposium, 2008, pp. 342–345.

[58] P. Rakwatin, N. Longepe, O. Isoguchi, M. Shimada, Y. Uryu, Mapping tropicalforest using ALOS PALSAR 50 m resolution data with multiscale GLCManalysis, in: IEEE International Geoscience and Remote Sensing Symposium,2010, pp. 1234–1237.

[59] H. Nguyen-Duc, T. Do-Hong, T. Le-Tien, C. Bui-Thu, A new descriptor for imageretrieval using contourlet co-occurrence, in: 2010 Third International Con-ference on Communications and Electronics (ICCE), 2010, pp. 169–174.

[60] M. Do, M. Vetterli, The contourlet transform: an efficient directional multi-resolution image representation, IEEE Trans. Image Process. 14 (12) (2005)2091–2106.

[61] T. Lindeberg, Scale-Space Theory in Computer Vision, Kluwer AcademicPublishers, New York, NY, USA, 1994.

[62] S. Lazebnik, C. Schmid, J. Ponce, A sparse texture representation using localaffine regions, IEEE Trans. Pattern Anal. Mach. Intell. 8 (2005) 1265–1278.

[63] UMD, Texture dataset, 2012. ⟨http://www.cfar.umd.edu/� fer/High-resolution-data-base/hr_database.htm⟩.

[64] P. Brodatz, Textures: a Photographic Album for Artists and Designers, DoverPublications, Inc., New York, USA, 1966.

[65] T. Ojala, T. Mäenpää, M. Pietikäinen, J. Viertola, J. Kyllönen, S. Huovinen, Outex—new framework for empirical evaluation of texture analysis algorithms, in:Proceedings of the 16th International Conference on Pattern Recognition, vol.1, Washington, DC, USA, 2002, pp. 701–706.

[66] VisTex, Texture dataset, 2012. ⟨http://vismod.media.mit.edu/vismod/imagery/VisionTexture/vistex.html⟩.

[67] OuTex, Texture dataset, 2012. ⟨http://www.outex.oulu.fi/index.php?page=classification#Outex_TC_00005⟩.

[68] V. Arvis, C. Debain, M. Berducat, A. Benassi, Generalization of the cooccurrencematrix for colour images: application to colour texture classification, ImageAnal. Stereol. 23 (2004) 63–72.

Fernando Roberti de Siqueira received his B.Sc. degreein Computer Science from the University of Campinas,Campinas, Brazil. He is currently a M.Sc. student in theInstitute of Computing at the University of Campinas,Brazil. His research interests include image processing,computer vision, data mining and machine learning.

William Robson Schwartz received his Ph.D. degree inComputer Science from University of Maryland, CollegePark, USA. He received his B.Sc. and M.Sc. degrees inComputer Science from the Federal University ofParana, Curitiba, Brazil. He is currently a professor inthe Computer Science Department at UniversidadeFederal de Minas Gerais, Brazil. His research interestsinclude computer vision, pattern recognition and imageprocessing.

Helio Pedrini received his Ph.D. degree in Electricaland Computer Engineering from Rensselaer Polytech-nic Institute, Troy, NY, USA. He received his M.Sc.degree in Electrical Engineering and his B.Sc. in Com-puter Science, both from the University of Campinas,Brazil. He is currently a professor in the Institute ofComputing at the University of Campinas, Brazil. Hisresearch interests include image processing, computervision, pattern recognition, computer graphics, compu-tational geometry.