LBP Project Report [Fadi]

Electrical Engineering Department

ELEG493

STP in ELEG: Image Processing

Final Project

Texture classification using Local Binary Patterns

Prepared by:

Fadi Al-Zir ID: 5261

Instructor: Dr. Hasan Al Marzouqi

Submission date: December 17th, 2015

Abstract

This is a report for the final course project in Image processing, namely, texture classification using local binary patterns (later referred to as LBP). The purpose of this project is to analyze and learn to implement the LBP technique in order to classify a given set of texture samples in MATLAB environment.

This report provides a detailed overview of the work done, as well as results and analysis of the taken approach.

The project implementation covers 11 texture classes taken from the Mondial-Marmi granite dataset, where 3 samples of each class were tested against other textures of the dataset, and the corresponding class of the texture was attempted to be determined. The results showed that LBP approach alone may be quite effective, in a certain fashion, in this task.

ii

Table of Contents

Abstract..................................................................................................................................... ii

Table of Contents.....................................................................................................................iii

1. Introduction..........................................................................................................................4

3. Theoretical Background........................................................................................................5

4. Experimental setup & procedure...........................................................................................7

5. Discussion............................................................................................................................9

6. Conclusion..........................................................................................................................11

6. References..........................................................................................................................12

6. Appendix............................................................................................................................13

iii

1. IntroductionLBP is defined as one of the techniques used for image classification purposes in the field of computer vision. [1]. More specifically, it is a simple and efficient approach that allows to summarize the distinctive features of a 2D image by a histogram of binary numbers, where each number would represent a certain pixel in an image. It is thought to had been first introduced in 1994, as an approach that could be implemented in 2D texture analysis. The advantage, and hence, later, the acceptance of LBP is explained by its simplicity, low computational costs and lastly, it’s relative quickness as compared to other techniques – criterions, vital in computer vision. [2].

Obviously, the approach has developed tremendously since its first introduction: nowadays there are numerous variations and modifications of the basic LBP approach, depending on the specific application and needs. Some of these are Volume, Center-Symmetric, Multi-Block, Direction coded, Centralized and many other.

LBP is usually used solely or as a part of an algorithm in such applications as face recognition and classification, facial expression recognition, biometrics (e.g. fingerprints) and finally, as it is in our case, texture recognition and classification.

4

3. Theoretical BackgroundThe idea of the LBP approach, as described before, is to summarize the structure of an image. This is done by comparing each pixel of the image with pixels in its neighborhood. The detailed approach is described in the next paragraphs.

First, for a basic LBP procedure, the image, that is to be processed, has to be converted to gray-scale. Then, each pixel of that image is taken separately as a center pixel and the testing neighborhood is defined. The center pixel intensity value as then taken as the threshold value and all neighborhood pixels undergo thresholding as follows: each of the surrounding pixels is compared to the center pixel, if the pixel intensity value is greater than that of the center one, it is denoted with 1, otherwise, it is denoted with a 0. More specifically, if the neighborhood size is chosen as 3 by 3 pixels (Figure 1), then each of the surrounding pixel values, after thresholding, is collected into a binary number, 8-bit in this case, which will represent the center pixel, and hence, a distinct feature of the tested image. In this fashion, you end up with a 28 possible binary number combinations, which are, in fact, called the Local Binary Patterns.

Figure 1: 3x3 Basic LBP Procedure

This procedure is applied to every pixel in an image, thus providing us with a set of binary numbers (patterns), each describing a distinct pixel and its 3x3 neighborhood. These patterns need to be somehow collated and summarized to become useful in analysis. In order to do that, a histogram of these patterns is created. In such a histogram, all equal patterns are assigned to their own bin, thus providing a good measure of which patterns are more dominant or occur more frequently in an image, and which do not.

However, the approach is not efficient at this point: taking into account the number of all possible pattern combinations, the computational cost as well as time consumption of such algorithm is too high. In order to tackle that, it is necessary to include only uniform patterns in our analysis. A binary pattern is called uniform when it contains a maximum of two transitions from 0 to 1 and vice versa. Furthermore, it is a known fact that 80-90% of patterns in normal images are uniform, while the impact of non-uniform patterns is negligible, which is why this is applicable to LBP. Finally, these patterns are all collated into a normalized histogram of uniform patterns (Figure 2), where the non-uniform, regardless of their value, are all placed into the last, here, 59th bin.

5

Figure 2: Uniform LBP Histogram

On the other hand, however, deriving the patterns from a square pixel neighborhood matrix around the center pixel is not effective, as it was noted soon after it had been proposed, because the fixed neighborhood fails to encode the details of the image that differ by scale. Thus the LBP operator has to be extended to use a variable neighborhood, which is essentially achieved by a Circular LBP approach. Here, the idea is to align a pre-defined number of neighboring pixels on a circle of a variable radius around the center pixel (Figure 3).

Figure 3: Circular LBP Procedure

Finally, we approach the more specific issue concerning our task – comparing such LBP histograms of texture sample images and classifying the latter based on their distinctive features. In order to accomplish this, we will need to compare LBP histograms to each other by calculating the histogram distance. Various techniques can be used for that purpose: Bin-to-bin comparisons, like Chi-squared distance, and cross-bin comparison like Bin-similarity matrix or Earth Moving Distance (EMD). In this project, we can use the Logarithmic Distance calculation, as follows:

LBPdistance=−[∑ LBPa 1∗log10 LBPa 2+¿∑ LBPb 1∗log10 LBPb 2 ¿] (Eq. 1)

Where a refers to the image that is to be compared, and b refers to the image that “a” is compared to

Furthermore, in the distance calculation, two mapping tables for the Circular LBP patterns were used: a mapping table for 8 samples (neighboring pixels) and radius of 1 – corresponds to a1 and b1, and a table for 24 samples and a radius of 3 – which are a2 and b2 in the equation above. It must be also noted that the equation (Eq.1) is sensitive to the order of input histograms: the LBP histogram that is taken as a base sample, i.e. the histogram that the input is compared to, must always be in the b section of the equation (Eq.1). While the input histogram must always be in the a section.

Thus, a MATLAB function was created (Appendix I). This function accepts any texture sample image as an input, and then calculates and compares the input image LBP histogram to each of the pre-loaded sample histograms of each texture class. Based on the lowest histogram distance, the function outputs a text message, telling the user which class does the input texture belong to. Hence, the input texture is assumed to be of the same class with the reference sample it was tested against, if their histogram distance was the least across all other classes.

6

Total Tests 33Failed Tests 4Successful Tests 29Success Rate (%) 87.88

4. Experimental setup & procedureIn order to verify the integrity and performance of the created algorithm, several experiments were conducted in MATLAB environment using the function described before. These experiments used the Mondial-Marmi granite data set (Appendix II).

In the first experiment, 3 different samples of each of the 11 texture classes (hence 33 test samples in total) were tested against a pre-loaded base sample of every texture class. The results of the test are displayed in the figures below (Figure 4-5), where the green cells represent a successful test (i.e. the texture was correctly recognized and classified), and consequently, red cells represent a failed test, with a name of the texture class that the input texture was assigned to instead of the correct one. As we can see, the developed LBP algorithm was in fact successful in classifying the samples, with an estimated success rate of 88%. In this experiment, the obtained results of uniform LBP approach may also be assumed to be applicable to a rotation-invariant modification.

Texture Type 1 2 3Acqua MarinaAzul CapixabaAzul Platino Napoletano NapoletanoBianco CristalBianco SardoGiallo Napoletano RosaPorrinoGiallo OrnamentaleGiallo SantaCeciliaGiallo VenezianoRosa BetaRosa Porrino RosaBeta

Sample Number

Figure 4: First experiment results (0 deg.)

The second experiment was conducted in order to evaluate the performance of the devised algorithm with the samples used in the first experiment, but rotated gradually by 15, 30, 60 and 90 degrees in either 2D direction. The purpose of this experiment is to examine the rotation-variant properties of the proposed approach.

The results observed were satisfying: up to 15 degrees of rotation, the MATLAB function successfully identifies and classifies most of the texture samples, thus producing exactly the same results that were observed in the first experiment (Figure 4-5). However, increasing the angle past that limit, tremendously decreases the total effectiveness of the algorithm. In particular, rotating test samples by 30 degrees, drops the success rate to 70% (Figure 6-7). Furthermore, rotating the samples by 60 degrees decreases the algorithm success rate to 67%, the absolute minimum in our trials (Figure 8-9). The same result is observed when the sample textures are rotated by 90 degrees

7

Figure 5: First experiment summary



Texture Type 1 2 3Acqua MarinaAzul CapixabaAzul Platino Napoletano Napoletano RosaBetaBianco CristalBianco SardoGiallo Napoletano RosaPorrinoGiallo Ornamentale BiancoCristalGiallo SantaCecilia Veneziano Veneziano OrnamentaleGiallo Veneziano OrnamentaleRosa BetaRosa Porrino RosaBeta

Sample Number

Figure 6: Second Experiment Results (30 degrees)

Texture Type 1 2 3Acqua MarinaAzul CapixabaAzul Platino Napoletano Napoletano RosaBetaBianco CristalBianco SardoGiallo Napoletano RosaPorrinoGiallo OrnamentaleGiallo SantaCecilia Ornamentale RosaPorrino OrnamentaleGiallo Veneziano Ornamentale Ornamentale OrnamentaleRosa BetaRosa Porrino RosaBeta

Sample Number

Figure 8: Second Experiment Results (60 and 90 degrees)

8

Figure 7: 2nd Experiment summary (30 degrees)

Figure 9: 2nd Experiment summary (60 and 90 degrees)

5. DiscussionIt is important to note that the results of the experiments with rotated texture samples mainly show that the proposed algorithm can be best used when texture comparison is done using rotation-invariant LBP histogram option. Hence the observed deterioration of effectiveness, along with random errors in classification, was expected.

First, we need to examine the common errors that have occurred across all variations of the previously described experiments. As we can see from the previously presented figures, the algorithm failed to classify 4 specific samples across all experiments: “Azul Platino” - samples 1 & 2, “Giallo Napoletano” - sample 3, and finally “Rosa Porrino” - sample 1 (Figure 10, placed accordingly).

Figure 10: Failed texture samples

As it may be seen from the experiment results, these textures were always identified with the wrong classes: First two “Azul Platino” samples were identified to be of class “Giallo Napoletano”. The third sample, “Giallo Napoletano” was identified to be of class “Rosa Porrino”. Finally, the fourth sample in question, “Rosa Porrino” was identified as “Rosa Beta”. Reference samples of the texture classes, which the test samples were associated with, are displayed below (Figure 11, placed accordingly).

There seems to be no obvious reason why such errors would occur, especially considering the fact that other similar samples were correctly identified and classified. To a human eye, it is clear that “Azul Platino” (first two samples in Figure 10) and “Giallo Napoletano” (first sample in Figure 11) are different textures, at least by general color and size of impurities. However, it is not that obvious of a difference in computer vision, and this is specific to the proposed approach. First, the images are converted to gray-scale, hence the effect of color of the texture may be marginal in the classification process, depending on the characteristics of the texture itself (amount of grain, impurities etc.). Furthermore, in gray-scale, the texture is mostly (and as intended) judged by the features that are described by each

9

Figure 11: Reference texture samples

pixel and its immediate neighborhood. Hence these two textures may seem to have slightly more similarities in structure, as compared to the similarities between the test sample and its actual corresponding reference sample. For instance, the impurities in “Giallo Napoletano” are bigger in size than those observed in “Azul Platino”, but that may not always be fully reflected in the LBP histogram. Possible solutions to this issue would be to either modify the number of samples and the radius in the Circular LBP approach, or increase the resolution of the images, so that the image structure could be described in more detail (all samples presented were provided to the function in a 544*544 resolution).

On the other hand, however, we can see that the algorithm effectiveness for particular classes remained 100% throughout both experiments. These classes are “Acqua Marina”, ”Azul Capixaba”, ”Bianco Cristal”, ”Bianco Sardo”, and ”Rosa Beta” (see Appendix II). These texture classes were correctly handled by the proposed function regardless of the rotation. If we examine these textures, we can easily see the reason for such results: the first two classes, “Acqua Marina” and ”Azul Capixaba” are considerably different from all other classes by structure: a plain white texture with minor impurities. They are easily distinguished from each other as well: the former contains a considerable amount of minor but dense grain across the whole sample, while the latter is a more blurry plain-white texture with minor light-yellow impurities. On the other hand, the last three texture classes, mentioned above, differ from the others by the high amount of small grain-like impurities across the texture. Furthermore, they are differentiated from each other mostly by the density of these impurities.

Finally we examine the textures that were fully and successfully classified in the first experiment, but resulted in errors after rotation of samples: “Giallo Santa Cecilia” and “Giallo Veneziano”. Here, the reason for the classification errors is clearer: both texture classes contain distinct line-like impurities, which change their orientation with the sample rotation, hence their classification becomes completely wrong after a considerable rotation angle increase. Furthermore, the former has a greater number of more clearly defined line-like impurities, which is the feature that this texture class is differentiated by, as compared to the latter, and which is the reason why it is already falsely classified at 30 degree rotation.

10

6. ConclusionTo summarize, the project objectives of learning and analyzing the LBP technique application in texture classification were met. An algorithm was developed to perform the required task and was adequately tested on a real-life texture data set, showing satisfactory and educating performance.

To note several characteristics of the proposed algorithm:

It is successful in classifying textures that are captured at the same angle as the base samples

Can be safely used to successfully classify textures that were captured at angle variations of ±15º from the reference sample

Comparison of a single test sample takes up to 8-9 seconds to process Function accepts only one test sample at a time Effectiveness of the algorithm depends on the number of pre-loaded reference

samples and variety of provided texture classes – both variables are easily modifiable Accuracy of classification may decrease with sample rotation, depending on the

characteristics of the test texture

The proposed, indeed, is a considerably basic approach, as compared to the present day achievements and standards in the area. However, it is essential to note the next steps that could be taken in developing the proposed algorithm. For instance, as we may easily notice, textures can be easily differentiated, first and foremost, by their main color, which is why it is necessary to enchance the algorithm to be able to accurately handle and compare RGB images. Furthermore, to increase the accuracy and reliability of the algorithm, it may be supplied with more reference samples of each texture class. It may also be necessary to increase the resolution of the processed images. While this will increase the computational time and cost, it will potentially provide means for more accurate comparison and classification due the fact that more details of texture features will be recorded. Finally, it would make sense to increase the factors of the Circular LBP approach, as discussed before.

11

6. References[1] T. Ojala, M. Pietikäinen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions". 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 1, pp. 582 – 585.

[2] M. Pietikäinen, A. Hadid, G. Zhao and T. Ahonen, “Computer Vision Using Local Binary Patterns”, 2011

12

6. AppendixAppendix I: MATLAB algorithm for texture classification

%% Image processing, Fall 2015%% Project: Texture classification using LBP%% Written by: Fadi Al-Zir% Senior Student% Electrical Engineering Department% Petroleum Institute, Abu Dhabi, U.A.E.% [email protected] function cmpr=texture_class(itest) %input a texture image to test warning('off','MATLAB:bitshift:NDiscontinueSupport');testimg=rgb2gray(itest); %convert the input texture image to grayscale%% Texture Class Samples AcquisitionT11=imread('AcquaMarina.bmp');T1=rgb2gray(T11); %acquire each texture sample and convert it to grayscaleT22=imread('AzulPlatino.bmp'); T2=rgb2gray(T22);T33=imread('BiancoCristal.bmp'); T3=rgb2gray(T33);T44=imread('BiancoSardo.bmp'); T4=rgb2gray(T44);T55=imread('GialloNapoletano.bmp'); T5=rgb2gray(T55);T66=imread('GialloOrnamentale.bmp'); T6=rgb2gray(T66);T77=imread('GialloSantaCecilia.bmp'); T7=rgb2gray(T77);T88=imread('GialloVeneziano.bmp'); T8=rgb2gray(T88);T99=imread('RosaBeta.bmp'); T9=rgb2gray(T99);T1010=imread('RosaPorrino.bmp'); T10=rgb2gray(T1010);T1111=imread('AzulCapixaba.bmp'); T11=rgb2gray(T1111); %% Comparison by texture classz=zeros(11,1); %creating a zero vector to accomodate class comparisonsmapping1=getmapping(8,'u2'); %assigning mapping functionsmapping2=getmapping(24,'u2');%Note that the getmapping function might take some time to process z(1,:)=lbp_distance(testimg,T1,mapping1,mapping2); %calculating LBP distance with each class samplez(2,:)=lbp_distance(testimg,T2,mapping1,mapping2);z(3,:)=lbp_distance(testimg,T3,mapping1,mapping2);z(4,:)=lbp_distance(testimg,T4,mapping1,mapping2);z(5,:)=lbp_distance(testimg,T5,mapping1,mapping2);z(6,:)=lbp_distance(testimg,T6,mapping1,mapping2);z(7,:)=lbp_distance(testimg,T7,mapping1,mapping2);z(8,:)=lbp_distance(testimg,T8,mapping1,mapping2);z(9,:)=lbp_distance(testimg,T9,mapping1,mapping2);z(10,:)=lbp_distance(testimg,T10,mapping1,mapping2);z(11,:)=lbp_distance(testimg,T11,mapping1,mapping2); [X,Y] = min(z); %record the image class that has the minimal LBP distance as compared to tested texture %% Display Test Resultif Y==1 disp('The input texture is of class Acqua Marina') elseif Y==2 disp('The input texture is of class Azul Platino') elseif Y==3

13

disp('The input texture is of class Bianco Cristal') elseif Y==4 disp('The input texture is of class Bianco Sardo') elseif Y==5 disp('The input texture is of class Giallo Napoletano') elseif Y==6 disp('The input texture is of class Giallo Ornamentale') elseif Y==7 disp('The input texture is of class Giallo Santa Cecilia') elseif Y==8 disp('The input texture is of class Giallo Veneziano') elseif Y==9 disp('The input texture is of class Rosa Beta') elseif Y==10 disp('The input texture is of class Rosa Porrino') elseif Y==11 disp('The input texture is of class Azul Capixaba') end end

14

Appendix II: Mondial-Marmi Granite Data set

This texture data set may be viewed and downloaded at the following web-page: http://dismac.dii.unipg.it/mm/ver_1_1/index.html

15

http://dismac.dii.unipg.it/mm/ver_1_1/index.html

Documents

LBP Project Report [Fadi]