Statistical Modeling of Texture Wavelet Coefficients

Daniela-Ecaterina Cristea

Department of Communications University "Politehnica" of Timisoara, Faculty of ETc

Timisoara, Romania E-mail: daniela.cristea@etc.upt.ro

Abstract-This paper focuses on texture analysis using the

wavelet transform. The statistical modeling of the wavelet

coefficients with the Generalized Gaussian Distribution (GGD)

raises the problem of estimating its parameters. Our aim is to

examine the estimated values which enable the fitting of the

wavelet coefficients histograms to a probability density function

(PDF) of the GGD. The experimental results show that in most

situations the fitted PDF does not approach the special cases of

this family of distributions. However, results obtained in some

cases lead us to think to the Laplacian distribution as a suitable

one.

Keywords- Generalized Gaussian Distribution; texture analysis;

wavelet coefficient histogram; Maximum Likelihood Estimation

I. INTRODUCTION

Texture analysis is very important when we want to classify images in which repeating fundamental elements (patterns) can be found. A useful tool for this purpose is the wavelet transform applied to the image, which produces a set of coefficients organized into subbands. The dependencies between wavelet coefficients have been studied in image compression for many years. Simon celli shows in [1] that the wavelet coefficients are statistically dependent. Two properties of the wavelet transform indicate that the value of one coefficient is likely to influence the values of its neighbors in a direct proportional way and the values of the coefficients tend to spread over the scales.

We want to see how we can characterize textures through the statistics of the wavelet detail coefficients. Based on the work of Do and Vetterli [2], we will use the GGD for modeling the coefficients and the Maximum Likelihood (ML) technique for estimating the GGD parameters, a and �.

Our goal is to see how the wavelet coefficients distribution looks by representing a histogram of the coefficients of each subband and see under what conditions it fits a GGD. We also want to check if we can extend the GGD fit to a whole set of coefficients representing one random texture or even all classes of textures. Existing research suggest that a Laplacian PDF (which is a particular case of GGD) fit could be convenient even if it is not the most accurate [3].

The organization of this paper is as follows. Section II describes the theoretical tools use to achieve our purpose. In section III we present the experimental results and discussions on singular images and on a large texture image database. Section IV concludes the paper.

978·1·4673·1176·2/12/$31.00 ©2012 IEEE

II. MODELLING THE WAVELET COEFFICIENTS

Every image will be transformed into wavelet coefficients using the 2D discrete wavelet transform (DWT) n-Ievel decomposition. The number of subbands is 3n+ 1. In the figure Fig.l we present a 3-level decomposition example applied to one image.

"A model PDF can characterize the statistical behavior of a signal. For multimedia signals, the generalized Gaussian distribution is often used" [4].

We want to model the wavelet coefficients by using the independent and identically distributed generalized Gaussian distribution (GGD) [2], given by:

p«x;a,�) =

�exp(-(lx l/a)P) (1) .

2ar(l/�)

where ro is the Gamma function [2]. This model is used in image restoration, de noising, texture classification. In equation (1), a models the width of the PDF and � models the decreasing speed of the distribution.

n=3�

n=2 -7

n=1-7

£ 4H 9V8D

6V 5D

3V

IH

2D

Figure 1. The 2D DWT 3·level decomposition, with horizontal·H, vertical· V

and diagonal-D detail subbands (top); Original image to be decomposed (bottom left); The 3-level image decomposition (bottom right).

Sometimes a is called the scale parameter and � the shape parameter, which is a measure of the sharpness of the PDF peak. The experiments from [5-7] showed that if these two parameters of the GGD are varied we achieve a good PDF approximation for the marginal density of coefficients for a certain subband for several types of wavelet transforms.

The GGD presents two special cases: the Gaussian and Laplacian PDFs. For �=1 we obtain the Laplacian distribution:

I P (x;a,l)=--exp(-lxl/a); r(1)=l (2) x 2ar(l)

For �=2 and a= .ficr we obtain the Gaussian distribution: 2 2 2 P (x;a,2) = exp( -x I a ) (3)

x 2ar(l/2) But r 1/2 = J;, so:

1 ( 2 I 2) a2 = 2cr2 (4) p)x;a,2) = c exp -x a ; vna

The GGD model presents a few limitations because it only approximates first order statistics of wavelet coefficients, while the higher-order statistics are not taken into account [8].

These first-order statistics are derived from the image histogram and we can use them as texture features [7]. In [5],

Mallat presents a family of exponential functions that models the detail histograms of natural textured images.

So far we have decomposed the image into subbands by using a wavelet transform, thus obtaining the wavelet coefficients. We have modeled their distribution by the GGD in (1). We now need to find the two unknown parameters a and �. To accomplish these we will use the Maximum Likelihood (ML) estimators. In [9], we can see that this estimator is superior (compared to others estimators) for heavy-tailed distribution (the case of the distribution coefficients from a subband).

For the lID data xl'x2, ... , xL the log-likelihood function is given by:

L L(x; a, 13) = log p(x;; a, 13) = I logp(x;; a, 13) (5)

[=]

l Image/images r. Wavelet coefficients for one/all subbands

// 1 Wavelet decomposition l ML estimation J n levels a, 13 (GGD)

� Data generation with GGD having

r+ a and � as parameters

For the GGD:

Inp(x;a,�)=ln � exp(-Ixil/a)� (6) f 2ar(1/13)

The maximization of (6) with respect to a and � yields the ML estimates for a and �, which have the expressions [2]:

1 a= - 2:lxil

A (�L I3Jl3 Li=l

( J �::Ixlloglxil (ft L • J I I H lIn I-' "I 113 0 1+-'I' - - - +- - L. x = � � �::Ixl � L i=l I

i=l

(7)

(8)

In [10], Kay proposes to determine � by using the Newton­Raphson iterative procedure with the initial guess from the moment method described in [6].

We want to see how the fits are when we try to superpose the histogram of wavelet sub bands coefficients over a plot of the fitted GGD using the ML estimator. We will present different cases and some conclusions. First of all, to better understand the results, we present a block diagram (Fig.2), representing the comparison between the histogram of the wavelet coefficients subbands and the PDF of the GGD computed with the ML estimates of the parameters a and � (which are estimated from the wavelet coefftcients subbands).

III. EXPERIMENTAL RESULTS

The textures that we used were obtained from the MIT Vision Texture (VisTex) database [1 1]. It contains 41 textures classes, real world 128 x 128 images from different natural scenes. Each class contains 16 subimages (textures), so we have a test database of 656 texture images. The compactly supported orthonormal wavelets introduced by Ingrid Daubechies [12] made DWT practicable. We chose the db1 wavelet (the same as Haar wavelet, the first and the simplest: a step function) and the db4 wavelet, with three levels of decomposition.

Histogram of wavelet coefficients for one/all subbands

l Comparison J

PDF based on generated data and on estimated parameters

Figure 2. Block diagram of the experimental results

All images used were normalized to have zero mean and unitary standard deviation. In the fIrst case, we select image Fabric15. We estimated the two GGD parameters from one subband. We chose the 7th wavelet subband and after that we made a comparison between the wavelet coeffIcients histogram of the 7th subband and the PDF of GGD computed with the ML estimates of a and p. What we obtained is showed in Fig.3. The shape parameter P is equal to one for "db4" and smaller than one for Haar, so a Laplacian density can be fItted to this empirical histogram in the fIrst case (Fig.3a).

In the next step, we kept the same image, but this time, we estimated the scale and shape parameters for each of the 10 subbands (1 subband for approximation and 9 subbands for detail). We left out the histogram plots for brevity and we present the estimation results in Table 1

1.4

1. 2

0.8

0.6

0.4

0. 2

o L-_A!l!l!I!I!!II[iIlIIW ·2 . 5 ·2 · 1 . 5 ·1 ·0 5 0 0. 5 1. 5 2 2 . 5

1.6 1-----,-------,--T----r-;:;:::::::::�==::::::;_J

1.4

1. 2

0.8

0.6

0.4

0. 2

0 ·3 - 2 -1 2 3

Figure 3. Wavelet 7�' subband coefficient histogram fitted with a GGO. Example for the Fabric 15 subimage of size 128xl28 using a) db4(top) and Haar (bottom). The estimated parameters are: a) a=0.3152 and �=1.0019; b) a=0.2769 and �=0.8298

TABLE!.

Subband

Approx.

lH

20

3V

4H

50

6V

7H

80

9V

SCALE AND SHAPE PARAMETERS ML ESTIMATES USING DB4 AND HAAR WAVELETS

db4 Haar

a � a �

7.7596 2.8740 7.5331 3.4803

1.5052 1.2805 0.3781 0.5939

1.3796 0.8939 0.8836 0.7624

1.7039 1.0157 0.3989 0.6014

0.6362 0.9950 0.2827 0.6477

0.2757 0.6054 0.1951 0.5551

0.6399 0.8431 0.5185 0.8035

0.3152 1.0019 0.2769 0.8298

0.1570 0.6878 0.1524 0.6370

0.3250 1.0557 0.2963 0.9511

We can observe that for the approximation coeffIcients, the density is closer to the Gaussian, for db4. For one of the other 9 subbands, the shape parameter P is close to one (9V subband for the "Haar" wavelet), or even one (3V, 4H, 7H, 9V for the "db4" wavelet), so a Laplacian density can be fItted to the corresponding histogram. We repeated these measurements for one image from each texture class and we obtained similar results.

In FigAa, b we kept the same image and we showed the comparison between the wavelet coeffIcients histogram and the PDF of the GGD computed with the ML estimates of a

and p. In this case we have used all of the 10 wavelet subbands together, one subband for approximation and nine subbands for detail. The shape parameter P is close to 0.5 in both cases (P=0.5724 for db4 - FigAa and p=0.5662 for Haar - FigAb), so we cannot conclude that we have a Laplacian density which fIts to the histogram.

Next, we tried to see what the impact of the approximation coeffIcients is. For this we have eliminated these coeffIcients, keeping only the detail coeffIcients. The results are shown in FigAc, d. Compared to the previous test, we can observe a small enhancement for the value of the shape parameter, in the way that it is somewhat closer to the Laplacian (P=0.6431 for db4 - FigAc and p=0.6269 for Haar - FigAd).

In the following case we used all the 656 images from the VisTex database. We compared the histograms of all the coeffIcients from all the 10 subbands with the PDF of the GGD computed with the ML estimates of a and P (Fig. 4e, f).

Then, we eliminated the approximation coeffIcients and used the remaining detail coeffIcients to build the histogram and to estimate the parameters.

The results are presented in Fig.6g, h. The shape parameter is close to 0.5 (P=OA462 for db4 - FigAe and P=OA695 for

= ...... l��� = ...... c::::Jltstop1l

" -frtHGGD " -fiIHGGD " -fiIHGGD -frtl!'dGGD

12

DB ,. DB

DB

OS OS no

" " " " " 02 02

� IL 02 02

'. '. '. '. a b c d

-- � I :::01 -=-:0 -r",wooo -r",w OGO

e f g h

Figure 4. Wavelet coefficient histogram and fitted GGDs for different cases

Haar - Fig.4t) with a slight increase in the case when we eliminate the approximation coefficients (P=0.4951 for db4 -

Fig.4g and p=O.5239 for Haar - Fig.4h). Again the Laplacian

density does not fit to the histogram.

IV. CONCLUSIONS

Texture analysis implies a large amount of data to be processed; therefore the use of transforms can simplify our work. One possibility is the wavelet transform because it

assures the distribution of the useful information in a relatively

small number of coefficients.

We wanted to see how to model the wavelet coefficients

statistics. Related research in this field suggested that the GGD model is appropriate. This is the reason for which we determined the GGD parameters by using the ML estimation

method.

We explored the statistical framework for texture analysis,

where wavelet coefficients in each subband were

independently modeled by a GGD. We have demonstrated that the GGD PDF using the estimated parameters can be fitted to the wavelet coefficient histogram from each subband.

We found that the estimations are different for each

subband, obtaining different values for the shape parameter. In

most cases the values were smaller than one, except for the

highest horizontal frequency scale (subband 7), when the GGD model contains the Laplacian PDF as special case using

P=l. Furthermore, we have observe that when using the db4

wavelet, the shape parameter for some subbands leads us to a

Laplacian PDF as opposed to the haar wavelet when in all

cases P is smaller than one. Therefore, we cannot say that the

Laplacian PDF it is an adoptable model for all the cases, for

all subband coefficients (or for all the subbands) or for all type of texture.

REFERENCES

[I] E. P. Simoncelli, "Bayesian denoising of visual images in the wavelet domain," in Bayesian inference in Wavelet Based Models, P. MUller and B. Vidakovic, Eds. New York: Springer-Verlag, 1999.

[2] Minh N. Do and M. Vetterli, "Wavelet-Based Texture Retrieval Using Generalized Gaussian Density and Kullback-Leibler Distance", iEEE Transactions on image processing, 11(2): 146-158, February 2002.

[3] L.Sendur and I.Selesnick, "Bivariate shrinkage functions for wavelet­based denoising exploiting interscale dependency", iEEE Transactions on Signal Processing, 50(11), November 2002

[4] J.-R.Ohm, "Multimedia Communications Technology", Ed. Springer, page: 53, 2004

[5] S. Mallat, "A theory for multiresolution signal decomposition: the wavelet representation", IEEE Trans. Patt. Recog. and Mach. Intell., 11(7): 674- 693, July 1989.

[6] K. Sharifi and A. Leon-Garcia, "Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video", iEEE Trans. Circuits Syst. Video Techno/., vol. 5, pp. 52-56, 1995.

[7] G. Van de Wouwer, P. Scheunders, D. Van Dyck, "Statistical texture characterization from discrete wavelet representations", iEEE Transactions on image Processing , 8(4): 592-598, 1999.

[8] Juan Liu, "A hierarchical statistical model for image estimation", ECE497PM Project Report, December 16, 1998

[9] M. K. Varanasi and B. Aazhang, "Parametric generalized Gaussian density estimation," J. Acoust. Soc. Amer., vol. 86, pp. 1404-1415, 1989.

[10] S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993

[II] http://vismod.media.miLeduJpublYisTexlVisTex.tar.gz

[12] Daubechies, 1.," Orthonormal bases of compactly supported wavelets", Comm. Pure Appl. Math., 41: 909-996, 1988.