A Probabilistic Image Model for Smoothing and Compression

C.H. Li P.C. YuenDepartment of Computer Science

Hong Kong Baptist UniversityHong Kong

chli@comp.hkbu.edu.hk pcyuen@comp.hkbu.edu.hk

P.K.S. TamDepartment of Electronic and Information Engineering

The Hong Kong Polytechnic UniversityHong Kong

enptam@polyu.edu.hk

Abstract

In this paper, the problem of edge preserving smoothingin image processing is tackled by combining a noise cor-ruption model and a region and edge image model. Thederivation of the probability model for the first order dif-ference in the gray levels of the region pixels and edge pix-els lead to a non-linear filter with coefficients as functionsof the estimated noise variance and edge intensity. Such amodel-based approach allows the design of improved filtersfor noise filtering and image compression. Experimental re-sults demonstrate the improved performance of the filter forboth synthetic and natural images.

Keywords: image processing, model-based filter, imagecompression, edge-preserving smoothing

1 Introduction

Smoothing is widely used in the early stage of a visionsystem. Its purposes are of twofold. The observed datamay be corrupted by noise from various sources and haveto be smoothed before further processing. The data maycontain features which are relevant to the problem at handand need to be smoothed to reduce the complexity for sub-sequent processing.

Traditionally, smoothing is often accomplished by con-volving the data with a Gaussian kernel. However, suchsmoothing is often not ideal for processing of visual datacontaining a lot of discontinuities. Those discontinuitiesare often not preserved in location and in strength in suchsmoothing. Nonlinear image processing techniques have

been introduced into digital image processing. In partic-ular, the use of order statistics based techniques and ro-bust statistics have widely been applied in image enhance-ment and image filtering. Furthermore, adaptive smooth-ing algorithms have been developed involving the designof special operators which adapt to the local characteris-tics of the particular type of images. For example, Kuan etal.[1] proposed the locally linear minimum mean square er-ror (LLMMSE) estimator to smooth out noise in flat regionswhile retaining discontinuities. Ragarajan [2] proposed amodel-based approach for image filtering. Recently, ad-vanced techniques based on modeling of differential equa-tions and probabilistic considerations have been developedfor solving the problem[3],[4] which provide improved re-sults in the area of edge-preserving smoothing. A numberof order statistics based filter have been proposed for imagefiltering with good results. The�-trimmed mean filter hasbeen proposed by Bednar and Watt [5] as a compromise be-tween the median filter and the moving average filter. Thestandard M-filter (STM) which is based on the Huber esti-mator has also been shown to be effective in image filter-ing [6]. However, a major disadvantage of the STM filter isthe high computational requirement in solving the implicitequation.

In this paper, an examination of a noise corruption pro-cess and a region and edge image model will be first dis-cussed. The probability distribution for the first order differ-ence in the pixel levels of the image is then calculated. Sucha probability characterization is then incorporated into thenon-linear diffusion framework as the probabilistic nonlin-ear diffusion filter (PND). While most order-statistics filtersrely on the rejection of outliers in filtering, the PND filter

1

operates by estimating the probability of a pixel being in aregion or at an edge. Such a model-based approach to im-age filtering allows useful statistical information about theimage to be incorporated into the design of image filters.Experimental results with synthetic and natural images willbe presented to show the effectiveness of the PND filter.

2 Probability Model for Region and EdgeRepresentation Model

In this paper, the usual two-dimensional image model ona rectangular grid is assumed with an additive white Gaus-sian noise corruption. The ideal image is a rectangular arrayof real functionx on the two dimensional plane. For sim-plicity in notation, the pixel in the image can be referencedby an integer indexi wherei 2 [1;M ] andM is the totalnumber of pixels in the image. The noise corrupted imageis denoted asy and is related tox by

y = x+ ��; (1)

where� is the standard deviation (s.d.) of the white Gaus-sian noise and� denotes an array of random values in thesame dimension as the image with a Gaussian probabil-ity density function with unit standard deviation and zeromean.

In building the edge and region model for the ideal im-age, we have to assume a model that allows the specificationof the edge and region pixels. In this paper, a piece-wiseconstant model for the ideal image [7] is assumed. Theimage is composed of connected regions where the pixelshave identical gray values. A straight forward approach isto model the joint probabilities of a number of neighboringpixels in a local clique. However, such a joint probabilitydistribution is difficult to be estimated accurately in an im-age with moderate or high complexity. Moreover the com-putational cost in handling joint probability distributions issignificant. Instead of attempting to directly model the jointprobability distribution of the gray values of an image, it isproposed to model the distribution neighboring of the graylevel differences of neighbouring pairs.

Consider a neighboring pair of pixels iny indexed byiandj respectively, the difference in gray levels is modeledas a random variable denoted bydi;j where

di;j = yi � yj : (2)

As the random variablesyi andyj are assumed as comingfrom the same region and have the same mean, the den-sity function of their differences is thus zero mean with twotimes the variance of each individual random variable. Thusthe density function ofdi;j when(i; j) 2 R is given by

f(di;j = g j(i; j) 2 R) =1

2p��

exp(� g2

4�2); (3)

whereR is the set containing all neighboring pairs(i; j)with pixelsi andj having the same mean gray levels.

When the pair(i; j) resides on the boundary betweentwo regions, the random variablesyi andyj have differentmeans which relates to the edge gray level difference be-tween the two regions. Denoting this gray level differenceash, the density functiondi;j when(i; j) 2 E is given by

f(di;j = g j(i; j) 2 E; h) =1

2p��

�exp(� (g � h)2

4�2)

�

(4)whereE is the set containing all neighboring pairs of(i; j)with pixels i andj having different levels which equal toh. The choice of the value ofh in this model should corre-sponds to the characteristics of the image to be filtered. Asmultiple edges of different strength often coexist in a sin-gle image, the value ofh can be chosen as the average edgestrength or the predominant edge strength in the image.

The density function of alldi;j in the entire image isgiven by

f(di;j = g) = prf(di;j = g j(i; j) 2 R) +pe2f(di;j = g j(i; j) 2 E; h) +

pe2f(di;j = g j(i; j) 2 E;�h); (5)

wherepr is the proportion of the pairs of pixels comingfrom a region andpe is the proportion of the pixel pairswhich lie on the boundary between two regions. As aneighboring pair either comes from a region or an edge,pe+pr = 1. The values ofpe andpr can be estimated fromobserving the percentage of edges in the image. This valuecan be calculated or estimated simply by counting the num-ber of edge pixels in an image wherepe is given by dividingthe number of edge pixels by the total number of pixels. Anassumed value ofpe of 5% agrees well with intuition andgives good results for image with moderate complexity. InEqn.5, the two edge probability terms arise from the sym-metry of edge pair, where a rising edge pair(i; j) wouldgive rise to a falling edge for the pair(j; i) with the sameedge strength and the sign of edge strength reversed.

The probability of a pixel pair being a region is given by

P (R jdi;j = g) =prf(di;j = g j(i; j) 2 R)

f(di;j = g); (6)

whereP (R jdi;j = g) is the conditional probability of thepixel pair(i; j) being a region pair when the gray level dif-ference between the pair isg.

Substituting Eqn.3 and Eqn.4 into Eqn.6 gives

P (R jdi;j = g) =

�pr exp(� x2

4�2)�=

2

�pr exp(� g2

4�2) +

pe2

�exp(� (g � h)2

4�2) + exp(� (g + h)2

4�2)

��:

(7)

Similarly, the conditional probability of a pixel pair(i; j)being an edge is given by

P (E jdi;j = g) =�pe2f(di;j = gj(i; j) 2 E; h)

+pe2f(di;j = g j(i; j) 2 E;�h)

�f(di;j = g): (8)

Substituting Eqn.3 and Eqn.4 into Eqn.8 gives

P (E jdi;j = g) =�pe2

�exp(� (g � h)2

4�2) + exp(� (g + h)2

4�2)

��.hpr exp(� g2

4�2) +

pe2

�exp(� (g � h)2

4�2) + exp(� (g + h)2

4�2)

�i: (9)

Figure 1 shows the probabilityP (E jdi;j = g) andP (R jdi;j = g) whenpr = 0:95; pe = 0:05; h = 20 and� = 20:0. It can be seen that the probability of a pixel pairbeing from a region is a maximum when the difference be-tween the pixel pair is zero. This probability decreases asthe difference between the pixel pair increases.

3 Probabilistic Nonlinear Diffusion

In the nonlinear diffusion approach to image smooth-ing [3],[8], a simple numerical scheme is

yt+1i = yti + �

Xj:(i;j)2N

c(di;j)di;j (10)

where� is a small positive constant that satisfies0 < � <1, t represents the time or the number of iterations,c(:) isthe nonlinear diffusion function andN is the set of pairsof pixels which are in a four-nearest neighborhood. In theanisotropic diffusion approach, the functionc(:) is chosenfrom experience and it takes the form of either

c(di;j) =1

1 + (jdi;j jK

)2(11)

or the form of

c(di;j) = exp

��( jdi;j j

K)2�: (12)

−500 −400 −300 −200 −100 0 100 200 300 400 5000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

g

P(R

|di,j

=g)

(a)

−500 −400 −300 −200 −100 0 100 200 300 400 5000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

g

P(E

|di,j

=g)

(b)

Figure 1. (a) Probability of pixels in a Region(b) Probability of pixels at an Edge.

3

In the probabilistic approach, we set the nonlinear diffu-sion rate to be proportional to the probability of the pixelpair being from the same region. Thus if a pixel pair hasa high probability of coming from the same region, a highdiffusion rate is set to allow for inter-region smoothing. Onthe contrary, if the pixel pair has a low probability of beingin the same region indicating that it is likely to be an edge,a low diffusion rate would be set for the pixel pair. Thus thefollowing diffusion rate is adopted,

c(di;j) = P (R jdi;j = g): (13)

The diffusion rate can be computed and stored in a one-dimension table to minimize the computation requirementof calculating Eqn. 7. The look-up table for speeding upthe diffusion process will be further discussed in the resultsSection. The diffusion rate� also affects the speed of ex-ecuting the algorithm by affecting the number of iterationsneeded for the filtering the noise. There is a trade-off be-tween the performance and the number of iterations needed.A slower rate of diffusion may lead to a better quality at theexpense of a larger number of iterations. In this paper, weaim at the development of a fast and efficient image filterand thus we look for a large value of� that enables a goodfiltering result without significant performance degradation.We choose the criteria for setting the value of� in relationwith the total change in gray levels permitted in each itera-tion. The value of� is set to restrict the maximum changearising from a single pair interaction to be less than 10 graylevels, that is

� =10

maxd(jd c(d)j) (14)

where the maximization is taken over the set of values ofd 2 [�256; 256]. Experimental results in the next Sectionindicate that this scheme of selection of� only requires lessthan 10 iterations even for image with severely corruptedwhite Gaussian noise.

4 Results and Discussions

A synthetic image has been generated to test the perfor-mance of the algorithm. The test image is generated usinga piece-wise constant model. Both rectangles and circles ofdifferent gray values and different sizes are included in theimage. Circles are included since their edges contain seg-ments of different directions and blocking artifacts in anyspecific orientation or size can be easily detected by look-ing at their circumferences. The reference image is thencorrupted with additive white Gaussian noise of s.d. of 20.0and 40.0. The performance of the probabilistic nonlineardiffusion is evaluated against the mean filter,�-trimmedmean filter and the standard M filter [9].

The PND filter is computed via a table-lookup methodby storing the entries of�dc(d) where� is calculated from

Table 1. Performance summary on a syntheticimage with Gaussian noise of s.d. 20.0

RMSE n ERMSE n MAE nmean 12.74 2 42.02 1 6.73 2�-T.M. 12.21 1 38.32 1 6.71 3STM 7.41 4 21.23 1 4.45 5PNF 6.30 4 13.42 2 3.93 9

Eqn.14 and ford 2 [�256; 256]. Using this table-lookupapproach, only 4 subtractions, 4 lookup operations and 4additions are required for each pixel in each iteration. Thus,the proposed diffusion algorithm is fast and easy to imple-ment.

The evaluation criteria include measuring the root meansquared errors (RMSE), edge root mean square error(ERMSE), mean absolute errors (MAE)and visual quality.The ERMSE measures the root mean square error of onlythe edge pixels in the image and can thus be used to judgethe performance of the algorithm in preserving and elimi-nating noise at edges. The MAE is widely employed in thedesign of non-linear filters in image processing[10].

All filters assume a window size of 3x3. The datareject coefficient� is set as 1/9 and 2/9 in turn. TheSTM filter has a scale parameterb and is set at the values5; 10; 20; 40. For the PND filter, the parameters for test are(h; �). The values of the parameters are chosen from the set[(60; 20); (60; 30); (60; 40); (100; 20); (100; 30); (100; 40)].For each set of parameters selected for the algorithms, fortyiterations of each algorithm is applied and the minimumRMSE achieved is selected for representing the perfor-mance of the algorithm. The number of iterations taken toreach the minimum RMSE, the minimum ERMSE and theminimum MAE are also tabulated.

Table 1 shows the results of smoothing the synthetic im-age corrupted with Gaussian noise of s.d. 20.0. The meanfilter and the�-trimmed mean filter achieve a low MSEwhile failing to give a low ERMSE. The�-trimmed meanfilter gives slightly lower errors than the mean filter in allthree measures. The STM filter performs much better thanboth the mean filter and the�-trimmed mean filter.

From Table 2, we find that similar results are obtainedwhen the s.d. of the Gaussian noise is increased to 40.0.There are proportional increases in the number of iterationsneeded to attain minimum RMSE. The PND filter attainsa much lower ERMSE than other algorithms indicating thebetter edge retaining ability of the algorithm. Thus in thecase of the synthetic image, the PND filter performs betterthan the classical algorithms.

Table 3 shows the results of filtering the Lenna imagecorrupted with Gaussian noise of s.d. 20.0. As the Lenna

4

Table 2. Performance summary on a syntheticimage with Gaussian noise of s.d. 40.0

RMSE n ERMSE n MAE nmean 15.11 2 43.46 1 9.28 5�-T.M. 15.20 2 42.02 1 10.35 3STM 11.49 7 31.68 2 7.53 8PNF 10.16 9 23.04 4 6.46 13

Table 3. Performance summary on the Lennaimage with Gaussian noise of s.d. 20.0

RMSE n ERMSE n MAE nmean 9.45 1 13.68 1 6.67 2�-T.M. 9.40 2 12.96 1 6.77 2STM 8.92 2 13.10 1 6.45 2PNF 8.73 2 12.06 2 6.43 3

image contains both textures and smoothly varying regions,it demonstrates the performances of the algorithms whenthe piece-wise constant assumption is violated. In this case,the minimum errors attained by the four algorithms aremuch closer to each other. The PND filter achieves lowererrors than the other algorithms.

The results when the Lenna image is corrupted withGaussian noise of s.d. 40.0 is shown in Table 4. Whenthe s.d. of the Gaussian noise is raised to 40.0, the fourfilters attain errors close to each other. Thus in the casewhere the piece-wise constant assumption is not valid, theperformance of the PND filter is comparable with classicalalgorithms.

Figure 2 shows the results of the STM filter and the PNDfilter on the Lenna image. The filtered images are visuallysimilar to each other. This is not unexpected, as the STMand PND filter have similar errors as shown in Table 3. TheSTM filter gives more homogeneous regions while the PNDfilter preserves more details.

To investigate the effect of the filter on compression andstorage of images, the jpeg compression algorithm is ap-

Table 4. Performance summary on the Lennaimage with Gaussian noise of s.d. 40.0

RMSE n ERMSE n MAE nmean 12.49 3 16.97 2 9.13 4�-T.M. 12.58 3 17.02 2 9.20 4STM 12.58 3 16.97 2 9.38 4PNF 12.29 6 16.71 5 9.00 8

(a) (b)

(c) (d)

Figure 2. (a) Lenna image, (b) the image con-taminated with Gaussian noise of s.d. 20.0.,(c) filtered image using STM, and (d) filteredimage using PND.

Table 5. Performance summary on JPEG com-pression of the synthetic image and theLenna image with Gaussian noise of s.d.20.0 (File size in bytes)

Quality Factor 100 90 80Synthetic Image 65808 30776 22067

Filtered Syn. Image 36853 15709 11231Lenna Image 66575 31492 22671

Filtered Lenna Image 42362 17432 12343

plied to the noisy images and the filtered images. The filesizes of the compressed jpeg files for both the filtered im-ages and the noisy images at various quality factor are tab-ulated in Table 5.

The filtered image shows significant reductions in thecompressed file sizes at various quality factors for jpeg com-pression. The reductions are particularly significant in thesynthetic image where the filter model holds more vigor-ously.

The purpose of the experimental simulations is not todemonstrate exhaustively which algorithms gives the small-est RMSE or MAE, since the parameters in each of the algo-rithms may need a lot of fine tuning for minimizing individ-ual signal-to-noise ratio and to the particular type of imagesinvolved. However the results clearly demonstrate the ef-fectiveness of the PND filter in comparison with differenttypes of filters with varying complexity.

5

5 Conclusion

In this paper, an efficient nonlinear filter has been de-rived from a probabilistic framework. The modeling of thenoise corruption process and the region and edge pixelspairs allows the computation of the relative probability ofa pixel pair being an edge or coming from a homogeneousregion. Such probability characterization is then incorpo-rated into a non-linear diffusion framework as the PND fil-ter. Experimental results with synthetic and natural imagehave demonstrated the effectiveness of the PND algorithmin smoothing and compression.

References

[1] D. T. Kuan, A. A. Sawchuk, T. C. Strand, and PierreChavel. Adaptive noise smoothing filter for imageswith signal-dependent noise.IEEE Trans. PatternAnal. Machine Intell., 7:165–177, 1985.

[2] A. Rangarajan, R. Chellappa, and Y. T. Zhou. Amodel-based approach for filtering and edge detectionin noisy images.IEEE Trans. on Circuits and Systems,37:140–144, 1990.

[3] P. Perona and J. Malik. Scale space and edge detectionusing anisotropic diffusion.IEEE Trans. Pattern Anal.Machine Intell., 12:629–639, 1990.

[4] C. H. Li and C. K. Lee. Image smoothing using para-metric relaxation.Graphical Models and Image Pro-cessing, 57:161–174, 1995.

[5] J. B. Bednar and T. L. Watt. Alpha-trimmed meansand their relationship to the median filters.IEEETrans. Acoust., Speech, Signal Process., 32:145–153,1987.

[6] S. R. Peterson, Y. H. Lee, and S. A. Kassam. Somestatistical properties of alpha-trimmed mean and stan-dard type m-filters.IEEE Trans. Acoust., Speech, Sig-nal Process., 36:707–713, 1988.

[7] D. Geman and G. Reynolds. Constrained restorationand the recovery of discontinuities.IEEE Trans. Pat-tern Anal. Machine Intell., 14(3):367–383, 1992.

[8] P. Perona, T. Shiota, and J. Malik. Anisotropic diffu-sion. In Bart M. ter Haar Romeny, editor,Geometry-Driven Diffusion in Computer Vision, volume 1, pages73–92. 1994.

[9] I. Pitas and A. N. Venetsanopoulos. Order statisticsin digital image processing.Proceedings of the IEEE,80:1893–1921, 1992.

[10] I Tabus, D. Petrescu, and M. Gabbouj. A trainingframework for stack and boolean filtering-fast optimaldesign procedures and robustness case study.IEEETrans. Image Processing, 5(6):809–826, 1996.

6