Rank algorithms for picture processing

COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 35,234-258 (1986)

Rank Algorithms for Picture Processing

VLADIMIR KIM AND LEONID YAROSLAVSKII

Institute of Information Transmission Problems, The USSR Academy of Sciences, 19, Yermolovoy str., GSP-4, 101447, Moscow, USSR

Received May 17,1985; revised March 25,1986

A class of nonlinear rank algorithms is introduced based on computation of local rank statistics of pictures. These algorithms are superior to the liuear ones in the simplicity of local adaptation and are free from their basic disadvantage, spatial inertia. Application of rank algorithms to a wide range of picture processing problems such as smoothing, standardization, detail enhancement, extraction of details and their boundaries, coding, etc. is described. 6 1986 Academic Press. Inc.

1. INTRODUCTION

The digital techniques for picture processing that are taking shape now may be classified from the standpoint of the use of computer facilities into two groups-structured and non-structured ones. Structured techniques are those that make use of large program blocks dealing with pixel vectors rather than with individual pixels. Non-structured techniques are those where operational blocks cannot be larger than the usual arithmetic and logic operations over individual pixels. As a rule, they appear at the initial stage of solution of meaningful picture processing problems, and with the progress of the solution they take a structured form. Structuring of picture processing techniques is a prerequisite for enhancement of their computational effectiveness, and for design of picture processing computers and their software.

The linear filtration methods relying on the fast algorithms of convolution and spectral analysis [l] that already have been shaped sufficiently may serve as typical examples of structured techniques. Linear filtration of discrete signal V = { V, }, k = O,l,..., N - 1 is describable as

where hk( V,) is a linear function of Vk:

0.1)

and v, is a weighted sum of the rest of the signal elements. Those that are involved in the weighted sum with non-zero weights form the so-called neighborhood of the given element.

The present paper is devoted to the development of the class of structured non-linear algorithms performing the transformation

(1.3)

where cj~~(V~lk) is, generally, a function defined by some subset of so-called ranks or

234 0734-189X/86 $3.00 Copyright 0 1986 by Academic Press. Inc. All rights of reproduction in any form resewxl.

RANK ALGORITHMS 235

rank (order) statistics formed by signal element in some spatial neighborhood of the given element. Therefore, we refer to this class as rank algorithms.

In order to explain this notion let us introduce the following notations which will be required subsequently:

(k, 1)-coordinates of current pixel. V,, ,-quantized initial value of the videosignal in pixel (k, I); I’,, , E [0, Q - 11;

Q is the number of signal quantization levels; (k, 1) are the co’ordinates of the pixel on a discrete raster.

pkIk. ,-quantized resulting videosignal in element (k, 0 after transformation. S-neighborhood-spatial neighborhood of element (k, II), defined as set of

pixels spatially surrounding the “central” element (k, /) including itself on a raster. N,--size of the S-neighborhood, i.e., the quantity of its constituent elements. { h,(q)}-histogram of pixels values over the S-neighborhood, q = 0.. . ~

Q - 1. V,( r )-ordered in increasing order sequence of pixels of S-neighborhood (a

variational row); values V,(r) are called rth rank (order) statistics over S-neighborhood:

K(r) I V,(r + 1); r=O,l,..., NY-l.

r,(V)-the rank of a pixel having value V, i.e., the number of this pixel in the variational row generated over the S-neighborhood.

M-neighborhood-defined in some manner as’s subset of pixels from S- neighborhood. Variations of the M-neighborhood: “KNKneighborhood” is a neighborhood formed from K-nearest by value neighbors of a pixel (k, I):

i

p+K KAY= I/,(r): C IV,.,- V,(r)! = r$n ;

‘=p 1

“KSN-neighborhood” is a neighborhood formed from K spatially nearest neighbors of a pixel (k, 1); “E~neighborhood”: EV= {V,(r): Ii/,(r) - V,J 5 E,,}; “ER- neighborhood”: ER = {V,(r): Ir - r,(l/,,,)l I E,}.

rKNY( I’)-the rank of V in the ordered sample generated over the KNV- neighborhood.

rEv( I’)-the rank of T/ is the ordered sample generated over the EI’-neighborhood.

V,(r)-the value with rank r of an element in the ord’ered sample over the M-neighborhood.

NM---size of the M-neighborhood. MEAN(M)-the arithmetic mean value of the element,s of the M-neighbor-

hood:

MEAN(M) =

236 KIM AND YAROSLAVSKII

MED(M)-the median over the M-neighborhood:

MED(M) = V,( r = (NM - 1)/2)

CUT( M)-“cut” over the S-neighborhood:

V v,(O s v/&,5 V,(R); CUT(M) = V;;i,, Vk,l < GW;

VM(R)~ vk,, ’ vM(R)-

RAND(M)-random value generated by a generator of random numbers whose distribution histogram coincides with the histogram of pixels values over M-neighborhood.

MIN(M), MAX(M)-the minimal and maximal values in M-neighborhood.

Evidently, one may determine any r th order statistics V(r) of the given neighborhood S through the local histogram {h,(q)} by solving equation

V,(r) c Gd = r. q=o

(1.4)

There exists a fast recursive algorithm [l] for computation of local histograms for neighborhoods of each pixel at successive scamring of the picture by an aperture covering the desired neighborhood. Therefore, computational complexity of the rank filtration algorithms is all but independent of the neighborhood size. Moreover, further simplifications are possible in computations of particular rank statistics and their derivatives, in particular, because of informational redundancy of pictures. Therefore, rank algorithms computationally are not more complex than linear filtration ones.

The most important feature of rank algorithms is that they are essentially adaptive or, to put it more exactly, they are locally adaptive because their parameters, by definition, are functions of the local characteristic of pictures-their local histogram. From the viewpoint of local adaptivity, they not only do not give way to linear filtration algorithms, but even are superior to them. At the same time, rank algorithms are free from the characteristic disadvantage of linear filtration methods, their spatial inertiality which manifests itself in the influence of individual details on the resulting picture at the distance of the order of filter aperture, e.g., in blurring of detail boundaries at picture smoothing, in distortion of detail forms at their extraction from the background, etc.

Since rank algorithms reorder picture data into a l-dimensional ordered sequence, they might appear at first glance to have the basic disadvantage of not making use of the spatial relations between picture elements. Rank algorithms are indeed invariant to signal dimensionality. But no matter how strange it may appear, this property of rank algorithms is rather an advantage than a disadvantage, it is one more aspect of their adaptive nature. The natural structure of picture data is such that spatial relations between pixels (defined, for example, by their membership in one detail) manifest themselves in the variational row through the parameters of a

RANK ALGORITHMS 237

FIG. 1. On representation of spatial relations through local hystogram:: (1’1 original picture; (2). (3) pictures obtained from a generator of random numbers by algorithms: Vk, , == RAND(S) and V&, , = RAND( EV), accordingly; S = 15 x 15 pixels; E,, = 10.

conditional histogram of signal distribution in the neighborhood of the given elements, e.g., through the histograms over the EV- or KNV-neighborhood. It is striking indeed how informative the local histograms and especially conditional histograms are (see Fig. 1).

The term “rank algorithms” appeared in picture processing comparatively re- cently. However, some picture processing algorithms that actually belong to this class have existed for a rather long time. For instance, in [2] the concept of adaptive amplitude transformations was introduced which included then adaptive mode quantization, local histogram equalization, and equalization of histograms raised to some power that was referred to as power intensification. The median filtration algorithms [4, 5, 61 proposed by Tukey in 1971 [3] has found application to pulse noise filtration and picture smoothing from about the mid-70’s. There are also extremal filtration algorithms using the maximal and minimal vlaues over a neighborhood. It is clear now that all of them are representatives of a large family of rank algorithms [ 71.

The aim of this paper is to outline this family and illustrate: applications of the rank algorithms to main picture processing tasks. Of course, each specific algorithm


has its own substantiation and is optimal in a definite sense. But we would emphasize rather the structural unity of the rank algorithms, the possibility to construct them from a small set of elementary operations and notions like MEAN, MED, CUT, EV-, KNV-, KSN-neighborhoods than go to formal substantiation of specific algorithms. For substantiation we shall refer mainly to a similarity and relationship of the rank algorithms to the so-called robust methods of mathematical statistics. Operation of the algorithms is illustrated by specimens of air, space, and medical pictures, and also by test pictures. All the pictures shown below in illustrations have 512 x 512 elements and 256 quantization levels.

2. ALGORITHMS FOR PICTURE SMOOTHING

The notion of picture smoothing has two senses: for correction of signal distortions introduced by the imaging system, it is suppression of (additive, pulse, etc.) noise due to the imperfections of imaging systems; for picture preparation, it is elimination of (usually small) details handicapping perception of the desired objects in pictures. With the correction of distortions caused by the imaging system, it is the so-called “source” picture, i.e., the output of imaging system, that is smoothed. At preparation, smoothing may be used at any stage of the picture processing procedure.

The notion of smoothing always implies some concept of an “ideally smooth” signal which is the aim of smoothing.

The notion of smoothing also implies a concept of what is to be suppressed. Let us call the suppressed portion of the signal “noise.” We shall discuss rank smoothing algorithms for two most characteristic noise models, additive and pulse ones.

Smoothing for additive model. In the additive model, the observed signal is assumed to be the sum of useful signal and noise. It is more natural and easy to substantiate rank algorithms for additive noise smoothing from the standpoint of piecewise constant model of picture signal and cluster analysis as adaptive mode quantization algorithms [l, 21. Indeed, any S-neighborhood of an “ideal” piecewise constant picture, i.e., a “patch-work” picture with arbitrarily situated patches having constant signal value within each patch, has the following histogram of videosignal distribution:

hXq) = CWG - 4t) 9 cw

where qt is the signal value in a spot with number t and area M,. The histogram of the distribution of a videosignal in any S-neighborhood of the observed “non- smooth” picture differs from h:(q) in that its “pure” modes defined by the delta functions in (2.1) are blurred by some factors (noise, distortions, foreign details, etc.). Therefore, it may be written as

h,(q) = c4A,(q - 4th (2.2)

where A, is some unimodal function characterizing mode fuzziness. Thus, h,(q) is, generally speaking, a multimodal function where primary modes

{ A,( q - ql)} manifest themselves as some “clusters” or local maxima.

RANK ALGORITHMS 239

Now, smoothing may be defined as estimation of the shift parameter q1 of the cluster to which the given element belongs. In order to estimate it, it is necessary to determine cluster boundaries. This may be done either by determining boundaries of all the clusters followed by testing where the given element is located [l, 2,8], or by “cluster growing” [9], i.e., by determining the boundaries of the cluster including the current element by growing the cluster from some “growth clenter” related to the current element, The first way is more general, but also is computationally more difficult because it requires determination of the boundaries of all the clusters rather than of the desired one only. In a practice it can be used only in a “jumping window” mode of processing when analysis and estimating is made over all pixels in the window. In a “sliding window” mode when all pixels in the window are analyzed but only the central element of the window is estimated, “cluster growing” method is more preferable. Possible versions of choosing “growth center” and M-neighborhood dete rmining the boundaries of that cluster to which the given pixel belongs are presented as rows and columns entries of Table 1 accordingly. Table 1 itself presents possible versions of estimating algorithms. It is evident from the table that estimates of growth centers may be made iteratively, e.g., in the following way:

&!;j = MELAN( KSN( I’, , ,)) ;

&!‘k’f’l = MEAN( KNV( ir,‘i;” )), i = 2,3,... (2.3)

With increase of the number of iterations, the resulting estimate approach that for the adaptive mode quantization algorithm described in [l, 2,8].

The choice of the neighborhood type is defined by the available a priori information on the processed picture.

The choice of KNV-neighborhood allows one to take into consideration a priori information about geometrical dimensions of the details to be preserved. As a rule K should be of the order of the area of the details to be preserved by smoothing.

The choice of EV-neighborhood allows one to take into consideration the a priori information about minimal overfalls to be preserved, or information about the variance of noise to be suppressed.

ER-neighborhood finds application in the pulse noise filtration algorithms [lo] and in the boundary extractions algorithms to be described below.

KSN-neighborhood in principle may be built around each tylpe of growth center shown in the Table 1, but there is no reasonable reason to use it otherwise than only

TABLE 1 -__

M-neighborhood

Grows center KSN-neighborhood KN V-neighborhood E V-neighborhood ER-neighborhood

V

A>:hmetic KWG,,) KNV(v,,,) WV,,,) ER(l/k,,)

- KNV(MEtAN(M)) EV(MEAN(M)) ER(MFiAN(M)) mean over M

Median - KN V(MED( M)) EV(MED(M)l ER(MED( M)) over M

Cut over M KNV(CUT( M)) EV(CUT( M)) ER(CUT( M))


at first iteration of the algorithms. In a partial case KSN-neighborhood may coincide with S-neighborhood.

As for the size of S-neighborhood, it is recommendable that it be of the order of the double size of the minimal detail to be preserved by smoothing. Owing to the adaptivity of the rank algorithms, the form of S-neighborhood weakly affects the quality of smoothing.

The choice of the estimate of a current picture element over a cluster as well as the position of the “growth center” are defined by the nature of the distribution of noise to be suppressed. If the distribution is of Gaussian type, a better estimate is provided by the arithmetic mean; if it has heavier tails, it is recommendable to use median or cut [ll]. It should be noted that in the literature on mathematical statistics the cut over E&neighborhood is referred to as winsorizing and the mean over ER-neighborhood is referred to as trimmed mean. Sometimes, for rank picture processing algorithms cuts over KNV- and EV-neighborhoods are preferable.

Thus, a whole family of rank smoothing algorithms may result from choosing various means of estimation, cluster growth center, and neighborhoods. Some representatives of this family may be found in the literature. For instance, [12] describes an algorithm for averaging over KV-nearest neighbors which may be written in our notation as

fiklk,[ = MEAN(KNY(Vk,,)). (2.4

A “sigma-filter” operating according to the following algorithm is described in [13]:

with E” = l.Sa, where u is standard deviation of Gaussian noise which is assumed to be known. A similar filter with some internal window M like KSN-neighborhood is described in [14] for smoothing speech signals:

&, = MEAN(a,(MED(M))). P-6)

Evidently, median filtration which is now one of the most popular smoothing method [3, 4, 5, 6, 15, 161:

&, = MED(S) P-7)

is one the simplest methods of the same family. Figures 2(l)-(5) show examples of smoothing by algorithms (2.5) and

&, , = MEAN( ZW(MED( K&V( V’, [)))). (2.8)

In the latter case, KSN-neighborhood is the 3 X 3 neighborhood of the central element. Figures 3(l)-(3) show another example of smoothing by (2.8). Comparison of Figs. 2(3) and 2(5) reveals how the concept of noise varies depending on the smoothing algorithm parameters: suppressed is either a weakly correlated component which may be regarded as noise of the videosignal (Fig. 2(3)), or, at stronger smoothing, that component which contains small texture details of the picture.

RANK ALGORITHMS 241

FIG. 2. (1) The initial picture: (2) smoothing by (2.5), over 7 x 7 S-neighborhood, eV = 10: (3) 17 times amplified difference between 1 and 2; (4) smoothing of (1) by (2.8) over 1.7 x 17 S-neighborhood, M-neighborhood is 3 X 3 KSN-neighborhood of a central element, K = 90: (5) 3 times amplified difference between 1 and 4; (6) the result of boundary extraction by (4.1). 9 X 9 S-neighborhood. L = 30, R = 52: (7) the result of applying (4.3), 9 x 9 S-neighborhood; (8) the result of boundary extraction by (4.1) for 9 x 9 S-neighborhood, E,, = 11. M: 3 x 3 KSN-neighborhood, SMTH = MEAN.

Smoothing for the pulse noise model. It is assumed in the pulse noise model that a random variable is substituted with some probability for the signal element. Obviously, pulse noise smoothing implies detection of distorted signal elements and their subsequent estimation through non-distorted ones. Generally, pulse noise smoothing algorithm should have two passes-for marking the: distorted elements and for estimation of their smoothed values. For the sake of simplicity, however, the algorithm may be made to have a single pass combining detection and estimation.

The problem of marking picture elements as noise-distorted and non-distorted is that of detecting noise outburst or testing the hypothesis that the central element of S-neighborhood either belongs to the same sample as the given majority of the rest of neighborhood elements, or falls out of it. This is a rather standard problem of the mathematical statistics, and it is usually recommended to solve it by means of rank statistics algorithms [ll].


FIG. 2-Continued.

A simplest rank technique for testing whether the central element of S-neighborhood belongs to the sample from the majority of the rest of neighborhood elements is testing whether rank r,(V,, ,) lies within the ER-neighborhood of the median (,‘ voting algorithm,” see [lo]) defined as depending on the probability of pulse noise occurrence per pixel,

kv-/A - (4 - WI < Er. (2.9)

If it is concluded that there is no noise, otherwise the element (k, I) is marked as noise-distorted. In essence, this technique of noise detection assumes that as a rule the pulse noise takes extremal values. It should be noted that as the test of the hypothesis that the element belongs to the given sample, the rank is a special case of the Wilcoxon test [17] for the presence of shift between two samples with identical distributions.

Let us note now, that the parameter (N, - 1)/2 in (2.9) is nothing but the rank of median and the last term may be considered smoothed estimation of the value of the current pixel. So the criterion (2.9) may be rewritten in more general form as Psv-k,,) - ~,(sMTww)l < El.9 where SMTH(M) is a smoothed value of V,, , obtained by means of one of the above smoothing techniques for the additive model.

RANK ALGORITHMS 243

FIG. 3. (1) The initial picture; (2) smoothing by (2.8), for 15 x 15 S-neighborhood. 3 x 3 M-neighborhood. K = 60; (3) 6 times amplified difference between (1) and (2); (4) the result of boundary extraction in smoothed picture by (2.12) with subsequent thresholding for 9 x 9 S-neighborhood, Z, = 31, R = 51, lower threshold = 2, upper threshold = 6; (5) the result of boundary extraction b,v (2.12) in non-smoothed picture, algorithm parameters as above: (6) superposition of (4) over (1).


Such test criterion means that the test ,%-neighborhood is build symmetrically around the rank r,(SMTH(M)). This corresponds to the assumption that in S-neighborhood undistorted videosignal values are distributed symmetrically around some characteristic value, estimated by the SMTH(M). Of course it is not neces- sarily so in real pictures. Consequently the best criterion for testing whether the current pixel is noisy is testing whether this pixel belongs to the KNV-neighborhood of the SMTH(M):

V,, , E KNV(SMTH( S)) (2.10)

given K is also defined depending on the probability of noise outbursts. The hypothesis about presence or absence of noise outburst in the central element

of S-neighborhood may be also tested by its magnitude rather than its rank. For instance, the sign of difference

A = sy - ] V,., - SMTH(M)] (2.11)

may be used as a test, where ey is selected depending on distribution of noise and scatter of signal values. This technique better corresponds to those features of pictures as signals that manifest themselves in the fact that values of signal in geometrically neighboring pixels are as a rule close to one another.

The threshold sy may be either chosen right away for the whole picture, or adjusted adaptively depending on the local scatter of signal values. The value of the interquantile distance over .s,-neighborhood may be used, for example, as local scatter estimate:

QDISP(S) = V((N, - 1)/2 + Ed) - v,((N, - 1)/2 - a,) (2.12)

which,is known [18] to be stable-to-distribution scale estimate. Following detection, the pixels marked as pulse noise outbursts should be

replaced by their estimates. Estimation may be made through the values obtained via one or more smoothings over the neighborhood of these elements, elements marked at detection of noise outbursts being eliminated from the neighborhood.

Thus, the two following types of pulse noise smoothing algorithms are possible:

A “J =

Vk,,, V,,, E KNV(SMTH,bW)); SMTH,(M,), V,,, 4 KNV(SMTH,(M,));

(2.13)

LI

‘J = Kc,,, &v 2 I h,, - SMTH,(W)I ;

SMTH,( M,), &v < IV,,, - SMTH,@f,); (2.14)

where SMTH,(M,) means smoothing over the neighborhood from which elements to be corrected are eliminated.

Characteristic errors of the pulse noise smoothing are those of false detection resulting in undesirable smoothing of details, and those of passing that lead to the occurrence of non-smoothed noise outbursts in the picture. The percentage of these errors depends on the thresholds: with their reduction, the portion of false detections increases while that of passings decreases. One has also to bear in mind that

RANK ALGORITHMS 245

the amount of false detections and passings also grows due to the plausibility of multiple great noise outbursts in the S-neighborhood rather than of a single one. Therefore, in order to enhance the quality of pulse noise smoothing, it is expedient to do it iteratively beginning with higher thresholds and reducing them at each iteration as great noise outbursts are eliminated [l, lo].

3. ENHANCEMENT OF PICTURE DETAILS

Enhancement of picture details is opposite to smoothing. If ,the latter erases the differences between details, the former makes them more conspicuous. Therefore, picture detail enhancement is also called “enhancement of local contrasts.” In essence, it is the basic operation of picture preparation.

Local contrasts are enhanced by measuring the differences between the signal in each picture element and those in surrounding elements, and by amplifying these differences.

The most popular and evident method of determination and ienhancement of the differences is the so-called non-sharp masking where one computes the difference between the values of pixels and the values averaged over the neighborhoods of these elements, amplifies it, and adds to the averaged picture.

This operation may be described by the following general formula:

&,, = kvk,, - Vk,J + v,J (3.1)

where v,,, is the sum of weighted elements in S-neighborhood and g is gain. It should be noted that from this formula follows the possibility of generalizing the non-sharp mask method to rank algorithms. It consists of using, instead the weighted mean over the S-neighborhood, the smoothed signal value SMTH(M) obtained by means of the above rank smoothing algorithms:

k., = A,, + SMTH(M), (3.2)

where

d k,l 1 = ‘k I - SMTH(M). (3.3)

The advantages of the non-sharp masking with rank rather than linear smoothing -adaptivity and lower spatial inertia follow from the advantages of rank smoothing.

Examples of application of this algorithm are shown in Figs. 4(l)-(4) and 5(l)-(4).

The aim of processing the picture shown in Fig. 4 was enhancement of its texture over field areas. Smoothing was done via the algorithm (2.6). The difference picture shown in Fig. 4(3) shows which texture elements were enhanced.

The aim of processing the picture shown in Fig. 5 was contrast enhancement of details smaller than the width of ribs in the picture. Smoothing was done via the following algorithm:

SMTH(kf) = MED(fwv(vk,,j). (3.4)


FIG. 4. (1) The initial picture; (2) smoothing by (2.6) for 15 x 15 S-neighborhood, M: 3 x 3 KSN-neighborhood, ev = 10; (3) 6 times amplified difference between (1) and (2); (4) detail enhancement by (3.2). (2) was used as smoothed picture; (5) sliding equatization of 1 for 15 X 15 S-neighborhood; (6) detail enhancement by (3.12), a = 0.5, p = 0.5.

RANK ALGORITHMS 247

FIG. 5. (1) The initial picture; (2) smoothing by (2.4) for 125 x 125 S-neighborhood, K = 7000: (3). (4) detail enhancement by (3.2), g = 3 and 5, respectively; (5) 12 times amplified difference between (1) and (2).


The smoothed and difference pictures (Figs. 5(2) and (5)) demonstrate that smoothing has not practically distorted the boundaries of the ribs, but details smaller than the rib width were suppressed. This means that only they have their contrast enhanced in the resulting picture.

Another way of local contrast enhancement is equalization of local histograms and its generalization, power intensification [l, 21. In doing so, the picture is subjected to elementwise nonlinear transformation F( V,,,) whose steepness in the point vk,, is proportional to some power p of the value of local histogram in the point q = V,,,:

AfP%,, = ~mlq=vk.,. (3.5)

The local histogram equalization corresponds to p = 1:

wwc,, = ~s(9)lq-vk.,- P-6) Solution of this difference equation shows that the transformed value of V,,, is defined at equalization as follows:

“k.1

f/c,, = f’(V,,,) = cl c h,(q) + ~23 q=o

(3.7)

where ci is normalization constant, and c2 is shift constant. These constants being self-evident, we omit them below. Therefore, (3.7) may be rewritten

“k.1

64 = c ed.

q=o (3.8)

It follows from this formula that local histogram equalization is nothing but substitution for the pixel value V,,, of its rank in the variational row constructed over the S-neighborhood:

c,, = cw-k,,). (3.9)

At power intensification, rank is obviously determined through histogram values raised to power p rather than through the histogram itself.

Figure 6(l)-(4) present examples of local histogram equalization with different S-neighborhood sizes. Figures 6(2), (3), and (4) show dependence of enhanced- contrast details on the size of S-neighborhood.

Of course, power intensification is a special case of signal transformation whose steepness is proportional to some function of local histogram value

(3.10)

Such a transformation may be referred to as f-equalization. Of ‘great practical interest is a version of the f-equalization with f being linear function

f@,(d) = e(9) + P9 (3.11)

RANK ALGORITHMS 249

FIG. 6. (1) The initial picture; (2) sliding equalization over 125 X 125 S-neighborhood; (3) sliding equalization, 35 x 35 S-neighborhood; (4) sliding equalization, 15 X 15 S-neighborhood; (5) smoothing of 4 by (2.8) 15 x 15 S-neighborhood, 3 X 3 M-neighborhood, K = 100: (6) smoothing of (5) subJected to sliding equalization with 19 x 19 S-neighborhood by (2.8) having the same parameters as for (5); (7) sliding equalization (19 x 19 S-neighborhood) of (6); (8) the final binary picture.

where CY and /3 are some constants. In this case, the transformed picture obviously is a weighted sum of the equalized and initial pictures:

(3.12)

Figure 4(6) demonstrates an example of “linear” f-equalization according to (3.12) where the original picture (Fig. 4(l)) and the equalized one (Fig. 4(5)) are summed with weights 0.5 and 0.5.

The “rank” treatment of equalization enables some natural generalizations due to the changes in the form of the neighborhood: “equalization over EV ‘and’ equalization over KNV.”

At equalization over El?

(3.13)


FIG. 6-Continued.

At equalization over KNV:

(3.14)

Equalization over EV- or KNV-neighborhood allows one to avoid the influence on the transformed details of details belonging to another “cluster” but occurring in the S-neighborhood. The difference between local histogram equalization and that over EV is illustrated by the test picture in Fig. 7. It may be seen that at equalization over EV the boundaries of the square are all but suppressed, while standard equalization enhances them along with enhancement of small contrast texture.

Comparison of standard equalization and those over EV- and KNV-neighborhoods on a real picture may be found in Fig. 8. One can see that equalization over KNV produces results closer to the standard equalization which is due to the fact that under the chosen values of K and E” the size of M-neighborhood where the ranks are calculated is closer to the whole S-neighborhood than at equalization over EV. By varying K and E” one may vary transformation from pure equalization (at K = N, or E y = Q/2 to degeneration at k = 1 or .sy = 0.

The choice between EV- and KNV-neighborhoods is defined, like in smoothing, by the a priori information about signal scatter and size of picture details to be contrast-enhanced.

RANK ALGORITHMS 251

FIG. 7. (1) The original model picture; (2) graph of a row of (1); (3) sliding equalization of (1) over (15 x 15 S-neighborhood); (4) graph of a row of (3); (5) sliding equalization of (1) over EV-neighborhood (15 x 15 S-neighborhood); (6) graph of a row of 5.

Another possibility of equalization generalization lies in replacement of V,, , by some function of rank

(3.15)

rather than by its rank over one or another neighborhood. Such a. processing may be


FIG. 8. (1) The original; (2) sliding equalization (15 X 15 S-neighborhood) of (1); (3) sliding equalization of (1) over KNV-neighborhood by (3.18), 15 x 15 S-neighborhood, K = 100; (4) sliding equalization of (1) over EV-neighborhood, 15 x 15 S-neighborhood, E,, = 10.

exemplified by the histogram hyperbolization algorithm [20]:

C,, = exd~sWk,J). (3.16)

Indeed, in doing so In I$, , will have uniform distribution required at hyperbolization.

The above discussion on equalization refers to power intensification as well. It is possible to perform it over EV and KiVV. Of special interest is power intensification at P = 0. It follows from (3.5) that in this case the transformed signal is determined as follows. At power intensification over S-neighborhood, P = 0:

L- V,,, - MIN(S). (3.17)

This formula describes the well-known algorithm of the so-called linear correction of videosignal.

RANK ALGORITHMS

At power intensification (P = 0) over KNV:

253

(3.18)

At power intensification (P = 0) over EV:

6,, = V,,, - MIN(EV(I/,,,))

I,t has no sense because as a rule MIN(EV(Vk,,)) = Vk,, -’ E”, and, therefore, V,, , = ey independently of V,, ,.

This brings to mind the fact that EV and KNV-neighborhoods are estimates of the cluster position in the variational row. In this case, V,, , may be regarded as an estimate of a cluster “growth center.” Hence, a smoothed estimate SMTH(M) should be involved in the general case instead of V,, ,, for instance:

vk,, = r,(SMTH(M)); (3.19)

&, = w,(SMTH(M)) + pSMTH(M); (3.20)

l$ I = V,,, - MIN( KN(SMTH(M))); (3.21)

fk,, = Vk,, - MIN( EV(SMTH( M))). (3.22)

Thus, one may consider two groups of rank algorithms for local contrast enhancement: algorithms of the difference type like (3.2) with rank srnoothing and purely rank algorithms (3.13)-(3.22).

Formally (3.18), (3.21), and (3.22) also may be classified as difference-type algorithms, but at the same time they bridge these two groups because there seems to be no basic difference between them. Their kinship is seen, for example, in the algorithm (3.22).

The difference-type detail enhancement algorithms correspond to the concept of details additively imposed on the background. But the concept of details as if cut into the background better corresponds to the notion of details and background. It follows from this that detail detection should precede contrast enhancement like in the pulse noise filtration algorithms. This may be done by means of switching algorithms like

p k,[

= SMTH(M), I&,,1 5 &vi

i i&c,, + SMTH(M), Id/c.,1 ’ Ev.

Algorithms of this kind may be exemplified by the algorithm [19]

(3.23)

V k.I = i

MED(S), IV,,, - MED(S)/ 2 E,,;

id’,,, - MED(S)) + MED(S), 1 V,., - MED(S)I < E”; (3.24)

where a value proportional to the interquantile distance determined through ranks NJ4 and 3NJ4 (the so-called interquartile distance) is proposed as an adaptive threshold E “.

Detail enhancement through enhancement of local contrasts, no difference is made between picture details, and their difference from the background is defined by the neighborhood parameters: S-neighborhood, EV-neighborhood, KNV- neighborhood. Therefore, detail enhancement is only a part of picture processing


procedure which might be called picture detail extraction and into which parameters are introduced in one or another form describing some details as useful and other ones as unnecessary. This means that the result of local contrast enhancement in principle should be subjected to smoothing for elimination of unnecessary details and for preservation of useful ones.

Depending on the final aim, two kinds of picture detail extraction algorithms may be indicated: detail extraction with preservation of the “background,” and separation of details from the background, i.e., their representation against a constant background. These algorithms are obtained by uniting the above local contrast enhancement and smoothing algorithms into a 2-stage procedure that enhances the local contrast over some &-neighborhood and smoothing over some S,-neighborhood. As a rule, S,-neighborhood is a part of the &-neighborhood. Let us demonstrate several examples of such procedures.

Procedures with detail extraction and background preservation:

&, = ~SMTH(S,(f(r,,(SMTH(S,))))) + BSMTH(S,); I$, = SMTH(S,(I$, - MIN(KN(SMTH(S,)))));

pkk,, = SMTH(S,(gd,,, + SMTH(S,))).

Procedures with separation of details from the background:

&,, = SMTH(S,(f(r,,(SMTH(S,)))));

(3.25)

(3.26)

(3.27)

(3.28)

&, = SMTH(S,(V,,, - SMTH(S,))). (3.29)

Figure 6(5) presents an example of equalized picture smoothing by the following algorithm:

tikk,, = MED( KNV(MED( M))). (3.30)

Notably, some of the above detail extraction procedures coincide in special cases with the procedures used in the mathematical statistics for hypothesis testing. For instance, (3.28) in a special case

c,, = M~NWsc(~k,,))) (3.31)

coincides with the well-known Wilcoxon test [17]. In the case of f(r) = r’, where t is the exponent, we obtain the statistics of the

Tamura test [17]:

c,,= M~N(S,hwk,,))). (3.32)

In the case of f(r) = sign( rsC( V,, ,) - (N,, - 1)/2) we obtain the so-called median test [17]:

64 = M~N(Ssb4~scWk,J - UC - WW). (3.33)

Let us now discuss several natural generalizations of the above rank algorithms for local contrast enhancement and for picture detail extraction.

It may be readily demonstrated that linear local contrast enhancement algorithms (3.2) may be implemented by means of parallel simplest filters computing the local

RANK ALGORITHMS 255

mean

V,,, - i ~Y,MELAN(S,) + : a,MEAN&), (3.34) u=l u=l

where (Y, are weight-selected so as to provide the best filtration effect ([21]). The similarity between (3.2) and (3.12) as well as (3.20) allows one to suggest the

following more general parallel rank algorithm for local contrast enhancement via equalization:

hk., = f ~J,,(SMTH(~)) (3.35) U’l

where rsu is rank, computed over the neighborhood S,. Such a weighted summation of the results of equalization over neighborhoods of different sizes enables one to balance the degree of contrast enhancement of details having different sizes.

Another possibility for generalization of the detail extraction rank algorithms follows from the general form of formulas (3.25)-(3.27) where procedure pairs “smoothing-local contrast enhancement” occur, and it would be natural to con- tinue them iteratively. Owing to the signal quantization effects, such a successive repetition of these pairs converges rather quickly to the binary preparation. Exam- ples of pictures obtained through such a processing are shown in Figs. 6(6), (7) and (0

4. EXTRACTION OF PICTURE DETAIL BOUNDARIES

The relation between “local enhancement-detail extraction” algorithms and rank test statistics as well as their obvious similarity to the above pulse noise outburst detection algorithms casts a new light on the sense of these algorithms. It follows that the “local contrast enhancement-detail extraction” algorithms may be treated as those of testing hypothesis about the disagreement of the central element of SC-neighborhood with a sample defined by a subset over EV- or KNV-neighborhood of the elements of S-neighborhood. A signal estimated by them. may be treated as a test whose value in each pixel of a processed picture is presented to the user as a prepared picture.

Such a treatment leads to generalization of detail extraction algorithms to detection of details and their boundaries. In the above algorithms, simplest point tests were used for testing whether a given pixel belongs to a given statistical sample, such as the difference between the value of the central element of neighborhood and the estimated mean over the neighborhood (difference-type algorithms), or the number of elements in the given neighborhood whose value does not exceed that of the central element, i.e., the rank of the central element in the given neighborhood (rank algorithms). The degree of disagreement was regarded as detail contrast.

For rank detection of picture details and their boundaries ir; seems neces%uy to measure the degree of statistical disagreement between the distribution characteristics of the elements of all the neighborhoods of the central element and given statistical characteristics describing signal distribution within a. detail. In doing so, the neighborhood size is chosen to be of the order of detail size ‘(for detail detection) or detail boundary neighborhood (for boundary detection). To measure the dis-


agreement degree, one may employ various goodness-of-fit tests including those used in mathematical statistics.

In this case, detection as such is a comparison of the measured degree of agreement with the threshold. At picture preparation, it would be reasonable, to visualize the value of agreement rather than the binary result of thresholding alone. Detection is then performed visually by the human operator.

Rank detection algorithms based on the comparison of signal histograms are, evidently, insensitive to the spatial “entanglement” of picture elements. Fortunately, spatial “entanglement” usually is not among possible picture transformations, therefore, the danger to “confuse” a detection of a picture with a sequence of independent samples having the same distribution as signal sample distribution on the detail is negligible. At the same time, rank detection algorithms are stable to the widespread signal distortions such as monotonic changes of value at amplitude distortions, distribution contamination, changes in orientation.

Let us consider several examples of possible rank algorithms for picture detail boundary extraction. For the “piecewise constant” picture model, it is the 2-mode nature of the local histogram that is boundary attribute. Characteristics of local histogram scatter such as, for instance, interquantile distance (2.12) where ER- neighborhood is chosen sufficiently small, may be used as 2-mode appearance test. It is recommendable for practical purposes to take E, = &.

A more general algorithm for computation of the interquantile distance over the M-neighborhood is exemplified by

fk,, = MAX(ER(SMTH(M))) - MIN(ER(SMTH(M))), (4.1)

which, basically, is capable of extracting finer boundaries. Naturally, it is expedient, prior to applying boundary extraction algorithms, to smooth pictures in order to improve the agreement between the picture and “piecewise constant” model.

Figure 2(6) shows an example of boundary extraction by the “interquantile distance” algorithm. For comparison, Fig. 2(7) shows the result of processing the same picture by the local variance algorithm,

&, = (MEAN(S(Q, - MEAN(S))‘)“‘. (4.2) This example demonstrates the spatial inertia of linear algorithms and how it is

eliminated by the rank algorithms. An example of using an algorithm of the (4.1) type may be seen in Fig. 2(8). Another example of using an interquantile distance algorithm for outlining detail boundaries (Figs. 3(4), (5), (6)) illustrates the role of smoothing before boundary extraction.

5. OTHER APPLICATIONS OF RANK ALGORITHMS

Apart from smoothing, detail enhancement, extraction of details and their boundaries, the rank algorithms may be used for solution of many other, more special picture processing problems such as diagnostics of the characteristics of video-signal distortions, picture standardization, determination of the videosignal statistical characteristics, measurement of texture features, and picture coding.

Automatic diagnostics of distortion and noise parameters. Automatic diagnostics of the parameters of videosignal distortions and noise may be built around detection and measurement of anomalies in videosignal statistical characteristics [lo, 221. In

RANK ALGORITHMS 257

each case, that statistical characteristic of the distorted vide,osignal is measured which enables the easiest detection of the anomalies caused by distortions. Anoma- lies in videosignal characteristics may be detected by means of rank algorithms such as the “voting algorithm” for testing whether the analysed sample element belongs to the given number of extremal (greatest or smallest) values of the ordered sample. Such algorithms were described in detail in the above discussion of the rank smoothing algorithms.

Picture standardization is reduction of picture characteristics to given ones. Histogram standardization, i.e., transformation of a videosignal reducing its distribution histogram to a given form, may be rather easily done by means of rank algorithms such as the following simple algorithm:

where Vs.(r) is the value of an element of the ordered sample constructed via the given (standard) histogram and having rank r. Depending on particular require- ments, histogram standardization may be either global, or local. In the former case, computations make use of the histogram of all the standard picture; and in the latter case, that of the sliding fragment of the standard picture is used.

Determination of the statistical characteristics of videosignal and measurement of texture features. Adaptive properties of the rank algorithms make them a convenient tool for measuring local statistical characteristics of pictures such as local mean, local variance, and other distribution moments. Obviously, these and other similar characteristics of local histograms are at the same time texture characteristics of pictures.

Rank algorithms may be used for estimation of not only histogram texture features, but also for estimation of texture features related to the local spatial statistical characteristics of pictures. The number of local extrema in the neighborhood S of the processed element is one of the simplest features of this sort [23, 241. Some texture features occur at characterization of the spatial distribution of local extrema, i.e., mean distance between them, variance of the distance betwen them, etc.

Features characterizing spatial distribution of ranks in the processed fragment, are of more general nature. In particular, the number of sign changes of the first derivative over a fragment of equalized picture under assigned direction of scanning at determination of the sign of the first derivative is also a texture feature. A number of texture features may be suggested as parameters of the spatial distribution of the elements of the above El/- and KNV-neighborhoods, in particular, moments of their mutual distance distribution.

Picture coding. Applicability of rank algorithms to picture coding is due to the possibility of using the adaptive mode quantization algorithms [l, 21 for fragment- wise processing. In this case, one analyses the histogram of pixel distribution within a fragment (or block as they say in the coding), determines cluster boundaries that are chosen as boundaries of quantization intervals, and quantizes the fragment samples according to the determined boundaries. As a rule, if the fragment is not too large, the number of quantization levels, Q,, of fragment samples is far less than the number of quantization levels, Q, chosen from the condition of high-quality picture reproduction. It may be readily computed that the number of bits required


for reproduction of values of IV, fragment samples is equal to the sum of QJog,Q bits for transmission of the quantization table and of N,log,Q, bits for transmission of the quantization level number, i.e., one sample requires an average of log,Q, + (QslogzQ)/Ns bits instead of log,Q bits without adaptive fragment-wise quantization. Hence, it is reasonable to increase fragment area until the number of quantization levels Q, is several units at most. Experimental fragment-wise mode quantization [l, 21 has demonstrated that this is possible for fragments of up to 30 x 30 elements. This means that the value of the order of 1 to 2 bits per element is the estimate of the potentialities of picture coding.

REFERENCES 1. L. P. Yaroslavskii, Vvedenie v tsyfrovuyu obrabotku izobrazhenii (Introduction to Digital Picture

Processing), Moscow, Sov. Radio, Moscow, [Russian]; Engl. transl. Digital Picture Processing, An Introduction, Springer-Verlag, Berlin/Heidelberg/New York, 1985.

2. T. P. Belikova and L. P. Yaroslavskii, Ispol’zovanie adativnykh amplitudnykh preobrazovanii dlya preparirovaniya izobrazhenii (Use of the adaptive amplitude transformations for picture preparation), Voprosy radioelectroniki, ser. Obschchetekhnicheskaya, vyp. 14, 1974, 88. [Russian]

3. J. W. Tukey, Exploratory Data Analysis, Addison-Wesley, New York, 1971. 4. T. S. Huang, Introduction, in: Two-Dimensional Digital Signal Processing, II, Topics in Appl. Phys.

Vol. 43, Springer-Verlag, New York/Berlin, 1981. 5. W. K. Pratt, Digital image Processing, Wiley, New York, 1978. 6. B. R. Frieden, A new restoring algorithm for preferential enhancement of edge gradients, J. Opt.

Sot. Amer. 66,1976,280. 7. L. P. Yaroslavskii, Editor’s introduction, in Bysttye algorithmy v obrabotke izograzhenii (Fast

algorithms for picture processing) (Russian transl. of T. S. Huang, Ed.), Two Dimensional Digital Signal Processing, II, Topics in Appl. Phys. Vol. 43, Springer-Verlag, New York, 1981.

8. A. Rosenfeld, Picture Processing by Computers, Academic Press, New York, 1969. 9. E. A. Patrick, Fundamentals of Pattern Recognition, Prentice-Hall, Englewood Cliffs, N.J., 1972.

10. L. P. Yaroslavskii, Theory and MethoaYs of Digital Processing in Optical and Holographic Systems, Doctoral thesis, GOI, Leningrad, 1982. [Russian]

11. M. G. Kendall and A. Stuart, Inference and Relationship, Griffin, London, 1967. 12. L. S. Davis and A. Rosenfeld, Noise cleaning by iterated local averaging, IEEE Trans. SMCJ, 1978,

705-710. 13. J. S. Lee, Digital image smoothing and the sigma filter, Comput. Vision Graphics Image Process 24,

1983, 255-269. 14. C. A. Pomalaza-Raez and C. D. McGillem, An adaptive, nonlinear edge-preserving filter, IEEE

Trans. Acoust. Speech Signal Process. ASSP-32, No. 3, 1984, 571-576. 15. B. I. Justusson, Median filtering: Statistical properties, in Two-Dimensional Digital Signa/ Processing,

II (T. S. Huang, Ed.), Topics Appl. Phys. Vol. 43, Springer-Verlag, New York 1981. 16. S.-G. Tyan, Median filtering: Deterministic properties, Two-Dimensional Digital Signal Processing, II

(T. S. Huang, Ed.), Topics in Appl. Phys. Vol. 43, Springer-Verlag, New York, 1981. 17. J. Haek and 2. Sidak. Theory of Rank Tests Academia, Prague, 1967. 18. H. A. David, Order Statistics, Wiley, New York, 1970. 19. I. Scollar, B. Weidner, and T. S. Huang, Image enhancement using the median and the interquartile

distance, Comput. Vision Graphics Image Process. 25, No. 2, 1984, 236-251. 20. W. Frei, Image enhancement by histogram hyperbolization, Comput. Graphics Image Process. 6,

1977, 256. 21. L. P. Yaroslavskii, On the possibility for parallel and recursive implementation of digital filters,

Radiotechnika, 1984, No. 3, 87. [Russian] 22. L. P. Yaroslavskii, Automatic diagnostics of picture distortions and methods of adaptive correction

of imaging systems, in 14th Intl Congr. Rapid Photography and Photonics, Moscow, 1980, p. 260. [Russian]

23. A. Rosenfeld and E. B. Troy, in Proc. Commun. Conf., Univ. of Missouri-Rolla, Oct. 1970, Sect 10-l. 24. 0. R. Mitchell, C. R. Myers, and W. Boyne, A max-mitt measure for image texture analysis, IEEE

Trans. Comput. C-26, No. 4, 1977, 408.

Documents

Rank algorithms for picture processing