An overview of median and stack filtering - TUNImoncef/publications/overview-of-median.pdf · AN OVERVIEW OF MEDIAN AND STACK FILTERING* Moncef Gabbouj, 1 Edward J. Coyle, 2 and Neal

CIRCUITS SYSTEMS SIGNAL PROCESS VOL. 11, NO. 1, 1992

AN OVERVIEW OF MEDIAN AND STACK FILTERING*

Monce f Gabbouj, 1 Edward J. Coyle, 2 and Neal C. Gallagher, Jr. 2

Abstract. Within the last two decades a small group of researchers has built a useful, nontrivial theory of nonlinear signal processing around the median-related filters known as rank-order filters, order-statistic filters, weighted median filters, and stack filters. This required significant effort to overcome the bias, both in education and research, toward linear theory, which has been dominant since the days of Fourier, Laplace, and "Convolute."

We trace the development of this theory of nonlinear filtering from its beginnings in the study of noise-removal properties and structural behavior of the median filter to the recently developed theory of optimal stack filtering.

The theory of stack filtering provides a point of view which unifies many different filter classes, including morphological filters, so it is discussed in detail. Of particular importance is the way this theory has brought together, in a single analytical framework, both the estimation-based and the structural-based approaches to the design of these filters.

Some recent applications of median and stack filters are provided to demonstrate the effectiveness of this approach to nonlinear filtering. They include: the design of an optimal stack filter for image restoration; the use of vector median filters to attenuate impulsive noise in color images and to eliminate cross luminance and cross color in TV images; and the use of median-based filters for image sequence coding, reconstruction, and scan rate conversion in normal TV and HDTV systems.

1. Introduction

Linear filters have long been the pr imary tool for signal and image processing. They are easy to implement and analyze and, perhaps most importantly, the linear filter which minimizes the mean squared error criterion can usually be found in closed form. Furthermore, they are optimal a m o n g the class of

* Received January 2, 1991. 1 Research Institute of Information Technology, Tampere University of Technology, P.O.

Box 527, SF-33101 Tampere, Finland. 2 School of Electrical Engineering, Purdue University, West Lafayette, Indiana 47907, USA.

8 GABBOUJ, COYLE, AND GALLAGHER

all filtering operations when the noise being considered is additive and Gaussian.

Unfortunately, a small deviation from this Gaussian assumption some- times leads to a severe deterioration in the performance of linear filters. In the many applications in which non-Gaussian noise arises, linear methods have thus proven to be inadequate for signal smoothing and noise reduction. One such case occurs in the presence of speckle noise. Other types of non-Gaussian and/or signal-dependent noise also cause problems. We believe that these cases occur more frequently than not; therefore, linear methods are not completely satisfactory when dealing with real signals and noise rather than simply computer simulations.

For a concrete example, consider the experiment conducted by Dr. J. Johnson of Redstone Arsenal. The image shown in Figure 1.1 is the original

I

Figure 1.1. Image before median filtering. This is an original two-dimensional image obtained by Dr. J. Johnson of Redstone Arsenal with an experimental laser imaging system at the far infrared wavelength of 1.2 mm. The image was obtained by scanning the laser over a scene consisting of a highly reflective copper corner cube sitting on a table with a forest in the background. If there was no noise, the image would appear as a black square, the return from the copper corner cube, on a gray background, the return from the forest and the table. The location of the cube is not apparent, though, because of speckle noise, thermal noise in the receiver, fluctuation in the laser beam's intensify, and other noise sources.

AN OVERVIEW OF MEDIAN AND STACK FILTERING 9

two-dimensional image obtained with an experimental laser imaging system at the far infrared wavelength of 1.2 ram. The image was obtained by scanning a laser beam over a scene consisting of a highly reflective copper corner cube sitting on a wooden table with a forest in the background. In the absence of any noise, the image would appear as a solid black square, the return from the cube, on a gray background, the return from the forest and the table. However, the picture in Figure 1.1 shows no trace of any meaningful object. This is due to the speckle-noise, thermal noise in the receiver, fluctuations in the intensity of the laser beam, and other noise sources.

Attempts with numerous linear filters on this image all failed. One reason for these failures is that both the impulsive structures created by the speckle and the edges of the cube are high-frequency components. Any linear filter designed to suppress the impulses would necessarily blur the edges in the image as well.

Another reason for their failure was mentioned earlier: linear filters are optimal only when the noise is additive and Gaussian, while speckle noise is far from Gaussian, is not additive, and is not even independent of the signal. So what should be done to solve this problem?

The answer is to use a filter that is not linear. There are, however, many classes of nonlinear filters, and the task of choosing the right class is itself a challenge. Each class of filters is good in certain applications. The user could consult a look-up table to determine which filter or class of filters best fits the problem at hand.

One filter that would certainly appear in any such catalogue would be the median filter--the filter suggested by Gallagher for the image in Figure 1.1. The median filter, or "running median" as it was mentioned in the first publication in which it appeared [129] and was elaborated on in [130], consists of a window, usually of odd width, which is stepped one sample at a time along a signal. At each position of the window, the sample values inside are ranked according to their magnitude and the middle element in this ranking is defined to be the output.

Typically, the window is assumed to have width 2N + 1 where N is any positive integer. Suppose that the window is centered on the kth sample in the input sequence and that the 2N + 1 points in the window, in time order, are specified by the vector

(Xk -N , X k - N + ~, " ' , Xk, . . . , Xk+N).

We want to find Yk, where

Yk = m e d i a n ( x k - N, Xk " N + 1 . . . . . Xk + N) (1.1)

which is the output of the median filter when the window is centered on the kth sample of the signal.

First, the samples in the window are reordered according to their rank,


5 4 c ~ . : 3 2 1 0

5 - �9 [ I _ - s : 4 o e I l

3 - - --1 -. = o Input 2 -~ - - - i~ - - • l-t_ g

0 I �9

Filter MOIilXl

�9 ,~ : O~tput

o 4 : . " - -

Figure 1.2. The operation of the window width 3 median filter, o: appended point.

with x(i) denoting the sample of ith rank. The samples in the window, in rank order, would then be

(X(1), X ( 2 ) , ' ' ' , X ( 2 N + I ) ) '

Suppose, for instance, that N = 2 and that the samples, in time order, in the window are

(Xk_2, Xk_l, Xk, Xk+l, X k + 2 ) = (8 , 1, 6, 4, 1);

in rank order they would be

(x(1), x(21, x(3), x(43, x(s)) = (1, 1, 4, 6, 8).

The median value of 2N + 1 samples is given by X(N+ 1), which for the example just given would be x(3 ) = 4. We thus have Yk = 4. The window is then moved so it is centered over the (k + 1)st sample in the input sequence and the output value Yk+, is computed by following the above procedure.

Figure 1.2 shows the input and output for a window width 3 median filter, and also shows how the window moves along the signal. Note from the figure that the median filter can preserve edge structures while deleting impulsive strtictures.

Note that near the beginning or end of a finite length sequence, some output values will be undefined because some samples will be missing from the window. One technique [130] for circumventing this problem is to repeat the endpoints of the sequence a sufficient number of times to fill up the window even when it is centered on an endpoint. If the window has width 2N + 1, then the endpoints must be repeated N times. In Figure 1.2, note that each endpoint must be repeated once since the window has width 3, which means that N = 1.

We are, of course, also interested in applying the median filter to two-dimensional signals like the one in Figure 1.1. There is a great deal of freedom in extending the median to two dimensions. One of the most

AN OVERVIEW OF MEDIAN AND STACK FILTERING l 1

Figure 1.3. Image after median filtering. Upon our suggestion, Dr. Johson and his colleagues applied the median filter to the noise corrupted image in Figure 1.1. The location of the copper corner cube, which was obscured in the original image, is now apparent as the dark region in the center near the top. None of the other filtering techniques that were tried produced results as good as these.

common methods is called separable median filtering [773, [86], [90], [98]. To filter the image in Figure 1.1 with a 3 x 3 separable median filter, a window width 3 median filter is first run along each of the rows of the image to produce an intermediate image, and the median is then run along each of the columns of this intermediate image to produce the final image shown in Figure 1.3. Note that the location of the copper corner cube, which was obscured in the original image, now very clearly appears as the dark region in the center near the top of the picture.

Now, the question is: Why did the median filter perform so well in this application? For the last few decades, this question has attracted the attention of many researchers. The result of this attention is a large number of papers, dissertations, and books written on median and median-related filters.

In the following three subsections we present a literature review on this subject. Due to lack of space, some of the work in "this field might not be listed here, please refer to the recent book by Pitas and Venetsanopoulos [107] and the bibliography in [34] for additional references.


5 1 l -'o0 4 c - e : : I - ' : I

2 I *

0 I : : : : - - I ~ t e r Motion

5 i � 9 1 4 9 4 c = = = == 3 �9 ! 2

�9 o Output

Figure 1.4. A median filter root signal. The signal shown above is a root signal for a window width 5 median filter. Roots for this filter consist of alternating nonincreas- ing and nondecreasing regions separated from each other by constant-valued regions at least three points long. Notice that sharp edges are allowed in a root signal, but not impulses.

1.1. Theoretical analysis of median filters

In the 1970s Tukey [129], [130, p. 210] introduced the "running median" as a tool for smoothing discrete data. Since then its name has changed to "median filter," and it has been used in several areas of digital signal processing, including speech processing [66], [117], image enhancement [67], [111], [121], and biochemical data analysis [18].

Some of the earliest theoretical results on the behavior of the median filter concerned the existence and nature of its invariant signals, or root signals. An example of a root signal is provided in Figure 1.4, in which it is easy to see that the input and output signals are the same.

In [130] several examples are given which demonstrate that successive passes of the median filter of window width 3 will eventually drive a signal of finite length to a signal which is invariant to further passes of the filter. The locally monotone nature of these finite-length fixed points was observed in [17], [130], [134] and then proven for all median filters of odd window width by Tyan [131], [132] and Gallagher and Wise [54]. If the signals being considered are infinite in length, then not all root signals are locally monotone; those that are not must be bivalued [131], [132].

Gallagher and Wise [54] also showed that any finite-length signal is filtered to a root signal after a finite number of passes of a median filter of a fixed window width. This is an important result--it ties median-type filters to neural nets since such convergence behavior is the primary characteristic of associative memories. The root signals represent eigenfunctions for the filter and are therefore fundamental to the properties of the filtering operation.

Results on the number of binary and multilevel root signals for median filters for various window widths were provided in [9], [45], [138], and [-144]. The tightest known bound on the number of passes of the median


necessary to reach a root can be found in [138]: if the filter window width is 2N + 1 and the signal has length L, then at most

i2{ 72)i passes of the filter are required to reach a root signal. The effect of a number of passes of a median filter, and the choice of the number of passes to achieve a desired effect, has been addressed in [11]. The behavior of roots and convergence properties of the two-dimensional separable median filter were characterized in [98].

Studies of the root signals and convergence behavior of the median filter form a significant part of what is now called the study of the structural behavior of median filters and median-type filters [34], ]-49]. The goal is to determine the type and number of root signals, whether every signal is filtered to a root signal, and which structures 3 are preserved, created, modified, or deleted by median-type filters.

The early studies of the structural behavior of the median filter comple- mented results known for quite some time in the statistics literature regarding the (nonrunning) median and other order statistics (see, for instance, [63, [36], [64], and [122]):

1. The median is the least absolute error estimate of the center of a distribution [67]. This says that the median of lid samples, x 1, x2, . . . , x,, of a random variable X achieves the minimum in

min ~ ] x - a l . a i=1

2. The median of lid samples of a double exponential random variable is the maximum likelihood estimator of the mean of that random variable [67].

3. The conditional median at each time instant t is the minimum mean absolute error estimator of the signal value at time t, where the conditioning is on the past history up to time t of the noise-corrupted observations of the signal (see, for instance, [126]).

4. As shown in [96], the running median is, with high probability, a maximum a posteriori estimator of a constant signal in symmetric impulse noise.

Some properties of the second moment and covariance of sequences filtered by the running median were derived by Justusson [67] and Kuhl- mann and Wise [72]. To be specific, they derived the bivariate distribution function for median-filtered sequences of independent, second-order random

3 A structure is any local, or global, variation of interest in the magnitude of a signal or image [34].


variables. Ataman et al. [16] showed that, under certain conditions, median filters can remove impulsive plus Gaussian white noise better than Harming filters. In [99] the output distribution of the one-dimensional median filter was derived for several cases including the kth-order output distribution with any input distribution. Liao et al. [77] derived the output distribution for the separable and the square window two-dimensional median filter for several cases, including first order and multivariate output distributions with white noise, and general multivariate input images.

Equipped with the above theoretical results, the success of median filters was becoming well understood and it was becoming clear which applications were well suited for median filtering. For instance, its ability to preserve edges and delete impulsive structures combined with its robustness with respect to different noise types is why it was able to recover the copper corner cube in the experiment discussed earlier.

All of the above praise might lead us to believe that the median is the perfect filtering operation, which is not true. The edges of the cube in Figure 1.3 are very sharp but they suffer from edge jitter; the final image is not a clean square. Edge jitter in median filtering has been studied in [24], and [140]. When images are filtered with a median filter, the phenomenon known as streaking also arises [22].

These shortcomings, and the desire to find other filters similar to the median but which allow more design choices than just the window width, has motivated a great deal of work on generalizations of the median filter.

1.2. Generalization o f median-type filters : theoretical analysis

Soon other filters related to median filters were exploited and analyzed. Two papers by Nodes and Gallagher [97], [99] extended the median filter to nth rank-order filters. These filters operate exactly like the median except that they put out x(,), the nth largest sample in the window, instead of the sample of middle rank. The simplest examples of rank-order filters other than the median are the filters which put out x(1), the minimum, and X(2N+I) , the maximum, values in the window. Rank-order filters all possess the convergence property of the median filter, but as shown in [97] a nonmedian rank-order filter reduces any sequence to a constant value after repeated filtering.

The recursive median filter was introduced by Nodes and Gallagher in [97]. Letting the input sequence be denoted by {Xk} and the output sequence by {Yk}, the recursive median of window width 2N + 1 is defined by replacing some of the input samples in (1.1) by previously derived output samples as follows:

Yk = median(yk-N, Y(k-N)+ l . . . . . Y k - 1 , Xk, Xk+ 1 . . . . . Xk+N). (1.2)


One very interesting result concerning this filter [97] is that just one pass of the filter produces a sequence which is invariant to any subsequent passes; in other words, a root signal is reached after only one pass of the recursive median filter. A recursive median filter has exactly the same set of root signals as the (nonrecursive) median filter of the same window width, but a given input signal might be filtered to different roots by the two filters.

In 1984 an important theoretical tool for analyzing median and median- type filters was developed by Fitch et al. [43]. They showed that all rank-order-based filters possess a limited superposition property called the threshold decomposi t ion [43], [44], which says that median filtering any sequence whose elements take on values in the set Q = {0, 1 , . . . , M - 1} is equivalent to decomposing the signal into binary sequences by thresholding at each level from 1 through M - 1, filtering each resulting binary sequence by a (binary) median filter, and then adding up the results.

To express this property more precisely, define T~[.] to be the operator which thresholds its argument at the level i:

{~ if x < i , (1.3) T~(x) = if x > i.

For instance, T3(5) = 1 and T3(2 ) = 0. We also apply thresholding to vectors, and define it as follows, even though it means a slight abuse of notation:

T/[(x1, x2 . . . . , x,)] = [T/(x 0, T/(x2) . . . . . T/(x,)]. (1.4)

For example, we would have T3((6, 2, 1, 3, 7)) = (1, 0, 0, 1, 1). Then, suppose a median filter of window width 3 whose input sequence

takes on values in Q. The threshold decomposition states that, for every k,

M e d i a n ( x k - 1, Xk, Xk + 1) -~- Median Ti((Xk- 1, Xk, Xk + 1 \ i = 1

M - 1

= ~, Median(Ti[(Xk- 1, Xk, Xk + 1)]). (1.5) i=1

This weak superposition property, which is illustrated in the example provided in Figure 1.5, allows the analysis of the effects of the median on multivalued sequences to be reduced to the study of its effects on binary

�9 sequences. This weak superposition property of median filters is so powerful that

several researchers use it as part of the definition of median-type filters. Arce [7] used the threshold decomposition to obtain closed-form expressions for the statistics of recursive median filters. The deterministic properties of the recursive separable median filter were studied by McLoughlin and Arce [86]. Using threshold decomposition, they derived the root structure of the recursive separable median filter. In recognition of the importance of the


1 1 0 2 3 3 1 2 2 - - ' - ~ - " ~ 1 1 1 2 3 3 2 2 2

Threshold at 1,2. and 3 Add binary outputs

t 0 0 0 0 1 1 0 0 0 ----~B~,~s.n~'~-~ 0 0 0 0 1 1 0 0 0

0 0 0 1 1 1 0 1 1 ---~s~-y~.~,s.~--> 0 0 0 1 1 1 1 1 1

1 1 0 1 1 1 1 1 1 ---~Bm-~.~e,tn~r}---> l l l l l l l l l

Figure 1.5. Median filtering by threshold decomposition. The 4-valued input at the upper left is median filtered the old way--by ranking and sorting the samples in the window--by following the heavy arrows. It is median filtered the new way--by threshold decomposition--by following the slender arrows. The binary signal on the lower left is obtained by thresholding the input signal at level 1; the second from the bottom by thresholding at level 2; etc. The output signal is produced by adding together the binary outputs of the binary median filters.

threshold decomposition, it was made one of the defining properties of stack filters [137], which will be discussed in much greater detail later.

Several other filters closely related to median filters have been introduced. These include the Separable Median Filter (SMF) mentioned above. Narendra [90] investigated the performance of the SMF in image noise smoothing. He showed that the output of the SMF is different from the nonseparable median filter. The performance of the SMF and the nonseparable median filter in image noise smoothing is about the same, at least for small window widths, but the SMF has a simpler implementation. This, however, depends on the hardware used.

The next generalization of median filters used linear combinations of order statistics. Bovik et al. [-23] combined the properties of averaging and median filters in a class of nonlinear filters, called either order-statistic filters or L-filters, whose output is a linear combination of the order statistics of the input sequence. If the n samples in the window, in rank order, are (x m, x(2 ~ . . . . . x(,)), then the output is computed as follows:

~ alx(1), (1.6) i=1

where the coefficients, ai, i = 1, 2 . . . . . n are any real numbers. The goal in [-23] was to pick the ai's to minimize the mean squared error (MSE) between the output of the filter and some desired signal, given noise-corrupted observations of that signal. The key problem was the computation of the correlation matrix of the order statistics.

It was shown in [-23] that the optimal (under the MSE criterion) order-statistic filter tends toward the median filter as the noise becomes more impulsive. A theory of order-statistic filters and their relationship to linear FIR filters was later investigated by Longbotham and Bovik [83].

A N OVERVIEW OF MEDIAN AND STACK FILTERING 17

Some recent work on order-statistic filters and their generalizations, the C-filters [55], [69], and Lf-filters [103], has concerned the development of optimal adaptive filtering algorithms. The result is an algorithm to train an order statistic, LE-filter, or C-filter to minimize the MSE criterion. This problem reduces to training the linear section of the filter, which then allows the well-developed theory of adaptive linear filtering I-3], [60], [62] to be brought to bear. These adaptive filters should prove useful in signal and image processing.

A class of filters which are an interesting subclass of order-statistic filters are the alpha- tr immed mean filters [19]. These filters average a subset of the samples in the filter window: the points excluded are those of very high or very low rank. For instance, in a window of size 7, the largest and smallest samples might be dropped, and then the remaining five samples are summed and divided by 5. A strict definition of the alpha-trimmed mean is as follows: select a value between 0 and 0.5 for a parameter e and then compute

1 _]) , - r ~ ]

n - 2r~n ~, x , ) , (1.7) i = Fern-1 + 1

where I x ] is the smallest integer which is larger than x. Bednar and Watt [19] explained the relationship between alpha-trimmed

mean filters and median filters and provided a new explanation of the convergence of repeated median filtering of an arbitrary sequence to a root sequence. Peterson et al. [106] later published a follow-up in which they investigated some statistical properties of alpha-trimmed mean and standard type-M filters.

The above combinations of linear and nonlinear filtering operations seem to work well in certain applications. Several papers were published in this area, we cite only two for lack of space. Lee and Kassam [74] described a nonlinear filter called the Modified Trimmed Mean filter (MTM). They showed that M filters can offer a more favorable combination of the running mean and median filters than can L-filters (order-statistic filters). They also showed that an MTM filter is a data-dependent modification of L-filters.

The next papers to introduce new classes of filters based on the median filter were by Nieminen et al. [93] and Heinonen and Neuvo [61]. The goal of these papers was to improve the detail preservation capability of median filters for both signal and image processing. These new filter classes were called FIR-median hybrid filters and multilevel median filters (later called multistage median filters in I-8]).

In the first level of operation of these filters, a median filter (or an FIR filter) of appropriate window width is applied in all directions along which lines, edges, or other details in the image are to be preserved. The result can be considered to be several intermediate images, each one produced by a median oriented in a particular direction. The second level of operation is


then a pointwise maximum, median, or minimum over all of the intermediate images.

To provide a simple example, suppose the window is a 5 x 5 square window consisting of pixels xi,j, where the indexing is relative to the center of the window. The pixel xo, o is in the center of the window. One possible version of the multistage median is called the max/median, and is given by

/ median(x~176176176176176 i,nedia (x2, 2,x 2

max|median(x2,o,Xl,o,Xo,o,X-t,o,X_2,o) | (1.8) ~median(x2,2, X l,o, Xo, o, x _ 1,o, x_ 2, o) ! I \Xo,o /

which has medians oriented vertically, horizontally, and diagonally at _+ 45 ~ It can thus preserve positive image features, such as lines oriented along the directions of any one of the medians [10]. The dual filter is the rain/median which can preserve negative image features. More sophisticated versions of the filters can preserve many different types of details in images and are effective at combating noise. For other properties of these filters and comparisons with other edge-preserving operations, please see [8], [61], and [93].

These filters were later extended to include a number of different types of linear subfilters to produce Predictor Median Hybrid (PMH) filters [-95] and Linear Median Hybrid (LMH) filters [14]. These filters effectively utilize the design freedom provided by the linear subfilters and were effectively used as edge detectors in [92].

All of the generalizations of the median filter introduced above considered discrete-time signals. Fitch et al. [-46] extended the definition of the median filter to allow continuous-time input signals; the resulting filter was called the analog median filter. This filter would be preferred to the standard discrete median in cases with a large sample size or when the associated statistics are simpler in the continuum. Longbotham and Bovik [82] then explored the relationship between analog and digital order-statistic filters.

Another set of generalizations of the median filter allowed its window width to change in an adaptive fashion. Lin and Willson [78] and Pomalaza- Raez and McGillem [108] proposed algorithms employing adaptive-length median filters to improve impulse-noise rejection. These algorithms can achieve significantly better image quality than fixed-length median filters. Another advantage of these filters is that they are easily integrated into efficient hardware realizations for median filters.

Some other examples of adaptive rank-order-based filters include: gradient median filters [15]; the adaptive minimum MSE point estimators considered in [70]; signal adaptive median (SAM) filters [20]; edge gradient enhancing order-statistic filters [73]; and adaptive order statistic and trimmed mean filters [37], [120].


The second paper to explore the adaptive median approach was by Nieminen et al. [94]. In this paper, the authors presented median-type filters with adaptive substructures suitable for filtering signals with rapidly varying characteristics. The adaptive filter substructures were used to estimate the current signal value from future signal values and from past input or output signal values.

Getting back to the threshold decomposition of the median filter, Fitch et al. [44] showed that this property was a powerful tool that can be used to analyze the whole class of rank-order filters, not just median filters as in [43].

Arce and McLoughlin [10] also used this tool to analyze the behavior of their max/median filters. They showed that max/median filters were superior to conventional median filters since the roots of the max/median filters consist of very low-resolution features which are suppressed by the latter filters. Arce and Foster [8] later generalized this work to multistage median filters, which they showed to be a combination of max/median and min/median filters. These filters also obey the threshold decomposition, which makes their theoretical analysis much simpler.

A class of nonlinear rank algorithms was introduced by Kim and Yaroslavskii [71]. They were based on the computation of local rank statistics of pictures.

Another important generalization of median filters is the class of rank- order smoothers and processors introduced in [25] and [26]. They are defined as the solution, obtained via dynamic programming, to a cost minimization problem, and take the form of a maximum of minima; i.e., a lattice polynomial. The output of a rank-order smoother is thus restricted to be one of the input points observed by the filter. The significance of this work is that it shows how regular sets and automata theory can be applied to this class of nonlinear operators [25], [26]. More recent work on this approach to nonlinear smoothing has concerned root signals of rank-order processors [27] and autoregressive equivalents of rank-order processors [28].

The final class of filters discussed here is the class called stack filters [137], which contain the class of rank-order smoothers introduced in [25] since stack filters are also maxima of minima, but the operators whose output is either always 0 or always M -- 1 are allowed when the input takes values in Q = {0 , 1 . . . . . M - 1} .

More fundamental than this slight difference in the class of operations that are considered is the differing motivations behind the definitions of the two filter classes. Specifically, stack filters were defined as having the threshold decomposition property and the ordering property [89] called the stacking property in [137]. The stacking property then led to a connection to positive Boolean functions [56].

In Figure 1.6 we show the threshold decomposition architecture of a


2 2 0 1 1 3 2 2 2 i " ' ~ ' ~ 2 2 1 1 1 3 2 2 2

t 1 1,2, and 3 A d d b | n a ! y o u t p u t s T h r e s h o d a t

0 0 0 0 0 I 0 0 0 ~ " ~ ] ~ 0 0 0 0 0 1 0 0 0

1 1 0 0 0 1 1 1 1 ~ " ] ~ 1 1 0 0 0 1 1 I 1

1 1 0 1 1 1 1 1 1 ~ , ~ [ ~ 1 1 1 1 1 1 1 1 1

Figure 1.6. Threshold decomposition architecture and the stacking property shown for one of the asymmetric median filters, there are two of them and both are stack filters. The input signal in the upper left is filtered by the stack filter SB by following the heavy arrows. This operation is equivalent to, by following the slender arrows, thresholding the input at levels 1, 2, and 3; each of these binary threshold signals is filtered by the positive Boolean function B; and finally the binary outputs are added together. At each time instant, these binary outputs appear as a column of l's supporting a column of O's, i.e., they obey the stacking property.

specific stack filter known as the asymmetric median filter. Mathematically, this figure states the following superposition property for the stack filter Sz(. ) based on the positive Boolean function f( .) :

S f ( x k - 1, xk, x~ + 1) -- Sz Ti((xk- 1, xk, x~ + 1))

M - 1

= ~ sf(r~((xk_l, x~, xk+0)) i = 1

M - 1

= ~ f(Ti((xk 1, Xk, Xk+ 1)))" (1.9) i=1

The difference between this filter and the median filter shown in Figure 1.5 is entirely in the Boolean function used on each threshold level. As long as this Boolean function is positive, the filter possesses the stacking property and the threshold decomposition property. The fact that these filters were maxima of minima was then proven via morphology in [84]; it is a consequence of [89], which shows that maxima and minima are the multilevel generalization of the logical and and logical or operations used in positive Boolean functions.

One of the primary advantages of stack filters is that a composition of stack filters is again a stack filter, which follows since the composition of two positive Boolean functions is another positive Boolean function. Perhaps the most important advantage of stack filters is the fact that the optimal stack filter can be found via a linear program when the goal is to minimize


the mean absolute error criterion [33], [34]. An adaptive algorithm is now available for minimum mean absolute error stack filtering [81]; it eliminates the need for complex computation of the statistics of the signal and noise processes, provided an appropriate training sequence is available.

There are three other fields of study which are very closely related to the field of median-type filters. One is the field of mathematical morphology--see [58], [84], [85], [123], [124], and [127] for the basic theory, the link to median-type filters, and some of the most recent results. Some work in morphology which is very closely related to signal-processing concepts is the work on morphological filters with multiple structuring elements in [127] and the work in [128] on the statistical and structural behavior of morphological filters. The link between signal processing and mathematical morphology is laid out in [84] and [85].

The primary difference between the fields of morphology and stack filtering is one of goals: in morphology the goal is to develop a theory for shape analysis, description, recognition, and manipulation; in median-type filtering the goal is to design filters for optimal estimation, detection, and manipulation of the structural characteristics of signals and images. In other words, noise and its effects are the central, although not the only concerns, in the field of median-type filters.

The second field which is also closely related to median-type filters is the field of cellular automata; see, for instance, [41], [112]-[116], and [139]. Researchers in this area call the rank-order operations by the name "ranking transforms." A cellular automaton is a logical operator, or a series/parallel combination of logical operators. When a positivity constraint is applied to these logical operators, they then are very similar to stack filters and increasing morphological filters. For an introduction to this area, please see [112] and [114]-[116] and references therein.

One class of filters which has grown out of the work on cellular automata is the class of E-filters [ 113]. They are a class of nonlinear digital convolution filters which operate, in their simplest form, like rank-order filters since on each threshold level in the threshold decomposition architecture they act as threshold logic gates. Parallel and cascade combinations of E-filters will yield stack filters. In [114] many interesting results are obtained by using a transfer function/frequency domain approach to the analysis of the effects of these filters. One of the most interesting results is that the response of these filters to a single sinusoid exhibits very sharp cutoff. The response to a sum of sinusoids of different frequencies is, however, not so easily determined since these filters are nonlinear. Interesting applications of these filters are also found in [114].

The final field related to median-type filters is the field of neural networks, although this connection has only begun to be exploited [32], [53], [135], [136], [141], [146], [147]. The fields are similar since they are both based on threshold logic operators--every binary rank-order operator is a thres-


hold logic gate. If the bits in the window of the nth binary rank-order operator R,(.) of window width 2N + 1 are (x l , x 2 , . . . , x 2 N + l ) , then its output is a 1 or 0 as follows:

R,~(x,, x 2 . . . . , x2N+ 1) =

2N+I

if ~. xi >-- n, i - 1

2N+ 1 if ~' x ~ < n .

i= l

(1.1o)

Stack filters can thus be implemented as parallel and cascade combinations of threshold logic gates. In fact, any neural net can be implemented by a generalized stack filter [80], and any generalized stack filter can be implemented by a neural network.

The difference between stack filters and neural networks is the way the threshold logic operations are put together to build more powerful operators. In neural networks, individual threshold logic gates are interconnected and all studies focus on the pattern of interconnects and the weights used in these interconnects. Median-type filters, and particularly stack filters, start from a more global point of view--the threshold logic gates are buried inside positive Boolean functions and are thus subject to an important restriction on their operation. Results on the convergence properties of symmetric threshold functions have been very helpful in analyzing the convergence behavior of a large subclass of stack filters [135], [-136].

As can be seen from the above discussion, researchers in many different fields have arrived at essentially the same class of filtering operations, although by different paths. Each field has much to learn from the other, and this special issue should foster better communication between researchers in morphology and median-type filtering.

1.3. Implemen ta t ion a lgor i thms

Before discussing further properties of rank-order-based filters, we review several types of algorithms to implement different filters.

The median filter, for one and two dimensions, was shown to be relatively easy to implement in software and hardware, see, for instance, [1], [2], [12], [17], [30], [35], [42], [59], [63], [76], [100], [118], [125], [133], and [142]. Huang et al. [63] presented a fast algorithm for two-dimensional median filtering based on sorting and updating the gray level histogram of the picture elements in the window. For window size m x n, the time requirement was O(n). Another algorithm for two-dimensional median filtering was developed by Ahmad and Sundararajan [1].

Several fast algorithms for real-time median filtering of signals and images were suggested. These include an algorithm developed by Ataman et al. [17]. This algorithm determines the kth bit of the median by inspecting the k


most-significant bits of the samples. They also compared the speed and hardware complexity of their algorithm with other fast methods for median filtering. The second algorithm was developed by Rao and Rao [118]. The algorithm for the analog version of the filter above was done by Li and Holmes [76].

Another algorithm to compute the median or any other rank-order value that was faster than an algorithm which computes the sample average was presented by Danielsson [35]. This algorithm computes the median by examining the bits of the arguments columnwise starting with the most- significant bits.

Parallel implementation of two-dimensional median filtering algorithms was proposed by Ahmad and Sundararajan [2].

VLSI implementations for rank-order-based filters have been undertaken by several authors, among them we cite [42], [59], and [100]. Special purpose processors were designed by Vainio and Neuvo [133]. These processors can be used for several variations of the FIR median hybrid filters, rank-order filters, and running sum averaging operations.

VLSI implementation of multilevel median filters was developed by Arce et al. [12]. A new bit-serial compare/exchange module was used as the building block of a systolic array yielding and efficient implementation with low area complexity which allows a single-chip VLSI realization.

Chen [30] introduced a bit-serial architecture for a class of nonlinear filters which contains all functions defined by stack filters. The digital realization is simple and modular in structure well suited for VLSI/LSI implementation.

Finally, it is important to mention the significant effort in the field of cellular automata to construct machines capable of implementing operations like the median-type filters discussed above. See [115] for an overview and further references on this topic.

2. Stack filters

The preceding two subsections demonstrate how large the field of median- type filtering has become and how it is so closely related with many other fields. In the remainder of this paper the theory of optimal stack filtering is reviewed and some recent theoretical advances and applications of median- type filters are mentioned.

The goal in the definition of stack filters [137] was to develop a class of strictly digital nonlinear filters which had the two properties considered at that time to be the essential features of the median filter: the threshold decomposition and the stacking properties. The hope was that the class of filters would have many of the desirable characteristics of the median filter, be large enough to include essentially every operation "of interest," small


enough to allow development of an e~cient optimization procedure, and simple enough to allow easy implementation. The extent to which these goals have been achieved, and the contribution of the work on stack filters to the now very large area concerned with filtering operations of this type can be judged from what follows.

Of the many classes of median-type filters introduced above, we now focus on just one, the class of stack filters. We give their explicit definition and some basic properties and then review recent results on optimal stack filtering.

2.1. The definition of stack filters

A stack filter is a sliding window nonlinear filter whose output at each window position is the result of a superposition of the outputs of a stack of positive Boolean functions operating on thresholded versions of the samples appearing in the filter's window.

Stack filters share two of the fundamental properties of median filters, which are the weak superposition property known as the threshold decomposition [43], [44] and the ordering property [893 called the stacking property in [43], [44], and [1373.

The threshold decomposition is a superposition property that was defined for the median filter in (1.5) and illustrated in Figure 1.5. It was shown for a particular stack filter in Figure 1.6. This property states that stack filtering a multilevel signal is the same as first decomposing it into a set of binary signals by thresholding, then filtering each binary signal with a binary stack filter, and, finally, adding the results of these operations.

This is a weak superposition property since it works only if the input is decomposed by thresholding. Even though it is a weak superposition property, the fact that stack filters obey any superposition property is significant, since, as can be seen with linear filters, such properties are essential for simplifying the study of filtering operations.

The second property of stack filters is called the stacking property, which is an ordering property. It states that the results of applying the binary stack filter to each of the binary signals obtained by thresholding the original signal will have a specific structure to them. As will be seen later, this structure is what makes the theory of minimum mean absolute error stack filtering possible.

Because these two properties are crucial to the optimization algorithms for this class of filters, the class was named stack filters.

Since these properties form the foundation for all that follows, they are defined below.

2.1.1. The threshold decomposition of signals. Formally, the threshold decomposition is defined as follows:


Definition 2.1. Let S t be any arbitrary time continuous bounded signal of finite length. Let SN = ($1, $2 . . . . . SN) denote the sampled and quantized, to M levels, version of St. Then the threshold decomposition of SN (M-valued sequence) is the set of (M -- 1) binary sequences, called threshold signals, t t , t 2 . . . . . t M_,, whose elements are defined by

10 if Sk > i, ti(k)= if Sk < i (2.1)

for i = 1, 2 . . . . , M - 1 .

The above decomposition of a signal into threshold signals can be seen on the left-hand side of Figures 1.5 and 1.6.

2.1.2. The stackin9 property for signals and Boolean functions. The stacking property is best described as follows: if the binary output signals of each of the Boolean operators on the threshold levels in Figures 1.5 and 1.6 are piled on top of each other according to their threshold level, the result is always a column of l's supporting a column of O's (see the right-hand side of Figures 1.5 and 1.6). This relatively simple property has some very important consequences and interpretations, which will be given below after it is more precisely defined.

We must first begin with a definition of the stacking property for sequences.

Definition 2.2. An ordered set of L binary sequences Sl, s2 . . . . . s L, in which all the sequences are of the same length, is said to obey the stacking property if

S 1 ~ S 2 ~ . . . ~ S L.

The stacking property for Boolean functions is stated in terms of the stacking property for binary sequences.

Definition 2.3. A window width n binary operator f ( . ) is said to have the stacking property if

f(x) > f(y) whenever x > y,

where x and y are two binary sequences of length n.

The following theorem states a necessary and sufficient condition for a Boolean function to satisfy the stacking property.


Theorem 2.1 [56]. An n-input binary fi,.nction f ( ' ) will satisfy the stackin9 property

f (x ) >_ f (y ) whenever x >_ y (2.2)

if and only if f ( ' ) is a positive Boolean function. Such a Boolean function has a unique minimum sum of products representation which has no complements.

A complete proof of the above theorem is found in [87]. Figure 1.6 illustrates how the threshold decomposition and the stacking property are used to filter a 4-valued signal by an asymmetric median filter (which is also a stack filter).

The stacking property has several consequences and interpretations:

1. It makes possible the direct VLSI implementation of the threshold decomposition architecture for all rank-order operators [59]. It allows the summing of the output of the Boolean functions to be performed by a space-efficient binary search for the threshold level of the highest 1 in the stack of l 's at the outputs of the Boolean functions. This architecture has since been improved upon; see, for instance, [30].

2. The stacking property can be interpreted as a consistency condition in the context of estimation [33], [79], [81]. Suppose that the goal of the filter on level k of the threshold decomposition architecture is to make the best decision, 0 or 1, at time t as to whether the signal it is estimating is less than k or not. Because of the stacking property, the level-crossing decisions made by the filters on different levels are constrained to be consistent with each other. The filter on level 3 will not decide the signal is less than 3, by putting out a 0, while the filter on level 5 decides the signal is greater than or equal to 5 by putting out a 1.

3. Any operation from Q " ~ Q (Q is the set of input values) has a representation in a threshold decomposition architecture with the operators on all the threshold levels satisfying the stacking property relative to each other [80].

The stacking property for a binary function f(-) is called the increasing property in morphology [58], [84], [123], [124]. What is of interest when comparing the two approaches to nonlinear filtering, is that the stacking property is not required of digital morphological operators [58], [84], [123], [124] when they are implemented in the threshold decomposition architecture; even though each such filter has a representation in this architecture in which this property is obeyed [79]. Instead, morphologists would begin with a binary operator, say f( ' ) , and then define the following multilevel operation for integer-valued input:

Y(m) = max{l: f(wl[m]) = 1)}, (2.3) 1

A N OVERVIEW OF MEDIAN AND STACK FILTERING 27

where Y(m) is the output of the digital morphological operator at point m and [m] is the window vector whose middle index is m.

Using the above two properties of stack filters, we can get a mathematical expression for the output of a stack filter operating on an input process. Let R(j) be the process at the input of a stack filter. Assume R(j) takes on values in Q = {0, 1, 2 . . . . . M - 1}. A window of width n slides, by increments of one sample, across the input process RU). At each time instant j, the stack filter S:(') maps the samples in the window, which are

to some integer Sf(R,(j)) in Q. The mapping S:(-): Q" ~ Q defining the stack filter is required to have the threshold decomposition structure

S f(R,,(j)) =

where, as in (1.4), we define

M - 1

f(Tk(R,,(j)) ), (2.5) k = l

(2.6)

in which

{~ if R(j)>_m, (2.7) T,,(R(j)) -- else,

and f(-) is the Boolean function used on each level in the architecture. The Boolean function f ( ' ) is required to possess the stacking property [137].

Although (2.3) and (2.5) are equivalent when the logical operator f( .) is a positive Boolean function, the difference in emphasis on the stacking property is one of the primary differences between these two approaches to image processing.

Any digital filter that can be realized by (2.5) with f( .) being a positive Boolean function possesses the threshold decomposition property [137]. This weak superposition property is made precise in the second equality in the following equation:

S:(R,(j)) = S: Tk(R,(j = ~ Sf(Tk(R,(j)))" ~ f(Tk(R,(j))). (2.8) k 1 k = l k = l

The next section is devoted to the optimality theory that was built around these filters. This theory i s an extension of the theory in [31] for rank-order filters, which may itself be regarded as another interpretation of the results in [21].

28 GABBOUJ, COYLE, AND (JALLAGHER

s c t ~ n(t) .

N ( t ) S[(') = S(') = Best, Est imate of S(t)

Figure 2.1. Structure of the optimal stack filtering problem. The received process R(t) is the result of the noise process N(t) acting on the input process S(t) through a memoryless function g(', .). The output of the stack filter S~(') is the best estimate of S(t) based on n samples of R(t), R,(t).

2.2. Opt imal f i l tering over the class o f s tack f ihers

The optimal filtering problem over the class of stack filters can be stated as shown in Figure 2.1. The process R(j) at the input of a stack filter is assumed to be a corrupted version of some desired process S(j). The corruption may be caused either by a noise process N(j) or by some intentional operation, such as a modulation scheme.

At each time instant j, the stack filter output is an estimate, called ~(j), of the desired process S(j). This estimate is based on the observed sequence R,(j) in the window of the stack filter; thus,

S(j) = Sy(R,(j)).

The goal is to pick a stack filter from the class of window width n stack filters such that the average mean absolute error per time unit between the filter's output and the desired signal is minimized. If S(j) and R(j) are jointly stationary, then the cost to be minimized is

E[I S(j) - Sf(R,(j))[]. (2.9)

The rationale of using the absolute error criterion is that it nicely reduces the estimation error of the stack filter to the sum of the decision errors incurred by the Boolean filters on each level of the threshold decomposition architecture. To illustrate this fact, let sk(j) = Tk(S(j)); then

[ M-I f(Tk(R"(J))) 1 E[IS( j ) - Sr ] = E ~ Sk(j) -- k=l

by threshold decomposition,

by the stacking property,

1 ] = E [sk(j) -- f(Tk(R,(j)))l L k = l

M - 1

2 k=l

E[Isk(j) -- f(Tk(R,(j)))[]. (2.10)

AN OVERVIEW OV MEDIAN AND STACK FILTERING 29

Because of the above reduction, and since a stack filter is completely defined by the Boolean function on each level of its architecture, the cost function to be minimized can be expressed in terms of a linear function of the output variables of the Boolean function f ( . ) [33], [79], [80].

Let the output o f f ( ' ) , when the length n binary sequence s~ is at its input, be called the decision variable P~(ltsj)E{0, 1}. This decision variable specifies the probability that the filter output is a 1 when the vector sj appears in the filter's window. The Boolean function f(-) can therefore be represented as a length 2" vector PI, whose kth entry is PI(1 ]Sk). With these definitions, the mean absolute error in the last equation in (2.10) can be reformulated as the following cost function:

2 n

Cost = ~ Cj-Pr(llsj). (2.11) ) = i

Cj can be interpreted as the cost incurred by f ( . ) for deciding a 1 when seeing sj. Note that this cost function is a linear function of the filter decision variables.

The stacking constraints can be expressed as a set of inequalities in terms of these decision variables; specifically,

Pf(llsi) <_ Pf(llsj) if S i ~ S j , (2.12)

where for any two length n real sequences s and t, s < t if and only if each entry of s is less than or equal to the corresponding entry in t.

The optimal filtering problem over the class of stack filters under the mean absolute error criterion can therefore be formulated as a zero-one integer linear program-- the goal being to determine whether each Py(1 Is j) should be 0 or 1.

By exploiting the structure of the constraint matrix, which was shown to be TUM (totally unimodular [104]) [50], this zero-one integer linear program can be formulated as the following linear program [33]:

2 n

minimize ~ Cj'PI(1 rsj) (2.13) j = l

subject to the constraints : Py(llsl) _< Pf(llsj) if si < sj, (2.14)

0 _< PI(1 [si) _< 1 for all i. (2.15)

The linear programming formulation of the optimization problem has a very nice interpretation in terms of the behavior of the Boolean operator f(.). The constraint in (2.15) implies that the filter is allowed to randomize its decisions [33]. The quantity Pf(l [sj) is the probability the filter puts out a 1 when the binary sequence sj is observed. The filter is thus allowed to make soft decisions.

Even with this extension to soft decisions, there is always an optimal filter


which makes hard decisions. This follows from the total unimodularity of the constraint matrix consisting of (2.14) and (2.15).

It is worth mentioning here, since it affects the complexity of the algorithms developed in [47], that the stacking constraint in (2.14) has many redundant equations. This redundancy can be reduced by considering the set of all local stacking constraints [33]; that is, a decision variable P1(llsj) is subject to a stacking constraint by another decision/variable Ps(1 Isi) only if sj < (_>)sl, and dH(s i, s j)---= 1, where dn(sl, s j) is the Hamming distance between the binary sequences s i and sj. This reduces the set of inequalities in (2.14) to a much smaller set of inequality constraints [33]. The reduction of the number of constraints was discussed in more detail in [50].

The theory of estimation under the mean absolute error criterion presented in the preceding section was extended in [80] to the class of filters known as Generalized Stack Filters (GSF). In GSFs each Boolean function can be fed threshold signals from more than one threshold level, and different Boolean functions can be used on different threshold levels. The stacking property is then a property between Boolean operators on adjacent levels in the threshold decomposition architecture, not a property of a single Boolean operator. In fact, as some examples in [80] show, the Boolean operators used in a GSF need not be positive for their outputs to stack.

An algorithm is provided in [80] to determine a GSF which minimizes the MAE between the output of the filter and a desired input signal. The algorithm is a linear program whose complexity depends on the window width of the filter and the number of threshold levels observed by each of the filters in the threshold architecture.

2.3. Application: An optimal stack filter example [81]

We now present an example to show how the adaptive stack filter algorithm developed in [81] can be used in an application in image processing.

Figure 2.2 shows the original "bridge" image. Figure 2.3 shows the image in Figure 2.2 corrupted by additive impulsive noise. Both images are 512 x 512 and are quantized into 256 gray levels. The image restoration problem is to find a filter which can do the "best" job of recovering the original from its corrupted version.

A quarter of the original image and the corrupted image were used to train a stack filter with 4 x 4 window using the adaptation algorithm. Figure 2.4 shows the image at the output of the stack filter with the corrupted image at its input. It is clear that the stack filter does a good job of removing the noise while preserving the details of the original image. A comparison between the performance of this optimal stack filter with several other filters can be found in [34].

Another application of GSFs to suppress shot noise and demodulate an AM radio signal can be found in [80].

(e)

Figure 3.1. (a) The original color image with impulsive noise added to each component. (b) The output of the median filter of window width 7 when applied separately to each component. (c) The output of the vector median of window width 7. (d) Cross color and cross luminance in a digitized test image. (e) The output of the vector median filter.

facing page 30

(c)

(d)

Figu

re 3

.2. (

a) T

he o

rigi

nal

imag

e: p

orti

on o

f th

e fif

th f

ram

e of

"C

ost

Gir

ls"

sequ

ence

. (b

) O

utpu

l of

the

mul

tile

vel

med

ian

[5].

(c)

Oal

put

of th

e w

eigh

ted

med

ian

filte

r [5

]. (

d) O

utpu

t of

the

prev

ious

pix

el r

epet

itio

n al

gori

thm

.

Figure 2.2. The original image.

Figure 2.3. The original image after corruption by impulsive noise.


Figure 2.4. The output image of the optimal 4 x 4 adaptive stack filter.

3. Recent developments in stack and median-based filtering

Increasing efforts have been devoted to the theory, implementation, and application of stack and median-based filtering. In the following sections we review some of these recent developments.

3.1. Theoretical advances

The class of weighted median filters is an immediate generalization of the standard median filter where the output is taken to be the median of some weighted input samples [67]. It was shown in [143] that a positive Boolean function and hence a stack filter can be represented by a weighted median filter if and only if it is linearly separable and self-dual.

Yli-Harja et al. [143] analyzed the properties of weighted median filters using threshold logic and stack-filter representations. In particular, they showed that a cascade of weighted median filters can always be expressed as a stack filter; and that if the composite positive Boolean function corresponding to the cascade is linearly separable, then the cascade can be


realized as a single weighted median filter with increased window size. This allows a much simpler computation of the output distribution of stack filters and enables the proof of unbiasedness of self-dual stack filters.

Two particular subclasses, called Class 1 and Class 2, of n-dimensional weighted median filters were developed and studied in details in [51]. The classification is based on the shapes and the weights of the filter windows. Both Class 1 and Class 2 filters are shown to suppress impulsive noise and yet have almost no effect on the noncorrupted parts of the signal. This makes these filters well suited for preprocessing purposes.

The most important feature of these filters is perhaps the fact that Class 1 recursive filters and Class 2 nonrecursive filters are idempotent filters [51], an important concept in filtering which, unfortunately, is not discussed further here due to space limitation.

Other advances in this area brought about the development of Vector Median (VM) filters 4 [13]. In many current applications, signals consist of several separate components, each carrying information on different properties of the signal. Multispectral satellite images and standard color images in TV systems are typical examples. These signals can be processed componentwise; however, this method discards all correlation between different components of the vector signal and therefore is not effective. In the next section we show how VM filters are effective in combating such defects. Two applications of VM filters to process color images are presented in the next section.

More formally, VM is defined as follows [13]:

Definition 3.1. The vector median of the N-dimensional vectors sl, s2 , . . . , s, is Svm such that Svm~ {si: i = 1, 2 . . . . , n} and, for a l l j = 1, 2 . . . . . n,

i=1 i=1

where II JPp is the Lp norm (p == 1 o r 2).

The VM is defined in such a way that its output is always one vector from the input sample vectors. This facilitates the study of the root structure of VM filters.

The structure of its root signals, and the convergence behavior of the median filter proved to be powerful analytical tools in analyzing the behavior of the filter [54]. This theory has since been extended to stack filters [49], where a graph-theoretic approach was used to prove necessary and sufficient conditions for a stack filter to have roots and an algorithm was devised to determine the roots of convergent stack filters.

4 This is not to be confused with the vector median filters VMFN• M of Kasparis and Eichmann [68].


This analysis allowed us to study the structural behavior of stack filters. This in itself is a good step toward understanding how stack filters work. However, it would be even more interesting to see how this theory of root signals for stack filters could be combined with the theory of minimum mean absolute error stack filtering developed previously. This new, unified theory was developed in [47]-[49]. It allows the designer to pick a stack filter which minimizes noise subject to constraints on its structural behavior. The extensions-of the results in [49] to two-dimensional signals can be found in [34].

The convergence behavior of stack filters has been extensively studied in [52], [105], [135], [136], and [145]. In [52] stack filters were partitioned into four different types, namely type-0 through type-3, based on the structure of the on-set and the off-set of the positive Boolean function defining the stack filter. It was shown that stack filters of type-0 through type-2 possess the convergence behavior, while type-3 stack filters do not all share this property. Examples of cyclic behavior in type-3 stack filters were given. Type-3 stack filters were then decomposed into a type-I and a type-2 stack filter. Furthermore, it was shown that the root signal set of any type-3 stack filter is the intersection of the root sets of the type-1 and the type-2 stack filters from which the type-3 filter is constructed. The rates of convergence for convergent stack filters were also determined for several appending strategies commonly used in the literature.

Patel et al. described the fixed-point structure (i.e., root structure) for three and four variable positive Boolean functions and described an algorithm to generate root structures of larger positive Boolean functions [105].

Wendt [135], [136] studied the convergence behavior of those nonrecursive and recursive one-dimensional stack filters and multidimensional stack filters which can be represented by symmetric threshold functions for binary inputs. Based on similar theorems on the convergence of neural networks, he showed that any n-dimensional signal, when surrounded by an arbitrary fixed boundary, will be filtered to either a root or will oscillate between two signals when filtered by any n-dimensional stack filter in this class in a finite number of passes. When the filter is used recursively, any signal is reduced to a root in a finite number of passes no matter how the filter moves over the signal.

Order-statistic filters is another expanding area of research. Lee and Tantaratana [75] proposed adaptive order-statistic filters whose outputs are based on decisions of hypothesis tests. Their Modified Median Filter (MMF) was designed to reduce edge shifting associated with median filters. It generates the output by modifying the median output based on a comparison of the median value to the current input value whenever an edge is detected by applying a hypothesis test. The main advantage of this filter is that it reduces the output mean square error caused mainly by edge shifting. The M M F was then combined with an L-type structure in order to suppress nonimpulsive and impulsive noise.


Finally, Prasad and Lee proposed an alternative study of stack filters based on selection probabilities, namely rank and sample selection probabilities [109], [110]. They showed that selection probabilities can be used as parameters to describe a stack filter and devised a method to compute these probabilities from the Boolean function defining a stack filter.

K2. New applications

3.2.1. Impulsive no&e attenuation in color images [102]. RGB or YUV signals, with their three spectral components, are the most common multispectral signals. Such signals can be thought of as vector-valued signals. The standard approach of processing these signals is to design a filter for scalar signals and apply it to each component separately. However, the signal components are, in this case, correlated and componentwise processing is not able to make use of such information.

Figure 3.1(a) shows the original image where impulsive noise has been added to each component. This image is then filtered by a median filter of window width 7. The result is shown in Figure 3.1(b). This is an example of distortion caused by componentwise processing. The median filter removes those impulses on fiat signal areas; however, it causes an edge to shift to the left when an impulse is present in front of the edge. This causes the output color to be different from that of the input.

As mentioned earlier, VM is a potential candiate to filter such signals since it uses the correlation between different components of the vector- valued signal.

Figure 3.1(c) is the output of the window width 7 VM filter (the input signal is shown in Figure 3. l(a)). Note that edge jittering causes no additional colors in the filtered image since the output of the filter is always one of the input signals.

3.2.2 Cross luminance and cross color elimination. Artifacts caused by cross color are very annoying in TV receivers. These artifacts are due to imperfect separation of the luminance and the chrominance components of TV signals traditionally done by notch filters. However, notch filters do not remove all color information from the signal since their design is a compromise between luminance bandwidth reduction and color separation. These filters are required to have sufficient rejection near the subcarrier frequency in order to prevent subcarrier dot structures in large areas of saturated color. However, some of the chrominance signal still enters the luminance channel causing interference effects on the luminance, called cross color, in sharp color edges in the horizontal direction. The chrominance circuitry also accepts all luminance signal near the subcarrier frequency causing cross luminance effects in areas with fine luminance details. These artifacts are shown in Figure 3.1(d). Cross luminance is clearly seen as running dots


between color bars and cross color as color dots near the white lines at the top of the picture.

The VM filters developed in [13] are good candidates to combat these artifacts. Samples of the vector-valued input signal are processed as vectors as opposed to componentwise scalar processing. The vector median operation inherently uses the correlation between the signal components, thus giving the filter some desirable properties. Since different components in the color video signal correlate strongly, the use of VM filters is a natural choice.

The performance of VM filters for color video signals were investigated in [101]. The authors used recursive VM filters for cross luma and cross color cancellation, then combined the VM with linear filtering which resulted in improved cross luma and cross color attentuation and good noise attenuation as well. Figure 3.1(e) shows the result of applying a VM filter to the test image of Figure 3.1(d). The cross chrominance is effectively removed, while the interfering signal is completely eliminated.

3.2.3. Scan rate conversion and HDTV. In current television systems, low field rate and interlaced display standard result in visible artifacts such as large area and line flicker and line crawl [91]. One way to partially remove these anomalies is to up-convert the picture scan rate in the receiver. Two methods are typically used to perform this task: field rate doubling and interlaced to progressive conversion both of which require interpolation of new picture information. The structure of the interpolator has a direct and significant effect on the image quality produced. The simplest interpolator consists of repeating previous information resulting in less than satisfactory performance. Other interpolators based on nonlinear operations were developed, tested, and resulted in much better performance. Several of these interpolators are reviewed here.

Volume III of the Proceedings of the Third International Workshop on H D T V ( A u g u s t 30 to September 1, 1989, Torino) contains several successful applications of median and median-based filters to HDTV [29], [38], [40], [57], [119]. These papers concentrated on reducing the artifacts discussed above.

Haavisto et al. [57] presented up-conversion algorithms based on an adaptive weighted median filter structure and a simple motion detector. The filter weights were adapted according to the output of the motion detector. The adaptive nature of the filter assures good performance in both stationary and moving picture areas. The algorithm can be applied to 50-100 Hz field rate conversion as well as to a noninteger conversion by a noninteger factor of 1.5, from 50 Hz to 75 Hz. The algorithm was tested using the VTE DVSR 100 Image Sequencer and produced remarkable results. Also, with fixed weights, the results were rather good even when no motion detection was used.

Rather than using a motion detector, Doyle et al. [39] showed that a


more cost effective solution would be to use a median filter. This has already been applied to an IDTV receiver [88]. Doyle and Looymans [38] later developed a detailed directional dependent filter based on median filtering. The output of the direction detection algorithm as well as the original interlaced input data were used to interpolate the missing lines in the progressive scan output. The interpolation was done using a three-point median filter with two pixels from the present field, being derived from the direction selection algorithm, and the corresponding pixel of the previous field. This combination enhances the performance of the filter in moving and stationary areas. The major advantage was the performance complexity tradeoff. These results can be improved by using a reliable motion detector.

Capodiferro [29] evaluated the performance of the median filter in simple interlaced to progressive conversion of HDTV sequences with and without motion compensation. In comparison with linear filters, median filters were more effective in simple and accurate deinterlacing.

Dreier [40] used a combination of frame repetition and median filtering for a scan conversion technique from 50 Hz to 100 Hz field rate which resulted in a screen display with no large area flicker and considerably reduced line flicker. An adaptive scheme was also used to control the switching between median and linear interpolation depending on the horizontal picture contents.

3.2.4. Image sequence coding. Another successful application of median- based filters is in image sequence coding and reconstruction [4], [5], [65]. Alp et al. [5] used multidimensional median operations to reconstruct image sequences which were coded using field-quincunx coding with the addition of an offset from frame to frame. The three-dimensional median-based structures for offset quincunx decoding of image sequences produced rather perfect reconstruction in still pictures and good performance in slow moving scenes in the sequence compared with existing methods. Figure 3.2 shows several examples of image reconstruction, in the presence of slow motion, using median-based multidimensional filters. In particular, this example shows that a properly designed median structure performs well under motion without any kind of motion detection algorithms.

4. Conclusion

In this paper we have traced the development of a new theory of nonlinear signal processing that has been built around the median-related filters known as stack filters. It was shown that these filters are generalizations of the median filter and that they have inherited both its noise removal properties and its interesting structural behavior.

Several fields of study which are closely related to the field of median-type


filters were briefly reviewed. The pr imary differences between these related fields and the field of stack filtering were also pointed out.

One of these related fields is mathemat ical morphology. As pointed out earlier, each field has much to learn from the other, and this special issue should foster better communica t ion between researchers in mathemat ical

morphology and median- type filtering.

Acknowledgment

The authors wish to thank the personnel of the Signal Processing Labora tory at Tampere Universi ty of Technology for prepar ing some of the art work in this paper. Thanks are also due to Prof. Jean Hsang Lin of the Universi ty of Delaware for providing the opt imal stack filter example.

References

[1] M. O. Ahmad and D. Sundararajan, A fast algorithm for two-dimensional median filtering, IEEE Transactions on Circuits and Systems, vol. 34, no. 11, pp. 1364-1374, November 1987.

[2] M. O. Ahmad and D. Sundararajan, Parallel implementation of median filtering algorithm, IEEE International Symposium on Circuits' and Systems, 1988, pp. 1499-1502.

[3] S. T. Alexander, Adaptive Signal Processing: Theory and Applications, New York~ Springer-Verlag, 1986.

[4] B. Alp, T. Jarske, K. Oist/imr, and Y. Neuvo, Digital image sequence processing using the median operation, Siihk6 Electricity and Electronics, June 1990, to appear.

[5] B. Alp, J. Juhola, Y. Neuvo, and T. Jarske, Multidimensional reconstruction of quincunx coded image sequences, Proceedings of the 1990 Picture Coding Symposium, Cambridge, MA, March 26-28, 1990, pp. 4.5-1-4.5-3.

[6] D. F. Andrews et al., Robust Estimation of Location, Princeton, NJ: Princeton University Press, 1972.

[7] G. R. Arce, Statistical threshold decomposition for recursive and nonrecursive median filters, IEEE Transactions on Information Theory, vol. 32, no. 2, pp. 243-253, March 1986.

[8] G. R. Arce and R. E. Foster, Detail-preserving ranked-order-based filters for image processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, no. 1, pp. 83-98 , January 1989.

[9] G. R. Arce and N. C. Gallagher, State descriptions of the root signal set of median filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 30, no. 12, pp. 894-902, December 1982.

[t0] G. R. Arce and M. P. McLoughlin, Theoretical analysis of the max/median filter, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, no. 1, pp. 60-69, January 1987.

[11] G. R. Arce and R. L. Stevenson, On the synthesis of median filtering systems, IEEE Transactions on Circuits and Systems, vol. 34, no. 4, pp. 420429, April 198%

[12] G. R. Arce, P. J. Warter, and R. E. Foster, Theory and VLSI implementation of multilevel median filters, Proceedings of the 1988 IEEE International Symposium on Circuits and Systems, Helsinki, June 1988, pp. 2795-2798.

[13] J. Astola, P. Haavisto, and Y. Neuvo, Vector median filters, Proceedings o f the IEEE, vol. 78, no. 4, pp. 690-710, April 1990.


[14] J. Astola, P. Heinonen, and Y. Neuvo, Linear median hybrid filters, IEEE Transactions on Circuits and Systems, vol. 36, no. 11, pp. 1430-1438, November 1989.

[15] J. Astola, O. Yli-Harja, P. Heinonen, and Y. Neuvo, Gradient median filter, Proceedings o f the 1987 IEEE International Symposium on Circuits and Systems, Philadelphia, PA, May 1987, pp. 246-251.

[16] E. Ataman, V. K. Aatre, and K. M. Wong, Some statistical properties of median filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 5, pp. 1073-1075, October 1981.

[17] E. Ataman, V. K. Aatre, and K. M. Wong, A fast method for real-time median filtering, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 28, no. 4, pp. 415-421, August 1980.

[18] J. A. Bangham, Data-sieving hydrophobicity plots, Analytical Biochemistry, vol. 174, pp. 142-145, 1988.

[19] J. B. Bednar and T. L. Watt, Alpha-trimmed means and their relationship to median filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 32, no. 1, pp. 145-153, February 1984.

[20] R. Bernstein, Adaptive nonlinear filters for simultaneous removal of different kinds of noise in images, IEEE Transactions on Circuits and Systems, vol. 34, no. 11, pp. 1275-1291, November 1987.

[21] P. Bloomfield and W. L. Steiger, Least Absolute Deviations, Boston: Birkhauser, 1983. [22] A.C. Bovik, Streaking in median filtered images, IEEE Transactions on Acoustics, Speech,

and Signal Processing, vol. 35, no. 4, pp. 493-503, April 1987. [23] A. C. Bovik, T. S. Huang, and D. C. Munson, Jr., A generalization of median filtering

using linear combinations of order statistics, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 31, no. 6, pp. 1342-1350, December 1983.

[24] A. C. Bovik, T. S. Huang, and D. C. Munson, Jr., The effect of median filtering on edge estimation and detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9, no. 2, pp. 181-194, March 1987.

[25] A. R. Butz, A class of rank order smoother, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, no. 1, pp. 157-165. February 1986.

[26] A. R. Butz, Regular sets and rank order processors, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 2, pp. 241-246, February 1986.

[27] A. R. Butz, Root signals of rank order processors, Proceedings of the 1990 International Conference on Circuits and Systems, New Orleans, LA, May 1-3, 1990.

[28] A. R. Butz, Autoregressive equivalents of rank order processors, IEEE Transactions on Acoustics, Speech and Signal Proceedings, vol. 38, no. 6, pp. 1050-1052, June 1990.

[29] L. Capodiferro, Interlaced to progressive conversion by median filtering, Proceedings Third International Workshop on HDTV, Torino, August 30 to September 1, 1989, vol. III.

[30] K. Chen, Realizations of a class of nonlinear filters using a bit-serial approach, Proceed- ings o f the 1988 IEEE International Symposium on Circuits and Systems, Helsinki, June 1988, pp. 1749-1752.

[31] E. J. Coyle, Rank order operators and the mean absolute error criterion, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, no. 1, pp. 63-76, January 1988.

[32] E. J. Coyle and N. C. Gallagher, Stack filters and neural networks, Proceedings o f the 1989 International Symposium on Circuits and Systems, Portland, OR, May 8-11, 1989, pp. 975-978.

[33] E. J. Coyle and J.-H. Lin, Stack filters and the mean absolute error criterion, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 36, no. 8, pp. 1244-1254, August 1988.

[34] E. J. Coyle, J.-H. Lin, and M. Gabbouj, Optimal stack filtering and the estimation and structural approaches to image processing, IEEE Transactlons on Acoustics, Speech, and Signal Processing, vol. 37, no. 12, pp. 2037-2066, December 1989.


[35] P.-E. Danielsson, Getting the median faster, Computer Graphics and Image Processing, vol. 17, no. 1, pp. 71-78, 1981.

[36] H. A. David. Order Statistics, Wiley: New York, 1970. [37] R. Ding and A. N. Venetsanopoulos, Generalized homomorphic and adaptive order

statistic filters for the removal of impulsive and signal-dependent noise, IEEE Transac- tions on Circuits and Systems, vol. 34, no. 8, pp. 948 955, August 1987.

[38] T. Doyle and M. Looymans, Progressive scan conversion using edge information, Proceedings of the Third International Workshop on HDTV, Torino, August 30 to Septebmer 1, 1989, vol. III.

[39] T. Doyle-et al., Median filtering of television images, Proceedings of the International Conference on Consumer Electronics, 1986.

[40] H.-L Dreier, Line flicker reduction by adaptive signal processing, Proceedings of the Thirdlnternational Workshop on HDTV, Torino, August 30 to September 1, 1989, vol. III.

[41] M. J. B. Duff and T. J. Fountain, Cellular Logic Image Processing, Academic Press: London, 1986.

[42] J. P. Fitch, Software and VLSI algorithms for generalized rank order filtering, IEEE Transactions on Circuits and Systems, vol. 34, no. 5, pp. 553-559, May 1987.

[43] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., Median filtering by threshold decomposition, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 32, no. 6, pp. 1183-1188, December 1984.

[44] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., Threshold decomposition of multidimensional ranked order operations, IEEE Transactions on Circuits and Systems, vol. 32, no. 5, pp. 445-450, May 1985.

[45] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., Root properties and convergence rates of median filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, no. 2, pp. 230-240, February 1985.

[46] J. P. Fitch, E. J. Coyle, and N. C. Gallagher, Jr., The analog median filter, IEEE Transactions on Circuits and Systems, vol. 33, no. 1, pp. 94-102, January 1986.

[47] M. Gabbouj, Estimation and structural-based approach for the design of optimal stack filters, Ph.D. Thesis, Purdue University, West Lafayette, IN, December 1989.

[48] M. Gabbouj and E. J. Coyle, Design of optimal stack filters with structural constraints under the mean absolute error criterion, Proceedings of the IEEE 32nd Midwest Symposium on Circuits and Systems, Champaign, IL, August 1989, pp. 55-58.

[49] M. Gabbouj and E. J. Coyle, Minimum mean absolute error stack filtering with structural constraints and goals, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 6, pp. 955-968, June 1990.

[50] M. Gabbouj and E. J. Coyle, On the LP problem which finds the MMAE optimal stack filter, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 39, no. 11, November 1991.

[51] M. Gabbouj, P. Haavisto, and Y. Neuvo, Recent advances in median filtering, Proceedings of the 1990 Bilkent International Conference on New Trends in Communications, Control, and Signal Processing, Bilkent University, Ankara, July 2-5, 1990, vol. II, pp. 1080-1094.

[52] M. Gabbouj, P.-T. Yu, and E. J. Coyle, Convergence behavior and root signal sets of stack filters, Circuits, Systems, and Signal Processing, this issue, pp. 171-193.

[53] N. C. Gallagher and E. J. Coyle, Stack filters, threshold decomposition, and neural networks, Proceedings of the Asilomar Conference on Signals, Systems, and Computing, November 1987.

[54] N. C. Gallagher, Jr., and G. L. Wise, A theoretical analysis of the properties of median filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1136-1141, December 1981.

[55] P. P. Ghandi, S. R. Peterson, and S. A. Kassam, Frequency/selective signal restoration using non-linear combination filters, Proceedings o f the 1988 IEEE International Conference on Acoustics, Speech, and Signal Processing, New York, April 1988, pp. 1522-1525.

AN OVERVIEW OF MEDIAN AND STACK FILTERINO 41

[56] E. N. Gilbert, Lattice theoretic properties of frontal switching functions, Journal of Mathematical Physics, vol. 33, pp. 57-67, April 1954.

[57] P. Haavisto, J. Juhola, and Y. Neuvo, Scan rate up-conversion using adaptive weighted median filtering, Proceedings Third International Workshop on HDTV, Torino, August 30 to September 1, 1989, vol. III.

[58] R. M. Haralick, S. R. Sternberg, and X. Zhuang, Image analysis using mathematical morphology, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9, no. 4, pp. 532-550, July 1987.

[59] R. G. Harber, S. C. Bass, and G. W. Neudeck, VLSI implementation of a fast rank order filtering algorithm, Proceedings of the 1985 IEEE International Conference on Acoustics, Speech, and Signal Processing, Tampa, FL, March 1985.

[60] S. Haykin, Adaptive Filter Theory, Englewood Cliffs, NJ: Prentice-Hall, 1986. [61] P. Heinonen and Y. Neuvo, FIR-median hybrid filters, IEEE Transactions on Acoustics,

Speech, and Signal Processing, vol. 35, no. 6, pp. 832-838, June 1987. [62] M. L. Honig and D. G. Messerschmitt, Adaptive Filters." Structures, Algorithms and

Applications, Boston: Kluwer Academic, 1984. [63] T. S. Huang, G. J. Yang, "and G. Y. Tang, A fast two-dimensional median filtering

algorithm, IEEE Transactions on Acoustics, Speech, and Signal Processin 9, vol. 27, no. 1, pp. 13-18, February 1979.

[64] P. J. Huber, Robust Statistics, New York: Wiley, 1981. [65] T. Jarski, K. Saarinen, J. Juhola, and Y. Neuvo, Quincunx coding for picture memories,

Proceedings of the 131st SMPTE Technical Conference, Los Angeles, CA, October 21-25, 1989.

[66] N. S. Jayant, Average and median-based smoothing techniques for improving digital speech quality in the presence of transmission errors, IEEE Transactions on Communica- tions, vol. 24, no. 9, pp. 1043-1045, September 1976.

[67] B. I. Justusson, Median filtering: statistical properties, in Two-Dimensional Digital Signal Processin 9 H: Transforms and Median filters, chap. 4, pp. 161-196. Topics in Applied Physics, vol. 42, T. S. Huang, ed., New York: Springer-Verlag, 1981.

[68] T. Kasparis and G. Eichmann, Vector median filters, Signal Processing, vol. 13, pp. 287299, October 1987.

[69] S. A. Kassam and S. R. Peterson, Nonlinear finite moving window filters for signal restoration, IEEE Pacific R I M Conference on Communications, Computers and Signal Processing, Canada, June 1987.

[70] R. Kasturi, J. Walkup, and T. Krile, Adaptive point estimation in signal-dependent noise, IEEE Transactions on Systems, Man, and Cybernetics, vol. 15, no. 3, pp. 352-359, May-June 1985.

[71] V. Kim and L. Yaroslavskii, Rank algorithms for picture processing, Computer Vision, Graphics, and Image Processing, vol. 35, no. 2, pp. 234-258, 1986.

[72] F. Kuhlmann and G. L. Wise, On second moment properties of median filtered sequences of independent data, IEEE Transactions on Communications, vol. 29, no. 9, pp. 1374-1379, September 198l.

[73] Y. H. Lee and A. T. Faro, An edge-gradient enhancing adaptive order statistic filter, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, no. 5, pp. 680-695, May 1987.

[74] Y. H. Lee and S. A. Kassam, Generalized median filtering and related nonlinear filtering techniques, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, no. 3, pp. 672-683, June 1985.

[75] Y.-H. Lee and S. Tantaratana, Decision-based order statistic filters, IEEE Trans- actions on Acoustics, Speech, and Signal Processing, vol. 38, no. 3, pp. 406-420, March 1990.

[76] J. S. J. Li and W. H. Holmes, Analog implementation of median filtecs tor real-time signal processing, IEEE Transactions on Circuits and Systems, vol. 35, no. 8, pp. 1032-1033, August 1988.


[77] G.-Y. Liao, T. A. Nodes, and N. C. Gallagher, Jr., Output distributions of two- dimensional median filters, IEEE Transactions on Acoustics, Speech, and Signal Process- ing, vol. 33, no. 4, pp. 1280-1295, October 1985.

[78] H.-M. Lin and A. N. Willson, Jr., Median filters with adaptive length, IEEE Transactions on Circuits and Systems, vol. 35, no. 6, pp. 675 690, June 1988.

[79] J.-H. Lin and E. J. Coyle, Optimal nonlinear filtering under the mean absolute error criterion, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, New York, April 1988, pp. 1439-1442.

[80] J.-H. Lin and E. J. Coyie, Minimum absolute error estimation over the class of generalized stack filters, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, no. 4, pp. 663-678, April 1990.

[81] J.-H. Lin, T. M. Sellke, and E. J. Coyle, Adaptive stack filtering under the mean absolute error criterion, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 6, pp. 938-954, June 1990.

[82] H. G. Longbotham and A. C. Bovik, Relating analog and digital order statistic filters, Proceedings of the 1988 IEEE International Conference on Acoustics, Speech, and Signal Processing, New York, April 1988, pp. 1526-1529.

[83] H. G. Longbotham and A. C. Bovik, Theory of order statistic filters and their relationship to linear FIR filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, no. 2, pp. 275 287, February 1989.

[84] P. A. Maragos and R. W. Schafer, Morpho|ogical filters--Parts I and II, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, no. 8, pp. 1153-1184, August 1987.

[85] P. A. Maragos and R. W. Schafer, Morphological systems for multi-dimensional signal processing, Proceedings of the IEEE, vol. 78, no. 4, pp. 690-710, April 1990.

[86] M. P. McLoughlin and G. R. Arce, Deterministic properties of the recursive separable median filter, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, no. 1, pp. 98-106, January 1987.

[87] S. Muroga, Threshold Logic and Its Applications, New York: Wiley, 1971. [88] S. Naimpally et al., Integrated digital IDTV receiver with features, Proceedings of the

International Conference on Consumer Electronics, 1988. [89] Y. Nakagawa and A. Rosenfeld, A note on the use of local min and max operations in

digitial picture processing, IEEE Transactions on Systems, Man, and Cybernetics, vol. 8, no. 8, pp. 632-635, August 1978.

[90] P. M. Narendra, A separable median filter for image noise smoothing, IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 3, no. 1, pp. 20-29, January 1981.

[91] Y. Neuvo, Interpolators for TV scanning rate conversions, Proceedings o f the 1st World Electronic Media Symposium ITU COM 89, Palexpo0 Geneva, October 4-7, 1989, pp. 203-207.

[92] Y. Neuvo, P. Heinonen, and I. Defee, Linear-median hybrid edge detectors, IEEE Tran'sactions on Circuits and Systems, vol. 34, no. 11, pp. 1337-1343, November 1987.

[93] A. Nieminen, P. Heinonen, and Y. Neuvo, A new class of detail-preserving filters for image processing, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 74-90, January 1987.

[94] A. Nieminen, P. Heinonen, and Y. Neuvo, Median-type filters with adaptive substructures, IEEE Transactions on Circuits and Systems, vol. 34, no. 7, pp. 842-847, July 1987.

[95] A. Neiminen and Y. Neuvo, Filtering of nonstationary signals using predictor median hybrid structures, Signal Processing, vol. 13, pp. 165-176, September 1987.

[96] T. A. Nodes, Theoretical results on the properties of median type operations, Ph.D. Dissertation, School of Electrical Engineering, Purdue University, West Lafayette, IN, December 1982.

[97] T. A. Nodes, and N. C. Gallagher, Jr., Median filters: some modifications and their


properties, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 30, no. 5, pp. 739-746, October 1982.

[98] T. A. Nodes, and N. C. Gallagher, Jr., Two dimensional root structures and convergence properties of the separable median filter, 1EEE Transactions on Acoustics, Speech, and Sional Processing, vol. 31, no. 12, pp. 1350-1365, December 1983.

[99] T. A. Nodes, and N. C. Gallagher, Jr., The output distribution of median type filters, IEEE Transactions on Communications, vol. 32, no. 5, pp. 532-541, May 1984.

[100] K. Oflazer, Design and implementation of a single-chip 1-D median filter, 1EEE Transactions on Acoustics, Speech, and Signal Processing, vol. 31, no. 5, pp. 1164-1168, October 1983.

[101] K. Oist~im6 and Y. Neuvo, Vector median operation for color image processing, Proceedings of the 1990 SPIE/SPSE Symposium on Electronic Imaging, Santa Clara, CA, February 11-16, 1990, pp. 2-12.

[102] K. Oist/im6 and Y. Neuvo, Video signal processing using vector median, SPIE Conference on Visual Communications and Image Processing, Lausanne, October 2-4, 1990, pp. 1171-1183.

[103] F. Palmieri and C. G. Boncelet, Jr., A class of nonlinear adaptive filters, Proceedings of the 1988 1EEE International Conference on Acoustics, Speech, and Signal Processing, New York, April 1988, pp. 1483-1486.

[104] C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, Englewood Cliffs, N J: Prentice-Hall, 1982.

[105] P. Patel, H. Longbotham, and N. Barsalou, Fixed point analysis of stack filters, Proceedings o f the 1990 SPIE/SPSE Symposium on Electronic Imaging, Santa Clara, CA, February 11-16, 1990, pp. 194-203.

[106] S. R. Peterson, Y. H. Lee, and S. A. Kassam, Some statistical properties of alpha-timmed mean and standard type M filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, no. 5, pp. 707-713, May 1988.

[107] I. Pitas and A~ N. Venetsanopoulos, Nonlinear Digital Filters: Principles and Applications, Boston: Kluwer, 1990.

[108] C. Pomalaza-Raez and C. D. McGillem, An adaptive nonlinear edge-preserving filter, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 32, no. 3, pp. 571-576, June 1984.

[109] M. K. Prasad and Y. H. Lee, Selection filters and commutativity with memoryless nonlinearities, Proceedings of the 1990 International Symposium on Circuits and Systems, New Orleans,,LA, May 1 3, 1990, pp. 1743-1746.

[110] M. K. Prasad and Y. H. Lee, Stack filters, and selection probabilities, Proceedings of the 1990 International Symposium on Circuits and Systems, New Orleans, LA, May 1-3, 1990, pp. 1747-1750.

[111] W. K. Pratt. Digital linage Processing. New York: Wiley, 1978. [112] K. Preston, Jr., Multidimensional logical transforms, IEEE Transactions on Pattern

Analysis and Machine Intelligence, vol. 5, September 1983. [113] K. Preston, Jr., E-filters, IEEE Transactions on Acoustics, Speech and Signal Processing,

vol. 31, no. 4, pp. 861-876, August 1983. [114] K. Preston, Jr., Detection of weak subpixel targets using mesh connected cellular

automata, IEEE Transactions on Aerospace and Electronic Systems, vol. 26, no. 3, May 1990.

[115] K. Preston, Jr., and M. J. B. Duff, Modern Cellular Automata--Theory and Applications, New York: Plenum, 1984.

[116] K. Preston, Jr., M. J. B. Duff, S. Levialdi, P. E. Norgren, and J.-I. Toriwaki, Basics of cellular logic with some application in medical image processing, Proceedings of the 1EEE, vol. 67, pp. 826-856, May 1979.

[117] L. R. Rabiner, M. R. Sambur, and C. E. Schmidt, Application of nonlinear smoothing algorithm to speech processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 23, no. 6, pp. 552-557, December 1975.


[118] V. V. B. Rao and K. S. Rao, A new algorithm for real-time median filtering, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, no. 6, pp. 1674-1675, December 1986.

[119] M. Renfors, T. Huuhtanen, A. Nieminen, and T. Koivunen, Linear and non-linear filters for sampling structure conversion of two-dimensional sequences, Proceedings of the Third International Workshop on HDTV, Torino, August 30 to September 1, 1989, vol. III.

[-120] A. Restrepo and A. C. Bovik, Adaptive trimmed mean filters for image restoration, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, no. 8, pp. 1326-1337, August 1988.

[121] A. Rosenfeld and A. C. Kak, Digital Picture Processing, vol. 1, New York: Academic Press, 1982.

[122] A. E. Sarhan and B. G. Greenberg (eds.), Contributions to Order Statistics, Wiley: New York, 1962.

[123] J. Serra, Image Analysis and Mathematical Morphology, New York: Academic Press, 1982:

[124] J. Serra, Image Analysis and Mathematical Morphology, vol. 2. Academic Press: New York, 1988.

[125] M. I. Shamos, Robust picture processing operators and their implementation as circuits, Proceedings of the Image Understanding Workshop, Pittsburgh, PA, November 1978.

[126] D. L. Snyder, Random Point Precesses, New york: Wiley, 1975. [127] J. Song and E. J. Delp, The analysis of morphological filters with multiple structuring

elements, Computer Vision, Graphics, and Image Proeessin 9, vol. 50, pp. 308-328, 1990. [128] R. L. Stevenson and G. R. Arce, Morphological filters: statistics and further syntactic

behavior, IEEE Transactions on Circuits and Systems, vol. 34, no. 11, pp. 1292-1305, November 1987.

[129] J. W. Tukey, Nonlinear (nonsuperposable) methods for smoothing data, Conference Record, 1974 EASCON, p. 673 (abstract only).

[130] J. W. Tukey, Exploratory Data Analysis, Addison-Wesly: Reading, MA, 1977. [131] S. G. Tyan, Fixed points of running medians, Technical Report, Department of Electrical

Engineering and Electrophysics, Polytechnic Institute of New York, 1977. [-132] S. G. Tyan, Median filtering: deterministic properties, in Two-Dimensional Digital Signal

Processing, H: Transforms and Median Filters, chap. 5, pp. 197-217, Topics in Applied Physics, vol. 42, T. S. Huang, ed., New York: Springer-Verlag, 1981.

1-133] O. Vainio and Y. Neuvo, A signal processor for median-based algorithms, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, no. 9, pp. 1406-1414, September 1989.

[134] P. F. Velleman, Robust nonlinear data smoothing, Technical Report 89, Series 2, Department of Statistics, Princeton University, 1975.

[135] P. D. Wendt, Convergence properties of multi-dimensional stack filters, Proceedings of the 1990. SPIE/SPSE Symposium on Electronic Imaging, Santa Clara, CA, February 11-16, 1990, pp. 204-211.

[136] P. D. Wendt, Nonrecursive and recursive stack filters and their filtering behavior, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, no. 12, pp. 2099-2107, December 1990.

[137] P. D. Wendt, E. J. Coyle, and N. C. Gallagher, Jr., Stack filters, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, no. 4, pp. 898-911, August 1986.

[138] P. D Wendt, E. J; Coyle, and N. C. Gallagher, Jr., Some convergence properties of median filters, IEEE Transactions on Circuits and Systems, vol. 33, no. 3, pp. 276-286, March 1986.

[139] S. Wolfram, Theory and Applications of Cellular Automata, Philadelphia, PA: World Science, 1986.

[140] G. J. Yang and T. S. Huang, The effect of median filtering on edge location estimation, Computer Graphics and Image Processing, vol. 15, no. 3, pp. 224-245, 1981.


[141] L. Yin, J. Astola, and Y. Neuvo, Adaptive weighted order statistic filters using back- propagation algorithm, Proceedings of the 1990 IEEE International Symposium on Circuits and Systems, New Orleans, LA, May 1-3, 1990, pp. 499-502.

[t42] O. Yli-Harja and J. Astola, The bimodal approximation of the standard median filter, Proceedings of the International Conference on Digital Signal Processing, Florence, September 1987, pp. 164-168.

[143] O. Yli-Harja, J. Astola, and Y. Neuvo, Analysis of the properties of median and weighted median filters using threshold logic and stack filter representation, IEEE Transactions on Acoustics, Speech, and Signal Processin 9, vol. 39, no. 2, pp. 395-410, February 1991.

[144] D. H. Yom and S. Ann, Directed graph representation for the root signal set of median filters, Proceedings o f the IEEE, vol. 75, pp. 1542-1544, November 1987.

[145] Pao-Ta Yu and E. J. Coyle, Convergence behavior and N-roots of stack filters, 1EEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 9, pp. 1529-1544, September 1990.

[146] Pao-Ta Yu and E. J. Coyle, The classification and associative memory capability of stack filters, submitted to IEEE Transactions on Acoustics, Speech, and Signal Processin 9.

[147] Pao-Ta Yu and E. J. Coyle, On the existence and design of the best stack filter based associative memory, Proceedings of the 1990 International Symposium on Circuits and Systems, New Orleans, LA, May 1-3, 1990, pp. 2495-2499.

Documents

An overview of median and stack filtering - TUNImoncef/publications/overview-of-median.pdf · AN OVERVIEW OF MEDIAN AND STACK FILTERING* Moncef Gabbouj, 1 Edward J. Coyle, 2 and Neal