14
1876 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005 Polyphase Antialiasing in Resampling of Images Daniel Seidner, Senior Member, IEEE Abstract—Changing resolution of images is a common opera- tion. It is also common to use simple, i.e., small, interpolation ker- nels satisfying some “smoothness” qualities that are determined in the spatial domain. Typical applications use linear interpolation or piecewise cubic interpolation. These are popular since the in- terpolation kernels are small and the results are acceptable. How- ever, since the interpolation kernel, i.e., impulse response, has a fi- nite and small length, the frequency domain characteristics are not good. Therefore, when we enlarge the image by a rational factor of , two effects usually appear and cause a noticeable degra- dation in the quality of the image. The first is jagged edges and the second is low-frequency modulation of high-frequency com- ponents, such as sampling noise. Both effects result from aliasing. Enlarging an image by a factor of is represented by first in- terpolating the image on a grid times finer than the original sam- pling grid, and then resampling it every grid points. While the usual treatment of the aliasing created by the resampling operation is aimed toward improving the interpolation filter in the frequency domain, this paper suggests reducing the aliasing effects using a polyphase representation of the interpolation process and treating the polyphase filters separately. The suggested procedure is simple. A considerable reduction in the aliasing effects is obtained for a small interpolation kernel size. We discuss separable interpolation and so the analysis is conducted for the one-dimensional case. Index Terms—Aliasing, antialiasing, enlargement, interpolation, polyphase filters, resampling. I. INTRODUCTION T HE essence of this paper is about introducing a new, efficient, and simple approach for analyzing and reducing aliasing effects in enlargements, caused by the interpolation filters. We consider the case where a separable interpolation kernel is used for resampling images, i.e., changing the number of pixels in the image. We concentrate on enlargement of images by a rational factor of . Since the interpolation we discuss is separable, we conduct the analysis in one dimension and use the time and frequency domains (as is common in signal pro- cessing) instead of spatial and spatial frequency domains. Sec- tion II of this paper introduces the relation between continuous time and discrete interpolation and defines the terms to be used in the paper. Section III discusses the aliasing effects created by resampling and explains the reason for aliasing effects based on the polyphase scheme of interpolation described in Section II. In Section IV, we conduct a mathematical analysis of polyphase in- terpolation and suggest some means for measuring the aliasing. Possible procedures for correcting the resampling process are Manuscript received August 11, 2003; revised August 16, 2004. The associate editor coordinating the review of this manuscript and approving it for publica- tion was Dr. Eli Saber. The author is with the Creo IL, Ltd., Herzlia 46103, Israel (e-mail: dani. [email protected]). Digital Object Identifier 10.1109/TIP.2005.854493 described in Section V. Results of applying the procedures on images are given in Section VI, demonstrating a distinct reduc- tion of the aliasing effects. II. RESAMPLING FORMULATION In this section, we first introduce the resampling scheme in a similar manner as is discussed in [1]. This is followed by a polyphase representation as in [2] or [3], which is usually used in order to reduce computations. We start the analysis by assuming that the input sequence represents the samples of a continuous time signal . Let us suppose that the signal is bandlimited in the frequency domain to . The Nyquist frequency required for sampling the signal without loss of information is, therefore, , and the appropriate sampling period is . We represent sampling by multiplying by , where (1) The sampled signal, denoted , is given by (2) The Fourier transform (FT) of , denoted , is given by (3) Using (2), it is easy to see that , the FT of , equals (4) Using the FT of and the convolution property of FT, it is easy to see that we also have (5) The sequence is the values of the signal at the sam- pling locations , i.e., . The spectrum of the sequence, denoted , is the discrete time Fourier trans- form (DTFT) of and is given by (6) 1057-7149/$20.00 © 2005 IEEE

Polyphase antialiasing in resampling of images

  • Upload
    d

  • View
    218

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Polyphase antialiasing in resampling of images

1876 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

Polyphase Antialiasing in Resampling of ImagesDaniel Seidner, Senior Member, IEEE

Abstract—Changing resolution of images is a common opera-tion. It is also common to use simple, i.e., small, interpolation ker-nels satisfying some “smoothness” qualities that are determined inthe spatial domain. Typical applications use linear interpolationor piecewise cubic interpolation. These are popular since the in-terpolation kernels are small and the results are acceptable. How-ever, since the interpolation kernel, i.e., impulse response, has a fi-nite and small length, the frequency domain characteristics are notgood. Therefore, when we enlarge the image by a rational factor of( ), two effects usually appear and cause a noticeable degra-dation in the quality of the image. The first is jagged edges andthe second is low-frequency modulation of high-frequency com-ponents, such as sampling noise. Both effects result from aliasing.Enlarging an image by a factor of ( ) is represented by first in-terpolating the image on a grid times finer than the original sam-pling grid, and then resampling it every grid points. While theusual treatment of the aliasing created by the resampling operationis aimed toward improving the interpolation filter in the frequencydomain, this paper suggests reducing the aliasing effects using apolyphase representation of the interpolation process and treatingthe polyphase filters separately. The suggested procedure is simple.A considerable reduction in the aliasing effects is obtained for asmall interpolation kernel size. We discuss separable interpolationand so the analysis is conducted for the one-dimensional case.

Index Terms—Aliasing, antialiasing, enlargement, interpolation,polyphase filters, resampling.

I. INTRODUCTION

THE essence of this paper is about introducing a new,efficient, and simple approach for analyzing and reducing

aliasing effects in enlargements, caused by the interpolationfilters.

We consider the case where a separable interpolation kernel isused for resampling images, i.e., changing the number of pixelsin the image. We concentrate on enlargement of images by arational factor of . Since the interpolation we discuss isseparable, we conduct the analysis in one dimension and usethe time and frequency domains (as is common in signal pro-cessing) instead of spatial and spatial frequency domains. Sec-tion II of this paper introduces the relation between continuoustime and discrete interpolation and defines the terms to be usedin the paper. Section III discusses the aliasing effects created byresampling and explains the reason for aliasing effects based onthe polyphase scheme of interpolation described in Section II. InSection IV, we conduct a mathematical analysis of polyphase in-terpolation and suggest some means for measuring the aliasing.Possible procedures for correcting the resampling process are

Manuscript received August 11, 2003; revised August 16, 2004. The associateeditor coordinating the review of this manuscript and approving it for publica-tion was Dr. Eli Saber.

The author is with the Creo IL, Ltd., Herzlia 46103, Israel (e-mail: [email protected]).

Digital Object Identifier 10.1109/TIP.2005.854493

described in Section V. Results of applying the procedures onimages are given in Section VI, demonstrating a distinct reduc-tion of the aliasing effects.

II. RESAMPLING FORMULATION

In this section, we first introduce the resampling scheme ina similar manner as is discussed in [1]. This is followed by apolyphase representation as in [2] or [3], which is usually usedin order to reduce computations.

We start the analysis by assuming that the input sequencerepresents the samples of a continuous time signal . Let ussuppose that the signal is bandlimited in the frequencydomain to . The Nyquist frequency required forsampling the signal without loss of information is, therefore,

, and the appropriate sampling period is . Werepresent sampling by multiplying by , where

(1)

The sampled signal, denoted , is given by

(2)

The Fourier transform (FT) of , denoted , is givenby

(3)

Using (2), it is easy to see that , the FT of , equals

(4)

Using the FT of and the convolution property of FT, it iseasy to see that we also have

(5)

The sequence is the values of the signal at the sam-pling locations , i.e., . The spectrum ofthe sequence, denoted , is the discrete time Fourier trans-form (DTFT) of and is given by

(6)

1057-7149/$20.00 © 2005 IEEE

Page 2: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1877

Suppose we want to reconstruct the continuous signalfrom the samples . For that, we build the signal fromthe sequence using (2) and then convolve it with a contin-uous time filter with an impulse response , producing

(7)

It is clear that when is bandlimited to , anideal low-pass filter (LPF), with a passband of ,reconstructs exactly. Using an interpolation filter which isnot bandlimited to introduces aliasing effectsin . When we resample, we actually want the values of

at other sampling points, i.e., at . Thus, the outputsequence of the resampling, denoted , is given by

(8)

The enlargement factor is . When this factor is a rationalnumber , , we can write as

(9)

We denote the sequence created by sampling atby , i.e.,

(10)

Since, for a given , depends on and of , we coulddenote it by to remind us that it is sampled times fasterthan . However, to keep the equations simple, we used thisnotation only in the measures suggested later in Section IV toemphasize their dependece on .

Equation (9) is equivalent to applying the scheme depicted inFig. 1, where the expander output, denoted , is given by

integerotherwise

(11)

and so we have

(12)

Since is not zero only for , weonly need the elements having . This, together withreplacing with , gives

(13)

Fig. 1. Resampling by a rational factor of L=M .

The decimator output sequence is actually every thelement of the decimator input sequence , i.e.,

(14)

From (10) and because , it is clear that (9) and(14) are identical. This means that the scheme of Fig. 1 can beused for resampling, instead of reconstructing the continuoustime signal and actually resampling it. A similar analysiscan be found in [1].

We already noticed that is not zero only for, and, so, we could save com-

putation time, i.e., reduce the number of multiplications, bymultiplying only the nonzero values of . Thus, insteadof having the expander followed by a single filter , wecan use a different scheme, called a polyphase representation(see [2] and [3]) where we split into filters where

. The filter is given by decimation of, shifted by , by a factor of

(15)

The DTFT of can be, therefore, written as

(16)

Since , we have

(17)

The relation of and the expander output is givenby . Similarly, means that expandedversions of -s are used. Thus, we still perform timesmore multiplications than required. However, if we first applythe -s, i.e., calculate where

(18)

Page 3: Polyphase antialiasing in resampling of images

1878 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

we have, after rearranging (17) and using (18)

(19)

and, so, we can produce the same result of (17), applyingexpanders after the filters, which are then followed by

delay elements, as depicted in Fig. 2. The analysis above isequivalent to applying one of the Noble Identities [2], [3] whichallows us to replace an expander followed by an expanded ,with the original followed by an expander.

In this new scheme, we perform only multiplicationscompared to the original scheme of Fig. 1. We still performtimes more multiplications than necessary, since, after we pro-duce , we decimate it by a factor of , which means thatwe only needed to calculate the -s, i.e., of the mul-tiplications. Other more computational-efficient schemes can bebuilt so that the number of multiplications is minimal [3], but thescheme of Fig. 2 is perfectly suitable for our analysis.

Note that this scheme, with slight changes, is still efficientwhen the resampling is performed by a computer program.When a computer program is used for the calculation, every

is performed by the same filtering routine, using a dif-ferent set of coefficients. We apply the routine to the inputsequence values and produce a single output value. In such acase, instead of calculating all of the values of the sequence

, we calculate only every th value.

III. ALIASING IN COMMON INTERPOLATION FILTERS

When the filter is not bandlimited to ,we have aliasing effects. In image processing, common inter-polation functions are zero-order hold, linear interpolation [4],and piecewise cubic interpolation [5], [6]. These functions spanalong a small number of samples. Thus, we have short sequences

and, therefore, a short processing time. These interpola-tion filters were determined according to some “smoothness”requirements in the time domain (linear interpolation is contin-uous; piecewise cubic interpolation using four sample points canalso have a continuous first derivative, and if six sample pointsare used, we can reconstruct the first four terms of a Taylor se-ries representation of the sampled signal [6]). Unfortunately, allof these filters always produce aliasing since they have an in-finite bandwidth. Note that, usually, a wider impulse response

enables improved performance, i.e., the filter has a FT(or the discrete filter has a DTFT) closer to the ideal LPFrequired for exact reconstruction [5], [6].

The common way of looking at the aliasing effects is to lookat at the frequency domain using the FT of the recon-structed continuous signal followed by sampling at

, or at the DTFT of the output sequence given by

(20)

Fig. 2. Polyphase representation of the interpolation part.

This results from the decimation by (see [1]), whereis given by

(21)

The conventional approach is, therefore, to improve sothe aliasing is reduced. One possible way of doing this is to add aLPF in series to the interpolation filter . This should be donewithout changing the phase properties of the interpolation filter

. It can easily be done by convolving with some LPFthat has no phase shift. The resulting filter will have a better fre-quency response, but will be longer. A better approach is, there-fore, to desert the “smoothness” qualities and replace witha LPF designed to block all frequencies above . However,it is difficult to specify the exact relation between that LPF andthe aliasing effects. This means that we do not exactly knowthe LPF design specifications required to reduce the aliasing,and certainly not the specifications required for the best perfor-mance, i.e., the maximal reduction of the aliasing. It seems thatwe lack a systematic approach to analyzing the quality of thatLPF in terms of aliasing effects.

In this paper, we address that problem and suggest a differentapproach to analyze and reduce the aliasing. The aliasing ef-fects are easily and better understood when the polyphase rep-resentation described earlier is used. We consider the outputsof the polyphase filters as different sequences, de-noted , having the DTFT of , and analyze them sep-arately. This approach produces better results than the conven-tional approach, especially when a relatively small number ofcoefficients is used.

To demonstrate our procedure we choose to use the simplelinear interpolation as an example, i.e., we use which isgiven by

otherwise.(22)

Page 4: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1879

Thus, is

(23)

and is, therefore, given by

otherwise.(24)

Let us look at for where . Theseare depicted in Fig. 3. It is clear that these filters have differentfrequency responses. Thus, the signals are influenceddifferently by the filters . This is so since the aredecimated versions of , shifted by . Each of them is, there-fore, affected differently by the decimation (from to ),and the aliasing in each of them, caused by that decimation, re-sults in a considerably different shape of for different-s.

Actually, the different shapes of the -s is the reason forhaving aliasing effects.

This statement is inaccurate since we did not take the phaseinto account. We will, therefore, rephrase that statement, takingalso the phase into account, later on in the paper. For the timebeing, this statement can still be used for explaining the aliasingeffects.

The differences between the -s cause two main effectswhich are easily noticeable in images, and easily understood bythe polyphase approach. One is jagged edges and the second ishigh-frequency noise modulation. Let us explain these effectsusing Fig. 3.

We first discuss the high-frequency noise modulation. Con-sider an image with a constant level. During the image acquisi-tion, a wide band noise is added to that constant level by the A/Dcircuit or the sensor noise. When enlarging the image by a factorof say 16/15, i.e., and , we actually use all of thefilters to calculate 16 consecutive output samples “from”an interval of 15 consecutive input samples and their neighbors.If we start the resampling at , i.e., , thenext value should be , and the one following it

should be , since we advance along thetime axis by increments of . This means that we calcu-late using the values of and and the coefficients of

. Then, we calculate using the values of andand the coefficients of . We use only the values of and

since these two points belong to the intervalwhich starts with and ends with . Inlinear interpolation, we have only two coefficients in the -s,and so we only need the two values at the ends of the interval.For , we use the values of and , since we are in thesecond interval where , and the coefficients of

. Similarly, we use for , for , etc.,until we get back to at . This pattern repeats itselfevery 16 consecutive output samples.

Thus, the output samples are taken from 16 images (se-quences) that are each produced by a different filter

Fig. 3. H (!)-s of linear interpolation (L = 8).

. is an all-pass system, and, so, it does not alterany of the frequencies of the added noise. On the other hand,

looks totally different. Its frequency response is. It has a zero in , and, so, it

attenuates the higher frequencies of the noise. The other filtersare somewhere in between these two. Going along the

output image and changing the filters, therefore, produces alow-frequency modulation of the high frequencies of the noise.Actually, all frequencies are affected, but since the differencesbetween the filters are larger in high frequencies, the effectis stronger in high frequencies. In Fig. 11, this phenomenonis demonstrated in a small part of the Lena image to whichwe added white Gaussian noise prior to enlarging it by 16/15using linear interpolation (the image was sharpened afterwards,in order to intensify the modulation effect, so it is easier toobserve in print). Note that if an image needs to be shifted by5/16 of a pixel, for example, we only use the output of .In such a case, there is some degradation of the image, mainlyin high frequencies, but there is no aliasing. Aliasing appearsonly when we use more than one polyphase filter, each withdifferent frequency responses.

The other noticeable effect is jagged edges. If we enlarge animage by a factor of 2, i.e., and , the even samplesof the output signal are duplicates of the input samples, whilethe odd output samples are “interpolated” by averaging the twoadjacent input samples. Thus, the high-frequency content of theodd pixels is blurred. In a case in which an area of the imageincludes high frequencies, e.g., edges, the difference betweenthe odd and even pixels is noticeable. The even pixels have morecontrast than the odd, blurred pixels. This is easily seen in edgesthat are almost horizontal or vertical, as in Fig. 14. In this figure,we used linear interpolation in the x and y directions to enlargethe image by a factor of 2 in each direction.

This effect is amplified when sharpening is applied to theimage following the enlargement. A 3 3 sharpening matrixactually adds the difference between a pixel and its surroundingsto the pixel value. If that pixel is the average of its surrounding,which is exactly the case for the odd row and column pixels, thepixel is not changed. Other pixels, for example the even rowand column pixels, will be “sharpened,” i.e., their values are

Page 5: Polyphase antialiasing in resampling of images

1880 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

changing. This increases the contrast differences between theeven and odd pixels even more.

Again, the two effects described above are caused by thevery same reason: differences between the interpolation filters

.Note that, in order to reduce these effects, the interpolation

filters are usually shifted by more, i.e.,instead of .

This means that we sample not exactly on the grid; thus,the resulting image is slightly shifted. This reduces the effects,since the difference between the -s is reduced.

IV. POLYPHASE ANALYSIS OF INTERPOLATION FILTERS

In this section, we suggest measures for estimating the qualityof an interpolation filter. These measures were first suggestedin [7]. We could define the desired interpolation filter byspecifying its desired frequency response . In the casethat is bandlimited to , the polyphase fil-ters, will have the same amplitude response, denoted

. In such a case, each of the polyphase filters has a fre-quency response given by

(25)

Since all of the polyphase filters have the same amplitude andthe appropriate phase, we expect no aliasing. It is easy to seethat, in such a case, is given by

otherwise.(26)

Thus, it is really bandlimited, and, so, no aliasing occurs.Here, we suggest a measure, the Interpolation Error Index,

denoted , for quality of an interpolation filter . Theinterpolation error index is defined as the average deviation ofthe -s from the desired interpolation filter of (25)

(27)

Before we continue the analysis, we need to check the relationbetween this measure and the regular measures used in filterdesign. Let us concentrate on square error, i.e., on . Theregular mean-square error (MSE) measure is given by

(28)

where is the desired frequency response of the inter-polation filter. It is easy to see that when is bandlim-ited to , as in (26), then the two measures satisfy

.

Although, for interpolation filters, we want to be ban-dlimited to , and so the two measures are basicallyequal, these two measures represent different approaches.

Using (28) is equal, in principle, to an analysis finding theerror energy at the output of the interpolation filter , as-suming that the signal is a WSS white noise process. An anal-ysis following this principle can be found in [8], although, there,the energy error of (instead of ) is calculated. Equa-tion (27), for square error, i.e., for , finds the error energyof each polyphase filter compared to the appropriate polyphasefilter of an ideal interpolation filter satisfying (26) or (25), andthen adds those errors together. Again, we assume that the signalis a WSS white noise process.

The MSE index measures the overall square error ofthe interpolation filter. It is not always a good measure for devi-ation from the desired filter because it averages large deviations,a pitfall which minimax criterion avoids. We still remain withthis measure since it enables us to concentrate on measuring thealiasing expected from the interpolation filter, and not just anoverall error measure. For that, we now split the quality index

into two parts. One part is a measure of the aliasing andthe other of the average deviation of the actual filter from the de-sired one. Although it is possible to split the in a similarmanner, we chose to use , since it allows us to minimizethe aliasing effects using the polyphase approach.

We denote the amplitude that brings the error indexto a minimum for a given set of filters by .

It is easy to see, by differentiating in respect to , thatis given by

(29)

We refer to as the average amplitude function of thepolyphase filters. From (16), it is easy to see that

for . We still prefer to use the no-tation in the following equations, in order to makethem self explanatory. Note that when is symmetric, i.e.,

, then is real. Using some algebra, and(29), we can see that for a square error is given by

(30)

We, therefore, define the Amplitude Index which in-dicates the deviation of the average amplitude functionfrom the desired amplitude function , as

(31)

Page 6: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1881

We also define the Aliasing Index , which indicates thedeviation of the -s from the average amplitude functionwith the appropriate phase factor, as

(32)

So, now, we have separate measures for the aliasing and forthe deviation of the average amplitude from the desired ampli-tude function.

Note that our previous statement that “the different shapesof the -s is the reason for having aliasing effects” is notaccurate. It should be that the differences of the -s, (witha phase factor of ), from , the average of thepolyphase filters, are the reason for having aliasing effects.

Suppose we want to compare the aliasing effects producedby 2 interpolation filters. To do that, we want to eliminate othereffects, e.g., the different frequency responses of the two inter-polation filters. Therefore, we should add a sharpening circuitin series to the interpolation filters, thereby bringing the twoimages to the same average frequency contents. Similarly, fora correct comparison of aliasing effects in different interpola-tion filters, the two measures and should be normal-ized. measures the difference from but differentinterpolation filters have different -s. In order to com-pensate for different -s, we should “equalize” or “nor-malize” the interpolation filter so that the normalized polyphasefilters have . Therefore, we define the NormalizedAliasing Index as

(33)

This is equivalent to a series LPF equalizer, with a frequencyresponse of for , which equalizes theaverage amplitude to 1.

Similarly, for comparing the deviation of from thedesired for two different -s, we need to normalize

as follows:

(34)

where is the Normalized Amplitude Index. Note that, usu-ally, when interpolation filters are designed, we are interested in

as an indication of the aliasing, and in as an indica-tion of the deviation from the desired amplitude function .In such cases, is not required.

Let us apply the normalized aliasing and the amplitude in-dices to a few common interpolation functions. We assume thatthe desired amplitude function equals 1, as in an idealLPF, and do the comparison for (in this case, we have

, since ).

We start with the simple zero-order hold interpolation, alsocalled nearest neighbor interpolation, and given by

otherwise.(35)

The normalized aliasing index is and the ampli-tude index for is 0.0376 ( is 0.2613).

The normalized aliasing and amplitude indices for the first-order hold interpolation, also called linear interpolation, whichis describe by (23) are 0.0670 and 0.0840, respectively ( is0.1193).

For the four-point cubic piecewise interpolation of [5] and[6], also called third-order cubic interpolation [9], given by

(36)we have a normalized aliasing index of 0.0620 and an amplitudeindex of 0.0407 ( is 0.0789).

Finally, for the six-point cubic piecewise interpolation of [6],also called fourth-order cubic interpolation [9], given by

(37)we find a normalized aliasing index of 0.0504 and an amplitudeindex of 0.0334 ( is 0.0645).

In Figs. 4 and 5, we see the frequency responses of the-s of the four- and six-point piecewise cubic inter-

polation filters for . From comparing the differencesbetween the -s in Figs. 3–5, we conclude that the ex-pected aliasing from linear interpolation is higher than the oneexpected from four-point piecewise cubic interpolation, which,in turn, is higher than the aliasing expected from six-pointpiecewise cubic interpolation. This is also what we find whenthese three interpolation filters are used for enlargement of im-ages. Thus, it seems that the normalized aliasing index agreeswith these conclusions and is useful for comparing the aliasingcaused by interpolation filters.

We also note that the linear interpolation polyphase filtersshown in Fig. 3 are not only more different than their averagerelative to those of Figs. 4 and 5 (thus causing aliasing effects),but also demonstrate a higher attenuation of high frequencieswhen compared to the other two interpolation filters. The de-viation of the average of the linear interpolation filters, fromthe ideal LPF having amplitude of 1, is larger than this of theaverage of the four-point piecewise cubic interpolation filters,which, in turn, is larger than the deviation of the average of thesix-point cubic piecewise filters from 1. Again, this agrees withthe amplitude index of the above filters (the average amplitude

equals the average of the -s when the phaseof the polyphase filters -s is the expected linear phase of

Page 7: Polyphase antialiasing in resampling of images

1882 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

Fig. 4. H (!)-s of four-point piecewise cubic interpolation (L = 8).

Fig. 5. H (!)-s of six-point piecewise cubic interpolation (L = 8).

; when the phase of the -s is close to the expectedphase, we can use the average of the , although not asaccurate as , for a rough estimation of the aliasing andthe amplitude indices). In practice, we see that images enlargedby these three interpolation filters follow this observation.

When two interpolation filters have an equal normalizedaliasing index, this suggests that they produce a similar amountof aliasing effects. Therefore, we may use the normalizedaliasing index for comparing aliasing of two filters that weredesigned using equal or different amplitude functions. Notethat, since using neutralizes the amplitude deviationfrom the desired amplitude , there could be cases in whichtwo interpolation filters have the same normalized aliasingindex, although one of them is close to the desired andthe other is far away from it. We should, therefore, give theappropriate attention to both the normalized aliasing indexand the amplitude index, when comparing interpolation filters.The appropriate attention depends on the application. If thealiasing is the issue, we use the normalized aliasing index. Ifthe amplitude is the important issue, we rely on the amplitudeindex. As an example, one may consider an image processing

system, having a resampling module followed by a sharpeningmodule. If this is the case, then the sharpening module can beused to control the amplitude, while the interpolation filtersshould be adjusted so they have a good, i.e., small, normalizedaliasing index.

V. ANTIALIASING PROCEDURES

Unfortunately, as we had already seen in Figs. 3–5, the filtersusually do not satisfy (25). In most cases, each of the

polyphase filters has a different amplitude function, anda phase that is not necessarily equal to , but close to it. Inmost cases, that phase is not linear.

Therefore, we want to correct those filters so that aliasingis reduced. Our approach is to analyze and modify each of thepolyphase filters separately. Adjusting meanschanging . This, of course, changes , but we do not tryto adjust and directly, since it is difficult to modify

and understand the implications on the polyphase filters, i.e., on the aliasing.

Our plan is to equalize the polyphase filters , thus re-ducing the normalized aliasing index. We offer a few proceduresfor adjusting the polyphase filters (some of them also appear in[10], [11]).

A. Procedure-I: Polyphase Equalization

We could add an equalizing filter in series to eachof the polyphase filters . In theory, the equalizing filtersshould satisfy

(38)

In practice, it is much easier to avoid tampering with the phase.Thus, we only correct the amplitude of . This is done byignoring the phase (i.e., assuming that has the exact phaseof ), and using only the magnitude of , i.e., we

(39)

Usually, the amplitude of is positive and does not changeits sign in the interval . We choose an thatis similar in nature, and, so, is a positive real func-tion, i.e., it has a zero phase. Having a zero phase is actuallydesired, since we do not want to alter the phase characteristicof , and so we need to have a zero phase equalizing filter

. For designing that equalizer, we could use any con-ventional LPF design technique that produces linear phase fil-ters, e.g., frequency sampling or optimal minimax filter design,e.g., the Parks–McClellan algorithm. In case we want the lengthof our final polyphase filter to be coefficients, we need to findthe best approximation to using coefficients for

, since, in our case of linear interpolation, has twocoefficients.

As an example, for the case of linear interpolation, we havechosen as our . Here, and in all

Page 8: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1883

of the following examples, we use , i.e., six coefficientsper each polyphase filter. The resulting filters, after equaliza-tion by frequency sampling, are depicted in Fig. 6. For these fil-ters, we have (which is approximately 4.3 timesbetter than six-point cubic interpolation and 5.7 times better thanlinear interpolation). The result of applying these new filters tothe same noisy image that was used for producing Fig. 11 isshown in Fig. 12. These pictures were equally sharpened so thecomparison is fair. We clearly see in Fig. 12 that the noise mod-ulation is completely removed.

B. Procedure-II: Add a Series LPF to

In this procedure, we add a linear phase LPF , havingcoefficients, in series to the original , having

coefficients. We specify the filter as having apassband of about and use any algorithm,say the Parks–McClellan algorithm, to find . We thenconvolve that filter with h[n]. The resulting filter is our new in-terpolation filter having coefficients. This is exactly thesame as one of the classical approaches we mentioned in Sec-tion III. So, what is the difference?

The difference is in the analysis of the resulting filter. In our procedure, the next stage is to split the

resulting filter into new polyphase filters , find their fre-quency response -s, calculate the normalized aliasingindex and the amplitude index, and then decide whether theyare satisfactory. If they are not, we should change thespecifications and repeat the process.

The problem here is that we do not know how to change thosespecifications in order to reduce the aliasing. Thus, we shoulduse an unsystematic trial and error search until we are satisfied.

C. Procedure-III: Use Only a LPF

In this procedure, we totally drop the linear interpolation filter, and, as in the classical approach, look for the “best” LPF.

The question is “best” in what sense?In case we are looking for minimal aliasing effects, we want

the of the filter to be minimized. Thus, the design proce-dure is as follows. We find a linear phase LPF , having

coefficients and split the result into polyphase fil-ters , each having coefficients. Then, we find their fre-quency response , calculate the normalized aliasing indexand the amplitude index, and decide whether they are satisfac-tory. If they are not, we should change the specifications of thefilter and repeat the process.

Again, we have the problem of not knowing what changes inthe specifications will produce the best result, i.e., as close aspossible to the desired . So, eventually, again we have toapply an unsystematic trial and error search.

It seems that Procedures II and III are not systematic enough.The difficulty lies in the fact that we do not know how thespecifications influence the aliasing index, which we want tominimize.

Thus, in the next two procedures, we reverse our approach.We will design each of the polyphase filters separately.

Fig. 6. H (!)-s of linear interpolation equalized by procedure-I, A(!) =cos(!=2) (L = 8).

D. Procedure-IV: MSE Polyphase Filters

In this procedure, we discuss designing optimal MSEpolyphase filters, i.e., polyphase filters minimizing the inter-polation error index . This is easy to do analytically. Wejust need to calculate the inverse DTFT of the desiredgiven by (25), and use the center coefficients as . Thisis so since the inverse DTFT is the Fourier Series coefficientsof . Note that since, when is bandlimited to

, we have , we could also calculatethe inverse DTFT of the desired and take the

center coefficients as . As an example, we discussthe case of an ideal LPF, in which for ,i.e., . Using the inverse DTFT of , wehave

(40)

for . We clearly see that the resultingequals

(41)

for , which are the center coeffi-cients of the ideal LPF.

Designing FIR filters using plain MSE criterion is notcommon, since the resulting filters have large deviations fromthe desired amplitude near the cutoff frequency. The MSEcriterion “allows” large deviations from the desired frequencyresponse at certain frequencies, since it averages the error. Inour case, using the MSE criterion produces polyphase filtershaving similar effects which result with a medium normalizedaliasing index.

Furthermore, if the input sequence has a constant level, wewant the output sequence to have the same constant level. Inorder to avoid any “ripple,” we must verify that the sum of co-efficients in is 1, i.e., . This condition is notmet when optimal MSE design is used. We can normalize the

Page 9: Polyphase antialiasing in resampling of images

1884 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

-s by dividing them by their sum of coefficients. The nor-malized filters have no ripple in constant level inputs, but theyare not MSE optimal anymore. This normalization causes onlyslight change in the normalized aliasing index.

Even if we agree to normalize the -s, we still have toaddress the large deviations from we have in the frequencyresponses of the polyphase filters, since those deviations appearalso in low frequencies. In case we leave the situation as is, wewill see a “ripple” or “modulation” in low-frequency areas ofthe enlarged image.

When designing FIR filters, windowing is usually used to re-duce these deviations. For interpolation filters, applying plainwindowing to is not the right procedure, since it widens thefilter , thus causing aliasing effects. We can apply a sim-ilar technique, in which we make the shape of “smoother”as we get close to , i.e., have a gradual change as opposed tothe abrupt jump from 1 to 0 of an ideal LPF. This is almost as ifwe apply a window to a narrower filter.

As an example, we took , similar tothe example used for Procedure-I. The inverse DTFT of

is

(42)

The resulting six coefficient filters, after normalization,should be compared to those designed by Procedure-I. They areexpected to be closer to the desired , sincethey have the optimal for that (prior to normal-ization). The resulting filters are depicted in Fig. 7, and lookmuch closer to than those of Fig. 6. Their normalizedaliasing index is , which also suggests that theyare much closer to their average.

As in window design, the shape of determines the char-acteristics of the filter. From the cases we have tried, it seemsthat a “smoother” enables reaching a lower aliasing index

.In the following design procedure, we apply minimax de-

sign techniques. These are usually considered superior to win-dowing in FIR filters design, since using the minimax criterionprevents large deviations from the desired frequency response.Minimax design results with equiripple filters that have the min-imal number of coefficients for a specific maximal deviation, orhave the minimal maximal deviation of the frequency responsefrom the desired one, for a given number of coefficients.

E. Procedure-V: Minimax Polyphase Filters

In this procedure, we design optimal minimax polyphase fil-ters. A minimax optimality criteria for the filters couldbe

(43)

where is a nonnegative weighting function which is com-monly used in specifying filters. Note that, since al-ways has a zero in , it is recommended to use an ampli-tude function which also has a zero in . We expectthe resulting polyphase filters to be close to the desired filters of

Fig. 7. H (!)-s of optimal MSE LPF, A(!) = cos(!=2) (L = 8).

(25) and, therefore, have a good normalized aliasing index anda good amplitude index.

Designing optimal polyphase filters according to (43) is notso easy. However, changing the criterion slightly, and splittingit into two parts as in the two following equations:

(44)

and

(45)

makes it very easy to implement. This is easier, since we knowfrom the symmetry properties of the DTFT that the real partof is the DTFT of , where is the even partof . Similarly, the imaginary part of is the DTFTof , where is the odd part of . Thus, we use(44) and the Parks–McClellan, or a similar algorithm, to find anoptimal symmetric filter [again, we recommend forcing

to be 1, so no modulation occurs when the image is con-stant]. Then, we use (45) and the Parks–McClellan algorithmto find an optimal anti-symmetric filter . We combine thetwo filters by adding the even and odd filters

(46)

The difference between the criterion of (43) to the criterionof (44) and (45) is in the shape of the range of the allowed error,which is round for (43) and rectangular for (44) and (45) (see[12]). We named filters designed that way almost linear phase(ALP) FIR filters.

In Fig. 8, we see the six-coefficient ALP polyphase filtersdesigned for of the filters of six-point piecewisecubic interpolation. The weighting function is 50 for ,climbing linearly to 150 at , staying at 150 until

, linearly decreasing back to 50 at , andcontinuing to decrease linearly to 10 at and to 0 at

. This results with and an of 0.0503.

Page 10: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1885

Fig. 8. H (!)-s of optimal ALP for A(!) = A of the fourth-order cubicinterpolation (L = 8).

In Fig. 9, we see the six coefficient ALP polyphase filters de-signed for , which is equal to the DTFT of the sequence[0.0, 0.1, 0.6, 0.6, 0.1, 0.0], without any weighting func-tion for -s, and with a weighting function of 50 for

, decreasing linearly to 10 at and further to 0at , for the -s. This results in , whichis much better that that of Fig. 8, but with (and

). In calculating the ALP filters above, we useda low-resolution search, with resolution of only 0.01.

From the above examples, we see that the shape ofplays a main role in determining the performance of an inter-polation filter. The normalized aliasing index can be reducedwhen a “softer” is specified, at the expense of increasingthe amplitude index. Eventually, one must decide which com-promise is acceptable for the specific application for which theinterpolation filter is designed.

Let us discuss the ultimate interpolation filter. For a given, we could ask for the minimal under some restric-

tions on , or the deviation of from , etc. Wecould even go further and look for the which bringsto a minimum, under some restrictions on . We did not dothat in this paper. Instead, we tried to get closer to the given

. We used the MSE index or the minimax criterionof (44) and (45) as our measures.

The first two design procedures mentioned above, are subop-timal according to both measures, since they restrict their filtersto a linear interpolation filter in series to a zero phase filter. Thereis no reason in the world to assume that a filter bringing toa minimum, or satisfying (44) and (45), is built that way.

The third procedure uses common methods, e.g., thePark–McCllelan algorithm, for designing the filter. Unless theunsatisfactory MSE design is used, we do not know what arethe appropriate specifications required for in order toproduce the optimal -s. Thus, for the Park–McCllelanalgorithm, which finds a minimax “optimal” for a desired

, there is also no reason to expect the resulting polyphasefilters to be optimal in the sense of having a minimal

or to reach the minimax error of (44) and (45), or even

Fig. 9. H (!)-s of optimal ALP—another example for procedure-V (L = 8).

just to have a good , although an equiripple “optimal”design of that is done.

Procedure-IV leads to an optimal . Procedure-V leads tooptimal ALP filters, which are optimal according to the crite-rion of (44) and (45). However, these two optimal criteria arenot optimal in the sense that we do not find the ultimate interpo-lation filter having a minimal , for a given and givenrestrictions on .

To summarize, in this section, we introduced a few designprocedures meant to achieve polyphase filters having a low nor-malized aliasing index and a low-amplitude index. Note that theshape of is the main factor determining whether a goodnormalized aliasing index can be reached or not.

Also note that there are other factors beside aliasing that areimportant when interpolation filters are designed, and whichwere only partially discussed above.

One such factor is a low ripple. By that, we mean that a con-cave or convex gradual change in a signal, should stay concaveor convex after the interpolation. Similarly, a straight line inwhich we have should result with a straight linein , and the particular case of a constant level signal shouldresult in the same constant level at . We have already men-tioned that in order not to produce a ripple at a constant level,the sum of coefficients in should be 1, i.e., . It iseasy to show that Procedures I and II create no ripple in straightlines. For the other procedures, one must check the straight lineripple of the resulting filters. In order to reduce ripple effects,one should verify that is symmetric, i.e., .

One other factor is the “ringing” effects in an abrupt changein , e.g., a step function. An overshoot before a sudden dropin the signal, and an undershoot after that drop, are actuallygood in images. However, no secondary undershoot should ap-pear prior to the overshoot, and no secondary overshoot shouldappear after the undershoot.

This paper focuses on the aliasing effects, but the rippleand ringing should also be taken into consideration when thepolyphase filters are designed separately. The importance ofthese issues increase when the enlarged image is processedagain.

Page 11: Polyphase antialiasing in resampling of images

1886 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

Fig. 10. Part of Lena with additive white Gaussian noise.

Fig. 11. Low-frequency modulation of high-frequency noise produced bylinear interpolation (L = 16,M = 15).

VI. IMAGE EXAMPLES

In this section, we demonstrate applying several interpola-tion filters on several images. We start with a noisy image ofLena, Fig. 10, which is enlarged by a factor of ,using linear interpolation. The result is given in Fig. 11, aftersharpening. We clearly see the low-frequency modulation of thehigh-frequency noise. In Fig. 12, we see the same noisy image,enlarged by the interpolation filter calculated according to Pro-cedure-I, after the same amount of sharpening. The -s ofthe polyphase filters used, are depicted in Fig. 6. In Fig. 13, wesee the result of applying the polyphase filters of Fig. 7, whichwere calculated using Procedure-IV, after equal sharpening. Wesee that, in both cases, the aliasing effect of noise modulation istotally removed by the new filters.

One might complain that the comparison of two coefficientlinear interpolation filters with six coefficient interpolation fil-ters, is not fair. Well, the six-point piecewise cubic interpolationalso produces similar noise modulation. It is similar in nature but

Fig. 12. Result of filters equalized by Procedure-I, (L = 16M = 15).

Fig. 13. Result of filters designed by Procedure-IV, (L = 16M = 15).

less noticeable. We used the linear interpolation since it demon-strates the noise modulation much better in print.

For the aliasing effects of jagged edges, we compare thesix-point piecewise cubic interpolation with the ALP filters wefound using Procedure-V. The filters of Fig. 5 are supposed todemonstrate the aliasing effects. Those of Fig. 9 should demon-strate a substantial reduction in the aliasing effects, while thefilters of Fig. 8 are expected to be somewhere in between. Weapplied the filters of Figs. 5, 8, and 9 to two images: the raysand the clock.

The rays image was enlarged by 2. We can see severe aliasingeffects produced by the linear and six-point cubic interpola-tion, in Figs. 14 and 15. We see less aliasing in Figs. 16 and17 (Fig. 18 is the original image). It seems that the amount ofjaggedness found in these figures agrees with the normalizedaliasing indices. The degradation in high frequencies also agreeswith the amplitude index.

The clock image was enlarged by 4 using the same interpo-lation filters. Again we, see similar results. Note that we see a

Page 12: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1887

Fig. 14. Enlargement by 2 (L = 2,M = 1) using linear interpolation.

Fig. 15. Enlargement by 2 using six-point piecewise cubic interpolation.

Fig. 16. Enlargement by 2 using the ALP filters of Fig. 8.

pattern in the background of the upper part of Fig. 19 and someremains of that pattern in Fig. 20. This pattern is due to aliasing.This can be easily verified using a longer and better interpola-tion filter. The pattern does not appear in Fig. 21 (Fig. 22 is theoriginal image).

The decision of whether the filters of Fig. 9 are “better” thanthose of Fig. 8 or not depends on the application (e.g., are we

Fig. 17. Enlargement by 2 using the ALP filters of Fig. 9.

Fig. 18. Original rays image.

going to sharpen the image after the resampling?) and on theamount of aliasing tolerated by the viewer.

Following these examples, we conclude that the polyphaseanalysis is a good and efficient approach for substantially re-ducing aliasing effects in enlargements.

VII. CONCLUSION

Aliasing in enlargement of images has two main effects: peri-odic modulation of high-frequency noise and jagged edges. Theclassical solution to this problem is to improve the interpola-tion filter. However, we lack a systematic analysis of the qualityof that LPF in terms of aliasing effects. In this paper, we intro-duced a new approach for anti-aliasing analysis in enlargements.The basic idea of this approach is to separately analyze each ofthe polyphase filters used for interpolation. We explained thatthe aliasing effects result from the differences between the fre-quency responses of the polyphase filters, and defined the Nor-malized Aliasing Index, which measures the aliasing expectedfrom an interpolation filter. We also defined the amplitude index,measuring the deviation of the interpolation filter from the de-sired amplitude response.

Since the aliasing effects result from the differences betweenthe frequency responses of the polyphase filters, equalizingthose filters reduces the aliasing. A few simple procedures for

Page 13: Polyphase antialiasing in resampling of images

1888 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 11, NOVEMBER 2005

Fig. 19. Enlargement by 4 (L = 2,M = 1) using six-point piecewise cubicinterpolation.

Fig. 20. Enlargement by 4 using the ALP filters of Fig. 8.

polyphase equalization are described. The resulting filters wereused for enlargement of a few images having the effects men-tioned above, demonstrating a considerable reduction in thealiasing effects, which was obtained for a small interpolationkernel size.

Fig. 21. Enlargement by 4 using the ALP filters of Fig. 9.

Fig. 22. Original Clock image.

REFERENCES

[1] A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing.Englewood Cliffs, NJ: Prentice-Hall, 1999.

[2] P. P. Vaidyanathan, “Multirate digital filters, filter banks, polyphasenetworks, and applications: a tutorial,” Proc. IEEE, vol. 78, no. 1, pp.56–93, Jan. 1990.

[3] , Multirate Systems and Filter Bank. Englwood Cliffs, NJ: Pren-tice-Hall, 1993.

[4] W. K. Pratt, Digital Image Processing. New York: Wiley, 1978.

Page 14: Polyphase antialiasing in resampling of images

SEIDNER: POLYPHASE ANTIALIASING IN RESAMPLING OF IMAGES 1889

[5] S. K. Park and R. A. Schowengerdt, “Image reconstruction by parametriccubic convolution,” Comput. Vis., Graph. Image Process., vol. 23, no. 3,pp. 258–272, Sep. 1983.

[6] R. G. Keys, “Cubic convolution interpolation for digital image pro-cessing,” IEEE Trans. Acoust. Speech Signal Process., vol. ASSP-29,no. 6, pp. 1153–1160, Dec. 1981.

[7] D. Seidner, “Polyphase analysis of aliasing effects in enlargements,” inProc. ICASSP, vol. 2, May 2004, pp. 561–564.

[8] F. Mintzer and B. Liu, “Aliasing error in the design of multirate filters,”IEEE Trans. Acoust. Speech Signal Process., vol. ASSP-26, no. 1, pp.76–88, Feb. 1978.

[9] E. Meijering and M. Unser, “A note on cubic convolution interpolation,”IEEE Trans. Image Process., vol. 12, no. 4, pp. 477–479, Apr. 2003.

[10] D. Seidner, “Noise equalization in resampling of images,” in Proc. IEEE22th Convention Israel, Dec. 2002, pp. 75–77.

[11] , “Polyphase antialiasing in enlargements,” in Proc. 5th IASTEDInt. Conf. Signal Image Processing, Aug. 2003, pp. 332–336.

[12] , “Optimal FIR filters with almost linear phase,” in Proc. ICASSP,vol. 6, Apr. 2003, pp. 1–4.

Daniel Seidner (S’98–M’99–SM’04) received theB.Sc., M.Sc., and Ph.D. degrees in electrical engi-neering from Tel-Aviv University, Israel, in 1981,1990, and 2000, respectively.

He has been with Scitex Corporation, Ltd., nowCreo IL, Ltd., Herzliya, Israel, since 1981. Until2000, he was a Senior Digital Design Engineer andan Image Processing Researcher in the R&D groupof the Input System Division and participated inthe development of the world’s first flatbed CCDscanner for pre-press systems. He holds several

patents in the area of color scanners and image processing. Since 1997, he hasbeen a Senior Scitex Fellow, now a Senior Creo Fellow. Since 2000, he hasbeen a part of the R&D team of Leaf, Creo’s digital photography division. Heis also a member of the Department of Computer Science at the College ofManagement, Rishon-LeZion, Israel, and the Efi Arazi School of ComputerScience in the Interdisciplinary Center, Herzliya. His research interests consistof signal and image processing and computer architecture.