Image interpolation by Estimation and Deconvolutionof Wavelet Approximate Subband

Hiroshi Yasuno, Werapon Chiracharit, Kosin ChamnongthaiDepartment of Electronic and Telecommunication Engineering, Faculty of Engineering

King Mongkut's University of Technology ThonburiBangkok, Thailand

Abstract-This paper proposes a method of image interpolationby estimation and deconvolution of wavelet approximate subband. Estimation HLIn this method, wavelet approximate subband that contains muchinformation is firstly interpolated by interpolation methods suchas cubic spline in order to find the decimated low-pass component .eKuHighof the image. The low-pass component is then deconvoluted for LH resolutiongetting the original image. To confirm the effectiveness of Imageproposed method, the experimental results performed withstandard test images reveal that proposed method outperformsconventional methods. The best result in peak-signal-to-noise-ratio is 39.27 dB by our proposal with cubic spline interpolation.

I. INTRODUCTION

Image interpolation plays an important role in screen display, (a) The IWT-based method. Normally, horizontal, verticaldigital TV, etc. The final goal of research in image and diagonal detail subbands are estimated.interpolation is to represent an original image with sharpness.Wavelet-based image interpolation is considered to representan original image in good sharpness comparing with tradition Estimation Undecimated Deconvolution Highspatial image interpolation methods. In addition, modem LL > resolutionimprovements of hardware performance enable us to adopting Il Imagemore heavy calculation for getting better quality highresolution image. Hence, the availability of wavelet based (b) The BWT-based method. BWT stands for backwardimage interpolation become higher than the passed time. wavelet transform. It estimates only undecimated LL subband.

Kinebuchi, et al proposed HMT (Hidden Markov Tree)based wavelet image interpolation [1]. Their method achieves Fig 1. The concept of wavelet-based image interpolation.their required parameters by using EM (Expectation LLdenoteapproximatesubbandwhichistheinput.Maximization) algorithm from an original image. Kim, et al and thatalso utilized HMT, but their method does not need a training perfectly guarantees the output images are the same asprocess because their parameters are estimated statically from the input ones in every execution.only input image [2]. These stochastic model based methods In this paper, since approximate subband is obtained byincluding an element of randomness have following three convolution and down sampling of the original imagedisadvantages. The first is that different results are obtained respectively, we try to go in the inverse way, in order to obtain

en tes.

Tthe perfect original image. That is we firstly estimate anevenifhite inaceputais e resame. Thesecod is a slightcpossibisty undecimated approximate subband, which is the result of ato achieve unacceptable results. The third iS a necessity of post covlto'fteoiia mg,b neplto nprocessing such as Gaussian low-pass filtering. Especially, the exraolation, an secorndl deconolut itheretation rslfirst~~~~~~~~~~~~~~anXheodfaueaent utbefrdgtlT n

extrapolation, and secondly deconvolute the estimation resultfirst and the second feature are not suitable for digital TV and by solving SLE (simultaneous linear equations). In addition,so on. On the other hand, Temizel, et a! estimated horizontal there is no need to perform a training process on the originaland vertical detail subbands by using regression analysis [3]. image, and the post processing such as Gaussian low-passChang, et al proposed a spatially adaptive algorithm based filtering, that is the advantages of the proposed method.wavelet image interpolation [4]. Their methods are free from Obviously, the same input generating the same output isthe disadvantage of randomness, but it is impossible to guaranteed.represent an original image perfectly because of using of zeroIIPROSDMTDcoefficients as diagonal detail subband. Hence, we need amethod which has a possibility to represent an original image We propose a different way from the existing major wavelet-

based image interpolation methods, which consist of the

978-1-4244-2336-1/08/$25.00 © 2008 IEEE154

Input Estimation Deconvolution(a) Extensi (b) (c) (d)

L........... Interpolation|Solving SLE& &

Extrapolation em oving

I_______________I______ extra regionFig. 2. Calculation step of the proposed method. The size of all four rectangles corresponds to the image size on each step.(a) The input image which is an approximate subband. (b) The result of input extension.(c) The result of estimation. (d) The output image which is the result of

the deconvolution and removing extended values.

estimation of detail subbands and IWT (inverse wavelet After calculating (2), undecimated approximate subband R' istransform), see Fig. 1(a). So, we call its as IWT-based method. gotten by one data at head is removed if L is odd.On the other hands, we named our proposal BWT-based Antithetically, if L is even, R'= R . Hence, L', which ismethod because it is inverse way of Equation (1), see Fig. 1. (b). length of R', is L -1 or L, see Equation (4).BWT stands for backward wavelet transform. Our proposal isalso one kind of inverse wavelet transforms but it uses only the L -I if L is oddvalues of low-pass analysis filter of wavelet transform. L t i L i (4)s

CL if L iS evenA (t 2)FLA T' denotes matrix according R'. If R' and T' are known,

A denotes an approximate subband which show as LL. FLA isunknown input signal X can be calculated by solving SLE,

LA see Equation (6), because (5) contains Mvariables and MN length analysis filter of wavelet transform. X represents M linear equations. On (6), vector X is according to a row orlength signal. 0 is a the convolution operator. (1 2) denotes column on Fig.2 (d) before removing extra region, R' isthe decimation by a factor of 2. according to a row or column on Fig.2 (c), and T''1 is the

Fig.2 shows the whole step of my proposal. How to realize inverse matrix of filter matrix T'.deconvolution is explained at first in the following because itgives some suggestion about our estimation, and then, the R x=T'X (5).implementation of the estimation is the second term. The lastpart is the input extension. X= T-1. R' (6).

A. DeconvolutionWe think deconvolution on 1-D signal because 2-D The example of (5) with N=2 and M=6 is Equation (7) which is

deconvolution is realized by applying 1-D deconvolution to removed head data of R because L is odd; L =2 + 6 -1 =7.row and column direction. Equation (2) is convolutionexpressed by a matrix multiplication. A result vector of rj' f 0 0 0 0 XOconvolution is R which length is L and L = N +M -1 r' 0 1 0° ° XIT denotes a L x M matrix which contains elements of FLA . 2 fAn element of matrix T is shown as t(x, y) and f, is an r3' 0 0 f fo 0 0 X2 (7).element of FLA, see Equation (3). 14 0 0 0 A f° 0 X3

R~~~T*X (2). 1~~~~~~~~~~r 0 0 0 0l f f0 x4R = T *X (2). r5' 0 0 0 0 0 f° X4

r') "I° ° ° j X5j

O,x<y

t(X, y) = ,A y . X . y ± N-i1 n = ±+1- x~(3), For calculating the 2-D case, an original image is achieved by{f 0,xn ± N solving SLE about column and row direction.

B. Estimationnote: (x,y)EZ, 0<x<L-i1,0.<y.<M -i1. Firstly, we think about the result vector R' ={tr/}

1'=0,i,---,L'-i1. (Note: the suffix 1' of element of R' is

2008 International Symposium on Communications and Information Technologies (ISCIT 2008) 155

and B by the result of testing the width of range 0 to 15. Thepixel value 128 is chosen as the source image of region B. Thisvalue is selected by minimizing the maximum difference in the

LL A border of region A and B. The estimation is applied to theextended input, and then, finally, extra pixels which come from

A the extension, are removed after the deconvolution.B

Fig. 3. Input extension. LL is the region of inputted low III. EXPERIMENT AND RESULTresolution image. Region A where is filled symmetrically We tested following 4 kinds 1-D interpolation methods inabout border to inputted image. Region B is filled by fixed this experiment; linear, cubic hermite, cubic spline and FFTvalue which comes from approximate subband of 4x4 constant based interpolation. The width of region A and B are set 2 andpixel value images. 7. The tested images were Lena, Baboon, Peppers and Zone

plate which were 512 x 512 size gray scale images. These 3standard test images were gotten from the term "Standard test

renumbered for starting at 0. An approximate subband can be image"o ikiped aere onverte to graysareimag, , image" on Wikipedia, and were converted to grayscale imageobtained by removing element r/ 1 2k-i by Matlab function rgb2gray. The input 256 x 256 images werek =0,1, L' /2. Hence, we must estimate these decimated LL subband which generated from an original image by Haarvalues from approximate subband. Value rr2 k-1 'wavelet transform.0 < k < (L' /2) - 1, must be interpolated, and only one value For confirming effectiveness of proposed method, we testedrL-1 must be extrapolated. bicubic interpolation (note: when applying bicubicNow, we think about interpolation. Normally, we consider

the result of two dimensional wavelet transform is in the interpation w as i. nput image iz t thsame range of original image, because input image iS not inspatial-frequency domain. But, the wavelet subband can be the range of 0 to 255), zero filling method which set zerosassumed to be still in the spatial domain which is just down coefficients to all detail subbands, and regression analysissampled, because each wavelet subbands are obtained by 2-D based wavelet image interpolation method which is proposedconvolution and down sampling. Hence, we can apply general by Temizel et al at 2005 [3]. The results PSNR values and the1-D or 2-D interpolation methods to our estimation, and then, mean time of ten executions are shown in Table 1. The meanthe results of interpolation are able to be considered as the low- time of ten execution is shown as "Mean time" in Table 1, andpass filtered images. its unit is seconds. We mesured the mean time of ten execution

Extrapolated data has different feature and it is very about all images and methods, but its have no differentimportant, see equation (7), because extrapolated data has only between images. All tested methods were implemented asone term. We should use another method between interpolation functions of Matlab. The rows which is marked as BWT areand extrapolation. A feature of wavelet transform is used for the result of our porposal with the used interpolation method.extrapolation. When coefficients of n level and n+1 level Subjective evaluation is shown as Fig. 4. The result of linearwavelet transform are plotted, fairly similar waveforms are and zero filling method is the same not only in subjective, butobserved if magnitudes and number of data are ignored. For also the mean square error between these two results was 0.this reason, we think wavelet transform keep the ratio of n data. Noise on the result of BWT using FFT based interpolation isTwo mean ratio between end and end-I of each column and like a mesh. BWT using cubic spline interpolation is betterrow of the convolution result are calculated by applying than the others, but diagonal lines of hat are not smooth yet.convolution to input image. Extrapolated data is achieved by Bicubic interpolation lacks sharpness, and its representation ofmultiplying the end data of input and this ratio. eyelash is worse than the BWT using cubic spline.

In this experiment, we tested 4 kinds of 1-D well known In addition, the result based for choosing the width of inputinterpolation methods, hence, above Interpolation and extension region A and B is shown as Table 2.extrapolation are applied to each row and column for treating2-D signal.. IV. DISCUSSION

C. Input extension The values, which are decimated by factor of 2 when obtainThe input image is extended by filling symmetrically about an approximate subband from convolution result and which are

border to the input and the fixed value which comes from not on the exactly middle of backward and forward value, canapproximate subband of 4x4 constant pixel values image, see be considered as the high frequency information, which isFig. 3. The region A is for avoiding noise in the region of according to the detail subbands, in our method. Because, theoriginal. Region B stabilizes the result of deconvolution. The zero filling method is considered that it have no thighboth regions should have enough width to avoid noise and frequency information because of using zeros coefficients asstabilizing result, because results of a few interpolation all detail subbands, and then, the results of our method withmethods generate some noise near the points where pixel linear interpolation equal to the result of zero filling method.values change large. We set 2 and 7 as the width of region A The feature of linear interpolation is that the estimated values

156 2008 International Symposium on Communications and Information Technologies (ISCIT 2008)

Table 1. PSNR comparison of 4 tested images (dB) and mean time of ten executions (sec).

Baboon Lena Peppers Zone plate Mean time (sec)

Linear BWT 23.35 31.56 36.12 10.30 1.9

Cubic hermite BWT 18.49 27.42 31.65 8.78 2.6

Cubic spline BWT 23.99 35.32 39.27 10.04 3.4

FFT BWT 21.42 27.04 23.33 9.51 1.8

Regression IWT 23.28 32.58 36.38 10.30 1.3

Zero filling IWT 23.35 31.56 36.12 10.30 0.1

Bicubic Spatial 23.12 33.26 31.09 10.09 0.7

(a) Original (b) Linear: 31.56 dB (c) Cubic hermite: 27.42 dB (d) Cubic spline: 35.32 dB

(e) FFT: 27.04 dB (f) Rigression: 32.58 dB (g) Zero filling: 31.56 dB (h) Bicubic: 33.26 dBanalyais

Fig. 4. Subjective evaluation of image Lena. Images are extracted from the 512 x 512 result images.

are always in the exactly middle value of the backward and other three interpolation methods in our experiment are not inforward value. On the other hand, interpolated values of the the exactly middle value of the backward and forward value.

2008 International Symposium on Communications and Information Technologies (ISCIT 2008) 157

Table 2. PSNR comparison of different input extension on Lena (dB).A and B mean the width of input extension region A and B.

0 1 2 3 4 5 6 7 8 9 100 31.45 16.97 19.59 25.82 33.27 35.05 35.19 35.20 35.20 35.21 35.201 29.50 16.70 19.61 25.84 33.39 35.14 35.30 35.31 35.32 35.31 35.312 32.74 16.89 19.61 25.82 33.38 35.15 35.31 35.32 35.32 35.32 35.323 31.56 16.99 19.63 25.83 33.38 35.15 35.31 35.32 35.32 35.32 35.32

A 4 B 31.60 16.89 19.57 25.78 33.36 35.14 35.31 35.32 35.32 35.32 35.325 31.54 16.92 19.61 25.80 33.37 35.15 35.31 35.32 35.32 35.32 35.326 32.20 16.82 19.56 25.77 33.36 35.15 35.31 35.32 35.32 35.32 35.327 31.26 16.84 19.62 25.82 33.37 35.15 35.31 35.32 35.32 35.32 35.328 32.22 16.99 19.78 25.95 33.43 35.16 35.31 35.32 35.32 35.32 35.329 32.22 16.87 19.71 25.91 33.41 35.15 35.31 35.32 35.32 35.32 35.3210 30.99 16.76 19.66 25.88 33.39 35.15 35.31 35.32 35.32 35.32 35.32

Especially, the waveforms of the result of cubic spline Only Haar wavelet is tested in this paper. Another kind ofinterpolation sometimes have overshoot and undershoot. That wavelets such as Daubechies 7/9 should be tested in the future,is to say, estimated values are out of the range of the backward but, we think concentrate Haar wavelet is not bad idea by theand forward value. We think these values represent the high reasons of following two. At first, in the wavelet based imagefrequency information. In the fact, cubic spline shows better interpolation field, wavelet type is only related to the easinessPSNR values than other methods excepting Zone plate in our of estimation. If the estimation is perfect, an original image isexperiment. Hence, an essential purpose of our method is how gotten by every proposed methods and/or every wavelet type.to estimate the values which are not exactly middle point of In fact, original image is represented from the true convolutionthe backward and forward value. result of the Haar analysis low-pass filter by our deconvolution.The tested wavelet based methods have the same trend on Hence, we think that wavelet type is not so important, and then,

the PSNR values. The ascending order of PSNR values of the essential problem in wavelet based image interpolation filed istested image is Zone plate, Baboon, Lena and Peppers. This the matching between the wavelet type and estimation method.order suggests that wavelet based image interpolation methods The second is simplicity of Haar wavelet. Its filter length isare suitable for images which contain strong low frequency only two. So using Haar wavelet is advantage for hardwarecomponents. But traditional bicubic interpolation is weak to implementation such as FPGA. Because our deconvolutionrepresent low frequency components, because the PSNR value matrix contains many zero values by the reason of using onlyof Lena is less than the peppers in the case of bicubic. On the analysis low pass filter. Hence, we think that improving Haarother hand, all tested methods are weak to zone plate image. wavelet based method has big impact in practical problem.This image contains very strong high frequency components The problem of Haar wavelet is also its simplicity. Thatbecause of being created by sinusoid functions. How to filter length is only two. So, the relationship between therepresent such a high frequency is an important feature work. coefficients of Haar wavelet is less than the others'. It leads

Seeing Table 2, it is known that the width of input extension difficulty of estimating lost information. In fact, the result ofregion B influence PSNR strongly, and the effect of the width IWT based method using regression analysis in our experimentof region A is small. But, unfortunately, we don't understand is worse than the result of the paper [3] in the case of usingwhy the input extension leads good results. In this proposal, Daubechies 7/9 filter bank. But we think that possibility ofonly 1-D well known interpolation methods are used. These Haar wavelet is shown by our experimental results.interpolation methods are not under the influence on farther Assuming coefficients of an approximate subband ofpoints. Hence the interpolation results are almost the same wavelet transform as low resolution image is a little bitbetween an extended input and a not extended input on the unpractical. Normally, when we resize a low resolution imageregion of original approximate subband. The difference is only to high resolution, we can not know the true pixel values ofon the extended region and near the tail region of input. This high resolution. Saying other words, we are impossible tofact means that our input extension don't give weak effect for know the correct coefficients of an approximate subband of aestimation. But it influences our deconvolution largely. high resolution image. Making wavelet based methods moreMaking clear the mechanism of effectiveness of input practical, we need to develop a method to estimate coefficientsextension is one of the future works. of approximate subband from normal image which pixel values

may be in the range of 0 to 255.

158 2008 International Symposium on Communications and Information Technologies (ISCIT 2008)

V. CONCLUSION

We proposed the BWT-based image interpolation methodwhich consists of the estimation of an undecimatedapproximate subband, the deconvolution of the estimationresult by solving SLE, and input extension. Cubic spline is themost suitable interpolation method for our proposal in thisexperiment. The best results in PSNR are 39.27 dB done on thePeppers image with cubic spline interpolation.

REFERENCES

[1] K. Kinebuchi, D.D. Muresan, T.W. Parks, "Imageinterpolation using wavelet based hidden Markov trees",IEEE Int. Conf. ICASSP 2001, Volume 3, pp.1957-1960,7-11 May 2001

[2] Sang Soo Kim, II Kyu Eom, Yoo Shin Kim, "ImageInterpolation Based on Statistical Relationship BetweenWavelet Subbands", IEEE Int. Conf. Multimedia and Expo2007, pp.1723-1726, 2-5 July 2007

[3] A. Temizel and T. Vlachos, "Wavelet domain imageresolution enhancement", IEE Proceedings, Vision, Imageand Signal Processing, Volume 153, Issue 1, pp. 25-30, 9Feb. 2006

[4] S.G. Chang, Z. Cvetkovic, M. Vetterli, "Locally adaptivewavelet-based image interpolation", IEEE Trans. ImageProcessing, Volume 15, Issue 6, pp. 1471-1485, June 2006

[5] W.K. Carey, D.B. Chuang, S.S. Hemami, "Regularity-Preserving Image Interpolation", IEEE Trans. ImageProcessing, Volume 8, Issue 9, pp. 1293-1297, Sept. 1999

[6] M.S. Crouse, R.D. Nowak, R.G. Baraniuk, "Wavelet-basedstatistical signal processing using hidden Markov models",IEEE Trans. Signal Processing, Volume 46, Issue 4, pp.886-902, April 1998

[7] S. Matsuyama "Data Interpolation for the Discrete WaveletTransform ", Research Publications, The Research Centerfor Computing and Multimedia Studies, Hosei University,Vol. 16, 2003