Upload
hiroaki
View
216
Download
2
Embed Size (px)
Citation preview
Deblurring for Enlarged ImageShotaro Moriya, Satoshi Yamanaka, Yoshiki Ono, Koji Minami, and Hiroaki Sugiura
Mitsubishi Electric CorporationAdvanced Technology R&D Center
Kyoto, Japan
Abstract—When you use a high resolution display, input imagesare enlarged in some cases. Many chips designed for digital TVsand displays enlarge input images by linear methods and outputblurred images. Therefore a deblurring method for an enlargedimage is desired. We propose a new deblurring method for anenlarged image. Experimental results show the proposed methodoutputs a visually pleasant image.
I. INTRODUCTION
When you use a high resolution display, input images areenlarged in some cases. Watching a DVD video on an LCDTV ready for HDTV is a typical example. If the input imageis enlarged by a Linear enlargement method such as Bicubic[1] and Bilinear, an output image seems blurred becauseit cannot generate a higher frequency component than theNyquist frequency of the input image.
Sophisticated enlargement methods such as super resolutioncould solve this problem [2]. However many LSIs designed forconsumer products implement linear enlargement methods andthey output blurred images. Therefore signal processing thatdeblurs an enlarged image is desired.
Greenspan suggested two kinds of signal processing basedon the Laplacian pyramid [3], [4]. One is an enlargementmethod and the other is a deblurring method. His enlargementmethod is improved by some researchers [5]–[7]. This reportfocuses on his deblurring method and propose a new methodthat deblurs an enlarged image.
II. CONVENTIONAL METHOD
A. Greenspan’s Method
The Laplacian pyramid is a sequence of high frequencycomponent images obtained by a multiresolution analysis [8].If there is an edge in an input image, there are zero crossingpoints at the same points in these high frequency componentimages and these images have a self-similarity around thezero crossing points [3], [4]. Greenspan thought if he founda nonlinear process that approximated to this self-similarity,a high frequency component that a blurred image didn’t havecould be predicted by the nonlinear process. His nonlinearprocess is composed of two high pass filters (HPFs) and aclipping process. Let G0(x, y) denote a blurred image, TheGreenspan’s deblurring method is described as follows.
1) Extract a high frequency component L0(x, y) by highpass filtering G0(x, y) by a first HPF.
2) Create Lclip0 (x, y) by clipping L0(x, y). This process is
expressed as
Lclip0 (x, y) =
⎧⎪⎨⎪⎩
T if(L0(x, y) > T )
−T if(L0(x, y) < −T )
L0(x, y) otherwise
(1)
where T is a threshold.3) Extract a high frequency component L−1(x, y) by high
pass filtering Lclip0 (x, y) by a second HPF.
4) Create deblurred image G−1(x, y) by adding G0(x, y)and L−1(x, y).
B. Problems
Greenspan’s deblurring method has some problems whenit is applied to a linearly enlarged image. Fig. 1 depicts ablock diagram of a linear upscaler. It is expressed as anupsampler followed by a low pass filter (LPF). Fig. 2a and2b show Fourier spectrum of an input signal to the upscalerand an upsampled signal respectively. The Fourier spectrumof the upsampled signal is a periodic function because theupsampler replicates the Fourier spectrum of the input signal.In other words, the upsampler generates imaging componentsin the frequency domain [9]. The following LPF aims to cutthese imaging components. If the LPF is ideal, the imagingcomponents are removed completely as shown in Fig. 2c.But it is impossible to realize an ideal LPF and some of theimaging components remain as alias signals like Fig. 2d. Ifyou apply the Greenspan’s deblurring method to an enlargedimage, its first HPF extracts these alias signals. This is a firstproblem.
Greenspan’s method uses the clipping process as a nonlinearprocess, and its recommended value for the threshold T in(1) is given by multiplying the maximum value of L0 by aconstant value [4]. However this threshold is not effective formany natural images because the clipping process works foran edge with the maximum contrast and some edges withclose contrasts to it but doesn’t work for edges with smallcontrasts [5]. If the clipping process doesn’t work, his methodis equal to a combination of two HPFs and extracts the imagingcomponents only. Therefore it cannot make an effective highfrequency component. This is a second problem.
Additionally, even though the clipping process works, thesecond HPF that follows the clipping process might causeringing [6]. This is a third problem.
2011 IEEE International Conference on Consumer Electronics - Berlin (ICCE-Berlin)
978-1-4577-0234-1/11/$26.00 ©2011 IEEE 249
Fig. 1. Diagram of an upscaler.
(a) Input signal. (b) Upsampled signal.
(c) Upsampled signal followed byideal LPF.
(d) Upsampled signal followed bynon-ideal LPF.
Fig. 2. Fourier spectrum. Fn means the Nyquist frequency of the inputsignal.
III. PROPOSED METHOD
In order to solve the first problem, we use a band pass filter(BPF) instead of the first HPF. The alias signals generated byan enlargement are cut by the BPF. Also we use another non-linear process reported by authors [7] instead of clipping. Thisprocess works for every zero crossing point, i.e. every edge.Therefore this process has stronger effect than Greenspan’smethod. The proposed method is described as follows.
1) Extract a horizontal frequency component HL0(x, y) byband pass filtering G0(x, y) by a horizontal BPF.
2) Create a nonlinearly processed signal HLnl0 (x, y) by
making slopes in the neighborhoods of zero crossingpoints in HL0(x, y) steeper. HLnl
0 (x, y) is given by
HLnl0 (x,y)=
⎧⎪⎨⎪⎩
k·HL0(x,y) if(HL0(x,y)·HL0(x+1,y)<0)
k·HL0(x,y) if(HL0(x,y)·HL0(x−1,y)<0)
HL0(x,y) otherwise(2)
where k is a constant.3) Create a horizontal high frequency component
HL−1(x, y) by high pass filtering HLnl0 (x, y).
4) Extract a vertical frequency component VL0(x, y) byband pass filtering G0(x, y) by a vertical BPF.
5) Create a nonlinearly processed signal VLnl0 (x, y) by
making slopes in the neighborhoods of zero crossing
(a) Part of original image.
(b) Part of GB0 , i.e. enlarged image by Bicubic.
(c) Part of GG−1, i.e. deblurred image by Greenspan’s method.
(d) Part of GP−1, i.e. deblurred image by the proposed method.
Fig. 3. Comparative results of the original image, GB0 , GG
−1, and GP−1.
These images are based on N6RGB(ISO-sRGB) in SCID [10].
points in VL0(x, y) steeper. VLnl0 (x, y) is given by
VLnl0 (x,y)=
⎧⎪⎨⎪⎩
k·VL0(x,y) if(VL0(x,y)·VL0(x+1,y)<0)
k·VL0(x,y) if(VL0(x,y)·VL0(x−1,y)<0)
VL0(x,y) otherwise(3)
6) Create a vertical high frequency component VL−1(x, y)by high pass filtering VLnl
0 (x, y).7) Create deblurred image G−1(x, y) by adding G0(x, y),
HL−1(x, y), and VL−1(x, y).
IV. EXPERIMENTAL RESULTS
We carried out an image simulation composed of followingsteps.
1) Reduce the width and the height of an original imageby half. We use the Bicubic method to resize image.
2) Make a blurred image GB0 by enlarging the reduced
image twice by Bicubic.3) Create a deblurred image GG
−1 by applying theGreenspan’s method to GB
0 .
250
−2Fn −Fn 0 Fn 2Fn
−6
−4
−2
0
Frequency
Four
ier
Spec
trum
( lo
g sc
ale
)
BicubicGreenspanProposed
(a) Horizontal Fourier spectra.
−2Fn −Fn 0 Fn 2Fn
−6
−4
−2
0
Frequency
Four
ier
Spec
trum
( lo
g sc
ale
)
BicubicGreenspanProposed
(b) Vertical Fourier spectra.
Fig. 4. Comparative results of Fourier spectra. The dash line with circles (◦) is a Fourier spectrum of GB0 . The dash line with plus signs (+) is a Fourier
spectrum of GG−1. The dash line with dots (·) is a Fourier spectrum of GP
−1. Fn is the Nyquist frequency of the reduced image.
4) Create a deblurred image GP−1 by applying the proposed
method to GB0 .
5) Compare GB0 , GG
−1, and GP−1.
Fig. 3 shows comparative results of the original image,GB
0 , GG−1, and GP
−1. We use a luminance component ofN6RGB(ISO-sRGB) in SCID [10] as the original image. Colorconversion from RGB to luminance(Y) is expressed as [11]
Y = 0.299R+ 0.587G+ 0.114B (4)
GB0 seems blurred the most. GG−1 seems the blockiest. GP−1 is
visually pleasant the most.Fig. 4 show comparative results of horizontal Fourier spectra
and vertical Fourier spectra of GB0 , GG−1, and GP−1. A hori-
zontal Fourier Spectrum HH and a vertical Fourier SpectrumHV are calculated by
HH(fH) =∑fH
H(fH , fV ) (5)
HV (fV ) =∑fH
H(fH , fV ) (6)
where H(fH , fV ), fH , and fV mean a Fourier spectrum of adigital image, a horizontal frequency, and a vertical frequencyrespectively.
When you observe the Fourier spectrum of GB0 , you can
find local peaks in areas where fH or fV are larger thanFn or smaller than −Fn. These peaks are originally fromimaging components. These peaks becomes higher in theFourier spectrum of GG
−1. This means that Greenspan’s methodincreases the imaging component of GB
0 and is not suited foran enlarged image. On the other hand, these local peaks arevanished in the Fourier spectrum of GP
−1. This means thatthe proposed method generates a new frequency component
without increasing the imaging component. Thus the proposedmethod is preferable to Greenspan’s method.
V. CONCLUSION
We propose a new deblurring method that is suitable for anenlarged image. The experimental results show the proposedmethod outputs a visually pleasant image. Additionally it isconfirmed that the proposed method is able to generate aneffective high frequency component without increasing animaging component that an enlarged has in a Fourier spectrum.
REFERENCES
[1] R. G. Keys, “Cubic convolution interpolation for digital image process-ing,” IEEE Trans. Acoust., Speech, Signal Process., vol. 29, no. 6, pp.1153–1160, Dec. 1981.
[2] S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution imagereconstruction : a technical overview,” IEEE Signal Process. Mag.,vol. 20, no. 3, pp. 21–36, May 2003.
[3] H. Greenspan and C. H. Anderson, “Image enhancement by non-linearextrapolation in frequency space,” in Proc. SPIE Image and Video II,Feb. 1994, pp. 2–13.
[4] H. Greenspan, C. H. Anderson, and S. Akber, “Image enhancementby nonlinear extrapolation in frequency space,” IEEE Trans. ImageProcess., vol. 9, no. 6, pp. 1035–1048, Jun. 2000.
[5] M. Shimizu, M. Nakashizuka, and Y. Iiguni, “Image enhancement andzooming by morphological operations in laplacian pyramid representa-tion,” IEICE Technical Report, vol. 106, no. 260, pp. 31–36, Sep. 2006.
[6] A. Shimura, K. Arakawa, and R. Taguchi, “A digital image enlargingmethod without edge effect by using epsilon-filter,” The Transactions ofthe Institute of Electronics, Information and Communication Engineers.A, vol. J86-A, no. 5, pp. 540–551, May 2003.
[7] S. Moriya, S. Yamanaka, S. Minami, and H. Sugiura, “Resolution en-hancement based on laplacian pyramid,” IEEE Trans. Consum. Electron.,vol. 56, no. 3, pp. 1830–1836, Aug. 2010.
[8] P. J. Burt and E. H. Adelson, “The laplacian pyramid as a compactimage code,” IEEE Trans. Commun., vol. 31, no. 4, pp. 532–540, Apr.1983.
[9] L. Tan, Digital Signal Processing : Fundamentals and Applications.Academic Press, 2008.
251
[10] Graphic Technology – Prepress Digital Data Exchange – StandardColour Image Data (XYX/SCID), Japanese Standards Association JISX 9204:2004, 2004.
[11] Studio Encoding Parameters of Digital Television for Standard 4:3 andWide Screen 16:9 Aspect Ratios, ITU-R Recommendations BT.601-6,2007.
252