Upload
lynguyet
View
224
Download
3
Embed Size (px)
Citation preview
An Adaptive Richardson-Lucy Algorithm for Single
Image Deblurring Using Local Extrema Filtering
Jiunn-Lin Wu*, Chia-Feng Chang and Chun-Shih Chen
Department of Computer Science and Engineering, National Chung Hsing University,
Taichung, Taiwan 402, R.O.C.
Abstract
Motion Blur is one of the common artifacts in digital photographing. With the population of
handheld camera and smart phone, image deblurring becomes an important problem. Richardson-
Lucy algorithm is well-known deconvolution algorithm. But the ringing artifacts usually appear while
the estimated point spread function is not accurate. In this paper, we proposed an improved
Richardson-Lucy deconvolution algorithm. Before deconvolution step, we separate the blurred image
into smooth part and edge part which is called the edge map. The blurred image and edge map are then
used for image deblurring. By using the proposed edge map, the ringing artifacts in the deblurred
image are significantly reduced while preserving the sharp edge information.
Key Words: Motion Blur, Deconvolution, Richardson-Lucy, Ringing Artifacts, Edge Map
1. Introduction
Motion Blur is common artifact which produces dis-
appointing blurred image under dim light in digital pho-
tography. It is caused by camera shaking or the target
object moving during exposure time. Recently, handheld
camera and smart phone are getting more and more po-
pular; restoring a blurred image is becoming an important
issue. Figure 1 shows the example of image deblurring.
In general, motion blur can be separated into two
types. The first one is camera shake caused the image
blurred; the second one is the target object moving. A
blurred image can be viewed as an unblurred image con-
volution with point spread function (PSF) which is the
movement of camera or the movement of target object. If
the point spread function is shift-invariant, the image
deblurring problem can be transformed into image de-
convolution. According to the point spread function is
known or not, the deconvolution can be classified into
non-blind deconvolution [1�4] and blind deconvolution
[5�8]. Blind deconvolution is an ill-pose problem, it uses
only blurred image to restore image. Therefore image-
pair deblurring method [9,10] is proposed to solve mo-
tion blur problem, but the additional image is hard to ac-
quire. Nonblind deconvolution uses known point spread
function and blurred image to restore deblurred image.
Wiener filter [11] and Richardson-Lucy deconvolution
[2] are well-known nonblind deconvolution algorithms
which are good at image deblurring. But if the point
spread function is not accurate, the deblurring result will
appear undesired ringing artifacts at the smooth region
around edges. To solve this problem, Wang et al. [3] pro-
posed using different weight values for smooth region
and edge region during the deblurring process to sup-
press ringing. But it still failed around some edges. Zhao
et al. [12] suggested that using only two weight values
for describing smoothness information of image is not
enough.
Journal of Applied Science and Engineering, Vol. 16, No. 3, pp. 269�276 (2013) DOI: 10.6180/jase.2013.16.3.06
*Corresponding author. E-mail: [email protected] Figure 1. Example of image deblurring.
In this paper, we proposed an adaptive image de-
blurring algorithm using local extrema filtering. To re-
duce ringing artifacts from deblurred image, the ex-
trema-based multiscale decomposition filter is used to
distinguish the edge region and smooth region, and then
edge map is obtained. According to Zhao et al. sugges-
tion, it is better that existing more than two weight values
in edge map instead of only two weight values in edge
map. Thus we use blurred image and edge map which
contains more than two weight values are both used to
deconvolution procedure. Finally the deblurred image is
obtained with less ringing.
The rest of the paper are organized as follows. Re-
lated works which is mentioned in Introduction are de-
scribed in detail. Then according to related works our
proposed method is presented. And then experiments are
designed for verifying our deblurring algorithm. Finally,
we conclude this paper.
2. Related Works
In general, a blurred image can be modeled as
(1)
where I is blurred image; L is latent unblurred image; k
is point spread function; n is noise and � is convolution
operator. According to point spread function is known
or not, image deconvolution can be separated into non-
blind deconvolution and blind deconvolution.
Richardson-Lucy algorithm [2] is a well-known it-
erative deconvolution method for image deblurring. A
more clearly deblurred image is generated for each time
of iteration. After several iterations, a sharp and un-
blurred result image can be obtained. According to Bayes’
theorem, (1) can be transform into
(2)
Then the formulation of Richardson-Lucy algorithm
can be defined as
(3)
where * is correlation coefficient operator, � is convo-
lution operator, t is the tth of iteration, Lt is current itera-
tion deblurred image, Lt+1 is the next iteration deblurred
image, k is PSF and I is blurred image. If the point
spread function is not accurate enough, the ringing ar-
tifacts will appear in the result image. Figure 2 shows
that if the point spread function is not precise enough,
then the ringing artifacts will be generated in the result
image. Therefore, to reduce ringing artifacts is an im-
portant issue for image deblurring. Several methods
have been proposed to solve this problem.
Shan et al. [13] suggested that the smooth region in
the blurred image is also smooth in deblurred image.
Therefore, Shan included a constraint about the smooth
region called local prior during the restoration process.
The formulation of local prior is defined as
(4)
where i is pixel position in image, L is latent unblurred
image, I is blurred image � is the standard deviation and
� is smooth region in image. Figure 3 shows the corre-
sponding smooth region of original image.
For this reason Wang et al. proposed improved Rich-
ardson-Lucy algorithm using local prior. It includes an
adaptive weight map called edge map during deblurring
process. Before deconvolution, the blurred image is used
to extract the edge map which indicates where the smooth
or edge region it. And then the edge map and blurred
270 Jiunn-Lin Wu et al.
Figure 2. Example of the ringing artifacts. (a) Original im-age. (b) Blurred image. (c) The result of Richard-son-Lucy algorithm.
Figure 3. Example of edge map. (a) Blurred Image. (b) Thecorresponding smooth region of blurred image.
image are used as inputs for each time of iteration. When
the ringing is appear during deconvolution process, the
edge map will reduce the effect of ringing by assigning
the smooth region where ringing appeared large weight
to update it. Then the ringing artifacts can be efficiently
reduced from result.
From Figure 4, we can notice that Wang’s method
reduces most ringing artifacts, but the ringing artifacts
around the region where between two edges is not re-
duced successfully. In next section, we will discuss this
problem, and then proposed our deblurring algorithm.
3. Proposed Method
In this section, we will propose an adaptive Richard-
son-Lucy deconvolution algorithm using local extrema
filtering. Observe on the result which is deblurred by
Wang et al, the region between two edges is not fully
described by Wang’s edge map. Consequently, the ring-
ing artifacts still exist in deconvolution result. To get an
appropriate edge map is an important issue for obtaining
a good deconvolution result. As a result, there are two
problems for extracting edge map, first is well descrip-
tion edge map about the smooth and edge region; the
other is only two weight values in edge map is not enough.
Extrema-based multiscale decomposition (EMD) [14]
is good at separating detail and mean layer. Compare
with existing edge-preserving image decomposition al-
gorithms, EMD can extract the detail from input image
while preserving edge mean and contrast properly. Its
idea is getting the local extrema such that maxima or
minima, and then getting the mean layer by averaging
maximum layer and minima layer, the mean layer is also
called smooth layer. Then we extract the detail layer by
subtracting smooth layer from input image.
As mentioned before the edge map must have good
description at smooth region, thus we improved EMD
algorithm for purpose. First EMD is used for separating
the detail layer and mean layer, and then the contour is
extracted from mean layer by calculate the gradient of
input image, called smooth map. For further edge map
information, the gradient of detail layer is also used for
describing the detail part called detail map. Finally,
Combining the detail map into smooth map, then the
result map is the proposed edge map. The flowchart of
our improved EMD method is shown in Figure 6.
In Figure 7(a), we can observe that the smooth map
contains most contours, so we can extract the contours
which are strong edges (Figure 7(c)) from smooth map.
And the detail map (Figure 7(b)) contains most texture
information, thus we extract further detail information
(Figure 7(d)) from detail map. But the detail map also
contains undesired noise information, so a simple thre-
shold method is used for filtering out the noise. Compare
proposed edge map with the edge map extracted by
Wang’s method in Figure 8; the region between two
strong edges is clear defined in our proposed edge map.
Wang’s edge map only separate the blurred image into
smooth region and edge region, it is too strict to define
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring 271
Figure 4. Comparison of deblurring results. (a) The resultusing Richardson-Lucy. (b) The result using Wang’smethod. (c) The corresponding magnitude of (a).(d) The corresponding magnitude of (b).
Figure 5. Overview of extrema-based multiscale decomposi-tion algorithm [12].
the edge map using only two weight values. For exam-
ple, some regions contain ringing artifacts or noise may
be classified as an edge region. As a result, the ringing
artifacts will still exist in result image. On the other hand,
our edge map defines the smooth region and edge region
more flexible, so the deblurring result can be reduced
more ringing artifacts.
After obtained a reasonable edge map, blurred image
and edge map are both used as inputs for deconvolution.
During the deconvolution process, the edge map is used
as the local prior of Richardson-Lucy algorithm, and then
the cost function Epl (L) = �log(pl (L)) can be defined as
(5)
where L is latent unblurred image, I is blurred image,
M is edge map which describes the smoothness infor-
mation of the image, �x is the partial derivative operator
in x direction and �y is partial derivative operator in y
direction.
Combine (5) with Richardson-Lucy algorithm, it can
be defined as
(6)
where �Epl (Lt) is defined as
(7)
where * is correlation coefficient operator, �x* is conju-
gate matrix of �x, �y* is conjugate matrix of �y, t is the
number of iteration, Lt is current deblurred image, Lt+1
is the deblurred image of next iteration, k is PSF, I is
blurred image, M is edge map and � is the predefined
parameter. The comparison of the proposed method
with Wang’s approach is shown in Figure 9; we observe
that the ringing artifacts within red rectangle are sig-
272 Jiunn-Lin Wu et al.
Figure 6. Flowchart of our improved EMD method. Figure 7. Intermediate results of our improved EMD. (a)Smooth map. (b) Detail map. (c) The result of gra-dient of smooth map and then use threshold me-thod. (d) The gradient of detail map.
Figure 8. Comparison between our method and others. (a)Proposed edge map. (b) Wang’s edge map.
nificantly reduced in our result. In the other word, our
deblurred image has better visual result than Wang’s
method with similar PSNR. It is obvious that the ring-
ing artifacts can be efficiently reduced by using the pro-
posed edge map in the deconvolution step. The flow-
chart of our proposed method is shown in Figure 10.
4. Experiments and Discussions
The aim of the proposed method is to reduce the
ringing artifacts in the deblurred images. Therefore, in
this section, we will use several well-known natural im-
ages which failed in Richardson-Lucy algorithm, and the
synthesis images to demonstrate the effectiveness of the
proposed method.
For all test images, at first the traditional Richard-
son-Lucy algorithm is used as a failed example with
ringing artifacts. Then several deconvolution methods
with edge map are used to recover blurred image and re-
duce ringing artifacts. Then we will compare our result
with the traditional Richardson-Lucy result and other
methods in detail. For the first experiment, we use the
blurred image “old man” which its size is 800 � 532 and
PSF which its size is 27 � 27 to verify our method. There
are many wrinkles on the old man face and hands, so we
can observe these two parts to verify our method. Figure
11 shows the undesired result of Richardson-Lucy algo-
rithm. And then we compare our result and edge map
with Wang’s method in Figure 12. We can observe that
compare with Wang’s method, our edge map provides
more edge information. Therefore we can reduce more
ringing artifact while preserving the edge.
Figure 13 is a magnitude view of our result and
others. Compare with traditional Richardson-Lucy al-
gorithm, our proposed method and Wang’s method can
efficient reduce the ringing artifacts, but our method has
better image vision than Wang’s. Next experiment we
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring 273
Figure 9. Comparison of deconvolution result (a) Originalimage (512 � 512). (b) PSF (21 � 21). (c) Wang’method with 40 times iteration (PSNR is 27.16). (d)Our proposed method with 40 times iteration(PSNR is 26.72).
Figure 10. The flowchart of our proposed method.
Figure 11. The result of traditional Richardson-Lucy decon-volution algorithm using “Old man”. (a) Blurredimage and PSF. (b) Deblurred result.
use blurred image “flower” which its size is 701 � 494
and bigger PSF which its size is 35 � 35 to verify our
method. The “flower” has more texture then “old man”.
Figure 14 is the result of Richardson-Lucy algorithm.
We can observe that Figure 15(a) has less ringing ar-
tifacts, but some region is failed to recover sharp edge. In
contrast, our result in Figure 15(c) has sharper result.
From Figure 16, we can see that our method has the best
image vision of the three. Next, we want use an image
which contains a lot of narrow detailed to verify the ro-
bustness of proposed method. So we use he blurred im-
age “doll” which its size is 800 � 558 and bigger PSF
which its size is 41 � 41 to verify our method. Figure 17
is the result of Richardson-Lucy algorithm. From Figure
18, the detail part is well preserved, and the ringing ar-
tifacts are reduced successfully.
274 Jiunn-Lin Wu et al.
Figure 12. Comparison results with other methods using “Oldman” (a) Wang’s result. (b) Wang’s edge map. (c)Our proposed method. (d) Our corresponding edgemap.
Figure 13. Close-ups which is extracted from Figures 11(b)and 12(a), (c). (a) Richardson-Lucy algorithm. (b)Wang’s method. (c) Our proposed method.
Figure 14. The result of traditional Richardson-Lucy decon-volution algorithm using “Flowers”. (a) Blurredimage and PSF. (b) Deblurred result.
Figure 15. Comparison results with other methods using“Flowers” (a) Wang’s result. (b) Wang’s edge map.(c) Our proposed method. (d) Our correspondingedge map.
Figure 16. Close-ups which is extracted from Figures 14(b)and 15(a), (c). (a) Richardson-Lucy algorithm. (b)Wang’s method. (c) Our proposed method.
The close-ups show proposed method reduces more
ringing artifacts where near the contour of the doll’s face
than other deconvolution methods.
The processing time of deblurring using all natural
images is show in Table 1.
Finally, we use the synthesis images shown in Figure
20 to verify our proposed method. The synthesis images
include, “Lena”, “Airplane” and “Fruit”. The image size
of burred “Lena”, burred “Airplane” and burred “Fruit”
are 490 � 490, 486 � 486 and 490 � 458 respectively. The
size of corresponding PSF are 23 � 23, 19 � 19 and 23 �
23 respectively. The result comparisons of the proposed
method with Wang’s algorithm are shown in Figure 20. It
is obvious that the proposed method has better visual re-
sult than Wang’s approach. Table 2 shows that the pro-
posed method has similar PSNR with Wang’s method.
The processing times for deblurring examples is shown in
Table 2. We implemented our method with Visual C++.
Net language. The testing environment was as PC running
Window 7 64 bit version with AMD Phenom II X4 945
3.4 GHz. 4 GB Ram. The processing time of the proposed
method is too long to be waiting for. The FFT (Fast Fou-
rier transform) which is mentioned in many existence
deconvolution methods can be used to solve this problem.
Adaptive Richardson-Lucy Algorithm for Single Image Deblurring 275
Figure 17. The result of traditional Richardson-Lucy decon-volution algorithm using “Doll”. (a) Blurred imageand PSF. (b) Deblurred result.
Figure 18. Comparison results with other methods using“Doll” (a) Wang’s result. (b) Wang’s edge map. (c)Our proposed method. (d) Our corresponding edgemap.
Figure 19. Close-ups which is extracted from Figures 17(b)and 18(a), (c). (a) Richardson-Lucy algorithm. (b)Wang’s method. (c) Our proposed method.
Table 1. The comparison of processing time between the
proposed method and other approaches
Processing time (second)
Image Proposed
method
Wang’s
methodRichardson-Lucy
Old man 610.5927 178.457 139.32
Flower 556.0106 200.890 170.90
Doll 816.6494 342.269 0302.936
Figure 20. Deblurring result of synthesis image. First row:blurred image and its PSF. Second row: the result ofproposed method. Third row: the result of Wang’smethod. (a) “Lena”. (b) “Airplane”. (c) “Fruit”.
To reduce time complexity of the proposed algo-
rithm will be the main task for future work. Our method
shares common limitation with others deconvolution
method. Huge PSF, bright spot, severe noise, and non-
uniform blur will cause the deblurring failed. To solve
these problems in the proposed method will be interest-
ing future work.
5. Conclusions
In this paper, we proposed an adaptive Richardson-
Lucy deconvolution algorithm using local extrema filter-
ing. First, the local extrema filtering called EMD is used
to extract the edge map of the image which contains well
definition on smooth regions, and then the edge map and
blurred image are both used as inputs for deblurring al-
gorithm. With the effort of edge map, the ringing artifact
can be reduced for each time of iteration. Finally, a shaper
deblurred image without ringing can be obtained. The
experiments show that our proposed method can reduce
the ringing artifacts effectively while preserving the edge
of the image.
Acknowledgements
This research is partially supported by National Sci-
ence Council Grant NSC 101-2221-E-005-088.
References
[1] Levin, A., Fergus, R., Durand, F. and Freeman, W. T.,
“Image and Depth from a Conventional Camera with a
Coded Aperture,” ACM Trans. Graph., Vol. 26, No. 3,
p. 70 (2007). doi: 10.1145/1276377.1276464
[2] Richardson, W. H., “Bayesian-Based Iterative Method
of Image Restoration,” JOSA, Vol. 62, No. 1, pp.
55�59 (1972). doi: 10.1364/JOSA.62.000055
[3] Wang, Y., Feng, H., Xu, Z., Li, Q. and Dai, C., “An Im-
proved Richardson-Lucy Algorithm Based on Local
Prior,” Optics & Laser Technology, Vol. 42, No. 5, pp.
845�849 (2010). doi: 10.1016/j.optlastec.2010.01.001
[4] Yuan, L., Sun, J., Quan, L. and Shum, H., “Progressive
Inter-Scale and Intra-Scale Non-Blind Image Decon-
volution,” ACM Trans. Graph., Vol. 27, No. 3 (2008).
doi: 10.1145/1360612.1360673
[5] Levin, A., Weiss, Y., Durand, F. and Freeman, W. T.,
“Understanding and Evaluating Blind Deconvolution
Algorithms,” Proc. CVPR, pp. 1964�1971 (2009). doi:
10.1109/CVPRW.2009.5206815
[6] Kundur, D. and Hatzinakos, D., “Blind Image De-
convolution,” IEEE Signal Processing Magazine, Vol.
13, No. 3, pp. 43�64 (1996). doi: 10.1109/79.489268
[7] Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T.
and Freeman, W. T., “Removing Camera Shake from a
Single Photograph,” ACM Trans. Graph., Vol. 25, No.
3, pp. 787�794 (2006). doi: 10.1145/1141911.1141956
[8] Xu, L. and Jia, J., “Two-Phase Kernel Estimation for
Robust Motion Deblurring,” Lecture Notes in Com-
puter Science, Vol. 6311, Computer Vision � ECCV,
pp. 157�170 (2010). doi: 10.1007/978-3-642-15549-
9_12
[9] Yuan, L., Sun, J., Quan, L. and Shum, H., “Image
Deblurring with Blurred/Noisy Image Pairs,” ACM
Trans. Graph., Vol. 26, No. 3 (2007). doi: 10.1145/
1276377.1276379
[10] Zhuo, S., Guo, D. and Sim, T., “Robust Flash De-
blurring,” in Proc. CVPR, pp. 2440�2447 (2010). doi:
10.1109/CVPR.2010.5539941
[11] Gonzalez, R. C. and Woods, R. E., Digital Image
Processing, 2nd ed., Prentice Hall (2002).
[12] Zhao, J. F., Feng, H. J., Xu, Z. H. and Li, Q., “An Im-
proved Image Restoration Approach Using Adaptive
Local Constraint,” Optik, Vol. 123, pp. 982�985 (2012).
doi: 10.1016/j.ijleo.2011.07.014
[13] Shan, Q., Jia, J. and Agarwala, A., “High-Quality Mo-
tion Deblurring from a Single Image,” ACM Trans.
Graph., Vol. 27, No. 3 (2008). doi: 10.1145/1360612.
1360672
[14] Subr, K., Soler, C. and Durand, F., “Edge-Preserving
Multiscale Image Decomposition Based on Local Ex-
trema,” in SIGGRAPH Asia, Singapore (2008).
Manuscript Received: Feb. 27, 2013
Accepted: Jun. 28, 2013
276 Jiunn-Lin Wu et al.
Table 2. The comparison between the proposed method
and Wang’s method
PSNR (dB)Processing time
(second)Image
Proposed
method
Wang’s
method
Proposed
method
Wang’s
method
Lena 28.9105 29.2214 320.7099 77.227
Airplane 27.7854 27.7699 298.3994 61.108
Fruit 30.7137 30.8389 298.7241 71.791