Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Yao WangPolytechnic University, Brooklyn, NY11201
http://eeweb.poly.edu/~yao
Image Filtering: Noise Removal, Sharpening,
Deblurring
©Yao Wang, 2006 EE3414: Image Filtering 2
Outline
• Noise removal by averaging filter• Noise removal by median filter• Sharpening (Edge enhancement)• Deblurring
©Yao Wang, 2006 EE3414: Image Filtering 3
Noise Removal (Image Smoothing)
• An image may be “dirty” (with dots, speckles,stains)• Noise removal:
– To remove speckles/dots on an image– Dots can be modeled as impulses (salt-and-pepper or
speckle) or continuously varying (Gaussian noise)– Can be removed by taking mean or median values of
neighboring pixels (e.g. 3x3 window)– Equivalent to low-pass filtering
• Problem with low-pass filtering– May blur edges– More advanced techniques: adaptive, edge preserving
©Yao Wang, 2006 EE3414: Image Filtering 4
Example
©Yao Wang, 2006 EE3414: Image Filtering 5
Averaging Filter
• Replace each pixel by the average of pixels in a square window surrounding this pixel
• Trade-off between noise removal and detail preserving:– Larger window -> can remove noise more effectively, but
also blur the details/edges
©Yao Wang, 2006 EE3414: Image Filtering 6
Example: 3x3 average
100100100100100
100195205200100
100200200195100
100203205200100
100100100100100
100100100100100
100144166144100
100168200167100
100145167144100
100100100100100
©Yao Wang, 2006 EE3414: Image Filtering 7
Example
©Yao Wang, 2006 EE3414: Image Filtering 8
Weighted Averaging Filter
• Instead of averaging all the pixel values in the window, give the closer-by pixels higher weighting, and far-away pixels lower weighting.
• This type of operation for arbitrary weighting matrices is generally called “2-D convolution or filtering”. When all the weights are positive, it corresponds to weighted average.
• Weighted average filter retains low frequency and suppresses high frequency = low-pass filter
∑∑−=−=
−−=L
Lk
L
Lllnkmslkhnmg ),(),(),(
©Yao Wang, 2006 EE3414: Image Filtering 9
Graphical Illustration
©Yao Wang, 2006 EE3414: Image Filtering 10
Example Weighting Mask
111
111
111
121
242
121
×91
×161
All weights must sum to one
©Yao Wang, 2006 EE3414: Image Filtering 11
Example: Weighted Average
100100100100100
100195205200100
100200200195100
100203205200100
100100100100100
100100100100100
100156175156100
100175201174100
100158176156100
100100100100100
121
242
121
×161
100100100100100
100144166144100
100168200167100
100145167144100
100100100100100
111
111
111
×91
©Yao Wang, 2006 EE3414: Image Filtering 12
Filtering in 1-D: a Review
• Continuous-Time Signal
• Extension to discrete time 1D signals
dtethfH
fHfSfG
dtshthtstg
ftj∫
∫
∞
∞−
−
∞
∞−
=
=
−=∗=
π
τττ
2)()( :responsefrequency Filter
)()()( :DomainFrequency
)()()()()( :Domain Time
2/ toscorrespond 1/22121 range at thelook toneedsonly periodic, is )(
)()( :(DTFT) responsefrequency Filter )()()( :Domain Frequency
)()()()()( :n)convolutio(linear Domain Time
2
s
fnj
n
m
f)/,/(-ffH
enhfHfHfSfG
mnsmhnhnsns
∈
==
−=∗=
−∑
∑
π
©Yao Wang, 2006 EE3414: Image Filtering 13
Filtering in 2D
).2/1,2/1(),2/1,2/1( region square at thelook toneedsonly periodic, is ),(
),(),(
Filter 2D theof responseFrequency ),(),(),(domain frequency 2DIn
),(),(),(
nConvolutioLinear 2D averaging Weighted
21
21
)(221
212121
1
0
211
0
1
0
1
0
−∈−∈
=
=
−−=
=
∑∑
∑∑
=
+−
=
==
ffffH
enmhffH
ffHffSffG
lnkmslkhnmg
k
kn
nfmfjl
lm
k
kk
l
ll
π
©Yao Wang, 2006 EE3414: Image Filtering 14
Frequency Response of Averaging Filters
• Averaging over a 3x3 window
=
0000001110011100111000000
91),( nmh
( )
( )
( )
( ) ( ) ( )
( )( ) ( )( )212222
22222222
))1()1((2))1()0((2))1()1((2
))0()1((2))0()0((2))0()1((2
))1()1((2))1()0((2))1()1((221
2cos212cos219111
91
19111
911
919191
91),(
2211
21111211
212121
212121
212121
ffeeee
eeeeeeee
eee
eee
eeeffH
fjfjfjfj
fjfjfjfjfjfjfjfj
ffjffjffj
ffjffjffj
ffjffjffj
ππππππ
ππππππππ
πππ
πππ
πππ
++=++++=
+++⋅+++++=
+++
+++
++=
−−
−−−−
⋅+⋅−⋅+⋅−⋅+−⋅−
⋅+⋅−⋅+⋅−⋅+−⋅−
−⋅+⋅−−⋅+⋅−−⋅+−⋅−
©Yao Wang, 2006 EE3414: Image Filtering 15
( ) ( )2211
2121
2cos2131)(,2cos21
31)(
)()(),(
ffHffH
fHfHffH
ππ +=+=
=
Sketch H(f1)
05
1015
2025
0
10
20
30-0.5
0
0.5
1
©Yao Wang, 2006 EE3414: Image Filtering 16
Frequency Response of Weighted Averaging Filters
[ ]
( )( ) 2
22
2
)1/()2cos(2)2cos(2),(
;111
1
)1(1
11
11
)1(1
bvbubvuH
bbb
bbbb
b
bH
+++=
+=
+=
ππ
05
1015
2025
0
10
20
300
0.2
0.4
0.6
0.8
1
©Yao Wang, 2006 EE3414: Image Filtering 17
Averaging vs. Weighted Averaging
[ ]
( )( ) 9/)2cos(21)2cos(21),(
;111111
111111111
91
vuvuH
H
ππ ++=
=
= [ ]
( )( ) 2
22
2
)1/()2cos(2)2cos(2),(
;111
1
)1(1
11
11
)1(1
bvbubvuH
bbb
bbbb
b
bH
+++=
+=
+=
ππ
05
1015
2025
0
10
20
300
0.2
0.4
0.6
0.8
1
05
1015
2025
0
10
20
30-0.5
0
0.5
1
b=2
©Yao Wang, 2006 EE3414: Image Filtering 18
Interpretation in Freq Domain
Low-passed image spectrum
Noise spectrum
Filter response
f0
Original image spectrum
Noise typically spans entire frequency range, where as natural images have predominantly lower frequency components
©Yao Wang, 2006 EE3414: Image Filtering 19
Median Filter
• Problem with Averaging Filter – Blur edges and details in an image– Not effective for impulse noise (Salt-and-pepper)
• Median filter:– Taking the median value instead of the average or weighted
average of pixels in the window• Median: sort all the pixels in an increasing order, take the middle one
– The window shape does not need to be a square– Special shapes can preserve line structures
• Order-statistics filter– Instead of taking the mean, rank all pixel values in the window, take
the n-th order value.– E.g. max or min
©Yao Wang, 2006 EE3414: Image Filtering 20
Example: 3x3 Median
100100100100100
100195205200100
100200200195100
100203205200100
100100100100100
100100100100100
100100195100100
100200200200100
100100200100100
100100100100100
Matlab command: medfilt2(A,[3 3])
©Yao Wang, 2006 EE3414: Image Filtering 21
Example
©Yao Wang, 2006 EE3414: Image Filtering 22
Original Image Corrupted Image Filtered Image
Matlab Demo: nrfiltdemo
Can choose between mean, median and adaptive (Wiener) filter with different window size
©Yao Wang, 2006 EE3414: Image Filtering 23
Noise Removal by Averaging Multiple Images
©Yao Wang, 2006 EE3414: Image Filtering 24
Image Sharpening
• Sharpening : to enhance line structures or other details in an image
• Enhanced image = original image + scaled version of the line structures and edges in the image
• Line structures and edges can be obtained by applying a difference operator (=high pass filter) on the image
• Combined operation is still a weighted averaging operation, but some weights can be negative, and the sum=1.
• In frequency domain, the filter has the “high-emphasis” character
©Yao Wang, 2006 EE3414: Image Filtering 25
Frequency Domain Interpretation
sharpened image spectrum
Filter response (high emphasis)
f0
Original image spectrum
©Yao Wang, 2006 EE3414: Image Filtering 26
Highpass Filters
• Spatial operation: taking difference between current and averaging (weighted averaging) of nearby pixels– Can be interpreted as weighted averaging = linear convolution– Can be used for edge detection
• Example filters
– All coefficients sum to 0!
;111181111
;111181111
;010141
010;
010141010
−−−−−−−−
−
−−−
−
−
©Yao Wang, 2006 EE3414: Image Filtering 27
Example Highpass Filters
05 10
1520 25
0
10
20
300
2
4
6
8
05
1015
2025
0
10
20
300
2
4
6
8
10
12
−−−
−
010141
010
−−−−−−−−
111181111
©Yao Wang, 2006 EE3414: Image Filtering 28
Example of Highpass Filtering
Original image Isotropic edge detection Binary image
©Yao Wang, 2006 EE3414: Image Filtering 29
Designing Sharpening Filter Using High Pass Filters
• The desired image is the original plus an appropriately scaled high-passed image
• Sharpening filterhs fff λ+=
),(),(),( nmhnmnmh hs λδ +=
x
f(x)
x
fs(x)=f(x)+ag(x)
x
g(x)=f(x)*hh(x)
©Yao Wang, 2006 EE3414: Image Filtering 30
Example Sharpening Filters
x
f(x)
x
fs(x)=f(x)+ag(x)
x
g(x)=f(x)*hh(x)
.1010181
010
41
010141
010
41 =
−−−
−=⇒
−−−
−= λwithHH sh
.11111161111
81
111181111
81 =
−−−−−−−−
=⇒
−−−−−−−−
= λwithHH sh
©Yao Wang, 2006 EE3414: Image Filtering 31
Example of Sharpening
−−−−−−−−
=1111161111
81
sH
−−−−−−−−
=111181111
41
hH
©Yao Wang, 2006 EE3414: Image Filtering 32
Example of Sharpening
4=λ
8=λ
©Yao Wang, 2006 EE3414: Image Filtering 33
Challenges of Noise Removal and Image Sharpening
• How to smooth the noise without blurring the details too much?
• How to enhance edges without amplifying noise?• Still a active research area
©Yao Wang, 2006 EE3414: Image Filtering 34
Wavelet-Domain Filtering
Courtesy of Ivan Selesnick
©Yao Wang, 2006 EE3414: Image Filtering 35
Feature Enhancement by Subtraction
Taking an image without injecting a contrast agent first. Then take the image again after the organ is injected some special contrast agent (which go into the bloodstreams only). Then subtract the two images --- A popular technique in medical imaging
©Yao Wang, 2006 EE3414: Image Filtering 36
Image Deblurring
• Noise removal considered thus far assumes the image is corrupted by additive noise– Each pixel is corrupted by a noise value, independent of
neighboring pixels• Image blurring
– When the camera moves while taking a picture– Or when the object moves– Each pixel value is the sum of surrounding pixels � The
blurred image is a filtered version of the original• Deblurring methods:
• Inverse filter: can adversely amplify noise• Wiener filter = generalized inverse filter• Many advanced adaptive techniques
©Yao Wang, 2006 EE3414: Image Filtering 37
Example of Motion Blur
©Yao Wang, 2006 EE3414: Image Filtering 38
Inverse Filtering vs. Wiener Filtering
©Yao Wang, 2006 EE3414: Image Filtering 39
Constrained Least Squares Filtering
©Yao Wang, 2006 EE3414: Image Filtering 40
What Should You Know
• How does averaging and weighted averaging filter works?• How does median filter works?• What method is better for additive Gaussian noise?• What method is better for salt-and-pepper noise?• How does high-pass filtering and sharpening work?• For smoothing and sharpening:
– Can perform spatial filtering using given filters– Deriving frequency response is not required, but should know the
desired shape for the frequency responses in different applications• What is the challenge in noise removal and sharpening?• What causes blurring? • Principle of deblurring: technical details not required
©Yao Wang, 2006 EE3414: Image Filtering 41
References
Gonzalez and Woods, Digital image processing, 2nd edition, Prentice Hall, 2002. Chap 4 Sec 4.3, 4.4; Chap 5 Sec 5.1 – 5.3 (pages 167-184 and 220-243)