Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
CoE4TN3Image Processing
Chapter 5Image Restoration
2
Image Restoration• Similar to image enhancement, the ultimate goal of restoration
techniques is to improve an image• Restoration: a process that attempts to reconstruct or recover a
degraded image by using some a priori knowledge of the degradation phenomenon.
• Technique: model the degradation -> apply the inverse process to recover the original image
• Enhancement technique are heuristic while restoration techniques are mathematical
3
Degradation Model
• Problem: given g(x,y) and H find an approximate of f(x,y). • Some statistical knowledge of n(x,y) is available.
g(x,y)=h(x,y)*f(x,y)+n(x,y)
4
Noise model• Principal sources of noise in digital images: during image
acquisition, during image transmission• Image acquisition: image sensor might produce noise because
of environmental conditions or quality of sensing elements• Image transmission: interference in the channel • Assumptions: noise is independent of spatial coordinates
(except for periodic noise) and independent of the image • Spatial description of noise: statistical behavior of the values
of the noise (PDF)• Most common PDFs found in image processing: Gaussian
noise, Rayleigh noise, Erlang (Gamma) noise, Exponential noise, Uniform noise, Impulse noise
5
6
Noise model• Different PDFs provide useful tools for modeling a broad
range of noise corruption situations:• Gaussian noise: due to factors such as electronic circuit noise,
sensor noise (due to poor illumination or high temperature)• Rayleigh noise: model noise in range imaging • Exponential and Gamma: laser imaging• Impulse noise: found in quick transients (e.g., faulty
switches)
7
Noise model
8
Noise model
9
Noise model
10
Periodic Noise• Periodic noise: from electrical or electromechanical
interference during image acquisition• Frequency domain filtering can be used to remove this noise• Fourier transform of a pure sinusoid is a pair of conjugate
impulses• In the Fourier transform of an image corrupted with periodic
noise should have a pair of impulses for each sine wave
11
Noise model
12
• Parameters of periodic noise are typically estimated by inspection of the FT of the image
• Estimation of parameters of the PDF of noise:– If the imaging system is available, one simple way is to capture a set
of images of “flat” environments• Flat environments: a solid gray board illuminated uniformly
– If only the images are available, the parameters of the PDF can be estimated from small patches of reasonably constant gray levels
Noise model
13
Noise model
14
Restoration in the presence of noise• When the only degradation is noise:
g(x,y)=f(x,y)+n(x,y)G(u,v)=F(u,v)+N(u,v)
• Spatial filtering is the method of choice in this case: Mean filters, Order-statistics filters, Adaptive filters
15
Mean filters• Sxy set of coordinates in a subimage of size mXn• Arithmetic mean filter:
– Smoothes local variations (noise is reduced because of blurring)
• Geometric mean filter:
– Achieves smoothing comparable to arithmetic mean, but tends to lose less image details
∑∈
=xySts
tsgmn
yxf),(
^),(1),(
mn
Sts xy
tsgyxf
1
),(
^),(),(
= ∏
∈
16
Mean filters• Harmonic mean filter:
– Suitable for salt noise, does well with Gaussian noise
• Contraharmonic mean filter
– Negative Q: Suitable for salt noise– Positive Q: Suitable for pepper noise
∑∈
=
xySts tsg
mnyxf
),(
^
),(1),(
∑
∑
∈
∈
+
=
xy
xy
Sts
Q
Sts
Q
tsg
tsgyxf
),(
),(
1
^
),(
),(),(
17
18
19
20
Order-statistics filters• Order-statistics filters: spatial filters whose response is based
on ordering (ranking) the pixels in the subimage• Median filters:
– Effective for salt and pepper noise
• Max and Min filters
– Max filter: useful for finding brightest points in an image (remove pepper noise)
– Min filter: useful for finding darkest points in an image (remove salt noise)
)},({),(),(
^tsgmedianyxf
xySts ∈=
)},({max),(),(
^tsgyxf
xySts ∈= )},({min),(
),(
^tsgyxf
xySts ∈=
21
Order-statistics filters • Midpoint filter:
– Works best for Gaussian and uniform noise
• Alpha-trimmed mean filter• d/2 lowest and d/2 highest gray-levels are removed
– Useful for combination of salt-pepper and Gaussian noise
)}],({min)},({max[21),(
),(),(
^tsgtsgyxf
xyxy StsSts ∈∈+=
∑∈−
=xyStsr tsg
dmnyxf
),(
^),(1),(
22
23
24
Adaptive filters• Adaptive filters: behavior of the filter changes based on
statistical characteristics of the image inside the subimage(Sxy)
• Adaptive filters have superior performance• Price: increase in filter complexity
25
Adaptive, local noise reduction filter• Response of the filter is based on four quantities:1. g(x,y) 2. : variance of noise 3. mL: mean of pixels in Sxy
4. σ2L: variance of pixels in Sxy
• The behavior of the filter:– If is zero, filter should return g(x,y)– If σ2
L is high (edges) the filter should return g(x,y)– If the two variances are almost equal, the filter should return the mean
of the pixels in Sxy
2ησ
2ησ
]),([),(),( 2
2^
LL
myxgyxgyxf −−=σση
26
27
Adaptive median filtering• Median filters perform well as long as the density of impulse
noise is not large• Adaptive median filter:
– Handle dense impulse noise– Smoothes non-impulse noise– Preserves details
• zmin: minimum gray level in Sxy
• zmax: maximum gray level in Sxy
• zmed: median gray level of Sxy
• zxy: gray level at coordinate (x,y)• Smax: maximum allowed size of Sxy
28
Adaptive median filtering• A
– A1=zmed-zmin
– A2=zmed-zmax
– If A1>0 and A2<0 go to B else increase the window size– If window size<Smax repeat A– Else output zxy
• B– B1=zxy-zmin
– B2=zxy-zmax
– If B1>0 and B2<0 output zxy
– Else output zmed
29
Adaptive median filtering
30
Periodic noise reduction• Bandreject filters remove or attenuate a band of frequencies
=
101
),( vuH
2),(
2),(
2
2),(
0
00
0
WDvuD
WDvuDWD
WDvuD
+>
+≤≤−
−<
n
DvuDWvuD
vuH 2
20
2 ),(),(1
1),(
−
+
=
220
2
),(),(
21
1),(
−−
−= WvuDDvuD
evuH
31
Periodic noise reduction
32
Periodic noise reduction
33
• A bandpass filter performs the opposite of a bandreject filter. – Hbp(u,v)=1-Hbr(u,v)
• A notch filter rejects (or passes) frequencies in predefined neighborhood about a center frequency
• Due to symmetry of the FT, notch filters must appear in symmetric pairs about the origin in order to obtain meaningful results
Periodic noise reduction
34
−
−=20
21 ),(),(21
1),( DvuDvuD
evuH
n
vuDvuDD
vuH
+
=
),(),(1
1),(
21
0
=10
),( vuHotherwise
DvuDDvuD 0201 ),(,),( ≤≤
Periodic noise reduction
20
201 )()(),( vvuuvuD −+−=
20
202 )()(),( vvuuvuD +++=
35
Periodic noise reduction
36
37
Estimation of degradation • Since degradations are modeled as being the result of
convolution, restoration is sometimes called deconvolution. • Estimation of the degradation function:
1. Observation2. Experimentation 3. Mathematical modeling
38
Estimation by image observation• We are given a degraded image without any knowledge of H.• We look at a small section of the image containing simple
structures (e.g., part of an object and the background)• Using sample gray levels of the object and background, we
can construct an unblurred image of the subimage ),(^
yxf s
),(
),(),( ^
vuF
vuGvuH
s
ss =
39
Estimation by Experimentation • If equipment similar to the equipment used to acquire the
degraded image is available it is possible a to obtain an accurate estimate of the degradation.
• The idea is to obtain the impulse response of the degradation by imaging an impulse (small dot of light) using the system
AvuGvuH ),(),( =
40
Estimation by modeling • Approach: derive a mathematical model starting from basic
principles• Example: Images blurred by motion
41
42
Inverse filtering
43
Wiener filtering
44
Wiener filtering
45
A Simple Example• 1-D signal• h is a 2 point signal (h(0) and h(1))• f is also a 2 point signal• g is observed: find f(0) and f(1) from g(0), g(1), g(2)
h(x)f(x)
n(x)
g(x)
46
A Simple Example
)2()1()1()2()1()1()0()0()1()1(
)0()0()0()0(
)2()1()1()0()2()2()1()1()0()0()1()1(
)0()1()1()0()0()0(
)()()()(
)()(*)()(1
0'
''
nhfgnhfhfg
nhfg
nhfhfgnhfhfgnhfhfg
xnxhxxfxg
xnxfxhxg
x
+=++=
+=
++=++=
+−+=
+−=
+=
∑=
47
A Simple Example
• If the noise is not ignored, we have five unknowns and only three equations.
• The original signal (f) cannot be determined uniquely. • This is an example of an ill-conditioned problem. • Some other knowledge about f is necessary.
})]1()0(min{[
)2()1()1()2()1()1()0()0()1()1(
)0()0()0()0(
2ff
nhfgnhfhfg
nhfg
−
+=++=
+=
48
Constrained Least Square Filtering• g=Hf+n• Find a restoration filter such that the output of the filter is
smooth:
• After a lengthy procedure (which has been removed in the new edition of the book) using the fact that the matrix H is circular it can be shown that:
22^
^2
1
0
1
0
2^
2
)1,()1,(),1(),1(),(4),(
)],([
nfHg
yxfyxfyxfyxfyxfyxf
yxfCM
x
N
y
=−
−−+−−−+−≈∇
∇= ∑∑−
=
−
=
49
Constrained Least Square Filtering
• P(u,v) is the Fourier transform of the Laplacian operator:
γ is a parameter.• p(x,y) should be padded to a proper size.• Although mathematical method exists for finding γ, in most
practical cases γ is selected by trial and error.
22
*
),(),(),(),(
vuPvuHvuHvuM
γ+=
−−−
−=
010141
010),( yxp
50
Constrained Least Square Filtering
51
Geometric Mean Filter• Generalization of Wiener filter:
α and β : positive real constants α=1 : inverse filterα=0 : parametric Wiener filter α=0, β=1 : Wiener filterα=1/2, β=1 : spectrum equalization filter
α
α
β
−
+
=
1
2
*
2
*
),(),(
),(
),(),(),(),(
vuSvuS
vuH
vuHvuHvuHvuM
f
n
52
Geometric Transformations• Geometrical transformations: modify the spatial relationships
between pixels in an image• Geometrical transformations: called rubber-sheet
transformations– They may be viewed as printing an image on a sheet of rubber and
then stretching this sheet according to some predefined rules
53
Geometric Transformations• Geometrical transformation consists of two basic operations:
1. Spatial transformation: defines the rearrangement of pixels on the image plane
2. Gray level interpolation: deals with the assignment of gray levels to pixels in the spatially transformed image
54
Spatial Transformations• Image f with pixels coordinates (x,y) has undergone
geometric distortion to produce an image g with coordinates (x’,y’)– x’=r(x,y), y’=s(x,y)
• Example: x’=r(x,y)=x/2, y’=s(x,y)=y/2– Distortion is a shrinking of the size of f(x,y) by one-half in both
directions.
55
Spatial Transformations• If r(x,y) and s(x,y) are known analytically: the inverse of r
and s is applied to g(x’,y’) to recover f(x,y).• In practice finding a single set of r(x,y) and s(x,y) is not
possible• Solution: spatial relocation is formulated by the use of
tiepoints. • Tiepoints: a set of pixels whose locations in distorted and
corrected images are known
56
Spatial Transformations• Suppose the geometrical distortion process within the region
is modeled by a pair of bilinear equations:– x’=r(x,y)=c1x+c2y+c3yx+c4
– y’=s(x,y)=c5x+c6y+c7yx+c8
– 8 known tiepoints, 8 unknown ci
– The model is used for all the points inside the region
57
Spatial Transformations
for x=1 to horizontal size {for y=1 to vertical size {
x’=r(x,y)=c1x+c2y+c3yx+c4
y’=s(x,y)=c5x+c6y+c7yx+c8
f^(x,y)=g(x’,y’)}}
58
Gray-level Interpolation• Depending on the values of ci x’ and/or y’ can be noninteger
for integer values of (x,y)– x’=r(x,y)=c1x+c2y+c3yx+c4
– y’=s(x,y)=c5x+c6y+c7yx+c8
• g is a digital image and its pixel values are defined only at integer values of (x,y).
• We need inferring gray-level values at noninteger locations (gray-level interpolation)
59
Gray-level Interpolation• Simplest scheme: nearest neighbor approach (zero-order
interpolation)1. Mapping (x,y) to (x’,y’)2. Selection of closest integer coordinate neighbor to (x’,y’) 3. Assign the gray-level of this nearest neighbor to the pixel at (x,y)
60
Gray-level Interpolation• Nearest neighbor interpolation: simple to implement, has the
drawback of producing undesirable artifacts• Example: distortion of straight edges in an image• More sophisticated techniques: better results, costly in terms
of computations• A reasonable compromise: bilinear interpolation approach
61
Gray-level Interpolation• (x’,y’): a noninteger coordinate• v(x’,y’): the gray-level at (x’,y’)• v(x’,y’)=ax’+by’+cx’y’+d• The four unknowns (a,b,c,d) are determined using the gray-
levels of four neighbors of (x’,y’)• When the coefficients (a,b,c,d) have been determined,
v(x’,y’) is computed and this values is assigned to the location (x,y).
62
Geometric Transformations