Chapter 5 Image Restoration and Reconstructioneem.eskisehir.edu.tr/cihant/EEM...

EEM 463 Introduction to Image Processing

Chapter 5Image Restoration and

Reconstruction

Fall 2017

Assist. Prof. Cihan Topal

Anadolu University, Dept. EEE

Slides Credit: Frank (Qingzhong) Liu

11/24/2017 2

Image Restoration

► Image restoration: recover an image that has beendegraded by using a prior knowledge of the degradationphenomenon.

► Model the degradation and applying the inverse process inorder to recover the original image.

► The principal goal of restoration techniques is to improvean image in some predefined sense.

► Although there are areas of overlap, image enhancement islargely a subjective process, while restoration is for themost part an objective process.

11/24/2017 3

A Model of Image Degradation/Restoration Process

►Degradation

Degradation function H

Degraded image is in the spatial domain.

Additive noise ),( yx

11/24/2017 4

A Model of Image Degradation/Restoration Process

If is a process, then

the degraded image is given in the spatial domain by

( , ) ( , ) ( , ) ( , )

H linear, position-invariant

g x y h x y f x y x y

11/24/2017 5

A Model of Image Degradation/Restoration Process

The model of the degraded image is given in

the frequency domain by

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v

11/24/2017 6

Basics of Filtering of the Frequency (Fourier) Domain

11/24/2017 7

11/24/2017 8

11/24/2017 9

Example: Phase Angles

11/24/2017 10

Example: Phase Angles and The Reconstructed

11/24/2017 11

2-D Convolution Theorem

1

0

1-D convolution

( ) ( ) ( ) ( )M

m

f x h x f m h x m

1 1

0 0

2-D convolution

( , ) ( , ) ( , ) ( , )M N

m n

f x y h x y f m n h x m y n

0,1,2,..., 1; 0,1,2,..., 1.x M y N

( , ) ( , ) ( , ) ( , )f x y h x y F u v H u v

( , ) ( , ) ( , ) ( , )f x y h x y F u v H u v

11/24/2017 12

The Basic Filtering in the Frequency Domain

Why is the spectrum at almost ±45 degree stronger than the spectrum at other directions?

11/24/2017 13

The Basic Filtering in the Frequency Domain

► Modifying the Fourier transform of an image

► Computing the inverse transform to obtain the processed result

1( , ) { ( , ) ( , )}

( , ) is the DFT of the input image

( , ) is a filter function.

g x y H u v F u v

F u v

H u v

11/24/2017 14

The Basic Filtering in the Frequency Domain

► In a filter H(u,v) that is 0 at the center of the transform and 1 elsewhere, what’s the output image?

11/24/2017 15

The Basic Filtering in the Frequency Domain

11/24/2017 16

Summary: Steps for Filtering in the Frequency Domain

1. Given an input image f(x,y) of size MxN, obtain the padding parameters P and Q. Typically, P = 2M and Q = 2N.

2. Form a padded image, fp(x,y) of size PxQ by appending the necessary number of zeros to f(x,y)

3. Multiply fp(x,y) by (-1)x+y to center its transform

4. Compute the DFT, F(u,v) of the image from step 3

5. Generate a real, symmetric filter function*, H(u,v), of size PxQ with center at coordinates (P/2, Q/2)

*generate from a given spatial filter, we pad the spatial filter, multiply the expaddedarray by (-1)x+y, and compute the DFT of the result to obtain a centered H(u,v).

11/24/2017 17

Summary: Steps for Filtering in the Frequency Domain

6. Form the product G(u,v) = H(u,v)F(u,v) using array multiplication

7. Obtain the processed image

8. Obtain the final processed result, g(x,y), by extracting the MxN region from the top, left quadrant of gp(x,y)

1( , ) ( , ) ( 1)x y

pg x y real G u v

An Example: Steps for Filtering in the Frequency Domain

11/24/2017 18

11/24/2017 19

Noise Sources

► The principal sources of noise in digital images arise during image acquisition and/or transmission

Image acquisition

e.g., light levels, sensor temperature, etc.

Transmission

e.g., lightning or other atmospheric disturbance in wireless network

11/24/2017 20

Noise Models (1)

►White noise

The Fourier spectrum of noise is constant

► With the exception of spatially periodic noise, we assume

Noise is independent of spatial coordinates

Noise is uncorrelated with respect to the image itself

11/24/2017 21

Noise Models (2)

Gaussian noiseElectronic circuit noise, sensor noise due to poor illumination and/or high temperature

Rayleigh noiseRange imaging

11/24/2017 22

Range Imaging (1)

Short for High Dynamic Range Imaging. HDRI is an imaging technique that allows for a greater dynamic range of exposure than would be obtained through any normal imaging process.

It is now popularly used to refer to the process of tone mapping* together with bracketed** exposures of normal digital images, giving the end result a high, often exaggerated dynamic range

* Tone mapping is a technique used in image processing and computer graphics to map a set of colours to another; often to approximate the appearance of HDRI in media with a more limited dynamic range** bracketing is the general technique of taking several shots of the same subject using different or the same camera settings

http://en.wikipedia.org/wiki/High_dynamic_range_imaginghttp://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html

11/24/2017 23

Range Imaging (2)

The intention of HDRI is to accurately represent the wide range of intensity levels found in real scenes ranging from direct sunlight to shadows

HDRI, also called HDR (High Dynamic Range) is a feature commonly found in high-end graphics and imaging software

http://en.wikipedia.org/wiki/High_dynamic_range_imaginghttp://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html

11/24/2017 24

Range Imaging: Examples (1)

Tower Bridge in Sacramento,

CA

http://en.wikipedia.org/wiki/High_dynamic_range_imaging

11/24/2017 25

Range Imaging: Examples (2)

Sydney Harbour Bridge HDRi produces greater detail and fewer shadowshttp://en.wikipedia.org/wiki/High_dynamic_range_imaging

11/24/2017 26Old Saint Paul’s Wellinton, New Zealand

11/24/2017 27

http://en.wikipedia.org/wiki/Image:HDRI-Example.jpg

11/24/2017 28

Noise Models (3)

Erlang (gamma) noise: Laser imaging

Exponential noise: Laser imaging

Uniform noise: Least descriptive; Basis for numerous random number generators

Impulse noise: quick transients,

such as faulty switching

11/24/2017 29

Gaussian Noise (1)

2 2( ) /2

The PDF of Gaussian random variable, z, is given by

1 ( )

2

z zp z e

where, represents intensity

is the mean (average) value of z

is the standard deviation

z

z

11/24/2017 30

Gaussian Noise (2)

2 2( ) /2

The PDF of Gaussian random variable, z, is given by

1 ( )

2

z zp z e

70% of its values will be in the range

95% of its values will be in the range

)(),(

)2(),2(

11/24/2017 31

Rayleigh Noise

2( ) /

The PDF of Rayleigh noise is given by

2( ) for

( )

0 for

z a bz a e z ap z b

z a

2

The mean and variance of this density are given by

/ 4

(4 )

4

z a b

b

11/24/2017 32

Erlang (Gamma) Noise

1

The PDF of Erlang noise is given by

for 0 ( ) ( 1)!

0 for

b baza z

e zp z b

z a

2 2

The mean and variance of this density are given by

/

/

z b a

b a

11/24/2017 33

Exponential Noise

The PDF of exponential noise is given by

for 0 ( )

0 for

azae zp z

z a

2 2

The mean and variance of this density are given by

1/

1/

z a

a

11/24/2017 34

Uniform Noise

The PDF of uniform noise is given by

1 for a

( )

0 otherwise

z bp z b a

2 2

The mean and variance of this density are given by

( ) / 2

( ) /12

z a b

b a

11/24/2017 35

Impulse (Salt-and-Pepper) Noise

The PDF of (bipolar) impulse noise is given by

for

( ) for

0 otherwise

a

b

P z a

p z P z b

If either or is zero, the impulse noise is calleda bP P

unipolar

if , gray-level will appear as a light dot,

while level will appear like a dark dot.

b a b

a

11/24/2017 36

11/24/2017 37

Examples of Noise: Original Image

11/24/2017 38

Examples of Noise: Noisy Images(1)

11/24/2017 39

Examples of Noise: Noisy Images(2)

11/24/2017 40

Periodic Noise

► Periodic noise in an image arises typically from electrical or electromechanical interference during image acquisition.

► It is a type of spatially dependent noise

► Periodic noise can be reduced significantly via frequency domain filtering

11/24/2017 41

An Examples of Periodic Noise

11/24/2017 42

Examples: Notch

Filters (1)

0

A Butterworth notch

reject filter D =3

and n=4 for all

notch pairs

11/24/2017 43

Examples: Notch Filters

(2)

11/24/2017 44

11/24/2017 45

Estimation of Noise Parameters (1)

The shape of the histogram identifies the closest PDF match

11/24/2017 46

Estimation of Noise Parameters (2)

Consider a subimage denoted by , and let ( ), 0, 1, ..., -1,

denote the probability estimates of the intensities of the pixels in .

The mean and variance of the pixels in :

s iS p z i L

S

S

1

0

12 2

0

( )

and ( ) ( )

L

i s i

i

L

i s i

i

z z p z

z z p z

11/24/2017 47

Restoration in the Presence of Noise Only

Spatial Filtering

Noise model without degradation

( , ) ( , ) ( , )

and

( , ) ( , ) ( , )

g x y f x y x y

G u v F u v N u v

11/24/2017 48

Spatial Filtering: Mean Filters (1)

Let represent the set of coordinates in a rectangle

subimage window of size , centered at ( , ).

xyS

m n x y

( , )

Arithmetic mean filter

1 ( , ) ( , )

xys t S

f x y g s tmn

11/24/2017 49

Spatial Filtering: Mean Filters (2)

1

( , )

Geometric mean filter

( , ) ( , )xy

mn

s t S

f x y g s t

Generally, a geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process

11/24/2017 50

Spatial Filtering: Mean Filters (3)

( , )

Harmonic mean filter

( , )1

( , )xys t S

mnf x y

g s t

It works well for salt noise, but fails for pepper noise.It does well also with other types of noise like Gaussian noise.

11/24/2017 51

Spatial Filtering: Mean Filters (4)

1

( , )

( , )

Contraharmonic mean filter

( , )

( , )( , )

xy

xy

Q

s t S

Q

s t S

g s t

f x yg s t

Q is the order of the filter.

It is well suited for reducing the effects of salt-and-pepper noise. Q>0 for pepper noise and Q<0 for salt noise.

11/24/2017 52

Spatial Filtering: Example (1)

11/24/2017 53

Spatial Filtering: Example (2)

11/24/2017 54

Spatial Filtering: Example (3)

11/24/2017 55

Spatial Filtering: Order-Statistic Filters (1)

( , )

Max filter

( , ) max ( , )xys t S

f x y g s t

( , )

Median filter

( , ) ( , )xys t S

f x y median g s t

( , )

Min filter

( , ) min ( , )xys t S

f x y g s t

11/24/2017 56

Spatial Filtering: Order-Statistic Filters (2)

( , )( , )

Midpoint filter

1 ( , ) max ( , ) min ( , )

2 xyxy s t Ss t Sf x y g s t g s t

11/24/2017 57

Spatial Filtering: Order-Statistic Filters (3)

( , )

Alpha-trimmed mean filter

1 ( , ) ( , )

xy

r

s t S

f x y g s tmn d

We delete the / 2 lowest and the / 2 highest intensity values of

( , ) in the neighborhood . Let ( , ) represent the remaining

- pixels.

xy r

d d

g s t S g s t

mn d

11/24/2017 58

11/24/2017 59

11/24/2017 60

11/24/2017 61

Spatial Filtering: Adaptive Filters (1)

Adaptive filters

The behavior changes based on statistical characteristics of the image inside the filter region defined by the mхn rectangular window.

The performance is superior to that of the filters discussed

11/24/2017 62

Adaptive Filters:

Adaptive, Local Noise Reduction Filters (1)

2

: local region

The response of the filter at the center point (x,y) of

is based on four quantities:

(a) ( , ), the value of the noisy image at ( , );

(b) , the variance of the noise corrupti

xy

xy

S

S

g x y x y

2

ng ( , )

to form ( , );

(c) , the local mean of the pixels in ;

(d) , the local variance of the pixels in .

L xy

L xy

f x y

g x y

m S

S

11/24/2017 63

Adaptive Filters:

Adaptive, Local Noise Reduction Filters (2)

2

2

The behavior of the filter:

(a) if is zero, the filter should return simply the value

of ( , ).

(b) if the local variance is high relative to , the filter

should return a value cl

g x y

ose to ( , );

(c) if the two variances are equal, the filter returns the

arithmetic mean value of the pixels in .xy

g x y

S

11/24/2017 64

Adaptive Filters:

Adaptive, Local Noise Reduction Filters (3)

2

2

An adaptive expression for obtaining ( , )

based on the assumptions:

( , ) ( , ) ( , ) L

L

f x y

f x y g x y g x y m

11/24/2017 65

11/24/2017 66

Adaptive Filters:

Adaptive Median Filters (1)

min

max

med

max

The notation:

minimum intensity value in

maximum intensity value in

median intensity value in

intensity value at coordinates ( , )

maximum all

xy

xy

xy

xy

z S

z S

z S

z x y

S

owed size of xyS

11/24/2017 67

Adaptive Filters:

Adaptive Median Filters (2)

med min med max

max

The adaptive median-filtering works in two stages:

Stage A:

A1 = ; A2 =

if A1>0 and A2<0, go to stage B

Else increase the window size

if window size , re

z z z z

S

med

min max

med

peat stage A; Else output

Stage B:

B1 = ; B2 =

if B1>0 and B2<0, output ; Else output

xy xy

xy

z

z z z z

z z

11/24/2017 68

Adaptive Filters:

Adaptive Median Filters (2)

med min med max

max

The adaptive median-filtering works in two stages:

Stage A:

A1 = ; A2 =

if A1>0 and A2<0, go to stage B

Else increase the window size

if window size , re

z z z z

S

med

min max

med

peat stage A; Else output

Stage B:

B1 = ; B2 =

if B1>0 and B2<0, output ; Else output

xy xy

xy

z

z z z z

z z

The median filter output is an impulse

or not

The processed point is an impulse or not

11/24/2017 69

Example:Adaptive Median Filters

Bilateral Filtering – 1 / 3

Bilateral Filtering – 2 / 3

Bilateral Filtering – 3 / 3

Bilateral Filtering - Example

11/24/2017 73

Input Gauss Filtered Bilateral Filtered

11/24/2017 74

Periodic Noise Reduction by Frequency

Domain Filtering

The basic idea

Periodic noise appears as concentrated bursts of energy in the Fourier transform, at locations corresponding to the frequencies of the periodic interference

Approach

A selective filter is used to isolate the noise

11/24/2017 75

Perspective Plots of Bandreject Filters

11/24/2017 76

A Butterworth bandreject filter of order 4, with the appropriate radius and

width to enclose completely the noise

impulses

11/24/2017 77

Perspective Plots of Notch Filters

11/24/2017 78

11/24/2017 79

Several interference components are present, the methods discussed in the preceding sections are not always acceptable because they remove much image information The components tend to have broad skirts that carry information about the interference pattern and the skirts are not always easily detectable.

11/24/2017 80

Optimum Notch Filtering

It minimizes local variances of the restored estimated

Procedure for restoration tasks in multiple periodic interference

Isolate the principal contributions of the interference pattern

Subtract a variable, weighted portion of the pattern from the corrupted image

( , )f x y

11/24/2017 81

Optimum Notch Filtering: Step 1

Extract the principal frequency components of the interference pattern

Place a notch pass filter at the location of each spike.

( , ) ( , ) ( , )NPN u v H u v G u v

1( , ) ( , ) ( , )NPx y H u v G u v

11/24/2017 82

Optimum Notch Filtering: Step 2 (1)

Filtering procedure usually yields only an approximation of the

true pattern. The effect of components not present in the estimate

of ( , ) can be minimized instead by subtracting from ( , )

a weighte

x y g x y

d portion of ( , ) to obtain an estimate of ( , ):

( , ) ( , ) ( , ) ( , )

x y f x y

f x y g x y w x y x y

One approach is to select ( , ) so that the variance of the estimate ( , )

is minimized over a specified neighborhood of every point ( , ).

w x y f x y

x y

11/24/2017 83

Optimum Notch Filtering: Step 2 (2)

2

2

The local variance of ( , ):

1( , ) ( , ) ( , )

(2 1)(2 1)

a b

s a t b

f x y

x y f x s y t f x ya b

11/24/2017 84

Optimum Notch Filtering: Step (3)

2

2

2

The local variance of ( , ):

1( , ) ( , ) ( , )

(2 1)(2 1)

( , ) ( , ) ( , )1

(2 1)(2 1) ( , ) ( , ) ( , )

(1

(2 1)(2 1)

a b

s a t b

a b

s a t b

f x y

x y f x s y t f x ya b

g x s y t w x s y t x s y s

a b g x y w x y x y

g

a b

2

, ) ( , ) ( , )

( , ) ( , ) ( , )

a b

s a t b

x s y t w x y x s y s

g x y w x y x y

Assume that w(x,y) remains essentially constant over the

neighborhood gives the approximation

w(x+s, y+t) = w(x,y)

11/24/2017 85

Optimum Notch Filtering: Step (4)

2

2

The local variance of ( , ):

( , ) ( , ) ( , )1( , )

(2 1)(2 1) ( , ) ( , ) ( , )

a b

s a t b

f x y

g x s y t w x y x s y sx y

a b g x y w x y x y

22

22

( , )To minimize ( , ) , 0

( , )

for ( , ), the result is

( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( , )

x yx y

w x y

w x y

g x y x y g x y x yw x y

x y x y

11/24/2017 86

Optimum Notch Filtering: Example

11/24/2017 87

Optimum Notch Filtering: Example

11/24/2017 88

Optimum Notch Filtering: Example

11/24/2017 89

Optimum Notch Filtering: Example