Phase Preserving Denoising of Images

Phase Preserving Denoising of Images

Mirko S.

Adv. Seminar: Image Processing (SS 2015)

Computer Vision Lab Dresden

TU Dresden

Peter Kovesi, Department of Computer Science, The University of Western Australia

29.06.2015 0

Roadmap

• Motivation

• Fourier Transform

• Wavelet Transform

• Phase Preserving Denoising

29.06.2015 1

Roadmap

• Motivation




29.06.2015 2

Motivation

• image noise

- random variation of brightness in images

- noise: „unwanted signal“

• salt-and-pepper noise

- caused by analog-to-digital converters,

- bit errors in transmission, …

- sparse -> much of the original signal is uneffected

-> eliminated by interpolating around outliers; median filter

29.06.2015 3

Motivation

• Gaussian noise

- produced by the sensor of the digital camera

- low light

- short exposure

- bad sensor quality

- additive, pixel-wise independent,

- independent of intensity

- present at every pixel -> difficult to eliminate

29.06.2015 4

Motivation

• linear smoothing filters

- weighted averages

- blurs the image

• anisotropic diffusion

- evolve image under smoothing partial differential equation

- preserves edges better

29.06.2015 5

Motivation

• nonlinear filters

- median

- bilateral filter

• Wavelet shrinkage

- transform image to domain where noise is easier identifiable

- remove/reduce responses that are likely due to noise

- transform back to recover denoised image

29.06.2015 6

Roadmap

• Recap: Complex Numbers & Convolution


• Denoising by Wavelet-Shrinkage


29.06.2015 7

Complex Numbers

• generalization of real numbers

• complex conjugate

• polar form

- magnitude

- phase

• Euler‘s formula

29.06.2015 8

iyxz

22 yxzr

),(atan2 xy

12 i

ireirz )sin(cos

iyxz *

"Complex number illustration modarg" by Complex_number_illustration.svg: Original uploader was Wolfkeeper at en.wikipediaderivative work: Kan8eDie (talk) - Complex_number_illustration.svg. Licensed under CC BY-SA 3.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Complex_number_illustration_modarg.svg# /media/File:Complex_number_illustration_modarg.svg

Convolution and Cross-Correlation

• convolution

29.06.2015 9

dtgftgf )()())((

"Convolution of box signal with itself2" by Convolution_of_box_signal_with_itself.gif: Brian Ambergderivative work: Tinos (talk) - Convolution_of_box_signal_with_itself.gif. Licensed under CC BY-SA 3.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Convolution_of_box_signal_with_itself2.gif#/media/File:Convolution_of_box_signal_with_itself2.gif


• convolution

• cross-correlation

- „sliding inner-product“

- provides a measure of similarity

- usefull for pattern recognition

29.06.2015 10

dttgtfgf )()())(#( *

dtgftgf )()())((

b

adxxgxfgf )()(, *


• convolution and cross-correlation are closely related

- for Hermitian function equal

• Hermitian function

- real part is even-symmetric

- imaginary part is odd-symmetric

29.06.2015 11

gtfgf )(# *

gfgf #

)()( * xfxf

https://www.mathsisfun.com/algebra/images/sine-cosine-graph.gif

ixexf )(

Roadmap

• Motivation




29.06.2015 12

Fourier Transform

• decomposes signal into its frequencies

• FT: time domain -> frequency domain

• iFT: frequency domain -> time domain

29.06.2015 13

dxexfF ix 2)()(

deFxf xi2)()(

Fourier Transform

• for the discrete case (Discrete FT):

- time domain: values at sampling points

- frequency domain: complex coefficients (weights of frequencies + phase)

29.06.2015 14

Fourier Transform

29.06.2015 15

real

imaginary

magnitude

phase

FT

Fourier Transform

29.06.2015 16

iFT

iFT

iFT

iFT

DC

Fourier Transform

• Discrete Fourier Transform (DFT): O(n²)

- for every coefficient consider every sample

• Fast Fourier Transform (FFT): O(n*log(n))

- limited to power-of-two sizes

- reason why FT is affordable for many applications in the first place

• „ Fourier Transform“ refers to:

- the transformation itself

- frequency domain representation of a function

29.06.2015 17

Fourier Transform

• convolution theorem:

convolution in time domain <-> multiplication in frequency domain

• band-pass filter easy to realize in frequency domain

• uncertainty principle:

- concentrated info in one domain is spread out in the other domain

- perfect bandpass filter -> heavy ringing artifacts

- problematic for extracting localized frequency information

29.06.2015 18

Fourier Transform

29.06.2015 19

FT low-pass

iFT

Fourier Transform

29.06.2015 20

Roadmap

• Motivation




29.06.2015 21

Wavelet Transform

• FT provides no local frequency information

- what part of the image resulted in a particular freq. response is unknown

• wavelets: bandpass filter kernels with good localization

- localized both in time and frequency domain

• FT: „frequency“ <-> WT: „scale“

- small scale <-> high frequencies; details

- large scale <-> low frequencies; coarse features

29.06.2015 22

"MexicanHatMathematica" by JonMcLoone - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:MexicanHatMathematica.svg#/ media/File:MexicanHatMathematica.svg

Wavelet Transform

• continuous wavelet transform (CWT)

• wavelets are derived from a mother wavelet

- translation + scaling

29.06.2015 23

dtttfs s )()(),( *

,

dsdtstf s )(),()( ,

s

t

sts

1)(,

Wavelet Transform

• wavelet properties: admissibility condition

- implies that the FT vanishes at the zero frequency

- wavelet features band-pass like spectrum

- filter integrates to zero in time domain

- must oscillate, i.e. wavelet is a wave

29.06.2015 24

d

2)(

0)0( 0)( dtt

Wavelet Transform

• wavelet properties: regularity conditions

- concentration in both time and frequency domain

- imply smoothness

- moments of the wavelet:

- vanishing moments: moments that are (close to) zero

-> high number of vanishing moments == good quality

-> reduce the number of required coefficients

29.06.2015 25

dtttM p

p )(

Wavelet Transform

• wavelet properties: regularity conditions

- write Taylor series with moments (here translation = 0):

- for n vanishing moments: coefficients decay as fast as s^(n+2)

29.06.2015 26

...

!2

)0(

!1

)0()0(

1)0,( 3

2

)2(2

1

)1(

0 sMf

sMf

sMfs

s

)(

!

)0(... 21

)(nn

n

n

sOsMn

f

Wavelet Transform

• discrete wavelets

- reduce the redundancy of the CWT

- wavelets can only be scaled and translated at discrete steps

- j, k: integers

- usually s0 = 2, tau0 = 1

29.06.2015 27

j

j

jkj

s

skt

st

0

00

0

,

1)(

Wavelet Transform

• results in series of wavelet coefficients

- instead of infinite amount like in the case of the CWT

• wavelet series decomposition

• reconstruction through summation of weighted wavelets

- if wavelets orthogonal

29.06.2015 28

kj

kj tkjtf,

, )(),()(

otherwise 0

and if 1)()( *

,,

nk mj dttt nmkj

Wavelet Transform

• condition for reconstruction:

- energy of coefficients between positive bounds

- A > 0, B < infinity

- A, B independent of f(t)

- A == B -> wavelets behave like orthonormal basis

- A != B -> different wavelets for analysis and reconstruction

29.06.2015 29

2

,

2

,

2, fBffA

kj

kj

Wavelet Transform

• how many scales do we need?

- time compression of wavelet results in stretch and shift of spectrum

-> cover frequency domain with band-pass filters (band-pass filter bank)

• how to cover all low frequencies?

29.06.2015 30

aF

aatfF

1)}({

Wavelet Transform

• how to cover all low frequencies?

- use only finite amount of wavelets

- cover the remaining spectrum with low-pass filter

-> analytically sum up all remaining wavelets

-> results in scaling function (father wavelet)

29.06.2015 31

kj

kj tkjt,

, )(),()(

1)( dtt

Wavelet Transform

• wavelet transform realizable per iterated filter bank

• Fast Wavelet Transform (FWT): O(n)

29.06.2015 32

"Jpeg2000 2-level wavelet transform-lichtenstein" by Alessio Damato - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Jpeg2000_2-level_wavelet_transform-lichtenstein.png#/media/File:Jpeg2000_2-level_wavelet_transform-lichtenstein.png

Wavelet Transform

• Wavelet Shrinkage

- features have localized wavelet response

- noise has distributed wavelet response

- identify expected amplitude response of noise

- reduce all coeffiecients by this amount (hard/soft-thresholding)

- transform back to recover denoised image

29.06.2015 33

Roadmap

• Motivation




29.06.2015 34

Phase Preserving Denoising

• how to measure quality of denoising?

-> compare denoised result with noise-free original

• root-mean-square error (RMSE)

- root of sum of squared differences

-> simple offset can result in high RMSE

-> not perceptually meaningful

• structure of image depends primarily on phase information

-> preserve the phase information

29.06.2015 35

n

i

ii ban

RMSE1

2)(1


29.06.2015 36


29.06.2015 37


• need to extract local phase and amplitude information

- conventional wavelet transform provides no phase

-> use even/odd-symmetric filter pair in quadrature

-> use FFT and iFFT for fast convolution

29.06.2015 38

])(,)([)](),([ o

n

e

nnn MxIMxIxoxe

22 )()()( xoxexA nnn

))(),((atan2)( xexox nnn


• both convolutions can be combined to a single one

- utilize linearity of FT: FFT(A + B) = FFT(A) + FFT(B)

-> multiply odd-symmetric with i and add to even-symmetric

29.06.2015 39


• Log-Gabor wavelets - Gaussian transfer functions on log frequency scale

29.06.2015 40 "Log-vs-gabor" by Sfdodge - Own work. Licensed under CC BY-SA 4.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Log-vs-gabor.png#/media/File:Log-vs-gabor.png


• Log-Gabor wavelets - Gaussian transfer functions on log frequency scale

29.06.2015 41

2

2

0

2

0

2

0

2

)(exp

))/(log(2

))/(log(exp),(

f

fffG

f

))/log(()2log(/22 0fB ff

)2log(22 B


29.06.2015 42

aniso bandpass filter directional part logGabor filter

* =


• Log-Gabor wavelets

- Gaussian transfer functions on log frequency scale

- maintain zero DC for all bandwidth; Gabor wavelets do not

- fit statistics of natural images better than other wavelets

- good localization in frequency and time domain

- ~2 octaves minimize spatial extend -> good choice

- no analytical time domain representation

- not an orthonormal basis -> reconstruction only up to scale

29.06.2015 43


• construct Log-Gabor filter bank in frequency domain

29.06.2015 44 http://www.hindawi.com/journals/vlsi/2014/651943/fig7/


• construct Log-Gabor filter bank in frequency domain

• results in vectors of complex wavelet coefficients

- encode both amplitude and phase information

- per pixel: one complex coefficient per scale per orientation

29.06.2015 45


• next step: determine the threshold

• what filter response to expect from pure noise?

-> Gaussian noise in real and imaginary parts of coefficients

-> what distribution follows the amplitude response?

29.06.2015 46


• what distribution follows the amplitude response?

29.06.2015 47

),( VUY

22 2/

22

1);(

u

U euf 22 2/

22

1);(

v

V evf Yx :

)(

2

1);( 222/2/

2

2222

vuxeedvduxf vu

2222 2/

20

2/2

02)(

2

1);(

xr e

xerxrdrdxf


• what distribution follows the amplitude response?

-> Rayleigh distribution

29.06.2015 48

2

2

2

2)( g

x

g

ex

xR

"Rayleigh distributionPDF" by Krishnavedala - Own work. Licensed under CC0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Rayleigh_distributionPDF.svg#/media/File:Rayleigh_distributionPDF.svg


• Rayleigh distribution

- only dependent on mean or variance

• how to derive a threshold?

-> determine noise response at diff. scales

-> set threshold T to a multiple of

-> standard deviation beyond mean

- k typically in range 2 - 3

29.06.2015 49

2

2

2

2)( g

x

g

ex

xR

2

gr

22

2

4gr

rr kT


• how to determine the noise amplitude response?

- noise frequencies cover spectrum uniformly

- smallest scale filters cover largest part of spectrum

-> smallest scale filters produce strongest noise response

- only at feature points will the response differ from noise

-> smallest scale filters responses follow primarily Rayleigh distr.

- some „contaminations“ due to features

29.06.2015 50



-> estimate mean of Rayleigh distr. from smallest filters amplitude response

- estimate median first; derive estimated mean from median afterwards

- more robust than directly estimating the mean from the coefficients

29.06.2015 51

2

12

2

2

0 2

dxex

g

x

median

g

)2/1ln(2 gmedian

2

gr




29.06.2015 52

2

12

2

2

0 2

dxex

g

x

median

g

)2/1ln(2 gmedian

')2/1ln(2

2/)( medianAE

g

g

N

'

)2/1ln(2

1median




29.06.2015 53

')2/1ln(2

1)( medianAE N

2/

)(

N

g

AE

2

gr

22

2

4gr

rr kT


• how to determine responses for other scales?

- noise frequency coverage is uniform

-> wavelets gather energy from noise as function of bandwidth

-> in 2d response is proportional to filter center frequency

-> solution: scale response estimate of smallest scale filters

29.06.2015 54


• apply derived thresholds to coefficients

- soft or hard thresholding?

• sum even-sym. responses to recover denoised image

- no need to transform back to frequency domain first

29.06.2015 55

rr kT

Phase Preserving Denoising - Results

• 1. result

- synthetic 8 bit grayscale image

- Gaussian white noise, standard deviation of 80 gray levels

- k = 2

29.06.2015 56

rr kT


• 1. result

29.06.2015 57


• 1. result

- single scanline; incorrect vertical scaling becomes apparent

29.06.2015 58


• 2. result

29.06.2015 59

Phase Preserving Denoising - Conclusion

• denoising scheme based on wavelet shrinkage

- using complex valued wavelets

• preserves perceptually important phase information

• derives appropriate threshold (semi-)automatically

• does not incorporate any optimizations

- contructed to satisfy constraint

- could be further developed f.i. to minimize amplitude spectrum distortion

29.06.2015 60

Discussion

29.06.2015 61

* =

Sources • Phase Preserving Denoising of Images: http://www.csse.uwa.edu.au/~pk/research/pkpapers/denoise.pdf

• What Are Log-Gabor Filters and Why Are They Good?: http://www.csse.uwa.edu.au/~pk/research/matlabfns/PhaseCongruency/Docs/convexpl.html

• A Really Friendly Guide to Wavelets: http://polyvalens.pagesperso-orange.fr/clemens/wavelets/wavelets.html

• Log Gabor Filter: https://en.wikipedia.org/wiki/Log_Gabor_filter

• Wavelet transform: https://en.wikipedia.org/wiki/Wavelet_transform

• Rayleigh distribution: https://en.wikipedia.org/wiki/Rayleigh_distribution

• Image Noise: https://en.wikipedia.org/wiki/Image_noise

• Complex Number: https://en.wikipedia.org/wiki/Complex_number

• Dot Product: https://en.wikipedia.org/wiki/Dot_product

• Fourier Transform: https://en.wikipedia.org/wiki/Fourier_transform

• Cross-Correlation: https://en.wikipedia.org/wiki/Cross-correlation

• Hermitian function: https://en.wikipedia.org/wiki/Hermitian_function

• Convolution: https://en.wikipedia.org/wiki/Convolution

29.06.2015 62

http://www.csse.uwa.edu.au/~pk/research/pkpapers/denoise.pdf

http://www.csse.uwa.edu.au/~pk/research/matlabfns/PhaseCongruency/Docs/convexpl.html

http://polyvalens.pagesperso-orange.fr/clemens/wavelets/wavelets.html



https://en.wikipedia.org/wiki/Log_Gabor_filter

https://en.wikipedia.org/wiki/Wavelet_transform

https://en.wikipedia.org/wiki/Rayleigh_distribution

https://en.wikipedia.org/wiki/Image_noise

https://en.wikipedia.org/wiki/Complex_number

https://en.wikipedia.org/wiki/Dot_product

https://en.wikipedia.org/wiki/Fourier_transform

https://en.wikipedia.org/wiki/Cross-correlation



https://en.wikipedia.org/wiki/Hermitian_function

https://en.wikipedia.org/wiki/Convolution

Software

Phase Preserving Denoising of Images