FourierFourier TransformTransforminin

Image ProcessingImage Processing

Joonas KonttinenJoonas KonttinenPrzemysPrzemysłław Pykaaw PykaMika KangasMika Kangas

LappeenrantaLappeenrantaUniversity of TechnologyUniversity of Technology

ContentsIntroductionIntroductionDiscreteDiscrete Fourier Fourier TransformTransform in in imageimage processingprocessingPropertiesProperties of the Fourier of the Fourier TransfromTransfromFastFast Fourier Fourier TransformTransformApplicationsApplications of the Fourier of the Fourier TransfromTransfrom in in digitaldigitalimageimage processingprocessingImageImage processingprocessing examplesexamples

IntroductionIntroduction

Fourier Transform is an integral transform of one function into Fourier Transform is an integral transform of one function into another.another.

Jean Baptiste Joseph Fourier (1768Jean Baptiste Joseph Fourier (1768--1830), a French mathematician and 1830), a French mathematician and

physicist.physicist.

There is a whole family of Fourier Transfrom variants which havThere is a whole family of Fourier Transfrom variants which have e

many scientific applications.many scientific applications.

Linearity

Fourier transform isFourier transform islinear!linear!

It possesses the properties ofIt possesses the properties ofhomogeneityhomogeneity

andandadditivityadditivity..

Homogeneityk * x[] k * x[] ↔↔ k * X[]k * X[]

Additivityxx11[] + x[] + x22[] [] ↔↔ XX11[] + X[] + X22[][]

Scaling

If function wider in the xIf function wider in the x--direction, itdirection, it’’s spectrum s spectrum wiil become smaller in the xwiil become smaller in the x--direction.direction.

The amplitude will also be changed.The amplitude will also be changed.

f(a*x) f(a*x) ↔↔ (1/a) * F(w/a)(1/a) * F(w/a)

Time ShiftingSignal with symmetry aroundSignal with symmetry arounda vertical axis a vertical axis –– linear phaselinear phase.

A shift in the time domainA shift in the time domaincorresponds to changingcorresponds to changingthe slope of the phasethe slope of the phase

Duality of Symetry

Apart from some scaling factors at least.Apart from some scaling factors at least.Because of this property, for example, the Because of this property, for example, the

spectrum of a rectangular pulse is a sinc spectrum of a rectangular pulse is a sinc function and at the same time the spectrum of function and at the same time the spectrum of

a sinc function is a rectangular pulse.a sinc function is a rectangular pulse.

if f(x) <if f(x) <----> F(w)> F(w)then F(x) <then F(x) <----> f(> f(--w)w)

The Discrete Fourier transform in image processing...The Fourier Transform is an important image processing tool whicThe Fourier Transform is an important image processing tool which is h is used to decompose an image into its sine and cosine components.used to decompose an image into its sine and cosine components.

The output of the transformation represents the image in the FoThe output of the transformation represents the image in the Fourier or urier or frequency domain, while the input image is the spatial domain frequency domain, while the input image is the spatial domain equivalent.equivalent.

In the Fourier domain image, each point represents a particularIn the Fourier domain image, each point represents a particularfrequency contained in the spatial domain image.frequency contained in the spatial domain image.

In image processing, The Fourier Transform is used in a wide raIn image processing, The Fourier Transform is used in a wide range of nge of applications, such as image analysis, image filtering, image applications, such as image analysis, image filtering, image reconstruction and image compression.reconstruction and image compression.

How does it work?From the variants of the Fourier Transform Discrete Fourier TraFrom the variants of the Fourier Transform Discrete Fourier Transform (DFT) nsform (DFT)

is the variant used in digital image processing.is the variant used in digital image processing.

The DFT is the sampled Fourier Transform. That is why DFT does The DFT is the sampled Fourier Transform. That is why DFT does have all the have all the

frequences which form the image, but only a set of samples whichfrequences which form the image, but only a set of samples which is large is large

enough to fully describe the spatial domain image.enough to fully describe the spatial domain image.

The number of frequencies corresponds to the number of pixels inThe number of frequencies corresponds to the number of pixels in the spatial the spatial

domain image, i.e. the image in the spatial and Fourier domain adomain image, i.e. the image in the spatial and Fourier domain are of the same re of the same

size.size.

How does it work?

For a square image of size NFor a square image of size N××N, the twoN, the two--dimensional DFT is dimensional DFT is given by:given by:

f(i,j) is the image in the spatial domain and the exponential tef(i,j) is the image in the spatial domain and the exponential term rm is the basis function corresponding to each point F(k,l) in the is the basis function corresponding to each point F(k,l) in the Fourier space. Fourier space.

The equation can be interpreted as: the value of each point F(k,The equation can be interpreted as: the value of each point F(k,l) l) is obtained by multiplying the spatial image with the is obtained by multiplying the spatial image with the corresponding base function and summing the result. corresponding base function and summing the result.

How does it work?

The basis functions are sine and cosine waves with increasing The basis functions are sine and cosine waves with increasing frequencies, i.e. F(0,0) represents the DCfrequencies, i.e. F(0,0) represents the DC--component of the image component of the image which corresponds to the average brightness and F(Nwhich corresponds to the average brightness and F(N--1,N1,N--1) 1) represents the highest frequency.represents the highest frequency.

In a similar way, the Fourier image can be reIn a similar way, the Fourier image can be re--transformed to the transformed to the spatial domain. The inverse Fourier transform is given by:spatial domain. The inverse Fourier transform is given by:

How does it work?

To obtain the result for the previous equations, a double sum To obtain the result for the previous equations, a double sum has to be calculated for each image point. However, because has to be calculated for each image point. However, because the Fourier Transform is separable, it can be written as:the Fourier Transform is separable, it can be written as:

wherewhere

How does it work?

Using those last two formulas, the spatial domain image is firstUsing those last two formulas, the spatial domain image is firsttransformed into an intermediate image using N onetransformed into an intermediate image using N one--dimensional Fourier dimensional Fourier Transforms.Transforms.

This intermediate image is then transformed into the final imagThis intermediate image is then transformed into the final image, again e, again

using N oneusing N one--dimensional Fourier Transforms.dimensional Fourier Transforms.

Expressing the twoExpressing the two--dimensional Fourier Transform in terms of a series of dimensional Fourier Transform in terms of a series of

2N one2N one--dimensional transforms decreases the number of required dimensional transforms decreases the number of required

computations.computations.

How does it work?How does it work?

The ordinary oneThe ordinary one--dimensional DFT still has complexity which can be dimensional DFT still has complexity which can be reduced with the use of Fast Fourier Transform (FFT) to compute reduced with the use of Fast Fourier Transform (FFT) to compute the onethe one--dimensional DFTs.dimensional DFTs.

It is a significant improvement, in particular for large imagesIt is a significant improvement, in particular for large images..

There are various forms of the FFT and most of them restrict thThere are various forms of the FFT and most of them restrict the size of the e size of the

input image that may be transformed, often to where n is an inteinput image that may be transformed, often to where n is an integer.ger.

How does it work? ...magnitude and phaseHow does it work? ...magnitude and phaseThe Fourier Transform produces a complex number valued output imThe Fourier Transform produces a complex number valued output image which age which

can be displayed with two images, either with the real and imagican be displayed with two images, either with the real and imaginary part or with nary part or with magnitude and phase.magnitude and phase.

In image processing, often only the magnitude of the Fourier TIn image processing, often only the magnitude of the Fourier Transform is ransform is

displayed, as it contains most of the information of the geometrdisplayed, as it contains most of the information of the geometric structure of the ic structure of the

spatial domain image.spatial domain image.

The Fourier image can also be reThe Fourier image can also be re--transformed into the correct spatial domain after transformed into the correct spatial domain after

some processing in the frequency domain...(both magnitude and phsome processing in the frequency domain...(both magnitude and phase of the imgae ase of the imgae

must be preserved for this).must be preserved for this).

The Fourier domain image has a much greater range than the imaThe Fourier domain image has a much greater range than the image in the spatial ge in the spatial

domain. Hence, to be sufficiently accurate, its values are usualdomain. Hence, to be sufficiently accurate, its values are usually calculated and ly calculated and

stored in float values.stored in float values.

Fast Fourier Transform

1)1) Decomposed an Decomposed an NN point time domain signal point time domain signal into into NN time domain signals each composed of time domain signals each composed of a single point.a single point.

2)2) Calculate the Calculate the NN frequency spectra frequency spectra corresponding to these corresponding to these NN time domain time domain signals.signals.

3)3) The The NN spectra are synthesized into a single spectra are synthesized into a single frequency spectrum. frequency spectrum.

How the FFT works?How the FFT works?

Fast Fourier TransformDecompositionDecomposition

16 point signal is decomposed into 16 signal each containing a s16 point signal is decomposed into 16 signal each containing a single pointingle point

LogLog22NN stages required in this decompositionstages required in this decomposition

Fast Fourier TransformDecompositionDecomposition

The decompositionThe decompositionis nothing moreis nothing morethan a than a reorderingreorderingof the samples in the signal.of the samples in the signal.

Fast Fourier TransformCalculatingCalculating

The next step is to find the frequency spectra of The next step is to find the frequency spectra of the 1 point time domain signals. the 1 point time domain signals.

The frequency spectrum of a 1 point signal is The frequency spectrum of a 1 point signal is equal toequal to itselfitself..

Don't forget that each of the 1 point signals is Don't forget that each of the 1 point signals is now a frequency spectrum, and not a time now a frequency spectrum, and not a time

domain signaldomain signal.

Fast Fourier Transform

The last step is to combine the The last step is to combine the NN frequency spectra in the exact reverse orderfrequency spectra in the exact reverse orderthat the time domain decomposition took place.that the time domain decomposition took place.

The last stage results in the output of the FFT, a 16 point freqThe last stage results in the output of the FFT, a 16 point frequency spectrum.uency spectrum.

SynthesisSynthesis

Fast Fourier TransformSynthesisSynthesis

When the time domain signal is diluted with zeros, the frequencyWhen the time domain signal is diluted with zeros, the frequency domain is domain is duplicated.duplicated.

If the time domain signal is also shifted by one sample during tIf the time domain signal is also shifted by one sample during the dilution,he dilution,the spectrum willthe spectrum willadditionalyadditionalybe multipliedbe multipliedby sinusoid.by sinusoid.

Fast Fourier TransformSynthesisSynthesis

Synthesis Flow Diagram Synthesis Flow Diagram for combiningfor combiningtwo 4 point spectra into a single 8 point spectrumtwo 4 point spectra into a single 8 point spectrum..

ButterflyButterfly –– basic calculation element in FTTbasic calculation element in FTT

Fast Fourier TransformAlghoritmAlghoritm

Loops:Loops:

1)1) runs through the runs through the LogLog22NN stages stages 2)2) moves through each of the individualmoves through each of the individual

frequency spectra in the stage being worked onfrequency spectra in the stage being worked on3) uses the butterfly to calculate the points3) uses the butterfly to calculate the points

in each frequency spectrain each frequency spectra

Overhead boxesOverhead boxesdetermine the beginning and ending indexesdetermine the beginning and ending indexesfor the loops, as well as calculatingfor the loops, as well as calculatingthe sinusoids needed in the butterfliesthe sinusoids needed in the butterflies

Fourier transform applications

Used in many scientific areasUsed in many scientific areasPhysicsPhysicsNumber theoryNumber theorySignal processing (inc. image processing)Signal processing (inc. image processing)Probability theoryProbability theoryStatisticsStatisticsCryptographyCryptographyAcousticsAcousticsOpticsOpticsEtc, etcEtc, etc……

Fourier transform in signal processing

Includes: (for example) audio and image signal processingIncludes: (for example) audio and image signal processing

Audio and image signal are very alike (1D vs. 2D)Audio and image signal are very alike (1D vs. 2D)Both can be continuous (analog) or discrete (digital)Both can be continuous (analog) or discrete (digital)Same kind of filters can be usedSame kind of filters can be used

Fourier transform is used for a easier way to apply different Fourier transform is used for a easier way to apply different kind of filters to the signalkind of filters to the signal

Filters are used for:Filters are used for:Removing unwanted frequencies from signalRemoving unwanted frequencies from signalRemoving noise from signalRemoving noise from signalAltering signal somehowAltering signal somehow

1D and 2D Fourier transform comparison

Basic sine signal:Basic sine signal:

Fourier transforms of that signal:Fourier transforms of that signal:

Position of peaks will depend on signal Position of peaks will depend on signal frequencyfrequency

1D and 2D Fourier transform comparison

Important part:Important part:Sine signal with added positive constant:Sine signal with added positive constant:

Fourier transform of the same signal:Fourier transform of the same signal:Notice the extra peak at originNotice the extra peak at origin

1D and 2D Fourier transform comparison

On 2D images, you are watching the On 2D images, you are watching the waveform from top viewwaveform from top viewYou can imagine that the white parts are You can imagine that the white parts are top of the wave and black parts are the top of the wave and black parts are the bottom parts (see picture on the right)bottom parts (see picture on the right)Fourier transform:Fourier transform:

1D and 2D Fourier transform comparison

In the 2DIn the 2D--image Fourier transforms, there is always a image Fourier transforms, there is always a peak at originpeak at originThe value at origin is the average intensity/color of The value at origin is the average intensity/color of the imagethe imageNo negative valuesNo negative valuesIn the 2DIn the 2D--image Fourier transform, higher image Fourier transform, higher frequencies represent detail in a picture and lower frequencies represent detail in a picture and lower frequencies represent the gradient changes in the frequencies represent the gradient changes in the picturepictureThat feature is used to apply filters to imagesThat feature is used to apply filters to images

Image processing

When a image is taken by a image sensor, it isnWhen a image is taken by a image sensor, it isn’’t t always usable in raw formatalways usable in raw format

There can be noise or other disturbance in the image There can be noise or other disturbance in the image It can be caused by system it self or some external sourceIt can be caused by system it self or some external source

Useful information have to be somehow separated from Useful information have to be somehow separated from the original imagethe original imageImage information have to be altered to some other formImage information have to be altered to some other formImage could be to big for transfer Image could be to big for transfer

Image compressionImage compression

Image filtering

Image filtering workflow with Fourier Image filtering workflow with Fourier transform:transform:

1.1. Image is converted to spectral form with Fourier Image is converted to spectral form with Fourier transform (Usually Fast Fourier transform is transform (Usually Fast Fourier transform is used).used).

2.2. Filter is applied to spectral imageFilter is applied to spectral image3.3. Image is converted back to its original form Image is converted back to its original form

using inverted Fourier transform using inverted Fourier transform

Few filter examples

Low pass filter = Image blurLow pass filter = Image blur

Few filter examplesHigh pass filter = EdgesHigh pass filter = Edges

Can be used in edge detectionCan be used in edge detection

Few filter examples

Sharpening = boosting high frequency pixelsSharpening = boosting high frequency pixels

Few filter examplesRemoving unwanted frequenciesRemoving unwanted frequencies

MoreMore……

Steven W. Smith Steven W. Smith „„The Scientist and Engineer's The Scientist and Engineer's Guide to Digital Signal ProcessingGuide to Digital Signal Processing””

http://student.kuleuven.be/~m0216922/CG/fourier.htmlhttp://student.kuleuven.be/~m0216922/CG/fourier.htmlhttp://www.cee.hw.ac.uk/hipr/html/fourier.htmlhttp://www.cee.hw.ac.uk/hipr/html/fourier.htmlhttp://www.cs.unm.edu/~brayer/vision/fourier.htmlhttp://www.cs.unm.edu/~brayer/vision/fourier.htmlhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MARSHALhttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MARSHALL/node15.html#SECTION00141000000000000000L/node15.html#SECTION00141000000000000000http://micro.magnet.fsu.edu/primer/java/digitalimaging/processinhttp://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/fouriertrg/fouriertransform/ansform/

Bibliography