Convolution Non-linear filters Histograms 2Problem: edge smoothing of linear filters 29 The median filter Example: “salt and pepper” noise Original image Filtered with the median

1

Vision industrielle 2002/2003

Christian [email protected]

Département Génie ProductiqueINSA de Lyon

Session 4 - Image Processing

2

Table of contents

Introduction Motivation, humanvision, history, digital images,methodology

Image Acquisition Imageformation, sensors, cameras,scanners, illumination, color

Fourier TransformationNoise, mage filtering in thefrequency domain, correlation

Image Processing linear andnon linear filtering, histograms

Image Analysis thresholding,gradients, laplacian, edgedetection, region segmentation

HistogramsNon-linear filtersConvolution

Mathematical morphology,3D binary images, erosion,dilation, neighborhoodtransformations, stereo vision,shape from X

QuestionsQuestions to the Lecture

PDF files: http://rfv.insa-lyon.fr/~wolf/enseignement

3

Local and semi-local smoothing

Instead of restoration (by inverting the degradationeffects), local treatment on the image enhances its quality.

Examples:• Salt and pepper noise• Holes or cuts in the object contours• ...

HistogramsNon-linear filtersConvolution 4

Linear noise filtering: example

Replace each pixel by the mean of its 4 neighbors:

f'(x,y)=( f(x-1,y-1) + f(x,y-1) + f(x-1,y) + f(x,y) )/ 4

1 1

1 1

1( , ) ( , )9

u v

u vF x y f x v y u

= =

=− =−

= + +∑ ∑

Replace each pixel by the mean of its 9 neighbors:

Original imagewith noise

Filtered image


5

Convolution: introduction

=

111111111

91h

−

=1001

rh

−

=0110

lh

Mean filter

Roberts filter


1D convolutionThe continuous case:

The discrete case:

Commutative operation!

x

Filtered pixel

h(x) = ”filter kernel”origin

f

h


7

2D convolution

h0 M-1

0

N-1

f xx→

y↓

y

f⊗h x

y×

The discrete case:


Why flip the kernel?

f⊗hfh

flipped kernel

Reason: so that f⊗h = h⊗f


9

The impulse response (IR)“2D Dirac image”: 1D case: 2D case:

The filter kernel equals the response of the filter to a Dirac impulse:

Transfer function: the Fourier transform of the impluse response: H=FT(h)Contains for each frequency the complex factor by which a periodicstructure is multiplicated.

The impulse response of theoperator ∅:


PropertiesCommutativity:

Distributivity (over addition):

Linearity:

α,β ... scalars

Associativity (combination of filters):


11

PropertiesShift invariance:

Shift operator:


Linear and shift-invariant operators

Thought experiment: what kind of linear and shift-invariantoperators can there exist?

Let’s consider an operator ∅with following properties:

Linearity

Shift invariance

h is the impulse response of this operator

13

A linear andshiftinvariantoperator isequivalentto aconvolutionwith theimpulseresponse oftheoperator!!!


Iteratively convolving an image N times by a filter of sizeM corresponds to the application of a single convolutionof a filter of size M* = 2 × ( (M-1) / 2 ) × N ) +1

3

1 2 11 2 4 216

1 2 1g

=

=

1464141624164624362464162416414641

2461

5g

2 x a filter of size 3 = 1 x filter of size 53 x a filter of size 3 = 1 x filter of size 7

⇔2 ×

Iterative application

One kernel: may reduce complexitySeveral kernels: may reducing programming effort


15

SeparabilityThe Gaussian and the average filter are separable:

f g f g g f g gx y y x⊗ = ⊗ ⊗ = ⊗ ⊗(( ) ) ( ) )

Advantage: Computational complexity for a MxN filter:Non-separable implementation: MxN operationsSeparable implementation: M+N operations

1 1 1 1 11 1 1 1 1

1 1 1 1 1 1251 1 1 1 11 1 1 1 1

B

=

11

1 1511

Bx

=

[ ]1 1 1 1 1 15

By =

1 2 11 2 4 216

1 2 1G

=

11 241

Gy =

[ ]1 1 2 14

Gx = [ ]1 1 2 12 1 2 1 2 4 21 1 2 1

× =

since

since [ ]

1 1 1 1 1 11 1 1 1 1 1

1 1 1 1 11 1 1 1 1 11 1 1 1 1 11 1 1 1 1 1

× =


Convolution in the frequency domain

Convolution in the spatial domain corresponds to amultiplication in the frequency domain:

)v,u(H).v,u(F)y,x(h)y,x(f ⊗

Can lead to a large decrease in computational complexity(depending on the filter operations)

Example: filter bank etc.


17

Border treatmentWhat do we do with missing pixels on the border?

0 M-1

• Cyclic replacement (only option in thefrequency domain!!)

• Zeros

• Interpolation (linear, quadratic, ...)


g =

19

1 1 11 1 11 1 1

1 111 14

g =

The obvious filter, corresponds to a convolution with the following kernel:

F(x,y)=f⊗g ⇔1 1

1 1

1( , ) ( , )9

u v

u vF x y f x v y u

= =

=− =−

= + +∑ ∑

The average filter (“blur”)


19

g =

125

1 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 1

Bigger kernel = more blur

Computational complexity!

Increasing the filter size (“blur more”)


g =

116

1 2 12 4 21 2 1

Discrete approximation of a 2D Gaussianfilter:

Weighted mean, the center pixels are moreimportant than the border pixels

G x yx y

x yx ye( , ) =

− +

1

2

12

2 2

πσ σσ σ

=

1464141624164624362464162416414641

2461

5g

Gaussian low pass filter (“soften”)


21

Original image Average filter Gaussian filter

Comparison: Gaussian & average filter


Gaussian low pass filter: details


23

Sharpen (rehaussement)Sharpened image = (A-1) original image + high pass filtered image

−−−−−−−−

111181111

−−−

−

010141010 Laplacian filter (high pass filter)

−−−−−−−−

111191111

A=1: standard laplacian filterA>1: a part of the original image is added to the high pass

A=2

Noise!!!


Strong sharpenStronger sharpening lets the noise in the image appear!

Example: scanner noise appears after sharpening


25

Adaptive smoothingThe coefficients in the filter mask depend on the data:

Position in the maskPixel to filter

Discontinuity function:

Normalization constant:


Problem: edgesmoothing oflinear filters

26

The min/max filterGoal: Contrast enhancementThe original pixel value is compared to the minimum of thepixels in the neighborhood (=m) and the maximum (=M):

1218

25 32

25 36 57

4819Minimum m=12Maximum M=57The center value 19 is closer to m, i.e. the new value ism=12.


27

Min/max: example

Original image Filtered image


The median filterNon-linear filter developed in order to preserveedges. Chooses the median value of the theneighboring pixels, i.e. the value at rank K/2 in theordered list of gray values.

1218

25 32

25 36 57

484

12 18 25 3225 36 574841 K/2 K

replace 4 by 25

The median filter is very efficient against noise, avoidsblurring and preserves colors transitions around the objects.


Problem: edgesmoothing oflinear filters

29

The median filterExample: “salt and pepper” noise

Original image Filtered with the median filter


The “mode” filter

1218

25 32

25 36 57

484

The mean value is not taken from the whole ordered list L ofgray values, but from a well chosen compact part of size K/2:

12 18 254

12 18 25 25

18 25 3225

12 18 25 3225 36 57484

...

From the chosen interval, themedian value is taken


31

The symmetric nearest neighbor filterGoal: smoothing regions while preserving edges and thinstructures.

P2

P1

P3 P4

Q4 Q3 Q2

Q1M

Step 1: For each couple (Pi, Qi):

Step 2:

mean or median


ExamplesOriginal image SNN 1 iteration SNN 3 iterations


33

Image enhancement

The image enhancement problem is not well defined. When isan image better than another?

Contrast enhancement:• Frequency filtering (as in image restoration)• Histogram modification• Color processing• Contrast techniques


Histogram techniquesStatistical distribution of the pixel values f(x,y)independent of their position: Appearancefrequency of each gray level (color)

Histo(i)=Card{Pixel(x,y)=i}

grayvalues

Pixel count

Operations:f’(x,y) = T(f(x,y))

T is a function on the gray value,independent of the pixel position!


35

Gray level correction functions f’(x,y) = T(f(x,y))

Stretch(recadrage)

Dilation of thebright zones

Dilation of thedark zones

Suppression ofthe 2 highestbits in the grayvalue.

7 6 5 4 3 2 1 0


Gray value correction functionsCorrection ofthe non-linearity of filmor monitor

Inversion of theimage

Extraction of anintensitywindow


37

Redistribute the colors [a..b]between [0..255]

83 gray levels are redistributed into the whole dynamic range of 0..255

Histogram stretch


Redistribute the colors linearly between [0..255]: The graylevels most frequently used get a higher dynamic range

Histogram equalization

The cumulative distribution function

Exponential

Cubic

Logarithmic

Alternative histogram shapes (“histogram specification”):

A uniform histogram is created!


39

Histogram equalization• Contrast is increased at the most populated range of

brightness values of the histogram (or "peaks").• Automatically reduces the contrast in very light or dark

parts of the image associated with the tails of a normallydistributed histogram.

• Equalization maximizes the entropy in the image.• The histogram is not totally flat, due to the discrete

nature of the histogram. A really flat histogram can becomputed by separating classes.


Histogram equalization: example


41

Histogram equalization: example


Iterative histogram peak sharpening(Amincissement itératif d’histogramme)

Shift pixels between histograms so that bins that are largerthan their neighborhood receive pixels from thisneighborhood.

For each bin i, consider the neighborhood [i-m, i+m]:

if H(i) > A A = mean (H(i-m),...,H(i+m)):

r = (H(i)-A)/H(i)move rH(i+j) pixels from i+j to i+j-1 for j = 1..mmove rH(i-j) pixels from i-j to i-j+1 for j = 1..m

i-m i i+m

43

Pseudo ColorsA color image is created from agrayscale image by assigning colors todifferent gray values.

Background: humans are able todistinguish only a few gray values butmillions of different colors.


False colorsModification of the colors of animage which is originally incolor:

• Create unnatural colors(purple sun, green sky,...)

• Change the spectrum tomatch human perception

• Emphasize single objects bychanging their colors(contrast against thebackground).

• ...

Example: composite image


Documents

Convolution Non-linear filters Histograms 2Problem: edge smoothing of linear filters 29 The median filter Example: “salt and pepper” noise Original image Filtered with the median