Embed Size (px)
Citation preview
1
DREAMDREAM
PLANPLANIDEAIDEA
IMPLEMENTATIONIMPLEMENTATION
3
Introduction to Image ProcessingIntroduction to Image Processing
Dr. Kourosh KianiEmail: [email protected]: [email protected]: [email protected]: www.kouroshkiani.com
Present to:Amirkabir University of Technology (Tehran
Polytechnic) & Semnan University
4
Lecture 05
Mask processing orSpatial filter
IMAGE FILTERING METHODS
Spatial Domain
Frequency Domain (i.e., uses Fourier Transform)
POINT PROCESSING METHODS
Convert a given pixel value to a new pixel value based on some predefined function.
AREA PROCESSING METHODS
Need to define:1. Area shape and size2. Operation
output image
•Area shape is typically defined using a rectangular mask.
• Area size is determined by mask size.e.g., 3x3 or 5x5
• Mask size is an important parameter!
AREA SHAPE AND SIZE
TERMINOLOGY
Neighborhood operation work with values of the image pixels and corresponding values of a sub image.
The sub image is called: Filter Mask Kernel Template Window
The values in a filter sub image are referred to as coefficients, rather than pixels.
Our focus will be on masks of odd sizes, e.g. 3x3, 5x5,…
OPERATION
Typically linear combinations of pixel values. e.g., weight pixel
values and add them together.
Different results can be obtained using different weights. e.g., smoothing,
sharpening, edge detection).
mask
EXAMPLE
0.50.5 00
0
0 00
mask
8
Modified image dataLocal image neighborhood
6 141 81
5 310
1
COMMON LINEAR OPERATIONS
Correlation
Convolution
CORRELATION
A filtered image is generated as the center of the mask visits every pixel in the input image.
g(i,j)
h(i,j)
f(i,j)filteredimage
n x n mask
HANDLING PIXELS CLOSE TO BOUNDARIES
0 0 0 ……
……
……
……
….0
• Option 1: Zero padding
• Option 2: Wrap around
• Option 3: Reflection
0 0 0 1 2 3 4 5 0 0 0
4 10 16 22 22
1 2 3*
3 4 5 1 2 3 4 5 1 2 3 4 5
19 10 16 22 23 5
3 2 1 1 2 3 4 5 5 4 3 2
7 10 16 22 27 5
CONVOLUTION PROPERTIES
Commutative:A*B = B*A
Associative:(A*B)*C = A*(B*C)
Linear: A*(B+C)= A*B+ A*B
Shift-InvariantA*B(x-x0,y-yo)= (A*B) (x-x0,y-yo)
THE DELTA KERNEL
otherwise
xxifxx
0
1 00
0 1 0 x
A(x)*(x) = A(x)
otherwise
yyxxifyyxx
0
&1, 00
000 0 00 1 00 0 0
),( yx
A(x,y)*(x,y) = A(x,y)
Due to shift-invariance:
A(x,y)*(x-x0,y-y0) = A(x-x0,y-y0)
0 0 0
0 1 00 0 0
0 0 0
0 0 00 0 1
(x,y) (x-1,y-1)
1 2 3
4 5 67 8 9
A
*
0 0 0
0 0 00 0 1
(x-1,y-1)
=0 0 0
0 1 20 4 5
A(x-1,y-1)
(Zero padding)
9 7 8
3 1 26 4 5
A(x-1,y-1)
(Wrap around)=
CORRELATION – EXAMPLE
GEOMETRIC INTERPRETATION OF CORRELATION
Suppose x and y are two n-dimensional vectors:
The dot product of x with y is defined as:
Correlation generalizes the notion of dot product
1 2 1 2( , ,..., ) ( , ,..., )n nx x x x y y y y
1 1 2 2. ... n nx y x y x y x y
. | || |cos( )x y x y using vectornotation:
x
y
θ
GEOMETRIC INTERPRETATION OF CORRELATION (CONT’D)
cos(θ) measures the similarity between x and y
Normalized correlation (i.e., divide by lengths)
2 2
2 2
2 2 2 22 1/2 2 1/2
2 2 2 2
( , ) ( , )
( , )
[ ( , )] [ ( , )]
n n
n nk l
n n n n
n n n nk l k l
h k l f i k j l
N i j
h k l f i k j l
. | || |cos( ) cos( )| || |
x yx y x y or
x y
NORMALIZED CORRELATION
• Measure the similarity between images or parts of images.
=
mask
APPLICATION : TV REMOTE CONTROL
• Traditional correlation cannot handle changes due to:
• size• orientation • shape (e.g., deformable objects).
NORMALIZED CORRELATION
?
CONVOLUTION
Same as correlation except that the mask is flipped, both horizontally and vertically.
h * f = f * h
Notation:
1 2 3
4 5 6
7 8 9
7 8 9
4 5 6
1 2 3
9 8 7
6 5 4
3 2 1
H V
2 2 2 2
2 2 2 2
( , ) ( , ) ( , ) ( , ) ( , )
n n n n
n n n nk l k l
g i j h k l f i k j l h i k j l f k l
For symmetric masks (i.e., h(i,j)=h(-i,-j)), convolution is equivalent to correlation!
CORRELATION/CONVOLUTION EXAMPLES
Correlation:
Convolution:
A = [10 20 30;40 50 60;70 80 90]A =10. 20. 30. 40. 50. 60. 70. 80. 90. H = [1 2 3;4 5 6;7 8 9]H =1. 2. 3. 4. 5. 6. 7. 8. 9.
Correlation result = (10 x 1) + (20 x 2) + (30 x 3) + (40 x 4) + (50 x 5) + (60 x 6) + (70 x 7) + (80 x 8) + (90 x 9) = 2850
Convolution and Correlation in Image Processing is quite similar except that the kernel are rotated 180 degree between these 2 operation.
Convolution and Correlation in Image Processing
A = [10 20 30;40 50 60;70 80 90]A =10. 20. 30. 40. 50. 60. 70. 80. 90. H = [1 2 3;4 5 6;7 8 9]H =1. 2. 3. 4. 5. 6. 7. 8. 9.H180= 9 8 76 5 43 2 1
Convolution and Correlation in Image Processing
Convolution result = (10 x 9) + (20 x 8) + (30 x 7) + (40 x 6) + (50 x 5) + (60 x 4) + (70 x 3) + (80 x 2) + (90 x 1) = 1650
In the real Image convolution and correlation, the kernel is sliding over every single pixel and performs above operation to form a new image
Convolution and Correlation in Image Processing
A = [10 20 30;40 50 60;70 80 90]A =10. 20. 30. 40. 50. 60. 70. 80. 90. H = [1 2 3;4 5 6;7 8 9]H =1. 2. 3. 4. 5. 6. 7. 8. 9.
corr_result = imfilter(A,H)corr_result =1350. 1680. 1950. 2520. 2850. 3120. 3150. 3480. 3750. -->conv_result = imfilter(A,rot90(H,2))conv_result =750. 1020. 1350. 1380. 1650. 1980. 2550. 2820. 3150.
Convolution and Correlation in Image Processing
Most of the time the choice of using the convolution and correlation is up to the preference of the users, and it is identical when the kernel is symmetrical. For example, a smoothing filter which is:
Convolution and Correlation in Image Processing
TWO 3X3 SMOOTHING LINEAR FILTERS
1 1 1
1 1 1
1 1 1
1 2 1
2 4 2
1 2 1
9
1 16
1
Standard average Weighted average
000
010
000
H
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
010
111
010
5
1H
Origin
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
Blur
00100
01110
11111
01110
00100
13
1H
Origin
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
Blur
Motion Blur Origin
100000000
010000000
001000000
000100000
000010000
000001000
000000100
000000010
000000001
9
1H
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
Motion blur is achieved by blurring in only 1 direction
Find Edges
00000
00000
00211
00000
00000
H
Origin
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
A filter of 5x5 instead of 3x3 was chosen, because the result of a 3x3 filter is too dark on the current image. Note that the sum of all the elements is 0 now, which will result in a very dark image where only the edges it detected are colored.
111
191
111
H
Origin
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
Sharpen
111
171
111
H
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
Extra Edge
Here's a filter that shows the edges excessively
Origin
110
101
011
H
kk=imread(‘nice.jpg');corr_result = imfilter(kk,H); imshow(corr_result)
OriginEmboss
An emboss filter gives a 3D shadow effect to the image, the result is very useful for a bumpmap of the image
30 10 20
10 25 250
20 25 30
10 ,10 ,20 ,20 ,25 ,25 ,30 ,30 ,250
min max
Min filter:
Max filter
yxNnmnmfyxf ,,,min,
yxNnmnmfyxf ,,,max,
Min & Max Filters
30 10 20
10 25 250
20 25 30
10 ,10 ,20 ,20 ,25 ,25 ,30 ,30 ,250
median
The Median Filter
yxNnmnmfmedyxf ,,,,
The Median Filter
AVERAGE FILTER FOR NOISE REDUCTION
Noisy image
3x3 average 5x5 average 7x7 average median
AVERAGING FILTER VS. MEDIAN FILTER
Filtering is often used to remove noise from imagesSometimes a median filter works better than an averaging filter
Original ImageWith Noise
Image AfterAveraging Filter
Image AfterMedian Filter
NOISY IMAGES
Original
Gaussian noise
salt & pepper
Median filter3x3 window
Gaussian blurstd=1.5
Salt & Pepper Noise:
Median filter5x5 window
Gaussian blurstd=1.5
Gaussian Noise:
RESULT OF MEDIAN FILTER
Noise from Glass effectRemove noise by median filter
CONVOLUTION Separable Filter 1 2 1
12 4 2
161 2 1
0 1 0 0 0 01 1
= 0 2 0 1 2 1 .4 4
0 1 0 0 0 0V H
h
h h
PROPERTIES OF GAUSSIAN
Convolution with self is another Gaussian
• Special case: convolving two times with Gaussian kernel of width is equivalent to convolving once with kernel of width
* =
PROPERTIES OF GAUSSIAN
• Separable kernel: a 2D Gaussian can be expressed as the product of two 1D Gaussians.
PROPERTIES OF GAUSSIAN
2D Gaussian convolution can be implemented more efficiently using 1D convolutions:
PROPERTIES OF GAUSSIAN (CONT’D)
row get a new image Ir
Convolve each column of Ir with g
EXAMPLE
2D convolution(center location only)
The filter factorsinto a product of 1D
filters:
Perform convolutionalong rows:
Followed by convolutionalong the remaining column:
*
*
=
=
O(n2)
O(2n)=O(n)
CONVOLUTION Complexity of non-separable filter: N2
Complexity of separable filter: 2N
Complexité (nb mult/add versus N)
0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Non séparable
Séparable
HOW DO WE CHOOSE THE MASK WEIGHTS?
Depends on the application. Usually by sampling certain functions and
their derivatives.
Gaussian1st derivativeof Gaussian
2nd derivativeof Gaussian
Good forimage smoothing
Good for image sharpening
NORMALIZATION OF MASK WEIGHTS
Sum of weights affects overall intensity of output image. Positive weights
Normalize them such that they sum to one. Both positive and negative weights
Should sum to zero (but not always)
1/9 1/16
• Idea: replace each pixel by the average of its neighbors.
• Useful for reducing noise and unimportant details.
•The size of the mask controls the amount of smoothing.
SMOOTHING USING AVERAGING
SMOOTHING USING AVERAGING
Trade-off: noise vs blurring and loss of detail.
3x3 5x5 7x7
15x15 25x25
original
GAUSSIAN SMOOTHING
Idea: replace each pixel by a weighted average of its neighbors
Mask weights are computed by sampling a Gaussian function
Note: weight values decrease with distance from mask center!
GAUSSIAN FILTER
The Gaussian kernel is separable and symmetric.
To construct the kernel we must sample and quantize! (basically, we “image” the function)
The kernel below is an example where sigma = 1.
1 4 7 4 1
4 16 28 16 4
7 28 49 28 7
4 16 28 16 4
1 4 7 4 1
GAUSSIAN SMOOTHING
mask size depends on σ :
• σ determines the degree of smoothing!σ=3
GAUSSIAN SMOOTHING - EXAMPLE
= 30 pixels= 1 pixel = 5 pixels = 10 pixels
AVERAGING VS GAUSSIAN SMOOTHING
Averaging
Gaussian
IMAGE SHARPENING
Idea: compute intensity differences in local image regions.
Useful for emphasizing transitions in intensity (e.g., in edge detection).
1st derivativeof Gaussian
EXAMPLE
Zooming as a Convolution Application Example
1 3 2
4 5 6
Zerointerlace
1 0 3 0 2 0
0 0 0 0 0 0
4 0 5 0 6 0
0 0 0 0 0 0
Convolve
1 1 1
1 1 1
1 1 1
The value of a pixel in the enlarged image is the average of the value of around pixels. The difference between insert 0 and original value of pixels is “smoothed” by convolution
Original Image
(1+0+3+0+0+0+4+0+5) ÷(1+1+1+1+1+1+1+1+1) = 13/9
Zooming as a Convolution Application Example
f
y
EXAMPLE
f
x
Questions? Discussion? Suggestions?
81
