Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
82
Geometric transformationsExample comparing f1, f2, and f3
Shift: x+0.25
à Linear( )( )( ) ( )( ) ( ) ( )( )( )( )
0
1
2
3
4
5
6
0
0
1 0.25 10 7.5
1 0.75 10 1 0.25 20 17.5
0.25 20 0.75 30 27.5
0.25 30 7.5
0
f x
f x
f x
f x
f x
f x
f x
=
=
= - × =
= - × + - × =
= × + × =
= × =
=
83
Geometric transformationsExample comparing f1, f2, and f3
Shift: x+0.25
à Bi-cubic( )( ) ( )
( ) ( )( )
( ) ( )( )( )
0
2 31
2 32
2 3
2 33
2 3
2 3
0
4 8 1.25 5 1.25 1.25 10
1.41
1 2 0.25 0.25 10
4 8 1.25 5 1.25 1.25 20
6.09
1 2 0.75 0.75 10
1 2 0.25 0.25 20
4 8 1.25 5 1.25 1.25 30
16.56
f x
f x
f x
f x
=
= - × + × - ×
= -
= - × + ×
+ - × + × - ×
=
= - × + ×
+ - × + ×
+ - × + × - ×
=
84
Geometric transformationsExample comparing f1, f2, and f3
Shift: x+0.25
à Bi-cubic ( ) ( )( )( )
( ) ( )( )
( ) ( )
2 34
2 3
2 3
2 35
2 3
2 36
1 2 0.75 0.75 20
1 2 0.25 0.25 30
4 8 1.75 5 1.75 1.75 10
32.18
1 2 0.75 0.75 30
4 8 1.75 5 1.75 1.75 20
7.97
4 8 1.75 5 1.75 1.75 30
1.41
f x
f x
f x
= - × + ×
+ - × + ×
+ - × + × - ×
=
= - × + ×
+ - × + × - ×
=
= - × + × - ×
= -
85
Geometric transformations
86
Geometric transformations
Non-affine (non-linear) transformation models!
87
Overview topics today…
Distance measures
Geometric transformations
Book à Chapter 4 & Chapter 5 (Section 5.3)
Convolution (filtering)
88
Convolution (filtering)
Filtering
Book à Section 5.3
Convolution
Smoothing
Linear operatorsDenoising
Sharpening
Kernel
Window Filter
Point spread function
Impulse response
Pre-processing
Blurring
Super-resolution
Separable
Prewitt
Sobel
Canny
RobinsonKirsch
Scale-space
Fourier
Deconvolution
Anisotropic filter
Image derivative
Laplacian
Edge
Restoration
High and low-pass filter
Wiener
89
FilteringDictionary
90
FilteringIn digital image processing!
“A digital filter is ‘a system’ that performsmathematical operations on a sampled, discretedigital image to reduce or enhance certain aspects ofthat image.”
“Filter”
91
FilteringIntroduction
Ø Single pixel transformations à signal intensity andbrightness transformations (histogram equalization)
Ø Global (all pixels) analyses à geometric (rigid, affine,projective,…) transformations
Ø Neighborhood processing (local region of pixels) àspatial filtering
92
FilteringSpatial filtering of a (2D) digital image
Linear filtering methods
Non-linear filtering methods
93
FilteringDefinition Linear spatial filteringØ Consider filter H on input image f(x,y), producing
output image g(x,y):
( ) ( ), ,H f x y g x y=é ùë û
94
FilteringDefinition Linear spatial filteringØ Consider filter H on input image f(x,y), producing
output image g(x,y):
Ø H is called linear if (for real values ai and aj):
“Additivity” & “Homogeneity”
( ) ( ), ,H f x y g x y=é ùë û
( ) ( ) ( ) ( )( ) ( )
, , , ,
, ,i i j j i i j j
i i j j
H a f x y a f x y a H f x y a H f x y
a g x y a g x y
é ù é ù+ = +é ùë ûë û ë û= +
95
FilteringExample Linear spatial filteringØ Sum operator “S” is linear
( ) ( ) ( ) ( )( ) ( )
( ) ( )
, , , ,
, ,, ,
i i j j i i j j
i i j j
i i j j
a f x y a f x y a f x y a f x y
a f x y a f x ya g x y a g x y
é ù é ù+ = +é ùë ûë û ë û= += +
å å åå å
96
FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear; consider ‘images’:
1
0 22 3
f é ù= ê úë û
2
6 54 7
f é ù= ê úë û
97
FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear; consider ‘images’:
Ø Find an example of a1 and a2 where the followingequality does not hold:
1
0 22 3
f é ù= ê úë û
2
6 54 7
f é ù= ê úë û
1 2 1 2
0 2 6 5 0 2 6 5max max max
2 3 4 7 2 3 4 7a a a aì ü ì ü ì üé ù é ù é ù é ù
+ = +í ý í ý í ýê ú ê ú ê ú ê úë û ë û ë û ë ûî þ î þ î þ
98
FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear:
1 2
1 2 1 2
,0 2 6 5 0 2 6 5max max max
2 3 4 7 2 3 4 7
a aa a a a
"ì ü ì ü ì üé ù é ù é ù é ù+ = +í ý í ý í ýê ú ê ú ê ú ê ú
ë û ë û ë û ë ûî þ î þ î þ
?
99
FilteringAlgorithm for 2D Linear spatial filtering1. Select a ‘center’ pixel (x,y) in the image
2. Perform some operation that involves the ‘neighboringpixels’ of (x,y)
3. The result of this operation is called the “response” ofthis process at point (x,y)
4. Repeat the procedure for each point (x,y) in the image
100
FilteringAlgorithm for 2D Linear spatial filtering1. Select a ‘center’ pixel (x,y) in the image
5 3 1 41 5 6 16 1 5 71 5 1 8
4451
7 5 7 1 6
1 2 3 11 2 1 2
97
2192
1211
1 9
41
21
101
FilteringAlgorithm for 2D Linear spatial filtering2. Perform some operation that involves the ‘neighboring
pixels’ of (x,y), e.g., the 8-connectivity neighborhood:
5 3 1 41 5 6 16 1 5 71 5 1 8
4451
7 5 7 1 6
1 2 3 11 2 1 2
97
2192
1211
1 9
41
21
102
FilteringAlgorithm for 2D Linear spatial filtering2. Perform some operation that involves the ‘neighboring
pixels’ of (x,y), e.g., the 8-connectivity neighborhood:
“Average” operation: 1 à (2+1+2+3+1+4+5+6+1)/9 = 2.8
5 3 1 41 5 6 16 1 5 71 5 1 8
4451
7 5 7 1 6
1 2 3 11 2 1 2
97
2192
1211
1 9
41
21
103
FilteringAlgorithm for 2D Linear spatial filtering3. The result of this operation is called the “response” of
this process at point (x,y)
“Average” operation: 1 à 2.8
5 3 2.8 41 5 6 16 1 5 71 5 1 8
4451
7 5 7 1 6
1 2 3 11 2 1 2
97
2192
1211
1 9
41
21
104
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
105
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0
0 0 00
106
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0 0 0
107
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0 0 0
108
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0 0 0
109
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0
0 0 00
110
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0
00
111
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
112
FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
“Average” operation on 3 x 3 neighborhood
0 0 0
00
113
Filtering2D Linear spatial filteringØ Formal algebraic notation:
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
11 12 13
21 22 23
31 32 33
2 1 23 46 11
5
f f ff f f f
f f f
é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û
114
Filtering2D Linear spatial filteringØ Formal algebraic notation:
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
11 12 13
21 22 23
31 32 33
2 1 23 46 11
5
f f ff f f f
f f f
é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û
11 12 13
21 22 23
31 32 33
1 1 11 11 1
1 11
9
w w ww w w w
w w w
é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û
Mask / filter mask / window / kernel
115
Filtering2D Linear spatial filteringØ Formal algebraic notation:
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
11 12 13
21 22 23
31 32 33
2 1 23 46 11
5
f f ff f f f
f f f
é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û
11 12 13
21 22 23
31 32 33
1 1 11 11 1
1 11
9
w w ww w w w
w w w
é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û
Mask / filter mask / window / kernel
Response value g à ( )3 3
1 12.8ij ij
i jg w f
= =
= =åå
116
Filtering2D Linear spatial filteringØ Formal algebraic notation at any position (x,y):
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
( ),f x y
( ),w a b
117
Filtering2D Linear spatial filteringØ Formal algebraic notation at any position (x,y):
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
( ),f x y
( ),w a b
Filtered image g(x,y)
118
Filtering2D Linear spatial filteringØ Formal algebraic notation at any position (x,y):
5 3 1 41 5 6 16 1 5 7
445
1 2 3 11 2 1 2
97
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
1.9 2.8 2.8 3.3
2.3 3.7 3.7 4.1
1.4 2.7 2.8 3.1
2.4
2.8
1.9
0.7 1.1 1.2 2.6
1.6 2.1 2.1 3.6
2.1
3
( ),f x y
( ),w a b
Filtered image g(x,y)
Neighborhood size: 3 x 3
𝑔 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏,
-./,
,
0./,
119
2D Linear spatial filteringØ Formal algebraic notation at any position (x,y) and
with general size of mask w being (2r+1) x (2s+1) :
Filtering
fN1 fNM
f11 …
…
f1M
w00
wr-s wrs
w-r-s …
…
w-rs
( ) ( ), ; , with 1,..., ; 1,...,xy xyf f x y g g x y x N y M= = = =
gN1 gNM
g11 …
…
g1M
𝑔 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏1
-./1
2
0./2
120
Filtering2D Linear spatial filtering (average)
Original image 3 x 3 neighborhood
121
Filtering2D Linear spatial filtering (average)
Original image 5 x 5 neighborhood
122
Filtering2D Linear spatial filtering (average)
Original image 7 x 7 neighborhood
123
Filtering2D Linear spatial filtering (average)
Original image 15 x 15 neighborhood
124
Filtering2D Linear spatial filtering (average)
Original image 31 x 31 neighborhood
125
Filtering2D Linear spatial filtering (average)
Original image 101 x 101 neighborhoodà blurring!
126
Filtering
( ) ( )( ) ( ) ( ), , , ,a b
r s
r sg x y x y f x a yw b w a bf
= =- -
= = - -* å å
Spatial filtering ~= “convolution” (finite)
à “Filtered” image (intensity values)g
à “Original” / “input” image (intensity values)f
à “Convolution” mask / filter / window / kernel (weighting coefficients)
w
127
Spatial filtering ~= “convolution” (infinite)
Filtering
( ) ( )( ) ( ) ( ), , , ,a b
g x y x y f x a y b w a bf w= =
¥ ¥
-¥ -¥
= = - -* å å
à “Filtered” image (intensity values)g
à “Original” / “input” image (intensity values)f
à “Convolution” mask / filter / window / kernel (weighting coefficients)
w
128
5 min break!
129
Filtering
( ) ( )( ) ( ) ( ), , , ,a b
g x y f w x y f x a y b w a b¥ ¥
=-¥ =-¥
= * = - -å å
Linear spatial filtering = convolution
A convolution (f*w) is nothing more than:a weighted sum of image values f
around location (x,y) where the weightsare defined by the coefficients in w
130
Linear spatial filtering = convolution
Flip sign à correlation(correlation = convolution with 180 deg rotation of kernel)
(correlation = convolution if the kernel is symmetric)
Filtering
𝑔 𝑥, 𝑦 = 𝑓°𝑤 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏5
-./5
5
0./5
( ) ( )( ) ( ) ( ), , , ,a b
g x y f w x y f x a y b w a b¥ ¥
=-¥ =-¥
= * = - -å å
131
Filtering
( ) ( )( ) ( ) ( ), , , ,a b
g x y f w x y f x a y b w a b¥ ¥
=-¥ =-¥
= * = - -å å
Discrete Û continuous 2D convolution
( ) ( )( ) ( ) ( ) , , , , d dg x y f w x y f x a y b w a b a b¥ ¥
-¥ -¥
= * = - -ò ò
132
Filtering
( ) ( )( ) ( ) ( )a
g x f w x f x a w a¥
=-¥
= * = -å
Discrete Û continuous 1D convolution
( ) ( )( ) ( ) ( ) dg x f w x f x a w a a¥
-¥
= * = -ò
133
Filtering
f g g f* = *
Convolution: properties
( ) ( )f g h f g h* * = * *
( ) ( ) ( )f g h f g f h* + = * + *
( ) ( ) ( ) a f g a f g f a g* = * = *
( )d df dgf g g fdr dr dr
* = * = *
à Commutative
à Associative
à Distributive
à Scaling
à Linearity
134
Filtering
f g g f* = *
Convolution: properties
( )( ) ( ) ( ) ( )( ) ( ) ( ) & ¥ ¥
=-¥ =-¥
* = - * = -å åa a
f g x f x a g a g f x g x a f a
?
135
Filtering
f g g f* = *
Convolution: properties
( )( ) ( ) ( ) ( )( ) ( ) ( ) & ¥ ¥
=-¥ =-¥
* = - * = -å åa a
f g x f x a g a g f x g x a f a
( )( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )( )
x b
x a a x
a a
x a b a x b f g x f b g x b
b a f a g x a g x a f a
g x a f a g x a f a g f x
¥
- =-¥
¥ ¥
- =-¥ - =-¥-
¥ ¥
- =-¥ =-¥
- ® Þ = - Þ * = -
® Þ - = -
= - = - = *
å
å å
å å
136
FilteringConvolution: properties (cont’d)
( )( ) ( )f x f xd* = à Delta “sifting”
137
FilteringConvolution: properties (cont’d)
( )( ) ( )f x f xd* =
à Shift property
à Delta “sifting”
( )( ) ( )( )( )( )
( )
a
a
a
f g x a f g x
f g x
with f f x a
* - = *
= *
= -
138
Filtering1D convolution: Example 1
f * g
f g
139
Filtering1D convolution: Example 1
fg
f * g
140
Filtering1D convolution: Example 1
fg
f * g
141
Filtering1D convolution: Example 1
fg
f * g
142
Filtering1D convolution: Example 1
fg
f * g
143
Filtering1D convolution: Example 1
fg
f * g
144
Filtering1D convolution: Example 1
fg
f * g
145
Filtering1D convolution: Example 1
f g
f * g
146
Filtering1D convolution: Example 1
f g
f * g
147
Filtering1D convolution: Example 1
f g
f * g
148
Filtering1D convolution: Example 1
f g
f * g
149
Filtering1D convolution: Example 1
f g
f * g
150
Filtering1D convolution: Example 1
f g
f * g
151
Filtering1D convolution: Example 2
f g
f * g ?
Infinite peak at “0”
152
Filtering1D convolution: Example 2
f g = d
( )( ) ( )( ) ( )x xg x f ff dº * =*
153
Filtering1D convolution: Example 3
fg
aa af * g ?
b
154
Filtering1D convolution: Example 3
f
aa
f * g
b
( ) for x
0 for xb a
f xa
ì <ï= í ³ïî
( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷
è ø è ø
g
a
155
Filtering1D convolution: Example 3
( )( ) ( ) ( ) dg sg ff x x s s¥
-¥
* º -ò
( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷
è ø è ø( )
for x0 for xb a
f xa
ì <ï= í ³ïî
156
Filtering1D convolution: Example 3
( )( ) ( ) ( )
( )
( ) ( )
d
d2 2
d d2 2
2 2
f g x x s s
a af x s s s s
a af x s s s f x s s s
a af
g s
x x
f
f
d d
d d
¥
-¥
¥
-¥
¥ ¥
-¥ -¥
* º -
é ùæ ö æ ö= - - + +ç ÷ ç ÷ê úè ø è øë û
æ ö æ ö= - - + - +ç ÷ ç ÷è ø è ø
æ ö æ ö= - + +ç ÷ ç ÷è ø è ø
ò
ò
ò ò
( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷
è ø è ø( )
for x0 for xb a
f xa
ì <ï= í ³ïî
157
Filtering1D convolution: Example 3
( )( )2 2a af g x f x f xæ ö æ ö* = - + +ç ÷ ç ÷
è ø è ø
b
( ) for x
0 for xb a
f xa
ì <ï= í ³ïî
2af xæ ö-ç ÷
è ø
0 32a
b
2af xæ ö+ç ÷
è ø
032a-
158
Filtering1D convolution: Example 3
b
2af xæ ö-ç ÷
è ø
0 32a
b
2af xæ ö+ç ÷
è ø
032a-
b
b
32a3
2a-
12a1
2a-
159
Filtering1D convolution: Example 3
b
b
32a3
2a-
12a1
2a-
( )g x
( )
2 for x2
3 for x and x2 2
30 for x2
ab
ag x b a
a
ì <ïïï ì ü= ³ <í í ý
î þïï
³ïî
160
Filtering1D convolution: Example 4
f
aa
f * g ? à homework (optional)
g
b
11
b
161
Filtering1D convolution: code snippet
Input = “x”; filter = “h”; output = “y”
162
Filtering2D convolution: code snippet
Input = in;filter = kernel;output = out