Slide credit Fei Fei Li. Image filtering Image filtering: compute function of local neighborhood at...
Preview:
Citation preview
- Slide 1
- Slide credit Fei Fei Li
- Slide 2
- Slide 3
- Slide 4
- Image filtering Image filtering: compute function of local
neighborhood at each position Really important! Enhance images
Denoise, resize, increase contrast, etc. Extract information from
images Texture, edges, distinctive points, etc. Detect patterns
Template matching
- Slide 5
- 111 111 111 Slide credit: David Lowe (UBC) Example: box
filter
- Slide 6
- 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00
0000000000 00 0000000 0000000000 0 0000000000 0000000000 000 00 000
00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Credit:
S. Seitz Image filtering 111 111 111
- Slide 7
- 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00
0000000000 00 0000000 0000000000 010 0000000000 0000000000 00090 00
000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000
Image filtering 111 111 111 Credit: S. Seitz
- Slide 8
- 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00
0000000000 00 0000000 0000000000 01020 0000000000 0000000000 00090
00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000
Image filtering 111 111 111 Credit: S. Seitz
- Slide 9
- 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00
0000000000 00 0000000 0000000000 0102030 0000000000 0000000000
00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000
0000000000 Image filtering 111 111 111 Credit: S. Seitz
- Slide 10
- 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00
000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111
Credit: S. Seitz
- Slide 11
- 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00
000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111
Credit: S. Seitz ?
- Slide 12
- 0102030 50 0000000000 0000000000 00090 00 000 00 000 00 000 0
00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111
111 Credit: S. Seitz ?
- Slide 13
- 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00
0000000000 00 0000000 0000000000 0102030 2010 0204060 4020 0306090
6030 0 5080 906030 0 5080 906030 0203050 604020 102030 2010 00000
Image filtering 111 111 111 Credit: S. Seitz
- Slide 14
- What does it do? Replaces each pixel with an average of its
neighborhood Achieve smoothing effect (remove sharp features) 111
111 111 Slide credit: David Lowe (UBC) Box Filter
- Slide 15
- Smoothing with box filter
- Slide 16
- Practice with linear filters 000 010 000 Original ? Source: D.
Lowe
- Slide 17
- Practice with linear filters 000 010 000 Original Filtered (no
change) Source: D. Lowe
- Slide 18
- Practice with linear filters 000 100 000 Original ? Source: D.
Lowe
- Slide 19
- Practice with linear filters 000 100 000 Original Shifted left
By 1 pixel Source: D. Lowe
- Slide 20
- Practice with linear filters Original 111 111 111 000 020 000 -
? (Note that filter sums to 1) Source: D. Lowe
- Slide 21
- Practice with linear filters Original 111 111 111 000 020 000 -
Sharpening filter - Accentuates differences with local average
Source: D. Lowe
- Slide 22
- Sharpening Source: D. Lowe
- Slide 23
- Other filters 01 -202 01 Vertical Edge (absolute value)
Sobel
- Slide 24
- Other filters -2 000 121 Horizontal Edge (absolute value)
Sobel
- Slide 25
- Filtering vs. Convolution 2d filtering 2d convolution
f=imageg=filter
- Slide 26
- Key properties of linear filters Linearity: filter(f 1 + f 2 )
= filter(f 1 ) + filter(f 2 ) Shift invariance: same behavior
regardless of pixel location filter(shift(f)) = shift(filter(f))
Any linear, shift-invariant operator can be represented as a
convolution Source: S. Lazebnik
- Slide 27
- More properties Commutative: a * b = b * a Conceptually no
difference between filter and signal But particular filtering
implementations might break this equality Associative: a * (b * c)
= (a * b) * c Often apply several filters one after another: (((a *
b 1 ) * b 2 ) * b 3 ) This is equivalent to applying one filter: a
* (b 1 * b 2 * b 3 ) Distributes over addition: a * (b + c) = (a *
b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b)
Identity: unit impulse e = [0, 0, 1, 0, 0], a * e = a Source: S.
Lazebnik
- Slide 28
- Weight contributions of neighboring pixels by nearness 0.003
0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097
0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022
0.013 0.003 5 x 5, = 1 Slide credit: Christopher Rasmussen
Important filter: Gaussian
- Slide 29
- Smoothing with Gaussian filter
- Slide 30
- Smoothing with box filter
- Slide 31
- Gaussian filters Remove high-frequency components from the
image (low-pass filter) Images become more smooth Convolution with
self is another Gaussian So can smooth with small-width kernel,
repeat, and get same result as larger-width kernel would have
Convolving two times with Gaussian kernel of width is same as
convolving once with kernel of width 2 Separable kernel Factors
into product of two 1D Gaussians Source: K. Grauman
- Slide 32
- Separability of the Gaussian filter Source: D. Lowe
- Slide 33
- Separability example * * = = 2D convolution (center location
only) Source: K. Grauman The filter factors into a product of 1D
filters: Perform convolution along rows: Followed by convolution
along the remaining column:
- Slide 34
- Separability Why is separability useful in practice?
- Slide 35
- Practical matters What about near the edge? the filter window
falls off the edge of the image need to extrapolate methods: clip
filter (black) wrap around copy edge reflect across edge Source: S.
Marschner
- Slide 36
- Practical matters What is the size of the output? MATLAB:
filter2(g, f, shape) shape = full: output size is sum of sizes of f
and g shape = same: output size is same as f shape = valid: output
size is difference of sizes of f and g f gg gg f gg g g f gg gg
full samevalid Source: S. Lazebnik
- Slide 37
- 2006 Steve Marschner 37 Median filters A Median Filter operates
over a window by selecting the median intensity in the window. What
advantage does a median filter have over a mean filter? Is a median
filter a kind of convolution? Slide by Steve Seitz
- Slide 38
- 2006 Steve Marschner 38 Comparison: salt and pepper noise Slide
by Steve Seitz
- Slide 39
- Why does the Gaussian give a nice smooth image, but the square
filter give edgy artifacts? GaussianBox filter
- Slide 40
- Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, Hybrid
Images, SIGGRAPH 2006 Hybrid Images,
- Slide 41
- Why do we get different, distance-dependent interpretations of
hybrid images? ? Slide: Hoiem
- Slide 42
- Why does a lower resolution image still make sense to us? What
do we lose? Image:
http://www.flickr.com/photos/igorms/136916757/http://www.flickr.com/photos/igorms/136916757/Slide:
Hoiem
- Slide 43
- Thinking in terms of frequency
- Slide 44
- Jean Baptiste Joseph Fourier (1768-1830) had crazy idea (1807):
Any univariate function can be rewritten as a weighted sum of sines
and cosines of different frequencies. Dont believe it? Neither did
Lagrange, Laplace, Poisson and other big wigs Not translated into
English until 1878! But its (mostly) true! called Fourier Series
there are some subtle restrictions...the manner in which the author
arrives at these equations is not exempt of difficulties and...his
analysis to integrate them still leaves something to be desired on
the score of generality and even rigour. Laplace Lagrange
Legendre
- Slide 45
- A sum of sines Our building block: Add enough of them to get
any signal g(x) you want!
- Slide 46
- Frequency Spectra example : g(t) = sin(2f t) + (1/3)sin(2(3f)
t) = + Slides: Efros
- Slide 47
- Frequency Spectra
- Slide 48
- = + =
- Slide 49
- = + =
- Slide 50
- = + =
- Slide 51
- = + =
- Slide 52
- = + =
- Slide 53
- =
- Slide 54
- Example: Music We think of music in terms of frequencies at
different magnitudes Slide: Hoiem
- Slide 55
- Other signals We can also think of all kinds of other signals
the same way xkcd.com
- Slide 56
- Fourier analysis in images Intensity Image Fourier Image
http://sharp.bu.edu/~slehar/fourier/fourier.html#filtering
- Slide 57
- Signals can be composed +=
http://sharp.bu.edu/~slehar/fourier/fourier.html#filtering More:
http://www.cs.unm.edu/~brayer/vision/fourier.html
- Slide 58
- Fourier Transform Fourier transform stores the magnitude and
phase at each frequency Magnitude encodes how much signal there is
at a particular frequency Phase encodes spatial information
(indirectly) For mathematical convenience, this is often notated in
terms of real and complex numbers Amplitude:Phase:
- Slide 59
- Computing the Fourier Transform Continuous Discrete k =
-N/2..N/2 Fast Fourier Transform (FFT): NlogN
- Slide 60
- The Convolution Theorem The Fourier transform of the
convolution of two functions is the product of their Fourier
transforms Convolution in spatial domain is equivalent to
multiplication in frequency domain!
- Slide 61
- Properties of Fourier Transforms Linearity Fourier transform of
a real signal is symmetric about the origin The energy of the
signal is the same as the energy of its Fourier transform See
Szeliski Book (3.4)
- Slide 62
- Filtering in spatial domain 01 -202 01 * =
- Slide 63
- Filtering in frequency domain FFT Inverse FFT = Slide:
Hoiem
- Slide 64
- Why does the Gaussian give a nice smooth image, but the square
filter give edgy artifacts? GaussianBox filter Filtering
- Slide 65
- Gaussian
- Slide 66
- Box Filter
- Slide 67
- Why does a lower resolution image still make sense to us? What
do we lose? Image:
http://www.flickr.com/photos/igorms/136916757/http://www.flickr.com/photos/igorms/136916757/
Sampling
- Slide 68
- Throw away every other row and column to create a 1/2 size
image Subsampling by a factor of 2
- Slide 69
- 1D example (sinewave): Source: S. Marschner Aliasing
problem
- Slide 70
- Source: S. Marschner 1D example (sinewave): Aliasing
problem
- Slide 71
- Sub-sampling may be dangerous. Characteristic errors may
appear: Wagon wheels rolling the wrong way in movies Checkerboards
disintegrate in ray tracing Striped shirts look funny on color
television Source: D. Forsyth Aliasing problem
- Slide 72
- Aliasing in video Slide by Steve Seitz
- Slide 73
- Source: A. Efros Aliasing in graphics
- Slide 74
- Sampling and aliasing
- Slide 75
- When sampling a signal at discrete intervals, the sampling
frequency must be 2 f max f max = max frequency of the input signal
This will allows to reconstruct the original perfectly from the
sampled version good bad vvv Nyquist-Shannon Sampling Theorem
- Slide 76
- Anti-aliasing Solutions: Sample more often Get rid of all
frequencies that are greater than half the new sampling frequency
Will lose information But its better than aliasing Apply a
smoothing filter
- Slide 77
- Algorithm for downsampling by factor of 2 1.Start with image(h,
w) 2.Apply low-pass filter im_blur = imfilter(image,
fspecial(gaussian, 7, 1)) 3.Sample every other pixel im_small =
im_blur(1:2:end, 1:2:end);
- Slide 78
- Anti-aliasing Forsyth and Ponce 2002
- Slide 79
- Subsampling without pre-filtering 1/4 (2x zoom) 1/8 (4x zoom)
1/2 Slide by Steve Seitz
- Slide 80
- Subsampling with Gaussian pre-filtering G 1/4G 1/8Gaussian 1/2
Slide by Steve Seitz
- Slide 81
- Why does a lower resolution image still make sense to us? What
do we lose? Image:
http://www.flickr.com/photos/igorms/136916757/http://www.flickr.com/photos/igorms/136916757/
- Slide 82
- Why do we get different, distance-dependent interpretations of
hybrid images? ?
- Slide 83
- Salvador Dali invented Hybrid Images? Salvador Dali Gala
Contemplating the Mediterranean Sea, which at 30 meters becomes the
portrait of Abraham Lincoln, 1976
- Slide 84
- Slide 85
- Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns,
Hybrid Images, SIGGRAPH 2006 Hybrid Images,
- Slide 86
- Early processing in humans filters for various orientations and
scales of frequency Perceptual cues in the mid-high frequencies
dominate perception When we see an image from far away, we are
effectively subsampling it Early Visual Processing: Multi-scale
edge and blob filters Clues from Human Perception
- Slide 87
- Hybrid Image in FFT Hybrid ImageLow-passed ImageHigh-passed
Image
- Slide 88
- Why do we get different, distance-dependent interpretations of
hybrid images? ? Perception
- Slide 89
- Things to Remember Sometimes it makes sense to think of images
and filtering in the frequency domain Fourier analysis Can be
faster to filter using FFT for large images (N logN vs. N 2 for
auto- correlation) Images are mostly smooth Basis for compression
Remember to low-pass before sampling
- Slide 90
- Sharpening revisited What does blurring take away? original
smoothed (5x5) detail = sharpened = Lets add it back:
originaldetail +