Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
Applications of Image Filters
Computer Vision
CS 543 / ECE 549
University of Illinois
Derek Hoiem
02/04/10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Credit: S. Seitz
],[],[],[,
lnkmglkfnmhlk
++=∑
[.,.]h[.,.]f
Review: Image filtering
111
111
111
],[g ⋅⋅
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
++=∑
[.,.]h[.,.]f
Image filtering
111
111
111
],[g ⋅⋅
Credit: S. Seitz
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
++=∑
[.,.]h[.,.]f
Image filtering
111
111
111
],[g ⋅⋅
Credit: S. Seitz
Filtering in spatial domain-101
-202
-101
* =
Filtering in frequency domain
FFT
FFT
Inverse FFT
Sharpening revisited• What does blurring take away?
original smoothed (5x5)
–
detail
=
sharpened
=
Let’s add it back:
original detail
+ α
Application: Hybrid Images
• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Application: Hybrid Images
• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Today’s class
• How to use filters for– Matching
– Denoising
– Anti‐aliasing
• Image representation with pyramids
• Texture– What is it?
– How to represent it?
Matching with filters
• Goal: find in image
Matching with filters
• Goal: find in image
• Method 1: SSD
Input 1- sqrt(SSD) Threshold at 0.8
Matching with filters
• Goal: find in image
• Method 1: SSD
• Method 2: Normalized cross‐correlation
Input Normalized X-Correlation Threshold at 0.5
Noise
Gaussian Additive Noise
+ =
Noisy Image
Gaussian noise
• Mathematical model: sum of many independent factors
• Assumption: independent, zero‐mean noise
Source: M. Hebert
Noise
• Salt and pepper noise: contains random occurrences of black and white pixels
• Impulse noise: contains random occurrences of white pixels
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Original
Gaussian noise
Salt and pepper noise
Impulse noise
Source: S. Seitz
Denoising
Additive Gaussian Noise
Gaussian Filter
Smoothing with larger standard deviations suppresses noise, but also blurs the image
Reducing Gaussian noise
Source: S. Lazebnik
Reducing salt‐and‐pepper noise by Gaussian smoothing
3x3 5x5 7x7
Alternative idea: Median filtering
• A median filter operates over a window by selecting the median intensity in the window
• Is median filtering linear?Source: K. Grauman
Median filter• What advantage does median filtering have over Gaussian filtering?– Robustness to outliers
Source: K. Grauman
Median filterSalt-and-pepper noise Median filtered
Source: M. Hebert
• MATLAB: medfilt2(image, [h w])
Median vs. Gaussian filtering3x3 5x5 7x7
Gaussian
Median
Throw away every other row and column to create a 1/2 size image
Subsampling by a factor of 2
• 1D example (sinewave):
Source: S. Marschner
Aliasing problem
Source: S. Marschner
• 1D example (sinewave):
Aliasing problem
• Sub‐sampling may be dangerous….
• Characteristic errors may appear: – “Wagon wheels rolling the wrong way in movies”
– “Checkerboards disintegrate in ray tracing”
– “Striped shirts look funny on color television”
Source: D. Forsyth
Aliasing problem
Aliasing in video
Slide by Steve Seitz
Source: A. Efros
Aliasing in graphics
Sampling and aliasing
• When sampling a signal at discrete intervals, the sampling frequency must be ≥ 2 × fmax;
• fmax = max frequency of the input signal.
• This will allows to reconstruct the original perfectly from the sampled version
good
bad
v v v
Nyquist‐Shannon Sampling Theorem
Anti‐aliasing
Solutions:
• Sample more often
• Get rid of all frequencies that are greater than half the new sampling frequency– Will lose information
– But it’s better than aliasing
– Apply a smoothing filter
Algorithm for downsampling by factor of 2
1. Start with image(h, w)
2. Apply low‐pass filterim_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))
3. Sample every other pixelim_small = im_blur(1:2:end, 1:2:end);
Anti‐aliasing
Forsyth and Ponce 2002
Subsampling without pre‐filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Slide by Steve Seitz
Subsampling with Gaussian pre‐filtering
G 1/4 G 1/8Gaussian 1/2
Slide by Steve Seitz
Gaussian pyramid
Source: Forsyth
Laplacian filter
Gaussianunit impulse
Laplacian of Gaussian
Source: Lazebnik
Laplacian pyramid
Source: Forsyth
Computing Gaussian/Laplacian Pyramid
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original from the laplacian pyramid?
Related idea: 2d wavelets
2d Wavelets
Matlab: wavemenu
Image representation
• Pixels: great for spatial resolution, poor access to frequency
• Fourier transform: great for frequency, not for spatial info
• Pyramids/wavelets: balance between spatial and frequency information
Major uses of image pyramids
• Compression
• Object detection– Scale search– Features
• Detecting stable interest points
• Registration– Course‐to‐fine
Texture
Source: Forsyth
Texture and Material
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
Texture and Orientation
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
Texture and Scale
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
What is texture?
• Regular or stochastic patterns caused by bumps, grooves, and/or markings
How can we represent texture?
• Measure frequencies at various orientations and scales
Overcomplete representation: filter banks
LM Filter Bank
Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
Filter banks
• Process image with each filter and keep responses (or squared/abs responses)
Representing texture
• Idea 1: take simple statistics (e.g., mean, std) of various absolute filter responses
Can you match the texture to the response?
Mean abs responses
FiltersA
B
C
1
2
3
Representing texture by mean abs response
Mean abs responses
Filters
Representing texture• Idea 2: take vectors of filter responses at each pixel and
cluster them, then take histograms (more on in coming weeks)
Things to remember
• When matching using a filter, normalized cross correlation is preferred
• Use Gaussian or median filter for denoising
• Beware of aliasing – use lowpass filter to downsample
• Laplacian pyramids and wavelets provide spatial/frequency information
• Filter banks provide overcomplete representation, good for modeling/recognizing texture
Next class
• Edges and lines
Questions