Applications of Image Filtersdhoiem.web.engr.illinois.edu/courses/vision... · Gaussian smoothing...

Applications of Image Filters

Computer Vision

CS 543 / ECE 549 

University of Illinois

Derek Hoiem

02/04/10

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Credit: S. Seitz

],[],[],[,

lnkmglkfnmhlk

++=∑

[.,.]h[.,.]f

Review: Image filtering

111

111

111

],[g ⋅⋅

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

],[],[],[,

lnkmglkfnmhlk

++=∑

[.,.]h[.,.]f

Image filtering

111

111

111

],[g ⋅⋅

Credit: S. Seitz

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

],[],[],[,

lnkmglkfnmhlk

++=∑

[.,.]h[.,.]f

Image filtering

111

111

111

],[g ⋅⋅

Credit: S. Seitz

Filtering in spatial domain-101

-202

-101

* =

Filtering in frequency domain

FFT

FFT

Inverse FFT

Sharpening revisited• What does blurring take away?

original smoothed (5x5)

–

detail

=

sharpened

=

Let’s add it back:

original detail

+ α

Application: Hybrid Images

• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Application: Hybrid Images

• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Today’s class

• How to use filters for– Matching

– Denoising

– Anti‐aliasing

• Image representation with pyramids

• Texture– What is it?

– How to represent it?

Matching with filters

• Goal: find       in image

Matching with filters

• Goal: find       in image

• Method 1: SSD

Input 1- sqrt(SSD) Threshold at 0.8

Matching with filters

• Goal: find       in image

• Method 1: SSD

• Method 2: Normalized cross‐correlation

Input Normalized X-Correlation Threshold at 0.5

Noise

Gaussian Additive Noise

+ =

Noisy Image

Gaussian noise

• Mathematical model: sum of many independent factors

• Assumption: independent, zero‐mean noise

Source: M. Hebert

Noise

• Salt and pepper noise: contains random occurrences of black and white pixels

• Impulse noise: contains random occurrences of white pixels

• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution

Original

Gaussian noise

Salt and pepper noise

Impulse noise

Source: S. Seitz

Denoising

Additive Gaussian Noise

Gaussian Filter

Smoothing with larger standard deviations suppresses noise, but also blurs the image

Reducing Gaussian noise

Source: S. Lazebnik

Reducing salt‐and‐pepper noise by Gaussian smoothing

3x3 5x5 7x7

Alternative idea: Median filtering

• A median filter operates over a window by selecting the median intensity in the window

• Is median filtering linear?Source: K. Grauman

Median filter• What advantage does median filtering have over Gaussian filtering?– Robustness to outliers

Source: K. Grauman

Median filterSalt-and-pepper noise Median filtered

Source: M. Hebert

• MATLAB: medfilt2(image, [h w])

Median vs. Gaussian filtering3x3 5x5 7x7

Gaussian

Median

Throw away every other row and column to create a 1/2 size image

Subsampling by a factor of 2

• 1D example (sinewave):

Source: S. Marschner

Aliasing problem

Source: S. Marschner

• 1D example (sinewave):

Aliasing problem

• Sub‐sampling may be dangerous….

• Characteristic errors may appear: – “Wagon wheels rolling the wrong way in movies”

– “Checkerboards disintegrate in ray tracing”

– “Striped shirts look funny on color television”

Source: D. Forsyth

Aliasing problem

Aliasing in video

Slide by Steve Seitz

Source: A. Efros

Aliasing in graphics

Sampling and aliasing

• When sampling a signal at discrete intervals, the sampling frequency must be ≥ 2 × fmax; 

• fmax = max frequency of the input signal.

• This will allows to reconstruct the original perfectly from the sampled version

good

bad

v v v

Nyquist‐Shannon Sampling Theorem

Anti‐aliasing

Solutions:

• Sample more often

• Get rid of all frequencies that are greater than half the new sampling frequency– Will lose information

– But it’s better than aliasing

– Apply a smoothing filter

Algorithm for downsampling by factor of 2

1. Start with image(h, w)

2. Apply low‐pass filterim_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))

3. Sample every other pixelim_small = im_blur(1:2:end, 1:2:end);

Anti‐aliasing

Forsyth and Ponce 2002

Subsampling without pre‐filtering

1/4 (2x zoom) 1/8 (4x zoom)1/2

Slide by Steve Seitz

Subsampling with Gaussian pre‐filtering

G 1/4 G 1/8Gaussian 1/2

Slide by Steve Seitz

Gaussian pyramid

Source: Forsyth

Laplacian filter

Gaussianunit impulse

Laplacian of Gaussian

Source: Lazebnik

Laplacian pyramid

Source: Forsyth

Computing Gaussian/Laplacian Pyramid

http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html

Can we reconstruct the original from the laplacian pyramid?

Related idea: 2d wavelets

2d Wavelets

Matlab: wavemenu

Image representation

• Pixels: great for spatial resolution, poor access to frequency

• Fourier transform: great for frequency, not for spatial info

• Pyramids/wavelets: balance between spatial and frequency information

Major uses of image pyramids

• Compression

• Object detection– Scale search– Features

• Detecting stable interest points 

• Registration– Course‐to‐fine

Texture

Source: Forsyth

Texture and Material

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

Texture and Orientation

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

Texture and Scale

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

What is texture?

• Regular or stochastic patterns caused by bumps, grooves, and/or markings

How can we represent texture?

• Measure frequencies at various orientations and scales

Overcomplete representation: filter banks

LM Filter Bank

Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html

Filter banks

• Process image with each filter and keep responses (or squared/abs responses)

Representing texture

• Idea 1: take simple statistics (e.g., mean, std) of various absolute filter responses

Can you match the texture to the response?

Mean abs responses

FiltersA

B

C

1

2

3

Representing texture by mean abs response

Mean abs responses

Filters

Representing texture• Idea 2: take vectors of filter responses at each pixel and 

cluster them, then take histograms (more on in coming weeks)

Things to remember

• When matching using a filter, normalized cross correlation is preferred

• Use Gaussian or median filter for denoising

• Beware of aliasing – use lowpass filter to downsample

• Laplacian pyramids and wavelets provide spatial/frequency information

• Filter banks provide overcomplete representation, good for modeling/recognizing texture

Next class

• Edges and lines

Questions