Applications of Image Filters

Applications of Image Filters

Computer VisionCS 543 / ECE 549

University of Illinois

Derek Hoiem

02/04/10

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Credit: S. Seitz

],[],[],[,

lnkmglkfnmhlk

[.,.]h[.,.]f

Review: Image filtering111

111

111

],[g

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

],[],[],[,

lnkmglkfnmhlk

[.,.]h[.,.]f

Image filtering111

111

111

],[g

Credit: S. Seitz

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

],[],[],[,

lnkmglkfnmhlk

[.,.]h[.,.]f

Image filtering111

111

111

],[g

Credit: S. Seitz

Filtering in spatial domain-101

-202

-101

* =

Filtering in frequency domain

FFT

FFT

Inverse FFT

=

Sharpening revisited• What does blurring take away?

original smoothed (5x5)

–

detail

=

sharpened

=

Let’s add it back:

original detail

+ α

Application: Hybrid Images

• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Application: Hybrid Images

• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006

Today’s class

• How to use filters for– Matching– Denoising– Anti-aliasing

• Image representation with pyramids

• Texture– What is it?– How to represent it?

Matching with filters• Goal: find in image

Matching with filters• Goal: find in image• Method 1: SSD

Input 1- sqrt(SSD) Threshold at 0.8

Matching with filters• Goal: find in image• Method 1: SSD• Method 2: Normalized cross-correlation

Input Normalized X-Correlation Threshold at 0.5

Noise

Gaussian Additive Noise

+ =

Noisy Image

Gaussian noise• Mathematical model: sum of many independent

factors• Assumption: independent, zero-mean noise

Source: M. Hebert

Noise• Salt and pepper

noise: contains random occurrences of black and white pixels

• Impulse noise: contains random occurrences of white pixels

• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution

Source: S. Seitz

Denoising

Additive Gaussian Noise

Gaussian Filter

Smoothing with larger standard deviations suppresses noise, but also blurs the image

Reducing Gaussian noise

Source: S. Lazebnik

Reducing salt-and-pepper noise by Gaussian smoothing

3x3 5x5 7x7

Alternative idea: Median filtering• A median filter operates over a window by

selecting the median intensity in the window

• Is median filtering linear?Source: K. Grauman

Median filter• What advantage does median filtering have

over Gaussian filtering?– Robustness to outliers

Source: K. Grauman

Median filterSalt-and-pepper noise Median filtered

Source: M. Hebert

• MATLAB: medfilt2(image, [h w])

Median vs. Gaussian filtering3x3 5x5 7x7

Gaussian

Median

Throw away every other row and column to create a 1/2 size image

Subsampling by a factor of 2

• 1D example (sinewave):

Source: S. Marschner

Aliasing problem

Source: S. Marschner

• 1D example (sinewave):

Aliasing problem

• Sub-sampling may be dangerous….• Characteristic errors may appear:

– “Wagon wheels rolling the wrong way in movies”

– “Checkerboards disintegrate in ray tracing”– “Striped shirts look funny on color television”

Source: D. Forsyth

Aliasing problem

Aliasing in video

Slide by Steve Seitz

Source: A. Efros

Aliasing in graphics

Sampling and aliasing

• When sampling a signal at discrete intervals, the sampling frequency must be 2 fmax;

• fmax = max frequency of the input signal.

• This will allows to reconstruct the original perfectly from the sampled version

good

bad

v v v

Nyquist-Shannon Sampling Theorem

Anti-aliasing

Solutions:• Sample more often

• Get rid of all frequencies that are greater than half the new sampling frequency– Will lose information– But it’s better than aliasing– Apply a smoothing filter

Algorithm for downsampling by factor of 2

1. Start with image(h, w)2. Apply low-pass filter

im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))

3. Sample every other pixelim_small = im_blur(1:2:end, 1:2:end);

Anti-aliasing

Forsyth and Ponce 2002

Subsampling without pre-filtering

1/4 (2x zoom) 1/8 (4x zoom)1/2

Slide by Steve Seitz

Subsampling with Gaussian pre-filtering

G 1/4 G 1/8Gaussian 1/2

Slide by Steve Seitz

Gaussian pyramid

Source: Forsyth

Laplacian filter

Gaussianunit impulse

Laplacian of Gaussian

Source: Lazebnik

Laplacian pyramid

Source: Forsyth

Computing Gaussian/Laplacian Pyramid

http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html

Can we reconstruct the original from the laplacian pyramid?

Related idea: 2d wavelets

2d Wavelets

Matlab: wavemenu

Image representation

• Pixels: great for spatial resolution, poor access to frequency

• Fourier transform: great for frequency, not for spatial info

• Pyramids/wavelets: balance between spatial and frequency information

Major uses of image pyramids

• Compression

• Object detection– Scale search– Features

• Detecting stable interest points

• Registration– Course-to-fine

Texture

Source: Forsyth

Texture and Material

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

Texture and Orientation

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

Texture and Scale

http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/

What is texture?

• Regular or stochastic patterns caused by bumps, grooves, and/or markings

How can we represent texture?

• Measure frequencies at various orientations and scales

Overcomplete representation: filter banks

LM Filter Bank

Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html

Filter banks• Process image with each filter and keep

responses (or squared/abs responses)

Representing texture• Idea 1: take simple statistics (e.g., mean, std)

of various absolute filter responses

Can you match the texture to the response?

Mean abs responses

FiltersA

B

C

1

2

3

Representing texture by mean abs response

Mean abs responses

Filters

Representing texture• Idea 2: take vectors of filter responses at each pixel and

cluster them, then take histograms (more on in coming weeks)

Things to remember

• When matching using a filter, normalized cross correlation is preferred

• Use Gaussian or median filter for denoising

• Beware of aliasing – use lowpass filter to downsample

• Laplacian pyramids and wavelets provide spatial/frequency information

• Filter banks provide overcomplete representation, good for modeling/recognizing texture

Next class

• Edges and lines

Questions