Lec05 Features

8/8/2019 Lec05 Features

1/46

Lecture 5: Feature detection and matching

CS4670: Computer VisionNoah Snavely


2/46

Announcements

Quiz on Monday

Next project: Feature detection and matching(TBA)

Newsgroup: cornell.class.cs4670

Late policy: 1 free day over the semester,otherwise -30% per day


3/46


4/46

Feature extraction: Corners and blobs


5/46

Motivation: Automatic panoramas

Credit: Matt Brown


6/46


7/46

Why extract features?

Motivation: panorama stitching

We have two images how do we combine them?


8/46




Step 1: extract features

Step 2: match features


9/46




Step 1: extract features

Step 2: match features

Step 3: align images


10/46

Image matching

by Diva Sian

by swashford


11/46

Harder case

by Diva Sian by scgbt


12/46

Harder still?

NASA Mars Rover images


13/46

NASA Mars Rover images

with SIFT feature matches

Answer below (look for tiny colored squares)


14/46

Feature Matching


15/46


16/46

Invariant local features

Find features that are invariant to transformations geometric invariance: translation, rotation, scale

photometric invariance: brightness, exposure,

Feature Descriptors


17/46

Advantages of local features

Locality

features are local, so robust to occlusion and clutter

Quantity

hundreds or thousands in a single image

Distinctiveness:

can differentiate a large database of objects

Efficiency

real-time performance achievable


18/46

More motivation

Feature points are used for:

Image alignment (e.g., mosaics)

3D reconstruction

Motion tracking

Object recognition

Indexing and database retrieval

Robot navigation

other


19/46

Snoop demo

What makes a good feature?


20/46

Want uniqueness

Look for image regions that are unusual

Lead to unambiguous matches in other images

How to define unusual?


21/46

Local measures of uniqueness

Suppose we only consider a small window of pixels

What defines whether a feature is a good or bad

candidate?

Credit: S. Seitz, D. Frolova, D. Simakov


22/46

Local measure of feature uniqueness

flat region:

no change in all

directions

edge:

no change along the

edge direction

corner:

significant change in

all directions

How does the window change when you shift it?

Shifting the window in any direction causes a big

change

Credit: S. Seitz, D. Frolova, D. Simakov


23/46

Consider shifting the window Wby (u,v)

how do the pixels inW change?

compare each pixel before and after bysumming up the squared differences (SSD)

this defines an SSD error E(u,v):

Harris corner detection: the math

W


24/46

Taylor Series expansion ofI:

If the motion (u,v) is small, then first order approximation is good

Plugging this into the formula on the previous slide

Small motion assumption


25/46

Corner detection: the math


define an SSD error E(u,v):W


26/46



define an SSD error E(u,v):W

Thus, E(u,v) is locally approximated as a quadratic error function


27/46

The surface E(u,v) is locally approximated by a quadratic form.

The second moment matrix

Lets try to understand its shape.


28/46

Horizontal edge:

uv

E(u,v)


29/46

Vertical edge:

uv

E(u,v)


30/46

General case

We can visualize H as an ellipse with axis lengths

determined by the eigenvalues ofH and orientation

determined by the eigenvectors ofH

direction of the

slowest change

direction of the

fastest change

(Pmax)-1/2

(Pmin)-1/2

const][ !

-

v

uHvu

Ellipse equation:Pmax, Pmin : eigenvalues ofH


31/46


Eigenvalues and eigenvectors of H

Define shift directions with the smallest and largest change in error

xmax = direction of largest increase in E

Pmax = amount of increase in direction xmax xmin = direction of smallest increase in E

Pmin

= amount of increase in direction xmin

xmin

xmax


32/46

Corner detection: the mathHow are Pmax, xmax, Pmin, and xmin relevant for feature detection?

Whats our feature scoring function?


33/46

Corner detection: the mathHow are Pmax, xmax, Pmin, and xmin relevant for feature detection?

Whats our feature scoring function?

Want E(u,v) to be large for small shifts in all directions

the minimum ofE(u,v) should be large, over all unit vectors [uv]

this minimum is given by the smaller eigenvalue (P

min) ofH


34/46

Interpreting the eigenvalues

P1

P2

Corner

P1

and P2

are large,

P1 ~ P2;Eincreases in all

directions

P1 and P2 are small;Eis almost constant

in all directions

Edge

P1 >> P2

Edge

P2 >> P1

Flat

region

Classification of image points using eigenvalues ofM:


35/46

Corner detection summaryHeres what you do

Compute the gradient at each point in the image

Create the H matrix from the entries in the gradient

Compute the eigenvalues.

Find points with large response (Pmin > threshold)

Choose those points where Pmin is a local maximum as features


36/46

Corner detection summaryHeres what you do

Compute the gradient at each point in the image

Create the H matrix from the entries in the gradient

Compute the eigenvalues.

Find points with large response (Pmin > threshold)

Choose those points where Pmin is a local maximum as features


37/46

The Harris operator

Pmin is a variant of the Harris operator for feature detection

The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22

Very similar to Pmin but less expensive (no square root)

Called the Harris Corner Detector or Harris Operator

Lots of other detectors, this is one of the most popular


38/46

The Harris operator

Harrisoperator


39/46

Harris detector example


40/46

f value (red high, blue low)


41/46

Threshold (f > value)


42/46

Find local maxima of f


43/46


44/46

Weighting the derivatives

In practice, using a simple window Wdoesnt

work too well

Instead, well weighteach derivative value

based on its distance from the center pixel


45/46

Questions?


46/46

Documents

Lec05 Features