Lecture 5Template matching

Slides by:Clark F. Olson

2

Comparing images

The simplest form of image comparison is a pixel-by-pixel comparison of some rectangular region.

In template matching, all of a smaller image (the template) is compared to all possible subregions of a larger image.

This is a form of appearance-based matching.

3

Template matching

Can find objects by comparing pixels (and aggregating)For example, sum of differences between corresponding pixels

Face recognition often uses a variation of this technique

In general, this requires an example image of the object you are looking for.Sometimes called a template or exemplar

General strategy:Consider all possible positions of template in search image

If match is close enough, save/output

4

Comparing pixels

How should the images be compared?• Sum of absolute differences• Sum of squared differences• Correlation (similar to convolution, but kernel is not flipped) • Unfortunately, these will fail if there is significant difference in:

Illumination

Sensor

Object color

Viewpoint

Etc.

5

Face finding

• Finding faces is an important special case

• The basic idea is very similar to filtering:– Filters tend to have strong

responses at locations that look similar to them.

– Use a face filter!

• Good implementations must be able to handle:– different lighting– different sizes– different orientations

6

Face finding

H. Rowley, S. Baluja, T. Kanade, “Neural network-based face detection”, IEEE Conf. on Computer Vision and Pattern Recognition. © 1996 IEEE.

77

H. Rowley, S. Baluja, T. Kanade, “Neural network-based face detection”, IEEE Conf. on Computer Vision and Pattern Recognition. © 1996 IEEE.

• If we know exactly what something looks like then it is easy to find.

• Objects look different under varying conditions:– Changes in lighting or color– Changes in viewing direction– Changes in size / shape

• A single exemplar is unlikely to succeed reliably!– However, it is impossible to represent all appearances of an object.

8

Recognition by finding patterns

9

Appearance-based matching

Appearance-based techniques use example images (templates or exemplars) of the objects to perform recognition (as opposed to extracted features).

I include edge images in this definition, although this is considered feature-based by some.

10

Example

?=

?=

Frontal faces are fairly easy to find (and sometimes classify)

However, changes to lighting and background cause problems.

11

Edge matching

Changes in lighting and color usually don’t have much effect on image edges.

12

Strategy:Detect edges in template and image

Compare edge images to find the template

Must consider range of possible template positions

Edge matching

13

Edge matching measures

What measure should we use to compare edge images?

Can count number of overlapping edges.Not robust to changes in shape

Better: count number of template edge pixels with some distance of an edge in the search image.

Best: • Determine probability distribution of distance to nearest edge in search

image (if template at correct position)• Estimate likelihood of each template position generating image

14

Matching gradientsOne way to be robust to illumination changes, but not throw away as much information is to compare image gradients.Matching is performed like matching greyscale images.

Simple alternative: use (normalized) correlation.

15

Different viewpoints

What if we don’t know the viewpoint?• Non-frontal faces• Objects in arbitrary orientation

A partial solution: linear transformations model small changes in viewpoint.

A better solution: use templates that model all possible view directions.• Computationally expensive

16

Large modelbases

If we have many potential templates that we are looking can we search efficiently?

One approach is based on eigenvectors of the templates.eigenfaces

http://www-white.media.mit.edu/vismod/demos/facerec/basic.html

• If we want to be insensitive to changes in illumination, sensor, and object color, we must use a robust measure.

• One option is to change the image to make them comparable:– Gradients– Entropy

Original image Local entropy

17

Robust measures

18

Different sensors

infrared image

CCD image