Region labelling

Region labelling

Giving a region a name

Image Processing and Computer Vision: 6 2

Introduction

Region detection isolated regions

Region description properties of regions

Region labelling identity of regions


Contents

Template matching Rigid Non-rigid templates

Graphical methods Eigenimages Statistical matching Syntactical matching


Template matching

Define a template a model of the object to be

recognised Define a measure of similarity

between template and similar sized image region


Measure dissimilarity between image f[i,j] and template g[i,j]

Place template on image and compare corresponding intensities

Need a measure of dissimilarity

Last is best....

i, j Rmax f g f g

i, j R 2

f g i, j R

Similarity


Expanding

If f and g fixed-fg a good measure of mismatch

fg a good measure of match

Compute match between template and image with cross-correlation

2

f g i, j R 2

f i, j R 2

g 2i , j R fg

i , j R

M i, j g k, l l n

ln

k m

km

f i k, j l


g is constant, f varies and so influences MNormalisation

C is maximum where f and g are same.Limitations

number of templates required rotation and size changes partial views

C i, j g k, l

l n

ln

k m

k m

f i k, j l

2fl n

ln

k m

km

i k, j l


0

20

40

60

80

100

1201

26

51

76

10

1

12

6

15

1

17

6

20

1

22

6

25

1

27

6

30

1

32

6

35

1

37

6

40

1

42

6

Position

No

n-N

orm

alis

ed

Co

rre

lati

on

Template

Input Output


Flexible Templates

Shapes are seldom constant Variation

in shape itself in image of same shape viewpoint

Non-rigid representations capture variability


Structure

Flexible image structures Linked by virtual springs


Recognition Deform image structure

To equate model and image Move image structures

To colocate model and image Matching

externalexternalinternalinternal EWEWEtotal


Learning the model Accuracy of model determines

success Model

For each control point average, variance of location

To be learnt with minimum external variation

size, orientation, inconsistency of location


Parametric Models

Parametrically define the shape straight line, circle, parabola, …

Update parameters to match model and object


Example – Face tracking

Eyes and mouth circles and parabolas locations, sizes, orientations

Templates define image structures


Flexible templates, EigenImages

Attempt to capture intrinsic variability of data

Mathematical representation of variation


Take samples from a population plot values of parameters on a

scatter diagram

Mathematical Foundation


Rotate axes: one axis encodes most of information other axis encodes remainder

Generalise to multiple dimensions


Images

Use outline co-ordinates image values

As the variables Normalise as much variability


Hand Eigenimages


Hand Gestures


Range of Eigenimages


Face Eigenimages


Recognition

Retain n eigenvectors with largest eigenvalues

Form dot product of these with image data

Find nearest neighbour from training set


Statistical Classification Methods

Derive characteristic feature measurements from image

Form a feature vector that identifies object as belonging to a predefined class

Need decision rules to make classification


Linear Discriminant Analysis

Samples from different classes occupy different regions of feature space

Can define a line separating them


Feature 1

Featu

re 2

Class A

Class B


Decision

d(X) = F2 - mF1 - c

d(X) > 0 for points in class Ad(X) = 0 for points on lined(X) < 0 for points in class B


height

weight

jockeys

basketball players?

jd 2

iu ijf i1

N

Rd minj1

N

jd

Nearest Neighbour Classifier

Assign the new sample to the population whose centroid is closest.


Most Likely

Incorporate range of possible class values

2

2

xxCp

A

A


Take population variation into account

Assume prior probability of observing class j is P(j)

e.g. 10% of population are jockeys

Assume a conditional probability distribution for each feature, x, of each population p(x|j).

height

weight

jockeys

basketball players?

Bayesian Classifiers


P j | x p x|

j P j p x|

j P j j1

N

Multiply these curves by P(j) to give probability of a measurement belonging to each class.

Divide by total probability of measuring x, to normalise.

This gives the probability of the sample being from each class.

x

p

p(x|1)

p(x|2)


Syntactic Recognition

Objects’ structure (outline) can be described linguistically Primitive shape elements = words Grammatically correct sentences = a

valid shape


Shape Grammar A set of pattern primitives

terminal symbols A set of rules that define combinations

of primitives (sentences) the grammar

A start symbol represents a valid object

Non-terminal symbols represent substructures in the shape


Recognition

Grammar is generative Recognition is degenerative

Recognition uses rules in reverse Terminal symbols are rewritten until a

valid start symbol is attained


Chromosome Grammar

armpartright

armpartleft

armpartleft pair arm

partright armpair arm

sidepair armpair arm

pair armsidepair arm

pair armpair armchromosomesubmedian

c

c


Chromosome Grammar

d

b

b

b

a

b

b

side

side

sideside

sideside

arm

armarm

armarm


The Primitives

a b c d

a bc

b

ab

bb

b

b

b

a

ad

d

c


Example

a bc

b

ab

bb

b

b

b

a

ad

d

c


<submedian chromosome>

d b a b c b a b d b a b c b a b

<side><side> <arm><arm> <arm>c <arm>c

<side><side> <arm><arm> <arm>c <arm>cb b

<side><side> <arm> <right part><arm> <right part>

<side> <arm pair><side> <arm pair>

<arm pair><arm pair>


Evaluation Classification rate Confusion matrix


Classification Rate How often does the

classifier get the correct answer?

Selection of training and test data must be carefully done.


Confusion matrix C(i,j) = number of times

pattern i was recognised as class j.

Want off-diagonal elements to be zero.


Summary

Template matching Deformable templates Flexible templates Statistical classification

Documents

Region labelling