Shape Description

8/6/2019 Shape Description

1/53

1. Shape detection1.1 Line detection

xcos + ysin =

Data: points(xi,yi)

Formpairs of points (random selection?) to find candidate line

parameters

From each pair of points find line parameters: (i, i)

Parameter sets from real lines tend to cluster in parameter space

Find local maxima in parameter space

Valid maxima are supported by a minimum number of points

Approaches: Hough Transform; RANSAC; Mean Shift

Shape description

xcos + ysin =

x

y


2/53

1.2 Circle detection

3 points define a circle

Data: points(xi,yi)

Form subsets of 3 points (random selection?) to find candidate

circle parameters (x,y,r).

From each subset find a parameter vector (xi,yi,ri)

Parameter sets from real circles tend to cluster in parameter

space

Find local maxima in parameter space

Valid maxima are supported by a minimum number of points

Approaches: Hough Transform; RANSAC; Mean Shift

Shape description


3/53

Robust estimation

Why do we need robust methods in CV?

Goal of CV: extract photometric, geometric orsemantic information from images

Related tasks: filtering, segmentation, featureextraction, registration, motion estimation,tracking etc.

Context: noisy data, image ambiguity, multipleobjects


4/53

Robust estimation

Many robust methods in CV...

Voting approach RANSAC, MLESAC, NAPSAC

LMedS,

Hough,

MINPRAN,

M estimators,

Kernel density estimators


5/53

Introduction

Detecting the presence and state of an object:-need a description of the object (model)

-need to estimate parameters of the model from (noisy) the data-wrong data may influence badly the results-non-robust methods are very sensitive to outlier data (not actually

belonging to the model sought)

To check if a point is on a line we need theparameters of the line: a chicken and eggproblem.


6/53

Introduction

Example: detect lines in the imageLS fit

)( nmxy iii

N

i

i

1

2min


7/53


8/53

Introduction

Example: detect lines in the presence ofmultiple lines (segmentation needed)

To check if a point is on a line we need theparameters of theline: a chicken and egg problem.


9/53


10/53

Influence function Huber

2

2

1, | |2

( )1

| | , | |2

u u ku

k u k u k

, | |( )

( ), | |

u u ku

ksign u u k


11/53

ML estimation:

Define

Minimize loss Maximize (log)likelihood to see

the experimental data(ML)

( ) log ( )u p u

( ) ( )

( )

log ( ) min

log ( ) max

( ) ( | )

s s

s s s s

ss s

s

W W

W

s s

r p r

p r

p r p

y y

y x y x

y

y x

yy x


12/53

Minimize loss = Maximize likelihood

- ()K()Min Max


13/53

ML estimation:

Assumingp(u) is Gaussian

Assumingp(u) is Laplacian

Whenp(u) cannot be modelled by known distributions,

nonparametric (kernel) density estimation can be used

ML estimation: find the mode(s) ofp(u).

21( )2

u u

( ) | |u u


14/53

Robust Image Processing

Estimating the pdf from samples

Ex. Count samples in a window aroundx.


15/53

The Mean shift algorithm [Comaniciu, Meer]

Start search somewhere


16/53


Mean shift vector

Centroid of points in

the window


17/53


Translate the window with the mean shift vector


18/53


19/53


The mean shift vector always points to

centroid


20/53


Translate the window with the mean shift vector


21/53


Find the new mean shift vector


22/53



23/53



24/53



25/53



26/53

The Mean shift algorithm [Comaniciu, Meer 2002]

- gradient ascent

- convergence granted

Window centre on centroid point.

Mean shift vector = zero

Convergence achieved


27/53

Multivariate probability density estimation

( )/

( ) x R

x x

R

p dk NP

pV V d

y y

xy

1, | | 1/ 2, 1,2,...,( )

0

i for u i d

Kotherwise

u

1

-0.5 0.5 u

K(u)

1


28/53

Hypercube kernel formulation

Isotropic kernels

radial symmetry

Epanechnikov kernel profile Gauss kernel profile

1

1 1 ( )

Ni

di

p K N h h

x x

x

)||(|| 2, xx kcK dkR

1,0

10,1)(

x

xxxkE 0),

2

1exp()( xxxkN


29/53


30/53

Finding local maxima of the pdf.

Themean shiftalgorithm

xxx

xxx

xm

n

i

i

n

i

i

i

Gh

hg

h

g

1

2

1

2

, )(

)(2)()( ,,

2,

,, xmxx Ghdg

dkGhKh

chcpp

)(

)(

2

1)(,

,2

,x

xxmGh

Kh

Ghp

pch


31/53

Finding local maxima of the pdf.Themean shiftalgorithm

Set y0 = xc, then iterate until convergence:

2

1

1 2

1

, 1,2,...

Nj i

i

i

j Nj i

i

gh

j

gh

y xx

yy x


32/53

Mean shift filter


33/53

Mean shift filter

Detail preserving image smoothing


34/53

Detail preserving image smoothing

Multiscale mode filter [Gui 2008]

- Improves bias/variance compromise of the mean shift filter


35/53

Mean shift segmentation

Approach: group all data samples (pixels) converging to a local mode of the pdf

into the same region.

Can be thought of as a robust clustering method, based on nonparametric

density estimation.

No threshold needed. Only a scale parameter (acting like a soft threshold).

Number of clusters (regions) does not have to be known beforehand (in contrast

to k-means clustering).

Scale less critical than threshold, still the main problem: scale selection


36/53

Mean shift segmentation

Examples


37/53

Mean shift segmentationVideo segmentation examples

From: I. Gu, V. Gui,Joint Space-Time-Range Mean Shift Based Image and Video Segmentation,Invited Paper in Y-J.Zhang, Ed., Advances in Image and Video Segmentation IRM Press,

Hershey, London, Melbourne, Singapore, 2006, pp 113-140

Background estimation


38/53

Background estimationVideo surveillance

People

Counting

Application


39/53

Background estimation

xk the feature vector of the frame k.

The image background is modeled at pixel level

b is the most often seen color maximizes pdf

)}(max{arg xb p


40/53

Robust Image Registration


41/53

Robust Image RegistrationSIFT features

Mean shift estimation (ML) of geometric transform

parameters


42/53

Human Machine InterfaceCAMSHIFT TRACKER + TRAJECTORY FILTERING


43/53

HMI based on finger detectionand tracking


44/53

Robust finger tracking

camouflage test

(640x480 pixels, 2ms/frame)


45/53

HMI dynamic gesturesdata = (x,y,)

350

400

450

500

100

150

200

250

80

90

100

XY

Z

350

400

450

500

550

100

150

200

90

100

YX

Z

400

450

500

150

200

250

300

859095

XY

Z


46/53

Boundary descriptors

Chain code (Freeman)

0

1

2

3

0

1

2

3

4

5

6

7

a) b)

566670


47/53


Chain code properties Invariance to translation Dosired: invariance to scale and rotation (2D) Aproximate: shape number

Ratio of minor and major axes of the enclosing rectangle

Region area A, perimeter, P, complexity C=4pi2

*A/P2

Minimum areaenclosing rectangle

Major and minoraxes Re-sampling ondesired number of

cells diferential andnormalized chaincodeEx: code F =5666700022334

Fdif:1001100201011

sn = Fdif norm: 0011002010111


48/53


Histogram of contour directions Histogram of contour curvatures

Polygonal approximations (sequential encoding) New vertex selected as most distant point from currentpolygonal approximation

Interest boundary points (high curvature)

A

B

C D

A

B

C D

E

F


49/53


Signatures (polar coordinate representation)

r

r

r

2

r

2

x N x

yN

y

c nn

N

c nn

N

1

1

0

1

0

1

,

.

Moments: p kp

k

N

Nr

1

0

1

[ ] mN

r mr kp

k

N

1

10

1

[ ]


50/53


Fourier descriptors (approximators)

1

0

0

, contour point sequence

1 2exp{ }, 0,1,..., 1

centroid, t , 0 translation invariant

rotation angle : exp{ }

rescaling factor :

n n n

N

u n

n

c c u

u u

u u

z x jy

it z un u N N N

t x jy u

t i t

s t st


51/53


Fourier

descriptors(approximators)example


52/53

Boundary/shape descriptors

Moments =

(approximators)

Centroid

Central moments

Other invariant

Moments Hu, Zernike

( , )p q

pq

x y

x y f x y

10 01c c

00 00

, .x y

m x x y y f x ypqp q

yx

( ) ( ) ( , )c c


53/53

Region based descriptors

Min, Max, Mean, Median level

variance

texture descriptors

granularity

orientation

regularity

co-occurrence matrices

histograms of: gray levels,

gradient magnitude/orientations,

colour histograms

Documents

Shape Description