7/31/2019 Mean Shift Seg

1/15

Mean Shift Segmentation

Jan Kybic

winter semester 2007

7/31/2019 Mean Shift Seg

2/15

Mean shift segmentation overview

No assumptions about probability distributions rarelyknown

Spatial-range domain x, y, f(x, y) normally f(x, y) Find maxima in the

x, y, f

space clusters close in space

and range correspond to classes.

7/31/2019 Mean Shift Seg

3/15

Mean shift procedure

Goal: Find local maxima of the probability density (density modes)given by samples.

Initial window

New window

New

centroid

Oldcentroid

Mean shift

1. Start with a random region of interest.

2. Determine a centroid of the data.3. Move the region to the location of the new centroid.

4. Repeat until convergence.

7/31/2019 Mean Shift Seg

4/15

Kernel estimation

K(x) = ckkx2

(radial symmetry)

Epanechnikov kernel (other choices possible)

k(r) =1 r for r 1

0 otherwise(profile, r = x2)

Kernel density estimator

f(x) = 1n hd

ni=1

Kx xi

h

7/31/2019 Mean Shift Seg

5/15

Mean shift procedure

At density maxima f = 0

f(x) =

1

n hd

n

i=1K

x xi

h

f(x) = 2 ckn h(d+2)

ni=1

(x xi) k

x xih 2

=2 ck

n h(d+2) n

i=1 gi ni=1 xi gini=1 gi x

for g(r) = k(r), gi = g||(x xi)/h||

2

7/31/2019 Mean Shift Seg

6/15

Mean shift procedure

At density maxima f = 0

f(x) =

1

n hd

ni=1

K

x xi

h

0 = f(x) = 2 ckn h(d+2)

ni=1

(x xi) kx xih 2

=2 ck

n h(d+2) n

i=1gi

ni=1 xi gi

ni=1 gi x

mean shift vector must be 0 at optimum

for g(r) = k(r), gi = g||(x xi)/h||

2

7/31/2019 Mean Shift Seg

7/15

Mean shift procedure (2)

Mean shift vector

m(x) = ni=1 xi gi

ni=1 gi xgi = g

||(x xi)/h||

2 g(r) = k(r)

Successive locations yj of the kernel:

yj+1 =n

i=1

xi g

yj xi

h

2 n

i=1

g

yj xi

h

2

7/31/2019 Mean Shift Seg

8/15

Mean shift procedure (2)

Mean shift vector

m(x) =

ni=1 xi gini=1 gi

x

gi = g||(x xi)/h||

2

g(r) = k(r)Successive locations yj of the kernel:

yj+1 =n

i=1 xi gyj xi

h 2

n

i=1 gyj xi

h 2

Theorem: If k is convex and monotonically decreasing, thesequence

yjj=1,2,...

converge andf(yj)

j=1,2,...

increasesmonotonically.

For Epanechnikov kernel convergence in finite number of steps.

7/31/2019 Mean Shift Seg

9/15

Mean shift mode detection

Points from a basin of attraction converge to the same mode.

Algorithm:

1. Using multiple initializations covering the entire feature space,identify modes (stationary points).

2. Using small random perturbation, retain only local maxima.

7/31/2019 Mean Shift Seg

10/15

Mean shift mode detection example

20 30 40 50 60 70 80 90 10020

0

20

40

60

80

100

L*

u*

20 30 40 50 60 70 80 90 10020

0

20

40

60

80

100

L*

u*

7/31/2019 Mean Shift Seg

11/15

Mean shift discontinuity preserving filtering

Combine spatial and range values

K(x) = ([xs xr]) =c

hds hpr

k

xs

hs

2

k

xr

hr

2

,

Algorithm:

1. For each image pixel xi, initialize yi,1 = xi.

2. Iterate the mean shift procedure until convergence.

3. The filtered pixel values are defined as zi = (xsi, y

ri,con); the

value of the filtered pixel at the location xsi is assigned theimage value of the pixel of convergence yri,.

7/31/2019 Mean Shift Seg

12/15

Mean shift discontinuity preserving filtering

7/31/2019 Mean Shift Seg

13/15

Mean shift segmentation

1. Mean shift discontinuity preserving filtering

2. Determine the clusters {Cp}p=1,...,m by grouping all zi, which

are closer than hs in the spatial domain and hr in the rangedomain, i.e. merge the basins of attractions.

3. Assign class labels to clusters

4. If desired, eliminate regions smaller than P pixels.

7/31/2019 Mean Shift Seg

14/15

Mean shift segmentation examples

7/31/2019 Mean Shift Seg

15/15