EE462 MLCV
1
Lecture 3-4Clustering (1hr)Gaussian Mixture and EM (1hr)
Tae-Kyun Kim
EE462 MLCV
2
2D data vectors (green) are grouped to two homogenous clusters (blue and red).Clustering is achieved by an iterative algorithm (left to right). The cluster centers are marked x.
Vector Clustering
EE462 MLCV
3
``
RGB
Pixel Clustering (Image Quantisation)Image pixels are represented by 3D vectors of R,G,B values.The vectors are grouped to K=10,3,2 clusters, and represented by the mean values of the respective clusters.
EE462 MLCV
4
dim
ensi
on
D………
……
or raw pixels
…
K codewords
Patch Clustering (BoW in Lecture 9-10) Image patches are harvested around feature points in a large number of images.They are represented by finite dimensional vectors, and clustered to form a visual dictionary.
SIFT
20
20D=400
EE462 MLCV
5……
Image ClusteringWhole images are represented as finite dimensional vectors.Homogenous vectors are grouped together in Euclidean space.
EE462 MLCV
6
K-means vs GMM
Hard clustering: a data point is assigned only one cluster.
Soft clustering: a data point is assigned multiple Gaussians probabilistically.
Two representative techniques are k-means and Gaussian Mixture Model (GMM).K-means assigns data points to the nearest clusters, while GMM assigns data to the Gaussian densities that best represent the data.
EE462 MLCV
7
Matrix and Vector Derivatives
EE462 MLCV
8
EE462 MLCV
9
K-means Clustering
EE462 MLCV
10
EE462 MLCV
11
till converge
EE462 MLCV
12
K=2
μ1
μ2
rnk
EE462 MLCV
13
Convergence proof (yes)Global minimum (no)
EE462 MLCV
14
V=V=
EE462 MLCV
15
Statistical Pattern Recognition Toolbox for Matlab
http://cmp.felk.cvut.cz/cmp/software/stprtool/
…\stprtool\probab\cmeans.m…\stprtool\probab\cmeans_tk.m
EE462 MLCV
16
Mixture of Gaussians
EE462 MLCV
17
EE462 MLCV
18
EE462 MLCV
19
EE462 MLCV
20
Maximum Likelihood
s.t.
EE462 MLCV
21
EE462 MLCV
22
objective ftn. f(x)constraints g(x)
max f(x) s.t. g(x)=0 max f(x) + g(x)
http://en.wikipedia.org/wiki/Lagrange_multiplier
EE462 MLCV
23
EE462 MLCV
24
till converge
EE462 MLCV
25
EE462 MLCV
26
EE462 MLCV
27
Statistical Pattern Recognition Toolbox for Matlab
http://cmp.felk.cvut.cz/cmp/software/stprtool/
…\stprtool\visual\pgmm.m…\stprtool\demos\demo_emgmm.m
EE462 MLCV
28
Supplementary Material
EE462 MLCV
29
Information Theory (for Lecture 7-8)
EE462 MLCV
30
EE462 MLCV
31
Advanced topic (optional)
http://www.iis.ee.ic.ac.uk/~tkkim/mlcv/lecture_clustering_em.pdf
EE462 MLCV
32
EM Algorithm in General
EE462 MLCV
33
EE462 MLCV
34