Intelligent Database Systems Lab
國立雲林科技大學National Yunlin University of Science and Technology
Advisor : Dr. Hsu Graduate : Sheng-Hsuan Wang Author : Sitao Wu
Tommy W.S. ChowDepartment of Information Management
Clustering of the self-organizing map using a clustering validity index based on inter-cluster and intra-cluster density
Pattern Recognition Volume: 37, Issue: 2, February, 2004, pp. 175-188.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Outline Motivation Objective Introduction SOM and Clustering Clustering of the SOM using local clustering validity
index and preprocessing of the SOM for filtering Experimental results Conclusions Personal opinion Review
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Motivation
Classical clustering methods based on the SOM.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Objective
Preprocessing techniques Filtering out noises and outliers.
A new two-level SOM-based clustering algorithm. Clustering validity index based on inter-cluster and intra-
cluster density.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Introduction
Self-Organizing Map, SOM. Clustering algorithms. two-level SOM-based clustering. In this paper, a new two-level algorithm for clusterin
g of the SOM is proposed. SOM. Agglomerative hierarchical clustering.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
SOM and Clustering
SOM and visualization. Clustering algorithms. Clustering of the SOM.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
SOM and visualization
Initial Step. Training Step.
Find the winner from (1). Update the winner and neighborhood according to (2).
(3) ))(2
||||exp()(
(2) )]()()[()()()1(
(1) ||})()({||minarg)(
2
2
t
rrth
tmtxthttmtm
tmtxtc
icci
iciii
ii
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
SOM and visualization
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Clustering algorithms
The categories of clustering methods Hierarchical Partitioning Density-based Grid-based Model-based
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Clustering of the SOM
Agglomerative hierarchical clustering of the SOM. Merging criterion : Inter-cluster distance, Inter-cluster and
intra-cluster density. Filtering noises and outliers before clustering of the SOM.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Clustering of the SOM using local clustering validity index and preprocessing of the SOM for filtering
Global clustering validity index for different clustering algorithms.
Merging criterion using the CDbw. Preprocessing before clustering of the SOM. Clustering of the SOM. The algorithm of clustering of the SOM.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Global Clustering validity index for different clustering algorithms
Three types of methods used to cluster validity: External criteria. Internal criteria. Relative criteria.
compact and well-separated clusters The newly proposed multi-representation clustering
validity index.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
CDbw
The notations in the clustering validity index A set of representation points represents th
e i th cluster. stdev(i) is a standard deviation vector of the i th cluster. The p th component of stdev(i) is defined by
The average standard deviation is given by
)1/()()(1
2
i
n
k
pi
pk
p nmxistdevi
c
i
cistdevstdev1
2 /||)(||
},...,,{ 21 iiriii vvvV
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
(5) otherwise.
stdev,||||
0
1),(
),()(
(4) 1.c ,)(1
)(_
1
1 1
ijlijl
n
lijlij
c
i
r
jij
vxvxf
vxfvdensity
vdensityc
cdenIntra
i
i
(7) otherwise.
)/2,||)(||||)(||(||||
0
1),(
),()(
(6) 1,c ),(||)(||||)(||
|)(_)(_||)(_
1
1 1
jstdevistdevuxuxf
uxfudensity
udensityjstdevistdev
jrepcloseirepclosecdenInter
ijkijk
nn
kijkij
ij
c
i
c
ij
ji
CDbw – Intra_den & Inter_den
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
The definition of the clusters’ separation
The overall clustering validity index, which is called “Composing Density Between and With clusters”.
c
i
c
ij cdenInter
jrepcloseirepclosecSep
1 1
(8) 1.c ,)(_1
||)(_)(_||)(
(9) )()(_)( cSepcdenIntracCDbw
CDbw
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Merging criterion using the CDbw
To find the pair of clusters with minimal value of the CDbw.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Preprocessing before clustering of the SOM
1. Labeling.
2. Compute the distance deviation : devj=||wj - mj||, mean_dev, and std_dev.
3. If devj > mean_dev + std_dev, exclude the neuron j.
4. Compute distances : disj(xi)=||xi - wj||, mean_disj, and std_disj.
5. If disj(xi) > mean_disj + std_devj, filter out the input vector xj.
6. Compute the number of data belonging to the jth cluster : numj, mean_num, and std_num.
7. If numj < mean_num - std_num, exclude the neuron j.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Clustering of the SOM
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
The algorithm of Clustering of the SOM
1. Train input data by the SOM.
2. Preprocessing before clustering of the SOM.
3. Cluster SOM by using the agglomerative hierarchical clustering. The merging criterion is the CDbw.
4. Find the optimal partition of the input data according to the CDbw.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Experimental results 200 2D synthetic data set.
With some noises and outliers. Use k-means, four HCA, and the proposed algorithm.
150 Iris data set. Three classes with 50 points each. Use single-linkage and proposed clustering algorithm.
1780 15D synthetic data set. Generating 20 uniformly distributed random 15D points.
178 Wine data set. Three classes are 59, 71, and 48, respectively.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
2D synthetic data set
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
2D synthetic data set
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Iris data set
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
15D synthetic data set
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Wine data set
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Conclusions
In this paper, we propose a new SOM-based clustering algorithm.
The clustering validity index locally to determine which pair of clusters to be merged.
The preprocessing steps for filtering out noises and outliers.
The experimental results better than other clustering algorithms on the SOM.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Personal opinion
This method more precise than others. We can consider the entropy or other index besides distan
ce and density.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Review
Self-Organizing Map, SOM. Clustering methods. Two-level Clustering. Clustering Validity index – CDbw. The preprocessing steps.