Upload
allen-austin
View
213
Download
0
Embed Size (px)
Citation preview
Adaptive fuzzy Adaptive fuzzy clusteringclustering
Kharkiv National University of Radio Electronics
Control Systems Research Laboratory
Crisp Approach to Clustering
AB
C
1x
2x
Fuzzy Approach to Clustering
1x
2x
AB
C
),( BAw
),( CBw),( CAw
),,( CBAw
Fuzzy Approach to Clustering
Input Data SetNkXxxxxX kN ,...2,1,},,,...,,{ 21
Goal Function of Clustering
N
k
m
jjkjkjjk cxdwcwE
1 1
2,, ),(),(
Batch Approaches
m
jjk Nkw
1, ,...,1,1
N
kjk mjNw
1, ,...,1,0
mNWwW jk dim},{ ,
Batch Fuzzy Clustering Algorithms: Probabilistic ApproachFuzzy clustering as a constrained optimization problem
nN RxxxX }...,{ ,2,1 m Divide a data set into clusers.
N
k
m
jjkkjjk cxdwcwE
1 1
2, ),,(,),(
,,...,1,11
, Nkwm
jjk
.,...,1,01
, mjNwN
kjk
(1)
(2)
(3)
Lagrange function:
N
k
m
j
N
k
m
jjkkjkjkkjjk wcxdwcwL
1 1 1 1,
2,, .1),(),,( (4)
Batch Fuzzy Clustering Algorithms: Probabilistic ApproachDegrees of membership:
,
)),((
))((
1
1
1
2
1
1
2
,
m
llk
jkjk
cxd
cxdw (5)
prototypes:
N
kjk
N
kkjk
j
w
xw
c
1,
1,
(6)
A Bkx 2,kw1,kw
px
2,pw1,pw
1x
2x
5.02,1,2,1, ppkk wwww
Batch Fuzzy Clustering Algorithms: Possibilistic Approach
2, , ,
1 1 1 1
( , ) ( , ) (1 ) .N m m N
k j j k j k j j k jk j j k
E w c w d x c w
0j
.
),(
1,
1
2,
N
kjk
N
kjkjk
j
w
cxdw
Unconstrained optimizationModified objective function:
Scalar parameters determine the distance at which the degrees of membership are equal to 0.5:
(8)
(7)
Batch Fuzzy Clustering Algorithms: Possibilistic Approach
,),(
1
1
1
12
,
j
jkjk
cxdw
.
1,
1,
N
kjk
N
kkjk
j
w
xw
c
Possibilistic algorithmDegrees of membership:
prototypes:
(10)
(9)
Adaptive Fuzzy Probabilistic Algorithms
m
j
m
jjkkjkjkkjjkk wcxdwcwL
1 1,
2,, ).1(),(),,(
,
)),((
)),((
1
1
1
,2
1
1
,2
,
m
llkk
jkkjk
cxd
cxdw
).( ,1,,,1 jkkjkkjkjk cxwcc
One-step Largange function:
Degrees of membership:
prototypes:
(13)
(11)
(12)
Unsurprised fuzzy competitive learning (UFCL) algorithm
Adaptive Fuzzy Probabilistic Algorithms
, 1 ,
1,,
( ), ,
, ,
k j k k k j
k jk j
c x c j lc
c j l
l
1
2
1,
2,
,
m
llkk
jkkjk
cx
cxw
).( ,12,,,1 jkkjkkjkjk cxwcc
UFCL is an extension of the Kohonen competitive learning rule:
where
When , UFCL transforms into the Gradient-based fuzzy c-means (GBFCM) algorithm
prototypes:
(16)
(14)
(15)
is a number of the “winning” prototype.
Degrees of membership:
Adaptive Fuzzy Possibilistic Algorithms
m
j
m
jjkjjkjkjjkk wcxdwcwE
1 1,
2,, .)1(),(),(
,),(
1
1
1
12
,
j
jkjk
cxdw
).( ,1,,,1 jkkjkkjkjk cxwcc
One-step objective function:
Adaptive possibilistic algorithmDegrees of membership:
prototypes:
(19)
(18)
(17)
Adaptive Fuzzy Possibilistic Algorithms
,2
,
,
jkkj
jjk
cxw
).( ,12,,,1 jkkjkkjkjk cxwcc
When β=2Degrees of membership:
prototypes:
(20)
(21)
Adaptive Fuzzy Possibilistic Algorithms
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
wkj
mu=0.1mu=0.2mu=0.3
Possibilistic membership function for 3.01.0,2
Matrix Fuzzy C-means Algorithm
1 2
2,{ ( )} , 1,2,... ,n ni iX x k R k N
1
1( ), ( ) ( ( ) ( )),
NT
k
x x k x k Tr x k x kN
1 1
( , ) ( ) ( ( ) )( ( ) )N m
Tj j j j j
k j
E u c u k Tr x k c x k c
1
1,m
jj
u
1
0 ( ) , 1,..., .N
jk
u k N j m
Input Data Set(22)
.Goal Function of Clustering
.Batch Approaches(25)
. (26)
(23)
(24)
Matrix Fuzzy C-means Algorithm
Karush-Kuhn-Tucker equation system:
1 2
1
1
( ( ), , ( ))( ) ( ( ), ) ( ) 0,
( )
( ( ), , ( ))( ) 1 0,
( )
( ( ), , ( ))2 ( )( ( ) ).
( )
j jj j
j
mj j
jjj
Nj j
j jkj
L u k c ku k D x k c k
u k
L u k c ku k
k
L u k c ku k x k c
c k
O
(27)
Matrix Fuzzy C-means Algorithm
Fuzzy clustering algorithm:
12 1
12 1
1
112 1
1
1
1
( ( ( ), ))( ) ,
( ( ( ), ))
( ) ( ( ), ) ,
( ) ( ).
( )
jj m
ll
m
ll
N
jk
j N
jk
D x k cu k
D x k c
k D x k c
u k x kc
u k
(28)
Matrix Fuzzy C-means Algorithm
J. Bezdek’s FCM-algorithm:
1
1
1
2
1
2
1
( ( ( ) )( ( ) ) )( ) ,
( ( ( ) )( ( ) ) )
( ) ( ).
( )
Tj j
j mT
l ll
N
jk
j N
jk
Tr x k c x k cu k
Tr x k c x k c
u k x kc
u k
. (29)
Experiments
MaxSelector
1x
2x
nx
mw
2w
1w
Class number
Classification system based on fuzzy clustering (the number of the rule nodes is equal to the number of classes)
Experiments
Data Fuzzy c-means
Batch possibilistic
Park-Dagher Recursive possibilistic
Iris 7,4% 7,6% 6,9% 7,8%Wine 3,8% 4,4% 3,9% 4,3%
Data Fuzzy c-means
Batch possibilistic
Park-Dagher Recursive possibilistic
Iris, class 3 33,3% 14,0% 33,3% 15,3%Iris, class 1 38,0% 6,0% 38,0% 6,7%Wine, class 3 29,0% 19,6% 29,0% 19,6%Wine, class 1 35,9% 23,6% 35,9% 23,4%Thyroid, class 3
18,1% 20,9% 18,6% 11,6%
Thyroid, class 1
15,4% 17,2% 15,5% 5.6%
Table 1: Classification with known number of classes (error rate on testing data)
Table 2: Classification with one unknown class (error rate on testing data)
Robust Probabilistic Fuzzy Clustering Algorithms
N
kj
m
jjkjjk
R ckxDkwcwE1 1
,, ),()(,
,,,1,1)(1
,
m
jjk Nkkw
.,,1,)(01
,
N
kjk mjNkw
,1 ,)(,
1
1
pckxcxD
pn
i
pjiijk
, ,k jx jic i )1( njk cx ,
Subject to constraints
(31)
(32)
where are th components of -vectors respectively.
(33)
(30)
Robust Probabilistic Fuzzy Clustering Algorithms
,s
cx
s),sSe(c),cp(x
i
ii
iiiii
2sech
2
1
icis
i
iiiiii
cxcxf
coshln),(
i
iiii
xxxf
tanh)()(
n
i
n
i i
jiiijuiij
R ckxckxfckxD
1 1
)(coshln),(),(
(34)
where and are the parameters that define the center and width of the distribution respectively.
, (35)
(36)
(37)
Robust Probabilistic Fuzzy Clustering Algorithms
.)(
coshln),(),(1 1 11 1
N
k
m
j
n
i i
jiiij
N
k
m
jj
Rjjj
E ckxwckxDwckwE
N
k
m
jj
i
jiiN
k
m
j
n
iijjj kwk
ckxkwkckwL
1 11 1 1
1)()()(
coshln)()(,),(
.0))(,),((
,0)(
))(,),((
,0)(
))(,),((
kckwL
k
kckwL
kw
kckwL
jjc
jj
j
jj
j
Objective function for robust clustering
Lagrange function
The system of Kuhn-Tucker equations
(40)
(39)
(38)
Robust Probabilistic Fuzzy Clustering Algorithms
N
kj
Rcjjjc
m
ll
R
ml l
R
jR
j
ckxDwkckwL
ckxDk
ckxD
ckxDkw
jj1
1
1
1
1
11
1
1
1
0),())(,),((
,),()(
,
),(
),(()(
m
jjj
Rm
jjjjk kwkckxDkwkckwL
11
)1)()((),()())(,),((
Local Lagrange function
(42)
(41)
Robust Probabilistic Fuzzy Clustering Algorithms
,)()(
tanh)()()(
)(,),()()()1(
,
),(
),(()(
11
1
1
1
i
jiijji
ji
jjkjiji
ml l
R
jR
prj
kckxkwkkc
c
kckwLkkckc
ckxD
ckxDkw
jikc , i jk
Arrow-Hurwitz-Uzawa procedure
where is the -th component of the -th prototype vector calculated at the -th step.
(43)
Robust Possibilistic Fuzzy Clustering Algorithms
N
kj
m
jj
N
kj
m
jjjjj kwckxDkwckwE
111 1
)(1),()(),),((
N
kj
Rcjjjjc
Nk j
Nk i
Rj
j
j
jR
posj
ckxDwckwE
kw
ckxDkw
ckxDkw
jj1
1
1
1
1
1
.0),(),),((
,)(
),()(
,),(
1)(
The criterion
The system of Kuhn-Tucker equation
(45)
(44)
Robust Possibilistic Fuzzy Clustering Algorithms
n
i
m
jji
i
jiii
m
jj
m
jjj
m
jj
Rjjjj
Rk
kwckx
kw
kwckxDkwckwE
1 11
11
)(1)(
coshln)(
)(1),()(,),(
,)()(
tanh)()()(,),(
)()()1(
,),(
1)(
1
1
1
i
jijji
ji
jjjkjiji
j
jR
posj
kckxkwkkc
c
ckwEkkckc
ckxDkw
Local Lagrange function
Adaptive possibilistic fuzzy clustering algorithm
(47)
(46)
Robust Possibilistic Fuzzy Clustering Algorithms
12 ))(1()( cxxp ii Experiments (48)
Complete data set (left) and its central part (right)
Robust Possibilistic Fuzzy Clustering Algorithms
Cluster prototypes layout obtained by different algorithms
Robust Possibilistic Fuzzy Clustering Algorithms
Table 3: Classification results
AlgorithmClassification error rate
Training Checking
Bezdek’s fuzzy c-means 17.1 % (1229 samples) 16.6 % (299 samples)
Probabilistic robust clustering (41) 15.6 % (1127 samples) 15.6 % (281 samples)
Possibilistic robust clustering (47) 15.2 % (1099 samples) 14.6 % (263 samples)