[IEEE 2005 International Conference on Neural Networks and Brain - Beijing, China (13-15 Oct. 2005)] 2005 International Conference on Neural Networks and Brain - A modified neural network based on subtractive clustering for bidding system

  • Published on

  • View

  • Download

Embed Size (px)


  • A modified neural network based onsubtractive clustering for bidding system

    Min Han, Yingnan Fan, Wei GuoSchool of Electronic and Information Eng., Dalian Univ. of Technology, Dalian, China, 116023

    E-mail: minhan@dlut.edu.cn

    Abstract-The paper presents a modified neural networkbased on subtractive clustering (NN-SC). It can be used toestimate the mark-up of construction bidding system. Inrecent years, many neural fuzzy approaches to model areproposed. But they are limited for complex and arbitrary incomputation and structure. In this paper, the NN-SC isproposed to overcome the drawbacks mentioned above andhave fuzzy inference and self-learning ability. It usessubtractive clustering to generate rules and form rulebase.With rule inference steps, it is convenient to determine thedegree of applicability for each rule. Therefore, it has highdegree of transparency, compact structure and computationaleffi'ciency. And based on neural network, nonlinear mappingbetween input and output is accomplished. With thesimulation, it is proven that the proposed network is validand has good performance.

    Index Terms-neural network; subtractive clustering;rule; biding system


    Recently, neural fuzzy approach to system modeling hasbecome a popular research focus. The key advantage ofneural fuzzy approach over traditional ones lies on that theformer doesn't require a mathematical description of thesystem while modeling [1]. Moreover, in contrast to pureneural network or fuzzy methods, neural fuzzy methodpossesses both of their advantages; it brings the low-levellearning and computational power of neural networks intofuzzy systems and provides the high-level human-likethinking and reasoning of fuzzy systems into neuralnetworks. A general fuzzy logic system contains fourmajor components [2]: fuzzifier, rule base, inferenceengine, and defuzzifier. Rule base and defuzzifier are themost important procedure in the four phases because theycan improve the transparence of neural network. AlthoughFuzzy logic introduced into neural network has manyadvantages, it exists some disadvantages that have limitedits application, such as the increasing computation of fourcomponents in fuzzy logic system and problem ofobtaining the rulebase. In addition, the choice ofmembership function of fuzzy sets is rather arbitrary foranybody [3]. To remove the shortcomings of neural fuzzysystem, a modified neural network based on subtractiveclustering is proposed.The main issues associated with a neural fuzzy system

    are 1) parameter estimation, which involves determining theparameters of premises and consequences, and 2) structureidentification, which concerns partitioning the input spaceand determining the number of fuzzy rules for a specificperformance [4]. The parameters can be adjusted by BPalgorithm or hybrid learning algorithm. In general, thestructure identification involves initial rule generation in theform of IF-THEN in terms of fuzzy sets of data dimensions [5]which would approximate the final rule base. To get fuzzyrules, several algorithms have been developed to generatefuzzy rules from numerical training data.

    For a given data set, different fuzzy rules can be obtainedusing different identification methods. Grid partitioning (GP)[6] and fuzzy clustering methods are used to identify theantecedent part of the neural fuzzy system popularly.Subtractive clustering [7] is an efficient fuzzy clusteringalgorithm. Compared with GP method, it need not optimize.Therefore, it is a good choice for the initialization of neuralfuzzy network. Fuzzy c-means and other optimization-basedclustering techniques would lead to excessive computer workbecause they perform a necessary optimization phase beforenetwork training. Moreover, progressive clustering andcompatible cluster merging algorithms are computationallyexpensive and need metrics for validation of individualclusters [8].Therefore, despite of their potential, they are toocomplex to train a network. Chiu's algorithm belongs to theclass of potential function methods, being more precisely, avariation of the mountain method. In this class of algorithms,a set of points are defined as possible group centers, each ofthem being interpreted as an energy source. In subtractiveclustering, the center candidates are the data samplesthemselves [9], which overcomes the main limitation of themountain method. In fact, there, the candidates are defined ina grid, will lead to "curse of dimensionality" problems. In thispaper, the rules of modified neural network are generated bysubtractive clustering algorithm to design a new structure.Based on the research of neural fuzzy systems, a modified

    neural network is proposed. The new structure of neuralnetwork is derived from the neural fuzzy network based onT-S model [3]. It is parallel and based on the rules. In thispaper, the structure of the model is obtained by means ofsubtractive clustering. For adopting the new method to definethe degree of applicability for IF-THEN rules, it does not usefuzzy membership function, so the fuzzifier and defuzzifierare needless. Hence, the structure of network is more compact.At the same time, the complexity of computation is decreased.

    0-7803-9422-4/05/$20.00 2005 IEEE128

  • It is obvious that the modified neural network model notonly has the characteristics of fuzzy systems, but also hassome advantages of neural network. To illustrateperformance of this network, it is used to the constructionbidding system.The paper is organized as follows. In section II, the

    subtractive clustering is introduced, including the choice ofparameters and acquisition of rules. Section III describesthe structure and algorithm of the modified neural network.The simulation results and analyses are provided in sectionIV. Finally, the conclusions are given in section V.


    Let Zvbe a set ofN data samples, zI, Z2,v', ZN, defined inan m+n space, where m denotes the number of inputs and ndenotes the number of outputs. In order to make the rangeof values in each dimension identical, the data samples arenormalized so that they are limited by a hypercube.As it was referred, it is admitted that each of the samples

    defines a possible cluster center. Therefore, the potentialassociated to zi is Eq.(l):

    N 12i (Zi, ZN) =Eexp(-aIz,-z I )j=l

    i=l,2,*--,N; i.j (1)4


    where ra >0 is the radii and defines the neighborhoodradius of each point. Thus, points zj located out of theradius of zi will have a smaller influence in its potential.Consequently, the effect of points close to zi will growwith the proximity. Hence, points with a denseneighborhood will have higher associated potentials. Aftercomputing the potential for each point, the one with thehighest potential is selected as the first cluster center. Next,the potential of all the remaining points is reduced.Defining z4 as the first group center and denoting itspotential as P7, the potential of the remaining points isreduced as in Eq.(2):

    P v P - Pj exp(-fiz -Z11j2)i=1,2,---,N; i#j (2)4rb

    where the radii r,>0 defines the neighborhood radius withsensitive reductions in its potential. Accordingly, pointsclose to the selected center will have their potentialsreduced in a more significant manner. So the probability ofbeing selected as centers diminishes. This procedure hasthe advantage of avoiding the concentration of identical

    clusters in denser zones. Therefore, rb is selected in order tobe slightly higher than ra to avoid closely spaced clusters, forexample, rb=l.5ra. With the reduction of potential for all ofthe candidates, the one with the highest potential is selectedas the second cluster. Then, the potential of the remainingpoints is again reduced. Generically, after the rthl group isdetermined, the potential is reduced as follows Eq.(3):

    (3)The procedure of center selection and potential reduction is

    repeated until the following stopping criterion [10] isreached:

    1) lf Pk > EpI accept zk as the next cluster center andcontinue.

    2) If Pk < ,E,reject 4 and finish the algorithm.3) Let dmi, be the shortest distance between z4 and all the

    centers already foundIf dm,I,/r +IP/i .1, accept zk as the next cluster

    center and continueOtherwise, reject Z and assign it the potential 0.0,

    select the point with higher potential as the new Zk,and repeat the test.

    Where k==[l , ,r], is the upper threshold that the point isselected as a center with no doubts and c is the lowerthreshold below that the point is definitely rejected. The thirdcase is the point characterized by a good tradeoff betweenhaving a sufficiently high potential and being distant enoughfrom the clusters determined before.As it can be understood from the description of the

    algorithm, the number of clusters obtained is notpre-specified. However, it is important to note that the radiiparameter is directly related to the number of clusters found.Thus, a small radius will result in a high number of rules,which, if excessive, may be overfitting. On the other hand, ahigher radius will result in a smaller number of clusters,which may generate underfitting. Models with reducedrepresentation are accurate. In practice, it is necessary to testseveral values for radii and select the most adequate oneaccording to the results obtained. This implies an importantadvantage over optimization and other classes of clusteringalgorithms when little information is known regarding theoptimal number of clusters. Another advantage of subtractiveclustering is that the algorithm is noise robust, since outliersdo not significantly influence the choice of centers, due totheir low potentials.

    With subtractive clustering, each of the obtained clusterswill constitute a prototype for a particular behavior of thesystem under analysis. So, each cluster can be used to definea fuzzy rule capable of describing the behavior of the systemin some region of the input-output space. The rules can beused to design the structure of the neural network.




    A. Determination ofthe Degree ofApplicabilityThe ability of the modified neural network is analogous

    to the neural fuzzy network based on T-S model, while thestructure is a neural network, and it has learning ability.Based on the rules obtained by subtractive clustering candescribe the behavior of the system, it is a main part of thenetwork that the degree of applicability for each rule isdetermined. According to the input vector, the degree ofapplicability can be determined for each IF-THEN rule ateach time sample.

    In neural fuzzy systems, the degree of applicability foreach rule is determined by means of choice of fuzzymembership function and computation of fuzzymembership degree. In this paper, Gauss function ischosen as fuzzy member function and shown in Eq.(4):

    1k = exp[ (x 2- )21I = 1,2,...m;k = 1,2, ... n (4)

    Where lIk denotes the membership degree for kth languageterm of xi, X4=[xl, ", xl,", x,,m] is input vector, xIk and olkdenote the center and width of Gaussian membershipfunction, n, denotes the number of membership function.Therefore, the maximum of P1k is obtained and denoted by

    P11i = maxulk I = 1, 2,**,m (5)Then, the degree of the applicability uj for the jth rule

    is defined:m m

    Pi=flu or p,=flp (6)1 1 =1-

    Fuzzy sets for the linguistic terms can be defined bydifferent number and type membership functions, so thedetermination of the degree of applicability is ratherarbitrary and subjective, being dependent on someempirical knowledge and prior information [3]. In order toovercome these drawbacks, the Euclidean distancebetween input vector Xi and center C is used to determinedegree of applicabilityu,uJX1) for IF-THEN rules in Eq.(7):

    Pi (Xi) = I-jlix - c11i=1,2, -N; j = 1,2, --,R (7)

    Where Cj is the center generated by subtractive clusteringalgorithm, R is the number of rules. When the input vectorXi approaches the center Cj more closely, the degree ofapplicability will be closer to unity and the premises inIF-THEN rules are more closely fulfilled. Using thismethod, the network needn't carry out fuzzifier anddefuzzifier step, so the structure of the neural network ismore compact and the efficiency of computation isimproved.

    B. Structure ofthe ModifiedNeural NetworkThe multi-input and multi-output system is composed of

    many multi-input and single-output systems in parallel way,so in this paper the network is designed to multi-input andsingle-output system. The structure of the modified neuralnetwork based on is shown in Fig. 1.

    Consequent subnetwork


    r 1,layer layer layer laye // OutputlayerIL

    _-- . _ _ ff,

    I I

    Input layerRule layer

    Antecedent subnetwork

    Fig. 1 Structure of the modified neural networkThe network is made up of two subnetworks: the

    antecedent subnetwork and the consequent subnetwork. Theantecedent network is designed to match the "IF" part offuzzy rules and generate the degree of applicability of eachrule for the input vector. And the consequent network isapplied to compute the "THEN" part of fuzzy rules. Theoutput of the whole neural network is weighted sum of theconsequent value of all rules, which the weights are thedegrees of applicability for each rule.The antecedent network contains two layers: input layer

    and rule layer. Input layer passes on the input data to rulelayer. In rule layer, the degree of applicability of each rule forthe input vector is determined by Eq.(8):

    22 = 1_ II(I2W-C,)J2 2 (8)where Ij22, o22 denote the input and output of the 2th layer(i.e. rule layer) in antecedent subnetwork, o is theconnection weights between input layer and rule layer. Thenumber of nodes in this layer is equal to the number of rules.The degree of applicability is viewed as weight of middleoutput layer ofthe consequent subnetwork.The consequent subnetwork contains five layers: input

    layer, hidden layer, middle output layer, multiplier layer andoutput layer. The structure is different from standard BPneural network because a middle output layer is inserted andthere is no given teach signal.The input layer is the first layer of input layer in

    consequent subnetwork. And input and output can be denotedas flows:

    I>=X;i O =11


  • The second layer is hidden layer, in which the nonlinearmapping is implemented from input data to output value.And input and output can be denoted as flows:

    NkIk =Z i XOik; k f(Ik)

    k=1The third layer is middle output layer. The number of

    nodes equals to the number of rules. The output is theconsequent value of each rule. And Input and output can bedenoted, respectively:

    RI= X

    j=l oil = f(I' )The fourth layer is multiplier layer. In this layer, the

    degree of applicability as weight products the output of themiddle output layer as the consequent value of rules. Thenumber of nodes also equal to the number of rules. So, theinput and output are:

    I, = &j X 022. o4 = 4where o22 = uj is the degree of applicability for the jthrule, i.e. one of the output in antecedent subnetwork.

    The fifth layer is output layer.R R

    2I15 =L...


View more >