NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society

Two Approaches to Data-Driven Design of

Evolving Fuzzy Systems: eTS and FLEXFJS

Plamen Angelov, Edwin Lughofer, Erich Peter Klement

Abstract-In this paper two approaches for the incrementaldata-driven learning of one of the most effectie fuzzy model,namely of so-called Takagi-Sugeno type, are compared. Thealgorithms that realise these approaches include not only adap-tation of linear parameters in fuzzy systems appearing in therule consequents, but also incremental learning and evolution ofpremise parameters appearing in the membership functions (i.e.fuzzy sets) in sample mode together with a rule learning strategy.In this sense the proposed methods are applicable for fast modeltraining tasks in various industrial processes, whenever there is ademand of online system identification in order to apply modelsrepresenting nonlinear system behaviors to system monitoring,online fault detection or open-loop control. An evaluation of theincremental learning algorithms are included at the end of thepaper, where a comparison between conventional batch modellingmethods for fuzzy systems and the incremental learning methodsdemonstrated in this paper is made with respect to modelqualities and computation time. This evaluation is based on highdimensional data coming from an industrial measuring processas well as from a known source on the Internet, which underlinesthe usage of the new method for fast online identification tasks.

Index Terms- Incremental learning, adaptation of parameters,evolving Takagi-Sugeno fuzzy systems, rule learning, onlineidentification

I. INTRODUCTION AND STATE OF THE ART

Nowadays Takagi-Sugeno fuzzy systems play an importantrole for system modelling and identification tasks due to theircapability for approximating any nonlinear dependency to acertain degree of accuracy between some physical, chemicalor medical variables occurring in the real world. They arealso applied in applications areas such as system analysis(as they gain linguistic interpretable models in form of rulebases and therefore may yield a better understanding of someunderlying system behaviors), prediction, control (as theyprovide a good interpretation about local behaviors of a controlsystem through observing the steepness of the hyper-planesin different directions), fault detection or simply simulation.A specific task is the on-line identification of Takagi-Sugenofuzzy systems, where the training is in an incremental manner,taking into account the newly loaded data points. Opposed toa re-building of the systems from time to time by sendingall the data measured so far into a conventional batch trainingalgorithm, the incremental learning approach guarantees a fasttraining of the fuzzy systems, which can be indispensable

P. Angelov is with the Department of Communication Systems, Info-lab2l, Lancaster Univeristy, Lancaster, LA1 4WA, United Kingdom (e-mail:p.angelov @lancaster.ac.uk)

E. Lughofer and E. P. Klement are with the Department Knowledge-basedMathematical Systems, Johannes Kepler University of Linz, A-4040 Linz,Austria (e-mail: {edwin.lughofer,ep.klement}@jku.at)

within an online system. In addition to an on-line fuzzy systeman evolving fuzzy systems includes the adaptation of theknowledge-base, which is the basis of the model structure. Thetype of the membership functions which also detrmines themodels structure are usually not adapted because the Gaussiantype that is often assumed ensures the greatest generality (itassumes a normal data distribution within the cluster). Thistype of leaming prevents a virtual memory overload in the caseof a very huge amount of data (e.g. a database containing fourmillion measurements and 100 different continuous variables)and improving process security by preventing extrapolationto new operating conditions and by filling large interpolationholes.As Takagi-Sugeno fuzzy systems possess linear parame-

ters in the rule consequent functions (see Section ?? for adetailed definition), a recursive algorithm for updating linearconsequent parameters is quite often demonstrated in literature[?], [?] and will be also described shortly in subsequentsections. However, the adaptation of the linear rule consequentparameters alone is not satisfactory [?], as it is not capable toincorporate a description about new system behaviors or newoperating conditions by means of additional fuzzy sets andrules in the input space or to adapt the existing descriptions(antecedents of the fuzzy rules).

In this paper two recently introduced approaches for theevolving of Takagi-Sugeno fuzzy systems, eTS and FLEXFISare demonstrated and compared. They both exploit incrementalrecursive clustering for online training of fuzzy partitionsas well as rule bases, with the possibility of adjoining new

fuzzy sets and rules whenever there is a change in the input-output data pattern. Here also lies the difference betweenthese two approaches: where eTS takes into account theaccumulated spatial proximity between all data point seen so

far, FLEXFIS gives priority to the plasticity in the well known'plasticity/stability dilema' and adjoins new cluster centres ateach new data point that is not assigned to an existing cluster.In this sense, eTS is more robust while FLEXFIS is more

flexible. The role of the previous information in FLEXFIS isonly indirect through the cluster centers, while in eTS it alsoinfluences significantly the accumulated proximity measure,

called potential to form a new cluster centre. FLEXFIS exploitsvector quantization and adaptive resonance theory networks(ART), combines and extends them in a reasonable way to an

incremental learning mode for nonlinear parameters and therule structure in evolving Takagi-Sugeno fuzzy systems. Inboth approaches this online training strategy for the nonlinearpart is combined with the recursive adaptation of linear para-

meters appearing in the rule consequents to a complete data-0-7803-9187-X/05/$20.00 02005 IEEE. 31

driven incremental learning algorithm. In doing so, they areboth practicable for taking into account each new incomingdata point separately for the model building process andtherefore feasible for online learning and identification tasksof various industrial processes, where models should be up-to-date as fast as possible.

II. PROBLEM STATEMENTThe product t-norm together with Gaussian functions leads

to some favorable properties, namely steady differentiablemodels, equivalency to radial basis functions neural networks[?], favorable interpolation properties due to infinite supportand an easy extraction of the parameters for the Gaussian fuzzysets (which is for instance not the case for sigmoidal functions,which also possess infinite support). Due to these reasons,this combination is taken into account, leading to the specificcase of the Takagi-Sugeno fuzzy system with multiple inputvariables (xl, ..., xp) and a single output variable y defined by

c

fx) ==Ei () (1)

with the basis functions

e _P (x-C 2 (2)

2k=1 k

and consequent functions

i= ZU'io + w7ix1 + wi2x2 + ... + WipXp (3)

This special form of a Takagi-Sugeno fuzzy system is alsocalled FBFN (i.e. Fuzzy Basis Function Networks).

A. What to evolve?When inspecting the formulation of a Takagi-Sugeno fuzzy

system as in (??) we have to realize that principally threedifferent kinds of components should be evolved over time:

* Consequent Parameters: they appear in the rules conse-quents as output weights (wio,Iwi ..., wip)

* Premise Parameters: they appear in the input membershipfunctions as centers (cij) and widths (onj)

. Rule Base: this concerns not only the amount of rulesgiven by the summation limit C in (??), but also thenumber of fuzzy sets per input dimension, which influ-ences the amount of rules and readability of the wholesystem.

Consequent parameters are linear parameters, premise para-meters are nonlinear ones, hence two completely differentapproaches are demanded for evolving and will be described inthe following sections. Within all approaches the adjustmentof the parameters should be carried out in a way such thatpreviously learned relationships should not be forgotten whileapproximating new relationships (described by the new incom-ing points) as narrow as possible, meaning that the approxi-mation function obtained by the evolving mechanism shouldnot cause a significantly worse quality than an approximationobtained from conventional batch learning algorithms. This

will be underlined in Section ??, where some well-knownbatch leaming methods will be compared with the incrementalapproaches proposed in this paper.

III. ETS - BAsIc FACTSEvolving Takagi-Sugeno (eTS) models [?], [?] are flexible

model constructs that adapt their parameters and evolves theirrule-base in on-line mode. They have been applied to anumber of benchmark and real world problems for time-seriesprediction [?], model-based process control, classification andtarget recognition [?], packetized speech error recovery, multi-sensor autonomous systems [?], etc. The eTS approach canbe interpreted as a neural network with five layers [?]. Themain difference between the eTS and a conventional fuzzysystem described by the Takagi-Sugeno model as in (??)-(??)is that in eTS neither the number of rules i = 1, 2,, C, neitherthe linguistic terms (model antecedents) in all p dimensions,neither the model consequent parameters as in (??) are knownor fixed.The eTS approach tries to extract this information directly

from measurement data and this in online mode applyingincremental learning. Hereby, the rule-base in eTS evolvestracking the changing data pattern. It can grow by adjoiningnew rules formed around focal points. The prototypes of suchnew focal points are the data points with very high spatialconcentration, measured by its potential [?] or alternatively byits scatter [?]. The scatter (a measure similar to the variancein a cluster, but applying to all input-output data vectors,z = (Y, ) for the new data point) is given recursively byM?:

(k -1)Z 1zk -2 'i1i Zk/3jk + -Yk

Sk (k-1)(p + 1)

(4)p+1

-Yk = -Yk-1 +Zj(k-1)j=1

3jk = /j(k-1) + Zj(k-1)

(5)It is compared to the scatter measured at each of the existingfocal points/cluster centres/neurons, which is updated recur-sively to take into account the influence of each new datasample:

p+lSk* Stki + E(Zjk -Zj(k-1))2i 1J..., Ck=k - I

j=1(6)

If the scatter at the new data sample is lower than the scatterat the existing focal points, that is:

IF (Sk" < S )Vi THEN (Fomi a new rule atzk) (7)

The new rule that is added to the rule-base is described by:

Rulec+j :IF x1 IS bL(c+1)1 AND X2 IS p,C+1)j AND...AND xp IS p(C+1)p THENW(C+1)O + W(C+l)lXl + ... + W(C+l)pxp (8)

32

with 11(c+1)j described by c(C+±)j set to:

C(C+1)3 = Xkj Vj E f{1,.p}(9)

When local identification criteria is satisfied then the linearparameters W(c+1)j of the weighted RLS are initialised with

c

C(C+ )j- Cjj i V) (10)i-1

The case of global identification is also possible and it isconsidered in detail in [?]. The rule-base can also be modifiedby replacing an existing rule by a new one formed around amore potential prototype:

IF (Snew < Sk) AND (d < ') THEN (Replace)a = minr ltIZjk-Zj,I12 i= ,...,C;j =1,...,p+ 1

The rule that replaces an existing one is described as in (??)by using rule number i instead of using the new (C+ 1)th rule,and where the linear parameters W(new)j are taken from therule to be replaced uwij.The distance d to the nearest of the existing rules is

compared to the cluster width to avoid forming new rules tooclose to existing cluster centers.

The possibility to delete rules that are already accepted inthe rule-base based on i) the similarity of the individual fuzzysets that forms them (projections of the data clusters) [?]; ii)the number of data points in each cluster (cluster population)[?] has also been studied. In summary, eTS is a Takagi-Sugenomodel as defined in (??)-(??) with flexible, evolving rule-basethat learns from data starting from scratch. The input data x,a vector with dimensionality p, is collected incrementally. Theoverall model outputs, y are formed by superposition of localsub-models as demonstrated in (??)-(??). Once the output(s)prediction y is made it can be used in real-time until thereal output(s) are collected. Based on the real outputs andinputs the clustering is updated, modified or simplified (ifnecessary conditions are satisfied). This concludes the rule-base evolution. At each step the consequent parameters arealso adapted using a modification of the well-known recursiveleast squares (RLS) procedure [?] as defined in (??)-(??) bysetting A to 1.

IV. FLEXFIS - BASIC FACTSWhile eTS described in the previous section extends the

subtractive clustering for on-line recursive and evolving fuzzyrule-base modelling, FLEXFIS exploits vector quantization inan open-loop variant and combined with the idea of ART-networks, i.e. to take into account the distance of newly loadedpoints to all the cluster centers generated so far. This distanceis compared with a so-called vigilance parameter and if it isgreater a new cluster is set, if this is not the case, the nearestcluster (also called winner neuron cwin) is updated for a new

incoming point with respect to its width w by exploitingrecursive variance formula [?] for each dimension j:

kiOrwinj = (ki-1) win,j +ki Acwinj + (Cwin,j -Xkj)2 (11)

where ACwin,j is the distance of the old prototype to the newprototype of the winner neuron in the jth dimension and kwinis the amount of data points lying nearest to cluster cwin andcan therefore be simply updated through counting. The centerof the winner neuron is updated by

(new) _(old) + . - -old)C-in -Cwin +(x- Cwin J (12)with 77 the learning gain.From the clusters the fuzzy sets are extracted by projection

and the rules are obtained by simply connecting fuzzy setsbelonging to one cluster with a t-norm. Hence, in the case anew cluster is born, automatically a new rule (the C + lth) isborn, see (??), where the width of the new cluster a(C+1)j isinitialized with:

c(C+±)j = * ran.ge(fj) Vj E {1,./p} (13)and where the linear weights w(C+i)j are set initially to 0 forstart values to weighted RLS. From this point of view, a clusterupdate always belongs to a rule update. For details about thislearning strategy of the antecedent part in a Takagi-Sugenofuzzy system and evaluation results with respect to a perfor-mance comparison on 2-dimensional data with- conventionalgenfis2 (as implemented in MATLAB [?], [?]) refer to [?].

This learning strategy for the rule base and the antecedentparts therein is combined with the recursive weighted leastsquares approach for updating the linear consequent parame-ters. The weighted version of the well known RLS [?] has to betaken here, as for each newly born rule a separate estimationof consequent parameters is initiated, not disturbing the thealready estimated parameters of the other rules. This separateestimation and adaptation is only valid in regions, where thebasis functions (??) are close to 1, hence a weighted leastsquares (for estimation) and a recursive weighted least squares(for adaptation) has to be applied, where the values of the basisfunctions act as weights in the formula RLS formula, i.e. forthe ith rule. A forgetting factor, A is usually used [?], due towhich a forgetting of older data points (because they are notso important any longer) is possible. If setting this factor to1, no forgetting is carried out and all data points influence theupdate of the parameters with equal intensity. Forgetting factorhas not been used in eTS, although it can be incorporated inthe same way.

Wzi(k + 1) = wi(k) + (k)(y(k + 1)-r(k + 1)w, (k)) (14)-y(k) =- Pi(k)F(k + 1)

I(x( + r(k + 1)Pi(k)F(k + 1)

Pi(k + 1) = (I- -y(k)FT(k + 1))P,(k)1

(15)

(16)

with Pi(k) = (Ri(k)TQ,(k)Ri(k))-l the inverse weightedinverse Hesse matrix and r(k + 1) = [1 xi (k + 1) x2(k +1) ... xp(k+ 1)]T the regressor values of the (k + ith) datapoint, which are obviously the same for all i rules.

33

Moreover, the weighted and separate variant for obtaininglinear parameters (also called local estimation approach) leadsto some advantages over the non-weighed one such as fastcomputation time, a higher stability for the inverse Hessematrix and a better interpretability of rule consequent functionsetc., see also [?] and [?]. Finally the combination of open-loopclustering and weighted RLS leads to the incremental learningalgorithm FLEXFIS as proposed in [?] [?]. An important facttherein is, that the ranges of the input variables have to beknown in advance (this is due to (??) and that VQ-ARTclustering is only reasonable with normalized data), thereforea purely data-driven incremental learning from scratch iscurrently not possible, because ranges are usually not knownand hence have to be estimated out of data. Investigations onthis point are carried out in Section ??. Another essential pointis the incorporation of so-called correction terms, which ensurean exact optimal solution of linear parameters in the leastsquares sense during incremental learning process. Duringonline training unfortunately this explicit calculation was notpossible (because this would have lost the incremental learningnature of the algorithm), but it could be shown, that byomitting these correction terms during the online learning (bysimply setting them to 0), the algorithm still converges fora typical parameter adjustment, especially with respect to rin (??) (so does not break out and deliver nonsense solutions)and delivers after all a suboptimal solution in the least squaressense. The problem of local and global learning and the- sub-optimality is also addressed in [?].The method is successfully implemented, verified and eval-

uated (compared to other methods) within an online faultdetection framework [?] for measurements recorded at enginetest benches.

V. ADVANCED ASPECTSA. Advanced Aspects for eTS

Initially, the concept of evolving rule-based models [?]has been partially recursive (in respect to the consequentparameters only) and thus an on-line rule-base evolution wasonly possible for the price of using a sliding window. Inthe eTS concept [?], [?] both the consequent parameters andthe rule-base are recursively updated. This is achieved bydecomposition of the identification problem and the sequentialupdate of the both parts of the fuzzy model. A modification ofthe original RLS algorithm was also required as well as a newformula for the recursive calculation of the potential of eachnew data point to be a new focal point. Some improvementsover the basic concept of the eTS models developed recentlyand under development currently include:

. eTS has been extended to the multi-input-multi-output(MIMO) case [?];

. Vector radius of clusters - allows hyper-ellipsoidal clus-ters, but still alongside the main axes, because of the useof Euclidean distance as similarity measure [?];

* Online normalization of the data samples using a recur-sive formula [?];

* Computational and conceptual simplification of the no-tion of Potential into the notion of Scatter that is relatedto the variance [?];

* Online adaptation of the cluster radius - under develop-ment;

* Online fuzzy sets simplification leading to improvedtransparency [?];

. Evolving classification based on the evolving clustering(one part of the eTS algorithm) and 'winner take all'.This modification was applied to on-line classification ofEEG signals [?];

. Adaptive target recognition using evolving classifier thatlearns on-line in real-time and takes multi-sensor readingsto learn classification rules from scratch [?];

This is a brief list of the recent modifications implemented inthe eTS algorithm or under development currently. Due to thespace limitations more details are omitted and can be find inthe respective references.

B. Advanced Aspects for FLEXFISThe advanced aspects for FLEXFIS mainly concern three

issues:. Improving interpretability for Takagi-Sugeno fuzzy mod-

els trained with FLEXFIS in incremental mode.. Improving process security by neglecting outliers with

the incremental training process* The possibility for incremental learning from scratch

For the first point clear ideas exist based on a fuzzy set andrule merging strategy during online process within a chapterof an early published phd-thesis [?]. For the second point apossible simple approach is the so-called rule base updateprocrastination strategy: whenever a new sample comes inwhich lies far away from the previously estimated clusters,a new cluster is born immediately, but not a new rule andhence no new fuzzy sets. Is is waited for more data pointsappearing in the same region until a new rule is set. This isbased on the assumption that the more data points in a newregion occur the more likely it gets that they denote a newoperating condition and no fault. The precarious thing is thatthe assumption mentioned above is not always true, when asystematic error within the system occurs. In such cases twopossible workarounds can help:

. Filtering of dynamic measurements with signal analysismethods in intelligent sensors as it was proposed in [?],which also effects a filtering or 'cleaning' of stationarymeasurements which are elicited through averaging ofdynamic ones.The exploitation of analytical and knowledge-based mod-els in a complete fault detection framework as demon-strated in [?] for delivering a fault detection statement forany loaded or online recorded data point. This statementcan be taken as input for the incremental learning methodand reacted upon it such that points classified as faultyare not incorporated into the adaptation process. Froman application example based on a specific data set weknow that the detection rate (i.e. the number of correctlydetected faults) can then be increased by about 15-20%.

For the incremental learning from scratch a vague idea exists,which is based on an adaptive vigilance parameter p in

34

dependency of the nature (quality and amount) of data pointsloaded so far.

VI. COMPARISON OF METHODSA. Comparative Analysis

While eTS filters out the outliers in a natural manner, asthe mechanism for new cluster formation in eTS is robust andconservative, because it is based on the accumulated prox-imity measure between all previous data samples, FLEXFISprovides a rule base procrastination strategy (see previoussection), where all newly loaded points are incorporated intothe clusters, but not into the rules and rules' premise parts.Furthermore, both strategies are very fast, as they can evolvefuzzy systems in sample mode manner, which makes themsuitable for real-time applications with high sampling rates(the packetised speech problem for example considers voicedata sampled at 20 ms), which will be also underlined inSection ??. Both approaches are non-iterative and use a com-bination of incremental clustering and recursive least squares,respectively recursive weighted least squares. The incrementalclustering variants are different: while eTS builds upon thesubtractive clustering, FLEXFIS exploits vector quantizationwith adaptive resonance theory network. In both cases itis shown that with respect to a specific parameter settingscheme a suboptimal solution in the least squares sense canbe achieved for the linear parameters in the rules consequents[?] [?]. Incremental learning from scratch is only possible foreTS, for FLEXFIS an initial fuzzy model has to be built upwith the first few dozen of data points, where the more initialpoints are used the more accurate the incremental learning atthe beginning (after this few dozen of points) and hence thebetter the approximation behavior.

B. Test ResultsIn this section test results are produced from some real

recorded data sets. These test results include not only acomparison between the incremental learning variants eTS andFLEXFIS, but also among other well-known batch learningvariants for Takagi-Sugeno fuzzy systems, including:

* ANFIS: Adaptive Neuro-Fuzzy Inference Systems [?], [?]. FMCLUST: cluster-based learning method [?] for a prac-

tical description of the usage of the method in MATLAB. genfis2: cluster-based learning method, see [?], [?]

and should underline the better usability for online identifica-tion and prediction tasks.

In table ?? model quality results on the auto-mpg data fromthe UCI-repositoryl are shown. The data concerns city-cyclefuel consumption in miles per gallon and consists of eightattributes including one class attribute (target channel), namelythe so-called 'miles per gallon', and seven input attributes,Five input dimensions were enough to describe the relationshipsignificantly, the results are shown in Table ??: FMCLUST,genfis2 extended and FLEXFIS show almost similar resultswith respect to approximation accuracy on fresh test and areeven slightly better than the best method demonstrated in

lhttp://www.ics.uci.edu/ mlearn/MLRepository.html

TABLE IMODEL ACCURACY OF DIFFERENT METHODS WHEN APPLYING TO

auto-mpg DATA FROM UCI REPOSITORY

Method-

Correl [ CPU,s7FMCLUST 0.917 1.3ANFIS 0.730 5.0genfis2 conv. 0.855 0.6genfis2 ext. 0.916 0.5genfis2 ext. conv. adapt 0.210 1.9FLEXFIS sample mode 0.912 3.1eTS 0.924 0.4

TABLE IICOMPARISON OF FuzzY MODEL BUILDING METHODS WITH RESPECT TO

QUALITY AND COMPUTATION SPEED

Method Quality Comp. Time Comp. TimeTest online 1 online 2

up-to-date up-to-dateevery 100 p. each point

FMCLUST 0.902 62m 41s Not poss.ANFIS 0.872 >genfis2 Not poss.genfis2 conv. 0.893 38m 31s Not poss.genfis2 ext. 0.904 34m 13s Not Poss.genfis2 ext. c. a. 0.818 3m lOs 3m lOsFLEXFIS 0.881 4m 36s Not poss.batch mode 100FLEXFIS 0.856 lOm 57s lOm 57ssample mode

[?] (LAPOC-VS), which achieves a correlation coefficient ofbetween 0.89 and 0.90 (Note: this is known due to a statementof the co-author of the paper). Obviously, eTS is superior toall the others with respect to approximation quality, whichis quite a strong property, as it can be also used for onlinetraining. As incremental training method it also is preferable toFLEXFIS with respect to computation time, which was around4 times higher (it should be noted that the experiments wereperformed on different PCs - FLEXFIS was run on 1.6GHzmachine, while eTS was run on a 3.2GHz PC). ANFIS andgenfis2 extended with conventional adaptation are more orless forgettable. Note: With conventional adaptation here it ismeant that only the rule consequents are trained incrementalmanner and no rule evolving and fuzzy set adaptation strategyis included, which is quite often proposed in literature, seealso Section ??. As obviously the first few dozen data pointsare not distributed sufficiently over the input space, the resultswith respect to this approach are useless.The computation results of genfis2 comy and genfis2 ext. are

quite good, when training with the complete data set at once.However, in the case of online modelling and identificationthe computation time will suffer, when trying to build up themodels for each data block or even more for each data sample,see Table ?? - here 62 up-to 5-dimensional models are builtfrom a high-dimensional data set: it can be clearly seen thatthe computation speed of conventional offline methods suffersextremely opposed to the evolving variant FLEXFIS. This isbecause a re-building of all the 62 fuzzy models has to becarried out for each block containing 100 data points (3rdcolumn) or single points (=real online mode: 4th column). This

35

TABLE IECOMPARISON OF EVOLVING VARIANTS WHEN APPROXIMATING A

DYNAMIC HIGH-DIMENSIONAL RELATIONSHIP

Method Quality3 features

/ No. of Rules_ CPU,s

eTS 0.904 / 4 / 0.81FLEXFIS 0.892 / 5 I 2.23

Quality4 features

/ No. of Rules/ CPU, s

0.906 / 4 I 0.800.903 / 5 / 2.76

Quality5 features

I No. of Rules/ CPU, s

0.915 / 3 / 0.760.911 I 5 1 3.18

makes them hardly applicable in fast identification processesand even totally inapplicable for online identification tasks,where fuzzy models have to be up-to-date for each small buffercontaining just a couple of points or even for each single point(see column 4). This is because, for batch modelling methods,single point update would mean training the model with kpoints (initial model), k+1, k+2, k+3, ..., k+N points. So, inthe case of this large diesel engine data consisting of 1810samples to perform 1710 re-estimations with 101, 102, 103,... 1810 data points, summing up to a computational effort ofhours or days. The eTS approach has not been tested on thisproprietary data.

Another test was performed on building a model for adynamic relationship in form of a prediction model, whosetarget was defined by the emission channel NOX for a carengine. This task emerged from the demand of saving expenseson a measurement sensor for NOX at an engine test bench bydescribing this channel through a formula that relate othervariables. It turned out, that at least 4 inputs (some originalchannels and their time delays) were needed in order to obtaina correlation higher than 0.9, see Table ??. The input channelsfor approximating NOX at time instant k consisted of thefollowing list of channels (in the order they were selected):N = Engine Rotation SpeedP2offset = Pressure in Cylinder number 2Te = Engine Output TorqueNd = Speed of the Dynanometertogether with their appropriate delays yielding a dynamicmodel in form of a four-step-ahead prediction of NOX de-scribed by

NOX(k) =f(N(k - 4), P2offset(k - 5), Te(k -5),Nd(k - 6), N(k -6)) (17)

where one step back denotes exactly one second. Whenreducing this input space to the first four or even the first threeinputs the approximation accuracy did not suffer significantlyas stated in Table ??, where the correlation coefficient is againtaken as quality measure on a fresh test data set. Moreover,in this case the complexity of the model (number of rules andfuzzy sets) can be reduced as firstly a dimension reductionalways leads to a better transparent and readable fuzzy model.

VII. CONCLUSION

Concluding, in this paper two approaches for evolvingfuzzy systems were described and compared among eachother and also among conventional well-known batch learning

variants based on various real-recorded data sets. Hereby,the focus were not only sharpened on approximation qualityand complexity of the obtained models, but also on thecomputation time for building them up, as this is an essentialpoint whenever fuzzy models should be trained in online modefor various online application cases. The results of both, eTSand FLEXFIS are encouraging and improved model qualitycould be achieved in comparison with the batch learningmethods, but more importantly both possess an incrementallearning mechanism, which makes it possible to update themodels during an online process whenever they have to.

36