Research ArticleNonlinear System Identification Using Quasi-ARX RBFN Modelswith a Parameter-Classified Scheme

Lan Wang1 Yu Cheng2 Jinglu Hu3 Jinling Liang4 and Abdullah M Dobaie5

1Wuxi Institute of Technology Wuxi 214121 China2Ant Financial Services Group Hangzhou 310099 China3Graduate School of Information Production and Systems Waseda University Kitakyushu 808-0135 Japan4School of Mathematics Southeast University Nanjing 210096 China5Department of Electrical and Computer Engineering King Abdulaziz University Jeddah 21589 Saudi Arabia

Correspondence should be addressed to Jinling Liang jinllianggmailcom

Received 15 June 2017 Accepted 6 November 2017 Published 11 December 2017

Academic Editor Dimitri Volchenkov

Copyright copy 2017 Lan Wang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Quasi-linear autoregressive with exogenous inputs (Quasi-ARX) models have received considerable attention for their usefulnessin nonlinear system identification and control In this paper identification methods of quasi-ARX type models are reviewed andcategorized in threemain groups and a two-step learning approach is proposed as an extension of the parameter-classifiedmethodsto identify the quasi-ARX radial basis function network (RBFN) model Firstly a clustering method is utilized to provide statisticalproperties of the dataset for determining the parameters nonlinear to the model which are interpreted meaningfully in the senseof interpolation parameters of a local linear model Secondly support vector regression is used to estimate the parameters linear tothe model meanwhile an explicit kernel mapping is given in terms of the nonlinear parameter identification procedure in whichthe model is transformed from the nonlinear-in-nature to the linear-in-parameter Numerical and real cases are carried out finallyto demonstrate the effectiveness and generalization ability of the proposed method

1 Introduction

Many real-world systems exhibit complex nonlinear char-acteristics and hence cannot be identified directly by linearmethods In the last two decades nonlinear models suchas neural networks (NNs) radial basis function networks(RBFNs) neurofuzzy networks (NFNs) and multiagent net-works have received considerable research attention fornonlinear system identification [1ndash4] However from a userrsquospoint of view the conventional nonlinear black-box modelshave been criticized mostly for not being user-friendly (1)they neglect some good properties of the successful linearblack-boxmodeling such as the linear structure and simplic-ity [5 6] (2) an easy-to-use model is to interpret propertiesof nonlinear dynamics rather than being treated as vehiclesfor adjusting fit to the data [7]Therefore careful modeling isneeded for a model structure favorable to certain applica-tions

To obtain the nonlinear models favorable to applicationsa quasi-linear autoregressive with exogenous inputs (quasi-ARX) modeling scheme has been proposed with two partsincluded a macro-part and a core-part [14] As shown inFigure 1 the macro-part is a user-friendly interface favorableto specific applications and the core-part is used to representthe complicated coefficients of the macro-part To this endby using Taylor expansion or othermathematical transforma-tion techniques a class of ARX-like interfaces is constructedas macro-parts in which useful properties of linear modelscan be introduced while their coefficients are represented bysome nonlinear models such as RBFNs In this way a quasi-ARX predictor linear with input variable 119906(119905) can be furtherdesigned where 119906(119905) in the core-part is replaced skillfullyby an extra variable Thereafter a nonlinear controller canbe generated directly from the quasi-ARX predictor whichis similar to the simple linear control method [15 16]In contrast complex nonlinear controller design should

HindawiComplexityVolume 2017 Article ID 8197602 12 pageshttpsdoiorg10115520178197602

2 Complexity

Macro-partinterface for specific application

Core-partrepresenting coecients of

macro-part

Core-partnonlinear model

Macro-partARX-like linear structure

An exampleBasic idea

sum

(a) (b)

Figure 1 Quasi-ARX modeling Basic idea of the quasi-ARX modeling is shown in (a) where a macro-part and a core-part are included inthe constructed model An example is illustrated in (b) An ARX-like linear structure works as macro-part for specific application whosecoefficients are parameterized by a flexible RBFN model

be considered in NN based control methods where twoindependent NNs are often contained the one used forpredictor and the other used for controller [17]

Actually similar block-typemodels have been extensivelystudied and named in several forms according to theirfeatures such as the state-dependent parameter models [18ndash20] and local linear models [10 21] Basically identificationmethods can be categorized into three schemes

(1) Hierarchical identification scheme quasi-ARXmodelstructure can be considered as an ldquoARX submodel+ NNrdquo when NNs are utilized in the core-part [1516] and a hierarchical method has been proposed toidentify the ARX submodel and the NN by a dual-loop scheme where parameters in the ARX submodelare fixed and treated as constants in one loop withthe NN trained by a back propagation (BP) algorithm(only a small number of epochs are implemented)then the resultant NN is fixed to estimate the param-eters of the ARX submodel in another loop The twoloops are executed alternatively to achieve a greatapproximation ability for nonlinear systems

(2) Parameter-classified identification scheme when thenonlinear basis function models are embedded in thecore-part of the quasi-ARXmodels all the parameterscan be classified as nonlinear (eg the center andwidth parameters in the embeddedRBFNs) and linear(eg the linear weights in the embedded RBFNs) tothe model A structured nonlinear parameter opti-mizationmethod (SNPOM) has been presented in [9]to optimize both the nonlinear and the linear param-eters simultaneously for a RBF-type state-dependentparameter model and improvement has been furthergiven in [19 22] On the other hand by using heuristicprior knowledge the authors in [14 23] estimate the

nonlinear parameters of a quasi-ARX NFN modeland the least square algorithm is used to estimate thelinear parameters Similarly a prior knowledge hasbeen used for nonlinear parameters in a quasi-ARXwavelet network (WN) model where identificationcan be explained in an integrated approach [24 25]

(3) Global identification scheme in this category all theparameters in the quasi-ARX models are optimizedregardless of the parameter features and model struc-ture For instance a hybrid algorithm of particleswarm optimization (PSO) with diversity learningand gradient descent method has been proposedin [10] to identify the WN-type quasi-ARX modelwhich is always used in time series prediction More-over NN [26] and support vector regression (SVR)[13] are applied respectively to identify all the quasi-ARX model parameters

In this paper specific efforts are made to extend thesecond identification scheme based on classifying the modelparameters Compared with the other schemes this oneexplores the model properties deeply and provides a promis-ing solution to a wide range of basis function embeddedquasi-ARX models It is known that SNPOM is an efficientoptimization method fallen into this category which makesgood use of the model parameters feature and gives impres-sive performance in time series prediction and nonlinearcontrol However this technique is still considered as a ldquonon-transparentrdquo approach since it is aimed at data-fitting onlyand model parameters are difficult to be interpreted alongwith physical explanation of real world or nonlinear dynam-ics of systems [7] Therefore it may constrain further devel-opment of the model In contrast a prior knowledge basednonlinear parameter estimation makes sense to interpretsystem properties meaningfully especially with respect to the

Complexity 3

quasi-ARX RBFN model as discussed later in Section 3 Theuseful prior knowledge can evolve a quasi-ARX model froma ldquoblack-boxrdquo tool into a ldquosemianalyticalrdquo one [27] whichmakes some parameters interpretable by our intuition justfollowing the principle of application favorable in quasi-ARXmodeling Owing to this fact nonlinear parameters are deter-mined in terms of prior interpretable knowledge and linearparameters are adjusted to fit the data It may contribute tolow computational cost and high generalization of the modelas parallel computation Nevertheless the problem is how togenerate useful prior knowledge for an accurate nonlinearparameter estimation

In the current study a two-step approach is proposedto identify the quasi-ARX RBFN model for the nonlinearsystems Firstly a clusteringmethod is applied to generate thedata distribution information for the system whereby centerparameters of the embedded RBFN are determined as clustercenters and thewidth parameter of eachRBF is set in terms ofdistance fromother nearby centersThen it is straightforwardto utilize the linear SVR for linear parameter estimation Themain purpose of this work is to provide an interpretable iden-tification approach for the quasi-ARX models which can beregarded as complementary to the identification procedures[6 9 13] Compared with the heuristic prior knowledge usedin quasi-ARXNFNmodel identification the clustering basedmethod gives an alternative approach to prior knowledgefor nonlinear parameter estimation and the quasi-ARXRBFN model is interpreted as a local linear model withinterpolationMoreover when linear SVR is applied for linearparameter estimation identification of the quasi-ARX RBFNmodel can be treated as an SVR with novel kernel mappingand associated feature space and the kernelmapping is equiv-alent to the nonlinear parameter estimation procedure whichis transformed from a nonlinear-in-nature model to thelinear-in-parameter one Unlike the SVR-based method [9]the kernel function proposed in this study takes an explicitmapping which is effective in coping with potential overfit-ting for some complex and noisy learning tasks [28] Finallyin the proposed method nonlinear parameters are estimateddirectly based on the prior knowledge to some extent it canbe considered as an algorithmic approach for initialization ofSNPOM

The remainder of the paper is organized as followsSection 2 introduces a quasi-ARX RBFN modeling schemeSection 3 proposes the identification method of the quasi-ARX RBFN model Section 4 investigates two numericalexamples and a real case Finally some discussions andconclusions are made in Section 5

2 Quasi-ARX RBFN Modeling

Let us consider a single-input-single-output (SISO) nonlin-ear time-invariant system whose input-output dynamics isdescribed as

119910 (119905) = 119892 (120593 (119905)) + 119890 (119905) (1)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 119899119910) 119906(119905 minus 1) 119906(119905 minus 119899119906)]119879119906(119905) isin R 119910(119905) isin R and 119890(119905) isin R are the system input output

and a stochastic noise of zero-mean at time 119905 respectively 119899119906and 119899119910 are the unknown maximum delays of the input andoutput respectively 120593(119905) isin R119899 with 119899 = 119899119910 + 119899119906 is the regres-sion vector composed of the delayed input-output data 119892(sdot)is an unknown function (black-box) describing the dynamicsof system under study which is assumed to be continuouslydifferentiable and satisfies 119892(0) = 0

Performing the Taylor expansion to 119892(120593(119905)) at 120593(119905) = 0one has

119910 (119905) = 119890 (119905) + 119892 (0) + (1198921015840 (0))119879 120593 (119905)+ 12120593119879 (119905) 11989210158401015840 (0) 120593 (119905) + sdot sdot sdot

(2)

Then (1) is reformalized with an ARX-like linear structure

119910 (119905) = 120593119879 (119905) 120579 (120593 (119905)) + 119890 (119905) (3)

where

120579 (120593 (119905)) = 1198921015840 (0) + 1211989210158401015840 (0) 120593 (119905) + sdot sdot sdot= [1198861119905 sdot sdot sdot 119886119899119910 119905 1198870119905 sdot sdot sdot 119887119899119906minus1119905]119879

(4)

In (4) coefficients 119886119894119905 = 119886119894(120593(119905)) and 119887119895119905 = 119887119895(120593(119905)) arenonlinear functions of 120593(119905) for 119894 = 1 2 119899119910 and 119895 =0 1 119899119906 minus 1 thus it can be represented by RBFN as

120579 (120593 (119905)) = Ω0 + 119872sum119895=1

Ω119895N (119901119895 120593 (119905)) (5)

where 119901119895 includes the center parameter vector 120583119895 and thewidth parameter120590119895 of the 119895th RBFN(119901119895 120593(119905))119872 denotes thenumber of basis functions utilized and Ω119895 = [1205961119895 120596119899119895]119879is a connection matrix between the input variables andthe associated basis functions According to (3) and (5) acompact representation of quasi-ARX RBFN model is givenas

119910 (119905) = 119872sum119895=0

120593119879 (119905) Ω119895N (119901119895 120593 (119905)) + 119890 (119905) (6)

in which the set of RBFs with scaling parameter 120582 (the defaultvalue of 120582 is 1) is

N (119901119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205821205902119895 ) 119895 = 01 119895 = 0

(7)

3 Parameter Estimation ofQuasi-ARX RBFN Model

From (6) and (7) it is known that 119901119895 (ie 120583119895 120590119895) for 119895 =1 119872 and are 119872 nonlinear parameters for the modelwhereas Ω119895 (119895 = 0 119872) become linear when all thenonlinear parameters are determinedfixed In the followingthe clustering method and SVR are respectively applied toestimate those two types of parameters

4 Complexity

y(t)

y(t) = T(t)((t))

yj(t) = T(t)Ωj

(t)

(t)

(pj (t)) =

1 2 3

(p j

(t)) exp(minus

(t) minus j2

2j

)

= T(t)3sum

j=1

Ωj(pj (t))(pj (t))=

3sumj=1

yj

Figure 2 A local linear interpretation for the quasi-ARX RBFNmodel A one-dimensional nonlinear system is approximated bythree linear submodels whose operating areas are decided by theassociated RBFs meanwhile the RBFs also provide interpolationsor weighs for all the linear submodels dependent on the operatingpoints A main interpolation is obtained from a linear submodelwhen the operating point is near to the center of the associated RBFwhile only a minor one can be received when the operating point isfar from the corresponding RBF

31 Nonlinear Parameters Estimation The choice of thecenter parameters plays an important role in performance ofthe RBF-type model [29] In this paper these parameters areestimated by means of prior knowledge from the clusteringmethod rather than by minimizing the mean square ofthe training error It should be mentioned that using theclusteringmethod for initializing the center parameters is nota new idea in RBF-type models and sophisticated clusteringalgorithms have been proposed in [30 31] In the presentwork nonlinear parameters are estimated in a clusteringway which have meaningful interpretations From this pointof view (6) is investigated as a local linear model with 119872submodels 119910119895 = 120593119879(119905)Ω119895 (119895 = 1 119872) and the 119895thRBF N(119901119895 120593(119905)) is regarded as a time-varying interpolationfunction for associated linear submodel to preserve the localproperty Figure 2 gives a schematic diagram to illustrate thequasi-ARX RBFN model via a local linear mean

In this way the local linear information of the datacan be generated by means of clustering algorithm wherethe number of clusters (linear subspaces) is equivalent tothe number of RBF neurons and each cluster center is setas the center parameter of the associated RBF In order todetermine appropriately the operating area of each locallinear submodel width of each RBF is set to well coverthe corresponding subspace Generally speaking we can setthe width parameters of the RBF neurons according to thedistances among those centers For instance a proper width

parameter 120590119895 of certain RBF can be obtained as a mean valueof distances from its center 120583119895 to its nearest two others From(7) one knows that an excessive small value of the widthparameters may result in insufficient local linear operatingareas for all data while a wide-shape setting will make all theRBFs overlapped and hence the local property of each linearsubmodel is weakened

Remark 1 Figure 2 only gives a meaningful interpretation ofthe model parameters In real applications since the data dis-tribution is complex and the exact local linear subspaces maynot exist the clustering partition approach is used to provideseveral rational operating areas and the scaling parameter120582 can be set to adjust the width parameters for goodweighting to each associated area

32 Linear Parameters Estimation After estimating and fix-ing the nonlinear parameters (6) can be rewritten in a linear-in-parameter manner as

119910 (119905) = Φ119879 (119905) Θ + 119890 (119905) (8)

where Φ(119905) is an abbreviation ofΦ(120593(119905)) withΦ (119905) = [120593119879 (119905) N1 (119905) 120593119879 (119905) N119872 (119905) 120593119879 (119905)]119879 (9)

Θ = [Ω1198790 Ω1198791 Ω119879119872]119879 (10)

in which since 119901119895 in the 119895th RBF has already been estimatedwe represent the 119895th RBF N(119901119895 120593(119905)) by a shorten form asN119895(119905) in (9) Therefore the nonlinear system identificationproblem is reduced to a linear regression one with respect toΦ(119905) and all the linear parameters are denoted by ΘRemark 2 As a result of nonlinear parameter estimationΦ(119905) plays an important role in transforming the quasi-ARX RBFN models from nonlinear-in-nature to linear-in-parameter with respect to Θ Accordingly it also transformsthe nonlinear mapping from the original input space of 119892(sdot)into a high feature space that is 120593(119905) rarr Φ(119905) This explicitmapping will be utilized for an inner-product kernel in thelater part

In the following the linear parameters are estimated bya linear SVR considering the structural risk minimizationprincipal as

min J ≜ 12Θ119879Θ + 119862119873sum119905=1

(120585119905 + 120585lowast119905 ) (11)

subject to

119910 (119905) minus Φ119879 (119905) Θ le 120598 + 120585119905minus 119910 (119905) + Φ119879 (119905) Θ le 120598 + 120585lowast119905 (12)

where119873 is the number of observations 120585119905 ge 0 and 120585lowast119905 ge 0 areslack variables 119862 is a nonnegative weight determining howmuch the prediction errors are penalized which exceeds the

Complexity 5

threshold value 120598 The solution can be transformed to find asaddle point of the associated Lagrange function

L (Θ 120585119905 120585lowast119905 120572119905 120572lowast119905 120573119905 120573lowast119905 )≜ 12Θ119879Θ + 119862

119873sum119905=1

(120585119905 + 120585lowast119905 )+ 119873sum119905=1

120572119905 (119910 (119905) minus Φ119879 (119905) Θ minus 120598 minus 120585119905)+ 119873sum119905=1

120572lowast119905 (minus119910 (119905) + Φ119879 (119905) Θ minus 120598 minus 120585lowast119905 )minus 119873sum119905=1

(120573119905120585119905 + 120573lowast119905 120585lowast119905 )

(13)

where 120572119905 120572lowast119905 120573119905 and 120573lowast119905 are nonnegative parameters tobe designed later The saddle point could be acquired byminimizingL with respect to Θ 120585lowast119905 and 120585119905

120597L120597Θ = 0 997904rArr Θ = 119873sum119905=1

(120572119905 minus 120572lowast119905 )Φ (119905) (14a)

120597L120597120585lowast119905 = 0 997904rArr 120573lowast119905 = 119862 minus 120572lowast119905 (14b)

120597L120597120585119905 = 0 997904rArr 120573119905 = 119862 minus 120572119905 (14c)

Thus one can convert the primal problem (11) into anequivalent dual problem as

max W (120572119905 120572lowast119905 )≜ minus12

119873sum119905119896=1

(120572119905 minus 120572lowast119905 ) (120572119896 minus 120572lowast119896 )Φ119879 (119905) Φ (119896)+ 119873sum119905=1

(120572119905 minus 120572lowast119905 ) 119910 (119905) minus 120598 119873sum119905=1

(120572119905 + 120572lowast119905 )(15)

subject to

119873sum119905=1

(120572119905 minus 120572lowast119905 ) = 0 120572119905 120572lowast119905 isin [0 119862] (16)

To do this the training results 119905 and lowast119905 are obtained from(15) and the linear parameter vector Θ is then obtained bythe training value

Θ = 119873sum119905=1

(119905 minus lowast119905 )Φ (119905) (17)

In the above way contributions of the SVR-based linearparameter estimation method can be concluded as follows

(1) The robust performance for parameter estimation isintroduced because of the structural risk minimiza-tion of SVR

(2) There is no need to calculate the linear parameterΘ directly Instead it becomes a dual form of thequadratic optimization which is represented by uti-lizing 120572119905 and 120572lowast119905 depending on the size of the trainingdata It is very useful to alleviate the computationalcost especially when themodel suffers from the curse-of-dimensionality

(3) Identification of quasi-ARX model is specified as anSVR with explicit kernel mapping Φ(119905) which hasbeen mentioned in Remark 2 To this end the quasi-ARX RBFN model is reformalized as

119910 (119905) = Φ119879 (119905) 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)Φ (1199051015840)= 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)K (119905 1199051015840) (18)

where 1199051015840 is time of training data and a quasi-linear kernelwhich is explicitly explained in the following remark isdefined as an inner product of the explicit nonlinearmappingΦ(119905)

K (119905 1199051015840) = Φ119879 (119905) Φ (1199051015840)= 120593119879 (119905) 120593 (1199051015840) 119872sum

119894=0

N119894 (119905)N119894 (1199051015840) (19)

Remark 3 The quasi-linear kernel name is twofold Firstly itis derived from the quasi-ARX modeling scheme Secondlyfrom (19) it is known that when 119872 is as small as zero thekernel is reduced to a linear one and nonlinearity of thekernel mapping is improved when increasing the value of119872Comparedwith conventional kernels andwith implicit kernelmapping the nonlinear mapping of the quasi-linear kernelis turnable by119872 which also reflects the nonlinearity of thequasi-ARX RBFNmodels in the sense of the number of locallinear subspaces utilized A proper value of119872 is essentiallyhelpful to cope with the potential overfitting which will beshown in the following simulations

4 Experimental Studies

In this section identification performance of the aboveproposed approach to quasi-ARX RBFN model is evaluatedby three examples The first one is an example to show theperformance of quasi-ARX RBFN model for time series pre-diction Second a rational system generated from Narendraand Parthasarathy [17] is simulated with a small amount oftraining data which is used to demonstrate the generalizationof the proposed quasi-linear kernel At last an examplemodeling a hydraulic robot actuator is carried out for ageneral comparison

In the nonlinear parameter estimation procedure affinitypropagation (AP) clustering algorithm [32] is utilized topartition the input space and automatically generate the sizeof clusters in terms of data distribution where Euclidean

6 Complexity

distance is evaluated as the similarity between exemplarsThen centers of all clusters are selected as the RBF centerparameters in the quasi-ARXmodel and thewidth parameterof a certain RBF is decided as the mean value of distancesfrom the associated center to the nearest two others For thelinear parameter estimation LibSVM toolbox [33] is appliedwhere ]-SVR is used with default ] setting by Matlab 76Finally the model performance is evaluated by root meansquare error (RMSE) as

RMSE = radicsum119905 (119910 (119905) minus 119910 (119905))2119870 (20)

where 119910(119905) is the prediction value of the system output 119910(119905)and119870 is the number of regression vectors

41 Modeling the Mackey-Glass Time Series The time seriesprediction on the chaotic Mackey-Glass differential equationis one of the most famous benchmarks for comparing thelearning and generalization abilities of different models Thistime series is generated from the following equation

119889119909 (119905)119889119905 = 119886119909 (119905 minus 120591)1 + 11990910 (119905 minus 120591) minus 119887119909 (119905) (21)

where 119886 = 02 119887 = 01 and 120591 = 17 which are the most oftenused values in the previous research and the equation doesshow chaotic behavior with them To make the comparisonsfair with the earlier works we will predict 119909(119905 + 6) usingthe input variables 119909(119905) 119909(119905 minus 6) 119909(119905 minus 12) and 119909(119905 minus 18)Two thousand data points are generatedwith initial conditiontaken as 119909(119905) equiv 12 for 119905 isin [minus17 0] based on the fourth-orderRungendashKutta method with time step Δ119905 = 01 Then onethousand input-output data pairs are selected from 119905 = 201to 119905 = 1200 which is shown in Figure 3 The first 500 datapairs are used as training data while the remaining 500 areused to predict 119909(119905 + 6) followed by

119909 (119905 + 6) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (22)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (23)

where 120593(119905) = [119909(119905 minus 18) 119909(119905 minus 12) 119909(119905 minus 6) 119909(119905)]119879The prediction of the Mackey-Glass time series using

a quasi-ARX RBFN model starts where 20 clusters areobtained from the AP clustering algorithm and thus 20RBFneurons are correspondingly constructed Thereafter SVR isused for linear parameter estimation in which the super-parameter 119862 is set as 100 The predicted result is comparedwith the original time series of test data in Figure 4 whichgives a RMSE of 00091

In Figure 4 the predicted result fits the original datavery well however it is still not as good as the resultsfrom some famous modelsmethods listed in Table 1 Since

0 100 200 300 400 500 600 700 800 900 100004

06

08

1

12

14

t

x(t)

Figure 3 Time series generated from the Mackey-Glass equation

0 50 100 150 200 250 300 350 400 450

002040608

11214

Original dataPredicted dataError

t

x(t)

Figure 4 Prediction result with the quasi-ARX RBFN model

no disturbance is contained in this example it is foundthat the prediction performance can be easily improved byminimizing the training prediction error In the comparisonlist SNPOM for RBF-AR model hybrid learning methodfor local linear wavelet neural network (LLWNN) andgenetic algorithm (GA) for RBFN are all optimization-basedidentification methods and it is relatively easy for them toachieve small RMSEs of the prediction by iterative trainingHowever these methods are much more time-costing incomparison with only 6 seconds by the proposed methodfor the quasi-ARX RBFN model In addition although the119896-means clustering method for RBFN is implemented in adeterministic way and shows efficient result the number ofRBF neurons used is as big as 238 In fact a small predictionRMSE obtained from these methods does not mean goodidentification of the models since overtraining may happensome times

In the present example we confirm the effectiveness ofthe optimization-based method given above and propose ahybrid approach for identification of the quasi-ARX RBFNmodel where prediction result from the proposed methodcan be further improved by SNPOM (the function ldquolsqnon-linrdquo in the Matlab Optimization Toolbox is used [9]) It isseen that the prediction RMSE can be improved to 21 times10minus3 by only 15 iterations of implementation in SNPOMand the result becomes compatible with others Howeversuch optimization is not always effective especially in model

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Complexity 3

quasi-ARX RBFN model as discussed later in Section 3 Theuseful prior knowledge can evolve a quasi-ARX model froma ldquoblack-boxrdquo tool into a ldquosemianalyticalrdquo one [27] whichmakes some parameters interpretable by our intuition justfollowing the principle of application favorable in quasi-ARXmodeling Owing to this fact nonlinear parameters are deter-mined in terms of prior interpretable knowledge and linearparameters are adjusted to fit the data It may contribute tolow computational cost and high generalization of the modelas parallel computation Nevertheless the problem is how togenerate useful prior knowledge for an accurate nonlinearparameter estimation

In the current study a two-step approach is proposedto identify the quasi-ARX RBFN model for the nonlinearsystems Firstly a clusteringmethod is applied to generate thedata distribution information for the system whereby centerparameters of the embedded RBFN are determined as clustercenters and thewidth parameter of eachRBF is set in terms ofdistance fromother nearby centersThen it is straightforwardto utilize the linear SVR for linear parameter estimation Themain purpose of this work is to provide an interpretable iden-tification approach for the quasi-ARX models which can beregarded as complementary to the identification procedures[6 9 13] Compared with the heuristic prior knowledge usedin quasi-ARXNFNmodel identification the clustering basedmethod gives an alternative approach to prior knowledgefor nonlinear parameter estimation and the quasi-ARXRBFN model is interpreted as a local linear model withinterpolationMoreover when linear SVR is applied for linearparameter estimation identification of the quasi-ARX RBFNmodel can be treated as an SVR with novel kernel mappingand associated feature space and the kernelmapping is equiv-alent to the nonlinear parameter estimation procedure whichis transformed from a nonlinear-in-nature model to thelinear-in-parameter one Unlike the SVR-based method [9]the kernel function proposed in this study takes an explicitmapping which is effective in coping with potential overfit-ting for some complex and noisy learning tasks [28] Finallyin the proposed method nonlinear parameters are estimateddirectly based on the prior knowledge to some extent it canbe considered as an algorithmic approach for initialization ofSNPOM

The remainder of the paper is organized as followsSection 2 introduces a quasi-ARX RBFN modeling schemeSection 3 proposes the identification method of the quasi-ARX RBFN model Section 4 investigates two numericalexamples and a real case Finally some discussions andconclusions are made in Section 5

2 Quasi-ARX RBFN Modeling

Let us consider a single-input-single-output (SISO) nonlin-ear time-invariant system whose input-output dynamics isdescribed as

119910 (119905) = 119892 (120593 (119905)) + 119890 (119905) (1)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 119899119910) 119906(119905 minus 1) 119906(119905 minus 119899119906)]119879119906(119905) isin R 119910(119905) isin R and 119890(119905) isin R are the system input output

and a stochastic noise of zero-mean at time 119905 respectively 119899119906and 119899119910 are the unknown maximum delays of the input andoutput respectively 120593(119905) isin R119899 with 119899 = 119899119910 + 119899119906 is the regres-sion vector composed of the delayed input-output data 119892(sdot)is an unknown function (black-box) describing the dynamicsof system under study which is assumed to be continuouslydifferentiable and satisfies 119892(0) = 0

Performing the Taylor expansion to 119892(120593(119905)) at 120593(119905) = 0one has

119910 (119905) = 119890 (119905) + 119892 (0) + (1198921015840 (0))119879 120593 (119905)+ 12120593119879 (119905) 11989210158401015840 (0) 120593 (119905) + sdot sdot sdot

(2)

Then (1) is reformalized with an ARX-like linear structure

119910 (119905) = 120593119879 (119905) 120579 (120593 (119905)) + 119890 (119905) (3)

where

120579 (120593 (119905)) = 1198921015840 (0) + 1211989210158401015840 (0) 120593 (119905) + sdot sdot sdot= [1198861119905 sdot sdot sdot 119886119899119910 119905 1198870119905 sdot sdot sdot 119887119899119906minus1119905]119879

(4)

In (4) coefficients 119886119894119905 = 119886119894(120593(119905)) and 119887119895119905 = 119887119895(120593(119905)) arenonlinear functions of 120593(119905) for 119894 = 1 2 119899119910 and 119895 =0 1 119899119906 minus 1 thus it can be represented by RBFN as

120579 (120593 (119905)) = Ω0 + 119872sum119895=1

Ω119895N (119901119895 120593 (119905)) (5)

where 119901119895 includes the center parameter vector 120583119895 and thewidth parameter120590119895 of the 119895th RBFN(119901119895 120593(119905))119872 denotes thenumber of basis functions utilized and Ω119895 = [1205961119895 120596119899119895]119879is a connection matrix between the input variables andthe associated basis functions According to (3) and (5) acompact representation of quasi-ARX RBFN model is givenas

119910 (119905) = 119872sum119895=0

120593119879 (119905) Ω119895N (119901119895 120593 (119905)) + 119890 (119905) (6)

in which the set of RBFs with scaling parameter 120582 (the defaultvalue of 120582 is 1) is

N (119901119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205821205902119895 ) 119895 = 01 119895 = 0

(7)

3 Parameter Estimation ofQuasi-ARX RBFN Model

From (6) and (7) it is known that 119901119895 (ie 120583119895 120590119895) for 119895 =1 119872 and are 119872 nonlinear parameters for the modelwhereas Ω119895 (119895 = 0 119872) become linear when all thenonlinear parameters are determinedfixed In the followingthe clustering method and SVR are respectively applied toestimate those two types of parameters

4 Complexity

y(t)

y(t) = T(t)((t))

yj(t) = T(t)Ωj

(t)

(t)

(pj (t)) =

1 2 3

(p j

(t)) exp(minus

(t) minus j2

2j

)

= T(t)3sum

j=1

Ωj(pj (t))(pj (t))=

3sumj=1

yj

Figure 2 A local linear interpretation for the quasi-ARX RBFNmodel A one-dimensional nonlinear system is approximated bythree linear submodels whose operating areas are decided by theassociated RBFs meanwhile the RBFs also provide interpolationsor weighs for all the linear submodels dependent on the operatingpoints A main interpolation is obtained from a linear submodelwhen the operating point is near to the center of the associated RBFwhile only a minor one can be received when the operating point isfar from the corresponding RBF

31 Nonlinear Parameters Estimation The choice of thecenter parameters plays an important role in performance ofthe RBF-type model [29] In this paper these parameters areestimated by means of prior knowledge from the clusteringmethod rather than by minimizing the mean square ofthe training error It should be mentioned that using theclusteringmethod for initializing the center parameters is nota new idea in RBF-type models and sophisticated clusteringalgorithms have been proposed in [30 31] In the presentwork nonlinear parameters are estimated in a clusteringway which have meaningful interpretations From this pointof view (6) is investigated as a local linear model with 119872submodels 119910119895 = 120593119879(119905)Ω119895 (119895 = 1 119872) and the 119895thRBF N(119901119895 120593(119905)) is regarded as a time-varying interpolationfunction for associated linear submodel to preserve the localproperty Figure 2 gives a schematic diagram to illustrate thequasi-ARX RBFN model via a local linear mean

In this way the local linear information of the datacan be generated by means of clustering algorithm wherethe number of clusters (linear subspaces) is equivalent tothe number of RBF neurons and each cluster center is setas the center parameter of the associated RBF In order todetermine appropriately the operating area of each locallinear submodel width of each RBF is set to well coverthe corresponding subspace Generally speaking we can setthe width parameters of the RBF neurons according to thedistances among those centers For instance a proper width

parameter 120590119895 of certain RBF can be obtained as a mean valueof distances from its center 120583119895 to its nearest two others From(7) one knows that an excessive small value of the widthparameters may result in insufficient local linear operatingareas for all data while a wide-shape setting will make all theRBFs overlapped and hence the local property of each linearsubmodel is weakened

Remark 1 Figure 2 only gives a meaningful interpretation ofthe model parameters In real applications since the data dis-tribution is complex and the exact local linear subspaces maynot exist the clustering partition approach is used to provideseveral rational operating areas and the scaling parameter120582 can be set to adjust the width parameters for goodweighting to each associated area

32 Linear Parameters Estimation After estimating and fix-ing the nonlinear parameters (6) can be rewritten in a linear-in-parameter manner as

119910 (119905) = Φ119879 (119905) Θ + 119890 (119905) (8)

where Φ(119905) is an abbreviation ofΦ(120593(119905)) withΦ (119905) = [120593119879 (119905) N1 (119905) 120593119879 (119905) N119872 (119905) 120593119879 (119905)]119879 (9)

Θ = [Ω1198790 Ω1198791 Ω119879119872]119879 (10)

in which since 119901119895 in the 119895th RBF has already been estimatedwe represent the 119895th RBF N(119901119895 120593(119905)) by a shorten form asN119895(119905) in (9) Therefore the nonlinear system identificationproblem is reduced to a linear regression one with respect toΦ(119905) and all the linear parameters are denoted by ΘRemark 2 As a result of nonlinear parameter estimationΦ(119905) plays an important role in transforming the quasi-ARX RBFN models from nonlinear-in-nature to linear-in-parameter with respect to Θ Accordingly it also transformsthe nonlinear mapping from the original input space of 119892(sdot)into a high feature space that is 120593(119905) rarr Φ(119905) This explicitmapping will be utilized for an inner-product kernel in thelater part

In the following the linear parameters are estimated bya linear SVR considering the structural risk minimizationprincipal as

min J ≜ 12Θ119879Θ + 119862119873sum119905=1

(120585119905 + 120585lowast119905 ) (11)

subject to

119910 (119905) minus Φ119879 (119905) Θ le 120598 + 120585119905minus 119910 (119905) + Φ119879 (119905) Θ le 120598 + 120585lowast119905 (12)

where119873 is the number of observations 120585119905 ge 0 and 120585lowast119905 ge 0 areslack variables 119862 is a nonnegative weight determining howmuch the prediction errors are penalized which exceeds the

Complexity 5

threshold value 120598 The solution can be transformed to find asaddle point of the associated Lagrange function

L (Θ 120585119905 120585lowast119905 120572119905 120572lowast119905 120573119905 120573lowast119905 )≜ 12Θ119879Θ + 119862

119873sum119905=1

(120585119905 + 120585lowast119905 )+ 119873sum119905=1

120572119905 (119910 (119905) minus Φ119879 (119905) Θ minus 120598 minus 120585119905)+ 119873sum119905=1

120572lowast119905 (minus119910 (119905) + Φ119879 (119905) Θ minus 120598 minus 120585lowast119905 )minus 119873sum119905=1

(120573119905120585119905 + 120573lowast119905 120585lowast119905 )

(13)

where 120572119905 120572lowast119905 120573119905 and 120573lowast119905 are nonnegative parameters tobe designed later The saddle point could be acquired byminimizingL with respect to Θ 120585lowast119905 and 120585119905

120597L120597Θ = 0 997904rArr Θ = 119873sum119905=1

(120572119905 minus 120572lowast119905 )Φ (119905) (14a)

120597L120597120585lowast119905 = 0 997904rArr 120573lowast119905 = 119862 minus 120572lowast119905 (14b)

120597L120597120585119905 = 0 997904rArr 120573119905 = 119862 minus 120572119905 (14c)

Thus one can convert the primal problem (11) into anequivalent dual problem as

max W (120572119905 120572lowast119905 )≜ minus12

119873sum119905119896=1

(120572119905 minus 120572lowast119905 ) (120572119896 minus 120572lowast119896 )Φ119879 (119905) Φ (119896)+ 119873sum119905=1

(120572119905 minus 120572lowast119905 ) 119910 (119905) minus 120598 119873sum119905=1

(120572119905 + 120572lowast119905 )(15)

subject to

119873sum119905=1

(120572119905 minus 120572lowast119905 ) = 0 120572119905 120572lowast119905 isin [0 119862] (16)

To do this the training results 119905 and lowast119905 are obtained from(15) and the linear parameter vector Θ is then obtained bythe training value

Θ = 119873sum119905=1

(119905 minus lowast119905 )Φ (119905) (17)

In the above way contributions of the SVR-based linearparameter estimation method can be concluded as follows

(1) The robust performance for parameter estimation isintroduced because of the structural risk minimiza-tion of SVR

(2) There is no need to calculate the linear parameterΘ directly Instead it becomes a dual form of thequadratic optimization which is represented by uti-lizing 120572119905 and 120572lowast119905 depending on the size of the trainingdata It is very useful to alleviate the computationalcost especially when themodel suffers from the curse-of-dimensionality

(3) Identification of quasi-ARX model is specified as anSVR with explicit kernel mapping Φ(119905) which hasbeen mentioned in Remark 2 To this end the quasi-ARX RBFN model is reformalized as

119910 (119905) = Φ119879 (119905) 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)Φ (1199051015840)= 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)K (119905 1199051015840) (18)

where 1199051015840 is time of training data and a quasi-linear kernelwhich is explicitly explained in the following remark isdefined as an inner product of the explicit nonlinearmappingΦ(119905)

K (119905 1199051015840) = Φ119879 (119905) Φ (1199051015840)= 120593119879 (119905) 120593 (1199051015840) 119872sum

119894=0

N119894 (119905)N119894 (1199051015840) (19)

Remark 3 The quasi-linear kernel name is twofold Firstly itis derived from the quasi-ARX modeling scheme Secondlyfrom (19) it is known that when 119872 is as small as zero thekernel is reduced to a linear one and nonlinearity of thekernel mapping is improved when increasing the value of119872Comparedwith conventional kernels andwith implicit kernelmapping the nonlinear mapping of the quasi-linear kernelis turnable by119872 which also reflects the nonlinearity of thequasi-ARX RBFNmodels in the sense of the number of locallinear subspaces utilized A proper value of119872 is essentiallyhelpful to cope with the potential overfitting which will beshown in the following simulations

4 Experimental Studies

In this section identification performance of the aboveproposed approach to quasi-ARX RBFN model is evaluatedby three examples The first one is an example to show theperformance of quasi-ARX RBFN model for time series pre-diction Second a rational system generated from Narendraand Parthasarathy [17] is simulated with a small amount oftraining data which is used to demonstrate the generalizationof the proposed quasi-linear kernel At last an examplemodeling a hydraulic robot actuator is carried out for ageneral comparison

In the nonlinear parameter estimation procedure affinitypropagation (AP) clustering algorithm [32] is utilized topartition the input space and automatically generate the sizeof clusters in terms of data distribution where Euclidean

6 Complexity

distance is evaluated as the similarity between exemplarsThen centers of all clusters are selected as the RBF centerparameters in the quasi-ARXmodel and thewidth parameterof a certain RBF is decided as the mean value of distancesfrom the associated center to the nearest two others For thelinear parameter estimation LibSVM toolbox [33] is appliedwhere ]-SVR is used with default ] setting by Matlab 76Finally the model performance is evaluated by root meansquare error (RMSE) as

RMSE = radicsum119905 (119910 (119905) minus 119910 (119905))2119870 (20)

where 119910(119905) is the prediction value of the system output 119910(119905)and119870 is the number of regression vectors

41 Modeling the Mackey-Glass Time Series The time seriesprediction on the chaotic Mackey-Glass differential equationis one of the most famous benchmarks for comparing thelearning and generalization abilities of different models Thistime series is generated from the following equation

119889119909 (119905)119889119905 = 119886119909 (119905 minus 120591)1 + 11990910 (119905 minus 120591) minus 119887119909 (119905) (21)

where 119886 = 02 119887 = 01 and 120591 = 17 which are the most oftenused values in the previous research and the equation doesshow chaotic behavior with them To make the comparisonsfair with the earlier works we will predict 119909(119905 + 6) usingthe input variables 119909(119905) 119909(119905 minus 6) 119909(119905 minus 12) and 119909(119905 minus 18)Two thousand data points are generatedwith initial conditiontaken as 119909(119905) equiv 12 for 119905 isin [minus17 0] based on the fourth-orderRungendashKutta method with time step Δ119905 = 01 Then onethousand input-output data pairs are selected from 119905 = 201to 119905 = 1200 which is shown in Figure 3 The first 500 datapairs are used as training data while the remaining 500 areused to predict 119909(119905 + 6) followed by

119909 (119905 + 6) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (22)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (23)

where 120593(119905) = [119909(119905 minus 18) 119909(119905 minus 12) 119909(119905 minus 6) 119909(119905)]119879The prediction of the Mackey-Glass time series using

a quasi-ARX RBFN model starts where 20 clusters areobtained from the AP clustering algorithm and thus 20RBFneurons are correspondingly constructed Thereafter SVR isused for linear parameter estimation in which the super-parameter 119862 is set as 100 The predicted result is comparedwith the original time series of test data in Figure 4 whichgives a RMSE of 00091

In Figure 4 the predicted result fits the original datavery well however it is still not as good as the resultsfrom some famous modelsmethods listed in Table 1 Since

0 100 200 300 400 500 600 700 800 900 100004

06

08

1

12

14

t

x(t)

Figure 3 Time series generated from the Mackey-Glass equation

0 50 100 150 200 250 300 350 400 450

002040608

11214

Original dataPredicted dataError

t

x(t)

Figure 4 Prediction result with the quasi-ARX RBFN model

no disturbance is contained in this example it is foundthat the prediction performance can be easily improved byminimizing the training prediction error In the comparisonlist SNPOM for RBF-AR model hybrid learning methodfor local linear wavelet neural network (LLWNN) andgenetic algorithm (GA) for RBFN are all optimization-basedidentification methods and it is relatively easy for them toachieve small RMSEs of the prediction by iterative trainingHowever these methods are much more time-costing incomparison with only 6 seconds by the proposed methodfor the quasi-ARX RBFN model In addition although the119896-means clustering method for RBFN is implemented in adeterministic way and shows efficient result the number ofRBF neurons used is as big as 238 In fact a small predictionRMSE obtained from these methods does not mean goodidentification of the models since overtraining may happensome times

In the present example we confirm the effectiveness ofthe optimization-based method given above and propose ahybrid approach for identification of the quasi-ARX RBFNmodel where prediction result from the proposed methodcan be further improved by SNPOM (the function ldquolsqnon-linrdquo in the Matlab Optimization Toolbox is used [9]) It isseen that the prediction RMSE can be improved to 21 times10minus3 by only 15 iterations of implementation in SNPOMand the result becomes compatible with others Howeversuch optimization is not always effective especially in model

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

4 Complexity

y(t)

y(t) = T(t)((t))

yj(t) = T(t)Ωj

(t)

(t)

(pj (t)) =

1 2 3

(p j

(t)) exp(minus

(t) minus j2

2j

)

= T(t)3sum

j=1

Ωj(pj (t))(pj (t))=

3sumj=1

yj

Figure 2 A local linear interpretation for the quasi-ARX RBFNmodel A one-dimensional nonlinear system is approximated bythree linear submodels whose operating areas are decided by theassociated RBFs meanwhile the RBFs also provide interpolationsor weighs for all the linear submodels dependent on the operatingpoints A main interpolation is obtained from a linear submodelwhen the operating point is near to the center of the associated RBFwhile only a minor one can be received when the operating point isfar from the corresponding RBF

31 Nonlinear Parameters Estimation The choice of thecenter parameters plays an important role in performance ofthe RBF-type model [29] In this paper these parameters areestimated by means of prior knowledge from the clusteringmethod rather than by minimizing the mean square ofthe training error It should be mentioned that using theclusteringmethod for initializing the center parameters is nota new idea in RBF-type models and sophisticated clusteringalgorithms have been proposed in [30 31] In the presentwork nonlinear parameters are estimated in a clusteringway which have meaningful interpretations From this pointof view (6) is investigated as a local linear model with 119872submodels 119910119895 = 120593119879(119905)Ω119895 (119895 = 1 119872) and the 119895thRBF N(119901119895 120593(119905)) is regarded as a time-varying interpolationfunction for associated linear submodel to preserve the localproperty Figure 2 gives a schematic diagram to illustrate thequasi-ARX RBFN model via a local linear mean

In this way the local linear information of the datacan be generated by means of clustering algorithm wherethe number of clusters (linear subspaces) is equivalent tothe number of RBF neurons and each cluster center is setas the center parameter of the associated RBF In order todetermine appropriately the operating area of each locallinear submodel width of each RBF is set to well coverthe corresponding subspace Generally speaking we can setthe width parameters of the RBF neurons according to thedistances among those centers For instance a proper width

parameter 120590119895 of certain RBF can be obtained as a mean valueof distances from its center 120583119895 to its nearest two others From(7) one knows that an excessive small value of the widthparameters may result in insufficient local linear operatingareas for all data while a wide-shape setting will make all theRBFs overlapped and hence the local property of each linearsubmodel is weakened

Remark 1 Figure 2 only gives a meaningful interpretation ofthe model parameters In real applications since the data dis-tribution is complex and the exact local linear subspaces maynot exist the clustering partition approach is used to provideseveral rational operating areas and the scaling parameter120582 can be set to adjust the width parameters for goodweighting to each associated area

32 Linear Parameters Estimation After estimating and fix-ing the nonlinear parameters (6) can be rewritten in a linear-in-parameter manner as

119910 (119905) = Φ119879 (119905) Θ + 119890 (119905) (8)

where Φ(119905) is an abbreviation ofΦ(120593(119905)) withΦ (119905) = [120593119879 (119905) N1 (119905) 120593119879 (119905) N119872 (119905) 120593119879 (119905)]119879 (9)

Θ = [Ω1198790 Ω1198791 Ω119879119872]119879 (10)

in which since 119901119895 in the 119895th RBF has already been estimatedwe represent the 119895th RBF N(119901119895 120593(119905)) by a shorten form asN119895(119905) in (9) Therefore the nonlinear system identificationproblem is reduced to a linear regression one with respect toΦ(119905) and all the linear parameters are denoted by ΘRemark 2 As a result of nonlinear parameter estimationΦ(119905) plays an important role in transforming the quasi-ARX RBFN models from nonlinear-in-nature to linear-in-parameter with respect to Θ Accordingly it also transformsthe nonlinear mapping from the original input space of 119892(sdot)into a high feature space that is 120593(119905) rarr Φ(119905) This explicitmapping will be utilized for an inner-product kernel in thelater part

In the following the linear parameters are estimated bya linear SVR considering the structural risk minimizationprincipal as

min J ≜ 12Θ119879Θ + 119862119873sum119905=1

(120585119905 + 120585lowast119905 ) (11)

subject to

119910 (119905) minus Φ119879 (119905) Θ le 120598 + 120585119905minus 119910 (119905) + Φ119879 (119905) Θ le 120598 + 120585lowast119905 (12)

where119873 is the number of observations 120585119905 ge 0 and 120585lowast119905 ge 0 areslack variables 119862 is a nonnegative weight determining howmuch the prediction errors are penalized which exceeds the

Complexity 5

threshold value 120598 The solution can be transformed to find asaddle point of the associated Lagrange function

L (Θ 120585119905 120585lowast119905 120572119905 120572lowast119905 120573119905 120573lowast119905 )≜ 12Θ119879Θ + 119862

119873sum119905=1

(120585119905 + 120585lowast119905 )+ 119873sum119905=1

120572119905 (119910 (119905) minus Φ119879 (119905) Θ minus 120598 minus 120585119905)+ 119873sum119905=1

120572lowast119905 (minus119910 (119905) + Φ119879 (119905) Θ minus 120598 minus 120585lowast119905 )minus 119873sum119905=1

(120573119905120585119905 + 120573lowast119905 120585lowast119905 )

(13)

where 120572119905 120572lowast119905 120573119905 and 120573lowast119905 are nonnegative parameters tobe designed later The saddle point could be acquired byminimizingL with respect to Θ 120585lowast119905 and 120585119905

120597L120597Θ = 0 997904rArr Θ = 119873sum119905=1

(120572119905 minus 120572lowast119905 )Φ (119905) (14a)

120597L120597120585lowast119905 = 0 997904rArr 120573lowast119905 = 119862 minus 120572lowast119905 (14b)

120597L120597120585119905 = 0 997904rArr 120573119905 = 119862 minus 120572119905 (14c)

Thus one can convert the primal problem (11) into anequivalent dual problem as

max W (120572119905 120572lowast119905 )≜ minus12

119873sum119905119896=1

(120572119905 minus 120572lowast119905 ) (120572119896 minus 120572lowast119896 )Φ119879 (119905) Φ (119896)+ 119873sum119905=1

(120572119905 minus 120572lowast119905 ) 119910 (119905) minus 120598 119873sum119905=1

(120572119905 + 120572lowast119905 )(15)

subject to

119873sum119905=1

(120572119905 minus 120572lowast119905 ) = 0 120572119905 120572lowast119905 isin [0 119862] (16)

To do this the training results 119905 and lowast119905 are obtained from(15) and the linear parameter vector Θ is then obtained bythe training value

Θ = 119873sum119905=1

(119905 minus lowast119905 )Φ (119905) (17)

In the above way contributions of the SVR-based linearparameter estimation method can be concluded as follows

(1) The robust performance for parameter estimation isintroduced because of the structural risk minimiza-tion of SVR

(2) There is no need to calculate the linear parameterΘ directly Instead it becomes a dual form of thequadratic optimization which is represented by uti-lizing 120572119905 and 120572lowast119905 depending on the size of the trainingdata It is very useful to alleviate the computationalcost especially when themodel suffers from the curse-of-dimensionality

(3) Identification of quasi-ARX model is specified as anSVR with explicit kernel mapping Φ(119905) which hasbeen mentioned in Remark 2 To this end the quasi-ARX RBFN model is reformalized as

119910 (119905) = Φ119879 (119905) 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)Φ (1199051015840)= 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)K (119905 1199051015840) (18)

where 1199051015840 is time of training data and a quasi-linear kernelwhich is explicitly explained in the following remark isdefined as an inner product of the explicit nonlinearmappingΦ(119905)

K (119905 1199051015840) = Φ119879 (119905) Φ (1199051015840)= 120593119879 (119905) 120593 (1199051015840) 119872sum

119894=0

N119894 (119905)N119894 (1199051015840) (19)

Remark 3 The quasi-linear kernel name is twofold Firstly itis derived from the quasi-ARX modeling scheme Secondlyfrom (19) it is known that when 119872 is as small as zero thekernel is reduced to a linear one and nonlinearity of thekernel mapping is improved when increasing the value of119872Comparedwith conventional kernels andwith implicit kernelmapping the nonlinear mapping of the quasi-linear kernelis turnable by119872 which also reflects the nonlinearity of thequasi-ARX RBFNmodels in the sense of the number of locallinear subspaces utilized A proper value of119872 is essentiallyhelpful to cope with the potential overfitting which will beshown in the following simulations

4 Experimental Studies

In this section identification performance of the aboveproposed approach to quasi-ARX RBFN model is evaluatedby three examples The first one is an example to show theperformance of quasi-ARX RBFN model for time series pre-diction Second a rational system generated from Narendraand Parthasarathy [17] is simulated with a small amount oftraining data which is used to demonstrate the generalizationof the proposed quasi-linear kernel At last an examplemodeling a hydraulic robot actuator is carried out for ageneral comparison

In the nonlinear parameter estimation procedure affinitypropagation (AP) clustering algorithm [32] is utilized topartition the input space and automatically generate the sizeof clusters in terms of data distribution where Euclidean

6 Complexity

distance is evaluated as the similarity between exemplarsThen centers of all clusters are selected as the RBF centerparameters in the quasi-ARXmodel and thewidth parameterof a certain RBF is decided as the mean value of distancesfrom the associated center to the nearest two others For thelinear parameter estimation LibSVM toolbox [33] is appliedwhere ]-SVR is used with default ] setting by Matlab 76Finally the model performance is evaluated by root meansquare error (RMSE) as

RMSE = radicsum119905 (119910 (119905) minus 119910 (119905))2119870 (20)

where 119910(119905) is the prediction value of the system output 119910(119905)and119870 is the number of regression vectors

41 Modeling the Mackey-Glass Time Series The time seriesprediction on the chaotic Mackey-Glass differential equationis one of the most famous benchmarks for comparing thelearning and generalization abilities of different models Thistime series is generated from the following equation

119889119909 (119905)119889119905 = 119886119909 (119905 minus 120591)1 + 11990910 (119905 minus 120591) minus 119887119909 (119905) (21)

where 119886 = 02 119887 = 01 and 120591 = 17 which are the most oftenused values in the previous research and the equation doesshow chaotic behavior with them To make the comparisonsfair with the earlier works we will predict 119909(119905 + 6) usingthe input variables 119909(119905) 119909(119905 minus 6) 119909(119905 minus 12) and 119909(119905 minus 18)Two thousand data points are generatedwith initial conditiontaken as 119909(119905) equiv 12 for 119905 isin [minus17 0] based on the fourth-orderRungendashKutta method with time step Δ119905 = 01 Then onethousand input-output data pairs are selected from 119905 = 201to 119905 = 1200 which is shown in Figure 3 The first 500 datapairs are used as training data while the remaining 500 areused to predict 119909(119905 + 6) followed by

119909 (119905 + 6) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (22)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (23)

where 120593(119905) = [119909(119905 minus 18) 119909(119905 minus 12) 119909(119905 minus 6) 119909(119905)]119879The prediction of the Mackey-Glass time series using

a quasi-ARX RBFN model starts where 20 clusters areobtained from the AP clustering algorithm and thus 20RBFneurons are correspondingly constructed Thereafter SVR isused for linear parameter estimation in which the super-parameter 119862 is set as 100 The predicted result is comparedwith the original time series of test data in Figure 4 whichgives a RMSE of 00091

In Figure 4 the predicted result fits the original datavery well however it is still not as good as the resultsfrom some famous modelsmethods listed in Table 1 Since

0 100 200 300 400 500 600 700 800 900 100004

06

08

1

12

14

t

x(t)

Figure 3 Time series generated from the Mackey-Glass equation

0 50 100 150 200 250 300 350 400 450

002040608

11214

Original dataPredicted dataError

t

x(t)

Figure 4 Prediction result with the quasi-ARX RBFN model

no disturbance is contained in this example it is foundthat the prediction performance can be easily improved byminimizing the training prediction error In the comparisonlist SNPOM for RBF-AR model hybrid learning methodfor local linear wavelet neural network (LLWNN) andgenetic algorithm (GA) for RBFN are all optimization-basedidentification methods and it is relatively easy for them toachieve small RMSEs of the prediction by iterative trainingHowever these methods are much more time-costing incomparison with only 6 seconds by the proposed methodfor the quasi-ARX RBFN model In addition although the119896-means clustering method for RBFN is implemented in adeterministic way and shows efficient result the number ofRBF neurons used is as big as 238 In fact a small predictionRMSE obtained from these methods does not mean goodidentification of the models since overtraining may happensome times

In the present example we confirm the effectiveness ofthe optimization-based method given above and propose ahybrid approach for identification of the quasi-ARX RBFNmodel where prediction result from the proposed methodcan be further improved by SNPOM (the function ldquolsqnon-linrdquo in the Matlab Optimization Toolbox is used [9]) It isseen that the prediction RMSE can be improved to 21 times10minus3 by only 15 iterations of implementation in SNPOMand the result becomes compatible with others Howeversuch optimization is not always effective especially in model

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Complexity 5

threshold value 120598 The solution can be transformed to find asaddle point of the associated Lagrange function

L (Θ 120585119905 120585lowast119905 120572119905 120572lowast119905 120573119905 120573lowast119905 )≜ 12Θ119879Θ + 119862

119873sum119905=1

(120585119905 + 120585lowast119905 )+ 119873sum119905=1

120572119905 (119910 (119905) minus Φ119879 (119905) Θ minus 120598 minus 120585119905)+ 119873sum119905=1

120572lowast119905 (minus119910 (119905) + Φ119879 (119905) Θ minus 120598 minus 120585lowast119905 )minus 119873sum119905=1

(120573119905120585119905 + 120573lowast119905 120585lowast119905 )

(13)

where 120572119905 120572lowast119905 120573119905 and 120573lowast119905 are nonnegative parameters tobe designed later The saddle point could be acquired byminimizingL with respect to Θ 120585lowast119905 and 120585119905

120597L120597Θ = 0 997904rArr Θ = 119873sum119905=1

(120572119905 minus 120572lowast119905 )Φ (119905) (14a)

120597L120597120585lowast119905 = 0 997904rArr 120573lowast119905 = 119862 minus 120572lowast119905 (14b)

120597L120597120585119905 = 0 997904rArr 120573119905 = 119862 minus 120572119905 (14c)

Thus one can convert the primal problem (11) into anequivalent dual problem as

max W (120572119905 120572lowast119905 )≜ minus12

119873sum119905119896=1

(120572119905 minus 120572lowast119905 ) (120572119896 minus 120572lowast119896 )Φ119879 (119905) Φ (119896)+ 119873sum119905=1

(120572119905 minus 120572lowast119905 ) 119910 (119905) minus 120598 119873sum119905=1

(120572119905 + 120572lowast119905 )(15)

subject to

119873sum119905=1

(120572119905 minus 120572lowast119905 ) = 0 120572119905 120572lowast119905 isin [0 119862] (16)

To do this the training results 119905 and lowast119905 are obtained from(15) and the linear parameter vector Θ is then obtained bythe training value

Θ = 119873sum119905=1

(119905 minus lowast119905 )Φ (119905) (17)

In the above way contributions of the SVR-based linearparameter estimation method can be concluded as follows

(1) The robust performance for parameter estimation isintroduced because of the structural risk minimiza-tion of SVR

(2) There is no need to calculate the linear parameterΘ directly Instead it becomes a dual form of thequadratic optimization which is represented by uti-lizing 120572119905 and 120572lowast119905 depending on the size of the trainingdata It is very useful to alleviate the computationalcost especially when themodel suffers from the curse-of-dimensionality

(3) Identification of quasi-ARX model is specified as anSVR with explicit kernel mapping Φ(119905) which hasbeen mentioned in Remark 2 To this end the quasi-ARX RBFN model is reformalized as

119910 (119905) = Φ119879 (119905) 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)Φ (1199051015840)= 119873sum1199051015840=1

(1199051015840 minus lowast1199051015840)K (119905 1199051015840) (18)

where 1199051015840 is time of training data and a quasi-linear kernelwhich is explicitly explained in the following remark isdefined as an inner product of the explicit nonlinearmappingΦ(119905)

K (119905 1199051015840) = Φ119879 (119905) Φ (1199051015840)= 120593119879 (119905) 120593 (1199051015840) 119872sum

119894=0

N119894 (119905)N119894 (1199051015840) (19)

Remark 3 The quasi-linear kernel name is twofold Firstly itis derived from the quasi-ARX modeling scheme Secondlyfrom (19) it is known that when 119872 is as small as zero thekernel is reduced to a linear one and nonlinearity of thekernel mapping is improved when increasing the value of119872Comparedwith conventional kernels andwith implicit kernelmapping the nonlinear mapping of the quasi-linear kernelis turnable by119872 which also reflects the nonlinearity of thequasi-ARX RBFNmodels in the sense of the number of locallinear subspaces utilized A proper value of119872 is essentiallyhelpful to cope with the potential overfitting which will beshown in the following simulations

4 Experimental Studies

In this section identification performance of the aboveproposed approach to quasi-ARX RBFN model is evaluatedby three examples The first one is an example to show theperformance of quasi-ARX RBFN model for time series pre-diction Second a rational system generated from Narendraand Parthasarathy [17] is simulated with a small amount oftraining data which is used to demonstrate the generalizationof the proposed quasi-linear kernel At last an examplemodeling a hydraulic robot actuator is carried out for ageneral comparison

In the nonlinear parameter estimation procedure affinitypropagation (AP) clustering algorithm [32] is utilized topartition the input space and automatically generate the sizeof clusters in terms of data distribution where Euclidean

6 Complexity

distance is evaluated as the similarity between exemplarsThen centers of all clusters are selected as the RBF centerparameters in the quasi-ARXmodel and thewidth parameterof a certain RBF is decided as the mean value of distancesfrom the associated center to the nearest two others For thelinear parameter estimation LibSVM toolbox [33] is appliedwhere ]-SVR is used with default ] setting by Matlab 76Finally the model performance is evaluated by root meansquare error (RMSE) as

RMSE = radicsum119905 (119910 (119905) minus 119910 (119905))2119870 (20)

where 119910(119905) is the prediction value of the system output 119910(119905)and119870 is the number of regression vectors

41 Modeling the Mackey-Glass Time Series The time seriesprediction on the chaotic Mackey-Glass differential equationis one of the most famous benchmarks for comparing thelearning and generalization abilities of different models Thistime series is generated from the following equation

119889119909 (119905)119889119905 = 119886119909 (119905 minus 120591)1 + 11990910 (119905 minus 120591) minus 119887119909 (119905) (21)

where 119886 = 02 119887 = 01 and 120591 = 17 which are the most oftenused values in the previous research and the equation doesshow chaotic behavior with them To make the comparisonsfair with the earlier works we will predict 119909(119905 + 6) usingthe input variables 119909(119905) 119909(119905 minus 6) 119909(119905 minus 12) and 119909(119905 minus 18)Two thousand data points are generatedwith initial conditiontaken as 119909(119905) equiv 12 for 119905 isin [minus17 0] based on the fourth-orderRungendashKutta method with time step Δ119905 = 01 Then onethousand input-output data pairs are selected from 119905 = 201to 119905 = 1200 which is shown in Figure 3 The first 500 datapairs are used as training data while the remaining 500 areused to predict 119909(119905 + 6) followed by

119909 (119905 + 6) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (22)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (23)

where 120593(119905) = [119909(119905 minus 18) 119909(119905 minus 12) 119909(119905 minus 6) 119909(119905)]119879The prediction of the Mackey-Glass time series using

a quasi-ARX RBFN model starts where 20 clusters areobtained from the AP clustering algorithm and thus 20RBFneurons are correspondingly constructed Thereafter SVR isused for linear parameter estimation in which the super-parameter 119862 is set as 100 The predicted result is comparedwith the original time series of test data in Figure 4 whichgives a RMSE of 00091

In Figure 4 the predicted result fits the original datavery well however it is still not as good as the resultsfrom some famous modelsmethods listed in Table 1 Since

0 100 200 300 400 500 600 700 800 900 100004

06

08

1

12

14

t

x(t)

Figure 3 Time series generated from the Mackey-Glass equation

0 50 100 150 200 250 300 350 400 450

002040608

11214

Original dataPredicted dataError

t

x(t)

Figure 4 Prediction result with the quasi-ARX RBFN model

no disturbance is contained in this example it is foundthat the prediction performance can be easily improved byminimizing the training prediction error In the comparisonlist SNPOM for RBF-AR model hybrid learning methodfor local linear wavelet neural network (LLWNN) andgenetic algorithm (GA) for RBFN are all optimization-basedidentification methods and it is relatively easy for them toachieve small RMSEs of the prediction by iterative trainingHowever these methods are much more time-costing incomparison with only 6 seconds by the proposed methodfor the quasi-ARX RBFN model In addition although the119896-means clustering method for RBFN is implemented in adeterministic way and shows efficient result the number ofRBF neurons used is as big as 238 In fact a small predictionRMSE obtained from these methods does not mean goodidentification of the models since overtraining may happensome times

In the present example we confirm the effectiveness ofthe optimization-based method given above and propose ahybrid approach for identification of the quasi-ARX RBFNmodel where prediction result from the proposed methodcan be further improved by SNPOM (the function ldquolsqnon-linrdquo in the Matlab Optimization Toolbox is used [9]) It isseen that the prediction RMSE can be improved to 21 times10minus3 by only 15 iterations of implementation in SNPOMand the result becomes compatible with others Howeversuch optimization is not always effective especially in model

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

6 Complexity

distance is evaluated as the similarity between exemplarsThen centers of all clusters are selected as the RBF centerparameters in the quasi-ARXmodel and thewidth parameterof a certain RBF is decided as the mean value of distancesfrom the associated center to the nearest two others For thelinear parameter estimation LibSVM toolbox [33] is appliedwhere ]-SVR is used with default ] setting by Matlab 76Finally the model performance is evaluated by root meansquare error (RMSE) as

RMSE = radicsum119905 (119910 (119905) minus 119910 (119905))2119870 (20)

where 119910(119905) is the prediction value of the system output 119910(119905)and119870 is the number of regression vectors

41 Modeling the Mackey-Glass Time Series The time seriesprediction on the chaotic Mackey-Glass differential equationis one of the most famous benchmarks for comparing thelearning and generalization abilities of different models Thistime series is generated from the following equation

119889119909 (119905)119889119905 = 119886119909 (119905 minus 120591)1 + 11990910 (119905 minus 120591) minus 119887119909 (119905) (21)

where 119886 = 02 119887 = 01 and 120591 = 17 which are the most oftenused values in the previous research and the equation doesshow chaotic behavior with them To make the comparisonsfair with the earlier works we will predict 119909(119905 + 6) usingthe input variables 119909(119905) 119909(119905 minus 6) 119909(119905 minus 12) and 119909(119905 minus 18)Two thousand data points are generatedwith initial conditiontaken as 119909(119905) equiv 12 for 119905 isin [minus17 0] based on the fourth-orderRungendashKutta method with time step Δ119905 = 01 Then onethousand input-output data pairs are selected from 119905 = 201to 119905 = 1200 which is shown in Figure 3 The first 500 datapairs are used as training data while the remaining 500 areused to predict 119909(119905 + 6) followed by

119909 (119905 + 6) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (22)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (23)

where 120593(119905) = [119909(119905 minus 18) 119909(119905 minus 12) 119909(119905 minus 6) 119909(119905)]119879The prediction of the Mackey-Glass time series using

a quasi-ARX RBFN model starts where 20 clusters areobtained from the AP clustering algorithm and thus 20RBFneurons are correspondingly constructed Thereafter SVR isused for linear parameter estimation in which the super-parameter 119862 is set as 100 The predicted result is comparedwith the original time series of test data in Figure 4 whichgives a RMSE of 00091

In Figure 4 the predicted result fits the original datavery well however it is still not as good as the resultsfrom some famous modelsmethods listed in Table 1 Since

0 100 200 300 400 500 600 700 800 900 100004

06

08

1

12

14

t

x(t)

Figure 3 Time series generated from the Mackey-Glass equation

0 50 100 150 200 250 300 350 400 450

002040608

11214

Original dataPredicted dataError

t

x(t)

Figure 4 Prediction result with the quasi-ARX RBFN model

no disturbance is contained in this example it is foundthat the prediction performance can be easily improved byminimizing the training prediction error In the comparisonlist SNPOM for RBF-AR model hybrid learning methodfor local linear wavelet neural network (LLWNN) andgenetic algorithm (GA) for RBFN are all optimization-basedidentification methods and it is relatively easy for them toachieve small RMSEs of the prediction by iterative trainingHowever these methods are much more time-costing incomparison with only 6 seconds by the proposed methodfor the quasi-ARX RBFN model In addition although the119896-means clustering method for RBFN is implemented in adeterministic way and shows efficient result the number ofRBF neurons used is as big as 238 In fact a small predictionRMSE obtained from these methods does not mean goodidentification of the models since overtraining may happensome times

In the present example we confirm the effectiveness ofthe optimization-based method given above and propose ahybrid approach for identification of the quasi-ARX RBFNmodel where prediction result from the proposed methodcan be further improved by SNPOM (the function ldquolsqnon-linrdquo in the Matlab Optimization Toolbox is used [9]) It isseen that the prediction RMSE can be improved to 21 times10minus3 by only 15 iterations of implementation in SNPOMand the result becomes compatible with others Howeversuch optimization is not always effective especially in model

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Complexity 7

Table 1 Results of different models for Mackey-Glass time series prediction

Model Method Number of neurons RMSEAutoregressive model Least square 5 019FNT [8] PIPE Not provided 71 times 10minus3RBF-AR [9] SNPOM 25 58 times 10minus4LLWNN [10] PSO + gradient decent algorithm 10 36 times 10minus3RBF [11] 119896-means clustering 238 13 times 10minus3RBF [12] GA 98 15 times 10minus3Quasi-ARX RBFN model Proposed 20 91 times 10minus3Quasi-ARX RBFN model Proposed + SNPOM 20 21 times 10minus3

simulations on testing data such as in the model 119909(119905) =119891(119909(119905 minus 1) 119909(119905 minus 2)) where 119909(119905 minus 1) is the prediction valueof 119909(119905 minus 1) In the following a rational system is evaluatedby simulated quasi-ARX RBFN models to show advantagesof the proposed method

42 Modeling a Rational System Accurate identificationof nonlinear systems usually requires quite long trainingsequences which contain a sufficient amount of data from thewhole operating region However as the amount of data isoften limited in practice it is important to study the iden-tification performance for shorter training sequences with alimited amount of dataThe systemunder study is a nonlinearrational model described as

119910 (119905) = 119891 (119910 (119905 minus 1) 119910 (119905 minus 2) 119910 (119905 minus 3) 119906 (119905 minus 1) 119906 (119905 minus 2)) + 119890 (119905) (24)

where

119891 (1199091 1199092 1199093 1199094 1199095) = 1199091119909211990931199095 (1199093 minus 1) + 11990941 + 11990922 + 11990923 (25)

and 119890(119905) isin (0 001) is the white noiseDifficulty of this example lies in the fact that only 100

samples are provided for training which is created by 100random sequences distributed uniformly in the interval[minus1 1] while 800 testing data samples are generated from thesystem with input

119906 (119905)=

sin(2120587119905250) if 119905 le 50008 sin(2120587119905250) + 02 sin(212058711990525 ) otherwise

(26)

The excited training signal 119906(119905) and system output 119910(119905) areillustrated in Figure 5

In this case 9 clusters are automatically obtained fromthe AP clustering algorithm then the nonlinear parameters120583119895 and 120590119895 (119895 = 1 9) of the quasi-ARX RBFN model areestimated as Section 3 described SVR is utilized thereafterfor linear parameter estimation where super-parameters are

10 20 30 40 50 60 70 80 90 100minus1

minus050

051

10 20 30 40 50 60 70 80 90 100minus1

01

t

t

y(t)

u(t)

Figure 5 Training data for rational system identification

set with different values for testing Following the trainingthe simulated model is

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (27)

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381722119895 ) (28)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 119910(119905 minus 119899) denotes the simulated result in the previous 119899step Figure 6 simulates the quasi-ARX RBFN model on thetesting data which gives a RMSE of 00379 under the super-parameter 119862 = 10

Due to the fact that identification of the quasi-ARXRBFNmodel can be regarded as an SVR with quasi-linear kernel ageneral comparison is given to show advantages of the quasi-ARX RBFN model from SVR-based identification Not onlythe short training sequence but also a long sequence with1000 pairs of samples which is generated and implemented inthe same manner as the short one is applied for comparingTable 2 presents the comparison results of the proposedmethod (ie SVR with quasi-linear kernel) SVR with linearkernel SVR with Gaussian kernel and quasi-ARX modelidentified directly by an SVR (Q-ARX SVR) where variouschoices of SVR super-parameters119862 and 120574 for Gaussian kernel

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

8 Complexity

Table 2 Simulated results of the SVR-based methods for rational system

Method Super-parameters RMSE119862 120574 (Gaussian) Short training sequence Long training sequence

Proposed1 - 00546 0028710 - 00379 00216100 - 00423 00216

SVR + linear kernel1 - 00760 0071010 - 00764 00708100 - 00763 00708

SVR + Gaussian kernel

1

001 01465 00560005 00790 0042601 00808 0042105 00895 00279

10

001 00782 00376005 00722 0040901 00866 0036505 00699 00138

100

001 00722 00352005 00859 0037601 00931 0031305 01229 00340

Q-ARX SVR [13]

1

001 00698 00362005 00791 0038401 00857 0034505 00749 00116

10

001 00783 00412005 00918 0032801 00922 0024205 01483 00338

100

001 00872 00400005 01071 0023701 08186 0016605 01516 00487

0 100 200 300 400 500 600 700 800minus1

minus08minus06minus04minus02

0020406

t

y(t)

System outputSimulated output

Figure 6 Simulated result with the quasi-ARX RBFN model forrational system

are provided From the simulation results under a short train-ing sequence (100 samples) it is seen that when the design

parameters are optimized SVR with quasi-linear kernelperformsmuch better than the oneswithGaussian kernel andlinear kernel and the quasi-linear kernel also performs littlesensitively with respect to the SVR super-parameter settingMoreover although the Q-ARX SVR method utilizes thequasi-ARXmodel structure it only provides a similar simula-tion RMSE to SVRwithGaussian kernel However these sim-ulation results cannot be resorted to refute the effectivenessof the SVR with Gaussian kernel and Q-ARX SVR methodfor nonlinear system identification In the simulations for along training sequence (1000 samples) it is found that Q-ARX SVR method outperforms all the others and SVR withGaussian kernel also performsmuchbetter than the oneswithquasi-linear and linear kernel

On the other hand from the perspective of the per-formance variation caused by different training sequenceshistograms of simulated error for SVR-based methods aregiven in Figure 7 where performance of simulations is illus-trated using respectively the short training sequence and the

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Complexity 9

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 005 010

200

400

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

minus03 minus025 minus02 minus015 minus01 minus005 0 005 010

500

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with short training sequence

(a)

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

minus025 minus02 minus015 minus01 minus005 0 0050

200

400

Simulated error

QminusARX SVR

SVR + Gaussian kernel

SVR + linear kernel

SVR + quasi-linear kernel

Simulated results with long training sequence

(b)

Figure 7 Histograms of the simulated errors The horizontal coordinate in each subfigure denotes the simulated error of the model whoseelements are binned into 10 equally spaced containers Four models are trained by using both short (a) and long (b) training sequences thenthe simulated performance variation can be investigated by comparison

long training sequence It indicates that the SVR with linearkernel has themost robust performance to amount of trainingdata and the robust performance is also found in the quasi-linear kernel compared with the Gaussian kernel and Q-ARX SVR method where significant deterioration is foundin the simulations when a limited amount of training samplesare used This result implies that Gaussian kernel and Q-ARX SVR may be overfitted since the implicit nonlinearmapping is carried out which has strong nonlinear learningability but with no idea about how ldquostrongrdquo the nonlinearityneed is In contrast the truth behind the impressive androbust performance of the quasi-linear kernel is that priorknowledge is utilized in the kernel learning (nonlinearparameter estimation) and a number of parameters aredetermined in terms of data distribution where complexityof the model (nonlinearity) is tunable according to thenumber of local linear subspaces clustered In other wordsthe quasi-ARX RBFN model performs in a local linear wayhence it can be trained in a multilinear way better thansome unknown nonlinear approaches for the situation withinsufficient training samples

Moreover the RBF-AR model is utilized with SNPOMestimation method for this identification problem wherethe number of RBF neurons are determined by trail-and-error whose initial values are given randomly Consideringrandomness of the algorithm ten runs are implementedexcept that the results fail to be simulated and the maximum

iterations value in SNPOM is set to 50 Consequently fourRBFs are selected for RBF-AR model which gives a meanRMSE of 00696 using short training sequence comparedwith the result of 00336 when the long training one isutilized Although the parameter setting for this methodmaynot be optimal we can generate the same conclusion forthe Q-ARX SVR method which is overfitted in the case oftraining by short sequence

43 Modeling a Real System This is an example modeling ahydraulic robot actuator where the position of the robot armis controlled by a hydraulic actuator The oil pressure in theactuator is controlled by the size of the valve opening throughwhich the oil flows into the actuator What we want to modelis the dynamic relationship between the position of the valve119906(119905) and the oil pressure 119910(119905)

A sample of 1024 pairs of 119910(119905) 119906(119905) has been observedas shown in Figure 8 The data is divided into two equalparts the first 512 samples are used as training data and therest are used to test the simulated model For the purposeof comparison the regression vector is set as 120593(119905) = [119910(119905 minus1) 119910(119905minus2) 119910(119905minus3) 119906(119905minus1) 119906(119905minus2)]119879 We simulate the quasi-ARX RBFN model on the testing data by

119910 (119905) = 120593119879 (119905) Ω0 + sum119895=1

120593119879 (119905) Ω119895N (120583119895 119895 120593 (119905)) (29)

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

10 Complexity

0 100 200 300 400 500 600 700 800 900 1000minus2

minus1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000minus4

minus2

0

2

4

t

t

y(t)

u(t)

Figure 8 Measurements of 119906(119905) and 119910(119905)

0 50 100 150 200 250 300 350 400 450 500minus4

minus3

minus2

minus1

0

1

2

3

4

t

y(t)

Measured dataSimulated data

Figure 9 Simulated result with quasi-ARXRBFNmodel for the realsystem

with

N (120583119895 119895 120593 (119905)) = exp(minus10038171003817100381710038171003817120593 (119905) minus 1205831198951003817100381710038171003817100381721205822119895 ) (30)

where 120593(119905) = [119910(119905 minus 1) 119910(119905 minus 2) 119910(119905 minus 3) 119906(119905 minus 1) 119906(119905 minus 2)]119879and 120582 is set as 50 heuristically due to the complex dynamicsand data distribution in this case which insures that the RBFsare wide enough to cover the whole space well Similar settingof 120582 can also be found in the literature for the same purpose[34 35]

To determine the nonlinear parameters of the quasi-ARXRBFN model AP clustering algorithm is implemented and11 clusters are generated automatically Then SVR is utilizedfor the linear parameter estimation Finally the model isidentified and simulated in Figure 9 by the testing data whichgives a RMSE of 0462 This simulation result is comparedwith the ones of linear ARXmodel NNWN and SVR-basedmethods shown in Table 3 From Table 3 it is known that theproposed method outperforms the others for the real systemIn addition RBF-ARmodel with SNPOMestimationmethod

Table 3 Comparison results for the real system

Model Super-parameters RMSE119862 120574 (Gaussian)ARX model - 1016NN [1] - 0467WN [6] - 0529

SVR + quasi-linear kernel1 - 04625 - 048710 - 0491

SVR + Gaussian kernel

1

005 106001 082802 064305 1122

5005 085001 074002 056205 0633

10

005 077501 066502 060805 1024

Q-ARX SVR [13]

1

005 073701 059202 080105 0711

5

005 060901 060002 071505 0890

10

005 059301 063202 123105 1285

fails to be simulated in this case where the number of RBFneurons is tested from 3 to 6 and their initial values are givenrandomly

5 Discussions and Conclusions

The proposed method has a twofold role in the quasi-ARXmodel identification For one thing the clustering methodhas been used to uncover the local linear information ofthe dataset Although similar methods have appeared in theparameter estimation of RBFNs meaningful interpretationhas been given here to the nonlinear parameters of quasi-ARX model in the manner of multilocal linear model withinterpolations In fact explicit local linearity does not alwaysexist in many real problems whereas clustering can provideat least a rational multidimensional space partition approachIn the future a more accurate and general space partitionalgorithm is to be investigated for identification of quasi-ARXmodels For another SVR has been utilized for the modelrsquos

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Complexity 11

linear parameter estimation meanwhile a quasi-linear ker-nel is deduced and performed as a composite kernel Theparameter119872 in the kernel function (19) corresponds to theamount of subspaces partitioned which is therefore preferrednot to be a big value to cope with the potential overfitting

In this paper a two-step learning approach has been pro-posed for identification of quasi-ARXmodel Unlike the con-ventional black-box identification approaches prior knowl-edge is introduced and makes sense for the interpretabilityof quasi-ARXmodels By minimizing the training data errorlinear parameters to the model are estimated In the simula-tions the quasi-ARX model is denoted in the form of SVRwith quasi-linear kernel which shows great approximationability as optimization-basedmethods for quasi-ARXmodelsbut outperforms them when the training sequence is limitedFinally the best performance of the proposed method hasbeen demonstrated with a real system identification problem

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China underGrants 81320108018 and 31570943and the Six Talent Peaks Project for the High Level Personnelfrom the Jiangsu Province of China underGrant 2015-DZXX-003

References

[1] J Sjoberg Q Zhang L Ljung et al ldquoNonlinear black-boxmod-eling in system identification a unified overviewrdquo Automaticavol 31 no 12 pp 1691ndash1724 1995

[2] I Machon-Gonzalez and H Lopez-Garcia ldquoFeedforward non-linear control using neural gas networkrdquo Complexity Article ID3125073 11 pages 2017

[3] J P Noel and G Kerschen ldquoNonlinear system identificationin structural dynamics 10 more years of progressrdquo MechanicalSystems and Signal Processing vol 83 pp 2ndash35 2017

[4] G Nagamani S Ramasamy and P Balasubramaniam ldquoRobustdissipativity and passivity analysis for discrete-time stochasticneural networks with time-varying delayrdquo Complexity vol 21no 3 pp 47ndash58 2016

[5] I Sutrisno M A Jamirsquoin J HU and M H Marhaban ldquoAself-organizing Quasi-linear ARX RBFN model for nonlineardynamical systems identificationrdquo SICE Journal of ControlMeasurement and System Integration vol 9 no 2 pp 70ndash772016

[6] J Hu K Hirasawa and K Kumamaru ldquoA hybrid quasi-ARMAX modeling and identification scheme for nonlinearsystemsrdquo Research Reports on Information Science and ElectricalEngineering of Kyushu University vol 2 no 2 pp 213ndash218 1997

[7] L Ljung System Identification Theory for the User Prentice-Hall Englewood Cliffs NJ USA 2nd edition 1999

[8] Y Chen B Yang J Dong and A Abraham ldquoTime-series fore-casting using flexible neural tree modelrdquo Information Sciencesvol 174 no 3-4 pp 219ndash235 2005

[9] H Peng T Ozaki V Haggan-Ozaki and Y Toyoda ldquoA parame-ter optimization method for radial basis function type modelsrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 14 no 2 pp 432ndash438 2003

[10] Y Chen B Yang and J Dong ldquoTime-series prediction using alocal linear wavelet neural networkrdquo Neurocomputing vol 69no 4ndash6 pp 449ndash465 2006

[11] C Harpham and C W Dawson ldquoThe effect of different basisfunctions on a radial basis function network for time seriesprediction a comparative studyrdquo Neurocomputing vol 69 no16-18 pp 2161ndash2170 2006

[12] H Du and N Zhang ldquoTime series prediction using evolvingradial basis function networks with new encoding schemerdquoNeurocomputing vol 71 no 7-9 pp 1388ndash1400 2008

[13] H T Toivonen S Totterman and B Akesson ldquoIdentificationof state-dependent parameter models with support vectorregressionrdquo International Journal of Control vol 80 no 9 pp1454ndash1470 2007

[14] J Hu K Kumamaru and K Hirasawa ldquoA quasi-ARMAXapproach to modelling of non-linear systemsrdquo InternationalJournal of Control vol 74 no 18 pp 1754ndash1766 2001

[15] J Hu and K Hirasawa ldquoAmethod for applying neural networksto control of nonlinear systemsrdquo in Neural Information Process-ing Research and Development vol 152 of Studies in Fuzzinessand Soft Computing pp 351ndash369 Springe Berlin Germany2004

[16] L Wang Y Cheng and J Hu ldquoStabilizing switching controlfor nonlinear system based on quasi-ARX RBFN modelrdquo IEEJTransactions on Electrical and Electronic Engineering vol 7 no4 pp 390ndash396 2012

[17] K S Narendra andK Parthasarathy ldquoIdentification and controlof dynamical systems using neural networksrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 1 no 1 pp4ndash27 1990

[18] P C Young P McKenna and J Bruun ldquoIdentification ofnon-linear stochastic systems by state dependent parameterestimationrdquo International Journal of Control vol 74 no 18 pp1837ndash1857 2001

[19] M Gan H Peng X Peng X Chen and G Inoussa ldquoA locallylinear RBF-network-based state-dependent AR model fornonlinear time series modelingrdquo Information Sciences vol 180no 22 pp 4370ndash4383 2010

[20] A Janot P C Young and M Gautier ldquoIdentification andcontrol of electro-mechanical systems using state-dependentparameter estimationrdquo International Journal of Control vol 90no 4 pp 643ndash660 2017

[21] A Patra S Das S N Mishra and M R Senapati ldquoAn adaptivelocal linear optimized radial basis functional neural networkmodel for financial time series predictionrdquo Neural Computingand Applications vol 28 no 1 pp 101ndash110 2017

[22] M Gan H Peng and L Chen ldquoA global-local optimizationapproach to parameter estimation of RBF-type modelsrdquo Infor-mation Sciences vol 197 pp 144ndash160 2012

[23] Y Cheng LWang and J Hu ldquoIdentification of quasi-ARXneu-rofuzzy model with an SVR and GA approachrdquo IEICE Trans-actions on Fundamentals of Electronics Communications andComputer Sciences vol E95-A no 5 pp 876ndash883 2012

[24] Y Cheng Study on Identification of Nonlinear Systems UsingQuasi-ARXModels [PhD dissertation]WasedaUniversity June2012

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

12 Complexity

[25] Y Cheng L Wang and J Hu ldquoQuasi-ARX wavelet network forSVR based nonlinear system identificationrdquo Nonlinear Theoryand Its Applications IEICE vol 2 no 2 pp 165ndash179 2011

[26] B M Akesson andH T Toivonen ldquoState-dependent parametermodelling and identification of stochastic non-linear sampled-data systemsrdquo Journal of Process Control vol 16 no 8 pp 877ndash886 2006

[27] B-G Hu H-B Qu Y Wang and S-H Yang ldquoA generalized-constraint neural network model associating partially knownrelationships for nonlinear regressionsrdquo Information Sciencesvol 179 no 12 pp 1929ndash1943 2009

[28] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press Cambridge Mass USA 2001

[29] C Panchapakesan M Palaniswami D Ralph and C ManzieldquoEffects of moving the centers in an RBF networkrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no 6pp 1299ndash1307 2002

[30] J Gonzalez I Rojas H Pomares J Ortega and A Prieto ldquoAnew clustering technique for function approximationrdquo IEEETransactions on Neural Networks and Learning Systems vol 13no 1 pp 132ndash142 2002

[31] A Guillen H Pomares I Rojas et al ldquoStudying possibility ina clustering algorithm for RBFNN design for function approx-imationrdquo Neural Computing and Applications vol 17 no 1 pp75ndash89 2008

[32] B J Frey and D Dueck ldquoClustering by passing messagesbetween data pointsrdquo American Association for the Advance-ment of Science Science vol 315 no 5814 pp 972ndash976 2007

[33] C Chang and C Lin ldquoLIBSVM a Library for support vectormachinesrdquo httpwwwcsientuedutwsimcjlinlibsvm

[34] YOussar andGDreyfus ldquoInitialization by selection forwaveletnetwork trainingrdquo Neurocomputing vol 34 no 4 pp 131ndash1432000

[35] Y Bodyanskiy and O Vynokurova ldquoHybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identificationrdquo Infor-mation Sciences vol 220 pp 170ndash179 2013

Submit your manuscripts athttpswwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of