A Predictive Model ofNonlinear System Basedon Generalized Regression Neural Network

Yibin Song, Ying Ren

School ofComputer Science, Yantai University, Shandong, P. R. China 264005Email: sybwAytu.edu.cn

Abstract -Generalized regression Neural Network (GRNN) isusually applied to the Function approximation. Based on theprinciple of GRNN, this paper presents a method for thepredictive model of nonlinear complex system. The presentedalgorithm is applied to the training and predicting process ofthe nonlinear model. The- simulations show the presentedmethod has good effects on predicting the dynamic process ofthe nonlinear model, and could be applied on the predictioncontrol for nonlinear systems satisfactorily.

L DITRODUCTION

It is known the Generalized Regression Neural Network(GRNN) is much efficient method for fitting orapproximating the output of complex plants. GRNN isusually made up of a layer of radial base network and alayer of linear network. It is known that Predictive Controlis referred to as model-based control. It combines thecontrol strategy with predictive model, optimizing algorithmand feedback adjusting. The predictive control for linearsystems had been resolved perfectly, but it's difficult for thenon-linear complex systems. To realize predictive modelaccurately for the non-linear dynamic process is the first keywork. According to the active I/O information, thepredictive model based on GRNN could be used to describethe dynamic action of nonlinear plants and to predict theintending outputs. The information is to be the base ofpredictive control. So, setting the predictive model is theprecondition for realizing predictive control of complex. Forthe structure-known linear models, there are many knownmethods for getting predictive information. However, it isdifficult to obtaining predicting information from thestructure-unknown and definition-deficiently nonlinearcomplex plants. So, the predictive control will be difficult torealize usually. The approximating ability of GRNN willmakes it possible to get the predictive model quickly andaccurately.On the other hand, GRNN is similar as multi-layer BP-

NN in network structure. The BP-NN needs usually largeamounts of calculation of weights for getting the globalapproach. The main characteristics of GRNN are its outputhas the local linear relation to the parameters. It is a typicallocal approaching network. So, its learning speed is morequickly than that of BP-NN, especially for the functionapproaching and model prediction for structure-

unknown objects. It is a powerful tool for realizing thepredictive control ofthe complex plants or systems.

Based on the principle of GRNN, this paper presents amethod for the predictive model of nonlinear complexsystem. The presented algorithm is applied to the predictingprocess of a nonlinear model. The simulations show thatGRNN has good effects on predicting the dynamic processof the complex model, and could be applied on theprediction control for nonlinear systems satisfactorily.

H. PREDICTWE CONTOL BASED ON GRNN

A. GRNN structure analysisThe typical structure of GRNN is shown as Fig. 1. It is

often used in function approximation for complex models. Itconsists of a layer of radial base network and a layer oflinear network. Direct mappings are adopted by GRNNbetween input layer and hidden layer. But the mappingbetween hidden layer and output layer adopts the weightedlinearly sum of hidden layers as the mapping mode (shownas Fig. 1). In Fig. 1, LW1,i E QxR are elements ofthe ithrow ofthe weighting matrix of first layer. ".*" stands for thedot product of input Vectors. LW2, E QxQ is the weightmatrix. al(E Qxl) is the output of first layer. R is thedimension of network input. Q is the neuron number ofevery layer and is also the number of training samples.Nprod presents the normal dot matrix for the calculating theoutput vectors n2(E QxI). a2(E QxI) is the output ofGRNN.GRNN adopts direct mapping from input cell to hidden

layer. But the mapping between hidden layer and outputlayer adopts the linearly weighted sum of hidden layers asthe mapping mode (shown as in Fig. 1). This structure ofNN can reduce the complexity of computational problemsso as to speed up the learning process. It is suitableespecially for accomplishing the function approximating andmodel identification quickly.

Usually, the Neural Network Predictive Control (NNPC)system consists of predictive model, looping optimizing andfeedback adjusting. Taking the errors between intendingexpectation and prediction as the input value, the predictivecontroller realizes the desired performance. The function ofNN predictor model is to provide the output predictive

0-7803-9422-4/05/$20.00 C2005 IEEE 2009

information accurately and quickly. According to the general structure of NNPC system based on GRNN isprediction 5(t) and the expectation r(t) , the designed shown as Fig. 2, where r(t) is the expectation of output,target function J could be optimized by the Nonlinear y(t) is actual output and y(t + k) is the output prediction.Optimuzer so as to reach the intending control goal. The

Figure 1: Structure ofGRNN

B. Predictive Model for Nonlinear System Based onGRNN

In general, GRNN is made up of two layers, i.e. hiddenlayer and output layer. It is similar to the BP-NN in themodel frame. The differences are the mapping function ofGRNN adopts the Generalized Regression Algorithm for thetraining of Neural Network, which has the features of itsoutput having the local linear relation to the parameters.

T'he network training could avoid the calculation ofnonlinear optimizing and, therefore, there will not be theproblem of local particle. So, the topological structure ofneural network can be defined during the learning process,which is more quickly than BP-NN. To complete predictivecontrol, the steps ofNNPC are following:I I I I Obtain the expectation of output sequence

r(t + j) (j=Kl, Kj+1,..., P I1 2 1 1 Create the prediction by RBF-NN model

y'(t + Alt) (j= K1, K1+I, ..., P)and then, to modify the errors:

y(t + jlit) = y(t + jlt) + aj[y(t + ]) - y(t + ijlt - 1)]I j= K, K+l, ..., P I

1 3 I1 Calculate prediction errors:e(t + j) = r(t + j)-y(t + jlt) I j= K1, K+Il, ..., P I1 4 1 1 Optimize the desired error goal function J so as to

get the optimal control sequence:u(t+ j) Ij=O, 1,2,3, ...,MI

1 5 I1 Put the u(t) to the system, and back to step (1).It is known the key function of GRNN predictor is to

realize the prediction of system output quickly andaccurately. In GRNN, the hidden layer adopts the nonlinearoptimizing strategy. So, the learning speed of the functionparameters will be slower accordingly.

Oppositely, the output layer adopts the linear optimizingstrategy for the adjusting of weights. The learning speed isquicker than that of the hidden layer. The input vectors oflower dimensions are mapped to the hidden space of higherdimensions first. The hidden cells select the generalizedregression basis fimction to realize the vector conversion,and then, sorted or identified by output layer. The form ofbase function F can be described as:

NF(X)= fir(I( X-Xi IP (1)i=.

Where V(jj X - X, 11) is the generalized radial basis

fimction (RBF), usually uses Gaussian fimction, (j *denotes the Euclid Norm.

r(t) 91:~~~~~~~~~Yt

|pFilter P

Figure 2: Structure ofNNPC System

To take known data Xi E R' as the center ofRBF and 3is radial-symmetrical to the center point. IfGauss function isadopted as RBF, it can be described as:

(2)G4| X - ti 112) = exp(- d || X ti 112)Where, M is the number of center, ti is the center ofRBF

and dm is the maximal distance between selected centers.The mean-square-error ofRBF takes 7=dm/[K

According to the different ways for selection of RBF

2010

center, GRNN is usually applied the methods as randomselection, self-organization learning (SOL) or supervisorylearning, etc. This paper applies Orthogonal-least-square(OLS) learning algorithm to choose the center of RBF so asto complete the network training. In OLS algorithm, thecentral position of RBF is set automatically. The weights ofoutput layer can be calculated by error-correction-learningalgorithm. So, it is known the OLS algorithm is a mixedlearning algorithm essentially. Its function is to adjust thecenter ofRBF to the key area of input space.

Clustering least distance is the learning goal of OLSalgorithm. The steps of training algorithm are:(1) Pre-elect randomly Mas the cell number of hidden layer;(2) Pre-elect a set of center vector t,( 1 < i <M ) for Radial

Base Function. Distribute input samples X.(n=1,2,-N) to every ti and to get the regression

operators matrix P:P = [pi (n)] = G(J|Xn - ti l|) (3)

(n-=, 2, ,N, I=1, 2, .., M)(3) Orthogonalize the matrix P by the following equations:

ui PiTC U P (4)

ui . Uik -IUk-Eai*Ui

(I < i < k; k = 2 .M)where Mi is the number of input sample in 0 i

(4) Calculate gi and -i by using the following equation:

uT *d

2 Tg i eu .uit i

=

T,T

(1. i< M)

coefficient for the signal prediction of complex models. Theadaptive spread coefficient is described as:

Sc=Sc+k; e ~ (7)

Where, k4 is the fitting coefficient,

; = Imin(err) - err*,err* is the desired learning error.

In order to get a suitable spread factor, the relative errorof learning ; is introduced into the training algorithm. Thevalue of Sc can be modified adaptively with the trainingerrors. Simulations show the amended algorithm canimprove the training performance and simplify the structureof neural network satisfactorily.

III. SIMULATION RESULTS

According to the sampling data of a nonlinear plant, thispaper applies a generalized regression neural network tostructure a predictive model for a nonlinear system. Supposea nonlinear system can be described by following equations:Input:

2ku)3k0)u(k)=0.25 -[Sin( ))+Sin( )]250 100

Nonlinear plant:

y(k -1) y(k - 2) u(k -1)[y(k - 3)-1) +u2(k - 2)]+y2(k-)+y2(k-2)

Samples of Output of the complex plant

(5)

(1. i< M)Get the upper triangle matrix A, and then calculate the

linking weights Wby the equation: A W = g .

Where g=[g1g2. gM]T(5) Check the whether the following equation is met. If itdoesn't be met, turn back to step (2) for reselecting thecenter of RBF.

M1- £i < P

i=l(6)

Where 0 < p < 1 is the fixed tolerance.The spread coefficient (Sc) is one of important parameters

in the GRNN learning algorithm. The different decision ofSc will affect the learning performance directly. In fact, asuitable Sc can get smoother process for the signalapproximating obviously. This paper presents an improvedGRNN learning method based on the adaptive spread

0.2

0L0

., It

100

I

! I1.

.--200

If300 400 500

Figure 3. Learning result ofGRNN for 500 pairs samples

Suppose the system structure is unknown. GRNN needsto model the unknown plant by learning the sampling data.To Sample respectively 500 pairs and 800 pairs of modelI/O data, from a nonlinear system, as the learning samplesfor the GRNN. The training results and predictive output ofGRNN are shown in Fig. 2 and Fig. 3. Where, '+' stands forsampling data, shows the results after learning by theGRNN. The learning mean variance is 1.7576e-005 and1.3860e-005 respectively. Meanwhile, after trainingfinished, some pairs of new samples are sent to the trained

2011

0.

i

.-L

GRNN so as to test its performance. Simulations show thetrained GRNN has the fine generalization ability obviously(shown in Fig. 5).

It is shown the training algorithm based on GRNN is aneffective way to realize the predictive control for thecomplex systems. The main strong feature of GRNN is itsspeediness and accuracy on function approximating ofcomplex model, especially suitable for constructing thepredictor model. It can provide the predict information forthe predictive control system quickly and accurately.

Samples of Output of the complex plant

3

2

-1

-2

-3

xio Training errors ofGRNN

0 200 460 600 800

Figure 6. Learning errors of the trained GRNN

IV. CONCLUSIONS

Fi

OA The excellent characteristics of GRNN algorithm are itsquick convergence and accurate learning results. This paper

0.2 presents a training algorithm based on GRNN for the neuralnetwork predictor. After applying to the modeling of a

typical nonlinear system and simulations, it is found that the0° 200 400 wo go

GRNN has a quicker learning process and an excellenttraining performance. In the aspect of function fitting and

igure 4. Learning result ofGRNN for 800 pairs samples signal prediction, GRNN has more excellent performancethan that of normal BP-NN. It is more suitable for getting

Samples from the complex plant the predictive model for complex plants. Therefore, GRNN0.2 is an effective way for the realization of the real-time1.1s6 / \ ~ predictive control for complex systems.

Preditive output. by the trained GRNN0.2 I

0.15 2

0.1

0.05

0 20 40 60 80 100

Figure 5. Generalization ability ofthe designed GRNN

It can be found the learning algorithm based on GRNNshows a clear advantage on learning performances,especially on the model formation of predictor. It canimprove performances obviously. It proved that the RBF-NN is more suitable for the modeling and identifying ofcomplex plants. It is an effective method for the predictionand control ofcomplex systems.

REFERENCES

[1] Philippe De Wilde, Neural Network Models, An analysis. Springer-Verlag, 1996.

[2] Specht D F, A general regression neural network. IEEE Trans. onNeural Networks, 2(6), 1991, pp. 568-576.

[3] Norio Baba, A new approach for finding the global minimum of errorfunction of neural networks. IEEE Trans. on Neural Networks, 3,1990, pp. 535-549.

[4] M. Bianchini, P. Frasconi and M. Gori, Learning without localminima in radial basis function networks, IEEE Trans. on NeuralNetworks, 6(3), 1995, pp.749-756.

[5] Kosmatopulos E B, high-Order Neural Network Structures forIdentification of Dynamical Systems. IEEE Trans. on NeuralNetworks. 6(2), 1995, pp. 422-431.

[6] Younes C, Suranjan P, Leonard F, A generalized regression neuralnetwork and its application for leaf wetness prediction to forecastplant disease. Chemometrics and Intelligent Laboratory System, 48,1999, pp. 47-58.

2012

KAjA WV~~~~~~~~~~~~

r

e4