The preference of Fuzzy Wavelet Neural Network to ANFIS

In Identification of Nonlinear Dynamic Plants with Fast

Local Variation

Mehrnoosh Davani poor Maryam Zekri, Farid Sheikholeslam

Department of Electrical and Computer Engineering Department of Electrical and Computer Engineering

Isfahan University of Technology Isfahan University of Technology

Isfahan, Iran Isfahan, Iran m.davanipoor@ec.iut.ac.ir mzekri@cc.iut.ac.ir, sheikh@cc.iut.ac.ir

Abstract— This paper presents a Fuzzy Wavelet Neural Network (FWNN) for identification of a system with fast local variation. The FWNN combines wavelet theory with fuzzy logic and neural networks. An effective clustering algorithm is used to initialize the parameters of the FWNN. Learning fuzzy rules in this FWNN is based on gradient decent method. The performance of the FWNN structure is illustrated by applying to a nonlinear dynamic plant which has fast local variation then compared with Adaptive Neuro-Fuzzy Inference System (ANFIS) model. Simulation results indicate remarkable capabilities of the proposed identification method for plants with fast local variation.

Keywords; Wavelet neural networks; Fuzzy wavelet neural networks; Adaptive Neuro-Fuzzy System; System Identification

I. INTRODUCTION The techniques of artificial intelligence based on fuzzy logic and neural networks are frequently applied together [1, 2]. The reasons to combine these two paradigms come out of the difficulties and inherent limitations of each isolated paradigm. Generally, when they are used in a combined way, they are called Neuro-Fuzzy Systems. Numerous different neural and fuzzy structures are proposed for solving identification problems [3, 4]. A well known structure is Adaptive Neuro-Fuzzy Inference System (ANFIS) [5]. In ANFIS, initial membership functions can be set up intuitively. Then, a set of fuzzy If-Then rules is generated by learning process to approximate a desired data set. In ANFIS model, the number of membership functions assigned to each input variable is chosen empirically, i.e., by examining the desired data set and/or by trial and error. Therefore, there are no simple ways to advance determining of the minimal number of hidden nodes, necessary to achieve a desired performance

level. On the other hand, the ANFIS generates a complete rule base and therefore, suffers from the curse of dimensionality [6]. Most of Neuro-Fuzzy systems utilize the back- propagation algorithm in learning step. The back-propagation algorithm that implements a nonlinear gradient optimization procedure can be trapped at a local minimum and converges slowly. On the other hand wavelet transform reveals properties of the function in localized regions of the joint time-frequency space. Wavelet neural network (WNN) can avoid local minimum.

A fuzzy wavelet neural network (FWNN) combines wavelet theory with fuzzy logic and neural networks. Recently, this combination was applied in function approximation, identification and control [7, 8, 9, 10].

Wavelet transform has the ability to analyze non-stationary signals to discover their local details. On the other hand fuzzy logic allows to reduce the complexity of the data and to deal with uncertainty. Also Neural Networks (NNs) have self-learning capability that increases the accuracy of the model. Therefore their combination allows us to develop a system with fast learning capability that can describe nonlinear systems that are characterized with uncertainties and local variation, perfectly.

In this paper these methodologies are combined to construct a FWNN for system identification. This FWNN structure is tested for identification of a dynamic plant. Also an ANFIS model is used to identify that plant. Because the output signal has fast local variation, the structure of ANFIS model becomes much more complex with respect to the structure of FWNN. The reason is that in ANFIS model, firstly, THEN-part of TSF (Takagi-Sugeno Fuzzy) model is a linear combination of the input variables which is essentially a global function and

Proceedings of ICEE 2010, May 11-13, 2010 978-1-4244-6760-0/10/$26.00 ©2010 IEEE

)(,....:1

11 xwythenAisxAisxIfRq

jjijii

iqq

ii �=

Ψ=

secondly, the number of statements in this linear combination is the same for all fuzzy rules. But note that the FWNN model, by using less fuzzy rules, utilizes the sub-WNNs with different resolution levels and various number of wavelets for each sub-WNN, captures different behaviors (global or local) of approximated signal. As the results of simulation show the proposed approach can identify the plant with fast local variation well meanwhile the ANFIS model with much more fuzzy rules is not able to identify the same plant. This paper is organized as follows: Section 2 describes the FWNN structure, Section 3 includes learning algorithm. In Section 4 we compare the results of proposed approach with those of the ANFIS model. Concluding remarks are given in Section 5.

II. FUZZY WAVELET NEURAL NETWORK A wavelet family are defined in the following form

qjforaabx

ax jj

jjj ,...,10)()(

2/1=≠

−Ψ=Ψ

− (1)

In (1), )(xjΨ is obtained from a single mother wavelet function by dilation and translations (a, b). Here q is the dimension of input vector.

A typical fuzzy wavelet neural network for approximating a function can be described by a set of fuzzy rules [11]:

(2)

where )1( qjx j ≤≤ is the j th input and iy is the output of

the local model for rule iR , which is equal to the linear combination of a finite set of wavelets.

jiΨ is defined such as (1). The structure of fuzzy wavelet network for approximating function is given in Fig.1.

The output of the network is calculated as

(3) where

and c is the total number of fuzzy rules and q is the number of inputs.

Most works done in the wavelet networks uses simple wavelets. In this paper, Morlet wavelet is used that is represented in (4).

)4(

Also all the membership functions are Gaussian-type function defined as (5).

(5)

In which ijc and i

jσ determine the center and the half- width of the corresponding membership function respectively.

III. LEARNING ALGORITHM

The training process of the FWNN is included two steps. The first step is determining the initial values of network parameters. The second step is updating these parameters. A good initialization of the parameters of the FWNN enables to obtain fast convergence. A number of methods are proposed in literature for initialization of the wavelets, such as the orthogonal least square procedure [7, 8] and the clustering method [12]. An optimal initial choice of the dilation and the translation parameters of the wavelets increases the training speed and results in fast convergence. Here we have initialize the wavelet parameters with an effective clustering algorithm in FWNN to reduce the number of fuzzy rules and complexity of the structure. In this clustering algorithm it isn’t necessary to guess the number of clusters and consequently the number of fuzzy rules in FWNN.

In learning step, a gradient based learning algorithm is used for updating the parameters in FWNN. In addition, a momentum is used to speed up the learning process. The parameters to be updated are the parameters of the membership functions in the second layer of the network in Fig.1, ( ijc and ijσ ) and the parameters of wavelets ( ija , ijb

and iw for i=1,..,c and j=1,…,q) in the consequent part. In the learning step our goal is to minimize the cost function

which is calculated as follows:

(6)

uandu d are Here the desired and the current output values of the network, respectively.

IV. SIMULATION RESULTS In this section a number of simulations is carried out in

order to compare the performance of the proposed structure with that of the ANFIS model. We applied both structures to a nonlinear dynamic plant. Because the output signal has fast local variation, the structure of ANFIS model becomes much more complex with respect to the structure of FWNN. We construct the FWNN with only two fuzzy rules. As the results of simulations show the proposed approach can identify the plant with fast local variation well meanwhile the ANFIS model with much more fuzzy rules is not able to identify the same plant. In addition the CPU time of running program is reduced in FWNN. Here simulations are done by Matlab

qqi xxxxandw ℜ∈=ℜ∈ ],....,,[ 21

)2/exp().5cos()( 2xxx −=Ψ

]/)(exp[)( 22 ij

ijjj

ij cxxA σ−−=

)()(1

j

q

j

iji xAx ∏

=

=μ

qjci ,....,2,1,,....,2,1 ==

�

�

=

== c

ii

c

iii y

u

1

1

μ

μ

)()(21 uuuuE d

Td −−=

Figure 1. Structure of FWNN

toolbox. As an example the below nonlinear dynamic plant is considered:

(7) For fast local variation the following excitation signal is

used: (8)

The network was trained for 200 epochs with 300 time steps in each epoch. The results of simulations are depicted in Figs.2-4. In Fig.2 (a) the plant’s actual output and the output of FWNN are shown. Fig.2 (b) illustrates the plant’s actual output and the output of ANFIS for sixteen fuzzy rules. As these figures show the FWNN with only two fuzzy rules can identify the plant well meanwhile the ANFIS model with much more fuzzy rules (sixteen) is not able to do that. Fig.3 (a) and (b) show the error signal in FWNN and ANFIS model, respectively. The preference of the FWNN’s performance with respect to ANFIS can be clearly seen from these figures, especially in points which local variation is very fast. Also we tried to identify that plant by using ANFIS model with twenty five fuzzy rules, however as the Fig.4 (a, b) show the result is not satisfactory again. The results of comparison between these two structures are given in table.1. We can see from this table that the CPU time is reduced to 2.93 (sec) when the FWNN is applied.

V. CONCLUSION In this paper a fuzzy wavelet neural network structure was discussed to solve the problem of system identification. The proposed structure incorporates the advantages of wavelet function, neural networks and fuzzy logic. Therefore we could develop a system with fast learning capability that can identify nonlinear dynamic plants which are characterized with

uncertainties and fast local variation, perfectly. The structure was tested for identification of a nonlinear dynamic plant which had fast local variation. Also the ANFIS model was used to identify the same plant. It was shown that the proposed approach with only two fuzzy rules identify the plant with fast local variation meanwhile the ANFIS model with much more fuzzy rules is not able to do that.

REFERENCES

[1] J. Shing, R. Jang, C. T. Sun, "Neuro-Fuzzy modeling and

control," Proc. IEEE, 1995. [2] C. S. Quang, W. J. Lee. "A TSK-Type Neuro-Fuzzy network approach

to system modeling problems," IEEE Trans. On Sys. Man and Cyberntic, Vol. 35, No. 4, 2005.

[3] V. Topalov, G. L. Cascella, V. Giordano, F. Cupertino, O. Kaynak, "Sliding mode Neuro-Adaptive control of electrical drives," IEEE Trans. Indust. Electron., V. 54, No. 1, pp. 671-679, 2007.

[4] M.J. Er, Y. Gao, "Robust adaptive control of robot manipulators using generalized fuzzy neural networks," Trans. On Industrial Electronics, Vol. 50, No.3, 2003.

[5] J. S. R.. Jang, C. T. Sun, E. Mizutani, "Neuro-Fuzzy and soft computing," Prentice-Hall New Jersey, 1997.

[6] C. Harris, X. Hong, Q. Gan, "Adaptive modeling, estimation and fusion from Data," Springer, Berlin, 2002.

[7] W.C. HO Daniel, Ping-An Zhang, and Jinhua Xu. "Fuzzy wavelet networks for function learning," IEEE Transactions on Fuzzy Systems, Vol. 9, No. 1, 2001.

[8] M. Zekri, S. Sadri, F. Sheikholeslam. "Adaptive fuzzy wavelet network control design for nonlinear systems," Fuzzy Sets and systems Vol. 159, p.p. 2668-2695, 2008.

[9] E. Karatepe, M. Alci, "A new approach to fuzzy wavelet system modelling", International Journal of Approximate Reasoning," Vol. 40, No. 3, pp. 302-322, 2005.

[10] S. Srivastavaa, M. Singha, M. Hanmandlub, A. N. Jha, "New fuzzy wavelet neural networks for system identification and control," Applied Soft Computing, Vol. 6, pp. 1-17, 2005.

[11] R .H. Abiyev, O. Kaynak, "Fuzzy wavelet neural networks for identification and control of dynamic plants-A novel structure and a

)2

)1()(exp(25)(

22 −+−= kykukyπ

))20/cos()100/(sin(8)( kkku ππ +×=

Figure 2. Results of identification, where solid line denotes the output of the plant, dashed line in (a) denotes the FWNN output and in (b) denotes

the ANFIS output (ANFIS has 16 fuzzy rules)

Figure 4. (a) Error signal between the actual output and the FWNN's output, (b) Error signal between the actual plant and the ANFIS output with

25 fuzzy rules

comparative study," IEEE Transactions on Industrial Electronics, Vol. 55, No. 8, 2008. [12] T. Q. D. Khao, L. M. Phuong, P. T. T. Binh, N. T. H. Lien,

"Application of wavelet and neural network to Long-Term load forecasting," International Conference on Power System Technology, Powercon2004, Singapore, PP.840-844, 2004

Table I. Comparison of the proposed FWNN and ANFIS model

Figure 3. (a) Error signal between the actual output and the FWNN's output, (b) Error signal between the actual plant and the ANFIS output with

16 fuzzy rules

CPU Time Min Error

Number of Parameters

Number of Rules

2.93 (sec) 0.0235 18 2 FWNN

14.11 (sec) 0.0681 112 16 ANFIS(1)

30.844 (sec)

0.0627 175 25 ANFIS(2)