A New Fuzzy Neural Network with Trapezoidal Fuzzy Weights

Jee-Haeng Lee and Sung-Bae Cho

Computer Science Department, Yonsei University134 Shinchon-dong, Sudaemoon-ku, Seoul 120-749, Korea

Phone: +82-2-361-2720 Fax: +82-2-365-2579Email: [easygo, sbcho]@candy.yonsei.ac.kr

Key Word: fuzzy neural network, trapezoidal membership function, learning algorithm

AbstractFuzzy neural networks can be used to solve the

problems that have both fuzzy number and real number information. This paper proposes a new learning algorithm of fuzzy neural networks with trapezoidal fuzzy weights. We construct the trapezoidal fuzzy weights by composition of two triangles, and devise a learning algorithm using the two triangular membership functions. This model can learn a variety of fuzzy numbers efficiently. The results of computer simulations on numerical data show that the proposed fuzzy neural networks produce better results than the conventional methods.

1. Introduction

Neural networks have been widely applied to solve various problems. However, conventional neural networks can handle only numerical data represented by real number vectors. Therefore, it is very difficult to represent linguistic information from human experts and ambiguous information with conventional neural networks. To express linguistic and ambiguous information, fuzzy set theory has been used in various ways [1]. Since fuzzy neural networks can use fuzzy numbers as well as real numbers, and represent linguistic information better than standard neural networks, they can be applied to many problems. Various approaches to construct fuzzy neural networks have been proposed, but there are few architectures and learning algorithms that can be used as standard models [2].

This paper proposes a learning algorithm and architecture of fuzzy neural networks which utilize trapezoidal fuzzy numbers as weights. We construct a trapezoid as composition of two triangles, and devise a learning algorithm with the two triangular membership functions effectively. The main reason of using new weights is to solve the problems that have triangular and trapezoidal fuzzy numbers together for inputs and outputs. This paper shows new architecture and learning algorithm of the fuzzy neural network. We show

simulation results that compares different fuzzy neural networks for learning fuzzy numbers.

2. Fuzzy Neural Networks

2.1 Architecture

The input-output relation of a two-layer feed-forward neural network can be formulated as follows. This neural network has input, hidden, and output units. The fuzzy weights and fuzzy biases are trapezoidal fuzzy numbers, and the inputs and target outputs can be any shape of fuzzy numbers [3,4]. Fig. 1 shows the architecture of this fuzzified neural network.

Fig. 1. Architecture of fuzzy neural network.

Input units: ,

Hidden units: ,

nH

1

i

nI

j

1

nO

k

1

Input units Hidden units Output units

Fuzzy weights Wji

Fuzzy biases Fuzzy weights Wkj

Fuzzy biases

Output units: ,

where Xpi is a fuzzy input, Wji and Wkj are trapezoidal fuzzy weights, and j and k are trapezoidal fuzzy biases.

2.2 Fuzzy Numbers

The operations of fuzzy numbers can be defined by the extension principle of Zadeh. Our approach is based on the level sets of fuzzy numbers where arbitrary number of h can be used. The fuzzy outputs are numerically calculated by interval arithmetic for level sets of fuzzy weights and fuzzy inputs [1]. For example, Fig. 2 represents a fuzzy number denoted by 6 level sets where h is 0.0, 0.2, …, 1.0.

Fig. 2. Fuzzy numbers in fuzzy neural network.

3. Learning Algorithm

3.1 Cost Function

Let Tp be the nO dimensional fuzzy target output and Opk be the actual output. Then, the cost function for the kth output of level h is as follows.

Then cost function of h level sets is

Hence, we can write the cost function of (Xp, Tp) as

3.2 Derivation of Learning Algorithm

In this section we derive a learning algorithm of the neural network from the cost function described in the previous section. The fuzzy weights and biases are adjusted to minimize the value of cost function. Since the adjustment should not distort the trapezoidal shape of weights and biases, we do not change the h-level set of the fuzzy weights independently [3,4]. In this paper, we construct a trapezoid as composition of two symmetric triangles (shown in Fig. 3). We can update each trapezoidal fuzzy weights without destructing its shape by the movement of the two triangles and the change of their width. Then, let the trapezoidal fuzzy weights have six parameters as follows.

Fig. 3 shows the parameters of trapezoidal fuzzy weights.

Fig. 3. Trapezoidal fuzzy weights.

We derive a rule of adjustment of the weights for each parameter. According to the standard back-propagation algorithm, parameters are updated by the following formula, and the other weights are updated in the same way.

1

0

Mem

bership

Mem

bership

1

x0

h

where t is the index of number of iterations through learning, is positive real constant, and is positive real constant less than 1.0. Then the derivatives are written as follows.

and

where

.

In this way, the amount of adjustment is calculated. Finally, the weights can be updated as follows.

Since the two triangles in a trapezoidal fuzzy weight are symmetric, can be calculated easily.

and are modified in the same way. An

example for the adjustment of weights is illustrated in Fig. 4. All the weights and biases of the fuzzy neural network are modified in this way.

Fig. 4. Modification of weight.

When fuzzy neural network is updated, any weights should hold following constraints.

, for all ,

.If the inequalities are not satisfied after updated, the modification the two weights are swaped.

4. Simulation Results

Fuzzy neural networks can learn fuzzy numbers. We construct various fuzzy numbers that have shapes of triangle only, trapezoid only, and the mixture of both triangle and trapezoid. Every experiment has done on the same condition that learning rate is 0.5, momentum constant is 0.5, 10 hidden nodes and 6-level sets, 0.0, 0.2, …, 1.0. The neural networks have been learnedfor 3000 times to make a comparison with other models.

First experiment is to learn simple triangular fuzzy numbers. Let us consider next fuzzy if-then rules :

If x is small then y is small.If x is large then y is large.

Fig. 5(a) shows the membership functions of the fuzzy numbers, and Fig. 5(b) shows actual outputs.

(a) Membership functions.

1

h

0

1

h

0

(b) Actual outputs.Fig. 5. Learning triangular fuzzy numbers.

Next experiment is to learn simple triangular fuzzy numbers. Let us consider next fuzzy if-then rules :

If x is medium small then y is medium small.If x is medium large then y is medium large.

Fig. 6(a) shows membership functions of the fuzzy numbers, and Fig. 6(b) shows actual outputs.

Final experiment is to learn the mixed fuzzy numbers of triangle and trapzoid. Let us consider next fuzzy if-then rules :

If x is medium small then y is medium small.If x is medium large then y is medium large.If x is middle then y is middle.

Fig. 7(a) shows the membership functions of the fuzzy numbers, and Fig. 7(b) shows actual outputs.

(a) Membership functions.

(b) Actual outputs.Fig. 6. Learning trapezoidal fuzzy numbers.

(a) Membership functions.

(b) Actual outputs.Fig. 7. Learning mixed fuzzy numbers.

Regarding the modeling power, we cannot conclude that the proposed fuzzy neural network is better than other neural network with pure triangular and trapezoidal fuzzy weights. However, with respect to the mean-squared error, our model is better than other models. Table 1 shows the mean-squared errors after 3000 iterations. For further information, refer to our recent publication [5].

Types of weight

Shapes of fuzzy numbersfor inputs and outputs

Triangle Trapezoid MixtureTriangle 0.00083 0.00762 0.00860

Trapezoid 0.00103 0.00866 0.00969Proposed 0.00148 0.00656 0.00796

Table 1. Mean-squared errors for learningvarious shapes of fuzzy numbers.

5. Conclusions

In this paper, we propose a new fuzzy neural network with trapezoidal fuzzy weights which are made by two triangular fuzzy weights, and derive a learning algorithm for the neural network. With computer simulations, our model has high fitting ability for more complex fuzzy numbers.

Fuzzy neural network can learn both numerical and linguistic values represented by fuzzy numbers, so it can be applied to various problems where human knowledge can be provided. However, there are models of fuzzy neural networks regarded as standard models. A general architecture and algorithms should be reported with

various experiments in many environments.

References

[1] A. Kaufmann, M.M. Gupta, Introduction to Fuzzy Arithmetic: Theory and Applications, Van Nostrand Reinhold, New York, 1985.

[2] J.J. Buckley, and Y. Hayashi, “Fuzzy neural networks: A survey,” Fuzzy Sets and Systems, vol. 66, pp. 1-13, 1994.

[3] H. Ishibuchi, M. Morioka, and I.B. Turksen, “Learning by fuzzified neural networks,” Internat. J. Approximate Reasoning, vol. 13, pp. 327-358, 1995.

[4] H. Ishibuchi, K. Kwon, and H. Tanaka, “A learning algorithm of fuzzy neural networks with triangular fuzzy weights,” Fuzzy Sets and Systems, vol. 71, pp. 277-293, 1995.

[5] K. H. Lee, and S. B. Cho, “A learning algorithm of fuzzy neural networks with trapezoidal fuzzy weights,” Proc. of Asian Fuzzy Systems Symposium, Masan, Korea, June 1998.