A Concise Functional Neural Network Computingthe Largest (Smallest) Eigenvalue and one

Corresponding Eigenvector of a Real SymmetricMatrix

Yiguang Liu, Zhisheng YouInstitute of Image & Graphics,

Sichuan University,Chengdu. 610064. China

E-mail: lygpapers(yahoo.com.cn

Abstract-Quick extraction of eignpairs of a real symmetricmatrix is very important in engineering. Using neuralnetworks to complete this operation is in a parallel mannerand can achieve high performance. So, this paper proposes avery concise functional neural network (FNN) to compute thelargest (or smallest) eigenvalue and one its eigenvector. Whenthe FNN is converted into a differential equation, thecomponent analytic solution of this equation is obtained. Usingthe component solution, the convergence properties are fullyanalyzed. On the basis of this FNN, the method that cancompute the largest (or smallest) eigenvalue and one itseigenvector whether the matrix is non-definite, positivedefinite or negative definite is designed. Finally, threeexamples show the validity of the method. Comparing withother neural networks designed for the same aim, theproposed FNN is very simple and concise, so it is very easy tobe realized.

I. INTRODUCTION

Using neural networks to compute eigenvalues andeigenvectors of a matrix has parallel and quick features,quick computation of eigenvalues and eigenvectors hasmany applications such as primary component analysis(PCA) for real time image compression, or adaptive signalprocessing etc. So, many research works about this fieldhave been reported [1-14]. Recently, a FNN model isproposed to compute eigenvalues and eigenvectors of a realsymmetric matrix [13]. It is as follow

dx(t) = Ax(t)-xT(t)x(t)x(t), (1)dt

Liping CaoSichuan University Library,

Sichuan University,Chengdu. 610064. China

()=Ax(t)-sgn[x (t-(t)X(t) (2)dit

where definitions of A, t and x(t) are same to those offormula (1), sgn[xi (t)] retums 1 if xi (t) > O , 0 if

x, (t) = 0 and -I if x, (t) < 0 . Let I denote a suitable

dimensional identity matrix. When A - Isgn[xT(t) x(t)is looked as synaptic connection weights which obviouslyare functions of x(t), the neuron activation functions areassumed as pure linear functions, formula (2) describes afunctional neural network (FNN (2) ). Evidently, FNN (2) issimpler than FNN (1), this feature is important for electronicdesign.

II. SOME PRELIMINARIES

Since matrix A is real symmetrical, all eigenvalues aredenoted as 2~ 2.22. ... >2An I correspondingeigenvectors and eigensubspaces are denoted as Pl....,* Pnand V1,---,Vn . Assume matrix A has m < n different

eigenvalues, and denote them as al > a2 > .. > am' let

Ki (i = 1,...,m) denote the sum of algebraic multiplicityof cr .. ,cri and let Ko =0 , AO = 0, obviously

Km = n. Let

where t20 x=(xi, ,xn)TeRn represents the

tat f nern,Mti A ERR"nxn .states ofnuos arxAeRfXlrequiring eigenpaircalculation is symmetric. This paper gives an improvedform of this model described as

ui -VKil+l D --@-VKi ( = 1,...,m), (3)where ' (D ' denotes the direct sum operator. Obviously

U, is the eigensubspace corresponding to vi.Lemma 1: l +a 222+a 2*.*2>An +a are

0-7803-9422-4/05/$20.00 ©2005 IEEE1334

eigenvalues of A+ al (aE R) and p, ... p, are the

eigenvectors corresponding to .+.a,... , + a.Proof: From

Apui = ipui i = 1,*--,n .

It follows thatApui + ap, Aipji + apli, i =1, *.*.* n

i.e.

(A + aI)pi =(Ai + a),u,, i =1, * * n .

Therefore, this lemma is correct. This completes the proof.Let 4 denote the equilibrium vector of FNN (2). 4

ought to exist, because 4 = 0 is an invariable equilibriumvector. When 4 exists, there exists

4=lm x(t) (4)and

nA4 = |{I4I4when 4 . 0, 4 is an eigenvector and the corresponding

eigenvalue 2 isn

i= (5)

HI. COMPONENT ANALYTIC SOLUTION

Theorem 1: Denote Si - xi(t) denotes the

projection value of x(t) onto Si, then the analytic solution

ofFNN (2) x, (t) can be presented as

xi(t)= + xi(0)exp(ijt) (6)1 + xi (o jexp(2141T

j=lProof: Using similar approach in [13], we can easily

prove this theorem.

IV. CONVERGENCE ANALYSIS

Using component analytic solution formula (6) and thesimilar proving ways in [13], we can obtain the followingtheorems about convergence ofFNN (2).

Theorem 2: If x(0).0O and x(0) E V,, then 4=0 orn

4E Vi.If4E Vi,then 2, =Z|JkI.k=l

Theorem 3: If x(0) .0, x(0) Vi (i =1, ,n), and

n

4.0,then 4EU1, 2 =ZI4k .k=1

Theorem 4: When A is replaced with -A, If x(O) . 0,x(O) Vi (i=1,..., n) and 4 0 , then 4E Um

n

in E |kk=1

Theorem 5: Whether the sign of A is reversed, FNN (2)can not directly compute 2n of positive definite matrix and

il ofnegative definite matrix.

Theorem 6: Let IA2 1 denote the minimal integer not less

than 2 . For positive definite matrix A, if x(O) .0 and

x(O) 0 Vi (i = 1, **, n) , replacing A with

-(A-F2 jI) makes 4 0 andn

An=-Z I +F21Ii=l

(7)

Theorem 7: Let L2J denote the maximal integer not

more than 2n . With negative definite matrix A , if

x(0).0 and x(O) 0 Vi (i = 1, ,n), replacing A with

(A-L2n JI) will make 4 0 andn

Al =Z@i+L2nIj (8)i=l

Theorem 8: For FNN (2), there must have4 = 0 or thatn

Z I equal to one positive eigenvalue..i=1

V. STEPS TO COMPUTE Al, .., An

In this section, steps are designed to compute the largesteigenvalue and the smallest eigenvalue of a real symmetricmatrix B E R "xn with FNN (2), the steps are:

Step 1: When A is replaced with B and 4 . 0,n

then Al=E4ji=1

Step 2: When A is replaced with -B and 4 0,n

we get 2n =-Z 4,i If il has been obtained, goto step 5. Otherwise, go to step 3. If 4 = 0, go tostep 4.

Step 3: Through replacing A with (B -L2,n jI), we

1335

nget Al = l,I+Li and go to step 5.

Step 4: Through replacing A with - (B-F1),n

we get in--g + [A- I and go to step 5.i=l

Step 5: End.

VI. SIUMJLATIONS

We give three examples to test the validity of the methodfor computing the smallest eignevalue and the largesteigenvalue of a real symmetric matrix. The simulatingplatform is Matlab.

Convergence of modulus of components of x(t)0.71

Ea)

a)0

CL

E3

0

1-

S

0.6

0.5

0.4

0.3

0.2 -

0.1 .

O, . .U 0 11. 1U0 20Number of Iterations

Fig. 1. The trajectories of components ofx whenA is replaced with B

Convergence of)>(t)l

-x!-w

4

3.5 -

3

2.5

2-

1.5

0 50 100 150 200

Convergence of modulus of components of x(t)0.35

e-l- 0.3x 0.

0

C 0.21c

0)

a

"c 0.2E

0.14)

10 0. 1

E 0.05

0-0 50 100Number of Iterations

150 200

Fig. 3. The trajectories of components ofx whenA is replaced with -B

Convergence of-ENA(t)l0

-0.2

-0.4

-0.6

-0.86

-1.2

-1.40 50 100 150 200

a

Fig. 4. The trajectories of An(t)=- |xi(twhen A is Replaced with -B.

Example 1: A real '

generated as0.5972 0.38580.3858 0.70150.8341 0.2719

B = 0.7884 0.68830.5024 0.41960.3249 0.20610.5769 0.6175

7 x 7 symmetric matrix is randomly

0.83410.27190.95660.39360.74040.67840.3759

0.78840.68830.39360.24730.21190.34700.4511

0.50240.41960.74040.21190.31430.53660.2305

0.32490.20610.67840.34700.53660.54140.6682

0.57690.61750.37590.45110.23050.66820.5030

n

Fig. 2. The trajectories of 2& (t) = E lxi (t wheni=1

A is replaced with B

Through step 1, we get= (0.5851 0.4572 0.6304 0.4575 0.4404 0.4783 0.4893)T

nSo, i = |jiJ =3.5382 . The trajectories of xi(t)

n(i=1,2, * ,7) and (t) =yjExi (t| are shown in Fig. 1

i=l

and Fig. 2. From Fig. 1 and 2, we can see that FNN (2)

1336

quickly reaches equilibrium state and the largest eigenvalueis obtained soon. Through step 2, we get= (-0.1301 -0.0931 0.0269 0.1453 0.0858 -0.0916 0.0848)

n

and 2n = =-0.6576. The trajectories of x,(t)i=1

n

(i = 1,2,* * *,7) and n (t) =- xi (ti are shown in Fig.3 and Fig. 4. Using mathlab to directly compute theeigenvalues of B gives the true largest eigenvalue4 = 3.5379 and smallest eigenvalue i',n = -0.6576.Therefore, the results computed by the method are very

close to the true values, the absolute difference values are

- =13.5382 - 3.53791 = 0.0003,and

An = - 0.6576 - (- 0.6576 = 0.0000.This verifies that using this method to compute the largest

eigenvalue and the smallest eigenvalue of B is successful.Example 2: A positive matrix is

( 0.6495 0.5538 0.5519 0.5254'0.5538 0.8607 0.7991 0.55830.5519 0.7991 0.8863 0.45080.5254 0.5583 0.4508 0.6775)

Convergence of modulus of components of x(t)AH

s 0.7x

0

,. 0.6c

o 0.5Eo 0.4

0.3co

= 0.20oE 0.1,

U,50 100

Number of Iterations150 200

Fig. 5. The trajectories of components of x whenA is replaced with C

Convergence of EIN(t)l3

2.5

2

1.5 D

0.50 50 100 150 20(

Fig. 6. The trajectories of

Replaced with C.

10n

Ixi(t when A isi=l

The true eigenvalues of C computed by Matlab arei'= (0.0420 0.1496 0.3664 2.5160). From step1, we can get 2 = 2.5160 , the trajectories of the

n

components of x(t) and E xi (t which will approach toi=1

are shown in Fig. 5 and 6. Using step 2, the trajectoriesn

of components of x(t) and 2in (t)=-g xi (tI are showni=l

in Fig. 7 and; = l.Oe - 5 * (0.1229

8. The equilibrium vector0.2827 - 0.2183 - 0.1342)T

n

and 2n =-|Il=-7.5811e-6 . Obviously, An isi=1

very close to zero. So, step 4 is used to compute the smallesteigenvalue. Replacing A with

c=-(C-F, I)=- 2.3505 0.5538 0.5519 0.52540.5538 -2.1393 0.7991 0.55830.5519 0.7991 -2.1137 0.45080.5254 0.5583 0.4508 -2.3225)

nthe trajectories of components of x(t) and -Zx(44

i=l

which will approach to the smallest eigenvalue of C' areshown in Fig. 9 and 10. In the end, we get the equilibriumvector s =(0.4168 1.0254 -0.9303 -0.5854)T and thesmall eigenvalue of C

1337

I

Convergence of modulus of components of x(t)

Number of Iterations

Fig. 7. The trajectories of components ofx whenA is replaced with -C

in = - i*1+FAI1= -2.9579 + 3= 0.0421

=2.5160 and A* = 0.0421 are the resultscomputed by FNN (2), comparing with the true values2 = 2.5160 and An = 0.0420, we can see that A1 is

very close to 2i, the absolute difference is 0.0000, and

A* is very near to 2n,A the absolute difference is 0.0001.This verifies that using step 4 to compute the smallest

eigenvalue of a real positive symmetric matrix is valid.10

Convergence of-ZII>(t)l

D 50 100 150 200

Fig. 8. The trajectories ofn

- |xi (tA when A isi=l

Replaced with -C.Convergence of modulus of components of x(t)

Number of Iterations

Fig. 9. The trajectories of components ofx whenA is replaced with C'

Convergence of -EZI(t)l

-1

-1.5-

-2

-2.5

0 50 100 150 201

Fig. 10. The trajectories

is Replaced with C'.

10n

of -Z xi (t when Ai=l

Example 3: Matrix D = -C is used to test the validity ofFNN (2).Obviously, D is negative defimite and its true eigenvalues

are 2'= (- 2.5160 - 0.3664 - 0.1496 - 0.0420) .Replacing A with D, from step 1, the obtained eigenvalueis 7.5811 le- 6, and we cannot decide whether this value isthe true largest eigenvalue. Through step 2, the smallesteigenvalue is obtained 24 = -2.5160. Using step 3, we

get 2l =-0.0421. Comparing 24 and A to 24 and 2;,respectfully, it can be found that using step 3 to compute thelargest eigenvalue is nice.x(0) of example 1 is

[0.2892 0.1175 0.0748 0.2304 0.2502 0.2001 0.2275]Tand x(0) of example 2 and 3 is

[0.2892 0.1175 0.0748 0.2304]T. Repeating thethree examples with the other initial values, the obtainedresults are all very close to the corresponding true values.

VI. CONCLUSION

1338

0.:

ol~ 0.a

0

0

-3.m

1.3-25 -

1.2

'5

0 50 100 150 20(

0.1

0

-0.1

-0.2

-0.3xLw -0.4

-0.5

-0.6

-0.7

-0.8

x

0

-3

c

a)c

-E0a

-00

'k.r

ri

This paper proposes a very concise functional neuralnetwork presented by a differential dynamical system, usingthe component analytic solution of this system gives theconvergence behavior. This network is adaptive to extractthe largest positive eigenvalue or the smallest negativeeigenvalue. Detail steps are designed to compute the largesteigenvalue and smallest eigenvalue of a real symmetricmatrix using this neural network. This method is adaptive tonon-definite, positive definite, or negative definite matrix.Three examples show the validity of this method, B is anon-definite, C is positive definite and D is negativedefinite, the results obtained by this method are all veryclose to the corresponding true values. Comparing withother approach based on neural networks, this finctionalneural network is very concise and can be realized easily.

REFERENCES

[1] N. Li, A Matrix Inverse Eigenvalue Problem and Its Application, LinearAlgebra and Its Applications. Vol. 266, pp.143-152, Nov. 1997.

[2] F.-L. Luo, R. Unbehauen and A. Cichocki, A Minor ComponentAnalysis Algorithm, Neural Networks, vol. 10, Issue 2, pp. 291-297,Mar. 1997.

[3] C. Ziegaus, E.W. Lang, A Neural Implementation of the JADEAlgorithm (nJADE) Using Higher-order Neurons, Neurocomputing,vol. 56, pp. 79-100, Jan. 2004.

[4] F.-L. Luo, R. Unbehauen, Y.-D. Li, A Principal Component AnalysisAlgorithm with Invariant Norm, Neurocomputing, vol.8, Issue 2, pp.213-221, Jul. 1995.

[5] J. Song, Y. Yam. Complex Recurrent Neural Network for Computingthe Inverse and Pseudo-inverse of the Complex Matrix, AppliedMathematics and Computing, vol. 93, Issues 2-3, pp. 195-205, Jul.1998.

[6] H. Kakeya and T. Kindo, Eigenspace Separation of AutocorrelationMemory Matricesfor Capacity Expansion, Neural Networks, vol. 10,Issue 5, pp. 833-843, Jul. 1997.

[7] M. Kobayashi, G. Dupret, 0. King and H. Samukawa, Estimation ofSingular Values of Very Large Matrices Using Random Sampling.Computers and Mathematics with Applications, Vol. 42, Issues 10-11,pp. 1331-1352, Nov.-Dec. 2001.

[8] Y. Zhang, F. Nan and J. Hua, A Neural Networks Based ApproachComputing Eigenvectors and Eigenvalues Of Symmetric Matrix,Computers and Mathematics with Applications, vol. 47, Issues 8-9, pp.1155-1164, Apr.-May 2004.

[9] F.-L. Luo, Y.-D. Li, Real-time Neural Computation of the EigenvectorCorresponding to the Largest Eigenvalue of Positive Matrix,Neurocomputing, Vol. 7, Issue 2, pp. 145-157, March 1995

[10] V. U. Reddy, G. Mathew and A. Paulraj, Some Algorithms forEigensubspace Estimation, Digital Signal Processing, Volume 5,Issue 2, pp. 97-115, April 1995.

[11] R. Perfetti and E. Massarelli, Training Spatially Homogeneous FullyRecurrent Neural Networks in Eigenvalue Space, Neural Networks,vol. 10, Issue 1, 125-137, Jan. 1997.

[12] Y. Liu, Z. You, L. Cao and X. Jiang, A Neural Network Algorithm forComputing Matrix Eigenvalues and Eigenvectors, Journal ofSofiware, vol. 16, pp. 1064-1072, Jun. 2005.

[13] Y. Liu, Z. You and L. Cao, A Simple Functional Neural NetworkforComputing the Largest and Smallest Eigenvalues and CorrespondingEigenvectors of a Real Symmetric Matrix, Neurocomputing. vol. 67,pp 369-383, Aug. 2005.

[14] Y. Liu, Z. You and L. Cao, A Functional Neural Network forComputing the Largest Modulus Eigenvalues and TheirCorresponding Eigenvectors of an Anti-symmetric Matrix,Neurocomputing. vol 67, pp 384-397, Aug. 2005.

1339