2469

Artificial Neural Networks for the generation and estimation of chaotic signals

A. Miiller and JMH. Elmirghani

School of Engineering University of NoriJnnnbria

Newcastle upon Tyne, NE1 8ST, UK

Abstract Dynamic feedback, inversion and LMS estimation have been established for the estimation of an information signal encoded onto a chaotic carrier. The poor resultant Smg of the mvered signal limits the applicability of these methods. Two novel chaotic codinglaecoding strategies based on artificial neural networh (ANN) and radial bgsis functions WF) have been developed and the resultant performance has been assessed. The results indicate that the nonlinear predictor (ANN-RBF-NLP) offers pdormance independent of the channel SNR (for S m l O dB) and offm 4 dB improved Smg compared to the LMS. Pseudo-chaotic sequences generated using an A" and estimated m a dynamic feedback manner (ANN-RBF-DF) have resulted in a sysrem with an Sm, that is linearly depndent on the channel SNR and offering for example U) dB improved Sl'&g compared to the LMS at a channel SNR of 40 dB.

Introduction

Chaotic coding (CC) has been proposed for communication mainly inspired by the results in [l]. Chaotic signals are weakly conelated and have a noise like power spectral density (PSD) and this makes them useful in several communication applications. kamples of communication applications that can benefit from this form of PSD include channel estimation and echo cancellation [2]. A wide variety of CC-methods based on discrete dynamical systems OS) have been investigated, e.g. modulation and demcdulation using the simple inversion method, least mean square (LIvlS) and recursive least square @LS)-algorithms [3], dynamic programming methods 141 and dynamic feedback @F) concept [SI. A drawback of all these methods is their poor performance in the presence of channel noise which has prevented CC from being seriously applied in Communifation sy;tems. Let {x,} be thejth coordinate of a flow generated by the dynamical system @S)

values oj E 6, = b~ ,..., s!] in a chaotic region ci xk = f ( x k - l , ~ o , . . . , ~ n ) , Which iS f01 m e w

chaotic, then for time series generation one state xk ,k = I,. ..,n of the flow is employed. CC-methods may be described as schemes of the form

= f [ Uk-17 .. ., = h(lk)? ... > o n = h ( l k ) ]

+ Sl(4 1 xk ='E+&('k) (1)

f:%"+W". g , , g,, k%+% where I , is the signal to be coded.

Chaotic Modulation

Consider, for example, the one dimwsional logistic map 0

Xk+I = j l x k (l - % k (2)

which is for the bifurcation parameter 0, =a€ Q = [3.7,4] chaotic. Choosing

a = a, = h ( i k ) , h(t) = 3.95+ 0.05~ g, = g, = o the signal dynamics are "modulated" with respect to I, so that the coding regime (1) reads more precisely as

Demodulation (estimation of a 2 for k ) of (3) may be achieved by the simple inversion [6]

(which is highly sensitive to additive observation noise) or by employing the tracking capability of the simple LMS algorithm [31

0-7803-4984-9/98/$10.00 01998 IEEE

2470

with & =3.95,0<2/VAR(x,(l-xk)) and the more complex RLS algaithm which is nearly equivalent to the LMS in this case due to the weak aurocoIIelation of {x, 1 171.

Dvnamic Feed Back

F i investigations on synchronised chaotic systems have led to coding regimes where

("chaotic masking") which are not applicable in communication. Investigations in [5] have led to the DF method a scheme with g z ( Z k ) r O and

~ ~ ( g , ( Z , ) ) ~ ~ ~ ( x , ) t o e n s u r e o p e r a t i o n i n Q, showing an SNR-improvement compared to chaotic masking. Using tho LM a DF-coding regime is given by

h( t )=cons t ,g , ( z , )~ O,RMS(gZ(Zk))<< RMS(&)

xk =x~~la(i-x~..l)+o.~, . (6)

with IZ,l c 1,a ='3.8. For damding the prediction error is used :

i, = io(x, - x,,n(l -xk-l 1). (7)

Artificial Neural Networks

In noise impaired real life applications dewding strategies merging robustness and accuracy have to be developed. Since ANN operate in finite dimensional spaces noise reduction methods based on ANNs were developed [SI. Applying a radial basis function (RBB ANN, noise reduction may be achieved by projecting the athactor of a DS on an n-dimensional manifold with a noise component (having an infinite degree of freedom) onto a linear subspace U(%") c ~'(3") spanned by

the RBFs of the ANN where m has to be c h m m accoTdance to Takem embedding themem 191: De 2 2n+ 1, De being the embedding dimension.

Two new chaotic dngkkccdhg methods have been developibasedonANN.

III the first methcd, applying an on-line aaining algorithm using the error

e: = (x,+, - (0.05& + 3.95)x, (1 - X, )r (8)

Fq. 1. RBF-ANN antfiguration for the LM where qi are the RBFs, wi the hidden layer weights and b, the biases

where m was chosen to be 3 with n=l for the logistic map. As an activation function for the hidden neurons Gausian RBFs

were employed where x, =(X,,...,X-*~+,) is the input, C, the center vector and z, is a symmetric norm weighting matrix so that, with linear output neurons having the activatim function fi (x) = w,xi, the output oisgivenas

247 1

FEg. 2.

- e

o 0.4 0.2 0.3 0.4 0.5 m.l;.e 0.7 0.8 0.9 1 1.1

Amactor of the LM (5000 points) rewmstructed by a RBF-ANN after Idtraining cycles

As trainiig algorithm for the new RBF-ANN-NLP

was used, where tl; is the derivative of 7,. with respect toitsargumens .&[k] ,ci[k] and w,[k]arethevalues at time step k. Obviously also the chaotic attractor may only be approximated mu(%"). In the second method, instead of projecting the original ath-actor we generate a pseudo chaotic time series using the local approximation of the DS based on a RBF-ANN (fRBFANN:%I" --f %) that has been trained off-line

with the original attractor. As coding strategy the DF scheme is now employed so that for the LM with .2.2-=3.8,m=3 and II,IC~g,(Z,)=0.31, and

employing a RBF-ANN as m Fig. 1 the resulting coding and decodiag regime for the RBF-ANN-DF scheme is

xk = f R B F . A " . D F ( x ~ - 1 ~ x ~ - ~ ~ x ~ - 3 )+0.31k (13)

The large amplitude value of 0.3 is possible because of the local approximation of the LM, i.e. though the LM is unimcdal x, > 1 and x, < 0 still result m a pseudo chaotic amactor. The attractor being pseudo chaotic due to the finite number of orbits. Since now the amactor is

reconsttuction of the dynamics at the receiver should be possible and noise reductiiw may be expected when noise is present The reconstructed amactor for I, = 0 of the RBF-ANN trained with the LM is depicted in Fig. 2.

&&&

The two newly developed methods, RBF-ANN-NLP and RBF-ANN-DF, are compared to the inversion, LMS and DF methods. To achieve objective and comparable tests, I,, consisted of NSOOO sample of a constant

Ik = 0.9 anda speech signal 1, = s, ,Is,1 I 1 obtained by 8 lcHz sampling from the sentence "very very nice"

2472

0 10 20 30 40 50 60 70

Fig. 3. Resulting SNR in dB when transmitting a constant signal with I , = 0.9 ChrmnelSNRiUdB

0 10 20 30 40 50 60 70

Fig. 4. Resulting SNR in dB when transmitting a speech signal with I I , IS 1 ChannelSNRhdB

for various channel SNR values of x,. The resulting signal SNR, for i, defined by

was evaluated. For the LMS, inversion and the RBF- ANN-NLP decoding methods, modulation of the LM is done by (3) at the transmitter and at the receiver (4), (5) and (9) are applied to decode the noise impaired receiver signal r, = x, + n, to get Ik where n, is a white noise process. When a DF model is employed, x, is generated ria (6) and decoded in accoIdimce to (7). Using a RBF-

..

(1.5)

2473

ANN-DF model, (13) is used for g e n d g {xk} and (14) for decoding after a RBF-A” has been with the LM (16 epochs).

Results

Results are depicted in Fig. 3 for a constant signal and in Fig. 4 for a speech signal. figs. 3 and 4 clearly i l lmate that the inversion methcd (4) is only applicable in the absence of noise whereas the RBF-ANN-W scheme is nearly independent of the channel SNR value provided the channel SNR is greater than about 10 dB. It achieves a SML,of 9 dB for It =0.9 andaSN%,of 5 dB for the speech signal. Employing the LMS results in an improved Smg compared to the REF-ANN-NLP method when demding a constant signal but when decoding a speech signal a maximal S%g =4 dB is reached due to the dynamics of Ik = sk . A characteristic of the DF methods is their linear dependence on the channel SNR (for the LM) together with their independence of the nature of Ik (constant vs. time- varying). Dependence on the RMS-value of Ik. is however manifested. Therefore, in Figs. 3 and 4 the Smg evolution is similar except that the actual SNF&ig values are different due to the different amplitudes of I t . Generating {x,} by (6) and decoding via (7) results in a linearly increasing me Moreover, if REF-ANN-DF is used, thus generating {xk} by (13) and decoding I , via (14). the DF results are outperformed by a Si‘&g improvement of 40 dB.

-

Condosions

Two novel chaotic Coding/decdng strategies based on artificial neural networks (ANN) and radial basis functions (RBF) have been developed and the resultant performance has been compared to s e v d established methods including dynamic feedback inversion and LMS estimation. Results obtained for the Mimodal logistic map with speefh signals indicate that ANN RBF based nonlinear predictors (A”-REF-“) offer performance independent of the channel SNR (for SNR >10 dB) and offer 4 dB improved Swig compared to the LMS. Psuedo-chaotic sequences generated using an ANN and estimated in a dynamic feedback manner (ANN-REF-DF) have resulted in a system with an Sm, that is linearly dependent on the channel SNR and offering for example U) dB improved S w i g compared to the LMS at a channel SNR of40 dE.

[41

I71

References

L.M. Pecora, T.L. C~IIOU : Synchronisation in chaotic systems, Physics Review Len, Vol. 64, 1990, pp. 821-824

JMH. Ehnkghani, S.H. Ulna, RA. Cryan : Expimental evaluation of echo path modeling with chaotic coded speech, IEEE Trans. Sig. Roc. Vol. 45 No 10, pp. 2600-2604, Oct. 1997

I€ Lemg, J. Lam : Receiver design for chaotic modulation system using adaptive filters, Roc. SPlE on Chaotic Circuits in Communication, 1995, Vol. 2612, pp. 126-135

S. Kay : Asymptotic maximum likelihccd estimator pe-rfonnance for chaotic signals in noise, IEEE Trans. on Signal Processing, Vol. 43, No.4, April 1995, pp. 1009-1012

J. B. Kadtke, J. B. Brush : Adaptive methods for

2038 Cham in Conmnmications, 1993, pp. 182- 193

chaotic cormnrmication sySxem.5, SPIE Vol.

J.M.H. Elmirghani, RA. Csyan : New chaotic based communication technique with multiuser provision, Electron. Lett., 1994, 30, (15), pp. 1206-1207

E. Eweda : Cornparision of RLS, LMS and sign algorithms for aacking randomly time-varying channels, IEEE Trans. on Signal Processing, Vol. 42, N0.11, November 1994, pp. 2937-2943

J.C. Principe, J M Kno : Noise reduction m state space using the focused gamma neural network, SPIE Vol. 2038 Cham m Communications, 1993, pp. 326332

F. Takens : On the numerical determination of the dimension of an amactor, Dynamical Systems and BifurcationsSpringer-Verlag, Lecture Notes in Mathematics, Vol. 1125, pp. 99-106