IEEE SIGNAL PROCESSING LETTERS, VOL. 18, NO. 11, NOVEMBER 2011 659

An Adaptive Diffusion Augmented CLMSAlgorithm for Distributed Filteringof Noncircular Complex Signals

Yili Xia, Student Member, IEEE, Danilo P. Mandic, Senior Member, IEEE, and Ali H. Sayed, Fellow, IEEE

Abstract—An adaptive diffusion augmented complex least meansquare (D-ACLMS) algorithm for collaborative processing ofthe generality of complex signals over distributed networks isproposed. The algorithm enables the estimation of both secondorder circular (proper) and noncircular (improper) signals withina unified framework of augmented complex statistics. The anal-ysis shows that the performance advantage of the widely linearD-ACLMS over the strictly linear D-CLMS increases with thedegree of noncircularity while maintaining similar performancefor proper data. Simulations on both synthetic benchmark andreal world noncircular data support the approach.

Index Terms—Adaptive diffusion, distributed estimation, non-circular complex signals, widely linear modelling.

I. INTRODUCTION

A Number of modern applications, such as those arising inenvironmental monitoring, target localization, and sensor

networks, require new classes of estimation techniques, basedon distributed networks of nodes embedded with cooperativedata processing. In such cases, we can employ limited cooper-ation, incremental like techniques, or adaptive diffusion strate-gies to implement cooperation among individual adaptive nodes[1], [2]. These nodes are equipped with local learning capa-bilities: they produce local estimates for the parameter of in-terest and share information with only their neighbors. Whenlarger scale communication resources are available, distributedadaptive algorithms can be derived that exploit more fully thenetwork connectivity and increase the degree of cooperationamong nodes. Owing to their localized and real-time mode ofoperation, adaptive diffusion techniques offer enhanced robust-ness to link and node failures and are scalable.Standard complex-valued adaptive filtering algorithms are

normally considered generic extensions of the correspondingalgorithms in . However, consider a real-valued conditionalmean squared error (MSE) estimator , which esti-mates the signal in terms of another observation . For zeromean, jointly normal and , the optimal solution is the linear

Manuscript received July 06, 2011; revised August 24, 2011; acceptedSeptember 02, 2011. Date of publication September 15, 2011; date of currentversion September 26, 2011. The work of A. H. Sayed was supported in part byNSF Grants CCF 0942936 and 1011918. The associate editor coordinating thereview of this manuscript and approving it for publication was Dr. Hsiao-ChunWu.Y. Xia and D. P. Mandic are with the Department of Electrical and Electronic

Engineering, Imperial College London, London SW7 2BT, U.K. (e-mail: yili.xia06@imperial.ac.uk; d.mandic@imperial.ac.uk).A. H. Sayed is with the Department of Electrical Engineering, University of

California, Los Angeles, CA 90095 USA (e-mail: sayed@ee.ucla.edu).Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/LSP.2011.2168390

model given by , where is a vectorof fixed filter coefficients, the regressorvector, and the vector transpose operator.In the complex domain, since both the real and imagi-

nary parts of complex variables are real, we have, , and thus

, where theoperators and extract respectively the real and imagi-nary parts of a complex variable. Substituteand to arrive at

(1)

leading to the widely linear estimator for complex-valued data[3]–[5]

(2)

where and are complex-valued coefficient vectors. Such awidely linear estimator is optimal for the generality of complexsignals (both proper and improper), and it simplifies into thestrictly linear model for proper data. In practice, thewidely linear estimate in (2) is based on a regressor vector pro-duced by concatenating the input vector with its conjugate, to give an augmented input vector ,

together with the corresponding augmented coefficient vector. The corresponding augmented co-

variance matrix then becomes [6]

(3)and contains the full second order statistical information. Hence,in addition to the standard covariance matrix ,we also need to consider the pseudo-covariance matrix,

; processes with vanishing pseudo-covariance,, are termed second order circular (proper). The augmentedcovariance matrix in (3) considers the improperness of the ob-servation process in the widely linear estimation problem in(2). A more general solution where the estimate and observationare not jointly proper can be found in [7].The augmented complex statistics has opened the possibility

to design adaptive filtering algorithms based on widely linearmodels, suitable for the processing of both circular and non-circular signals. These algorithms are usually termed “widelylinear” or “augmented” algorithms, such as the WL-LMSalgorithm in communications [8], and the augmented CLMS(ACLMS) algorithm in adaptive filtering area [9], [10].In this letter, the Diffusion ACLMS (D-ACLMS) algorithm is

introduced in order to provide cooperative adaptive estimationof noncircular complex-valued signals, where the distributednodes cooperate for the estimation of the same global task. Theadvantage of the widely linear D-ACLMS over the strictly linear

1070-9908/$26.00 © 2011 IEEE

660 IEEE SIGNAL PROCESSING LETTERS, VOL. 18, NO. 11, NOVEMBER 2011

Fig. 1. Topology of a distributed network with nodes.

D-CLMS is illustrated over simulations on both benchmark andreal-world noncircular wind signals.

II. DERIVATION OF D-ACLMS ALGORITHM

Consider a network with distributed nodes, where eachnode has access to the local information at timeinstant , where is the desired signal, and is a re-gressor input vector of length , defined as

, both associated with node . As illustratedin Fig. 1, each node in the network has access only to theirpeer neighbors, and the diffusion protocols allow every nodeto combine information collected from its neighborhood ,which is the set of all the nodes linking to node , includingitself. By linearly combining the weight estimates in the neigh-borhood of node , and following the derivation in [2], we canmotivate replacing the local weights for standard and conjugateparts, that is and , with the weighted estimates

(4)

where the italic symbols with underscores denote the combinedweights vectors and at node for some combiningcoefficients and satisfying . Thisaggregate estimation at node can be interpreted as a weightedlinear combination which fuses information from nodes acrossthe network into node . The resulting aggregated weight vectors

and at node , can then be fed into its local adap-tive filter in order to respond to local information and updatethe local weights and . This way, the resulting co-operative adaptive network provides a peer-to-peer estimationframework that is robust to node and link failures and exploitsnetwork connectivity.In the context of widely linear adaptive filtering, the output

signal at node of a cooperative filter is given by

(5)

whereas the corresponding output error at the node is de-fined as . Based on the global cost func-tion we wish to optimize

, using thecalculus [11], the local weight updates for the standard part andconjugate part at each node can be derived as1

(6)

(7)

1Notice that the standard Cauchy–Riemann derivative of a real function ofcomplex variables, such as the error power in the cost function does not exist,however, the calculus shows that the steepest descent is in the direction ofthe conjugate weight vector, i.e., [11].

TABLE ICOMPARISON OF THE COMPUTATIONAL COMPLEXITIES OF THE PROPOSEDD-ACLMS AND THE STRICTLY LINEAR D-CLMS (PER ITERATION)

The computational complexities, in terms of real multiplica-tions and additions2, of the proposed D-ACLMS and the strictlylinear D-CLMS [1] are summarised in Table I, where is thenumber of the neighboring nodes of node .

III. STABILITY ANALYSIS OF D-ACLMS ALGORITHM

To provide insight to the convergence of the D-ACLMS al-gorithm, we shall next investigate the role of the degree of co-operation and network topology on the system performance. Tothat end, rewrite the local and global quantities as

in terms of the quantities that appear in (4), (6), and (7), and de-note by , a diagonal matrixcontaining the local step-sizes.To evaluate performance, without loss in generality, consider

an improper teaching signal , given by [5]

(8)

where and are respectively the unknown optimal localstandard and conjugate weight vectors, and the symboldenotes samples of doubly white3 noise with variance , statis-tically independent of the input sequence . The D-ACLMSalgorithm can now be written in a compact form

where is the transition matrix and isthe network connection matrix representing the networktopology: a nonzero entry means that node is connectedto node . Moreover, , whereis an column vector.The standard and conjugate weight error vectors for all the

nodes are given by

(9)

2Note that one complex multiplication requires four real multiplications andtwo real additions, while one complex additions requires two real additions.3The term “doubly white” refers to the white and mutually statistically inde-

pendent real and imaginary parts, with variance .

XIA et al.: ADAPTIVE DIFFUSION AUGMENTED CLMS ALGORITHM 661

where , , and their evolution isgoverned by

Therefore, the evolution of the augmented weight error vectoris described by

(10)

and can be written more compactly as

where .Under the assumption that the noise is statistically indepen-

dent of the regressor vectors and the independence among theregressor vectors over space and time, upon applying the statis-tical expectation operator on both sides of the above equation,the evolution of the mean weight error vector is governed by

(11)

where is block diagonal andis the augmented covariance matrix as de-

fined in (3). Hence, for the expression (11) to converge, thatis, , , for stability in themean , where are the eigenvalues for

and . Following the analysis in [1], itcan be shown that . In other words,the spectral radius of is generally smaller than that of ,and therefore cooperation under the diffusion protocol has a sta-bilizing effect on the network.

A. Performance Dependence on the Degree of Noncircularity

The strictly linear D-CLMS can be summarised as [1]

(12)

and the evolution of its weight error vector follows the samegeneric form as that in (11), with the standard covariance matrixreplacing the augmented covariancematrix. Since it does not ac-count for the pseudocovariance, it cannot explain the noncircu-larity (rotation dependent distribution) of input signals, and per-forms suboptimally for improper data. Locally, for every nodein the network, the analysis in [5], [10] shows that the advantageof the widely linear ACLMS over standard CLMS is more pro-nounced with higher degrees of noncircularity. Since the diffu-sion topology facilitates mutual communication between nodes,

Fig. 2. Network topology used in the simulations.

and each distributed node performs local adaptive estimation onthe same global task, the performance advantage of D-ACLMSover D-CLMS is also more pronounced when processing im-proper signals, whereas for proper signals their performancesare identical, as shown in the simulations.

IV. SIMULATIONS

The proposed D-ACLMS was compared with the standardD-CLMS in a one step ahead prediction setting of both properand improper signals. Out of the several methods to select thecombination coefficients , such as the Metropolis [1],Laplacian [12] and nearest neighbor rules [1], we used theMetropolis network topology described by (see Fig. 2)

ififfor

(13)where and are respectively the numbers of nodes in theneighborhood of nodes and . A network with nodeswas considered, with filter length , and the global step size

. Learning curves comparing D-CLMS andD-ACLMSwere produced by averaging 100 iterations of independent trials,performed on benchmark complex-valued stable circular andnoncircular signals. Single trial simulations were performed onreal-world noncircular wind signals4.The benchmark circular complex signal used in simulations

was a stable autogressive AR(4) process, with coefficient vectorand complex-valued doubly

white Gaussian driving noise , with zero mean and unitvariance [5]. The benchmark improper complex signal was anautoregressive moving average ARMA(4,1) complex process,which combined the improper MA(1) model in [13] with theabove stable AR(4) circular process, to give

(14)where , and the driving noisecovariance and pseudoco-variance , with the degree ofimproperness .Another benchmark noncircular signal considered was the

Ikeda signal (nonlinear and with coupled states), given by

(15)

4In , the wind vector can be represented as ,where denotes the speed and the direction.

662 IEEE SIGNAL PROCESSING LETTERS, VOL. 18, NO. 11, NOVEMBER 2011

TABLE IICOMPARISON OF DEGREES OF NONCIRCULARITY

FOR THE VARIOUS CLASSES OF SIGNALS

TABLE IIICOMPARISON OF PREDICTION GAINS FOR THE VARIOUS

CLASSES OF CIRCULAR AND NONCIRCULAR SIGNALS

where and .The real world noncircular wind signals were with different

dynamics, identified as regions high, medium or low based onthe changes in the wind intensity, for more detail see [5].For a quantitative measurement of the degree of non-

circularity of a complex vector , we used the index[14], where denotes

the matrix determinant operator; the degree of noncircularityis normalised to within with the value of zero indicatingperfect circularity. Table II illustrates the degrees of noncircu-larity for the signals considered.The standard prediction gain

was employed to assess the performance, where and anddenote respectively the variance of the input signal

and the forward prediction error which is the averagederror over all the nodes. Table III compares averaged predictiongains Rp (dB) over 100 independent trials for the D-CLMSand D-ACLMS trained cooperative networks. Observe thatfor the circular AR(4) signal, the performances of the stan-dard algorithms (noncooperative CLMS [15] and cooperativeD-CLMS) were similar to those of their widely linear counter-parts (ACLMS and D-ACLMS), whereas for the noncircularsignals, there was an improvement in the prediction gain whenthe widely linear algorithms were employed. In all the cases,due to the cooperative mode of operation, the diffusion algo-rithms outperformed their single node counterparts.Fig. 3 shows the evolution of mean-square errors (MSEs)

for the noncircular ARMA process and Ikeda map for all thealgorithms considered—observe that the diffusion algorithmsachieved both faster convergence and smaller MSEs than theirnoncooperative counterparts, and that the widely linear algo-rithms outperformed the strictly linear ones.

V. CONCLUSION

We have introduced a diffusion ACLMS (D-ACLMS) algo-rithm for distributed adaptive estimation of general complex-valued signals (both circular and noncircular) in a cooperativefashion. The advantage of the diffusion topology over nonco-operative structures in terms of convergence speed and smallerMSEs has been illustrated via analysis and simulations on bothsynthetic and real world noncircular complex signals.

Fig. 3. Comparison of MSEs of the noncooperative CLMS and ACLMS andcooperative D-CLMS and D-ACLMS algorithms on noncircular signals (a)Noncircular ARMA and (b) Ikeda map.

REFERENCES

[1] C. G. Lopes and A. H. Sayed, “Diffusion least-mean squares over adap-tive networks: Formulation and performance analysis,” IEEE Trans.Signal Process., vol. 56, no. 7, pp. 3122–3136, 2008.

[2] F. S. Cattivelli and A. H. Sayed, “Diffusion LMS strategies for dis-tributed estimation,” IEEE Trans. Signal Process., vol. 58, no. 3, pp.1035–1048, 2010.

[3] B. Picinbono and P. Chevalier, “Widely linear estimation with com-plex data,” IEEE Trans. Signal Process., vol. 43, no. 8, pp. 2030–2033,1995.

[4] T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation. UpperSaddle River, NJ: Prentice-Hall, 2000.

[5] D. P. Mandic and S. L. Goh, Complex Valued Nonlinear Adaptive Fil-ters: Noncircularity, Widely Linear and Neural Models. Hoboken,NJ: Wiley, 2009.

[6] P. J. Schreier and L. L. Scharf, “Second-order analysis of impropercomplex random vectors and process,” IEEE Trans. Signal Process.,vol. 51, no. 3, pp. 714–725, 2003.

[7] J. Navarro-Moreno, M. D. Estudillo-Martínez, R. M. Fernández-Al-calá, and J. C. Ruiz-Molina, “Estimation of improper complex-valuedrandom signals in colored noise by using the Hilbert space theory,”IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 2859–2867, 2009.

[8] R. Schober, W. H. Gerstacker, and L. H. J. Lampe, “Date-aided andblind stochastic gradients for widely linear MMSEMAI supression forDS-CDMA,” IEEE Trans. Signal Process., vol. 52, no. 3, pp. 746–756,2004.

[9] D. P. Mandic, S. Javidi, S. L. Goh, A. Kuh, and K. Aihara, “Complex-valued prediction of wind profile using augmented complex statistics,”Renewable Energy, vol. 34, no. 1, pp. 196–210, 2009.

[10] D. P. Mandic, Y. Xia, and S. C. Douglas, “Steady state analysis ofthe CLMS and augmented CLMS algorithms for noncircular complexsignals,” in Proc. 44th Asilomar Conf. Signals, Sys., Comp., 2010, pp.1635–1639.

[11] K. Kreutz-Delgado, “The complex gradient operator and the cal-culus,” Lecture Suppl. ECE275A, pp. 1–74, 2006.

[12] R. Bru, L. Elsner, and M. Neumann, “Convergence of infinite productsof matrices and inner-outer interation schemes,” Elec. Trans. Numer.Anal., vol. 2, pp. 183–193, 1994.

[13] J. Navarro-Moreno, “ARMA prediction of widely linear systems byusing the innovations algorithm,” IEEE Trans. Signal Process., vol.56, no. 7, pp. 3061–3068, 2008.

[14] P. J. Schreier, “Bounds on the degree of impropriety of complexrandom vectors,” IEEE Signal Process. Lett., vol. 15, pp. 190–193,2008.

[15] A. H. Sayed, Fundamentals of Adaptive Filtering. Hoboken, NJ:Wiley, 2003.