Dsp 2

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

105

Transform domain robust Variable Step size Griffiths

Adaptive Algorithm for System Identification S.V. Narasimhan

1 and S. Roopa

2

1Digital signal processing and systems Group

Aerospace Electronics & Systems Division

National Aerospace Laboratories, Bangalore-560017,

(Council of Scientific & Industrial Research (CSIR), New Delhi)

E-mail: [email protected]

[email protected]

2Department of E.C.E.,

2B.M.S College of Engineering, Bnagalore-560018

Abstract— A transform domain robust variable

step size Griffiths’ LMS algorithm (TVGLMS) is proposed

for system identification. For the TVGLMS, the robust

variable step size has been achieved by using the Griffiths’

gradient which uses cross-correlation between the desired

signal contaminated with observation noise and the input

and the discrete cosine transform (DCT). The presentation

considers both cases with and without desired signal

decomposition by the DCT. The proposed algorithm is

found to be having better convergence error/

misadjustment ( by 10 dB for colour and white observation

noise) compared to that of ordinary transform domain

LMS (TLMS) algorithm, both in the presence of white/

colored observation noise. The reduction in convergence

error achieved by the new algorithm with desired signal

decomposition is found to lower than that by without

decomposition. (By 8 dB for colour and 3 dB for white

observation noise)

Keywords: Robust Transform domain LMS algorithm, DCT

Griffiths’ LMS algorithm,, Variable step size LMS algorithm .

1. INTRODUCTION

The LMS adaptive algorithm due to its simplicity is widely

used for many applications like system identification, channel

equalization, echo cancellation, noise removal, adaptive

coding/compression and active noise control. However the

LMS algorithm, suffers from slow convergence rate when the

eigenvalue spread of the input autocorrelation matrix is large.

To over come this, algorithms are proposed which

orthogonalize the input bringing down the eigenvalue spread.

In this direction transform domain approach is quite popular.

In this, the orthogonalization of the input by the transform

aligns the weight vector with the principal eigenvectors of the

error surface there by removing the cross coupling which

exists among the weights involved in adaptation and further

the scaling of these orthogonal components results in

reduction in eigenvalue spread. basically the orthogonalization

enables to use different step sizes for the adaptive filters of

different components of the input depending on their power

(which is same as scaling). in view of this, the stepsize used

for different components will be significantly large than that

used for the transversal LMS algorithm as it uses the total

power of the input for stepsize normalization in the

normalized LMS algorithm (NLMS).though the use of larger

stepsize for different components results in faster convergence

rate., it also results in higher convergence error/

misadjustment. Hence to have smaller misadjustment, it is

necessity to reduce large values of the stepsize for different

components as the convergence is approached. Therefore to

achieve both fast convergence rate and low misadjustment, a

variable step-size LMS (VSS) algorithm is required. In VSS,

the step-size is varied from a large to the small value [1, 3, 4-

6, 7] as the convergence is approached.. In [9], the step-size is

made as a function of the error energy, however in the

presence of observation noise this will result in larger step-

size. In [1], step-size is made proportional to the

autocorrelation of the error and this is valid only for the white

observation noise. The correlation LMS [6], uses cross-

correlation between input and error and this suffers from

negative step-size and adaptation stalling. In Okello’s [5]

VSS, the step-size is a function of the sum of the squared

cross-correlations between the error and the delayed inputs

corresponding to the weights. This is free from the problems

of [6] but may be sluggish for non stationary signal. In the

recent algorithm by Zhang [7], step-size is made proportional

to the squared norm of the smoothed gradient vector. All

these algorithms attempt to make the step-size robust to

observation noise, but not the gradient for the weight vector

adaptation. A variable step size Griffiths LMS algorithm

(VGLMS) which not only uses the step size but also gradient

for weight vector which are robust to observation noise has

been proposed [8]. The VGLMS achieves this by using the

cross-correlation between the desired signal and the input.

However, the VGLMS algorithm for a given maximum step

size, has a slower convergence rate, due to replacement of

instantaneous correlation between input and the error by the

Griffiths averaged gradient. This motivates to apply VGLMS

approach to algorithms which have faster convergence rate

for inputs with large eigenvalue spread. Hence, it is of interest

to extend this algorithm to transform domain adaptation as this

will not only provide additional convergence speed over the

variable step size but also a smaller misadjustment. A

transform domain Griffiths’ algorithm has been applied for


106

adaptive blind demodulation of DS/CDMA signals [9].

However this is very specific to the application and cannot be

use for system identification type of applications.

In this paper, a transform domain robust variable

step-size LMS (TVGLMS) algorithm has been proposed.. In

the TVGLMS, the robust variable step size and the robust

gradient are achieved by using the cross-correlation between

the desired signal and different transform components of the

input. In this, as a special case, use of transformed

components of the desired signal instead of complete desired

signal is also considered. The performance of the proposed

algorithm is found to be better for both white and colored

observation noise situations even at low SNR. Use of

transformation for the desired signal has better performance as

it reduces the observation noise for the individual components

are reduced.

II. BACKGROUND

In this section the background material required viz., the

system identification and the Griffiths’ variable step size

algorithm will be briefly reviews for the purpose of reference

and notational convenience.

A. System identification

Consider a system )(zP with impulse response

[ ])1()1()0()( −= MpppnP � and with white noise

input [ ]TMnxnxnxnX )1()1()()( +−−= � . For this, the

system output ∑−

=

−=′1

0

)()()(

M

k

knxkpnd and the observation

noise )n(o corrupts it to

give a desired signal )n(o)n(d)n(d +′= .The

normalized LMS (NLMS) adaptation rule for this is given by

)()()(

)()()1( knxne

nME

nnwnw

xkk −+=+

µ (1a)

)()1()1()( 2nxnEnE xxxx ββ −+−= , 10 << xβ

)()()(

0

knxnwny

M

k

k −=∑=

)(nEx is power of the input computed

recursively, )(ny is the adaptive filter output and )(nµ is the

adaptation step size.

Since )()()( nyndne −= and )()()( nondnd +′= ,,

equation .(1a) becomes

[ ]

)()()(

)(

)()()()(

)()()1(

knxnonME

n

knxnyndnME

nnwnw

x

xkk

−µ

+−−′µ

+=+ (1b)

In Equation. (1c), the presence of last term in is not desirable

as only the expectation of the product,. { } 0)()( =− knxnoE

Since )(no and )(nx are uncorrelated to each other. However,

the instantaneous value of the product, { } 0)()( ≠− knxno .

Hence in the LMS algorithm as it uses only instantaneous

values of the variables involved, last term due to the

observation noise )(no will disturb the convergence process

and even it make the adaptation to diverge. Hence the

suppression of the observation noise for the LMS adaptation is

essential. Further, a large step size results in larger

convergence error /misadjustment, hence it is required to have

a larger step size prior to convergence and a smaller step size

after convergence.

If the observation noise )(no is not there or if it is of

small magnitude compared to plant output )(nd ′ , after many

iterations )(nW will converge to )(nP and will be a good

estimate of )(nP for a whit noise input )(nx . .However if

)(nx is not a white noise but colored characterized by

fundamental and its harmonics, the system )(zP will be

identified only at those frequencies. In view of this,

)(nW will be different from )(nP and )(zW will match )(zP

only at the frequencies of the input. For such inputs the

convergence of )(nW to )(nP will be slow due to large

spectral range / eigenvalue spread. In such cases, it is required

to orthogonalize the input to bring down the Eigen value

spread using a orthogonal transform.

B. Griffiths’ variable step size algorithm (VGLMS)

To remove the effect of observation noise on the

adaptation process, Griffiths modified the NLMS algorithm. If

[ ] { }[ ])()()()()()(G knxnondEknxndEnk −+′=−= ,

Then )(G nk , 1,,0 −= Mk … , represents the cross-correlation

between the desired signal and the input. Further as )(nx and

)(no are independent,

[ ] 0)()( =− knxnoE .

Therefore, )],()([)( knxndEnk −′=G 1,,0 −= Mk …

and [ ] [ ])()()()()( knxnyEnGknxneE k −−=− .

If [ ])()()()( knxnynnQ kk −−= G , 1,,0 −= Mk …

)(nQk is referred to as Griffiths’ cross-correlation and is free

from the effect of observation noise component, )(no .

Fig. 1 System

Identification

++

−)n(e

)n(x

)n(y)z(W

+)z(P

)n(O

+

)n(d

)n(d ′

+


107

Therefore the LMS adaptation algorithm Equation n.(1a) with

Griffiths cross-correlation is [4]

)(

)()()()1(

nME

nQnnwnw

x

kkk

µ+=+ (2a)

Okello [10] proposed an algorithm which provides a

variable step-size based on the cross-correlation )(nU k ,

between input and the error.

[ ] ( )[ ])()()()()()( knxnyndEknxneEnU k −−=−= (3)

Since 0)]()([ =− knxnoE , )(nU k is free from observation

noise. In this, to avoid negative step-size, the variable step-

size )(nµ is given by [10]

∑−

=

+−=1

0

2 )()1()(

M

k

k nUnµαnµ γ ( ) 1,0 << γα (4)

. The expectation operation in )(nU k , reduces the effect of

observation noise on step size. Further, 0)(

1

0

2 ≥∑−

=

M

k

k nU in

Equation. (4), ensures positive step-size and the presence of

)1( −nµα avoids stalling [1]. This being a NLMS algorithm,

for stability the initial value for )(nµ is set close to unity [7]

and the lower limit for )(nµ is fixed to prevent adaptation

stalling and to provide minimum misadjustment tolerable.

The Griffiths’ and Okello’s algorithms are combined

to realize VGLMS algorithm []. In the VGLMS algorithm,

)(nU k in Okello’s algorithm is replaced by the Griffiths’

cross correlation [ ])()()()( knxnynnQ kk −−= G . The

expectation operation for )()( knxny − is removed due to use

of instantaneous quantities in LMS algorithm and also as

observation noise is not directly present in this term. It is

important to note that as )(ny is a time varying quantity and

averaging of this quantity in )(nQk makes the adaptation

process sluggish, justifying the use of )(nQk over )(nU k .

The VGLMS rule for the individual coefficients is

[ ])()()(ˆ)(

)()()1( knxnynG

nME

nnwnw k

xkk −−+=+

µ (5a)

[ ]∑−

=

−−+=+1

0

2)()()(ˆ)()1(

M

k

k knxnynGnn σµαµ (5b)

( ) 10),()(1)(ˆ)1(ˆ <<−−+=+ ggkgk knxndnGnG βββ (5c)

Here gβ decides the cut-off frequency and hence the degree

of rejection of the observation noise. If

[ ])()()(ˆ)(

1)( knxnynG

nMEn k

vk −−=∇ ,

Then Eqn. (5a) can be expressed as

)n()n()n(w)n(w kkk ∇+=+ µ1 (5d)

The second term of Eqn..5(d) is the product of )(nµ and the

weight vector adaptation gradient )(n∇ . The VGLMS

algorithm uses both the noise step size and noise free gradient

whereas other variable step size algorithms use only the noise

free step size.

III PROPOSED TRANSFORM DOMAIN VARIABLE

STEP SIZE GRIFFITHS LMS ALGORITHM

(TVGLMS)

A TVGLMS without desired signal decomposition

The basic transform domain LMS algorithm using

discrete cosine transform is shown in Fig. 2. For the input

)(nx , the M2 point DCT of produces orthogonal

components 1,,0),( −= MknX k … (the remaining M points

are symmetrical) and these form inputs to single coefficient

adaptive filters 1,,0),( −= MknWk … . The output of these

adaptive filters 1,,0),( −= Mknyk … are summed (with a

gain factor of 2) to take care of symmetrical components) on

sample to sample basis to get the estimate )(ny of the desired

signal )n(d . The error signal )()()( nyndne −= is used for

the adaptation of the weights 1,,0),( −= MknWk … using

the normalized LMS algorithm as

)()()(

)(2)()1(

2nXne

n

nnwnw k

k

kkσ

µ+=+ ,

1,2,1,0 −= Mk … (6)

)()1()1()(222

nXnn kkk ββσσ −+−= 10 << β

)(nµ is the step size used for the adaptation. )(2 nkσ is the

input power to adaptive filter 1,,0),( −= Mknwk … and is

+

)n(X1

)(1 nw

)(nx

)(nd ′

)n(o

)n(y0

)(nd

+

_)(ne

+

)n(w2

)n(wM 1−

)n(X1

)n(XM 1−

)n(y1

)n(yM 1−

)z(PD

CT

)n(y

+

+

+

+

Fig.2 DCT transform domain LMS adaptive filter for

system identification


108

estimated recursively. The transform domain provides faster

convergence compared to transversal filter since

)()(2

1

0

2nn k

M

k

k σσ ∑−

=

<< and the gradient step size will be

relatively large for the former. However due to large step size

the convergence error o the transform domain LMS can be

larger than that for transversal filter. Since )()()( nyndne −=

Equation. (6) Can be written as

[ ] )()()()(

)(2)()1(

2nXnynd

n

nnwnw k

k

kk −+=+σ

µ

Further as

)()()( nondnd +′= ,

[ ] )()()(

2)()()(

)(

)(2)()1(

22nXno

nnXnynd

n

nnwnw k

k

k

k

kkσ

µ

σ

µ+−′+=+ (7)

The adaptation for the weight )(nwk gets affected by the last

term on the right hand side of Equation (7). Applying the

Griffiths algorithm to this case,

[ ])()()()(

)(2)()1(

2nXnynG

n

nnwnw kk

k

kk −+=+σ

µ (8)

Where

1,,0)],()([)]()([

)]()([)(

−=+′=

=

MknXnoEnXndE

nXndEnG

kk

kk

…

0)]()([ =nXnoE k as the observation noise )n(o and input

component )(nX k are uncorrelated. Hence use of Griffith’s

gradient as in Equation (8) will remove the effect of

observation noise on the weight adaptation process. Further,

the VGLMS can be used for reducing the step size as

converges approaches. Defining

[ ])()()()( nXnynGnQ kkk −= , 1,,0 −= Mk … (9a)

)(nQk is Griffiths cross-correlation in the context of

transform domain LMS algorithm. Further )(nGk is estimated

recursively as

( ) 10),()(1)(ˆ)1(ˆ <<−+=+ gkgkgk nXndnGnG βββ (9b)

Similar to VGLMS algorithm, the step size )n(µ can be

adapted as

∑−

=

+−=1

0

2 )()1()(

M

k

k nQnµαnµ γ ( ) 1,0 << γα (10)

This TVGLMS algorithm will be referred to as TVGLMS-1

algorithm.

B TVGLMS with desired signal decomposition

For adaptation the weights )(nwk , if the desired

signal is orthogonally decomposed by transform, then

individual errors )(nek will be directly available. Use of these

errors )(nek is advantageous to that of overall error )n(e .

This is because; the overall error )(ne will contain not only

the required error component but also other error components

which act as observation noise for the weight adaptation under

consideration. This not only facilitates faster convergence but

also smaller convergence error/ misadjustment. In view of

this, the application of VGLMS algorithm to the case where

the desired signal is decomposed is of interest.

Fig.3 shows the schematic for TVGLMS with desired

signal decomposition. Here )(nd is decomposed by another

DCT of same number of points ( M2 ) to get the

components )n(dk , 1,,0 −= Mk … . Further the outputs of

the adaptive filters )(nyk are not summed but are remained to

get the individual errors )(nek as

)()()( nyndne kkk −= , 1,,0 −= Mk …

The TVGLMS algorithm (Equations. (8), (9), and (10) for this

case is

[ ])()()()(

)(2)()1(

2nXnynG

n

nnwnw kkk

k

kk −+=+σ

µ (11)

[ ])()()()( nXnynGnQ kkkk −= , 10 −= M,,k … (12a)

( ) 10),()(1)(ˆ)1(ˆ <β<β−+β=+ gkkgkgk nXndnGnG (12b)

The step size )(nµ is adapted as

∑−

=

+−=1

0

2 )()1()(

M

k

k nQnµαnµ γ ( ) 1,0 << γα (13)

)n(X1

)(1 nw

)(nx )(nd′

)n(o

)n(y0

)(nd

+_

)n(eM 1−

+

)n(w2

)n(wM 1−

)n(X1

)n(XM 1−

)n(y1

)n(yM 1−

)z(P

+

+

_

_

)n(d0

)(1 nd

)n(dM 1−

)n(e0

)n(e1

DC

T

DC

T

+

+

+

Fig.3 DCT transform domain LMS adaptive filter for System

identification with desired signal decomposition


109

This TVGLMS algorithm using components of the desire

signal in addition the components of the input signal will be

referred to as TVGLMS-2 algorithm.

Both TVGLMS-1(Equations. (8), (9) and (10)) and

TVGLMS-2 (Equations. (11), (12), and (13)) algorithms not

only use a gradient but also use a variable step size which is

robust to observation noise. In both cases, a high value of γ

enables better tracking for non-stationary signal, whereas a

small γ is acceptable for stationary signals. Further the noise

free gradient and step-size, significantly improve the

convergence performance. The TVGLMS-2 algorithm may

perform better than that of TVGLMS-2, due to use of

appropriate errors for the individual weight adaptations.

IV. SIMULATION RESULTS

The performance of the propose algorithms is illustrated for

the system identification. The smoothed ensemble average

square error (SEASE) expressed in dB, )(nSdξ

))((log10)( 10 nn SSd ξ=ξ

)(nSξ is a smoothed )(nξ by a moving average filter.

∑=

=ξK

i

i neK

n

1

2)(

1)(

)()()( 'nyndne iii −= - Error for i

th realization and is

derived using the system output without the observation noise

and K is the number of realizations used in getting )(nξ .

The plant chosen is an acoustic path of impulse

response length 128=N The DCT used is of N2 points.

The values of 50=K . The observation noise used is both

white (SNR=1dB)and coloured noise (SNR= -4dB). The

coloured noise is generated by filtering white noise by a

transfer function

21 8.01

1)(

−− +−=

zzzH .

The input used is sum of three sinusoids of frequencies

50,100,150 Hz respectively and additive random noise ( SNR

=18dB) . The parameters used for TVGLMS without and with

desired signal decomposition are given in Table -1 and Table

– 2, respectively.

Table-1

Parameter

µ gβ minµ maxµ β σ α

Value 1e-4 0.99 1e-6 1e-4 0.99 1e-11 0.9999

Table-2

Parameter

µ gβ minµ maxµ β σ α

Value 0.01 0.99 0.001 0.01 0.99 1e-11 0.9999

-

The convergence error for both white and colored

observation noise is reduced by 10 dB by the proposed

TVGLMS algorithm (without desired signal decomposition)

over the TLMS algorithm. However, the convergence rate is

somewhat reduced due to the averaging of the gradient

involved in the proposed algorithm. The reduced convergence

error is due to the robust variable step-size and this variation

from a high value to a low value is shown in the figure.

The convergence error for white and colored

observation noise is reduced by 3 dB and 8 dB, respectively

by the proposed TVGLMS algorithm (with desired signal

decomposition) over the TLMS algorithm. However, the

convergence rate is faster than that for without decomposition

due to removal of other components which act as observation

noise for the specific component. Even the noise also gets

removed from the corresponding error due to decomposition.

For the same reason the convergence error reduction is less

compared to that by TVGLMS without desired signal

decomposition. The reduced convergence error is due to the

robust variable step-size and this variation from a high value

to a low value is shown in the figure.


110

V. CONCLUSION

In this paper a new efficient robust transform domain

Griffiths’ variable step size LMS (TVGLMS) algorithm was

proposed for system identification both in the presence of

white/ coloured observation noise. The new algorithm due to

the input orthogonalization by the DCT, improves the slow

convergence rate of VGLMS based on transversal adaptive

filter. In this algorithm, both the gradient and the stepsize used

are robust to observation noise. Further, as same gradient is

used for both the weight vector and step size adaptation, it is

computationally efficient. The TVGLMS algorithm results in

a lower misadjustment as the gradient is free from observation

noise and also as the stepsize decreases towards convergence.

The TVGLMS algorithm that decomposes the desired signal

in addition to the input, has a better performance both in terms

of convergence speed and the error magnitude, due to

provision of removal observation noise for that component.

The proposed algorithms are compared with the transform

domain algorithm with out Griffith’s gradient and without

stepsize variation, and are found to be significantly superior to

the latter in terms of misadjustment/ convergence error. The

reduction in convergence error by the TVGLMS over TLMS,

without desired signal decomposition is better than that

provided by the one with decomposition.

ACKNOWLEDGMENT

I thank Ms Veena for her help in choosing the parameters for

this new proposed algorithm.

REFERENCES

[1] T.Abolnasar, K.M.Mayyas, “A robust variable step-size

LMS type algorithm: Analysis & simulation”, IEEE

Transactions on Signal Processing, Vol. 45(3), 1997, pp. 631-

637

[2] L. J. Griffiths, “A simple adaptive algorithm for real time

processing of in antenna arrays”, Proc. IEEE, vol. 57, pp.

1696-1704, October 1967

[3] R. W. Harris, D. Chabries and F. Bishop, “A variable step

(VS) adaptive filter algorithm,” IEEE Trans. Acoustics,

Speech, and Signal Processing , Vol. APPS-34, No.2, pp. 309-

316, Apr. 1986.

[4] R.H. Kwong and E.W. Johnston, "A variable step-size

LMS algorithm", IEEE Trans. Signal Process., vol. 40, no. 7,

pp. 1633-1642, July 1992.

[5] James Okello et. al., “A new modified variable step-size

for the LMS algorithm”, IEEE International Symposium on

circuits and systems, pp. 170-173, Vol. 5, 1998

[6} T.J. Shan and T. Kailath, “Adaptive algorithms with an

automatic gain control feature,” IEEE Trans. on Circuits And

Systems, CAS-35, 1988, pp. 122-127

[7] Yonhhang Zhang, Ning Li, Jonathon. A. Chambers and

Yanling Hao, New Gradient based Variable Step-Size LMS

algorithms, EURASIP Journal on Advances in Signal

Processing Vol. 2008, Article ID529480

[8] S.V.Narasimhan, S. Veena, Lokesha.Variable step-size

Griffiths’ algorithm for improved performance of Feed

forward/ Feedback Active Noise Control

[9] Ho-Chi Hwang and Che-Ho Wei, Adaptive blind

demodulation of DS/CDMA signals with transform domain

Griffiths’ algorithm, IEEE International symposium on

circuits and systems, (ISCAS-99) 30th

May-2nd

June 2, 1999

Documents

Dsp 2