An Affine Projection Algorithm with Two Numbers of Input Vectors

NamWoong Kong, MoonSoo Chang, PooGyeon Park�, Sang Woo KimElectrical and Computer Engineering DivisionPohang University of Science and Technology

Pohang, Kyungbuk, 790-784, Korea�pandakon, gravi, ppg, swkim�@postech.ac.kr

Abstract

This paper presents an affine projection algorithm (APA)with a large number of input vectors at the first stageand with a small number of input vectors at the secondstage, where the transition is performed by using the cri-terion for computing the optimum number of input vectorsin the dynamic selection APA (DS-APA). The proposed al-gorithm has fast convergence at the first stage and a smallsteady-state estimation error at the second stage, which per-forms like APAs with the selective input vectors includingDS-APA. However, the proposed algorithm has only twofixed numbers of input vectors and low complexity, whichis more applicable for hardware implementation compar-ing to APAs with the selective input vectors. Simulationsillustrate the performance of the proposed algorithm.

1. Introduction

The affine projection algorithm (APA) has been pro-posed to improve the convergence performance of the leastmean square (LMS) type adaptive filter with highly corre-lated input signal [1]. The APA updates the coefficients ofadaptive filters on the basis of the last input vectors [2]. Itconverges quickly but has a large steady-state estimation er-ror when many input vectors are used. On the other hands, itconverges slowly but has a small steady-state estimation er-ror when the number of input vectors is small [3], [4]. Thismakes us conclude that the number of input vectors greatlyaffects the convergence performance in the APA.

Recently, several algorithms focusing on the number ofinput vectors have been proposed to improve the perfor-mance of the APA [5]-[7]. The set-membership affine pro-jection algorithm with variable data-reuse factor (SM-APAvdr) [5] is based on the set-membership affine projectionalgorithm. This algorithm shows that the convergence per-formance can be improved by varying the number of inputvectors. In addition, the affine projection algorithm with se-

lective regressors (SR-APA) [6] focuses on selecting a sub-set of input regressors at every iteration. The affine projec-tion algorithm with dynamic selection of input vectors (DS-APA) [7] deals with selecting the input vectors and varyingthe number of the input vectors. The hardware for select-ing input vectors is difficult to implement with DS-APA.Furthermore, the complexity and cost of the hardware in-crease because of additional memory spaces for selectinginput vectors.

In this paper, we propose the modified APA with twofixed numbers of input vectors. The proposed algorithmdrop the number of input vectors to reduce the steady-stateestimation error at the switching point. We find the switch-ing point using the criterion, which is employed to obtainthe optimum number of input vectors in the DS-APA [7].When the optimum number, which is obtained by the cri-terion, becomes equal or less than a half of the maximumnumber, the proposed algorithm drops the number of inputvectors from maximum to two. The proposed algorithm hasfast convergence like DS-APA and a smaller steady-state es-timation error than the conventional APA [2] and DS-APA[7]. Furthermore, the hardware can be simply implementwith low cost because of using the two fixed input vectors.

The paper is organized as follows. Section II briefly re-views the concept of an affine projection algorithm. SectionIII shows that the proposed algorithm, an affine projectionalgorithm with two numbers of input vectors, is derived.And Section IV presents the simulation results of the pro-posed algorithm about the several systems. Finally, we con-clude in the Section V.

2. Affine Projection Algorithm (APA)

Consider data �� that arise from the unknown system

�� � u�w� � �� (1)

where w� is an known column vector that we expect to esti-mate, �� accounts for measurement noise with variance ��

�,

Fifth International Conference on Fuzzy Systems and Knowledge Discovery

978-0-7695-3305-6/08 $25.00 © 2008 IEEE

DOI 10.1109/FSKD.2008.470

272

and u� denotes ��� row input regressor vectors as follows:

u� � � �� ���� � � � ��������

Let us define w� as an estimate for w� at iteration �, and weobtain the APA update equation such that

w� � w��� � �U�

� �U�U�

� ���e� (2)

where

U� �

�����

u�u���

...u�����

����� d� �

�����

������

...������

�����

e� � d� � U�w���, and � is the step size.

3. APA with Two Numbers of Input Vectors

S.-J. Kong and K.-Y. Hwang and W.-J. Song proposedthe method calculating the optimum number of the inputvectors [7]. In this chapter, we present APA with two fixednumbers of input vectors and the method to find the switch-ing point by using the criterion of the DS-APA.

���� ������ ��� �� ��� ���� ����� �

The update recursion equation (2) can be rewritten interms of the weight-error vector, �w� � w� � w� such that

�w� � �w��� � �U�

� �U�U�

� ���e�� (3)

Squaring both sides and taking expectations, we can obtainthe mean-square deviation (MSD) satisfies

��w���� ��w����

���Re

��

e���U�U

�

� ���U� �w���

� ���e���U�U�

����e�

� ��w����

� � (4)

where is a function of input vectors. If we maximize the, then the MSD will undergo the largest decrease fromiteration �� � �� to iteration �. From the maximizing the ,we can obtain the fastest convergence speed [7], [8]. canbe written as

� ������e���U�U�

����e�

� ���

�Tr���U�U�

����

��

We assume that the noise sequence �� is identically andindependently distributed and statistically independent ofthe regression data U�, and the dependency of w̃��� on pastnoise is neglected [4], [7], [8].

We replace the expected value with an instantaneousvalue because it is impossible to know the exact expectedvalue. is rewritten as follows:

� ��� ��e�� �U�U�

� ���e� � ����Tr

��U�U

�

� ���

��

From the above equation, we can find the input vectorsto make the largest decrease of the MSD by maximizing .However, the practical implementation of can be com-putationally very complex because of the subset selectionof the input vector. To simplify the calculation of U�U�

�,

we assume that the diagonal components of U�U�

�are much

larger than off-diagonal components. Although the solu-tion obtained by the assumption is not exactly same as thatthrough the full implementation, it shows the satisfactoryconvergence performance and low computational complex-ity.

From the above assumption, we can approximate as

� ��� ����� ��� � � � ����

�(5)

where

�� �������� ��

���� ��

�u�����

����� � ���� � u���w���

for � � � � � � � � �.From (5), we can find that if ��

���� � ��

��� � ��,

then u��� contributes to obtain the maximum value of .However, if ������ � ����� � ��, then the value of isdcrease by u���. Therefor, the optimum number of inputvectors should be the same as the number of errors satisfy-ing ������ � ������ �� at every iteration.

���� � ������ ���� ����

The proposed algorithm is the combination of conven-tional APA with � input vectors and with two input vec-tors, where � is the maximum number of input vectors.Therefore, the algorithm has two stage. We use the APAwith the number of input vectors which is � at the firststage and two at the second stage. The important part ofthis algorithm is to find the switching point. The switchingpoint means the moment changing the number of input vec-tors from � to . To find the point, we use the criterion ofthe DS-APA as follows [7] :

������� �

�

� ��� � � � � � � � � � ��� (6)

Let us define ���� as the number of input vectors satis-fying the criterion (6). If ���� � ��, then we compul-sively change the number of input vectors from � to two

273

in the APA update equation. The proposed algorithm hasa good performance when the number of input vectors, �,is two after the switching point � satisfying ���� � ��.We experimentally confirm the above result. After all, thenumber of input vectors of the proposed algorithm is � atthe first stage (�=� ) and two at the second stage (�= ).Table 1 shows the computational complexity of the conven-tional APA, DS-APA and the proposed algorithm for everyiteration.

Table 1. Computational Complexity of APA,DS-APA and proposed algorithm

ConventionalAPA DS-APA Proposed

Algorithm

Multiplication����������

� ��� (*) (**)

Comparison - � � � �

���� - �������� (***)

��� � ������ �� ������������� ������

���� � ������ � ���� ������������� ������

���� �

�� if � � switching point

� if � � switching point

�� � �� � ���� �

�� if � � switching point

� if � � switching point.

From the Table 1, we know that the proposed algorithmhas less complexity than the conventional APA and DS-APA.

4. Simulation Results

In this section, we illustrate the performance of the pro-posed algorithm by carrying out computer simulations in achannel estimation. The channel of the unknown system isgenerated by moving average model with 32 taps. We as-sume that both adaptive filter and the unknown channel havethe same number of taps. The input signals are obtainedby filtering a white, zero-mean, Gaussian random sequencethrough the following systems :

����� ��

�� �� ���

����� �� � ������

� � ������ � �������

0 1000 2000 3000 4000 5000 6000 7000 8000−25

−20

−15

−10

−5

0

Number of iterations

MS

D (

dB)

(a) μ=0.1, APA (K=16)(b) μ=0.5, APA (K=16)(c) μ=0.01, APA (K=16)(d) DS−APA (K=16)(e) Proposed APA

(b) μ = 0.5

(a) μ = 0.1(d) μ = 0.1

(c) μ = 0.01

(e) μ = 0.1

Figure 1. The mean-square deviation of con-ventional APA, DS-APA and proposed algo-rithm (The input signal is generated by G1,SNR = 30dB)

0 1000 2000 3000 4000 5000 6000 7000 80004

6

8

10

12

14

16

Number of iterations

Num

ber

of in

put v

ecto

rs

Switching Point

Figure 2. The number of input vectors to findthe switching point

274

0 1000 2000 3000 4000 5000 6000 7000 8000−45

−40

−35

−30

−25

−20

−15

−10

−5

0

Number of iterations

MS

D (

dB)

(a) μ=0.1, APA (K=16)(b) μ=0.5, APA (K=16)(c) μ=0.01, APA (K=16)(d) DS−APA (K=16)(e) Proposed APA

(b) μ = 0.5 (a) μ = 0.1 (d) μ = 0.1

(c) μ = 0.01 (e) μ = 0.1

Figure 3. The mean-square deviation of con-ventional APA, DS-APA and proposed algo-rithm (The input signal is generated by G2,SNR = 30dB)

The SNR used in the simulations is defined by

SNR � �� log��

��������

�������

where ���� � u�w�. The simulation results are obtained byensemble averaging over 100 independent trials.

Figure 1 and Figure 3 show the MSD of the proposedalgorithm with K=16 and SNR=30dB. When the numberof taps of the unknown system is 32 and the input signalsare generated by G1 and G2, the simulation is performed.We confirm that the proposed algorithm has the fast con-vergence speed as the DS-APA and a small misadjustmenterror compared to the conventional APA and DS-APA.

Figure 2 shows the number of input vectors by the en-semble average. When the number of input vectors satisfy-ing the criterion (6) becomes less than a half of the maxi-mum number of input vectors, we drop the number of inputvectors from K=16 to K=2 at the moment.

5. Conclusions

This paper proposed an affine projection algorithm(APA) with maximum input vectors at the first stage andwith two input vectors at the second stage, where the tran-sition is performed by using a criterion for calculating theoptimum number of input vectors in the DS-APA. We haveconfirmed that the proposed algorithm had the fast conver-gence speed as the DS-APA and more smaller steady-stateestimation error than the conventional APA and the DS-APA in the simulation results. Because the proposed al-gorithm has only two fixed numbers of input vectors and

low complexity, it is more applicable for practical hardwareimplementation compared to APAs with the selective inputvectors. The future study will be the subject of the APAwith multi-stage and a variable step size.

Acknowledgment

This research was supported by Ministry of Knowledgeand Economy, Republic of Korea, under the ITRC (Infor-mation Technology Research Center) support program su-pervised by IITA (Institute for Information Technology Ad-vancement). (IITA-2008-C1090-0801-0004)

This work was supported by the Brain Korea 21 Projectin 2008.

References

[1] S. Haykin, Adaptive filter theory, 4th ed. Upper SaddleRiver, NJ:Prentice-Hall, 2002.

[2] K. Ozeki and T. Umeda, ”An adaptive filtering algo-rithm using an orthogonal projection to an affine sub-space and its properties,” Electron. Commun. Jpn., vol.67-A, no. 5, pp. 19-27, 1984.

[3] S. J. M. de Almeida, J. C. M. Bermudez, N. J. Ber-shad, and M. H. Costa, ”A statistical analysis of theaffine projection algorithm for unity step size and au-toregressive inputs,” IEEE Trans. Circuits Syst. I, Reg.Papers, vol. 52, no. 7, pp. 1394-1405, Jul. 2005.

[4] H.-C. Shin and A. H. Sayed, ”Mean-square perfor-mance of a family of affine projection algorithm,”IEEE Trans. Signal Process., vol. 52, no. 1, pp. 90-102, Jan. 2004

[5] S. Werner, P. S. R. Diniz, and J. E. W. Moreira, ” Set-membership affine projection algorithm with variabledata-reuse factor,” in Proc. 2006 IEEE Int. Symp. Cir-cuits and Systems, May 2006, pp. 261-264.

[6] K.-Y. Hwang and W.-J. Song, ”An affine projectionadaptive filtering algorithm with selective regressors,”IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 54, no.1, Jan. 2007.

[7] S.-J. Kong and K.-Y Hwang, and W.-J. Song, ”Anaffine projection algorithm with dynamic selection ofinput vectors,” IEEE Signal Process. Lett., vol. 14, no.8, pp. 529-532, Aug. 2007.

[8] H.-C. Shin, A. H. Sayed, and W.-J. Song, ”Variablestep-size NLMS and affine projection in algorithm,”IEEE Signal Process. Lett., vol. 11, no. 2, pp. 132-135, Feb. 2004.

275