Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
New Constrained Affine-ProjectionAdaptive-Filtering Algorithm
Md. Zulfiquar Ali Bhotto and Andreas Antoniou
Department of Electrical and Computer EngineeringUniversity of Victoria, Victoria, BC, Canada
Emails: [email protected], [email protected]
ISCAS 19-23 May 2013
Frame # 1 Slide # 1 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Outline
� Objectives
Frame # 2 Slide # 2 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Outline
� Objectives
� Two versions of a new constrained affine-projection (CAP)algorithm, PCAP-I and PCAP-II, are proposed as follows:
– derivation of PCAP-I algorithm– derivation of the PCAP-II algorithm– discussion on proposed and conventional CAP algorithms.
Frame # 2 Slide # 3 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Outline
� Objectives
� Two versions of a new constrained affine-projection (CAP)algorithm, PCAP-I and PCAP-II, are proposed as follows:
– derivation of PCAP-I algorithm– derivation of the PCAP-II algorithm– discussion on proposed and conventional CAP algorithms.
� Simulation results
– system identification application– interference-suppression application for direct-sequence
code-division multiple access (DS-CDMA) communicationsystems
Frame # 2 Slide # 4 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Outline
� Objectives
� Two versions of a new constrained affine-projection (CAP)algorithm, PCAP-I and PCAP-II, are proposed as follows:
– derivation of PCAP-I algorithm– derivation of the PCAP-II algorithm– discussion on proposed and conventional CAP algorithms.
� Simulation results
– system identification application– interference-suppression application for direct-sequence
code-division multiple access (DS-CDMA) communicationsystems
� Conclusions
Frame # 2 Slide # 5 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
Frame # 3 Slide # 6 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
Frame # 3 Slide # 7 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
– yield an unbiased solution
Frame # 3 Slide # 8 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
– yield an unbiased solution– require a reduced computational effort
Frame # 3 Slide # 9 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
– yield an unbiased solution– require a reduced computational effort– yield a reduced steady-state misalignment
Frame # 3 Slide # 10 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
– yield an unbiased solution– require a reduced computational effort– yield a reduced steady-state misalignment– require fewer iterations to converge
Frame # 3 Slide # 11 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Objectives
� Develop adaptive-filtering algorithms that can be used forapplications where the weight vector must satisfy certainconstraints.
� The developed algorithms should
– yield an unbiased solution– require a reduced computational effort– yield a reduced steady-state misalignment– require fewer iterations to converge
compared to the constrained normalized least-mean square(CNLMS), known CAP, and set-membership CAP (CSMAP)algorithms.
Frame # 3 Slide # 12 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm
� The two versions of the CAP algorithm are obtained bysolving the minimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2
Frame # 4 Slide # 13 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm
� The two versions of the CAP algorithm are obtained bysolving the minimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2
subject to the constraint
Cw = f
Frame # 4 Slide # 14 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm
� The two versions of the CAP algorithm are obtained bysolving the minimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2
subject to the constraint
Cw = f
where dk ∈ Rl×1 is the desired signal vector, Xk ∈ Rm×l isthe input signal matrix, C ∈ Rp×m is a constraint matrix, andf ∈ Rp×1 is a constraint vector.
Frame # 4 Slide # 15 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm
� The two versions of the CAP algorithm are obtained bysolving the minimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2
subject to the constraint
Cw = f
where dk ∈ Rl×1 is the desired signal vector, Xk ∈ Rm×l isthe input signal matrix, C ∈ Rp×m is a constraint matrix, andf ∈ Rp×1 is a constraint vector.
� The gradient of J(w) at points wk and wk−1 satisfies theinequality
‖∇J(wk)−∇J(wk−1)‖2 ≤ λk,max(XkXTk )‖wk − wk−1‖2
Frame # 4 Slide # 16 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm
� The two versions of the CAP algorithm are obtained bysolving the minimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2
subject to the constraint
Cw = f
where dk ∈ Rl×1 is the desired signal vector, Xk ∈ Rm×l isthe input signal matrix, C ∈ Rp×m is a constraint matrix, andf ∈ Rp×1 is a constraint vector.
� The gradient of J(w) at points wk and wk−1 satisfies theinequality
‖∇J(wk)−∇J(wk−1)‖2 ≤ λk,max(XkXTk )‖wk − wk−1‖2
where λk,max is the maximum eigenvalue of XkXTk .
Frame # 4 Slide # 17 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm, Cont’d...
� We can, therefore, approximate J(w) = 0.5‖XTk w − dk‖2 at
wk as
J(w) ≈ J(wk) = J(wk−1) + (wk − wk−1)T∇J(wk−1)
+1
2μk‖wk − wk−1‖22
where J(w) < J(wk) and μk is called the Lipschitz constant.
Frame # 5 Slide # 18 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm, Cont’d...
� We can, therefore, approximate J(w) = 0.5‖XTk w − dk‖2 at
wk as
J(w) ≈ J(wk) = J(wk−1) + (wk − wk−1)T∇J(wk−1)
+1
2μk‖wk − wk−1‖22
where J(w) < J(wk) and μk is called the Lipschitz constant.
� The PCAP-I algorithm can now be obtained by solving theminimization problem
minimizewk
J(wk−1)+(wk−wk−1)T∇J(wk−1)+
1
2μk‖wk−wk−1‖22
Frame # 5 Slide # 19 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm, Cont’d...
� We can, therefore, approximate J(w) = 0.5‖XTk w − dk‖2 at
wk as
J(w) ≈ J(wk) = J(wk−1) + (wk − wk−1)T∇J(wk−1)
+1
2μk‖wk − wk−1‖22
where J(w) < J(wk) and μk is called the Lipschitz constant.
� The PCAP-I algorithm can now be obtained by solving theminimization problem
minimizewk
J(wk−1)+(wk−wk−1)T∇J(wk−1)+
1
2μk‖wk−wk−1‖22
subject to the constraint
Cwk = f
Frame # 5 Slide # 20 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm, Cont’d...
� The solution of the problem at hand can be obtained by usingthe Lagrange multiplier method as
wk = Z [wk−1 + μkXkek ] + t
Frame # 6 Slide # 21 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New CAP Algorithm, Cont’d...
� The solution of the problem at hand can be obtained by usingthe Lagrange multiplier method as
wk = Z [wk−1 + μkXkek ] + t
where Z ∈ Rm×m is a matrix given by
Z = I− CT (CCT )−1C
and t ∈ Rm×1 is a vector given by
t = CT (CCT )−1f
Frame # 6 Slide # 22 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� For the PCAP-I algorithm, we solve the minimization problem
minimizeμk
F (μk) = 0.5‖XTk wk − dk‖2
to obtain
μk =eTk X
Tk ZXk(dk − XT
k [Zwk−1 + t])
eTk XTk ZXkX
Tk ZXkek
which can be used in the update formula
wk = Z [wk−1 + μkXkek ] + t
Frame # 7 Slide # 23 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� For the PCAP-I algorithm, we solve the minimization problem
minimizeμk
F (μk) = 0.5‖XTk wk − dk‖2
to obtain
μk =eTk X
Tk ZXk(dk − XT
k [Zwk−1 + t])
eTk XTk ZXkX
Tk ZXkek
which can be used in the update formula
wk = Z [wk−1 + μkXkek ] + t
� With the initialization w0 = t the update formula of thePCAP-I algorithm simplifies to
wk = wk−1 + μkZXkek
Frame # 7 Slide # 24 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� For the PCAP-I algorithm, we solve the minimization problem
minimizeμk
F (μk) = 0.5‖XTk wk − dk‖2
to obtain
μk =eTk X
Tk ZXk(dk − XT
k [Zwk−1 + t])
eTk XTk ZXkX
Tk ZXkek
which can be used in the update formula
wk = Z [wk−1 + μkXkek ] + t
� With the initialization w0 = t the update formula of thePCAP-I algorithm simplifies to
wk = wk−1 + μkZXkek
where
μk =eTk X
Tk ZXkek
eTk XTk ZXkX
Tk ZXkek
Frame # 7 Slide # 25 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� The solution of the system of equations Cw = f can beobtained as
w = Vrω + C+f
where C+ denotes the Moore-Penrose pseudo-inverse of C,Vr ∈ Rm×r is a matrix consisting of the last m = r − pcolumns of V which is obtained by using using thesingular-value decomposition of C, and ω ∈ Rr×1 is a vector.
Frame # 8 Slide # 26 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� The solution of the system of equations Cw = f can beobtained as
w = Vrω + C+f
where C+ denotes the Moore-Penrose pseudo-inverse of C,Vr ∈ Rm×r is a matrix consisting of the last m = r − pcolumns of V which is obtained by using using thesingular-value decomposition of C, and ω ∈ Rr×1 is a vector.
� With this solution, the optimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2subject to the constraint
Cw = f
Frame # 8 Slide # 27 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-I Algorithm, Cont’d...
� The solution of the system of equations Cw = f can beobtained as
w = Vrω + C+f
where C+ denotes the Moore-Penrose pseudo-inverse of C,Vr ∈ Rm×r is a matrix consisting of the last m = r − pcolumns of V which is obtained by using using thesingular-value decomposition of C, and ω ∈ Rr×1 is a vector.
� With this solution, the optimization problem
minimizew J(w) = 0.5‖XT
k w − dk‖2subject to the constraint
Cw = f
can be expressed as
minimizeω J(ω) = 0.5‖XT
k Vrω + XTk C
+f − dk‖2
Frame # 8 Slide # 28 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-II Algorithm
� The PCAP-II algorithm is obtained by solving theminimization problem
minimizeω J(ω) = 0.5‖XT
k Vrω + XTk C
+f − dk‖2
by using the same steps as for the PCAP-I algorithm.
Frame # 9 Slide # 29 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
New PCAP-II Algorithm
� The PCAP-II algorithm is obtained by solving theminimization problem
minimizeω J(ω) = 0.5‖XT
k Vrω + XTk C
+f − dk‖2
by using the same steps as for the PCAP-I algorithm.
� The update formula of the PCAP-II algorithm becomes
ωk = ωk−1 + μkVTr Xkek
where
μk =eTk X
Tk VrV
Tr Xkek
eTk XTk VrV
Tr XkX
Tk VrV
Tr Xkek
Frame # 9 Slide # 30 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Discussion
� The PCAP-I and PCAP-II algorithms do not require theinverse of XT
k ZXk and hence they require less computationthan the conventional CAP algorithm.
Frame # 10 Slide # 31 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Discussion
� The PCAP-I and PCAP-II algorithms do not require theinverse of XT
k ZXk and hence they require less computationthan the conventional CAP algorithm.
� The PCAP-II algorithm requires reduced computation ascompared to the PCAP-I algorithm due to the reduceddimensions of ωk and VT
r .
Frame # 10 Slide # 32 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Simulation Results
� MSD learning curves for system identification application:
0 100 200 300 400 500 600 700
−35−30
−25−20
−15−10
−50
5
Number of iterations
MSD
, dB
CNLMSCAPSMCAPPCAP−IPCAP−II
Frame # 11 Slide # 33 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Simulation Results, Cont’d...
Table: Average CPU Time, in Microseconds
CNLMS CAP SMCAP PCAP-I PCAP-II
27 61 8 38 36
Frame # 12 Slide # 34 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Simulation Results, Cont’d...
� Learning mean output-error (MOE) curves for DS-CDMAinterference suppression application:
0 200 400 600 800 1000 1200 1400 1600 1800 20000
5
10
15
20
25
Number of iterations
MO
E, d
B
CNLMSCAPSMCAPPCAP−IPCAP−II
Frame # 13 Slide # 35 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Conclusions
Two new closely related CAP adaptive-filtering algorithms, thePCAP-I and PCAP-II, that use a new step size have beenproposed. The new algorithms
� produce an unbiased output in applications where the desiredsignal is unavailable or not required
Frame # 14 Slide # 36 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Conclusions
Two new closely related CAP adaptive-filtering algorithms, thePCAP-I and PCAP-II, that use a new step size have beenproposed. The new algorithms
� produce an unbiased output in applications where the desiredsignal is unavailable or not required
� require reduced computational effort than the conventionalCAP algorithm
Frame # 14 Slide # 37 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Conclusions
Two new closely related CAP adaptive-filtering algorithms, thePCAP-I and PCAP-II, that use a new step size have beenproposed. The new algorithms
� produce an unbiased output in applications where the desiredsignal is unavailable or not required
� require reduced computational effort than the conventionalCAP algorithm
� yield reduced steady-state misalignment relative to theCNLMS algorithm as well as some recent CAP algorithms
Frame # 14 Slide # 38 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm
Conclusions
Two new closely related CAP adaptive-filtering algorithms, thePCAP-I and PCAP-II, that use a new step size have beenproposed. The new algorithms
� produce an unbiased output in applications where the desiredsignal is unavailable or not required
� require reduced computational effort than the conventionalCAP algorithm
� yield reduced steady-state misalignment relative to theCNLMS algorithm as well as some recent CAP algorithms
� offer faster convergence than the CNLMS algorithm.
Frame # 14 Slide # 39 Bhotto & Antoniou New CAP Adaptive-Filtering Algorithm