Adaptive Filters Chapter · Adaptive Filters Introduction The term adaptive filter implies changing the characteristic of a ... 1.Simon Haykin, Adaptive Filter Theory, fourth edition,

er

Adaptive FiltersIntroduction

The term adaptive filter implies changing the characteristic of afilter in some automated fashion to obtain the best possible signalquality in spite of changing signal/system conditions. Adaptivefilters are usually associated with the broader topic of statisticalsignal processing. The operation of signal filtering by definitionimplies extracting something desired from a signal containingboth desired and undesired components. With linear FIR and IIRfilters the filter output is obtained as a linear function of theobservation (signal applied) to the input. An optimum linear fil-ter in the minimum mean square sense can be designed to extracta signal from noise by minimizing the error signal formed bysubtracting the filtered signal from the desired signal. For noisysignals with time varying statistics, this minimization process isoften done using an adaptive filter.

For statistically stationary inputs this solution is known as aWiener filter.1

1.Simon Haykin, Adaptive Filter Theory, fourth edition, Prentice Hall, 2002.

Σ

WienerFilter

+

-e n[ ]

y n[ ]

d n[ ]

x n[ ]

DesiredSignal

Observation

MMSEEstimateof d[n]

ErrorSignal

Chapt

9

ECE 5655/4655 Real-Time DSP 9–1

Chapter 9 • Adaptive Filters

Wiener Filter

• An M tap discrete-time Wiener filter is of the form

(9.1)

where the are referred to as the filter weights

– Note: (9.1) tells us that the Wiener filter is just an M-tapFIR filter

• The quality of the filtered or estimated signal is deter-mined from the error sequence

• The weights , are chosen such that

(9.2)

is minimized, that is we obtain the minimum mean-squarederror (MMSE)

• The optimal weights are found by setting

(9.3)

• From the orthogonality principle1 we choose the weightssuch that the error is orthogonal to the observations(data), i.e.,

(9.4)

1.A. Papoulis, Probability, Random Variables, and Stochastic Processes,third edition, McGraw-Hill, 1991.

y n[ ] wmx n m–[ ]m 0=

M 1–

∑=

wm

y n[ ]e n[ ] d n[ ] y n[ ]–=

wm m 0 1 … M 1–, , ,=

E e2 n[ ]{ } E d n[ ] y n[ ]–( )2{ }=

wm∂∂ E e2 n[ ]{ } 0 m, 0 1 … M 1–, , ,= =

e n[ ]

E x n k–[ ] d n[ ] y n[ ]–( ){ } 0 k, 0 1 … M 1–, , ,= =

9–2 ECE 5655/4655 Real-Time DSP

Wiener Filter

– This results in a filter that is optimum in the sense of mini-mum mean-square error

• The resulting system of equations

(9.5)

or

(9.6)

for are known as the Wiener-Hopf ornormal equations

– Note: is the autocorrelation function of and is the cross-correlation function between and

• In matrix form we can write

(9.7)

where is the correlation matrix associated with

(9.8)

is the optimum weight vector given by

wmE x n k–[ ]x n m–[ ]{ }m 0=

M 1–

∑ E x n k–[ ] d n[ ]( ){ }=

wmφxx m k–[ ]m 0=

M 1–

∑ φxd k–[ ]=

k 0 1 … M 1–, , ,=

φxx k[ ] x n[ ]φxd k[ ] x n[ ]d n[ ]

Rxxwo pxd=

Rxx M M×

x n[ ]

Rxx

φxx 0[ ] φxx M 1–[ ]

φxx M– 1+[ ] φxx 0[ ]

= . . .

. . .

. . .

. . .

. . .

wo



(9.9)

and is the cross-correlation vector given by

(9.10)

• The optimal weight vector is given by

(9.11)

• As a matter of practice (9.11) can be solved using sample sta-tistics, that is we replace the true statistical auto- and cross-correlation functions with time averages of the form

(9.12)

(9.13)

where N is the sample block size

Adaptive Wiener Filter

• In an adaptive Wiener filter the error signal is fed backto the filter weights to adjust them using a steepest-descentalgorithm

wo wo0 wo1 … woM 1–

T=

pxd

pxd φxd 0[ ] φxd 1–[ ] … φxd 1 M–[ ]T

=

wo Rxx1– pxd=

φxx k[ ] 1N---- x n k+[ ]x n[ ]n 0=

N 1–

∑≅

φxd k[ ] 1N---- x n k+[ ]d n[ ]n 0=

N 1–

∑≅

e n[ ]



• With respect to the weight vector , the error can beviewed as an M dimensional error surface, that due to thesquared error criterion, is convex cup (a bowl shape)

• The filter decorrelates the output error so that signals incommon to both and in a correlation sense arecancelled

w e n[ ]

w0

w1

Mea

n S

quar

e E

rror

Optimum at [1,1]

Error Surface for M = 2

e n[ ]d n[ ] x n[ ]



• A block diagram of this adaptive Wiener (FIR) filter is shownbelow

Least-Mean-Square Adaptation

• Ideally the optimal weight solution can be obtained by apply-ing the steepest descent method to the error surface, but sincethe true gradient cannot be determined, we use a stochasticgradient, which is simply the instantaneous estimate of and from the available data, e.g.,

(9.14)

(9.15)

where

(9.16)

• A practical implementation involves estimating the gradientfrom the available data using the least-mean-square (LMS)algorithm

Σ+

-e n[ ]

y n[ ]

d n[ ]

x n[ ]

DesiredSignal

Observation

MMSEEstimateof d[n]

ErrorSignal

WienerFilter

Adaptive Wiener FIlter

AdaptiveAlgorithm

Rxxpxd

Rxx x n[ ]xT n[ ]=

pxd x n[ ]d n[ ]=

x n[ ] x n[ ] x n 1–[ ] … x n M– 1+[ ]T

=



• The steps to the LMS algorithm, for each new sample at timen, are:

– Filter to produce:

(9.17)

– Form the estimation error:

(9.18)

– Update the weight vector using step-size parameter :

(9.19)

• For algorithm stability, the step-size must be chosen suchthat

(9.20)

where

(9.21)

• In theory, (9.20) is equivalent to saying

(9.22)

where is the maximum eigenvalue of

x n[ ]

y n[ ] wm n[ ]x n m–[ ]m 0=

M 1–

∑ wT n[ ]x n[ ]= =

e n[ ] d n[ ] y n[ ]–=

μ

w n 1+[ ] w n[ ] μx n[ ]e n[ ]+=

μ

0 μ 2tap-input power--------------------------------------< <

tap-input power E x n k–[ ] 2{ }k 0=

M 1–

∑=

0 μ 2λmax-----------< <

λmax Rxx



Adaptive Filter Variations1

• Prediction

• System Identification

• Equalization

1.B. Widrow and S. Stearns, Adaptive Signal Processing, Prentice Hall, NewJersey, 1985.

Σ

e n[ ]y n[ ]

d n[ ]

x n[ ]Delay Adaptive

Filter

s n[ ]

+-

Σ

e n[ ]y n[ ]

d n[ ]

x n[ ]

Plant

AdaptiveFilter

s n[ ]

+-

Σ

e n[ ]y n[ ]

d n[ ]

x n[ ]Plant/ Adaptive

Filter

s n[ ]

+-

Delay

Σ++

Noise

Channel

TrainingPattern


Adaptive Line Enhancement

• Interference Canceling


• A relative of the interference canceling scheme shown above,is the adaptive line enhancer (ALE)

• Here we assume we have a narrowband signal (say a sinu-soid) buried in broadband additive noise

• The filter adapts in such a way that a narrow passband formsaround the sinusoid frequency, thereby suppressing much ofthe noise and improving the signal-to-noise ratio (SNR) in

Σe n[ ]y n[ ]

d n[ ]

x n[ ]Adaptive

Filter

+-

Signal +

Interference’

Interference

Σe n[ ]

y n[ ]x n Δ–[ ]Adaptive

Filter

x n[ ] NB n[ ] BB n[ ]+=

z Δ–

+

-

y n[ ]



Example: MATLAB ALE Simulation

• A simple MATLAB simulation is constructed using a singlesinusoid at normalized frequency plus additivewhite Gaussian noise

(9.23)

• The SNR is defined as

(9.24)

function [n,x,y,e,wo,F,Wo] = lms_ale(SNR,N,M,mu)

% [n,x,y,e,wo,F,Wo] = lms_ale(SNR,N,M,mu)

%

%*******LMS ALE Simulation************

% SNR = Sinusoid SNR in dB

% N = Number of simulation samples

% M = FIR Filter length

% mu = LMS step-size

%

% n = Index vector

% x = Noisy input

% y = Filtered output

% e = Error sequence

% wo = Final value of weight vector

% F = Frequency response axis vector

% Wo = Frequency response of filter

%**************************************

% Mark Wickert, 4/27/98

%

% Sinusoid SNR = (A^2/2)/noise_var

n = 0:N; % actually length N+1

x = 1*cos(2*pi*1/20*n); % Here A = 1, Fo/Fs = 1/20

x = x + sqrt(1/2/(10^(SNR/10)))*randn(1,N+1);

% White Noise -> Delta = 1, so delay x by one sample

fo 1 20⁄=

x n[ ] A 2πfon[ ]cos w n[ ]+=

SNR A2

2σw2----------=



xd = filter([0 1],1,x);

% Initialize output vector y to zero

y = zeros(1,N+1);

% Initialize error vector e to zero

e = zeros(1,N+1);

% Initialize weight vector to zero

wo = zeros(1,M);

% Initialize filter memory to zero

z = zeros(1,M-1);

% Initialize a vector for holding xd of length M

xdm = zeros(1,M);

for k=1:N+1

% Filter one sample at a time

[y(k),z] = filter(wo,1,x(k),z);

% Form the error sequence

e(k) = x(k) - y(k);

% Update the weight vector

wo = wo + 2*mu*e(k)*xdm;

% Update vector used for correlation with e(k)

xdm = [xd(k) xdm(1:M-1)];

end %end loop on time index k

% Create filter frequency response

[Wo,F] = freqz(wo,1,512,1);

Wo = 20*log10(abs(Wo));

• A simulation is run using 1000 samples, dB, M =64, and

» [n,x,y,e,wo,F,Wo] = lms_ale(10,1000,64,.01/64);

» plot(n,e.^2)

» subplot(211)

» plot(n,x)

» axis([600 1000 -1.5 1.5])

» subplot(212)

» plot(n,y)

» plot(F,Wo)

SNR 10=μ 0.01 64⁄=



Convergence Occursin Here (~275 samples)

600 650 700 750 800 850 900 950 1000-1.5

-1

-0.5

0

0.5

1

1.5ALE x[n] & y[n] for SNR = 10 dB, mu = 0.01/64

x[n]

600 650 700 750 800 850 900 950 1000

-1

-0.5

0

0.5

1

y[n]

Index n


C6x Code Examples

• A C version of the above MATLAB code would be very sim-ilar except all of the vector operations would have to bereplaced by for loops

• TBD

C6x Code Examples

A Two Channel Input Signal + Signal or Signal + Noise Can-celler

• In this first example a modified version of Chassaing’sAdaptnoise.c program is considered

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5-45

-40

-35

-30

-25

-20

-15

-10

-5

0

5ALE Freq. Resp. for SNR = 10 dB, mu = 0.01/64

|Wo(

f)|d

B

Normalized Frequency f

Sinusoid Frequency



• In this program the following system is implemented

• Notice that signal2 is present at both inputs in the exact sameform; in a real application the input would be similar tosignal2 as seen in , but not exactly the same

• Floating point C code is shown below:// Welch, Wright, & Morrow, // Real-time Digital Signal Processing, 2011// Modified by Mark Wickert February 2012 to include GPIO ISR start/stop postings

///////////////////////////////////////////////////////////////////////// Filename: Adaptive_ISRs.c//// Synopsis: Interrupt service routine for codec data transmit/receive/////////////////////////////////////////////////////////////////////////

#include "DSP_Config.h"

// Function Prototypeslong int rand_int(void); // Data is received as 2 16-bit words (left/right) packed into one// 32-bit word. The union allows the data to be accessed as a single // entity when transferring to and from the serial port, but still be // able to manipulate the left and right channels independently.

#define LEFT 0#define RIGHT 1

Σe n[ ]y n[ ]

d n[ ]

x n[ ]Adaptive

Filter

+-

Signal1 +

Signal2

Signal2 (or noise)

x n[ ]d n[ ]


C6x Code Examples

volatile union {Uint32 UINT;Int16 Channel[2];

} CodecDataIn, CodecDataOut;

/* add any global variables here *///#define beta 1E-10;//rate of convergence#define N 64 //# of weights (coefficients)#define LEFT 0 //left channel#define RIGHT 1 //right channelfloat w[N]; //weights for adapt filterfloat delay[N]; //input buffer to adapt filtershort output; //overall outputshort out_type = 1;//output type for sliderfloat alpha = 10;//alpha from slider

interrupt void Codec_ISR()///////////////////////////////////////////////////////////////////////// Purpose: Codec interface interrupt service routine //// Input: None//// Returns: Nothing//// Calls: CheckForOverrun, ReadCodecData, WriteCodecData//// Notes: None///////////////////////////////////////////////////////////////////////{

/* add any local variables here */WriteDigitalOutputs(1); // Write to GPIO J15, pin 6; begin ISR timing pulseshort i;float yn=0, E=0, dplusn=0, desired=0, noise=0;float beta = 1E-11;beta *= alpha;//scale beta by slider alpha value

if(CheckForOverrun())// overrun error occurred (i.e. halted DSP)return; // so serial port is reset to recover

CodecDataIn.UINT = ReadCodecData();// get input data samples

/* add your code starting here */ desired = (float) CodecDataIn.Channel[ LEFT]; //input left chan noise = (float) CodecDataIn.Channel[ RIGHT]; //input right chan



dplusn = desired + noise; //desired+noise delay[0] = noise; //noise as input to adapt FIR

for (i = 0; i < N; i++) //to calculate output of adaptive FIR { yn += (w[i] * delay[i]); //output of adaptive filter }

E = (desired + noise) - yn; //"error" signal=(d+n)-yn

for (i = N-1; i >= 0; i--) //to update weights and delays { w[i] = w[i] + beta*E*delay[i]; //update weights delay[i] = delay[i-1]; //update delay samples } if (out_type == 1) //if slider in position 1 { output = (short) E; //error signal as overall output } else if (out_type == 2) { output = (short) dplusn;//desired+noise }

CodecDataOut.Channel[ LEFT] = output;//overall output resultCodecDataOut.Channel[RIGHT] = 0;/* end your code here */

WriteCodecData(CodecDataOut.UINT);// send output data to portWriteDigitalOutputs(0); // Write to GPIO J15, pin 6; end ISR timing pulse

}

//White noise generator for filter noise testinglong int rand_int(void){

static long int a = 100001;

a = (a*125) % 2796203;return a;

}

// Welch, Wright, & Morrow, // Real-time Digital Signal Processing, 2011 ///////////////////////////////////////////////////////////////////////// Filename: main.c//


C6x Code Examples

// Synopsis: Main program file for demonstration code/////////////////////////////////////////////////////////////////////////


int main(){

// call StartUp for application specific code// defined in each application directoryStartUp();

// main stalls here, interrupts drive operation while(1) {

; } }

// Welch, Wright, & Morrow, // Real-time Digital Signal Processing, 2011

///////////////////////////////////////////////////////////////////////// Filename: StartUp.c//// Synopsis: Adaptive filter initialize/////////////////////////////////////////////////////////////////////////


#define N 64 //# of weights (coefficients)extern float w[N];//weights for adapt filterextern float delay[N];//input buffer to adapt filter

void StartUp(){

InitDigitalOutputs();// initialize filter weights and delay states

DSP_Init();

int T = 0; for (T = 0; T < N; T++) //init variables { w[T] = 0.0; //init weights of adaptive FIR delay[T] = 0.0; //init buffer for delay sampless }}



• The left channel input (desired) is added to the right chan-nel input (noise) and forms the LMS input (dplusn)

• The right channel input by itself forms the input (noise)

• A direct form FIR filter is implemented using floats in thefirst for loop

• In the second for loop the filter tap weights w[n] areupdated using coefficient beta which can be adjusted usinga GEL file; the filter history in delay[n] is also updated inthis loop

• A second GEL slider controls which filter output goes to theleft codec channel

• This adaptive filter was tested with two sinusoid inputs, oneat 1 kHz as the desired signal and one at 2.5 kHz the noisesignal

• Analog outputs were captured using the PC sound card and

d n[ ]

x n[ ]

GEL Slider to togglebetween dplusnand the error E

GEL slider to set the value of coefficientbeta between 1E-12 and 1E-10


C6x Code Examples

spectrum analyzed using MATLAB

0 500 1000 1500 2000 2500 3000 3500 4000−60

−50

−40

−30

−20

−10

0

10

20

30

Frequency

Pow

er S

pect

rum

Mag

nitu

de (

dB)

0 500 1000 1500 2000 2500 3000 3500 4000−60

−50

−40

−30

−20

−10

0

10

20

30

Frequency

Pow

er S

pect

rum

Mag

nitu

de (

dB)

Desired 1.0 kHz Sinusoid + 2.5 kHz Noise

Error Signal Output: Noise Removed



Adaptive Line Enhancer

• In this first example a modified version of Chassaing’sAdaptpredict.c program is considered

• In this program the following system is implemented

• Floating point C code is shown below: //Adaptpredict_AIC23.c Adaptive predictor to cancel interference

//using interrupts

// Enhanced version of Chassaing program

// by Mark Wickert Spring 2009

//**********************************************************

#include "DSK6713_AIC23.h"

//set sampling rate {8,16,24,32,44.1,48,96}

Uint32 fs=DSK6713_AIC23_FREQ_8KHZ;

#define DSK6713_AIC23_INPUT_MIC 0x0015

#define DSK6713_AIC23_INPUT_LINE 0x0011

Uint16 inputsource=DSK6713_AIC23_INPUT_LINE; // select input

//Uint16 inputsource=DSK6713_AIC23_INPUT_MIC; // select input

//***********************************************************

// For left and right channel processing

// left and right help in packed 32 bit word

#define LEFT 0

#define RIGHT 1

union {Uint32 combo; short channel[2];} AIC23_data;

Σe n[ ]

y n[ ]x n Δ–[ ]Adaptive

Filter

x n[ ] NB n[ ] BB n[ ]+=

z Δ–

+

-


C6x Code Examples

// Program globals

#define N 64 //# of coefficients of adapt FIR

#define Ns 10 //length splusn buffer

float splusn[Ns]; //buffer wideband signal+interference

float w[N]; //buffer for weights of adapt FIR

float delay[N]; //buffer for input to adapt FIR

Uint32 out_type = 1;//output type for slider

float alpha = 10;//alpha from slider

interrupt void c_int11() //interrupt service routine

{

short i;

float yn, E;//yn=out adapt FIR, error signal

float wb_signal;//wideband desired signal

float noise; //external interference

short output;//output to codec

float beta = 1E-13;//rate of convergence

beta *= alpha;//scale beta by slider alpha value

AIC23_data.combo = input_sample(); //in l&r as 32-bit

wb_signal = (float) AIC23_data.channel[LEFT]; //desired on l

//noise = (float) AIC23_data.channel[RIGHT]; //noise on r

noise = 0.1*((short) rand_int());//intern noise

splusn[0] = wb_signal + noise; //wb signal+interference

delay[0] = splusn[3]; //delayed input to adapt FIR

yn = 0; //init output of adapt FIR

for (i = 0; i < N; i++)

yn += (w[i] * delay[i]); //output of adapt FIR filter

E = splusn[0] - yn; //(wb+noise)-out adapt FIR

for (i = N-1; i >= 0; i--)

{

w[i] = w[i]+(beta*E*delay[i]);//update wgts of FIR

delay[i+1] = delay[i]; //update buf delay samps

if (i < Ns-1) {

splusn[i+1] = splusn[i]; //update buf corr wb

}



}

//out_type = 2;

/*

if (out_type == 1)//if slider in position 1

{

output = ((short)E);//WB signal

//output = ((short)wb_signal);//WB signal

}

else if (out_type == 2)

{

output =((short)yn);//NB signal

//output =((short)wb_signal);//NB signal

}*/

output =((short)yn);

output_sample(output);//output resultto left chan.

return;

}

void main() //main function

{

int T = 0;

for (T = 0; T < N; T++) //init variables

{

w[T] = 0.0; //init weights of adaptive FIR

delay[T] = 0.0; //init buffer for delay samples

if (T < Ns)

{

splusn[T] = 0; //init wideband+interference

}

}

comm_intr(); //init DSK, codec, McBSP

while(1); //infinite loop

}

//16 bit word random number generator

int rand_int(void)

{

static int a = 100001;

a = (a*125) % 2796203;


C6x Code Examples

return a;

}

• The inputs are assumed to be a wideband signal, e.g., noise,and a narrowband signal, e.g., a sinusoid

• The inputs arrive at the left and right codec inputs and aresummed in code as wb_signal + noise

• An input delay buffer of Ns samples is created in the arraysplusn[], here Ns = 10

• splusn[3] serves as , so here

• A GEL file allows the left channel of the codec to outputeither the wideband signal at setting 1 or the narrowband sig-nal at setting two

• The adaptation coefficient is set to be

x n Δ–[ ] Δ 3=

10 12–



• A 1.8 kHz sinusoid is input from a function generator andwideband noise is input from another function generator, thefrequency response of the converged filter weights w[n] isdisplayed in CCS

• The corresponding spectra of and are captured viathe PC sound card

y n[ ] e n[ ]

0 500 1000 1500 2000 2500 3000 3500 4000−60

−50

−40

−30

−20

−10

0

10

20

30

Frequency

Pow

er S

pect

rum

Mag

nitu

de (

dB)

The Wideband Signal Output


C6x Code Examples

0 500 1000 1500 2000 2500 3000 3500 4000−60

−50

−40

−30

−20

−10

0

10

20

30

Frequency

Pow

er S

pect

rum

Mag

nitu

de (

dB)

The Narrowband Signal Output




Documents

Adaptive Filters Chapter · Adaptive Filters Introduction The term adaptive filter implies changing the characteristic of a ... 1.Simon Haykin, Adaptive Filter Theory, fourth edition,