Digital Signal Processing II Chapter 3: FIR & IIR Filter Design Marc Moonen Dept. E.E./ESAT, K.U.Leuven [email protected]

Digital Signal Processing II

Chapter 3: FIR & IIR Filter Design

Marc Moonen

Dept. E.E./ESAT, K.U.Leuven

[email protected]

www.esat.kuleuven.be/scd/

mailto:[email protected]

DSP-IIVersion 2010-2011 Chapter-3: FIR/IIR Filter Design p. 2

PART-I : Filter Design/Realization

• Step-1 : define filter specs

(pass-band, stop-band, optimization criterion,…)

• Step-2 : derive optimal transfer function

FIR or IIR design

• Step-3 : filter realization (block scheme/flow graph)

direct form realizations, lattice realizations,…

• Step-4 : filter implementation (software/hardware)

finite word-length issues, …

question: implemented filter = designed filter ?

Chapter-3

Chapter-4

Chapter-5


Chapter-3 : FIR & IIR Filter Design

• FIR filters– Linear-phase FIR filters– FIR design by optimization Weighted least-squares design, Minimax design

– FIR design in practice `Windows’, Equiripple design, Software (Matlab,…)

• IIR filters– Poles and Zeros– IIR design by optimization Weighted least-squares design, Minimax design

– IIR design in practice Analog IIR design : Butterworth/Chebyshev/elliptic Analog->digital : impulse invariant, bilinear transform,… Software (Matlab)


FIR Filters

FIR filter = finite impulse response filter

• Also known as `moving average filters’ (MA)• N poles at the origin z=0 (hence guaranteed stability) • N zeros (zeros of B(z)), àll zero’ filters• corresponds to difference equation

• impulse response

NNNzbzbb

z

zBzH ...

)()( 1

10

][....]1[.][.][ 10 Nkubkubkubky N

,...0]1[,][,...,]1[,]0[ 10 NhbNhbhbh N


Linear Phase FIR Filters

• Non-causal zero-phase filters :

example: symmetric impulse response

h[-L],….h[-1], h[0] ,h[1],...,h[L]

h[k]=h[-k], k=1..L

frequency response is

i.e. real-valued (=zero-phase) transfer function

kL

L

kk

L

Lk

kjj kdekheH0

. ).cos(....].[)(

xee jxjx cos.2



• Causal linear-phase filters = non-causal zero-phase + delay

example: symmetric impulse response & N even

h[0],h[1],….,h[N]

N=2L (even)

h[k]=h[N-k], k=0..L

frequency response is

= i.e. causal implementation of zero-phase filter, by

introducing (group) delay

L

kk

LjN

k

kjj kdeekheH00

. ).cos(.....].[)(

kN

Lj

ez

L ezj

0



Type-1 Type-2 Type-3 Type-4N=2L=even N=2L+1=odd N=2L=even N=2L+1=oddsymmetric symmetric anti-symmetric anti-symmetric

h[k]=h[N-k] h[k]=h[N-k] h[k]=-h[N-k] h[k]=-h[N-k] zero at zero at zero atLP/HP/BP LP/BP BP HP

PS: `modulating’ Type-2 with 1,-1,1,-1,.. gives Type-4 (LP->HP)PS: `modulating’ Type-4 with 1,-1,1,-1,.. gives Type-2 (HP->LP)PS: `modulating’ Type-1 with 1,-1,1,-1,.. gives Type-1 (LP<->HP)PS: `modulating’ Type-3 with 1,-1,1,-1,.. gives Type-3 (BP<->BP)

PS: IIR filters can NEVER have linear-phase property ! (proof see literature)

L

kk

Nj kdej0

2/ ).cos(.).2

sin(.

L

kk

Nj kde0

2/ ).cos(..

L

kk

Nj kde0

2/ ).cos(.).2

cos(

1

0

2/ ).cos(.).sin(L

kk

Nj kdje

,0 0


Filter Specification

0 0.5 1 1.5 2 2.5 30

0.2

0.4

0.6

0.8

1

1.2

Passband Ripple

Stopband Ripple

Passband Cutoff -> <- Stopband Cutoff

Ex: Low-pass

S

PP1

S


FIR Filter Design by Optimization

(I) Weighted Least Squares Design :• select one of the basic forms that yield linear phase

e.g. Type-1

• specify desired frequency response (LP,HP,BP,…)

• optimization criterion is

where is a weighting function

)(.).cos(..)( 2/

0

2/ AekdeeH NjL

kk

Njj

)(.)( 2/ d

Njd AeH

),...,(

2

,...,

2

,...,

0

00)()()(min)()()(min

L

LL

ddF

ddddj

dd dAAWdHeHW

0)( W



• …this is equivalent to

= `Quadratic Optimization’ problem

...

)cos(...)cos(1)(

)().().(

)().().(

...

}.2..{min

0

0

10

),...,( 0

Lc

dcAWp

dccWQ

dddx

pxxQx

T

d

T

LT

ddF

TTx

L

pQxOPT .1


0 0.5 1 1.5 2 2.5 30

0.2

0.4

0.6

0.8

1

1.2

Passband Ripple

Stopband Ripple

Passband Cutoff -> <- Stopband Cutoff


• Example: Low-pass design

optimization function is

i.e.

...

band-stop

)(.

band-pass

1)(),...,( 2

0

2

0

dAdAddFS

P

L

band)-(stop ,0)(

band)-(pass ,1)(

Sd

Pd

A

A

,)( ,1)( SP WW



• a simpler problem is obtained by replacing the F(..) by…

where the wi’s are a set of (n) selected sample frequencies

This leads to an equivalent (`discretized’) quadratic optimization function:

+++ : simple - - - : unpredictable behavior in between sample freqs.

i

TTid

L

iT

iL pxxQxA

d

d

cWddF .2..)(:).()(),...,(

2

0

0

Compare to p.10

... ,)().().( ,)().().( i

iidii

iT

ii cAWpccWQ

i

idiiL AAWddF2

0 )()().(),...,(

pQxOPT .1



• This is often supplemented with additional constraints, e.g. for pass-band and stop-band ripple control :

• The resulting optimization problem is :

minimize : (=quadratic function)

subject to (=pass-band constraints)

(=stop-band constraints)

= `Quadratic Programming’ problem

LT

L

dddx

ddF

...

...),...,(

10

0

SS

PP

bxA

bxA

.

.

ripple) band-stop is ( ,..., freqs. band-stopfor ,)(

ripple) band-pass is ( ,..., freqs. band-passfor ,1)(

S2,1,S,

P2,1,P,

SSiS

PPiP

A

A



(II) `Minimax’ Design :• select one of the basic forms that yield linear phase

e.g. Type-1



where is a weighting function

)(.).cos(..)( 2/

0

2/ AekdeeH NjL

kk

Njj

)(.)( 2/ d

Njd AeH

)()().(maxmin)()().(maxmin 0,...,0,..., 00

dddd

jdd AAWHeHW

LL

0)( W



• this is equivalent to

• the constraint is equivalent to a so-called `semi-definiteness’ constraint

where D>=0 denotes that the matrix is positive semi-definite

)cos(...)cos(1)(

...

with

0for )().().(

subject to

min

10

22

Lc

dddx

AxcW

T

LT

dT

x

01))().().((

))().().(()(

dT

dT

AxcW

AxcWD

Skip this slide


Filter Design by Optimization

• a realistic way to implement these constraints, is to impose the constraints (only) on a set of sample frequencies :

i.e. a `Semi-Definite Programming’ (SDP) problem, for which efficient interior-point algorithms and software are available.

0

)(...00

:::

0...)(0

0...0)(

subject to

min

2

1

m

x

D

D

D

Skip this slide



• Conclusion: (I) weighted least squares design

(II) minimax design

provide general `framework’, procedures to translate filter design problems into standard optimization problems

• In practice (and in textbooks): emphasis on specific (ad-hoc) procedures :

- filter design based on `windows’

- equiripple design


FIR Filter Design using `Windows’

Example : Low-pass filter design• ideal low-pass filter is

• hence ideal time-domain impulse response is

• truncate hd[k] to N+1 samples :

• add (group) delay to turn into causal filter

0

1)(

C

CdH

kk

kdeeHkh

c

ckjdd

j

)sin(

....).(2

1][ .

otherwise 0

2/2/ ][][

NkNkhkh d



Example : Low-pass filter design (continued)• PS : it can be shown that the filter obtained by such time-domain

truncation is also obtained by using a weighted least-squares design procedure with the given Hd, and weighting function

• truncation corresponds to applying a `rectangular window’ :

• +++: simple procedure (also for HP,BP,…)

• - - - : truncation in time-domain results in `Gibbs effect’ in frequency domain, i.e. large ripple in pass-band and stop-band (at band edge discontinuity), which cannot be reduced by increasing the filter order N.

][].[][ kwkhkh d

otherwise 0

1][

2/2/ NkNkw

1)( W



Remedy : apply windows other than rectangular window:• time-domain multiplication with a window function w[k] corresponds to

frequency domain convolution with W(z) :

• candidate windows : Han, Hamming, Blackman, Kaiser,…. (see textbooks, see DSP-I)

• window choice/design = trade-off between side-lobe levels (define peak pass-/stop-band ripple) and width main-lobe (defines transition bandwidth)

][].[][ kwkhkh d

)(*)()( zWzHzH d


FIR Equiripple Design

• Starting point is minimax criterion, e.g.

• Based on theory of Chebyshev approximation and the àlternation theorem’, which (roughly) states that the optimal d’s are such that the `max’ (maximum weighted approximation error) is obtained at L+2 extremal frequencies…

…that hence will exhibit the same maximum ripple (èquiripple’)• Iterative procedure for computing extremal frequencies, etc. (Remez

exchange algorithm, Parks-McClellan algorithm) • Very flexible, etc., available in many software packages• Details omitted here (see textbooks)

)(maxmin)()().(maxmin 0,...,0,..., 00 EAAW

LL ddddd

2,..,1for )()(max0 LiEE i


FIR Filter Design Software

• FIR Filter design abundantly available in commercial software

• Matlab: b=fir1(n,Wn,type,window), windowed linear-phase FIR design, n is filter

order, Wn defines band-edges, type is `high’,`stop’,…

b=fir2(n,f,m,window), windowed FIR design based on inverse Fourier transform with frequency points f and corresponding magnitude response m

b=remez(n,f,m), equiripple linear-phase FIR design with Parks-McClellan (Remez exchange) algorithm

See exercise sessions


IIR filters

Rational transfer function :

•

N poles (zeros of A(z)) , N zeros (zeros of B(z))

• infinitely long impulse response• stable iff poles lie inside the unit circle• corresponds to difference equation

= also known as ÀRMA’ (autoregressive-moving average)

NN

NN

NNN

NNN

zaza

zbzbb

azaz

bzbzb

zA

zBzH

...1

...

...

...

)(

)()(

11

110

11

110

][....]1[.][.][....]1[.][ 101 NkubkubkubNkyakyaky NN

'`

1

'`

10 ][....]1[.][....]1[.][.][AR

N

MA

N NkyakyaNkubkubkubky


IIR Filter Design

+++• low-order filters can produce sharp frequency response

• low computational cost

- - -• design more difficult• stability should be checked/guaranteed• phase response not easily controlled

(e.g. no linear-phase IIR filters)• coefficient sensitivity, quantization noise, etc. can be a

problem (see Chapter-5)


IIR filters

Frequency response versus pole-zero location : (cfr. frequency response is z-transform evaluated on the unit circle)

Example-1 :

Low-pass filter

poles at

j2.08.0

DC (z=1)

Nyquist freq (z=-1)

pole pole

ReIm

pole near unit-circle introduces `peak’ in frequency response

hence pass-band can be set by pole placement


IIR filters

Frequency response versus pole-zero location :

Example-2 :

Low-pass filter

poles at

zeros at

j2.08.0

j66.075.0 DC

Nyquist freq

zero

pole pole

zero near (or on) unit-circle introduces `dip’ (or transmision zero) in freq. response

hence stop-band can be emphasized by zero placement


IIR Filter Design by Optimization

(I) Weighted Least Squares Design :• IIR filter transfer function is



where is a weighting function• stability constraint :

)(dH

),...,,,...,(

2

,...,,,...,

10

10)()()(min

NN

NN

aabbF

dj

aabb dHeHW

0)( W

NN

NN

zaza

zbzbb

zA

zBzH

...1

...

)(

)()(

11

110

1,0)( zzA



(II) `Minimax’ Design :• IIR filter transfer function is



where is a weighting function• stability constraint :

)(dH

0)( W

NN

NN

zaza

zbzbb

zA

zBzH

...1

...

)(

)()(

11

110

1,0)( zzA

)()().(maxmin 0,...,,,..., 10

dj

aabb HeHWNN



These optimization problems are significantly more complex than those for the FIR design case… :

• Problem-1: presence of denominator polynomial leads to non-linear (non-quadratic) optimization– A possible procedure (`Steiglitz-McBride’) consists in iteratively

(k=1,2,…) minimizing

…which leads to iterative Quadratic Programming, etc..

(similar for minimax)

2

1

2

,1,,0,

)(

)()(

)().()()(),...,,,...,(

k

k

jd

jkNkkNkk

A

WW

deAHeBWaabbF

Skip this part



These optimization problems are significantly more

complex than those for the FIR design case… :

• Problem-2: stability constraint (zeros of a high-order polynomial are related to the polynomial’s

coefficients in a highly non-linear manner)

– Solutions based on alternative stability constraints, that e.g. are affine functions of the filter coefficients, etc…

– Topic of ongoing research, details omitted here



• Conclusion: (I) weighted least squares design

(II) minimax design

provide general `framework’, procedures to translate filter design problems into ``standard’’ optimization problems

• In practice (and in textbooks): emphasis on specific (ad-hoc) procedures :

- IIR filter design based analog filter design (s-domain

design) and analog->digital conversion

- IIR filter design by modeling = direct z-domain design

(Pade approximation, Prony, etc., not addressed here)


Analog IIR Filter Design

Commonly used analog filters :• Lowpass Butterworth filters - all-pole filters (H) characterized by a specific

magnitude response (G): (N=filter order) - poles of G(s)=H(s)H(-s) are equally spaced on circle of radius

- Then H(jw) is found to be monotonic in pass-band & stop-band, with `maximum flat response’, i.e. (2N-1) derivatives are zero at

N

c

N

c

jHjG

ssHsHsG

2

2

2

2

)(1

1)()(

)(1

1)().()(

c

poles of H(s)

N=4

poles of H(-s)

,0

N

c



Commonly used analog filters :• Lowpass Chebyshev filters (type-I) all-pole filters characterized by magnitude response

(N=filter order)

is related to passband ripple

are Chebyshev polynomials:

)(1

1)()(

...)().()(

22

2

cNT

jHjG

sHsHsG

)(xTN

)()(.2)(

...

12)(

)(

1)(

21

22

1

0

xTxTxxT

xxT

xxT

xT

NNN

Skip this slide



Commonly used analog filters :• Lowpass Chebyshev filters (type-I) All-pole filters, poles of H(s)H(-s) are on ellipse in s-plane Equiripple in the pass-band

Monotone in the stop-band • Lowpass Chebyshev filters (type-II) Pole-zero filters based on Chebyshev polynomials Monotone in the pass-band Equiripple in the stop-band

• Lowpass Elliptic (Cauer) filters Pole-zero filters based on Jacobian elliptic functions Equiripple in the pass-band and stop-band

(hence) yield smallest-order for given set of specs )(1

1)(

2

2

cNU

jH

Skip this slide



Frequency Transformations :• Principle : prototype low-pass filter (e.g. cut-off frequency = 1

rad/sec) is transformed to properly scaled low-pass, high-pass, band-

pass, band-stop,… filter • example: replacing s by moves cut-off frequency to

• example: replacing s by turns LP into HP, with cut-off frequency

• example: replacing s by turns LP into BP

• etc...

C

s

C

sC

C

).(

.

12

212

s

s


Analog -> Digital

• Principle : design analog filter (LP/HP/BP/…), and then convert it to a digital filter.

• Conversion methods:

- convert differential equation into difference equation

- convert continuous-time impulse response into discrete-time

impulse response

- convert transfer function H(s) into transfer function H(z)

• Requirement: the left-half plane of the s-plane should map into the inside of the unit circle in the z-plane, so that a stable analog filter is converted into a stable digital filter.


Analog -> Digital

• Conversion methods: (I) convert differential equation into difference equation : -in a difference equation, a derivative dy/dt is replaced by a `backward difference’ (y(kT)-y(kT-T))/T=(y[k]-y[k-1])/T, where T=sampling interval. -similarly, a second derivative, etc… -eventually (details omitted), this corresponds to replacing s by (1-1/z)/T

in Ha(s) (=analog transfer function) :

-stable analog filters are mapped into stable digital filters, but pole location for digital filter confined to only a small region (o.k. only for LP or BP)

T

zsa sHzH 11)()(

jws-plane z-plane

j

1 0

10

zs

zs

Skip this slide


Analog -> Digital

• Conversion methods:

(II) convert continuous-time impulse response into discrete-time impulse response :

-given continuous-time impulse response ha(t), discrete-time impulse

response is where T=sampling interval.

-eventually (details omitted) this corresponds to a (many-to-one) mapping

-aliasing (!) if continuous-time response has significant frequency

content above the Nyquist frequency

).(][ Tkhkh a

s-planejw

z-planej

1

sTez

1/

10

zTjs

zs

Skip this slide


Analog -> Digital

• Conversion methods: (III) convert continuous-time system transfer function into

discrete-time system transfer function : Bilinear Transform -mapping that transforms (whole!) jw-axis of the s-plane into unit circle

in the z-plane only once, i.e. that avoids aliasing of the frequency components.

-for low-frequencies, this is an approximation of -for high frequencies : significant frequency compression (`warping’) (sometimes pre-compensated by `pre-warping’)

s-planejw

z-planej

1

)1

1.(

21

1)()(

z

z

Tsa sHzH

1.

10

zjs

zs

sTez


IIR Filter Design Software

• IIR filter design considerably more complicated than FIR design (stability, phase response, etc..)

• (Fortunately) IIR Filter design abundantly available in commercial software

• Matlab: [b,a]=butter/cheby1/cheby2/ellip(n,…,Wn),

IIR LP/HP/BP/BS design based on analog prototypes, pre-warping,

bilinear transform, …

immediately gives H(z) (oef!)

analog prototypes, transforms, … can also be called individually

filter order estimation tool

etc... See exercise sessions

Documents

Digital Signal Processing II Chapter 3: FIR & IIR Filter Design Marc Moonen Dept. E.E./ESAT, K.U.Leuven [email protected]