Digital Audio Signal Processing DASP - KU Leuvendspuser/dasp...– Multi-channel Wiener filter (=spectral+spatial filtering) 2 Digital Audio Signal Processing Version 2017-2018 Lecture-4:

1

Marc Moonen Dept. E.E./ESAT-STADIUS, KU Leuven

[email protected] homes.esat.kuleuven.be/~moonen/

Digital Audio Signal Processing

DASP

Lecture-4: Noise Reduction-III Adaptive Beamforming

Digital Audio Signal Processing Version 2017-2018 Lecture-4: Adaptive Beamforming 2 / 34

Overview

•  Beamforming / Recap

•  Adaptive Beamforming –  Adaptive beamforming goal & set-up –  LCMV beamformer –  Generalized sidelobe canceler

•  Multi-channel Wiener filter for multi-microphone noise reduction –  Multi-microphone noise reduction problem –  Multi-channel Wiener filter (=spectral+spatial filtering)

2


F1(ω)

F2 (ω)

FM (ω)

z[k]

y1[k]

+ : dM

θ

dM cosθ

Recap 1/4

Filter-and-sum Beamforming Classification:

Fixed beamforming: data-independent, fixed filters Fm e.g. matched filtering, superdirective, … Adaptive beamforming: data-dependent filters Fm e.g. LCMV-beamformer, generalized sidelobe canceler

= This lecture

= Previous lecture


Recap 2/4 Data model: source signal

•  Microphone signals are filtered versions of source signal S(ω) at angle θ

•  Stack all microphone signals (m=1..M) in a vector d is `steering vector’

[ ]TjM

j MeHeH )()(1 ).,(...).,(),( 1 θωτθωτ θωθωθω −−=d

)( ..),(),(shift phase dep.-pos.

)(

pattern dir.

ωθωθω θωτ SeHY mjmm

!"!#$!"!#$−=

)().,(),( ωθωθω SdY =

3


Recap 3/4

Data model: source signal + noise •  Microphone signals are corrupted by additive noise

Noise correlation matrix is

•  Output signal after `filter-and-sum’ is

[ ]TMNNN )(...)()()( 21 ωωωω =N

})().({)( Hnoise E ωωω NNΦ =

)()().,(),( ωωθωθω NdY += S

)()}.,().({),().(),().(),(1

* ωθωωθωωθωωθω SYFZ HHM

mmm dFYF ===∑

=


Recap 4/4 Definitions:

•  Array directivity pattern = `transfer function’ for source at angle θ

•  Array Gain = improvement in SNR for source at angle θ ( -π<θ< π )

White Noise Gain =array gain for spatially uncorrelated noise

Directivity =array gain for diffuse noise

),().()(),(),( θωω

ωθω

θω dFHSZH ==

)().().(),().(

),(2

ωωω

θωωθω

FΓFdF

noiseH

H

input

output

SNRSNR

G ==

Iwhitenoise =Γ

⎟⎟⎠

⎞⎜⎜⎝

⎛ −=Γ

cddf ijsdiffuse

ij

)(sinc)(

ωω

4


Overview





Adaptive beamforming Goal & Set-up

•  Adaptive filter-and-sum structure: –  Aim is to minimize noise output power, while maintaining a chosen

response for a given steering angle (and/or other linear constraints, see below). –  This is similar to operation of a matched filter beamformer (in white

noise) or superdirective beamformer (in diffuse noise), see Lecture-3, (**) p.22 & p.30, but now noise field is unknown & adapted to!

F1(ω)

F2 (ω)

FM (ω)

z[k]

y1[k]

+ : yM [k]

5


Adaptive beamforming Goal & Set-up

•  Adaptive filter-and-sum structure: –  Implemented as adaptive FIR filter :

[ ] ]1[...]1[][][ T+−−= Nkykykyk mmmmy

∑=

==M

mm

Tm

T kkkz1

][][][ yfyf

[ ]TTM

TT kkkk ][][][][ 21 yyyy …=

[ ]TTM

TT ffff …21=[ ] ... T

1,1,0, −= Nmmmm ffff

yM [k]

F1(ω)

F2 (ω)

FM (ω)

z[k]

y1[k]

+ : yM [k]


Overview




6


LCMV beamformer

LCMV = Linearly Constrained Minimum Variance –  f designed to minimize output power (variance) (read on..) z[k] :

–  To avoid desired signal cancellation, add (J) linear constraints

Example: fix array response for steering angle ψ & sample freqs ωi

For sufficiently large J constrained output power minimization approximately corresponds to constrained output noise power minimization (if source angle is equal to steering angle) (why?)

Ryy[k]= E{y[k].y[k]T}{ } fRf

ff⋅⋅= ][min][min 2 kkzE yy

T

FT (ωi ).d(ωi,ψ) =Lecture4-p18

dT (ωi,ψ).f =1 i =1..J

JJMNMNT ℜ∈ℜ∈ℜ∈= × bCfbfC ,, .


LCMV beamformer

LCMV = Linearly Constrained Minimum Variance –  f designed to minimize output power (variance) (read on..) z[k] :

–  To avoid desired signal cancellation, add (J) linear constraints

–  Solution is (obtained using Lagrange-multipliers, etc..):

Ryy[k]= E{y[k].y[k]T}{ } fRf

ff⋅⋅= ][min][min 2 kkzE yy

T

( ) bCRCCRf 111 ][][ −−− ⋅⋅⋅⋅= kk yyT

yyopt

JJMNMNT ℜ∈ℜ∈ℜ∈= × bCfbfC ,, .

7


Overview





Generalized sidelobe canceler (GSC)

GSC = Adaptive filter formulation of the LCMV-problem Constrained optimisation is reformulated as a constraint pre-processing,

followed by an unconstrained optimisation, as follows: –  LCMV-problem is

–  Define `blocking matrix’ Ca,, columns spanning the null-space of C

–  Define ‘quiescent response vector’ fq satisfying constraints

–  Parametrize all f’s that satisfy constraints (verify!)

i.e. filter f can be decomposed in a fixed part fq and a variable part Ca. fa

bfCfRff

=⋅⋅⋅ Tyy

T k ,][min

f = fq −Ca.fa

)( JMNMNa

−×ℜ∈C

JJMNMN ℜ∈ℜ∈ℜ∈ × bCf ,,

CT .Ca = 0

fq =C.(CT .C)−1b

fa ∈ℜ(MN−J )

8


Generalized sidelobe canceler (GSC)

GSC = Adaptive filter formulation of the LCMV-problem Constrained optimisation is reformulated as a constraint pre-processing,

followed by an unconstrained optimisation, as follows: –  LCMV-problem is

–  Unconstrained optimization of fa : (MN-J coefficients)

bfCfRff

=⋅⋅⋅ Tyy

T k ,][min JJMNMN ℜ∈ℜ∈ℜ∈ × bCf ,,

).].([.).(min aaqyyT

aaq ka

fCfRfCff −−

f = fq −Ca.fa

fa ∈ℜ(MN−J )


GSC (continued)

–  Hence unconstrained optimization of fa can be implemented as an adaptive filter (adaptive linear combiner), with filter inputs (=‘left- hand sides’) equal to and desired filter output (=‘right-hand side’) equal to –  LMS algorithm :

Generalized sidelobe canceler

}))..][().][({min...).].([.).(min

2][~][

aa

kd

qaaqyyT

aaq

T

aakkEk fCyfyfCfRfCf

ky

TTff

!"!#$!"!#$ΔΔ==

−==−−

][~ ky

][kd

])[..][][..(][..][]1[][~][][~

kkkkkk a

k

aT

kd

Tq

k

Taaa

T

fCyyfyCffyy!"!#$"#$!"!#$ −+=+ µ

9


Generalized sidelobe canceler GSC then consists of three parts: •  Fixed beamformer (cfr. fq ), satisfying constraints (‘LC’) but not yet

minimum variance (‘MV’) ), creating `speech reference’

•  Blocking matrix (cfr. Ca), placing spatial nulls in the direction of the speech source (at sampling frequencies) (cfr. C’.Ca=0), creating MN-J `noise references’

•  Multi-channel adaptive filter (linear combiner) your favourite one, e.g. LMS

][kd

][~ ky



A popular & significantly cheaper GSC realization is as follows

Note that some reorganization has been done: the blocking matrix now generates (typically) M-1 (instead of MN-J) noise references, the multichannel adaptive filter performs FIR-filtering on each noise reference (instead of merely scaling in the linear combiner). Philosophy is the same, mathematics are different (details on next slide).

Postproc

y1

yM

10



•  Math details: (for Delta’s=0)

[ ][ ]

[ ]][.~][~

]1[~...]1[~][~][~

~...00:::0...~00...0~

][...][][][

]1[...]1[][][

][.][.][~

:1:1

:1:1:1

permuted,

21:1

:1:1:1permuted

permutedpermuted,

kk

Lkkkk

kykykyk

Lkkkk

kkk

MTaM

TTM

TM

TM

Ta

Ta

Ta

Ta

TMM

TTM

TM

TM

Ta

Ta

yCy

yyyy

C

CC

C

y

yyyy

yCyCy

=

+−−=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

=

=

+−−=

==

=input to multi-channel adaptive filter

select `sparse’ blocking matrix such that :

=use this as blocking matrix now



•  Blocking matrix Ca (for scheme p.19)

–  Creating (M-1) independent noise references by placing spatial nulls in look-direction

–  different possibilities

(broadside steering)

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

−

−

=

100101010011

TaC

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−−

−−

=

111111111111

TaC 0

20004000

60008000 0 45 90 135 180

0

0.5

1

1.5

Angle (deg)

Frequency (Hz)

Griffiths-Jim

Walsh

f Tq = 1 1 1 1!"

#$

11



•  Problems of GSC:

–  Impossible to reduce noise from steering angle direction

–  Reverberation effects cause signal leakage in noise references

–  Adaptive filter should only be updated when no speech is present to avoid signal cancellation Hence requires voice activity detection (VAD)


Overview




12


ym[k]= sm[k]+ nm[k],m =1...M

Multi-Microphone Noise Reduction Problem

(some) speech estimate

speech source

noise source(s)

microphone signals ][kn

][ks

? ][ks⌢

speech part noise part

M=

num

ber o

f mic

roph

ones

compare to Lecture-2



Will estimate speech part in microphone 1 (*) (**)

? ][1 ks⌢

(*) Estimating s[k] is more difficult, would include dereverberation (**) This is similar to single-microphone model (Lecture-2), where additional microphones (m=2..M) help to get a better estimate

][kn

][ks

ym[k]= sm[k]+ nm[k],m =1...M

13



•  Data model: Frequency domain.. See Lecture-3 on multi-path propagation

(with q left out for conciseness)

Hm(ω) is complete transfer function from speech source position to m-the microphone

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+=

+=

)(

)()(

)(.

)(

)()(

)(

)()(

)()().( )()()(

2

1

2

1

2

1

ω

ω

ω

ω

ω

ω

ω

ω

ω

ω

ωωω

ωωω

MMM N

NN

S

H

HH

Y

YY

S

!!!

NdNSY


Overview




14


Multi-Channel Wiener Filter (MWF)

•  Data model:

•  Will use linear filters to obtain speech estimate

•  Wiener filter (=linear MMSE approach)

Note that (unlike in DSP-CIS) `desired response’ signal S1(w) is obviously unknown here (!)

)()().( )( ωωωω NdY += S

)().()().()(ˆ1

*1 ωωωωω YFH

M

mmm YFS ==∑

=

})().()({min2

1)( ωωωω YFFHSE −



•  Wiener filter solution is (see DSP-CIS)

–  All quantities can be computed ! –  Special case of this is single-channel Wiener filter formula (Lecture-2) –  In practice, use alternative to ‘subtraction’ operation (see literature)

( ))}().({)}().({.)}().({

...

.)}().({)}().({)(

*1

*1

1

)0)}().({(with

lationcrosscorre

*1

1

ationautocorrel*

1

ωωωωωω

ωωωωω

ωω

NEYEE

SEE

H

NE

H

NYYY

YYYF

S

−=

=

=

−

=

−

!! "!! #$!! "!! #$

compute during speech+noise periods

compute during noise-only periods

15


•  Note that… MWF combines spatial filtering with single-channel spectral filtering (as in Lecture-2)

if

then…

! "#$

%&%'(

)noise vectorsteering

2

1

)()(.)(

)(

)(

)()(

ωωω

ω

ω

ωω

Nd

Y

+=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

S

Y

YY

M

E{N(ω).NH (ω)} =Φnoise(ω)



…then it can be shown that

①  represents a spatial filtering (*) Compare to superdirective & delay-and-sum beamforming (Lecture-4) –  Delay-and-sum beamf. maximizes array gain in white noise field –  Superdirective beamf. maximizes array gain in diffuse noise field –  MWF maximizes array gain in unknown (!) noise field. MWF is operated without invoking any prior knowledge (steering vector/noise field) ! (the secret is in the voice activity detection… (explain))

(*) Note that spatial filtering can improve SNR, spectral filtering never improves SNR (at one frequency)


F(ω) =α(ω)scalar!.Φnoise

−1 (ω).d(ω)

Φnoise−1 (ω).d(ω)

16


…then it can be shown that

①  represents a spatial filtering (*)

②  represents an additional `spectral post-filter’ i.e. single-channel Wiener estimate (p.9), applied to output signal of spatial filter (prove it!)


α(ω) = ... =S(ω) 2 .H1

*(ω)dH (ω).Φnoise

−1 (ω).d(ω)( ). S(ω) 2 +1

)(ωα

F(ω) =α(ω)scalar!.Φnoise

−1 (ω).d(ω)

Φnoise−1 (ω).d(ω)


Multi-Channel Wiener Filter: Implementation

•  Implementation with short-time Fourier transform (see Lecture-2)

•  Implementation with time-domain linear filtering:

ymT [k]= ym[k] ym[k −1] ... ym[k − L +1]"

#$%

⌢s1[k]= ymT [k].fm

m=1

M

∑][kn

][ks

][1 ks⌢

][1 ky

filter coefficients

ym[k]= sm[k]+nm[k]

17


Solution is…

}].[][{min2

1 fyf kksE T−[ ][ ]TT

MTT

TTM

TT

kkkk ][...][][][

...

21

21

yyyy

ffff

=

=

[ ][ ] [ ]]}[.][{]}[.][{.}][.][{

]}[.][{.}][.][{

11

1

1

1

knkEkykEkkE

kskEkkE

T

T

nyyy

yyyf

−=

=

−

−

compute during speech+noise periods

compute during noise-only periods




Solution is… PS: Single-channel version of this is related/similar to single channel

subspace method, see Lecture-2, p.32 (check!)

}].[][{min2

1 fyf kksE T−[ ][ ]TT

MTT

TTM

TT

kkkk ][...][][][

...

21

21

yyyy

ffff

=

=

[ ][ ] [ ]]}[.][{]}[.][{.}][.][{

]}[.][{.}][.][{

11

1

1

1

knkEkykEkkE

kskEkkE

T

T

nyyy

yyyf

−=

=

−

−



Documents

Digital Audio Signal Processing DASP - KU Leuvendspuser/dasp...– Multi-channel Wiener filter (=spectral+spatial filtering) 2 Digital Audio Signal Processing Version 2017-2018 Lecture-4: