Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Blind Source Separation using Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
May, 2015
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Non-negative Matrix Factorization (Lee and Seung, 2001)
V ≈ V = WH
I V and V are spectrograms ∈ <≥0,M×N
I W are the bases where ∈ <≥0,M×K
I H is the activation matrix where H ∈ <≥0,K×N
I Positive → Meaningful
Example:
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Considering β-divergence as cost function
Definition:
dβ(x |y) =
1
β (β−1)
(xβ + (β − 1) yβ − β x yβ−1
)β ∈ R\{0, 1}
x (log x − log y) + (y − x) β = 1
xy− log x
y− 1 β = 0
Observations:
I Note that for different β some common cost functions are defined:
I β = 2: Euclidean distance
I β = 1: generalised Kullback-Leibler divergence
I β = 0: Itakura-Saito divergence
Gradient Descend and Multiplicative Updates:
I b = a− γ∇F (a)
I for γ small enough, then F (a) ≥ F (b)
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Prior Subspace Analysis: composing W
W is formed by two parts:
I Fixed Trained Patterns (Wtar ) where Wt ∈ <≥0,M×Ktar
I Adaptive Patterns (Wbgnd ) where Wt ∈ <≥0,M×Kbgnd
(Note that Ktar + Kbgnd = K)
Example:
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Objectives:
I Flexible Audio Source Separation Toolbox (FASST) (Ozerov et al. 2012)
is a general purpose framework able to model SS problems.
FASST: source model (I). Based on an excitation-filter model:
V ≈ V = WH =∑j
Vj ; Vj = Vexj � Vft
j
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
FASST: source model (II). Based on an excitation-filter model:
Vj = Vexj � Vft
j = Eexj Pex
j � Eftj Pft
j =
= (Wexj Uex
j Gexj Hex
j )� (Wftj Uft
j Gftj Hft
j )
Vexj is further decomposed as:
I Spectral patterns (Eexj ) are modulated by time activations (Pex
j ).
Spectral patterns (Eexj ) are linear combinations of:
I Narrow band spectral patterns Wexj with weights Uex
j .
Time activations Pexj are a combination of:
I time-localized patterns Hexj with weights Gex
j .
The filter spectral power Vftj is similarly expressed.
EXTREMELLY VERSATILE! Fixed and learnt basis.
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
FASST: source model (III). Based on an excitation-filter model:
Vj = Vexj � Vft
j = Eexj Pex
j � Eftj Pft
j =
= (Wexj Uex
j Gexj Hex
j )� (Wftj Uft
j Gftj Hft
j )
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Optimization problem: decomposition
FASST estimation of the sources is achieved in two steps:
I Step 1. Given initial parameter values θinit , the model parameters θ are
estimated from the mixture V using an iterative Generalized Expectation
Maximization (GEM) algorithm; where each iteration is achieved by
means of Multiplicative Update (MU) rules inspired from the NMF
methodology.
I Step 2. Given the mixture V and the estimated model parameters θ,
source estimates Vj are computed using Wiener filtering.
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Decomposition block diagram
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Implementation details (I)
I FASST is set for separating the bass, drums, melody and the remaining
instruments from a stereo music recording.
I Is available online written in Matlab.
Relies on two main concepts:
I Having prior knowledge about the sources to label them automatically
(PSA).
I Joint modelling of all sources instead of a cascading approach (separating
the drums, then the melody, etc.).
The sound scene is modelled with 12 sources (Vj ):
I 4 sources representing the bass.
I 4 sources drums.
I 4 sources modelling other instruments.
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Implementation details (II)
The bass and drums sources are constrained to Vj = Wexj Gex
j with:
I Gexj being adaptive
I Wexj being fixed and pre-trained (with the same framework) from isolated
bass and drum samples from the RWC music database.
The melody and to the remaining instruments are constrained to
Vj = Wexj Uex
j Gexj where:
I Uexj and Gex
j are adaptive.
I Wexj is a fixed dictionary representing the narrow band spectral patterns
modelling the harmonic pitch.
Considering:
I ERB spectrogram (providing higher low-frequency resolution than the
STFT desired for modelling drums and bass).
I With random initialization to all the adaptive parameters.
The melody is reconstructed by choosing the most energetic source of the 4
sources modelling other instruments.
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
BSS Evaluation (I)
I Emmanuel Vincent, Remi Gribonval, Cedric Fevotte. Performance
measurement in blind audio source separation. IEEE Transactions on
Audio, Speech and Language Processing, Institute of Electrical and
Electronics Engineers (IEEE), 2006.
I Objective measures accepted for the community.
Formalization of the problem:
starget(n) = starget(n) + einterf (n) + enoise(n) + eartif (n)
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
BSS Evaluation (II)
Formalization of the objective measures:
I Source to Distortion Ratio (SDR):
SDR = 10log10‖starget(n)‖2
‖einterf (n) + enoise(n) + eartif (n)‖2
I Source to Interference Ratio (SIR):
SIR = 10log10‖starget(n)‖2
‖einterf (n)‖2
I Sources to Noise Ratio (SNR):
SNR = 10log10‖starget(n) + einterf (n)‖2
‖enoise(n)‖2
I Sources to Artifacts Ratio (SAR):
SAR = 10log10‖starget(n) + einterf (n) + enoise(n)‖2
‖eartif (n)‖2
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Outline
Background
NMF
Cost function
PSA
FASST
Objectives and source model
Optimization problem: decomposition
Implementation
Evaluation
Try it!
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Download and use FASST
I http://bass-db.gforge.inria.fr/fasst/
I Download Version 1 (Matlab)
I Uncompress
I Run:
EXAMPLE prof rec sep drums bass melody FULL(’Shannon Hurley Sunrise prof prod mix.wav’, ’example data’,
’example data’)
I And... HAVE FUN! :)
Blind Source
Separation using
Non-negative Matrix
Factorization: FASST
Jordi Pons i Puig
Background
NMF
Cost function
PSA
FASST
Objectives and source
model
Optimization problem:
decomposition
Implementation
Evaluation
Try it!
Thanks
Thanks for your attention!