A Novel Approach to Blind Source Extraction Based on Skewness

Qingyan Shi1,2

, Renbiao Wu2, Shuyan Wang

2

1. Key Lab for Radar Signal Processing, Xidian University, Xi’an, Shanxi 710071, P.R.China; 2. Tianjin Key Lab for Advanced Signal Processing, Civil Aviation University of China,

Tianjin 300300, P.R.China Email: rbwu@cauc.edu.cn

Abstract

Blind signal extraction (BSE) is an efficient way to recover the source signals from the observed signals. In this paper, a new adaptive algorithm of blind signal extraction based on the skewness was introduced for the signals whose probability distribution is not symmetric. The algorithm cooperated with the deflation procedure realizes the extraction of source signals one bye one. Only third-order statistics are used in the novel algorithm, so it can reduce the computational burden effectively. Computer simulations confirm the validity and performance of the algorithm.

1. Introduction

Blind signal extraction (BSE) is to estimate one

source or a selected number of the sources (smaller

than the number of sources) with particular desired

properties or characteristics. It has received wide

attention in the fields such as wireless communications,

speech enhancement, remote sensing, biomedical

signal analysis and processing (EEG, MEG, ECG).

BSE and Blind source separation (BSS) are similar,

and their difference lies in using the BSE we can

extract the original source signals one by one, without

knowing the number of the source signals[1][2]. BSE

approach has the following advantages over BSS

method: 1) Signals can be extracted in a special order

according to the stochastic features of the source

signals. 2) Only interesting signals need be extracted. 3)

The learning algorithms for BSE are local and

biologically plausible, which are simpler than the

simultaneous BSS algorithms. 4) The method is very

flexible, and in each stage of extraction different

criterion and the corresponding algorithm to extraction

sources with special features can be adopted[2].

In the last ten years, blind signal extraction has

developed rapidly. Delfosse and Loubaton proposed

BSE in 1995[3], who solved the problem of extracting

source signal for the number of sources is unknown or

the number of sensors is greatly larger than the number

of sources. In succession, Cichlcki et al realized the

sequential BSE by making use of kurtosis[4]. In 2000

Cruces proposed a new blind signal extraction

algorithm which is able to recover simultaneously an

arbitrary number (which is less than the number of

sources) signals[5], additionally which is robust for

Gaussian noise. In the same year, Cichocki put

forward the correlated signals extraction based on

second-order statistics[6]. In 2002, Yuanqing Li and

Jun Wang gave a solvability analysis of BSE, also

presented a principle and cost function based on

fourth-order cumulants[7]. Wei Liu et al introduced a

new cost function based upon normalized prediction

error, and the source signal with the highest linear

predictability is extracted first [8].

For the signals with non-zero skewness, we

proposed a novel blind signal extraction method based

on the skewness. The skewness is coming from the

third-order cumulants, and the kurtosis is defined by

the fourth-order cumulants, so the new method can

reduce the computational burden effectively.

This paper is organized as follows. In section 2 we

specify the idea of BSE. After introducing the concept

of the skewness, we derived a new learning algorithm

in section 3. Section 4 presents the simulation results,

and at last section 5 draws the conclusions.

2. Model of blind signal extraction

Fig.1 shows the structure of blind signal

extraction[1]. The observed signals are the linear

instantaneous mixture of the source signals:

)()()( kkk nAsxT

m kxkxkxk )]()(),([)( 21xT

n kskskst )](),(),([)( 21

)(kA n

)(ks

)()(1 kk Qxx

(1)

where is a vector of the

observed signals, s is the

unknown source signals, n is a vector of additive

noise, is an unknown m mixing matrix. Every

observed signal is only the linear combination of the

.

To simplify the algorithm, the prewhitening can be

applied

(2)

ICSP2006 Proceedings

____________________________________

0-7803-9737-1/06/$20.00 ©2006 IEEE

mnRQ is whitening matrix which makes

IxxR }{ TX E after prewhitening the estimation

becomes easier.

Considering the Fig. 1, the first extraction unit can

be described by

)()()( 11 kkky T xw (3)

The unite successfully extracts a source signal, say

the th source signal, if , which satisfies

the relation , where denotes the th

column of a nonsingular diagonal matrix.

j *11 )( wwTjeAw *1 je j

nnBSE contains two sections[6]: the first is the

extraction, and the second is the deflation. The two

different types of processing units are connected with

each other alternately. According to the models and

criteria the BSE can be mainly classified into two

types: the first is the algorithm based on higher-order

statistics, and the second is the algorithm based on the

concept of linear predictability. The algorithm based

on the third-order statistics was mainly studied in this

paper.

Q 1w1w

( )ks ( )kx 1( )kx 2 ( )kx

1( )y k

A

Figure1. Implementation of extraction and deflation

principles

LAE: learning algorithm for extraction

LAD: learning algorithm for deflation

3. Blind signal extraction based on skewness

3.1. Criteria of signal extraction

Central Limit Theorem shows that under certain mild

conditions the distribution of a sum of independent

random variables tends towards a Gaussian

distribution. This means that a sum of several variables

typically has a distribution that is closer to Gaussian

than any of the original random variables[2]. Kurtosis

measures the flatness of the signals. A distribution

with negative kurtosis is called sub-Gaussian. A

distribution with positive kurtosis is referred to as

super-Gaussian, and a distribution with zero-kurtosis is

Gaussian. Normalized kurtosis was considered as one

of the simplest measures of non-Gaussianity of

extracted signal[2][4]. Therefore, to find vector that

maximizes the non-Gaussianity of the extracted signal

can separate an independent signal, the criterion was

formulated

1w

)(4

1)( 1411 ykJ w (4)

where is the normalized kurtosis, for zero-mean

signals which is defined by

4k

3}{

}{)(

2

1

2

4

114 yE

yEyk (5)

When the distribution of a signal is close to

Gaussian, it is close to symmetry, so we can utilize the

unsymmetry to measure the non-Gaussianity roughly.

Skewness is a measure of symmetry of the probability

distribution. For zero-mean signals the skewness can

be defined by

32

1

3

13

)( yE

yE (6)

When it equals zero, the probability distribution is

symmetrical. Negative values for the skewness

indicate that the data are skewed left, which means that

the left tail is long relative to the right tail. Similarly,

positive values for the skewness indicate that the data

are skewed right, and skewed right means that the right

tails is long relative to the left tail.

Similarly with the above criteria Eq. (4), we get a

cost function based on the skewness

31 )(wJ (7)

For the source signals with non-zero skewness, we can

find the vector that maximizes the unsymmetry of

the output .

1w

1y

3.2 Adaptive signal extraction algorithm based on skewness

(1) Signal extraction

Applying the standard gradient descent approach to

minimize the cost function (7), we have

)(

)(3

)(

2

1

3

11

2

12

1

3

1

2

12

3

1

2

1

31

1

11

1

yEyEyEyEyEyE

sign

Jdt

d

xx

www

(8)

where , at the same time 3 can be absorbed by

the learning rate . Applying the stochastic

approximation technique, we obtain an on-line

learning formula

01

1

1

2

12

5

1

3

1

2

1

2

131

1

)(

)( xw

yE

yyEyEysign

dtd

(9)

Exerting a simple approximation for the derivative

using the Euler approximation, we obtain the discrete-

time learning rule

)()]([)()()(

)1(

11311

1

kkyfsignkkk

xww

(10)

where the nonlinear function is defined as )]([ 1 kyf

)(

)(

)()()()]([ 1

2

5

2

3211 ky

km

kmkmkykyf (11)

The higher-order moments , and the skewness

can be calculated by the following formula

2m 3m

(12) )()1()1()( 1 kaykmakm ppp

and 3

2

33

))((

)()(

kmkm

k (13)

where .]1,0(aComparing with the algorithm based on kurtosis[4]

the new method omits the calculation of the forth-

order moments and reduces the computation efficiently.

(2) Deflation

After the signal ( ) had been

extracted, deflation technique is needed to remove the

previously extracted signal from the mixed signals.

This can be easily carried out by on-line learning

transformation

)()( ksky ij ],1[ ni

)21 ( ),(~)()(1 ,,jkykk jjjj wxx (14)

where Tmjjjj kxkxkxk )](),(),([)( ,12,11,11x

,)](),(),([)( ,2,1,

Tmjjjj kxkxkxkx

)()( kky jTjj xw .

jw~ can be estimated by the following cost function

m

p

pjjjj xJ1

,12

12

1)()~( xw (15)

From the above equation, we can see, the loss

function is energy. When the extracted signal has

been removed from the mixtures, the energy function

reaches minimum.

jy

Using the gradient descent to Eq. (15), we can

derive the learning rule

)()()(~)(~)1(~1 kkykkk jjjjj xww (16)

The two procedures can be continued by turns until

all the estimated source signals are recovered

successfully.

4. Simulation results

To demonstrate the validity and performance of the

proposed algorithm, the following computer

simulation was performed. In the simulations the

original signals were mixed by the

randomly chosen mixing matrix

T],,[ 321 sssS

77.065.046.0

79.04289.05028.0

68.07095.08318.0

A

From (a) to (b), Fig. 2 shows the original source

signals , the mixed signals T],,[ 321 sssST

321 ,, xxxX , and the extracted signals

T321 ,, yyyY . The algorithm (10) and (16) were

applied to extract source signals. The initial learning

rats for extraction and deflation are set to be 0.6 and

0.06. After two procedures of extraction and two

procedures of deflation, the sources signals can be

separated successfully. The coefficients of source

signals and the extracted signals are 0.9895, 0.9726,

0.9913 respectively. The comparison of the original

source signals and the extracted signals confirms the

validity of the algorithm.

5. Conclusions

A novel method of adaptive algorithm for blind

signal extraction is described for the source signals

whose probability distribution is not symmetric. The

new method based on third-order cumulants has less

computation and more stable than the algorithm using

the fourth-order cumulants. The validity and

performance of the algorithm are confirmed through

computer simulations.

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1s1

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1s2

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1s3

(a)

0 200 400 600 800 1000 1200 1400 1600 1800 2000-2

-1

0

1x1

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1x2

0 200 400 600 800 1000 1200 1400 1600 1800 2000-1

-0.5

0

0.5

1x3

(b)

0 200 400 600 800 1000 1200 1400 1600 1800 2000-50

0

50y1

0 200 400 600 800 1000 1200 1400 1600 1800 2000-10

-5

0

5

10y2

0 200 400 600 800 1000 1200 1400 1600 1800 2000-4

-2

0

2

4y3

(c)

Figure 2. Simulation results for the mixture of three source

signals

(a) the source signals

(b) the mixed signals

(c) the extracted signals

Acknowledgement

The research is supported by the National Natural

Science Foundation of China under grant 60325102.

References

[1] Simon Haykin. “Unsupervised Adaptive

Filtering Volume I: Blind Separation”, John Wiley & Sons Ltd, England, 2000.

[2] A. Cichocki, S. Amari. “Adaptive Blind Signal

and Image Processing”, John Wiley & Sons Ltd,

England, 2002.

[3] N.Delfosse, P.Loubaton. “Adaptive Blind

Separation of Independent Sources: a Deflation

Approach”, Signal Processing, 45, pp.59-83,

1995.

[4] A.Cichocki, R. Thawonmas and S. Amari.

“Sequential Blind Signal Extraction in Order

Specified by Stochastic Properties”, Electronics Letters, 33(1), pp.64-65, 1997.

[5] S.Cruces, A. Cichocki, and L. Castedo, “Blind

Source Extraction in Gaussian Noise”, in proceedings of the 2nd International Workshop on Independent Component Analysis and Blind Signal Separation(ICA’2000), Helsinki, Finland,

pp.63-68, June, 2000.

[6] A.Cichocki, T.Rutkowski, A. K. Barros and Oh

Sang-Hoon, “A Blind Extraction of Temporally

Correlated but Statistically Dependent Acoustic

Signals”, Proc. IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing, Sidney, Australia, pp.455-464, 2000.

[7] Yuanqing Li, Jun Wang. “Sequential Blind

Extraction of Instantaneously Mixed Sources”,

IEEE Transaction on signal processing, 50(5),

pp.997-1006, May, 2002.

[8] Wei Liu, Mandic Danilo P, Cichocki Andrzej,

“A Class of Novel Blind Source Extraction

Algorithms Based on a Linear Predictor”, IEEE International Symposium on Circuits and Systems, 4, pp.3599-3602, May, 2005.