Author's copy - produced at Lamar University · Author's copy - produced at Lamar University SPECTRAL ESTIMATION ... by using signal processing techniques. ... to have optimal resolution

Author's copy - produced at Lamar University

SPECTRAL ESTIMATION OF ELECTROENCEPHALOGRAM SIGNAL USING

ARMAX MODEL AND PARTICLE SWARM OPTIMIZATION

A Thesis

Presented to

The Faculty of College of Graduate Studies

Lamar University

In Partial Fulfillment

of the Requirements of the Degree

Master of Engineering Science

by

Bijaya Gautam

Date

November 2010


SPECTRAL ESTIMATION OF ELECTROENCEPHALOGRAM SIGNAL USING

ARMAX MODEL AND PARTICLE SWARM OPTIMIZATION

BIJAYA GAUTAM

ABSTRACT

The spectrum estimate of an electroencephalogram (EEG) signal is crucial to

various medical applications. Since the non parametric approach of spectrum estimate is

not reliable for short length data, the parametric approach of spectrum estimate is widely

suggested. In the presented work, a parametric ARMAX model was implemented to find

the spectrum estimate of the EEG signal. Performance of the ARMAX model (based on

the spectrum estimate of the EEG signal) was contrasted with parametric methods (e.g.,

AR model and ARMA) and non parametric method (e.g., periodogram). All the

programming and visualization were performed in a MATLAB environment.

Coefficients of ARMAX and ARMA model were estimated using the particle

swarm optimization (PSO) technique based on swarm intelligence. The PSO algorithm

adapted with the system identification technique and statistics yielded highly satisfactory

results in finding the coefficients of the ARMAX and ARMA models. AR model

parameters were found using the Modified covariance method.

The EEG spectral estimate obtained by the proposed method was able to depict

distinct spectral components not resolved by other methods, even compared to models of

higher orders. ARMAX models of the order three were generally found suitable for EEG


analysis, while the ARMA-based method generally required seventh order to produce

decent spectral estimates. AR-based analyzers were unable to produce spectral estimates

of sufficiently high resolution. The periodogram-based estimator, while showing decent

spectral resolution for high frequency content, failed to resolve low-frequency EEG

components.

Based on these observations, we recommend the ARMAX model for spectral

analysis of EEG.


ii

TABLE OF CONTENTS

Contents Page

List of Figure………………………………………………………………………………v

Chapter

1. Introduction………………………………………………………………………..1

1.1 Background ............................................................................................... 1

1.2 Research Motivation .................................................................................. 2

1.3 Objective ................................................................................................... 2

2. Methods in EEG Signal Analysis………………………………………................4

2.1 Spectral Analysis ....................................................................................... 4

2.1.1 Nonparametric methods ......................................................... 5

2.1.2 Parametric methods ................................................................ 6

3. ARMAX Model…………………………………………………………………..9

3.1 AR Model ................................................................................................ 10

3.2 ARX Model ............................................................................................. 11

3.3 ARMA Model .......................................................................................... 12

3.4 ARMAX .................................................................................................. 13

4. Parameters Estimation…………………………………………………………...15

4.1.1 Models.................................................................................. 15

4.1.2 Estimation ............................................................................ 16

4.1.3 Algorithm ............................................................................. 16


iii

4.1.4 Validation ............................................................................. 16

4.1.5 Model Fit .............................................................................. 17

5. Particle Swarm Optimization…………………………………………….............19

5.1 Introduction of Particle Swarm Optimization ......................................... 19

5.2 PSO Algorithm Flow Diagram ................................................................ 22

6. Experiments and Results…………………………………………………………24

6.1 EEG Signal .............................................................................................. 24

6.2 White Noise ............................................................................................. 25

6.3 Exogenous Input ...................................................................................... 25

6.4 PSO Based ARMAX Parameter Estimation Process .............................. 27

6.4.1 PSO initialization ................................................................. 28

6.4.2 Loss Function ....................................................................... 30

6.4.3 Global and Self best position ............................................... 32

6.4.4 Optimization ........................................................................ 34

7. Spectral Analysis………………………………………………………………...38

7.1 White Noise ............................................................................................. 38

7.2 Exogenous Input ...................................................................................... 39

7.3 Frequency Response of Filters A, B and C ............................................. 41

7.4 ARMAX Model Output ........................................................................... 43

7.5 Performance Comparison for different model orders .............................. 45

7.6 Comparison with an ARMA model ......................................................... 46

7.7 Comparison with an AR model and with the Periodogram method. ....... 49


iv

8. Conclusion……………………………………………………………………….54

8.1 Conclusion ............................................................................................... 54

8.2 Future Work ............................................................................................ 54

9. Reference………………………………………………………………………...56

10. Appendix...……………………………………………………………….………63

10.1 Modified Covariance Method ........................................................ 63

10.2 Model validation data……………………………………………..65


v

LIST OF FIGURES

Figure Number Figure Name Page

Figure 3-1 General Linear Model ................................................................................. 9

Figure 3-2 AR Model .................................................................................................. 10

Figure 3-3 ARX Model ............................................................................................... 11

Figure 3-4 ARMA Model ........................................................................................... 12

Figure 3-5 ARMAX model ......................................................................................... 13

Figure 5-1 PSO Algorithm Flow Diagram (Schutte 2005) ........................................ 22

Figure 6-1 EEG Signal ............................................................................................... 24

Figure 6-2 White Noise realization ............................................................................ 25

Figure 6-3 Exogenous Signal ..................................................................................... 26

Figure 6-4 ARMAX Parameters as Swarm of PSO ................................................... 29

Figure 6-5 AFPE of Each Swarm .............................................................................. 31

Figure 6-6 Global FPE at Each Iteration.................................................................... 33

Figure 6-7 Model Validation ..................................................................................... 35

Figure 6-8 MSE During Model Validation ................................................................ 36

Figure 7-1 Noise Spectral Estimate ........................................................................... 39

Figure 7-2 PSD Estimate of Exogenous Input ........................................................... 40

Figure 7-3 Frequency Response of filters C and B. ................................................... 42

Figure 7-4 Frequency Response of filter A................................................................ 43

Figure 7-5 Spectral Estimate of EEG......................................................................... 44

Figure 7-6 EEG spectral estimates for different model orders .................................. 45


vi

Figure 7-7 EEG spectral estimates by ARMA and ARMAX methods .................... 47

Figure 7-8 Different Order ARMA Model. .............................................................. 48

Figure 7-9 PSD Comparison for Order 3 .................................................................. 50




Bijaya 1

CHAPTER 1

Introduction

1.1 Background

The word Electroencephalography (EEG) was derived from the Greek words

“electro” (related to electron or electrical), ‘enkephalos’ (marrow in the head)

(thefreedictionary 2010), and ‘graphy’ (“writing” or a ‘field of study) (graphy-Wikipedia

2010). Therefore, the literal translation of EEG would be the writing, or study of the

electrical signals in the brain. Electroencephalograph would record the electrical activity

taken from the human scalp (using sensors of a special type) over, usually, a period of

time (Paul 2006). The sensors would be placed in multiple locations of the subject’s

scalp. Recordings of the electrical signals are simultaneously performed for all channels.

From the signal processing viewpoint, EEG is a spatial and non-stationary time-series

process (EEG- Wiki 2010). An analysis of the EEG signals is one of the key areas of

biomedical data processing due to the information contained in these signals. EEG could

be extremely beneficial for studying the conditions and status of the human brain (BSA-

EE Herald 2010).

Similar to other naturally-generated signals, EEG also contains information that

could be extracted by using signal processing techniques. Various types of such

techniques have been developed to analyze EEG signals. An accurate analysis could

provide valuable clinical, psychological, and physical information in reference to the

brain. In particular, EEG waveforms would disclose information about certain changes


Bijaya 2

due to drugs, emotions, thoughts, and diseases. Measurements of these changes available

in EEG waveforms in real-time may be used to determine the status and conditions of the

subject (Niederhauser, et al. 2003).

1.2 Research Motivation

In the study of EEG, the usual task is to estimate the spectrum of the signal. Short

Time Fourier Transform Method (Spectrogram Method), based on Fourier

transformation, is a traditional approach to this study, but, “it has a serious drawback,

which is the implicit assumption of stationary within each segment and unsatisfactory

time/frequency resolution” (Maghsoudi and White 1993).

Parametric spectral analysis methods based on autoregressive (AR) and auto

regressive moving average (ARMA) models have been suggested as a better approach for

the spectral estimates of the EEG signal (Tseng, Chong and Kuo 1995). This research

will focus on applying the "auto regressive moving average with the exogenous input"

(ARMAX) method to estimate the spectrum of the EEG. There has not been much

scholarly work done with ARMAX in the spectral estimation of EEG, and ARMAX may

allow for a more accurate and flexible representation of the EEG time series.

1.3 Objective

The basic objective of this research was to test the suitability of an ARMAX

model for a spectral analyss of the EEG signal. In order to perform this analysis,

parameters of the ARMAX model needed to be estimated. For this purpose we propose


Bijaya 3

employing an innovative approach, the Particle Swarm Optimization (PSO) technique.

The PSO algorithm was implemented and its performance was evaluated using

MATLAB.


Bijaya 4

CHAPTER 2

Methods in EEG Signal Analysis

2.1 Spectral Analysis

The time series signal could also be represented in the frequency domain

(Estimation theory 2010). Power-frequency distribution of the signal sometimes becomes

more important than its time domain representation. Spectral estimation would portray

the distribution of the power contained in a signal over the frequency within a finite set of

data (Murugesan and Sukanesh 2009). Knowledge of the power spectral density (PSD) in

the signal is useful in various situations e.g. detection of a signal masked in wideband

noise (S D-Wiki 2010).

The Wiener–Khinchin theorem defines the concept of a spectrum for a wide sense

stationary (WSS) process. The theorem states, “The PSD S(f) of a wide-sense stationary

random process x(τ) is the Fourier transform of its autocorrelation function R(τ)”, as

shown in Equation 2-2 (S D-Wiki 2010).

��τ� = � x�t + τ�x�t�dt =�� x�t�x�t − τ�dt�� Equation 2-1

Thus,

�� = � �� = Ϝ{��} Equation 2-2

If R(τ) is known for all τ, evaluating the power spectrum would be

straightforward. The two basic problems for spectral estimation of the EEG data are:


Bijaya 5

a. The amount of data is always limited.

b. Data is frequently corrupted by noise or contaminated with an interfering

signal (Hayes 1996).

The two most abundant methods of spectrum estimation are as follows.

(Bingham, Godfrey and Tukey 1967).

a) Nonparametric methods relied on the direct use of the available data.

b) Parametric methods relied on the model assumed for the signal generation.

Therefore, the choice between parametric methods and nonparametric methods to

obtain a PSD estimate is a choice between straightforward (less accurate) nonparametric

estimators and a computationally complex (yet more accurate) parametric PSD estimator.

2.1.1 Nonparametric methods

Nonparametric methods would use signals to determine the power spectrum

density. If x[n] is a finite-length signal, the simplest method of a PSD estimate is the

periodogram P(ejw

) (Schuster 1898) that is based on Fourier transform as shown in

Equation 2-3.

�� = �� ∗�� = �� | #��$|� Equation 2-3

Here, X(e^jw) is DTFT of x[n] and N denotes the total number of samples. Welch

modified the now commonly used periodogram in 1967. His modification is known as

Welch's method (Welch 1967).


Bijaya 6

Akin has used the periodogram method of spectral estimation for clinical

applications (Akin and Kiymik 2000). Bullock and his colleagues used an additive

periodogram to determine the stochastic nature of EEG signal and to monitor the

temporal fine structure of the EEG data (Bullock, Enright and Chong 1998). Sign

periodogram was successfully used to identify seizure precursors from the depth of the

EEG (Niederhauser, et al. 2003). Recently, Qihou Zhou and colleagues reported a time-

frequency analysis method by using an adaptive length periodogram technique for EEG

data processing. This technique was reported to have optimal resolution in both the

frequency and time domain. (Qihou, Matthew and Jade 2008). The nonparametric

methods - like periodogram, Bartlett, Welch, and Blackman Tukey’s - are generally

limited by the length of data. A few other general limitations known about nonparametric

methods are (Marple 1989):

a. Correlation is assumed to be zero beyond N. N is thethe number of available

data samples.

b. If two frequencies are separated by ∆f, then we need N ≥ �∆' data samples to

resolve them.

c. Resolution limit imposed by N also causes bias.

2.1.2 Parametric methods

In the Parametric methods, the frequency domain output of a model (usually

driven by white noise) is viewed as the power spectral density of the signal. Some

examples would be the Yule-Walker autoregressive (AR) structure and the Burg method


Bijaya 7

(Bingham, Godfrey and Tukey 1967). Parametric methods would produce better

resolution than nonparametric methods when applied to a short length signal (Brown and

Hwang 1983 and Tseng, Chong and Kuo 1995).

Subspace methods, also known as high-resolution methods of spectral estimation, would

use an Eigen-analysis or Eigen-decomposition of the correlation matrix (Spectral

Analysis S E M 2010). Multiple signal classification (MUSIC) method and the

eigenvector (EV) method are examples of subspace algorithms. Subspace techniques are

found useful for the line spectra and can be used when sinusoids are corrupted by noise

with low signal-to-noise ratios (Stoica and Eriksson 1995).With an accuracy of 95.6%,

Melancholia disease has been diagnosed by Enping using the Eigen vector method to

EEG signals (En'ping, Sheng and Shini 2009).

Reports are suggesting to use the autoregressive moving average (ARMA) models

for the EEG analysis (Maghsoudi and White 1993). In the method proposed by

Maghsoudi, coefficients of the ARMA model are identified and used to depict the

waveform. Extended Least Squares and Recursive Extended Least Squares methods were

used to estimate the parameters of the first order ARMA model. The identified model

was then used to describe the time and frequency-domain properties of the EEG

waveforms. Parametric methods suffer from the rise of computation time. However, the

benefit of the ARMA method (over the nonparametric estimation) is its ability to track

the time-varying process (Maghsoudi and White 1993).

The evaluation of the performance of parametric methods for spectral estimation

of EEG was conducted by Tseng (Tseng, Chong and Kuo 1995). The Akaike information

criterion (AIC) (Akaike 1974) was used for determining the orders of AR and ARMA


Bijaya 8

models. The tests had suggested that the AR model would require a higher model order

(8.67 on average) than the ARMA model order (6.17 on average). It was also found that

about 96% of the total EEG segments each 1.024 seconds long were efficiently

represented by the AR model, and only about 78% could be represented by the ARMA

model.


Bijaya 9

CHAPTER 3

ARMAX Model

The system shown in Figure 3-1 is called a general linear model and could be

described using the Equation 3-1 (Candy 2006).

Figure 3-1 General Linear Model

)�*�+�,� = ��,�-�*� + .�,�/�* − 0� Equation 3-1

When A(q), B(q) and C(q) are the polynomials of corresponding filters’ coefficients.

k is delay,

e(n) indicates white noise and u(n) represents the deterministic signal.

��,�-�*� resembles the stochastic part of the system and

.�,�/�* − 0� resembles the deterministic part of the system.

y(n) indicates the output of the system.


Bijaya 10

General linear model structure is shown in Figure 3-1. A simple change in

parameters could yield a completely new model. Interestingly, a nonlinear optimization

method that requires serious calculation without any global convergence could be used to

estimate the parameter of the general linear model (Instruments 2010).

By setting the filter polynomials A (q), B (q) or C (q) in Equation 3-1, equal to

one or zero, we could derive somewhat simpler models such as the ARX, AR and ARMA

structures. Selection of the appropriate model would depend entirely on the

characteristics of the problem since these models having different characteristics and

applications.

3.1 AR Model

A simple AR model is shown in figure 3-2 and is defined by Equation 3-2. AR

model uses only A(q) filter. e(n) and y(n) indicates white noise input and system output

respectively.

Figure 3-2 AR Model

)�*�+�,� = ��,� Equation 3-2


Bijaya 11

The AR model is used when current output is dependent only on the previous

outputs. There are different types of parameter estimation techniques for the AR model.

One of the techniques is the modified covariance method. This method is described in

appendix I (Hayes 1996).

3.2 ARX Model

Addition of exogenous signal .�,� as an input to a simple AR model give rise to

ARX model as shown in Figure 3-3 and described by equation 3-3. ARX model uses

A(q) filter and B(q) filter with k delay samples.

Figure 3-3 ARX Model

)�*�+�,� = ��,� + .�,�/�* − 0� Equation 3-3

This model could give the best result among the polynomial estimations as the

model is obtained as a solution of a linear equation in analytic form. “The solution is

unique and always satisfies the global minimum loss function.” (Instruments 2010).


Bijaya 12

3.3 ARMA Model

Combining Autoregressive (AR) and moving average (MA) processes, a highly

flexible class of uni-variate processes called the ARMA process was introduced by Box

and Jenkins (Jenkins and Box 1970). A block diagram of an ARMA model is shown in

Figure 3-4.

.

Figure 3-4 ARMA Model

)�*�+�,� = ��,�-�*� Equation 3-4

ARMA model incorporates A(q) filter (Autoregressive component) and C(q) filter

(Moving Average component). The ARMA model does not have exogenous input and is

a special case of ARMAX models.


Bijaya 13

3.4 ARMAX

The autoregressive moving average model with the exogenous inputs (ARMAX

model) incorporates the disturbance dynamics known as exogenous input. This model is

widely used in the presence of dominating disturbances that appear early in the process,

such as the input. The ARMAX model could be significantly more efficient at

management of disturbance modeling than the ARX model (Instruments 2010).

Figure 3-5 ARMAX model

The diagram in Error! Reference source not found.5 can be represented by

quation 3-5 (System Identification Toolbox 2010).

+�,� + 1�+�, − 1�+. . . +145+�, − ,5� = 6�.#, − ,7� + ⋯ + 645.�, − ,7 −,9 + 1� + :��, − 1� + ⋯ + :4;��, − ,;� + ��,�� Equation 3-5

Where, y(n) Output at sample n.

na Number of poles of filter A.

nb Number of zeroes of filter B plus 1.

nc Number of poles of filter C.

nk is the dead time in the system. For discrete systems with no dead time nk =1.


Bijaya 14

u (n) is exogeneous input.

e(n) is whit noise.

The parameters na, nb, and nc are the orders of the ARMAX model; therefore,

filter polynomials A, B and C can be written as follows:

)�*� = 1 + 1�*�� + ⋯ + 14<*�4<

/�*� = 6� + 6�*�� + ⋯ + 64=*�4=>�

-�*� = 1 + :�*�� + ⋯ + :4?*�4? Equation 3-6


Bijaya 15

CHAPTER 4

Parameters Estimation

Depending upon repeatability and extraneous variations, there are two types of

signals; the deterministic signal and random signal. Deterministic signals are repeatable

and do not vary extraneously. Random signals are not repeatable and vary extraneously

(Candy 2006). The real world’s experimental input-output data were used to estimate

parameters of mathematical models. The process of estimating parameters of the model

with the real world’s experimental input-output data is based on the statistical theory

(Ljung 2010). The theory is centered on the following concepts. (Ljung 2010)

4.1.1 Models

The model represents a relation between system input and the output (Candy

2006). Mathematical expression or a graph or table may be used to define this

relationship. We denote the model by ‘m’ (Ljung 2010).

Depending upon the nature of the problem, the model could be deterministic or

probabilistic. Usually, models contain information such as the process of signal

generation, characteristics of noise, and the deterministic and probabilistic structure, etc.

(Candy 2006).


Bijaya 16

4.1.2 Estimation

Information about the object to be modeled, the nature of the problem and the

observed data can be used to select an appropriate model. This process of selection is

called ‘model estimation’. Estimation data, usually known as training data obtained from

experiments, are used in model selection. We denote the estimation of data by zen (Ljung

2010). Where, ‘n’ is the size of the data.

4.1.3 Algorithm

Various factors must be considered while determining a correct estimation

algorithm e.g computation time, error, data length etc. According to Candy, “a

preconceived knowledge of the similar problem structure of the estimator greatly

influences the resulting algorithm” (Candy 2006). Performance of the algorithm can be

assessed by evaluating the estimation error. To yield a good estimation algorithm, the

estimation error must be minimized (Ljung 2010).

4.1.4 Validation

Excluding the evaluation set of data, a model should be useful to the other set of

data that was never used for training purposes. Data sets used for this purpose are called

validation data. We denote validation data by zCD (Ljung 2010).


Bijaya 17

4.1.5 Model Fit

The model fit is a scalar value representing how well a model is able to fit the

data set Z. We denote the model fit by F(m,Z) (Ljung 2010).

With a model parameters optimization algorithm, it is not difficult to find a model

that would represent the estimation data set. The actual performance of a model is given

by the model fit value with the validation data E�F, HI4�. The average fit of the validation

data is known as ‘expected fit’ E�F, HI4� ,and is generally smaller than the fit of the

estimation data known as ‘actual fit’ E�F, HJ4� (Ljung 2010).

The data set HJ4 is represented by a model FK , estimated from an estimation data

set HJ4 ,in a model set M using model estimation techniques as described above. The

model fit of FK with the estimation data HJ4 is given by E�FK, HJ4� and model fit of FK to

validation data HI4 and given by E�FK, HI4�.

E�FK, HI4� = E�FK, HJ4� + ��-�L�, M� Equation 4-1

The mean square error between the expected and actual fit is known as the fitness

of the model (Ljung 2010). C is the complexity of the model that would decrease with the

increase of N (Ljung 1999). Since FK is a random variable of the set M, Equation 4-1 is

probabilistic. The expectation of this function Nℱ PPP�FK, HI4� is given by various forms as

shown in Equation 4-2.


Bijaya 18

Nℱ PPP�FK, HI4� ≈ �>RS��RS ℱ�FK, HJ4� ……..(a)

≈ 1 + �T� ℱ�FK, HJ4� ……..(b)

≈ ��RS�U ℱ�FK, HJ4� ..……(c) Equation 4-2

Equation 4-2(a) is known as Akaike’s Final Prediction Error (FPE); this function

is used to determine the model order and acts as a criterion function. Equation 4-2(b) is

the Akaike’s Information Criterion (AIC) (Akaike 1974), and Equation 4-2 (c) is the

generalized cross-validation (GCV) criterion, (Wahba and Craven 1979). ‘d’ is the

complexity measure of the model set.


Bijaya 19

CHAPTER 5

Particle Swarm Optimization

5.1 Introduction of Particle Swarm Optimization

The Particle Swarm Optimization is one of the stochastic optimization techniques.

PSO is based on the social and personal behavior of a swarm. It was first introduced by

Eberhart and Kennedy in 1995 (Eberhart and Kennedy 1995). The researchers described

how PSO can be applied to a nonlinear optimization problem through the simulation of a

social system characterized by swarms (e.g. bees). PSO would continuously search for a

new position for each swarm based on their social best position and self best position. Shi

and Eberhart introduced a third factor, inertia weight, responsible for keeping a swarm in

its original direction (Eberhart and Shi 1998). In the same year, Angeline developed a

“hybrid PSO with the addition of a standard selection process from evolutionary

computations in order to produce a high-quality solution within a short computation

time” (Angeline 1998). PSO is being used for various applications around the world such

as solving a computationally complex -“travelling salesman problem” (Wang, et al.

2003).

PSO and evolutionary computation techniques, like a Genetic Algorithm, are

especially similar. Both algorithms would work by searching solutions enhanced with

updating generations. However, there is nothing to the equivalent of ‘evolution operators’

in PSO, which is common in Genetic Algorithm. Most importantly, PSO does not have a

global gradient (Jenigiri 1996). Research on PSO is evolving around the world and

different hybrid modules of PSO are also being proposed. Jiao Wei and colleagues have


Bijaya 20

successfully incorporated a mutation in PSO (Jiao, Liu and Xian 2008). Gudsiz has

compared PSO with Artificial Neural Network (ANN) and efficiently cited ANN (Gudsiz

and Venayagamoorthy 2003). PSO is also being used in system identification problems.

Huang and colleagues have demonstrated that PSO can be used for identifying an

ARMAX Model for Short- term Load Forecasting (Huang, Huang and Wang 2005).

The Basic PSO algorithm is described by Equation 5-1 and Equation 5-2

(Eberhart and Shi 1998). The position of an individual particle is updated using Equation

5-2 with the velocity calculated in Equation 5-1.

V7>�W = XV7W + Y� .� × [�6�\�7W − �7W ] + Y�.� × [^6�\�W − �7W ] Equation 5-1

�7>�W = �7W + V7>�W Equation 5-2

Where, �7W - Position of the ith

particle during kth

iteration.

V7W - Velocity of the ith

particle during kth

iteration.

�6�\�7W - Individual best fitness achieved by a ith

particle till kth

iteration.

^6�\�W- Global best fitness of ith

particle till kth

iteration.

X - inertial coefficient.

Y�and Y�- Cognitive and social parameters.

.�and .� - Random numbers between 0 and 1.

Equation 5-1 is the summation of three different velocity update parameters. Each

term has its own important role in optimizing the PSO algorithm.

a) Inertial component: The first term XV7W is known as an inertia component

that is responsible for preserving the original direction of particle. (Blodin


Bijaya 21

2009). Generally, the inertial coefficient w may be chosen between 0.8 and 1.2

(Eberhart, Kennedy and Shi 2001).

b) Cognitive component: The second term Y� .�[�6�\�7W − �7W ], is called the

cognitive component. Blodin defines this component as “the memory of the

particle, forcing it to move toward the search space that has achieved the best

individual fitness (pbesti) ever” (Blodin 2009). The cognitive coefficient Y� is

chosen close to 2. This coefficient is responsible for the step size, measuring

what a particle would take to reach its ‘individual best solution’ (Eberhart,

Kennedy and Shi 2001).

c) Social component: The third term Y�.�[^6�\�W − �7W ] is known as the social

component. This component is also known as the memory of the particle. This

term would force the particle to move toward the search space that has

achieved the best global fitness (gbesti) (Blodin 2009). The social coefficient

Y� is also chosen close to 2. Unlike the cognitive component, the social

component would drive a particle toward the best global candidate solution

with a step size of Y� (Eberhart, Kennedy and Shi 2001).

.� and .� in the cognitive and social component are random values responsible

for stochastic behavior of the velocity update. Presence of these stochastic parameters

causes each particle to move in a semi-random manner toward the direction of the pbest

of the particle and gbest of the swarm (Blodin 2009).


Bijaya 22

5.2 PSO Algorithm Flow Diagram

The PSO algorithm flow diagram is shown in Figure 5-1 (Kennedy and Eberhart

1997) and (Angeline 1998).

Figure 5-1 PSO Algorithm Flow Diagram (Schutte 2005)


Bijaya 23

The basic process is outlined as followed:

1. Initialization

a. Specify number of iteration (Kmax), Y�, Y� and w.

b. Randomly initialize all particles’ positions Ri .

c. Randomly initialize all particle’s velocity vi such that v

i < v

max.

d. Set k=1

2. Optimization

a. Evaluate loss function value àb using design space coordinates cab .

b. If àb < èfghb then iefghab = cab .

c. If àb < èfghj then kefghab = cab

d. Update all the particle velocities vi.

e. Update all the particle position Ri.

f. Check stopping condition

a. Yes: display result and quit.

b. No: increase i.

g. Check condition if i is less than total number of particle

a. Yes: Set i=1 and increase k then go to 2(a).

b. No: Go to 2(a)


Bijaya 24

CHAPTER 6

Experiments and Results

6.1 EEG Signal

The EEG data used in this research was obtained from Henri Begleiter from the

Neurodynamics Laboratory at the State University of New York Health Center in

Brooklyn. “These data were recorded to examine EEG correlates of genetic

predisposition to alcoholism. It contains measurements from 64 electrodes placed on

subject's scalps which were sampled at 256 Hz (3.9-msec epoch)” (Begleiter 1999). For

our experiment, we used 30 samples i.e. 0.1172 Sec length of signal from randomly

selected data set. An example of the experimental data is shown in Figure 6-1.

Figure 6-1EEG Signal


Bijaya 25

6.2 White Noise

The discrete-time random process represented by a random vector w is a white

noise vector with the mean vector l� and autocorrelation function ��that would satisfy

the Equation 6-1 and Equation 6-2 (white noise 2010 ).

l� = N{X} = 0 Equation 6-1

�� = N{XXn} = o�p Equation 6-2

Where, I is identity matrix. Figure 6-2 shows the white noise used for the

experiment.

Figure 6-2 White Noise realization

6.3 Exogenous Input

Unlike ARMA, an ARMAX model requires an exogenous input. This input

should be a deterministic signal. In this study, the exogenous input .�� was formed by


Bijaya 26

the summation of three sinusoids of f1, f2 and f3 as represented by Equation 6-3 and

shown Figure 6-3. While the EEG signal has been sampled with sampling frequency fs =

256 Hz i.e. sampling time Ts= 0.0039 Sec, the exogenous signal was generated for 30

epochs (0.1172 Seconds). In this research, the sum of pure sinusoidal waves of the

frequencies 5 Hz, 10 Hz and 18 Hz is taken as an exogenous Input.

.�� = sin�2t�� + sin�2t�� + sin�2t�� Equation 6-3

Figure 6-3 Exogenous Signal


Bijaya 27

6.4 PSO Based ARMAX Parameter Estimation Process

First, we needed to verify that our algorithm would work properly. For this

purpose, the ARMAX model with the following arbitrarily selected parameters was

created of the order of 3, 5 and 7.

A= [1 -0.1400 -0.4244 0.3963 0.3921 0.3764 0.0796 0.0321]

/ = [-0.2347 0.1348 -0.2084 -0.0903 0.3789 -0.0220 -0.1044 0.0021]

- = [1 0.0137 0.1123 0.1459 -0.0266 -0.1099 -0.1540 0.0419] Equation 6-4

Equation 6-4 represents the ARMAX model from order 7. Models of Orders 5 and

3 may be obtained by excluding appropriate higher order coefficients from the same

filters A, B and C. The ARMAX model of a particular order was simulated with

exogenous input u(n) and white noise e(n). The output of the model y(n) was recorded

along with u(n) in a data set Ze, as shown in Equation 6-5. So obtained output of the

model y(n) was used as the ‘target data’ to train an ARMAX model.

Ze ={ [+�(n) u1(n)], [+�(n) u2(n)], [+u(n) u3(n)]…..[ +W (n),ui(n)]} Equation 6-5

One hundred of data sets were generated so that each set contained a pair of

exogenous input u(n) and training data y(n). With these training data sets and noise e(n),

ARMAX model parameters were estimated using the PSO technique. The objective of

finding the ARMAX model parameters is to validate the PSO algorithm being used.


Bijaya 28

Parameters estimated using PSO are referred to as )K , /P and -K . The estimation process

was performed for the orders 3, 5, and 7.

A new ARMAX model was created with [)v /P -K ] and simulated with exogenous

input and white noise. The simulated output of the ARMAX model with parameters [)v /P

-K ] was compared with the output of the ARMAX model with parameters [A B C]. If

both of the outputs were similar, we concluded that the PSO technique could produce the

ARMAX parameters imitating a nonlinear system (like an EEG sequence). Once the PSO

technique was validated, we used real EEG signals to estimate the ARMAX model

parameters. Analyzing the model enables us to produce a spectrum of that model that can

be used as a spectral estimate of a real EEG signal.

The PSO technique as described in chapter 5 was used to estimate the ARMAX

model parameters. The following estimation steps were followed with each data set.

6.4.1 PSO initialization

The initial trial parameter vectors Ri were randomly generated, where Ri =

uniform(a,b)d, i=1,2,……k, d = n+m+r are uniformly distributed in [a,b] in the d-

dimensional space. Here, n,m and r were the orders of filters A, B and C respectively.

For example, if all filters’ orders n, m and r equal to three, then d=3+3+3=9. The

dimension of random matrix would be a random matrix of 9 X Ri. Each element of Ri is

uniform distribution between integers ‘a’ and ‘b’. In this matrix, each Ri column is

potential ARMAX parameters. With these possible parameters, an ARMAX model is

simulated with white noise (as discussed in chapter 6-2) and with an exogenous input (as

discussed in chapter 6-3).


Bijaya 29

Figure 6-4 ARMAX Parameters as Swarm of PSO

The third order ARMAX model represented by Equation 3-1 can also be

described as:

+�,� = 1 − 1�+�, − 1�−1�+�, − 2�−1u+�, − 3� + 6�.�, − 1� + 6�.�, − 2�+ 6u.�, − 3� + :��, − 1� + :��, − 1� + :��, − 1� + ��,�

Equation 6-6

In this step, a number of tentative ARMAX models Ri with the same model

structure of n, m and r but with different combinations of parameter vectors were


Bijaya 30

established. In the initialization stage ‘i’ the number of the ARMAX model was created

as shown in Error! Reference source not found.. Each Ri is called a ‘current position’

f swarm according to PSO terminology.

6.4.2 Loss Function

‘ith

’ model output was obtained by simulating each ARMAX model represented

by Ri with exogenous input ui . New data sets Hx i containing ui and +yi were created as

specified by Equation 6-7.

Hx i=( [+y1 (n) u1(n)], [+y2 (n) u2(n)], [+y3 (n) u2(n)]…..[ +yi (n),ui(n)] Equation 6-7

Here, +y and u are the output and the input of ARMAX model Ri.

The difference between the output +z generated with the training data and a real

output y is known as a residual or prediction error (Equation 6-8). A model producing

small prediction error is considered a good model (Rahiman, Taib and Salleh 2007).

ε�n� = y�n� − yy�n� Equation 6-8

y(n) is the target data and yy�n� is the predicted output of the ARMAX model Ri. The loss

function of ith

model Ri is calculated using a prediction error }�,� and a transpose of }�,�

(Ljung 2001) as follows:


Bijaya 31

~�� = det [�4 �}�,� × }�,��] Equation 6-9

The Akaike Final Prediction Error (AFPE) is obtained using Equation 6-10.

)E�N = ~��, �� × �> ��R�� Equation 6-10

The loss function shows the accuracy of the fit for the model. The AFPE of each

Ri is the accuracy for the fit termed ‘Swarm fitness’ of the swarm Ri. .Among all the

Swarm fitness of Ri, at Kth

iteration, the swarm fitness that is the absolute minimum (i.e.

nearest to zero) is called the ‘global best swarm fitness’ (Gbestk). Figure 6-5 illustrates

fitness (AFPE) of each Ri.

Figure 6-5 AFPE of Each Swarm


Bijaya 32

As shown in Figure 6-5, for each Ri an AFPE is evaluated. The minimum AFPE is

chosen as the global best fitness ‘^6�\�’ at that step. Analysis indicates that the AFPE is

at its minimum at the 5th

position of swarm for the case shown in Figure 6-5. Therefore,

the 5th

position swarm (R5) has the minimum AFPE among all Ri. If Ri keeps updating

with iterations, Gbest changes accordingly.

6.4.3 Global and Self best position

Each particle Ri remembers its own loss function and designates the minimum

function. This is called the self best position �6�\�W. The particle with the best loss

function among all Ri is referred to as the global best position ^6�\�. In the first iteration,

each particle’s Ri is set directly to �6�\�Wand the particle with the best FPE value among

�6�\� is set to ^6�\�. An example of how �6�\�W and ^6�\� change in each iteration has

been shown in Table 1.

Table 1. An Example of global and local fitness

Gbest

�� /�6�\��

�� /�6�\��

��u�� /�6�\�u

………..

��W�� /�6�\�W

Iteration

(Step k)

0.2

0.256 -0.421 0.20

………..

0.436

1 0.256 -0.421 0.20 0.436

0.17

0.17 -0.514 0.21

………..

0.201

2 0.17 -0.421 0.20 0.201

0.073

0.104 -0.073 0.243

……

0.154

3 0.104 -0.073 0.20 0.154


Bijaya 33

The first iteration �6�\�W of each Ri is set directly equal to the AFPE of Ri. R3 is

the minimum AFPE at the first step, so gbest is set to R3AFPE. During the second iteration,

a new gbest is obtained, but pbest is not changed for all the swarms. R2AFPE and R3AFPE

are higher than the first iteration, so pbest of these two swarms are not changed during

the second iteration. During the third iteration, pbest of all the swarms are changed except

from R3. Similarly, pbest and gbest are updated until the predefined number of iterations

is completed. Figure 6-6 illustrates the change in gbest during iteration.

Figure 6-6 Global FPE at Each Iteration


Bijaya 34

Figure 6-6 shows that at each step of the swarm (at each iteration), ^6�\� is

decreasing. This indicates that the difference between the target signal and the model

output are decreasing at each step, and all particles are converging toward the solution in

search space.

6.4.4 Optimization

The optimization of the PSO algorithm is based on the velocity and the position

update step. Velocity would refer to the difference between the swarm positions at kth

iteration and at (k+1)th

iteration. The Velocity of the Swarm is calculated according to

the global and local best positions using Equation 5-1.

When the new velocity and the previous position are known, an offspring

vector is calculated as the sum of the velocity and the previous position as shown in

Equation 5-2. Offspring vector is the new position of the swarm.

The new position is set as the current position of a swarm at kth

iteration. Now, a

new set of Ri is ready to simulate the ARMAX model. Steps 6.4.2 to 6.4.4 are repeated

until the number of iteration assigned is completed. Because of this, Ri has the best

solution, and Gbest in the population has been obtained. Vectors of Ribest containing

coefficients [)v /P -v ] are regarded as a solution, or as an optimized model. The optimized

model should more precisely pass diagnostic checks. Otherwise, the model identification

process described above should be repeated until an appropriate model is obtained.

The best solution obtained using PSO algorithm has to pass the validation test. To

validate the model, the validation data V(n) is used. This data should never be used to

train the model. The model is simulated, and this time the output of model is compared


Bijaya 35

with V(n) instead of target data y(n). The Mean Square Error (MSE) between validation

data and the model output is obtained by using Equation 6-11.

MSE�,� = �� ,� − +y�,�� Equation 6-11

When V(n) is the validation data, yy�n� is the predicted output of the ARMAX

model with [)v. /P -v ], and N is the length of data.

Figure 6-7 demonstrates the validation data from the initial coefficients [A B C]

and simulated output of the PSO based coefficients [)v. /P -v ].

Figure 6-7 Model Validation


Bijaya 36

It can be observed that the model output follows the validation data very closely.

This observation indicates that the ARMAX model created with [)v. /P -v ] has succeeded

to depict the target signal.

Figure 6-8 shows the mean square error (MSE) defined by Equation 6-11. It is

seen that the MSE between ��,�and +y�,� is very small. This indicates that the

performance of the estimated model is satisfactory with a good model fit.

Figure 6-8 MSE During Model Validation

The process of the ARMAX parameter estimation and validation was repeated for

100 independent data sets. Therefore, we needed to record the MSE as a single valued


Bijaya 37

score so that we could compare the MSE with two different results of simulation. The

combination of the Akaike final prediction error (Akaike 1974) and Cramer’s rule of

determinant (Weisstein 2010) yields Equation 6-12 giving us a single valued score of the

MSE.

L�N�� = abs[�� L�N × L�N��] Equation 6-12

MSE(det) is calculated from the score taken from the best ARMAX parameters.

This score signifies how well the model output may resemble the validation data. When

the MSE(det) is smaller, the model performance is better. The simulation of the results for

one hundred data sets, and their respective MSE(det) is presented in Appendix II. The

average for the MSE(det) was found to be as small as 2.42 E-06. This result indicates the

functionality and convergence of the PSO algorithm and ARMAX model.


Bijaya 38

CHAPTER 7

Spectral Analysis

7.1 White Noise

Theoretically, the spectral density of a white noise is expected to be a constant

(Brown and Hwang 1983). Since power is equally distributed in all frequencies, the total

power of the signal should be infinite, making it impossible to generate in practice.

“Since, an infinite-bandwidth white noise signal is only theoretically possible” (white

noise 2010 ), in this experiment, we dealt with white noise by using a flat spectrum over a

defined wide band.

Equation 6-1 and Equation 6-2 do imply that a “white noise random process has

zero mean and infinite power at zero time shifts. Autocorrelation function of random

process is the Dirac delta function,” (white noise 2010 ) and the Fourier transform of the

autocorrelation function is also the power spectral density (PSD) (Papoulis 1991).

��X� = �� Equation 7-1

A sample pseudo-random noise process has been generated, and its PSD was

estimated by the periodogram method. The estimate of noise PSD is shown in Figure 7-1


Bijaya 39

Figure 7-1 Noise Spectral Estimate

Since we were using a finite sample of the noise signal, the power spectral density

was estimated by using a periodogram as shown in Figure 7-1. The graph is only an

estimate of the noise spectrum, not a true noise spectrum.

7.2 Exogenous Input

The frequency domain representation of .�� described by Equation 6-3 could be

obtained using the Fourier transform. The Fourier transform of .��is demonstrated in

Equation 7-2.

Ϝ{.��} = .#��$ = �2 [�� + �� − �� − �� + �� + �� − �� − ��

+�� + �u� − �� − �u�] Equation 7-2


Bijaya 40

In this project, the sum of pure sinusoidal waves of frequencies 5 Hz, 10 Hz and

18 Hz was used as an exogenous input. PSD of the exogenous signal was estimated by

using the periodogram method and is shown in Figure 7-2.

Figure 7-2 PSD Estimate of Exogenous Input

In the Figure 7-2, we observe three distinct peaks that are due to the three

frequency components f1, f2, and f3 described in the Equation 6-3 and Equation 7-2. It

should be noticed that the exact mirror image of the spectrum shown in the positive


Bijaya 41

frequency range also exists in the negative frequency range. In the entire spectrum, the

total six main peaks exist as described in Equation 7-2.

7.3 Frequency Response of Filters A, B and C

The ARMAX parameters would resemble three filters: A, B and C. Each filter

has coefficients computed with the PSO algorithm as explained in CHAPTER 5. The

frequency responses of these filters are computed using the discrete Fourier transform

method. For a discrete set of complex or real values x[n], where n is integer, the DTFT of

x[n] is

�� = ∑ ��D[,]�4�� Equation 7-3

The frequency response of each filter was obtained with 512 point DFT. The

frequency responses of filters B and C are shown in Figure 7-3. The product of filter B,

the exogenous input and product of noise, and filter C (an ARMA part of the ARMAX

model) are also shown in the same figure.


Bijaya 42

Figure 7-3 Frequency Response of filters C and B.

It is seen in Figure 7-3 that the filter C is a low pass filter. Since the EEG may be

viewed as a signal having most information within the lower frequency band, it is

reasonably predictable that filter C is low pass filter. The sum of an ARMA part and an

exogenous part is shown in Figure 7-3. This result is filtered by the filter A. The

frequency response of filter A is also shown in Figure 7-4.


Bijaya 43

Figure 7-4 Frequency Response of filter A

It could be seen in Figure 7-4 that filter 1/A is a band stop filter. Filter responses

of these filters are not distinct; they vary with the length of data.

7.4 ARMAX Model Output

When the sum of the exogenous part and the ARMA part is filtered with the filter

A, the final output of the ARMAX model is obtained. This output is the spectral

representation of y(n) i.e. Y(ejw

), thus the spectral estimate of the EEG analyzed. The

output of the third order ARMAX model for 0.2 seconds (51 samples) of the EEG data is

shown in Figure 7-5.


Bijaya 44

Figure 7-5 Spectral Estimate of EEG

The spectral estimate indicates that the EEG signal has higher energy at lower

frequency, and the power is decreasing as the frequency increases. Jorge Baztarrica

Ochoa in his thesis on classification of EEG signals for brain computer interface

application depicts that EEG signal upto 4Hz known as ‘Delta type’ is normally present

with high amplitude in all adults and babies during sleeping (Ochoa 2002). Magnitude of

amplitude at frequency range 4-7 Hz, known as ‘Theta type’ suggests the drowsiness and

arousal in older children and Adults. Similarly frequency range of 8-12 Hz known as

‘Alpha type’ is associated with closing of eyes and state of relaxation. Frequency range


Bijaya 45

of 12-30 Hz known as ‘Beta type’, corresponds to the activeness, alertness of mind

(Kirmizi-Alsan, et al. 2006). Since our EEG data is of alcoholic patient it is expected to

have theta type EEG signal along with delta (sleeping), alpha (relaxed, closing eyes) and

beta (alert) types. Figure 7-5 shows that magnitude of delta, theta and alpha type EEG

signal are higher than beta type, which clearly suggests the patient being less active.

7.5 Performance Comparison for different model orders

Similar experiments were performed for the model orders 3, 5 and 7. The results

of all three orders are compared in Figure 7-6 for the same 0.2 Seconds (51 samples)

fragment of EEG.

Figure 7-6 EEG spectral estimates for different model orders


Bijaya 46

As seen in Figure 7-6, an increase in the model order results in spectral estimates

of a seemingly higher resolution. In order to recommend the ARMAX model over other

parametric or DFT-based spectral estimation methods, the results above were compared

with the Modified Covariance AR estimator, the ARMA model, and the periodogram

method. The procedure of the parameter estimation of Modified Covariance method is

described in Appendix II.

7.6 Comparison with an ARMA model

ARMA models have been used by many researchers to estimate the EEG

spectrum for various applications. The ARMA is a special case of an ARMAX model. By

keeping an exogenous input equal to zero, the same procedure used for the parameter

estimation of ARMAX was implemented to estimate the parameters of an ARMA model.

Once the parameters were found, the spectral estimation was performed similarly to the

ARMAX spectral estimator. The final result obtained for the third order ARMA model is

compared with the result of the third order of the ARMAX model.


Bijaya 47

Figure 7-7 EEG spectral estimates by ARMA and ARMAX methods

Figure 7-7 shows that an estimate based on an ARMA model may seem less

informative than one of an ARMAX model because the ARMA model-based estimator

has depicted only the general outline of the power distribution. Whereas, the ARMAX

model-based estimator was able to distinguish between different components of the

spectrum, and higher power is observed at lower frequencies. EEG contains most of its

information in lower frequencies; therefore, a better resolution at lower frequencies is

desirable for an EEG analysis. The spectral estimates obtained by the third order ARMA

model are insufficient to provide enough information to resolve these components in the

spectrum. Spectral estimations using ARMA models with different orders were

conducted next, and the results are presented in Figure 7-8.


Bijaya 48

Figure 7-8 Different Order ARMA Model.

As seen in Figure7-8, when the model order increases, spectral resolution

becomes better. However, we may conclude that the resolution of these spectral estimates

is still insufficient. Tseng suggested that the higher order ARMA models are needed to

represent an EEG signal (Tseng, Chong and Kuo 1995). Our results agree with Tseng’s

conclusion. Therefore, the ARMAX model is deemed more appropriate for an EEG

analysis than the ARMA model.


Bijaya 49

7.7 Comparison with an AR model and with the Periodogram method.

Next, the comparison of an ARMAX-based spectral estimator was performed

with an AR model and the periodogram method. The modified covariance technique was

used to estimate the coefficients of the AR model. Comparisons were conducted for

different orders of AR and ARMAX models using different fragments of EEG sequence.


Bijaya 50

a) Fragment 1.

Order of AR model =3

Order of ARMAX model=3

Start time= 23 Seconds.

Data length = 0.13 Seconds

Number of samples= 34.

Figure 7-9 shows a comparison of power spectral densities obtained using

periodogram, AR and ARMAX model of fragment 1.

Figure 7-9 PSD Comparison for Order 3


Bijaya 51

b) Fragment 2.




Data length = 0.1 Seconds

Number of samples= 26





Bijaya 52

c) Fragment 3.




Data length = 0.9 Seconds.

Number of samples= 24.





Bijaya 53

Spectral estimates obtained using different estimators are not equal to the true

spectrum of a signal. However, they resemble the true spectrum. Since, the research is

concerned in obtaining clear information of power distribution along frequency domain

an estimator that gives a high spectral resolution is considered better than the estimator

with a low spectral resolution. Figures 7-9, 7-10, and 7-11 indicates that the spectral

estimates obtained by an ARMAX model are somewhat similar to the spectral estimates

obtained using other methods. On the other hand, the ARMAX-based estimates show a

higher resolution. The AR method has only depicted general trends in power distribution.

Similar to ARMA, the low-order AR models are barely suitable for an EEG spectral

analysis. A DFT-based periodogram method shows a higher-resolution estimate of power

distribution. However, a close inspection reveals that the periodogram at lower

frequencies is unable to resolve different spectral components. The lack of distinct

components at lower frequencies would make the periodogram not ideal for EEG

analysis. On the other hand, spectral estimates obtained with the ARMAX model show a

higher spectral resolution even at lower frequencies. At higher frequencies, the ARMAX-

based estimates were very similar to the periodogram-based estimates.


Bijaya 54

CHAPTER 8

Conclusions and Future Work

8.1 Conclusions

In this thesis, an ARMAX model based on a parametric method of spectral

estimation was successfully developed and applied to the analysis of EEG signals.

Coefficients of the ARMAX model were estimated by using a novel optimization

technique, known as particle swarm optimization. The coefficients were designed,

implemented, and verified with different types of data sets. This optimization technique

was found very useful while estimating the coefficients of ARMA and ARMAX models.

The performance of the ARMAX spectral estimator was compared with other parametric

and nonparametric techniques such as: the periodogram method, ARMA-based analyzer,

and an AR-based spectral estimator. The estimates obtained using the ARMAX model

were found superior to other methods, especially in a low frequency diapason. We have

observed that an ARMAX model-based estimator may show higher spectral resolution at

lower frequencies. Finally, it was illustrated that the ARMAX model of a significantly

lower order - compared to ARMA or AR models - may be sufficient for an accurate

spectral estimation.

8.2 Future Work

In the future, the selection of the model order of a proposed method needs to be

optimized. Such an optimization would allow avoiding over-ordering the estimator while


Bijaya 55

maintaining sufficient spectral resolution. Perhaps, traditional order selection criteria

(Akaike or MDL, for instance) may be modified for the ARMAX.

The study of the effects of short data fragments (while using a model parameters

estimation process) is another important area of future research. Reliable spectral

estimation for short data sequences is of great interest in many fields. Therefore, knowing

practical limits of the proposed method (i.e., the minimum duration of data sequence still

yielding acceptable estimates) would be important for its applications.

This research may result in development of a common tool capable for estimating the

spectrum of all the channels of EEG simultaneously. The latter may be used to identify

the characteristics of interest in the EEG.

Finally, the proposed spectral estimation algorithm may also be applied for the

analysis of other naturally-generated signals, such as speech or seismological data.

Appropriate tests would be necessary before recommending the ARMAX model for those

applications.


Bijaya 56

CHAPTER 9

Reference

Akaike H. "A new look at the statistical model identification." IEEE Transactions on

Automatic Control (IEEE Transactions on Automatic Control) 19, no. 6 (1974):

716-723.

Akin A, and Kiymik K. "Application of Periodogram and AR Spectral Analysis to EEG

Signals." Journal of Medical Systems (Springer Netherlands) 24, no. 4 (2000):

247-256.

Angeline P. J. "Using selection to improve particle swarm optimization." International

Conference on Evolutionery Computation. Anchorage, AK: IEEE, 1998. 84-89 .

Aydın S. "Comparison of Power Spectrum Predictors in Computing Coherence Functions

for Intracortical EEG Signals." Annals of Biomedical Engineering, 2008: 192-

200.

Begleiter H. EEG correlates of genetic predisposition to alcoholism. EEG Data, at

Brooklyn: Neurodynamics Laboratory at the State University of New York Health

Center , 1999.

Bingham M. D, Godfrey, and Tukey J. W. "Modern techinques of power spectrum

estimation." IEEE transaction on Audio and Electoacoustics, 1967: 55-66.

Blodin J. "Particle swarm optimization." A tutorial, 2009.

Brown R.G., and Hwang P. Y. C. Introduction to Random Signal Analysis and Kalman

Filtering. John Willey & Sons Inc, 1983.


Bijaya 57

BSA-EE Herald. 2010. http://www.eeherald.com/section/design-guide/dg100008.html

(accessed 05 06, 2010).

Bullock T.H., Enright J.T, and Chong K.M. "Forays with the additive periodogram

applied to the EEG." 5th Joint Symp on Neural Computation. Sa Diego:

University of california, 1998. 25-28.

Candy J. V. Model-Based Signal Processing. New Jersey: John Wiley & Sons, Inc, 2006.

Eberhart R., Kennedy J, and Shi Y. Swarm intelligence. morgan kaufmann, 2001.

Eberhart R., and Kennedy J. "Particle swarm optimisation." IEEE International

Conference on Neural Networks (IEEE Int Conf. Neural Netw) 4 (1995): 1942-

1948.

Eberhart R., and Shi Y "A modified particle swarm optimizer." IEEE. Anchorage, AK,:

IEEE International. Conference on Evolutionary Computation, 1998. 69-73.

EEG- Wiki. 2010. http://bmet.wikia.com/wiki/EEG (accessed 06 02, 2010).

En'ping L., Sheng Z., and Shini Q. "Application of eigenvector estimation and SVM for

EEG Signals Classification." 9th International Conference on Electronic

Measurement & Instruments ICEMI '09. Beijing: IEEE, 2009. 4-182 - 4-185.

Estimation theory. Wikimedia Foundation, Inc. 6 5, 2010.

http://en.wikipedia.org/wiki/Parametric_estimating (accessed 12 4, 2009).

graphy-Wikipedia. 07 3, 2010. http://en.wikipedia.org/wiki/-graphy (accessed 07 07,

2010).

Gudsiz V.G., and Venayagamoorthy G.K. "Comparison of particle swarm optimization

and backpropagation as training algorithms for neural networks." IEEE Swarm


Bijaya 58

Intelligence Symposium 2003 (SIS 2003). Indianapolis, Indiana, USA: IEEE,

2003. 110-117.

Hayes M. H. Statistical Digital Signal Processing and Modeling. John Wiley & Sons Inc,

1996.

Huang C.M., Huang C.J., and Wang M.L. "A Particle Swarm Optimization to Identifying

the ARMAX Model for Short-Term Load Forecasting." IEEE Transactions on

Power Systems (IEEE TRANSACTIONS ON POWER SYSTEMS) 20, no. 2 pp

1126-1133 (2005): 1126-1133.

Instruments, National. Selecting a Model Structure in the System Identification Process.

National Instruments Corporation. 2010.

http://zone.ni.com/devzone/cda/tut/p/id/4028 (accessed 01 05, 2010).

Jenigiri S. "Comparative study of efficiency of genetic algorithms and particle swarm

optimization technique to solve permutation problems." internal Rapport,

Computer Society of India,. Mysore: University of Mysore, 1996. 490-507 .

Jenkins M., and Box G. E. P. "Time series analysis: forecasting and control." Holden-

day. San Francisco, CA: University of Wisconsut. Madison. WI and University of

Lancaster, England, 1970.

Jiao W., Liu G., and Xian L.D. "Elite Particle Swarm Optimization with mutation." 7th

International Conference on -System Simulation and Scientific Computing, 2008.

ICSC 2008. Asia Simulation Conference. Beijing: IEEE, 2008. 800 - 803.

Kennedy, J, and Eberhart R.. "A discrete binary version of the particle swarm algorithm."

International Conference on Systems, Man, and Cybernetics. Piscataway, NJ:

IEEE , 1997. 4104 - 4108.


Bijaya 59

Ljung L, System Identification, Theory for the User. Upper Saddle River, N.J: Prentice-

Hall, 1999.

Ljung L and Glad T. "On global identifiability of arbitrary." Automatica, 1994: 265–276.

Ljung L. "Estimating linear time invariant models of non-linear time-varying systems."

European Journal of control, 2001: 203–219.

Ljung L. "Perspectives on System Identification" Annual Reviews in Control, 2010: 1–12.

Maghsoudi and White C.D. "Real-time identification of parameters of the ARMA model

of the human EEG waveforms." Biomed Sci Instrum. San Antonio: US National

Library of Health, 1993.

Mandic D. "Lecture 2: Nonparametric Spectrum." www3.imperial.ac.uk. 2009.

http://www.commsp.ee.ic.ac.uk/~mandic/SE_ASP_LN/SE_ASP_Lecture_2_Non

parametric_SE.pdf (accessed 06 2010, 25).

Murugesan M and Sukanesh R. "Towards Detection of Brain Tumor in

Electroencephalogram Signals Using Support Vector Machines." International

Journal of Computer Theory and Engineering. IACSIT, 2009. 1793-8201.

Niederhauser J.J, Esteller R, Echauz J, Vachtsevanos G, and Litt B.. "Detection of seizure

precursors from depth-EEG using a sign periodogram transform." IEEE

Transactions on Biomedical Engineering (IEEE transaction on Biomedical

Engineering) 50, no. 4 (2003): 449-458.

Papoulis, A. Probability, Random Variables and Stochastic Processes. McGraw-Hill

Companies, 1991.


Bijaya 60

Paul S. R. Electroencephalography. 03 1, 2006.

http://www.scholarpedia.org/article/Electroencephalogram (accessed 10 01,

2010).

Qihou Z., Matthew B., and Jade M. "Analysis of EEG Data Using an Adaptive

Periodogram Technique." International Conference on BioMedical Engineering

and Informatics (International Conference on BioMedical Engineering and

Informatics) 2 (2008): 353-357.

Rahiman M. H. F., Taib M. N., and Salleh Y. M. "Model order criterions and model order

selection based on data from the essential oil extraction system." 3rd

International Colloquium on Signal Processing and its Applications. Melaka,

Malaysia: CSPA, 2007.

S D-Wiki. Wikimedia Foundation, Inc. 06 05, 2010.

http://en.wikipedia.org/wiki/Spectral_density (accessed 07 4, 2010).

Schuster A. "On the investigation of hidden periodicities with application to a supposed

26 day period of meteorological phenomena." Journal of Geophysical Research

(Terrestrial Magnetism and Atmospheric Electricity, ) 3, 13-41 (1898): 13-41.

Schutte J, F. "The Particle Swarm Optimization Algorithm." Lecture note on Structural

Optimization Fall 2005, 2005.

Spectral Analysis S E M. 2010.

http://www.spectralanalysis.net/spectral_estimation_method.asp (accessed 08 22,

2010).


Bijaya 61

Stoica P. and Eriksson A. "MUSIC estimation of real-valued sine-wave frequencies."

Elsevier (Elsevier North-Holland, Inc) 42, no. 2 Pages: 139 - 146 (1995): 139-

146.

System Identification Toolbox. 2010.

http://www.mathworks.com/access/helpdesk/help/toolbox/ident/ref/armax.html

(accessed 04 12, 2010).

thefreedictionary. 2010. http://www.thefreedictionary.com/encephalo- (accessed 08 26,

2010).

Tseng C. R.C., Chong F.C, and Kuo T.S. "Evaluation of parametric methods in EEG

signal analysis." Medical Engineering and Physics (Medical Engineering &

Physics) 17, no. 1 (1995): 71-78.

Wahba G, and Craven. "Smoothing noisy data with spline functions: estimating the

correct degreeof smoothing by the method of generalized cross-validation."

Numer math, 1979: 377-403.

Wang K.P., Huang L., Zhou C.G., and Pang W. "Particle swarm optimization for

traveling salesman problem." IEEE. Piscataway,NJ,USA: International

Conference on Machine Learning and Cybernetics, 2003. 1583 - 1585.

Weisstein, E. W. Cramer's Rule. 2010. http://mathworld.wolfram.com/CramersRule.html

(accessed 07 22, 2010).

Welch, P.D. "The Use of Fast Fourier Transform for the Estimation of Power Spectra: A

Method Based on Time Averaging Over Short, Modified Periodograms." IEEE

Trans. Audio Electroacoustics (IEEE Trans. Audio Electroacoust) AU-15 Pgs.70-

73 (1967): 70-73.


Bijaya 62

white noise. Wikimedia Foundation, Inc. May 25, 2010 .

http://en.wikipedia.org/wiki/White_noise (accessed July 02, 2010).


Bijaya 63

CHAPTER 10

Appendix

10.1 Modified Covariance Method

“An autoregressive (AR) process xn is represented as the output of an all-pole

filter that is driven by unit variance white noise” (Hayes 1996):

�¡� = 9��>∑ 5¢£¤¢¥¢¦§ Equation 10 -1

The power spectrum of pth

order AR process is

��¨� = |9�||�>∑ 5¢£¤©¢ª¥¢¦§ | Equation 10-2

If 17 is found, the estimate of the power spectrum may be formed using Equation 10-3.

�«��¨� = |9«�||�>∑ 5y¢£¤©¢ª¥¢¦§ | Equation 10-3

Covariance method of coefficient estimation:

Coefficient 17can be found by solving a set of linear equations presented below


Bijaya 64

¬��1,1� ��2,1� … ��®, 1��1,2� ��2,1� … ��®, 2�⋮ ⋮ ⋱ ⋮��1, ®� ��2, ®� … ��®, ®�± ²1�1�⋮1³´ = ²��0,1��0,2�⋮��0, ®�´ Equation 10-4

Where, ��0, ~� is autocorrelation of x(n) at l shift lag defined by Equation 10-4.

��0, ~� = ∑ xD�µxD�µ∗��4�³ Equation 10 -5

AR parameters in the Modified Covariance method are found by using slightly

different method of computing ��0, ~�.

Here ��~, ®� is computed using Equation 10-6.

��0, ~� = ¶ xD�µxD�µ∗ + xD�·xD�·>¸∗��4�³ Equation 10 -6


Bijaya 65

10.2 Model validation data

Original parameter

A B C

1 -0.2559 -0.2165 0.3316 0.1541 0.1142 1 0.2896 0.1763

Calculated parameters

)v /P -v MSE

Error

1 -0.4232 0.0021 -0.0677 0.4672 0.3228 1 -0.014 0.3944 7.90E-07

1 -0.2669 -0.1365 0.3124 0.1319 0.0856 1 0.1934 0.1167 4.54E-06

1 -0.3627 -0.1492 0.0926 0.072 0.0477 1 0.1396 0.2345 4.23E-06

1 -0.2613 -0.1572 0.0836 0.2721 0.4231 1 0.3801 0.2032 9.80E-07

1 -0.2252 -0.1513 0.287 0.0218 0.2677 1 0.22 0.2973 1.38E-06

1 -0.2685 -0.1831 0.3254 0.252 0.2005 1 0.056 0.157 1.23E-06

1 0.0548 -0.2496 0.2936 0.3207 -0.0032 1 0.3937 0.3029 2.55E-06

1 -0.1512 -0.3913 0.4621 0.311 -0.0737 1 -0.0018 0.362 1.27E-06

1 -0.2407 -0.1211 0.2976 0.1112 0.121 1 0.3043 0.1525 8.69E-06

1 -0.2777 -0.1546 0.3843 0.3218 0.3255 1 0.329 0.1776 2.02E-06

1 -0.2827 -0.1294 0.4505 0.193 0.1635 1 0.2363 0.1396 6.20E-07

1 -0.3184 -0.2581 0.1593 0.1114 -0.1462 1 0.3267 0.1345 3.69E-06

1 -0.2663 -0.1963 0.4013 0.3752 0.3055 1 0.1826 0.1259 2.49E-06

1 -0.1833 -0.1389 0.1681 0.2507 0.2729 1 0.2221 0.293 2.10E-06

1 -0.2476 -0.3464 0.1909 0.1249 0.1927 1 0.1599 0.1855 1.69E-06

1 -0.0647 -0.3346 0.2274 0.1858 0.3278 1 0.2315 -0.1291 2.00E-06

1 -0.2796 -0.0118 0.4399 0.2201 0.2484 1 0.3352 0.2401 3.37E-06

1 -0.0415 -0.213 0.1674 0.3487 0.2082 1 0.2418 0.1424 1.74E-06

1 -0.3133 -0.1276 0.1816 0.2345 0.2598 1 0.1321 0.2209 1.23E-06

1 -0.1737 0.0012 0.246 0.1019 0.0621 1 0.1794 0.3332 2.16E-06

1 -0.334 -0.172 0.4526 0.0419 0.2739 1 0.3057 0.1538 6.60E-07

1 -0.2335 -0.005 0.335 0.2534 0.4041 1 0.238 0.2148 4.47E-06

1 -0.0506 -0.2706 0.2276 0.2849 -0.1544 1 0.3379 0.4486 8.50E-07

1 -0.2285 -0.0547 0.3577 0.1972 0.2091 1 0.305 0.3094 8.30E-07

1 -0.2052 -0.23 0.2951 0.2919 0.1445 1 0.074 0.2237 3.08E-06

1 -0.2057 -0.0989 -0.0628 0.1464 0.0808 1 0.1583 0.2503 1.82E-05

1 -0.2431 -0.097 0.4395 0.3221 0.2123 1 0.3798 0.1942 3.70E-07

1 -0.2014 -0.1597 0.3559 -0.0578 0.3458 1 0.3129 -0.0435 6.58E-06

1 -0.2212 -0.088 0.3364 0.1196 0.1615 1 0.4584 0.2636 1.27E-06

1 -0.4243 -0.0981 0.3182 0.2438 0.1603 1 0.3084 0.266 8.10E-07

1 -0.1033 -0.2001 0.072 0.2345 0.2595 1 0.3956 0.0936 6.67E-06

1 -0.2227 -0.1064 0.185 0.0826 0.1773 1 0.3483 0.2909 4.21E-06

1 -0.2716 -0.1718 0.2614 0.1871 0.1763 1 0.0824 0.0902 8.50E-07

1 -0.3643 -0.2561 0.4286 0.102 -0.3282 1 0.2401 0.3866 5.70E-07

1 -0.1215 -0.1663 0.2238 0.1756 0.0445 1 0.3695 0.1907 1.85E-06


Bijaya 66

1 -0.3266 -0.266 0.4491 0.2763 0.0758 1 -0.0869 0.0644 2.93E-06

1 -0.2083 -0.1694 0.5363 0.3864 0.2578 1 0.1027 0.1173 1.14E-06

1 -0.1177 -0.1737 0.2394 0.4211 0.1353 1 0.1169 0.2549 5.02E-06

1 -0.2573 -0.2142 0.2721 0.2921 0.4155 1 -0.022 0.1186 3.40E-07

1 -0.1705 -0.1543 0.3064 0.1803 0.3609 1 0.2226 0.0372 1.29E-06

1 -0.2138 0.0494 0.0983 0.1232 0.1052 1 0.0509 0.1392 4.06E-06

1 -0.066 -0.1571 0.2643 0.2888 0.1807 1 0.1152 0.4421 3.80E-06

1 -0.2761 -0.1889 0.3722 -0.1797 0.2972 1 0.3768 -0.1622 1.90E-06

1 -0.2581 -0.2215 0.3345 0.2453 0.1651 1 0.0764 0.1981 8.30E-07

1 -0.3479 -0.1813 0.3109 0.4092 -0.0042 1 -0.0695 0.2035 1.38E-06

1 -0.2142 -0.245 0.226 0.1881 0.1887 1 0.2264 0.2622 1.16E-06

1 -0.2062 -0.0494 0.3374 0.0693 -0.0011 1 0.1763 0.4417 3.02E-06

1 -0.2885 -0.136 0.3395 0.2799 0.0388 1 0.0331 0.1445 3.73E-06

1 -0.1945 -0.0584 0.3527 0.2887 0.098 1 0.2658 0.251 2.73E-06

1 -0.4927 -0.1012 0.3129 0.1961 0.1906 1 0.0132 0.2162 9.70E-07

1 -0.1101 -0.086 0.2824 0.3244 0.1515 1 0.4651 0.2309 1.14E-06

1 -0.3601 -0.0696 0.3847 0.1065 0.1966 1 0.0104 0.2774 3.88E-06

1 -0.2934 -0.0266 0.1766 0.0664 0.2651 1 0.2779 0.459 3.50E-06

1 -0.1421 -0.3131 0.4044 0.3607 0.1331 1 0.2163 0.2176 2.74E-06

1 -0.2191 -0.2019 0.3368 0.1402 0.1725 1 0.0849 0.3294 5.50E-07

1 -0.2708 -0.1214 0.25 -0.0838 0.3578 1 0.321 0.2095 1.09E-06

1 -0.2763 -0.2336 0.4102 0.3595 -0.0169 1 0.1605 0.0497 2.51E-06

1 -0.2503 -0.1272 0.2344 0.1777 0.0245 1 0.4227 0.3672 4.14E-06

1 -0.274 -0.1675 0.2896 0.1102 -0.1795 1 -0.0744 0.2651 4.85E-06

1 -0.3395 -0.2213 0.2855 0.2107 -0.0688 1 0.2748 0.0668 1.49E-06

1 -0.0647 -0.3479 0.1748 0.2749 0.1581 1 0.2982 0.1778 1.14E-06

1 -0.2089 -0.309 0.1488 0.3011 0.2507 1 0.3536 -0.1034 8.50E-07

1 -0.2117 -0.2827 -0.017 0.4557 0.3089 1 0.2645 0.2419 3.10E-06

1 -0.1804 -0.0574 0.3159 0.2402 0.2505 1 0.2152 0.1979 1.63E-06

1 -0.1697 -0.1303 0.3348 0.1275 0.2396 1 0.2182 0.2762 2.44E-06

1 -0.3934 -0.168 0.1679 0.3404 0.2194 1 -0.0384 0.0981 4.05E-06

1 -0.0427 -0.3704 0.0425 0.4029 0.0262 1 0.2851 0.2577 1.50E-06

1 -0.3161 -0.0768 0.3532 0.0938 -0.026 1 0.2752 0.1823 3.53E-06

1 -0.2918 0.182 0.0869 0.1178 0.5084 1 0.3329 0.164 4.83E-06

1 0.1246 -0.1851 -0.0912 0.4266 0.1289 1 0.2201 0.2705 2.04E-06

1 -0.1481 -0.1029 0.0963 0.3644 0.4425 1 0.1548 0.1746 1.48E-06

1 -0.1203 -0.1254 0.291 0.1962 0.1567 1 0.3526 0.3883 8.00E-07

1 -0.2658 -0.2443 0.4261 0.1387 0.2924 1 0.4114 0.1364 7.80E-07

1 -0.394 0.0153 0.1129 0.2352 0.2837 1 0.0368 0.0296 3.06E-06

1 -0.2643 -0.1864 0.3664 0.1292 -0.0715 1 -0.0868 0.0036 1.87E-06

1 -0.2464 -0.1596 0.2285 0.358 0.0878 1 0.1229 0.0433 2.07E-06

1 -0.2447 -0.2584 0.2548 0.3947 0.0711 1 0.1746 0.2808 1.37E-06

1 -0.2594 -0.0782 0.3213 0.2401 0.2977 1 0.0224 0.0116 2.70E-06

1 -0.1584 -0.1805 -0.0188 0.3855 0.1988 1 0.298 0.2911 9.80E-07

1 -0.3779 -0.1716 0.3979 -0.0466 0.2143 1 0.2017 0.3523 1.00E-06

1 -0.14 -0.3255 0.1902 -0.0317 0.3965 1 0.3076 -0.2083 2.78E-06


Bijaya 67

1 -0.2942 -0.1173 0.1582 -0.1898 0.4259 1 0.3108 0.3565 2.18E-06

1 -0.3483 -0.2246 0.2737 0.0385 -0.087 1 0.27 0.4246 4.11E-06

1 -0.1706 -0.1707 0.2167 0.3927 0.0089 1 0.0691 0.2426 1.44E-06

1 -0.1851 -0.184 0.2991 0.2795 0.0585 1 0.3033 0.317 1.37E-06

1 -0.1351 -0.1902 0.1976 0.3616 -0.0986 1 0.2892 0.3352 9.10E-07

1 -0.2416 -0.1982 0.3262 0.0242 0.1713 1 0.4756 0.1756 4.51E-06

1 -0.1546 -0.0644 0.2075 0.415 0.4179 1 0.3003 0.1008 1.06E-06

1 -0.1281 -0.2326 0.294 0.253 0.0909 1 0.2113 -0.0758 3.15E-06

1 -0.2153 -0.1466 0.0114 0.075 0.1419 1 0.2396 0.3055 8.30E-07

1 -0.2185 -0.3155 0.3708 0.1964 0.0308 1 0.482 -0.1052 2.03E-06

1 -0.168 -0.2442 0.0391 0.2377 0.3429 1 0.0213 0.0549 1.21E-06

1 -0.3965 -0.0474 0.4189 0.0802 0.1569 1 0.1597 0.2761 2.42E-06

1 -0.3378 -0.2568 0.3865 0.3608 0.2716 1 0.2252 -0.2687 1.74E-06

1 -0.2987 -0.0679 0.2769 -0.0133 -0.0366 1 0.2053 0.4378 3.53E-06

1 -0.1052 -0.2972 0.1159 0.0787 0.2906 1 0.3528 0.2463 1.36E-06

1 -0.3129 -0.0095 0.4474 0.3099 0.205 1 0.0726 0.0867 2.34E-06

1 -0.2311 -0.002 0.1099 0.3496 0.2476 1 0.2746 0.4717 9.00E-07

1 -0.0679 -0.247 0.2314 0.0495 0.2301 1 0.2389 0.0629 1.93E-06

1 -0.2124 -0.0836 0.3697 0.0622 0.2616 1 0.3602 0.3699 3.00E-07

Average MSE 2.42E-06

Documents

Author's copy - produced at Lamar University · Author's copy - produced at Lamar University SPECTRAL ESTIMATION ... by using signal processing techniques. ... to have optimal resolution