Blind and Total Identiﬁcation of ARMA M

7/30/2019 Blind and Total Identication of ARMA M

1/8

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 46, NO. 6, DECEMBER 1999 1233

Blind and Total Identification of ARMA Modelsin Higher Order Cumulants Domain

Hong-Zhou Tan, Member, IEEE, and Tommy W. S. Chow, Member, IEEE

Abstract A novel recursive algorithm for identifying ordersand parameters of ARMA models driven by a sequence of non-Gaussian random signals is investigated. The input sequence isassumed to be unobservable and the conditions are based onproperties of the model output cumulants of the third order. Inevery cycle of updating the model order, the proposed algorithmminimizes a quadratic cost function to determine the parameters.The novelty of the approach is that the model orders and param-eters are all estimated without a priori knowledge; the system isblind. The identification process is said to be total because themodel parameters together with the model order are estimatedin the same process. Owing to its order-recursive nature, theproposed algorithm requires little computational complexity and

exhibits fast convergence behavior. Simulation results verify thatGaussian noises present at the output do not have noticeableeffects on the identifiability and the accuracy of estimation.

Index Terms Autoregressive moving average models, blindand total identification, convergence, order recursion, third-ordercumulants.

I. INTRODUCTION

IMPORTANT applications that deal with linear stochastic

autoregressive moving average (ARMA) models include

geophysics, sonar, radar, image processing, electrical system

fault diagnosis, and control. The problem of identification of

ARMA models has increasingly been emphasized in the sys-

tem theory, time-series analysis, adaptive control, and signal

processing literature [1], [5], [10]. Such parametric modeling

methods are primarily based on the second-order statistics,

like autocorrelations of various lags. In recent years, many

methodologies are based on the higher order statistics, for

example, the third- or the fourth-order cumulants, and have

been developed to characterize real physical random processes

[7], [12]. The rationale behind the utilization of higher order

statistics is twofold. First, higher order statistics preserve

the true phase character of the identified models, that is,

they carry full model information (both model magnitude

and model phase). Identification methods based on the higher

order statistics can, thus, reconstruct not only minimum-phase models, but nonminimum-phase models as well, i.e.,

so-called generic models. Second, because all cumulant spectra

of order greater than two are identically zero [11], in the

higher order cumulants domain the noises are eliminated if

Manuscript received February 10, 1998; revised December 29, 1998.Abstract published on the Internet August 20, 1999. This work was supportedby the Hong Kong RGC CERG Project 9040214.

The authors are with the Department of Electronic Engineering, CityUniversity of Hong Kong, Kowloon, Hong Kong.

Publisher Item Identifier S 0278-0046(99)08492-0.

the received signals are Gaussian-noises corrupted. This makes

the identification of the linear stochastic model under low

signal-to-noise ratio (SNR) possible [3].

The identification methods are usually categorized as to

whether the input signals are available or not, and whether the

inputs are Gaussian or not when they are available. As a result,

blind system identification is a fundamental signal processing

technology that retrieves unknown information regarding the

system by analyzing the characteristics of its output only. It is

particularly suitable in those situations where the only avail-

able data are the signals generated from an unknown system

excited by an unknown input, hence, the word blind. Somerecent work on blind identification of an ARMA model with

higher order cumulants and their related spectra can be found

in the literature. Batch-type or time-recursive blind algorithms

[5][7], [9], [12][14] for identifying ARMA models were

suggested under the assumptions that the orders of the handled

model must be determined or known a priori [9], in spite of

favorable performance under model mismatch conditions.

In most practical cases, however, an unknown system con-

sists of unknown system order and its corresponding param-

eters. The system identification is said to be total because

the model parameters, as well as the model order, can be

estimated all together, or equivalently; the poles and zeros

of the model transfer functions can be determined completely.In [4], a blind and total identification method was developed

to deal with AR submodels subordinate to ARMA models

where MA submodel estimations were unblind. A total system

identification technique was then used to estimate ARMA

parameters as well as orders simultaneously in [2], where

inputoutput pair observations were used to obtain the esti-

mated third-order cumulants and crosscumulants, which does

not preserve the blind property of identification. Here, based

on the third-order noisy output cumulants and model-order

recursive strategy, we focus specifically on questions about the

blind and total algorithm for estimating nonminimum-phase

ARMA models of which orders are not givena priori

. Theprocedure begins with a first-order AR and MA model and

updates the model order with an increment of one. At each

updated order, the corresponding parameters are estimated by

using the least-squares algorithm to minimize the well-defined

cost function. The magnitude of the cost function decreases

as the order increases. When the updated order reaches the

true order, the magnitude of the cost function is minimum

and remains constant, even if the order is further increased.

The AR order and the MA order of the whole model are then

determined by monitoring the variation of the cost-function

02780046/99$10.00 1999 IEEE


2/8

1234 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 46, NO. 6, DECEMBER 1999

characteristic curve. This enables us to determine the AR

and MA order of nonminimum-phase ARMA models. When

the AR and MA order are determined, their corresponding

parameters estimations can be obtained accordingly.

The remainder of this paper is organized as follows. In

Section II, we describe the statement of the problem and

preliminaries briefly. Cumulant-based order-recursive identifi-

cation algorithms of AR and MA submodels are proposed and

analyzed, respectively, in Sections III and IV. In Section V,

asymptotically convergent behavior of the algorithm is ana-

lyzed, and numerical simulation paradigms are presented in

Section VI. Finally, conclusions are drawn in Section VII.

II. STATEMENT OF PROBLEM AND PRELIMINARIES

Consider a linear stochastic ARMA model with in-

putoutput pair which is described by

(1)

where the excited sequence of the model is azero-mean non-Gaussian process with and

The model (1) contains unknown parameters

and unknown

orders which satisfies and

For observations of output signals, there

seem to exist several unknown additive noises. Assume here

the model output port is corrupted by Gaussian noises of

unknown pdf, The available information-bearing

measurements are, thereby, in the form of

(2)

where additive noises are zero-mean, white (or col-

ored) Gaussian processes, which are independent of modeloutput For measured random signals in noisy envi-

ronment its third-order cumulants are just equal to its

third-order moments that are defined as

(3)

Combining (2) with (3), with the feature of higher order

cumulants for Gaussian signals in mind, we can express (3) as

(4)

The above expression implies that, in the higher order cu-

mulants domain, the effect of Gaussian noises, which infects

the non-Gaussian signals, are eliminated. Subsequently, the

clean signals are utilized for the identification of ARMAmodels.

The basic idea in this paper is to introduce an identification

method for determining the parameters

and orders of model (1),

given finite output measurements

alone; herein, is the length of the observed signals. The

parameter estimation is performed model-order recursively

until the estimated parameters together with the model order

converge and reach their optimal values. In this paper, this

approach in identifying the parameters and order of the ARMA

models is said to be total.

III. AR SUBMODELS IDENTIFICATION

In this section, we briefly describe an order-recursive algo-

rithm for the on-line identification of parameters and orders of

AR submodels which belong to ARMA models. Further details

and theoretical background of this method can be found in [4].

The one-dimensional (1-D) cumulant slice for ARMA

models in (1) is constrained by the following AR delin-

eation [1]:

for (5a)

Considering the property of (4), we express the above equation

as

for (5b)

One may define as the estimated corre-

sponding to the estimates of parameters and orders,

Equation (5b) is rewritten as follows:

for (6)

The nonlinear cost function is constructed in matrix form as

follows, where definitions are given in (8a)(8e), shown at the

bottom of the next page:

(7)

Take the derivative of with respect to parameters and

let all the derivatives equal zero

(9)

The order-recursive solution of the least square isobtained as the above conditions are all satisfied, i.e.,

(10a)(10c), as shown at the bottom of the next page. Therein,

the corresponding matrices are (11a)(11f), shown at the

bottom of the next page.

Equations (10a)(10c) together constitute the proposed al-

gorithm. The implementation of the algorithm begins by using

a first testing order of AR submodels to approximate the ob-

served output sequences. In accordance with the convergence

analysis described in Section V, we will illustrate that the

value of the cost function drops with increases in the testing

order. Also, when the recursive order is equal to or greater than

the true order, the value of the cost function and the estimated

parameters remain unchanged. Thus, the correct estimate of

the AR submodel is obtained, and the testing order is equal

to its true one.

IV. MA SUBMODELS IDENTIFICATION

To estimate the MA submodel parameters and orders, the

so-called residual-time-series method [7] is used in this paper.

The output observations are applied as inputs to the th

order filter with transfer function

where are the AR parameters previously computed. The


3/8

TAN AND CHOW: BLIND AND TOTAL IDENTIFICATION OF ARMA MODELS 1235

residual time series is, thus, given by

(12)

Rewriting (1) with instead of and considering (2)

and (12), we can obtain

(13)

In other words, (13) is an MA submodel of ARMA model

(1), and it is worth noting that the submodel output

contains an additive Gaussian noise term. It is worth noting

that the following expression is apparently tenable due to the

cumulant property:

(14)

The following closed-form formulas for in MA model

(13) is deduced according to [5]:

(15)

(8a)

......

......

(8b)

(8c)

(8d)

(8e)

(10a)

(10b)

(10c)

(11a)

(11b)

...

...... (11c)

(11d)

(11e)

... (11f)


4/8


(a) (b) (c)

Fig. 1. The error between true and estimated output third-order cumulants where lags SNR 0.1 dB. (a) Example 1. (b) Example2. (c) Example 3.

Herein, refers to the autocor-

relation statistics of noisy residual time series and

It is easy to verify that the

autocorrelation can be expressed as

(16a)

Substituting (16a) into the left-hand side (LHS) of (15), we

lead to

for (16b)

Therefore, the right-hand side (RHS) of (15) is given by

for

(17)

As per the AR submodel expression of (6), we define

as the estimated cumulants of the residual time series

corresponding to the estimated MA parameters and orders

Thus, from (17), the following equation

is obtained:

for

(18)

The corresponding cost function is constituted as that of AR

submodel

(19a)

where definitions are given in (19b)(19d), shown at the

bottom of the page. Comparing (19a) with (7), by replacing

and with and we can utilize

the AR order-recursive algorithm of (10a)(10c) to identifyMA parameters and orders.

V. CONVERGENCE ANALYSIS

In this section, we demonstrate that the minimization of

the cost function yields an asymptotically convergent estimate

of ARMA parameters and orders. If the estimation of both

the AR and the MA parts subordinate to the ARMA model

has convergent behaviors, it is said that the estimation of

the ARMA model is convergent according to the proposed

algorithm. For brevity, the convergence analysis for the AR

submodel estimation is presented here. Further details can be

found in [1].Considering (11a)(11e) and the second term of the RHS

of (10c) , we obtain

(20a)

(20b)

Combining (10a) and (10c) with (20a) and (20b), we get

(21a)

(21b)

where...

......

......

(19b)

(19c)

(19d)


5/8


(a) (b)

(c)

Fig. 2. Cost-function trajectory. (a) Example 1. (b) Example 2. (c) Example 3.

(a) (b)

(c) (d)

Fig. 3. Convergence curve of model parameters. (a) Example 1. (b) Example 2. (c) AR part of example 3. (d) MA part of example 3.

Assume that the true (optimum) value of the estimated

AR parameter vector is

Also assume that a zero-mean additive error vector is

introduced [1]. The following equation is obtained from (7):

(22)

Assume the error vector is independent of While

the testing recursive order is equal to or greater than the true

order, we have

(23)


6/8


(24)

Considering inequality (21a), an asymptotic convergence is

found to be

(25a)

Likewise, the cost function is also convergent, i.e.,

(25b)

In the same way as above, MA submodels identification

converge along with the convergence of the cost function

Hence, we can conclude that the total order-recursive

identification of the ARMA models has a strong convergent

characteristic.

The identification procedure begins with a first-order AR

and MA model and updates the model order with an incrementof one. At each test order, the corresponding parameters are

estimated by using the least-squares algorithm to minimize the

well-defined cost function. The magnitude of the cost function

decreases as the order increases. When the test order reaches

the true order, the magnitude of the cost function is minimum

and remains constant if the test order is further increased.

Thus, we can conclude that the estimates of ARMA order

are just the value at the last turning point of the cost-function

characteristic curve, where the corresponding parameters are

also estimated. It is also the very order-recursive (not time-

recursive) feature that the proposed algorithm exhibits less

computational complexity and faster convergent performance

than those of the LMS or RLS algorithms based on higherorder statistics [1], [5].

VI. NUMERICAL SIMULATIONS

In this section, the performance of the blind and total algo-

rithm for ARMA models is tested by computer simulations.

As we know from Sections II and IV, the definition of the

third-order cumulants involves expectation operations. They

cannot be computed in an exact manner from real data and

must be approximated by replacing expectations with sample

averages. Assuming the length of is

its conventional sample third-order cumulants denoted by

are estimated as [11]

(26)

In the following three experiments of this section, 1024

output signals are sampled and the driving signal

is assumed to be zero-mean exponentially distributed, with

and The additive Gaussian noise

which is colored and independent of the model output

sequence is produced at SNR 0.1 dB. To verify the

theoretical results, 30 Monte Carlo runs were performed. The

TABLE ITRUE AND ESTIMATED PARAMETERS OF ARMA(3, 1), WITH CORRESPONDING

MEANS AND VARIANCES, UNDER SNR 0.1 dB, 30 MONTE-CARLO RUNS

TABLE IITRUE AND ESTIMATED PARAMETERS OF ARMA(3,2), WITH CORRESPONDINGMEANS AND VARIANCES, UNDER SNR 0.1 dB, 30 MONTE-CARLO RUNS

final results are summarized in tables where the true and

estimated parameters (mean and variance) are compared. Their

correspondent cost-function characteristic trajectories are also

plotted.

Example 1: This is a true nonminimum-phase model

ARMA(3,1)

with poles at and a zero at

Example 2: This is a nonminimum-phase ARMA(3,2)

model

Its poles are located at 0.5, 0.3, 0.6, and zeros at 2 and 0.5.


7/8


TABLE IIITRUE AND ESTIMATED PARAMETERS OF ARMA(4, 3), WITH CORRESPONDING

MEANS AND VARIANCES, UNDER SNR 0.1 dB, 30 MONTE CARLO RUNS

Example 3: This is an identified nonminimum phase

ARMA(4,3) model

with four poles at and three zeros

at 2, 0.5, and 1.

The convergence of the parameters is shown in Fig. 3. The

upper order bounds of the models for Examples 13 are 5,

5, and 6, respectively. The three-dimensional (3-D) graphicsof the errors between the true and estimated output third-

order cumulants for Examples 1-3 are, respectively, depicted

in Fig. 1(a)(c). The cost-function characteristic curves of the

AR part and MA part and are, respectively, shown in

Fig. 2(a)(c). Clearly, at the beginning, the magnitudes of

and decrease rapidly, then they remain the same values

after some order updates, where for Example

1, for Example 2, and for

Example 3. Thus, it is rather straightforward to determine

that the estimated orders of the AR part and the MA part

are the same as their own true ones. It is also worth noting

that, in Fig. 2 after and the decreases of the cost

functions are unnoticeable. It is also interesting to point out thevery slight numerical changing in the cost function when

and are contributed by the estimate errors illustrated in

Fig. 1. Estimated parameters at their correspondent points over

30 Monte Carlo runs are listed in Tables IIII, together with

the correspondent means and variances. From the obtained

results, the mean values of the parameters evaluated by the

proposed algorithm are good approximations of the true values

of the corresponding true ARMA models. It is also clear

that the proposed algorithm runs faster and is simpler than

the celebrated LMS or RLS methods based on higher order

statistics.

VII. CONCLUSION

New aspects of the blind and total system identification

of linear stochastic ARMA models have been presented in

this paper. An important feature of the proposed method is

its order-recursive implementation of estimating procedures.

A detailed analysis of the model approximation has indicated

that the so-called total identification happens even if the

model orders and parameters are all unavailable a priori.The utilization of phase information-bearing higher order

cumulants guarantees the reconstruction of generic ARMA

models and the elimination of the effect attributed to the

presence of Gaussian noises in the output signals.

In the case of identifying ARMA models, the parameters and

orders of the AR part and MA part were separately estimated.

Firstly, an order-recursive model equation associated with the

AR part was built to determine its parameters and orders.

A residual-time-series sequence was then produced and the

MA part can be identified totally by using an order-recursive

formula similar to that of the AR submodel. In comparing our

proposed technique with the LMS and the RLS approaches

[1], [5], simulation paradigms showed that the convergent rateof the proposed algorithm is much faster than that of the LMS

and the RLS algorithms. The computational complexity of our

proposed algorithm is also less than that of the LMS and the

RLS algorithms.

REFERENCES

[1] T. W. S. Chow, H.-Z. Tan, and G. Fei, Third-order cumulant RLS algo-rithm for nonminimum ARMA systems identification, Signal Process.,vol. 61, no. 1, pp. 2338, Aug. 1997.

[2] T. W. S. Chow and H.-Z. Tan, Recursive scheme for ARMA parameterand order estimation with noisy input-output data, Proc. Inst. Elect.

Eng.Vision, Image and Signal Processing, vol. 146, no. 2, pp. 6572,Apr. 1999.

[3] T. W. S. Chow, G. Fei, and S. Y. Cho, Higher-order cumulants-based

least squares for nonminimum-phase system identification, IEEE Trans.Ind. Electron., vol. 44, pp. 707716, Oct. 1997.

[4] T. W. S. Chow and H.-Z. Tan, Semiblind identification of nonminimumphase ARMA models via order recursion with higher-order cumulants,

IEEE Trans. Ind. Electron., vol. 45, pp. 663771, Aug. 1998.[5] B. Friedlander and B. Porat, Adaptive IIR algorithm based on high-

order statistics, IEEE Trans. Acoust., Speech, Signal Processing, vol.37, pp. 485495, Apr. 1989.

[6] G. B. Giannakis, On the identifiability of non-Gaussian ARMA modelsusing cumulants, IEEE Trans. Automat. Contr., vol. 35, pp. 1826, Jan.1990.

[7] G. B. Giannakis and J. M. Mendel, Identification of nonminimum phasesystems using higher order statistics, IEEE Trans. Acoust., Speech,Signal Processing, vol. 37, pp. 360377, Mar. 1989.

[8] , Cumulant-based order determination of non-Gaussian ARMAmodels, IEEE Trans. Acoust., Speech, Signal Processing, vol. 38, pp.14111423, Aug. 1990.

[9] G. B. Giannakis and A. Swami, Identifiability of general ARMAprocesses using linear cumulant-based estimators, Automatica, vol. 28,pp. 771779, July 1992.

[10] R. Johansson, System Modeling and Identification. Englewood Cliffs,NJ: Prentice-Hall, 1993.

[11] C. L. Nikias and M. R. Raghuveer, Bispectrum estimation: A digitalsignal processing framework, Proc. IEEE, vol. 75, pp. 869891, July1987.

[12] A. Swami and J. M. Mendel, ARMA parameter estimation using onlyoutput cumulants, IEEE Trans. Acoust., Speech, Signal Processing, vol.38, pp. 12571265, July 1990.

[13] J. Vidal and J. A. R. Fonollosa, Adaptive blind system identificationusing weighted cumulant slices, Int. J. Adapt. Contr. Signal Processing,vol. 10, pp. 213237, Mar.June 1996.

[14] X. D. Zhang, Y. Song, and Y. D. Li, Adaptive identification of non-minimum phase ARMA models using higher order cumulants alone,

IEEE Trans. Signal Processing, vol. 44, pp. 12851288, May 1996.


8/8


Hong-Zhou Tan (M98) received the B.S. degree inelectrical engineering from Chongqing ArchitectureUniversity, Chongqing, China, and the M.S. degreein automatic control and the Ph.D. degree in elec-tronic engineering from South China University ofTechnology, Guangzhou, China, in 1985, 1991, and1998, respectively.

From 1985 to 1988, he was with ChongqingArchitecture University. In 1991, he became a Lec-turer in the Automation Department, South China

University of Technology. Since 1996, he has beenwith the Department of Electronic Engineering, City University of Hong Kong,Kowloon, Hong Kong. His primary research interests include higher orderspectra analysis, signal processing, adaptive filtering and control, and systemsidentification.

Tommy W. S. Chow (M93) received the B.Sc.(Hons.) degree and the Ph.D. degree from the Uni-versity of Sunderland, Sunderland, U.K.

He is currently an Associate Professor in the De-partment of Electronic Engineering, City Universityof Hong Kong, Kowloon, Hong Kong. His currentresearch activities include artificial neural networksand applications, signal processing, and machinefault analysis.

Documents

Blind and Total Identiﬁcation of ARMA M