Asymptotic theory of mixed time averages and kth-order cyclic-moment and cumulant statistics

216 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 41, NO. 1, JANUARY 1995

Asymptotic Theory of Mixed Time Averages and Kth-Order Cyclic-Moment and Cumulant Statistics

Amod V. Dandawatk, Member, IEEE, and Georgios B. Giannakis, Senior Member, IEEE

Abstruct- We generalize Parzen’s analysis of “asymptotically stationary” processes to mixtures of deterministic, stationary, nonstationary, and generally complex time series. Under certain mixing conditions expressed in terms of joint cumulant summability, we show that time averages of such mixtures converge in the mean-square sense to their ensemble averages. We ad- ditionally show that sample averages of arbitrary orders are joidtly complex normal and provide their covariance expressions. These conclusions provide us with statistical tools that treat random and deterministic signals on a common framework and are helpful in defining generalized moments and cumulants of mixed processes. As an important consequence, we develop consistent and asymptotically normal estimators for time-varying, and cyclic- moments and cumulants of kth-order cyclostationary processes and provide computable variance expressions. Some examples are considered to illustrate the salient features of the analysis.

Index Terms- Cyclostationarity, cumulants, higher order statistics, mixed spectra, almost periodic time series, consistency, and asymptotic normality.

I. INTRODUCTION AND BACKGROUND COMMON practice in the implementation of probabilis- A tic expectations is to replace them with sample averages,

and establish consistency of the resulting estimators. For finite data sets, the proximity of the sample estimators to their expected values can be determined from the asymptotic distribution. This is useful in implementing and evaluating theoretical time-domain signal processing schemes that employ moments and cumulants which are defined using the probabilistic expectations.

For stationary processes, consistency of third- and fourth- order sample moment and cumulant estimators was established, respectively, by [35], [34] and [24]. However, consistency of sample moment and cumulant estimators of an arbitrary mixture of k 2 3, deterministic, stationary, and nonstationary processes was left as an open problem. Such combinations are of interest, for example, in spectral analysis of mixed processes such as sinusoids in noise [26], and system identification with deterministic inputs and stationary measurement noise [27].

The objectives of this paper are: i) to study the asymptotic properties of sample averages which consist of a mixture

Manuscript received May 3, 1993; revised May 10, 1994. Parts of the results were presented at the International Conference on Acoustics, Speech, and Signal Processing, Minneapolis, MN, April 27-31, 1993. This research was funded by the Office of Naval Research under Grant N00014-93-1-0485.

The authors are with the Department of Electrical Engineering, University of Virginia, Charlottesville, VA 22903-2442 USA.

IEEE Log Number 9406573.

of deterministic, stationary, and nonstationary complex sequences, in arbitrary combinations, ii) to establish consistency and asymptotic normality of kth-order statistics of cyclostationary processes, and iii) to provide a common framework for treatment of a class of deterministic and random complex signals, such as processes with mixed spectra, via generalized moments and cumulants.

Earlier results on the asymptotic properties of sample averages of not necessarily stationary processes were provided by Parzen [31], who showed, under certain mixing conditions, that for a real process z ( t )

where mgs. represents convergence in the mean-square sense (for extensions of these notions to cyclostationary processes using a “nonprobabilistic” framework, see [ 141). However, Parzen did not study the asymptotic distribution of sample averages and made no connection with the common framework which can be used for estimation of certain nonstationary statistics.

Under some mixing conditions, distinct from those of Parzen’s, we establish that for a mixture of ( k + 1) discrete- time complex processes, { z m ( t ) } i = ” , which may be random or deterministic, a relationship analogous to (1) holds. Further, we prove asymptotic normality of sample averages and provide covariance expressions. With the help of certain examples, we show the simplicity of our mixing conditions and also illustrate the applicability of these ideas in defining (generalized) moments and cumulants of processes with mixed spectra. Further, we use our results for developing consistent and asymptotically normal single-record estimators for the ‘ statistics of a class of nonstationary processes to be termed as kth-order (almost) cyclostationary. Computable variance expressions are also provided. Our analysis treats estimation of kth-order moments and cumulants of stationary processes as a special case.

The joint cumulant, cum (21, . . . , z p } of random variables { z1, . . . , z p } , is defined as the coefficient of ( - - j ) p u l . . . up in the Taylor series expansion of In E{exp [ j ( z lu l+. . .+zpup)]} (see [5, p. 191). Our mixing condition can be stated in terms of the joint cumulants of{z,(t)}i=, as follows:

Assumption 1 . Z : Let { z ~ ( t ) } ~ = O be ( k + 1) sequences (deterministic or random) such that all their joint cumulants of

0018-9448/95$04.00 0 1995 IEEE

I

DANDAWAT6 AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS

all orders satisfy Vm E Z

where

and * denotes complex conjugation. Intuitively, Assumption 1.1 implies that samples of the

processes { z m ( t ) } ~ , o that are well separated in time are approximately independent. Mixing conditions of this form were used by Brillinger and Rosenblatt [6] for developing consistent polyspectrum estimators for stationary processes and are met in practice, for example, by stable linear processes (see Example 1 of Section 11-B). Apart from (2) we also assume the following:

Assumption 1.2:

For k = 1, zo(t + T O ) = z ( t ) , and zl(t + 7-1) = z(t + T ) ,

Assumption 1.2 defines Parzen's class of asymptotically stationary processes z ( t ) , which accept generalized harmonic analysis [31]. For stationary processes using the time invari- ance of the ensemble moment we infer that

so that Assumption 1.2 in this case translates to requiring that the kth-order cross moment of the ( k + 1) processes to exist. As will be seen later on, Assumption 1.2 is satisfied also by kth-order cyclostationary processes.

Assumptions 1.1 and 1.2 dictate the class of signals that are of interest to us in this paper.

Asymptotic properties of sample averages are defined in Section I1 and some examples are presented to interpret the results. Consistent and asymptotically normal estimators for statistics of kth-order (almost) cyclostationary processes are presented along with their asymptotic covariances in Section 111, and finally, conclusions are drawn in Section V. Since we deal with complex normal distributions throughout this paper a relevant discussion is provided in Appendix I. Proofs of all the Theorems are summarized in Section IV.

11. ASYMFTOTIC PROPERTIES OF SAMPLE AVERAGES

It is well known [31], that for a zero-mean stationary process E{z( t ) z ( t + T ) } may be z ( t ) , the covariance c g , ( t ; ~ )

estimated by the following sample average:

If the process is mixing then it can be shown that

-

217

where ,gs' represents convergence in the mean-square sense. This conclusion forms the basis for implementing theoretical signal processing schemes using finite data.

In this section, we study the asymptotic behavior of sample averages for a mixture of random and deterministic signals on a common framework. The objective of this study is to provide tools for statistical analysis of estimators based on sample averages of deterministic, stationary, and nonstationary discrete-time complex signals in any combination. Such sample averages are important, for example, in regression analysis (e.g., [19), system identification [27], and as will be seen in Section 111 for estimation of cyclic statistics.

Asymptotic results are derived in Section 11-A and some examples are considered in Section 11-B to illustrate potential applications.

A. Asymptotic Results

random complex signals, with joint moment' Let { ~ ~ ( t ) } k = ~ , be a mixture of ( k + 1) deterministic or

m,, Z k (t: TO,. . . T k ) 2 E{zo(t + TO). ' . 5 k ( t + T k ) }

and corresponding sample estimate

Mi.) ( T O , . .. , rk) 2 E zo(t + T O ) . . . z k ( t + T k ) (6) t=O

where T is the length of the available data samples.A Before presenting the asymptotic properties of M ( T ) we

prove the following lemma: Lemma 2.1: Let {zm( t )} i=o be ( k + l ) discrete-time com-

plex processes (deterministic or random) and define the product processes f,(t; 7,) of order K , as

K,

f n ( t ; T n ) n z , , r ( t + .r,,l>. %,l(t> E { ~ m ( t > > L o , 1=O

Tn (Tn,O,. . . ,Tn,K,) (7)

where T,,Z are arbitrary but fixed lags. With if Assumption 1.1 holds then, Vm i)

(11, . . . , &),

' Although we consider moments without conjugation of components, our analysis is applicable with or without conjugation of components.


Conclusions i) and ii) hold even when any of the f ’ s are conjugated.

Proof: See Appendix 11. As discussed in Section I, Assumption 1 . 1 implies joint

asymptotic independence of { 2 , (t)}kzo. Conclusion i) of Lemma 2 . 1 affirms that Assumption 1 . 1 is sufficient to impose joint asymptotic independence on products of ( 2 , ( t ) } i=o . In other words, Assumption 1 . 1 implies a mixing condition for the product processes defined in (7). Clearly, from (6) and (7), M ( T ) is a sample average of mixing processes f , and hence, a form of law of large number and a central limit theorem will be useful in studying its convergence properties and asymptotic distribution. These statements are proved in the next theorem and its corollary which summarize the main results of this section; namely, the asymptotic properties of generalized sample averages:

Theorem 2.1 If { ~ , ( t ) } k , ~ is a set of ( k f l ) discrete-time complex processes ( random or deterministic) which satisfy Assumptions 1 . 1 and 1 . 2 , then with

z,,i(t) E { X ~ ( ~ ) } ~ = ~ , T , A ( T , , ~ , . . . , T , J ~ ) k

the asymptotic cross-covariances of

f i [ M K l ( T m ) - M K , ( ~ T L ) ]

and

f i [ M K 2 ( T n ) - M K , ( ~ , ) ]

are given as

T+03 lim T cum { Mgi ( T ~ ) , M r j (T,), } = SZ~(T,, 7,) ( 1 2 )

and2

lim Tcum{Mr!(Tm),M~nT)(T,),} = S;*,’(T~,T,). ( 1 3 )

T-CC

The results of Theorem 2 . 1 imply that for a mixture of processes the sample averages of their products of arbitrary orders, K,, and arbitary lags, T,, are mean-square-sense consistent and asymptotically jointly normal as seen from the following discussion: From (9) we note that the covariance (also second-order cumulant with m = 1 in (9)) of the M g-) (T,)’s vanishes asymptotically proving consistency.

and et us assume that 1 T- l Kn

1 T-l

T-03 lim fi~[iiZZT,1(7,) - MK,(T,)] = 0.

This is at least true if { ~ ~ ( t ) } k , ~ are all stationary, since

M ( T ) @ , ) K , A - T n 2 , , l ( t + 74) t=O 1=0

(8) in this case = - f n ( t ; T n )

t=O K ,

it holds that E r n X , , l ( t + Tn,l)> 1=0

E{MiTn)(r,)} = {MK,(T,))

for all T from (8) and the definition of {MK,(T,)} following (11 ) . We show in Section 111-B (see (57)) that this assumption is also valid if { ~ , ( t ) } & , ~ are cyclostationary. The implication of this assumption is that

for any arbitrary but fixed orders K,, and lags ~ , , n = O,... ,m, even if any of the M K ~ ( T , ) ’ s are conjugated. Further, with

K ,

1=0 f m ( t ; T m ) A n E m , l ( t + Tm,l)

1 =o

. T-1

is asymptotically zero-mean. To infer asymptotic normality, we follow the approach in

[5, Lemma P4.5, p. 4031 (see also Appendix I) and start with (9) which implies that

<=-CC

(10) *In (1 1) and (13) the (*) in the superscript is merely notational.

DANDAWATE AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS 219

even when some of the fiK:(~,)’s are conjugated. Now from the properties of cumulants [5 , p. 191 it follows that cumulants

process x ( t ) , may be defined as

of f l M F ; ( r n ) are the same as those of mkz(71, . . . , 7k-1) ~ T-1

t=O

(16) To express the kth-order generalized cumulant ckz(~l , . . .,

Tk-l), consider the set of integers { 1 , 2 , . . . , k } . If P = (VI, . . . , up) denotes a partition of { 1 , 2 , . . . , k } , then [6]

where the nonrandom constant

~ ~ ~ ( 7 , ) A lim n;ZKT(Tn). T-02

Equation (14) with m 2 2 therefore shows that the joint cumulants of ~ ~ ( 7 1 , . . . , Tk-1) 2 X(-l)(p-’)(p - I)! m,, . . . , mvp

and

@[fi$?’(~n) - Mkn(7n)]

vanish asymptotically proving asymptotic normality of

(note that there are (m + 1) sample moments in (9)). Returning now to the proof of mean-square-sense conver-

gence and asymptotic normality of M$T!..zk (TO, . . . , ~ k ) , we first note from (6) and (8) that Mi:! . . , , (TO, . . . , ~ k ) is a special case of M ~ ? ( T ~ ) with K, = k{x,,l(t) = ~ l ( t ) } p = ~

and 7, (TO, . . . , ~ k ) . Since the limit as T + cc of E{Mi:!..zk (TO, . . . , Q)} exists by Assumption 1.2, unbiasedness follows trivially under the hypotheses of Theorem 2.1. Consistency and asymptotic normality follow as special cases of the results of Theorem 2.1. We summarize these observations in the following corollary:

Corollary 2. I : Under the hypotheses of Theorem 2.1 Mzo,. . . ,zk A (TI (TO, . . . , 71~) of (6) converges in the mean-square sense; i.e.,

lirn Mi:! _.., z k (70, . . . , 7rc) T+o=

. T-1

In addition, if

lim f i E [ f i g ! ( ~ ~ ) - M K , (7,)l = 0 T-CX

then f l [MkT!. . zk ( T ~ , . . . , 7 k ) - Mzo. . .zk (TO, . . . , ~ k ) ] is asymptotically complex-normal, with zero-mean and covariances given by (12) and (13) with K, = k , {x , ,~ ( t ) =

Note that Theorem 2.1 and Corollary 2.1 treat random and deterministic signals on a common framework, and therefore can be used to define (generalized) moments for processes involving both random and deterministic signals such as processes with mixed spectra. Using these (generalized) moments, one may define (generalized) cumulants for mixed processes. Specifically, the kth-order generalized moment of a mixed

zl(t)}f=, and 7, ( T O , . . . , n).

W

(17) where we sum over all the partitions of the form V I , . . . , vp, with p representing the number of members in each partition, and m, denoting the time-average

of the product of x( t + ~ k ) with subscript in v.

ments and cumulants define For constructing sample estimators of the generalized mo-

(18) 1 T-l E T { . . .} d - E{. . .}

t=O T

then, the kth-order (generalized) sample moment of a process z ( t ) , may be defined as

m(T) kz (71,. . . , 7k.1) kT{z( t )x ( t + 71). . ’ x ( t + T k - l ) }

and the (generalized) sample cumulant ?K), may be defined as in (17), with the m’s replaced by time-average ET of the product of x( t + r k ) with subscript in v.

Thus for example, with IC = 3 in (17), we find that (see also [SI)

tK’(71, 72) fiT{z(t)x(t + 71)x(t + 72))

- J%{z(t))&{z(t + 71)x(t + 7 2 ) )

- ET{x(t + 71)}fiT{x(t)x(t f 72))

- fiT{x(t + 72)}fiT(x(t)z(t f 71))

+ 2fiT{x(t)}fiT{z(t + 71)}ET{x(t f 72)).

(19)

The asymptotic properties of ?K) can be studied via Theorem 2.1, and cross-generalized moments and cumulants can be defined similar to the auto-versions given herein. Example 3, in the next subsection shows that importance of generalized cumulants.

Specializing to k = 1, zo(t + 70) = z ( t ) , and xl(t + 71) = x ( t + T ) , in Corollary 2.1, the mean-square-sense consistency of f i g ) , reduces to Parzen’s result (see (l)), under the stated mixing conditions. Next we present some examples to illustrate the applicability of our results.

B. Motivating Applications

The theme throughout this section is to present examples of processes that satisfy our mixing condition (2) and which

cannot be analyzed using conventional analysis tools for stationary processes, but can be handled using the results of Section 2.1.

Example 1 (Linear Processes)

A very common assumption made in the analysis of random processes is that they are generated by a linear filter. In this example we consider stable (non) stationary linear processes and show that they obey our mixing condition, and hence their sample averages can be analyzed via Theorem 2.1. Let { ~ ~ ( t ) } k = ~ be ( I C + 1) nonstationary linear processes generated by linear time-varying filters as follows:

where t(t) is an independent and identically distributed (i.i.d.) process with Icth-order cumulant 7ker bounded moments, and the filter kemels satisfying

consistency of sample cross-moments, since in this case, due to stationarity . T-1

= E { z o ( t + 7 ~ ) . . . z k ( t + T k ) } . Many communication transmission signals can be represented as an output of linear periodically time-varying system [13], and our results will be useful in implementing and analyzing relevant schemes. Note that for well-defined discrete periodic processes the right-hand side of (24) necessarily exists. Other applications include studying the radar ambiguity function [SI, [lo].

Example 2 (AM Signals) Amplitude-modulated (AM) signals and processes with

missing observations can be modeled as (e.g., 191, [171, [311, [321)

To check if {z,(t)}k=O are mixing in the sense of Assump- tion 1.1, let us consider:

where ([I, . . . &) and the last equality follows from the multilinearity of cumulants. Using the fact that cumulants of independent random variables vanish [5, p. 191 we obtain from (22), that Vl = 1, . . . IC

T L € 1

where the last inequality follows upon using (21). Therefore, { ~ ~ ( t ) } k = ~ are jointly mixing in the sense of Assumption 1.1.

Now if { ~ ~ ( t ) } k = ~ also satisfy Assumption 1.2 then from Theorem 2.1

~ T-1

t=O ~ T-I

m.s.s. 1

T+m T = lim - E{zo(t + T O ) . . . z k ( t + ~ k ) } . (24) t = O

If { z, (t)}&=o are stationary-linear processes then, h , ( t ; ~ ) = h , ( ~ ) in (20L and thus (24) Droves m.s.s.


~I - \ , ~ I I (28)

where b ( t ) is a known and deterministic modulating function for which

- T-1

exists, z ( t ) is a zero-mean (stationary) information signal, and w(t) is a zero-mean additive Gaussian noise of unknown covariance which is assumed independent of z ( t ) . Our objective is to recover the third-order cumulant c g , ( ~ l , 7 2 ) of z ( t ) , from

Due to the presence of a mixture of random and deterministic signals, y(t) in (25) is nonstationary as can be seen from its time-varying cumulant cky(t; T ) , which is given by

Y(t).

where c k , represents the cumulant of z ( t ) , IC 2 3. In deriving (26), we have used the independence of w(t) and z ( t ) , the fact that b ( t ) is deterministic, the multilinearity of cumulants, and the insensitivity of cumulants to additive Gaussian noise of unknown covariance [5, ch. 21. Due to this nonstationarity one cannot employ conventional stationary tools to estimate cky and hence to recover Ckl. However, if z ( t ) and v ( t ) are mixing, in the sense of Assumption 1.1, and supt Ib(t)l < B < 03, then y(t) is also mixing; i.e., for y(t) as in (25), Assumption 1.1 reduces to

C S u P l ~ l c k y ( t ; T ) I T t

7

Further

~ T-1

DANDAWATI? AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS

~

221

exists, so that y ( t ) satisfies Assumption 1.2 and hence from Theorem 2.1 and (26) we have that

T-1

Therefore

is a consistent and asymptotically normal estimator of provided b ( t ) is such that the inverse in (29) exists.

Example 3 (Mixed Processes)

s ( t ) and random component v ( t ) , given as Let y(t) be a mixed process with deterministic component

(30)

Let v ( t ) be zero-mean (perhaps correlated) Gaussian noise and assume that

y(t) = s ( t ) + v ( t ) .

1 T-l lim - s ( t ) = 0.

For example, s ( t ) in (30) may represent superposition of deterministic periodic sequences. Our goal is to estimate the third-order moment

T-or, T t = O

. T-1

of s ( t ) from y(t). As in Example 2, y ( t ) is nonstationary and we cannot employ conventional analysis to obtain estimators of 7 r ~ ~ ~ . However, if v ( t ) is mixing in the sense of Assumption 1.1, then

C S U P I ~ l c k i ( t t 7 1 1 = I ~ l % J ( ~ ) I < 00 (31) 7 r

where we have used the fact that cumulants, but not moments of deterministic sequences vanish according to the conventional definition of cumulants [5 , p. 191.

The sample average

1 T-l 7 4 ) ( 7 1 , 7 2 ) = T y(t)y(t + .l)Y(t + 72) (32)

t=O

is a consistent and asymptotically normal estimator of since from Theorem 2.1, using the Gaussian noise insensitivity of cumulants and the zero-mean assumption about s ( t ) , we obtain

Note that (33) is based on a single record of the observed data, which is important since conventionally, availability of multiple records is assumed to obtain consistent estimates of cumulants of mixed processes [38]. Clearly, if s ( t ) represents a superposition of complex exponentials with deterministic phase, then m 3 s ( ~ 1 , T ~ ) 0, unless frequency coupling is present [16], [41]. But the ideas of this example are still useful for single-record estimation based on the fourth-order generalized cumulant ~ 4 ~ ( 7 1 , 7 2 , 7 3 ) which is nonzero [2].

In the problem of harmonic retrieval in linear stationary non- Gaussian noise, one could exploit the fact that m3s(71, 7 2 ) =; 0 for uncoupled sinusoids, and use Theorem 2.1 to show that

t = O

= c3v(71 ,72) . (34)

Using ~ 3 , ( 7 1 , 72 ) , one can estimate the system generating ~ ( t ) and whiten the data to perform statistical tests for detecting sinusoids in non-Gaussian white noise. The frequency-domain counterpart of this approach was originally proposed by [26]. For a more comprehensive account of mixed processes and polyspectra the reader is referred to [41].

In conclusion, we have established mean-square consistency and asymptotic normality of generalized sample averages which show up in different signal processing scenarios, under a certain mild mixing condition. The simplicity of checking the validity of the mixing condition and the wide applicability of our results were illustrated through some examples.

111. ESTIMATION OF CYCLIC MOMENTS AND CUMULANTS

Having presented the asymptotic properties of sample averages of mixed processes, in this section, we apply the analysis of Section 11-A to derive consistent and asymptotically normal single-record estimators for the statistics of the class of Icth- order cyclostationary processes.

We start with some brief background on cyclostationary processes in Section 111-A and present the asymptotic results in Section 111-B.

A. Cyclostutionury Processes

A process is called kth-order cyclostationary if its cumulants up to order Ic exist and are almost periodic functions of time [ 1 11. Such processes occur commonly in communications, rotating machinery, biology, astronomy, meteorology, and economic time-series where almost periodicity occurs either due to natural rhythms or is artificially induced by man- made operations. Due to time-varying statistics, the process at hand is nonstationary and therefore special tools called cyclic- moments and cumulants have to be defined to emulate, in this case, the role played by the time-invariant moments and cumulants in the conventional stationary signal analysis.

Let ~ ( t ) , be a cyclostationary signal, with kth-order moment

7 r Q , ( t ; 7 ) 2 E{z( t ) z ( t + 71) ' . . z ( t + 7k-1)).

Moments of cyclostationary processes accept a Fourier Series


(FS) expansion with respect to t , as follows:

m k z ( t ; T ) = Mkz(a;T)ejat

a € q

- T-1

where the FS coefficients Mk,(a; T ) , are called the cyclic- moments at cycle frequency a and

AT A (0 5 a < 2 ~ : M ~ + ( Q I ; T ) # 0}

denotes the so-called set of cycles. Notice that Mk, has a form similar to the generalized

moments defined in (16); in fact, m k z ( T ) as defined in (16) and Mka: of (35) are identical for Q = 0. Also, from (15) and (35), the similarity between cyclic-moments and sample averages of mixed processes is apparent. Therefore, it is natural to use the analysis of Section 11-A for the cyclostationary case which we do in the next subsection.

We define the kth-order cumulant C k x ( t ; T ) of x ( t ) , as follows: With 70 2 0, let

mv(t; Tv) 2 E{x( t + Tvl). . . x ( t + T,,)}

where v is the set of lags v = { T ~ = ( T ~ ~ , . . . , T , ~ ) } , and

T ) = ( - 1 ) q p - 1)!mu1z(t; Tu1). . .mupz( t ; T u p ) U

(36) where, the summation on v extends over all partitions v = v1 U. . . U vp of { TO, . . . , Tk - I}, with p representing the number of groups in a partition. Analogous to the moments we may write

C k z ( t ; T ) = C k z ( a ; a E d ;

1 T-l Sk,(a; w ) A lim - S k , ( t ; w)e--jat. (39)

Finally, an important relationship exists between C k z and

T-00 T t=O

Mkz given as [ l l l

U a 1 , . . . , a p

x M v , z ( ~ p ; T ~ ~ ) V ( Q I - Q I - . . . - a p ) (40)

where the summation on v extends over all the partitions of {TO = O , T I , . . . , T ~ - ~ } , and q denotes the Kronecker comb (train) function

Va 2 { 1, Q = 0 (mod 2 ~ ) 0, else

A “nonprobabilistic” counterpart of (40) was derived in- dependently by [15], for studying effects of higher order nonlinearities on sinusoids.

Interest in cyclostationary processes has existed since the 1960s (see, e.g., [18] and references therein). They have been studied in several problems including hydrology [39], communications [ 131, radar, and array and multichannel processing

Estimation of the covariance of cyclostationary processes has been considered in [3], [23], the spectra in [21], polyspectra in [7], [ l l ] , and empirical testing for presence of cycle- frequencies were considered in [22]. A “nonprobabilistic” version of kth-order cyclostationarity was introduced inde- pendently in [15], but issues of consistency and asymptotic normality were not addressed (see [20] for criticism of the “nonprobabilistic” approach). Related problems in nonlinear time-series regression were considered in [ 191 and tutorials on the state-of-the-art methods for handling generally nonstationary processes can be found in [29].

Higher than second-order cyclic cumulants appear naturally

[11, [401.

with polynomial phase signals [36], [37]. Here we motivate the need for considering cyclic-cumulants of order 2 3 by considering the identification of mixed-phase tinle-varying MA models with known seasonal variation.

Example 4 (Mixed Phase TV-MA Models with Known Seasonal Variation)

Process of special ‘‘separable” form

(37) 1 T-l

C k z ( a ; T ) = lim - c k z ( t ; T)e-Jat T-ca T

t=O

where Ck,(a; T ) , represents the cyclic cumulant and

/ti A (0 5 Q < 2T: C k , ( a ; T ) # 0).

As with the stationary case, in which moments and cumulants have Fourier Transform (FT) counterparts called moment- spectrum and polyspectrum, respectively, we may define the time-varying moment-spectrum Mkz ( t ; U), and polyspectrum

Let y(t) be an observed signal which is generated as a linear

P

s k z ( t ; w) at “frequency” w A (w1, . . . , ~ k - ~ ) , as y(t) = h(t; n)€( t - n) + v(t) n=O

Mk,(t; w ) = m k , ( t ; 7 ) e - J W T ’ 9

T = b(t)h(n)t(t - n) + .(t) (42)

where b ( t ) is known and characterizes the seasonal variation S k z ( t ; U ) = C k z ( t ; T)e - IWT’ (38) n=O

T

where ’ represents transpose. The spectra h f k , and Skz have corresponding FS expansions with coefficients called cyclic- moment spectra M k z , and cyclic-polyspectra Skz, defined as By letting

in the generally mixed phase impulse response {h(n)}“,o which is driven by skewed i.i.d. variates ~ ( t ) and finally, w(t) represents stationary additive noise that is independent of t(t).

- T-1 Q

:r(t) 4 h(n)t(t - n) n = O

DANDAWATE AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS 223

it follows from the definition of cumulants and (42) that

C Q y ( t ; 7 1 , 7 2 ) = b ( t ) b ( t + T l ) b ( t + 7 2 2 ) C 3 z ( 7 1 : 7 2 ) + c 3 , ( 7 1 , 7 2 )

(43) from which we see that the third-order cyclic-cumulant of y ( t )

assuming - T-1

exists, is given as

c 3 y ( a ; 7 1 ; 7 2 ) = C3X(71,Q2)

~ T-1

where ~ T - I

is the Kronecker delta train with period 27r.

It is clear from (44) that

a # 0 (mod 27r) (45)

so by knowing b ( t ) , the third-order cumulant of z ( t ) can be obtained from C3y ( a ; 7 1 , 7 2 ) regardless of w ( t ) provided that a # 0 (mod 27r). Of course a must be such that

By knowing b ( t ) , the identification of the TV-MA system h( t ;n) = b(t)h(n) will be complete if we can recover {h(n)}:=o from ~ 3 ~ ( ~ 1 , ~ 2 ) . Using the multilinearity of cumulants, [5 , p. 191 and the i.i.d. nature of c ( t ) it follows that

4

C 3 , ( 7 1 , 7 2 ) = Y3t h(n)h(n + 7 1 ) h ( n + 7 2 ) (46) n=O

where ~3~ is the third-order cumulant of ~ ( t ) . It is easy to see from (46) that h(71) can be obtained from ~ 3 ~ ( 7 1 , 7 2 ) using

This however, is generally not possible using the cyclic- correlation of y ( t ) , since it yields the covariance of z ( t )

4

czz(7) = Y2F h(n)h(n + 7 ) (48) n = O

which is not uniquely related to h(n). In fact, the spectrum (the Fourier transform of c z X ) , is zero-phase regardless of the phase of h(7) and therefore is generally incapable of recovering

completely3 the phase of h(7). The third-order approach fails when t ( t ) is nonskewed, as with binary phase-shift-keyed communication signals, since ~3~ becomes zero. In this case a fourth-order counterpart of (46) has to be used to recover the TV-MA impulse response. This demonstrates importance of the third- and fourth-order cumulants at least in the case of identification of mixed-phase time-varying linear systems.

Notice that, at least theoretically, it is possible to recover h(71) from ~ ~ ~ ( 7 ~ ~ 7 ~ ) regardless of the presence of v ( t ) as long as it is stationary. This property of the cyclic-domain along with the phase-preserving nature of the higher order statistics motivates a joint exploitation of cyclostationary and higher order statistics via cyclic-cumulants. Cyclic-cumulants can be shown to be at least theoretically insensitive to generally cyclostationary Gaussian noise of (unknown) spectral characteristics-a property inherited from the Gaussian-noise insensitivity of higher order statistics. Some results on cyclic- cumulant-based identification, detection, and classification can be found in [7].

For implementation and performance evaluation of schemes based on cyclic-cumulants such as (46), it is necessary to develop consistent sample cyclic-cumulants and derive their asymptotic distribution which we do next using the results of Section 11.

B. Estimators and Asymptotic Results

Our goal is to develop consistent estimators for mkz, C k z ,

Mkx , and C k z . we first derive estimators for Mkz and m k x

and use them to define estimators for c k z and Ckz via (40) and (37). Let

be our estimator for M kz ( a ; 7). The following result presents asymptotic properties of ME).

Theorem 3.1: Let ~ ( t ) be a Icth-order cyclostationary process which satisfies Assumption 1.1 with zn,l(t) E { z ( t ) , z* ( t ) ) . If Mkz exists then, M g ' ( a ; T ) of (49) converges in the mean-square sense, i.e.,

lirn M E ) ( a ; T) ,gs. M k z ( a ; T ) . (50) T-CC

In addition,

f i [ M L ' ( Q ; T ) - Mkz(O!;T)]

is asymptotically complex-normal (see also Appendix I). More generally, with zn,l(t) E { z ( t ) , z*( t ) }

7, A (7n,0,'..,7n,Kn) and

"part of the phase can be recovered from the magnitude via Hilbert relations.

224 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 41, NO. 1 , JANUARY 1995

for any arbitrary but fixed orders K,, and lags T,, n = 0 , . . . , m, even when any of the Mgi(a ; 7,) are conjugated. Further, let

K , K ,

f m ( t ; T m ) 4 n Z , , d ( t + T ~ , l ) f , ( t ; T , ) I - I Z n , l ( t + T n , l ) 1=O 1=0

and the cyclic cross-spectra of f be defined as

The conclusion of Theorem 3.1 implies that f i E ) as defined in (49) is a consistent and asymptotically normal estimator of the cyclic moment M k x ( T ) for arbitrary orders k and lags 7. In particular, it follows from the properties of almost periodic functions [4, eq. (3), p. 141 applied to the deterministic sequence f l E [ M g l ( a ; 7,) - Mr!(a; T,) ]

that

(57)

This, along with the conclusions of Theorem 3.1, implies that f i [ M g ' ( a ; 7,) - Mg:(a; Tn)l is asymptotically normal with zero-mean and covariances given by (55) and (56).

From (35) it seems natural to estimate the time-varying kth-order moment of ~ ( t ) as

(58)

lim \/;i;~[MjTn'(a; 7,) - MiT,I(a; Ta)l = 0. T-CC

m(T) k x ( t ; 7 ) = M p ( a ; T ) e j ?

&AT

Now, since is a linear combination of ME' [cf. (%)I its consistency and asymptotic normality follows readily from that of ME) (cf. Theorem 3.1), provided that Ap has ajni te number of elements. The asymptotic covariances of f i E ) are then given from

T-00 lim T cum { rjLE)(t; T ) , hiy)(t; p ) }

The set dp has a finite number of cycles if, for example, mkx is periodic, or, almost-periodic with a finite number of cycles (e.g., a finite sum of periodic functions with perhaps incommensurate periods).

Next, to estimate Ckxr (40) suggests the following estimator:

Y el, ... , Q p

. . . M ~ ~ ~ ( a p ; r Y p ) v ( a - a1 - . . . - a,) (60)

which is a nonlinear combination of ME) and hence special care has to be taken to deduce the asymptotic properties of CL:) from those of ME'. The following theorem states the asymptotic properties of CK) in (60):

Theorem 3.2: Under the hypotheses of Theorem 3.1, if Ap has a finite number of cycles then CL:)(.; T ) as defined in (60) converges in the mean-square sense, i.e.,

In addition, f l [CE)(a; r ) - C k x ( a ; T ) ] is asymptotically complex-normal (see also Appendix I). More generally, under the hypotheses of the theorem

^(T) ^(TI cum { ' k O z ( aO; T O ) , ' ' ' > Ckmac (am; 7,)) = O(T-") (62)

even when any of the CiTL(a; T , ~ ) are conjugated. Further

= C(-l)("+"-"(;O - l)!(q - l)!

DANDAWATE AND GIANNAKIS ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS 225

and

Using the arguments analogous to the ones made after Theorem 3.1 we conclude from Theorem 3.2, that CE) is mean-square-consistent and asymptotically complex-normal. As in the case of the cyclic- and time-varying moments, we define the estimator of the time-varying cumulant as [see (37)]

Consistency and asymptotic normality of follows readily from that of CL:) (cf. Theorem 3.2), provided that Ai has a finite number of elements. The asymptotic covariance of EE) is then given as

where the limiting covariance inside the sums are given by (63) and (64).

C. Computation of Covariances

In Section 111-B, we developed cyclic-moment and cumulant . .

estimators, proved their consistency and asymptotic normality, and provided asymptotic covariance expressions. Such an analysis will be useful in implementing and evaluating schemes based on the cyclic statistics. In particular, it will be possible to derive variance bounds and confidence intervals for the cyclic-statistics-based estimators. In this regard, one question remains to be answered: how can we estimate the covariance of cyclic-statistics in practice from finite data sets of length T? We address this issue in the current section.

( a ; 7, ) as defined in ( 5 l), its covariances are given by (55) and (56). Recall also, that S Z ~ is the cross-cyclic-spectrum of the f processes defined

Recall from Theorem 3.1 that with

in (53) and (54), and therefore the required covariances of (55) and (56) can be estimated via cyclic-spectrum estimators, which in turn will yield the estimators for the covariance of cyclic-moments and cumulants to be seen later. We will use the results of [ 111 to develop estimators for the covariances and summarize the steps involved in the following:

Step 1: For a given process x ( t ) and set of lags T , and r,, compute

K ,

f m ( t ; r m ) 2 n xm,l(t + 7 m J ) 1 =o

Step 2: Form the cross-cyclic-periodograms as

and

which can be computed using the FFT.

using a spectral window W ( s ) as Step 3: Compute the smoothed cross-periodogram estimate

and

\ I ,

U- U A X ,

Step 4: The required covariances may be evaluated approximately using (55) and (56) as

(73)

For estimating the second-order cyclic-cumulant spectra of the process f in (70) and (71), we need to suppress the


frequencies that are cycles of the mean of f , analogous to the stationary case (see [5, p. 1421). Because the mean of f m is the K,th-order moment of x ( t ) (see the definition of f in Step l), the cycles of the mean of f ’ s are the precisely sets Am (see (35)), which we avoid in (70) and (71). For more details we refer the reader to [ 1 I].

In practice, it is usually sufficient to consider kth-order cyclic statistics for IC 5 4, further, for zero-mean processes it follows from (40) that

(74) (75)

C 2 z ( a ; T) = M 2 z ( a ; 7 )

CQz (a; 71 , 72) = MQX (a; 71 > 7 2 )

and

C 4 z ( a ; T 1 , 7 2 r 7 3 ) = M 4 z ( Q ; 7 1 , 7 2 , 7 3 )

- { M 2 x ( a - p; 7 1 ) M 2 z ( P ; 73 - 72)eJpT2

a a , - + M 2 x ( a - P; 72)M2x( /3; 71 - 73)ejpT3

+ MzZ(a - & 7 3 ) M 2 x ( P ; T 2 - 71)eJPT1}.(76)

NOW, since for Tn,o = 0, M L T ) ( a ; T n ) = MLT)(a! ;T) , the steps outlined previously in this section can be used for computing the covariances of the kth-order cyclic-moment estimators as well as the cyclic-cumulant estimators (for IC = 2,3) due to (74) and (75). Further, for a! AY it follows from (76) that

(77)

so that the covariance of the fourth-order cyclic-cumulant can also be computed using the outlined steps at least for this case. The general covariance expression for the fourth-order cyclic-cumulant estimate is derived in Appendix 111.

Although the results in this section were established for autocumulants, the estimators and asymptotic analysis carry over for cross-cumulants as well. More importantly, the estimators of this section can also be applied with appropriate modification to estimation of statistics of nonstationary processes whose moments or cumulants accept a basis expansion with known basis functions (not necessarily exponential) of the form (35) and (37), respectively. This extends the autocorrelation estimators for nonstationary processes (discussed in [ 131 and references therein), to higher order (cyclic) moments and cumulants and detailed derivations will be reported in the future.

c 4 x (a; 71 , 72 3 73) = M 4 z ( a ; 7 1 , 7 2 , 73) 1

Iv . PROOFS

A. Proof of Theorem 2.1

[ 5 , p. 191, to obtain We start with (8) and use the multilinearity of cumulants,

1 =o P

f m ( t , 7,)

Substituting to = t , t l - t o = (1, . . . , t , - t o = tm in (78)

and with ,$ A ((1, . . . E m ) , we obtain

where

and

where the summation on t , contains O ( T ) number of terms.

Using i) of Lemma 2.1 in (80) we infer

cum { M ~ ) ( T ~ ) , . . . , ME: ( T ~ ) } = O(T-m). (81)

The relation (81) holds even when any of the M F : ( T ~ ) ’ s are

conjugated. To find the covariances consider

T cum { ME: ( T ~ ) , fir: (7.)) K ,

Substituting tl = t and t2 - t l = E in (82) and with

t , 4 -min (0, E } , and t p T - 1 - max (0, E } , we obtain

Tcum { M ~ ! ( T ~ ) , M F ! ( T ~ ) }

DANDAWATG AND GIANNAKIS: ASYM€TOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS

~

221

Writing t p T-1 tu-1 T-1 c=c-e- c

t=t, t=O t=O t=t,+l

in (83) we obtain

We treat each of the three summations on t in the bracketed

expression of (84) separately. Consider first, the term

1 T- l T- l T1= -

T cum { f m ( t ; 7 m ) , f n ( t + I ; 7 n ) I E=-(T-1) t=O

fn(t + I ; T ~ ) } + O(T- ' ) (85) 1 where the last equality in (85) follows from ii) of Lemma 2.1.

Hence from (85) and (10)

Next, consider the term

~ T-1 t , -1

Now using the fact that t , 5 111 and i) of Lemma 2.1 we

observe that

1 " T2 F T s ~ P I I E l l C U m ~ ~ m ( ~ ; ~ m ) , f n ( ~ + ~ ; ~ ~ ) ~ l l

E = - 30

= O(T-l) (88)

from which it follows that

lirn T2 = 0. (89) T-w

Similarly one can show that T-1 T-1

T 3 = - c u m { f m ( t ; 7 m ) , f , ( t + ~ ; 7 , ) } E=-(T-l) t=tp+l

T

= O(T-1) . (90)

Hence from (86), (89), (90), and (84), it follows that

T-03 h i Tcum{M~' , ( . r , , ) ,M~i( . r , )} = S2f(7,,7,) (91)

and following the same steps once again we conclude that

completing the proof.

B. Proof of Theorem 3.1

Let zo(t) = z ( t ) , . . . , X k - l ( t ) = x( t + ~ k - l ) , and z k ( t ) = e-Jat. Any joint cumulant of ( ~ ~ ( t ) } k = ~ involving the deterministic z k ( t ) vanishes by properties of cumulants [5 , p. 191, and (zm(t)}&=O trivially satisfy Assumption 1.1. Further, since all cumulants of z ( t ) are absolutely summable, {zm (t)}F,do satisfy Assumption 1.1. Hence by Theorem 2.1 and Corollary 2.1, d ? [ M g ) ( a ; ~ ) - M k Z ( a ; 7 ) ] are jointly normal with respect to I C , Q , and T ,

lini M K ) ( a ; T ) m ~ s M ~ ~ ( ~ ; T ) (93) T-33

and (52) holds. To determine the covariance of M g ' ( a : ~ ) , use (49) and the multilinearity of cumulants to obtain,

Tcum {fig', (a; T m ) , M g ! ( p : .,I} 1 T - l

= - cum{fm(tl:7,,), f T L ( t 2 ; 7 , ) } e - 3 a t l e - 3 ~ t z .

t1,t,=O

(94) Substituting t l = t and t 2 - t l = [, in (94) and with t , 2 -min (0, t}, and t o a T - 1 - max (0, I } , we obtain

Writing T-1 tu-1 T-1

t=tu t=O t=O t=t,j+l

in (95) we obtain

Tcum { ~ ~ ! ( a ; 7 n ~ ) , M ~ ) ( ~ ; 7 n ) }

Consider first, the term

1 T - l T 1 = -

T <=-(T-l)


(99) Now using the fact that t , I 151 and i) of Lemma 2.1 we observe that

1 " T2 L T I~llcum{f,(t;7,),fn(t+~;7,)}l

E=- 00

= O(T-1) ( 100)

from which it follows that

lim T2 = 0. T-CC

which after using the Leonov-Shiryaev formula (see Appendix I), yields

(TI E{ M x u l z ( a l ; T ~ ~ ) . . . ML;L(a,; T ~ ~ ) } = C,; . . . Cu;

U'

(106) where the summation extends over all partitions (vi . . . v;) of {VI, . . . , up} and C,, denotes the joint cumulant of the M ( T ) ' ~ with subscripts in U'. Now since joint cumulants of fit:) of all orders 2 2 vanish asymptotically (cf. Theorem 3.1), we have from (106) that

T+lX lim ~ { M ~ ~ i ( a , ; 7 , , ~ ) . . . Mi:2(aP; T u p ) }

= T-CC lim ~ { ~ i ~ . ( a l ; 7 ~ ~ ) } . . . T-00 lim E { M ~ ~ ~ ( ~ ~ ; ~ ~ ~ ) }

= Mu1+(a1;7ul) ' . . . M u , z ( a p ; 7 u p ) (107)

where the last equality in (107) follows from the unbiasedness of M ( T ) ' ~ (cf. Theorem 3.1). From (107), (105), and (40)

T-CC lim ~ { C c ) ( a ; 7 ) } = cICZ(a;7) (108)

proving unbiasedness of Ckz (a; 7). Asymptotic normality: To prove consistency and asymp-

totic normality consider a cumulant of order m 2 2 given from (60) as

Similarly, one can show that

T 3 = - T

where the constant K, E,,, and Ea are defined appropriately in terms of the summations over Q and v, and constants that show up in (60). Using again the Leonov-Shiryaev formula

1 T-l T-l cum {fm(t; T,), fn(t + E ; 7,))

<=-(T-l) t=ta+l we may write

U"

where v" represents all the indecomposable partitions (see Appendix I) of

(103) - - SZf7,,7, ( a + P ; P>

and similarly the following holds:

( T ) G?o . . . MUpnkO lim Tcum { MEI, (a ; T,), M$:)(P; Tn), }

T'" ."

( a - P ; -P) (104) (1 11) - (*I - SZfT,, ,T,

.. (i., . * ($1 completing the proof. Mu1 I C , . * . Mup, IC,

C. Proof of Theorem 3.2 and Cu,, denote the cumulant of A@) with subscripts in v". Now if two rows are hooked, the group that hooks the two rows contains at least one element from each row which yields (due to (52)) a cumulant of order O(T- l ) . Now since all the m + l rows in an indecomposable partition must communicate this yields a cumulant (or product of cumulants) that is (are) at most O(T-,) for all partitions v" = (v?, . . . , vi) and hence,

Unbiasedness: From (60) we have that

E{CK)(a;*)} - - x(- l ) (p- l ) (p - '1'

. . . f ig: (ap ; Tup ) }

E { M % ( a l ; T u l )

(Io5)

U LYl'..OP

229 DANDAWATB AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS

from (1 10)

1=1

= O(T-").

Relationship (1 12) holds even when some of the components of the cumulant are conjugated which establishes (62) and hence mean-square consistency and asymptotic normality of

To determine the covariance consider, (from (60) and mul-

C ( T ) > S .

tilinearity of cumulants)

T cum { CKL ( a ; 7% 1 1 er; ( P ; 7,)) - - X(- l ) (P+q-Z)(p - l ) ! ( q - l ) !

I I "

. T cov { i q h 1 ; Tp1 ) . . . M K h p ; T/lp 1 1

*1 . .4 Jp 41..dJq

ML:L($l; T u 1 ) . ' . MEl(cbq; T J } . (113)

Now

where the summation zB is over all indecomposable partitions of

and CO are as defined in (1 10). Using the consistency of M ( T ) and the result from Theorem 3.1 that the cumulants of order n of M ( T ) are 0(Tpn+l), we obtain from (114) that

U 4 P

m l = l m z f l 2

mz=l m z f 'z

Similarly, it follows that

I I "

which completes the proof.

V. CONCLUSION

We proved that sample averages of arbitrary orders are asymptotically jointly normal and converge in the mean-square sense to their expectations. We also provided the asymptotic covariance expressions. Our results treated deterministic and random signals on a common framework, which is of interest in several examples of practical importance. The asymptotic analysis also enabled us to define and analyze (generalized) moments and cumulants for mixed processes based on a single record. Lastly, we developed consistent and asymptotically normal single-record estimators for cyclic- and time-varying moments and cumulants of cyclostationary signals. The relevant asymptotic covariance expressions and their computable forms were also provided. Throughout our analysis we em- ployed the mixing conditions based on absolute cumulant summability. It will be interesting to carry out the present analysis under alternative mixing conditions such as a-mixing.


APPENDIX I CUMULANTS AND COMPLEX NORMALITY: BASIC RESULTS

Two complex random variables (r.v.) 2 = X + j y and < = E + j 4 , where j 2 = -1, are said to be jointly normal if the real r.v. X, Y, E , 4 are jointly normal [5, pp. 89-90], [33, pp. 118-1231), [28, pp. 152-1531. The mean of the vector [X, Y, (, 41 can be obtained from the means of 2 and 5 while the cross-covariances can be found as

In this paper, we provide sufficient information for computing the means of 2 , C and the cumulants cum ( 2 , Z}, cum {Z,Z*}, cum {C, <}, cum {C, C*}, cum{Z,C}, and cum{2,<*}. It is understood that the joint probabilistic properties of 2 and ( are completely charac- terized by a 4-variate normal distribution of [X, y , E , 41.

The approach we have adopted to prove asymptotic complex normality in Theorems 2.1-3.2 can be found, for example, in the well-known works [6], [5], and [34] albeit in the context of (poly-) spectral estimators. According to this elegant approach, random variables say y1, . . . , y k are asymptotically jointly r"al if all the joint cumulants of y1, . . . , y k of all orders 23 (assuming they exist) vanish asymptotically (see [5, Lemma P4.5 and Theorem 4.4.1 pp. 403-4041).

Now, let

2,=x,+jyn, n = l , . . . , k ( 120)

be k-complex-valued random variables with real part X,, and imaginary part Y,. We say that {2n}k=1 are asymptotically jointly normal if all the joint cumulants of {X,, Y,, 72 = 1, . . . , IC} of all orders 2 3 vanish asymptotically. Now

2, 2: X,=-+- 2 2

and

2 2 Using the multilinearity of cumulants [5, p. 191, we can express the joint cumulants of {X,, y,, n = 1, . . . , I % } in terms of the joint cumulants of {2,, 2:, n = 1, . . . , IC}. For example

{ 21 ; 2;, 2 2 - 2; cum{X1,Y2} =cum ~ ~

2 1 2

= - [cum{21,22> -cum{21,2,*}

+ c ~ m { 2 ; , 2 ~ } -cum(2;,2,*}].

Thus to prove that the joint cumulants of {X,, y, , n = 1,. . . , IC} vanish asymptotically it is sufficient to show that the joint cumulants of {2,, 2:) n = 1, . . . , IC} vanish asymp-

totically, which in fact proves that {2n, n = 1,. . . , I C } are asymptotically jointly normal.

In our proofs we will frequently make use of the Leonov-Shiryaev identity [25], and the indecomposable partitions. For the readers convenience we include here both the concepts (as presented in [5 pp. 20-211) without proof.

Indecomposable partitions t.5, p . 201: Consider the following table

(1,1) . . ' '(LK1)

( M , 1) . ' . ( M , Kn4) (121)

and a partition PI U Pz U . . . U PM of its entries. We shall say that sets P,, , P,,, , of the partition, hook if there exist (m1,ICl) E P,, and ( m z , k 2 ) E P,!, such that ml = m2.

We shall say that the sets P,! and P,f/ communicate if there exists a sequence of sets Pml = P,, , P,, , . . . , P,, = PTnIt such that P,, and P,,+, hook for n = I, 2 , . . . , N - 1. A partition is said to be indecomposable if all sets communicate.

Leonov-Shilyaev identity [S p . 211: Consider a two-way array of random variables X m k , k = 1, . . . , K,;m = 1, . . . , M . Consider the M random variables

The joint cumulant cum {Yl , . . . , Y M } is then given by

cum{X,k,mk E vl}...cum{X,k,mlc E up} (123)

where the summation is over all indecomposable partitions v = v1 U . . . U vp of the table in (121).

v

APPENDIX I1 PROOF OF LEMMA 2.1

i) Let YO,^ zo,l(t + TO,^), and Y,,l z,,l(t + <, + T,J), n E { 1, . . . , m}. Using the definition of the product processes f , , and the properties of cumulants [5, ch. 21 we may write

cum { f 0 ( t ; 7 0 ) , fl ( t + <I ; 71) 1 ' ' ' fm ( t + <m ; 7,))

= cum { ~ , , l : (n , 1 ) E v l > v

. . . cum {Y,,J: (n , I ) E vp} (124)

where the summation on v is over all the indecomposable partitions of the following table:

From (124) we may write

23 I DANDAWATl? AND GIANNAKIS: ASYMPTOTIC THEORY OF TIME AVERAGES AND KTH-ORDER STATISTICS

Now each of cum{Y&: (n , l ) E U} is a joint cumulant of { ~ ~ ( t ) } & = ~ , and since Assumption 1 . 1 holds, the right-hand side of the inequality in (126) is finite, and hence

U = 2 ==+ rul = 7 2 rvz = 7 3 ru3 = 71

U = 3 j 7ul = 7 3 T~~ = 7 1 T,,, = 7 2 . (131)

where, the last equality in (129) follows upon using i). Hence from (129) and (128), ii) follows. Following the same steps it

of the f ’ s are conjugated.

p = l P E A T

can be shown that conclusions i) and ii) hold even when any . M (Pi P l l s - Pl lz 1 + %7, , p * 3 - p w 2 ( a + P ; P ) . M (11 - P ; ppl )] e j P P w z

232 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 41, NO. I , JANUARY 1995

It similarly follows that

REFERENCES

J. C. Allen and S. L. Hobbs, “Detecting target motion by frequency- plane smoothing,” in Proc. 26th Asilomar Con$ on Signals, Systems and Computers (Pacific Grove, CA, Oct. 26-28, 1992), pp. 1042-1047. J. Anderson and G. B. Giannakis, “HOS-based harmonic retrieval: A deterministic formulation,” in 6th SP Workshop on Statistical Signal and Array Processing (Victoria, BC, Canada, Oct. 7-9, 1992), pp. 128-131. P. L. Anderson and A. V. Vecchia, “Asymptotic results for periodic autoregressive moving-average processes,” J. Time Ser. Anal., vol. 14, pp. 1-18, Jan. 1993. A. S. Besicovitch, Almost Periodic Functions. New York: Dover, 1954. D. R. Brillinger Time Series: Data Analysis and Theory. San Francisco, CA: Holden Day, 1981. D. R. Brillinger and M. Rosenblatt, “Asymptotic theory of estimates of k-th order spectra,” in Spectral Analysis of Time Series, B. Harris, Ed. New York: Wiley, 1967, pp. 153-188. A. V. DandawatC, “Exploiting cyclostationarity and higher-order statistics in signal processing,” Ph.D. dissertation, Univ. of Virginia, 1993. A. V. DandawatC and G. B. Giannakis, “Ergodic results for nonstationary processes: cumulants, ambiguity functions and wavelets,” in Proc. 25th Conf on Information Sciences and Systems, 1991, pp. 976-983. - ,“Non-parametric cyclic polyspectral analysis of AM signals and processes with missing observations,” IEEE Trans. Inform. Theory, vol. 39, no. 6, pp. 1864-1876, Nov. 1993. __ , “Differential delay-Doppler estimation using second and higher-

order ambiguity functions,” Proc. Inst. Elec. Eng.--Pt. F, Radar Sig. Process., vol. 140, no. 6, pp. 410418, Dec. 1993. __ , “Nonparametric polyspectral estimators for kth-order (almost) cyclostationary processes,” IEEE Trans. Inform. Theory, vol. 40, no. I , pp. 67-84, Jan. 1994. -, “Statistical tests for presence of cyclostationarity,” IEEE Trans. Signal Process., pp. 2355-2369, Sept. 1994. W. A. Gardner, Statistical Spectral Analysis: A Nonprobabilistic Theory. Englewood Cliffs, NJ: Prentice-Hall. -, “Correlation estimation and time-series modeling for nonstationary processes,” Signal Process., vol. 15, pp. 31-41. 1988. W. A. Gardner and C. M. Spooner, “Higher-order cyclostationarity, cyclic cumulants and cyclic polyspectra,” in Inr. Symp. on Information Theory and Applications, Hawaii, 1990. G. B. Giannakis and G. Zhou, “Retrieval of random amplitude modulated harmonics using cyclic statistics,” in Proc. 27th Con6 on Informa- tion Science and Systems (Baltimore, MD, The Johns Hopkins Univ., Mar. 1993), pp. 650-655. -, “On amplitude modulated time-series, higher-order statistics and cyclostationarity,” in Higher-Order Stat. Signal Processing and Applications, E. Powers, B. Boashash, and A. Zoubir Eds. New York: Longman Chesire, 1994. E. G. GladySev, “Periodically correlated random sequences,” Sov. Math.,

E. J. Hannan, “Non-linear time-series regression,” J. Appl. Probab., vol.

M. J . Hinich, book review of Statistical Spectral Analysis: A Nonprob- abilistic Theory, in SIAM Rev., vol. 33, pp. 677-678, Dec. 1991. H. L. Hurd, “Nonparametric time series analysis of periodically correlated processes,” IEEE Trans. Inform. Theory, vol. 35, pp. 350-359, Mar. 1989. H. L. Hurd and N. L. Gerr, “Graphical methods for determining the presence of periodic correlation,” J. Time Ser. Anal., vol. 12, no. 4, pp. 337-350, 1991. H. L. Hurd and J. Leskow, “Strongly consistent and asymptotically normal estimation for the covariance for almost periodically correlated processes,” Statist. Decisions, vol. 10, pp. 201-225, 1992. P. T. Kim, “Consistent estimation of the fourth-order cumulant spectra density,” J. Time Ser. Anal., vol. 12, no. 1, pp. 63-71, 1991. V. P. Leonov and A. N. Shiryaev, “On a method of calculation of semi-invariants,” Theor. Prob. Appl., vol. 4, pp. 319-329, 1959. K-S. Lii and T-H. Tsou, “Detecting sinusoids in non-Gaussian noise,’’ 1. TimeSer. Anal., pp. 391-410, 1992. L. Ljung, System Identification: Theory,for the User. Englewood Cliffs, NJ: Prentice-Hall, 1987. M. LoCve, Probability Theory. Princeton NJ: D. Van Nostrand, 1963. C. Longo and B. Picinbono, Eds., Time and Frequency Representation of Signals. M. Pagano, “On periodic and multiple autoregressions,” Annals Statist., vol. 6, pp. 1310-1317, 1978. E. Parzen, “Spectral analysis of asymptotically stationary time series,” Bull. Inst. Int. Statist., vol. 39, pp. 87-103, 1961. -, “On spectral analysis with missing observations and amplitude modulation,” SANKHY-x: Indian J. Statist.: Ser. A , pp. 383-392, 1963. B. Picinbono, Random Signals and Systems. Englewood Cliffs, NJ: Prentice-Hall, 1993. M. Rosenblatt, Stationary Sequences and Random Fields. Birkhluser, 1985. M. Rosenblatt and J. W. Van Ness, “Estimation of the bispectrum,” Annals Marh. Statist., vol. 36, pp. 11 20-1 136, 1965. S. Shamsunder, “Array and multichannel signal processing using higher- order and cyclic statistics,” Ph.D. dissertation, Univ. of Virginia, Jan. 1994. S. Shamsunder, G. B. Giannakis, and B. Friedlander, “Estimating random amplitude polynomial phase signals: a cyclostationary approach,” IEEE Trans. Signal Process., Feb. 1995 (to appear). A. Swami and J. M. Mendel, “Cumulant-based approach to the harmonic retrieval and related problems,” IEEE Trans. Signal Process., vol. 39, pp. 1099-1 109, 1991. A. V. Vecchia, “Periodic autoregressive-moving average (PARMA) modeling with applications to water resources,” Water Resources Bull., vol. 21, pp. 721-730, Oct. 1985. G. Xu and T. Kailath, “Direction-of-arrival estimation via exploitation of cyclostationarity -A combination of temporal and spatial processing,” IEEE Trcms. Signal Process., vol. 40, July 1992. G. Zhou and G. B. Giannakis, “Fourier series moment polyspectra for mixed processes and coupled harmonics,” IEEE Trans. It$orm. Theory, Dec. 1993 (submitted).

vol. 2, pp. 385-388, 1961.

8, pp. 767-780, 1971.

New York: Springer-Verlag, 1989.

Documents

Asymptotic theory of mixed time averages and kth-order cyclic-moment and cumulant statistics