Improved nonparametric inference for multiple correlated

StatThe ISI’s Journal for the Rapid (wileyonlinelibrary.com) DOI: 10.1002/sta4.28Dissemination of Statistics Research

Improved nonparametric inference for multiplecorrelated periodic sequencesYing Suna,�, Jeffrey D. Hartb and Marc G. Gentonc

Received 14 July 2013; Accepted 21 July 2013

This paper proposes a cross-validation method for estimating the period as well as the values of multiple correlatedperiodic sequences when data are observed at evenly spaced time points. The period of interest is estimated condi-tional on the other correlated sequences. An alternative method for period estimation based on Akaike’s informationcriterion is also discussed. The improvement of the period estimation performance is investigated both theoreticallyand by simulation. We apply the multivariate cross-validation method to the temperature data obtained from mul-tiple ice cores, investigating the periodicity of the El Niño effect. Our methodology is also illustrated by estimatingpatients’ cardiac cycle from different physiological signals, including arterial blood pressure, electrocardiography,and fingertip plethysmograph. Copyright © 2013 John Wiley & Sons, Ltd.

Keywords: cross-validation; model selection; multiple sequences; nonparametric estimation; period

1 IntroductionA number of nonparametric methods for period estimation in univariate time series have been proposed recently,including Hall et al. (2000), Hall & Yin (2003), Hall & Li (2006), Genton & Hall (2007), and Hall (2008). In thesepapers, to estimate the period of a periodic function, an appropriate random spacing of time points is needed in orderto ensure consistency; in other words, unevenly spaced observations are needed. Considering that equally spaced dataare prevalent in time-series analysis, Sun et al. (2012) proposed a cross-validation (CV) based nonparametric methodfor evaluating periodicity when time points are evenly spaced and a periodic sequence is observed from the model

Yt D �t C "t, t D 1, : : : , n,

where Y1, : : : , Yn are the observations; �1, : : : ,�n are unknown periodic constants; and the errors "1, : : : , "n areindependent and identically distributed (i.i.d.), and mean-zero random variables.

To make statistical inference on periodic sequences, the traditional approach is to estimate the values of the periodicsequence via parametric models, for example, trigonometric regression, where the estimated periodic component isa linear combination of a finite number of unknown regressors. These unknown parameters can be estimated byfrequency domain methods based on the periodogram, such as in Walker (1971), Rice & Rosenblatt (1988), andQuinn & Thomson (1991). Moreover, Li (2012) proposed methods for detecting periodicity by using the quantileperiodogram when the noise is asymmetric. Another direction of statistical inference for the periodic behavior in thefrequency domain is periodicity testing (Canova, 1996). Typically, one says that a sequence exhibits periodicity if its

aDepartment of Statistics, University of Chicago, Chicago, IL 60637, USAbDepartment of Statistics, Texas A&M University, College Station, TX 77843, USAcCEMSE Division, King Abdullah University of Science and Technology, Thuwal 23955-6900, Saudi Arabia∗Email: [email protected]

Stat 2013; 2: 197–210 Copyright © 2013 John Wiley & Sons Ltd

Y. Sun et al. Stat(wileyonlinelibrary.com) DOI: 10.1002/sta4.28 The ISI’s Journal for the Rapid

Dissemination of Statistics Research

spectral density has a peak or a large mass at a certain frequency. Then a test is developed to determine whetherthe cycle is significant by comparing the filtered variance with respect to the frequency to the variance of the originalseries. As commented by Sun et al. (2012), although these frequency domain methods are suitable for identifyingsinusoidal functions that are highly correlated with the data, they are not appropriate for the problem of estimatingthe smallest period of a periodic function. The detailed discussion on the advantage of the CV method over the Quinnand Thomson periodogram-based method can be found in Sun et al. (2012). Furthermore, when little is known aboutthe structure of such periodic sequences, it is important to have nonparametric methods for period estimation.

In many real-world problems, multivariate time series are available. For instance, several different variables aresimultaneously recorded from a system under study, such as atmospheric temperature, pressure, and humidity inmeteorology, or heart rate, blood pressure, and respiration in physiology. A multivariate time series could also berecorded from one variable but in spatially extended systems, such as in studies of meteorology, where temperaturerecordings are obtained from probes or satellites at different spatial locations. In this paper, we use a similar frameworkas in Sun et al. (2012) to develop nonparametric methods for period estimation when multiple correlated periodicsequences are observed at evenly spaced time points. The basic idea is to borrow information from other correlatedsequences to improve the period estimation of interest.

One motivation for our methodology is the study of periodicity of El Niño effects, which are defined as sustainedincreases of at least 0.5 ıC in average sea surface temperatures over the east-central tropical Pacific Ocean. TheEl Niño phenomenon dramatically affects the weather in many parts of the world. It is therefore important to betterunderstand its appearance. Various climate models and statistical models attempt to predict El Niño as a part ofinterannual climate variability. Sun et al. (2012) have applied their method to a series of sea surface temperaturesto estimate the El Niño period. The El Niño Southern Oscillation (ENSO) is in the tropical Pacific Ocean area. In thispaper, we investigate whether the ENSO can be detected in the North Atlantic region by analyzing temperature datafrom different ice cores in Greenland.

Let d be the number of sequences. We consider the following model:

Yt D �t C "t, t D 1, : : : , n, (1)

where Yt D .Y1,t, : : : , Yd,t/T D .Xt, ZT

t /T is the observed vector with Xt D Y1,t and Zt D .Y2,t, : : : , Yd,t/

T, �t D

.�1,t, : : : ,�d,t/T is an unknown constant vector, and the error vectors "t D ."1,t, : : : , "d,t/

T, t D 1, : : : , n, are i.i.d. withmean zero, and covariance matrix † that can be partitioned into

† D

��21 � T

12

� 12 †22

�,

with �21 D Var.Xt/,†22 D Var.Zt/ and � 12 D Cov.Xt, Zt/. It is assumed that �s,i D �s,iCmps for i D 1, : : : , ps, m D1, 2, : : :, s D 1, : : : , d, and integers p1, : : : , pd, each of which is at least 2. Of interest is estimating p1, the period ofthe first sequence X1, : : : , Xn.

Our method of estimating the period of interest is also based on CV. The CV estimator proposed by Sun et al. (2012)for one periodic sequence evaluates a candidate period q by first “stacking,” at the same time point, all data that areseparated in time by a multiple of q and then computing a “leave-out-one-cycle” version of the variance for each ofthe q stacks of data. In other words, if we have k complete cycles where k D n=q, the stacked means of the otherk � 1 cycles are used to predict the ones left out, and then the averaged prediction errors are computed. For multiplecorrelated periodic sequences, suppose we are interested in estimating p1 in model (1). Instead of the stacked means,we propose to use the conditional means, that is, conditional on other correlated sequences, to predict the left-outcycle in CV. The prediction errors tend to be smallest when the period candidate q1 equals p1. As CV is typically used

Copyright © 2013 John Wiley & Sons Ltd 198 Stat 2013; 2: 197–210

Stat Multiple correlated periodic sequences

The ISI’s Journal for the Rapid (wileyonlinelibrary.com) DOI: 10.1002/sta4.28Dissemination of Statistics Research

as a model selection tool, we also consider model selection criteria for multiple periodic sequences other than CV inestimating a period.

Sun et al. (2012) have described the asymptotic behavior of the one sequence CV method. They showed that whenp1 is sufficiently large, the period estimator Op1 is virtually consistent, in the sense that limn!1 P

�Op1 D p1

�increases

to 1 as p1 increases. In this paper, we show that the multivariate CV has the same asymptotic properties. However, forfinite samples, our simulation studies indicate that stronger correlation between sequences, better period estimation ofother correlated sequences, and use of more correlated sequences all lead to substantial improvements in estimatingthe period of interest.

The rest of this paper is organized as follows. Section 2 describes the CV method of estimating the period andsequence values for multiple periodic sequences. Section 3 discusses asymptotic properties of the method. Simulationsmotivated by real data applications are reported in Section 4. Two applications to ice core data and physiologicalsignals data are presented in Section 5. Concluding remarks are provided in Section 6, and a derivation of maximumlikelihood estimates (MLEs) for bivariate sequence means is in the Appendix.

2 MethodologySuppose we observe multiple sequences at equally spaced time points from model (1), where for each s D 1, : : : , d,the sequence �s,1, : : : ,�s,n is periodic with (smallest) period ps and the "ts are i.i.d. random vectors with zero meansand covariance matrix †. We propose a methodology for estimating p1, the period of interest, in Section 2.1 and analternative based on Akaike’s information criterion (Akaike, 1973, AIC) in Section 2.2. Besides period estimation, wealso discuss the estimation of sequence values for multiple periodic sequences.

2.1. Multivariate cross-validation method for period estimationFor one periodic sequence, Sun et al. (2012) proposed the CV method for period estimation. Let q be a candidateinteger period with 2 6 q 6 Mn, where Mn is of smaller order than

pn. For each i D 1, : : : , q, they constructed

an estimator of �i by stacking all data that are separated in time by a multiple of q. So at time points i, the dataare Xi, XiCq, : : : , XiCqkq,i , where kq,i is the largest integer such that iC qkq,i 6 n. For each relevant q, i, and j, defineXqij D XiC.j�1/q. Let

CV.q/ D1

n

qXiD1

kq,iXjD1

�Xqij � NX

jqi

�2, (2)

where NXjqi is the average of Xqi`, ` D 1, : : : , kq,i, excluding Xqij. A period estimator Op is defined to be the minimizer of

CV.q/ for 2 6 q 6 Mn.

Now, suppose we observe multiple sequences at equally spaced time points from model (1). Let qs be a candidateinteger period of the s-th sequence. For the first sequence, let q D q1 for simplicity, and then NXj

qi is the average of Xqi`

excluding Xqij, i D 1, : : : , q, ` D 1, : : : , kqi and i C qkq,i 6 n. Define the averages for the sequences for s D 2, : : : , din a similar way. For i D 1, : : : , qs, let NYs,qsi be the average of Ys,qsi`, ` D 1, : : : , kqs,i and iC qskqs,i 6 n. Then definethe residual for each of the sequences to be O"s,qsi` D Ys,qsi` �

NYs,qsi. Now, by defining the residual separately for eachsequence, we have a residual vector O"t at each time point t, t D 1, : : : , n. Let

CVd.q/ D1

n

qXiD1

kq,iXjD1

hXqij �

°NXj

qi C �T12†

�122 O"qij

±i2, (3)

Stat 2013; 2: 197–210 199 Copyright © 2013 John Wiley & Sons Ltd



where O"qij D O"iC.j�1/q. Initially, we assume � 12 and †22 to be known and define a period estimator Op1 to be theminimizer of CVd.q/ given Op2, : : : , Opd, each of which is the minimizer of CV.q/ in (2). In fact, we can choose q1, : : : , qd

to minimize CVd.q/ in general.

The case of two sequences is a special case of the CV criterion in (3). If we observe two sequences ¹Xt : t D 1, : : : , nºand ¹Zt : t D 1, : : : , nº, criterion (3) can be simplified as

CV2.q/ D1

n

qXiD1

kq,iXjD1

�Xqij �

²NXj

qi C�1

�2� O"qij

³2, (4)

where �21 D Var.Xt/, �22 D Var.Zt/, � D Cov.Xt, Zt/, O"qij D O"iC.j�1/q, i D 1, : : : , q, j D 1, : : : , kq,i, r D 1, : : : , q2, and` D 1, : : : , kq2,r. Then we propose to estimate p1 in the following way:

S1. Apply the CV method (2) for one periodic sequence in Sun et al. (2012) to each sequence and obtain theperiod estimates Op.0/1 and Op2 for p1 and p2, respectively.

S2. Estimate the sequence values for each periodic sequence using stacked means corresponding to theestimated periods Op.0/1 and Op2. We show in the Appendix that these estimators of the sequence values arep

n-consistent.S3. Subtract the estimated sequence values from the observations, compute the sample covariance matrix from

residual vectors, and obtain O�1, O�2 and O�.S4. Construct NXj

qi CO�1O�2O� O"qij to predict Xqij and compute the averaged squared prediction errors in (4).

S5. Choose q to minimize (4) and obtain the period estimate Op.1/1 of the first sequence.S6. Repeat steps 2–5 and choose the period estimate Op1 D Op

.2/1 .

When "t has an elliptical distribution, the best linear predictor of Xt given Zt is �1,t C�1�2�"2,t. This motivates the

predictor NXjqi C

O�1O�2O� O"qij of Xqij. When � D 0, Equation (4) reduces to Equation (2) for the one-sequence case, and the

second sequence is not used. It is also worth pointing out that the multivariate CV method is computationally simplebecause the main computation of the prediction errors for each period candidate only involves calculating the stackedmeans.

2.2. Model selection criteria for multiple periods estimationCross-validation is typically used as a model selection tool. Sun et al. (2012) discussed period estimation for onesequence from a model selection point of view, as in fact each candidate period q corresponds to a model for thesequence consisting of the q parameters �1, : : : ,�q. Similarly, for multiple periodic sequences, if we assume that theerror vectors in model (1) are i.i.d. multivariate normal, then AIC has the form

AIC.q1, : : : , qd/ D �2 log L�O�t, O†jYt, t D 1, : : : , n

�C 2

dXsD1

qs, (5)

where O�t and O† are the MLEs of �t and †, respectively, when the periods are assumed to be q1, : : : , qd. MinimizingAIC.q1, : : : , qd/ provides estimators of p1, : : : , pd. For the one-sequence case, it is straightforward to verify that theMLEs of �1, : : : ,�q are the stacked means NYq1, : : : , NYqq. For multiple correlated sequences, instead of simple stackedmeans for each sequence, the MLEs depend on the covariance matrix †. The proof for the bivariate case is in theAppendix. We can see that only when � D 0 are the MLEs the usual stacked means. For � ¤ 0, the MLEs depend onthe value of � and the observations from both of the sequences.




3 Theoretical properties of the bivariate CV methodHere, we provide some theoretical evidence for the effectiveness of our bivariate CV method. First of all, we can arguethat, asymptotically, the proposed predictor of Xt is less noisy than that based on the one-series method of Sun et al.(2012). Let QX.t/q be the leave-out-one-cycle predictor of Xt using only data from the first sequence. The prediction errorin this case is

Xt � QX.t/q D "1,t C

��1,t � QX

.t/q

�,

and in the two-sequence case, it is

Xt �

�QX.t/q C

O�1

O�2O� O"2,t

�D "1,t �

�1

�2�"2,t C

��1,t � QX

.t/q

�C Op

�n�1=2

�,

where for the latter error we have used a consistency result to be discussed near the end of this section.

Now, for ıt D "1,t � .�1=�2/�"2,t,

Var.ıt/ D �21 C�21�22�2�22 � 2

�1

�2��1�2� D �

21

�1 � �2

�6 �21 .

Therefore, by using the second sequence, we can better predict Xt when � 6D 0, which in turn should lead to a lessnoisy CV criterion.

The first-stage CV criterion that uses information from both sequences is

CV2.q/ D1

n

nXtD1

�Xt � QX

.t/q �

O�1

O�2O� O"2,t

�2D CV1.q/ �

2

nO�1

O�2O�

nXtD1

�Xt � QX

.t/q

�O"2,t C

1

nO�21O�22O�2

nXtD1

O"22,t

D CV1.q/ �2

n�1

�2�

nXtD1

�Xt � QX

.t/q

�"2,t C

1

n�21�22�2

nXtD1

"22,t C �n

D CV1.q/ �2

n�1

�2�

nXtD1

��1,t � QX

.t/q

�"2,t C �n �

2

n�1

�2�

nXtD1

"1,t"2,t C1

n�21�22�2

nXtD1

"22,t.

Now, the last two summands on the right-hand side of the last equation are free of q and hence irrelevant.Because QX.t/q and "2,t are independent, we may use techniques as in Sun et al. (2012) to show that the effect of2n�1�2�Pn

tD1

��1,t � QX

.t/q

�"2,t is asymptotically negligible. The term �n is Op.n�1=2/, and hence negligible, if we can

show that the initial estimators of the two sequences of means arep

n-consistent uniformly in t. This is carried outin the Appendix. So, the bottom line is that, under the conditions of Sun et al. (2012), the two-series CV method isasymptotically equivalent to the one sequence method.

Although the two methods are asymptotically equivalent, the simulation results of the next section provide ampleevidence that the multiple series approach can improve upon the single series approach in moderate-sized samples.

4 Simulations4.1. Bivariate caseTo study whether another correlated periodic sequence improves the period estimation, we performed simulations withan intermediate value of the period, that is, p1 D 43 months, which is the estimate of the El Niño period obtained bySun et al. (2012). As it is well known that sea surface temperatures are negatively correlated with sea level pressures,




we create another series with p2 D 59, let �1 D �2 D 1 in model (1), and consider three cases, � D 0,�0.6,�0.8.For all cases, the set over which the objective function CV2.q/ was searched was taken to be ¹12, 13, : : : , 96º months,or 1 to 8 years, which is the possible range of El Niño period, and the algorithm S1–S5 proposed in Section 2.1 wasused to estimate p1. The number of replications of each setting is 1000.

The left panel of Figure 1 shows how the probabilities of choosing p1 D 43 increase as the sample size n increasesfor each � D 0,�0.6,�0.8 when p2 D 59. It is clear that the convergence is faster for larger correlation. We can alsosee that for finite samples, the second sequence does improve the estimation of p1, and more improvements couldbe obtained with stronger correlation. Then we consider different values of p2 when � D �0.8. The probabilities ofchoosing p1 D 43 are shown in the right panel of Figure 1 for p2 D 20, 59, 80. For a fixed sample size n, the probabilityof choosing p1 D 43 is higher for smaller values of p2 because of more available cycles of the second sequence. Thus,for a fixed value of �, how much the second series helps for estimating p1 depends on how well p2 is estimated as well.

Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

ρ = 0ρ = − 0.8ρ = − 0.6

Two Sequences

200 400 600 800 1000 200 400 600 800 1000

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

ρ = − 0.8

Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

p = 20p = 59p = 80

Figure 1. Left panel: P�Op1 D p1

�for the bivariate CV method at different correlation levels. Right panel: P

�Op1 D p1

�for the

bivariate CV method at different values of p2.

200 400 600 800 1000

0.0

0.2

0.4

0.6

0.8

1.0

Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

σ = 0.5σ = 1σ = 1.5

Figure 2. P�Op1 D p1

�for the bivariate CV method at different error levels of the second sequence. The values of �1 and �

are 1 and �0.8, respectively, in each case.




Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

One Seq.Two Seq.Three Seq.

200 400 600 800 1000

0.4

0.5

0.6

0.7

0.8

0.9

1.0

200 400 600 800 1000

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

p =p =59p =p =20

Figure 3. Left panel: P�Op1 D p1

�for the univariate, bivariate, and trivariate CV methods. Right panel: P. Op1 D p1/ for the

trivariate CV method when p2 D p3 D 59 and p2 D p3 D 20.

Now, suppose p1 D p2 D 43, � D �0.8, and �1 D 1. We consider different error levels of the second sequence.Figure 2 shows the probabilities of choosing p1 D 43 for the different error levels �2 D 0.5, 1, 1.5. The convergenceis slower for larger error levels, which is further evidence that how well p2 is estimated plays a role in the estimationof p1.

4.2. Trivariate caseSuppose we observe three periodic sequences from model (1). Let �1 D �2 D �3 D 1, �12 D �0.8, which is the sameas the correlation in the bivariate simulation, �13 D 0.6 and �23 D �0.5. Initially, we let p1 D 43, p2 D p3 D 59.The probabilities of choosing p1 D 43 are shown in the left panel of Figure 3. With three sequences, the convergenceis faster than either one or two sequences, which shows that one more correlated sequence further improves theestimation of p1, although the improvement from one to two sequences is apparently larger than improvements fromtwo to three. We also consider a situation with p2 D p3 D 20. The right panel of Figure 3 shows that the convergenceis faster for p2 D p3 D 20 than for p2 D p3 D 59. This is for the same reason as in the two sequences case, that is,more cycles are available for a smaller period when n is fixed.

4.3. AIC for bivariate caseFor the bivariate case, the AIC in (5) becomes

AIC.q1, q2/ D1

1 � �2

´1

�21

nXtD1

.Xt � O�1,t/2 C

1

�22

nXtD1

.Zt � O�2,t/2 �

2�

�1�2

nXtD1

.Xt � O�1,t/.Zt � O�2,t/

μC 2.q1 C q2/,

where q1 and q2 are the period candidates for p1 and p2, and we choose the period estimates as the minimizerof AIC.q1, q2/. Again, let p1 D 43, p2 D 59, �1 D �2 D 1, and � D �0.8. The set for q1 and q2 over which theAIC.q1, q2/ was searched was taken to be ¹12, 13, : : : , 96º, and the number of replications of each setting is 1000.In the simulation study, we assume that the parameters �1, �2, and � are known, and the mean parameters wereestimated by the closed form MLEs in the Appendix, which are a function of the observations, �, and the ratio of �1




200 400 600 800 1000

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Sample Size n

P(c

hoos

ing

corr

ect m

odel

)

One Seq.Two Seq.

Figure 4. P. Op1 D p1/ for the bivariate case by the AIC method compared with the CV method for one sequence.

and �2. Then the criterion AIC.q1, q2/ was minimized over the set ¹12, 13, : : : , 96º� ¹12, 13, : : : , 96º. The probabilitiesof choosing p1 D 43 are shown in Figure 4. Similarly, it is clear that the convergence is faster for the two-sequencecase.

5 Applications5.1. Ice core dataAn ice core is a core sample removed from an ice sheet. It contains ice formed over a range of years, which enablesthe reconstruction of local temperature records. The Greenland Ice Sheet Project (GISP) was a decade-long projectto drill ice cores in Greenland and has provided important insights into the climate variations and demonstratedthe significance of the climatic record stored in ice sheets. It started in 1971 at Dye 3, where a 372-m deep corewas recovered. After this, intermediate depth cores at various locations were drilled on the ice sheet. The first wasa 398-m core at Milcent and another was a 405-m core at the Crete station in 1974. More information aboutGISP is provided by the National Oceanic and Atmospheric Administration (http://www.ncdc.noaa.gov/paleo/icecore/greenland/gisp/gisp.html).

A problem of great interest is to search for a signal of the El Niño effect from ice core series, as the ENSO is notexpected in the North Atlantic region at all (North et al., 2012). However, North et al. (2012) also showed that theENSO can be detected when several millennia of data are accumulated by analyzing one time series of 3600 yearlydata from the Dye 3 station. In fact, such a long time series of ice core data is difficult to obtain; thus, instead of onlyusing one sequence, we estimate the period of the ice core data from Milcent (Y1) by borrowing information from theother two sequences from Dye 3 (Y2) and Crete (Y3) stations, respectively.

The annual data available for all three stations are from the years 1176–1872, 697 years in total. The three sequencesare positively correlated, and the sample correlations are �12 D 0.09, �13 D 0.28, and �23 D 0.20. The univariate CVmethod proposed by Sun et al. (2012) yields a period estimate of Op1 D 14 years for Milcent (see the left panel ofFigure 5 for the CV curve), Op2 D 3 years for Dye 3, and Op3 D 9 years for Crete. However, the estimate Op1 D 14 yearshas no clear interpretation in meteorology (North et al., 2012). Now, we add the other two sequences, Dye 3 andCrete, to estimate p1. As seen in the CV curve in the right panel of Figure 5, the multivariate CV method gives


http://www.ncdc.noaa.gov/paleo/icecore/greenland/gisp/gisp.html

http://www.ncdc.noaa.gov/paleo/icecore/greenland/gisp/gisp.html



1.02

1.04

1.06

1.08

1.10

year

CV

1(q)

14

0.94

0.96

0.98

1.00

1.02

year

CV

3(q)

10 20 30 40 50 10 20 30 40 504 14

Figure 5. Left panel: the plot of the CV curve for estimating the period of the Milcent sequence. Right panel: the plot of theCV curve for estimating the period of the SST sequence when using Milcent, Dye 3, and Crete sequences.

Op1 D 4 years, which is within the range of plausible El Niño periods; see Torrence & Webster (1999) and referencestherein. Even though the criterion CV3.4/ D 0.9409 is just slightly smaller than CV3.14/ D 0.9417, we are able todetect what appears to be a faint El Niño signal by borrowing information from nearby locations.

5.2. Physiological signals dataIn the intensive care unit (ICU), monitors are used to provide information about patients in numerical and waveformformats, such as arterial blood pressure (ABP), electrocardiography (ECG), and fingertip plethysmograph (PLE). Thesephysiological signals are often severely corrupted by noise, which makes the cardiac cycle estimation less accurate inthe diagnosis and tracking of medical conditions. Thus, it results in a high incidence of false alarms from ICU monitors(Li et al., 2008).

The frequency of the cardiac cycle is described by the heart rate, which is the number of heartbeats per unit of time,typically defined as beats per minute. Therefore, the signals from ABP, ECG, and PLE related to heartbeats all showa certain periodicity. The behavior of blood pressures is an important indicator of human health. We are interestedin studying its periodicity, in particular, identifying the period as well as a typical blood pressure cycle, which is notnecessarily the cycle between two heartbeats.

t (sec)

AB

P (

mm

Hg)

t (sec)

EC

G (

mV

)

0 10 20 30 40 50 60 0 10 20 30 40 50 60 0 10 20 30 40 50 60

7080

9010

011

012

013

0

0.0

0.2

0.4

0.6

−0.

50.

00.

5

t (sec)

PLE

(m

V)

Figure 6. Time-series plots of the three signals: the arterial blood pressure (ABP) with unit mmHg, electrocardiography(ECG), and fingertip plethysmograph (PLE) with unit millivolt (mV).




We randomly select one patient from the Multiparameter Intelligent Monitoring in Intensive Care database (fromhttp://www.physionet.org/physiobank/database/mimicdb/) with three signals: ABP (Y1), ECG (Y2), and PLE (Y3). Thedescription of this database can be found in Moody & Mark (1996) and Goldberger et al. (2000). The monitors take500 samples per second, and thus measurements are available every 0.002 s. We analyze the record of the threesignals of total length 1 min, that is, the sample size is N D 30, 000. The three sequences are shown in Figure 6, andthe sample correlations are �12 D 0.1183, �13 D 0.3944, and �23 D 0.0001.

The CV method for one sequence proposed by Sun et al. (2012) yields a period estimate of Op D 475� 0.002 D 0.95 sfor all three sequences separately, which means that within 1 min, such a periodic event happens 60=0.95 D 63.15times. Suppose we only have shorter sequences with a sample size n D 1500. For the ABP sequence, in this case, theCV method gives a period estimate of Op1 D 708 � 0.002 D 1.42 s, that is, 42.37 cycles; see the left panel of Figure 7for the CV curve. Notice that the CV curve is also locally minimized at 0.95 s.

t (sec)

CV

1(q)

t (sec)

CV

3(q)

0.6 0.8 1.0 1.2 1.4

010

020

030

040

050

060

0

0.95 1.420.6 0.8 1.0 1.2 1.4

010

020

030

040

050

060

0

0.95 1.42

Figure 7. Left panel: the plot of the CV curve for estimating the period of blood pressures from ABP sequence only. Rightpanel: the plot of the CV curve for estimating the period of blood pressures using ABP, ECG, and PLE sequences.

t (sec)

AB

P (

mm

Hg)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

7080

9010

011

012

013

0

0.0 0.2 0.4 0.6 0.8

7080

9010

011

012

013

0

t (sec)

AB

P (

mm

Hg)

Figure 8. Left panel: a plot of the cycles of data corresponding to a period of 1.42 s superimposed on each other. Rightpanel: a plot of the cycles of data corresponding to a period of 0.95 s superimposed on each other. The estimated meansare connected by the red lines.


http://www.physionet.org/physiobank/database/mimicdb/



The plot of the cycles of data superimposed on each other for Op1 D 1.42 s is in the left panel of Figure 8, whichcontains three peaks. Interestingly, if only one of ECG or ELP is used with ABP, the period estimate does not change,but if we use both with ABP, Op1 becomes 470 � 0.002 D 0.94 s. In other words, there are 63.83 cycles within 1 min,which is much closer to the estimate from the whole series of ABP; see the right panel of Figure 7 for the CV curve. Aplot of the cycles of data superimposed on each other for Op1 D 0.95 s is in the right panel of Figure 8, which includestwo peaks. By improving the period estimation of blood pressures as well as the periodic mean estimation, the cardiaccycle between two maximum blood pressures can also be better characterized.

6 DiscussionIn this paper, we have proposed a CV period estimator when two or more equally spaced periodic sequences areavailable. Using an idea similar to the CV method for one sequence, a leave-out-one-cycle version of CV is used tocompute an average squared prediction error given a particular period, or cycle length. The multivariate CV methoduses the conditional means, that is, conditional on other correlated sequences, to predict the left-out cycle in CV. Inthis way, the period estimation for a sequence X has been improved by borrowing information from other sequenceswith which X is correlated. In theory, we argue heuristically that the asymptotic behavior of the bivariate CV is thesame as that of the CV for one sequence. In our simulation studies, however, it was shown that for finite samples,the better the periods of the other correlated sequences were estimated, the more substantial improvements couldbe obtained in estimating the period of interest. We have also shown that more correlated sequences lead to moreimprovements, although the improvement from one to two sequences is apparently larger than improvements fromtwo to three, and so on. In addition to the CV method, we also considered a model selection criterion, AIC, to estimatethe period for multiple periodic sequences. Similarly, for finite samples, a simulation study showed an improvement inperiod estimation from using information in a correlated sequence. The asymptotic properties of the AIC method needfurther exploration.

Appendix AA.1. Deriving MLEs of sequence of meansFor the bivariate case, suppose we observe bivariate sequences from the following model:

²Xt D �1,t C "1,t,Zt D �2,t C "2,t,

where�"1"2

�t

� i.i.d. N2.0,†/, and † D�

�21 ��1�2��1�2 �22

�. The likelihood is of the form

L D1

1 � �2

²1

�21

nXtD1

.Xt � �1,t/2 C

1

�22

nXtD1

.Zt � �2,t/2 �

2�

�1�2

nXtD1

.Xt � �1,t/.Zt � �2,t/

³. (A.1)

Let q1 and q2 be the period candidates for Xt and Zt, respectively, and mi D �2,iCq2 D �2,iC2q2 D � � � for i D 1, : : : , q2,�i D �1,iCq1 D �1,iC2q1 D � � � for i D 1, : : : , q1. First, minimize

1

�22

nXtD1

.Zt � �2,t/2 �

2�

�1�2

nXtD1

.Xt � �1,t/.Zt � �2,t/ (A.2)




with respect to m1, : : : , mq2 . Define Zq2ij D ZiC.j�1/q2 , where i D 1, : : : , q2 and kq2i is the largest integer such thatiC q2kq2,i 6 n. Then (A.2) can be written as

A D1

�22

q2XiD1

kq2 iXjD1

.Zq2ij �mi/2 �

2�

�1�2

q2XiD1

kq2 iXjD1

.Xq2ij � �1,q2ij/.Zq2ij �mi/.

The first derivative

@A@miD �

2

�22

kq2 iXjD1

.Zq2ij �mi/C2�

�1�2

kq2 iXjD1

.Xq2ij � �1,q2ij/ D 0

givesmi D NZq2i � �

�2

�1. NXq2i � N�1,q2i/, (A.3)

where NZq2i is the average of Zq2i`, ` D 1, : : : , kq2i. Then plug (A.3) into Equation (A.1),

L1 D1

�21

q1XiD1

kq1 iXjD1

.Xq1ij � �i/2 C

1

�22

q2XiD1

kq2 iXjD1

.Zq2ij � NZq2i/2

��2

�21

q2XiD1

kq2i. NXq2i � N�1,q2i/2 �

2�

�1�2

q2XiD1

kq2 iXjD1

.Xq2ij � �1,q2ij/.Zq2ij � NZq2i/.

We then have @�1,q2ij=@�r D 1, if iC .j � 1/q2 D r C .` � 1/q1 for some `, otherwise 0, and @ N�1,q2i=@�r D Nir=kq2i,where Nir is the number of times iC .j � 1/q2 D r C .` � 1/q1 when j ranges from 1 to kq2i and ` ranges from 1 tokq1r. Then

@L1@�rD �

2

�21kq1r. NXq1r � �r/C

2�2

�21

� q2XiD1

NXq1iNir �

q2XiD1

1

kq2i

q1XkD1

NirNik�k

�C

2�

�1�2Sq1,q2,r,

where Sq1,q2,r DPkq1r

iD1

�Yq1rj � NY

q2q1rj

�. We can see that @L1=@�r D 0 if and only if

kq1r. NXq1r � �r/C �2

q2XiD1

1

kq2i

q1XkD1

NirNik�k D �2

q2XiD1

NXq2iNir C ��1

�2Sq1,q2,r.

This yields a set of q1 linear equations. Arrange them so that we can solve a smaller system, that is, q1 < q2. Thecoefficient of �r in the r-th equation is

�kq1r C �2

q2XiD1

1

kq2iN2ir,

and the coefficient of �k (k ¤ r) is

�2q2X

iD1

1

kq2iNirNik.

The equations are Bq1,q2� D bq1,q2 , where Bq1,q2.j, k/ D �2Pq2

iD11

kq2 iNijNik�kq1jI.j�k/ for I.0/ D 1, 0 otherwise, and

bq1,q2.j/ D �2

q2XiD1

NXq2iNij C ��1

�2Sq1,q2,j � kq1,j

NXq1j.

When q1 D q2, the MLEs of �1, : : : , �q1 and m1, : : : , mq2 are the usual stacked means. �




A.2. Proof of consistency of mean estimator for one-series methodConsider observations Y1, : : : , Yn from Yt D �tC"t, t D 1, : : : , n, where "1, : : : , "n are i.i.d. N.0, �2/, and the sequenceof means �1,�2, : : : is periodic with (smallest) period p0. Let O�q,n be the vector of stacked mean estimates of�n D .�1, : : : ,�n/ that correspond to assuming that the period is q. For example, if n D 10 and q D 3, O�q,n has theform

�O�31, O�

32, O�

33, O�

31, O�

32, O�

33, O�

31, O�

32, O�

33, O�

31

�. Let Op be the Sun et al. (2012) (SHG) estimator of p0, and, for simplicity

of notation, let O�n D . O�1, : : : , O�n/ D O�Op,n. Under the conditions of SHG, we will show that O�n isp

n-consistent for �n

in the sense that

ın D max16i6n

j O�i � �ij D Op

�n�1=2

�.

Two facts are key in proving consistency of O�n. First, although Op is not a consistent estimator of p0, SHG showthat, asymptotically, the support of Op is p0, 2p0, : : :. The second key is that, for any j, O�jp0

k isp

n-consistent for�k, k D 1, : : : , jp0, because each O�jp0

k is a sample mean of approximately n=.jp0/ i.i.d. random variables. So withprobability tending to 1, O�n is one of O�p0,n, O�2p0,n, : : :, any of which is

pn-consistent.

We need to show thatp

nın is bounded in probability. This means that for each " > 0, we must find M" and n" suchthat P.

pnın > M"/ 6 " for all n > n". Let En be the event that Op is in the set ¹p0, 2p0, 3p0, : : :º. We have

P�p

nın > M"

�D P

�pnın > M" \ En

�C P

�pnın > M" \ Ec

n

�6 P

�pnın > M" \ En

�C P

�Ec

n

�. (A.4)

SHG show that P.Ecn/! 0 as n!1, and so obviously there exists n1" such that P.Ec

n/ 6 "=2 for all n > n1".

SHG minimize the CV criterion over a set ¹1, : : : , Mnº, where Mn D o.p

n/. Let m be an integer such that 1 6 mp0 <Mn. For mn the largest integer such that mnp0 6 Mn, we have

P�p

nın > M" \ En�D P

�pnın > M" \ Op 2 ¹p0, 2p0, : : : , mp0º

�C P

�pnın > M" \ Op 2 ¹.mC 1/p0, : : : , mnp0º

�6 P

�pnın > M" \ Op 2 ¹p0, 2p0, : : : , mp0º

�C P. Op 2 ¹.mC 1/p0, : : : , mnp0º/. (A.5)

The theorem of SHG entails that P. Op 2 ¹.mC 1/p0, : : : , mnp0º/ can be made arbitrarily small by taking m sufficentlylarge. Therefore, there exist m" and n2" such that for all n > n2"

P. Op 2 ¹.m" C 1/p0, : : : , mnp0º/ 6"

4. (A.6)

Turning to the other probability on the right-hand side of (A.5),

P�p

nın > M" \ Op 2 ¹p0, 2p0, : : : , m"p0º�D

m"XjD1

P�p

nın > M" \ Op D jp0�6

m"XjD1

P�p

n max16k6jp0

j O�jp0k � �kj > M"

�

6m"XjD1

jp0XkD1

P�p

nj O�jp0k � �kj > M"

�6

m"XjD1

jp0XkD1

nM2"

�2

[ n=.jp0/], (A.7)

where [ x] denotes the largest integer not greater than x. The last inequality simply makes use of Markov’s inequalityand the fact that O�jp0

k is a sample mean of at least [ n=.jp0/] i.i.d. random variables.

It is straightforward to check that the right-hand side of (A.7) is bounded by n�2p20m3"=[ .n � m"p0/M2

" ]. Obviously,there exists n3" such that for all n > n3", n=.n �m"p0/ < 3=2. Therefore, we need .3=2/�2p20m

3"=M

2" 6 "=4, or

M" >p6�p0m

3=2"

p"

. (A.8)




Once m" is determined, we can obviously choose M" to satisfy (A.8), and so, putting together (A.4)–(A.7), for alln > max.n1", n2", n3"/ and M" satisfying (A.8), P.

pnın > M"/ 6 ", which completes the proof. �

AcknowledgementThe work of Professor Hart was supported by NSF grant DMS 1106694.

ReferencesAkaike, H (1973), ‘Information theory and an extension of the maximum likelihood principle’, in Petrov, BN & Csake,

F (eds), Second International Symposium on Information Theory, Akademiai Kiado, Budapest, 267–281.

Canova, F (1996), ‘Three tests for the existence of cycles in time series’, Ricerche Economiche, 50, 135–162.

Genton, MG & Hall, P (2007), ‘Statistical inference for evolving periodic functions’, Journal of the Royal StatisticalSociety Series B, 69, 643–657.

Goldberger, AL, Amaral, L, Glass, L, Hausdorff, JM, Ivanov, P, Mark, RG, Mietus, JE, Moody, GB, Peng, C& Stanley, HE (2000), ‘PhysioBank, physioToolkit, and physioNet: components of a new research resource forcomplex physiologic signals’, Circulation, 101, 215–220.

Hall, P (2008), ‘Nonparametric methods for estimating periodic functions, with applications in astronomy’, in Brito,P (ed.), COMPSTAT 2008: Proceedings in Computational Statistics, Physika-Verlag, Springer, Heidelberg, 3–18.

Hall, P & Li, M (2006), ‘Using the periodogram to estimate period in nonparametric regression’, Biometrika, 93,411–424.

Hall, P, Reimann, J & Rice, J (2000), ‘Nonparametric estimation of a periodic function’, Biometrika, 87, 545–557.

Hall, P & Yin, J (2003), ‘Nonparametric methods for deconvolving multiperiodic functions’, Journal of the RoyalStatistical Society Series B, 65, 869–886.

Li, T (2012), ‘Detection and estimation of hidden periodicity in asymmetric noise by using quantile periodogram’, inProceedings of 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IBMTJ Watson Res. Center, Yorktown Heights, NY, USA, 3969–3972.

Li, Q, Mark, RG & Clifford, GD (2008), ‘Robust heart rate estimation from multiple asynchronous noisy sources usingsignal quality indices and a Kalman filter’, Physiological Measurement, 29, 15–32.

Moody, GB & Mark, RG (1996), ‘A database to support development and evaluation of intelligent intensive caremonitoring’, Computers in Cardiology, 23, 657–660.

North, GR, Genton, MG & Chylek, P (2012), ‘Using ice core data to constrain climate sensitivity’, Manuscript.

Quinn, BG & Thomson, PJ (1991), ‘Estimating the frequency of a periodic function’, Biometrika, 78, 65–67.

Rice, JA & Rosenblatt, M (1988), ‘On frequency estimation’, Biometrika, 75, 477–484.

Sun, Y, Hart, JD & Genton, MG (2012), ‘Nonparametric inference for periodic sequences’, Technometrics, 54,83–96.

Torrence, C & Webster, P (1999), ‘Interdecadal changes in the ENOS–Monsoon system’, Journal of Climate, 12,2679–2690.

Walker, AM (1971), ‘On the estimation of a harmonic component in a time series with stationary independentresiduals’, Biometrika, 58, 21–36.


Documents

Improved nonparametric inference for multiple correlated