Time series, spring 2014 1

The course• Time series, stochastic processes, existence theorem,

stationarity and strict stationarity, autocovariance functions, multivariate normal distribution

• Trends and seasonality• Hilbert spaces• ARMA processes• Spectral methods• Prediction• Estimation, model selection, and model checking• Multivariate time series (perhaps)• Kalman filters• ARIMA, unit root , and stochastic volatility models

Course literature

(TS) Time Series: Theory and Methods, second edition (1991) P.J. Brockwell and R.A. Davis, Springer-Verlag, New York.

Complementary literature:(ITS): Introduction to Time Series and Forecasting, second edition (2002) P.J. Brockwell and R.A. Davis, Springer-Verlag, New York + material from new version of the book, soon to appear.

ITSM: The program package which comes with the books:http://www.stat.columbia.edu/~rdavis/Pune2015/itsm7pro.zip

Slides provided by Richard Davis

SSPSE: Stationary stochastic processes for scientists and engineers(2013) G. Lindgren, H. Rootzén, and M. Sandsten

Slides for TS ch 1, p 1-37

Exercises: TS 1.2, 1.4, 1.5, 1.6, 1.7, 1.8, 1.10, 1.11, 1.12, 1.13,1.15, 1.18

Australian red wine sales

(thou

sand

s)

1 9 8 0 1 9 8 2 1 9 8 4 1 9 8 6 1 9 8 8 1 9 9 0 1 9 9

0.5

1.0

1.5

2.0

2.5

3.0

Features: upward trendseasonal pattern (peak in July, trough in Jan)increase in variability

Time Series Plots: Examine for

• trend over time (does the series increase or decrease with time)

• regular seasonal (or cyclical) components

• constant variability over time• other systematic features of the data

Ex 1.1.4 (All-star baseball games, 1933-1995)

1 9 4 0 1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0 1 9 9 0

-2-1

01

2

1 if National league won-1 if American league won

Ex 1.1.6 (Accidental deaths, USA; DEATHS.TSM)

(thou

sand

s)

1 9 7 3 1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7

78

910

11

Features: slight trendseasonal component (peak in July)

Monthly accidental deaths

s.rv' N(0,.25) of sequence IIDan is N where2,...,200 1,= t, N )10cos(t/

t

t+=tX

Ex. (Signal Detection; SIGNAL.TSM)

Figure : red = estimated signal

black= true signal

0 5 1 0 1 5 2 0

-2-1

01

23

Stochastic (or random) variableDescribes the mechanism of drawing a number randomly from a bag of numbers,With different probabilities for different numbers

Stochastic processDescribes the mechanism of drawing numbers randomly from many bags of numbers

or (equivalently) the mechanism of drawing a function randomly from a bag of functions

3.49

7.14 8.72 1.03

Stochastic variables and processesΩ,𝐹𝐹,𝑃𝑃 Probability space. Ω is the ”bag of numbers”, 𝑃𝑃

determines the probability of choosing a particular set ofnumbers, 𝐹𝐹 contains all sets which have a probability

𝑋𝑋 Stochastic variable. A function Ω → 𝑹𝑹

𝑇𝑇 Parameter space. In this course 𝑇𝑇 is ”time”: 𝑹𝑹 or 0, 1, 2, … or 0, ±1, ±2, … or …

𝑋𝑋𝑡𝑡; 𝑡𝑡 ∈ 𝑇𝑇 Stochastic process. A collection (or family) of random variables

or, a function Ω → 𝑹𝑹𝑻𝑻 , i.e. a function-valued randomvariable

or, a distribution (probability measure) on the space offunctions T → 𝑹𝑹

time

x_t

0 5 10 15 20

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

A realisation (or sample function, or sample path, or sample field, or observation, or trajectory, or …) is the function

𝑥𝑥⋅ 𝜔𝜔 : 𝑇𝑇 → 𝑹𝑹𝑑𝑑𝑡𝑡 → ft(𝜔𝜔)

for 𝜔𝜔 fixed.

An example: 5 realisations 𝑋𝑋𝑡𝑡 𝜔𝜔1 ,𝑋𝑋𝑡𝑡 𝜔𝜔2 ,𝑋𝑋𝑡𝑡 𝜔𝜔3 ,𝑋𝑋𝑡𝑡 𝜔𝜔4 , 𝑋𝑋𝑡𝑡 𝜔𝜔5 of the stochastic process 𝑋𝑋𝑡𝑡 𝜔𝜔 = 𝐴𝐴 𝜔𝜔 cos(𝜋𝜋

4𝑡𝑡 +

Θ 𝜔𝜔 , with 𝐴𝐴 > 0,Θ ∼ 𝑈𝑈(0, 2 𝜋𝜋)

Terminology

random variablestochastic variable

random elementstochastic element

random processstochastic process

random fieldstochastic field

random vectorstochastic vector

Objectives of Time Series AnalysisModelling paradigm:

• set up family of probability models to represent data • estimate parameters of model• check model for goodness of fit

Applications of models:

• provides a compact description of the data• interpretation • prediction• hypothesis testing

Ex: Binary Process

Xt ~ IID

P[Xt = 1] = p, P[Xt = -1] = 1-p,

This is a family of SPs indexed by p. How do we know a probability space (Ω,F,P) exists? Answer given by Kolmogorov’s theorem (next slide). If p=.5, then

P[X1 = i1, X2 = i2 , . . .,Xn = in] =1/2n

for any n-tuple (i1, i2 , . . ., in) of ±1’s

Is this a good model for All Star baseball games?

Ex: (Random Walk). Let Xt ~ IID as in previous example

S0= 0, St= X1+ . . . + Xt , t ≥1

The finite dimensional distributions (fidis) of a stochastic process Xt, t ∈ T are the collection of functions

where x = (x1, . . . , xn)’ and t = (t1, . . . , tn)’ are vectors in 𝑹𝑹𝑛𝑛

and 𝑇𝑇𝑛𝑛, with t1< t2< … < tn.

)...,,()( 21 21 nttt xXxXxXPFn

≤≤≤=xt

Kolmogorov’s consistency teorem

Three realizations of a stochasticprocess. 𝐹𝐹2,5,8 𝑥𝑥1, 𝑥𝑥2, 𝑥𝑥3 is the probability to obtain a sample path which passes through all three vertical lines

Theorem (Kolmogorov): The family of probability distribution functions Ft, t∈ Tn , t1< t2< … < tn, n ≥1 are the fidis of a stochastic process if and only if

where t(i) and x(i) are the vectors of t and x with the ith

component removed (i=1,…,n).

(Nice) functions of stochastic processes are stochastic processes – this is not proved in book, but is typically how processes are shown to exist in course. Existence or not is a recurring and important problem in time series analysis.

))(()(lim )( iFF ixi

xx tt =∞↑

E.g.

So one direction of the theorem is obvious (the fidis of a SP must satisfy the consistency condition). Kolmogorov says that the consistency condition is also sufficient.

One can easily check that IID sequences exist by using Kolmogorov’s theorem. (Do this!)

),,(),,(

),,,(lim),,,(lim

4214,2,1

442211

4433221143214,3,2,133

xxxFxXxXxXP

xXxXxXxXPxxxxFxx

=≤≤≤=

≤≤≤≤=∞↑∞↑

Let Xt be a time series (or SP) with E Xt2 < ∞ . Then the

autovariance function (ACVF) is given by

The time series Xt is (weakly) stationary if

(i) E Xt2 < ∞ for all t

(ii) E Xt is independent of t (mean is constant)

(iii) γ(r,s)=γ(r+t,s+t) for all r, s, t (covariance only depends on distance between time points)

( )( )[ ]ssrrsr EXXEXXEXXCovsr −−==γ ),(),(

Stationarity and strict stationarity

• Replacing t with –s gives that

γ(r,s) = γ(r-s,s-s) = γ(r-s,0)

so that γ is a function of the time gap r-s. We hence write

γ(h)= γ(h,0) = Cov(Xt+h , Xt).

• Autocorrelation function (ACF). Note that γ(0) = Cov(Xt) so that ρ(h) = Cor(Xt+h , Xt)

= γ(h)/γ(0).

• in the literature, stationarity = weak stationarity

= stationarity in the wide sense= second-order stationarity

The time series Xt, t=0, ±1, ±2, …. is strictly stationary if for any k ≥ 1,

(X1, . . . , Xk) =d (X1+h, . . . , Xk+h)

for all integers h. (=d means the two vectors have the same distribution)

e.g.,

• (X1, X2 , X3 , X4)’ =d (X0, X1 , X2 , X3)’

=d (X200, X201 , X202 , X203)’

=d etc

• X1=d Xh for all integers h, so the stochastic variables in a strictly stationary stochastic process are identically distributed

• (X1, Xk)’ =d (X1+h, Xk+h)’

• If the time series Xt is strictly stationary with E Xt2 < ∞ , then

a) X1=d X2=d X3 …. and hence the mean must be constant,EX1= EX2= EX3 ….

b) (Xr, Xs)’ =d (X0, Xs-r)’ which implies that

Cov(Xr, Xs)= Cov(X0 , Xs-r) = γ(s-r).

(a) and (b) imply that Xt is stationary.

• The converse is not necessarily true:

let Zt ~ IID N(0,1) and Yt ~IID Exp(1) and define

Then 𝑋𝑋𝑡𝑡 is weakly stationary, but not strictly stationary. (Check this!)

odd is t ifeven is t if

,1,

−=

t

tt Y

ZX

Random walk

St= X1+ . . . + Xt , Xt ~ IID(0,σ2)

Is St stationary?

• ESt = 0

v(St+h , St) = Cov(X1+ . . . + Xt+h, X1+ . . . + Xt )

= Cov(X1+ . . . Xt+…+ Xt+h, X1+ . . . + Xt )

= Cov(St + St+h - St , St)

= Cov(St, St)

= t σ2

Since this ACVF depends on t, the process is not stationary.

Cosine process.

Xt= A cos(θt) + Bsin(θt),

where A and B are independent random variables with mean 0 and variance 1, and 𝜃𝜃 is a constant

EXt = 0

Cov(Xt+h , Xt)

= Cov(A cos(θ(t+h)) + Bsin(θ(t+h)), A cos(θt) + Bsin(θt))= cos(θ(t+h))cos(θt) + sin(θ(t+h))sin(θt) =cos(θh)

The process is weakly stationary (and in fact also strictly stationary)

IID noise

Xt ~ IID (0,σ2)

IID means independent and identically distributed. Both stationary and strictly stationary

White noise

Xt ~ WN (0,σ2)

means that the variables are uncorrelated but not necessarily IID.

IID WN

IID WN

0.h if 0h if

,0,

)(2

≠=

σ

=γ h

Moving Average; MA(1).

Xt = Zt + θ Zt-1 , Zt ~ WN(0,σ2)

This implies that the time series is stationary. Moreover,

1 |h| if1 h if

0 h if

,0,

,)(1)( 2

22

>±=

=

θσσθ+

=γ h

1 |h| if1 h if

0 h if

,0),1/(

,1)( 2

>±=

=

θ+θ=ρ h

Gaussian processesA random vector 𝐗𝐗 = 𝑋𝑋1, … ,𝑋𝑋𝑑𝑑 has a multivariate Gaussian distribution iff one of the following conditions hold:

• 𝛼𝛼, 𝑥𝑥 ≜ ∑𝑖𝑖𝑑𝑑 𝛼𝛼𝑖𝑖𝑋𝑋𝑖𝑖 has a univariate normal distribution for all 𝛼𝛼 ∈ 𝑅𝑅𝑑𝑑 .

• There exist a vector 𝝁𝝁 ∈ 𝑹𝑹𝑑𝑑 and a non-negative definite matrix Σ such that for all 𝜽𝜽 ∈ 𝑹𝑹𝑑𝑑

𝜙𝜙 𝜽𝜽 = 𝐸𝐸 𝑒𝑒𝑖𝑖𝜽𝜽𝑋𝑋 = e𝑖𝑖𝑖𝑖𝝁𝝁−12𝜽𝜽𝚺𝚺𝜽𝜽´

If Σ is positive definite then 𝑋𝑋 has the probability density1

2𝜋𝜋 𝑑𝑑 Σ 1/2 𝑒𝑒−12 𝒙𝒙−𝝁𝝁 Σ−1 𝒙𝒙−𝝁𝝁 ´

then 𝑋𝑋 is Gaussian.

Here 𝝁𝝁 = 𝐸𝐸 𝑿𝑿 and Σ = 𝐶𝐶𝐶𝐶𝐶𝐶 𝑿𝑿 .

We write 𝐗𝐗~𝑁𝑁𝑑𝑑 𝝁𝝁, Σ if 𝐗𝐗 has a d-variate Gaussian distribution with mean 𝑚𝑚 and covariance matrix Σ.

Important results, read TS

(i) if 𝐗𝐗~𝑁𝑁𝑑𝑑 𝝁𝝁, Σ and 𝐴𝐴 is a 𝑑𝑑 × 𝑑𝑑 matrix, then 𝐗𝐗𝐴𝐴~𝑁𝑁𝑑𝑑 𝝁𝝁𝐴𝐴,𝐴𝐴𝐴Σ𝐴𝐴

(ii) If 𝑿𝑿 = 𝑿𝑿1,𝑿𝑿2 with 𝑿𝑿1 = 𝑋𝑋1, … ,𝑋𝑋𝑛𝑛 , 𝑿𝑿2 = 𝑋𝑋𝑛𝑛+1, … ,𝑋𝑋𝑑𝑑 ,with mean vectors 𝝁𝝁1 and 𝝁𝝁2 and covariance matrix Σ =Σ1,1 Σ1,2Σ2,1 Σ2,2

, then the conditional distribution of 𝑿𝑿1 given 𝑿𝑿2 is n-variate normal with mean

𝝁𝝁1|2 = 𝝁𝝁1 + (𝑿𝑿2−𝝁𝝁2)Σ2,2−1Σ2,1

and covariance matrix

Σ1|2 = Σ1,1 − Σ1,2Σ2,2−1Σ2,1

• Xt is a Gaussian time series iff all the fidis are multivariate normal.

So if Xt is Gaussian, then for any 𝑛𝑛 the distribution of X= (X1, . . . , Xn) is multivariate normal with

mean vector: µn = E(X1, . . . , Xn) covariance matrix: Σn = Cov(X,X)

If Σn is nonsingular, then X has a pdf

f(x) = (2π)-n/2 |Σn|-1/2 exp-1/2 (x- µn)’ Σn-1 (x- µn)

• A Gaussian time series is completely determined by its mean and autocovariance functions (i.e. second order properties)

• A Gaussian time series is stationary iff it is strictly stationary

Classical Decomposition:

Xt = mt + st + Yt

mt trend component (slowly changing function of t )

st seasonal component (periodic with period d )

Yt random noise component

Remove trend and seasonal component, and then model the result (the residuals) as a stationary time series

Trends and seasonal components

Model with no seasonal component.

Xt = mt + Yt ,

where mt is a slowly varying function called the trend function and 𝑌𝑌𝑡𝑡 is the residual process. We assume E Yt = 0 -- this is a modelling assumption.

e.g. mt = a0 + a1 t + a2 t2

where coefficients are estimated by minimizing the sum of squares of the deviations between the observations 𝑥𝑥1, … , 𝑥𝑥𝑛𝑛and 𝑚𝑚, i.e. by minimizing

∑𝑡𝑡=1𝑛𝑛 𝑥𝑥𝑡𝑡 − 𝑚𝑚𝑡𝑡2

with respect to 𝑎𝑎0,𝑎𝑎1,𝑎𝑎2.

Method 1: trend estimation via least squares

Model: Xt = a0 + a1 t + a2 t2 + Yt

Ex. Population of USA.; USPOP.TSM

a0 = 6.96x105 , a2= 6.51x 105

Figure 1.8.

a1= -2.16 x 106,

(Milli

ons)

1 8 0 01 8 2 01 8 4 01 8 6 01 8 8 01 9 0 01 9 2 01 9 4 01 9 6 01 9 8 0

020

4060

8012

016

020

024

0

Forecast for year 2000: m2000 = 274.35 x 106

^ ^ ^

^

Ex. Lake Huron Levels (1875-1972); LAKE.TSM

Figure 1.9.

Model: Xt = a0 + a1 t + Yt

1 8 8 01 8 9 01 9 0 01 9 1 01 9 2 01 9 3 01 9 4 01 9 5 01 9 6 01 9 7 0

6.0

7.0

8.0

9.0

10.0

11.0

12.0

10.2-.0242 t

Residuals from the LS fit of Lake Huron data

1 8 8 0 1 9 0 0 1 9 2 0 1 9 4 0 1 9 6 0

-2-1

01

2

Note: residuals do not look IID

finite moving average filter: assume 𝑋𝑋𝑡𝑡 = 𝑚𝑚𝑡𝑡 + 𝑌𝑌𝑡𝑡, where 𝑚𝑚𝑡𝑡varies slowly (is smooth) and 𝑌𝑌𝑡𝑡 has mean zero

𝑊𝑊𝑡𝑡 =1

2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞

𝑋𝑋𝑡𝑡+𝑗𝑗

=1

2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞

𝑚𝑚𝑡𝑡+𝑗𝑗 +1

2𝑞𝑞 + 1𝑗𝑗 ≤𝑞𝑞

𝑌𝑌𝑗𝑗

≈ mt + 0

Method 2: smoothing to estimate trend

linear filter𝑋𝑋𝑡𝑡 𝑚𝑚𝑡𝑡 = 𝑊𝑊𝑡𝑡 = ∑𝑗𝑗 𝑎𝑎𝑗𝑗𝑋𝑋𝑡𝑡−𝑗𝑗

5-term moving average of strikes data STRIKES.TSM

(thou

sand

s)

1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0

3.5

4.0

4.5

5.0

5.5

6.0

Exponential smoothing

𝑚𝑚𝑡𝑡 = 𝑎𝑎𝑋𝑋𝑡𝑡 + 1 − 𝑎𝑎 𝑚𝑚𝑡𝑡−1, t= 2, … n𝑚𝑚1 = 𝑋𝑋1

0 1a

max smoothing no smoothing

Strike data a = .4

(thou

sand

s)

1 9 5 0 1 9 6 0 1 9 7 0 1 9 8 0

3.5

4.0

4.5

5.0

5.5

6.0

Classical Decomposition Model:

𝑋𝑋𝑡𝑡 = 𝑚𝑚𝑡𝑡 + 𝑠𝑠𝑡𝑡 + 𝑌𝑌𝑡𝑡

where, for seasons of length 𝑑𝑑

𝑠𝑠𝑡𝑡 = 𝑠𝑠𝑡𝑡+𝑑𝑑 and 𝑡𝑡=1

𝑑𝑑

𝑠𝑠𝑡𝑡 = 0 ,

and𝐸𝐸 𝑌𝑌𝑡𝑡 = 0

Estimation of both trends and seasonality

Step 1: Estimate the trend using a simple moving average of length q = d/2 (or (d-1)/2).

Step 2: Estimate sk, k =1,…,d using the average deviations from trend for each season.

Step 3: Deseasonalize the data by forming

dt = xt − st , t=1,…,n

Step 4: Fit a parametric function mt to the deseasonalizeddata dt.

Step 5: Calculate the estimated noise/residuals

Yt = xt − mt − st

^

^ ^ ^

The accidentals deaths data.data = blue boxes

st = red lines

(thou

sand

s)

1 9 7 3 1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7

78

910

11

Difference Operator 𝛻𝛻 ≔ 1 − 𝐵𝐵

𝛻𝛻𝑋𝑋𝑡𝑡 = 𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−1𝛻𝛻2𝑋𝑋𝑡𝑡 = 𝛻𝛻(𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−1)

= 𝑋𝑋𝑡𝑡 − 2𝑋𝑋𝑡𝑡−1 + 𝑋𝑋𝑡𝑡−2

Backward Shift Operator B :

B Xt = Xt-1

Bs Xt = Xt-s , s=0, + 1, . . ..

Method 3: differencing to eliminatetrends and seasonality

Seasonal Differencing 𝛻𝛻𝑑𝑑 ≔ 1 − 𝐵𝐵𝑑𝑑

𝛻𝛻𝑑𝑑𝑋𝑋𝑡𝑡 = 𝑋𝑋𝑡𝑡 − 𝑋𝑋𝑡𝑡−𝑑𝑑If 𝑠𝑠𝑡𝑡 is a seasonal component with period 𝑑𝑑 then

𝛻𝛻𝑑𝑑𝑠𝑠𝑡𝑡 = 𝑠𝑠𝑡𝑡 − 𝑠𝑠𝑡𝑡−𝑑𝑑 = 0

𝛻𝛻 applied to a trend function.

mt = a0 + a1 t

𝛻𝛻mt = mt - mt-1 = a0 + a1 t - (a0 + a1 (t-1))

= a1

Generally, 𝛻𝛻𝑘𝑘 applied to a polynomial of degree k gives a constant.

Differenced monthly accidental deaths.xt = xt - xt-12 , t = 13, . . . , 72.

(thou

sand

s)

1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7

-1.0

-0.5

0.0

0.5

∇12

Deseasonalized series still exhibits trend which we attempt to remove by differencing

Detrended and deseasonalized accidental deaths.

xt = xt - xt-1 - xt-12 + xt-13 , t = 14, . . . , 72.∇12∇

1 9 7 4 1 9 7 5 1 9 7 6 1 9 7 7 1 9 7 8 1 9 7

-500

050

010

00

Quantile regression

For heavytailed data it may sometimes be better to use quantile/median regression instead of least squares. On then proceeds exactly as in method 1, but uses quantile regression instead of least squares. References and some discussion may be found at https://en.wikipedia.org/wiki/Quantile_regression

State of the art seasonal adjustment program

X-13ARIMA-SEATS Seasonal Adjustment Program, available at https://www.census.gov/srd/www/x13as/

If γ(h) = Cov(Xt+h, Xt ) is the ACVF of a stationary time series Xt, then

• γ(0) ≥ 0,

• |γ(h)| ≤ γ(0)

• γ(h) = γ(−h) (i.e. γ(.) is an even function)

Proof: (i) follows from the fact that Var(Xt) ≥ 0. (ii) follows from the Cauchy-Schwarz inequality

|Cov(Xt+h, Xt)| ≤ Var(Xt+h)1/2 Var(Xt)1/2 = γ(0).

(iii) Follows from the fact that

γ(−h) = Cov(Xt-h, Xt ) = Cov(Xt+h, Xt ) = γ(h).

The ACVF of a stationary process

Non-negative definiteness: A real-valued function κ(h) defined on the integers is non-negative definite (nnd) iff

for all n and choices of vectors a = (a1, . . . , an)’ .

0)(1,

≥−κ∑=

j

n

jii ajia

Theorem: γ(h) is the ACVF of a/some stationary TS iff

1) γ(.) is an even function

2) γ(.) is nnd

Proof: 1) If γ(h) is the ACVF of a stationary TS Xt, then by previous result, γ(h) is even and with X = (X1, . . . , Xn)’, we have

0≤Var(a’X)=a’Cov(X,X)a = Σi,jaiCov(Xi,Xj)aj = Σi,jaiγ(i-j)aj

Now suppose γ(h) is even and nnd. We construct a stationary zero mean Gaussian TS with γ as its ACVF using Kolmogorov’s theorem. For t = (t1, . . . , tn)’ with t1< t2< … < tn define Ft as the multivariate normal distribution with mean zero and covariance matrix Σ𝑛𝑛= [γ(ti-tj)]i,j=1,…n. That is, the characteristic function associated with Ft is

φt(u)=exp(-u’ Σ𝑛𝑛u/2).

Since γ(h) is nnd, Kn is a nnd definite matrix and hence this is a legitimate characteristic function. The consistency condition in Kolmogorov’s theorem is equivalent to

which is easy to check holds, and this completes the proof.

))(()(lim )(0iiui

uu tt φ=φ→

For any given ACVF, we constructed a stationary Gaussian TS with the specified ACVF.

• It can be hard to verify that a given function γ(h) is an ACVF, by checking it is nnd. Instead, one might try to find a stationary TS with the designated ACVF.

Ex. Is γ(h) := cos (θh) an ACVF?

Ans. Yes. But verification of nnd property is difficult.

Easier to check that cos (θh) is the ACVF of

Xt = A cos (θh) + B sin (θh)

where A & B are uncorrelated (0,1) rv’s.

• The ACF ρ(h) has the same properties as γ(h) and ρ(0)=1.

Ex. The function

is an ACVF iff |ρ| ≤ .5.

To see this, if |ρ| ≤ .5, then γ is the ACVF of the MA(1) process with σ2=(1+θ2)-1 and θ=(2ρ)-1(1- sqrt(1-4ρ2)).

On the other hand, if |ρ| > .5, let a be the n-vector

(1,-1,1,-1,…)’. Then, for n large,

otherwise 1 if

0 if

,0,,1

)( ±==

ρ=γ h

hh

0)1(2,...)1,1,1(

100

01001001

,...)1,1,1(' <ρ−−=−

ρ

ρρρ

ρ

−= nnKn

aa

Observed data: x1 , . . . , xn

Sample mean :

Sample autocovariance function :

Sample autocorrelation function :

• Division by n instead of (n-h) ensures that sample cov

matrix is nnd.

∑=

−=n

ttxnx

1

1

0hfor ,))(()(ˆ1

1 ≥−−=γ ∑−

=+

−hn

thtt xxxxnh

)0(ˆ)(ˆ

)(ˆγγ

=ρhh

njin ji 1,)](ˆ[ˆ=−γ=Γ

The sample ACVF

• γ(h) is approximately the sample covariance function of (x1 , x1+h), . . . ,(xn-h , xn)

• If data are observations from IID noise, then

ρ(h) is approx N(0, n -1)

• For IID noise,

|ρ(h)| < 1.96 n -.5 with probability .95

Ex. 200 observations from IID N(0,1)).

0 5 0 1 0 0 1 5 0 2 0 0

-2-1

01

23

L a g

ACF

0 1 0 2 0 3 0 4 0

0.00.2

0.40.6

0.81.0

Sample ACF of WINE.TSM

L a g

ACF

0 1 0 2 0 3 0 4 0

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

Let residuals from the LS fit be denoted by

yt = xt - a0 - a1 t , t=1,...,98

L a g

ACF

0 1 0 2 0 3 0 4 0

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

ACF of residuals yt

A Model for the Lake Huron Data

Scatter plot of residuals (yt-1 , yt) showing regression line y = .791 x.

- 2 . 5 - 2 . 0 - 1 . 5 - 1 . 0 - 0 . 5 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5

-2.5

-1.5

-0.5

0.5

1.5

2.5

Model: Yt = .791 Yt-1 + Zt , Zt ~ WN(0,σ2) (Autoregressive or AR(1) model)