NonCausal Vector AR Processes with .1in Application to ...sistemas.fciencias.unam.mx/~ceactuaria/otros/... · is an iid sequence of continuous random vectors with mean 0 and covariance

NonCausal Vector AR Processes withApplication to Economic Time Series

Richard A. DavisColumbia University

(Joint work with Li Song)

2nd Congreso De ActuariaUNAM, Mexico City

January 24, 2013

Richard A. Davis (Columbia) 2nd Congreso De Actuaria January 24, 2013 1

Example: Walmart

log(Volume) of Walmart stock 12/1/03-12/31/04

Louvain 2011 2

1. Motivating Example (cont)

day

log-

volu

me

0 50 100 150 200 250

15.0

15.5

16.0

16.5

17.0

Log(volume) of Walmart stock 12/1/03-12/31/04


Example: Walmart (ACF/PACF)

Analysis of the ACF and PACF of the time series (n=274) suggests that {Xt } follows anAR(1) or AR(2). A causal AR(2) fit (using Gaussian MLE) is

Xt = .4455Xt−1 + .1025Xt−2 + Zt

The estimated residuals are uncorrelated but dependent as seen in the plots of the ACF ofthe absolute values and squares of the residuals

⇒ residuals follow an allpass model

Louvain 2011 14

7. Walmart revisited---residuals from noncausal model

Analysis of the ACF and PACF of the time series (n=274) suggests that {Xt} follows an AR (1) or AR(2).

A causal AR(2) fit (using Gaussian MLE) is

Xt .4455 Xt-1 .1025 Xt-2 Zt

The estimated residuals were uncorrelated but dependent.

lag (h)

acf o

f abs

val

ues

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

lag (h)

acf o

f squ

ares

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0


Example: Walmart (noncausal)

Maximum-likelihood model:

Xt = −2.0766Xt−1 + 2.0772Xt−2 + Zt (purely noncausal)

Zt ∼ IID Stable

α = 1.8335, β = .5650, σ = .4559, µ = 16.0030

Louvain 2011

lag (h)

acf o

f abs

val

ues

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

lag (h)

acf o

f squ

ares

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

CB’s simulatedCB’s simulated


Outline

Warm-up example with Walmart

Univariate AR models: causal and noncausal

Multivariate AR models: causal and noncausal

• Preliminaries

• Identifiability and likelihood function.

• Asymptotic properties of the estimators

• Examples

* term structure of interest rates

* fiscal foresight

Summary


AR Model:

The Model: Assume {Xt } is the stationary solution to the recursions

Xt = φ1Xt−1 + · · ·+ φpXt−p + Zt {Zt } ∼ IID(0, σ2)

(1 − φ1B − · · · − φpBp)Xt = Zt (B is backward shift operator)

φc(B)φnc(B)Xt = Zt ,

where φc(z) and φnc(z) are the respective causal and noncausal polynomials of the ARpolynomial, i.e.,

φc(z) causal means φc(z) , 0 for |z| ≤ 1.

φnc(z) noncausal means φc(z) , 0 for |z| > 1.

The process has the two-sided representation (one sided if the degrees of φc or φnc are 0)

Xt =∞∑

j=−∞

ψjZt−j


Univariate AR models: causal and noncausal models

Louvain 2011 10

4. AR models—causal and noncausal

Zt

Xt

Impulse response: causal & low frequency

jt-j

jt

ZX



Louvain 2011 11


Impulse response: noncausal & high frequency

Zt

Xt

jt-j

jt

ZX



Louvain 2011 12


Impulse response: mixed causal (low frequency) & noncausal (high frequency)

Zt

Xt

jt-j

jt

ZX


An Example: Muddy River–tributary to Sun River in Central Montana

Muddy Creek: surveyed every 15.24 meters, total of 5456m; 358 measurements


An Example: Muddy River–tributary to Sun River in Central Montana

Muddy Creek: surveyed every 15.24 meters, total of 5456m; 358 measurements

Best fitting (minimum AICC) ARMA model to residuals (after removal of quadratic trend):

Yt = .574Yt−1 + Zt − .311Zt−1, {Zt } ∼ WN(0, .0564)

.Residuals follow an allpass model AP(1), which suggests a noncausal model:

Noncausal ARMA(1,1) model:

Yt = 1.743Yt−1 + Zt − .311Zt−1, {Zt } ∼ WN(0, .0564)

.


Noncausal VAR

A multivariate time series {Xt } is a VAR(p) process if it is stationary and satisfies therecursions

Φ(B)Xt = Xt −Φ1Xt−1 − · · · −ΦpXt−p = Zt ,

where Xt := (Xt ,1, . . . ,Xt ,m)T is an m-dimensional stochastic process, Zt := (Zt ,1, . . . ,Zt ,m)T

is an iid sequence of continuous random vectors with mean 0 and covariance matrix Σ∗0.

Causal: All the roots of detΦ(z) are outside the unit circle.

Purely noncausal: All the roots of detΦ(z) are inside the unit circle.

Mixed: detΦ(z) has roots both inside and outside the unit circle.

Univariate vs multivariate: A multivariate AR(1) can be mixed!


Identifiability

A general VAR(p) model given by

Φ(B)Xt = Xt −Φ1Xt−1 − · · · −ΦpXt−p = Zt ,

may have zeros of detΦ(z) = det(I −Φ1z − · · · −Φpzp) that are both inside andoutside the unit circle.

Hannan (1970) showed that the spectral density matrix of Xt always has arepresentation which is the spectral density matrix of a causal VAR(p) process.

Noncausal models cannot be distinguished from causal models in the Gaussian case.

Adapting Theorem 1 from Chan and Tong (2006) on reversibility of vector-valued linearprocesses, one can show identifiability of possibly noncausal VAR models undersuitable conditions on the distribution of Zt .


Identifiability

From Chan and Tong (2006), we have

Lemma

Let {Xt } and {Xt } be two nonGaussian m-dimensional linear processes defined by

Xt =∞∑

j=−∞

CjZt−j and Xt =∞∑

j=−∞

Cj Zt−j ,

where

Cj and Cj square-summable

{Zt } ∼ IID(0,Σ) and {Zt } ∼ IID(0, Σ)

Then under some technical conditions on the distribution of Zt ,

{Xt }d= {Xt }

if and only if there exist an integer q and a matrix H such that for all t,

Zt−qd= HZt and Ct−q = Ct H . (1)


State-space representation

As is well known, a VAR(p) model can be re-expressed as a VAR(1) model.

Define two new processes {Yt } and Z∗t by

Yt =

Xt

Xt−1

...Xt−p+1

pm×1

and Z∗t =

Zt

0...0

pm×1

.

ThenYt = ΦY Yt−1 + Z∗t , (2)

where

ΦY =

Φ1 Φ2 · · · · · · Φp

Im Om · · · · · · Om

Om Im. . .

......

. . .. . .

...Om Om · · · Im Om

pm×pm

,

and Om is the m ×m matrix of zeros.



Zeros of det Φ(z) correspond to the reciprocals of the eigenvalues of ΦY .

causal(noncausal) roots of det Φ(z) correspond to eigenvalues of ΦY that areinside(outside) the unit circle.

Jordan canonical form of ΦY implies there exists a pm × pm invertible matrix A suchthat ΦY A = AJ, where

J =

λ1

s1. . .

. . .. . .

sl−1 λl

0 λl+1

sl+1. . .

. . .. . .

spm−1 λpm

,

and |λ1| ≤ |λ2| ≤ · · · ≤ |λl | < 1 < |λl+1| ≤ · · · ≤ |λpm |, {si} are 0 or 1



From Yt = ΦY Yt−1 + Z∗t and the fact that ΦY = AJA−1 we have

(I − JB)A−1Yt := (I − JB)Yt = A−1Z∗t := Zt

Since J is block diagonal (corresponding to the causal and noncausal components),we decompose Yt and Zt into the first l and last pm − l components, i.e.,

Yt = (YTt ,1, Y

Tt ,2)T and Zt = (ZT

t ,1, ZTt ,2)T

and obtain Yt ,1 − J1Yt−1,1 = Zt ,1

Yt ,2 − J2Yt−1,2 = Zt ,2

This implies Yt ,1 is a purely causal AR process and Yt ,2 is a purely noncausal ARprocess.



Representation for Yt :

Yt =∞∑

i=−∞

AFiA−1Z∗t−i ,

where

Fi =

Ji1

Opm−l

, i ≥ 0, Ol

−Ji2

, i ≤ −1.

Representation for Xt : Let Mi be the upper-left sub-matrix of AFiA−1, then

Xt =∞∑

i=−∞

MiZt−i .

Impulse response coefficients: {Mi} or {MiPL } where PL is the lower triangular matrix inthe Cholesky decomposition of Σ∗0.


Impulse response coefficients

Model (Xt ,1

Xt ,2

)=

(0.8 0.60.6 1.7

) (Xt−1,1

Xt−1,2

)+

(Zt ,1

Zt ,2

),

for which det(I −Φ1z) has zeros at z = .5 and z = 2. The corresponding component series{Yt ,1} and {Yt ,2} (found from Yt = A−1Xt ) are univariate noncausal and causal AR(1)processes given by

Yt ,1 = .5Yt−1,1 + Zt ,1

Yt ,2 = 2Yt−1,2 + Zt ,2

where {(Zt ,1, Zt ,2)′} is an iid sequence.


Impulse response coefficients(Xt ,1

Xt ,2

)=

(0.8 0.60.6 1.7

) (Xt−1,1

Xt−1,2

)+

(Zt ,1

Zt ,2

).

−10 −5 0 5 10−0.4

−0.2

0

0.2

0.4

0.6

0.8M11

−10 −5 0 5 10−0.4

−0.2

0

0.2

0.4

0.6

0.8M12

−10 −5 0 5 10−0.4

−0.2

0

0.2

0.4

0.6

0.8M21

−10 −5 0 5 10−0.4

−0.2

0

0.2

0.4

0.6

0.8M22


Non-Gaussian distributions and parameters

We consider a general elliptical class of distributions for Zt which has a density of theform fZ (z; ν) = det(Σ)−1/2f(zT Σ−1z; ν). Multivariate t-distribution is also included in thisclass.

Parameters

φ =

φ1

φ2

...φpm2

=

vec(Φ1)vec(Φ2)

...vec(Φp)

and σ = vech(Σ).

Let θ be the vector that contains all the unknown parameters, namely

θ =

φσν

.The state-space representation of the VAR(p) process described earlier plays a keyrole in deriving the likelihood function.


Non-Gaussian likelihood function

A complete likelihood function of Xt is given by

ln(θ) = log(L(X1, . . . ,Xn))

= log(p1(Yp,1)p2(Yn,2) · | det(A)|−1) +n∑

i=p+1

(log fZ (Zi(φ); ν) + κ(φ)) ,

where Yp,1 only depends on {Z−∞, . . . ,Zp}, Yn,2 only depends on {Zn+1, . . . ,Z∞} and Aonly depends on {Φ1, . . . ,Φp}.

The function κ(φ) is the reciprocal of the product of all roots of detΦ(z) = 0 that areinside the unit circle.

This complete likelihood can be approximated by

ln(θ) =n∑

i=p+1

(log fZ (Zi(φ); ν) + κ(φ)) :=n∑

i=p+1

gi(θ).


Asymptotic behavior of the score

Theorem 2:

Suppose that Zt has a nonGaussian density function and satisfying some regularity condi-tions. Then,

1√

n − p∂ln(θ0)

∂θ=

1√

n − p

n∑i=p+1

∂gi(θ0)

∂θ

d→ N(0,Iθθ(θ0)),

where the matrix Iθθ(θ0) is given by −E(∂2gp+1(θ0)

∂θ∂θT

)and is positive definite.


Technical complements

Identifiability of θ0 and consistency:The independence of Yp,1 and Yn,2 from Zp+1(φ0), . . . ,Zn(φ0) implies

ln(θ) = log(L(X1, . . . ,Xn))

= log(p1(Yp,1)p2(Yn,2) · | det(A)|−1) +n∑

i=p+1

(log fZ (Zi(φ); ν) + κ(φ)) ,

is also a complete likelihood for any choice of initial distributions p1(·) and p2(·) for Yp,1 andYn,2.

In fact, one can choose p1(·) and p2(·) independent of θ, and for such a complete likelihoodwe will have the following properties:

E ∂ln(θ0)

∂θ

= 0,

where E(·) denotes the expectation under the measure with the new choices of p1(·) andp2(·).


Technical complements (cont)

It follows that:

0 =1

n − pE

∂ln(θ0)

∂θ

=

E(∂ log(p1(Yp,1)p2(Yn,2) · | det(A)|−1)/∂θ

)n − p

+1

n − pE

(∂ln(θ0)

∂θ

)=

1n − p

n∑i=p+1

E(∂gi(θ0)

∂θ

).

But using an asymptotic stationarity argument (i.e., using stationarity of {∂gi(θ0)/∂θ} underthe stationary measure), we obtain the following relationship,

E(∂gi(θ0)

∂θ

)= lim

n→∞E

(∂gbn/2c(θ0)

∂θ

)= lim

n→∞E

(∂gbn/2c(θ0)

∂θ

)= lim

n→∞

1n − p

n∑i=p+1

E(∂gi(θ0)

∂θ

)= 0, (3)

which is the key result for identifiability and consistency.


Simulation example

Recall the mixed model given by(Xt ,1

Xt ,2

)=

(0.8 0.60.6 1.7

) (Xt−1,1

Xt−1,2

)+

(Zt ,1

Zt ,2

).

We simulated the {Xt } process based on bivariate t noise {Zt } with Σ0 = I2 and ν = 6.


Simulation results

True value Sample size 100 Sample size 200 Sample size 500 Sample size 1000φ1 0.8 0.8435 (0.3851) 0.8220 (0.2394) 0.8068 (0.1345) 0.8026 (0.0932)

(0.2985) (0.2106) (0.1330) (0.0940)φ2 0.6 0.5274 (0.2612) 0.5883 (0.1338) 0.5992 (0.0665) 0.5989 (0.0456)

(0.1487) (0.1049) (0.0662) (0.0468)φ3 0.6 0.5359 (0.5491) 0.5938 (0.3695) 0.5988 (0.2102) 0.5989 (0.1444)

(0.4614) (0.3254) (0.2055) (0.1452)φ4 1.7 1.6297 (0.5576) 1.6987 (0.3228) 1.7067 (0.1693) 1.7021 (0.1186)

(0.3839) (0.2708) (0.1710) (0.1029)σ1 1 1.1382 (0.5579) 1.0561 (0.2693) 1.0159 (0.1421) 1.0075 (0.0987)

(0.3115) (0.2197) (0.1387) (0.0980)σ2 0 -0.0663 (0.3529) -0.0115 (0.2060) -0.0024 (0.1130) -0.0024 (0.0770)

(0.2444) (0.1724) (0.1089) (0.0769)σ3 1 1.0226 (0.5093) 1.0264 (0.3137) 1.0108 (0.1760) 1.0040 (0.1233)

(0.4063) (0.2866) (0.1810) (0.1279)ν 6 6.7925 (3.5194) 6.4947 (2.2386) 6.1386 (1.1061) 6.0816 (0.7712)

(2.4436) (1.7235) (1.0884) (0.7692)

Table: The true and empirical mean of the MLE’s of φ1, φ2, φ3, φ4, σ1, σ2, σ3 and ν. The empiricalstandard errors are given in (· · · ) theoretical standard errors are given in (· · · ).


Term structure of interest rates

The raw data (1970:1 to 1998:4 [116 observations]) are taken from the website ofGregory Duffee.

The bivariate time series is constructed by

• ∆r = Change in the three-month interest rate (quarter-end yields on U.S.zero-coupon bonds);

• S = Spread between the ten-year and three-month interest rates.


Time series plot

Change in three−month interest rate

Time

V1

0 20 40 60 80 100 120

−6

−4

−2

02

Spread between the ten−year and three−month interest rates

Time

V2

0 20 40 60 80 100 120

−4

−2

02


ACF of residuals from VAR(3) causal fit

0 5 10 15

−0.

50.

00.

51.

0

Lag

AC

F

V1

0 5 10 15

−0.

50.

00.

51.

0

Lag

V1 & V2

−15 −10 −5 0

−0.

50.

00.

51.

0

Lag

AC

F

V2 & V1

0 5 10 15

−0.

50.

00.

51.

0

Lag

V2


ACF of squares of residuals from VAR(3) causal fit

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V1 & V2

−15 −10 −5 0

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V2 & V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V2


Noncausal fitted models

Model assumption CG CN PNN MXLog-likelihood -256.624 -237.345 -235.481 -229.054

Table: Comparison of log-likelihood.

The following table summarizes the results of the best fitted VAR(3) model of the data. Itturns out the detΦ(z) = 0 corresponding to the best fit has only one root inside the unitcircle and five roots outside the unit circle.

Φ1 Φ2 Φ3 Σ ν

0.789 0.009 0.434 0.110 0.728 -0.268 0.774 -0.448 2.806(0.289) (0.255) (0.202) (0.327) (0.128) (0.232) (0.245) (0.145) (0.715)-0.548 0.818 -0.209 0.014 -0.552 0.136 -0.448 0.393(0.170) (0.174) (0.128) (0.240) (0.079) (0.175) (0.145) (0.106)

Table: The MLE’s of the parameters and their associated standard errors for the interest rate data.


ACF of residuals from noncausal fit

0 5 10 15

−0.

50.

00.

51.

0

Lag

AC

F

V1

0 5 10 15

−0.

50.

00.

51.

0

Lag

V1 & V2

−15 −10 −5 0

−0.

50.

00.

51.

0

Lag

AC

F

V2 & V1

0 5 10 15

−0.

50.

00.

51.

0

Lag

V2


ACF of squares of residuals from noncausal fit

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V1 & V2

−15 −10 −5 0

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V2 & V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V2


Impulse response coefficientsTerm structure of interest rates

−20 −10 0 10 20

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5M11

−20 −10 0 10 20

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5M12

−20 −10 0 10 20

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5M21

−20 −10 0 10 20

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5M22


Fiscal foresight

Fiscal foresight (see Lanne and Saikkonen (2009)) is the notion that agents receivesignals in advance about prospective changes fiscal policy issues that may anticipatemovements in some macro economic time series.

Data are quarterly from 1955:1 to 2000:4 [184 observations] and are taken from theNational Income and Product Accounts (NIPA) tables.

All the components of national income are in real per capita terms and are transformedfrom their nominal values by dividing them by the gdp deflator and the populationmeasure.The trivariate time series is constructed by the differences of the GDP, the TotalGovernment Expenditure and the Total Government Revenue. More specifically,

1 GDP = Quarterly U.S. GDP data;2 Total Government Expenditure = Federal Defense Consumption Expenditures + Federal

Non Defense Consumption Expenditures + State and Local Consumption Expenditures +Federal Defense Gross Investment + Federal Non Defense Gross Investment + State andLocal Gross Investment;

3 Total Government Revenue = Total Government Receipts - Net Transfers Payments - NetInterest Paid.


Time series plot

Differenced US GDP

Time

V1

0 50 100 150

−5

05

Total government expenditure

Time

V2

0 50 100 150

−1.

00.

01.

0

Total government revenue

Time

V3

0 50 100 150

−6

−2

02

46


ACF of squares of residuals from VAR(2) causal fit

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V1 & V2

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V1 & V3

−15 −10 −5 0

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V2 & V1

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V2

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V2 & V3

−15 −10 −5 0

−0.

20.

00.

20.

40.

60.

81.

0

Lag

AC

F

V3 & V1

−15 −10 −5 0

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V3 & V2

0 5 10 15

−0.

20.

00.

20.

40.

60.

81.

0

Lag

V3


Noncausal fitted models

Model assumption CG CN PNN MXLog-likelihood -819.230 -802.865 -800.355 -791.270

Table: Comparison of log-likelihood.

The following table summarizes the results of the best fitted VAR(2) model of the data. Itturns out the detΦ(z) = 0 corresponding to the best fit has three roots inside the unit circleand three roots outside the unit circle.

Φ1 Φ2 Σ ν

3.516 3.710 -3.799 1.789 -6.268 -1.162 49.108 -12.746 -14.666(1.226) (3.631) (1.527) (0.924) (2.946) (0.673) (9.654) (5.728) (7.215)-0.512 -3.104 0.679 -0.586 2.658 0.295 -12.746 5.135 4.019 5.751(0.456) (1.267) (0.625) (0.311) (1.081) (0.312) (5.728) (3.373) (3.494) (1.399)-1.099 -1.811 2.918 -0.505 1.149 1.257 -14.666 4.019 9.376(0.475) (1.889) (0.762) (0.435) (1.534) (0.314) (7.215) (3.494) (5.203)

Table: The MLE’s of the parameters and their associated standard errors for the GDP data.


ACF of squares of residuals from noncausal fit

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

V1

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V1 & V2

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V1 & V3

−15 −10 −5 0

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

V2 & V1

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V2

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V2 & V3

−15 −10 −5 0

0.0

0.2

0.4

0.6

0.8

1.0

Lag

AC

F

V3 & V1

−15 −10 −5 0

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V3 & V2

0 5 10 15

0.0

0.2

0.4

0.6

0.8

1.0

Lag

V3


Impulse response coefficientsFiscal foresight

−10 −5 0 5 10

−1

−0.5

0

0.5

M11

−10 −5 0 5 10

−1

−0.5

0

0.5

M12

−10 −5 0 5 10

−1

−0.5

0

0.5

M13

−10 −5 0 5 10

−1

−0.5

0

0.5

M21

−10 −5 0 5 10

−1

−0.5

0

0.5

M22

−10 −5 0 5 10

−1

−0.5

0

0.5

M23

−10 −5 0 5 10

−1

−0.5

0

0.5

M31

−10 −5 0 5 10

−1

−0.5

0

0.5

M32

−10 −5 0 5 10

−1

−0.5

0

0.5

M33


Summary

We have developed a state-space representation for VAR models that can handle bothcausal and noncausal cases.

Developed a likelihood procedure for estimation of parameters in a possibly noncausalVAR framework.

Noncausal models may be useful in producing more independent looking residuals.

Noncausal models may provide additional insight (and creative explanations!) into theunderlying dynamics of a multivariate time series that are not otherwise detected viacausal models.


Documents

NonCausal Vector AR Processes with .1in Application to ...sistemas.fciencias.unam.mx/~ceactuaria/otros/... · is an iid sequence of continuous random vectors with mean 0 and covariance