Parameter estimation using third-order cumulants for INAR(p) processes

I. Silva and M. E. Silva Parameter estimation using third-order cumulants for INAR(p) processes

Parameter estimation using third-ordercumulants for INAR(p) processes

Isabel Silva1 Maria Eduarda Silva2,3

1Departamento de Engenharia Civil and CEC, Faculdade de Engenharia da Universidade do Porto

2Faculdade de Economia da Universidade do Porto

3Unidade de Investigao Matemtica e Aplicaes (UIMA), Universidade de Aveiro

Workshop on Integer-valued Time Series Modelling (WINTS09)

WINTS09 1 / 17


Contents

Introduction High-order statistics INteger-valued AutoRegressive (INAR) process

Parameter estimation based on third-order cumulants Cumulant third-order characterization of INAR(p) processes Estimation using cumulant Third-Order Recursion equation

Monte Carlo results and application to real data

Final remarks

WINTS09 2 / 17


High-Order Statistics (HOS)Moments and cumulants of order higher than two

Introduction WINTS09 3 / 17



Lack of Gaussianity and/or non-linearity




Lack of Gaussianity and/or non-linearity

Notation:{Xt} : kth-order stationary stochastic process

X(s1, . . . ,sk1) : kth-order joint moment of Xt,Xt+s1 , . . . , Xt+sk1function of k1 variables (s1, . . . ,sk1 R)

X = E[Xt], X(s1, . . . ,sk1) = E[XtXt+s1 . . .Xt+sk1 ]

CX(s1, . . . ,sk1) : kth-order joint cumulant of Xt,Xt+s1 , . . . , Xt+sk1function of k1 variables (s1, . . . ,sk1 R)

the coefficient of 12 . . .k in the Taylor series expansion (about (0, . . . ,0)) ofthe cumulant generating function of Xt,Xt+s1 , . . . , Xt+sk1



High-Order Statistics (HOS)Relations between joint moments and joint cumulants [Leonov and Shiryaev (1959)]CX = E[Xt] = XCX(k) = X(k)X2 = R(k), k ZCX(k,m) = X(k,m)X

(X(k)+ X(m)+ X(km)

)+2X3, k,m Z



High-Order Statistics (HOS)Relations between joint moments and joint cumulants [Leonov and Shiryaev (1959)]CX = E[Xt] = XCX(k) = X(k)X2 = R(k), k ZCX(k,m) = X(k,m)X

(X(k)+ X(m)+ X(km)

)+2X3, k,m Z

Symmetry properties [Mendel (1991)]

CX(m) = CX(m), m > 0CX(m,n) = CX(n,m) = CX(n,mn) = CX(nm,m), m,n > 0

n

m

0



INteger-valued AutoRegressive processes

INAR(p) [Latour (1998)]

Xt = 1 Xt1 +2 Xt2 + +p Xtp + et





Xt = 1 Xt1 +2 Xt2 + +p Xtp + et

0 i < 1, i = 1, . . . ,p1, and 0 < p < 1, such that pk=1 k < 1 thinning operation [Steutel and Van Harn (1979); Gauthier and Latour (1994)]

i Xti = Xtij=1 Yi,j, for i = 1, . . . ,p{Yi,j} (counting series): set of i.i.d. non-negative integer-valued r.v. withE[Yi,j] = i,Var[Yi,j] = 2i and E[Y3i,j] = i{et} (innovation process): sequence of i.i.d. non-negative integer-valued r.v.(independent of {Yi,j}) with E[et] = e,Var[et] = 2e and E[e3t ] = e





Xt = 1 Xt1 +2 Xt2 + +p Xtp + et

0 i < 1, i = 1, . . . ,p1, and 0 < p < 1, such that pk=1 k < 1 thinning operation [Steutel and Van Harn (1979); Gauthier and Latour (1994)]

i Xti = Xtij=1 Yi,j, for i = 1, . . . ,p{Yi,j} (counting series): set of i.i.d. non-negative integer-valued r.v. withE[Yi,j] = i,Var[Yi,j] = 2i and E[Y3i,j] = i{et} (innovation process): sequence of i.i.d. non-negative integer-valued r.v.(independent of {Yi,j}) with E[et] = e,Var[et] = 2e and E[e3t ] = e

Usually: Poisson INAR(p) process with binomial thinning operation



Cumulant third-order characterization of INAR(p) processes

[Silva and Oliveira (2004, 2005) and Silva (2005)]CX(0,0) = pi=1 pj=1 pk=1 ijk X(i j, i k)+3pi=1 pj=1 ji2X(i j)+ e

+3(eX)pi=1 pj=1 ijX(i j)+3X(eX)pi=1 i2 +23X6eX2 pi=1 i3e(e2 +2e )+ X pi=1 (i3ii23i )

CX(0,k) = pi=1 iCX(0,k i), k > 0

CX(k,k) = pi=1 pj=1 ijCX(k i,k j)+pi=1 i2CX(k i), k > 0

CX(k,m) = pi=1 iCX(k,m i), m > k > 0

Parameter estimation based on third-order cumulants WINTS09 6 / 17



[Silva and Oliveira (2004, 2005) and Silva (2005)]CX(0,0) = pi=1 pj=1 pk=1 ijk X(i j, i k)+3pi=1 pj=1 ji2X(i j)+ e

+3(eX)pi=1 pj=1 ijX(i j)+3X(eX)pi=1 i2 +23X6eX2 pi=1 i3e(e2 +2e )+ X pi=1 (i3ii23i )

CX(0,k) = pi=1 iCX(0,k i), k > 0

CX(k,k) = pi=1 pj=1 ijCX(k i,k j)+pi=1 i2CX(k i), k > 0

CX(k,m) = pi=1 iCX(k,m i), m > k > 0

INAR processes have a non-linear structure

1st and 2nd order cumulants are not sufficient to describe dependence structure




Cumulant Third-Order Recursion (TOR) equation [Silva (2005)]

CX(k,m)p

i=1

iCX(i k, im) = (k)p

i=1

i2CX(im), 0 k m,m 6= 0

where (a) ={

1 if a = 0,0 otherwise,

is the Kronecker delta function




Cumulant Third-Order Recursion (TOR) equation [Silva (2005)]

CX(k,m)p

i=1

iCX(i k, im) = (k)p

i=1

i2CX(im), 0 k m,m 6= 0

where (a) ={

1 if a = 0,0 otherwise,

is the Kronecker delta function

~w CX(k,k) = CX(0,k)

C3,X =

CX(0,0) CX(1,1) CX(p1,p1)CX(0,1) CX(0,0) CX(p2,p2)

.

.

.

.

.

.

.

.

.

.

.

.

CX(0,p1) CX(0,p2) CX(0,0)

1

2.

.

.

p

=

CX(0,1)CX(0,2)

.

.

.

CX(0,p)

= c3,X



Estimation using cumulant TOR equation

{X1, . . . ,XM,XM+1, . . . ,X2M, . . . ,X(B1)(M+1), . . . ,XN=BM}




{X1, . . . ,XM 1st block

,XM+1, . . . ,X2M 2nd block

, . . . ,X(B1)(M+1), . . . ,XN=BM Bth block

}

B blocks of M observations each

{X(1)1 , . . . ,X(1)M

1st block

,X(2)1 , . . . ,X(2)M

2nd block

, . . . ,X(B)1 , . . . ,X(B)M

Bth block

}




{X1, . . . ,XM 1st block

,XM+1, . . . ,X2M 2nd block

, . . . ,X(B1)(M+1), . . . ,XN=BM Bth block

}

B blocks of M observations each

{X(1)1 , . . . ,X(1)M

1st block

,X(2)1 , . . . ,X(2)M

2nd block

, . . . ,X(B)1 , . . . ,X(B)M

Bth block

}

For each block: X(i) = 1M

M

j=1

X(i)j

C(i)X (k,k) =1M

Mkj=1

(X(i)j X

(i))(

X(i)j+kX(i))2

, k = 0, . . . ,p1

C(i)X (0,k) =1M

Mkj=1

(X(i)j X

(i))2(

X(i)j+kX(i)), k = 1, . . . ,p




Overall third-order cumulant estimators:

CX(k,k) =1B

B

i=1

C(i)X (k,k), k = 0, . . . ,p1

CX(0,k) =1B

B

i=1

C(i)X (0,k), k = 1, . . . ,p





CX(k,k) =1B

B

i=1

C(i)X (k,k), k = 0, . . . ,p1

CX(0,k) =1B

B

i=1

C(i)X (0,k), k = 1, . . . ,p

To solve the system of linear equations in order to the coefficients


.

.

.

.

.

.

.

.

.

.

.

.

CX(0,p1) CX(0,p2) CX(0,0)

1

2.

.

.

p

=

CX(0,1)CX(0,2)

.

.

.

CX(0,p)





CX(k,k) =1B

B

i=1

C(i)X (k,k), k = 0, . . . ,p1

CX(0,k) =1B

B

i=1

C(i)X (0,k), k = 1, . . . ,p

To solve the system of linear equations in order to the coefficients


.

.

.

.

.

.

.

.

.

.

.

.

CX(0,p1) CX(0,p2) CX(0,0)

1

2.

.

.

p

=

CX(0,1)CX(0,2)

.

.

.

CX(0,p)

e = X(

1p

i=1

i

), 2e = VpX

p

i=1

2i , Vp = R(0)p

i=1

i R(i)



Monte Carlo results

To examine the small sample properties of the proposed estimation method

To compare its performance with other methods: YW, CLS, WHT and LS_HOS

Monte Carlo results and application to real data WINTS09 10 / 17


Monte Carlo results



1000 realizations of Poisson INAR(p) processes, with binomial thinning operation,for several orders, sample sizes and parameter values



Monte Carlo results




Sample properties of the cumulant TOR estimatorThe sample bias, variance and mean square error decrease as the sample size (ofthe block) increases Distribution is consistent and symmetric



Monte Carlo results




Sample properties of the cumulant TOR estimatorThe sample bias, variance and mean square error decrease as the sample size (ofthe block) increases Distribution is consistent and symmetricFor small sample size: evidence of departure from symmetry in the marginaldistributions, specially for values of the parameter near the non-stationary region



Monte Carlo results





Coefficients underestimated; innovation parameter overestimated



Monte Carlo results





Coefficients underestimated; innovation parameter overestimated

Fixed M statistics measures decrease as B increases and vice-versa



Monte Carlo results

TOR_1B TOR_2B TOR_4B TOR_1B TOR_2B TOR_4B2

1

0

1

2Bi

as(

1)


1

0

1

2

Bias

(2)


0

7

Bias

()N=60 N=200

Figure: Boxplots of the sample bias for the estimates obtained in 1000 realizations of 60 and 500 observations of theINAR(2) model: Xt = 0.6Xt1 +0.1Xt2 + et, where et Po(1).



Monte Carlo results

WHT CLS LS_HOS YW TOR_1B WHT CLS LS_HOS YW TOR_1B0.8

0.4

0

0.4

0.8Bi

as(

)

WHT CLS LS_HOS YW TOR_1B WHT CLS LS_HOS YW TOR_1B5

2.5

0

2.5

5

Bias

()(, )=(0.4, 3.0) (, )=(0.9, 3.0)

Figure: Boxplots of the sample bias for the estimates obtained in 1000 realizations of 200 observations of the INAR(1)models: Xt = 0.4Xt1 + et and Xt = 0.9Xt1 + et, where et Po(1).



Application to real data

1920 1930 1940 1950 1960 1970 19815

10

15

20

25

30

35

40

45

50

Num

ber o

f pla

nts

Figure: The number of Swedish mechanical paper and pulp mills, from 1921 to 1981 [Brnns (1995) and Brnns andHellstrm (2001)]



Application to real dataSimple INAR(1)It is not assumed the Poisson distribution for the innovation process:

X = 20.40 and S2 = 155.16




X = 20.40 and S2 = 155.16

Method e 2e x 2x MSECLS 0.9591 0.2017 15.2268 4.9279 192.2764 9.3254

LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224




X = 20.40 and S2 = 155.16


LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224

Mean and variance of the estimated models: x =e

1 and 2x =

(1 )(e + 2e )(1 )2(1+ )




X = 20.40 and S2 = 155.16


LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224

Mean and variance of the estimated models: x =e

1 and 2x =

(1 )(e + 2e )(1 )2(1+ )

MSE between the observations and the fitted models based on TOR_1B, LS_HOS andCLS estimates



Application to real data

1920 1930 1940 1950 1960 1970 19815

10

15

20

25

30

35

40

45

50

years

Num

ber o

f pla

nts

real dataCLSTOR_1B

Figure: The number of plants and the fitted values considering the TOR_1B and CLS estimatesMonte Carlo results and application to real data WINTS09 15 / 17


Final remarks

Advantage of HOS: capability to detect and characterize the deviations fromGaussianity and non-linearity of the processes

Final remarks WINTS09 16 / 17


Final remarks


INAR processes are non-Gaussian

Parameter estimation method: Estimation using cumulant TOR equation



Final remarks




The method does not assume any particular discrete distribution for the countingseries and for the innovation process



Final remarks





Monte Carlo results: cumulant TOR estimates provides acceptable results, interms of sample bias, variance and mean square error



Final remarks





Monte Carlo results: cumulant TOR estimates provides acceptable results, interms of sample bias, variance and mean square error

When used in the context of a non-Poisson real dataset the estimates that useHOS information provide a model with mean, variance and autocorrelationscloser to the sample values



References

BRNNS, K. (1995).Explanatory Variables in the AR(1) Count Data Model.Ume Economic Studies 381.

BRNNS, K. and HELLSTRM, J. (2001).Generalized Integer-Valued Autoregression.Econometric Reviews 20 (4), 425-443.

GAUTHIER, G. and LATOUR, A. (1994).Convergence forte des estimateurs des paramtres dunprocessus GENAR(p).Annales des Sciences Mathmatiques du Qubec 18,49-71.

LATOUR, A. (1998).Existence and stochastic structure of a non-negativeinteger-valued autoregressive process.Journal of Time Series Analysis 19, 439-455.LEONOV, V.P. and SHIRYAEV, A.N. (1959).On a method of calculation of semi-invariants.Theory of Probability and its Applications (translated byBrown, J.R.) IV, pp. 319-329.

MENDEL, J. M. (1991).Tutorial on higher-order statistics (spectra) in signalprocessing and system theory: theoretical results and someapplicationsProceedings of the IEEE 79, pp. 278-305.SILVA, I. (2005).Contributions to the analysis of discrete-valued timeseries.PhD Thesis. Universidade do Porto, Portugal.

SILVA, M. E. and OLIVEIRA, V. L. (2004).Difference equations for the higher-order moments andcumulants of the INAR(1) model.Journal of Time Series Analysis 25, 317-333.SILVA, M. E. and OLIVEIRA, V. L. (2005).Difference equations for the higher-order moments andcumulants of the INAR(p) model.Journal of Time Series Analysis 26, 17-36.STEUTEL, F. W. and VAN HARN, K. (1979).Discrete analogues of self-decomposability and stability.The Annals of Probability 7, 893-899.

References WINTS09 17 / 17

Documents

Parameter estimation using third-order cumulants for INAR(p) processes