Pad& Approximation and its Application in Time Series Analysis

Kuldeep Kumar

Department of Ecorwmics and Statistics

National University of Singapore

ABSTRACT

Since the appearance in 1970 of the book by Box and Jenkins, the use of the

auto regressive-moving average (ARMA) model has become widespread in many

fields for the analysis and prediction of time series data. However, one of the main

criticisms of these ARMA models is that they are difficult to specify. In this paper

we have proposed a new method based on the theory of Pad6 approximations for

the estimation of parameters and specification of the order of ARMA( p, 9) models.

Simulation results and results obtained from applying this method to real data sets

substantiate our claim for using this method.

1. INTRODUCTION

Since the 1970 work by Box and Jenkins [3], auto regressive-moving average (ARMA) models have become popular and important tools in the modeling and forecasting of time series data. These models are commonly

referred as Box- Jenkins models and the whole approach is usually referred to as the Box- Jenkins approach. Auto regressive moving average models of

order ( p, q) (abbreviated as ARMA( p, 9)) for an univariate time series 2, can be written as

Zt-f#JlZ,_l... -c$,Z,_~=~,-19,a,_,... -8,a,_, (1.1)

or d(B)Z,=O(B)a,

where a, is a sequence of independently and identically distributed random variables, commonly referred to as white noise, with mean zero

APPLIED MATHEMATICS AND COMPUTATZON 48:139-151 (1992) 139

0 Elsevier Science Publishing Co., Inc., 1992

655 Avenue of the Americas, New York, NY 10010 0096-3003/92/$05.00

140 KULDEEP KUMAR

and variance u,“.

4(R)=I-&B- *** -+rBP and O(B)=l-O,B- ... -0,Bq

where B is the backward shift operator such that BZ, = Z,_ 1. One of the main criticisms of these ARMA models is that they are

difficult to specify, especially if both p and 4 > 0. Box and Jenkins’ 1976 method [3] for the specification of the ARMA( p, 4) models is to examine the auto correlation function (ACF) and partial auto correlation function (PACF) of a given time series. The order of an auto regressive (AR) process can be specified by observing the “cut off’ in the PACF and the order of moving average (MA) process can be similarly determined by the behavior of the ACF. In the case of an ARMA( p, 4) model when both p and q are greater than zero, i.e., the process is a mixture of AR and MA components, the ACF and PACF do not uniquely determine p and q by simple inspection.

It is well known that model (1 .l) can be represented as either an infinite AR process or an infinite MA process. The model (1.1) can be written either as

?r( B)Z, = a,

or we can also write model (1.1) as

Where ?r( B) = F’( B)$( B) and g(B) = 4-‘( B)B( B) are infinite series in B.

In this paper we have used the estimates of ?r( B) and $(B) obtained by fitting an infinite AR model or infinite MA model to estimate the parame- ters and specify the order of an ARMA( p, q) model. The method proposed is new and is based on the theory of Pad& approximations given by the mathematician Pad& about a century ago. It may be mentioned that application of the theory of Pad& approximations in statistics is quite recent. For example, Philips [5] h as used a new method based on Pad& approximations for approximating the probability density function of eco- nomic estimators and test statistics. The mathematical details for the theory of Pad6 approximation can be found in Baker [l] and [2].

One of the important advantages of this method is that we can specify the ARMA( p, q) model and simultaneously we can also get the estimates of the parameters of the specified model. It may be mentioned that

Pade’ Approximation Time Series 141

Wahlberg [7] has recently proposed a technique for the estimation of ARMA models using high-order AR approximations. However, the tech- nique proposed here is different from that of Wahlberg.

In Section 2 of this paper we briefly outline the important definitions and theorems regrading Pad6 approximations. In Sections 3 and 4 we gi.ve theoretical results for the estimation and specification of ARMA models using the theory of Pad& approximations. In Section 5 we outline the method for using the so called C-table. Finally conclusions are drawn in Section 6.

2. THEORY OF PAD6 APPROXIMATIONS

Suppose that we are given a power series CL, C,Z’ representing a function f(Z) such that

(2.1)

DEFINITION 2.1. A Pad6 approximant [ p/q] to the power series f(Z) is defined as the rational function of two polynomials P,(Z) and Q,(Z) of degree p and 4 respectively, i.e.,

[p,q] _ wq _ %+%z+ *-’ +a,ZP Q,(Z) b,+b,Z+.-* +b,Z" P-2)

There are ( p + 1) numerator coefficients and (4 + 1) denominator coef- ficients in Equation (2.2) but usually in practice we take b, = 1, so there are ( p + o + 1) unknown coefficients in all. The power series f(Z) deter- mines the coefficients of the polynomial P,(Z) and Q,(Z) by the equation

cc,z’ = a,+ a,Z+ **- + u,zp b,+b,Z+ .-- +bqZY

+ o( zp+q+‘)

or

( b,+ b,Z+ --- +bqZ~)(Co+c,Z+C,Z2+-.)

= a, + u,Z + - * - + u,ZP +o( zp+q+‘) (2.3)

142 KULDEEP KUMAR

Equating the coefficients of Zp+‘, ZP+‘, . . . , Zp+q to zero, we find q linear equations for q unknown denominator coefficients as follows:

k&q+, + bq-1Cp_q+2 + a.. + b,Cp+1 = 0

bqCp-_q+2 + b,_ ICp_q+3 + . . . + boCp+2 = 0

(2.4)

We can find b,, b,_ 1,. . . b, by solving these equations. The numerator coefficients a,, a,, . . . , ap follow immediately from Equation (2.3) by equating the coefficients of 1, Z, Z2, . . . ZP on both sides of the equation

a, = Cl + b,C,

a2 = C, + b,C, + b,C,, P-5)

min( P, (I) aP= C,+ C biCp_j.

i=l

This system of equations is called Pad6 equations.

THEOREM 2.1. The [ p /q] Pad& approximant of C ,p”= ,, Ci Z i is given by

p[p’q’( z) [ piq1 = Q[Ph'( Z) (2.6)

Pad& Approximation Time Series 143

provided Q[P/q](O) # 0 where

cp-q+1 cp-q+2

cp-q+2 *. . cp-q+3-

c P+l

C P+2 I .

pw41 z = : () I CP c,+1 C P+4 P-Q p-4+1 c cp+i i= 0

,Fo cizq+i-l i$ociz

and

C p-q+1 Cp-,+z... C, C P+l

C p-q+2 qq+3.. * Cp+l C P+2

Q[P/q](Z) = : ;

CP C

P+l &,+,+I c,+,

2” zq-’ Z 1

(2.7)

P-8)

In Equation (2.7), if the lower index on a sum exceeds the upper, the sum if replaced by zero. It is common practice to display the approximation for various values of p and q in a table called the Pad& table.

From Equation (2.6) it is clear that Q [pfq1(O) # 0 is a sufficient condition for the existence of a [ p/q] Pad& approximant. Because the nonvanishing of Q[p’ql(0) is so important, a special symbol is exclusively reserved for this quantity.

DEFINITION 2.2.

C( p/q) = QIP’qI(0) =

C p-9+1 C,-,+z*~~ c,

C p-9+2 q-q+3 C

P+l

CP C

P+l C p+9- 1

(2.9)

It is convenient to display { c( p/q), p, q = 0, 1,2,. . . } in a table, called the C-Table, as shown in Table 1.

Owing to some convenient features of this table that are mentioned in a later theorem, this table has been used throughout this paper for the

144

P 9

KULDEEP KUMAR

TABLE 1 THE C-TABLE

2

0 W/O) C(l/O) wv) C(3/0f 1 V/l) CUD) C(2/1) (73/l) 2 W/2) W/2) C(2/2) ~(312)

specification of ARMA( p, 9) models. In practice it is not necessary to

calculate all the determinants. In fact, almost all the determinants can be calculated using the following simple relationships.

(a) The first row is by definition C( p/O) = 1, p = 0, 1,2,. . . (b) The second row is by definition C( p/l) = C,, p = 0, 1,2, . (c) The first column is C(O,9) = (- 1)q(4-1)‘zCOy, 9 = 0,1,2

(d) Most of the remaining entries can be calculated using the following

relationship:

C( P/9 +I) = C(P+l19)qP-v9)-C(P19)2

C( P/9 - 1) (2.10)

which is valid if C( p / 9 - 1) # 0.

We now sta.te the main theorem that has been found useful in the specification of ARMA( p, 9) models. The proof of the theorem has been

omitted but can be found in [l] and [2].

THEOREM 2.2. Suppose that a function f( z) is analytic at the origin and is uniquely determined by its MacLaurin series f ( z) = C,p”_, CiZi. Then the existence of an infinite block of zeros in the Padd table of Ct 0 C,Z’ is a necessary and suffwient condition for f( z) to be rational, let f( z) be of type [ p /9]. Then the corresponding block in the C-table is defined by C( p / 9 + l)#O, C(p+l/q)#O and C(p+i/q+j)=O; i,j=1,2,...,03.

The Pad& equations given in Equations (2.4) and (2.5) are found to be useful in the estimation of parameters a,, a,, . . . , up and b,, b,, . . . , b, of ARMA( p, 9) model using the estimates C,, C,, Cz, . . . , obtained by fitting a very high order AR model.

Time Series 145

3. THEORETICAL RESULTS: C-TABLE FOR ARMA( p, 9) MODEL

Pad& approximants were originally developed to approximate a power series by the ratio of two polynomials and the ideas can be used to find an ARMA( p, 9) model that best approximates an infinite AR or MA model, In this section we have obtained theoretically the structure of the C-table in the case of ARMA( p, 9) models. We have first obtained the C-table for the ARMA(l, 1) model.

Let us consider the ARMA(l, 1) model:

(3.1)

This model can be represented as an infinite AR model

a = -lB)z =+qz t (1-Q) t t

where ~(B)=~+?~,B+~,B’+...+R~B~+...

It can be seen easily that in this case,

xk=6:-‘(~,-+,) forall k=1,2,...

In the context of ARMA models we have Ck = ?Tk and using the determi- nant [Equation (2.9)] and relation [Equation (2.10)] we can find various entries of the C-table, for example,

c(1/2) = ; I I 1

E 2

= C, - Cl” = 6,(8, - 6i)

c(2/2) = E; I I

z = c,c, - cz” = 0 3

C(3/2) = C,C, - C,z = 0

146 KULDEEP KUMAR

Also using the relation [Equation (2.10)]

c( l/3) = c(o/2) x c(2/2) - c(1/2)2

C( I/I)

The other entries of the C-Table can be obtained similarly using the

recurrence relations. Substituting these entries in Table 1, we get the

C-Table for the ARMA(l, 1) model as shown in Table 2. The model given by Equation [3.I] can also be represented as an

infinite moving average model as

where 1+5(B)=l+$,B+rl/~B~+ ---

In this case C, = J/k = 4:-l($i - 8,); k = 1,2,. . . The entries of the C-table in this case can also be calculated as shown

here (interchanging 8 and 4) and the final structure of the C-table in this

case comes out as shown in Table 3. It is evident from Table 2 and Table 3 that in both the cases the infinite

block of zero starts at p = 2 and 9 = 2, so the ARMA model can be

specified by using either of the tables, i.e., by fitting either an infinite AR

model or an infinite MA model. However, in practice it is difficult to fit a high-order MA model and so usually we specify an ARMA( p, 9) model using the C-table obtained by fitting an infinite AR Model.

TABLE 2

C-TABLE IN THE CASE OF ARMA MODEL Zt = dlZt_l + a, - ela,_,

p 0 1 2 3 4 . . . 9

0 1 1 1 1 1 . . .

1 1 to,- 41) e,(b - h) efp, - 44 e:p, - 41) . e.

2 -1 9dh - d9 r-------_------_------_-------_

I 0 0 0

3 -1 -&e, - w j 0 0 0

I Zero

Pa& Approximation Time Series 147

TABLE 3 C-TABLE IN THE CASE OF ARMA(l,l) MODEL

2

The estimates of 8, and +1 can be obtained by solving the systems of Equations (2.4) and (2.5). In the case of the ARMA(l, 1) model as given in Equation (3.1):

a,=l, b,=l, a,=-&, b,= --13~, c, = 1

so b,C, + b,C, = 0

or ^

c, - e,c, = 0 62 +,

*el=_=-.-

Cl +1

Similarly a, = c, + b,

or

The C-Table in the case of an ARMA( p, 9) model for some p, 9 can be constructed in a similar fashion and has an infinite block of zeros starting

at ( p + I), (9 + I). It can be shown easily that the C-Table is also capable of specifying

pure AR or MA models, which are of course special cases of ARMA( p, 9) models when either p = 0 or 9 = 0. It can also be shown theoretically that

the C-Table is capable of specifying seasonal models to some extent. The detailed theoretical results for the models mentioned above can be seen in [4].

4. PAD6 TABLE FOR ARMA( p, 9) MODEL

In this section we have obtained the Pad6 table for the ARMA( p, 9) model when p = 1 and 9 = 1. As in Section 2, the Pad& approximation

148

[ p, 91 of an infinite series Cy=“=, C,Z’ is defined as

Wq’( Z) [ “‘I = Qb'/ql(Z)

where P[P’ql(Z) and Q[p/ql(Z) are given by respectively.

We can represent the ARMA(1, 1) model

KULDEEP KUMAR

Equations (2.7) and (2.8),

by an infinite power series of the form

Now let us obtain various entries of the Pad& table. It can be shown using tedious algebra that when p = 0, 9 = 0,

[o/o] = 2 = 1

In the case p = 1, 9 = I,

P[“l’( z) [l/l] = Q[‘/‘](Z)

1- +,z =p

1-e,z

Similarly for p = 2, 9 = 2,

P’2’2’( z) [2/2l = Q[2/2]( Z)

It can be shown that QL2i2](Z) = 0. So the Pad& approximation [2/2] does not exist. For p = 2, 9 = 1,

WI = l-&Z i-e,z

Padi Approximation Time Series 149

and the same result is obtained when q = 2, p = 1, i.e.,

l-&Z w4 = 1_B,z

Similarly, it can be shown that for p = 3, q = 1

P/11 = P[3’1y z) 1- &Z

Q[3’11( z) = 1-8,

The Pad6 approximant does not exist when p 2 2 and q 2 2.

Substituting the various approximations, we get the Pad& Table for the ARMA( 1,l) model as shown in Table 4.

It can be observed that Pad& table gives a constant pattern and a block of nonexisting entries when p 2 1 and q > 1 in the case of ARMA(l, 1) model and hence the model can be specified by looking at the constant pattern and block of nonexisting entries. This result can be easily general- ized for ARMA( p, q) models. However, simulation results show that it is difficult to specify an ARMA( p, q) model using the Pad6 table as com- pared with the C-table where a block zero starts at ( p + 1) and (q + 1). We can get the estimates of the parameters by looking at the Pad& table as well as the C-Table. Since the C-Table is easy to use and simple, we will discuss it further.

TABLE 4 PADfi TABLE FOR THE ARMA( 1,1) MODEL

P 4

2

0 1 l+(e,-qqz l+clz+c2z2 1+qz+c2z2

1 1

l--&Z 1-+,z l- +,z

l-(e,-+,)z l-e,z l-e,z 1-e,z 1

2 l-&Z

l-c,z-czzZ NE

i-e,z NE

NE

150 KULDEEP KUMAR

5. METHOD FOR USING THE C-TABLE

In this section we outline the method for using the the specification This methodology used in carrying out and applications real data sets for the specifica- tion of ARMA( p, 9) models using the C-Tables.

(1) Plot the ACF to see if the series is stationary. If the series is nonstationary, difference it appropriately so that the resulting series is stationary. To determine if the series is stationary and the required order of differencing, calculate standard deviations of Z,, VZ,, V’Z,, . . . , and choose the series having the least standard deviation.

(2) Fit a very-high-order AR model to the stationary series. The order may depend on the size of the series. Methods for fitting a high-order AR model are given in [3]. In [6], a different method for estimating the coefficient of an infinite AR model is also suggested. We can obtain the estimates of infinite MA coefficients $r, $a,. . . , by using estimates ,. ^

;;e2fn’[6,* obtained by fitting an infinite AR model using the method

(3) Obtain the C-Table (Table 1) by replacing C, by ?T~ for 9 = 1 and p = 1,2,3,. . . and obtain the rest of the entries of the C-Table using the recurrence relation [Equation (2.10)].

(4) Look for an infinite block of zeros in the resulting C-Table and so specify p and 9 corresponding to the boundaries between zero and nonzero entries.

(5) Estimate the parameters of the specified model in Step 4 by solving the system of Equations (2.4) and (2.5).

6. CONCLUSIONS

In this paper, we have proposed a simple method for the specification of mixed ARMA( p, 9) models based on Pad& approximants. The estimates obtained by fitting an infinite AR model are used to construct the C-Table, which clearly depicts the order of an ARMA( p, 9) model. Another advan- tage of this method is that we can simultaneously get estimates of the parameters of the specified model.

REFERENCES

1 G. A. Baker, Essentials of Pad6 Approximants, Academic Press, New York, 1975.

2 G. A. Baker and P. R. Graves-Morris, Pad& Approximants, Part I: Basic Theory, Addison Wesley, Reading, Massachusetts 1981.

Pa& Approximation Time Series 151

3 G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, Holden Day, San Francisco, 1970, 1976.

4 K. Kumar, Some Topics in the Identification of Time Series Models, Unpub- lished Ph.D. Thesis, Univ. of Kent, United Kingdom, 1986.

5 P. C. B. Philips, Best Uniform and Modified Pad& Approximants to Probability Densities in Econometrics, Cowles Foundation, Paper No. 557, Yale Univ., 1983.

6 T. M. Pukkila, On the identification of ARMA ( p, 9) model, Time Ser. Anal. Theor. and Prac. 1:81-103 (1982).

7 B. Wahlberg, Estimation of autoregressive moving average models Time Ser. Anal. 10283-299 (1989).