Austral. J . Statist., 25 (3), 1983, 449-452

ON A TEST OF FIT IN TIME SERIES MODELS’

STEPHEN BEVERIDGE Faculty of Business, University of Alberta, Edmonton, Alberta,

Canada T6G 2G1

summary It is shown that a test for adequacy of a fitted ARMA (p, 4) model

based on forecast errors, has the same asymptotic expected value whether the null hypothesis is true or false. A sampling study supports the conclusion in small to moderately large samples and indicates that for sample sizes commonly used by practitioners, the power of the test is very low.

Key words: Autoregressive-moving average models; Diagnostic checking; Time series analysis.

1. Introduction

A time series model test statistic based on the evaluation of forecast errors has been proposed for two purposes. Box and Tiao (1976) suggest the statistic for the detection of a change in model structure, in particular, the effect of an “intervention.” Bhattacharyya & Andersen (1974) developed the measure as a test of model adequ- acy and it recommended for this use by Granger & Newbold (1977, p. 161). However, while the first purpose is valid, the latter application is not appropriate.

The object of this communication is to show that contrary to Granger & Newbold’s statement, model misspecification will not in general result in an inflated value of the statistic. The problem is that when estimated values are used in the quantity, model inadequacy inflates both the numerator and denominator to the same degree. In large samples, the expected value of the test statistic is invariant under the null and alternative hypotheses.

2. Definition of the Test Statistic

Consider a stationary time series {zt}, which may be the d-th difference of a suitably transformed series, generated by the possibly

Manuscript received August 9, 1982; revised March 18, 1983.

450 STEPKEN BEVERXDGE

infinite-order moving average process

21 = $(B)y, (1) where I+!@) = 1 + J/,B + G2B2+. . . , B'y = y-', and {y} is a sequence of independent and identically distributed N(0, a') random deviates. It is readily shown that the forecast error, denoted by e,(k) , of the minimum mean square error forecast of z t + k conditioned on observa- tions z,,, z,,-~, . . . is given by

i -0

Now let e'={e,(l), . . . , e,(k)} , U'=(U. , ,+~, . . . , u,,+k), and T be a k X k lower triangular matrix with diagonal elements equal to unity and the i-th subdiagonal element equal to Since the transformation from the disturbance terms to the forecast errors is e = Tu, it follows that E(ee') = a2Z, where 2=TT'. Thus the quantity

is distributed as x 2 with k degrees of freedom. Additionally Box & Tiao (1976) show that an equivalent representation of (3) is

Q = e'S-'e/a2 (3)

Q = u'u/a2. (4)

In practice (3) , or (4) in conjunction with (2), can be used as a test statistic by replacing the $i and a' with values from the estimated model which has negligible effect in moderately long series. However, while Q can be helpful in indicating a change in model structure, it is transparent from (4) that when estimated values are employed the statistic provides no useful information about model misspecification.

3. Formal Analysis of the Problem

Suppose that an incorrect model has been estimated leading to autocorrelated residuals E, = tj(B)-'$(B)y, where 6 ( B ) is the pure moving average filter of the incorrectly assumed model. It is assumed that the probability limits of the parameter estimates have been inserted into the misspecified model and hence the aj are arbitrary fixed values. Defining the vector of one-step forecast errors as E'=

(E,+~,. . . , e n c k ) and the estimated residual variance as 6f= n-l xy=l E: with E(&f) = a:, using (4) we have 0 = E'E/&~ and conse- quently from the Taylor's series expansion with respect to uf,

1 E(E'E) COV (E'E, 6:) E(E'E) V X (6:) + third and higher moments. E(Q)=-- +

of UP

( 5 ) Since E(E'E) = ka;, it is clear that to a first order of approximation E(d) = k.

ON A TEST OF FIT IN TIME SERIES MODELS 45 1

The second order terms are both of order n-'. From Parzen (1962, p. 92) we have cov ( E : , E : ) = 2u:pt-jl, where pi is the i-th order autocorrelation coefficient of E,. Thus the covariance term

can be expressed as

Moreover, since all the quantities in (7) are positive, the second term on the right-hand side of (5) will tend to deflate 6. And because asymptotically var (62) = 2n-'u:, the last term in ( 5 ) becomes 2kn-'. It follows that E(Q) = k + O(n-'), even though an inadequate model has been estimated.

4. Simulated Results

A sampling study was conducted to determine how close the small to moderately large sample behaviour of the statistic is to its large sample properties. Series of lengths 46(750 replications) and S6(500 replications) were generated using first-and second-order autoregres- sive models. Only the mean was estimated (it had true value zero), with the last six observations saved for producing the post-sample forecasts, i.e. k = 6.

The results are displayed in Table 1. As expected the power of the

TABLE 1 Empirical power of Q. Assumed model: z , = p + y. True model: z, = 41z,- l + &z, -~+ u,

n=40 n=80

4, & Mean Var. 5%" 10% Mean Var. 5% 10%

0.3 0 6.97 22.50 10.5 16.4 6.36 17.05 7.4 11.4 0.5 0 7.33 33.05 13.1 18.8 6.40 22-70 9.4 14.0 0.7 0 8.16 65.94 18.5 23.2 6.58 38.75 11.8 17.2 0.9 0 11.69 263.3 26.7 31.5 8.24 121.3 16.6 21.8

-0.3 0 6.72 22.85 9.1 14.9 6.55 19.91 7.6 13.4 -0.7 0 6.97 41.68 14.5 19.6 6.78 32.85 14.0 18.6

1.6 -0.8 7.69 74.37 18.4 23.3 6.61 49.83 13.0 16.8 1.0 -0.5 6.81 30.00 14.1 19.9 6-26 23.58 11.8 12.6 0.4 0.3 7.01 42.92 13.2 18.1 6.14 30.94 9.6 13.8 0.4 -0.3 6.62 21.01 9.3 14.8 6.20 16.44 7.8 13.0 0.2 0.6 7.00 43.63 13.7 17.9 6.25 36.35 11.4 14.6

-0.4 -0.3 6.75 29-29 10.1 16.9 6.57 24.99 9.2 15-6

a Powers are expressed as percentages.

452 STEPHEN BEVERIDGE

Q criterion is dismally low; furthermore, as n increases the power of Q falls. Misspecification greatly inflates the variance but this effect is also reduced by enlarging the sample size. Estimating first-order models on the second-order series produced the same results.

Thus the sample study findings are consistent with the asymptotic derivation: the location of the Q-statistic effectively remains at k even though the null hypothesis is wrong. In other words, the test statistic has poor power properties and consequently model misspecification is not detectable.

Acknowledgement

The author is very grateful to the referee for helpful comments.

References

BHATTACHARWA, M. N. and ANDERSON, A. P. (1974). A post-sample diagnostic test for a time series model. Working paper, Department of Economics, University of Qweensland, Australia.

BOX, G . E. P. and TIAO, G. C. (1976). Comparison of forecast and actuality. Appl. Statist. 25, 195-200.

GRANGER, C. W. J. and NEWBOLD, P. (1977). Forecasting Economic Time Series. New York: Academic Press.

PAWN, E. (1962). Stochastic Processes. San Francisco: Holden-Day.