Embed Size (px)
Citation preview
'f~,
~;"0','.
...,-,.'{ ..f'f,.--'
DISTRIBUTIONS OF SOME SERIAL
CORRELATION COEFFICIENTS
by
Mohammed Moinuddin SiddiquiUniversity of North Carolina
Special report to the Office of NavalResearch of work at Chapel Hill underthe contract for research in probabilityand statistics. Reproduction in wholeor in ~art is permitted for any Durposeof the United states government.
Institut~ of StatisticsMimeogra~h Series No. 164January" 1957
•
ACKN01AJLEDGEMENT
I wish to ex?ress my feelings of gratitude to
Professor Harold Hotelling for his thought provoking
suggestions and valuable advice at various stages of
this research.
I also wish to express my thanks to the Institute
of Statistics and to the Office of Naval Research for
providing me With financial and other facilities, to
Mrs. Hilda ~attsoff for careful typing of the manuscript
and to Miss Elisabeth Deutsch for helping me in reading
and correcting proofs.
M. Moinuddin Siddiqui
ii
iii
..TABLE OF OONTENTS
..CHAPT'I:'R
ACKNO"\tJLEDGEMENT . . . .PAGE
ii
TABLE OF CONTE~ITS
NOTATION . .41 • • • • • • • • • • • • • • .. •
. . . " . . . . . . .iii
1. Introduction and summary ••••••
I. TEST CRITERIA . . . . . .. . .
1
1
2. The distribution of a sample in an autoregressivescheme • • • • • . 4
•
..
3. Least-squares estimates
4. Likelihood-ratio criteria . . . . .. . .13
16
II. DISTRIBUTION OF A FIRST SERIAL CORRELATION CO-
EFFICIENT NE.AR THE ENDS OF THE RANGE · . .. . 21
7. Bounds on the integral IN •••.••
*8. Bounds on Per ~ ro) ..•..
21
?4
29
34
37
39
42
55
. .
. . .
. . .
. .. . .. . . . .
• • •
· . . .
. . .. . . .
. . . . . . . . . . .
*Approximation to the value of r
Discussion about g(\)
2. Geometrical representation
*3. The value of r near the curve
1. Introduction
4.
5. Integral expression for p(r*~ rO) ••
6.
• ••••••• f • • • • • •
III. JACOBI POLYNOMIALS AND DISTRIBUTIONS OF ST£RIAL CORRE
tATION COEFFICIENTS 58
iv
CHPPTER PAGE
1. Introduction. • •
2. Jacobi polynomials
• • •
. . .II • • •
• • • • • . . . . . .58
59
3. Distributions of certain statistics as series ofJ Beobi polynomials •••••.• Ii • • • • • • •
7. Moments of r l •••• • •
8. Approximate distribution of r l •••• • •••
9. Asymptotic validity of the distribution of r l •
, .. . . . . .
. . , .
63
66
67
73
75
78
81
87
90
• • •
• •
. . . .4. Independence of r l and p • • • • •
5. Cumulants of ql • • • • • •
6. ~symptotic normality of q1 • • • •
10 0 Some other properties of the series (8.6)
11. Correction for the sample mean••
..
12. Distribution of the circular serial correlationcoefficient • • • • • • • • • • • • • • •• 92
13. Concluding remarks •••••• . . . . • • 97
IV. DISTRIBUTIONS OF PARTIAL fiND MULTIPLE SERIAL CORRELA-
. . . . . .,2. Cumulants of qs
1. Summary of the chapter
~ . . . . .
99
99
100
· .. . . . . . . .
. , . . . . . .. . .. . .. .TIOW COEFFICIENTS
108
111
. . . .. .. . .The distribution of r 2
3. The exact distribution of r 2s when N is even 103
1°54.
5. Bivariate moments of r l and r 2 ••••
6. The distribution of partial serial correlationcoeffici ant • • • • • • • • • • •• ••••.•
..
.'t
v
CHAPTER PAGE
1. The distributi.on of multiple serial correlationcoefficient • • • • • • • • • • • •
•
... 8. Extension to multivariate case . . .. . . . "
115
119
9. Bivariate distribution of r 1 and r 2 •• • 120
10. Concluding remarks • • • . . . . . .0 • • 123
LEAST-SQUARES ESTIMATION lNHF.N RESIDUALS ARE CORRELATEDv.1. Summary . . . . . . • • t • • • , • • • • • • •
124
124
2. Least-squares estimates and their covarianees •• 124
3. Bounds on a quadratic and a biU.near form •
. . . . . . . . . . . .
. . . . . .
130
135
137
140
143
145
. .
. .
· . .· . .
· . .
. .
. . .. . .
, . . , .Linear trend .' •••.
Bounds on covariances
Concluding remarks
6. Trigonometric functions
BIBLIOGRAPHY • • • • • •
..
..
•
•
•
...
'.
vi
NOTATION
The following notation and convention will be used, any depart-
ture being indicated at the proper place •
Numbers in L- 7 refer to the bibliography.-"Variate" stands for "random variable"
IIN(IJ.,o-) variate" means "a variate distributed normally with mean IJ.
and standard deviation (J.ll
"p(S)tr stands for "the probability of the statement S."
It Cx" stands for "the expectation of the variate x."
"a II stands for "the standard deviation of the variate x."X
\I = Gxr , i.e. the rth moment of the variate x.rlX
I': denotes the rth cumulant of the variate x.rt'xf(x) denotes the probability density function of x.
F(x) denotes the cumulative distribution function of x.
¢ (t) denotes the characteristic function of x.xX (t) denotes the cumulant generating function of x.x
.OV stands for "is approximately equal to. I'
~ stands for lIis asymptotically equivalent to."
o(r... ) stands for "a quantity of the order of magnitude of L. II
If A denotes a matrix then AI denotes its transpose.
If m is a positive number L-m_7 denotes the largest integer less
than or equal to m•
..
...
•
CHAPTER I
TEST CRITERIAI
1. Introduction and summary.
The distribution theory of serial correlation coefficients has
received considerable attention lately. R. L. Anderson /-2 72 obtain-- -ed exact distributions of serial correlation coefficients defined oir-
cularly at Hotellingts suggestion. The distributions thus obtained are
complicated and are given by different analytical functions in differ
ent parts of the range. Following Koopmans l method /-29 7 of smoothing- -the characteristic function of the variate, a simple approximation to
the exact distribution of the circular serial correlation coefficient
with known mean was found by Dixon /-9 7 and Rubin ~4S 7 for uncor-- - - -related normal variates. This aoproxirnation is accurate for large
sample Size, N, in the sense that fi~st N moments of the serial cor-
relation coefficient agree with the moments of the fitted distribution.
Leipnik r31 7 extended Dixon's distribution, following a method due- --
to Madow /-32 7, to the case of a circular autoregressive scheme of- -order 1. Quenouille L-43 7, T'Tatson ~S2 7 and Jenkins /-21, 22_7, all- - - -adopting the circular definition and the method of smoothing the char-
acteristic function of the variates, have made considerable progress
lSponsored by the Office of Naval Research under the contractfor research in probability and statistics at Chapel Hill. Reproduction in whole or in part is permitted for any purpose of the UnitedStates Government.
2Numbers in the square brackets refer to the Bibliography.
..
towards obtaining the joint distribution of several serial correlation
coefficients under null and non-null hypotheses. More recently
Daniels L-8_7, by the application of the method of steepest descents,
obtained the already known results and some new ones for circular and
non-circular statistics.
It is difficult to give a comprehGnsive history of the work
done in this field as the subject has been under extensive research
by scores of mathematicians and statisticians. Since the present study
is confined to the distribution of non-circular serial correlation co
efficients under the null hypothesis, no attempt has been made to in
clude all the names of the persons who have contributed to the prob
lems of estimation and testing; nor is any history of the classical
approach to time series analysis included. However, the bibliography
includes papers and books dealing with the general problem of time
series analysis. The names of B3rtlett L-" 6_7, Quenouille L-4l, 4~h
Von Neumann £49, 50, ,1_7, 1IIlhittle L-53_7:; T. 1,11. .~nderson L-3, 4_7,Kendall L-'?4, 2,_7, Mann and T.Tald ['33_7 may be specially no·ted in
this respect.
The major part of this paper deals with the distributions of
serial correlation coefficients and their functions, in uncorrelated
normal variates with known means. Since these coefficients are inde
pendent of scale parameter there is no loss of generality in assuming
the variance of the normal variates to be unity.
In this chapter the distribution ~f the sample from autore-
3
gressive schemes of orders land 2 will be obtained. A method of in-
verting the covariance matrix of the sample from a general order auto-
regressive scheme will be suggested. Estimates, which are approxima-
tions to least-squares estimates of the parameters in the autoregres-
sive schemes of orders 1 and 2 will be ?iven; and statistics' for test-
ing the null hypothesis,which are approximations to the likelihood
ratio oriteria, will be constructed. These are functions of first and
second serial correlation coefficients.
Two alternative definitions will be proposed for the first ser
ial correlation coefficient. r* is the ordinary correlation coeffi
cient between the sets (xl ,x2, ••• ,xN
_l ) and (x2 ,x3
, •.• ,xN) except that
the deviations will be taken from the population mean, which is assum-
ed to be zer~ rather than the sample mean. r l is the ratio of two
quadratic forms obtained by replacing the denominator of r* by
NE xi
2 • Bounds for P(r* > ro) . will be obtained in Chapter II, usingi=l -a geometrical method. A geometrical. approach was introduced into
statistical theory by Fisher ~12, 13_7 to obtain the distributions
of ordinary, multiple and partial correlation coefficients. Other
geometrical methods have been utilized by Rotelling ~17, 18_7 in ob
taining some important distributions. He also shows how to determine
the order of contact of frequency curves of some statistics, with the
variate axis near the ends of the range, even though the actual dis-
tribution is unknown.
In the third chapter a method will be developed to obtain dis-
4
tributions of serial correlation coefficients including those of par
tial and multiple serial correlation coefficients. Each of these dis
tributions is represented asa product of a beta distribution and a
series of Jacobi polynomials. The Hermite polynomials which are
associated with the normal distribution have long been in use in dis
tribution theory. Hotelling suggested the use of Laguerre polynomials
for approximating the distribution of positive definite quadratic
forms. Similarly, series of other functions such as Bessel functions
have been utilized for such pt~poses L-15_7.The theory of Jacobi polynomials is well developed and many re
sults will be quoted from Szego ~47_7 without pro9f. The same tech
nique will be extended to obtain a bivariate distribution in Chapter
IV, and could be extended to multivariate distributions as well. Since
the knowledge of moments is necessary for this purpose, a considerable
part of the third and fourth chapters will be devoted to calculating
the cumulants and moments of the variates. A comparison with the al
ready known distributions will be made to demonstrate the power of
the proposed methodo
In Chapter V, which is the last chapter, least-squares estima
tion of regression coefficients will be considered when the hypothesis
of the independence of residuals is not satisfied.
2. The distribution of a sample in an au~~regressive scheme.
Let xl' • '," xN be a realization of a time series at time
t = 1, ••• , N. It is assumed that the underlying model is an auto-
5
regressive scheme of order k, i.e.,
where St is independent of xt _l ' xt _'2' ... , and each e is a
N(O, a) variate, independent of all other ets.
In the case of aj
= 0 for j > 1, it is known that (see, for
instBi."'l.ce, 1:2°_7) for all t
,
2 e 2where Cl'x::: C; xt and is independent of t. It is also known that
the distribution of xl' •.• , xN is ~iven by
) ( ) ( '2 -N/'2 ( '2 1/2(2.2 dF xl' ... ,xN ,., 2na) l-al
)
_ 1 (N 2 2 N...l 2 N-l 7.ex·pL - 2r:i l ff~ctl ~ xi-2al ~ xixi+lS_7dxl ... ebeN •
We will study a second order scheme, i.e., a.=O for j > 2, in moreJ
detail. Changing the notation we put a = a1 and a 2 = -~ so that
an autoregressive scheme of order 2 is
Kendall L-26, Vol. II, p. 406 ff._7 has shown that the most general
solution of (2.3) in our notation is
6
(2.4)'/2
t/2 00 2~J sin jQXt=~ (A cos t ~+B sin t 9) + j:O (4~ _ a 2)r/"/. 6 t _j +l
Here A and B are arbitrary constants and 2 cos Q = a~-1/2, ~1/2
is taken with positive si(ln and it is assumed that 4~ > a2
• We also
assume that 0 < ~ < 1, the first ineauality being necessary for a
real non-vacuous prooess, and the second ineauality insuring that
Xt will not increase without limit. Further, if we suppose the
proc~ss to have been started a long time a~o, then the first term on
the right hand side of (2.4) may be considered to be damped out of
existenoe. Thus
The justification of representing xt as a sum of infinit~ly many
random variables is provided by the following theorem taken from Doob1s
Stochastic Pr-ocesses'/-lO 7, page 108:------------ -
00and if Z E ~ y ~ converges,
1 \ n)
"THEOREM. Let Y1' Y2' •••
with variances
be mutually independent random variables,2 '2 00 2 2
0'1' 0'2' ••• • Then, if i ~n = (J < co
00
Z y =x is convergent with1 n
probability 1 and also in the mean. Moreover
00 2 2 2E ~.x j = ~E ty t ; E tx J-E tx} = ct
n=l n
and if x = ~ [Yi-E fYil 7n 1 .J-
P~L.U.B.lx(~J)1 > e}~0'2/e2.\.. n n f_
Conversely, if (.i.mn-> 00
7
n 00 2i Yj = x exists, the series i a j
and00i E 1Yj ~ converge."
We need only to point out that in the notation of the theorem
above
,
(2.6)_. co[xt = E [Yj = 0
j=O,
(2.8)
Furthermore
-t 2 a k/ 2 2 kr. kr. Q (k 2)Ex x = ~ a;- cos e... cos e'" t cos - e 7t t+k (4t:l_a2) - 1 ...~ ~ - •
p 1 - 2~ cos ?Q + f3
If we define the autocorrelation coefficient of xt and Xs as
\
(2.9) p =It-st
we have r 26 7- -(2.10)
(2.11)
Pk satisfies the equation
8
with Q. = 1, (J - "'u - t - '"' t •
From (2.11) taking k=t and 8 in turn and solving for a and
~ we get
a ==P t (J s-2 - P s P t-2
p t-1 p s-2 - p t-2 PS - 1
In particular
(J t P 8-1 - p t-1 P s~ ... p t-1 p s-2 :--p t-2 p8-1
(2.12) a ::
~ =2P1 .... O 2., P 2.... - 1
I
:HE0R~ 2.1. Let A be the covariance matrix of x =(x1,x2' •.• ,xN)
where the model is (2.3), so that
(2.14) A = (J2r...
O2pi 1 Cl . •x j N··l
P1 1 PI • . P
N-2
I
PI 1 PN-3II °2 I
lp 1/-1 P I/-~ •
.PN-3 1
then
-1 1Ii =2
C1
1 -a
2-a l+a
~ -a-a~
o ~
~ 0
-a-a~ ~
2 2l+a +13 -a-ap
-a-a13 1+a2+13 2
·..o
o
o
o
o
o
o
9
,
o
o
o
o
o
o
o
o
·..·.. -a
-a
1
(2.16) (ii)
and
~A! =)2 {< .,~) t
(1-~ _ 1+~ -a J
(iii) if x denotes the transpose ofI
X , its distribution is
(2.17) ( )_( )-N/2 I I -1/2 /- 1 f A-I 7dxdF xl' ~ •••xl\T - 2n I A exp_ - -r x_
PROOF. The proof of part (i) is completed by actual multip1ic~ion,
taking into consideration that
(2.18)
and
Pi = a '::\_1 - ~ P i-2' 1:=2,3, ••. ,N-l
PI = a - ~ -P1
10
A-IThe nrocess by which we arrive at
The distribution of (S3, ••• ,eN) is
is the following.
( ) ( 2· -(N-?)/2 ./- I N 2 7dF &3'" .,eN = 2no ) exp_ - ';)(l ~ 6 i _ de) ••• deN •
Sunnosing xl and x2 given, make the transformation
The Jacobian of the transformation is 1, hence) given Xl and x2' the
distribution of (x3' •.. , xN) is
Now Xl and x2 are distributed as bivariate normal given by
where PI = ~, and ax is given by (2.7), hence
eXPL- 2~2 {(xi+X~)(1-~2)-?a(1+~)xIX2}_7rlx1dx2 •
Multipl~Ting (2.19) and (2.20) vie get (2.17) and also the value of IA I .
11
Thus parts (i1) and (iii) are proved.
Another method. l~Tith the help of (2.19) we write the matrix A-I- ,
with the first two rows and the first two columns blank and to be
filled in, asr---
* tl$o * * ·...... '* '* * ** * * * ·...... '* * i~ *
2 2 -0 0 0 0* * 1+0: +~ -a-a~ ...
-1 1 2 2 0 0 0il- * -a-aa Ha -+p ... 0A =~ ,
c .. •
J,fo * 0 0 • • • ,,". "a"'a~ 1+(12+~2 ...a ...o;~ ~
* * 0 0 ·...... a -a-np 1+0;2 -a
e * * 0 0 ·... " ... 0 ~ -0; 1
Now, we observe that the matrix A is persymmetric, i.e.,
A- lsymmetric about both diagonals, therefore has the same property.
l~Te fill up the first two rows and the first two columns with the help
of the last two rows and the last two columns, only writing them in
reverse order, and obtain (2.15). The proof is then completed by
actual multiplication.
If the underlying scheme is autoregression of order k given
by (2.1), A-l can be obtained easily by this method if N > 2k.
Thus,
(2.21) dF(xk+l,·",xNJ x1' ••• ,xk) =
12
If k is small we can find the distribution of (X1
, ..• ,xk),
and by multiplying it with (2.21) obtain not only the exponent but
also the value of I AI.
In any case, however, we can 1I\Trite A-I with the help of (2.21),
filling in the first k rows and the first k columns wi'bh the help
of the last k rows and the last k columns. Thus, for instance,
for k = 3,
1 -0:1 -0:2 -0: 03
2-c;. 1+0:1 -0:1-0:10::? -0:2-C:l c£3 -0:32 2-0:2 -CI1-0:10:2 I+CI1+0:2 -0:1-0:ICI2-0:20:3 -0:2-0:10:3
-1 -0:3 -0:2-CI10:3 -0:1-CII0:2-0:2CI3 222A = 1+0:1+0:2+0:
3 -0:1-0:10:2-0:20:3
0 -0: -0:2-0:10:3 -0:1-0:10::?-0:20:3 222
3 1+0:1+0:2+0:3
0 0 0 0 0
0 0 0 0 0
0 0
0 0
0 0
0 0
0 0 1"2
..Cf
2• 1+0:1 -0:1
. -0:1 1
13
J . lATise L-54_7 has given another method of inverting A in
the general case.
)_ Least-squares estimates.
In the analogy with the elementarv theory we find formally the
least-s"'uares estimates of the parameters i.n autoregressive schemes
of orders 1 and 2. This will serve as a guide in constructing simpler
statistics based on serial correlation coefficients.
For a Markov scheme
least-squares estimates (which are also the maximum-likelihood esti-
mates when xl is supposed
a =1
to be fixed)
N-li~lxixi+l
N-l 22: x.
i=l ~
2of a and (J are
and
(J .2)
respectively. The estimate of the variance. of al is
L. Hurwicz L-20_7 has shown that al is biased in small samples.
Mann and Wald ~J3_7 showed that this bias tends to zero as N tends
14
to infinity.
For an autoregressive scheme of order 2, that :l.s,
the least-squares estimates of 2a, ~, and ~ are
{ZXi _1Xi )(~i...2)-(~1-~i)(Zxi _2xi _l )a2 = 2 2 (
(ZXi_l)(~Xi_2)- ZXi _2xi _l )=D
(~xi_2Xi_l)(zxi ..lxi)-(ZXi_l)(ZXi_2xi)
D
and
respectively, where all the summations extend from 3 to N. The esti-
mate of the covariance matrix of a2 and b2 is
2sB = 1)
XXi..l~i"2
2ZXi_
l
Obviously these estimates are of litt~.e practical use. To
simplify we propose the following estimates instead, which are suggest-
ed by and approximated to the least-squares estimates.
1ATriting
N-?Z x.xi +21=1 ~
N 2p == Z xi
i=l
let us define a first and a second serial correlation coefficient as
and
if "P -I o.
We propose r l as an estimate of a in a first order autore
gressive scheme~
as an estimate of 2(] . Similarly an estimate of variance of r
lis
In the analogy of elementary theory we expect
t* =
to be distributed approximately as Student t, under the hypothesis
a :: O.
Similarly for a scheme of order 2, given by (2.3~
a =
b =
and
(3.6) 2s =p(1-2ri+2rir2-r~)
- (N-2 )(l-ri)
pel - R2
)
N-2-
16
will be our estimates of a, f3 and (J2. Here we shall define R as
a multiple correlation coefficient between xt ffild (Xt _l , xt _2).
It may be noted that
1 r l r 2r 1 r11.". r 1 1
2 "'2(3.1) 1 - R =:e 1 :1 Ir 1
Again, it is expected that when a = f3 =: 0,
* 2 l_R2F (N-2,2) =~ -7\-
N-.::: R':::
will be distributed, at least approximately, as F with N-2 and 2
degrees of freedom.
4. Likelihood-ratio criteria.
Consider the maximum of likelihood under the following hj~otheses,
for an autoretressive scheme of order 2 ~iven by (2.3),
HOO: a =: 0, ~ = 0
HIO : a r0, f3 =: 0
17
HOI: a = 0, ~ 10
Hll : a I 0, ~ I 0
Notation: gij(x) denotes the likelihood under Hij ; and &ij' aij , and
B.. the maximum likelihood estimates and. ~J
likeli:."1ood.
/'\g.. '~he maximized~J
Since we propose to study only null distributions, i.e.,
distributions of statist:i.cs under HOO ' we will not oonsider
any non-null hypothesis against another non-null hypothesis.
Furthermore, for mathematical simplifications we shall
assume that either xl' or xl and x 2 ) are givan. This re-,
suIts in a slight loss of information but saves many com-
plications.
( ) ( 2)-(N-l)/2 /- Igoo x2'·· •,xNtxl = 21£0' exp_-?20'
N~~O = ( 6 x~)/(N-I)
2 ~
( ) ( 2)-(N-l)/2 1- 1 N( 2 7glO X2'··· ,xNJ Xl = 2110' exp_ - 20'2 ~ Xi-axi _l ) _
N°120 = E(x.-ax. 1)2/(N-l)
2 ~ ~=
N-l Nalo = ( E xixi~l)/( Exi)
1 2
18
Hence the likelihood-ratio criteria Ala for testing HOC against
HIO is
1~ ---::T" rv
l-alO
for large N •.
We note that replacing ala by r l results in some loss of
power but since alO-"" r l this loss is negligible for large N. How
ever, this or some other approximation seems necessary for further
progress in the theory.
For testing HOO against Hal or Hil we will compare
" ( ) ,,2gij x3'.' .,xNfxl' x'2 • 0'00 could be considered modified accordingly.
19
Using BPproximat:l.ons similar to those used for ~Ol and ~Ol' we find
"~11 ::: •
Thus ~ for testing HOO against HOI' we ha've
2
(l )"N=2nl
and for HOO
against H11
1
(4.4)2
( l )N--2 ,....11
where
1= ----2
1 - R
20
Though AOl ' Ala and All are not exact likelihood-ratio oriteria,
and their use ~dll involve some loss of power, it seems that to de
velop any theory based on serial correlation coeffioients some suoh
approximations are to be used for maximum-likelihood estimates.
Other writers L-2, 9, 21, 43, 52_7 have used circular definitions of
serial oorrelation coefficients to oiroumvent some of the theoretioal
difficulties. We have preferred to avoid the oircularity definitions
and have dealt with non-circular statistios.
In the following chapters the distributions of these statistics
have been obtained. .Also an upper bound for P(r* ~ r O) has been ob-
tai.ned iAhere *r is a serial correlation coefficient defined on the
pattern of an ordinary correlation coefficient between two series.
CHAPTER II
DISTRIBUTION OF A FIRST SERIAL CORRELATION
COEFFICIENT NEAR THE ENDS OF THE RANGE
1. Introduction
If xl' x2' ••• , x~ are observations on the variates Xl' X2,
••• , XNt and the population means are known which may, without loss
of generality, be taken to be zero, the most natural definition of
the serial correlation coefficient with lag unity would be
N-lL x.x. 1
* . 1 1 1+1=
r = :=::::;::=:;;::;;===N..l 2 N-l 2(LX.)( Lx.+ l )i=l 1 i=l 1;
if the denominator i 0,
which is an analogue to the definition of the ordinary correlation
coefficient between two different sets of observations. However,
instead of taking deviations from the sample mean, we have taken
deViations from the known popUlation means which are assumed to be
zero. Due to the seemingly insurmountable mathematical difficulties
" *involved in obtaining" the distribution of r , several alternative
*definitions have been proposed as approximations to r • The most
suitable among the alternatives seems to be
22
Even this presents difficulties as far as exact distribution is con-
cerned. Further, r l never attains the values -1 and 1, its minimum
and maximum values being - cos N:l and cosN: l respectively. But it
*has the advantage of being (i) a good approximation to r for large
N, (ii) a ratio of two quadratic forms, (iii) 'independently distriN :2
buted of L xi on null hypothesis, and (iv) of not bringing anyi=l
unnatural assumptions about the universe, particularly in non-null
cases, as the circular definition or some other proposed definitions
do.
*If we consider the cummulative probability curves of rand
r l , for large N, the two curves will be very close in the middle of
the range but will deviate apart toward the upper end of the range.
This becomes clear if we observe that
~ ~ .1 2 ~ 2 1 ~2: x. > ~ Xl + Xc> + •••+x.. -1 + '7) xN >
i=l ~ - c. • c;.. N '- -
-1 n N-1 2~( r x.)( I x. 1)
'1'1 . 11+J.= ~=
i. e. Ir11 ~lr*1 , the inequality being true whenever at least
two XIS are different from zero and different from each other. Hence
for a given number r O between 0 and 1 we have p( Irll ~ ro)<P( Ir*l~ro)'
23
On the other hand we have
2 2r 1 ~
(--.)= (1 - ---2)(1*
r Lxi
Writing "N
2
= (1 - ~2)(1Ex.
1
we observe that if xi are independent
N(O,l) variates
and the variances of these quantities are 0(N-2). Therefore"N tends
to 1 in probability as N ->00. Hence from section 20.6, page 254
of Cramer's Mathematical Methods of Statistics L:7_7, we conclude that
the distribution function of r1
is asymptotically equivalent to that
of r* •
It is therefore desirable to consider the mathematical behavior
*of the distribution function of r near the ends of the range, and
*to use the distrib~tion function of r l for the values of r 1 (or r )
in the middle of the range. However, it may be pointed out that r l
is 'easier to calculate and can be used in the whole of its range.
*r
*In this chapter the distribution of r near the extremities
of the range is considered. The geometrical approach seemed to be
particularly suitable in obtaining the order of contact of the dis
*tribution curve at r =+ 1. It will be shown that if for a given
*number r O < land close to 1, p(r ~ r O) is expanded in a series of
powers of (1 - r O)' the first non-zero coefficient in the expansion
is that of the term (1 - r o)(N-2)/2. Upper and lower bounds for the
coefficient of this power in the expansion will be calculated. The
lower bound is positive and the upper bound gives an approximation to
*the upper bound of P(r ~ r O)'
2. Geometrical representation.
Let Xl' ""XN be N independent N(O,l) variates. Define
= ;:::;;::;:=::;;::;::::/ N-12 N-12/ ( fX.)( fX. 1)V 1 1 1 1+
*if the denominator is not equal to zero, then r is a variate with
For every set of observations Yl"."YN on these variates, we
take a point S in N-dimensional Euciidean space, which may be regarded
as a representation of sample space. S is determined if its co-
ordinates are taken to be (Y1, ,o.,YN). Denoting the origin by 0, we
25
see that the points S are distributed with spherical symmetry about
*O. Furthermore, a uni~ue value of r corresponds to all the points
*on a straight line OS, excepting the origin for which r is not de-
fined. Let the straight line OS meet the (N-l)-dimensional unitI
sphere in Q and Q , where Q is on the same s ide of the origin a.s S
is. Denoting by (xl'" .,XN) the Coordinates of Q, we have
(2.2)N\~ 2 1~ x. = ,
. 1 :I.:1.=
which may also be taken as the e~uat1on of the unit sphere. The
I *points Q and Q may be considered to determine a uni~ue value of r •
Considering only the point Q, it is easily seen that the distribution
of Q is uniform over the unit spherej that is, denoting the total
(N-l)-dimensional surface area of (2.2) by SN_l' the probability of
Q fall ing in an area A on the sphere is AISN-1' For a given r 0 in
L:-l,l_7 there exists a set of points on the unit sphere such that
*for each point in this set the correspoinding value of r lies in
the interval ~ro,1_7, and for no other point. If this set of point
covers an area A on the surface of the sphere (2.2), it follows that
*P(r 2: r 0) =A/sN_l •
26
*We observe that r = 1 if and only if
Xi = Axi _l , i = 2,3, ••• ,N and ~ > 0, Xl~O,
that is,
i-l Jxi = A Xl' i = 2,3, .•• ,N and ~ > 0, XliO•
Since the point (xl""'~) lies on (2.2), we obtain for the value
whence
I 2 2Nc = t (1 - ~ ) /( 1 - ~ ).
( - c,
N-l )Denote the variable point (c, hC, ••• ) A. c by P and
N-l- AC, ••• , - A c) by Pl. As A varies from zero to 00, each
of P and pI describes a curve for every point of which -- excepting
the two points of each curve obtained by A = °and 00 -- corresponds
*the value of r = 1.
27
Since both these curves are exactly alike, except for their
position in space, we confine our attention to the curve
(2.4) 1-1;:: A xl' 1 = 2, ••• , N, 0 < A. < co •
Further, from now on we reserve (x1' ••• 'xN) to denote the
point on curve (2.4) which corresponds to the parameter value A,
and we use (£l' ••• '£N) to denote any other point on the unit sphere.
To find -the probability of r* exceeding a given" value r a
which is close to 1, we consider the points within a "tube" of goe..
deaic radius 9 on the surface of the sphere (2.2) with its axial
curve (2.4) •.
Let the length of the curve (2.4) measured from Pa(l,a, ••• ,
0) to P(xl , ••• , xN) be denoted by 8, or more explicitly seA),
and an element of curve by ~s. Denoting by primes the differential
coefficient with respect to s, the direction cosines of the tangent
to the curve at Pare
where
,... , x
N,
We note that
t =L-(J."-1),i-2e ,1-] de _,1 1 2xi 1\ + I\. •• QA _/1\ , i= , , ••• , N •
(2.6 )N '2Z x = 1
1=1 i,
28
and sinceN 2Z xi:&< 1
i=l
we haveN I
(27) Z xixi :: 0i=l
Let the coordinate axes be rotated, so that the new coordinates
B, ~ and ~denote the column vectors (~l' .'., ~) and (~, ••• ,
rw) respectively.
The "1 axis:i.s how narallel to the tangent of the curve
at P and the ~2 axis coincides With the line OPe
The (N-J)-dimensional sphere given by the set of equations
"i = Yi sin 8, i = 3,4, ••• ,N,
with
29
lies entirely on the (N-l)-dimensional sphere
(2.10)N 2 NIT) =1= 1:~2
i=l i 1=1 i
The sphere (2.10) i8 the same as (2.?) with a change of nota
tion. Each point on (2.9) is at a geodesic distance Q from P
measured on the sphere (2.10). Further, since (2.9) lies in the
plane ~ = 0, this hypersphere is perpendicular to the tangent of
curve (? .)4) at P.,
Changing back to the original coordinates we have ~ = B T) or
,t i =X1T)1+XiT)2+b31T)3 + ••• + bNiTlN' i = 1, ••• ,N •
Eauations (2.9) become
(2.11)
with
N~j = xj cos Q + sin e 1: bijYi , j=1,2, ••• ,N
i=3
N 2Z Yi ,.. 1
i=3
3. *The value of r near the curve.
*tet us calculate the value of r corresponding to a point
(~"'.'~N) on the hypersphere (2.11), We have
since
N-1 2 ~ N-1Z t j = 1 - and l: ~ 1 = 1 - ';12.
j=l . j=l J+
30
Now
N-1 N-l N(3.1) Z ~j~" i= Z Lx.cOg G+sin e Z bi ·Yj. 7
j=l J~ j=l J i=3 J -
N_r Xj+lcOS g+sin e Z b . Y 7i=3 i,J+l i -
2 N-l= cos e z x,x j +lj=l J.
It is convenient to introduce here a double suffix summation
convention. We will reserve the letter j appearing at two places
as the suffix in a product term to represent summation over j
from 1 to N-l. The letters i and k appear ing twice as
suffices with stand for summaticn from 3 to N. Thus, for ex-
ample
and
meansN-lZ x,x.+1j=l J J
Now
31
Therefore,
Similarly
Rememhering that the point P(x1, •.• ,XN) is on the curve (2.4) for
each point of which r* ~ 1, we have
(3.4)
Since B is an orthogonal matrix, we obtain
bij xj +1 ~ A bijXj = - Ab~
N= - A biNc ,
1 1bi ,j+1Xj = ~ bi,j+lxj +l = - r bilxl
:: -f bil c •
1~Te note that N is not a dUlillTI:y suffix and does not stand for
sUl"f1mation.
Finally from (3.1) - (3.5) we obtain
32
(3.6) r*:: l-c
33
Now,
,
The coefficient of sin Q cos Q
= 0 •
After substituting the values from (3.1) in terms of >.. and(
replacing cos2e by 1 - sin? 0 and simplifying, we get
*(3.8) 1-;sin e
It is understood that terms of order sin?Q in (3.6) have been omitted
in (3.8) and in the later discussion.
4. Approximation to the value of *r •
34
,
As an approximation, replace the terms in the square bracket
in (3.8) by their expectations. Since Y3,o"'YN are Cartesian
coordinates of a point on (N-3)-dimensional unit sphere
N ')2: 'Y'''' = 1
i=3 i
we have from considerations of symmetry
;l 2 1t Yi = N~' i=),4"";iN ,
Rearranging the terms of (3.8), we get
Since B is orthogonal, we have
or
35
Similarly
and
and
where the double suffix summation convention is still being followed.
Hence,
I Irx .X. l+xjx'·l 7.- J J+ . J+ -
Now.
and
(4.4)
Therefore t
dxi i-2 i-I dxlax- = (i-l)X xl+A dX' 1=2, ••• , N
36
Furthermore"
2 N-I 2j-? A(A2N- 2_1)XjXjJ>_l = Axl E A =_.~- ,
- Y • I "\ <::'1\; 0,J= ~ -1
and, since, I ,
x. I = Ax j + x.A ,J+ J
we have
12 f t
= A(l-xN ) - A VN~ from (?6) and (2.7). Substituting from (4.4)·(4.7) in (4.2) we get
f 1Inserting the values of xl' xl' xN' xN' rearranging terms and
simplifying, we finally obtain
37
or
5. *Integral expression for Per ~ ro)'
To find the probability that r* ~ r O' where rO < 1 and
close to 1, we proceed in the following manner. For a given A, r Odetermines a unique positive value of sin 8; hence a unique value of
Qo ,ds by f SN_3 dG is the ratio of this area to the area of the unit
o
e in,the interval ~O, ~ _7, say eO(A). For a ~iven A, the
probability that a random point (~l' ••. , ~N) falls in the elemental
area of the surface of the sphere ~~i =1 given by the product of1
sphere; here denotes the surface area ofN 2 2~ Y'i = sin Q.
3
This ratio equals,~(N-2)
21C N-3sin 0 dQ
N-? ..'1_=-u.n2n
N-3sin e dO
~emembering that for r* ~ I there correspond two curves on the unitI
sphere, one traced by P and the other by P, and noting that
38
changing the signs of x's in (3.6) does not change the expression,
we get00 • 9
0P.)
P(r* ~ r o) .., N;2 J jf sinN- 3ed9dsb..).
° °*It may be pointed out that for X = ° and 00, r is not de-
fined, but the inclusion of these values in evaluating (,.1) does
not affect the probability as the integrand remains finite near these
values and is zero for X = 0 and 00, as will be seen later. Now,
(,.2) sinN- 30 cos QdQ
cos 13
• N~? 2 :.'s:m eO sin 13 3 sec 60/- °= .... 1- 2~' r(N-2)cos eO - N cos eo N sinN-' QO
if N is large and eO small.
Therefore,
,'l- ) 1Per .:: rO -- n
From (4.,) and (4.9)
* (l-r )(N-2)/2(,.3) Per ~ 1'0' ::: __0;....-__
n
00
J .N-2 ()s~n BOds A.
o
00
39
Let us denote the integral on -the right hand side by IN and note
that a change of variable Al =- 1/).. leaves the integral l..l.i'1chcn ged;
hence the integral from ° to 1 is the s arne as from 1 to 00. Thus
(,.4)
where
~ and
IN = 2 l l"g(\LT(N,.2)!2h(>.)d>.
°
We note here that E. s. ~eeping ~23_7 has studied a similar
but much less complicated integral.
6. Discussion about g(A).
Let us consider the behavior of the function g(A) in the
rm ge ["0, 1_7. Writing
(6.1)
and
(6.2) ( N-? ( N)( ) - l-y i l-yg2 y - N-- ~ .. ,
(l-y )
40
we have
(6.3)
Dividing out the factor (l-y) from the numerator and the
denominator of gl (y), we have
l+yNgl = '2 N-l > 0 for y > 0
y+y ..... ·+Y
1 r 2N-? ?N-3 ( N ( ) N-2 , 7:: D . t L y +?y + •••+ N-l)y - N-l Y ......-2Y-!r.enom~na or
Hence, from Descartes' rule of sign, dgl/dy = 0 can have
at most one positive root, as after supplying the missing term yN-l
we see that there is only one variation of the sign whether the co
efficient of yN-l be considered positive or negative. This root
is given by y = 1. It follows that for 0 < y < 1, dgl/dy has
a constant sign which is easily seen to be negative. Hence gl is
a monotcnically decreasing function of its argument in the range
(6.4)
Differentiating g?(y) with respect to y and arranging
terms,
dg2 yN-3(1_y)2 N-1 N-2 N-3---d. = - NI' 3 ;-rN-2)y -2y -2y - •.• -2y+(N-2) 7.
y (l-y -) - -
Let the expression in the square bracket be denoted by
41
<.
* Fromg2°
Descartes' rule of sign,
Now'
*g2 = 0 can have at most two positive roots o
and
= 0
Hence y = 1 is a repeated root of g;(y) =0
Theref<0re ,
,)
dg N-3 4 N-3()-2 = _ y (l-Yj ¢(y)=. -y 1-y ¢(y)dy (N-l) ( N-?)3l-y l+y+ •.•+Y .
,
where 0(Y) f. 0 for 0 < y < 1, and since- -* 2g2(Y) = (1-y) ¢(y) ,
¢(y) has the same sign as g;(y) which is easily seen to be posi
tive. Hence dg2!{jy < 0 for 0 < y < 1, and for 0:: y ~ 1, g2(Y)
is a monotonically decreasing function of y~ and
(6.5)
Since
g (n) = 1, g (1) =~2)N2 ? (N_l)2
\2 is a monotonically increasing function of A in /-0,1 7,- -,
we conclude from (6.3) that g(\) is a monotonically decreasing
function of \ in L-O, 1_7 and
(6.6) g(o) = 00,
42
( ) 1 2 1 (N-2)Ng 1 = 1+ 2'. N-1 + N-2 • (N-l)~
N 2= ('N-!) ,
Write
(6.7)
(7.1)
then 1'(A) is a monotonically increasing function of A inL-0,1_7
with
(6.8) ,+'(0) = 0 and ''/1(1) = (1 _ ~)2 •
7. Bounds on the integral IN'
From (5.4) and (6.1)1
IN = 2 rJ: 'ijt'(A)_7(N-2)f2h (A)dA •J
o
Now yeA) can be written as
2 ] A2N- 2 ~ m 2(1_,2N-4)"'Y'(A)=2A. (M .~. );-1+ w\o '"
l"'A2N - ;::2)(l-A2N)-
Hence
_ 1 N2,2N-2 1/~/ '" _7 C dA •- (1_A2/ - (I-A2N/
Put
43
then(12
00 sinh lW')x N-2 _ N sinh (1- i)x (N-2)/2rL sinh i -_7 L 1+ ) _T~ (N-2 sinh xo
r 1- 4 sinh2 ~
N
r N 2x . 2x) 7(N_?)/2_ 1+ N-2( cosh N - coth x s:mh T _
Now
00 ( h it th .• 1 X)N-2/- h2 x _2 cosech2~71/2~.t COSL N.w .eo x Ol.:D:n N-. _ cos N- If" U.I\,
)o
for every x, while for lxl < n ,
where B is the nth Bernoulli number. Therefore, for Ix J < 1t,n
246 2h x th '00 x 1- 1(1 x :x: 2x ) x O( -3)cos N - co X S1 N= - N + j - 45 + 945 + ••• + 2N2+ N
and
(N-2)1og(cosh ~ -coth x sinh i)= - (1+ ~ - ~+ ••• )+ ~(~ +~
44
and finally
x xN2 2 ~(1.4) (cosh N -ooth x sinh 'N) - "'exPL-(1+ -r -'[; + ••• )
Further
2 Ii 61+ .1!~(oosh~ - coth x sinh ~)=2r1- 1(::_ - ;..,.. -I- 92~ +
N-~ N ~ - N'3 4~
1 (X2 2x4 . 3... -2 - + -,-p + ••• )+ O(N- ) _7t
N 3 45
and
N-2 r N (2x . 2x) 7... ""2 log .t.. 1+ N-2 cosh 1r ... ooth x slnh N _
... )
N-2 1 x2 x4 2x6 1 ( 2 x4 2x6= - ....,...log 2 + '2(j - 4; + 945 + ••• )- TiN 2x - '9 + n~
+ ... ) + 0(N-2);
and so
(7.5) L-1+ }'J~2(cOSh ~ - ooth x sir.h ~L7 -(N-2)/2
2-(N-2)/2 i 1(x2
x4+ 2x
6+ ) 1(x2
x4 ) (-2).~"" exp _ ., '.3- - 45 ~ ... - N -r .... W ... +0 N _I"
Finally
246( 6) r ( -x/N) 7(N-2)/2 -1 /- x x x7• _ "f e _ =e exp_ - '6 ... 90 - '91.3 + 0 ••
3 2,,.2 x4 -·2... mJ(l + "'::9 - 30 + ••• )....O( N >-7
45
the expression being valid for Ix I < no Also
) r 2 x 2 . 2 1/2 N /- x2 137 4(7 .7 _ cosech N -:N cosech x_7 :: I/'J'"' _ 1-!O ... l.~OO x + •••
We split the range of the integral IN into the ranges
£0,1.] and L-1, 00_7. Denoting the in~jjegra1 from ° to 1 by
IN1 , we have f:::-om (7.4) - (7.7) to order N-1 ,
-1(7.8) I =.L
Nl lJ
/- x2 137 4 7
_ 1- !O + 12600 x + .""_ dx
a-:1 f1 x2 x4 x2 137x4=0 - (1- '0 + 40 + " .. )(1- yO + 12606 + ".. ) dx130
a-I f1 x2 x2 x4 x4 137 4=- ff 0 (1-?", - 10 + 40 + bo + 12600 x + •.• ) dx
_ a-I ( 1 1 1 1 131- ./3- 1- T8 - 3U ... "2"00 ... 300 ... b3000 - .".)
-1= ~(.9216) :: 0.196 •13
Hence, to 0(N-1 ),
I =0.196+2N/ 2N
o
Denote the second term on the right hand side by J and substitute
A = yl/2
. 1 -1/2dX ., '2 y dy ]
so thlt
2 N-l. )2 /rl N y t1-y 71 2 d
- ~ y •- (l_yN) -
The result (1.8) is sufficient to prove that the coefficient of
(l_r~)(N-2)/2 in the expansion of p(r~.:: rO) is non zero. 1nTe
now proceed to set bounds on J. Now, for 0 ~ y < 1 ,
.r N (N-I) 2N N 3N1+(N-2)y + 2 Y +(3)Y- + ... _7
=1_(N_2)yN-lCI_y)+ {(N;2)y2N-2_CN_2)2y2N-l+(N;1)y2N}
{(N-?) 3N-3 (N-2)2(N-3) 3N-? (N-2)2(N-1)3N_l
- 3 y- - -"'21 T + 2 r
- (~h':3NJ ~ ...
47
and
J:iilrthermore,
and hence
Therefore
e-1/ 2 _ Ny(1_~r-2) -(N-2)/2 1- ('1'2 < / 1..- - tr 7 < . in ·(l..-y) N-2) - (N-2 )(l-y) - (hy) t N-2
Using (7.10) - (7.12) in (7.9), we obtain
where-2/Ne
~(N-2) f (N,-3)/.?d~2 r y y.
- 0 (1_Y)(1~y)(N-2)1?
-(N-2)
48
-2/N+ ~ (N;?)y(5N-7)/2_(N_2)2y(5N-5)/2+(N;1)y(5N-3)/2dy
o (1_y)(1+y)(N-2)/2
e...2/ N
f (l-y) {y(5N-7)/~_i5N-5)/21dY
o (1_~J)2(1+y)\N-~)/2
We observe that we have to evaluate integrals of type
-2/Ne
fo
and
(7.16)
-?/Ne
Jo
where q = (N-2)/2, P = sq+b, s > 0 and b = 0(1) • Substituting
y = e-2/ Nz and expanding (1+e-2/ Nz)-q in the powers of (l-z) and
integrating term by term, we obtain
49
-2/N e...2(p+1)/N ~ r~r» r(p+1) ( 1 )rM(p;q,e )- ( -2/N q r=O q f,(p+r+2j 2/Nl+e ) . 1+e
e-2(p+1)!N 1= 21M q F(1,q,p+2, 2/q);
(p+1)(1+e- ) l+e r
and
-2/N _ e-2(p+1)/N 00 e-2k/ N ( , 1t(p,q,e )- -2/N X +k+1 F\1,q,p+K+2, 2/N)·
(l+e )q k=O P l+e
Now if s > 1 and c, t > 0,
) t t( t+1) 2 +F(l,t,st+c,x =1+ ts+c x + (ts+c)(ts+c+1) x
x x 2 x 3> 1 + - + -2 +'-3 + •.•
s s a
and
...1
=l-xTS
If t is large, then the right hand side of the last inequality =
2x x (1 )1+ - +;:;, '" 0 it. NoW, fors 6 (l-x)
we have
(1.17) 26 ( 1 ) 1+8+2s2
2s-1 < F 1, t, ts+e, 2TN <. 2l+e N 2s
Therefore, taking q =! - 1 and p = sq v b, to O(N-1), we find that2
and
(7.18)
Let L-q_7 denote the largest integer in q; also, let q1 be the
q 2qlargest integer in ~, q2 in IO and so on. Then
00 -2k/N 1~ e+k+1 F(1,q,p+k+2, - 2/N)
k=O P l+e
Lee7 -1 e-2k/N 1+•••+ ~ v . +1 F(1,q,P+Q9+2, 2/N)
k=q p q9 l+e9
12/N)
l+e
•••
+ e-?q/N(1_e-?q/N) ( ~+(S+1)+2(S+1)~ )(S+1)Q(1_e-2/N) 2(s+1)2
+-.2 I 2\
e, ., ll.2+s+( e+ 02) j + •••(S+ .2)
-1 -1 J -2 f J+ !:.e L- e j {S+2+2(8+1)2 + e 3l 8+3+2(8+2)2e (8+1) . (8+2)
•-r { 2(+ ••• + e. 3 . s+r+1+2(s+r) J
(s+r)+ •••
Similarly
(7.20)
e-·9(28+1.8)+•••+ (s+1)(28+1.8-1) _7
e For 8 = 1, p ~ q =~ - 1, we have
,l-,a-'" { -1 11 -2 22 -3 37 -4 56
+ 2 e. 1f +e '"'-1 + e • or; + e • W +
hence,
( -2/N) .629L p,q,e . < (N-1r;~ ·
2 I
Asimilar calculation
S,p and q,
shows that for the above values ot
Denoting the integrals on the right-hand side of (7,14) by Ql' Q2etc. as they occur in order, and neglecting th,e sign, we see that
Ql
= 2(N-2)/2 L(~:~J ¥, e-2/N) •
Hence from (7 .:?1) and (7.22)
0.54:? < ~ < 0.629
We may note here that J < 01 and hence
IN = .196 + J < .196 + .629 = .8:?5 •
The upper bound can be reduoced further by evaluating 02' Q3' •••
and we proceed to do this. Now,
°= 2(N-2)/2(N° 2)M(3N-.o5 N-2 -2/N)
2 - -2·~' -2 ' e
Hence, from (7.18), putting s = 3 and qN!, we get2
or
0.065 < Q2 < 0.066 •
Similar evaluations give the following results:
•
54
0.101 < Q3
< 0.103
...504 =< O(!.-), N
0.028 < Q~ < 0.0?9
0.0055 < Q6 < 0.0056 •
Now
Q = Ql - Q2 - Q3
+ Q4 + Q5 - Q6 - •.•
<.629-.065-.101+.029-.005= .487
and
Q > .542-.066-.103+.028-.006 ~ .395.
The latter terms diminish very rapidly and do not affect to the
second decimal place. Now,
e-1/2 Q < J < Q '
that is,
.?39 < J < .487 •
Since IN =< .196 + J, we have
These calculations are valid to
Hence the first term, PO' in the
*r =1 is given by
.'O(N ~) and to two decimal places.
i f p( ~l- ) bexpans on 0 r ~ rO
a out
p =~ (l-r )(N-2)/2o 1t 0
where IN is bounded as in (7.24).
8. *Bounds on Per ~ r o).
From (4.9), we have
sin Q = (1_r*)1/2 L- "f' (".L71/ 2
"
Hence fOr a given value of 'rO
' the maximum value of QO( ".), sayI
90, is given by
(8.1)
( )1/2~ l-rO
or
(8.2)
FUrthermore, from (,.2), we have to O(N-2),
i N-20s n 0<N-2
Therefore,
" N-2s:m 80<-----
(N-2)cos eO
1-nor
co
f N-2 )sin SOds(".
o
"
From (1.24) it follows that
0.43'(1 )(N-2)/2< P( * ) < 0.68~(1 )(N-2)/2-rO r > r O _v -r p
n - nrO 0
IllTe remember that the results EPPly to such values of r O
56
for which sin3 QO is negligible in comparison to 1. If, for in
stance, we consider 0.01 as negligible, we may take the maximum
value of sin 90 = .21, i.e., r O~ .978. In fact the bounds may be
valid even for much l~er values of r O' as (8.~) suggests that the
neglected terms in (3.6) may be of 0(sin4e) rather than 0(sin3e).
A similar argument shows that r~f- = .. 1 if and only if
If -1 < 1" < 0 andothat near r*= - 1,
i=2,3, ••• ,N and X < 0, xl f 0 •
is close to 1, it follows from (4.9)
,and if the integral corresponding to IN be denoted by IN '
~ ~ / £ 1f' (l.)jN-2 )/~().)dA •
-00
,_4 chen ge of variable from X to -X shows that ~ = IN.the first term, p~, in the expansion of p(r~ ~ r~) about
Therefore,~f-
r = -1 is
(8.4)
and
,p =o
IN ( f )(N-2)/2- 1 + r O 'n
It is instructive to compare (1.?5) with the probability of an
ordinary correlation coefficient r, based on a sample of size (N-I),
57
exceeding a given value r O close to 1.
If the population means are known the frequency function of
r for a sample of size N-l is given by
r(N-l)= _ 2 (1_r)(N-4)!2 t2- (l-r) ~ (N-4)!2
rn f(N;2) )
N-42T r(B)_ _ ~?_ (1 )(N-4)!2 {1 N-4(tl)+ - ..'
::z N 2 -r - - 2 •• • J •rn r( 2-) 2
Therefore the first term in the expansion of per ~ r O) is
Hence
(8~6)
This ratio tends to zero as N-> co •
CHAPTER III
JACOBI POLYNOMIALS AND DISTRIBUTIONS
OF SERIAL CORRELATION COEFFICIENTS
1. Introduction.
It was pointed out in section 1 of Chapter 2 that a statistic
closely related to r* is given by
(1.1) r =1
N-lZ xix'+l
i=l 1.
NZ x2
i=l i
•
which has the advantaqe of being a ratio of two quadratic forms. In
this chapter the approximate distribution of r l will be obtained in•
terms of a beta distribution and a series of Jacobi polynomials.
Denoting by ql the numerator of the right hand side of (1.1),
the characteristic roots of the matrix of ql will be utilized to
calculate the cumulants and the moments of ql' The asymptotic normal
ity of ql and of r l will be proved separately. Following a dis
cussion about the properties of Jacobi polynomials a method will be
developed to obtain an approximate frequency function of the ratio of
two quadratic forms. Of course this method has limited applicability
but suggests the possibility of using orthogonal polynomials in approx-
imating the distributions of certain statistics whose exact distribu-
tion cannot be obtained expressly in terms of known functions and inte-
grals. It can also be used for calculating percentile points, even
when the exact distribution is available, but is tedious to handle.
2. Jacobi polynomials.
~efore obtainin~ the distribution of r l it is necessary to
state some properties of Jacobi Do1ynomials. These properties are
given below without proof. For the proofs a reference can be made
to Orthogonal Polynomials, by G.Szego L-47_7.Write
f(x;a,~)= (l_x)a(l+x)~ . ;a,~ > -l,-l;x ~ 12a+13+1B(a+l,~+1)
The Jacobi polynomial p~a,~)(x) is defined as the orthogonal poly
nomial of degree r with we:i.ght function f(x;a,~). The normaliza-
tion is achieved by the restriction
Explicit1v
r mYo (r+a)(r+~)(x-l)r-m(_x+l) 2_ ~ ~, r:O,l, , •••.
O m r-m t:. cm=
It can be shown that
if r r/ k
if r = k = 0
if r =: k = 1
if r =k > 1
A very useful property is
60
'( )k (. ) B(a+1,13+1) kt (ll B)( )= -1 f x,a,13 B(a+k+l,~+k+l' 2k Pk " x
p(a,~)(x) satisfies the differential eauation.r
2 d 2 d(2.4) (I-x) --4 + /-~-a-(a+13+2)x _7 ~ +r(r+a+~+l)y = 0dx~ - aX
and is related to a hyper?eometric function by the equation
Changing the variable from x to z by
x = 2z - 1
dx= 2dz
let f(x;a,~)dx be transformed into g(z;a,13)dz and p~a,p)(x) to
Q(O:,P)(z). (Note. o(a,p)(z) has no oonneotion with the second solu-r - r
tion of (?I.d, a no'liation used by Szego). Then
and
,
(2.6)
Equation (2.3) go~s into
61
Now, consider the characteristic function of a variate x whose fre
quency function is g(x;a,~). We have
1
0') t)= tetx== JetXg(x;a,{3 )dx
°
The partial sums of the series under the integral sign are dominated
by the function e ~xl in /-0,1 7 which is integrable over this- -range. Therefore, we can integrate term by term and thus obtain
(2.8)co
= 2:r=O
r(p+1+r)f(a+@+?) t r- y
r(~+1)r(a~~+2+r) r.
== F(~+l, a+~+?, t)
where F({3+1, a+~+?, t) is the confluent hypergeometric function
whose Taylor series expansion about t=O is given by (2.8). The series
(2.8) is absolutely convergent and since a, ~ > -1,
<1+Jfr+J~l2 +
+ •.•
_ Q ItI••• - v •
Hence, F({3+1,a+{3+2,t) -is ~i entire function and so
From the inversion theorem of characteristic fun~tions, if
,0 ~ x < 1,/
( 0) 1 J -tx ) (l_X)I1X~2.1 ~ni je F{~+1~a+~+2~t dt- B(I1+1,~+1)
io
where (f ....dt stands for lim J ... d'ii, and i is imaginary) c ->00 -ie
unit.
Replacing a and ~ by a+k and p+k in (2.10)' and differ-
entiating both sides k times with respect to x we obtain
(2.11) 1 + k -txF( )2ni j- t e ~+k+l,~+o:+2k+?, t dt
= g(X;I1,/3) B(a+l,p+l) (a C!) )--:---~'""'--"(" k' Q ' to' (xB(a+k+l,/3+k+l ) • k
Similarly, the characteristic function of a variate y with frequency
function f(y;a,/3) is e-t F(p+1,a+/3+?,2t) and for -1 < y < 1 ,
and
•
,
63
3. Distributions of certain statistic~ as series of Jacobi poly-
nomia1s.
Let R be a statistic whose range is contained in ~o,1_7.
Let either its characteristic function or its exact moments up to a
certain order and a few dominating terms in all its moments be known.
Further, let it be possible to write its characteristic function 1
¢R(t), in the form
(3.1) ¢R(t)=aoF(~+I,a+S+2,t)+alt F(~+2,n~~+4,t)+ •••
+Bk tkF(~+k+l,a';'~+:'k+2, t)+ •.•
00 k= Z ~ t F(~+k+l,a+~+2k+2jt)
k=O
where a,~ > -1. It is possible that ak = 0 for all k ~ r, in
which case the series will terminate. Since IF(~+k+l, a+~+?k+2, !t/)Iitl
~ e' , for all k ~ 0, the series on the right hand side of (3.1)
converges absolutely if
1.... ' a,.. t I.J..iil . ~t. I
k ->00 iak_i! < 1;
•
and it converges absolutely for all values of t if
8klim - =0k -> 00 ak _1
Suppose either the series ().1) terminates or converges abso-
:}.ute1y f or all values of t. From the inversion theorem of char-
acteristic functions the freouency function of R is given by
64
Let us formally carry out the term by term integration. From
(2.11) we will then write, fQrmally, the frequency function of R as
in the range of R, and f(R)=O othe~~Jise. It will be seen that the
terms of the expansions (3.1) and (3.3) correspond by means of the
relation (2.11). This process is justified analytically in the case
when partial sums in the integral (3.2) are dominated by an integrable
~unction of t. However, in practical applications in most cases we
are seldom interested in the convergence properties of our expansions.
As Cramer states~ "What we really want to know is whether a small
number of terms - usually not more than two or three - suffice to
give good approximation to f(x) and F(x)" /-7, p. 224 7. It will be- -seen later that by this method we ob'tain expansions for frequency
functions and distribution functions of serial correlation coefficients
in series which are asymptotic, and have some other desirable proper-
ties; for example, the moments of the fitted series agree to a high
order of accuracy with the moments of the sta'Gistic under consideration.
Now the maximum value of Q~O:'~)(R) in the rmge fO,lJ is
max Q(a'~)(R) = (k1:Q) where q = max(a,~) •
O<R<l k ~
6,
Hence, by the ratio test, the series (3.3) converges for all values of
lim 14kakk -> 00 IBk_1
<1
A similar procedure shows that if.
000.4) ¢R(t) = e-t E akt~(f3+k+l,o;+f3+2k+2,2t)
k=O
then we can formally develop the frequency function of R as
This series converges if
lim I 2k81{ lk -> 00 I"ak-~ I < 1 •
The above results are summarized in the following theorems.
THEOREM 3.1. If R is a variate with range L-cl ,c2 _7 where 0 ~ 01
~ 02 ~. 1, and if the characteristic function of R, ¢R(t), can be
written as (3.1) and the partial sums are dominated by a function
~ (t) such th at, e 'tR~( t) is integrable over (-00, 00) then the fre
quency function of R is given by (3.3).
THEOREM 3.2. In 'theorem 3.1 replace
(i) 0 ~ c1 ~ c2 ~ 1 by -1 ~ c1 ~ 02 ~ 1,
In the following sections we will obtain distributions of
66
serial correlation coefficients by the application or the method de-
ve10ped in this section.
4. Independence of rl
and 1'.
Let xl' x2' •.. , xN be independent N(O, 1) variates. Define
N-1ql. = l xjx'+l '
j=l J
p =:N 2l x
j=l j,
and
THEOR~M 4.1. r l is distributed independently of p.
PROOF. The distribution of xl' •.• , xN is given by
2-N/2 -lxi /2
dF(xl ,···,xN)=(2n) e dx1 •.• dXN
=(2n)-N/2 e-P/ 2 dX1
e" dXN
Let1 '2x.=z.p /
J J
N 2, j =1,2, e." N, with l z. = 1
j=l J
The Jacobian of the transformation is ~1, p. 385_7
(N-l)/2P
T~erefore the distribution of p, zl' "', zN_l is given by
67
1
Hence the conmined variable (zl' ••• , zW) is distributed independently
of P, as zN can be expressed in terms of zl' ••• , zN.1 only. Now
and is, therefore, distributed independently of p.
COROLLARY./' k'., q(. 1;t kC P
5. Cumulants of ql .
Before proceeding to investigate the distribution of r l , we
will investigate the properties of the distribution of ql'
As a preliminary, we will state a theorem about, the cumulants
of a general quadratic form in normally distributed variates.
I
THEOREM 5.1. Let x =(~, "" xN) be distributed normally with
mean vector zero and covariance matrix Z, where Z is a positiveI ,
defin5.te. Let u = x Ax, where x is the transpose of x and A
is a real symmetric matrix. If the characteristic roots of AZ are
08
PROOF. The characteristic function of u, ¢. (t), is given byu
¢u( t)=(2n)-N/21 z \-1/2 j expL:' ~X 1Z-lX+tx I AX _7dX
:r(2n)-N/2 I Z 1-1/ 2 J expL--~x '(Z-1-2tA)x_7dx
:: t Z 1-1/? IZ-1_2tA i -1/2
"" 7T (l_2tcj)-1/?
j::l
Therefore~
1 NX (t) = - - E log(1-2tc.)
u 2 j=l J
If by cl we denote the characteristic root of (~E) which is largest
in magnitude, then for ~t 1 < --2:.--,2 l0ll
• Q.E.D.
Note: If Z is the identit.y mat rix, cl ' •.• , cN will be the
characteristic roots of A.
THEOREM ,~. The characteristic roots of the matrix Ql of the form
N-lql "" i:1XiXi+l are
•
(5.2)
PROOF. 1I\1e have
j'lAj = cos 'N+I ' j=l, 2, •• " N ~
1
0 0 0 0 ... 0 1
0 0 0 1 01 0 ... i1--- . - ...i
Write
DN= i -j.L 1 0 0 ·.. 0
II 1 "j.L 1 0 ·.. 0iI
0 1 0I 1 -j.L ·..I
I
"I •I
e I 0 0 0 0 -j.L,
Then it can be easily shown that
PutN
~ =- x ,
so that
or
Let
so that
x =
2x +j.Lx+l=-O
i(4 2)1/2-IJ. + -lJ.-2
lJ. = 2 cos ~
t¢ -i¢x = -e ,-e
70
Now
Introducing DO' which satisfies the difference enuation for N=2,i.e.,
we obtain
The most general solution of (,.4) is
where A and B are arbitrary constants. From the values of DO
and Dl' we obtain
A + B = 1~r>! -i~
-Ae .J..'P. Be = -IJ.
which give
and
B = -
A =>
.i¢e
2i sin ¢
21 sin ¢
r i(N+l)¢ ~i(N+l)¢ '1e -e -'-
..
DN =2i sin ¢
(_l)N sin (N + 1) ¢=
sin ¢
The characteristic roots of 2Ql are the roots of the equation DN= 0
in IJ.~ which is equivalent to the equation in ¢
71
sin(N+l)¢ - 0sin t3 -
This pives
¢ ~ i:r ' j : 1,2, ••• ,N
We note that ¢: 0 is not a solution. Since DN = 0 is an Nth de
gree equation in ~, the N roots are given by
2jn .
~j = cos N+l , j = 1, 2, ••. , N ,
and hence the characteristic roots of °1 are
A - jn j : 1, 2, N •j - cos 'N+! , ... ,
COROLLARY 1- The characteristic function of ql is
N jn 1/2r/J (t) = 1t (1-2t cos IIl+1 )- •ql j=l
GOROLL4RY 2. The limits of the variation of r1
are n+ cos N+l
Cumulants of ql
We observe that if N: 2m, th0n
AN_j +1 = - Aj , j = 1, 2, "0' m;
and if
N = 2m + 1 ,
A 1 = 0, A 1 =m+ N-j+ = 1, 2, ... , m
..In any case
e . From theorem 5.1,
N~ '\ 2.k+1 = 0 k - 0 1 2~ ~ , - , , , ...
j:l J
hence
(5.6)
Now
N I
_ 2r-1( -1)' ~ ,r.K' r:ql - r..... flo ,
j=l J
K' t1 1 ;=: 0, r = 0, 1, 2, ...rr+ :ql
ir.j -injN N .-._" -- 2rZ ).,2r = Z~(e N~l + e N+l)
j=l j j=l 2 r
2 2nij(k-r)1 r 2r N N+l= --- Z ( ) ~ e
22r k=O k j=l
72
If k-r = 0, + (N+1), + 2(N+l),- - ..., then
Otherwise,
NZ e
j=l
2j.t-( k-r )jN+l N
= Z 1 = Nj=l
NZ e
j=-l
Hence, if
2in(k-r)jN+I
2iR(k-r) 2in(k-r)N+Ie - e
?in(k-r)1 _ e N+l
= -1
(i) r ~ N,
1 /-(2r) 2r 2r) 7= ~r _ r N - 2 +(r _2
73
(ii) r = s(N+l)+k where k~ N and s is a natural number
(5.8)
+ 2( ?Nr )(N+l) - 22r _7r-s -s
Finally, from theorem 5.1,
if s(N+l) ~ r < (s+l)(N+l) where s = 1,2, •••
6. Asymptotic normality of ql'
THTi:OREM 6.1. If xl' x2' ..• is a seouence of independent
N-lN(O,l) variates and ql = .E xixi +l then ql' when standardized, is
1.=1
asymptotically N(O,l).
PROOF. First we note that the Central Limit Theorem is not applicable
in this case, as the successive terms in the SUIllil1ation are not inde-
pendent of each other. Secondly, the theorem 6.1 is valid for the
case when the variance of each x is the same finite quantity.
Let
(6.1)
74
From (5.6) and (5.9) we have,
/' q1 :: K l :: 0-..' :ql
"2 :ql:: K. = N-1 •?:q1
q1-IN=I
From (5.9) and (6.1) we have
(6.?) , t 2 N t 2r (? 1) tX ( t)= - 't. E _..:.!.:.-
z 2 r=2 (2r) t 2(N_l):r."
00 (s+l)(N+l)-l+ E E
s=l r=s(N+l)
t?r (2r-1)i
(2r) 1 . 2(N-ll
As N -> 00, for every r we have
lim K, = lim (?r-l)lN->00 2r i Z N->oo 2( N_l)r
, \~ (2r)( N+l)_22r l = 0 •\ r )
Furthermore, the double summation term in (6.2) recedes to infinity
as N -> 00; hence it follows that
limN ->00
Q.E.D.
The rapidity of converpence to normality could be measured by the
kurtosis, as the skevIness is zero.
75
The freouency function of ql' r(ql)' in type A Edgworth series up
to 0(N-2) is given by
1=:-
ffn
where
7.
H ( )_ r r(r-l) r-2 r(r-l)(r-2)(r-3) r-4x-x - x + -2 x - •••
r 2-lt 2 . 21
Moments of r l •
From theorem 4.1 the moments of r l are obtained by
where
utilizing the relations between moments and cumu1ants we obtain
W2k 1 =: 0, k =: 0, 1, 2, •.•+ :ql
For N.:: 4,
tl\I -- '. q
l ·Q - //-1"1 '. -and
(7.1)
76
Vt')o = N - 1 ,c·q
l
Y4" = 3N2+12N-27 ,·ql
Y6' = 15N3+225N?'"5?5N-2205 ,·ql
v8" ~ l05(W-l)4 + 1260(3N-5)(N-l)2+1260(3N_5)2·ql
+6720(5N-ll)(N-l)+l008(35N-53) •
In general the first two terms in the highest powers of N in v2k:ql
are
p is distributed as )(,2 with N degrees of freedom; hence
r(N )/} 2 r' - +r" r 2c· p =~- =N(N+2) ••• (N+2r-2)
r(-)2
, Therefore,
(7.3)
Y2k - = 0,' k = 1,2,3,i +J. :r
l
In general
y. :0
2:r,.I.
N-lN(N+2)
N(N+2)(N+4)(N+6 )
15N3+225N2+525N-2205N(N+2)(N+4)(N+6)(N+8)(N+IO)
(7.4)
77
1.3.5.... (2k-l) {Nk+k(3k_4)Nk- l ) +0(Nk- 2)
N(N+?) ... (W+4k-2)
-k)=- O(N •
Asymptotic normality of rl
.
Let
v2k+l :x = 0, k = 1,2, ... ,
lim "2k 1x = limN ->00 . N ->00
~.1.3.S... (2k-l)Nk/-l 0(1) 72k + -- NN
If ¢. (t;N)x
for every t
= 1.).5 ... (2k-l)
denotes the characteristic function of x, then
It follows that the distribution function of x converges to the dis
tribution function of a w(O, 1) variate. Hence
!!'EOREM 7 ~l. :i:'l' when standardi~ed, is 9sy:nptotic ally N( 0, 1).
The rapidity of approach to normality can be measured by kurtosis
as the skewness is zero. NoW
~ = v~ = 3N2+1?~~~~2 v~ N(N+2)(N+4)(N+6)
~2 - 3
( 2 10 1= 3 1- - • - • 0(_) )N Nt' N3
6 1= - - • 0(_)N N2
78
8. flnproximate distribution of r l •
From (7.3) and (7.4) the characteristic function of ri = z
can be written, if N > 6,
(8.1) ¢ (t) = 1. ~-l ~ + 3N2.l2N-27
z N(N+2) If N(N+2)(N+4)(N+6)
• l5{N3"1:;?rl+35N-47) ~N(N+2)(N+4)(N+6)(N+8)(N+lO) 3t
•
Let us write formally,
(8.2 )00 4
¢ ( t)=F(~ N+2 t)+ ~ a tkF( 2k+l N+ k+2 t)z 2' ?' k='l k -r ' - 2 ' .
Comparing (8.1) and (8.2) we determine SkIS. Writing (8.2) in detail,
2¢ (t) = 1 +..l !... + I.) ~ +•z N+2 11 (N+2)(N+4) 2l
1.).5 ..• (2k-l) t k•+ - - - + •••
(N+2) •.• (N+2k) k!
..-+ •••
23., ~+
(N+6)(N+B) 2!
79
+ a t r rl + ?r+l.!. +r - N+4r~2 1!
+ ~(-.;2r;,.-+.:;;1..::-;)(~2~r...;+)~)...;.•.;...•.;.,.;;(..,;.2k..,;.-...;:l;.:..)(N+4r+~) •.• (N+2r+2k)
t k- r........;..-- + ••• 7(k-r)! -
+
•,
or
~ t k 5 k:l (2r+l)(2r+) •.. (?k-l) kl t( 8 .:3) ¢z(t ) =1 + Lo -, ak+ Lo a J
k=l k! l r=O r (N+4r+?) ••• (N+?r+~k) (k-r)t
where aO = 1. Therefore, we obtain
a + kZl a (?r+l)(2r+) ••• (2k-l)k r=O r (N+4r+2) ••• (N+~r+2k)
k! = \I(k-r) 1 2k:r1
Solvinp for 81 , a~:" a), we get
1 N-la +-=-~-1 N+~ N(N+2)
or
1
hence}
•
a ... -2
)
2N(N+2)(N+4)(N+6)
45BJ
... - -)! N(N+2)(N+4)(N+6)(N+8)(N+IO)
Si..milarly,
80
Now, all the cumulants of q1 are ~nown; hence, it is theoretically
~ossib1e to calculate all the moments of q1 and therefore of r1
and z. Then we are able to determine a1, a2, .,0' to any desired
number. No systematic way of determining all the coefficients could
emerge because of the irregular behavior of the cumulants of q1 for
order greater than N. From corollary 2 of theorem ,.2 we see that
the range of r1 is £"- cos N:r ' cos N~I _7 and that of z is
-/- 2 11 7- 0, cos i+! _ . Hence from (3.1) and (3.3) an approximate distri-
button of z to the first four terms in the expansion is
( / /N+l 1
_ (l-z) N-1) 2z-1 2 _ B(~ , 2)r(z)- N 1 r I 1- N+3 3 Ql(z)
B(T)' 2) - N(N+2)B(T'-)2
3 B(.! , ~)
- N(N+?)(N+4~(N+6~B(2, ~~ Q2(z)2 2
45 B(l , R:1:)2 2 7
- N(N+?)(N+4)(N+6)(N+8)(N+IO)B(~ ,~) Q3(z)-
where the superscript~ for each polynomial Q are (N;l , -~) • Or
() (1_z)(N-l)/2z-1/2dZ L-- N+4 'I;
f v..dz --- .l. z_. B(N+l }-, "- - W ';2' '2)
N+8 : 2 6z(1-z) 3(1-z)2--:::--- ~ Z - - + --"-----l.2!N i\. (N+1) (N+1)(N+3) .
(J.-Z)
(N+l)
3(N+12)
23 .3 rN
81
2 'Itfor 0 < z < oos -1
- - N+
= o otheI'tvise.
Henee, an aporoxtmate distribution of r l is
(8.6)
+ _3(1-ri)2_ ~(N+l)(N+3) )
4t: 2(_ 2)2;>:rl l.-rl
(N+l)(N+3)
l5(1-ri)3 \-----\ 7(N+l)(N+3)(N+5) J -
= 0
1t 1tfor - cos N+:r .:: r l ::: oos N+J:
otherwise •
9. Asymptotic validity of the distribution of r l •
In the previous section we have given an expansion of ¢z(t),
first in a power series of t and then in a mixed series of powers of
t and hypergeometric functions. From this let ter series 1I1e have ob-
tained the distribution of r l in terms of a beta function and Jacobi
polynomials. Naturally questions about the behavior of series (8.5)
arise. The question of convergence of (8.5) and of (8.2) depends on
the knowledge of ak for large k" which in t~Tn depends on the speci
fication of all the moments of r.,. Such information is not available.L
in exact specifications. However, first few dominant terms in all the
82
moments can be specified. From this it is possible to prove the asymp-
totic validity of the distribution of 2z = r"t'
.J.This proof is carried
out in several steps.
§tep (1). 'oITe observe that cos ~ '" 1 + 0(N-2). Since we are goingL~+.1.
to consider the asymptotic behavior of the series we replace th~ range
of z = ri by i-o, I _7. This change will have negligibly small effect
on the value of probaryilities or of moments. ForexaIDple, considering
its effect on the kth moment from the first term of series (8.,),
I t zk-l/2(1_z)(N+l)/2 dz
2 ncos w.;:r
<
(1 2 n )(n-l)!2-cos 'i\T+I...
'B(~ 1)2 ' 2
= O( N~i/')N
Sim e t zk = O(N-k), this change is negligible for any k
less than N and as N --> 00 it will be negligible for all k.
S~ep (2). Let us write
where vO(z), v1(z), .0' are the successive terms in series (8.,); for
instance,
83
Furthermore, let us write
and so on; and So(t), sl(t), ••. for the characteristic functions and
FO(Z), Fl(Z), ••• for the cumulative distribution functions associated
with fO(Z)' fl(z), "" respectively; then
1 N+2so(t) = F(2' ~, t) ,
N+6 t), -,2
at cetera.
Step (31, Denote the sth moment of the distributions fO(Z)' fl(Z),
••• boy' v(O) vel) •.• , resp'ectively. Then v(O) is the coeffi-s:z' s:z' s:z
cient of ~ in the expansion of so(t), v~;~ is the coefficient ~f
s~ in the expansion of sl(t) and so on.s.
Step (4). From the relations among moments and cumulants of a distri-
bution and remembering that odd cumulants of ql are zero and that
every even cumulant is of order N, we find
>,
54
(9.1)
... .
Therefore,
v2 • =1.3 •••• (2s-1)(N-l)s~3.,••• (2s-1)S(S-1)(3N-')(N-l)s-2~l 2(N)s.ql s-
where R(N) stands for an unspecified polynomial of degree s in N.s
v2 • can be arranged in the following form:s·ql
v2s :ql=1.3., ••• (2s-1)L-N(N+2s+2) (N+2s+4) ••. (N+4s-2)
-s(N+4)(N+2B+4)(N+2s+6) •.• (N+4s- 2)+fs_2(N) _7.
Therefore, we can write
_ 1.3.' ... (2s-1) (2s-1)(2s-3) •• '.3.1.sv - - - - ------.-~~--
s:z (N+2)(N+4) •.• (N+2s) N(N+2)(N+6)(N+8) ••. (N+2s+2)
ls..2(N)"" ------..;.,-.;;..----
N(N+2)(N+4) ••• (N+4s-2)
We note that the first term is of O(N-s), the second term of O(N-s- l )
(-s~2 (0)and the third term of 0 N ). Also that the first term is ~
s~z
and the first two terms are equal to (1)v • , i.e., for s > 1,s·z
(9.2 )(0) fs_I(N)v = v =-- _
stz s:z N(N+2) ••• (N+4s-2)
85
Furthermore, it can be easily verified that
" =(1) _ (2)
=: ,,(3) =:l:z "l:Z - "l:z l:z
"2:z= ,,(2) = /3) ... ...
2~z ?:z
"32Z(J)
=:== "3~z... ,
That is, by taking more of the terms of the series (8.,) more of the
first moments of z become equal to the moments of the constructed
distribution.
Step. (5) . Now,
Therefore,
00
+ i:s"2
... svSi
N(N+2) ••• (N+4s-2)
lim ~N t¢z(t)-~o(t)) j =: limN -> 00 N ->00
i t I I,' F( 3 .N+6 {. )- -2' 2 ' vN+2 I
00 (s_2(N)+ i:. --",-=.~--
s==? (!J+4) ••. (N';'4s-2) st
R.'S(N) t S
N(N+~) ••• (N+4s-2) s!== 0 •
Thus 0z(t) is asymptotically equivalent to ~o(t), ~l(t), •.•
86
with successively better apnroximations and hence F(z) is asymptotic
ally equivalent to FO(Z), Fl(Z), •.• with successively better approxi
mations. lATe can improve our approximations by taking more terms of
the series (8.2), stopping before the term for which ak/~_l is no
longer of O(N-l ). Thus series (8.2) is to be regarded the sum of
a finite number of terms, stopping either at the term but one for
which the coefficient ak is smallest or at some earlier one when we
have already achie~d as much accuracy as we want. In our case, taking
one, two or three terms from series (8.2), which is the same as taking
the same number of terms from series (8.5), means that the moments of
constructed distributions agree to the moments of z within relative
orders N-l , N-2 and N-3 respectively. This is clear from step (4)
equat ions (9.2).
~tep (6). It can be demonstrated that if we integrate the series (8.5)
from 0 to zo' we actually obtain an asymptotic expansion. ~Triting
we have
f(z)dz
,Zo
F(zo) = j V6(Z)dZ +
o
,ZO/ vl(z)dZ+'0
...
Now,
87
in the notation of ~arl Pearson. Furthermore,
, .
!~(zo)l
•,
and similarly,
Therefore ~
We may, therefore, write
2 ) 1 N+l N+4 t 3 N+l 1 N+3) \P(rl < Zo .-J I ("~,-)- -2.(-~ I· (_,--)-1 (-,- \- Zo 2 2 N N+2) ~o 2 2 Zo 2 2 J
•
10. Some other' properties of the ser1.es (8.6).
(i) If the series for f(r l ) given on the right hand side of
(8.6) is written
where
(N+4)(1_r2/ N- 1 )/2 I 1 2)v (r )... _ . ~ ~ _ Jr 2 _~ ~1 1 2N B(l !.:!) (1 N+o£. j
2' 2
,
88
and so on, heuristic considerations show that over the effective range
v2(r1 ) ].vOCr
lJ == 0(;2)
To show this we observe that the variance of r is O(N-I),
so that values of f(rl ) outside some rcnge on either side of zero,
which is 0(N-I !2), are negligibly small. Thus r, may be regarded...as 0(N-1/ 2) over the effective range of rIo Hence, over this range,
i.e., for !rll < A/NI/2, where A is a constant independent of N,
we have
..., -and
and so on. This statement is further strengthened by comparing the
maxima of vO' vI' v2 ••• over the range of r l • It is found that
89
and so
max \ vII 1---- = 0(-)N
Similarly,
(ii) 1111'e make the transformation
t =
in (8.6). The frequency function of t is given by
= a otherwise .
This shows that t is cp?roximately a Student statistic with
(N+l) degrees of freedom.
11. Correction for the sample mean.
90
In almost all practical cases we do not know the population
mean of the XIS. It is, therefore, necessary to define serial cor
relation r1, corrected for the sample mean. Let
(11.1)
where
and let
(11.3 )
This correction complicates the mathematics considerably, as the char
acteristic roots of the matrix Ql of the form q1 are not tractable.
However, by straightforward expectations,
N-1--Nand
•
The independence of the distribution of r 1 from that of p-/ /
can be proved easily. Now, [p = N-l, Gp2= (N-1)(N+1), we obtain
(ll.h) t! - N-l 1 1v r l ... - N' 'N=!'" - N
91
From (3.4) and (3.5), if N > 3, we obtain
... 0 otherwise,
where cN_1 is the least and c1
is the highest characteristic root
of Ql' One root of Ql being zero, we always take cN = 0 ~ For
N= 2,
r l ; - 1/2 with probability 2.
For N'" 3, the characteristic roots of Ql are 0, 0, -2/3. Hence
2..2Yl/32 2
Y1+Y2
where Y1 and Y2 are independent N( 0, 1). Therefore,
(- ) - 3 ( 3 - )-1/2 3 - )-1/2 - / -f r 1 dr1 =;; - 2 r 1 (1+ 2 r 1 drl , -2 3 ~ r 1 ~ o.
For N = 4, the characteristic roots of Q'l are
92
c1 = ( jFJ -1)/4, c2 =- - 1/4, c3 = -( J5 +1)/4 •
For N=5,
c1 = 1/2, c2 =< (-fti -4)/10'03= -1/2'04= -( J2J. +4)/10
It 1.'8 th t (-Q) 2n d 2na guess a c1 1 = cos N+l ,an 0N_1 < - cos N+f •
12. Distr:i.bution of the circular serial correlation coefficient.
Consider
(12.1 ) =',....
p
where xl\T+1 = ~ and xl' ••• , ~ are independent N( 0, 1) variates.
The exact distribution of R1
is givan by R. L. Anderson L-2 _7.The characteristic function, ¢ (t), of c isc
(12.2)
Let
N-lrj; (t) =- 11" (1-2t oos 21!J.)-1/2
C j=1 N
21!-i • 1 2 1OJ = cos -N- , J =< " ••• , N-
N-1 k ' N-l ~ i·/N ~ 4~/N' k~. ~ ~ ( en J -en~~)",c='l{",e e
j=l j 2 j=l
•
..If k = 2m+1 < N
=_~ ~ (~) Nil e2nij(?r-k)/N •2 r=1 j=l
Hence
93
N~l e2nij(2r-2m-l)/N == _ 1 •j e l
If k == 2m < N
N-l 2m+lZ c.
j=-l J
1 k k=-:I{ Z-()
2 r=l r
=-1
(12.4) m-l 2m 1 r(?m)N _ 22m 7 .6 cj
=- :Tm _ m _j=l 2
For k ~ N, we oan proceed on lines similar to those used for calcu
lating the cumulants of ql.
For N > 2m
Hence for N > 4 ~
K = -1l:c
t\" = N-22:0
1<: = -83:c
K =- 6(3N-8) .4:c ;
and
v1 "·0
v?:o
= -1
= N-l
v = -3( N+l)3:0.
/'
94
Furtherml:"re,
t~ pr = (N-l)(N+l)(N+3) ... (N+?r-3) •
-Therefore the moments of HI' up to order 4, are
(12.4)
Henoe
1\1 - • - -I:R1 N-l
N-l 1v2,R
1= CN-l)(N+l) = N+l
v3:R1 = - IN:..13....(N-.+-)-)
1 t 1 t 2 3 t 3rbR
1(t) = 1- N..l TI + N+l -:IT - (N-l)(N+3) 3T + •••
which can be written as
•
t t t 2 t 3 1 t 1 t 2(12.,) ¢R1(t).... e- (1+ IT + -2; + 51 + ..• )(1- 1\T~l It + N{.l2r
3 t 3 3 t 4(N-l)(N+j) Ji + 1N+1HN+j) 4T + ••• )
-t( N-2 t N2-N-4 t
2
= e 1+ '1\T:I n + (N-!HN+lj 21
N3+3N2-10N-24 t 3 N3+5N2-14N-48 t 4+ -+-......,....., ""1.=+ ••• ).
(N2~1)(N+3) 3L (N2-1)(N+3) 4\.
Let also
(12.6) ¢nR (t)=e-t;-F(~,N_l,2t)+a1tF(~,N?1,2t)+a2t2F(N+2,N+3,2t)+ ••• 7.1 - 2 2 2 -
95
Comparing (12.5) and (12.6) we determine the values of al , a2~ a3
and
a4 as
t;. = 0
From (3.4) and (3.5), taldng ~. (N-4)/2, ~ = (N-2)/2, we obtain
Or,
= 0 otherwise,
where b5, b6
... can be determined in the way outlined above, and
. (N-2 N~i')where tne ~~perscripts of the polynomials are .---, t. The series2 2
(12.7) is exact up to 4 terms, that is, the first four moments of the
fitted series are equal to the moments of RIo The remaining moments
96
are approximated at least to 0{N-2). Neglecting b5P5(Rl ) and later
terms, close to 5 per cent values of the positive tail of the distri~
bution are oalculated for three values of N. It serves as a rough
comparison with the exact 5 per cent values given by R.L. Anderson
N Calculations Exact 5 per oentvalue
10 peRl .:: .36)= .0496 .360
30. peRl .:: .26)= .0483 .257
60 IP~l ~ .18)= .0610 .191
peRl ~ .20 ),* .0439
The closeness of the results is very satisfaotory. The calcu-
lations were carried out with the help of Karl Pearson's IITables of
the Incomplete Beta-Functions", which give values of the argument only
up to two decimal places. Since our object is to illustrate the theory
rather than providing a table of percentage point~,more refined calcu
lations were not attempted.
A proof of the asymptotic behaviottr of the distribution function
of Rl obtained from f(Rl ) can be developed on the lines of a proof
which is given in section 9 for the distribution of ri
•
A similar method applied to the circular serial correlation co-
efficient without mean correction, RI , where
97
gives an approximate distribution of Rl , if N is even, as
(12.8)
::: 0 otherwise e
The first N+4 moments of (12.8) are exactly equal to the moments of Rle
13. .90ncluding remark.!.
We conclude this chapter with the remark that the method of de-
veloping the distribution of a serial correlation coefficient in a
product of a beta distribution and a series of Jacobi polynomials
produces better approximations to the true distributions as compared
to the method of smoothing the characteristic function. This latter
method has been widely used by several authors. For example, Dixo~
~9 7 and Jenkins .;-21 7 have obtained smoothed distributions of cir-- - -cular serial correlation coefficients and their functions. If we com-
pare Dixon's distribution of the first circular serial correlation co-
efficient uncorrected for the mean, i.e., in our notation Rl , with
the distribution (12.8), we observe that the first term of the series
(12.8) is the same as Dixon's frequency function. But whereas in the
smoothing method there is no way of improving the accuracy for a fixed
sample size, our teohn1que provides a method of improving the accuracy
of the fitted distribution to any desired extent. Even for small sample
98
sizes, by taking sufficient nuraber of terms in the series; we can im-
prove the accuracy to any desired level, as the successive terms de-
crease very rapidly. This was demonstrated in section 12 by computing
the probabilities of the circular serial correlation coefficie.t cor
rected for the mean, RI , from (12.7) and comparing them with the exact
values given by R.L. Anderson L-2_7. Even for a sample of size 10,
and utilizing only first two terms of the series, we obtained a result
which is correct within 0(10-4).
Finally, it may be observed th&t approximate distributions of
partial, multiple and other ordinary serial correlation coefficients
can elso be obtained by the proposed methodo In the Case of first
two serial correlation coefficients and their functions, it will be
carried out in Chapter 4.
J
e.CHAPTER IV
DISTRIBUTIONS OF PARTIAL AND MULTIPLE
SERIAL CORRELATION COEFFICIENTS
1 c Strrllll1ary •
It is indicated by section 3 of Chapter 1 that we require the
distribution of a partial serial correlation coefficient, b, between
Xt and xt _2 eliminating xt _l ; and of a multiple serial correlation
coefficient, R, between xt and the combined variable (xt _l ' xt _2),
to test different hypotheses concerning an autoregressive soheme of
order 2
Since band R are functions of 1'1 and r 2, it is necessary
to investigate the joint distribution of 1'1 and r 2• A method which
is due to Koopmans L-28_7 yields the exact distribution of 1'2' when
the sample number, N, is even. It is also applicable to the deriva-
tion of exact distributions of serial correlations with even lags when
N is even. These distributions will be developed in the following two
sections 0
The distribution of rr. for N even is defined by diff~rent~
par ts of the range; furthermore;> its exact distribution for N odd is
not available. Therefore, an q>proximate distribution of r 2, N even
or odd, will be obtained in terms of a beta distribution and Jacobi
polynomials. In later sections approximate distributions of band
R will be obJuained 0
lno
? • Cumulants of q •______.....:5.
Let ¢s(tjN) and xs(t;N) denote the characteristic and cumulant
generating functions, respectively, of the form
N-sqs = ~ x(i)x(i+s),
i=l
where s is a natural number < N/2 and x( i), i=1,2, ••. ,N are inde-
pendent 1\T(O,l) variates. We will use the notation xCi) and xi in-
terchangeably. If C denotes the (N x N) matrix
0 1 0 ·.. 0 0
0 0 1 ·.. 0 0
C ::0
0 0 0 0 1
0 0 0 ·.. 0 0
then the matrix Qs of the quadratic form qs is
Q = (Cs + C's)/?s
,where C is the transpose of c.
Let ms be the greatest multiple of 6 in Nand j the
remainder, 60 that N= ms + j.
qs can be written as the sum of the following s quadratic
forms ~
rn-lq ~ ~ x(r+ks)x(r+k+l s); r = ly2, ••• yjsr k=O
m-2qsr "., k:OX (l:'+ks)x(r+k+l s); r""j+l, •.• , s •
,
101
If s = 1 then j = 0, and we have found the cumu1ants of q1. If
s > 1, then qsl' qs2' ~.t, qss are all forms like ql' and no x
which appears in one of them appears in any other of them; hence they
are mutually independent. Each of the first j forms involves (m+l)
of the x's and each of the las t (s-j) forms involves m of the x IS.
Therefore,
•
Or m+l . m
=L- )(' (1-2t cos k\.) -1/27j /- )( (1-2t cos ~) -1/27S-jk=l m+2 - - k=l m+J.-
which, incidentally, proves
THEOREM 2.1. The characteristic roots of the matrix Qs are
cos~, k=1,2, ••• ,m~1, each repeated j times;
kn ( )cos m;I' k=1,2,o.t,m, each repeated s-j times.
We observe that if N is a multiple of B, i.e., j = 0, there are only
m distinct roots of Q given bys
cos ~:1 ,k=1,2, ••• ,m, each repeated s times.
Since cos e is a decreasing function of e in the range
i-G, n _7, we have
kn kn(k+l)n k 1 ,cos -:-'2 > cos ~l > cos~~ J "= , 0.', nl.v_m.... JI'-~ In+£:
If j 10 and the distinct roots of Qs are denoted by 01' 02' __ .,
102
02m+l in decreasing order of magnitude, La.,
(2.6) °1, > 02 > 0'. > c2m+1 '
then
knc2k_1 : cos m+2' k=l, 2, •.• , ~l;
C2k = cos ~n:' k=l, 2, ••• m.
Taking the logarithm on both sides of (2.4)
If the rth cumulant of qs is denoted by K (q ~N), then it followsr s
from (5.6) of Chcpter 3 that
(2.8) K (q :N) = 0,' r= 0,1, •••2r+l s
Furthermore, if r ~ m, then from (5.9) of Chepter 3,
For r > m, we can similarly write down expressions for It 2r( qs :N)
with the help of equation (5.9) of Chapter 3. Hence, all the cumulants
of q are known.s
THEOREM 2.2. qs' when standardized, is an asymptotically N(O, 1)
variate.
The proof is similar to that given for the asymptotic normal-
ity of ql.
103
3. The exact distribution of r 28 when N is even.
and
Let NP - ~x·2- '-' i
i=l
JHEOREM 3~1. If N:::: 2sm + 2j, where 2sm is the greatest multiple
of 28 in N, so that j < s, then the freauenoy funotion f(y) of
r2s
=. y, when j :J 0, is given by
(3.1)
(N-2)/2 r-l _ cs - j - 1+-_.1:/ .1
( ) -. 8-J-s-j-l i=l ~w
= 0
if c? < y < c2 l' r=l, 2, ..•} m ,L-I' - r-
otherwise ,
fwhere 7t" indicates that from the product that factor is omitted
which involves the characteristic root with respect to which differ~
entiation is being performed.
If j ~ O~ the first summation is to be disregarded.
PROOF. We only consider the case j:f 0, j :::: 0 being a special case of
it.
104
The joint characteristic fUnction of p and Q2s denoted by
¢(t,Q), where t corresponds to p and Q to Q2s' is given by
From here we can proceed on the lines of the argument given by
Koopmans ~28_7 in his discussion of the distribution of a ratio of
tWQ quadratic forms. Without reproducing his arguments, we arrive at
the following integral expression for the distribution of y:z r 2"s '
f( ) . (N-2)/2y = 2ni
•
where Yy is any curve proceeding from z = y into the lower half
plane, crossing the real axis at a point z > 01' and returning to
z ~ y through the upper half-plane.
Now, if g(z) is a function analytic within and on a closed
curve Y, we have
I ( g(z)dz 1 2l r -1 •'2nI I = (r-l}t 1: g(a) , if a is within or on y,
! r () r-(z-a) (aY
,.. 0, if a :f.s outside of Y •
Hence the tb-aorem follows.
105
4. The distribution of r 2•
If N = 2m, the distribution of r 2 is obtained fr~m theorem
3.1 by taking s = 1 and j = 0 I
, if c2 2u+ < r 2 .::: c2u'
u =: 1, 2, ••• ,m..l
:: 0 otherwise,
where
unc2u "" cos jji;I , u". 1, 2, •.• , m.
The frequency function (4.1) is exact but of little practical
use if N is even moderately large, say 10 or greater, due to the
involved calculations for percentile points.
For N odd, the exact distribution can be put only in the in-
tegral form.
We therefore use the theorem 3.1 of Chapter 3 and obtain the
distribution of r 2 in terms of a beta distribution and Jacobi poly
nomials.
Follo'tl1ing the method whi.ch gives cumulants and moments of ql'
we obtain the moments of q2' if N.:: 6, up to order 6 as
v == 02k+1tq2
"2:Q2 :: N-2
k ""' 1, 2, ••.
106
V4:Q? ~ 3N2+6N-48
v6:Q? == 15N3+180N2-60N-3600.
In general, the term of highest power in N of
( ) k2k··1 N'"
Hence, the moment s of r 2 are
v2k • is 1.3.5 •••·Q2
k=1,2, ... ,
== N-2)\T(N+2)
N(N+2) (N+4)(N+6)
15N3+180N2-60N-3600== _ •...;;.,;;.;---.......;"-'--------N(N+? )(N+4) (N+6 )(N+8) (N+10)
and in general
On the lines of the proof given for the ;proach to normality of r l ,
it can be proved that r 2, when standardized, tends to a N(o,l) var-
iate as N-> co.
Writing 2 0z(t) c an be writtenr 2 == z,
¢ (t)=-l+ _N-2J. 3N2+6N-48 t 2
(4.2) ~+ -+ ... .Z N(N+::» It N(N+2)N+4)(N+6) ?1
Let us also write
(4.3) ri (t)= ~o F(2k+l': N+4k+~ J.-)'flZ ~O 8k . -r' 2 -, u
107
Comparing (4.2) and (4.3), we dete:CTilL,a up to 83
,
Hence from section 3 of Ohanter 3, the expansion of the frequency
function of z, up to third degree polynomial, is
= 0
if 0 < Z < 0062 nI-NT1 7 '-2- +1
otherwise •
(N-l,_ 1) (!-2.~,N2-1)Now, Q~ 2 (r~) can be expressed in terms of P2 (r2 ), and
thus we obtain
=
n nif - 006---- < r < cos . ..L-~ 7+1 - 2 - rN~. 7+)1
2 - - 2-
:' 0 otherwise •
103
Now, if Zo = r~, then from (4.4) ,
( ~ 2 ( )p r 2 ~ r O) = F(z ~ zO) = F Zo ' say.
1 N+l N+4 /- 3 N+1 1 N+l 7=I (-,-)- --r,;",;\ I (-,--)-I (~,--) •Zo 2 2 N,N+l) ~ Zo 2 2 Zo 2 2 -
Hence, apart from the fact that the difference
I (1 }l:!)_ I (o! !:];)Zo 2' 2 Zo 2' 2
will be considerably smaller than I (1~) we observe that theZo 2' 2 '
second term is O(N-l ) as compared to the first term; and therefore
the series is an asymptotic series. We can write
(4 6) I 2 2) (1 N+l (N+4)/- (3 N+l) 1 N·:·l) 7• P\r2 .:5 TO =3:. 2'"2,-r)- NnT+!J- I 2 ~2'T -I 2("2 s2 - ·
ZOo TO rO
Since a2 ::::.- 83 ::::.- 0, the error in (4.6) will be O(N-3) or less.
5. Bivariate moments of rl and r 2•
To find the bivariate moments of r1 and r 2, it is not possible
to use the joint characteristic function of ql and Q2' as the
matrices of these forms are not commatative and cannot be reduced to
their diagonal forms simultaneously. Hence we proceed with the straight-
forward method of calCUlating the expectations of products.
We first note that
109
B. Xi2r+1 = 0," r=O,1,2, ... ,
and, therefore, in taking the expectations we disregard all the terms
which contain odd powers of XiS as a factor. For example:
J 2 2 ~ N-l 2 N-2 2t,qlq~ ~ ~( Z x:-xi +l ) ( ~ x.xo+2)
c i=l ~ j:l J J
powers of XIS as a factor_7,
the double summation extends over values of i and j such that no
subscript of x is less than 1 or greater than N. Thus
or
e 2 2 2C~qlq2 : N +9N-30
. We have used the fact that· t x2=1, f x~3 •
The results of the calculations are the fol1owing~
lIn
t- 23 2 4 6 e 4G qlq2= 6 N + 8N-21 , (., qlq?= 0 ,
(: q~q2= 0, t qtq~= 3N3+108N2+159l\J-l~.J4, t:)qiq~= 0,
tqiq~= 3N3 + 66N2
+ 417N - 25?6, tqlq~ = 0
,c' k s'''''riting 'lJks = urlr2' we can summarize the moments, including
the univariate moments, up to order six as follows:
Order 1:
Order 2:
Order 3:
Order 4~
Order 5:
III
Order 6:
(5.1) _ 3N3+66N2+417N-2526))24- - N(N+2) ... (N+101 '''15 = 0 ,
_ 15N3+180N2-60N-3600"06- -N(N+2) •.. (N+IO)
6. The distribution of partial seri~~ correlation coefficient.
In an autoregressive scheme of order 2 ,
we have seen that
(6.1)
where PI and P 2 are population autocorrelation coeffioients wi th
lag 1 and 2 respectively. It is easy to see that -~ is the
partial autocorrelation between xt and xt _2 when xt _l is eliminat~
ed. Substituting the estimates of PI and P2
we get a consistent
estimate of ~ as
(6.2) b ::
Further, define the multiple autocorrelation between x andt
as
112
A sample multiple serial correlation R can be defined by
2 ( 2) 21 - R = l-b (1-r1 )=
"2Since r 1 < 1, the condition R > 0 gives
We already know that if - /- N+l 7m - - ,- ?-
from which we conclude that the limits of the variation of Rare
['0, 1 _7 and thoBe of bare .r- cos m.~ , 1 _7.
We determine approximate moments of band R by the following method.
Fr<ri' (6.2)
(6.5)
( k+2 (5+ 3 )r1 + •••
.."
113
Taking the expectations on both sides of (6.5) with the help of (,.1),
and rearranging the terms, we obtain
(6.6) ,: (1 . )k_ 1 k(k+l) + k(f'2k2-5k+6)
~. +0 - + ?(N+?) U{N+?)
k(k4~16k3+,lk2_76k+16) I 1... ---sN(N+2)(N+4) + O'N(N+2)(N+4J{l\J+6)'
This approximation for i(l+b)k is valid for k small in
compari.son to N, i. e ., for k < IN. For higher values of k we will
need more terms. OmittL1g the terms of o(N-4) we have
;:l
( (l+b)
and
Unfortunately it is not possible to determine exactly the moments of
(l~b). However, working with the approximations to the order N-4 for
these moments, let us ~~ite the characteristic function of (l+b) as
terminating the series at a2 " Proceeding in the manner of section 3
of Chapter 3 we obtain
N+3 _ N+3 31ii+'2 ... 81 - "N+"2 - N(N+2}(N+4) ,
or
114
a = 3 •1 - l\T( N+:? J( N+4 ) '
N+5 N+5 N+5 1 112a2 + ?a1 N+G +~ = N+2 - N(N+2J - N(N+2)(N+4) ~
or
Hence,
Now,
and
Furthermore,
and
0: 0 otherwise.
N+5 ( 2 7+ N+i I-b) _.
Hence, for large N, the effective ran~e of the variation of b is
115
O(N-1/2 ). Therefore, in thi.s ranl'.e, the second and third terms :ot·. the
series in the square bracket are O(N-51?) and O(N-l ) respectively.
-1/2)Ordinarily, we would expect the second term to be O(N ; but in
this particular case al is O(N-J ) instead of O(N-l). HowQyer
it can be proved, as in the case of r l , that the distribution func
tion of b obtained from (6.7) will be asymptotically valid repre-
sentation of the true distribution.
If we consider ncos------
r(N+l)I?+l 7- -.-..J 1 and denote by
[r(l+b)k the expectation of (1 + b)k with respeot to the function
(6.7), we find after rome reduction
f. (l+b)k= 1 + k(k+l) + k(kJ-2k?-5k+6 + O( 1 )v f 2{N+2) 6N(Nv?) 'N3
1'1 k e kA comparison with (6.6) shows that Cf(l+b) agrees with ~(l+b) to
O(N-3) •
7. The distribution of multiple serial correlation coefficient.
In section 6 a definition of a multiple serial correl~ion, R,
between xt and (xt _l ' xt _2) was given by the equation
Hence J
Expanding the factors, we obtain after rearranging
116
: 1 _ 2k ~ k(4k-l) + 0(1 )N+2 !,frN+2) ;J'
the approximat ion being good for k small as compared to N.
Writing T = 1 - R2 and the characteristic function of T,
¢T(t), in the form
rl (t)=FCM ~) (N+2 N+6'{IT 2' 2' t + ~t F""2 ' 2' t) + ••• ,
we can only determine al effectively as a2, 83
, •.. require more
exact knowledge of the moments of T. We therefore consider the
series for ¢T(t) to be terminated after two terms. omitting the
terms O(N-3) and less, we have
3N(N+2)
Thus the distribution of T may be written up to the first degree
Jacobi polynomial as
~ 0 otherwise.
Or, the distribution of R2 is, to the s&~e degree of approximation,
(7.3) f(R2)dR2= E(I_R2)(N-2)/2dR2rl+ 3(N+4)~(1_R2_~) 7. 2 - 4N 1'1+2 -
for 0 < R2 < I- - ,= 0 otherwise
First we observe that the moments of the distribution (7.3)
N-2 2agree to the order with the actual moments of R. Secondly,
since [. R2= O(N-I ) and the variance of R2is O(N-2}, :in the effective
117
range orR the second term in (7.3) is O(N-l ) as compared to the
first term. Thirdly, the cumulative distribution function of R2 is
or
If R2 is O(N-l ), the second term is O(N-l ) rel~ive to the firstoterm.
Another method
The kth moment of the distribution
iskx = N
T+'2T{" •
If 2k < N, this can be expanded as2k 4k2
(1 - ~~ + ---2 + ••• ). IfN
k is small compared to N, these moments agree with the moments of T
to order N-l • Let us, then, write an approximate distribution of T
as
where (N;2T - ¥) is first degree orthogonal polynomial with weight
. (w-2)!2 r 7i'uneticn T .. 'J and range 0,1 • From this, we have- -
118
j 1 *( N N N+2 NTf T)dT= m + '2 81(lmi - N+2)
o
Equating this to the first moment of T, we obtain
a = 3(N+~) + O( 1 ) •1 2N ;'-
( -2Therefore, negleoting terms 0 N ) in 81 ,
which agrees with (7.2). This series may be extended by taking addi-
tional terms and equating corresponding moments.
In (7.3), make the transformation
Then, the distribution of F is given by
N+2 )=(.N)2 F(N-2 /2 - 3(N+4)
2 (1+ ~)(N+2)72 L 1+ -fiN"-F-l 7
(1+ ¥) ,- ,O<F<oo... - ,
I 2zwhich shows that F is approximately distributed as Fisher s e ,
with Nand 2 degrees of freedom.
119
8. Extension to multivariate case.
We shall state a theorem suggested by theorems 3.1 and 3.2 of
Chapter 3. The proof is similar L~ nature.
THEOREM 8.1. Let ¢(tl , ••• , t u ' 91, "', 8v) be the joint character-
isttc function of variates zl' Y], •.• , y, where 0 ~ bi.. v
.:: Z i .:: c i .:: 1, i = 1, 2, .•. , u; and ,-.1.:: d j .:: Yj .:5 e j :; 1, j= 1,
2••.. , Vo Let it be possible to write
00
(8 .1) ¢( t l , ••. ,t ,91 , •.. ,0 ) = '. L; ai
. j .u v .' t' •. j' 0 l' .• J.u 1'" Jv
~l,·····~,:"'u,Jl"'~' va
v j7t 9 r F(6 +j +1, Y +6+2j +2,2Q), all a,~,y,6 > -1 ,r=l r r r r r r
the series (8.1) being absolutely convergent for all values of t l ,
"'J t u' el,.·.,Qv. If the partial sums of the series are dominated
by a function ~ of these variables such that ~ exp ltlzl+ •••+tuzu+
QlYl+ •• 0+8vYv l is integrable, then the joint frequency function of the
variates is
(8.2)v
ITr=l
......
00
L L; ai j O il' •• j'j •.. jl' ••• ~ 1""= - v
u B(a +l;P +1) i 1"IT ~_ .!_ s __
s=l B(n +1 +1,~ +i +1)s s s s
1:'0
v B(r +1,8 +1)IT r rr:l B(y +j +1,8 +j +1)'r r r r
within the range of the variates and zero otherwise.
9. Bivariate distribution of r 1 and
For a given r 1 the condition R ~ 0 gives the range of r 2
2as ["2r1
- 1, 1 _7. But since r2
in absolute value cannot exceed
cos 1t-1 wherem+
; - N.I,' 7m = ~....,;..:::, we hav'"e- 2-
Similarly, for a given r 2,
For a given r1
, the range of the variation of b is
n1- cos -~"'i'
( 1 m~·.1.- +- -';--0h' •
1 - 1'1
cos 1tl
)m+
As N -> 00 so does m and this range approaches to L--1, 1 _7.Furthermore, (I'J.b - [rl f.b ..... O. This indicates that r l and
b, though not independent, are asymptotically independently distri-
butedo Hence, we shall consider the joint distribution of rl
and
b as an intermediate step to finding the joint distribution of r land
and let ¢( t, Q) be 'bha joint characteristio
121
function of z and b. From the knowledge of the first few moments
of z and b let us write
where aOO=l. But
Therefore, we obtain the equations
e _ 1v z - alO + N+2
tZ(l+b)
and so on. Since the moments of (l+b) have been calculated to order
m-3 we do not attempt to determine coefficients beyor.d those listed
above. 1ftTe find to order
a - 3 a _ 101 - - N[fti. 2 ) (H+4)"' 10 - - flTt"N+2) ,
,
1?2
Now~
t' z(l+b) =~ - N(N+2~(N+4j + O(~)
The value of all turns out to be O(N-4),which we shall neglect.
Hence to the second order polynomials the joint distribution of z and
b is
in the range of z and b and zero otherwise. Making the transforma-
tion
we finally obtain the joint distribution of r l and r 2 as
of' , .~ _ r(1-2ri+r 2) (1-!'2) 1(N-I}/2 (1-r2) r""(9.,) ... (rl'r2Jdr1dj.2 -1- 2 - J' -T~ dr1"'.l.2
\. l-r1 (1-r1)
123
..
I-2ri+r2 1 N+8( ? a)(-im
l-rl
...
..
within the range of variation of T1 and r 2 , and zero otherwise.
The asymptotic behavior of the series (9.1) fer the character-
istic function of r l and b can be established in the same manner
as was given in section 9 of Chapter 3 fer the series representation
of the characteristic function of r l • Also, in the effective range
of r l and r 2, it can be shown that the terms in (9.5) decrease
rapidly and the first term may serve a good approximation for even
moderately large N.
10. Concluding remarks.
Starting from the joint distribution of r l and r 2 given by
(9.2), we can obtain an approximate distribution of any function of
these variates.
It may be noted that the distributions obtain::d in third and
fourth chapters are quite good even for small samples as the theory
permits development of the series to any desired order of approxima-
tion.
..., 6N) and 6 for itif3 trBnspose. In
assumed that N > P and tha~
y
CHAPTER V
• LEAST-SQUARES ESTIMATION lNHEN RESIDUALS ARE CORRELATED
1 • St'1.:mmary.
In this chapter we will study the effect on the least-squares
estimates of regression coefficients and of the variance of residuals
when the condition of independence of residuals is violated. The
maxima and minima of a bilinear and a quadratic form will be obtained.
These will be utilized to set up bounds on the variances and covar-
iances of the least-SQuares estimates of regression coefficients. The
cases of a linear trend and of trigonometric functions will be con-
sidered in more detail. The material in section 2 up to equation (2.11)
is not new except for its oresentation, and is adapted from Hotelling's
lectures on the theory of least squares and regression analysis. The
rest of the chapter is original.
2. feast-squares estimates and their covar~£~.
Let Y1' .", YN be observations on a variate and let
py~; ~ ~iXiy + 6v , y = 1, 2, "', N 0
'i=l I
The observations on Xl' "', xp are considered fix8d from sample to
sample. Let us write 6'::t (61
,
regression theory it is usually
(i) ['!J = 0
( nn) ;.,,,2 2 f 1 2 Nl.l L LI = 0' ~r or , , ••. , 1 ,Y
125
(iii) C6y "0 = ° for y ,,6 ,
(iv) 6 is distributed normally.
If NC~,~) stands for a normal distribution with mean vector ~ and
covariance matrix ~, the above conditions may be summ~ized as
(1) b is distributed as NCO, IN)' where IN is the N x N
identity matrix and ° stands for the N x 1 zero vector.
Write ,=: (Yl ' Y
N)Y . .,
(2.2) ,(~l' ~p)~ = ... ,
x(p x N) =:
,
l._.Xpl xp2 xpN
== (Xiy); i == 1,2, •.• ,p, y=1,2, .•• , N ,
a(p x p):=- xx'-1
c = a
g =: xy
and
We note that, ,
a =: a and c = e
We will use a prime on a letter to denote the transpose of a matrix
or a vector represented by that letter.
126
We suppose that N > P and that the rank of x is ~. Than
the least-squares estimates, b of ~ and 82 of a2, are given by
.. (2.3) b = cg
t ,
(2.4) 82 = if y - b gN-p
= y'y _ bIgn :I
where n.: N - p. Furthermore, if we write 6q = q - lq, where q
is a variate, then
Now,
B(p x p),:> 1 2
= C,6bl5b = (] C
• j; ' I 1t~b = Ccg = -':'(xx )- xy
= (xx t )-1 xx I ~
= ~
as £y:::: r(x'~ + 6) ~ x' f3. It is also known that under the condi-
tion (1)
~ 2 2cs = (j
We propose to consider these estimates when condition (iii) is
replaced by
(iii)* r /'), If = a2I...- y 6 P f y-s i '
~~ich means that condition (1) is replaced by
(2) 6. is distributed N( 0, (]2 p), where
(2.8)
and where
127
...-
peN x N) = 1 PI P2 • •• PM-1
I ·.. PPI PI N-2
P2p I ·.. P l'J_3I
. . . . . . . · . . . . , .P P N- 2 P N-3 I
N-l-'
N-I i 'i= IN + Z Pi(C + C ) ,
i=-l
C ::- r 0 I 0 ·.. 0 0,;
0 1 0 0I 0 ·..• • . . . . . . . •
e 0 0 0 ·.. 0 I
0 0 0 0 0 II' ..- ...-
We still find
Let
Then
,v=y-xb
I " ,v v =- (y - b x)(y - x b)
, ,=- yy-bg
and2s =-
,vv- .n
Also , ,v=x~+6-xb
= x' (~ - b) + I::.
128
I
= (IN - x ex) /).
as
lrV'rite
>.. is a root
b=~+cx/)..
12m :0 x ox; then ml = m and m = m. It follows that if
of m then >..:: >..2, so that, >.. = 0 or 1. 't1Triting tr
for trace of a matrix, we have
I Itr m = tr :it ex = tr xx 0 = tr I p
=p •
It follows that p roots of mare 1 and N-P = n of them are
zero. This can also be seen by observing that,
x(IN-m) = x - xx ox..
= 0 ,i.e., there are P relations among the columns of ~-m. Now,
e I I(/ V v:: ttl' v v
I I- t tr 6 (IN-m) (IN-m) 6-
= f. tr II I (IN-m) II
::: t tr M I (IN-m)
=0.2 tr P - ,l- tr P In
2 2=> N cr - C1 tr Pm.
) I 2 . 2 2Pm=- tr m =- P, ~:. v v = n (J , and (. s = C1 asIf p:: IN then tr
stated in (2.7).
If L:: (fij ), i, j = 1,2, ••• ,N,
tr L(ek + elk) :: 2
1s a matrix, we observe that
N..kZ f
1=1 k,k+i
and
129
Therefore,N-1 k 'k
tr P m ~ tr m P = tr /-mI + E f) ktr m( C +C )_7- N k=l
TIT-I N-k:: p +? E f) k Z m k+
k=l y=l y, Y
and
(2.10)
Now,
and
(2.11)
, N-1 N-kp 2 (! v v 2/-1 2 ~ 'I:'(6 =L-=a -- £J £Jn - n k""l y:1
" , .. 'It 6g15g = x t l5y5y x
2 t:: (1 X P X
<> I 2 IB = 1..ob6b = (1 C x p x c
Write
d(p x N)=-(dj
)= ex, j = 1, 2, ... , p, y:::: 1, 2,y ..., N ,
and let Bij = Bji be the element of B L~ ith row and jth.eo1umn.
We obtain from'4
2 ,B = (1 dPd
2 - N-l k 'k '= cr d/ T7~ + E P k( C +0 L7d
- .'1 1<=1
and
that
_ k 'k ' N-kL d(C +C )d _7ij = Z (d. d + d d )y=l JY i,k+y iy j,k+y
(2.12)
1.30
Bij=cov(bi'bj )
2 N N-l N-k=CJ r i: dj; d. + i: i: P k( dJ'ydi,k+y+di.y+dJ' ,k+Y) _7.
- "(=1 Y JY k=l y=l'
so that
Since eaoh bi is a linear function of 6's, the veotor b is
N(~,B).
If by a proper ohoioe of XIS we make
fXX = I ,
p
a = c = I , we havep
2 _ N-l N-k(2.1.3) Bij = CJ L 0i'+ i: i: P k(Xj; x. k+ +x j Xi k+ ) _7,
J k=l y.l Y J, Y Y , Y
where 0ij = 0 if i " j and 0ii = 1. .
.3. Bounds on a quadratic and a bilinear form.
We notice that i~ s~ and the Bij'S contain bilinear and quad
ratio forms in the x t s. We wish to set some uniform bounds on these
quantities for different sets of non-stochastic variables.
THEOR~M .3.1. Let A= (aij ) be an N X N real symmetric matrix. Let
al , a2' ••• , aN be the characteristic roots of A where
al :: a2 ::; ••• .:: aW_l .:: aN •
, f
If x = (xl' •..• xN), y ~ (Yl' ••• , YN) are row vectors in real
quantities with x and Y their transposed column vectors, then,
under the oonditions, (' ,
(i) x x=l, ii) y y=l, and (iii)x y=O, ~ xrAy has a maximum
(a~ral )/2 and a minimum -( '1J-al )/2
131
PROOF. Since the problem is invariant under any. simultaneous orthoga
anal transformation of x and y, we assume that A is a diagonal
matrix. Hence
Usi ng Lapl ace's method of multipliers, we write
The ma:.'Clmul11 and minirm:un values of )" and T are the same. For these
values
;;T = 0---- == a y. - my. - kxi ,i = 1, 2, .•• , NJ~xi ,i ~ ~
Therefore,
aT 2Zy. - = Zu; y - m == 0
1 aXi i i
Zx .J.!. == Ea .X~ - m == 0j 2fij J J
Zy. ~~- == )" - f = 0 ,J 01,-;
v
which give
f = k == 'X,
132
The equations (3.4) become
-Axi + (ai-v)Yi = 0, i = 1,2, ••• ,N ,
(ai-v)xi - AYi a 0, i = 1,2, •.• ,N •
t I iLet z =(x y). Then for a non-null solution in z of the above
equations it is necessary that
-AI
A-vI
or
A-vI II
-AI I!
a 0
NJr ~a.-(v+~) 7/-ai-(v-l) 7 = 0 •~1 ~ -- -
This means that either
v+l = ai for i =1 or 2 or ••• or N
Qr
v-A = ai for i = 1 or 2 or •.• or N.
Hence the stationary values of A are obtained from the above given
values of v + A and \I - A provided they are consistent with con-
ditions (1) - (iii). Now,
= .. 1.6ij
= .. >"61j,
133
° if i f ji,j=1,2, ••• ,N and 0ij ~ L- 1 if i:: j
The Messian, 1. e., the matrix of the second partial derivatives, H,
is given by
-AI A-vI(3.6) H(2N x 2N) >: L
A-vI -AI
The characteristic roots of H are given by the equation in k
= ° ,Loe. , k = (v-A)-a., a.-(v+A), i=1,2, ••. ,N.
~ ~~11 the ?N roots will be
non-negative if and only if
V-A = aN
Le., ifA = -(a -(1 ) /2
N
V = (a1 + aN)/2
Hence the mruhmum value of A is -( aN-a1
) /2. one vector for which
A attains its mil1tmum is
i.e.,
, .z =(1/ /2,0, uo,O,l/ n,l/./~,O,•.. ,O~-l/ .(2) ,
x' = (1/ I~,O, 0 .. ,0,1/ /?)
y'= (1/ \/2,0, ... ,O,.el/ ~).
Similarly, all the 2N roots of H are non-positive if
V + A :: a!IT
\I - A = a1
134
and the maximum value of A is (aN - al )/2. Thus
-(a -a1)/2 ~ A i < A < A = (a -al )/2 .N mn- -max N
THEOREM 3.2. In the notation of theorem 3.1 and under the same oondi--tiona
where , ,v=xAx=yAy
..PROOF •
If aN_1 = aN' the veotors
x* = (0" 0,
*y = (0, 0,
...,
...,0, 1/ /2, 1/ /2)
0,-1/ /2, 1/ .(2)
make v = aN and the maximum of v, in this case, is aN.
If a~_l I aN' v r aN' as this implies x~ = y~ = 1, xl = x2 =
•.• = xN_1 ~ Yl = Y2 = ••• = YN-l ~ O~ which is contrary to the condi-
tion x'y = 0. For the vectors x* and y* the value of v is
and for vmax
If 2 .. /:". xT\T < .L ~,
135
2Simi.larly, YN~ 1/2, i.e.,
(3.8 )
But
(XNYN)2 ... (xlYl+ ••• + xN_1YN_l)2
N-l 2 N-l 2~ ( Z x.)( z Y
1)
1 1. 1
It follows from (3.8)..
For these values of
N-l 2where Z xi ... 1/2.
1
... (l-X~)(l-Y~) ~ 1/4 for xN' YN ~ 1/2 •
and (3.9 )'thct for vmax ' x~ ... 1/2, Y~ ... 1/2.
B 2JCN a'l d YN
2 2v ... (tN/2 + (alx1+ ••• + (tN_lxN_l)
The maximum value of N~lCttXi under the condi1
~12 2tion Z xi ... 1/2 is ~_1/2 which is achieved for xN_l ... 1/2,
1
xl = ...... xN_2 ... O. Similarly, Y~-l'" 1/2, Yl = ...... YN-2 = O.
Thus the maximum value of v,
vmax .. (aN + aN_l )/2 ,
which is attained for z ... (x*y*). A similar proof shows that
Q.E.D.
4. Bounds on covariances.
In .this section we revert to the notction of section 2 and
..
1.::;6
define x, y, a, ~ etc., as in (2.?) •
If xi denotes the ith row of x and if by proper choice
of the XIS we make
NZ x. x. = 0••
y=l 1.Y JY 1.J,
then a '" c '" I and we get (2.13), i.e.,p
2 - N-l k kl' 7Bi . ::: 0' / 0 .. + Z P kXi (C + C )x j _ •
J - 1.J k=l
From theorem 2.1 of Chapter 4 the maximum characteristic root
.. of Ck+ C
lkis 2 cos
fore from theorem 3.1,
11 which is less than 2. There-
(4.1)2 I 2 i N-l \iB1.'j. - 0' 6.IJ. i < 20' ! Z Pk II
.J.. • ; k=l .
For first order autoregressive scheme
where 6 t satisfies the conditions given in section 2 of Chapter 1,
we have
Hence, if a > 0:;
k::: a c
(4.2)
and
•2( N< 11 < 20' a.-a. )
ij i-a •
137
Similarly with the help of (2.10) we can set bounds on Bij
for a
second order scheme. We note that (4.1) gives a uniform bound for,
all schemes and for all x's such that xx = I. In special casesp
we can improve these bounds to a considerable extent.
5. Linear trend.
Let us suppose that N = 2s+1 and consider the linear trend
in the form
where
(5.2) u2 = s(s+1)(26+1)/3
In the notation of section 2
(26+1)-1/2(5.3) x =ly
x2y = (y-s-l)/u, y=l, 2, ... , N ,
b1
= (2s+1)1/2 y = Nl / 2 Y
b =2'!,yy' - (s+1)Eyy y y y
u
where the summation over y is from 1 to N. Also
aac=JP
b = g
mij
=(x'x)1J'= -N1 + ~(2i-N-l)(2j-N-l)- N(N2_1)
"
138
p = 2
n == N-2~ 2 b2 • 2
f ~Yy - 1 - °2s2 = .!...! == .:..y _
n n
and
2 N N-l N-kBiJ· == CJ r t:, x. x. + t:, Z f)k(xi X. k+ +x. xi k+ ) 7,i,j==1,2 •
- y==l Joy JY k=l y==l Y J, Y JY , y-
After substituting the values of x1y and x 2y l<Te obtain
2 _ . N-l (N-k)(5---5) B11 == CJ Lh;l N P k _7
N-l~ k3 7~ P k -
k=l
•
and
2 2 4 N-1 2 2 N-1 4 N-1fa 91 ["1- if Z Pk + iiN(4+~) E kP k -'-? Z k3
p k _7.k==l N -1 k==l nN(N -1) k=l
kFor the first order autoregressive scheme, P k == Ct • Now, for any
number z =/1, we have
(5.6)q k z_zQ+1t:, z ==
k=l 1-z
•q kZ kz ::
k::l
( q+l q+2z- q+l)z +gz
(1_z)2
139
..
•
Hence for the first order scheme
2 4 N 8 N 1\1+1 4 a_NaN+aN+1 4 N-.1 3kr? s 1 a-a + a-Na +a + 2 2 . -, 2 Z k a.c,-:J= - ii I-a liN (1_a)2 nN(N-l) (I-a) nN(N -1) 1
'" I 4a- - n{l-a) ,
(5.7)
2 /- l+a 6aB22 ,.. (j 'l:'=7i - - 2 _7•
- - -a N(l-a)
If we put
If2 b2=N(N2..{)
we have
and under the first order scheme for by'
140
..
(5.8)
•
Since b1 and b2 are linear functions of Ars and since
their covariance is zero, we conclude that bi are independently
distributed as N(Si' Bii), i = I,? Aconsistent estimate of a
\. may be taken
N-l N 2r1 = Z v v +1 / Z v Q
1 Y Y 1 y
6. Trigonometric functions.
Consider
(6.1)
rr p+/ - z ~2. 1 sin 'A.y + b .,y=I,2, ••• ,N,\ N i=1 ~+ 1. Y
satisfies condition (2) of section 2.
Now
In our notation
141
(6.2)
x2• = cos 'X.. y1.,1 1.
x is a 'P+I x N matrix such thatI
xx = I 2P+1" Therefore
a = c = I 2p+1 '
and we find
Furthermore,
"
1 2 P=- + - ~ cos (y-6)AN N i=l i
\-\~
142
n = N - 2p - 1
and
i~ 2 2 2 N-l 2 N-lC s = CJ rl- N L; Pk + -nN E k P k
- k=l k=l
4 TIT-I P-!iN . E E (N-k)p k cos kAi 7.
K=l i=1
For the covariances of bi md bj
we obtain from (2.13)
2 N-l N-kEll = CJ .rl+2~1 N Pk _7
2 4 N-l N-kB2i 2i = CJ £1+ N Z ~ Pk coS(k+V)A. cos VAi _7
~ k=l r=l ~
and Bl ,2i+l and B2i+1 ,2i+l are obtainable from the expressions
for Bl ,2i and B2i,2i by replacing cosine by sine. Finally
2 N··l N-k 2 .B2i 2j+l = CJ ~ ~ L;_ - )eos VAi sin (k+y)X i, k=l "(:1 N l
N- lIf we consider P2' P 3' •.• to be negligible, we obtain to order
,
We can get more precise expressions if the underlying scheme is
known. For eXaIn1jle, if the underlying scheme is first order auto-
regressive scheme, we have
2 2 4 P cos A.~a[~ ~ 1 a a Z ( )..)
a~ - n{i-a) -if i=l 1-2a cos A.+a~ •).
~~ observe that the vector b is distributed as N(~,B).
7. Concluding remarks.
1~Te conclude this chapter with the remarks that in most practical
cases the underlying scheme for 6's will not be known. Even if it
is known, we will be required to estimate the parameters involved
144
2which will further reduce the degrees of freedom of s • Similarly,
'Bij'S will have to be estimated from the sample. It may not be very
difficult to estimate 02, but the question of estimating PI' P2,
••• remains open. ore suggestion obviously is to estimate these
quantities by sample serial correlations; or, in case when higher
order autocorrelations are functions of first k autocorrelations,
to estimate the first k autocorrelations by corresponding sample
serial correl~ions. In any case, the distribution of resulting esti-
mates will be extremely difficult to obtain. However, these estimates
will be a definite improvement over the estimates generally used when-
ever there is evidence of significant serial correlations among the
residuals.
£4_7
.[5 -7,e /-6 7-
£7 _7
i-8 _7
145
BIBLIOGRAPHY
A.itken} A.C., liOn Least Squares and Linear Combinations ofObservations", Proceedings of the Royal ~ociety of ~din
burgh, Vol. tV(1935),pp.42-48.
Anderson, R. L., "Distribution of the Serial Correlation Coefficient", Annals of Mathematical Statistics, Vol.XIII(1942), pp. 1-13. ---
Anderson, R. L., and Anderson!} T).r., "Distribution of the Circular Serial Correlation Coefficient for Residuals fromFitted Fourier Series"~ Annals of Mathematical Statistics,Vol.XXI(1950), pp. 59-81:- .
Anderson, T.W., "On the Theory of Testing Serial Correlation flJ
Skandir:~:..visk .Aktuarietidskrift, Vol. XXXI( 1948) ,pp .88-116.
Bartlett, H.S., "Some Aspects of the Time-Correlation Problemin regard to Tests of Significance fl
, The Journal of theRoyal Statistical Society, Vol. XCVIII, part III(1935),pp. 536..543.
Bartlett, M.S., 1l4pproximate Confidence Intervals tl , Biometrika,Vol. XL(1953), pp. 306-317.
Cramer, Harald, Mathematical Methods of Statistics, PrincetonUniversity Press, Princeton, 19510
Daniels, H.E., It ,o\Pproximate Distribution of Serial Correlation Coefficientsll., Biometrika, Vol. XLIII(1956), pp.169185 ~
Dixon, W. J., "Further Contributions to the Problem of SerialCorrelation t' , Annals of Mathematical Statistics, Vol.XV(1944), pp. 119-144.
Doob, J. L., Stochastic Processes,porated,-New York, 1953 ~_._-
John Wiley and Sons, Incor-
r127- -
Durbin, J., and 1A!atson, G. S., "Testing of Serial Correlationin Least Squares Regression 11, Biometrika, Vol.XXXVII(1950), pp. 409-428; Vol. XXXVIII{1951), pp. 159-178.
Fisher, R. A., "Frequency-Distribution of the Values of theCorrelation Coefficient in Samples from an IndefinitelyLarge Population It, ~iometrika, Vol.X(1915), pp. 507-521.
146
Fisher, R.A., ltThe General Sampling Distribution of theMultiple Correlation Coefficient/If, Proceedings of RoyalSocl.ety Se~ies A, Vol. CXXI(1928), pp. 654-673.
Ghurye, S. Go. "A Method of Estimat ing the Parameters of anAutoregressive Time Series ", Biometrika, XXXVII(1950),pp. 173-178.
Gurland. John, "Distribution of Definite and of IndefiniteQuadratic Forms ", .Annals of Mathematical Statistics,Vol. XXVI (1955), pp. 122-127.
/-16 7 Hart,- - B. I., "Significance Levels for the Ratio of the MeanSquare Successive Difference to the Variance", Annalsof Mathematical Statistics, Vol. XIII(1942b),pp. 445"41(1:"--
•
Hotelling, Harold, "An Application of Analysis Situs toStatistics ", Bulletin of the Amerioan MathematicalSociety, Vol.!XXIII(1927), pp. 467-476 .
Hotelling, Harold, "Tubes and Spheres in n-Spaces and aClass of Statistical Problems II, 1\merican Journal ofMathematics, Vol. LXI(1939), pp. 440-460.
P.L., "On the Asymptotio Distributions 6f Oertain Statistics used in Testing the Independence between SuccessiveObservati.ons from a Normal Population", Annals of Mathematical Statistics, Vol. XVII(1946), pp. 350"354.
Hurwicz, Leonid, "Least Squares Bias in Time Series ll , Chapter15 in Koopmans', T., Statistical Inference in DynamicEconomic Models, John-wIley and Sons, Incorporated,New york, 1950. •
Jenkins, G.M., "Tests of Hypothesis in the Linear Autoregressive Model. I. Null Hypothesis Distributions in theYule .Scheme fl , ~~, Vol. XLI(1954), pp. 405-419.
Jenkins, G.M., t~ests of Hypothesis in the Linear Autoregressive Model. II. Null Distributions for Higher OrderSohemes: Non-m!ll Distributions", Biometrika, Vol. XLIII(1956), pp. 186-199<
Keeping, E.S., "A. Significance Test for Exponential Regr.ess:ton tt,
Annals of Mathematical Statistics, Vol. XXII(1951), pp.180-198.
£29_7".
£3o_7
•e £31_7
r35·7- -
147
Kendall, M.G., "On Autoregressive Time Series", Biometrika,Vol. XXXIrI(1944), pp. 105-12'.
Kendall, M.G., Contributions to the Study of OscillatorTime-Series, Cambri ge University Press, 19 o.
Kendall, MG., The Advanced Theory of Statistics, Vols.I andII, Charles Griffin and Company Liniitea, London, 1948.
Kendall, M.G., "Note on Bias in the Estimation of Autocorrelationa", Biometrika, Vol. XI,I(1954), pp. 40.3-404._._--
Koopmans, Tjalling, "Serial Correlation and (...luadratic Forms inNormal Variables", Annals of Mathematical Statistics,Vol. XIII(1942), pp. 14-2.3.
Koopmans, TjaLling, Statistical Inference in Dynamic Economic'Models, John 'Wiley and Sons, Incoropor ated, New York;, 1950.
Lehmann, E.L., "On Optimum Tests of Composite Hypothesis with. One Constraint", A1mals of Mathematical Statistics, Vol.
XVIII(1947), pop. 4'73-494 •
Leipnik, R.13., "Distribution of the Serial Correlation Coefficient in a Circularly Correlated Universe", Annalsof Mathematical Statistics, Vol. XVIII(1947), pp. 80-87.
Madow, W.G., r~ote on the Distribution of the Serial Correlation Coefficient li , Annals of Mathematical Statistics,Vol. XVI(1945), pp. 308-310.
Mann, H.'!3. and lrlald, Abraham, "On the Statistical Treatmentof l:.1.near Stochastic Difference 'Equations", Econometrica,Vol. XI(1943), pp. 173-220.
Marriott, F.H.C., and Pope, J.A., "Bias in the Estimation ofAutocorrelations", Biomet!':t~, Vol. XLI(1954), pp. 390-402.
Moran, P.A.P., "Some Theorems on Time Series", Biometrika, Vol.XXXIV(1947), pp.281-29l} Vol.XXXV(1948),pp.25;-260.
Moran, P.A.P., III!. Test for the Serial Independence of Residuals",Biometrika, Vol. XXXVII(1950), pp. 178-181.
Orcutt, G.H., "A Study of the Autoregressive Nature of the TimeSeries used for Tinbergen's Model of the Economic Systemof the United States 1919-32 11, ,Journal of the Royal Sta~is
tical Societl S~pplement, Vol. X(1948), pp. 1-53-
r407- -
148
r38 7 Orcutt, a.H., en d James, S.F., "Testing the Significance of- - Correlation between Time Series", Biometrika, Vol.XX:XV(1949), Pp. 397-413.
Orcutt, a.H., and Cochrane, D., "~ Sampli.n(7 Study of the Meritsof Autoregressive and Reduced Form Transformations inRegression Analysis", Journal of the American Statistical~ssociation, Vol.XLIV(1949), pp. 356-372.
Pearson, Karl, Tables of the Incomplete Beta-Function, Cambridge University Press, 1934.
£41 7 Quenouille, M. H.• "Some Results in the Testing of the Serial- Correlation Coefficient", Biometrika, Vol. XXXV(1948), pp.
261-267.
Quenouille, M.H., "Approximate Tests of Correlation in TimeSeries If, Journal of the Royal Statistic a1 Society Supple~~ Vol. XI(1949a), pp. 68-84.
['44...7
Quenouil1e, M.H., "The J oin"1i Distributions of Serial Correlation Coefficients TI, Annals of Mathematical Statistics, Vol.XX(1949b), pp. ,61-571.
Rao, S.R., and Som, R.K., IIAutoregressive Series", secondarticle in Indian Statistical Institute's '~he Applicability of Large Sample Tests for Moving Average and Autoregressive Schemes to Series of Short Length rr, Sankhya,Vol. XI(19,1), pp. 239-2,6.
RUbin, Herman, liOn the Distribution of the Serial Correlat ionCoefficient", Annals of Ma"~hematical Statistics, Vol.XVI(1945), pp. 211-215. .
Sastry, A.S .R.. "Bias in Estimation of Serial Correlation Coefficients", Sankhya, Vol.XI(1951a),pp. 281-296.
Szego, aabor, OrthogoEal Polyoomials, American MathematicalSociety Colloquium PUblications, Vol.XXIII, New York,1939.
Tintner, Gerhard, Econometrics, John Wiley and Sons, Incorporated, New YorK: 19;2.
1:49_7 von Neumann, John, Kent,R.H., Bellinson,H.R., and Hart,B.I.,"The Iv1ean Square Successive Difference ", Annals of I'1athematical Statistics, Vol.XII(1941a), pp. 153-162.
149
von Neumm n, JOhn, "Distribution of the Ratio of the MeanSquare Successive Difference to the Variance ll ; Annalsof Mathematical Statistics, Vol. XTI(1941b), pp: 357=395.
151_7 von Neumann, John, !fA Further Remark on the Distribution ofthe Ratio of the Mean Square Successive Difference to theVariance tl , Annals of }Jfathematical Statistics, Vol.XIII(1942), pp. ~-B6.
r527- -r537- -£54-.7
r557- -
'" .(56_7
e£57_7
Watson, G.S., rQn the Joint Distribution of the Circular SerialCoefficients", ~iometrika, Vol-XLIII(1956), pp. 161-l68~
lNhittle, Peter, Hypothesis Testing in Time ~eries, Almquistand l,vikse1ls 'Roktryckari P,.B., Uppsa1a, 1951.
lnTise, J., liThe Autocorrelation Function and the Spectral DensityFunction", 12iome~~, Vol. XLII(1955), pp. 151-159.
Wold, Herman, A Study in the Analysis of Stationary Time Series,Almquist andWlksells, Uppsala, 1938 •
Yule; G.U., tlOn the Time-Correlation Pr oblemll3 .:!ournal of the
Royal Statistical Society, Vol. LXXXIV(1921), pp.496-537.
Yule, a.u., "Why do we Sometimes get Nonsense Correlationsbetween Time Series?", Journal of the Royal StatisticalSociety, Vol. LXX~IX(1920Y, pp. 1-64.
Yule, G.U., nOn a Method of Investigating Periodicities inDisturbed Series with Special Reference to Wo1fer'sSunspot Numbers", Philosophical Transa~ions A, Vol.CCXXVI(1927), pp. 267-29~. .-
