Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Lesson 6: Estimation of the AutocovarianceFunction of a Stationary Process
Umberto Triacca
Dipartimento di Ingegneria e Scienze dell’Informazione e MatematicaUniversita dell’Aquila,
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Introduction
The autocovariance function plays a key role in the construction ofthe DGP’s model of our time series.
Thus an important question is:
Is it possible to obtain a ’good’ estimate of theautocovariance function?
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Introduction
This question is not trivial since we observe only one realization ofthe process.
We cannot, for example, observe a second realization of Italianannual GDP for the period from 2001 to 2011.
In general, with a single realization we are unable to estimate amoment function of a stochastic process.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The Mean Function of a stochastic process
First, we consider the mean function.Definition. Let {xt ; t ∈ Z} be a stochastic process such thatVar(xt) <∞ ∀t ∈ Z. The function
µx : Z→ R
defined byµx(t) = E (xt)
is called Mean Function of the stochastic process {xt ; t ∈ Z}.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The Mean Function of a weakly stationary process
Let {xt ; t ∈ Z} be a weakly stationary stochastic process. Themean function is given by
µx(t) = µ ∀t ∈ Z
Thus, for a weakly stationary process, the mean function is theconstant function which takes always the same value µ.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean Function
How do we estimate µ?
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean Function
If were possible to obtain m independent realizations{x
(j)1 , ..., x
(j)T
}j = 1, ...,m
we could estimate the mean of {xt ; t ∈ Z} by averaging over therealizations
µ(δ)m =
1
m
m∑j=1
x(j)δ , δ ∈ {1, 2, ...,T}
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean Function
Since
E
[(µ
(δ)m − µ
)2]
=γx(0)
m
we have
limm→∞E
[(µ
(δ)m − µ
)2]
= 0
The estimator µ(δ)m converges in quadratic mean to µ as the
number of realizations increase.
This implies that µ(δ)m is consistent for µ.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean Function
Unfortunately, as we have already seen, it is impossible to obtainmultiple realizations. We have a single finite realization
x1, x2, . . . , xT
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean and Autocovariance Function
Then, the only way to perform the average is along the time axis.
⇓
The mean and covariance functions can be estimated by theircorresponding time averages:
xT =1
T
T∑t=1
xt
and
γx(k) =1
T
T∑t=k+1
(xt − xT )(xt−k − xT ) for k = 0, 1, . . . ,T − 1
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean and Autocovariance Function
Now the question is: are these estimators ‘good’?
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Estimation of Mean and Autocovariance Function
The answer is yes, if the stationary process is ergodic
Ergodic comes from Greek.From ergon ‘energy, work’and hodos‘way, path’.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
What is an ergodic process?
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
Definition. A stationary stochastic process is said ergodic, withrespect to a given population moment, if the sample (or time)moment for a single finite realization of length T converges inquadratic mean to the population moment as T increases to ∞.
Remark. Note that we cannot simply refer to a process as ergodic.Ergodicity must be related directly to the particular populationmoment, e.g. mean value, covariance, etc.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
In particular, a stationary process xt with mean µ is mean-ergodic if
limT→∞E[(xT − µ)2
]= 0
and it is covariance-ergodic if
limT→∞E
( 1
T
T∑t=k+1
(xt − xT )(xt−k − xT )− γx(k)
)2 = 0
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
ergodic processes
It is important to underline that covariance-ergodicity impliesmean-ergodicity, but not the reverse.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
ergodic processes
A simple example of mean and covariance-ergodic process is theprocess
ut ∼ i .i .d .N(0, 1)
Problem: Show that the process ut ∼ i .i .d .N(0, 1) is mean-ergodic
We observe that the process ut has no memory, in the sense thatthe value of the process at time t is uncorrelated with all pastvalues up to time t − 1.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
ergodic processes
It is important to note that not all stationary processes areergodic.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
Consider the stationary process {xt ; t ∈ Z}, with xt = A ∀t ∈ Z,where A is a random variable with mean 3 and variance 7.
This process is not mean-ergodic since the sample mean xT doesnot converge to µ as T →∞
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
In fact we have
limT→∞E[(xT − 3)2
]= 7 6= 0
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
The ergodicity is a matter of information contained in a singlerealization of a long duration of the process.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
If the process is not too persistent (ergodicity), so that eachelement of the realization x1, x2, . . . , xT will contain someinformation not available from the other elements, then a singlerealization of a long duration will be sufficient to obtain a goodestimate of its moments.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
We have seen that:
1 the process ut ∼ i .i .d .N(0, 1) is mean-ergodic;
2 the process xt = A, where A is a random variable with mean 3and variance 7 is not mean-ergodic.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
Consider the process ut ∼ i .i .d .N(0, 1) and let
u1, u2, . . . , uT
be a time series from this process. Each element of the time seriesu1, u2, . . . , uT will contain some information not available from theother elements. There are new information contained in thenew observations.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
Now, we consider the process xt = A, where A is a randomvariable with mean 3 and variance 7. We note that a time seriesfrom this process will has the form
a, a, ..., a
where with a we have denoted a realization of the random variableA.Thus in the time series
x1 = a, x2 = a, ..., xT = a
there is the same infomation contained in the first observation x1.There aren’t new information contained in the newobservations.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
The meaning of the ergodicity
The process ut has no memory, in the sense that the value of theprocess at time t is uncorrelated with all past values up to timet − 1. The process xt = A, instead, is too persistent.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
Now, we present some theoretical results, called ergodictheorems. These theorems give some necessary and/or sufficientcondition for mean-ergodicity of a stationary process.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
Theorem. (Slutsky’s Theorem) A stationary process xt with meanµ and autocovariance function γx(k) is mean-ergodic iff
limT→∞1
T
T−1∑k=0
γx(k) = 0
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
We note that
1
T
T−1∑k=0
γ(k) = Cov(xT , xT ).
Thus
limT→∞1
T
T−1∑k=0
γ(k) = 0 ⇐⇒ limT→∞Cov(xT , xT ) = 0
We have that a stationary process xt is mean-ergodic if and only ifas the sample size T is increased there is less and less correlation(covariance) between the sample mean xT and the last observationxT .
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
Corollary 1 (Sufficient condition for mean-ergodicity). Let xt be astationary process with mean µ and autocovariance function γx(k).If
limk→∞γx(k) = 0,
then xt is mean-ergodic.
Intuitively, this result tell us that if any two random variablespositioned far apart in the sequence are almost uncorrelated, thensome new information can be continually added so that the samplemean will approach the popolation mean.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
Corollary 2. (Sufficient condition for mean-ergodicity) Let xt be astationary process with mean µ and and autocovariance functionγx(k). If
∞∑k=0
|γx(k)| <∞,
then xt is mean-ergodic.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic Theorems
If∞∑k=0
|γx(k)| <∞,
we say that the autocovariance function is absolutely sommable
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodicity under Gaussianity
Let {xt ; t ∈ Z} be a stationary process with mean µ andautocovariance function γx(k). If the process is Gaussian, then1. the condition of absolute summability of covariance function
∞∑k=0
|γx(k)| <∞
is sufficient to ensure ergodicity for all moments.2. the condition
limk→∞γx(k) = 0
is necessary and sufficient.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodicity under Gaussianity
Heuristically, a Gaussian stationary process is ergodic if and only ifany two random variables positioned far apart in the sequence arealmost independently distributed. That is, for sufficiently large k ,xt and xt−k are nearly independent.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process
Ergodic processes
Conclusion
For stationary ergodic processes, we do not need to observeseparate independent realizations of the process in order to obtaina consistent estimate of its mean value or other moments.
A good estimate of the moments of the process can be obtainedconsidering only one sufficiently long realization of the process.
Umberto Triacca Lesson 6: Estimation of the Autocovariance Function of a Stationary Process