ISUP - LSTA

2002-3 Fevrier 2002

Doubly Stochastic Hilbertian processes

Serge GUILLAS

Rapport Technique n3 (2002)

Institut de Statistique de l’Universite de ParisLaboratoire de Statistique Theorique et Appliquee

Universite Paris VITour 45-55, Boıte 1584, place Jussieu75252 Paris Cedex 05

Rapport Technique n3 (2002)

Doubly Stochastic Hilbertian processes

Serge GUILLAS

Universite Paris VI, ISUP-LSTATour 45-55, Boıte 158

4, place Jussieu, 75252 Paris Cedex 05

Fevrier 2002

Ces documents de travail n’engagent que leurs auteurs.

Working papers only reflect the views of the authors.

Doubly Stochastic Hilbertian processes

S. GuillasUniversite Paris 6

Ecole des Mines de Douai

Fevrier 2002

Abstract

In this paper, we consider a Hilbert space valued autoregressive stochastic sequence(Xn) with several regimes. We suppose that the underlying process (In) which drivesthe evolution of (Xn) is stationary. Under some dependence assumptions on (In),we prove the existence of a unique stationary solution, and with a symmetric com-pact autocorrelation operator, we can state a Law of Large Numbers with rates andthe consistency of the covariance estimator. For the cross-covariance operators, theboundedness of (Xn) ensures the same type of consistency. An overall hypothesisstates that the regimes where the autocorrelation operator’s norm is greater than oneshould be rarely visited.

1 Introduction

Autoregressive Hilbertian processes of order one (ARH(1)) have been extensively in-vestigated by Bosq [4, 5], mainly because of the interest for prediction of continuoustime stochastic processes (e.g. electricity consumption [8], traffic [2], or climatic vari-ation [3]). The inclusion of exogenous variables in an additive way may improve themodel. Therefore, it has recently been considered theoretically [11] and applied topollution forecasting [9]. However, in the case of real-valued time series, several au-thors also examined the non-additive inclusion. For instance, we can mention regimemodels (according to Tjøsteim’s terminology [21]) of the form

Xt = θtXt + et (1)

where (et) is a noise and (θt) is a stochastic process. When θt is regulated by Xt−1, (1)results in ”(Self-Exciting) Threshold Autoregressive processes” (SETAR or possiblyTAR). On the other hand, if (θt) is a Markov chain having a finite state space, we saythat the model is doubly stochastic [20]. ”Random Coefficient Autoregressive Models”(RCA) [16] appear in the particular case where (θt) is i.i.d. .

Tong [22], [23] studied TAR processes and an application to the prediction of ozonecan be found in [15]. These nonlinear processes should fit better to phenomena withsudden changes such as river flows.

Petrucelli and Woolford [17] considered the real valued model

Zt = φ1Z+t−1 + φ2Z

−t−1 + εt, t = 1, 2, ...

1

where x+ = max(x, 0), x− = min(x, 0) and (εt) is a white noise. They showed that(Zt) is ergodic if and only if

φ1 < 1, φ2 < 1, φ1φ2 < 1,

which is less restrictive than|φ1| < 1, |φ2| < 1.

Similarly, Brandt [7] and Bougerol and Picard [6] made use of the negativity of thetop Lyapunov exponent in order to ensure the existence of a stationary solution, butYao [24] pointed out that this criterion is difficult to apply, proposing to look atthe spectral radius of the autocorrelation matrix in a multivariate framework. Seealso Pourahmadi [18] in this context, Franck and Zakoıan [10] for extensions to themultivariate ARMA case, Yao and Attali [25] for an investigation of the nonlinearcase, and Horst [14] for non-stationary (θt) and (et).

From our ARH(1) point of view, the model imposes conditions of the followingtype (see [5]):

∃j,∥∥ρj∥∥L < 1,

for the existence of a stationary solution to the equation

Xn = ρ(Xn−1) + εn,

where H is a real and separable Hilbert space - usually an infinite-dimensional one- with norm ‖.‖, ρ is a bounded linear operator on H, ‖.‖L is the linear norm oflinear operators on H, and (εn) is a strong Hilbertian white noise (SHWN), that is asequence of i.i.d. random variables with values in H satisfying

∀n ∈ Z, Eεn = 0, 0 < E ‖εn‖2 = σ2 < ∞.

Let (In)n∈Z be a strictly stationary sequence of random variables taking their valuesin 0, 1, and (εn)Z be a SHWN. Let ρ0 and ρ1 be two bounded linear operators onH. We consider the following model:

Xn = ρ0(Xn−1) + εn if In = 0Xn = ρ1(Xn−1) + εn if In = 1 . (2)

which can be also written down as

Xn = ρIn(Xn−1) + εn. (3)

In this paper, we extend doubly stochastic models to Hilbert space valued pro-cesses. When one of the two choices leads to an explosive case, for instance if

∥∥∥ρj1

∥∥∥ ≥ 1for all j, the alternative choice may stabilize the behavior as we will see hereafter. Notethat with this kind of sample paths, it seemed reasonable to study extremes for suchprocesses (see de Haan et al. [13] in the particular case of RCA models). In sec-tion 2, we establish the existence and uniqueness of a stationary solution under theassumption that (In)n∈Z is (m − 1)-dependent, with a positive integer m. Then, insection 3, we state a Law of Large Numbers with rates, under the assumption thatρIn

is symmetric and compact and (In)n∈Z is i.i.d. (RCA models). Finally, in section4, we show the consistency of some estimators of the covariance and cross-covarianceoperators. Note that our paper is devoted to the two regimes case for simplicity, butit is easy to generalize to the multiple regimes case.

2

2 Stationary solution

We start with two technical Lemmas. First, let us recall the following result of Bass[1] (see also [19]).

Lemma 2.1 (Bass 1955) Let Y1, ..., Yn be non-negative random variables. Then,

E (Y1...Yn) ≤∫ 1

0

QY1(u)...QYn(u)du

where QX denotes the quantile function of the r.v. |X| defined by

QX(u) = inf t, P (|X| > t) ≤ u .

We then infer the following upper bound from Bass Lemma.

Lemma 2.2 Let a and b be two positive real numbers. Let Y1, ..., Yn be identicallydistributed random variables such that

P (Y1 = a) = 1− q, P (Y1 = b) = q.

ThenE (Y1...Yn) ≤ (1− q) an + qbn. (4)

Proof. Apply Lemma 2.1:

E [Y1...Yn] ≤∫ 1

0

(QY (u))ndu,

with QY (u) := QY1(u). If b ≤ a then

P (Y1 > t) =

0 if t ≥ a1− q if t ∈ [b, a[1 otherwise

and if a ≤ b then

P (Y1 > t) =

0 if t ≥ bq if t ∈ [a, b[1 otherwise

.

Hence, in the case where b ≤ a,

QY (u) =

a if 0 ≤ u < 1− qb if 1− q ≤ u < 1

and in the case where a ≤ b,

QY (u) =

b if 0 ≤ u < qa if q ≤ u < 1 .

Thus, in general, ∫ 1

0

(QY (u))ndu = (1− q) an + qbn.

3

Theorem 2.1 Suppose that (In) is (m− 1)-dependent and that

m4 := E ‖ε0‖4< ∞.

Setq = P (In = 1),

andcm = (1− q) ‖ρ0‖8m + q ‖ρ1‖8m

. (5)

If cm < 1, then (2) has a stationary solution given by

Xn =∞∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j) , (6)

the series converges in L2H(P ) (that is in the mean square sense), with the convention

j−1∏p=0

ρIn−p = Id, pour j = 0.

Under the conditionE ‖X0‖4

< ∞,

this solution is unique.

Proof. For the existence:

∆l′

l :=

∥∥∥∥∥∥l′∑

j=l

(j−1∏p=0

ρIn−p

)(εn−j)

∥∥∥∥∥∥2

L2H(P )

=l′∑

j=l

∥∥∥∥∥(

j−1∏p=0

ρIn−p

)(εn−j)

∥∥∥∥∥2

L2H(P )

since the εn are orthogonals. Therefore,

∆l′

l =l′∑

j=l

E

⟨(j−1∏p=0

ρIn−p

)(εn−j) ,

(j−1∏p=0

ρIn−p

)(εn−j)

⟩

≤l′∑

j=l

E

∥∥∥∥∥j−1∏p=0

ρIn−p

∥∥∥∥∥2

L

‖εn−j‖2H

≤

l′∑j=l

(E

[j−1∏p=0

∥∥ρIn−p

∥∥4

L

])1/2 (E[‖εn−j‖4

H

])1/2

︸ ︷︷ ︸(m4)

1/2

It remains to find an upper bound to(E[∏j−1

p=0

∥∥ρIn−p

∥∥4

L

])1/2

. For that purpose,

remark that the r.v. Yp =∥∥ρIn−p

∥∥4

L are identically distributed, and use the (m− 1)-dependence of the sequence (In) and then of the sequence (Yn). We may now writedown:

Y1...Yj = Y1...YmYm+1...Y2mY2m+1...YkmYkm+1...Ykm+r

4

where j = km + r is the expression of the euclidean division of j by m (0 ≤ r < m).Let us denote

U1 = Y1...Ym

V1 = YmYm+1...Y2m

U2 = Y2m+1...Y3m

...

so that

Y1...YmYm+1...Y2mY2m+1...Ykm...Ykm+r =

U1V1U2...Ubk/2c+1Ykm+1...Ykm+r if k oddU1V1U2...Vk/2Ykm+1...Ykm+r if k even .

Hence, for odd k, and denoting Vbk/2c+1 = Ykm+1...Ykm+r

E

[j−1∏p=0

Yp

]≤

E

bk/2c+1∏i=0

U2i

1/2E

bk/2c+1∏i=0

V 2i

1/2

≤[(

(1− q) ‖ρ0‖8m + q ‖ρ1‖8m)bk/2c+1

]1/2 [((1− q) ‖ρ0‖8m + q ‖ρ1‖8m

)bk/2c]1/2

[(1− q) ‖ρ0‖8r + q ‖ρ1‖8r

]1/2

by Lemma 2.2. Accordingly, for odd k,

E

[j−1∏p=0

Yp

]≤[(1− q) ‖ρ0‖8r + q ‖ρ1‖8r

]1/2 [(1− q) ‖ρ0‖8m + q ‖ρ1‖8m

]bk/2c,

upper bound which holds also for even k’s by a similar argument. If we set

Mm = supr=0,...,m−1

[(1− q) ‖ρ0‖8r + q ‖ρ1‖8r

]1/2

andcm = (1− q) ‖ρ0‖8m + q ‖ρ1‖8m

we obtain

E

[j−1∏p=0

Yp

]≤ Mmcbk/2c

m ,

and (E

[j−1∏p=0

∥∥ρIn−p

∥∥4

L

])1/2

≤(Mmcbk/2c

m

)1/2

.

And since

j = km + r

k =j − r

m>

j −m

m=

j

m− 1,

5

we get (E

[j−1∏p=0

∥∥ρIn−p

∥∥4

L

])1/2

≤ K√

Mm

[(√

cm)b1/2mc]j

, (7)

with a K > 0. Consequently, if cm < 1, the upper bound is the general term ofa convergent series. Thus ∆l′

l tends to 0 when l, l′ tend to infinity and by Cauchycriterion, the series in (6) converges in L2

H(P ).Consider the stationary process

Wn =∞∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j) .

Since ρ0 and ρ1 are bounded,

Wn − ρIn(Wn−1) =

∞∑j=0

(j−1∏p=0

ρIn−p

)(εn−j)−

∞∑j=0

ρIn

(j−1∏p=0

ρIn−1−p

)(εn−1−j)

=∞∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j)−

∞∑j=0

(j∏

p=0

ρIn−p

)(εn−1−j)

=∞∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j)−

∞∑j′=1

j′−1∏p=0

ρIn−p

(εn−j′)

= εn.

Conversely, let (Xn) be a stationary solution of (2) satisfying

E ‖X0‖4< ∞,

then by a straightforward induction, it follows

Xn =

(k∏

p=0

ρIn−p

)(Xn−k−1) +

k∑j=0

(j−1∏p=0

ρIn−p

)(εn−j) (8)

and

E

∥∥∥∥∥∥Xn −k∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j)

∥∥∥∥∥∥2

≤ E

∥∥∥∥∥(

k∏p=0

ρIn−p

)(Xn−k−1)

∥∥∥∥∥2

≤ E

∥∥∥∥∥k∏

p=0

ρIn−p

∥∥∥∥∥2

L

‖Xn−k−1‖2

≤

E

∥∥∥∥∥k∏

p=0

ρIn−p

∥∥∥∥∥4

L

1/2 (E ‖Xn−k−1‖4

)1/2

.

The term(E ‖Xn−k−1‖4

)1/2

is constant because of the stationarity of (Xn). And theuse of the bound in (7) entails the conclusion:

E

∥∥∥∥∥∥Xn −k∑

j=0

(j−1∏p=0

ρIn−p

)(εn−j)

∥∥∥∥∥∥2

→ 0

6

when k →∞.

3 Law of large numbers

In the sequel, we will assume that ρIn is symmetric and compact of the form

ρIn =∞∑

j=0

Aj,Inej ⊗ ej

where (ej) is a fixed orthonormal basis of H, and (Aj,0)j , (Aj,1)j are two non increas-ing, non negative sequences tending to 0, such that for all j and all n

ρIn(ej) = Aj,In

ej .

Actually, only the eigenvalues are random, the change of regime is just in terms oflevel of influence.

The following Proposition will be a technical tool used in the proof of our results.It generalizes the property - holding in the real-valued context - that it is possible totake out of the conditional expectation random variables which are measurable withrespect to the considered σ-algebra.

Proposition 3.1 Let (Lj)j∈N be a sequence of real random variables defined on aprobability space (Ω,A, P ) such that

L =∞∑

j=0

Ljej ⊗ ej

is a bounded linear operator on H, (ej) being a fixed orthonormal basis of H. LetA0 be a sub-σ-algebra of A. Let X be an integrable H-valued random variable. If thesequence (Lj)j∈N is A0-measurable, then

EA0 [L(X)] = L(EA0 [X]

).

Proof. In order to show that L(EA0 [X]

)is the conditional expectation - see [5,

(1.34)] for the definition - of L(X) relative to A0, it is sufficient to show that for allA0-mesurable A

E[1AL

(EA0 [X]

)]= E [1AL(X)]

i.e. E[⟨

1AL(EA0 [X]

), ej

⟩]= E [〈1AL(X), ej〉] , j ∈ N

i.e. E[1ALj

⟨EA0 [X] , ej

⟩]= E [1ALj 〈X, ej〉] , j ∈ N

i.e. E[1ALjE

A0 〈X, ej〉]

= E [1ALj 〈X, ej〉] , j ∈ N.

But, since the random variables 1ALj and 〈X, ej〉 are R-valued, we recognize here thewell known result about conditional expectation:

EA0 [1ALj 〈X, ej〉] = 1ALjEA0 〈X, ej〉 .

7

The Law of Large Numbers will be stated under the hypothesis:

(In) i.i.d. (H1)

Note that under (H1), (In) is (m− 1)-dependent with m = 1. Such an influence onthe change of regime connects our model with RCA ones.

We will also assume the following hypothesis:

(In) and (εn) are independent. (H2)

This assumption tells us that the two influences (the index of the operator and theadditive noise) are not linked: we may talk of a double noise.

Set Sn = X1 + ... + Xn.

Theorem 3.1 (LGN) Under (H1) and (H2), under all the assumptions of Theorem2.1, and if

(1− q) ‖ρ0‖+ q ‖ρ1‖ < 1,

then

E

∥∥∥∥Sn

n

∥∥∥∥2

= O(1n

) (9)

andn1/4

(lnn)β

Sn

n→

n→∞0 a.s., β > 1/2. (10)

Proof. Note that

E ‖Xn + ... + Xn+p−1‖2 =∑

n≤j,j′≤n+p−1

E 〈Xj , Xj′〉 .

by stationarity of (Xn). We use now (8) with n = h and k = h − 1 to compute〈X0, Xh〉:

〈X0, Xh〉 =

⟨X0,

h−1∑j=0

(j−1∏p=0

ρIh−p

)(εh−j)

⟩+ 〈X0, ρIh

...ρI1 (X0)〉 . (11)

By another way

E

⟨X0,

h−1∑j=0

(j−1∏p=0

ρIh−p

)(εh−j)

⟩=

h−1∑j=0

E

[⟨X0,

(j−1∏p=0

ρIh−p

)(εh−j)

⟩]

=h−1∑j=0

E

[⟨∑i

〈X0, ei〉 ei,∑

i

Ai,Ih...Ai,Ih−j−1 〈εh−j , ei〉 ei

⟩]

=h−1∑j=0

E

[∑i

Ai,Ih...Ai,Ih−j−1 〈X0, ei〉 〈εh−j , ei〉

].

But ∑i

E[∣∣Ai,Ih

...Ai,Ih−j−1 〈X0, ei〉 〈εh−j , ei〉∣∣] < ∞.

8

Indeed, supposing without loss of generality that ‖ρ1‖ ≥ ‖ρ0‖ ,∑i

E[∣∣Ai,Ih

...Ai,Ih−j−1 〈X0, ei〉 〈εh−j , ei〉∣∣] ≤ ‖ρ1‖j

∑i

E [|〈X0, ei〉 〈εh−j , ei〉|]

≤ ‖ρ1‖jE

[∑i

|〈X0, ei〉 〈εh−j , ei〉|

]

≤ ‖ρ1‖j 12E

[∑i

|〈X0, ei〉|2 + |〈εh−j , ei〉|2]

≤ ‖ρ1‖j 12

(E ‖X0‖2 + E ‖ε0‖2

)< ∞.

Consequently,

E

⟨X0,

h−1∑j=0

(j−1∏p=0

ρIh−p

)(εh−j)

⟩=

h−1∑j=0

∑i

E[Ai,Ih

...Ai,Ih−j−1 〈X0, ei〉 〈εh−j , ei〉]

=h−1∑j=0

∑i

E[E[Ai,Ih

...Ai,Ih−j−1 〈X0, ei〉 〈εh−j , ei〉∣∣σ (I1, ..., Ih)

]]=

h−1∑j=0

∑i

E[Ai,Ih

...Ai,Ih−j−1E [ 〈X0, ei〉 〈εh−j , ei〉|σ (I1, ..., Ih)]].

Now, owing to the independence of X0 and εh−j with σ (I1, ..., Ih) ,

E [ 〈X0, ei〉 〈εh−j , ei〉|σ (I1, ..., Ih)] = E [〈X0, ei〉 〈εh−j , ei〉] = 0.

and therefore

E

⟨X0,

h−1∑j=0

(j−1∏p=0

ρIh−p

)(εh−j)

⟩= 0.

We manage the second term of (11) in a similar way

E 〈X0, ρIh...ρI1 (X0)〉 = E

[⟨∑i

〈X0, ei〉 ei,∑

i

Ai,Ih...Ai,I1 〈X0, ei〉 ei

⟩]

= E

[∑i

Ai,Ih...Ai,I1 〈X0, ei〉2

]=∑

i

E[Ai,Ih

...Ai,I1 〈X0, ei〉2]

because all the terms are non negative, so

E 〈X0, ρIh...ρI1 (X0)〉 =

∑i

E[E[Ai,Ih

...Ai,I1 〈X0, ei〉2∣∣∣σ (I1, ..., Ih)

]]=∑

i

E[Ai,Ih

...Ai,I1E[〈X0, ei〉2

∣∣∣σ (I1, ..., Ih)]]

.

But, owing to the independence of X0 with I1, ..., Ih,

E[〈X0, ei〉2

∣∣∣σ (I1, ..., Ih)]

= E[〈X0, ei〉2

].

9

Hence,E 〈X0, ρIh

...ρI1 (X0)〉 =∑

i

E [Ai,Ih...Ai,I1 ]E

[〈X0, ei〉2

].

Since

E [Ai,Ih...Ai,I1 ] ≤ E [A1,Ih

...A1,I1 ]

≤h∑

j=0

Cjhqj(1− q)h−j ‖ρ1‖j ‖ρ0‖h−j

= ((1− q) ‖ρ0‖+ q ‖ρ1‖)h,

we get

E 〈X0, ρIh...ρI1 (X0)〉 ≤ [(1− q) ‖ρ0‖+ q ‖ρ1‖]h

∑i

E[〈X0, ei〉2

]= [(1− q) ‖ρ0‖+ q ‖ρ1‖]h E ‖X0‖2

.

Accordingly,

E ‖Xn + ... + Xn+p−1‖2 ≤ 2pE ‖X0‖2p−1∑h=0

[(1− q) ‖ρ0‖+ q ‖ρ1‖]h .

As (1− q) ‖ρ0‖+ q ‖ρ1‖ < 1,

E ‖Xn + ... + Xn+p−1‖2 = O(p),

which entails (9), and applying [5, Cor. 2.3], we obtain (10).

4 Estimation of the covariance estimator

At this point, we use the same method as in [4]. Let us recall that the covarianceoperator and the empirical covariance operator are defined respectively by

C(.) = E [〈X0, .〉X0]

Cn(.) =1n

n∑t=1

〈Xt, .〉Xt.

Theorem 4.1 Under (H1) and (H2), under all the assumptions of Theorem 2.1, andif

α2 := (1− q) ‖ρ0‖2 + q ‖ρ1‖2< 1,

then

E ‖Cn − C‖2S = O

(1n

), (12)

‖Cn − C‖S →n→∞

0 a.s. (13)

10

Proof. Define the following random variables with values in the space S of Hilbert-Schmidt operators on H:

Zt = 〈Xt, .〉Xt − C, t = 1, 2, ...

Note that (Zt) is a stationary process. Moreover,

E ‖Zt‖2S ≤ 2E ‖〈Xt, .〉Xt‖2

S + 2 ‖C‖2S

≤ 2E ‖X0‖4 + 2 ‖C‖2N

by [5, (1.49), (1.55)]. Hence,

E ‖Zt‖2S ≤ 2E ‖X0‖4 + 2

(E ‖X0‖2

)2

=: 2M < ∞

by [5, (1.59)].Because of the stationarity,

E ‖Zn + ... + Zn+p−1‖2S ≤ 2pM + 2

p−1∑h=1

(p− h)E 〈Z0, Zh〉S .

Let us find an upper bound for E 〈Z0, Zh〉S . Use the relation [4, (25)] :

E 〈Z0, Zh〉S =∑j,l

E (〈X0, ej〉 〈X0, el〉 〈Xh, ej〉 〈Xh, el〉)−∑j,l

〈C(ej), el〉2 , (14)

then replace Xh by εh +ρIh(εh−1)+ ...+ρIh

...ρI2(ε1)+ρIh...ρI1(X0) owing to relation

(8). We only find in the first sum of (14) terms of the form

E(〈X0, ej〉 〈X0, el〉

⟨ρIh

...ρIp(εp−1), ej

⟩ ⟨ρIh

...ρIp(εp−1), el

⟩).

orE (〈X0, ej〉 〈X0, el〉 〈ρIh

...ρI1(X0), ej〉 〈ρIh...ρI1(X0), el〉)

Terms of the following form vanish for p 6= k : as a matter of fact,

E(〈X0, ej〉 〈X0, el〉

⟨ρIh

...ρIp(εp−1), ej

⟩〈ρIh

...ρIk(εk−1), el〉

)= E (〈X0, ej〉 〈X0, el〉)E

(⟨ρIh

...ρIp(εp−1), ej

⟩〈ρIh

...ρIk(εk−1), el〉

)owing to independence properties. But

E(⟨

ρIh...ρIp

(εp−1), ej

⟩〈ρIh

...ρIk(εk−1), el〉

)= E

[E[⟨

ρIh...ρIp

(εp−1), ej

⟩〈ρIh

...ρIk(εk−1), el〉

∣∣ (In)]]

= E[E[⟨

ρIh...ρIp(εp−1), ej

⟩∣∣ (In)]E [ 〈ρIh

...ρIk(εk−1), el〉| (In)]

]because the random variables ρIh

...ρIp(εp−1) and ρIh

...ρIk(εk−1) are conditionally in-

dependent with respect to (In) by the independence of εp−1 and εk−1. Now

E (〈ρIh...ρIk

(εk−1), ej〉) = E [E [ 〈ρIh...ρIk

(εk−1), ej〉| (In)]] (15)= E [〈E [ρIh

...ρIk(εk−1)| (In)] , ej〉]

= E [〈ρIh...ρIk

E [εk−1| (In)] , ej〉]

11

by Proposition 3.1. Owing to independence properties and knowing that ε0 is centered,

E [εk−1| (In)] = E [εk−1] = 0.

Terms of the following form

E (〈X0, ej〉 〈X0, el〉 〈ρIh...ρIk

(εk−1), ej〉 〈ρIh...ρI1(X0), el〉)

also vanish due to independence properties, in a similar way as in (15).For p = k, consider the terms of the form

E(〈X0, ej〉 〈X0, el〉

⟨ρIh

...ρIp(εp−1), ej

⟩ ⟨ρIh

...ρIp(εp−1), el

⟩)= E (〈X0, ej〉 〈X0, el〉) E

(⟨ρIh

...ρIp(εp−1), ej

⟩ ⟨ρIh

...ρIp(εp−1), el

⟩)as a result of independence properties. But

〈C(ej), el〉 = E (〈Xh, ej〉 〈Xh, el〉)= E [〈εh, ej〉 〈εh, el〉] + ... + E (〈ρIh

...ρI2(ε1), ej〉 〈ρIh...ρI2(ε1), el〉)

+ E (〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉)

by (8), the other terms involved in this relation yield vanishing expectations by use oftechniques of the same kind as seen before. Accordingly,∑

p

E(⟨

ρIh...ρIp(εp−1), ej

⟩ ⟨ρIh

...ρIp(εp−1), el

⟩)= 〈C(ej), el〉 − E (〈ρIh

...ρI1(X0), ej〉 〈ρIh...ρI1(X0), el〉) .

Thus,

E 〈Z0, Zh〉S =∑j,l

E (〈X0, ej〉 〈X0, el〉 〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉)

−∑j,l

〈C(ej), el〉E (〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉) .

12

Let us find upper bounds for each sum in absolute value. For the first one, note that∣∣∣∣∣∣∑j,l

E (〈X0, ej〉 〈X0, el〉 〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉)

∣∣∣∣∣∣=∑j,l

E[(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1) 〈X0, ej〉2 〈X0, el〉2

]=∑j,l

E[E[(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1) 〈X0, ej〉2 〈X0, el〉2

∣∣∣σ (I1, ..., Ih)]]

=∑j,l

E[(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1) E

[〈X0, ej〉2 〈X0, el〉2

∣∣∣σ (I1, ..., Ih)]]

=∑j,l

E[(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1) E

[〈X0, ej〉2 〈X0, el〉2

]]=∑j,l

E [(Aj,Ih...Aj,I1) (Al,Ih

...Al,I1)]E[〈X0, ej〉2 〈X0, el〉2

]≤[(1− q) ‖ρ0‖2 + q ‖ρ1‖2

]h∑j,l

E[〈X0, ej〉2 〈X0, el〉2

]≤[(1− q) ‖ρ0‖2 + q ‖ρ1‖2

]hE ‖X0‖4

.

For the second one, we reason as previously and obtain

E (〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉)= E [(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1) 〈X0, ej〉 〈X0, el〉]

= E [(Aj,Ih...Aj,I1) (Al,Ih

...Al,I1)]E [〈X0, ej〉 〈X0, el〉]= E [(Aj,Ih

...Aj,I1) (Al,Ih...Al,I1)] 〈C(ej), el〉 .

Hence, ∣∣∣∣∣∣∑j,l

〈C(ej), el〉E (〈ρIh...ρI1(X0), ej〉 〈ρIh

...ρI1(X0), el〉)

∣∣∣∣∣∣=∑j,l

〈C(ej), el〉2 E [(Aj,Ih...Aj,I1) (Al,Ih

...Al,I1)]

≤[(1− q) ‖ρ0‖2 + q ‖ρ1‖2

]h‖C‖2

S .

But‖C‖2

S ≤ ‖C‖2N =

(E ‖X0‖2

)2

,

so setting α2 =[(1− q) ‖ρ0‖2 + q ‖ρ1‖2

]|E 〈Z0, Zh〉S | ≤ αh

2

[E ‖X0‖4 +

(E ‖X0‖2

)2]

= Mαh2 .

13

Consequently, as α2 < 1,∣∣∣∣∣2p−1∑h=1

(p− h) E 〈Z0, Zh〉S

∣∣∣∣∣ ≤ 2M

p−1∑h=1

(p− h)αh2

≤ 2Mp

p−1∑h=1

= 2Mpα21− αp−1

2

1− α2

≤ 2Mpα2

α2 − 1.

In conclusion,E ‖Zn + ... + Zn+p−1‖2

S = O (p) ,

and therefore we get directly (12), and from [5, Cor. 2.3] we obtain (13).The ultimate goal is to estimate the autocorrelation operators ρ0 and ρ1. For that

purpose, we can not use the following relation (see [5, Th. 3.2]) which holds in thecase of ARH(1) processes:

D = ρC,

where D is the cross-covariance operator, as there are now two choices for ρ. Thisis why we define two cross-covariance operators Di and their respective empiricalestimators Di,n enabling us to use relations of the same kind. Let us set, for i = 0, 1

Di(.) = E [1I1=i 〈X0, .〉X1] , i = 0, 1.

This operator corresponds to the cross-covariance operator, but only in the case wherethe evolution from Xn to Xn+1 is carried out through the use of ρi. Note that for allx ∈ H,

Di(x) = E [1I1=i 〈X0, x〉 (ρI1 (X0) + ε1)]= E [1I1=i 〈X0, x〉 ρI1 (X0)] + E [1I1=i 〈X0, x〉 ε1]= E [E [1I1=i 〈X0, x〉 ρI1 (X0)| I1]] + E [E [1I1=i 〈X0, x〉 ε1| I1]]= E [1I1=iE [ 〈X0, x〉 ρI1 (X0)| I1]] + E [1I1=iE [ 〈X0, x〉 ε1| I1]]= E [1I1=iρI1E [ 〈X0, x〉X0| I1]] + E [1I1=iE [〈X0, x〉 ε1]]

using Proposition 3.1 for the first term and independence properties for the second.Hence,

Di(x) = E [1I1=iρI1E [〈X0, x〉X0]]= E [1I1=iρI1C(x)]= P (I1 = i)ρiC(x),

and setting qi = P (I1 = i), the relation comes out

Di = qiρiC (16)

The empirical estimator of Di is

Di,n(x) =1

n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉Xt+1,

In order to obtain the consistency, the assumptions will be stronger than those used forthe covariance operator: we will assume that the Xt are bounded for purely technicalreasons.

14

Theorem 4.2 Under (H1) and (H2), under all the assumptions of Theorem 2.1, ifthere is an A > 0 such that

‖Xt‖ < A, t ∈ Z,

and ifα2 := (1− q) ‖ρ0‖2 + q ‖ρ1‖2

< 1,

then

E ‖Dn,i −Di‖2S = O

(1n

), i = 0, 1 (17)

‖Dn,i −Di‖S →n→∞

0 a.s. , i = 0, 1. (18)

Proof. Fix i. Set

Z ′t = 1In+1=i 〈Xt, .〉Xt+1 −Di

= 1It+1=i 〈Xt, .〉 εt+1 + 1It+1=i 〈Xt, .〉 ρIt(Xt)− qiρiC

by relation (16). So

1n− 1

n−1∑t=1

Z ′t (19)

=1

n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉 εt+1 +1

n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉 ρIt(Xt)− qiρiC.

But the 1It+1=i 〈Xt, .〉 εt+1 are orthogonals in L2S because, for t < t′ without loss of

generality,

E⟨1It+1=i 〈Xt, .〉 εt+1,1It′+1=i 〈Xt′ , .〉 εt′+1

⟩S

= E∑i,j

1It+1=i1It′+1=i 〈Xt, ei〉 〈εt+1, ej〉 〈Xt′ , ei〉 〈εt′+1, ej〉 .

The inversion of the sum and the expectation is possible since∑i,j

E∣∣∣1It+1=i1It′+1=i 〈Xt, ei〉 〈εt+1, ej〉 〈Xt′ , ei〉 〈εt′+1, ej〉

∣∣∣≤ 1

2

∑i,j

E∣∣∣〈Xt, ei〉2 〈εt+1, ej〉2

∣∣∣+∑i,j

E∣∣∣〈Xt′ , ei〉2 〈εt′+1, ej〉2

∣∣∣

=12

[2E ‖X0‖2 ‖ε0‖2

]≤ 1

2E[‖X0‖4 + ‖ε0‖4

]< ∞.

Consequently,

E⟨1It+1=i 〈Xt, .〉 εt+1,1It′+1=i 〈Xt′ , .〉 εt′+1

⟩S

=∑i,j

E[1It+1=i1It′+1=i 〈Xt, ei〉 〈εt+1, ej〉 〈Xt′ , ei〉 〈εt′+1, ej〉

]=∑i,j

E[1It+1=i1It′+1=i 〈Xt, ei〉 〈εt+1, ej〉 〈Xt′ , ei〉

]E [〈εt′+1, ej〉]︸ ︷︷ ︸

=0

= 0.

15

Hence,

E

∥∥∥∥∥ 1n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉 εt+1

∥∥∥∥∥2

S

=1

(n− 1)2

n−1∑t=1

E∥∥1It+1=i 〈Xt, .〉 εt+1

∥∥2

S

≤ 1n− 1

E ‖X0‖2 ‖ε0‖2 = O

(1n

).

With regard to the second and third terms of (19), we can simply write down thefollowing identity

1It+1=i 〈Xt, .〉 ρIt(Xt) = 1It+1=i 〈Xt, .〉 ρi (Xt) a.s.,

which yields

1n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉 ρIt (Xt)− qiρiC (20)

= ρi

[1

n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉Xt − qiC

]

= ρi

[1

n− 1

n−1∑t=1

1It+1=i 〈Xt, .〉Xt −1

n− 1

n−1∑t=1

qi 〈Xt, .〉Xt

]

+ ρi

[1

n− 1

n−1∑t=1

qi 〈Xt, .〉Xt − qiC

].

The second term of the last right-hand side of (20) is merely equal in L2S -norm to

qiE ‖ρi (Cn − C)‖2S = O

(1n

)by Theorem 4.1. The first term in L2

S -norm is equal to

E

∥∥∥∥∥ρi

(1

n− 1

n−1∑t=1

(1It+1=i − qi

)〈Xt, .〉Xt

)∥∥∥∥∥2

S

= E

∥∥∥∥∥ 1n− 1

n−1∑t=1

(1It+1=i − qi

)〈Xt, .〉 ρi (Xt)

∥∥∥∥∥2

S

≤ 1(n− 1)2

E

(n−1∑t=1

∣∣1It+1=i − qi

∣∣ ‖〈Xt, .〉 ρi (Xt)‖S

)2

≤ 1(n− 1)2

E

(n−1∑t=1

∣∣1It+1=i − qi

∣∣ ‖ρi‖L ‖Xt‖2

)2

≤‖ρi‖2

LA4

(n− 1)2E

(n−1∑t=1

∣∣1It+1=i − qi

∣∣)2

=‖ρi‖2

LA4

(n− 1)2

n−1∑t=1

E[∣∣1It+1=i − qi

∣∣2] =‖ρi‖2

LA4

(n− 1)E[|1I0=i − qi|2

]

16

because the random variables∣∣1It+1=i − qi

∣∣ are i.i.d. and centered, therefore

E

∥∥∥∥∥ 1n− 1

n−1∑t=1

Z ′t

∥∥∥∥∥2

S

= O

(1n

),

which implies straightforward (17), and (18) with the help of [5, Cor. 2.3] but arguingin a similar way about Z ′n + ... + Z ′n+p−1.

Finally, with the outcomes of this section, it is possible to slightly modify theresults of [12] in order to obtain rates of convergence - such as n−1/4 in the generalcase - of an estimator denoted by ρn,i,a of ρi (for i = 0, 1). As the technique is similar,this topic is omitted.

References

[1] J. Bass (1955). Sur la compatibilite des fonctions de repartition. C.R. Acad. Sci.Paris 240, 839-841.

[2] P. Besse, H. Cardot (1996). Approximation spline de la prevision d’un processusfonctionnel autoregressif d’ordre 1. Revue Canadienne de Statistique/CanadianJournal of Statistics, 24, 467-487.

[3] P. Besse, H. Cardot, and D. Stephenson (2000). Autoregressive forecasting of somefunctional climatic variations. Scandinavian Journal of Statistics 27(4), 673–687.

[4] D. Bosq (1991). Modelization, nonparametric estimation and prediction for con-tinuous time processes, p. 509-529. In G. Roussas (Ed.), Nonparametric functionalestimation and related topics, Nato Asi Series C, Kluwer.

[5] D. Bosq (2000). Linear processes in function spaces. Lecture Notes in Statistics149, Springer.

[6] P. Bougerol, N. Picard. Strict stationarity of generalized autoregressive processes.Ann. Probab. 20 (1992), no. 4, 1714–1730.

[7] A. Brandt. The stochastic equation Yn+1 = AnYn+Bn with stationary coefficients.Adv. in Appl. Probab. 18 (1986), no. 1, 211–220.

[8] A. Cavallini, G.C. Montanari, M. Loggini, O. Lessi, M. Cacciari (1994). Nonpara-metric prediction of harmonic levels in electrical network. Proceed. IEEE ICHPS6, Bologna.

[9] J. Damon, S. Guillas (2002). The inclusion of exogenous variables in functionalautoregressive ozone forecasting. Environmetrics, to appear.

[10] C. Francq, J.-M. Zakoıan (2001), Stationarity of multivariate Markov–switchingARMA models, Journal Of Econometrics 102, n. 2 pp. 339-364

[11] S. Guillas (2000). Non-causalite et discretisation fonctionnelle, theoremes limitespour un processus ARHX(1). C. R. Acad. Sci. Paris, t. 331, Ser. I, 91-94.

[12] S. Guillas (2001). Rates of convergence of autocorrelation estimates for autore-gressive Hilbertian processes. Statistics and Probability Letters, 55, n. 3, pp. 281-291.

[13] L. de Haan, S. Resnick, H. Rootzen, C. de Vries (1989). Extremal behaviour ofsolutions to a stochastic difference equation with applications to ARCH processes.Stochastic Processes and their Applications 32, 213-224.

17

[14] U. Horst (2001). The stochastic equation Yt+1 = AtYt + Bt with non-stationarycoefficients. J. Appl. Probab. 38, no. 1, 80–94.

[15] G. Melard, R. Roy (1988). Modeles de series chronologiques avec seuils. Revuede Statistique appliquee XXXVI, n. 4, p. 5-24.

[16] D.F. Nicholls, B.G. Quinn (1982). Random coefficent autoregressive models: anintroduction. Lecture Notes in Statistics 11, Springer, New York.

[17] J.D. Petrucelli, S.W. Woolford (1984). A threshold AR(1) model. J. Appl. Prob.21, 270-286.

[18] M. Pourahmadi (1986). On stationarity of the solution of a doubly stochasticmodel. Journal of Time Series Analysis, Vol. 7, n. 2.

[19] E. Rio (2000). Theorie asymptotique des processus aleatoires faiblementdependants. Mathematiques et Applications 31, Springer.

[20] D. Tjøsteim (1986). Some doubly stochastic time series models. Journal of TimeSeries Analysis, Vol. 7, n. 1.

[21] D. Tjøsteim (1994). Non-linear time series: a selective review. Scandinavian Jour-nal of Statistics, Vol. 21, p. 97-130.

[22] H. Tong (1983). Threshold models in nonlinear time series analysis. Lecture Notesin Statistics, Vol. 21, Springer, Berlin.

[23] H. Tong (1990). Nonlinear times series. Oxford University Press.

[24] J. Yao (2001). On square-integrability of an AR process with Markov switching.Statist. Probab. Lett. 52 , no. 3, 265–270.

[25] J.-F. Yao, J.-G. Attali (2000). On stability of nonlinear AR processes with Markovswitching. Adv. in Appl. Probab. 32 , no. 2, 394–407.

18

LISTE DES RAPPORTS TECHNIQUES

N1 Fevrier 1999 Autoregressive Hilbertian ProcessDenis BOSQ

N2 Avril 1999 Using the Entringer numbers to count the alternating permuta-tions according a new parameter

Christiane POUPARD

N1 Mars 2000 Noncausality and functional discretization, limit theorems for anARHX(1) process

Serge GUILLAS

N2 Juin 2000 Sieve estimates via neural network for strong mixing processesJian tong ZHANG

N3 Juillet 2000 Technical tools to evaluate Hausdorff-Besicovitch measure of ran-dom fractal sets

Alain LUCAS

N1 Avril 2001 Rates of convergence of autocorrelation estimates for autoregres-sive Hilbertian processes

Serge GUILLAS

N2 Avril 2001 Sur l’estimation de la densite marginale d’un processus a tempscontinu par projection orthogonale

Arnaud FRENAY

N3 Juin 2001 Rate-optimal data-driven specification testing in regression mod-els

Emmanuel GUERRE et Pascal LAVERGNE

N4 Juin 2001 The inclusion of exogenous variables in functional autoregressiveozone forecasting

Julien DAMON et Serge GUILLAS

N5 Aout 2001 Estimation and simulation of autoregressive Hilbertian processeswith exogenous variables

Julien DAMON et Serge GUILLAS

N1 Janvier 2002 Functional asymptotic normality of the L2-deviation of the kerneldensity estimation indexed by class of weight functions

Fateh CHEBANA

N2 Janvier 2002 Optimal sampling for density estimation in continuous timeDelphine BLANKE et Besnik PUMO

N3 Fevrier 2002 Doubly Stochastic Hilbertian processesSerge GUILLAS

20