A class of strong laws for functionals of countable nonhomogeneous Markov chains

ELSEVIER Statistics & Probability Letters 22 (1995) 87-96

STATISTICS& PROBABILITY

LETTERS

A class of strong laws for functionals of countable nonhomogeneous Markov chains

Wen Liu*, Guoxin Liu Department of Mathematics, Hebei Institute of Technology, Tianjin 300130, China

Received September 1992; revised February 1994

Abstract

In this paper, we use an analytic approach to study the limit properties of {f.(X.)}, the functional of a countable nonhomogeneous Markov chain {X. }. A class of strong laws of large numbers for these processes, which are different from the usual ones, are obtained. In the theorems of this paper, the expectation E(f.(X~)) in the usual strong laws of large numbers is replaced by the conditional expectation E(f~(X.IX._ 1 )). Some classical strong laws of large numbers for sequences of independent random variables are implied by the results of this paper.

AMS 1980 subject classifications: Primary 60J10, Secondary 60F15.

Key words: Functional of nonhomogeneous Markov chain; Strong law of large numbers; Strong law; Conditional expectation

1. Introduction

Let {X., n >/0} be a nonhomogeneous Markov chains with state space S = { 1, 2 . . . . } and { f . (x.), n/> 0} be a sequence of real functions defined in S. Also let

Y.=f . (X . ) , n>~O (1.1)

be the functional of {X., n >10}. We refer to Rosenblatt-Roth (1963, 1964) for related research on the strong law of large numbers for this class of stochastic processes. The purpose of this paper is to present a class of strong laws, concerning the conditional expectation. Some classical strong laws of large numbers for sequences of independent random variables can be easily deduced from the theorem. In this paper, the analytic approach proposed by the first author (see Liu Wen, 1990) is used. It is different from the traditional probabilistic method. The crucial part is the application of Lebesgue's theorem on differentiability of monotone function to the study of a.e. convergence.

* Corresponding author.

0167-7152/95/$9.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0167-71 52(94)00053-B

88 W. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96

Throughout this paper we shall deal with the underlying probability space ([0, 1] ,~- ,P) , where ar is the class of Borel sets in the interval [0, 1), and P is the Lebesgue measure. We first give, in the above probability space, a realization of the Markov chain with initial distribution and transition matrices, respectively,

q(1), q(2), q(3) . . . . . (1.2)

P.=(p . ( i , j ) ) , i , j~S , n>~O. (1.3)

where p.( i , j )= P(Xn+ 1 =J I Xn = i).

Let

q(ni), i= 1, 2, 3,.. .

be the positive terms of (1.2), where nl <n2 <n3 < - . . . Divide the interval [0, 1) into countably (possibly finitely) many right-semiopen intervals: Ixo, Xo = n~, n2, n3 . . . . . i.e.

In1 = [0, q(nl)), In2 = [q(nl), q(nl)+q(n2)) . . . . .

which will be called intervals of order 0. In general, suppose Ixo . . . x , is an interval of order n, and the positive elements of the x. th row of the matrix P., in their natural order, are

p.(x . ,mi) , i= 1,2,3 . . . .

By dividing Ixo . . . x . into countably many right-semiopen intervals Ixo . . . x.xn+ ~ (x.+ ~ = mi, i = 1, 2, 3 . . . . ) at the ratio

p,,(x,,, ml): p,,(x,,, m2):, p,,(x,,, mz): ....

the intervals of order n + 1 are constructed. It is easy to see that

n - - 1

P(Ixo . . . x , , )=q(xo) l-[ p,,,(x,,, x,,,+l). (1.4) m=O

For n>~0, define a random variable X,: [0, 1)~S by

X,(~o)=x,, i fcoelxo. . .x , , . (1.5)

By (1.4) and (1.5), we have

n - - I

P ( X o = x o . . . . ,X , ,=x , , )=P( I xo . . . x , )=q (xo ) I-[ p,,(x,,,,x,,+ l). (1.6) m = O

Therefore, {X,, n/> 0} is a Markov chain with initial distribution (1.2) and transition matrices (1.3).

2. Fundamental lemma

Let {a., n~> 1} be a sequence of constants such that 0<a .T . Also for n~> 1, define the function f * ( x ) by

f , ( x ) = ~ f _ . ( x ) if .f.(x)l<<,a., (2.1) ~o if I f . (x ) l>a. .

W. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96

Let i~S, and let 2 be a nonzero constant. If P(X._x=i)>O, then let

b*(i )=E[f*(X.) lX._~=i] , n>,l.

Q.(2, i)= E {exp [2(f*(X,)-b*(i)) /a.] IX._1 = i}

= ~ p ._ l ( i , j )exp{2[ f*( j ) -b*( i ) ] /a . } . j = l

It is easy to see that

I[ f*(X.)-b*(i)] /a.I <~2.

Fundamental Lemma 1. For every nonzero constant 2,

{ [ f , . (X.) -b . (Xm-1)] /a~,} exp 2 Z~,= 1 * * lim exists and is finite a.e.

. I]"=~ Q..(~, X ._x)

89

(2.2)

(2.3)

(2.4)

Proo£ Let ~¢ denote the collection consisting of the interval [0, 1) and the intervals of all orders. Define a set function/~ on ~¢ by

P(Ixo.. .x.)exp {2 Y~:= 1 [f*(x. ,)-b*(xm-1)]/a, ,} #(Ixo. . .x.)= , n>>, 1; (2.5)

1-I"=1 O..(,t, X. -x)

!a(Ixo ) = ~ I~(I xox x ), (2.6)

#([0, 1 ) ) = ~ #(Ixo), (2.7) XO

where Y.x, denote the sum taken over all possible values of x~. Rewrite (2.3) as

Q.(2, x._ a) = ~ p,- x (X._l, x.) exp {2 I f * (x.) - b* (x._,)]/a. }. (2.8) Xn

By (2.5), (2.8) and (1.6), it is easy to see that

#(lxo... x.)= #(Ixo ... x._ x). (2.9) x .

By (2.6), (2.7) and (2.9), we have that/~ is sigma additive on ~¢. Define a monotone function, fx(t), on [0, 1) by

fa(t) = inf {/~(A): A = [0, t), A ea(~¢)},

where a ( d ) is the algebra generated by d . It is obviously that

~(lxo... x.) =fx(l + Xo... x.)-fa(1-Xo.. , x.) (2.10)

for any Ixo.. .x. , where I - xo . . . x . and I+xo.. .x. denote, respectively, the right and left endpoints of lxo ... x.. Let

t.(A,~o)=f~(I+xo...x.)--fa(l-xo...x.)= #(Ixo.. .x.) ¢oelxo...x., n>,l. (2.11) I+xo . . . x . - - I -xo . . . x . P(xo ..... x.)'


Let A(2) be the set of points of differentiability of fa. Then by the existence theorem for the derivative of a monotone function, P(A(2))= 1. Let o)eA(2), lxo. . , x. be the interval of order n that the ~o belongs to. If lim. P(Ixo. . . x , ) = 0, according to a property of the derivative (see Billingsley, 1986, p. 423), we have by (2.11)

lim t.(2, co)=f~(o)) < oo n

If lim. P(lxo. . . x.) = d > 0, then

lira t.(2, o)) = ( l / d ) lim I~(Ixo ... x.) < oo. n n

By (2.12) and (2.13), we have

lim t.(2, o)) exists and is finite, meA(2). n

By (1.6) and (2.5),

t.(2, ~o)_ exp {)1, ~"= x [ f*(Xn, ) -b*(xm-1)] /am}

I]~n=l Ore( "~', X m - 1 )

By (2.15) and (1.5), we have

{ • , } exp 2Z~,= 1 [ fm(X, . ) -bm(Xm-1)] /a , . t. (2, o ) =

1-I~,= 1Qm(2, X,.- a)

Noticing that P(A(2))= 1 by (2.14) and (2.16), we have

exp {2 Y~,=, [ f*(Xm)- -b*(X, . -1)] /am} lira - n

n 1 - [ . = 1 O . ( ~ , Xm-1)

This completes the proof of the lemma.

(2.12)

(2.13)

(2.14)

coe lxo ... x,. (2.15)

ooelxo . . .x . , o)e[0, 1). (2.16)

exists and is finite a.e.

3. Main result

Theorem 1. Let { Y., n~>0} be the functional of a countable nonhomogeneous Markov chain (X., n>~0} and defined by (1.11); let {~o.(x), n~>l} be a sequence of continuous even functions which are positive and nondecreasing in (0, oo). Let {a., n >/1 } be a sequence of constants such that 0 < ant and

E[tp(Yn)/qg(a,)] < oo. (3.1) n=l

(a) Zf as Ixl increases, ~o,,(x)/Ixl is nonincreasing, then

~, YJan converges a.e., (3.2) n = l

(I/an) ~, Ym~O a.e. (3.3) m = l

IV. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96 91

(b) I f as Ix I increases, tp.(x)/lxl is nondecreasing and ~p.(x)/x 2 is nonincreasing, then

[Y.--E(Y. IX._I)]/a . converges a.e., (3.4) . = 1

(i/a.) ~ [Y,.-E(YmlXm_I)]~O a.e. (3.5) m = l

Proof. Since the hypothesis (3.1) can be rewritten as

E [E [tp.( II.)1X._ 1 ]/tp.(a.)] < ~ (3.6) . = 1

and E[tp.(Y.)IX._I] is a nonnegative random variable (n ~> 1), we have

~E[tp.(Y.) lX._l]/ tp.(a. ) converges a.e. (3.7) . = 1

By the hypothesis: ~p.(x)T as Ixl increases, and by (3.1) we have

P(fm(x.,) ef*~(xm)) = e ( d ~ o ) m = l =1 f={X.)l > a .

~, [qam(fm(Xm))/tpm(am) ] P(dto) ~< E[tpm(Ym)]/tpm(am)< oo. (3.8) m = l f . ( X . ) l > a , r a = l

By (3.8) and Borel-Cantelli's lemma, we have

fm(X,,)#f*(Xm) only for a finite number of terms a.e.

Hence,

(1/am)[f~(Xm)--f*(X,.)] converges a.e. (3.9) m = l

By (2.2) and (2.3), we have

E { [2 ( f * (X . ) - b* (i))/a. ] [ X . _ 1 = i } = 0; (3 .10)

Q.(2, i ) -1 = E{ [exp (2(f* (Xn)-- b* (i))/a.)--I

- - 2 ( f * ( X . ) - - b* (i))/a.] [ X . _ 1 = i} . (3 .11)

Also by (2.2)

(i)l = f f ~ f*.(x)dFx., x._.(xli) <~ f_~ If*(x)ldFx., x.-l(xli). (3.12) Ib*

where Fx. Ix.-1 (x I i) = P(X. <. x lX._ 1 = i). the conditional distribution function relative to {X._ 1 = i}. By the inequalitied below

e X - l - x > ~ 0 for all x; l e X - l l ~ l x l e t, -t<~x<~t


and (3.11), (2.3), (2.4), (3.12) and (2.1), we have

0 ~< E { [exp (2(f* (X.)- b* (i))/a,)-I - 2 ( f* (X.)- b* (i))/a.] I X._ x = i} = Q.(2)-I

= E { [exp(2( f* (X.)-b* (i) )/ a.)-l ] IX.-1 =i}

. I ~ l exp (2(f* (x)-- b* (i) )/a,,)-I I dFx. Ix.-, (x I i) ~<

~<(1/a.) f~o ](2(f*(x)--b*(i))le21~ldFx"lx"-'(x[i)

f f ~ (1/a.)lf*(x)ldFx. lx._,(x Ii) ~212le 21at

=2121 eztal fy.(x)l<~a (1/a.)lf.(x)] dFx. lx.-l(xli) •

Now suppose the hypothesis in (a) holds. Obviously,

I f.(x) I/a,, <~ ~o.(f.(x))£o.(a.), since If,,(x) I ~< a,,.

By (3.13) and (3.14), we have

0~<Q,(2, i ) -1 ~<2121emr f q~.(f.(x))/~o.(a.)dFx.lx,_t(xl i) d;L(x)l <~a.

~<2121e ztal ~o.(f.(x))/~o.(a.)dFx, jx._,(xli)=2121e21~l E(~o.(YDlX._,=i)/~o,(aD. --O0

By (3.15) and (3.7), we have

~', [Q.(2, X . - 1 ) - I ] converges a.e. n=l

Hence,

f i Q . ( 2 , x . _ l ) converges a.e. n=l

By the fundamental lemma and (3.16), we have

{ ~ [f.(X.,-b.,(X.,_~)]/a,.} exists andisf ini tea.e . lim exp 2 * * m=l

Letting 2= 1, --1 in (3.17), we get, respectively,

l imexp{ ~ * * } [ f . (X,,,-b.,(X.,-D]/a,. exists and is finite a.e. n m=l

{ ~ [ f ~ (Xm -- b ~1~1 (Xlll -- 1 )]/am } exists and is finite a.e. lim. exp - , ,= 1

(3.13)

(3.14)

(3.15)

(3.16)

(3.17)

(3.18)

(3.19)

W. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96

(3.18) and (3.19) imply

(1/a,)[f*(Xm)-b*(X~_l)] converges a.e. n l = l

By (2.2), (2.1) and (3.14), we have

~. (1/a,)lb*(i)l m = l

~< ~ (1/am)~ Ifm(x) ldFx,,Ixm_,(xli) m = 1 f . (x ) l ~<a.

<<. ~. [¢Pm(fm(X)/tpm(a,~)]dFxmlx,~_l(xli)<<, EEtpm(fm(Xm))lXm-l=i)]fipm(am). m = l f.(x)l<~a, m = l

i . e . ,

(1/a.)lb*(i)l<~ ~ EE~o.(Ym)lX.-1)]/~.(am). m = l m = l

By (3.21) and (3.7),

(1/am)b*(Xm-1) converges a.e. r a = l

By (3.9), (3.20) and (3.22) we conclude

~, Ym/am converges a.e., m = Z

i.e., (3.2) is true. By Kronecker's lemma (3.3) follows from (3.2). Since

0~<eX-l-x<<.x2e t, - t ~ x ~ t .

by (3.11) and (2.4), we have

0 ~< Qn(,~, i ) -1 ~22e21~lE{[f*(Xn)/an'] 2] IXn_l =i}.

Hence

0 ~< Qn(2, Xn- 1)-1 ~< 22e21~lE { Ef* (Xn)/an] 2] [Xn- 1 }.

Suppose the hypothesis in (b) holds. Since tpn(x)/x2~ as I xl increases,

x2/a 2 <<. tpn(x)/~p,(an), as I xl ~< an.

Since tp,(x)T on (0, c~), we have

E f* (Xn)/an] 2 <<, ¢Pn I f* (Xn)q/q)n(an)~ tpn(Yn)/cpn(an)

By (3.23) and (3.24), we have

0 ~< Qn(2, x~_ 1)-1 <<,[22e21~l/tpn(an)]EEtPn(Yn)lX~_~}.

93

(3.20)

(3.21)

(3.22)

(3.23)

(3.24)

(3.25)

94

Also, by (3.25) and (3.7),

Y, [Q.(,~, x._ 0-1] . = 1

Hence

I~I Q.(,~, X.-x) n = l

IV. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96

converges a.e.

converges a.e.

Noticing that P(A(2))= 1 by the fundamental lemma and (3.26), we have

l imexp{2 ~ = , 1 [f*(X,.)-b*(Xm_~)]/a,,} exists and is finite a.e.

Imitating the proof of (3.20), by (3.27) we have

[f,~(X,~)-b.,(X,,-1)]/a,~ converges a.e. m = l

Let

b.(i)=E(Y, lX._~=i), n>~l.

Then E(Y. IX.-1)=b.(X.-x). By the hypothesis in (b), ~o,(x)/xT as Ixl increases. Hence

Ixl/a,. <~o,.(x)/~o,,,(a,.), as Ixl>~a,..

By (2.2), (3.29) and (3.30),

(1/a,,,)lb.,(i)-b*(i)l=(1/am)[ f~-oo [f,.(x)--f*(x)]ldFxmlx._,(xli)

~< (1/a") If.(x)l>a. If"(x)ldFx"lXm-l(xli)~iS.(x)i >.. [¢p,.(f,,,(x))/~o,.(a,.)]dfxmlx.,_~(xli)

~< [ 1/~om (a,.)] E [q~,,,(f,,(X,,,))lX,._ ~ = i], i.e.,

(l/a,.) I bin(X,,_ 1)- b*~(X,,_ 1 )1 ~ [ 1/O,.(a,,)] E [~o.(fm(Xm))lX.,- 1 ]"

By (3.31) and (3.7), we have

~(1/a,~)[b*(X,~-l)--bm(Xm-1)] converges a.e. r a = l

By (3.9), (3.28) and (3.32), we have

~(1/am)[fm(Xm-l)-bm(Xr,-1)] converges a.e., m = l

i.e., (3.4) holds. By Kronecker's lemma we get (3.5) from (3.4). This completes the proof.

(3.26)

(3.27)

(3.28)

(3.29)

(3.30)

(3.31)

(3.32)

W. Liu, G. Liu / Statistics & Probability Letters 22 (1995) 87-96 95

4. Some corollaries

Corollary 1. Let {Xn}, {Yn}, {an} be defined as in the theorem. And let {~pn(x), n~>l} be a sequence of continuous even functions such that they are positive on (0, oo) and as I xl increases, ~pn(X)/Ixl is nondecreasing and ~pn(x)/x 2 nonincreasing; and suppose (3.1) holds. Then

iff

/ff

• [Yn-E(Yn)]/an converges a.e.; I1=1

{E(Y, n = l

tXn-1)--E[E(YnlX.- a)]}/an

(1~an) ~ [Ym--E(Ym)]~O a.e. m = l

(1~an) m=l

{E(YmlXm-1)--E[E(Y~IX.-1)])

converges a.e.:

converges a.e.

Proof. Noticing that E [E(YnIXn_ 1)] = E(Yn), the corollary is obtained immediately from Theorem 1 (b).

Corollary 2 below points out that some classical results (including the results of Kolmogorov, Marcin- kiewicz and Zygmund, etc.) can be deduced from Theorem 1.

Corollary 2. Let {Xn, n>>. 1} be a sequence of independent random variables, {an, n>~ 1}, {~%(x), n~> 1) be the same as in the theorem, and

E[~on(Xn)]/~%(a,)< oo. (4.1) n = l

(a) If, as Ix[ increases, ~on(x)/Ixl is nonincreasing, then

~ Xn/an converges a.e., (4.2) n=l

(1/an) ~ Xm~O a.e. (4.3) m = l

(b) If, as Ixl increases, ~o.(x)/Ixl is nondecreasing and q)n(X)/X 2 is nonincreasing, then

~ [Xn-E(Xn)]/an converges a.e. (4.4) n = l

(1/an) ~ [Xm-E(Xm)]~O a.e., (4.5) m = l

Proof. Let

_~khn as khn<~Xn<(k+l)hn; Yn-~_khn as - - (k+ 1)hn<~Xn<-khn (4.6)


(k~>0; n>~ 1), where (h., n~> 1) is a sequence of positive numbers such that

~ h./a. < ~ . (4.7) n= l

Then { Y~/h. } is a sequence of independent integer-valued random variables. Hence { II. } can be considered as a functional of a Markov chain. By (4.6)

IY.-S~l<<.h., n>>.l. (4.8)

Since, as Ixl increases, q~.(x)T, and I Y.I ~<lX.I, by (4.6), we have

~o.(y.) ~< ~0.(x.).

Thus, by (4.1)

E[q),(Y,)]/tp,(a,)< ~, n= l

i.e., the condit ion (3.1) in the theorem is satisfied. Therefore, as the condit ions in (a) are satisfied, by (3.2), (3.3), (4.7) and (4.8), (4.2) and (4.3) are valid; as the condit ions in (b) are satisfied, by (3.4), (3.5), (4.7) and (4.8), (4.4) and (4.5) are valid.

Acknowledgement

The author would like to thank the referee for many valuable suggestions.

References

Billingsley, P. (1986), Probability and Measure (Wiley, New York). Chung, K.L. (1974), A Course in Probability Theory, 2nd ed. (Academic Press, New York). Liu Wen (1990), Relative entropy densities and a class of limit theorems of the sequence of m-valued random variables, Ann. Probab. 18,

829-839. Rosenblan-Roth, M. (1963), Some theorems co~acerning the law of large numbers for non-homogeneous Markov chains, Z.f . Wahrsch. 1,

433-445. Rosenblatt-Roth, M. (1964), Some theorems concerning the strong law of large numbers for non-homogeneous Markov chains, Ann.

Math. Statist. 35, 566-576.

Documents

A class of strong laws for functionals of countable nonhomogeneous Markov chains