6
Journal ofMatheraatical Sciences, VoL 93, No. 3, 1999 ONE-SIDED CONVERGENCE OF CONTINUOUS PROCESSES OF STOCHASTIC APPROXIMATION S. V. Komarov and T. P. Krasulina UDC 519.2 A modified continuous Robbins-Monro procedure is considered. Bibliography: 5 titles. w INTRODUCTION Let {Y(z), -co < z < co} be a family of random variables. Let P(Y(z) < y) = H(ylz ) and assume that there exist Er(z) = ,Ad(x), E(Y(x) - .A,4(z)) 2 = a~-(x). Let 0 denote the solution of the equation .A4(z) = O, which is assumed to be unique. D. Anbax [1] proposed the following modification of the Robbins-Monro procedure: x.+l = x. + ..(Y.(xn) - b.), (1) where X1 is a random variable, a,, > 0 and b,, _> 0 axe sequences of specially chosen real numbers, and P(rn(Xn) < y lX,,X~,... ,Xn) = P(Yn(X.) < u IX.) = H(y I X.). Anbax proved that Xn ~ 0 as n ~ co and that X,, exceeds 0 only finitelymany times with probability one. At present, A. V. Md'nikov has developed new methods based on semimaxtingale techniques. They make possible a general consideration of processes of stochastic approximation both with continuous and with discrete time. However, for the continuous analog of the Robbins-Mouro procedure no modification ensuring its one- sided convergence in one sense or another was suggested. By analogy with the process of Anbax (see [1]),we propose to consider the stochastic differential equation dX(t) = a(t) (:aCx(t)) - Kt)) dt + a(t)~(x(t)) dw(t), (2) where a(t) and b(t) axe some positive functions, w(t) is the Wiener random process, X(to) is a random variable, and t > to. w CONVERGENCE OF THE MODIFIED CONTINUOUS ROBBINS-MONRO PROCEDURE Theorem 2.1. Let the following conditions be satisAed: (1) the function b(t) is continuous for t > to; (2) .A4(x)(x - O) < 0 and I~(z)l + la(z)l __< g(1 + Izl) for ~ll x and some constant K; (3) for any positive N, there exists a KN such that I~(~) - ~(y)l + I~(~) - ~(y)l -< KNIz - yl for Ixl ~ g and lYl -< N; Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 228, 1996, pp. 201-208. Original article submitted April 20, 1996. 1072-3374/99/9303-0379522.00 Kluwer Academic/Plenum Publishers 379

One-sided convergence of continuous processes of stochastic approximation

Embed Size (px)

Citation preview

Journal ofMatheraatical Sciences, VoL 93, No. 3, 1999

O N E - S I D E D C O N V E R G E N C E O F C O N T I N U O U S P R O C E S S E S OF S T O C H A S T I C A P P R O X I M A T I O N

S. V. Komarov and T. P. Krasulina UDC 519.2

A modified continuous Robbins-Monro procedure is considered. Bibliography: 5 titles.

w INTRODUCTION

Let {Y(z), - c o < z < co} be a family of random variables. Let P ( Y ( z ) < y) = H(ylz ) and assume that there exist

E r ( z ) = ,Ad(x), E ( Y ( x ) - .A,4(z)) 2 = a~-(x).

Let 0 denote the solution of the equation .A4(z) = O, which is assumed to be unique. D. Anbax [1] proposed the following modification of the Robbins-Monro procedure:

x . + l = x . + . . ( Y . ( x n ) - b.) , (1)

where X1 is a random variable, a,, > 0 and b,, _> 0 axe sequences of specially chosen real numbers, and

P(rn(Xn) < y l X , , X ~ , . . . ,Xn) = P(Yn(X.) < u IX.) = H(y I X.) .

Anbax proved that Xn ~ 0 as n ~ co and that X,, exceeds 0 only finitely many times with probability one.

At present, A. V. Md'nikov has developed new methods based on semimaxtingale techniques. They make possible a general consideration of processes of stochastic approximation both with continuous and with discrete time.

However, for the continuous analog of the Robbins-Mouro procedure no modification ensuring its one- sided convergence in one sense or another was suggested.

By analogy with the process of Anbax (see [1]), we propose to consider the stochastic differential equation

dX(t) = a(t) (:aCx(t)) - Kt)) dt + a(t)~(x(t)) dw(t), (2)

where a(t) and b(t) axe some positive functions, w(t) is the Wiener random process, X(to) is a random variable, and t > to.

w CONVERGENCE OF THE MODIFIED CONTINUOUS ROBBINS-MONRO PROCEDURE

T h e o r e m 2.1. Let the following conditions be satisAed: (1) the function b(t) is continuous for t > to; (2) .A4(x)(x - O) < 0 and I ~ ( z ) l + la(z)l __< g ( 1 + Izl) for ~ll x a n d some constant K; (3) for any positive N , there exists a K N such that

I ~ ( ~ ) - ~ ( y ) l + I~(~) - ~(y)l -< KNIz - yl

for Ixl ~ g and lYl -< N;

Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 228, 1996, pp. 201-208. Original article submitted April 20, 1996.

1072-3374/99/9303-0379522.00 �9 Kluwer Academic/Plenum Publishers 379

(4) we have

o o o o

to to oO

a(t)b(t) dt < oo, a(t) >__ O, b(t) >__ O.

t o

Then for any random variable X(to) with E X (to) 2 < oo and independent o f w( t) - W(to), there exists a unique (up to stochastic equivalence in the strong sense, see [3]) solution X( t ) , to < t < c0, of Eq. (2) which is an a.s. continuous Markov process. Furthermore, X ( t ) converges to 0 as t ~ eo a.s. and

E X ( t ) ~ < e x p c ( t - t0 )Ex( t0 ) 2,

where c is a certain constant.

Proof. This directly follows from Theorem 3.8.2 of [31 with q(t ,z) = -b(t) , F ( z ) = .M(x), and a(t) = ~(t) = a(t).

w AUXILIARY RESULTS

Without loss of generality, we assume that 8 = 0 in what follows. To prove the one-sided convergence of the modified Robbins-Monro procedure, we need the following

lemmas.

L e m m a 3.1. Let X( t ) be a solution o f Eq. (2) and let the assumptions of Theorem 2.1 be fid61!ed. Furthermore, assume that

(a) there exist positive constants t(1 and Ks such that

Kllxl <_. I~(z)l ~<K~lxl form x;

(b) ~(t) = q, wh~e 2.4K1 > 1; (c) b2(t) <_ ~ for some c > 0 and all t > to > e, and EX2( to) < co. C t - - - -

Then EX2( t ) = o ( l ~ ) , t --* oo.

Proof. It is well known (see [3, Chap. 3, w that Eq. (2) determines a Markov random process X(t) with differential generator L acting on V(z) as follows:

LV(x) = a(t)(~(~) - b(t)) ~ + a~Ct)~2(x) d~V(~)

2 dz 2

For V(z) = z 2, we have

2.AKI(1 - - ~)x 2 + .Ab2(t) + t 2tK~ e

K(1 + z2)A 2 t2

for some positive ~. It follows from relation (3.5.5) of [3] that if e is such that

2AKI(1 - e ) > 1,

380

then for some 71 > 1 we have

Next, we have

d ~. Cl b2(t) EX( t ) 2 = EL[X(t) 2] <_ - EX( t ) 2 + ~ + c 2 - - ~

whence it follows that

Thus, we obtain

which proves the lernma.

L e m m a 3.2. Let 7 > 1/2.

cl c2 / ~ loglog s ds. EX(t)2 < T + ~ J,0 s~-~,

c' log log t c~ EX(t) 2< 1 7 +T'

Then, under the assumptions of Lemma 3.1,

t J / u - ~ X 2 ( s ) s ~-1 ds ~ 0 a.s. as t -~ co.

d to

where "Ln.t." implies

Proof. To prove Lemma 3.2, it is sufficient to check that

P ( sup tl/2-" f t X 2 ( s ) s T - l ds > ~ i.n.t.) = O, (3) ~2 ~ <t<2 TM Jto

means "in~nite re]tuber of times." The Chebyshev inequality, together with Lemma 3.1,

t / ft/m+t P sup t 1/2-7 f - - X~(s)s 7-1 ds > e < _C2m(1/2_7) log log_______~a ds ~2m<t_<2 m+l , / to -- ~" S2--7

{ cX2 -m12 log(m + I), 7 > I,

< c22-m/'(m + 1)log(m + 1), 7 = 1,

C32 -m(v-ll2) log(m + 1), 7 < I.

Applying the Borel-Cantelli lemma, we obtain (3), which completes the proof of Lemma 3.2.

L e m m a 3.3. Let 7 > 1/2 and let

I' ~(t) = t - , s , - ldw(s) ,

where w(a) is the Wiener random process. Then

limsup tll2y(t) < I t - -oo x/21og log t - ~ a . s .

t 37--I Proof. Consider the process ~(t) = f ~ dw(s). Obviously, this is a Caussiam process with variance 27t_ I

0

and zero mean. Similarly to the proof given in [4], it is easy to show that

L 3<, 2 ( 2 7 - l ) ] > 3 _< e - ~ .

381

Let h(t) = (2t loglogt) 1/9-. Fix numbers O, t, ~, a, and fl such that

i < 0 < o o , t = O n,

= (1 + 6 ) O - ( " - l ) " h ( O " - ' ) V f ~ - 1, D =

Then ( ~ = (1 + 6)(loglog 0 "-1) and e -~ = const-(n - 1) -1-6. Using the Borel-CanteUi lemma, we obtain

0 < 6 < 1 ,

o(n'l)('r " .1 )

2 v ~ - 1

P(m,_<atxt ( f o s u ' - I dw(tL) - 9-(27Ota2"-I ~ - - i) ,] >/~ i.n.t.) = O.

For sufficiently large n and 0 "-1 < t < 0 n, we have

I /o u "-~ dw(u) < max ~ ' -~ dw(u) < aO"O,-1) -._<o- - 2 ( 2 7 - 1)

Assuming that 0 $1 and/~ $ 0, we get

+ l~ < h(t)t'r ( ( i + r 1 ) - ~ 2 + "

t

f u,-1 dw(u) 1 l im sup o <

t - . oo h ( t ) t , - I - ~v~23"~ "

The proof is complete.

L e m m a 3.4. Let 7 > 1/2 and let the function or(z) satisfy the following two conditions: (a) =tim,~(=) = ,~o, (b) I~'(=) - ,r01 _< cl=l ~ for s o m e r E (0, 11. Then

t

t l /2 -" f "r 1 / s - (*(X(s)) - ~0) a~o(s) - , 0 to

Proof. As in Lemma 3.2, it is sufllcient to prove that

( I } P sup s ' - l ( ~ ( x ( 8 ) ) - ~ 0 ) d w C s ) > e i .n . t . = 0.

Using Theorem 1 from [5, w we get

P/ sup tl/2--r, [ t

A~ ( m - ' ) ~ f ' _< (loglogA1)e 2 s2" '2E(~(X(s)) - ao) 2 ds.

Now, consider

Using Lemma 3.1, we get

~ i s2"-2S(~(X(s)) - ~o) ~ as.

s 2 " - 2 E ( ~ ( X ( s ) ) - ~0) 2 ds ~ ~ 82"-2-~( log log 8) ~ ds.

382

Finally, we obtain

t112--7 t dw(s)l r

2k(1-27) f t j ~+t ~2(log k -t.- log log 2) (log log s) ~ s 2"~-2-~' as.

Using the Borel-CanteUi lemma as in the proof of Lemma 3.2, we verify the statement of Lemma 3.4.

Corol lary 3.1. Let the assumptions of Lemmas 3.3 and 3.4 be satis~ed. Then

llm sup sT_lq(X(s))dw(s ) < o'o t--.oo ~ / 2 1 o g log t - v f ~ - 1 a . s .

�9 w T H E MAIN RESULT

T h e o r e m 4.1. Assume that (1) the assumptions of Theorem 2.1 and Lemm~, 3.1 are satisfied; (2) the function ~(x) satisfies the assumptions of/.,emma 3.4; (3) ~ ( ~ ) = - ~ + 6(~), w h ~ e 6(~) = . 1 ~ + 61(~) and 6~(~) = O(~ ~) as ~ -~ 0 and 2 A . > 1;

0, t'or t < e, where D > (4) b(t) = Zn_~/2 /21og log t , eor t > ~,

Then we have (a) X ( O -~ o a.s. as t - ~ ~ ;

(b) P(ta: T(w) < oo) = 1, where T is a random moment such that X(t) < 0 for t >__ T.

Proof. Assertion (a) follows from the fact that the assumptions of Theorem 2.1 are satisfied and o o

f a(t)b(t)dt < oo. o

Further, using the It6 formula, we get

rr Jl ] x( t ) = t - ~ + ~ t -~ s ~ - b(s ) )ds + s " ~ - ' ( ~ ( x c s ) ) d ~ ( s ) . kJto

Using this relation, we obtain

tl/2X(t) t l /2-~AX(t) taA v t ~ U 2up -< llm2 P o

"~tl/2-a'A ft] + lim sup seM-16(X(s)) ds

"Atll~-a'a ft] q- llm sup saA-l(r(XCs)) dwCs)

-- llm ( f ' s .A-1 b(s)ds) AtlI2-"A

The first two sllmmands vanish by Lemmas 3.1 and 3.2. It remains to consider

.Atl/2-qA f t lira s ~a-a b(s) ds. t-'.O0 ~ Jto

The estimate

f,i + o0)), At -An s :'4-x b(s) ds = . . - - ti/2

383

Corollary 3.1, and condition (4) imply that

tl/2 lim sup X(t) < 0

t--cOO

a.s., which completes the proof of Theorem 4.1.

The authors are grateful to A. N. Borodin for his attention to this work.

Translated by A. V. Sudakov.

R E F E R E N C E S

1. D. A. Anbar, "A modified Robbins-Monro procedure approximating the zero of a regression function from below," Ann. Star., 5, 229-234 (1977).

2. A. V. Mel'nikov and A. E. Rodldna, "Consistent statistical estimation in semimartingale models of stochastic approximation," Ann. Aead.. Sci. Fenn., Set. A I Math., 17, 85-91 (1992).

3. M. Nevel'son and R. Khas'minskii, Stochastic Approzimation and Recurrent E~tiraation [in Russian], Naulm, Moscow (1972).

4. G. MeKean, Stochantic IntegraIa, Academic Press, New York-London (1969). 5. I. I. Gikhman and A. V. Skorokhod, Stochaatic Differential Equation, [in Russian], Kiev (1968).

384