Upload
s-v-komarov
View
212
Download
0
Embed Size (px)
Citation preview
Journal ofMatheraatical Sciences, VoL 93, No. 3, 1999
O N E - S I D E D C O N V E R G E N C E O F C O N T I N U O U S P R O C E S S E S OF S T O C H A S T I C A P P R O X I M A T I O N
S. V. Komarov and T. P. Krasulina UDC 519.2
A modified continuous Robbins-Monro procedure is considered. Bibliography: 5 titles.
w INTRODUCTION
Let {Y(z), - c o < z < co} be a family of random variables. Let P ( Y ( z ) < y) = H(ylz ) and assume that there exist
E r ( z ) = ,Ad(x), E ( Y ( x ) - .A,4(z)) 2 = a~-(x).
Let 0 denote the solution of the equation .A4(z) = O, which is assumed to be unique. D. Anbax [1] proposed the following modification of the Robbins-Monro procedure:
x . + l = x . + . . ( Y . ( x n ) - b.) , (1)
where X1 is a random variable, a,, > 0 and b,, _> 0 axe sequences of specially chosen real numbers, and
P(rn(Xn) < y l X , , X ~ , . . . ,Xn) = P(Yn(X.) < u IX.) = H(y I X.) .
Anbax proved that Xn ~ 0 as n ~ co and that X,, exceeds 0 only finitely many times with probability one.
At present, A. V. Md'nikov has developed new methods based on semimaxtingale techniques. They make possible a general consideration of processes of stochastic approximation both with continuous and with discrete time.
However, for the continuous analog of the Robbins-Mouro procedure no modification ensuring its one- sided convergence in one sense or another was suggested.
By analogy with the process of Anbax (see [1]), we propose to consider the stochastic differential equation
dX(t) = a(t) (:aCx(t)) - Kt)) dt + a(t)~(x(t)) dw(t), (2)
where a(t) and b(t) axe some positive functions, w(t) is the Wiener random process, X(to) is a random variable, and t > to.
w CONVERGENCE OF THE MODIFIED CONTINUOUS ROBBINS-MONRO PROCEDURE
T h e o r e m 2.1. Let the following conditions be satisAed: (1) the function b(t) is continuous for t > to; (2) .A4(x)(x - O) < 0 and I ~ ( z ) l + la(z)l __< g ( 1 + Izl) for ~ll x a n d some constant K; (3) for any positive N , there exists a K N such that
I ~ ( ~ ) - ~ ( y ) l + I~(~) - ~(y)l -< KNIz - yl
for Ixl ~ g and lYl -< N;
Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 228, 1996, pp. 201-208. Original article submitted April 20, 1996.
1072-3374/99/9303-0379522.00 �9 Kluwer Academic/Plenum Publishers 379
(4) we have
o o o o
to to oO
a(t)b(t) dt < oo, a(t) >__ O, b(t) >__ O.
t o
Then for any random variable X(to) with E X (to) 2 < oo and independent o f w( t) - W(to), there exists a unique (up to stochastic equivalence in the strong sense, see [3]) solution X( t ) , to < t < c0, of Eq. (2) which is an a.s. continuous Markov process. Furthermore, X ( t ) converges to 0 as t ~ eo a.s. and
E X ( t ) ~ < e x p c ( t - t0 )Ex( t0 ) 2,
where c is a certain constant.
Proof. This directly follows from Theorem 3.8.2 of [31 with q(t ,z) = -b(t) , F ( z ) = .M(x), and a(t) = ~(t) = a(t).
w AUXILIARY RESULTS
Without loss of generality, we assume that 8 = 0 in what follows. To prove the one-sided convergence of the modified Robbins-Monro procedure, we need the following
lemmas.
L e m m a 3.1. Let X( t ) be a solution o f Eq. (2) and let the assumptions of Theorem 2.1 be fid61!ed. Furthermore, assume that
(a) there exist positive constants t(1 and Ks such that
Kllxl <_. I~(z)l ~<K~lxl form x;
(b) ~(t) = q, wh~e 2.4K1 > 1; (c) b2(t) <_ ~ for some c > 0 and all t > to > e, and EX2( to) < co. C t - - - -
Then EX2( t ) = o ( l ~ ) , t --* oo.
Proof. It is well known (see [3, Chap. 3, w that Eq. (2) determines a Markov random process X(t) with differential generator L acting on V(z) as follows:
LV(x) = a(t)(~(~) - b(t)) ~ + a~Ct)~2(x) d~V(~)
2 dz 2
For V(z) = z 2, we have
2.AKI(1 - - ~)x 2 + .Ab2(t) + t 2tK~ e
K(1 + z2)A 2 t2
for some positive ~. It follows from relation (3.5.5) of [3] that if e is such that
2AKI(1 - e ) > 1,
380
then for some 71 > 1 we have
Next, we have
d ~. Cl b2(t) EX( t ) 2 = EL[X(t) 2] <_ - EX( t ) 2 + ~ + c 2 - - ~
whence it follows that
Thus, we obtain
which proves the lernma.
L e m m a 3.2. Let 7 > 1/2.
cl c2 / ~ loglog s ds. EX(t)2 < T + ~ J,0 s~-~,
c' log log t c~ EX(t) 2< 1 7 +T'
Then, under the assumptions of Lemma 3.1,
t J / u - ~ X 2 ( s ) s ~-1 ds ~ 0 a.s. as t -~ co.
d to
where "Ln.t." implies
Proof. To prove Lemma 3.2, it is sufficient to check that
P ( sup tl/2-" f t X 2 ( s ) s T - l ds > ~ i.n.t.) = O, (3) ~2 ~ <t<2 TM Jto
means "in~nite re]tuber of times." The Chebyshev inequality, together with Lemma 3.1,
t / ft/m+t P sup t 1/2-7 f - - X~(s)s 7-1 ds > e < _C2m(1/2_7) log log_______~a ds ~2m<t_<2 m+l , / to -- ~" S2--7
{ cX2 -m12 log(m + I), 7 > I,
< c22-m/'(m + 1)log(m + 1), 7 = 1,
C32 -m(v-ll2) log(m + 1), 7 < I.
Applying the Borel-Cantelli lemma, we obtain (3), which completes the proof of Lemma 3.2.
L e m m a 3.3. Let 7 > 1/2 and let
I' ~(t) = t - , s , - ldw(s) ,
where w(a) is the Wiener random process. Then
limsup tll2y(t) < I t - -oo x/21og log t - ~ a . s .
t 37--I Proof. Consider the process ~(t) = f ~ dw(s). Obviously, this is a Caussiam process with variance 27t_ I
0
and zero mean. Similarly to the proof given in [4], it is easy to show that
L 3<, 2 ( 2 7 - l ) ] > 3 _< e - ~ .
381
Let h(t) = (2t loglogt) 1/9-. Fix numbers O, t, ~, a, and fl such that
i < 0 < o o , t = O n,
= (1 + 6 ) O - ( " - l ) " h ( O " - ' ) V f ~ - 1, D =
Then ( ~ = (1 + 6)(loglog 0 "-1) and e -~ = const-(n - 1) -1-6. Using the Borel-CanteUi lemma, we obtain
0 < 6 < 1 ,
o(n'l)('r " .1 )
2 v ~ - 1
P(m,_<atxt ( f o s u ' - I dw(tL) - 9-(27Ota2"-I ~ - - i) ,] >/~ i.n.t.) = O.
For sufficiently large n and 0 "-1 < t < 0 n, we have
I /o u "-~ dw(u) < max ~ ' -~ dw(u) < aO"O,-1) -._<o- - 2 ( 2 7 - 1)
Assuming that 0 $1 and/~ $ 0, we get
+ l~ < h(t)t'r ( ( i + r 1 ) - ~ 2 + "
t
f u,-1 dw(u) 1 l im sup o <
t - . oo h ( t ) t , - I - ~v~23"~ "
The proof is complete.
L e m m a 3.4. Let 7 > 1/2 and let the function or(z) satisfy the following two conditions: (a) =tim,~(=) = ,~o, (b) I~'(=) - ,r01 _< cl=l ~ for s o m e r E (0, 11. Then
t
t l /2 -" f "r 1 / s - (*(X(s)) - ~0) a~o(s) - , 0 to
Proof. As in Lemma 3.2, it is sufllcient to prove that
( I } P sup s ' - l ( ~ ( x ( 8 ) ) - ~ 0 ) d w C s ) > e i .n . t . = 0.
Using Theorem 1 from [5, w we get
P/ sup tl/2--r, [ t
A~ ( m - ' ) ~ f ' _< (loglogA1)e 2 s2" '2E(~(X(s)) - ao) 2 ds.
Now, consider
Using Lemma 3.1, we get
~ i s2"-2S(~(X(s)) - ~o) ~ as.
s 2 " - 2 E ( ~ ( X ( s ) ) - ~0) 2 ds ~ ~ 82"-2-~( log log 8) ~ ds.
382
Finally, we obtain
t112--7 t dw(s)l r
2k(1-27) f t j ~+t ~2(log k -t.- log log 2) (log log s) ~ s 2"~-2-~' as.
Using the Borel-CanteUi lemma as in the proof of Lemma 3.2, we verify the statement of Lemma 3.4.
Corol lary 3.1. Let the assumptions of Lemmas 3.3 and 3.4 be satis~ed. Then
llm sup sT_lq(X(s))dw(s ) < o'o t--.oo ~ / 2 1 o g log t - v f ~ - 1 a . s .
�9 w T H E MAIN RESULT
T h e o r e m 4.1. Assume that (1) the assumptions of Theorem 2.1 and Lemm~, 3.1 are satisfied; (2) the function ~(x) satisfies the assumptions of/.,emma 3.4; (3) ~ ( ~ ) = - ~ + 6(~), w h ~ e 6(~) = . 1 ~ + 61(~) and 6~(~) = O(~ ~) as ~ -~ 0 and 2 A . > 1;
0, t'or t < e, where D > (4) b(t) = Zn_~/2 /21og log t , eor t > ~,
Then we have (a) X ( O -~ o a.s. as t - ~ ~ ;
(b) P(ta: T(w) < oo) = 1, where T is a random moment such that X(t) < 0 for t >__ T.
Proof. Assertion (a) follows from the fact that the assumptions of Theorem 2.1 are satisfied and o o
f a(t)b(t)dt < oo. o
Further, using the It6 formula, we get
rr Jl ] x( t ) = t - ~ + ~ t -~ s ~ - b(s ) )ds + s " ~ - ' ( ~ ( x c s ) ) d ~ ( s ) . kJto
Using this relation, we obtain
tl/2X(t) t l /2-~AX(t) taA v t ~ U 2up -< llm2 P o
"~tl/2-a'A ft] + lim sup seM-16(X(s)) ds
"Atll~-a'a ft] q- llm sup saA-l(r(XCs)) dwCs)
-- llm ( f ' s .A-1 b(s)ds) AtlI2-"A
The first two sllmmands vanish by Lemmas 3.1 and 3.2. It remains to consider
.Atl/2-qA f t lira s ~a-a b(s) ds. t-'.O0 ~ Jto
The estimate
f,i + o0)), At -An s :'4-x b(s) ds = . . - - ti/2
383
Corollary 3.1, and condition (4) imply that
tl/2 lim sup X(t) < 0
t--cOO
a.s., which completes the proof of Theorem 4.1.
The authors are grateful to A. N. Borodin for his attention to this work.
Translated by A. V. Sudakov.
R E F E R E N C E S
1. D. A. Anbar, "A modified Robbins-Monro procedure approximating the zero of a regression function from below," Ann. Star., 5, 229-234 (1977).
2. A. V. Mel'nikov and A. E. Rodldna, "Consistent statistical estimation in semimartingale models of stochastic approximation," Ann. Aead.. Sci. Fenn., Set. A I Math., 17, 85-91 (1992).
3. M. Nevel'son and R. Khas'minskii, Stochastic Approzimation and Recurrent E~tiraation [in Russian], Naulm, Moscow (1972).
4. G. MeKean, Stochantic IntegraIa, Academic Press, New York-London (1969). 5. I. I. Gikhman and A. V. Skorokhod, Stochaatic Differential Equation, [in Russian], Kiev (1968).
384