Math Meth Oper Res (1999) 49 :325±333

999

Bounds for a class of stochastic recursive equations

Zhen Liu1, Philippe Nain1, Don Towsley2

1 INRIA, B.P. 93, F-06902, Sophia Antipolis Cedex, France (e-mail: liu@sophia.inria.fr;nain@sophia.inria.fr)2Department of Computer Science, University of Massachusetts, Amherst, MA 01003, USA(e-mail: towsley@cs.umass.edu)

Abstract. In this note we develop a framework for computing upper and lowerbounds of an exponential form for a class of stochastic recursive equationswith uniformly recurrent Markov modulated inputs. These bounds generalizeKingman's bounds for queues with renewal inputs.

Key words: Queues, exponential bounds, general state space Markov chain

1 Introduction

Let �W;F� be a measurable space large enough to carry a R�-valued randomvariable (r.v.) X0, a sequence of E-valued r.v.'s fYn; n � 0; 1; . . .g and a se-quence of R-valued r.v.'s fxn; n � 0; 1; . . .g. The set R (resp. R�) denotes theset of all real numbers (resp. the set of all nonnegative real numbers) endowedwith the s-algebra B (resp. B�). We will assume that E is a general spaceendowed with the s-algebra E.

Let fF �x; ��; x A Eg be a family of probability measures on�E�R;E�B� and let h be a probability measure on �R� �E;B� � E�. Letm�A�1 h�R� � A�, A A E, be a probability measure on �E;E�.

We postulate the existence of a probability measure P on �W;F�, withexpectation operator E, such that for all A A E, B A B, C A B�,

P��X0;Y0� A C � A� � h�C � A� �1:1�

and

Manuscript received: March 1998

P��Yn � 1; xn� A A� B jX0;Y0; . . . ;Yn; x0; . . . ; xnÿ1�� F �Yn; A� B� �1:2�

for all n � 0; 1; . . .. The de®nition (1.2) implies, in particular, that Y �fYn; n � 0; 1; . . .g is a time-homogeneous Markov chain with state-space E,transition kernel given by Q�x;A�1F �x; A�R� for all x A E, A A E, andinitial probability distribution m.

We will assume that the Markov chain Y is aperiodic, positive Harris re-current (see [21, Theorem 13.0.1]) and will denote by p its invariant probabil-ity measure. Let fpn; n � 0; 1; . . .g be a family of probability measures on Erecursively de®ned by p0 1 m and pn�A� �

�E Q�x;A�pnÿ1�dx� for A A E,

n � 1; 2; . . .. In other words, pn is the probability distribution of Yn given thatthe probability distribution of Y0 is m.

On �W;F� we de®ne the new sequence fXn; n � 0; 1; . . .g of R�-valuedr.v.'s by

Xn�1 � max�0;Xn � xn�; n � 0; 1; . . . : �1:3�In the queueing literature the recursion (1.3) is called the Lindley's equation.For instance, Xn may represent the waiting time of the n-th customer in a ®rst-in-®rst-out G/G/1 queue, where xn is the di¨erence between the service time ofthe n-th customer and the interarrival time between the n-th and the �n� 1�-stcustomer.

The aim of this note is to compute exponential upper and lower bounds forthe tail distribution of Xn, both for every n � 1; 2; . . . and for the stationaryregime X of Xn (when it exists), namely, to ®nd real numbers a; an V 0; b; bn;y > 0 such that

aneÿys UP�Xn > s�U bneÿys �1:4�

aeÿys UP�X > s�U beÿys �1:5�for all sV 0; n � 1; 2; . . ..

When E is a singleton ± which implies that fxn; n � 0; 1; . . .g is a renewalsequence with common p.d.f. denoted as H ± the ®rst upper bound forP�X > s� was obtained by Kingman [17] via martingale theory. Successivere®nements by Kingman [18] and Ross [22] have ®nally given the followingbounds (see also Borovkov [3, p. 139]):

infs>0

g�s; y��� �

eÿy �s UP�X > s�U sups>0

g�s; y�� �

eÿys; sV 0 �1:6�

for all 0U yU y� � supfa > 0 : E�eax0 �U 1g, where g�s; y� :� �1ÿH�s��=�ys

ey�uÿs�H�du�. These bounds hold under the stability condition E�x0� < 0.When E is a ®nite set, and if the stability condition Ep�x0� < 0 holds, then

Liu, Nain and Towsley [20] have shown that the upper bound in (1.5) holdsfor every 0U yU y� � supfa > 0; r�a�U 1g with

b � sup�s; j� AA

Pk AE

p�fkg�F�k; f jg � �s;y��Pk AE

l�k; y� �ys

ey�uÿs�F �k; f jg � du� �1:7�

326 Z. Liu et al.

where �l�k; y�; k A E� is the unique positive left eigenvector of the matrix��R eyuF �k; f jg � du��k; j associated with its largest eigenvalue r�y� such thatP

k AE l�k; y� � 1. In (1.7) the set A is de®ned as A � f�s; j� A R� �E :

F �k; f jg � �s;y�� > 0 for some k A Eg. The lower bound (1.5) is obtained for

y � y� and with the coe½cient a de®ned as the r.h.s. of (1.7) after substituting``sup'' for ``inf '' in this expression. Transient bounds of the form (1.4) are alsoreported in [20].

Still for ®nite state-space E, steady-state bounds of the form (1.5) havebeen obtained by Asmussen and Rolski [2] in the context of risk theory. It isnot easy to compare analytically bounds in [2] to those in [20]. Numerical ex-periments conducted in [20] indicate that, in general, bounds in [20] are betterthan those in [2].

Upper bounds of the form (1.5) have already been derived by Du½eld [9]for arbitrary set E. Du½eld's approach, based on the maximal inequality forpositive super-martingales, does not yield very tight bounds in general (see[20, Section 3.4]) and does not seem to easily yield a lower bound. A moregeneral setting than the one in the present note is studied by Chang and Chenin [6]. In [6] the increment process fxn; n � 0; 1; . . .g in (1.3) is in the formxn � an ÿ c for all n � 0; 1; . . . with c a nonnegative constant. Under the as-sumption that the process fan; n � 0; 1; . . .g satis®es a sample criterion, Changand Chen derive lower and upper bounds for P�X > s�; bounds for intreerouting networks are also reported in [6]. As acknowledged by the authors,their bounds are not as tight as the bounds in [2] and in [20] when specializedto the case when fan; n � 0; 1; . . .g is a Markov arrival process.

The results in this note generalize the bounds in (1.6) and the bounds in[20] to the case when �E;E� is a general measurable space. Upper and lowerbounds for P�Xn > s� and P�X > s� are obtained through a uni®ed and simpleapproach.

Besides the theoretical interest of obtaining bounds like those in (1.4)±(1.5), bounds on the tail distribution of quantities such as bu¨er occupancyand response times can be used in the design of high speed networks. In ad-dition, bounds can also be used to develop policies for controlling the admis-sion of new applications or sessions to the network. The interested reader isreferred to [1, 5, 7, 8, 10, 11, 15, 16, 19, 20, 23] where these issues have beenlately addressed.

We conclude this section by introducing some further notation and bystating some preliminary results.

A ®rst step toward the generalization to general state-space E for theMarkov chain Y is to note that the process f�Yn�1;

Pni�0 xi�; n � 0; 1; . . .g is

a discrete-time Markov-Additive process [14] with MA-kernel given byF �x; A� B�. This observation will allow us to borrow several results from[14].

To this end, de®ne the transform

Q̂�x;A; y� ��yÿy

eyuF�x; A� du� �1:8�

for all x A E;A A E; y A R.Let F �1��x; A� B� � F�x; A� B� and F �i��x; A� B� � �E �yÿy F �iÿ1� �

Bounds for a class of stochastic recursive equations 327

�x; dy� du�F �y; A� �Bÿ u��; i V 2, and de®ne Q̂�i��x;A; y� � �yÿy eyuF �i��x; A� du� for all iV 1.

From now on we will assume that there exist a probability measure m on�E�R;E�B�, an integer i, and real numbers 0 < a1 U a2 <y such that

a1m�A� B�UF �i��x; A� B�U a2m�A� B�;Ex A E;A A E;B A B: �1:9�

The above condition is the ``recurrence hypothesis'' (3.1) in [14]. Condition(1.9) holds automatically when E is ®nite and when the Markov chain Y isirreducible and aperiodic. The interested reader is referred to [14, Section 7]for a discussion on cases where condition (1.9) holds.

De®ne m̂�A; y� � �R eyum�A� du� for all A A E, and let D � fy A R :m̂�E; y� <yg.

By applying Lemma 3.1 in [14] we deduce that for each y A D; Q̂��; �; y� hasa maximal simple eigenvalue r�y� > 0 with uniformly positive and boundedassociated left eigenmeasure (see [13, Theorem III.10.1]) l�y� � fl�A; y�;A A Eg. Recall that the left eigenmeasure l�y� satisfy the relationshipr�y�l�A; y� � �E Q̂�x;A; y�l�dx; y�, for all A A E. We will assume without lossof generality that the left eigenmeasure is chosen so that

l�E; y� � 1; Ey A D: �1:10�

2 Exponential bounds

The approach used in this paper generalized Kingman's in [18].Let fgn�s; ��; sV 0; n � 0; 1; . . .g, be a collection of measures on �E;E� such

that�E

� s

ÿygn�sÿ u; dx�F�x; A� du� � F�x; A� �s;y��pn�dx�U gn�1�s;A�

�2:1�

for all sV 0;A A E; n � 0; 1; . . ..The following technical lemma holds:

Lemma 2.1. Let Pn denote the property that

P�Xn > s;Yn A A�U gn�s;A� �2:2�

for all sV 0;A A E; n � 0; 1; . . ..If P0 is true, then Pn is true for all nV 0.

Proof. We will use an induction argument on n. Assume that Pm is true form � 0; 1; . . . ; n and let us show that Pn�1 is true.

We have for all sV 0;A A E,

328 Z. Liu et al.

P�Xn�1 > s;Yn�1 A A�

��

E

�yÿy

P�Xn > sÿ u;Yn�1 A A; xn A du jYn � x�pn�dx�

��

E

� s

ÿyP�Xn > sÿ u;Yn�1 A A; xn A du jYn � x�pn�dx�

��

E

�ys

P�Yn�1 A A; xn A du jYn � x�pn�dx�

��

E

� s

ÿyP�Yn�1 A A; xn A du jXn > sÿ u;Yn � x�

� P�Xn > sÿ u;Yn A dx�

��

EF �x; A� �s;y��pn�dx�

��

E

" � s

ÿyF�x; A� du�P�Xn > sÿ u;Yn A dx�

� F�x; A� �s;y��pn�dx�#

�2:3�

U�

E

� s

ÿygn�sÿ u; dx�F�x; A� du� � F �x; A� �s;y��pn�dx�

� ��2:4�

U gn�1�s;A� �2:5�

where (2.3), (2.4) and (2.5) follow from (1.2), the induction hypothesis and thede®nition (2.1), respectively. 9

Introduce the set G � faV 0 : r�a�U 1gXD. Observe that G is nonemptysince r�0� � 1 and m̂�E; 0� � 1 which implies that 0 A G.

We are now in position to prove the main result of this paper.

Proposition 2.1. Let y A G. For n � 0; 1; . . . ; de®ne

bn�y� � sup�s;A� AK0UmUn

�E F �x; A� �s;y��pm�dx��

E

�ys

ey�uÿs�F�x;A� du�l�dx; y� <y �2:6�

with K � f�s;A� A R� � E : F�x; A� �s;y�� > 0 for some x A Eg. If

h��s;y� � A�U b0�y�l�A; y�eÿys; EsV 0;A A E �2:7�

then

P�Xn > s;Yn A A�U b0�y�l�A; y�eÿys; EsV 0;A A E; n � 0; 1; . . . : �2:8�

Bounds for a class of stochastic recursive equations 329

In particular,

P�Xn > s�U bn�y�eÿys; EsV 0; n � 0; 1; . . . : �2:9�

Proof. Fix y A G. De®ne gn�s;A� � bn�y�eÿysl�A; y� for all sV 0;A A E;n � 0; 1; . . ..

Thanks to Lemma 2.1, the derivation of (2.8) will follow if we can showthat the mappings gn�s;A� de®ned above satisfy (2.1).

We have�E

� s

ÿygn�sÿ u; dx�F �x; A� du� � F �x; A� �s;y��pn�dx�

� bn�y�eÿys

�E

�yÿy

eyuF�x; A� du�l�dx; y� ÿ bn�y�

��

E

�ys

ey�uÿs�F�x; A� du�l�dx; y� ��

EF�x; A� �s;y��pn�dx�

U bn�y�eÿys

�E

�yÿy

eyuF�x; A� du�l�dx; y�;

from the definition of bn�y�

� bn�y�eÿys

�E

Q̂�x;A; y�l�dx; y�

� bn�y�eÿysr�y�l�A; y�U gn�1�y�

where the last inequality follows from r�y�U 1 and bn�y�U bn�1�y�. Thisproves (2.8).

For the proof of (2.9) simply observe that

P�Xn > s� � P�Xn > s;Yn A E�

U bn�y�eÿysl�E; y� from �2:8�

� bn�y�eÿys; EsV 0; n � 0; 1; . . .

by using the normalizing condition (1.10).We conclude this proof by showing that the constants b0�y�; b1�y�; . . . are

all ®nite for all y A G. This result follows from the inequalities

bn�y�U sup�s;A� AK0UmUn

�E F �x; A� �s;y��pm�dx��E F�x; A� �s;y��l�dx; y� U sup

0UmUnA AE

pm�A�l�A; y� <y �2:10�

where the last inequality follows from the positiveness of the left eigenmeasurel�y� (see Section 1). 9

330 Z. Liu et al.

From now on we will assume that there exists 0 < B <y such thatm�E; �B;y�� > 0. If this assumption does not hold then it can be shown from(1.9) and (1.3) that Xn !n 0 almost surely and the system becomes trivial.

De®ne y� � supfy A Gg. It is shown in [9] that when the set D is open theny� > 0 if the stability condition (see [4]) Ep�x0� < 0 is satis®ed, where Ep standsfor the expectation operator associated with a stationary Markov chain Y (i.e.m � p). In that case, y� is the largest exponential decay rate among all positivedecay rates such that r�y�U 1. Actually, y� is seen to be the largest exponen-tial decay rate among all y > 0 whenever (i) the set D is open, (ii) condition(1.9) holds and (iii) Ep�x0� < 0. This result is a direct consequence of theidentity limx!y�1=x� logP�X > x� � ÿy� that holds under the aforemen-tioned conditions (i)±(iii) (see Du½eld [9], Glynn and Whitt [12, Theorem 1]).

We now establish the transient lower bound.

Proposition 2.2. For n � 0; 1; . . ., de®ne

an � inf�s;A� AK0UmUn

�E F�x; A� �s;y��pm�dx��

E

�ys

ey ��uÿs�F�x; A� du�l�dx; y�� : �2:11�

Assume that Ep�x0� < 0;D is an open set and r�y�� � 1 (see Remark 2.1 below).If

h��s;y� � A�V a0l�A; y�eÿy �s; EsV 0;A A E �2:12�then

P�Xn > s;Yn A A�V anl�A; y�eÿy �s; EsV 0;A A E; n � 0; 1; . . . : �2:13�In particular,

P�Xn > s�V aneÿy �s; EsV 0; n � 0; 1; . . . : �2:14�

Proof. Let fdn�s; ��; sV 0; n � 0; 1; . . .g be a collection of measures on �E;E�such that�

E

� s

ÿydn�sÿ u; dx�F�x; A� du� � F�x; A� �s;y��pn�dx�V dn�1�s;A�

�2:15�

for all sV 0;A A E; n � 0; 1; . . . : As in Lemma 2.1 we can easily show that ifthe property

P�Xn > s;Yn A A�V dn�s;A�; Es > 0;A A E �2:16�

holds for n � 0 then it holds for all n � 0; 1; . . . :De®ne dn�s;A� � aneÿy �sl�A; y�� for all sV 0;A A E; n � 0; 1; . . . :If the mappings fdn�s;A�; sV 0;A A E; n � 0; 1; . . .g satisfy (2.15) then,

according to (2.16),

P�Xn > s;Yn A A�V aneÿy �sl�A; y��

Bounds for a class of stochastic recursive equations 331

for all s > 0;A A E; n � 0; 1; . . . ; which in turn implies with (1.10) that

P�Xn > s� � P�Xn > s;Yn A E�V aneÿy �s:

It remains to check that fdn�s;A�; sV 0;A A E; n � 0; 1; . . .g satisfy (2.15).This can be done by mimicking the proof of Proposition 2.1 and by using theassumption r�y�� � 1. The proof is omitted. 9

Remark 2.1. The equation r�y� � 1 has always one and only one solutiony � y� in DX �0;y� when the set D is open. This follows from the strictconvexity of r�y� on D (which itself is a consequence of the strict convexity oflog r�y� [14, Lemma 3.4(i)]), of limy!qDr�y� �y [14, Corollary 3.1], ofr�0� � 1, and of r 0�0� � Ep�x0� < 0.

We now turn to the derivation of steady-state bounds. As already men-tioned, we know that there exists a proper r.v. X such that P�Xn U s� !n

P�X U s� for all sV 0 independently of the initial distribution h of �X0;Y0�whenever Ep�x0� < 0 [9]. This result, combined with Propositions 2.1±2.2,yields the following:

Corollary 2.1. Assume that Ep�x0� < 0. Then, Ey A G,

P�X > s�U b�y�eÿys; EsV 0 �2:17�

where

b�y� � sup�s;A� AK

�E

F�x; A� �s;y��p�dx��E

�ys

ey�uÿs�F�x; A� du�l�dx; y� <y: �2:18�

If we further assume that D is an open set, then

P�X > s�V aeÿy �s; EsV 0 �2:19�

where

a � inf�s;A� AK

�E

F�x; A� �s;y��p�dx��E

�ys

ey ��uÿs�F�x; A� du�l�dx; y�� : �2:20�

As already noticed by Kingman [17] it is simple to construct examples wherethe coe½cient a in the lower bound is equal to 0. Instances where a > 0 arediscussed in [20].

References

[1] Artiges D, Nain P (1996) Upper and lower bounds for the multiplexing of multiclass Mar-kovian on/o¨ sources. Perf. Evaluation 27&28:673±698

[2] Asmussen S, Rolski T (1994) Risk theory in a periodic environment: the Cramer-Lundbergapproximation and Lundberg's inequality. Mathematics of Operations Research 2:410±433

332 Z. Liu et al.

[3] Borovkov AA (1976) Stochastic processes in queueing theory. Springer-Verlag, New York[4] Borovkov AA, Foss SG (1992) Stochastically recursive sequences and their generalizations.

Siberian Advances in Mathematics 2:16±81[5] Chang C-S (1994) Stability, queue length and delay of deterministic and stochastic queueing

networks. IEEE Trans. Aut. Contr. 39:913±931[6] Chang C-S, Cheng J (1995) Computable exponential bounds for intree networks with rout-

ing. Proc. of the Infocom'96 Conf., Boston, MA, Vol. 1, pp. 197±204[7] Cruz RL (1991) A calculus for network delay, part I: Network elements in isolation. IEEE

Trans. Inf. Theory 37:114±131[8] Cruz RL (1991) A calculus for network delay, part II: Network analysis. IEEE Trans. Inf.

Theory 37:132±141[9] Du½eld NG (1994) Exponential bounds for queues with Markovian arrivals. QUESTA

17:413±430[10] Elwalid AI, Mitra D (1993) E¨ective bandwidth of general Markovian tra½c sources and

admission control of high speed networks. IEEE/ACM Trans. on Networking 1:329±343[11] Gibbens RJ, Hunt PJ (1991) E¨ective bandwidths for the multi-type UAS channel. QUESTA

9:17±28[12] Glynn PW, Whitt W (1994) Logarithmic asymptotics or steady-state tail probabilities in a

single-server queue. In: Galambos J, Gani J (eds.) Studies in Appl. Prob., pp. 131±156[13] Harris TE (1989) The theory of branching processes. Dover Publications Inc., New York[14] Iscoe I, Ney P, Nummelin E (1985) Large deviations of uniformly recurrent Markov additive

processes. Adv. in Appl. Math. 6:373±412[15] Kelly FP (1991) E¨ective bandwidths at multi-class queues. QUESTA 9:5±16[16] Kesidis G, Walrand J, Chang C-S (1993) E¨ective bandwidths for multiclass Markov ¯uids

and other ATM sources. IEEE/ACM Trans. Networking 1:424±428[17] Kingman JFC (1964) A martingales inequality in the theory of queues. Proc. Camb. Phil.

Soc., 59:359±361[18] Kingman JFC (1970) Inequalities in the theory of queues. J. Roy. Stat. Soc., Series B 32:102±

110[19] Kurose JF (1992) On computing per-session performance bounds in high-speed multi-hop

computer networks. Proc. of the joint ACM Sigmetrics and Performance '92 Conf., Newport,RI, pp. 128±139

[20] Liu Z, Nain P, Towsley D (1997) Exponential bounds with applications to call admission.Journal of the ACM 44:366±394

[21] Meyn SP, Tweedie RL (1993) Markov chains and stochastic stability. Springer Verlag, Lon-don

[22] Ross SM (1974) Bounds on the delay distribution in GI/GI/1 Queues,'' J. Appl. Prob.11:417±421

[23] Yaron O, Sidi M (1993) Performance and stability of communication networks via robustexponential bounds. IEEE/ACM Trans. Networking 1:372±385

Bounds for a class of stochastic recursive equations 333