E n v i r o n m e n t I n t e rna t i ona l Vol. l, pp. 341-343. Pergamon Press Ltd. 1978. Printed in Great Britain.

Branching Processes, Markov Chains, and Compet ing Risks*

Wolfgang J. B(Jhler Johannes Gutenberg-Universitat Mainz and University of California. Berkeley, California, U.S.A

This paper re-proves and generalizes results by Hardin and Sweet on the time to absorption in certain Markov chains by using a theorem on the time to extinction in multitype branching processes. Renaming "absorption" as "death" and "state immediately before absorption" as "cause of death" makes possible an interpretation in terms of competing risks. This is of very limited value, however, due to the rather severe mathematical restrictions on the Marko~ chains at hand.

1. Introduction

Weesakul (61) and Neuts (63) have studied the probabilities for the time to absorption in a random walk on the integers {0, 1 . . . . b } where 0 is an absorbing barrier and b a semireflecting barrier, and have obtained the corresponding probability generating function. Weesakul (61) also gives the probabilities explicitly. In their paper, Hardin (69) have succeeded in deriving explicit expressions for the probabilities in two specific cases corresponding to the model considered by Neuts (63). The original purpose of the present study was purely mathematical, namely, to use branching process methods for the study of absorption probabilities in models like those treated by Hardin (69), Neuts (63), and Weesakul (61).

It was Neyman (Statistical Laboratory, University of California, Berkeley) who brought to my attention the fact that similar models with more than one absorbing state might have some applications in the theory of competing risks. Here each absorbing state would represent death from a specified cause. I have chosen a slightly different approach identifying the cause of death with the state immediately preceding absorption. The mathematical assumptions needed in our setup unfortunately are so strong that at present we seem to still be several steps removed from applicability to real-life situations. There is, however, some hope that similar methods might become applicable in a more realistic setup.

* This investigation was supported in part by Stiftung Volkswagenwerk and in part by grant USPHS NIH RO 1 ES01299-13.

2. Absorption probabilities

Consider a discrete-time Markov chain X o , X x ,X 2 . . . . with states 0, 1 ... . . b such that 0 is an absorbing state ("death") and any state ("of health") i ~ 0 can be reached from any other state j ~= 0 with positive probability in a finite number of steps. If we then delete the row and column corresponding to the state 0 from the transition matrix P, the resulting matrix M has a power M N with all of its elements positive. Such a matrix M is called primitive.

Closely related to our Markov chain is the b-type Gal ton-Watson process (startingwith one particle of type i, say) in which a particle of type j(1 < j < b) has precisely one descendant of type k with probability Pjk(1 < 1 <_ b) and no descendants at all with probability P;0- The Galton Watson process population will consist of one particle of t ype j if the Markov chain is in s ta te j and be extinct if the Markov chain is absorbed.

The conditions imposed on the Markov chain make the corresponding b-type Ga l ton-Watson process subcritical and positively regular. Its basic probability-generating function f is defined by

h "i S . / ( ' t . . . . . sh) = y P u s i + P i o . (1)

j = l

If we put ~' = (1, s 1 . . . . . s~) and J" = (1,J 1 ..... jh) we can write Equation (1) in the form

i'(s) = P~,, (2) and M is the matrix of expectations for this branching

~ rocess. The tth iterate of f is obviously the restriction of (s) = P'.~ to its last b coordinates. Also, deleting the 0th

row and column from pt yields M'. The probabilities of absorption at or before time t are the probabilities of

341

342 Wolfgang J. Biihler

extinction of the Ga l ton Watson process by that t ime and can be obtained as

b

. / i ( 0 ) = f i ( [ ) ) = 1 - 32 Mtk i = 1,2 . . . . . b, (3) k = l

which makes b

q(t; i) = 32 [M',k -1 --M'ik I (4) k = l

for the probabi l i ty of absorp t ion at t ime t. Thus, the study of absorp t ion probabil i t ies boils down to that of the cor responding M. The first of the following results, of course, is not new, but its present p roof may be of some mathemat ica l interest.

Theorem 1: For any Markov chain on [0, 1 . . . . . h ~, with absorb ing state 0 and with a primitive matr ix M we have: (i) the tail of the distr ibution of t ime to absorp t ion is

geometr ic with pa r amte r p, the largest eigenvalue of M;

0it asymptot ical ly , as t ~ , the condi t ional distr ibution of the posit ion of the particle at t ime t, given that it has not been absorbed yet, is given by the left eigenvector p of M belonging to the eigenvalue p and normal ized such that Pl +P2 + . . . + p b = 1.

Both s ta tements hold no mat te r what the initial posit ion i was.

ProoJ': (i) is a s t ra ightforward consequence of Equat ion (4) together with a theorem of Perron and Frobenius (Harris, 63). For (ii) we apply Ji~'ina's mul t i type version of Yaglom's condi t ional limit theorem for subcritical G a l t o n - W a t s o n processes showing that, for the initial posit ion i, the probabi l i ty generat ing function gi of the limiting dis tr ibut ion satisfies

gi 'J'(s)] = pgi(s)+ 1 - p . (5)

Tak ing into account that the g/ as well as f must be linear, compar i son of coefficients of the s r proves (ii).

Theorem 2: If the Matr ix M corresponding to the Markov chain happens to be an oscillation matr ix (Gan tmacher , 59), there are numbers p = Pl >P2 > ... >Ph > 0 and real numbers aii (1 < i,j <_ b) such that

b

q(t;i) = E airp~-I (6) j=l

ProoJ. M" can be written in the form

M" = p'~Ml + p ~ M z + ... + p~,Mh (7)

where M~M r = M~ 6 u and where p~ > p z > ... > p h > 0 are the b distinct eigenvalues of M. Combin ing Equat ions (4) and (7) yields Equat ion (6) after an easy manipula t ion .

Corollary: If all Pu with i = j are multiples of a given number p, as in, for example, the r a n d o m walks considered in Hard in (69), then Equat ion (6) can be written in the form

b

q(t;i) = p Y cir(1--~jp) ' - I (8) j - - 1

where cq>O (1 < i < b ) , 0<c¢ 1 <0(2< ... <0%< 1/b.

Proo f It is clear that the matr ices M and I - M have the same eigenvectors and in fact that the same M r occur in the decompos i t ion (7); the new eigenvalues 1 - P i must be of the form ~ p since I - - M is p times a matr ix N which is independent ofp. The coefficients air in Equat ion (6) are 1 - p~ times the ith row sum in M r which does not depend on p.

Remark. The p roof of the corol lary shows that it suffices to determine the spectral propert ies of M or 1 - M for a single value ofp. In the cases considered by Hardin (69) p = 112 would make M part icularly simple.

3. The state before absorp t ion

As pointed out in the introduct ion, trying to apply our study to the p rob lem of compet ing risks means identifying the nonabsorb ing states with states of "heal th" and the "'cause of death" with the state of the Markov chain immediately before absorpt ion. Thus we have to study the probabil i t ies

q(t; i , j ) = P(X t = O,X,_ 1 = j l X o = i). (9)

These satisfy the system of equat ions b

q ( t + l ; i , j ) = Y~ Pikq( t ;k , j ) (10) k = l

with the initial condi t ions q(1;i , j) = 6uPro. Writing q(t;j) for the vector whose ith coord ina te is q(t; i,j), we rewrite Equat ion (10) as

q ( t + l ; j ) = Mq( t ; j ) with q(1; j ) = ejPjo, (11)

which incidentally depends on j only through the initial condition. I tera t ion leads to

q(t; j) = M ' - le i Pro (12)

which can be combined with Equat ion (7) to give the solution

b

q(t;i , j) = X p~ I(Mk)ij Pro, (13) k = l

it" M is an oscillation matrix. If the matr ix M is nondegenerate , not necessarily an

oscillation matrix, it is easy to sum Equat ion (12) to obtain the overall frequencies for the "causes of death", namely,

P(X,-1 =.J~(o = i) = ( l - - M ) - X e j P i o (14)

In the case of an oscillation matr ix this again specializes to b

P(Xt I = j ] X o = i ) = y~ (Mk)iyPjo/(l--pk) ' (15) k = l

as can be seen most easily by summing Equat ion (13) over t.

The asympto t ic s ta tement contained in Equat ion (13), that the tail of the distr ibution of the t ime of death is geometr ic with pa ramete r p l - - t h e largest eigenvalue of M remains true under the weaker assumpt ion of primitivity of M.

Clearly, primitivity of M is a very restrictive assumpt ion.

Branching processes, Markov chains, and competing risks 343

It m a y n o t , howeve r , be c r uc i a l to t he s t r u c t u r e of t he resul t s , a n d ef for ts a re b e i n g m a d e to r e l ax th i s c o n d i t i o n w h i c h e s sen t i a l l y imp l i e s t he p o s s i b i l i t y of m o v i n g f r o m o n e s t a t e of h e a l t h to a n y o t h e r p o s s i b l y t h r o u g h s o m e i n t e r m e d i a t e s ta tes . I t m a y be p o i n t e d o u t t h a t s u c h a n a s s u m p t i o n is a l so i m p l i e d in s i m p l e r v e r s i o n s of m o d e l s o f t he F i x - N e y m a n (Fix, 51) type .

References

Fix E. and Neyman J. (1951) A simple stochasitc model of recovery, relapse, death and loss of patients, Hum. Biol. 23, 205 241. Gantmacher F. R. (1959)Matrizenrechnung, VEB Deutscher Verlag der Wissenschaften, Berlin. Hardin J. C. and Sweet A. L. (1969) A note on absorption probabilities

for a random walk between a reflecting and an absorbing barrier, J. Appl. Prob. 6, 224 226. Harris T. E. (1963) The Theory qfBranching Processes, Springer, Berlin. Neuts M. F. (1963) Absorption probabilities for a random walk between a reflecting and an absorbing barrier, Bull. Soc. Math. Belg. 15, 253 258. Weesakul B. (1961) The random walk between a reflecting and an absorbing barrier, Ann. math. Statist. 32, 765-769.

Discussion

Neyman: Are some probabilities equal to zero? Bf~ler: No. The condition which makes things work is that some power of the transition matrix is different from zero. So from any state one can go to any other state within a finite time.

No probabilities of zero are allowed in the case which I have considered. But this is not a real obstacle here. One can generalize this assumption.