Hindawi Publishing CorporationJournal of Probability and StatisticsVolume 2013 Article ID 424601 5 pageshttpdxdoiorg1011552013424601
Research ArticleOn the Study of Transience and Recurrence of the MarkovChain Defined by Directed Weighted Circuits Associated witha Random Walk in Fixed Environment
Chrysoula Ganatsiou
Department of Civil Engineering Section of Mathematics and Statistics School of Technological SciencesUniversity of Thessaly Pedion Areos 38334 Volos Greece
Correspondence should be addressed to Chrysoula Ganatsiou ganatsiootenetgr
Received 23 October 2012 Revised 2 March 2013 Accepted 27 March 2013
Academic Editor Man Lai Tang
Copyright copy 2013 Chrysoula GanatsiouThis is an open access article distributed under theCreativeCommonsAttributionLicensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
By using the cycle representation theory of Markov processes we investigate proper criterions regarding transience and recurrenceof the correspondingMarkov chain represented uniquely by directed cycles (especially by directed circuits) andweights of a randomwalk with jumps in a fixed environment
1 Introduction
A systematic research has been developed (Kalpazidou [1]MacQueen [2] Minping and Min [3] Zemanian [4] andothers) in order to investigate representations of the finite-dimensional distributions ofMarkov processes (with discreteor continuous parameter) having an invariant measure asdecompositions in terms of the cycle (or circuit) passagefunctions
119869119888
(119894 119895) = 1 if 119894 119895 are consecutive states of 119888
= 0 otherwise(1)
for any directed sequence 119888 = (1198941
1198942
119894119903
) (or 119888 =
(1198941
1198942
119894119903
1198941
)) of states called a cycle (or a circuit) 119903 gt 1of the corresponding Markov process The representationsare called cycle (or circuit) representations while the corre-sponding discrete parameter Markov processes generated bydirected circuits 119888 = (119894
1
1198942
119894119903
1198941
) 119903 gt 1 are called circuitchains
Following the context of the theory of Markov processesrsquocycle-circuit representation the present work arises as anattempt to investigate proper criterions regarding the prop-erties of transience and recurrence of the correspondingMarkov chain represented uniquely by directed cycles (espe-cially by directed circuits) and weights of a random walk
with jumps (having one elastic left barrier) in a fixed ergodicenvironment (Kalpazidou [1] Derriennic [5])
The paper is organized as follows In Section 2 we givea brief account of certain concepts of cycle representationtheory of Markov processes that we will need throughout thepaper In Section 3 we present some auxiliary results in orderto make the presentation of the paper more comprehensibleIn particular in Section 3 a randomwalk with jumps (havingone elastic left barrier) in a fixed ergodic environment isconsidered and the unique representations by directed cycles(especially by directed circuits) and weights of the corre-sponding Markov chain are investigated These representa-tions will give us the possibility to study proper criterionsregarding transience and recurrence of the abovementionedMarkov chain as it is described in Section 4
Throughout the paper we will need the following nota-tions
alefsym = 0 1 2 alefsymlowast
= 1 2
119885 = minus1 0 1 (2)
2 Preliminaries
Let us consider a denumerable set S Then the directedsequence 119888 = (119894
1
1198942
119894119903
1198941
) modulo the cyclic permuta-tions where 119894
1
1198942
119894119903
isin 119878 119903 gt 1 completely defines a
2 Journal of Probability and Statistics
directed circuit in 119878 The ordered sequence 119888 = (1198941
1198942
119894119903
)
associated with the given directed 119888 is called a directed cyclein 119878
A directed circuit may be considered as 119888 = (119888(119903) 119888(119903 +
1) 119888(119903 + V minus 1) 119888(119903 + V)) if there exists an 119903 isin 119885 such that1198941
= 119888(119903 + 0) 1198942
= 119888(119903 + 1) 119894V = 119888(119903 + V minus 1) 1198941
= 119888(119903 + V)where 119888 is a periodic function from 119885 to 119878
The corresponding directed cycle is defined by theordered sequence 119888 = (119888(119903) 119888(119903+1) 119888(119903+Vminus1))The values119888(119896) are the points of 119888 while the directed pairs (119888(119896) 119888(119896+1))119896 isin 119885 are the directed edges of 119888
The smallest integer 119901 equiv 119901(119888) ge 1 satisfying the equation119888(119903 + 119901) = 119888(119903) for all 119903 isin 119885 is the period of 119888 Adirected circuit 119888 such that 119901(119888) = 1 is called a loop (In thepresent work we will use directed circuits with distinct pointelements)
Let us also consider a directed circuit 119888 (or a directed cycle119888) with period 119901(119888) gt 1 Then we may define by
119869(119899)
119888
(119894 119895) = 1 if there exists an 119903 isin 119885 such that
119894 = 119888 (119903) 119895 = 119888 (119903 + 119899)
= 0 otherwise
(3)
the 119899-step passage function associatedwith the directed circuit119888 for any 119894 119895 isin 119878 119899 ge 1
Furthermore we may define by
119869119888
(119894) = 1 if there exists an 119903 isin 119885 such that 119894 = 119888 (119903)
= 0 otherwise(4)
the passage function associated with the directed circuit 119888 forany 119894 isin 119878 The above definitions are due toMacQueen [2] andKalpazidou [1]
Given a denumerable set 119878 and an infinite denumerableclass 119862 of overlapping directed circuits (or directed cycles)with distinct points (except for the terminals) in 119878 such thatall the points of 119878 can be reached from one another followingpaths of circuit edges that is for each two distinct points 119894
and 119895 of 119878 there exists a finite sequence 1198881
119888119896
119896 ge 1 ofcircuits (or cycles) of 119862 such that 119894 lies on 119888
1
and 119895 lies on 119888119896
and any pair of consecutive circuits (119888
119899
119888119899+1
) have at least onepoint in common Generally we may assume that the class 119862contains among its elements circuits (or cycles) with periodgreater or equal to 2
With each directed circuit (or directed cycle) 119888 isin 119862
let us associate a strictly positive weight 119908119888
which must beindependent of the choice of the representative of 119888 that isit must satisfy the consistency condition 119908cot119896 = 119908
119888
119896 isin 119885where 119905
119896
is the translation of length 119896 (that is 119905119896
(119899) equiv 119899 + 119896119899 isin 119885 for any fixed 119896 isin 119885)
For a given class 119862 of overlapping directed circuits (orcycles) and for a given sequence (119908
119888
)119888isin119862
of weights we maydefine by
119901119894119895
=sum119888isin119862
119908119888
sdot 119869(1)
119888
(119894 119895)
sum119888isin119862
119908119888
sdot 119869119888
(119894)(5)
the elements of a Markov transition matrix on 119878 if and onlyif sum119888isin119862
119908119888
sdot 119869119888
(119894) lt infin for any 119894 isin 119878 This means that agiven Markov transition matrix 119875 = (119901
119894119895
) 119894 119895 isin 119878 can berepresented by directed circuits (or cycles) and weights if andonly if there exists a class of overlapping directed circuits (orcycles) 119862 and a sequence of positive weights (119908
119888
)119888isin119862
suchthat the formula (5) holds In this case the Markov transitionmatrix 119875 has a unique stationary distribution 119901 which is asolution of 119901119875 = 119901 and is defined by
119901 (119894) = sum
119888isin119862
119908119888
sdot 119869119888
(119894) 119894 isin 119878 (6)
It is known that the following classes of Markov chainsmay be represented uniquely by circuits (or cycles) andweights
(i) the recurrent Markov chains (Minping and Min [3])(ii) the reversible Markov chains
3 Auxiliary Results
Let us consider aMarkov chain (119883119899
)119899ge0
onalefsymwith transitions119896 rarr (119896+1) 119896 rarr (119896minus1) and 119896 rarr 119896whose the elements ofthe corresponding Markov transition matrix are defined by
119875 (119883119899+1
= 0119883119899
= 0) = 1199030
119875 (119883119899+1
= 1119883119899
= 0) = 1199010
1199010
= 1 minus 1199030
119875 (119883119899+1
= 119896 + 1119883119899
= 119896) = 119901119896
119896 ge 1
119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
119896 ge 1
119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
119896 ge 1
(7)
such that 119901119896
+ 119902119896
+ 119903119896
= 1 0 le 119901119896
119903119896
le 1 for every 119896 ge 1 asit is shown in (Figure 1)
Assume that (119901119896
)119896ge0
and (119903119896
)119896ge0
are arbitrary fixedsequences with 0 le 119901
0
= 1 minus 1199030
le 1 0 le 119901119896
119903119896
le 1 for every119896 gt 1 If we consider the directed circuits 119888
119896
= (119896 119896 + 1 119896)1198881015840
119896
= (119896 119896)119896 ge 0 and the collections of weights (119908119888119896)119896ge0
and(119908119888
1015840
119896
)119896ge0
respectively then we may obtain that
119901119896
=119908119888119896
119908119888119896minus1
+ 119908119888119896+ 119908119888
1015840
119896
for every 119896 ge 1 (8)
with
1199010
=1199081198880
1199081198880+ 119908119888
1015840
0
(9)
Here the class 119862(119896) contains the directed circuits 119888119896
=
(119896 119896+1 119896) 119888119896minus1
= (119896minus1 119896 119896minus1) and 1198881015840
119896
= (119896 119896) Furthermorewe may define
119902119896
=119908119888119896minus1
119908119888119896minus1
+ 119908119888119896+ 119908119888
1015840
119896
for every 119896 ge 1 (10)
and 119903119896
= 119908119888
1015840
119896
(119908119888119896minus1
+119908119888119896+119908119888
1015840
119896
) such that 119901119896
+119902119896
+ 119903119896
= 1 forevery 119896 ge 1 with 119903
0
= 1 minus 1199010
= 119908119888
1015840
0
(1199081198880+ 119908119888
1015840
0
)
Journal of Probability and Statistics 3
1199030
1199010
11990211199031
1199011
11990221199032
1199012
11990231199033
middot middot middot 985747 minus 1
119903985747minus1
119901985747minus1
119902985747
985747
119903985747
middot middot middot0 1 2 3
Figure 1
The transition matrix 119875 = (119901119894119895
) with
119901119894119895
=suminfin
119896=0
119908119888119896sdot 119869(1)
119888119896
(119894 119895)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] for 119894 = 119895 (11)
119901119894119894
=
suminfin
119896=0
1199081198881015840119896
sdot 119869(1)
119888
1015840
119896
(119894 119894)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] (12)
where 119869(1)
119888119896
(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896
119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888
119896
and 119869119888
1015840
119896
(119894) = 1if 119894 is a point of the circuit 119888
1015840
119896
expresses the representationof the Markov chain (119883
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840
119899
)119899ge0
on alefsym whose elements of the corresponding Markovtransition matrix are defined by
119875 (1198831015840
119899+1
= 01198831015840
119899
= 0) = 1199031015840
0
119875 (1198831015840
119899+1
= 11198831015840
119899
= 0) = 1199021015840
0
1199021015840
0
= 1 minus 1199031015840
0
119875 (1198831015840
119899+1
= 119896 minus 11198831015840
119899
= 119896) = 1199011015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896) = 1199031015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896) = 1199021015840
119896
119896 ge 1
(13)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 0 lt 1199011015840
119896
1199021015840
119896
1199031015840
119896
le 1 for every 119896 ge 1as it is shown in (Figure 2)
Assume that (1199021015840
119896
)119896ge0
and (1199031015840
119896
)119896ge0
are arbitrary fixedsequences with 0 le 119902
1015840
0
= 1 minus 1199031015840
0
le 1 0 le 1199021015840
119896
1199031015840119896
le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840
119896
= (119896+1 119896 119896+1)119888101584010158401015840
119896
= (119896 119896) 119896 ge 0 and the collections of weights (119908119888
10158401015840
119896
)119896ge0
and (119908
119888
101584010158401015840
119896
)119896ge0
respectively then we may have
1199021015840
119896
=119908119888
10158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1 (14)
with
1199021015840
0
=119908119888
10158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(15)
Here the class 1198621015840
(119896) contains the directed circuits 11988810158401015840
119896
=
(119896 + 1 119896 119896 + 1) 11988810158401015840
119896minus1
= (119896 119896 minus 1 119896) and 119888101584010158401015840
119896
= (119896 119896)Furthermore we may define
1199011015840
119896
=119908119888
10158401015840
119896minus1
11990810158401015840119888119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1
1199031015840
119896
=119908119888
101584010158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
(16)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 for every 119896 ge 1 with
1199031015840
0
= 1 minus 1199021015840
0
=119908119888
101584010158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(17)
So the transition matrix 1198751015840
= (1199011015840
119894119895
) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883
1015840
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Consequently we have the following
Proposition 1 The Markov chain (119883119899
)119899ge0
defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights
Proof Let us consider the set of directed circuits 119888119896
= (119896 119896 +
1 119896) and 1198881015840
119896
= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888
119896minus1
119888119896
and 1198881015840
119896
andtwo circuits through 0 119888
0
11988810158400
The problem we have to manage is the definition of the
weightsWemay symbolize by119908119896
the weight119908119888119896of the circuit
c119896
and by1199081015840
119896
the weight119908119888
1015840
119896
of the circuit 1198881015840119896
for any 119896 ge 0Thesequences (119908
119896
)119896ge0
(1199081015840119896
)119896ge0
must be a solution of
119901119896
=119908119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199010
=1199080
1199080
+ 11990810158400
119903119896
=1199081015840
119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199030
=1199081015840
0
1199080
+ 11990810158400
119902119896
= 1 minus 119901119896
minus 119903119896
119896 ge 1
(18)
Let us take by 119887119896
= 119908119896
119908119896minus1
120574119896
= 1199081015840
119896
1199081015840
119896minus1
119896 ge 1As a consequence we may have
119887119896
=119901119896
119902119896
=119901119896
1 minus 119901119896
minus 119903119896
120574119896
=119903119896
119903119896minus1
119901119896minus1
119901119896
119887119896
for every 119896 ge 1
(19)
4 Journal of Probability and Statistics
119903998400
0
119902998400
1119902998400
0
119903998400
1
119901998400
1
119902998400
2
119903998400
2
119901998400
2119901998400
3119903998400
3
middot middot middot 985747 minus 1
119903998400
985747minus1
119901998400
985747
119902998400
985747minus1
985747
119903998400
985747
middot middot middot0 1 2 3
Figure 2
Given the sequences (119901119896
)119896ge0
and (119903119896
)119896ge0
it is clear that theabove sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
exist and are uniqueThismeans that the sequences (119908
119896
)119896ge0
and (1199081015840
119896
)119896ge0
are defineduniquely up to multiplicative constant factors by
119908119896
= 1199080
sdot 1198871
sdot sdot sdot 119887119896
1199081015840
119896
= 1199081015840
0
sdot 1205741
sdot sdot sdot 120574119896
(20)
(The unicity is understood up to the constant factors 1199080
1199081015840
0
)
Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840
119899
)119899ge0
defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights
Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908
10158401015840
119896
the weight 11990810158401015840
119888119896
of the circuit 11988810158401015840
119896
and by119908101584010158401015840
119896
the weight 119908101584010158401015840119888119896
of the circuit 119888101584010158401015840119896
for every 119896 ge 0 Thesequences (11990810158401015840
119896
)119896ge0
and (119908101584010158401015840
119896
)119896ge0
must be solutions of
1199021015840
119896
=11990810158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199021015840
0
=11990810158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199031015840
119896
=119908101584010158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199031015840
0
=119908101584010158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
119896 ge 1
(21)
By considering the sequences (119904119896
)119896
and (119905119896
)119896
where 119904119896
=
11990810158401015840
119896minus1
11990810158401015840
119896
119905119896
= 119908101584010158401015840
119896
119908101584010158401015840
119896minus1
119896 ge 1 we may obtain that
119904119896
=1 minus 1199021015840
119896
minus 1199031015840
119896
1199021015840119896
119905119896
=1199031015840
119896
1199031015840119896minus1
sdot1199021015840
119896
1199021015840119896minus1
sdot 119904119896
for every 119896 ge 1
(22)
For given sequences (1199021015840
119896
)119896ge0
(1199031015840
119896
)119896ge0
it is obvious that(119904119896
)119896ge1
(119905119896
)119896ge1
exist and are unique for those sequences thatis the sequences (119908
10158401015840
119896
)119896ge0
(119908101584010158401015840119896
)119896ge0
are defined uniquely upto multiplicative constant factors by
11990810158401015840
119896
=11990810158401015840
0
1199041
sdot 1199042
sdot sdot sdot 119904119896
119908101584010158401015840
119896
= 119908101584010158401015840
0
sdot 1199051
1199052
sdot sdot sdot 119905119896
(23)
(The unicity is based to the constant factors119908101584010158400
1199081015840101584010158400
)
4 Recurrence and Transience of the MarkovChains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
We have that for the Markov chain (119883119899
)119899ge0
there is a uniqueinvariant measure up to a multiplicative constant factor 120583
119896
=
119908119896minus1
+ 119908119896
+ 1199081015840
119896
119896 ge 1 1205830
= 1199080
+ 1199081015840
0
while for the Markovchain (119883
1015840
119899
)119899ge0
1205831015840119896
= 11990810158401015840
119896minus1
+11990810158401015840
119896
+119908101584010158401015840
119896
119896 ge 1with1205831015840
0
= 11990810158401015840
0
+119908101584010158401015840
0
In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing
Proposition 3 (i)TheMarkov chain (119883119899
)119899ge0
defined as aboveis positive recurrent if and only if
infin
sum
119896=1
1198871
sdot 1198872
sdot sdot sdot 119887119896
lt +infin (1199001199031
1199080
sdot
infin
sum
119896=1
119908119896
lt +infin)
infin
sum
119896=1
1205741
sdot 1205742
sdot sdot sdot 120574120581
lt +infin (1199001199031
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
lt +infin)
(24)
(ii) The Markov chain (1198831015840
119899
)119899ge0
defined as above is positiverecurrent if and only if
infin
sum
119896=1
1
1199041
sdot 1199042
sdot sdot sdot 119904119896
lt +infin (1199001199031
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
lt +infin)
infin
sum
119896=1
1199051
sdot 1199052
sdot sdot sdot 119905119896
= +infin (1199001199031
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin)
(25)
In order to obtain recurrence and transience criterionsfor the Markov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
we shall need thefollowing proposition (Karlin and Taylor [6])
Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent
Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892
119896
119896 ge 1) For the Markov chain (119883119899
)119899ge0
this is asolution of
1199010
sdot 1198921
+ 1199030
sdot 1198920
= 1198920
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119896 ge 1(26)
Journal of Probability and Statistics 5
Since Δ119892119896
= 119892119896
minus 119892119896minus1
for every 119896 ge 1 we obtain that
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot (Δ119892119896+1
+ 119892119896
) + 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot Δ119892119896+1
+ (119901119896
+ 119902119896
+ 119903119896
) sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
= 119892119896
119901119896
sdot Δ119892119896+1
minus 119902119896
sdot (119892119896
minus 119892119896minus1
) = 0
119901119896
sdot Δ119892119896+1
= 119902119896
sdot Δ119892119896
(27)
If we put 119898119896
= Δ119892119896
Δ119892119896+1
we get 119898119896
= 119901119896
119902119896
(with119901119896
= 1 minus 119902119896
minus 119903119896
) 119896 ge 1 which is the equation of definitionof the sequences (119904
119896
)119896ge1
and (119905119896
)119896ge1
(as a multiplicative factorof the (119904
119896
)119896ge1
) for the Markov chain (1198831015840
119899
)119899ge0
such that 1199021015840119896
=
119902119896
1199031015840119896
= 119903119896
for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883
119899
)119899ge0
are in correspondence with the weight representations of theMarkov chain (119883
1015840
119899
)119899ge0
such that
1199021015840
119896
= 119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
1199031015840
119896
= 119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
= 1 minus 119902119896
minus 119903119896
= 119901119896
for every 119896 ge 1
(28)
To express this kind of duality we will call the Markovchain (119883
1015840
119899
)119899ge0
the adjoint of the Markov chain (119883119899
)119899ge0
andreciprocally in the case that the relation (28) is satisfied
Equivalently for the Markov chain (1198831015840
119899
)119899ge0
the harmonicfunction 119892
1015840
= (1198921015840
119896
119896 ge 1) satisfies the equation
1199031015840
0
sdot 1198921015840
0
+ 1199021015840
0
sdot 1198921015840
1
= 1198921015840
0
1199021015840
119896
sdot 1198921015840
119896+1
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
119896 ge 1
(29)
Since Δ1198921015840
119896
= 1198921015840
119896
minus 1198921015840
119896minus1
for every 119896 ge 1 we have
1199021015840
119896
sdot (Δ1198921015840
119896+1
+ 1198921015840
119896
) + 1199011015840
119896
sdot 1198921015840
119896
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
(1199011015840
119896
+ 1199021015840
119896
+ 1199031015840
119896
) sdot 1198921015840
119896
+ 1199021015840
119896
sdot Δ1198921015840
119896+1
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
= 1198921015840
119896
1199021015840
119896
sdot Δ1198921015840
119896+1
= 1199011015840
119896
sdot (1198921015840
119896
minus 1198921015840
119896minus1
) = 1199011015840
119896
sdot Δ1198921015840
119896
(30)
If we put ℓ119896
= Δ1198921015840
119896+1
Δ1198921015840
119896
we get ℓ119896
= 1199011015840
119896
1199021015840
119896
(with1199021015840
119896
= 1minus1199011015840
119896
minus1199031015840
119896
) 119896 ge 1 which is the equation of the definitionof the sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
(as a multiplicative factorof the (119887
119896
)119896ge1
) for the Markov chain (119883119899
)119899ge0
such that 1199011015840119896
=
119901119896
1199031015840
119896
= 119903119896
for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899
)119899ge0
we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883
1015840
119899
)119899ge0
are in correspondencewith the weight representations of the Markov chain (119883
119899
)119899ge0
such that equivalent equations of (28) are satisfiedSo we may have the following
Proposition 5 TheMarkov chain (119883119899
)119899ge0
defined as above istransient if and only if the adjointMarkov chain (119883
1015840
119899
)119899ge0
is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are null recurrent simultaneouslyIn particular
(i) the Markov chain (119883119899
)119899ge0
defined as above is transientif and only if (111990810158401015840
0
) sdot suminfin
119896=1
11990810158401015840
119896
lt +infin and (1119908101584010158401015840
0
) sdot
suminfin
119896=1
119908101584010158401015840
119896
= +infin(ii) theMarkov chain (119883
1015840
119899
)119899ge0
defined as above is transientif and only if (1119908
0
) sdot suminfin
119896=1
119908119896
lt +infin and (11199081015840
0
) sdot
suminfin
119896=1
1199081015840
119896
lt +infin(iii) the adjointMarkov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are nullrecurrent if
1
1199080
sdot
infin
sum
119896=1
119908119896
=1
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
= +infin
1
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
=1
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin
(31)
Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3
Acknowledgment
C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion
References
[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995
[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981
[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982
[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991
[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999
[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Journal of Probability and Statistics
directed circuit in 119878 The ordered sequence 119888 = (1198941
1198942
119894119903
)
associated with the given directed 119888 is called a directed cyclein 119878
A directed circuit may be considered as 119888 = (119888(119903) 119888(119903 +
1) 119888(119903 + V minus 1) 119888(119903 + V)) if there exists an 119903 isin 119885 such that1198941
= 119888(119903 + 0) 1198942
= 119888(119903 + 1) 119894V = 119888(119903 + V minus 1) 1198941
= 119888(119903 + V)where 119888 is a periodic function from 119885 to 119878
The corresponding directed cycle is defined by theordered sequence 119888 = (119888(119903) 119888(119903+1) 119888(119903+Vminus1))The values119888(119896) are the points of 119888 while the directed pairs (119888(119896) 119888(119896+1))119896 isin 119885 are the directed edges of 119888
The smallest integer 119901 equiv 119901(119888) ge 1 satisfying the equation119888(119903 + 119901) = 119888(119903) for all 119903 isin 119885 is the period of 119888 Adirected circuit 119888 such that 119901(119888) = 1 is called a loop (In thepresent work we will use directed circuits with distinct pointelements)
Let us also consider a directed circuit 119888 (or a directed cycle119888) with period 119901(119888) gt 1 Then we may define by
119869(119899)
119888
(119894 119895) = 1 if there exists an 119903 isin 119885 such that
119894 = 119888 (119903) 119895 = 119888 (119903 + 119899)
= 0 otherwise
(3)
the 119899-step passage function associatedwith the directed circuit119888 for any 119894 119895 isin 119878 119899 ge 1
Furthermore we may define by
119869119888
(119894) = 1 if there exists an 119903 isin 119885 such that 119894 = 119888 (119903)
= 0 otherwise(4)
the passage function associated with the directed circuit 119888 forany 119894 isin 119878 The above definitions are due toMacQueen [2] andKalpazidou [1]
Given a denumerable set 119878 and an infinite denumerableclass 119862 of overlapping directed circuits (or directed cycles)with distinct points (except for the terminals) in 119878 such thatall the points of 119878 can be reached from one another followingpaths of circuit edges that is for each two distinct points 119894
and 119895 of 119878 there exists a finite sequence 1198881
119888119896
119896 ge 1 ofcircuits (or cycles) of 119862 such that 119894 lies on 119888
1
and 119895 lies on 119888119896
and any pair of consecutive circuits (119888
119899
119888119899+1
) have at least onepoint in common Generally we may assume that the class 119862contains among its elements circuits (or cycles) with periodgreater or equal to 2
With each directed circuit (or directed cycle) 119888 isin 119862
let us associate a strictly positive weight 119908119888
which must beindependent of the choice of the representative of 119888 that isit must satisfy the consistency condition 119908cot119896 = 119908
119888
119896 isin 119885where 119905
119896
is the translation of length 119896 (that is 119905119896
(119899) equiv 119899 + 119896119899 isin 119885 for any fixed 119896 isin 119885)
For a given class 119862 of overlapping directed circuits (orcycles) and for a given sequence (119908
119888
)119888isin119862
of weights we maydefine by
119901119894119895
=sum119888isin119862
119908119888
sdot 119869(1)
119888
(119894 119895)
sum119888isin119862
119908119888
sdot 119869119888
(119894)(5)
the elements of a Markov transition matrix on 119878 if and onlyif sum119888isin119862
119908119888
sdot 119869119888
(119894) lt infin for any 119894 isin 119878 This means that agiven Markov transition matrix 119875 = (119901
119894119895
) 119894 119895 isin 119878 can berepresented by directed circuits (or cycles) and weights if andonly if there exists a class of overlapping directed circuits (orcycles) 119862 and a sequence of positive weights (119908
119888
)119888isin119862
suchthat the formula (5) holds In this case the Markov transitionmatrix 119875 has a unique stationary distribution 119901 which is asolution of 119901119875 = 119901 and is defined by
119901 (119894) = sum
119888isin119862
119908119888
sdot 119869119888
(119894) 119894 isin 119878 (6)
It is known that the following classes of Markov chainsmay be represented uniquely by circuits (or cycles) andweights
(i) the recurrent Markov chains (Minping and Min [3])(ii) the reversible Markov chains
3 Auxiliary Results
Let us consider aMarkov chain (119883119899
)119899ge0
onalefsymwith transitions119896 rarr (119896+1) 119896 rarr (119896minus1) and 119896 rarr 119896whose the elements ofthe corresponding Markov transition matrix are defined by
119875 (119883119899+1
= 0119883119899
= 0) = 1199030
119875 (119883119899+1
= 1119883119899
= 0) = 1199010
1199010
= 1 minus 1199030
119875 (119883119899+1
= 119896 + 1119883119899
= 119896) = 119901119896
119896 ge 1
119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
119896 ge 1
119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
119896 ge 1
(7)
such that 119901119896
+ 119902119896
+ 119903119896
= 1 0 le 119901119896
119903119896
le 1 for every 119896 ge 1 asit is shown in (Figure 1)
Assume that (119901119896
)119896ge0
and (119903119896
)119896ge0
are arbitrary fixedsequences with 0 le 119901
0
= 1 minus 1199030
le 1 0 le 119901119896
119903119896
le 1 for every119896 gt 1 If we consider the directed circuits 119888
119896
= (119896 119896 + 1 119896)1198881015840
119896
= (119896 119896)119896 ge 0 and the collections of weights (119908119888119896)119896ge0
and(119908119888
1015840
119896
)119896ge0
respectively then we may obtain that
119901119896
=119908119888119896
119908119888119896minus1
+ 119908119888119896+ 119908119888
1015840
119896
for every 119896 ge 1 (8)
with
1199010
=1199081198880
1199081198880+ 119908119888
1015840
0
(9)
Here the class 119862(119896) contains the directed circuits 119888119896
=
(119896 119896+1 119896) 119888119896minus1
= (119896minus1 119896 119896minus1) and 1198881015840
119896
= (119896 119896) Furthermorewe may define
119902119896
=119908119888119896minus1
119908119888119896minus1
+ 119908119888119896+ 119908119888
1015840
119896
for every 119896 ge 1 (10)
and 119903119896
= 119908119888
1015840
119896
(119908119888119896minus1
+119908119888119896+119908119888
1015840
119896
) such that 119901119896
+119902119896
+ 119903119896
= 1 forevery 119896 ge 1 with 119903
0
= 1 minus 1199010
= 119908119888
1015840
0
(1199081198880+ 119908119888
1015840
0
)
Journal of Probability and Statistics 3
1199030
1199010
11990211199031
1199011
11990221199032
1199012
11990231199033
middot middot middot 985747 minus 1
119903985747minus1
119901985747minus1
119902985747
985747
119903985747
middot middot middot0 1 2 3
Figure 1
The transition matrix 119875 = (119901119894119895
) with
119901119894119895
=suminfin
119896=0
119908119888119896sdot 119869(1)
119888119896
(119894 119895)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] for 119894 = 119895 (11)
119901119894119894
=
suminfin
119896=0
1199081198881015840119896
sdot 119869(1)
119888
1015840
119896
(119894 119894)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] (12)
where 119869(1)
119888119896
(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896
119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888
119896
and 119869119888
1015840
119896
(119894) = 1if 119894 is a point of the circuit 119888
1015840
119896
expresses the representationof the Markov chain (119883
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840
119899
)119899ge0
on alefsym whose elements of the corresponding Markovtransition matrix are defined by
119875 (1198831015840
119899+1
= 01198831015840
119899
= 0) = 1199031015840
0
119875 (1198831015840
119899+1
= 11198831015840
119899
= 0) = 1199021015840
0
1199021015840
0
= 1 minus 1199031015840
0
119875 (1198831015840
119899+1
= 119896 minus 11198831015840
119899
= 119896) = 1199011015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896) = 1199031015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896) = 1199021015840
119896
119896 ge 1
(13)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 0 lt 1199011015840
119896
1199021015840
119896
1199031015840
119896
le 1 for every 119896 ge 1as it is shown in (Figure 2)
Assume that (1199021015840
119896
)119896ge0
and (1199031015840
119896
)119896ge0
are arbitrary fixedsequences with 0 le 119902
1015840
0
= 1 minus 1199031015840
0
le 1 0 le 1199021015840
119896
1199031015840119896
le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840
119896
= (119896+1 119896 119896+1)119888101584010158401015840
119896
= (119896 119896) 119896 ge 0 and the collections of weights (119908119888
10158401015840
119896
)119896ge0
and (119908
119888
101584010158401015840
119896
)119896ge0
respectively then we may have
1199021015840
119896
=119908119888
10158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1 (14)
with
1199021015840
0
=119908119888
10158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(15)
Here the class 1198621015840
(119896) contains the directed circuits 11988810158401015840
119896
=
(119896 + 1 119896 119896 + 1) 11988810158401015840
119896minus1
= (119896 119896 minus 1 119896) and 119888101584010158401015840
119896
= (119896 119896)Furthermore we may define
1199011015840
119896
=119908119888
10158401015840
119896minus1
11990810158401015840119888119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1
1199031015840
119896
=119908119888
101584010158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
(16)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 for every 119896 ge 1 with
1199031015840
0
= 1 minus 1199021015840
0
=119908119888
101584010158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(17)
So the transition matrix 1198751015840
= (1199011015840
119894119895
) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883
1015840
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Consequently we have the following
Proposition 1 The Markov chain (119883119899
)119899ge0
defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights
Proof Let us consider the set of directed circuits 119888119896
= (119896 119896 +
1 119896) and 1198881015840
119896
= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888
119896minus1
119888119896
and 1198881015840
119896
andtwo circuits through 0 119888
0
11988810158400
The problem we have to manage is the definition of the
weightsWemay symbolize by119908119896
the weight119908119888119896of the circuit
c119896
and by1199081015840
119896
the weight119908119888
1015840
119896
of the circuit 1198881015840119896
for any 119896 ge 0Thesequences (119908
119896
)119896ge0
(1199081015840119896
)119896ge0
must be a solution of
119901119896
=119908119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199010
=1199080
1199080
+ 11990810158400
119903119896
=1199081015840
119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199030
=1199081015840
0
1199080
+ 11990810158400
119902119896
= 1 minus 119901119896
minus 119903119896
119896 ge 1
(18)
Let us take by 119887119896
= 119908119896
119908119896minus1
120574119896
= 1199081015840
119896
1199081015840
119896minus1
119896 ge 1As a consequence we may have
119887119896
=119901119896
119902119896
=119901119896
1 minus 119901119896
minus 119903119896
120574119896
=119903119896
119903119896minus1
119901119896minus1
119901119896
119887119896
for every 119896 ge 1
(19)
4 Journal of Probability and Statistics
119903998400
0
119902998400
1119902998400
0
119903998400
1
119901998400
1
119902998400
2
119903998400
2
119901998400
2119901998400
3119903998400
3
middot middot middot 985747 minus 1
119903998400
985747minus1
119901998400
985747
119902998400
985747minus1
985747
119903998400
985747
middot middot middot0 1 2 3
Figure 2
Given the sequences (119901119896
)119896ge0
and (119903119896
)119896ge0
it is clear that theabove sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
exist and are uniqueThismeans that the sequences (119908
119896
)119896ge0
and (1199081015840
119896
)119896ge0
are defineduniquely up to multiplicative constant factors by
119908119896
= 1199080
sdot 1198871
sdot sdot sdot 119887119896
1199081015840
119896
= 1199081015840
0
sdot 1205741
sdot sdot sdot 120574119896
(20)
(The unicity is understood up to the constant factors 1199080
1199081015840
0
)
Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840
119899
)119899ge0
defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights
Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908
10158401015840
119896
the weight 11990810158401015840
119888119896
of the circuit 11988810158401015840
119896
and by119908101584010158401015840
119896
the weight 119908101584010158401015840119888119896
of the circuit 119888101584010158401015840119896
for every 119896 ge 0 Thesequences (11990810158401015840
119896
)119896ge0
and (119908101584010158401015840
119896
)119896ge0
must be solutions of
1199021015840
119896
=11990810158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199021015840
0
=11990810158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199031015840
119896
=119908101584010158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199031015840
0
=119908101584010158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
119896 ge 1
(21)
By considering the sequences (119904119896
)119896
and (119905119896
)119896
where 119904119896
=
11990810158401015840
119896minus1
11990810158401015840
119896
119905119896
= 119908101584010158401015840
119896
119908101584010158401015840
119896minus1
119896 ge 1 we may obtain that
119904119896
=1 minus 1199021015840
119896
minus 1199031015840
119896
1199021015840119896
119905119896
=1199031015840
119896
1199031015840119896minus1
sdot1199021015840
119896
1199021015840119896minus1
sdot 119904119896
for every 119896 ge 1
(22)
For given sequences (1199021015840
119896
)119896ge0
(1199031015840
119896
)119896ge0
it is obvious that(119904119896
)119896ge1
(119905119896
)119896ge1
exist and are unique for those sequences thatis the sequences (119908
10158401015840
119896
)119896ge0
(119908101584010158401015840119896
)119896ge0
are defined uniquely upto multiplicative constant factors by
11990810158401015840
119896
=11990810158401015840
0
1199041
sdot 1199042
sdot sdot sdot 119904119896
119908101584010158401015840
119896
= 119908101584010158401015840
0
sdot 1199051
1199052
sdot sdot sdot 119905119896
(23)
(The unicity is based to the constant factors119908101584010158400
1199081015840101584010158400
)
4 Recurrence and Transience of the MarkovChains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
We have that for the Markov chain (119883119899
)119899ge0
there is a uniqueinvariant measure up to a multiplicative constant factor 120583
119896
=
119908119896minus1
+ 119908119896
+ 1199081015840
119896
119896 ge 1 1205830
= 1199080
+ 1199081015840
0
while for the Markovchain (119883
1015840
119899
)119899ge0
1205831015840119896
= 11990810158401015840
119896minus1
+11990810158401015840
119896
+119908101584010158401015840
119896
119896 ge 1with1205831015840
0
= 11990810158401015840
0
+119908101584010158401015840
0
In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing
Proposition 3 (i)TheMarkov chain (119883119899
)119899ge0
defined as aboveis positive recurrent if and only if
infin
sum
119896=1
1198871
sdot 1198872
sdot sdot sdot 119887119896
lt +infin (1199001199031
1199080
sdot
infin
sum
119896=1
119908119896
lt +infin)
infin
sum
119896=1
1205741
sdot 1205742
sdot sdot sdot 120574120581
lt +infin (1199001199031
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
lt +infin)
(24)
(ii) The Markov chain (1198831015840
119899
)119899ge0
defined as above is positiverecurrent if and only if
infin
sum
119896=1
1
1199041
sdot 1199042
sdot sdot sdot 119904119896
lt +infin (1199001199031
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
lt +infin)
infin
sum
119896=1
1199051
sdot 1199052
sdot sdot sdot 119905119896
= +infin (1199001199031
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin)
(25)
In order to obtain recurrence and transience criterionsfor the Markov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
we shall need thefollowing proposition (Karlin and Taylor [6])
Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent
Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892
119896
119896 ge 1) For the Markov chain (119883119899
)119899ge0
this is asolution of
1199010
sdot 1198921
+ 1199030
sdot 1198920
= 1198920
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119896 ge 1(26)
Journal of Probability and Statistics 5
Since Δ119892119896
= 119892119896
minus 119892119896minus1
for every 119896 ge 1 we obtain that
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot (Δ119892119896+1
+ 119892119896
) + 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot Δ119892119896+1
+ (119901119896
+ 119902119896
+ 119903119896
) sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
= 119892119896
119901119896
sdot Δ119892119896+1
minus 119902119896
sdot (119892119896
minus 119892119896minus1
) = 0
119901119896
sdot Δ119892119896+1
= 119902119896
sdot Δ119892119896
(27)
If we put 119898119896
= Δ119892119896
Δ119892119896+1
we get 119898119896
= 119901119896
119902119896
(with119901119896
= 1 minus 119902119896
minus 119903119896
) 119896 ge 1 which is the equation of definitionof the sequences (119904
119896
)119896ge1
and (119905119896
)119896ge1
(as a multiplicative factorof the (119904
119896
)119896ge1
) for the Markov chain (1198831015840
119899
)119899ge0
such that 1199021015840119896
=
119902119896
1199031015840119896
= 119903119896
for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883
119899
)119899ge0
are in correspondence with the weight representations of theMarkov chain (119883
1015840
119899
)119899ge0
such that
1199021015840
119896
= 119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
1199031015840
119896
= 119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
= 1 minus 119902119896
minus 119903119896
= 119901119896
for every 119896 ge 1
(28)
To express this kind of duality we will call the Markovchain (119883
1015840
119899
)119899ge0
the adjoint of the Markov chain (119883119899
)119899ge0
andreciprocally in the case that the relation (28) is satisfied
Equivalently for the Markov chain (1198831015840
119899
)119899ge0
the harmonicfunction 119892
1015840
= (1198921015840
119896
119896 ge 1) satisfies the equation
1199031015840
0
sdot 1198921015840
0
+ 1199021015840
0
sdot 1198921015840
1
= 1198921015840
0
1199021015840
119896
sdot 1198921015840
119896+1
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
119896 ge 1
(29)
Since Δ1198921015840
119896
= 1198921015840
119896
minus 1198921015840
119896minus1
for every 119896 ge 1 we have
1199021015840
119896
sdot (Δ1198921015840
119896+1
+ 1198921015840
119896
) + 1199011015840
119896
sdot 1198921015840
119896
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
(1199011015840
119896
+ 1199021015840
119896
+ 1199031015840
119896
) sdot 1198921015840
119896
+ 1199021015840
119896
sdot Δ1198921015840
119896+1
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
= 1198921015840
119896
1199021015840
119896
sdot Δ1198921015840
119896+1
= 1199011015840
119896
sdot (1198921015840
119896
minus 1198921015840
119896minus1
) = 1199011015840
119896
sdot Δ1198921015840
119896
(30)
If we put ℓ119896
= Δ1198921015840
119896+1
Δ1198921015840
119896
we get ℓ119896
= 1199011015840
119896
1199021015840
119896
(with1199021015840
119896
= 1minus1199011015840
119896
minus1199031015840
119896
) 119896 ge 1 which is the equation of the definitionof the sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
(as a multiplicative factorof the (119887
119896
)119896ge1
) for the Markov chain (119883119899
)119899ge0
such that 1199011015840119896
=
119901119896
1199031015840
119896
= 119903119896
for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899
)119899ge0
we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883
1015840
119899
)119899ge0
are in correspondencewith the weight representations of the Markov chain (119883
119899
)119899ge0
such that equivalent equations of (28) are satisfiedSo we may have the following
Proposition 5 TheMarkov chain (119883119899
)119899ge0
defined as above istransient if and only if the adjointMarkov chain (119883
1015840
119899
)119899ge0
is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are null recurrent simultaneouslyIn particular
(i) the Markov chain (119883119899
)119899ge0
defined as above is transientif and only if (111990810158401015840
0
) sdot suminfin
119896=1
11990810158401015840
119896
lt +infin and (1119908101584010158401015840
0
) sdot
suminfin
119896=1
119908101584010158401015840
119896
= +infin(ii) theMarkov chain (119883
1015840
119899
)119899ge0
defined as above is transientif and only if (1119908
0
) sdot suminfin
119896=1
119908119896
lt +infin and (11199081015840
0
) sdot
suminfin
119896=1
1199081015840
119896
lt +infin(iii) the adjointMarkov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are nullrecurrent if
1
1199080
sdot
infin
sum
119896=1
119908119896
=1
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
= +infin
1
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
=1
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin
(31)
Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3
Acknowledgment
C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion
References
[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995
[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981
[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982
[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991
[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999
[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Probability and Statistics 3
1199030
1199010
11990211199031
1199011
11990221199032
1199012
11990231199033
middot middot middot 985747 minus 1
119903985747minus1
119901985747minus1
119902985747
985747
119903985747
middot middot middot0 1 2 3
Figure 1
The transition matrix 119875 = (119901119894119895
) with
119901119894119895
=suminfin
119896=0
119908119888119896sdot 119869(1)
119888119896
(119894 119895)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] for 119894 = 119895 (11)
119901119894119894
=
suminfin
119896=0
1199081198881015840119896
sdot 119869(1)
119888
1015840
119896
(119894 119894)
suminfin
119896=0
[119908119888119896sdot 119869119888119896(119894) + 119908
119888
1015840
119896
sdot 119869119888
1015840
119896
(119894)] (12)
where 119869(1)
119888119896
(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896
119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888
119896
and 119869119888
1015840
119896
(119894) = 1if 119894 is a point of the circuit 119888
1015840
119896
expresses the representationof the Markov chain (119883
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840
119899
)119899ge0
on alefsym whose elements of the corresponding Markovtransition matrix are defined by
119875 (1198831015840
119899+1
= 01198831015840
119899
= 0) = 1199031015840
0
119875 (1198831015840
119899+1
= 11198831015840
119899
= 0) = 1199021015840
0
1199021015840
0
= 1 minus 1199031015840
0
119875 (1198831015840
119899+1
= 119896 minus 11198831015840
119899
= 119896) = 1199011015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896) = 1199031015840
119896
119896 ge 1
119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896) = 1199021015840
119896
119896 ge 1
(13)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 0 lt 1199011015840
119896
1199021015840
119896
1199031015840
119896
le 1 for every 119896 ge 1as it is shown in (Figure 2)
Assume that (1199021015840
119896
)119896ge0
and (1199031015840
119896
)119896ge0
are arbitrary fixedsequences with 0 le 119902
1015840
0
= 1 minus 1199031015840
0
le 1 0 le 1199021015840
119896
1199031015840119896
le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840
119896
= (119896+1 119896 119896+1)119888101584010158401015840
119896
= (119896 119896) 119896 ge 0 and the collections of weights (119908119888
10158401015840
119896
)119896ge0
and (119908
119888
101584010158401015840
119896
)119896ge0
respectively then we may have
1199021015840
119896
=119908119888
10158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1 (14)
with
1199021015840
0
=119908119888
10158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(15)
Here the class 1198621015840
(119896) contains the directed circuits 11988810158401015840
119896
=
(119896 + 1 119896 119896 + 1) 11988810158401015840
119896minus1
= (119896 119896 minus 1 119896) and 119888101584010158401015840
119896
= (119896 119896)Furthermore we may define
1199011015840
119896
=119908119888
10158401015840
119896minus1
11990810158401015840119888119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
for every 119896 ge 1
1199031015840
119896
=119908119888
101584010158401015840
119896
119908119888
10158401015840
119896minus1
+ 119908119888
10158401015840
119896
+ 119908119888
101584010158401015840
119896
(16)
such that 1199011015840119896
+ 1199021015840
119896
+ 1199031015840
119896
= 1 for every 119896 ge 1 with
1199031015840
0
= 1 minus 1199021015840
0
=119908119888
101584010158401015840
0
119908119888
10158401015840
0
+ 119908119888
101584010158401015840
0
(17)
So the transition matrix 1198751015840
= (1199011015840
119894119895
) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883
1015840
119899
)119899ge0
by directed cycles (especially bydirected circuits) and weights
Consequently we have the following
Proposition 1 The Markov chain (119883119899
)119899ge0
defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights
Proof Let us consider the set of directed circuits 119888119896
= (119896 119896 +
1 119896) and 1198881015840
119896
= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888
119896minus1
119888119896
and 1198881015840
119896
andtwo circuits through 0 119888
0
11988810158400
The problem we have to manage is the definition of the
weightsWemay symbolize by119908119896
the weight119908119888119896of the circuit
c119896
and by1199081015840
119896
the weight119908119888
1015840
119896
of the circuit 1198881015840119896
for any 119896 ge 0Thesequences (119908
119896
)119896ge0
(1199081015840119896
)119896ge0
must be a solution of
119901119896
=119908119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199010
=1199080
1199080
+ 11990810158400
119903119896
=1199081015840
119896
119908119896minus1
+ 119908119896
+ 1199081015840119896
119896 ge 1 with 1199030
=1199081015840
0
1199080
+ 11990810158400
119902119896
= 1 minus 119901119896
minus 119903119896
119896 ge 1
(18)
Let us take by 119887119896
= 119908119896
119908119896minus1
120574119896
= 1199081015840
119896
1199081015840
119896minus1
119896 ge 1As a consequence we may have
119887119896
=119901119896
119902119896
=119901119896
1 minus 119901119896
minus 119903119896
120574119896
=119903119896
119903119896minus1
119901119896minus1
119901119896
119887119896
for every 119896 ge 1
(19)
4 Journal of Probability and Statistics
119903998400
0
119902998400
1119902998400
0
119903998400
1
119901998400
1
119902998400
2
119903998400
2
119901998400
2119901998400
3119903998400
3
middot middot middot 985747 minus 1
119903998400
985747minus1
119901998400
985747
119902998400
985747minus1
985747
119903998400
985747
middot middot middot0 1 2 3
Figure 2
Given the sequences (119901119896
)119896ge0
and (119903119896
)119896ge0
it is clear that theabove sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
exist and are uniqueThismeans that the sequences (119908
119896
)119896ge0
and (1199081015840
119896
)119896ge0
are defineduniquely up to multiplicative constant factors by
119908119896
= 1199080
sdot 1198871
sdot sdot sdot 119887119896
1199081015840
119896
= 1199081015840
0
sdot 1205741
sdot sdot sdot 120574119896
(20)
(The unicity is understood up to the constant factors 1199080
1199081015840
0
)
Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840
119899
)119899ge0
defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights
Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908
10158401015840
119896
the weight 11990810158401015840
119888119896
of the circuit 11988810158401015840
119896
and by119908101584010158401015840
119896
the weight 119908101584010158401015840119888119896
of the circuit 119888101584010158401015840119896
for every 119896 ge 0 Thesequences (11990810158401015840
119896
)119896ge0
and (119908101584010158401015840
119896
)119896ge0
must be solutions of
1199021015840
119896
=11990810158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199021015840
0
=11990810158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199031015840
119896
=119908101584010158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199031015840
0
=119908101584010158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
119896 ge 1
(21)
By considering the sequences (119904119896
)119896
and (119905119896
)119896
where 119904119896
=
11990810158401015840
119896minus1
11990810158401015840
119896
119905119896
= 119908101584010158401015840
119896
119908101584010158401015840
119896minus1
119896 ge 1 we may obtain that
119904119896
=1 minus 1199021015840
119896
minus 1199031015840
119896
1199021015840119896
119905119896
=1199031015840
119896
1199031015840119896minus1
sdot1199021015840
119896
1199021015840119896minus1
sdot 119904119896
for every 119896 ge 1
(22)
For given sequences (1199021015840
119896
)119896ge0
(1199031015840
119896
)119896ge0
it is obvious that(119904119896
)119896ge1
(119905119896
)119896ge1
exist and are unique for those sequences thatis the sequences (119908
10158401015840
119896
)119896ge0
(119908101584010158401015840119896
)119896ge0
are defined uniquely upto multiplicative constant factors by
11990810158401015840
119896
=11990810158401015840
0
1199041
sdot 1199042
sdot sdot sdot 119904119896
119908101584010158401015840
119896
= 119908101584010158401015840
0
sdot 1199051
1199052
sdot sdot sdot 119905119896
(23)
(The unicity is based to the constant factors119908101584010158400
1199081015840101584010158400
)
4 Recurrence and Transience of the MarkovChains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
We have that for the Markov chain (119883119899
)119899ge0
there is a uniqueinvariant measure up to a multiplicative constant factor 120583
119896
=
119908119896minus1
+ 119908119896
+ 1199081015840
119896
119896 ge 1 1205830
= 1199080
+ 1199081015840
0
while for the Markovchain (119883
1015840
119899
)119899ge0
1205831015840119896
= 11990810158401015840
119896minus1
+11990810158401015840
119896
+119908101584010158401015840
119896
119896 ge 1with1205831015840
0
= 11990810158401015840
0
+119908101584010158401015840
0
In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing
Proposition 3 (i)TheMarkov chain (119883119899
)119899ge0
defined as aboveis positive recurrent if and only if
infin
sum
119896=1
1198871
sdot 1198872
sdot sdot sdot 119887119896
lt +infin (1199001199031
1199080
sdot
infin
sum
119896=1
119908119896
lt +infin)
infin
sum
119896=1
1205741
sdot 1205742
sdot sdot sdot 120574120581
lt +infin (1199001199031
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
lt +infin)
(24)
(ii) The Markov chain (1198831015840
119899
)119899ge0
defined as above is positiverecurrent if and only if
infin
sum
119896=1
1
1199041
sdot 1199042
sdot sdot sdot 119904119896
lt +infin (1199001199031
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
lt +infin)
infin
sum
119896=1
1199051
sdot 1199052
sdot sdot sdot 119905119896
= +infin (1199001199031
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin)
(25)
In order to obtain recurrence and transience criterionsfor the Markov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
we shall need thefollowing proposition (Karlin and Taylor [6])
Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent
Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892
119896
119896 ge 1) For the Markov chain (119883119899
)119899ge0
this is asolution of
1199010
sdot 1198921
+ 1199030
sdot 1198920
= 1198920
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119896 ge 1(26)
Journal of Probability and Statistics 5
Since Δ119892119896
= 119892119896
minus 119892119896minus1
for every 119896 ge 1 we obtain that
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot (Δ119892119896+1
+ 119892119896
) + 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot Δ119892119896+1
+ (119901119896
+ 119902119896
+ 119903119896
) sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
= 119892119896
119901119896
sdot Δ119892119896+1
minus 119902119896
sdot (119892119896
minus 119892119896minus1
) = 0
119901119896
sdot Δ119892119896+1
= 119902119896
sdot Δ119892119896
(27)
If we put 119898119896
= Δ119892119896
Δ119892119896+1
we get 119898119896
= 119901119896
119902119896
(with119901119896
= 1 minus 119902119896
minus 119903119896
) 119896 ge 1 which is the equation of definitionof the sequences (119904
119896
)119896ge1
and (119905119896
)119896ge1
(as a multiplicative factorof the (119904
119896
)119896ge1
) for the Markov chain (1198831015840
119899
)119899ge0
such that 1199021015840119896
=
119902119896
1199031015840119896
= 119903119896
for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883
119899
)119899ge0
are in correspondence with the weight representations of theMarkov chain (119883
1015840
119899
)119899ge0
such that
1199021015840
119896
= 119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
1199031015840
119896
= 119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
= 1 minus 119902119896
minus 119903119896
= 119901119896
for every 119896 ge 1
(28)
To express this kind of duality we will call the Markovchain (119883
1015840
119899
)119899ge0
the adjoint of the Markov chain (119883119899
)119899ge0
andreciprocally in the case that the relation (28) is satisfied
Equivalently for the Markov chain (1198831015840
119899
)119899ge0
the harmonicfunction 119892
1015840
= (1198921015840
119896
119896 ge 1) satisfies the equation
1199031015840
0
sdot 1198921015840
0
+ 1199021015840
0
sdot 1198921015840
1
= 1198921015840
0
1199021015840
119896
sdot 1198921015840
119896+1
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
119896 ge 1
(29)
Since Δ1198921015840
119896
= 1198921015840
119896
minus 1198921015840
119896minus1
for every 119896 ge 1 we have
1199021015840
119896
sdot (Δ1198921015840
119896+1
+ 1198921015840
119896
) + 1199011015840
119896
sdot 1198921015840
119896
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
(1199011015840
119896
+ 1199021015840
119896
+ 1199031015840
119896
) sdot 1198921015840
119896
+ 1199021015840
119896
sdot Δ1198921015840
119896+1
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
= 1198921015840
119896
1199021015840
119896
sdot Δ1198921015840
119896+1
= 1199011015840
119896
sdot (1198921015840
119896
minus 1198921015840
119896minus1
) = 1199011015840
119896
sdot Δ1198921015840
119896
(30)
If we put ℓ119896
= Δ1198921015840
119896+1
Δ1198921015840
119896
we get ℓ119896
= 1199011015840
119896
1199021015840
119896
(with1199021015840
119896
= 1minus1199011015840
119896
minus1199031015840
119896
) 119896 ge 1 which is the equation of the definitionof the sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
(as a multiplicative factorof the (119887
119896
)119896ge1
) for the Markov chain (119883119899
)119899ge0
such that 1199011015840119896
=
119901119896
1199031015840
119896
= 119903119896
for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899
)119899ge0
we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883
1015840
119899
)119899ge0
are in correspondencewith the weight representations of the Markov chain (119883
119899
)119899ge0
such that equivalent equations of (28) are satisfiedSo we may have the following
Proposition 5 TheMarkov chain (119883119899
)119899ge0
defined as above istransient if and only if the adjointMarkov chain (119883
1015840
119899
)119899ge0
is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are null recurrent simultaneouslyIn particular
(i) the Markov chain (119883119899
)119899ge0
defined as above is transientif and only if (111990810158401015840
0
) sdot suminfin
119896=1
11990810158401015840
119896
lt +infin and (1119908101584010158401015840
0
) sdot
suminfin
119896=1
119908101584010158401015840
119896
= +infin(ii) theMarkov chain (119883
1015840
119899
)119899ge0
defined as above is transientif and only if (1119908
0
) sdot suminfin
119896=1
119908119896
lt +infin and (11199081015840
0
) sdot
suminfin
119896=1
1199081015840
119896
lt +infin(iii) the adjointMarkov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are nullrecurrent if
1
1199080
sdot
infin
sum
119896=1
119908119896
=1
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
= +infin
1
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
=1
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin
(31)
Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3
Acknowledgment
C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion
References
[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995
[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981
[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982
[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991
[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999
[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Journal of Probability and Statistics
119903998400
0
119902998400
1119902998400
0
119903998400
1
119901998400
1
119902998400
2
119903998400
2
119901998400
2119901998400
3119903998400
3
middot middot middot 985747 minus 1
119903998400
985747minus1
119901998400
985747
119902998400
985747minus1
985747
119903998400
985747
middot middot middot0 1 2 3
Figure 2
Given the sequences (119901119896
)119896ge0
and (119903119896
)119896ge0
it is clear that theabove sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
exist and are uniqueThismeans that the sequences (119908
119896
)119896ge0
and (1199081015840
119896
)119896ge0
are defineduniquely up to multiplicative constant factors by
119908119896
= 1199080
sdot 1198871
sdot sdot sdot 119887119896
1199081015840
119896
= 1199081015840
0
sdot 1205741
sdot sdot sdot 120574119896
(20)
(The unicity is understood up to the constant factors 1199080
1199081015840
0
)
Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840
119899
)119899ge0
defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights
Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908
10158401015840
119896
the weight 11990810158401015840
119888119896
of the circuit 11988810158401015840
119896
and by119908101584010158401015840
119896
the weight 119908101584010158401015840119888119896
of the circuit 119888101584010158401015840119896
for every 119896 ge 0 Thesequences (11990810158401015840
119896
)119896ge0
and (119908101584010158401015840
119896
)119896ge0
must be solutions of
1199021015840
119896
=11990810158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199021015840
0
=11990810158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199031015840
119896
=119908101584010158401015840
119896
11990810158401015840119896minus1
+ 11990810158401015840119896
+ 119908101584010158401015840119896
119896 ge 1 with 1199031015840
0
=119908101584010158401015840
0
119908101584010158400
+ 1199081015840101584010158400
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
119896 ge 1
(21)
By considering the sequences (119904119896
)119896
and (119905119896
)119896
where 119904119896
=
11990810158401015840
119896minus1
11990810158401015840
119896
119905119896
= 119908101584010158401015840
119896
119908101584010158401015840
119896minus1
119896 ge 1 we may obtain that
119904119896
=1 minus 1199021015840
119896
minus 1199031015840
119896
1199021015840119896
119905119896
=1199031015840
119896
1199031015840119896minus1
sdot1199021015840
119896
1199021015840119896minus1
sdot 119904119896
for every 119896 ge 1
(22)
For given sequences (1199021015840
119896
)119896ge0
(1199031015840
119896
)119896ge0
it is obvious that(119904119896
)119896ge1
(119905119896
)119896ge1
exist and are unique for those sequences thatis the sequences (119908
10158401015840
119896
)119896ge0
(119908101584010158401015840119896
)119896ge0
are defined uniquely upto multiplicative constant factors by
11990810158401015840
119896
=11990810158401015840
0
1199041
sdot 1199042
sdot sdot sdot 119904119896
119908101584010158401015840
119896
= 119908101584010158401015840
0
sdot 1199051
1199052
sdot sdot sdot 119905119896
(23)
(The unicity is based to the constant factors119908101584010158400
1199081015840101584010158400
)
4 Recurrence and Transience of the MarkovChains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
We have that for the Markov chain (119883119899
)119899ge0
there is a uniqueinvariant measure up to a multiplicative constant factor 120583
119896
=
119908119896minus1
+ 119908119896
+ 1199081015840
119896
119896 ge 1 1205830
= 1199080
+ 1199081015840
0
while for the Markovchain (119883
1015840
119899
)119899ge0
1205831015840119896
= 11990810158401015840
119896minus1
+11990810158401015840
119896
+119908101584010158401015840
119896
119896 ge 1with1205831015840
0
= 11990810158401015840
0
+119908101584010158401015840
0
In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing
Proposition 3 (i)TheMarkov chain (119883119899
)119899ge0
defined as aboveis positive recurrent if and only if
infin
sum
119896=1
1198871
sdot 1198872
sdot sdot sdot 119887119896
lt +infin (1199001199031
1199080
sdot
infin
sum
119896=1
119908119896
lt +infin)
infin
sum
119896=1
1205741
sdot 1205742
sdot sdot sdot 120574120581
lt +infin (1199001199031
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
lt +infin)
(24)
(ii) The Markov chain (1198831015840
119899
)119899ge0
defined as above is positiverecurrent if and only if
infin
sum
119896=1
1
1199041
sdot 1199042
sdot sdot sdot 119904119896
lt +infin (1199001199031
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
lt +infin)
infin
sum
119896=1
1199051
sdot 1199052
sdot sdot sdot 119905119896
= +infin (1199001199031
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin)
(25)
In order to obtain recurrence and transience criterionsfor the Markov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
we shall need thefollowing proposition (Karlin and Taylor [6])
Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent
Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892
119896
119896 ge 1) For the Markov chain (119883119899
)119899ge0
this is asolution of
1199010
sdot 1198921
+ 1199030
sdot 1198920
= 1198920
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119896 ge 1(26)
Journal of Probability and Statistics 5
Since Δ119892119896
= 119892119896
minus 119892119896minus1
for every 119896 ge 1 we obtain that
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot (Δ119892119896+1
+ 119892119896
) + 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot Δ119892119896+1
+ (119901119896
+ 119902119896
+ 119903119896
) sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
= 119892119896
119901119896
sdot Δ119892119896+1
minus 119902119896
sdot (119892119896
minus 119892119896minus1
) = 0
119901119896
sdot Δ119892119896+1
= 119902119896
sdot Δ119892119896
(27)
If we put 119898119896
= Δ119892119896
Δ119892119896+1
we get 119898119896
= 119901119896
119902119896
(with119901119896
= 1 minus 119902119896
minus 119903119896
) 119896 ge 1 which is the equation of definitionof the sequences (119904
119896
)119896ge1
and (119905119896
)119896ge1
(as a multiplicative factorof the (119904
119896
)119896ge1
) for the Markov chain (1198831015840
119899
)119899ge0
such that 1199021015840119896
=
119902119896
1199031015840119896
= 119903119896
for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883
119899
)119899ge0
are in correspondence with the weight representations of theMarkov chain (119883
1015840
119899
)119899ge0
such that
1199021015840
119896
= 119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
1199031015840
119896
= 119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
= 1 minus 119902119896
minus 119903119896
= 119901119896
for every 119896 ge 1
(28)
To express this kind of duality we will call the Markovchain (119883
1015840
119899
)119899ge0
the adjoint of the Markov chain (119883119899
)119899ge0
andreciprocally in the case that the relation (28) is satisfied
Equivalently for the Markov chain (1198831015840
119899
)119899ge0
the harmonicfunction 119892
1015840
= (1198921015840
119896
119896 ge 1) satisfies the equation
1199031015840
0
sdot 1198921015840
0
+ 1199021015840
0
sdot 1198921015840
1
= 1198921015840
0
1199021015840
119896
sdot 1198921015840
119896+1
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
119896 ge 1
(29)
Since Δ1198921015840
119896
= 1198921015840
119896
minus 1198921015840
119896minus1
for every 119896 ge 1 we have
1199021015840
119896
sdot (Δ1198921015840
119896+1
+ 1198921015840
119896
) + 1199011015840
119896
sdot 1198921015840
119896
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
(1199011015840
119896
+ 1199021015840
119896
+ 1199031015840
119896
) sdot 1198921015840
119896
+ 1199021015840
119896
sdot Δ1198921015840
119896+1
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
= 1198921015840
119896
1199021015840
119896
sdot Δ1198921015840
119896+1
= 1199011015840
119896
sdot (1198921015840
119896
minus 1198921015840
119896minus1
) = 1199011015840
119896
sdot Δ1198921015840
119896
(30)
If we put ℓ119896
= Δ1198921015840
119896+1
Δ1198921015840
119896
we get ℓ119896
= 1199011015840
119896
1199021015840
119896
(with1199021015840
119896
= 1minus1199011015840
119896
minus1199031015840
119896
) 119896 ge 1 which is the equation of the definitionof the sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
(as a multiplicative factorof the (119887
119896
)119896ge1
) for the Markov chain (119883119899
)119899ge0
such that 1199011015840119896
=
119901119896
1199031015840
119896
= 119903119896
for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899
)119899ge0
we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883
1015840
119899
)119899ge0
are in correspondencewith the weight representations of the Markov chain (119883
119899
)119899ge0
such that equivalent equations of (28) are satisfiedSo we may have the following
Proposition 5 TheMarkov chain (119883119899
)119899ge0
defined as above istransient if and only if the adjointMarkov chain (119883
1015840
119899
)119899ge0
is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are null recurrent simultaneouslyIn particular
(i) the Markov chain (119883119899
)119899ge0
defined as above is transientif and only if (111990810158401015840
0
) sdot suminfin
119896=1
11990810158401015840
119896
lt +infin and (1119908101584010158401015840
0
) sdot
suminfin
119896=1
119908101584010158401015840
119896
= +infin(ii) theMarkov chain (119883
1015840
119899
)119899ge0
defined as above is transientif and only if (1119908
0
) sdot suminfin
119896=1
119908119896
lt +infin and (11199081015840
0
) sdot
suminfin
119896=1
1199081015840
119896
lt +infin(iii) the adjointMarkov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are nullrecurrent if
1
1199080
sdot
infin
sum
119896=1
119908119896
=1
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
= +infin
1
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
=1
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin
(31)
Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3
Acknowledgment
C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion
References
[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995
[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981
[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982
[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991
[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999
[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Journal of Probability and Statistics 5
Since Δ119892119896
= 119892119896
minus 119892119896minus1
for every 119896 ge 1 we obtain that
119901119896
sdot 119892119896+1
+ 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot (Δ119892119896+1
+ 119892119896
) + 119902119896
sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
+ 119903119896
sdot 119892119896
= 119892119896
119901119896
sdot Δ119892119896+1
+ (119901119896
+ 119902119896
+ 119903119896
) sdot 119892119896
minus 119902119896
sdot 119892119896
+ 119902119896
sdot 119892119896minus1
= 119892119896
119901119896
sdot Δ119892119896+1
minus 119902119896
sdot (119892119896
minus 119892119896minus1
) = 0
119901119896
sdot Δ119892119896+1
= 119902119896
sdot Δ119892119896
(27)
If we put 119898119896
= Δ119892119896
Δ119892119896+1
we get 119898119896
= 119901119896
119902119896
(with119901119896
= 1 minus 119902119896
minus 119903119896
) 119896 ge 1 which is the equation of definitionof the sequences (119904
119896
)119896ge1
and (119905119896
)119896ge1
(as a multiplicative factorof the (119904
119896
)119896ge1
) for the Markov chain (1198831015840
119899
)119899ge0
such that 1199021015840119896
=
119902119896
1199031015840119896
= 119903119896
for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883
119899
)119899ge0
are in correspondence with the weight representations of theMarkov chain (119883
1015840
119899
)119899ge0
such that
1199021015840
119896
= 119875 (1198831015840
119899+1
= 119896 + 11198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896 minus 1119883119899
= 119896) = 119902119896
1199031015840
119896
= 119875 (1198831015840
119899+1
= 1198961198831015840
119899
= 119896)
= 119875 (119883119899+1
= 119896119883119899
= 119896) = 119903119896
1199011015840
119896
= 1 minus 1199021015840
119896
minus 1199031015840
119896
= 1 minus 119902119896
minus 119903119896
= 119901119896
for every 119896 ge 1
(28)
To express this kind of duality we will call the Markovchain (119883
1015840
119899
)119899ge0
the adjoint of the Markov chain (119883119899
)119899ge0
andreciprocally in the case that the relation (28) is satisfied
Equivalently for the Markov chain (1198831015840
119899
)119899ge0
the harmonicfunction 119892
1015840
= (1198921015840
119896
119896 ge 1) satisfies the equation
1199031015840
0
sdot 1198921015840
0
+ 1199021015840
0
sdot 1198921015840
1
= 1198921015840
0
1199021015840
119896
sdot 1198921015840
119896+1
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
119896 ge 1
(29)
Since Δ1198921015840
119896
= 1198921015840
119896
minus 1198921015840
119896minus1
for every 119896 ge 1 we have
1199021015840
119896
sdot (Δ1198921015840
119896+1
+ 1198921015840
119896
) + 1199011015840
119896
sdot 1198921015840
119896
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
+ 1199031015840
119896
sdot 1198921015840
119896
= 1198921015840
119896
(1199011015840
119896
+ 1199021015840
119896
+ 1199031015840
119896
) sdot 1198921015840
119896
+ 1199021015840
119896
sdot Δ1198921015840
119896+1
minus 1199011015840
119896
sdot 1198921015840
119896
+ 1199011015840
119896
sdot 1198921015840
119896minus1
= 1198921015840
119896
1199021015840
119896
sdot Δ1198921015840
119896+1
= 1199011015840
119896
sdot (1198921015840
119896
minus 1198921015840
119896minus1
) = 1199011015840
119896
sdot Δ1198921015840
119896
(30)
If we put ℓ119896
= Δ1198921015840
119896+1
Δ1198921015840
119896
we get ℓ119896
= 1199011015840
119896
1199021015840
119896
(with1199021015840
119896
= 1minus1199011015840
119896
minus1199031015840
119896
) 119896 ge 1 which is the equation of the definitionof the sequences (119887
119896
)119896ge1
and (120574119896
)119896ge1
(as a multiplicative factorof the (119887
119896
)119896ge1
) for the Markov chain (119883119899
)119899ge0
such that 1199011015840119896
=
119901119896
1199031015840
119896
= 119903119896
for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899
)119899ge0
we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883
1015840
119899
)119899ge0
are in correspondencewith the weight representations of the Markov chain (119883
119899
)119899ge0
such that equivalent equations of (28) are satisfiedSo we may have the following
Proposition 5 TheMarkov chain (119883119899
)119899ge0
defined as above istransient if and only if the adjointMarkov chain (119883
1015840
119899
)119899ge0
is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are null recurrent simultaneouslyIn particular
(i) the Markov chain (119883119899
)119899ge0
defined as above is transientif and only if (111990810158401015840
0
) sdot suminfin
119896=1
11990810158401015840
119896
lt +infin and (1119908101584010158401015840
0
) sdot
suminfin
119896=1
119908101584010158401015840
119896
= +infin(ii) theMarkov chain (119883
1015840
119899
)119899ge0
defined as above is transientif and only if (1119908
0
) sdot suminfin
119896=1
119908119896
lt +infin and (11199081015840
0
) sdot
suminfin
119896=1
1199081015840
119896
lt +infin(iii) the adjointMarkov chains (119883
119899
)119899ge0
and (1198831015840
119899
)119899ge0
are nullrecurrent if
1
1199080
sdot
infin
sum
119896=1
119908119896
=1
119908101584010158400
sdot
infin
sum
119896=1
11990810158401015840
119896
= +infin
1
11990810158400
sdot
infin
sum
119896=1
1199081015840
119896
=1
1199081015840101584010158400
sdot
infin
sum
119896=1
119908101584010158401015840
119896
= +infin
(31)
Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3
Acknowledgment
C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion
References
[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995
[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981
[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982
[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991
[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999
[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of