Hindawi Publishing CorporationJournal of Probability and StatisticsVolume 2013 Article ID 424601 5 pageshttpdxdoiorg1011552013424601

Research ArticleOn the Study of Transience and Recurrence of the MarkovChain Defined by Directed Weighted Circuits Associated witha Random Walk in Fixed Environment

Chrysoula Ganatsiou

Department of Civil Engineering Section of Mathematics and Statistics School of Technological SciencesUniversity of Thessaly Pedion Areos 38334 Volos Greece

Correspondence should be addressed to Chrysoula Ganatsiou ganatsiootenetgr

Received 23 October 2012 Revised 2 March 2013 Accepted 27 March 2013

Academic Editor Man Lai Tang

Copyright copy 2013 Chrysoula GanatsiouThis is an open access article distributed under theCreativeCommonsAttributionLicensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

By using the cycle representation theory of Markov processes we investigate proper criterions regarding transience and recurrenceof the correspondingMarkov chain represented uniquely by directed cycles (especially by directed circuits) andweights of a randomwalk with jumps in a fixed environment

1 Introduction

A systematic research has been developed (Kalpazidou [1]MacQueen [2] Minping and Min [3] Zemanian [4] andothers) in order to investigate representations of the finite-dimensional distributions ofMarkov processes (with discreteor continuous parameter) having an invariant measure asdecompositions in terms of the cycle (or circuit) passagefunctions

119869119888

(119894 119895) = 1 if 119894 119895 are consecutive states of 119888

= 0 otherwise(1)

for any directed sequence 119888 = (1198941

1198942

119894119903

) (or 119888 =

(1198941

1198942

119894119903

1198941

)) of states called a cycle (or a circuit) 119903 gt 1of the corresponding Markov process The representationsare called cycle (or circuit) representations while the corre-sponding discrete parameter Markov processes generated bydirected circuits 119888 = (119894

1

1198942

119894119903

1198941

) 119903 gt 1 are called circuitchains

Following the context of the theory of Markov processesrsquocycle-circuit representation the present work arises as anattempt to investigate proper criterions regarding the prop-erties of transience and recurrence of the correspondingMarkov chain represented uniquely by directed cycles (espe-cially by directed circuits) and weights of a random walk

with jumps (having one elastic left barrier) in a fixed ergodicenvironment (Kalpazidou [1] Derriennic [5])

The paper is organized as follows In Section 2 we givea brief account of certain concepts of cycle representationtheory of Markov processes that we will need throughout thepaper In Section 3 we present some auxiliary results in orderto make the presentation of the paper more comprehensibleIn particular in Section 3 a randomwalk with jumps (havingone elastic left barrier) in a fixed ergodic environment isconsidered and the unique representations by directed cycles(especially by directed circuits) and weights of the corre-sponding Markov chain are investigated These representa-tions will give us the possibility to study proper criterionsregarding transience and recurrence of the abovementionedMarkov chain as it is described in Section 4

Throughout the paper we will need the following nota-tions

alefsym = 0 1 2 alefsymlowast

= 1 2

119885 = minus1 0 1 (2)

2 Preliminaries

Let us consider a denumerable set S Then the directedsequence 119888 = (119894

1

1198942

119894119903

1198941

) modulo the cyclic permuta-tions where 119894

1

1198942

119894119903

isin 119878 119903 gt 1 completely defines a

2 Journal of Probability and Statistics

directed circuit in 119878 The ordered sequence 119888 = (1198941

1198942

119894119903

)

associated with the given directed 119888 is called a directed cyclein 119878

A directed circuit may be considered as 119888 = (119888(119903) 119888(119903 +

1) 119888(119903 + V minus 1) 119888(119903 + V)) if there exists an 119903 isin 119885 such that1198941

= 119888(119903 + 0) 1198942

= 119888(119903 + 1) 119894V = 119888(119903 + V minus 1) 1198941

= 119888(119903 + V)where 119888 is a periodic function from 119885 to 119878

The corresponding directed cycle is defined by theordered sequence 119888 = (119888(119903) 119888(119903+1) 119888(119903+Vminus1))The values119888(119896) are the points of 119888 while the directed pairs (119888(119896) 119888(119896+1))119896 isin 119885 are the directed edges of 119888

The smallest integer 119901 equiv 119901(119888) ge 1 satisfying the equation119888(119903 + 119901) = 119888(119903) for all 119903 isin 119885 is the period of 119888 Adirected circuit 119888 such that 119901(119888) = 1 is called a loop (In thepresent work we will use directed circuits with distinct pointelements)

Let us also consider a directed circuit 119888 (or a directed cycle119888) with period 119901(119888) gt 1 Then we may define by

119869(119899)

119888

(119894 119895) = 1 if there exists an 119903 isin 119885 such that

119894 = 119888 (119903) 119895 = 119888 (119903 + 119899)

= 0 otherwise

(3)

the 119899-step passage function associatedwith the directed circuit119888 for any 119894 119895 isin 119878 119899 ge 1

Furthermore we may define by

119869119888

(119894) = 1 if there exists an 119903 isin 119885 such that 119894 = 119888 (119903)

= 0 otherwise(4)

the passage function associated with the directed circuit 119888 forany 119894 isin 119878 The above definitions are due toMacQueen [2] andKalpazidou [1]

Given a denumerable set 119878 and an infinite denumerableclass 119862 of overlapping directed circuits (or directed cycles)with distinct points (except for the terminals) in 119878 such thatall the points of 119878 can be reached from one another followingpaths of circuit edges that is for each two distinct points 119894

and 119895 of 119878 there exists a finite sequence 1198881

119888119896

119896 ge 1 ofcircuits (or cycles) of 119862 such that 119894 lies on 119888

1

and 119895 lies on 119888119896

and any pair of consecutive circuits (119888

119899

119888119899+1

) have at least onepoint in common Generally we may assume that the class 119862contains among its elements circuits (or cycles) with periodgreater or equal to 2

With each directed circuit (or directed cycle) 119888 isin 119862

let us associate a strictly positive weight 119908119888

which must beindependent of the choice of the representative of 119888 that isit must satisfy the consistency condition 119908cot119896 = 119908

119888

119896 isin 119885where 119905

119896

is the translation of length 119896 (that is 119905119896

(119899) equiv 119899 + 119896119899 isin 119885 for any fixed 119896 isin 119885)

For a given class 119862 of overlapping directed circuits (orcycles) and for a given sequence (119908

119888

)119888isin119862

of weights we maydefine by

119901119894119895

=sum119888isin119862

119908119888

sdot 119869(1)

119888

(119894 119895)

sum119888isin119862

119908119888

sdot 119869119888

(119894)(5)

the elements of a Markov transition matrix on 119878 if and onlyif sum119888isin119862

119908119888

sdot 119869119888

(119894) lt infin for any 119894 isin 119878 This means that agiven Markov transition matrix 119875 = (119901

119894119895

) 119894 119895 isin 119878 can berepresented by directed circuits (or cycles) and weights if andonly if there exists a class of overlapping directed circuits (orcycles) 119862 and a sequence of positive weights (119908

119888

)119888isin119862

suchthat the formula (5) holds In this case the Markov transitionmatrix 119875 has a unique stationary distribution 119901 which is asolution of 119901119875 = 119901 and is defined by

119901 (119894) = sum

119888isin119862

119908119888

sdot 119869119888

(119894) 119894 isin 119878 (6)

It is known that the following classes of Markov chainsmay be represented uniquely by circuits (or cycles) andweights

(i) the recurrent Markov chains (Minping and Min [3])(ii) the reversible Markov chains

3 Auxiliary Results

Let us consider aMarkov chain (119883119899

)119899ge0

onalefsymwith transitions119896 rarr (119896+1) 119896 rarr (119896minus1) and 119896 rarr 119896whose the elements ofthe corresponding Markov transition matrix are defined by

119875 (119883119899+1

= 0119883119899

= 0) = 1199030

119875 (119883119899+1

= 1119883119899

= 0) = 1199010

1199010

= 1 minus 1199030

119875 (119883119899+1

= 119896 + 1119883119899

= 119896) = 119901119896

119896 ge 1

119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

119896 ge 1

119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

119896 ge 1

(7)

such that 119901119896

+ 119902119896

+ 119903119896

= 1 0 le 119901119896

119903119896

le 1 for every 119896 ge 1 asit is shown in (Figure 1)

Assume that (119901119896

)119896ge0

and (119903119896

)119896ge0

are arbitrary fixedsequences with 0 le 119901

0

= 1 minus 1199030

le 1 0 le 119901119896

119903119896

le 1 for every119896 gt 1 If we consider the directed circuits 119888

119896

= (119896 119896 + 1 119896)1198881015840

119896

= (119896 119896)119896 ge 0 and the collections of weights (119908119888119896)119896ge0

and(119908119888

1015840

119896

)119896ge0

respectively then we may obtain that

119901119896

=119908119888119896

119908119888119896minus1

+ 119908119888119896+ 119908119888

1015840

119896

for every 119896 ge 1 (8)

with

1199010

=1199081198880

1199081198880+ 119908119888

1015840

0

(9)

Here the class 119862(119896) contains the directed circuits 119888119896

=

(119896 119896+1 119896) 119888119896minus1

= (119896minus1 119896 119896minus1) and 1198881015840

119896

= (119896 119896) Furthermorewe may define

119902119896

=119908119888119896minus1

119908119888119896minus1

+ 119908119888119896+ 119908119888

1015840

119896

for every 119896 ge 1 (10)

and 119903119896

= 119908119888

1015840

119896

(119908119888119896minus1

+119908119888119896+119908119888

1015840

119896

) such that 119901119896

+119902119896

+ 119903119896

= 1 forevery 119896 ge 1 with 119903

0

= 1 minus 1199010

= 119908119888

1015840

0

(1199081198880+ 119908119888

1015840

0

)

Journal of Probability and Statistics 3

1199030

1199010

11990211199031

1199011

11990221199032

1199012

11990231199033

middot middot middot 985747 minus 1

119903985747minus1

119901985747minus1

119902985747

985747

119903985747

middot middot middot0 1 2 3

Figure 1

The transition matrix 119875 = (119901119894119895

) with

119901119894119895

=suminfin

119896=0

119908119888119896sdot 119869(1)

119888119896

(119894 119895)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] for 119894 = 119895 (11)

119901119894119894

=

suminfin

119896=0

1199081198881015840119896

sdot 119869(1)

119888

1015840

119896

(119894 119894)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] (12)

where 119869(1)

119888119896

(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896

119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888

119896

and 119869119888

1015840

119896

(119894) = 1if 119894 is a point of the circuit 119888

1015840

119896

expresses the representationof the Markov chain (119883

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840

119899

)119899ge0

on alefsym whose elements of the corresponding Markovtransition matrix are defined by

119875 (1198831015840

119899+1

= 01198831015840

119899

= 0) = 1199031015840

0

119875 (1198831015840

119899+1

= 11198831015840

119899

= 0) = 1199021015840

0

1199021015840

0

= 1 minus 1199031015840

0

119875 (1198831015840

119899+1

= 119896 minus 11198831015840

119899

= 119896) = 1199011015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896) = 1199031015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896) = 1199021015840

119896

119896 ge 1

(13)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 0 lt 1199011015840

119896

1199021015840

119896

1199031015840

119896

le 1 for every 119896 ge 1as it is shown in (Figure 2)

Assume that (1199021015840

119896

)119896ge0

and (1199031015840

119896

)119896ge0

are arbitrary fixedsequences with 0 le 119902

1015840

0

= 1 minus 1199031015840

0

le 1 0 le 1199021015840

119896

1199031015840119896

le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840

119896

= (119896+1 119896 119896+1)119888101584010158401015840

119896

= (119896 119896) 119896 ge 0 and the collections of weights (119908119888

10158401015840

119896

)119896ge0

and (119908

119888

101584010158401015840

119896

)119896ge0

respectively then we may have

1199021015840

119896

=119908119888

10158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1 (14)

with

1199021015840

0

=119908119888

10158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(15)

Here the class 1198621015840

(119896) contains the directed circuits 11988810158401015840

119896

=

(119896 + 1 119896 119896 + 1) 11988810158401015840

119896minus1

= (119896 119896 minus 1 119896) and 119888101584010158401015840

119896

= (119896 119896)Furthermore we may define

1199011015840

119896

=119908119888

10158401015840

119896minus1

11990810158401015840119888119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1

1199031015840

119896

=119908119888

101584010158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

(16)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 for every 119896 ge 1 with

1199031015840

0

= 1 minus 1199021015840

0

=119908119888

101584010158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(17)

So the transition matrix 1198751015840

= (1199011015840

119894119895

) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883

1015840

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Consequently we have the following

Proposition 1 The Markov chain (119883119899

)119899ge0

defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights

Proof Let us consider the set of directed circuits 119888119896

= (119896 119896 +

1 119896) and 1198881015840

119896

= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888

119896minus1

119888119896

and 1198881015840

119896

andtwo circuits through 0 119888

0

11988810158400

The problem we have to manage is the definition of the

weightsWemay symbolize by119908119896

the weight119908119888119896of the circuit

c119896

and by1199081015840

119896

the weight119908119888

1015840

119896

of the circuit 1198881015840119896

for any 119896 ge 0Thesequences (119908

119896

)119896ge0

(1199081015840119896

)119896ge0

must be a solution of

119901119896

=119908119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199010

=1199080

1199080

+ 11990810158400

119903119896

=1199081015840

119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199030

=1199081015840

0

1199080

+ 11990810158400

119902119896

= 1 minus 119901119896

minus 119903119896

119896 ge 1

(18)

Let us take by 119887119896

= 119908119896

119908119896minus1

120574119896

= 1199081015840

119896

1199081015840

119896minus1

119896 ge 1As a consequence we may have

119887119896

=119901119896

119902119896

=119901119896

1 minus 119901119896

minus 119903119896

120574119896

=119903119896

119903119896minus1

119901119896minus1

119901119896

119887119896

for every 119896 ge 1

(19)

4 Journal of Probability and Statistics

119903998400

0

119902998400

1119902998400

0

119903998400

1

119901998400

1

119902998400

2

119903998400

2

119901998400

2119901998400

3119903998400

3

middot middot middot 985747 minus 1

119903998400

985747minus1

119901998400

985747

119902998400

985747minus1

985747

119903998400

985747

middot middot middot0 1 2 3

Figure 2

Given the sequences (119901119896

)119896ge0

and (119903119896

)119896ge0

it is clear that theabove sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

exist and are uniqueThismeans that the sequences (119908

119896

)119896ge0

and (1199081015840

119896

)119896ge0

are defineduniquely up to multiplicative constant factors by

119908119896

= 1199080

sdot 1198871

sdot sdot sdot 119887119896

1199081015840

119896

= 1199081015840

0

sdot 1205741

sdot sdot sdot 120574119896

(20)

(The unicity is understood up to the constant factors 1199080

1199081015840

0

)

Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840

119899

)119899ge0

defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights

Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908

10158401015840

119896

the weight 11990810158401015840

119888119896

of the circuit 11988810158401015840

119896

and by119908101584010158401015840

119896

the weight 119908101584010158401015840119888119896

of the circuit 119888101584010158401015840119896

for every 119896 ge 0 Thesequences (11990810158401015840

119896

)119896ge0

and (119908101584010158401015840

119896

)119896ge0

must be solutions of

1199021015840

119896

=11990810158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199021015840

0

=11990810158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199031015840

119896

=119908101584010158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199031015840

0

=119908101584010158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

119896 ge 1

(21)

By considering the sequences (119904119896

)119896

and (119905119896

)119896

where 119904119896

=

11990810158401015840

119896minus1

11990810158401015840

119896

119905119896

= 119908101584010158401015840

119896

119908101584010158401015840

119896minus1

119896 ge 1 we may obtain that

119904119896

=1 minus 1199021015840

119896

minus 1199031015840

119896

1199021015840119896

119905119896

=1199031015840

119896

1199031015840119896minus1

sdot1199021015840

119896

1199021015840119896minus1

sdot 119904119896

for every 119896 ge 1

(22)

For given sequences (1199021015840

119896

)119896ge0

(1199031015840

119896

)119896ge0

it is obvious that(119904119896

)119896ge1

(119905119896

)119896ge1

exist and are unique for those sequences thatis the sequences (119908

10158401015840

119896

)119896ge0

(119908101584010158401015840119896

)119896ge0

are defined uniquely upto multiplicative constant factors by

11990810158401015840

119896

=11990810158401015840

0

1199041

sdot 1199042

sdot sdot sdot 119904119896

119908101584010158401015840

119896

= 119908101584010158401015840

0

sdot 1199051

1199052

sdot sdot sdot 119905119896

(23)

(The unicity is based to the constant factors119908101584010158400

1199081015840101584010158400

)

4 Recurrence and Transience of the MarkovChains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

We have that for the Markov chain (119883119899

)119899ge0

there is a uniqueinvariant measure up to a multiplicative constant factor 120583

119896

=

119908119896minus1

+ 119908119896

+ 1199081015840

119896

119896 ge 1 1205830

= 1199080

+ 1199081015840

0

while for the Markovchain (119883

1015840

119899

)119899ge0

1205831015840119896

= 11990810158401015840

119896minus1

+11990810158401015840

119896

+119908101584010158401015840

119896

119896 ge 1with1205831015840

0

= 11990810158401015840

0

+119908101584010158401015840

0

In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing

Proposition 3 (i)TheMarkov chain (119883119899

)119899ge0

defined as aboveis positive recurrent if and only if

infin

sum

119896=1

1198871

sdot 1198872

sdot sdot sdot 119887119896

lt +infin (1199001199031

1199080

sdot

infin

sum

119896=1

119908119896

lt +infin)

infin

sum

119896=1

1205741

sdot 1205742

sdot sdot sdot 120574120581

lt +infin (1199001199031

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

lt +infin)

(24)

(ii) The Markov chain (1198831015840

119899

)119899ge0

defined as above is positiverecurrent if and only if

infin

sum

119896=1

1

1199041

sdot 1199042

sdot sdot sdot 119904119896

lt +infin (1199001199031

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

lt +infin)

infin

sum

119896=1

1199051

sdot 1199052

sdot sdot sdot 119905119896

= +infin (1199001199031

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin)

(25)

In order to obtain recurrence and transience criterionsfor the Markov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

we shall need thefollowing proposition (Karlin and Taylor [6])

Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent

Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892

119896

119896 ge 1) For the Markov chain (119883119899

)119899ge0

this is asolution of

1199010

sdot 1198921

+ 1199030

sdot 1198920

= 1198920

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119896 ge 1(26)

Journal of Probability and Statistics 5

Since Δ119892119896

= 119892119896

minus 119892119896minus1

for every 119896 ge 1 we obtain that

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot (Δ119892119896+1

+ 119892119896

) + 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot Δ119892119896+1

+ (119901119896

+ 119902119896

+ 119903119896

) sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

= 119892119896

119901119896

sdot Δ119892119896+1

minus 119902119896

sdot (119892119896

minus 119892119896minus1

) = 0

119901119896

sdot Δ119892119896+1

= 119902119896

sdot Δ119892119896

(27)

If we put 119898119896

= Δ119892119896

Δ119892119896+1

we get 119898119896

= 119901119896

119902119896

(with119901119896

= 1 minus 119902119896

minus 119903119896

) 119896 ge 1 which is the equation of definitionof the sequences (119904

119896

)119896ge1

and (119905119896

)119896ge1

(as a multiplicative factorof the (119904

119896

)119896ge1

) for the Markov chain (1198831015840

119899

)119899ge0

such that 1199021015840119896

=

119902119896

1199031015840119896

= 119903119896

for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883

119899

)119899ge0

are in correspondence with the weight representations of theMarkov chain (119883

1015840

119899

)119899ge0

such that

1199021015840

119896

= 119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

1199031015840

119896

= 119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

= 1 minus 119902119896

minus 119903119896

= 119901119896

for every 119896 ge 1

(28)

To express this kind of duality we will call the Markovchain (119883

1015840

119899

)119899ge0

the adjoint of the Markov chain (119883119899

)119899ge0

andreciprocally in the case that the relation (28) is satisfied

Equivalently for the Markov chain (1198831015840

119899

)119899ge0

the harmonicfunction 119892

1015840

= (1198921015840

119896

119896 ge 1) satisfies the equation

1199031015840

0

sdot 1198921015840

0

+ 1199021015840

0

sdot 1198921015840

1

= 1198921015840

0

1199021015840

119896

sdot 1198921015840

119896+1

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

119896 ge 1

(29)

Since Δ1198921015840

119896

= 1198921015840

119896

minus 1198921015840

119896minus1

for every 119896 ge 1 we have

1199021015840

119896

sdot (Δ1198921015840

119896+1

+ 1198921015840

119896

) + 1199011015840

119896

sdot 1198921015840

119896

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

(1199011015840

119896

+ 1199021015840

119896

+ 1199031015840

119896

) sdot 1198921015840

119896

+ 1199021015840

119896

sdot Δ1198921015840

119896+1

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

= 1198921015840

119896

1199021015840

119896

sdot Δ1198921015840

119896+1

= 1199011015840

119896

sdot (1198921015840

119896

minus 1198921015840

119896minus1

) = 1199011015840

119896

sdot Δ1198921015840

119896

(30)

If we put ℓ119896

= Δ1198921015840

119896+1

Δ1198921015840

119896

we get ℓ119896

= 1199011015840

119896

1199021015840

119896

(with1199021015840

119896

= 1minus1199011015840

119896

minus1199031015840

119896

) 119896 ge 1 which is the equation of the definitionof the sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

(as a multiplicative factorof the (119887

119896

)119896ge1

) for the Markov chain (119883119899

)119899ge0

such that 1199011015840119896

=

119901119896

1199031015840

119896

= 119903119896

for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899

)119899ge0

we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883

1015840

119899

)119899ge0

are in correspondencewith the weight representations of the Markov chain (119883

119899

)119899ge0

such that equivalent equations of (28) are satisfiedSo we may have the following

Proposition 5 TheMarkov chain (119883119899

)119899ge0

defined as above istransient if and only if the adjointMarkov chain (119883

1015840

119899

)119899ge0

is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are null recurrent simultaneouslyIn particular

(i) the Markov chain (119883119899

)119899ge0

defined as above is transientif and only if (111990810158401015840

0

) sdot suminfin

119896=1

11990810158401015840

119896

lt +infin and (1119908101584010158401015840

0

) sdot

suminfin

119896=1

119908101584010158401015840

119896

= +infin(ii) theMarkov chain (119883

1015840

119899

)119899ge0

defined as above is transientif and only if (1119908

0

) sdot suminfin

119896=1

119908119896

lt +infin and (11199081015840

0

) sdot

suminfin

119896=1

1199081015840

119896

lt +infin(iii) the adjointMarkov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are nullrecurrent if

1

1199080

sdot

infin

sum

119896=1

119908119896

=1

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

= +infin

1

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

=1

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin

(31)

Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3

Acknowledgment

C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion

References

[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995

[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981

[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982

[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991

[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999

[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

2 Journal of Probability and Statistics

directed circuit in 119878 The ordered sequence 119888 = (1198941

1198942

119894119903

)

associated with the given directed 119888 is called a directed cyclein 119878

A directed circuit may be considered as 119888 = (119888(119903) 119888(119903 +

1) 119888(119903 + V minus 1) 119888(119903 + V)) if there exists an 119903 isin 119885 such that1198941

= 119888(119903 + 0) 1198942

= 119888(119903 + 1) 119894V = 119888(119903 + V minus 1) 1198941

= 119888(119903 + V)where 119888 is a periodic function from 119885 to 119878

The corresponding directed cycle is defined by theordered sequence 119888 = (119888(119903) 119888(119903+1) 119888(119903+Vminus1))The values119888(119896) are the points of 119888 while the directed pairs (119888(119896) 119888(119896+1))119896 isin 119885 are the directed edges of 119888

The smallest integer 119901 equiv 119901(119888) ge 1 satisfying the equation119888(119903 + 119901) = 119888(119903) for all 119903 isin 119885 is the period of 119888 Adirected circuit 119888 such that 119901(119888) = 1 is called a loop (In thepresent work we will use directed circuits with distinct pointelements)

Let us also consider a directed circuit 119888 (or a directed cycle119888) with period 119901(119888) gt 1 Then we may define by

119869(119899)

119888

(119894 119895) = 1 if there exists an 119903 isin 119885 such that

119894 = 119888 (119903) 119895 = 119888 (119903 + 119899)

= 0 otherwise

(3)

the 119899-step passage function associatedwith the directed circuit119888 for any 119894 119895 isin 119878 119899 ge 1

Furthermore we may define by

119869119888

(119894) = 1 if there exists an 119903 isin 119885 such that 119894 = 119888 (119903)

= 0 otherwise(4)

the passage function associated with the directed circuit 119888 forany 119894 isin 119878 The above definitions are due toMacQueen [2] andKalpazidou [1]

Given a denumerable set 119878 and an infinite denumerableclass 119862 of overlapping directed circuits (or directed cycles)with distinct points (except for the terminals) in 119878 such thatall the points of 119878 can be reached from one another followingpaths of circuit edges that is for each two distinct points 119894

and 119895 of 119878 there exists a finite sequence 1198881

119888119896

119896 ge 1 ofcircuits (or cycles) of 119862 such that 119894 lies on 119888

1

and 119895 lies on 119888119896

and any pair of consecutive circuits (119888

119899

119888119899+1

) have at least onepoint in common Generally we may assume that the class 119862contains among its elements circuits (or cycles) with periodgreater or equal to 2

With each directed circuit (or directed cycle) 119888 isin 119862

let us associate a strictly positive weight 119908119888

which must beindependent of the choice of the representative of 119888 that isit must satisfy the consistency condition 119908cot119896 = 119908

119888

119896 isin 119885where 119905

119896

is the translation of length 119896 (that is 119905119896

(119899) equiv 119899 + 119896119899 isin 119885 for any fixed 119896 isin 119885)

For a given class 119862 of overlapping directed circuits (orcycles) and for a given sequence (119908

119888

)119888isin119862

of weights we maydefine by

119901119894119895

=sum119888isin119862

119908119888

sdot 119869(1)

119888

(119894 119895)

sum119888isin119862

119908119888

sdot 119869119888

(119894)(5)

the elements of a Markov transition matrix on 119878 if and onlyif sum119888isin119862

119908119888

sdot 119869119888

(119894) lt infin for any 119894 isin 119878 This means that agiven Markov transition matrix 119875 = (119901

119894119895

) 119894 119895 isin 119878 can berepresented by directed circuits (or cycles) and weights if andonly if there exists a class of overlapping directed circuits (orcycles) 119862 and a sequence of positive weights (119908

119888

)119888isin119862

suchthat the formula (5) holds In this case the Markov transitionmatrix 119875 has a unique stationary distribution 119901 which is asolution of 119901119875 = 119901 and is defined by

119901 (119894) = sum

119888isin119862

119908119888

sdot 119869119888

(119894) 119894 isin 119878 (6)

It is known that the following classes of Markov chainsmay be represented uniquely by circuits (or cycles) andweights

(i) the recurrent Markov chains (Minping and Min [3])(ii) the reversible Markov chains

3 Auxiliary Results

Let us consider aMarkov chain (119883119899

)119899ge0

onalefsymwith transitions119896 rarr (119896+1) 119896 rarr (119896minus1) and 119896 rarr 119896whose the elements ofthe corresponding Markov transition matrix are defined by

119875 (119883119899+1

= 0119883119899

= 0) = 1199030

119875 (119883119899+1

= 1119883119899

= 0) = 1199010

1199010

= 1 minus 1199030

119875 (119883119899+1

= 119896 + 1119883119899

= 119896) = 119901119896

119896 ge 1

119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

119896 ge 1

119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

119896 ge 1

(7)

such that 119901119896

+ 119902119896

+ 119903119896

= 1 0 le 119901119896

119903119896

le 1 for every 119896 ge 1 asit is shown in (Figure 1)

Assume that (119901119896

)119896ge0

and (119903119896

)119896ge0

are arbitrary fixedsequences with 0 le 119901

0

= 1 minus 1199030

le 1 0 le 119901119896

119903119896

le 1 for every119896 gt 1 If we consider the directed circuits 119888

119896

= (119896 119896 + 1 119896)1198881015840

119896

= (119896 119896)119896 ge 0 and the collections of weights (119908119888119896)119896ge0

and(119908119888

1015840

119896

)119896ge0

respectively then we may obtain that

119901119896

=119908119888119896

119908119888119896minus1

+ 119908119888119896+ 119908119888

1015840

119896

for every 119896 ge 1 (8)

with

1199010

=1199081198880

1199081198880+ 119908119888

1015840

0

(9)

Here the class 119862(119896) contains the directed circuits 119888119896

=

(119896 119896+1 119896) 119888119896minus1

= (119896minus1 119896 119896minus1) and 1198881015840

119896

= (119896 119896) Furthermorewe may define

119902119896

=119908119888119896minus1

119908119888119896minus1

+ 119908119888119896+ 119908119888

1015840

119896

for every 119896 ge 1 (10)

and 119903119896

= 119908119888

1015840

119896

(119908119888119896minus1

+119908119888119896+119908119888

1015840

119896

) such that 119901119896

+119902119896

+ 119903119896

= 1 forevery 119896 ge 1 with 119903

0

= 1 minus 1199010

= 119908119888

1015840

0

(1199081198880+ 119908119888

1015840

0

)

Journal of Probability and Statistics 3

1199030

1199010

11990211199031

1199011

11990221199032

1199012

11990231199033

middot middot middot 985747 minus 1

119903985747minus1

119901985747minus1

119902985747

985747

119903985747

middot middot middot0 1 2 3

Figure 1

The transition matrix 119875 = (119901119894119895

) with

119901119894119895

=suminfin

119896=0

119908119888119896sdot 119869(1)

119888119896

(119894 119895)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] for 119894 = 119895 (11)

119901119894119894

=

suminfin

119896=0

1199081198881015840119896

sdot 119869(1)

119888

1015840

119896

(119894 119894)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] (12)

where 119869(1)

119888119896

(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896

119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888

119896

and 119869119888

1015840

119896

(119894) = 1if 119894 is a point of the circuit 119888

1015840

119896

expresses the representationof the Markov chain (119883

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840

119899

)119899ge0

on alefsym whose elements of the corresponding Markovtransition matrix are defined by

119875 (1198831015840

119899+1

= 01198831015840

119899

= 0) = 1199031015840

0

119875 (1198831015840

119899+1

= 11198831015840

119899

= 0) = 1199021015840

0

1199021015840

0

= 1 minus 1199031015840

0

119875 (1198831015840

119899+1

= 119896 minus 11198831015840

119899

= 119896) = 1199011015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896) = 1199031015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896) = 1199021015840

119896

119896 ge 1

(13)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 0 lt 1199011015840

119896

1199021015840

119896

1199031015840

119896

le 1 for every 119896 ge 1as it is shown in (Figure 2)

Assume that (1199021015840

119896

)119896ge0

and (1199031015840

119896

)119896ge0

are arbitrary fixedsequences with 0 le 119902

1015840

0

= 1 minus 1199031015840

0

le 1 0 le 1199021015840

119896

1199031015840119896

le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840

119896

= (119896+1 119896 119896+1)119888101584010158401015840

119896

= (119896 119896) 119896 ge 0 and the collections of weights (119908119888

10158401015840

119896

)119896ge0

and (119908

119888

101584010158401015840

119896

)119896ge0

respectively then we may have

1199021015840

119896

=119908119888

10158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1 (14)

with

1199021015840

0

=119908119888

10158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(15)

Here the class 1198621015840

(119896) contains the directed circuits 11988810158401015840

119896

=

(119896 + 1 119896 119896 + 1) 11988810158401015840

119896minus1

= (119896 119896 minus 1 119896) and 119888101584010158401015840

119896

= (119896 119896)Furthermore we may define

1199011015840

119896

=119908119888

10158401015840

119896minus1

11990810158401015840119888119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1

1199031015840

119896

=119908119888

101584010158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

(16)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 for every 119896 ge 1 with

1199031015840

0

= 1 minus 1199021015840

0

=119908119888

101584010158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(17)

So the transition matrix 1198751015840

= (1199011015840

119894119895

) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883

1015840

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Consequently we have the following

Proposition 1 The Markov chain (119883119899

)119899ge0

defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights

Proof Let us consider the set of directed circuits 119888119896

= (119896 119896 +

1 119896) and 1198881015840

119896

= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888

119896minus1

119888119896

and 1198881015840

119896

andtwo circuits through 0 119888

0

11988810158400

The problem we have to manage is the definition of the

weightsWemay symbolize by119908119896

the weight119908119888119896of the circuit

c119896

and by1199081015840

119896

the weight119908119888

1015840

119896

of the circuit 1198881015840119896

for any 119896 ge 0Thesequences (119908

119896

)119896ge0

(1199081015840119896

)119896ge0

must be a solution of

119901119896

=119908119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199010

=1199080

1199080

+ 11990810158400

119903119896

=1199081015840

119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199030

=1199081015840

0

1199080

+ 11990810158400

119902119896

= 1 minus 119901119896

minus 119903119896

119896 ge 1

(18)

Let us take by 119887119896

= 119908119896

119908119896minus1

120574119896

= 1199081015840

119896

1199081015840

119896minus1

119896 ge 1As a consequence we may have

119887119896

=119901119896

119902119896

=119901119896

1 minus 119901119896

minus 119903119896

120574119896

=119903119896

119903119896minus1

119901119896minus1

119901119896

119887119896

for every 119896 ge 1

(19)

4 Journal of Probability and Statistics

119903998400

0

119902998400

1119902998400

0

119903998400

1

119901998400

1

119902998400

2

119903998400

2

119901998400

2119901998400

3119903998400

3

middot middot middot 985747 minus 1

119903998400

985747minus1

119901998400

985747

119902998400

985747minus1

985747

119903998400

985747

middot middot middot0 1 2 3

Figure 2

Given the sequences (119901119896

)119896ge0

and (119903119896

)119896ge0

it is clear that theabove sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

exist and are uniqueThismeans that the sequences (119908

119896

)119896ge0

and (1199081015840

119896

)119896ge0

are defineduniquely up to multiplicative constant factors by

119908119896

= 1199080

sdot 1198871

sdot sdot sdot 119887119896

1199081015840

119896

= 1199081015840

0

sdot 1205741

sdot sdot sdot 120574119896

(20)

(The unicity is understood up to the constant factors 1199080

1199081015840

0

)

Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840

119899

)119899ge0

defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights

Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908

10158401015840

119896

the weight 11990810158401015840

119888119896

of the circuit 11988810158401015840

119896

and by119908101584010158401015840

119896

the weight 119908101584010158401015840119888119896

of the circuit 119888101584010158401015840119896

for every 119896 ge 0 Thesequences (11990810158401015840

119896

)119896ge0

and (119908101584010158401015840

119896

)119896ge0

must be solutions of

1199021015840

119896

=11990810158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199021015840

0

=11990810158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199031015840

119896

=119908101584010158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199031015840

0

=119908101584010158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

119896 ge 1

(21)

By considering the sequences (119904119896

)119896

and (119905119896

)119896

where 119904119896

=

11990810158401015840

119896minus1

11990810158401015840

119896

119905119896

= 119908101584010158401015840

119896

119908101584010158401015840

119896minus1

119896 ge 1 we may obtain that

119904119896

=1 minus 1199021015840

119896

minus 1199031015840

119896

1199021015840119896

119905119896

=1199031015840

119896

1199031015840119896minus1

sdot1199021015840

119896

1199021015840119896minus1

sdot 119904119896

for every 119896 ge 1

(22)

For given sequences (1199021015840

119896

)119896ge0

(1199031015840

119896

)119896ge0

it is obvious that(119904119896

)119896ge1

(119905119896

)119896ge1

exist and are unique for those sequences thatis the sequences (119908

10158401015840

119896

)119896ge0

(119908101584010158401015840119896

)119896ge0

are defined uniquely upto multiplicative constant factors by

11990810158401015840

119896

=11990810158401015840

0

1199041

sdot 1199042

sdot sdot sdot 119904119896

119908101584010158401015840

119896

= 119908101584010158401015840

0

sdot 1199051

1199052

sdot sdot sdot 119905119896

(23)

(The unicity is based to the constant factors119908101584010158400

1199081015840101584010158400

)

4 Recurrence and Transience of the MarkovChains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

We have that for the Markov chain (119883119899

)119899ge0

there is a uniqueinvariant measure up to a multiplicative constant factor 120583

119896

=

119908119896minus1

+ 119908119896

+ 1199081015840

119896

119896 ge 1 1205830

= 1199080

+ 1199081015840

0

while for the Markovchain (119883

1015840

119899

)119899ge0

1205831015840119896

= 11990810158401015840

119896minus1

+11990810158401015840

119896

+119908101584010158401015840

119896

119896 ge 1with1205831015840

0

= 11990810158401015840

0

+119908101584010158401015840

0

In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing

Proposition 3 (i)TheMarkov chain (119883119899

)119899ge0

defined as aboveis positive recurrent if and only if

infin

sum

119896=1

1198871

sdot 1198872

sdot sdot sdot 119887119896

lt +infin (1199001199031

1199080

sdot

infin

sum

119896=1

119908119896

lt +infin)

infin

sum

119896=1

1205741

sdot 1205742

sdot sdot sdot 120574120581

lt +infin (1199001199031

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

lt +infin)

(24)

(ii) The Markov chain (1198831015840

119899

)119899ge0

defined as above is positiverecurrent if and only if

infin

sum

119896=1

1

1199041

sdot 1199042

sdot sdot sdot 119904119896

lt +infin (1199001199031

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

lt +infin)

infin

sum

119896=1

1199051

sdot 1199052

sdot sdot sdot 119905119896

= +infin (1199001199031

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin)

(25)

In order to obtain recurrence and transience criterionsfor the Markov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

we shall need thefollowing proposition (Karlin and Taylor [6])

Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent

Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892

119896

119896 ge 1) For the Markov chain (119883119899

)119899ge0

this is asolution of

1199010

sdot 1198921

+ 1199030

sdot 1198920

= 1198920

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119896 ge 1(26)

Journal of Probability and Statistics 5

Since Δ119892119896

= 119892119896

minus 119892119896minus1

for every 119896 ge 1 we obtain that

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot (Δ119892119896+1

+ 119892119896

) + 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot Δ119892119896+1

+ (119901119896

+ 119902119896

+ 119903119896

) sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

= 119892119896

119901119896

sdot Δ119892119896+1

minus 119902119896

sdot (119892119896

minus 119892119896minus1

) = 0

119901119896

sdot Δ119892119896+1

= 119902119896

sdot Δ119892119896

(27)

If we put 119898119896

= Δ119892119896

Δ119892119896+1

we get 119898119896

= 119901119896

119902119896

(with119901119896

= 1 minus 119902119896

minus 119903119896

) 119896 ge 1 which is the equation of definitionof the sequences (119904

119896

)119896ge1

and (119905119896

)119896ge1

(as a multiplicative factorof the (119904

119896

)119896ge1

) for the Markov chain (1198831015840

119899

)119899ge0

such that 1199021015840119896

=

119902119896

1199031015840119896

= 119903119896

for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883

119899

)119899ge0

are in correspondence with the weight representations of theMarkov chain (119883

1015840

119899

)119899ge0

such that

1199021015840

119896

= 119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

1199031015840

119896

= 119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

= 1 minus 119902119896

minus 119903119896

= 119901119896

for every 119896 ge 1

(28)

To express this kind of duality we will call the Markovchain (119883

1015840

119899

)119899ge0

the adjoint of the Markov chain (119883119899

)119899ge0

andreciprocally in the case that the relation (28) is satisfied

Equivalently for the Markov chain (1198831015840

119899

)119899ge0

the harmonicfunction 119892

1015840

= (1198921015840

119896

119896 ge 1) satisfies the equation

1199031015840

0

sdot 1198921015840

0

+ 1199021015840

0

sdot 1198921015840

1

= 1198921015840

0

1199021015840

119896

sdot 1198921015840

119896+1

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

119896 ge 1

(29)

Since Δ1198921015840

119896

= 1198921015840

119896

minus 1198921015840

119896minus1

for every 119896 ge 1 we have

1199021015840

119896

sdot (Δ1198921015840

119896+1

+ 1198921015840

119896

) + 1199011015840

119896

sdot 1198921015840

119896

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

(1199011015840

119896

+ 1199021015840

119896

+ 1199031015840

119896

) sdot 1198921015840

119896

+ 1199021015840

119896

sdot Δ1198921015840

119896+1

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

= 1198921015840

119896

1199021015840

119896

sdot Δ1198921015840

119896+1

= 1199011015840

119896

sdot (1198921015840

119896

minus 1198921015840

119896minus1

) = 1199011015840

119896

sdot Δ1198921015840

119896

(30)

If we put ℓ119896

= Δ1198921015840

119896+1

Δ1198921015840

119896

we get ℓ119896

= 1199011015840

119896

1199021015840

119896

(with1199021015840

119896

= 1minus1199011015840

119896

minus1199031015840

119896

) 119896 ge 1 which is the equation of the definitionof the sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

(as a multiplicative factorof the (119887

119896

)119896ge1

) for the Markov chain (119883119899

)119899ge0

such that 1199011015840119896

=

119901119896

1199031015840

119896

= 119903119896

for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899

)119899ge0

we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883

1015840

119899

)119899ge0

are in correspondencewith the weight representations of the Markov chain (119883

119899

)119899ge0

such that equivalent equations of (28) are satisfiedSo we may have the following

Proposition 5 TheMarkov chain (119883119899

)119899ge0

defined as above istransient if and only if the adjointMarkov chain (119883

1015840

119899

)119899ge0

is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are null recurrent simultaneouslyIn particular

(i) the Markov chain (119883119899

)119899ge0

defined as above is transientif and only if (111990810158401015840

0

) sdot suminfin

119896=1

11990810158401015840

119896

lt +infin and (1119908101584010158401015840

0

) sdot

suminfin

119896=1

119908101584010158401015840

119896

= +infin(ii) theMarkov chain (119883

1015840

119899

)119899ge0

defined as above is transientif and only if (1119908

0

) sdot suminfin

119896=1

119908119896

lt +infin and (11199081015840

0

) sdot

suminfin

119896=1

1199081015840

119896

lt +infin(iii) the adjointMarkov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are nullrecurrent if

1

1199080

sdot

infin

sum

119896=1

119908119896

=1

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

= +infin

1

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

=1

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin

(31)

Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3

Acknowledgment

C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion

References

[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995

[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981

[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982

[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991

[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999

[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Journal of Probability and Statistics 3

1199030

1199010

11990211199031

1199011

11990221199032

1199012

11990231199033

middot middot middot 985747 minus 1

119903985747minus1

119901985747minus1

119902985747

985747

119903985747

middot middot middot0 1 2 3

Figure 1

The transition matrix 119875 = (119901119894119895

) with

119901119894119895

=suminfin

119896=0

119908119888119896sdot 119869(1)

119888119896

(119894 119895)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] for 119894 = 119895 (11)

119901119894119894

=

suminfin

119896=0

1199081198881015840119896

sdot 119869(1)

119888

1015840

119896

(119894 119894)

suminfin

119896=0

[119908119888119896sdot 119869119888119896(119894) + 119908

119888

1015840

119896

sdot 119869119888

1015840

119896

(119894)] (12)

where 119869(1)

119888119896

(119894 119895) = 1 if 119894 119895 are consecutive points of the circuit119888119896

119869119888119896(119894) = 1 if 119894 is a point of the circuit 119888

119896

and 119869119888

1015840

119896

(119894) = 1if 119894 is a point of the circuit 119888

1015840

119896

expresses the representationof the Markov chain (119883

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Furthermorewe consider also the ldquoadjointrdquoMarkov chain(1198831015840

119899

)119899ge0

on alefsym whose elements of the corresponding Markovtransition matrix are defined by

119875 (1198831015840

119899+1

= 01198831015840

119899

= 0) = 1199031015840

0

119875 (1198831015840

119899+1

= 11198831015840

119899

= 0) = 1199021015840

0

1199021015840

0

= 1 minus 1199031015840

0

119875 (1198831015840

119899+1

= 119896 minus 11198831015840

119899

= 119896) = 1199011015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896) = 1199031015840

119896

119896 ge 1

119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896) = 1199021015840

119896

119896 ge 1

(13)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 0 lt 1199011015840

119896

1199021015840

119896

1199031015840

119896

le 1 for every 119896 ge 1as it is shown in (Figure 2)

Assume that (1199021015840

119896

)119896ge0

and (1199031015840

119896

)119896ge0

are arbitrary fixedsequences with 0 le 119902

1015840

0

= 1 minus 1199031015840

0

le 1 0 le 1199021015840

119896

1199031015840119896

le 1 for every119896 ge 1 If we consider the directed circuits 11988810158401015840

119896

= (119896+1 119896 119896+1)119888101584010158401015840

119896

= (119896 119896) 119896 ge 0 and the collections of weights (119908119888

10158401015840

119896

)119896ge0

and (119908

119888

101584010158401015840

119896

)119896ge0

respectively then we may have

1199021015840

119896

=119908119888

10158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1 (14)

with

1199021015840

0

=119908119888

10158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(15)

Here the class 1198621015840

(119896) contains the directed circuits 11988810158401015840

119896

=

(119896 + 1 119896 119896 + 1) 11988810158401015840

119896minus1

= (119896 119896 minus 1 119896) and 119888101584010158401015840

119896

= (119896 119896)Furthermore we may define

1199011015840

119896

=119908119888

10158401015840

119896minus1

11990810158401015840119888119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

for every 119896 ge 1

1199031015840

119896

=119908119888

101584010158401015840

119896

119908119888

10158401015840

119896minus1

+ 119908119888

10158401015840

119896

+ 119908119888

101584010158401015840

119896

(16)

such that 1199011015840119896

+ 1199021015840

119896

+ 1199031015840

119896

= 1 for every 119896 ge 1 with

1199031015840

0

= 1 minus 1199021015840

0

=119908119888

101584010158401015840

0

119908119888

10158401015840

0

+ 119908119888

101584010158401015840

0

(17)

So the transition matrix 1198751015840

= (1199011015840

119894119895

) with elementsequivalent to that given by the above-mentioned formulas(11) (12) expresses also the representation of the ldquoadjointrdquoMarkov chain (119883

1015840

119899

)119899ge0

by directed cycles (especially bydirected circuits) and weights

Consequently we have the following

Proposition 1 The Markov chain (119883119899

)119899ge0

defined as abovehas a unique representation by directed cycles (especially bydirected circuits) and weights

Proof Let us consider the set of directed circuits 119888119896

= (119896 119896 +

1 119896) and 1198881015840

119896

= (119896 119896) for every 119896 ge 0 since only the transitionsfrom 119896 to 119896 + 1 119896 to 119896 minus 1 and 119896 to 119896 are possible There arethree circuits through each point 119896 ge 1 119888

119896minus1

119888119896

and 1198881015840

119896

andtwo circuits through 0 119888

0

11988810158400

The problem we have to manage is the definition of the

weightsWemay symbolize by119908119896

the weight119908119888119896of the circuit

c119896

and by1199081015840

119896

the weight119908119888

1015840

119896

of the circuit 1198881015840119896

for any 119896 ge 0Thesequences (119908

119896

)119896ge0

(1199081015840119896

)119896ge0

must be a solution of

119901119896

=119908119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199010

=1199080

1199080

+ 11990810158400

119903119896

=1199081015840

119896

119908119896minus1

+ 119908119896

+ 1199081015840119896

119896 ge 1 with 1199030

=1199081015840

0

1199080

+ 11990810158400

119902119896

= 1 minus 119901119896

minus 119903119896

119896 ge 1

(18)

Let us take by 119887119896

= 119908119896

119908119896minus1

120574119896

= 1199081015840

119896

1199081015840

119896minus1

119896 ge 1As a consequence we may have

119887119896

=119901119896

119902119896

=119901119896

1 minus 119901119896

minus 119903119896

120574119896

=119903119896

119903119896minus1

119901119896minus1

119901119896

119887119896

for every 119896 ge 1

(19)

4 Journal of Probability and Statistics

119903998400

0

119902998400

1119902998400

0

119903998400

1

119901998400

1

119902998400

2

119903998400

2

119901998400

2119901998400

3119903998400

3

middot middot middot 985747 minus 1

119903998400

985747minus1

119901998400

985747

119902998400

985747minus1

985747

119903998400

985747

middot middot middot0 1 2 3

Figure 2

Given the sequences (119901119896

)119896ge0

and (119903119896

)119896ge0

it is clear that theabove sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

exist and are uniqueThismeans that the sequences (119908

119896

)119896ge0

and (1199081015840

119896

)119896ge0

are defineduniquely up to multiplicative constant factors by

119908119896

= 1199080

sdot 1198871

sdot sdot sdot 119887119896

1199081015840

119896

= 1199081015840

0

sdot 1205741

sdot sdot sdot 120574119896

(20)

(The unicity is understood up to the constant factors 1199080

1199081015840

0

)

Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840

119899

)119899ge0

defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights

Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908

10158401015840

119896

the weight 11990810158401015840

119888119896

of the circuit 11988810158401015840

119896

and by119908101584010158401015840

119896

the weight 119908101584010158401015840119888119896

of the circuit 119888101584010158401015840119896

for every 119896 ge 0 Thesequences (11990810158401015840

119896

)119896ge0

and (119908101584010158401015840

119896

)119896ge0

must be solutions of

1199021015840

119896

=11990810158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199021015840

0

=11990810158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199031015840

119896

=119908101584010158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199031015840

0

=119908101584010158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

119896 ge 1

(21)

By considering the sequences (119904119896

)119896

and (119905119896

)119896

where 119904119896

=

11990810158401015840

119896minus1

11990810158401015840

119896

119905119896

= 119908101584010158401015840

119896

119908101584010158401015840

119896minus1

119896 ge 1 we may obtain that

119904119896

=1 minus 1199021015840

119896

minus 1199031015840

119896

1199021015840119896

119905119896

=1199031015840

119896

1199031015840119896minus1

sdot1199021015840

119896

1199021015840119896minus1

sdot 119904119896

for every 119896 ge 1

(22)

For given sequences (1199021015840

119896

)119896ge0

(1199031015840

119896

)119896ge0

it is obvious that(119904119896

)119896ge1

(119905119896

)119896ge1

exist and are unique for those sequences thatis the sequences (119908

10158401015840

119896

)119896ge0

(119908101584010158401015840119896

)119896ge0

are defined uniquely upto multiplicative constant factors by

11990810158401015840

119896

=11990810158401015840

0

1199041

sdot 1199042

sdot sdot sdot 119904119896

119908101584010158401015840

119896

= 119908101584010158401015840

0

sdot 1199051

1199052

sdot sdot sdot 119905119896

(23)

(The unicity is based to the constant factors119908101584010158400

1199081015840101584010158400

)

4 Recurrence and Transience of the MarkovChains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

We have that for the Markov chain (119883119899

)119899ge0

there is a uniqueinvariant measure up to a multiplicative constant factor 120583

119896

=

119908119896minus1

+ 119908119896

+ 1199081015840

119896

119896 ge 1 1205830

= 1199080

+ 1199081015840

0

while for the Markovchain (119883

1015840

119899

)119899ge0

1205831015840119896

= 11990810158401015840

119896minus1

+11990810158401015840

119896

+119908101584010158401015840

119896

119896 ge 1with1205831015840

0

= 11990810158401015840

0

+119908101584010158401015840

0

In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing

Proposition 3 (i)TheMarkov chain (119883119899

)119899ge0

defined as aboveis positive recurrent if and only if

infin

sum

119896=1

1198871

sdot 1198872

sdot sdot sdot 119887119896

lt +infin (1199001199031

1199080

sdot

infin

sum

119896=1

119908119896

lt +infin)

infin

sum

119896=1

1205741

sdot 1205742

sdot sdot sdot 120574120581

lt +infin (1199001199031

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

lt +infin)

(24)

(ii) The Markov chain (1198831015840

119899

)119899ge0

defined as above is positiverecurrent if and only if

infin

sum

119896=1

1

1199041

sdot 1199042

sdot sdot sdot 119904119896

lt +infin (1199001199031

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

lt +infin)

infin

sum

119896=1

1199051

sdot 1199052

sdot sdot sdot 119905119896

= +infin (1199001199031

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin)

(25)

In order to obtain recurrence and transience criterionsfor the Markov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

we shall need thefollowing proposition (Karlin and Taylor [6])

Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent

Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892

119896

119896 ge 1) For the Markov chain (119883119899

)119899ge0

this is asolution of

1199010

sdot 1198921

+ 1199030

sdot 1198920

= 1198920

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119896 ge 1(26)

Journal of Probability and Statistics 5

Since Δ119892119896

= 119892119896

minus 119892119896minus1

for every 119896 ge 1 we obtain that

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot (Δ119892119896+1

+ 119892119896

) + 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot Δ119892119896+1

+ (119901119896

+ 119902119896

+ 119903119896

) sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

= 119892119896

119901119896

sdot Δ119892119896+1

minus 119902119896

sdot (119892119896

minus 119892119896minus1

) = 0

119901119896

sdot Δ119892119896+1

= 119902119896

sdot Δ119892119896

(27)

If we put 119898119896

= Δ119892119896

Δ119892119896+1

we get 119898119896

= 119901119896

119902119896

(with119901119896

= 1 minus 119902119896

minus 119903119896

) 119896 ge 1 which is the equation of definitionof the sequences (119904

119896

)119896ge1

and (119905119896

)119896ge1

(as a multiplicative factorof the (119904

119896

)119896ge1

) for the Markov chain (1198831015840

119899

)119899ge0

such that 1199021015840119896

=

119902119896

1199031015840119896

= 119903119896

for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883

119899

)119899ge0

are in correspondence with the weight representations of theMarkov chain (119883

1015840

119899

)119899ge0

such that

1199021015840

119896

= 119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

1199031015840

119896

= 119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

= 1 minus 119902119896

minus 119903119896

= 119901119896

for every 119896 ge 1

(28)

To express this kind of duality we will call the Markovchain (119883

1015840

119899

)119899ge0

the adjoint of the Markov chain (119883119899

)119899ge0

andreciprocally in the case that the relation (28) is satisfied

Equivalently for the Markov chain (1198831015840

119899

)119899ge0

the harmonicfunction 119892

1015840

= (1198921015840

119896

119896 ge 1) satisfies the equation

1199031015840

0

sdot 1198921015840

0

+ 1199021015840

0

sdot 1198921015840

1

= 1198921015840

0

1199021015840

119896

sdot 1198921015840

119896+1

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

119896 ge 1

(29)

Since Δ1198921015840

119896

= 1198921015840

119896

minus 1198921015840

119896minus1

for every 119896 ge 1 we have

1199021015840

119896

sdot (Δ1198921015840

119896+1

+ 1198921015840

119896

) + 1199011015840

119896

sdot 1198921015840

119896

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

(1199011015840

119896

+ 1199021015840

119896

+ 1199031015840

119896

) sdot 1198921015840

119896

+ 1199021015840

119896

sdot Δ1198921015840

119896+1

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

= 1198921015840

119896

1199021015840

119896

sdot Δ1198921015840

119896+1

= 1199011015840

119896

sdot (1198921015840

119896

minus 1198921015840

119896minus1

) = 1199011015840

119896

sdot Δ1198921015840

119896

(30)

If we put ℓ119896

= Δ1198921015840

119896+1

Δ1198921015840

119896

we get ℓ119896

= 1199011015840

119896

1199021015840

119896

(with1199021015840

119896

= 1minus1199011015840

119896

minus1199031015840

119896

) 119896 ge 1 which is the equation of the definitionof the sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

(as a multiplicative factorof the (119887

119896

)119896ge1

) for the Markov chain (119883119899

)119899ge0

such that 1199011015840119896

=

119901119896

1199031015840

119896

= 119903119896

for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899

)119899ge0

we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883

1015840

119899

)119899ge0

are in correspondencewith the weight representations of the Markov chain (119883

119899

)119899ge0

such that equivalent equations of (28) are satisfiedSo we may have the following

Proposition 5 TheMarkov chain (119883119899

)119899ge0

defined as above istransient if and only if the adjointMarkov chain (119883

1015840

119899

)119899ge0

is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are null recurrent simultaneouslyIn particular

(i) the Markov chain (119883119899

)119899ge0

defined as above is transientif and only if (111990810158401015840

0

) sdot suminfin

119896=1

11990810158401015840

119896

lt +infin and (1119908101584010158401015840

0

) sdot

suminfin

119896=1

119908101584010158401015840

119896

= +infin(ii) theMarkov chain (119883

1015840

119899

)119899ge0

defined as above is transientif and only if (1119908

0

) sdot suminfin

119896=1

119908119896

lt +infin and (11199081015840

0

) sdot

suminfin

119896=1

1199081015840

119896

lt +infin(iii) the adjointMarkov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are nullrecurrent if

1

1199080

sdot

infin

sum

119896=1

119908119896

=1

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

= +infin

1

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

=1

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin

(31)

Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3

Acknowledgment

C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion

References

[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995

[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981

[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982

[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991

[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999

[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

4 Journal of Probability and Statistics

119903998400

0

119902998400

1119902998400

0

119903998400

1

119901998400

1

119902998400

2

119903998400

2

119901998400

2119901998400

3119903998400

3

middot middot middot 985747 minus 1

119903998400

985747minus1

119901998400

985747

119902998400

985747minus1

985747

119903998400

985747

middot middot middot0 1 2 3

Figure 2

Given the sequences (119901119896

)119896ge0

and (119903119896

)119896ge0

it is clear that theabove sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

exist and are uniqueThismeans that the sequences (119908

119896

)119896ge0

and (1199081015840

119896

)119896ge0

are defineduniquely up to multiplicative constant factors by

119908119896

= 1199080

sdot 1198871

sdot sdot sdot 119887119896

1199081015840

119896

= 1199081015840

0

sdot 1205741

sdot sdot sdot 120574119896

(20)

(The unicity is understood up to the constant factors 1199080

1199081015840

0

)

Proposition 2 The ldquoadjointrdquoMarkov chain (1198831015840

119899

)119899ge0

defined asabove has a unique representation by directed cycles (especiallyby directed circuits) and weights

Proof Following an analogous way of that given in the proofof Proposition 1 the problem we have also to manage hereis the definition of the weights To this direction we maysymbolize by 119908

10158401015840

119896

the weight 11990810158401015840

119888119896

of the circuit 11988810158401015840

119896

and by119908101584010158401015840

119896

the weight 119908101584010158401015840119888119896

of the circuit 119888101584010158401015840119896

for every 119896 ge 0 Thesequences (11990810158401015840

119896

)119896ge0

and (119908101584010158401015840

119896

)119896ge0

must be solutions of

1199021015840

119896

=11990810158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199021015840

0

=11990810158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199031015840

119896

=119908101584010158401015840

119896

11990810158401015840119896minus1

+ 11990810158401015840119896

+ 119908101584010158401015840119896

119896 ge 1 with 1199031015840

0

=119908101584010158401015840

0

119908101584010158400

+ 1199081015840101584010158400

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

119896 ge 1

(21)

By considering the sequences (119904119896

)119896

and (119905119896

)119896

where 119904119896

=

11990810158401015840

119896minus1

11990810158401015840

119896

119905119896

= 119908101584010158401015840

119896

119908101584010158401015840

119896minus1

119896 ge 1 we may obtain that

119904119896

=1 minus 1199021015840

119896

minus 1199031015840

119896

1199021015840119896

119905119896

=1199031015840

119896

1199031015840119896minus1

sdot1199021015840

119896

1199021015840119896minus1

sdot 119904119896

for every 119896 ge 1

(22)

For given sequences (1199021015840

119896

)119896ge0

(1199031015840

119896

)119896ge0

it is obvious that(119904119896

)119896ge1

(119905119896

)119896ge1

exist and are unique for those sequences thatis the sequences (119908

10158401015840

119896

)119896ge0

(119908101584010158401015840119896

)119896ge0

are defined uniquely upto multiplicative constant factors by

11990810158401015840

119896

=11990810158401015840

0

1199041

sdot 1199042

sdot sdot sdot 119904119896

119908101584010158401015840

119896

= 119908101584010158401015840

0

sdot 1199051

1199052

sdot sdot sdot 119905119896

(23)

(The unicity is based to the constant factors119908101584010158400

1199081015840101584010158400

)

4 Recurrence and Transience of the MarkovChains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

We have that for the Markov chain (119883119899

)119899ge0

there is a uniqueinvariant measure up to a multiplicative constant factor 120583

119896

=

119908119896minus1

+ 119908119896

+ 1199081015840

119896

119896 ge 1 1205830

= 1199080

+ 1199081015840

0

while for the Markovchain (119883

1015840

119899

)119899ge0

1205831015840119896

= 11990810158401015840

119896minus1

+11990810158401015840

119896

+119908101584010158401015840

119896

119896 ge 1with1205831015840

0

= 11990810158401015840

0

+119908101584010158401015840

0

In the case that an irreducible chain is recurrent there is onlyone invariant measure (finite or not) so we may obtain thefollowing

Proposition 3 (i)TheMarkov chain (119883119899

)119899ge0

defined as aboveis positive recurrent if and only if

infin

sum

119896=1

1198871

sdot 1198872

sdot sdot sdot 119887119896

lt +infin (1199001199031

1199080

sdot

infin

sum

119896=1

119908119896

lt +infin)

infin

sum

119896=1

1205741

sdot 1205742

sdot sdot sdot 120574120581

lt +infin (1199001199031

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

lt +infin)

(24)

(ii) The Markov chain (1198831015840

119899

)119899ge0

defined as above is positiverecurrent if and only if

infin

sum

119896=1

1

1199041

sdot 1199042

sdot sdot sdot 119904119896

lt +infin (1199001199031

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

lt +infin)

infin

sum

119896=1

1199051

sdot 1199052

sdot sdot sdot 119905119896

= +infin (1199001199031

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin)

(25)

In order to obtain recurrence and transience criterionsfor the Markov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

we shall need thefollowing proposition (Karlin and Taylor [6])

Proposition 4 Let us consider a Markov chain on alefsym whichis irreducible Then if there exists a strictly increasing functionthat is harmonic on the complement of a finite interval andthat is bounded then the chain is transient In the case thatthere exists such a function which is unbounded then the chainis recurrent

Following this direction we shall use a well-knownmethod-theorem based on the Foster-Kendall theorem (Kar-lin and Taylor [6]) by considering the harmonic function119892 = (119892

119896

119896 ge 1) For the Markov chain (119883119899

)119899ge0

this is asolution of

1199010

sdot 1198921

+ 1199030

sdot 1198920

= 1198920

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119896 ge 1(26)

Journal of Probability and Statistics 5

Since Δ119892119896

= 119892119896

minus 119892119896minus1

for every 119896 ge 1 we obtain that

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot (Δ119892119896+1

+ 119892119896

) + 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot Δ119892119896+1

+ (119901119896

+ 119902119896

+ 119903119896

) sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

= 119892119896

119901119896

sdot Δ119892119896+1

minus 119902119896

sdot (119892119896

minus 119892119896minus1

) = 0

119901119896

sdot Δ119892119896+1

= 119902119896

sdot Δ119892119896

(27)

If we put 119898119896

= Δ119892119896

Δ119892119896+1

we get 119898119896

= 119901119896

119902119896

(with119901119896

= 1 minus 119902119896

minus 119903119896

) 119896 ge 1 which is the equation of definitionof the sequences (119904

119896

)119896ge1

and (119905119896

)119896ge1

(as a multiplicative factorof the (119904

119896

)119896ge1

) for the Markov chain (1198831015840

119899

)119899ge0

such that 1199021015840119896

=

119902119896

1199031015840119896

= 119903119896

for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883

119899

)119899ge0

are in correspondence with the weight representations of theMarkov chain (119883

1015840

119899

)119899ge0

such that

1199021015840

119896

= 119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

1199031015840

119896

= 119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

= 1 minus 119902119896

minus 119903119896

= 119901119896

for every 119896 ge 1

(28)

To express this kind of duality we will call the Markovchain (119883

1015840

119899

)119899ge0

the adjoint of the Markov chain (119883119899

)119899ge0

andreciprocally in the case that the relation (28) is satisfied

Equivalently for the Markov chain (1198831015840

119899

)119899ge0

the harmonicfunction 119892

1015840

= (1198921015840

119896

119896 ge 1) satisfies the equation

1199031015840

0

sdot 1198921015840

0

+ 1199021015840

0

sdot 1198921015840

1

= 1198921015840

0

1199021015840

119896

sdot 1198921015840

119896+1

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

119896 ge 1

(29)

Since Δ1198921015840

119896

= 1198921015840

119896

minus 1198921015840

119896minus1

for every 119896 ge 1 we have

1199021015840

119896

sdot (Δ1198921015840

119896+1

+ 1198921015840

119896

) + 1199011015840

119896

sdot 1198921015840

119896

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

(1199011015840

119896

+ 1199021015840

119896

+ 1199031015840

119896

) sdot 1198921015840

119896

+ 1199021015840

119896

sdot Δ1198921015840

119896+1

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

= 1198921015840

119896

1199021015840

119896

sdot Δ1198921015840

119896+1

= 1199011015840

119896

sdot (1198921015840

119896

minus 1198921015840

119896minus1

) = 1199011015840

119896

sdot Δ1198921015840

119896

(30)

If we put ℓ119896

= Δ1198921015840

119896+1

Δ1198921015840

119896

we get ℓ119896

= 1199011015840

119896

1199021015840

119896

(with1199021015840

119896

= 1minus1199011015840

119896

minus1199031015840

119896

) 119896 ge 1 which is the equation of the definitionof the sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

(as a multiplicative factorof the (119887

119896

)119896ge1

) for the Markov chain (119883119899

)119899ge0

such that 1199011015840119896

=

119901119896

1199031015840

119896

= 119903119896

for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899

)119899ge0

we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883

1015840

119899

)119899ge0

are in correspondencewith the weight representations of the Markov chain (119883

119899

)119899ge0

such that equivalent equations of (28) are satisfiedSo we may have the following

Proposition 5 TheMarkov chain (119883119899

)119899ge0

defined as above istransient if and only if the adjointMarkov chain (119883

1015840

119899

)119899ge0

is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are null recurrent simultaneouslyIn particular

(i) the Markov chain (119883119899

)119899ge0

defined as above is transientif and only if (111990810158401015840

0

) sdot suminfin

119896=1

11990810158401015840

119896

lt +infin and (1119908101584010158401015840

0

) sdot

suminfin

119896=1

119908101584010158401015840

119896

= +infin(ii) theMarkov chain (119883

1015840

119899

)119899ge0

defined as above is transientif and only if (1119908

0

) sdot suminfin

119896=1

119908119896

lt +infin and (11199081015840

0

) sdot

suminfin

119896=1

1199081015840

119896

lt +infin(iii) the adjointMarkov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are nullrecurrent if

1

1199080

sdot

infin

sum

119896=1

119908119896

=1

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

= +infin

1

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

=1

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin

(31)

Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3

Acknowledgment

C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion

References

[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995

[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981

[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982

[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991

[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999

[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Journal of Probability and Statistics 5

Since Δ119892119896

= 119892119896

minus 119892119896minus1

for every 119896 ge 1 we obtain that

119901119896

sdot 119892119896+1

+ 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot (Δ119892119896+1

+ 119892119896

) + 119902119896

sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

+ 119903119896

sdot 119892119896

= 119892119896

119901119896

sdot Δ119892119896+1

+ (119901119896

+ 119902119896

+ 119903119896

) sdot 119892119896

minus 119902119896

sdot 119892119896

+ 119902119896

sdot 119892119896minus1

= 119892119896

119901119896

sdot Δ119892119896+1

minus 119902119896

sdot (119892119896

minus 119892119896minus1

) = 0

119901119896

sdot Δ119892119896+1

= 119902119896

sdot Δ119892119896

(27)

If we put 119898119896

= Δ119892119896

Δ119892119896+1

we get 119898119896

= 119901119896

119902119896

(with119901119896

= 1 minus 119902119896

minus 119903119896

) 119896 ge 1 which is the equation of definitionof the sequences (119904

119896

)119896ge1

and (119905119896

)119896ge1

(as a multiplicative factorof the (119904

119896

)119896ge1

) for the Markov chain (1198831015840

119899

)119899ge0

such that 1199021015840119896

=

119902119896

1199031015840119896

= 119903119896

for every 119896 ge 1 This means that the strictlyincreasing harmonic functions of the Markov chain (119883

119899

)119899ge0

are in correspondence with the weight representations of theMarkov chain (119883

1015840

119899

)119899ge0

such that

1199021015840

119896

= 119875 (1198831015840

119899+1

= 119896 + 11198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896 minus 1119883119899

= 119896) = 119902119896

1199031015840

119896

= 119875 (1198831015840

119899+1

= 1198961198831015840

119899

= 119896)

= 119875 (119883119899+1

= 119896119883119899

= 119896) = 119903119896

1199011015840

119896

= 1 minus 1199021015840

119896

minus 1199031015840

119896

= 1 minus 119902119896

minus 119903119896

= 119901119896

for every 119896 ge 1

(28)

To express this kind of duality we will call the Markovchain (119883

1015840

119899

)119899ge0

the adjoint of the Markov chain (119883119899

)119899ge0

andreciprocally in the case that the relation (28) is satisfied

Equivalently for the Markov chain (1198831015840

119899

)119899ge0

the harmonicfunction 119892

1015840

= (1198921015840

119896

119896 ge 1) satisfies the equation

1199031015840

0

sdot 1198921015840

0

+ 1199021015840

0

sdot 1198921015840

1

= 1198921015840

0

1199021015840

119896

sdot 1198921015840

119896+1

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

119896 ge 1

(29)

Since Δ1198921015840

119896

= 1198921015840

119896

minus 1198921015840

119896minus1

for every 119896 ge 1 we have

1199021015840

119896

sdot (Δ1198921015840

119896+1

+ 1198921015840

119896

) + 1199011015840

119896

sdot 1198921015840

119896

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

+ 1199031015840

119896

sdot 1198921015840

119896

= 1198921015840

119896

(1199011015840

119896

+ 1199021015840

119896

+ 1199031015840

119896

) sdot 1198921015840

119896

+ 1199021015840

119896

sdot Δ1198921015840

119896+1

minus 1199011015840

119896

sdot 1198921015840

119896

+ 1199011015840

119896

sdot 1198921015840

119896minus1

= 1198921015840

119896

1199021015840

119896

sdot Δ1198921015840

119896+1

= 1199011015840

119896

sdot (1198921015840

119896

minus 1198921015840

119896minus1

) = 1199011015840

119896

sdot Δ1198921015840

119896

(30)

If we put ℓ119896

= Δ1198921015840

119896+1

Δ1198921015840

119896

we get ℓ119896

= 1199011015840

119896

1199021015840

119896

(with1199021015840

119896

= 1minus1199011015840

119896

minus1199031015840

119896

) 119896 ge 1 which is the equation of the definitionof the sequences (119887

119896

)119896ge1

and (120574119896

)119896ge1

(as a multiplicative factorof the (119887

119896

)119896ge1

) for the Markov chain (119883119899

)119899ge0

such that 1199011015840119896

=

119901119896

1199031015840

119896

= 119903119896

for every 119896 ge 1 By considering a similarapproximation of that given before for the Markov chain(119883119899

)119899ge0

we may say that the strictly increasing harmonicfunctions of theMarkov chain (119883

1015840

119899

)119899ge0

are in correspondencewith the weight representations of the Markov chain (119883

119899

)119899ge0

such that equivalent equations of (28) are satisfiedSo we may have the following

Proposition 5 TheMarkov chain (119883119899

)119899ge0

defined as above istransient if and only if the adjointMarkov chain (119883

1015840

119899

)119899ge0

is pos-itive recurrent and reciprocally Moreover the adjoint Markovchains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are null recurrent simultaneouslyIn particular

(i) the Markov chain (119883119899

)119899ge0

defined as above is transientif and only if (111990810158401015840

0

) sdot suminfin

119896=1

11990810158401015840

119896

lt +infin and (1119908101584010158401015840

0

) sdot

suminfin

119896=1

119908101584010158401015840

119896

= +infin(ii) theMarkov chain (119883

1015840

119899

)119899ge0

defined as above is transientif and only if (1119908

0

) sdot suminfin

119896=1

119908119896

lt +infin and (11199081015840

0

) sdot

suminfin

119896=1

1199081015840

119896

lt +infin(iii) the adjointMarkov chains (119883

119899

)119899ge0

and (1198831015840

119899

)119899ge0

are nullrecurrent if

1

1199080

sdot

infin

sum

119896=1

119908119896

=1

119908101584010158400

sdot

infin

sum

119896=1

11990810158401015840

119896

= +infin

1

11990810158400

sdot

infin

sum

119896=1

1199081015840

119896

=1

1199081015840101584010158400

sdot

infin

sum

119896=1

119908101584010158401015840

119896

= +infin

(31)

Proof The proof of Proposition 5 is an application mainly ofProposition 4 as well as of Proposition 3

Acknowledgment

C Ganatsiou is indebted to the referee for the valuablecomments which led to a significant change of the firstversion

References

[1] S Kalpazidou Cycle Representations of Markov ProcessesSpringer New York NY USA 1995

[2] J MacQueen ldquoCircuit processesrdquo Annals of Probability vol 9pp 604ndash610 1981

[3] Q Minping and Q Min ldquoCirculation for recurrent markovchainsrdquo Zeitschrift fur Wahrscheinlichkeitstheorie und Ver-wandte Gebiete vol 59 no 2 pp 203ndash210 1982

[4] A H Zemanian Infinite Electrical Networks Cambridge Uni-versity Press Cambridge UK 1991

[5] Yv Derriennic ldquoRandom walks with jumps in random envi-ronments (examples of cycle and weight representations)rdquo inProbability Theory and Mathematical Statistics Proceedings ofThe 7th Vilnius Conference 1998 B Grigelionis J Kubilius VPaulauskas V A Statulevicius and H Pragarauskas Eds pp199ndash212 VSP Vilnius Lithuania 1999

[6] S Karlin and H Taylor A First Course in Stochastic ProcessesAcademic Press New York NY USA 1975

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of