Solutions to group exercises

1. (a) Truncating the chain is equivalent to setting transition probabilities to any state in {M+1,...} to zero. Renormalizing the transition matrix to make it stochastic we get

(b) We have so

pij∗=

pij

pikk=0

M

∑pijπ i =pjiπ j

pij∗ pikk=0

M

∑ π i =pji∗ pjkk=0

M

∑ π j

and yielding detailed

balance for the truncated chain.

2. Let Zn = (X2n,X2n+1). Then Zn is a Markov chain with transition matrix

Let Tk= P(hit (0,1) before (1,0)|start at k).

First step analysis yields the equations

π i∗ =

πi pikk=0

M

∑

πl plkk=0

M

∑l=0

M

∑

P =

p02 p0q0 q0q1 q0p1

q1p0 q1q0 p1q1 p12

p02 p0q0 q0q1 q0p1

q1p0 q1q0 p1q1 p12

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

(b) If q0=p1=p we get p01,01=1/2, fair coin.3. (a)

so

whence the differential equation follows by letting .

(b) Since P1(t)=1-P0(t) we get

T00 =p02T00 +p0q0T01 +q0p1T11

T01 =q1p0T00 +q1q0 +p12T11

T11 =q1p0T00 +q1q0T01 +p12T11

P0 (t + Δt) =P0 (t)(1−αΔt+ o(Δt)) +P1(t)(βΔt+ o(Δt))P1(t+ Δt) =P1(t)(1−βΔt+ o(Δt)) +P0 (t)(αΔt+ o(Δt))

P0 (t + Δt) −P0 (t)Δt

=−αP0 (t) + βP1(t) + o(1)

Δt → 0

′P0 (t) = −(α + β)P0 (t) + β

which can either be solved directly, or one can check that the given solution satisfies the differential equation.

(c) Letting we get

This value as a starting distribution also yields a marginal distribution that is free of t, so it behaves like a stationary distribution (which we will define later).

t→ ∞ P0 (∞) =β

α +β

Announcement

MathAcrossCampus Colloquium(http://www.math.washington.edu/mac/)

Evolutionary trees, coalescents, and gene trees: can mathematicians find the woods?

JOE FELSENSTEIN Genome Sciences, UW

Thursday, November 13, 2008, 3:30 Kane Hall 210 Reception to follow

The Markov property

X(t) is a Markov process if for any n

for all j, i0,...,in in S and any t0<t1<...<tn<t.

The transition probabilities

are homogeneous if pij(s,t)=pij(0,t-s).

We will usually assume this, and write pij(t).

P(X(t) =j X(tn) =i0 , ...,X(t0 ) =in)

=P(X(t) =j X(tn) =i0 )

pij (s, t) =P(X(t) =j X(s) =i), s ≤t

Semigroup property

Let Pt be [pij(t)]. Then Pt is a substochastic semigroup, meaning that

• P0 = I

• Ps+t = PsPt

• Pt is a substochastic matrix, i.e. has nonnegative entries with row sums at most 1.

Proof

(a) ?(b)

(c)

pij (s + t) =P(X(s + t) =j X(0) =i)

= P(X(s + t) =j X(s) =k,X(0) =ik∈S∑ )

P(X(s) =k X(0) =i)= P(X(s + t) = j X(s) = k)P(X(s) = k X(0) = i)

k∈S∑

= pkj (t)pik (s)k∈S∑

pik (t) =P(X(t) ∈S X(0) =i) ≤1k∈S∑

Standard semigroup

Pt,t≥0 is a standard semigroup ifas .

Theorem:

For a standard semigroup the transition probabilities are continuous.

Proof:

By Chapman-Kolmogorov

Unless otherwise specified we will consider standard semigroups.

Pt → It ↓0

limt→ s Pt =limt→ s PsPt−s =PsI=Ps

Infinitesimal generator

By continuity of the transition probabilities, Taylor expansion suggests

We must have gij≥0, gii≤0. Let G=[gij].Then (under regularity conditions)

G is called the infinitesimal generator of Pt.

pij (h) ≈gijh, i≠j

pii (h) ≈1+giih

limh↓0

(Ph −I) / h=G

Birth process

G =

Under regularity conditions we have

so we must have

−λ0 λ0 0 L

0 −λ1 λ1 L

0 0 −λ2 λ2

L L L L

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

pij (h)j∈S∑ =1=1+h gij

j∈S∑ + o(h)

G1T =0T

Forward equations

so

and or

pij (t +h) = pik (t)pkj (h)k∈S∑

= pik (t)(gikh+ o(h))k≠j∑+pij (t)(1+gjjh+ o(h))

pij (t +h) −pij (t)h

= pik (t)gkj + o(1)k∈S∑

′pij (t) = pik (t)gkjk∈S∑ ′P (t) = P(t)G

Backward equations

Instead of looking at (t,t+h] look at (0,h]:

so

pij (t +h) = pik (h)pkj (t)k∈S∑

= pkj (t)(gikh+ o(h))k≠j∑+pjj (t)(1+gijh+ o(h))

′P (t) = GP(t)

Formal solution

In many cases we can solve both these equations by

But this can be difficult to actually calculate.

Pt =tn

n!Gn

n=0

∞

∑ ≡exp(tG)

The 0-1 case

G=−α αβ −β

⎛⎝⎜

⎞⎠⎟

det(G−λI) =(α + λ)(β + λ)−αβ=λ(α +β + λ) =0

λ =0 or − (α + β)

Gx =0 ⇒ x=11⎛⎝⎜⎞⎠⎟

(G+ (α + β)I)x=0 ⇒ x=−αβ

⎛⎝⎜

⎞⎠⎟

G=−α 1β 1

⎛⎝⎜

⎞⎠⎟

−(α +β) 00 0

⎛⎝⎜

⎞⎠⎟

−α 1β 1

⎛⎝⎜

⎞⎠⎟

−1

0-1 case, continued

Thus

Pt =tk

k!k=0

∞

∑ Gk

=I+−α 1β 1

⎛⎝⎜

⎞⎠⎟

tK

k!k=1

∞

∑ −(α +β)k 00 0

⎛

⎝⎜⎞

⎠⎟−α 1β 1

⎛⎝⎜

⎞⎠⎟

−1

=−α 1

β 1

⎛⎝⎜

⎞⎠⎟

e−(α+β)t 0

0 1

⎛

⎝⎜⎞

⎠⎟

−1

α + β

β

α + β

1

α + β

α

α + β

⎛

⎝

⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟

=1

α + β

αe−(α+β)t α(1− e−(α+β)t )

β(1− e−(α+β)t ) α + βe−(α+β)t

⎛

⎝⎜⎞

⎠⎟

Marginal distribution

Let . Then for a starting distribution π(0) we have π(t)=π(0)Pt.

For the 0-1 process we get

πk(t) = P(X t = k)

(π0(0 ) 1−π0

(0 ) )1

α + βαe−(α+β)t α(1−e−(α+β)t )

β(1−e−(α+β)t ) α + βe−(α+β)t

⎛

⎝⎜⎞

⎠⎟

=(β

α +β+ π0

(0 ) −β

α +β⎧⎨⎩

⎫⎬⎭e−(α+β)t 1−L )

Exponential holding times

Suppose X(t)=j. Consider

Let be the time spent in j until the next transition after time t. By the Markov property, P(stay in j in (u,u+v], given stays at least u) is precisely P(stay in j v time units). Mathematically

Let g(v)=P(>v). Then we have g(u+v)=g(u)g(v), and it follows that g(u)=exp(-λu). By the backward eqn

and P(>v)=pjj(v).

P(X(t +u) =j,0 < u≤α X(t) =j)

P( > u+ v > u) =P( > v)

′p jj (0) = gjj

Jump chain

Given that the chain jumps from i at a particular time, the probability that it jumps to j is -gij/gii.

Here is why (roughly):

Suppose t<<t+h, and there is only one jump in (t,t+h] (likely for small h). Then

P(X(t +h) =j X(t) =i, jump in (t, t+h])

==pij (h) + o(h)

1−pii (h) + o(h)→ −

gijgii

Construction

The way the continuous time Markov chains work is:

(1) Draw an initial value i0 from π(0)

(2) If , stay in i0 for a random time which is

(3) Draw a new state from the distribution where

gi0i0< 0

Exp(−gi0i0 )

Ri0 ,k R jk =−gjk / gjj

Death process

Let gi,i-1 = i = - gi,i. The forward equation is

Write . Then

This is a Lagrange equation with sln

or

′pNk (t) = −μkpNk (t) + μ(k + 1)pN,k+1(t)

G(s; t) = skpNk (t)∑∂G(s; t)

∂t= −μ kskpNk (t) +∑ μ (k + 1)skpN,k+1(t)∑

=−s∂G(s; t)

∂s+ μ

∂G(s; t)

∂s

G(s; t) =(1−(1−s)e−t )N

pNk (t) =N

k⎛⎝⎜⎞⎠⎟e−kt (1−e−t )N−k