Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Basic Theory For SDE’s
W. C. Troy∗
July 11, 2008Table of Contents(1.) Background: general properties.
(a.) The Cauchy-Schwarz, Chebychev and Markov inequalities.
(b.) How to get the ODE associated with PX ≤ x+ t|X > t = P0 < X ≤ x.(c.) Stationary and WSS (wide sense stationary) random processes.
(d.) If X(t) is a WSS process then E (X(t)) = µ is constant.
(e.) The autocorellation function: if X(t) is WSS then E(X(t)X(s)) = RX(|t− s|).(f.) Important Example: white noise.
(g.) If X(t) is WSS then V ar (X(t)) = constant = σ2 = RX(0) − µ2.
(h.) Random processes with independent increments.
(i.) X(t) is mean square continuous at t ⇐⇒ RX(t, s) is continuous at s = t.
(j.) If X(t) is m. s. continuous at t then X(t) is continuous in probability at t.
(k.) First derivative properties of a random process.
(l.) Stochastic integral properties of a random process.
(m.) How to see where the greatest uncertainty of a process occurs?
(n.) The pdf’s for Z = Y +X and Z = Y −X.
(o.) Definition and computation of FX(x|a < X ≤ x).
(p.) Proof that ρ(x, t|y, t) = δ(x − y) when X(t) is continuous.
(q.) The interpretaion of∫
R1
∫
R2ρ(x′, t1, x′′, t2)dx′dx′′.
(r.) Principle X: Prob a < X ≤ b, c < Y ≤ d =∫ d
cProba < X ≤ b|Y = η fY (η)dη.
(2.) The Poisson process
(a.) Basic definition.
(b.) Proof that E(X(t)) = λt, E(X2(t)) = λt+ λ2t2 and V ar(X(t)) = λt.
(c.) Application: estimate the charge on the electron.
(d.) Proof that E(X(t)X(s)) = λmin(t, s) + λ2ts and Cov(X(t), X(s)) = λmin(t, s).
∗Department of Mathematics, University of Pittsburgh, Pittsburgh, PA 15260, U.S.A.
1
(e.) Variation, and coefficients of correlation and variation.
(f.) Proof that CV=1 for T1.
(g.) Proof that fTN (t) = λe−λt (λt)N−1
(N−1)! and E(T kN ) = N(N+1)..(N+k−1)
λk .
(h.) Principle X: Proba < X ≤ b, c < Y ≤ d =∫ d
c Prob a < X ≤ b|Y = η fY (η)dη.
(i.) Proof that ZN = TN − TN−1 is an exponential random variable.
(j.) If Z = TN − TM , N > M then fZ(z) = λN
(N−1−M)!e−λzzN−1−M .
(k.) Proof that Z2 = T2 − T1 and Z1 = T1 are independent.
(l.) Proof that Z = TN − TM , N > M and TM are independent.
(m.) Two proofs that E(y(t)) = λ for the spike train y(t) =∑∞
k=1 δ(t− Tk).
(n.) Two proofs that the spike train y(t) satisfies E(y(t)y(s)) = λ2 when s > t.
(o.) Proof that the spike train satisfies E(y(t)y(s)) = λδ(t − s) + λ2
(p.) Proof that y(t) satisfies E(y2(t)) = ∞ and V ar(y(t)) = ∞.
(q.) Proof that Probone spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s).
(3.) Spiking Theory
(a.) The Definition of Input Current.
(b.) Derivation of the correlation coefficient c.
(c.) The LIF model - Benji’s notes.
(d.) How to solve τ dVdt = (µ− V ) + σ dζ
dt .
(e.) How to analyze E(X(t)X(t+ τ) when X(t) solves an SDE?
(f.) Gardner’s eigenfunction approach to find ρ(x, t).
(g.) How to use the eigenfunction expansion to find ρ(x, t|x0, 0).
(h.) How to find the autocorrelation function from the eigenfunction expansion.
(i.) Example 1: the Ornstein-Uhlenbeck process.
(j.) The linear integrate and fire model.
(4.) The Wiener process W (t).
(a.) Start with conditional, then construct the single time and joint distributions of W (t).
(b.) Important properties ρ(w, 0) = δ(w) and ρ(w′, t|w, t) = δ(w′ − w).
(c.) Proof that ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ w(d.) Proof that E(W (t)W (s)) = min(t, s).
(e.) Proof that W (t) has independent increments.
(f.) Proof that W (t) −W (s) ∼√t− sN(0, 1) when t > s.
(g.) W (t) is continuous in mean square and in probability at time t.
(h.) Proof that W (t) does not have a mean square first derivative.
(i.) Proof that the generalized derivative of W (t) satisfies RW ′(t, s) = σ2δ(t− s).
(j.) Properties of the mean square integral Y =∫ t
t0W (α)dα.
(k.) Y (t) =∫ t
0X(η)dη is a Wiener process if X(t) is normal white noise.
(l.) Scaling: dWτ =√ǫdWt if τ = ǫt. (i.e. they have the same pdf’s)
2
(m.) dXt = f(Xt)dt+B(Xt)dWt becomes ǫdXτ = f(Xt)dτ +√ǫB(Xt)dWτ if τ = ǫt.
(5.) Doering’s derivation of the Fokker-Planck equation.
(a.) The stochastic ODE.
(b.) Evaluation of E(∆X(t)|X(t) = x) and E(
(∆X(t))2|X(t) = x)
.
(c.) The Chapman-Kolmogorov equation.
(d.) Use of the Chapman-Kolmogorov eq. to derive the Fokker-Planck eq.
(e.) Use of Ito’s Lemma to derive the Fokker-Planck eq.
(6.) Derivation of the Backwards Kolmogorov Equation.
(7.) Examples of the Fokker - Planck equation.
(a.) wienerSDEscalar.m solves the Wiener process SDE W ′ = σζ(t).
(b.) The Fokker-Planck pde for the Wiener process SDE.
(c.) The Ornstein-Uhlenbeck SDE X ′ = −γX + σζ(t).
(d.) The autocorellation and power functions for the Ornstein-Uhlenbeck process.
(e.) How to use the Ornstien-Uhlenbeck SDE to find ρ(x, t|y, s) (monahan.pdf notes)
(f.) The Fokker-Planck pde for systems of SDE’s.
(g.) How to find ρ(y, t|y0, 0) for dYdt = µ+ σζ(t), Y (0) = y0.
(8.) Ito’s Lemma for scalar equations and examples.
(a.) Informal proof of Ito’s Lemma.
(b.) Example 1. Solve dx(t)dt = λx+ µxζ(t) using Ito’s Lemma.
(c.) ItoExample1.m solves dx(t)dt = λx+ µxζ(t) and compares with the true sol.
(d.) Linear stability for dx(t)dt = λx+ µxζ(t)
(e.) higham3.m solves dx(t)dt = x+ .5xζ(t) 1000 times and graphs 5 solutions.
(f.) ItoExample2.m solves dx(t)dt = λx + µζ(t) and compares with the true sol.
(g.) Solve dx(t)dt = µx− x2 + σxζ(t) and find the Fokker-Planck eq. .
(h.) The Brownian Bridge SDE dx = k−x1−t + dW (t), x(0) = A
(i.) Numerical sol.and the Fokker-Planck pde for dx(t)dt = α− λx+ σ
√xζ(t)
(j.) Proof that∫ t
0 WNdW (s) = W N+1
N+1 − N2
∫ t
0 WN−1(s)ds and E
(
∫ t
0 WNdW (s)
)
= 0.
(k.) Proof that∫ t
0 g(s)dW (s) = g(t)W (t) −∫ t
0 W (s)dg(s) and E(
∫ t
0 g(s)dW (s))
= 0.
(l.) How to find ρ(V, t|v, 0) numerically from the SDE
(9.) A power law associated with dX = µXdt+ σXdW,X(0) = x0
(a.) Transform dX = µXdt+ σXdW,X(0) = x0 into dYdt = µ+ σζ(t), Y (0) = y0.
(b.) Find ρ(y, t|y0, 0) for dYdt = (µ− σ2
2 )dt+ σζ(t), Y (0) = y0.
(c.) Use moment generating functions to derive the power law.
(d.) The generalized power law.
(e.) The generalized power law when N=2
(f.) The generalized power law when N=3
3
(g.) A power law associated with the Gompertz equation.
(h.) The Gompertz equation with a delay.
(10.) Ito’s Lemma for systems and examples.
(a.) Stament of Ito’s Lemma for systems.
(b.) Example 1.∫ t
0etAdB(s) = etAB(t) −
∫ t
0AetAB(s)ds.
(c.) Example 2. Solve LQ′′ + RQ′ + 1CQ = G+ σζ(t)
(d.) Find the system of SDE’s satisfied by eix(t), where x(t) is the Wiener process.
(11.) Exit time problems following Oksendal
(a.) Use of Ito’s Lemma to derive the generator A of X(t).
(b.) Dynkin’s formula.
(c.) The exit time from a ball in RN for Brownian motion (exit1.m).
(d.) The Backward Kolmogorov Equation - Oksendal version.
(e.) The Backward Kolmogorov Equation - Wikipedia version.
(f.) The Feynman-Kac formula - Wikepedia version.
(g.) The Feynman-Kac formula - Oksendal version.
(12.) The first passage time - Gardner version
(a.) Basic ODE for first passage time. Exercises.
(b.) Basic ODE for Nth moment.
(c.) Example I. 1st and 2nd moments for diffusion.
(d.) Estimating integrals by the method of steepest descent.
(e.) Escape over a double-well potential barrier.
(f.) Why escape time is asymptotically an exponential random variable.
(g.) The probability, and mean time, of exit thru a particular end of [a, b].
(h.) Probability of exit thru x = 0 or x = 1 in dU = γU(1 − U)dt+ σ√
U(1 − UdW.
(i.) Example: finite time extinction in dx(t)dt = µx− x2 + σxpζ(t).
(k.) Stochastic Resonance - Benzi paper ideas (benzipaper1.pdf).
(13.) Large deviation theory - Freidlin-Wentzell theory
(a.) Maier’s notes on exit time asymptotics: santafewentzellnotes.pdf and maier2.pdf
(b.) Links between the Hamiltonian, Lagrangian and Action integral.
(c.) Example: dXt = 2λXt +√
2N b(X)dWt
(d.) Muratov - 2 emails, and paper Deville.pdf
(d.) Barbara Gentz estimates for large deviations - linuxmail.org
(14.) Continuum limits of Master equations.
(a.) Derivation of Ct = DCxx from C(x, t + τ) = λrC(x − ∆x, t) + λlC(x + ∆x, t).
4
(b.) Derivation of the Fokker-Planck equation for dU = γU(1 − U)dt+ σ√
U(1 − UdW.
(15.) Time of first passage problems for discrete problems.
(a.) Random walk: how to find the probability that XN = 0 for some N > 0.
(14.) White noise
(a.) Definition of white noise.
(b.) Example: uniform white noise.
(c.) Computing RX(τ) and SX(ω) for uniform white noise.
(16.) Research ideas and papers to get
(a.) Feynman extension.
(b.) Oscillations in predator-prey systems.
(c.) Neuro papers: Brunel-Hakim, Amit-Brunel, Brune
(d.) alternans
(17.) Use of the steady state pde
(18.) Population growth in a randomly varying environement.
(a.) The problem.
(b.) Put the extinction problem into the Central Limit Theorem setting.
(c.) Conditions which imply that E(Xt) → ∞, yet extinction occurs.
(d.) Proof that(
∏ti=1 li
)1/t
≤ 1t
∑ti=1 li when li ≥ 0.
(e.) A periodic sequence li such that ρ < 1 < λ and (Xt, Xt) → (0, 0) and as t→ ∞.
(f.) A periodic sequence li such that ρ = 1 < λ, Xt is periodic, Xt → constant as t→ ∞.
(19.) A noisy genetic swith model.
(20.) Coherence resonance in SIR (Kuske et al [13])
(a.) The problem.
(b.) Properties of the Bivariate normal distribution.
(c.) Properties of the Ornstein-Uhlenbeck process
1 Background: general properties.
(a.) The Cauchy-Schwarz, Chebychev and Markov inequalities.
The Cauchy-Schwarz inequality. Let X and Y be random vbariables. Then
|E (XY ) |2 ≤ E(X2)E(Y 2). (1.1)
5
To prove this we consider 0 ≤ E(
[X − sY ]2)
= E(X2) − 2sE(XY ) + s2E(Y 2). The first derivative
with respect to s of the right side is zero when s = E(XY )E(Y 2) . At this value we conclude that
0 ≤ E(X2) − (E(XY ))2
E(Y 2). (1.2)
From this a manipulation gives (3.461). It will be used below in part (h.).
Chebychev’s inequality. Let X be a random variable. If E(X) = µX and σ2 = V ar(X) exist then
P (|X − µX | ≥ a) ≤ σ2X
a2=V ar(X)
a2for each a > 0. (1.3)
The proof (Problem 2.35, p. 66 of Schaum’s) starts with
P (|X − µX | ≥ a) =
∫ µX−a
−∞fX(x)dx +
∫ ∞
µX+a
fX(x)dx =
∫
|x−µX |≥a
fX(x)dx. (1.4)
from the definition of σ2 we conclude that
σ2 =
∫ ∞
−∞(x− µx)2fX(x)dx ≥
∫
|x−µX |≥a
(x− µx)2fX(x)dx ≥ a2
∫
|x−µX |≥a
fX(x)dx. (1.5)
Combining (1.4) and (1.5) gives (1.3). It will be used below in part (i.).
Markov’s inequality. Let X be a random variable If fX = 0 for x ≤ 0 then
P (X ≥ a) ≤ µX
afor each a > 0. (1.6)
The proof (Problem 2.34, p. 66 of Schaum’s) starts with
µX =
∫ ∞
0
xfX(x)dx ≥∫ ∞
a
fX(x)dx ≥ a
∫ ∞
a
fX(x)dx. (1.7)
Thus,∫ ∞
a
fXdx = P (X ≥ a) ≤ µX
a. (1.8)
(b.) How to get the ODE associated with PX ≤ x+ t|X > t = P0 < X ≤ x.This follows [26], p. 73 - no. 2.53. Then PX ≤ x+ t|X > t = P0 < X ≤ x is equivalent to
PX ≤ x+ t and X > tPX > t = P0 < X ≤ x. (1.9)
which in turn leads to
PX ≤ x+ t and X > t = PX > tP0 < X ≤ x. (1.10)
Assume that X ≥ 0 and let FX(x) = P0 < X ≤ x. Then (1.10) becomes FX(x + t) − FX(x) =
(1 − FX(t))FX(x). Divide both sides by x and get
FX(x+ t) − FX(x)
x= (1 − FX(t))
FX(x)
x. (1.11)
6
Use the fact that FX(0) = 0 (since X ≥ 0) and let x→ 0 to get the ODE
F ′X(t) = F ′
X(0)(1 − FX(t)). (1.12)
Set µ = F ′X(0), solve this ODE and find that FX(t) and the associatd pdf fX(t) are
FX(t) = 1 − e−µt and fX(t) = F ′X(t) = µe−µt ∀t ≥ 0. (1.13)
(c.) Stationary and WSS (wide sense stationary) random processses.
A random process X(t) is a stationary (Schaum’s, p. 163) process if, for each set ti|i = 1, 2, .., n,
fX(x1, t1, x2, t2, .., xn, tn) = fX(x1, t1 + γ, x2, t2 + γ, .., xn, tn + γ) ∀γ. (1.14)
If the stationary condition holds only for n ≤ k, then the process X(t) is stationary to order k. If X(t)
is stationary to order 2, then X(t) is called a WSS (wide sense stationary) process, and therefore
fX(x1, t1) = fX(x1, t1 + τ) and fX(x1, t1, x2, t2) = fX(x1, t1 + η, x2, t2 + η) ∀t1, t2, τ and η (1.15)
Setting τ = −t1 and η = −t2 reduces (1.15) to
fX(x1, t1) = fX(x1, 0) and fX(x1, t1, x2, t2) = fX(x1, t1 − t2, x2, 0) ∀t1 and t2. (1.16)
Alternatively, we could have let η = −t1, in which case the second part of (1.15) reduces to
fX(x1, t1, x2, t2) = fX(x1, 0, x2, t2 − t1) ∀t1, t2. (1.17)
Thus, we conclude that if X(t) is a WSS process then
fX(x1, t1, x2, t2) = fX(x1, x2, |t2 − t1|) ∀t1, t2. (1.18)
(d.) If X(t) is a WSS process then E(X(t)) = µ is constant.
It follows from the first part of (1.17), and the definition of mean, that
E(X(t) =
∫ ∞
−∞x1fX(x1, 0)dx = µ ∀t. (1.19)
(e.) The autocorellation function: if X(t) is WSS then E(X(t)X(s)) = RX(|t− s|)
It follows from (1.18) that the autocoreelation function RX(t, s) satisfies
RX(t, s) = E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞x1x2f(x1, x2, |t− s|)dx1dx2 = RX(|t− s|) ∀t, s. (1.20)
This and (1.20) give the following result which we will use:
RX(t, t+ τ) = E (X(t)X(t+ τ)) = RX(τ) ∀t, τ. (1.21)
7
A continuous time WSS random variable X(t) is called white noise if
µX(t) = 0 and RX(τ) = σ2δ(τ) ∀τ ∈ R. (1.22)
In this case the frequency power spectrum SX(ω) satisfies
SX(ω) =
∫ ∞
−∞σ2δ(τ)dτ = σ2 ∀ω ∈ R. (1.23)
Since SX(ω) ≡ σ2, all frequencies contribute equally to power, hence X(t) is white noise. Section 19
gives further details about the connection between the uniform distribution and white noise.
(g.) If X(t) is WSS then V ar (X(t)) = constant = σ2 = RX(0) − µ2.
The variance is defined by
V ar (X(t)) = E(
[(X(t) − E(X(t))]2)
= E(
[X(t)]2)
− (E(X(t))2. (1.24)
Setting τ = 0 in (1.21), and using (1.19), we obtain
V ar (X(t)) = RX(0) − µ2 = constant = σ2 ∀t. (1.25)
(h.) Processes with independent, stationary increments.
Definition 1. A random process X(t), t ≥ 0 has independent increments if whenever 0 < t1 < t2 <
.. < tN , then X(0), X(t1) −X(0), X(t2) −X(t1), .., X(tN ) −X(tN−1) are independent.
Definition 2. If a random process X(t), t ≥ 0 has independent increments, and if X(t) − X(s)
has the same probability distribution as X(t + h) −X(s + h) ∀s, t, h ≥ 0, then X(t) is said to have
stationary independent increments (see Schaum’s, p. 163).
Basic properties. First, the assumption that X(t) has stationary, independent increments means
that if t > s then X(t) − X(s) has the same pdf as X(t + h) − X(s + h) ∀h. Setting h = −s, we
conclude that X(t) −X(s) and X(t− s) −X(0) have exactly the same pdf.
Next, if X(t), t ≥ 0 has stationary independent increments, and X(0) = 0, then
E (X(t)) = µX t ∀t ≥ 0, (1.26)
V ar (X(t)) = σ2t ∀t ≥ 0 and V ar (X(t) −X(s)) = V ar (X(t)) − V ar (X(s))∀t > s ≥ 0. (1.27)
Remarks. It follows from (1.19), (1.25), (1.26) and (1.27) that a process with stationary independent
increments is not a WSS process. The process with stationary independent increments that we are
interested in is the Wiener process which is described in the next section.
The proofs of (1.26)-(1.27) are from Problems 5.22 and 5.23, p. 181-182 of Schaum’s. First, define
f(t) = E (X(t)) = E (X(t) −X(0)) ∀t ≥ 0. (1.28)
8
Then
f(t+ s) = E (X(t+ s) −X(0)) = E (X(t+ s) −X(s) +X(s) −X(0)) (1.29)
The linearity of E implies that
f(t+ s) = E (X(t+ s) −X(s)) + (X(s) −X(0)) . (1.30)
Because X(t) has has stationary independent increments, this reduces to
f(t+ s) = E (X(t) −X(0)) + (X(s) −X(0)) = f(t) + f(s). (1.31)
The solution of f(t+ s) = f(t) + f(s) is f(t) = ct. Setting c = f(1) = E(X(1)) = µX gives
f(t) = E (X(t))) = µXt ∀t ≥ 0. (1.32)
Next, to prove (1.27) we define
g(t) = V ar (X(t)) = V ar (X(t) −X(0)) ∀t ≥ 0. (1.33)
Then
g(t+s) = V ar (X(t+ s) −X(s) +X(s) −X(0)) = V ar (X(t+ s) −X(s))+V ar (X(s) −X(0)) t > s ≥ 0.
(1.34)
This used the independent stationary increment property that X(t+ s) −X(s) and X(s)−X(0) are
independent, the property that V ar(Y +Z) = V ar(Y )+V ar(Z) if Y and Z are independent (Problem
4.43, pp 149-150 of Schaum’s). Since X(t+ s) −X(s) = X(t) −X(0), then (1.34) reduces to
g(t+ s) = V ar (X(t) −X(0)) + V ar (X(s) −X(0)) = g(t) + g(s) t > s ≥ 0. (1.35)
Thus, g(t + s) = g(t) + g(s) and as before it follows that g(t) = kt where k is a constant. Thus
V ar(X(t)) = kt, hence
V ar (X(t)) = σ2t, σ2 = V ar(1) ∀t ≥ 0. (1.36)
(i.) X(t) is m. s. continuous at t ⇐⇒ RX(t, s) is continuous at s = t.
Definition 1. A random process X(t) is mean square continuous at t ( Schaum’s, p. 209) if
limh→0
E([X(t+ h) −X(t)]2) = 0. (1.37)
Start with (Problem 6.1, p. 219 Schaum’s)
E(
[X(t+ h) −X(t)]2)
= E(
X2(t+ ǫ))
− 2E (X(t+ ǫ)X(t)) + E(
X2(t))
= RX(t+ ǫ, t+ ǫ) − 2RX(t+ ǫ, t) +RX(t, t).(1.38)
If RX(t, s) is continuous at s = t, then the right side of (1.38) tends to 0 as ǫ→ 0, hence (1.37) holds.
9
Next, suppose that X(t) is mean square continuous at t. Then
RX(t+ ǫ1, t+ ǫ2) −RX(t, t) = E (X(t+ ǫ1)X(t+ ǫ2)) − E(
X2(t))
= E ([X(t+ ǫ1) −X(t)] [X(t+ ǫ2) −X(t)])
+ E ([X(t+ ǫ1) −X(t)]X(t)) + E ([X(t+ ǫ2) −X(t)]X(t)) .
(1.39)
The Cauchy Schwarz inequality |E(XY )|2 ≤ E(X2)E(Y 2) applied to (1.39) implies that
|RX(t+ ǫ1, t+ ǫ2) −RX(t, t)|2 ≤ E(
[X(t+ ǫ1) −X(t)]2)
E(
[X(t+ ǫ2) −X(t)]2)
+ E(
[X(t+ ǫ1) −X(t)]2)
E(
X2(t))
+ E(
[X(t+ ǫ2) −X(t)]2)
E(
X2(t))
.(1.40)
The right side of (1.40) tends to 0 as (ǫ1, ǫ2) → 0, hence RX(t, t) is continuous.
(j.) If X(t) is m. s. continuous at t then X(t) is continuous in probability.
Definition. A process X(t) is continuous in probability (Schaum’s, Problem 5.62) at t if ∀ǫ > 0,
limh→0
P (|X(t+ h) −X(t)| > ǫ) = 0. (1.41)
Proof. We follow Problem 6.5, p. 220 of Schaum’s. First, recall Chebychev’s inequality (Problem
2.36, p. 66 of Schaum’s outline): Let X be a random variable such that µX and V ar(X) exist. Then
P (|X − µ| > a) ≤ σ2X
a2=V ar(X)
a2for each a > 0. (1.42)
Letting a = ǫ, we conclude from (1.42) that
P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t))| > ǫ) ≤ V ar(X(t+ h) −X(t))
ǫ2. (1.43)
To proceed further we need the following observation: if Z is a random variable with mean µ then
P (|Z − µ| > ǫ) =
∫ µ−ǫ
−∞f(z)dz +
∫ ∞
µ+ǫ
f(z)dz. (1.44)
Note that −|µ| ≤ µ and µ ≤ |µ|. Thus,
∫ −|µ|−ǫ
−∞f(z)dz +
∫ ∞
|µ|+ǫ
f(z)dz ≤∫ µ−ǫ
−∞f(z)dz +
∫ ∞
µ+ǫ
f(z)dz (1.45)
Translating (1.45) into probability terms gives
P (|Z| > |µ| + ǫ) ≤ P (|Z − µ| > ǫ) . (1.46)
Setting Z = X(t+ h) −X(t) and µ = µ(t+ h) + µ(t) in (1.46), we obtain
P (|X(t+ h) −X(t)| > |µ(t+ h) − µ(t)| + ǫ) ≤ P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t)| > ǫ) . (1.47)
To reduce this further we use the observe that if 0 < λ < ǫ then
P (|Z| > 2ǫ) ≤ P (|Z| > λ+ ǫ) . (1.48)
10
Let h > 0 be small enough that |µ(t+ h) − µ(t)| < ǫ. It then follows from as in (1.44)-(1.46) that
P (|X(t+ h) −X(t)| > 2ǫ) ≤ P (|X(t+ h) −X(t)| > |µ(t+ h) − µ(t)| + ǫ) . (1.49)
Combining (1.49) with (1.47), we conclude that
P (|X(t+ h) −X(t)| > 2ǫ) ≤ P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t)| > ǫ) . (1.50)
Finally, it follows from (1.43) and (1.48) that
P (|X(t+ h) −X(t)| > 2ǫ) ≤ V ar(X(t+ h) −X(t))
ǫ2. (1.51)
Replacing ǫ with ǫ2 in (1.51) gives
P (|X(t+ h) −X(t)| > ǫ) ≤ 4[V ar(X(t+ h) −X(t))]
ǫ2. (1.52)
Note that
V ar(X(t+ h) −X(t)) = E([X(t+ h) −X(t)]2) − (µ(t+ h) − µ(t))2 (1.53)
The continuity of µ(t), and the fact that X(t) is mean square continuous at t, imply that the right
side of (1.53) appoaches zero as h→ 0. This implies that
limh→0
P (|X(t+ h) −X(t)| > ǫ) = 0. (1.54)
(k.) First derivative properties of a random process
Definition. A random process X(t) has a mean square first derivative X ′(t) (p. 209 Schaum’s) if
limǫ→0
E
(
[
X(t+ ǫ) −X(t)
ǫ−X ′(t)
]2)
= 0. (1.55)
Property 1. If a random process X(t) has a mean square first derivative then
E (X ′(t)) = µ′X(t) ∀t ≥ 0. (1.56)
To prove this we follow (Schaum’s, p. 221, Problem 6.7) and obtain
E (X ′(t)) = limǫ→0E(
X(t+ǫ)−X(t)ǫ
)
= limǫ→0µX (t+ǫ)−µX (t)
ǫ = µ′X(t) ∀t ≥ 0.
(1.57)
Property 2. If a random process X(t) has a mean square first derivative then
RXX′(t, s) =∂RX(t, s)
∂s(1.58)
To prove this we follow Problem 6.7, p. 221 of Schaum’s and obtain
RXX′(t, s) = limǫ→0E(
X(t)X(s+ǫ)−X(s)ǫ
)
= limǫ→0E[X(t)X(s+ǫ)]−E[X(t)X(s)]
ǫ = ∂RX (t,s)∂s .
(1.59)
11
Property 3. If a WSS process X(t) has a mean square first derivative then
RXX′(t, t+ τ) =∂RX(s− t)
∂s|s=t+τ =
dRX(τ)
dτ. (1.60)
Property 4. If X(t) has a mean square first derivative then RX′(t, s) = ∂2RX(t,s)∂t∂s .
Remark. This property will be used to show that the Wiener process does not have a m. s. derivative.
To prove that RX′(t, s) = ∂2RX (t,s)∂t∂s start with Problem 6.7, p. 221 of Schaum’s and obtain
RX′(t, s) = E (X ′(t)X ′(t)) = limǫ→0
E
(
X(t+ ǫ) −X(t)
ǫX ′(t)
)
. (1.61)
This reduces to
RXX′(t, s) = limǫ→0E[X(t+ǫ)X′(s)]−E[X(t)X′(s)]
ǫ
= limǫ→0RXX′ (t+ǫ,s)−RXX′ (t,s)
ǫ |s=t = ∂2RX(t,s)∂t∂s .
(1.62)
Property 5. If a WSS process X(t) has a mean square first derivative then
RX′(t, t+ τ) =∂2RX(s− t)
∂t∂s|s=t+τ =
d2RX(τ)
dτ2. (1.63)
Property 6. If ∂2RX (t,s)∂t∂s |s=t exists then X(t) has a mean square first derivative.
The proof is in Porblem 6.6, p. 220 of Schaum’s.
Remarks. Our main application of these properties will be to the Wiener process W (t) which we will
show (in the next section) does not have a mean square first derivative. However, as we will see, W (t)
it will have a generalized function first derivative given by the delta function.
(l.) Stochastic integral properties of a random process.
Definition. Y (t) =∫ t
t0X(α)dα (p. 210 Schaum’s) is the mean square integral of X(t) if
lim∆ti→0
E
[
∫ t
t0
X(α)dα −∑
i
X(ti)∆ti
]2
= 0. (1.64)
Problem 6.11, p. 223 of Schaum’s proves that the mean square integral
∫ t
t0
X(α)dα exists ⇐⇒∫ t
t0
∫ t
t0
RX(α, β)dαdβ exists. (1.65)
(m.) How to see where the greatest uncertainty of a process occurs?
The greatest uncertainty occurs at the value of t where V ar(x(t)) is a maximum. For example, when
we solve the Brownian Bridge SDE, we find that if we impose x(0) = x(1) = 0 then E(x(t)) = 0 ∀t,and V ar(x(t)) = E(x2(t)) = t(1 − t). Thus, the maximum uncertainty occurs at t = 1/2.
(n.) The pdf’s for Z = Y +X and Z = Y −X.
12
First we follow ([26], p.123) and consider general Z = g(X,Y ). For a given z ∈ R define
DZ(z) = (x, y)|g(x, y) ≤ z. (1.66)
Let f(x, y) be the joint pdf for X and Y. Then the cdf FZ(z) is defined by
ProbZ ≤ z = FZ(z) =
∫ ∫
DZ(z)
f(x, y)dydx. (1.67)
If Z = Y +X then
DZ(z) = (x, y)| −∞ < x <∞, y ≤ z − x (1.68)
and
Prob Z ≤ z = FZ(z) =
∫ ∫
DZ (z)
f(x, y)dydx =
∫ ∞
−∞
∫ z−x
−∞f(x, y)dydx. (1.69)
The corresponding pdf for Z is the convolution
fZ(z) =d
dzProbZ ≤ z =
d
dzFZ(z) =
∫ ∞
−∞f(x, z − x)dx. (1.70)
If Z = Y −X then the pdf for Z is
fZ(z) =d
dzFZ(z) =
∫ ∞
−∞f(x, z + x)dx. (1.71)
Suppose that Y and X are functions of t and that Z = Y (t2) −X(t1). Then
fZ(z) =d
dz
∫ ∞
−∞
∫ z−x
−∞f(x, t1, y, t2)dydx =
∫ ∞
−∞f(x, t1, z + x, t2)dx. (1.72)
Remark. In order to make use of this formula we sometimes (see the Wiener process part) change
the order in the integrand and get
fZ(z) =
∫ ∞
−∞f(z + x, t2, x, t1)dx =
∫ ∞
−∞f(z + x, t2|x, t1)fX(x, t1)dx. (1.73)
(o.) Definition and computation of FX(x|a < X ≤ b).
This is from Schaum’s Outline, Ch. 2 - problems (2.50)-(2.53)). Recall that
P (X ≤ x) = FX(x) =
∫ x
−∞fX(η)dη, (1.74)
P (X ∈ (a, b]) = FX(b) − FX(a) =
∫ b
a
fX(η)dη, (1.75)
and the basic definition
FX(x|a < X ≤ b) = P (X ≤ x|a < X ≤ b) =P (X ≤ x and a < X ≤ b)
P (a < X ≤ b). (1.76)
To compute the right side of (1.76), start with
X ≤ x and a < X ≤ b ⇐⇒ X ∈ (−∞, x] ∩ (a, b]. (1.77)
13
Next, note that
(−∞, x] ∩ (a, b] =
φ, x ≤ a,
(a, x], a < x ≤ b,
(a, b], x > b.
(1.78)
Therefore,
X ∈ (−∞, x] ∩ (a, b] ⇐⇒
X ∈ φ, x ≤ a,
X ∈ (a, x], a < x ≤ b,
X ∈ (a, b], x > b.
(1.79)
From this we conclude that
P (X ∈ (−∞, x] ∩ (a, b]) =
P (X ∈ φ) = 0, x ≤ a,
P (X ∈ (a, x]) = FX(x) − FX(a), a < x ≤ b,
P (X ∈ (a, b]) = FX(b) − FX(a), x > b.
(1.80)
Finally, combine (1.76), (1.75) and (1.80), and get
FX(x|a < X ≤ b) =
0, x ≤ a,
FX (x)−FX(a)FX (b)−FX(a) , a < x ≤ b,
FX (b)−FX(a)FX (b)−FX(a) , x > b.
(1.81)
(p.) Proof that ρ(x, t|y, t) = δ(x− y) when X(t) is continuous
Assume that X(t) is a continuous random variable, and let φ ∈ C∞0 (R) be a test function with compact
support on an interval (a, b). To prove the result we need to show that
∫ ∞
−∞φ(x)ρ(x, t|y, t)dx =
∫ b
a
φ(x)ρ(x, t|y, t)dx = φ(y). (1.82)
Using Riemann sums, it follows that
∫ b
a
φ(x)ρ(x, t|y, t)dx = lim∆x→0
∑
i
φ(xi)ρ(xi, t|y, t)∆x. (1.83)
Consider the particular Riemann sum with the points xi defined by
x0 = y,
xi = y + i∆x, i 6= 0.
(1.84)
Then
14
∫ b
a φ(x)ρ(x, t|y, t)dx = lim∆x→0
∑i=Mi=−L φ(xi)ρ(xi, t|y, t)∆x
= φ(y)ρ(y, t|y, t)∆x+ lim∆x→0
∑i=−1i=−L φ(xi)ρ(xi, t|y, t)∆x
+ lim∆x→0
∑i=Mi=1 φ(xi)ρ(xi, t|y, t)∆x,
(1.85)
where M and L are large enough to insure that the interval (a, b) is contained in [y−L∆T, y+M∆t].
When i 6= 0 we observe that
φ(xi)ρ(xi, t|y, t)∆x = φ (xi) Prob
X(t) ∈ (xi − ∆x2 , xi + ∆x
2 )|X(t) = y
= 0, (1.86)
since X(t) = y /∈ (y + i∆x− ∆x2 , y + i∆x+ ∆x
2 ) when i 6= 0. We conclude that
∫∞−∞ φ(x)ρ(x, t|y, t)dx = lim∆x→0
∑i=Mi=−L φ(xi)ρ(xi, t|y, t)∆x
= lim∆x→0 φ(y)ρ(y, t|y, t)∆x
= lim∆x→0 φ (y) Prob
X(t) ∈ (y − ∆x2 , y + ∆x
2 )|X(t) = y
= φ(y)
(1.87)
since the condition X(t) = y implies that
Prob
X(t) ∈ (y − ∆x
2, y +
∆x
2)|X(t) = y
= 1 ∀∆x > 0.
(q.) The interpretation of∫
R1
∫
R2ρ(x′, t1, x′′, t2)dx′dx′′
According to Gardner [11], Ch. 5, we have the following interpreation:∫
R1
∫
R2ρ(x′, t1, x′′, t2)dx′dx′′ is the probability the particle is in R1 when t = t1, and the particle is
in R2 when t = t2.
(r.) Principle X: Proba < X ≤ b, c < Y ≤ d =∫ d
c Prob a < X ≤ b|Y = η fY (η)dη.
This principle provides a first step for proving results about Prob a < X ≤ b, c < Y ≤ d , which is
typically hard to do. For example, in the next section we use it to prove that, for the Poisson process,
(i) the interspike interval for the Poisson process is an exponential random,
(i) TN − TM and TM are independent random variables when N > M.
The proof of Principle X uses two basic concepts - Riemann sum and conditional probability:
Step I. Express Prob a < X ≤ b, c < Y ≤ d as a sum.
Let ∆η = (d−c)N > 0 and define ηi = c+ i∆η, 1 ≤ i ≤ N. Then
Proba < X ≤ b and c < Y ≤ d = Prob
a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η](1.88)
15
Step II To get a Riemann sum we use conditional probability and conclude that (2.29) reduces to
Proba < X ≤ b and c < Y ≤ d = Prob
a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b|Y ∈ [c, c+ i∆η]ProbY ∈ [c, c+ i∆η]
≈∑
i≥1 Proba < X ≤ b|Y = ηi fY (ηi)∆η
(1.89)
The last expression is a Riemann sum. Thus, in the limit as ∆t→ 0, we obtain the integral formula
Prob[a < X ≤ b and c < Y ≤ d] =
∫ d
c
Prob a < X ≤ b|Y = η fY (η)dη (1.90)
16
2 The Poisson Process
(a.) Basic definition This follows [26], pp. 170-171. A counting process X(t)|t ≥ 0 satisfies
(i) X(t) ≥ 0 ∀t ≥ 0, X(t) is integer valued, and X(0) = 0.
(ii) X(t) is the number of events in [0, t) ∀t ≥ 0, X(s) < X(t), 0 ≤ s < t
A counting process X(t) is a Poisson process with rate λ > 0 if
(iii) X(t) has stationary, independent increments, and
(iv) The number of X(t) in any interval (s, t+ s) of length t satisfies
Prob[X(s+ t) −X(s) = n] = e−λt (λt)n
n!∀t, s ≥ 0.
Example I. Let 0 < t < s. If ∆t, ∆s are small enough then
X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)
are independent random variables. Thus
Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = Prob X(t+ ∆t) −X(t) = 1Prob X(s+ ∆s) −X(s) = 1(2.1)
Evaluation of the right side of (2.1) gives
Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1 = 0 = λ2∆t∆se−λ(∆t+∆s). (2.2)
Similarly,
ProbX(t+ ∆t) −X(t) = 0 and X(s+ ∆s) −X(s)) = e−λ(∆t+∆s. (2.3)
(b.) Proof that E(X(t)) = λt, E(X2(t)) = λt+ λ2t2 and V ar(X(t)) = λt.
It follows from the basic definition that
E(X(t)) =∑∞
k=0 ke−λt (λt)k
k!
=∑
k≥1 e−λt (λt)k
(k−1)! = λt∑
k≥1 e−λt (λt)k−1
(k−1)!
= λt∑
j≥0 e−λt (λt)j
j! = λt.
(2.4)
Similarly,
E(X2(t)) =∑∞
k=0 k2e−λt (λt)k
k!
= λt∑
k≥1 ke−λt (λt)k−1
(k−1)! = λt∑
j≥0(1 + j)e−λt (λt)j
(j)!
= λt(
∑
j≥0 e−λt (λt)j
j! +∑
j≥0 je−λt (λt)j
j!
)
= λt(1 + λt).
(2.5)
17
Finally, combining (2.4) and (2.5), we obtain
V ar(X(t)) = E(X2(t)) − E2(X(t)) = λt(1 + λt) − λ2t2 = λt. (2.6)
(c.) Application: estimate the charge on the electron.
Let X denote the number of electrons passing a given point in time inteval [0, t]. Assume that X is a
Poisson random variable with λ > 0 denoting the mean value of X in [0, t]. Current is defined by
I = eX, (2.7)
where e is the charge on a single electron. Thus, current is a random variable which is a constant
multiple of the Poisson random variable X. We conclude that
E(I) = eλ and V ar(I) = V ar(eX) = e2λ.
This leads to the formula
e =V ar(I)
E(I).
Thus, the charge on the elctron can be computed by measuring the mean current and its variation at
a given spatial point.
Below we will show that the spike train for the Poisson process is
y(t) =
∞∑
k=1
δ(t− Tk), (2.8)
where Tk is a gamma random variable parameters λ and k. The mean of the spike train is
E(y(t)) =∞∑
k=1
E(δ(t− Tk)) =∞∑
k=1
∫ ∞
0
δ(η − Tk)fTk(η)dη =
∞∑
k=1
fTk(t). (2.9)
Since Tk is a gamma random variable parameters λ and k, we conclude that
E(y(t)) =∞∑
k=1
λ
(k − 1)!e−λt)(λt)k−1 = λ (2.10)
The Poisson process X(t) satisfies
X(t) =
∫ t
0
∞∑
k=1
δ(t− Tk) =
∫ t
0
y(η)dη. (2.11)
This implies that
X ′(t) =
∞∑
k=1
δ(t− Tk) = y(η). (2.12)
We conclude that the derivative of the Poisson process is the spike train, and that
E(y(t)) =d
dtE(X(t)) = E(X ′(t)) =
∞∑
k=1
E(δ(t− Tk)) = λ. (2.13)
18
Thus, for the current defined above in (2.7), we conclude that
dI
dt= eX ′(t) = e
∞∑
k=1
δ(t− Tk) = ey(η). (2.14)
(d.) Proof that E(X(t)X(s)) = λmin(t, s) + λ2ts and Cov(X(t), X(s)) = λmin(t, s).
Let t > s ≥ 0 and recall that X(t) −X(s) and X(t− s) have the same pdf. Thus,
E([X(t) −X(s)]2) = E([X(t− s)]2) = λ(t− s) + λ2(t− s)2. (2.15)
Also, an expansion gives
E([X(t) −X(s)]2) = E(X2(t)) + E(X2(s)) − 2E(X(t)X(s))
= λt+ λ2t2 + λs+ λ2s2 − 2E(X(t)X(s)).
(2.16)
Combining (2.15) and (2.16) gives the autocorrelation
E(X(t)X(s)) = λmin(t, s) + λ2ts. (2.17)
Next, from the definition of covariance we obtain
Cov(X(t), X(s)) = E(X(t)X(s)) −E(X(t))E(X(s)) = λmin(t, s) + λ2ts− λ2ts = λmin(t, s). (2.18)
(e.) The variation, and coefficients of correlation and variation.
From (2.17) it follows that E(X2(t)) = λt− λ2t2. Thus, the varaiation of X(t) is
σ2 = V ar(X(t)) = E(X2(t)) − E2(X(t)) =(
λt+ λ2t2)
− (λt)2 = λt. (2.19)
The coefficient of variation of two random variables X and Y is
ρ =Cov(X,Y )
√
V ar(X)√
V ar(Y ). if 0 < s ≤ t. (2.20)
According to Wikepedia, the covariance between two random variables X and Y is
Cov(X,Y ) = E(XY ) − E(X)E(Y ), (2.21)
and
Cov(aX, bY ) = abCov(X,Y ). (2.22)
According to Wikepedia, the autocovariance for the two random variables X(t) and X(s) is
KXX(t, s) = E(X(t)X(s)) − E(X(t))E(X(s)) if 0 < s ≤ t. (2.23)
For a wide sense stationary process, µ = E(X(t)) and
KXX(τ) = E(X(t)X(t+ τ)) − E(X(t))E(X(t+ τ)) = RXX(τ) − µ2 (2.24)
19
According to Wikepedia, cross correlation and Cov(X,Y ) are sometimes referred to as the same thing.
According to Wikepedia, the coefficient of variation between two random variables X and Y is
ρXY =Cov(X,Y )
σXσY, σ2
X = V ar(X). (2.25)
A nice discussion is in http : //www.tufts.edu/ gdallal/corr.htm where they tell you to always look at
scatter plots. The site http : //en.wikipedia.org/wiki/Image : Oldfaithful3.png has an interesting
scatterplot showing two different waiting times for the old faithful geyser eruption time.
According to Wikepedia, the sample correlation is
rXY =N∑
xiyi − xy√
N∑
x2i − (
∑
xi)2√
N∑
y2i − (
∑
yi)2(2.26)
According to http : //www.vias.org/tmdatanaleng/cccorrcoeff.html, for a sample correlation we
need the following assumptions on the random variables x and y are:
*(i) linear relationship between x and y
(ii) continuous random variables and both variables must be normally distributed
(iii) x and y must be independent of each other
According to Wikepedia, the coefficient of variation is
CV =
√
V ar(X(t)
E(X(t))=
√λt
λt=
1√λt
if 0 < s ≤ t. (2.27)
(f.) Proof that CV=1 for T1.
Recall that T1 is an exponential random variable with
fT1(t) = λexp(−λt),
hence
E(T1) =
∫ ∞
0
tfT1(t)dt =1
λand E(T 2
1 ) =
∫ ∞
0
t2fT1(t)dt =2
λ2.
Therefore,
CV =
√
V ar(T1)
E(T1)= 1 (2.28)
(g.) Proof that fTN (t) = λe−λt (λt)N−1
(N−1)! and E(T kN ) = N(N+1)..(N+k−1)
λk .
(i) The N-th passage time is the random variable TN which is defined by
TN = inft > 0|X(t) > N ∀N ≥ 1.
To find the pdf of TN , we let t ≥ 0 and compute
Prob[TN > t] = Prob[X(t) ≤ N − 1] =N−1∑
i=0
e−λt (λt)i
i!.
20
Thus,
Prob[TN ≤ t] = 1 −N−1∑
i=0
e−λt (λt)i
i!,
and therefore
fTN (t) =d
dtProb[TN ≤ t] = λe−λt (λt)N−1
(N − 1)!.
(ii) To show that E(T kN ) = N(N+1)..(N+k−1)
λk we use the moment generating function and compute
E(eζTN ) =
∫ ∞
0
eζtfTN (t)dt =
∫ ∞
0
eζtλe−λt (λt)N−1
(N − 1)!dt
This gives
E(eζTN ) =λN
Γ(N)
∫ ∞
0
tN−1e(ζ−λ)tdt =
(
λ
ζ − λ
)N
, ζ < λ.
Divide top and bottom by λ, use the binomial expansion and obtain
E(eζTN ) =
∞∑
i=0
N(N + 1)..(N + k − 1)
λk
ζk
k!
Fromn this we conclude that
E(T kN) =
N(N + 1)..(N + k − 1)
λk
(h.) Principle X: Prob a < X ≤ b, c < Y ≤ d =∫ d
c Proba < X ≤ b|Y = η fY (η)dη.
This principle, which is repeated from the previous section, provides a very important and useful
first step for proving results about Prob a < X ≤ b, c < Y ≤ d , which is typically hard to do. For
example, below we use it to prove that, for the Poisson process,
(i) the interspike interval for the Poisson process is an exponential random,
(i) TN − TM and TM are independent random variables when N > M.
The proof of Principle X uses two basic concepts - Riemann sum and conditional probability:
Step I. Express Prob a < X ≤ b, c < Y ≤ d as a sum.
Let ∆η = (d−c)N > 0 and define ηi = c+ i∆η, 1 ≤ i ≤ N. Then
Proba < X ≤ b and c < Y ≤ d = Prob
a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η](2.29)
Step II To get a Riemann sum we use conditional probability and conclude that (2.29) reduces to
Proba < X ≤ b and c < Y ≤ d = Prob
a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η]
=∑
i≥1 Proba < X ≤ b|Y ∈ [c, c+ i∆η]ProbY ∈ [c, c+ i∆η]
≈∑i≥1 Proba < X ≤ b|Y = ηi fY (ηi)∆η
(2.30)
21
The last expression is a Riemann sum. Thus, in the limit as ∆t→ 0, we obtain the integral formula
Prob[a < X ≤ b and c < Y ≤ d] =
∫ d
c
Prob a < X ≤ b|Y = η fY (η)dη (2.31)
(i.)Proof that ZN = TN − TN−1 is an exponential random variable.
Our goal is to prove that
Prob ZN = TN − TN−1 ≤ z = 1 − e−λz ∀z ≥ 0.
The proof uses (i.) the method of proof of Principle X, and (ii.) the key ”probability link”
ProbTN − TN−1 > z|TN−1 = τ = Prob X(z + τ) −X(τ) = 0 = e−λz. (2.32)
First, we follow the method of proof of Principle X and obtain and integral formula for ProbZN ≤ z .Let ∆τ > 0 be small and define
τi = i∆τ, i ≥ 0.
Note that the events ZN > z and ZN > z and TN−1 ∈ [0,∞)] are equivalent. Therefore
ProbZN > z = ProbZN > z and TN−1 ∈ [0,∞)]
= Prob ZN > z and TN−1 ∈ ∪i≥0[τi, τi+1)
=∑
i≥0 Prob ZN > z and TN−1 ∈ [τi, τi+1)
=∑
i≥0 Prob ZN > z|TN−1 ∈ [τi, τi+1)Prob TN−1 ∈ [τi, τi+1)
≈∑i≥0 Prob ZN > z|TN−1 = τi fTN (τi)∆τ
≈∫∞0
ProbZN > z|TN−1 = τ fTN (τ)dτ
(2.33)
Thus, in the limit as ∆t→ 0, we obtain the integral formula
ProbZN > z =
∫ ∞
0
ProbZN > z|TN−1 = τ fTN (τ)dτ (2.34)
From this and (2.32) we conclude that
Prob ZN > z =∫∞0 ProbZN > z|TN−1 = τ fTN (τ)dτ
=∫∞0
Prob X(z + τ) −X(τ) = 0 fTN (τ)dτ
=∫∞0 e−λzfTN (τ)dτ = e−λz
∫∞0 fTN (τ)dτ
= e−λz
(2.35)
From this we conclude that ProbZN ≤ z = 1 − e−λz as claimed.
22
(j.) If Z = TN − TM , N > M then fZ(z) = λN−M
(N−1−M)!e−λzzN−1−M .
The first step is to obtain a formula for ProbZ > z . Using the same method as in the previous
subsection, we conclude that
Prob Z > z = ProbZ > z and TM ∈ [0,∞)] =∫∞0
ProbZ > z|TM = τ fTM (τ)dτ.
(2.36)
Next, we observe the key probability equivalence
ProbTN − TM > z|TM = τ =
N−1−M∑
i=0
ProbX(z + τ) −X(τ) = i =
N−1−M∑
i=0
e−λz (λz)i
i!. (2.37)
Combining (2.36) and (2.37) gives
ProbZ > z =
∫ ∞
0
N−1−M∑
i=0
e−λz (λz)i
i!fM (τ)dτ =
N−1−M∑
i=0
e−λz (λz)i
i!
∫ ∞
0
fM (τ)dτ. (2.38)
This, and the fact that∫∞0 fM (τ)dτ = 1, imply
ProbZ ≤ z = 1 −N−1−M∑
i=0
e−λz (λz)i
i!
∫ ∞
0
fM (τ)dτ = 1 −N−1−M∑
i=0
e−λz (λz)i
i!. (2.39)
Finally, it follows from (2.39) that
fZ(z) =d
dzProb Z ≤ z =
λN−M
(N − 1 −M)!e−λzzN−1−M . (2.40)
(k.) Proof that Z2 = T2 − T1 and Z1 = T1 are independent.
Our goal is to prove the following result: if x > 0 and y > 0 then
Prob Z2 ≤ x, T1 ≤ y = Prob Z2 ≤ xProbT1 ≤ y .
The proof of this property consists of two steps: I. use the method of proof of Principle X to develop
an integral formula involving a product of a conditional probability and a simpler probability density
function, and II. reduce the conditional probability part of the integrand to a function which is
independent of the variable of integration so that it can can be taken outside of the integral
Step I. First, note that the range of T1 is 0 ≤ T1 ≤ y. Thus, as in the proof of Principle X, we
subdivide the interval [0, y] into subintervals. Let N > 1 be large and set ∆y = yN . Define
τi = i∆y, 0 ≤ i ≤ N.
23
We observe that the events [Z2 ≤ x and T1 ≤ y] and [Z2 ≤ x and T1 ∈ [0, y]] are equivalent. Then
Prob[Z2 ≤ x and T1 ≤ y] = Prob[Z2 ≤ x and T1 ∈ [0, y]]
= Prob[Z2 ≤ x and T1 ∈ ∪1≤i≤N [τi−1, τi)]
=∑
1≤i≤N Prob[Z2 ≤ x and T1 ∈ [τi−1, τi)]
=∑
1≤i≤N Prob[Z2 ≤ x|T1 ∈ [τi−1, τi)]Prob[T1 ∈ (τi−1, τi]]
≈∑
1≤i≤N Prob[Z2 ≤ x|T1 = τi−1]fT1(τi−1)∆τ
≈∫ y
0Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ
(2.41)
Thus, in the limit as ∆t→ 0, we obtain the integral formula
Prob[Z2 ≤ x and T1 ≤ y] =
∫ y
0
Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ. (2.42)
Step II. From this integral formula we conclude that
Prob[Z2 ≤ x and T1 ≤ y] =∫ y
0Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ
=∫ y
0 Prob[T2 − T1 ≤ x|T1 = τ ]fTN (τ)dτ
=∫ y
0(1 − Prob[T2 − T1 > x|T1 = τ ]) fTN (τ)dτ
=∫ y
0 (1 − e−λx)fTN (τ)dτ
= (1 − e−λx)∫ y
0fTN (τ)dτ
= (1 − e−λx)(1 − e−λy) = Prob[Z2 ≤ x]Prob[T1 ≤ y].
(2.43)
(l.) Proof that Z = TN − TM , N > M and TM are independent.
Our goal is to prove the following result: if x > 0 and y > 0 then
ProbZ ≤ x, 0 ≤ TM ≤ y = Prob Z ≤ xProb0 ≤ TM ≤ y .
The proof uses (i.) Principle X, and (ii.) the fact that the Ti’s increase as i increases implies the key
”probability link”
Prob TN − TM > x|TM = τ =
N−1−M∑
i=0
Prob X(z + τ) −X(τ) = i =
N−1−M∑
i=0
e−λz (λz)i
i!. (2.44)
First, we conclude from Principle X that
Prob Z ≤ x and 0 ≤ TM ≤ y =
∫ y
0
ProbZ ≤ x|TM = τ fTM (τ)dτ. (2.45)
24
From this and (2.44) we conclude that
ProbZ ≤ x and 0 ≤ TM ≤ y =∫ y
0ProbZ ≤ x|TM = τ fTM (τ)dτ
=∫ y
0 ProbTN − TM ≤ x|TM = τ fTM (τ)dτ
=∫ y
0(1 − ProbTN − TM > x|TM = τ) fTM (τ)dτ
=∫ y
0
(
1 −∑N−1−Mi=0 ProbX(z + τ) −X(τ) = i
)
fTM (τ)dτ
=(
1 −∑N−1−Mi=0 e−λz (λz)i
i!
)
∫ y
0 fTM (τ)dτ
= Prob Z ≤ xProb0 ≤ TM ≤ y .
(2.46)
(m.) Two proofs that E(y(t)) = λ for the spike train y(t) =∑∞
k=1 δ(t− Tk).
Method I. This method relies on knowing the exact form of the pdf’s of the random variables Tk.
The spike train for the Poisson process is
y(t) =
∞∑
k=1
δ(t− Tk), (2.47)
where Tk is a gamma random variable parameters λ and k. The mean of the spike train is
E(y(t)) =∞∑
k=1
E(δ(t− Tk)) =∞∑
k=1
∫ ∞
0
δ(t− η)fTk(η)dη =
∞∑
k=1
fTk(t). (2.48)
Since Tk is a gamma random variable parameters λ and k, we conclude that
E(y(t)) =
∞∑
k=1
λ
(k − 1)!e−λt(λt)k−1 = λ (2.49)
Method II. This method relies on knowing the exact formula for E(X(t)). The Poisson process X(t)
satisfies E(X(t)) = λt. Furthermore,
X(t) =
∫ t
0
∞∑
k=1
δ(t− Tk) =
∫ t
0
y(η)dη, (2.50)
hence
X ′(t) =
∞∑
k=1
δ(t− Tk) = y(η). (2.51)
We conclude that the derivative of the Poisson process is the spike train, and that
E(y(t)) =d
dtE(X(t)) = E(X ′(t)) = λ =
∞∑
k=1
E(δ(t− Tk)). (2.52)
(n.) Two proofs that the spike train y(t) satisfies E(y(t)y(s)) = λ2 when s > t.
25
Method I. This method relies on knowing the exact formula for E(X(t)X(s)). For the Poisson process
we recall from (2.17) that
E(X(t)X(s)) = λmin(t, s) + λ2ts. (2.53)
From this it follows that
E(y(t)y(s)) = E(X ′(t)X ′(s)) =∂2
∂t∂sE(X(t)X(s)) = λ2. (2.54)
Method II. This method proceeds in two steps: step 1 notes thatX(t) has independent, stationary
increments and step 2 uses these two properties to formulate a reasonable approach to compute
E(X ′(t)X ′(s)).
Remarks:
(i) Is this a reasonable approach to use to compute the autocorellation E(y(t)y(t + τ)) for a spike
train associated with a solution of a LIF SDE?
(ii) Is it true that the arguments given below imply that E(y(t)y(t+ τ)) = E(y(t))E(y(t + τ))
First, since X(t) is the Poisson process, we conclude that if 0 < t < s and ∆t, ∆s are small enough
then
X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)
are independent random variables.
Secondly, this fact allows us to compute E(X ′(t)X ′(s)). That is,
E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)
E
((
X(t+ ∆t) −X(t)
∆t
)(
X(s+ ∆s) −X(s)
∆s
))
(2.55)
becomes
E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)
E
((
X(t+ ∆t) −X(t)
∆t
))
E
((
X(s+ ∆s) −X(s)
∆s
))
. (2.56)
Because the Poisson process is stationary and X(0) = 0, this further reduces to
E(y(t)y(s)) = E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)
E
((
X(∆t)
∆t
))
E
((
X(∆s)
∆s
))
= λ2. (2.57)
26
(o.) Proof that the spike train satisfies E(y(t)y(s)) = λδ(t− s) + λ2
The first step is to recall that
E(X(t)X(s)) =
λs+ λ2ts, if t > s,
λt+ λ2ts, if t < s,(2.58)
and that
E(y(t)y(s)) = E
(
dX(t)
dt
dX(s
ds
)
. (2.59)
Thus,
E (X(t)y(s)) = E
(
X(t)dX(s
ds
)
=∂
∂sE (X(t)X(s)) . (2.60)
Combining (2.58) with (2.60) gives
E(X(t)y(s)) =
λ+ λ2t, if t > s,
0 + λ2t, if t < s.(2.61)
This can be written as
E (X(t)y(s)) = λH(t− s) + λ2t, (2.62)
Where H(t−s) is the heaviside function. From (2.63), and the fact that H ′(t−s) = δ(t−s), it follows
that
E (y(t)y(s)) = λδ(t− s) + λ2. (2.63)
27
(p.) Proof that E(y2(t)) = ∞ and V ar(y(t)) = ∞
The spike train stisfies
y(t) =d
dtX(t) = lim
∆t→0
(
X(t+ ∆t) −X(t)
∆t
)
Therefore,
E(y2(t)) = lim∆t→0
E
(
(
X(t+ ∆t) −X(t)
∆t
)2)
= lim∆t→0
1
(∆t)2E(
(X(t+ ∆t) −X(t))2)
(2.64)
Because X(t) is stationary this further reduces to
E(y2(t)) = lim∆t→0
1
(∆t)2E(
(X(∆t))2)
= lim∆t→0
λ∆t
(∆t)2= lim
∆t→0
λ
∆t= ∞ (2.65)
Thus, to compute V ar(y(t), we combine (2.52) and (2.65), and obtain
E(y2(t)) − E2(y(t)) = ∞− λ2 = ∞. (2.66)
(q.) Proof that Probone spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s).
Let 0 < t < s. If ∆t, ∆s are small enough then
X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)
are independent random variables. Thus,
Prob one spike in (t, t+ ∆t) and one spike in (s, s+ ∆s)
is equivalent to
Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1
= ProbX(t+ ∆t) −X(t) = 1Prob X(s+ ∆s) −X(s) = 1(2.67)
Evaluation of the right side of (2.67) gives
Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1 = λ2∆t∆se−λ(∆t+∆s). (2.68)
Thus,
Prob one spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s). (2.69)
28
3 Spiking Theory
(a.) The definition of input current.
This follows de la Rocha et al [2]. Corellated fluctuating currents resembling synaptic activity are
injected into the somata of twenty unconnected cortical neurons. The total input current to cell i is
Ii(t) = µi + σi
(√1 − cζi(t) +
√cζc(t)
)
, (3.1)
where µi is temporal average of the current. The term
σi
(√1 − cζi(t) +
√cζc(t)
)
represents Gaussian fluctuations with a temporal structure consistent with that in-vivo (how are they
consistent?), and is composed of two weighted factors: ζi(t), which is independent for each cell, and
ζc(t), which is common to all cells. The parameter σi sets the variance of the total input current. The
weight of the shared fluctuations between adjacent pairs of cells is set by the ”corelation coefficient”
c, where 0 ≤ c ≤ 1.
Q1. Are the cells adjacent in reality?
(b.) Derivation of the correlation coefficient c.
This follows the derivation in the first part of the Methods Section of [2]. Let k denote trial number,
with 1 ≤ k ≤ 100. Then total input current to cell i is
Iki (t) = µi + σi
(√cζk
c (t) +√
1 − cζki (t)
)
(3.2)
The cross correlation function of two random variables X(t) and Y (t) is, according to p. 211 of
Schaums’,
E (X(t)Y (t+ τ)) , τ ∈ (−∞,∞). (3.3)
They define cross correlation function of a pair of currents by
Gkk′
ij = E(
Iki (t)Ik′
j (t+ τ))
− µiµj . (3.4)
Q2. This is slightly differently from (3.3) since they subtract off µiµj . Why do they define Gkk′ij as a
cross correlation?
When the right side of (3.4) is computed, the independence of ζki (t), ζk
j (t) and ζkc (t) force several
terms to be zero, and they get
Gkk′
ij = δkk′σiσj [δij + (1 − δij)c]E(
ζkc (t)ζk′
c (t+ τ))
, (3.5)
where δij is the Kronecker delta. This formula seems right.
Q3. What is the range of t? Is t ≥ 0 since t represents time?
29
Let k = k′ and i 6= j. Then (3.5) becomes
Gkkij = σiσjcE
(
ζkc (t)ζk′
c (t+ τ))
= σiσjcδ(τ). (3.6)
If k = k′ and i = j then (3.5) becomes
Gkkii = σiσiδ(τ). (3.7)
Q4. Are these calculations right?
They claim the corellation coefficient of a pair of input currents is
cij =
∫∞−∞Gkk
ij (τ)dτ∫∞−∞Gkk′
ii (τ)dτ= c. (3.8)
However, I seem to get
cij =
∫∞−∞Gkk
ij (τ)dτ∫∞−∞Gkk
ii (τ)dτ=σiσj
σiσic =
σj
σic. (3.9)
Q5. Which of these two answers is right? Also, why are they integrating? The definition
of corelatation coefficient does not seem to involve an integral.
To get c from the computation should (3.9) be
cij =
∫∞−∞Gkk
ij (τ)dτ(
∫∞−∞Gkk
ii (τ)dτ)1/2 (
∫∞−∞Gkk
jj (τ)dτ)1/2
=σiσj
σiσjc = c. (3.10)
30
(c.) The LIF model - Benji’s notes
These notes are a summary of our discussions about Beji Lindner’s thesis starting on Feb. 6, 2008.
The Leaky Integrate and Fire (LIF) model with trasmembrane constant τ is the equation
τdV
dt= (µ− V ) + σ
dζ
dt, (3.11)
together with the reset condition
V (t−) = vT and V (t+) = vR. (3.12)
Remark. If σ =√
2D then we get Benji’s equation
D =σ2
2. (3.13)
We set τ = 1 and study the simpler equation
dV
dt= (µ− V ) + σ
dζ
dt, . (3.14)
Remarks.
(i) Typical values for vT and vR are
vT = 1 and vR = 0. (3.15)
(ii) If µ < 1 and σ = 0 then there can be no spike since V (t) just increases to µ as t→ ∞.
(iii) If µ > 1 and σ = 0 then we have a deterministic train of equally spaced spikes.
(iv) The law of large numbers links the mean of the interspike intervals (a deterministic number) with
a limiting sum of random variables as follows: Let T be the random variable which denotes the time
the solution V takes to go from V = vR to V = vT , i.e. T is a first passage time. Also, let ti denote
the spike times (random variables). The quantities ti − ti−1 are i.i.d. random variables representing
the interspike intervals, each with mean E(ti − ti−1) = E(T ). Because of the law of large numbers,
Benji (equation (2.10)) can conclude that
E(T ) = limN→∞
1
N
N∑
i=0
(ti − ti−1). (3.16)
The Fokker-Planck equation associated with problem (3.14)-(3.23) is
∂P
∂t=
∂
∂V
(
(V − µ) +σ2
2
∂
∂V
)
P + ν(t)δ(V − vR). (3.17)
31
Question 1. Does Benji set V (0) = 0 ? If yes, then P (V, t) = (V, t|0, 0) is the solution of the
Fokker-Planck equation? Brent says that he himself assumes that P (V, 0) = δ(V ), and therefore his
P (V, t) = (V, t|0, 0). He also pointed out that Benji is only interested in the stationary pdf case, and
therefore Benji does not assume a specific intitial value for V (0).
Let P (V, t) denote the solution. Since the range of values of solutions of (3.14)-(3.23) is −∞ < V (t) <
vT , then P (V, t) satisfies∫ vT
−∞P (V, t)dV = 1 ∀t ≥ 0. (3.18)
Question 2. Do we interpret (3.18) as
Prob−∞ < V (t) < vT =
∫ vT
−∞P (V, t)dV = 1? (3.19)
Next, since we need the integral in (3.18) to converge, then it is reasonable to assume that
limV →−∞
P (V, t) = 0. (3.20)
The absorbing boundary condition at V = vT is
P (vT , t) = 0. (3.21)
32
(d.) How to solve τ dVdt = (µ− V ) + σ dζ
dt .
The Leaky Integrate and Fire (LIF) model with trasmembrane constant τ is the equation
τdV
dt= (µ− V ) + σ
dζ
dt, (3.22)
together with the reset condition
V (t−) = vT and V (t+) = vR. (3.23)
Next, multiply (3.22) by dt/τ and get
dV =(µ− V )
τdt+ σ
√
1
τdW. (3.24)
The numerical solution. We let vR = 0, vT = 1 and assume that V (0) = 0. The time step is
h = dt. The numerical Euler algorithm to solve (3.24) in matlab is
V (i+ 1) = V (i) +(µ− V (i))
τh+ σ
√
h
τrandn, V (1) = 0, (3.25)
A reasonable neuronal value for τ is τ = .1 However, we let τ = 1, and (3.25) becomes
V (i+ 1) = V (i) +(µ− V (i))
τh+ σ
√hrandn, V (1) = 0, (3.26)
together with the rest condition. We use time step h = .01 for experimental purposes, and h = .001
for publication.
The refractory period. After a spike, when we reset V to vR we need to keep V (t) = vR = 0
for a short interval of t values called the refractory period. We set taurefractory = .1 in our matlab
program.
33
(e.) How to analyze E (X(t)X(t+ τ)) when X(t) solves an SDE?
Our goal is to develp a useful formula for the autocorellation function E (X(t)X(t+ τ)) . We do this
in two parts:
I. Basic properties of E (X(t)X(s)) when s ≥ 0 and t > s.
II. The derivation of the formula for E (X(t)X(s)) .
Part I. Basic properties of E (X(t)X(s)) when s ≥ 0 and t > s.
Our goal here is to analyze basic properties of E (X(t)X(s)) when X(t) solves the SDE
dX
dt= f(X) + σζ(t), X(s) = x, and t > s ≥ 0. (3.27)
where ζ(t) is white noise.
The Fokker-Planck equation for the conditional probability density function associated with (3.27) is
∂
∂tρ(x′, t|x, s) = − ∂
∂x′(f(x′)ρ(x′, t|x, s)) +
σ2
2
∂2
∂x′2ρ(x′, t|x, s), t > s ≥ 0. (3.28)
When s = t the solution ρ(x′, s|x, s) is required to satisfy
ρ(x′, s|x, s) = δ(x′ − x) (3.29)
The function E (X(t)X(s)) is defined by
E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, t;x, s)dx′dx. (3.30)
Note that ρ(x′, t;x, s) can be written as
ρ(x′, t;x, s) = ρ(x′, t|x, s)ρ(x, s) (3.31)
Substituting (3.31) into (3.30) transforms (3.30) into the more amenable form
E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, t|x, s)ρ(x, s)dx′dx. (3.32)
Note that the function ρ(x′, t|x, s) in the integrand is the solution of problem (3.28)-(3.29).
Question 1. On the right side of the SDE (3.27) the only place that t appears is in the white noise
function ζ(t). Does the property allow us to ”shift to the left” by amount s and conclude that
ρ(x′, t|x, s) = ρ(x′, t− s|x, 0) = ρ(x′, τ |x, 0), τ = t− s. (3.33)
If (3.33) holds then substitution of (3.33) into (3.32) gives
E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, t− s|x, 0)ρ(x, s)dx′dx. (3.34)
Part II. The derivation of the formula for E (X(t)X(s)) .
34
Replace t and s in (3.34) with t+ τ and t, respectively, and transform (3.34) into
E (X(t+ τ)X(t)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, τ |x, 0)ρ(x, t)dx′dx. (3.35)
The function ρ(x′, τ |x, 0) solves the the Fokker-Planck equation
∂
∂tρ(x′, τ |x, 0) = − ∂
∂x′(f(x′)ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0), τ > 0. (3.36)
At τ = 0 the function ρ(x′, τ |x, 0) satisfies the ususal condition
ρ(x′, 0|x, 0) = δ(x′ − x). (3.37)
Also, the function ρ(x′, τ |x, 0) is pdf for the solution X(τ) of the SDE
dX
dτ= f(X) + σζ(τ), X(0) = x, and τ ≥ 0. (3.38)
Question 2. Next, we need to know what problem the function ρ(x, t) solves. I think that
ρ(x, t) = ρ(x, t|0, 0), (3.39)
which is the Fokker-Planck pdf for the solution of
dX(t)
dt= f(X(t)) + σζ(t), X(0) = 0, and t > 0. (3.40)
Substitute (3.39) into (3.35) and get
E (X(t+ τ)X(t)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, τ |x, 0)ρ(x, t|0, 0)dx′dx. (3.41)
This formula allows us to see the contributions from both τ and t to the of the autocorellation function
E (X(t+ τ)X(t)) .
Remarks. In order to use (3.41) to analyze the behavior of E (X(t+ τ)X(t)) when t >> 1, we need
to determine the behavior of ρ(x, t|0, 0) when t is large. One possibility is that ρ(x, t|0, 0) approaches
a stationary solution as t → ∞. A stationary solution of the Fokker-Planck equation (3.28) is a time
independent solution ρS(x′) of
0 = − d
dx′(f(x′)ρS(x′)) +
σ2
2
d2
dx′2ρS(x′), (3.42)
where ρS(x′) is bounded, ρS ∈ C2(R), and∫ ∞
−∞ρS(x′)dx′ = 1. (3.43)
Note that (3.42) can be written as
0 = − d
dx′J(x′), (3.44)
where J(x′) is the time independent flux
J(x′) = f(x′)ρS(x′) − σ2
2
d
dx′ρS(x′). (3.45)
35
(f.) Gardner’s eigenfunction approach to finding ρ(x, t).
Goal. We develop Gardner’s eigenfunction exapansion method ([11], pp. 129-131) to construct solu-
tions of the Fokker-Planck equation associated with the SDE
dX
dt= A(X) +
√
B(x)ζ(t), X(s) = x0, (3.46)
where ζ(t) is white noise. The associated Fokker-Planck equation is
∂
∂tρ(x, t) = − ∂
∂x(A(x)ρ(x, t)) +
1
2
∂2
∂x2(B(x)ρ(x, t)) . (3.47)
We assume that ρ(x, t) solves (3.47) on the interval a < xb, that a < b are finite, and that ρ(x, t)
satisfies the reflective boundary conditions
0 = −A(x)ρ(x, t) +1
2
∂
∂x(B(x)ρ(x, t)) when x = a, x = b. (3.48)
Remark. In the next subsection we show how to use the expansion method to construct ρ(x, t|x0, 0),
the particular solution associated with the SDE (3.46).
Step I. Find the stationary solution ρs(x), which solves
0 = − ∂
∂x(A(x)ρs(x)) +
1
2
∂2
∂x2(B(x)ρs(x)) . (3.49)
Assuming that ρs(x) satisfies reflective boundary conditions implies that
0 = −A(x)ρs(x) +1
2
∂
∂x(B(x)ρs(x)) ∀x ∈ [a, b]. (3.50)
This equation plays a key role in Step IV below.
Step II. Define the function q(x, t) associated with ρs(x) and ρ(x, t) by setting
ρ(x, t) = ρs(x)q(x, t). (3.51)
Combining (3.47), (3.50) and (3.51) shows that q(x, t) satisfies the backwards equation
∂
∂tq(x, t) = A(x)
∂
∂xq(x, t) +
B(x)
2
∂2
∂x2q(x, t). (3.52)
Step III. Find the eigenvalues and eigenfunctions associated with ρ(x, t) and q(x, t). These satisfy
ρ(x, t) = Pλ(x)e−λt and q(x, t) = Qλ′ (x)e−λ′t. (3.53)
It is important for Step IV below to observe that, if λ = λ′, then
Pλ(x) = ρs(x)Qλ(x). (3.54)
Next, it is easily verified that Pλ(x) and Qλ′ (x) satisfy the ODE’s
−λPλ(x) = − d
dx(A(x)Pλ(x)) +
1
2
d2
dx2(B(x)Pλ(x)) , (3.55)
36
and
−λ′Qλ′ (x) = A(x)
d
dxQλ′ (x) +
B(x)
2
d2
dx2Qλ′ (x). (3.56)
The boundary conditons astisfied by Pλ(x) are
0 = −A(x)Pλ(x) +1
2
d
dx(B(x)Pλ(x)) when x = a, x = b. (3.57)
When λ = λ′= 0 the eigenfunctions are
P0(x) = ρs(x) and Q0(x) = 1 ∀x ∈ (a, b). (3.58)
Furthermore, by scaling ρs(x), we conclude that
∫ b
a
P0(x)Q0(x)dx =
∫ b
a
ρs(x)dx = 1 (3.59)
Step IV. Show that the eigenfunctions satisfy the orthognality conditions
∫ b
a
Pλ(x)Qλ′ (x)dx 6= 0 when λ = λ′, (3.60)
and∫ b
a
Pλ(x)Qλ′ (x)dx = 0 when λ 6= λ′. (3.61)
First, when λ = λ′we see that
∫ b
a
Pλ(x)Qλ(x)dx =
∫ b
a
ρs(x) (Qλ(x))2dx 6= 0. (3.62)
This is consistent with (3.59).
Next, multiply (3.55) by Qλ′ (x), and (3.56) by Pλ(x), subtract the two equations, and integrate the
resulting equation from x = a to x = b to get
(λ′ − λ)
∫ b
a
Pλ(x)Qλ′ (x)dx =
[
Qλ′ (x)
(
−A(x)Pλ(x) +1
2(B(x)Pλ(x))′
)
− 1
2B(x)Pλ(x)Q′
λ′ (x)
]b
a(3.63)
To prove (3.61) we nee to show that the right side of (3.63) is zero when λ 6= λ′.
It follows from (3.57) that the first term on the right side of (3.63) is zero. This reduces (3.63) to
(λ′ − λ)
∫ b
a
Pλ(x)Qλ′ (x)dx =
[
−1
2B(x)Pλ(x)Q′
λ′ (x)
]b
a
(3.64)
It remains to show that the right side of (3.64) is zero when λ 6= λ′. For this we develop an expression
for B(x)Pλ(x)Q′λ′ (x). First, it follows from (3.54) that, if λ is replaced by λ
′, then
ρs(x) =Pλ′ (x)
Qλ′ (x)and ρ′s(x) =
P ′λ′ (x)
Qλ′ (x)− Pλ′ (x)
Q2λ′ (x)
Q′λ′ (x). (3.65)
37
Recall from (3.50) that
0 = −A(x)ρs(x) +1
2
∂
∂x(B(x)ρs(x)) ∀x ∈ [a, b]. (3.66)
Substitute (3.65) into (3.66) and obtain
0 =
(
−A(x) +1
2B′(x)
)
Pλ′ (x)
Qλ′ (x)+B(x)
2
(
P ′λ′ (x)
Qλ′ (x)− Pλ′ (x)
Q2λ′ (x)
Q′λ′ (x)
)
. (3.67)
Multiplying (3.67) by Q2λ′ leads, after an algerbraic mainipulation, to
1
2B(x)Pλ′ (x)Q′
λ′ (x) =
(
−A(x) +1
2B′(x)
)
Pλ′ (x)Qλ′ (x) +B(x)
2P ′
λ′ (x)Qλ′ (x) (3.68)
Rearranging terms gives
1
2B(x)Pλ′ (x)Q′
λ′ (x) =
(
−A(x)Pλ′ (x) +1
2(B(x)Pλ′ (x))
′)
Qλ′ (x) (3.69)
Finally, multiply (3.69) by Pλ(x) and divide by Pλ′ (x), and get
1
2B(x)Pλ(x)Q′
λ′ (x) =Pλ(x)
Pλ′(x)
(
−A(x)Pλ′ (x) +1
2(B(x)Pλ′ (x))′
)
Qλ′ (x) (3.70)
Recasll from (3.57) that Pλ′ (x) satisfies reflective boundary condtions, hence
[
1
2B(x)Pλ(x)Q′
λ′ (x)
]b
a
=
[
Pλ(x)
Pλ′(x)
(
−A(x)Pλ′ (x) +1
2(B(x)Pλ′ (x))
′)
Qλ′ (x)
]b
a
= 0. (3.71)
This completes the proof that the right side of (3.64) is zero, and therefore the proof of of the
orthogonality condition (3.61) is complete.
Step V. We now develop the eigenfunction exapsnion for a solution of the Fokker-Planck equa-
tion (3.47) with reflective boundary conditions. The expansion is
ρ(x, t) =∑
λ
AλPλ(x)e−λt. (3.72)
To find a specific Aλ′ , multiply (3.72) by Qλ′ (x), integate from x = a to x = b, and get
∫ b
a
ρ(x, t)Qλ′ (x)dx =∑
λ
Aλ
∫ b
a
Pλ(x)Qλ′ (x)dxe−λt. (3.73)
Because of the orthogonality conditions (3.61)-(3.62), it immediately follows from (3.74) that
Aλ′ =
∫ b
a
ρ(x, 0)Qλ′ (x)dx. (3.74)
(g.) How to use the eigenfunction expansion to find ρ(x, t|x0, 0).
38
As above assume that ρ(x, t|x0, 0) satisfies reflective boundary conditions. Recall that ρ(x, 0|x0, 0) =
δ(x− x0). Then (3.74) reduces to
Aλ′ =
∫ b
a
ρ(x, 0|x0, 0)Qλ′ (x)dx =
∫ b
a
δ(x− x0)Qλ′ (x)dx = Qλ′ (x0). (3.75)
In turn, this implies that ρ(x, t|x0, 0) is given by
ρ(x, t|x0, 0) =∑
λ
Qλ(x0)Pλ(x)e−λt. (3.76)
(h.) How to find the autocorrelation function from the eigenfunction expansion.
As above assume that ρ(x, t|x0, 0) satisfies reflective boundary conditions, so that
ρ(x, t|x0, 0) =∑
λ
Qλ(x0)Pλ(x)e−λt. (3.77)
The autocorrelation function is
RX(τ) =
∫ b
a
∫ b
a
xx0ρ(x, τ |x0, 0)ρs(x0)dxdx0e−λτ =
∑
λ
∫ b
a
∫ b
a
xx0Qλ(x0)Pλ(x)ρs(x0)dxdx0e−λτ .
(3.78)
Recall that
Pλ(x) = ρs(x)Qλ(x). (3.79)
Combining (3.78) and (3.79) gives
RX(τ) =∑
λ
(
∫ b
a
xQλ(x)ρs(x)dx
)(
∫ b
a
x0Qλ(x0)ρs(x0)dx0
)
e−λτ . (3.80)
From this, and the fact that Q0(x) = 1, we conclude that
RX(τ) =
(
∫ b
a
x(x)ρs(x)dx
)2
+∑
λ6=0
(
∫ b
a
xQλ(x)ρs(x)dx
)2
e−λτ . (3.81)
It follows from (3.81) that
limτ→∞
RX(τ) =
(
∫ b
a
xρs(x)dx
)2
. (3.82)
Remark 1. If a = −b < 0, and ρs(x) is an even function, then
RX(τ) =∑
λ6=0
(
∫ b
a
xQλ(x)ρs(x)dx
)2
e−λτ , (3.83)
and therefore
limτ→∞
RX(τ) = limτ→∞
∑
λ6=0
(
∫ b
a
xQλ(x)ρs(x)dx
)2
e−λτ = 0 (3.84)
39
Remark 2. This property will hold for the Ornstein-Uhlenbeck process where a = −∞, b = ∞, and
ρs(x) = exp(−kx2/2). Here, as Gardner shows ([11], pp.),
ρ(x, t|x0, 0) =∑
λ
Hλ(x)Pλ(x)e−λt, (3.85)
where λ = n ≥ 0, and Hλ(x) is the nth order Hermite polynomial. Thus, since H0(x) = 1, then
P0(x) = ρs(x) and we conclude that
limt→∞
ρ(x, t|x0, 0) = H0(x)P0(x) = ρs(x). (3.86)
Recall that Pλ(x) = Qλ(x)ρs(x). This and the orthogonality condition imply that
∑
λ
∫ ∞
−∞xQλ(x)ρs(x)dx =
∑
λ
∫ ∞
−∞H1(x)Pλ(x)dx =
∫ ∞
−∞H1(x)P1(x)dx =
∫ ∞
−∞x2ρs(x)dx. (3.87)
This implies that RX(τ) has the simplified form
RX(τ) =
(∫ ∞
−∞x2ρs(x)dx
)2
e−κτ , (3.88)
where κ > 0 is a constant (see the next section).
(i.) The autocorrelation function for the Ornstein-Uhlenbeck process.
The Ornstein-Uhlenbeck procsess is modelled by the SDE
dX(t)
dt= −γX + σζ(t), X(0) = x, and t ≥ 0. (3.89)
The Fokker-Planck problem for this process consists of the pdf
∂
∂tρ(x′, t|x, 0) = γ
∂
∂x′(x′ρ(x′, t|x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, t|x, 0), τ > 0. (3.90)
with initial condition
ρ(x′, 0|x, 0) = δ(x′ − x). (3.91)
Stationary solution. Setting f(x′) = −γx in (3.42) gives the ODE
0 =d
dx′(γx′ρS(x′)) +
σ2
2
d2
dx′2ρS(x′) (3.92)
An integration reduces (3.92) to the first order equation
c1 = γx′ρS(x′) +σ2
2
d
dx′ρS(x′). (3.93)
We assume that ρS(x′) satisfies the conditions
lim|x′|→∞
x′ρS(x′) = lim|x′|→∞
d
dx′ρS(x′) = 0. (3.94)
40
Substitution of (3.94) into (3.93) shows that c1 = 0, and therefore (3.93) simplifies to
0 = γx′ρS(x′) +σ2
2
d
dx′ρS(x′). (3.95)
Solving (3.95) subject to conditions (3.43) and (3.94), we obtain the stationary solution
ρS(x′) =1
σ
√
γ
πexp
(
− γ
σ2x′2)
. (3.96)
We use (3.96) to study the behavior of E (X(t+ τ)X(t)) when t >> 1. We assume that
limt→∞
ρ(x, t|0, 0) = ρS(x). (3.97)
It follows from (3.41) that
limt→∞
E (X(t+ τ)X(t)) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.98)
where the function ρ(x′, τ |x, 0) solves the Fokker-Planck equation
∂
∂tρ(x′, τ |x, 0) =
∂
∂x′(γx′ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0), τ > 0, (3.99)
and
ρ(x′, 0|x, 0) = δ(x′ − x). (3.100)
Finally, since
RX(τ) = limt→∞
E (X(t+ τ)X(t)) , (3.101)
we conclude that
RX(τ) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.102)
Properties of RX(τ).
Our goal here is to show how to use (3.102) to determine the behavior of RX(τ) at τ = 0 and τ = ∞,
and also to obtain a closed form formula for RX(τ).
First, It follows from (3.102) that
RX(0) =
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, 0|x, 0)ρS(x)dx′dx. (3.103)
Substituting ρ(x′, 0|x, 0) = δ(x− x′) into (3.103), we obtain
RX(0) =
∫ ∞
−∞x2ρS(x)dx. (3.104)
Combining (3.96) with (3.104), we obtain the well known ’fluctuation-dissipation’ equation
RX(0) =1
σ
√
γ
π
∫ ∞
−∞x2exp
(
− γ
σ2x2)
dx =σ2
2γ. (3.105)
41
Next, to determine the behavior of RX(τ) as τ → ∞, we need to assume that
limτ→∞
ρ(x′, τ |x, 0) = ρS(x′). (3.106)
Combining (3.102) with (3.106) gives
limτ→∞
RX(τ) =
∫ ∞
−∞
∫ ∞
−∞x′xρS(x)ρS(x′)dx′dx =
∫ ∞
−∞x′ρS(x′)dx′
∫ ∞
−∞xρS(x)dx = 0, (3.107)
since ρS is an even fucntion.
Derivation of the ODE satisfied by RX(τ).
To obtain a closed form formula for RX(τ) we derive and solve an associated ODE initial value
problem. First, a differentiation of both sides of (3.102) with respect to τ gives
d
dτRX(τ) =
∫ ∞
−∞
∫ ∞
−∞x′x
∂
∂τρ(x′, τ |x, 0)ρS(x)dx′dx (3.108)
Combining (3.99) with (3.108), we obtain
d
dτRX(τ) =
∫ ∞
−∞
∫ ∞
−∞x′x
(
∂
∂x′(γx′ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0)
)
ρS(x)dx′dx (3.109)
Integration by parts reduces (3.109) to
d
dτRX(τ) = −γ
∫ ∞
−∞
∫ ∞
−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.110)
This, together with (3.102), further reduces (3.110) to
d
dτRX(τ) = −γRX(τ). (3.111)
Solving (3.111), with the initial condition RX(0) = σ2
2γ , gives the well known autocorellation function
RX(τ) =σ2
2γe−γτ . (3.112)
42
(j.) Example 2: the LIF model.
The Leaky Integrate and Fire (LIF) model consists of the equation
dX
dt= (µ−X) + σ
dζ
dt, X(0) = x ∈ (−∞, vT ), (3.113)
together with the reset condition
X(t−) = vT and X(t+) = vR. (3.114)
Typical values for vT (threshold) and vR (reset) are
vT = 1 and vR = 0. (3.115)
Observations.
(i) If µ < vT and σ = 0 then there can be no spike since X(t) just increases to µ as t→ ∞.
(ii) If µ > vT and σ = 0 then we have a deterministic train of equally spaced spikes.
The Fokker-Planck problem associated with problem (3.113)-(3.114) consists of the pde
∂
∂tρ(x′, t|x, 0) =
∂
∂x′((x′ − µ)ρ(x′, t|x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, t|x, 0) + ν(t)δ(x′ − vR), (3.116)
with initial condition
ρ(x′, 0|x, 0) = δ(x′ − x). (3.117)
In addition, since the range of values of X(t) is −∞ < X(t) <∞, the normalizing condition is
∫ vT
−∞ρ(x′, t|x, 0)dx′ = 1. (3.118)
Stationary solution. A stationary solution of (3.116) is a solution ρS(x′) of
0 =d
dx′((x′ − µ)ρS(x′)) +
σ2
2
d2
dx′2ρS(x′) + νsH
′(x′ − vR), (3.119)
where we have used the fact that the Heaviside function H(x′ − vR) satisfies H ′(x′ − vR) = δ(x′ − vR)
.
Aside. Note that (3.42) can be written as
0 = − d
dxJ(x′), (3.120)
where J(x′) is the time independent flux
J(x′) = (µ− x′)ρS(x) − σ2
2
d
dx′ρS(x′) − νsH(x′ − vR). (3.121)
The stationary solution must satisfy
limx′→−∞
ρS(x′) = limx′→−∞
(x′ − µ)∂
∂x′ρS(x′) = 0 and
∫ vT
−∞ρS(x′)dx′ = 1. (3.122)
43
Our approach is to find functions ρ1(x′) and ρ2(x
′) such that
ρS(x′) =
ρ1(x′), −∞ < x′ < vR,
ρ2(x′), vR ≤ x′ < vT .
(3.123)
First, we let x′ < vR where ρS(x′) = ρ1(x′) and (3.119) reduces to
0 =d
dx′((x′ − µ)ρ1(x
′)) +σ2
2
d2
dx′2ρ1(x
′) + νsH′(x′ − vR), x′ < vR. (3.124)
An integration of (3.124) from −∞ to x′, together the first two conditions in (3.122), gives
(x′ − µ)ρ1(x′) +
σ2
2
d
dx′ρ1(x
′) = 0, x′ < vR. (3.125)
The solution of this equation is
ρ1(x′) = k1exp
(
− (x′ − µ)2
σ2
)
, x′ < vR, (3.126)
where k1 is a constant that remains to be determined.
Next, we let vR < x′ < vT where ρS(x′) = ρ2(x′) and (3.119) reduces to
0 =d
dx′((x′ − µ)ρ2(x
′)) +σ2
2
d2
dx′2ρ2(x
′) + νsH′(x′ − vR), vR < x′ < vT . (3.127)
An integration of (3.127) gives
(x′ − µ)ρ2(x′) +
σ2
2
d
dx′ρ2(x
′) = −νS + k2, vR < x′ < vT . (3.128)
An integration of (3.128) from x′ to vT , subject to the condition ρ2(vT ) = 0, gives
ρ2(x′) =
2
σ2(νS − k2)exp
(−(x′ − µ)2
σ2
)∫ vT
x′exp
(
(η − µ)2
σ2
)
dη, vR < x′ < vT . (3.129)
We need to determine the constants k1 and k2 in (3.126) and (3.129). For this we have make use of
two matching conditions at x′ = vR. The first is the continuity condtion
ρ1(v−R) = ρ2(v
+R). (3.130)
Combining (3.126), (3.129) and (3.130), we find that k1 and k2 satisfy
k1 =2
σ2(νS − k2)
∫ vT
vR
exp
(
(η − µ)2
σ2
)
dη. (3.131)
Next, we derive the jump condition at vR which is satisfied by ρ′1(vR) and ρ′2(vR). For this we let
ǫ > 0 and integrate (3.119) from vR − ǫ to vR + ǫ, and find that the stationary solution ρS satisfies
(vR + ǫ− µ)ρS(vR + ǫ) +σ2
2ρ′S(vR + ǫ) − (vR − ǫ− µ)ρS(vR − ǫ) − σ2
2ρ′S(vR − ǫ) = −νS. (3.132)
44
Letting ǫ→ 0+ reduces (3.133) to the jump condition
ρ′S(v+R) − ρ′S(v−R) = −2νS
σ2. (3.133)
Thus, in terms of ρ1 and ρ2, this becomes
ρ′2(v+R) − ρ′1(v
−R) = −2νS
σ2. (3.134)
From (3.125) we get
ρ′1(v−R) = − 2
σ2(vR − µ)ρ1(v
−R ). (3.135)
Likewise, from (3.128) we conclude that
ρ′2(v+R) =
2
σ2(−νS + k2) −
2
σ2(vR − µ)ρ2(v
+R). (3.136)
We substitute (3.136) into (3.135), and make use of (3.130), to conclude that
2
σ2(−νS + k2) = −2νS
σ2. (3.137)
Thus, it must be the case that k2 = 0. This and (3.131) imply that
k1 =2νS
σ2
∫ vT
vR
exp
(
(η − µ)2
σ2
)
dη and k2 = 0. (3.138)
Substituting these values into (3.126) and (3.129) gives
ρ1(x′) =
2νS
σ2exp
(
− (x′ − µ)2
σ2
)∫ vT
vR
exp
(
(η − µ)2
σ2
)
dη, −∞ < x′ < vR, (3.139)
and
ρ2(x′) =
2νS
σ2exp
(−(x′ − µ)2
σ2
)∫ vT
x′exp
(
(η − µ)2
σ2
)
dη, vR < x′ < vT . (3.140)
Substituting (3.139) and (3.140) into (3.123), we obtain the stationary solution
ρS(x′) =
2νS
Nσ2 exp(
− (x′−µ)2
σ2
)
∫ vT
vRexp
(
(η−µ)2
σ2
)
dη, −∞ < x′ < vR,
2νS
Nσ2 exp(
−(x′−µ)2
σ2
)
∫ vT
x′ exp(
(η−µ)2
σ2
)
dη, vR ≤ x′ < vT ,(3.141)
where N is the ’normalizing’ constant
N =∫ vR
−∞2νS
σ2 exp(
− (x′−µ)2
σ2
)
∫ vT
vRexp
(
(η−µ)2
σ2
)
dηdx′
+∫ vT
vR
2νS
σ2 exp(
−(x′−µ)2
σ2
)
∫ vT
x′ exp(
(η−µ)2
σ2
)
dηdx′.(3.142)
Remarks. In our derivation of the ODE satisfied by RX(τ) we may need the following insights about
the behavior of ρ1(x′) and ρ2(x
′). Setting k2 = 0 in (3.128), we find that
ρ′2(vR) =2
σ2(−νS + (µ− vR)ρ2(vR)) , ρ2(vT ) = 0 and ρ′2(vT ) = −2νS
σ2. (3.143)
45
Properties of RX(τ).
Following the method of the previous subsection, we find (correct?) that
RX(τ) =
∫ vT
−∞
∫ vT
−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx, (3.144)
where ρS(x) is the stationary solution defined above in (3.141), and ρ(x′, τ |x, 0) solves the Fokker-
Planck equation
∂
∂τρ(x′, τ |x, 0) =
∂
∂x′((x′ − µ)ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0) + νSδ(x
′ − x), τ > 0, (3.145)
subject to the initial condition
ρ(x′, 0|x, 0) = δ(x′ − x). (3.146)
Integrating (3.145) gives
∫ vT
−∞
∂
∂τρ(x′, τ |x, 0)dx′ =
σ2
2ρx′(vT , τ |x, 0) + νS . (3.147)
Note: the ’flux’ J(x′, τ) satisfies
.∂
∂τρ(x′, τ |x, 0) = − ∂
∂x′J(x′, τ) (3.148)
and is given by
J(x′, τ) = −(
(x′ − µ)ρ(x′, τ |x, 0) +σ2
2
∂
∂x′ρ(x′, τ |x, 0) + νSH(x′ − x)
)
(3.149)
Derivation of the ODE satisfied by RX(τ).
To obtain a closed form formula for RX(τ) we derive and solve an associated second order ODE.
First, a differentiation of both sides of (3.144) with respect to τ gives
d
dτRX(τ) =
∫ vT
−∞
∫ vT
−∞x′x
∂
∂τρ(x′, τ |x, 0)ρS(x)dx′dx. (3.150)
Substituting (3.145) into (3.150) gives
R′X(τ) =
∫ vT
−∞
∫ vT
−∞x′(
∂
∂x′((x′ − µ)ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0) + νSδ(x
′ − x)
)
xρS(x)dx′dx.
(3.151)
Integration the first term by parts, and evaluating the third term, shows that
R′X(τ) = −
∫ vT
−∞∫ vT
−∞ x′ρ(x′, τ |x, 0)xρS(x)dx′dx+ µ∫ vT
−∞
(
∫ vT
−∞ ρ(x′, τ |x, 0)dx′)
xρS(x)dx
+ σ2
2
∫ vT
−∞∫ vT
−∞ x′ ∂2
∂x′2 ρ(x′, τ |x, 0)xρS(x)dx′dx+ νS
∫ vT
−∞ x2ρS(x)dx.
(3.152)
Using the definition of RX(τ) given in (3.144), and the fact that
∫ vT
−∞ρ(x′, τ |x, 0)dx′ = 1, (3.153)
46
we find that (3.152) reduces to
R′X(τ) = −RX(τ) + µ
∫ vT
−∞ xρS(x)dx + νS
∫ vT
−∞ x2ρS(x)dx
+ σ2
2
∫ vT
−∞∫ vT
−∞ x′ ∂2
∂x′2 ρ(x′, τ |x, 0)xρS(x)dx′dx.
(3.154)
We need to simplify the double integral in (3.154). An integration by parts gives
∫ vT
−∞x′
∂2
∂x′2ρ(x′, τ |x, 0)dx′ = vT
∂
∂x′ρ(vT , τ |x, 0) −
∫ vT
−∞
∂
∂x′ρ(x′, τ |x, 0)dx′. (3.155)
Since∫ vT
−∞
∂
∂x′ρ(x′, τ |x, 0)dx′ = ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0, (3.156)
then (3.155) becomes∫ vT
−∞x′
∂2
∂x′2ρ(x′, τ |x, 0)dx′ = vT
∂
∂x′ρ(vT , τ |x, 0). (3.157)
Substitution of (3.157) into (3.154) leads to
R′X(τ) = −RX(τ) + µ
∫ vT
−∞ xρS(x)dx + νS
∫ vT
−∞ x2ρS(x)dx
+ vT σ2
2
∫ vT
−∞ ρx′(vT , τ |x, 0)xρS(x)dx.
(3.158)
To proceed further we need to derive a second helpful identity. For this we integrate (3.145) from −∞to vT and obtain
∫ vT
−∞
∂
∂τρ(x′, τ |x, 0)dx′ =
∫ vT
−∞
(
∂
∂x′((x′ − µ)ρ(x′, τ |x, 0)) +
σ2
2
∂2
∂x′2ρ(x′, τ |x, 0) + νSδ(x
′ − x)
)
dx′.
(3.159)
Interchanging the integral and derivative on the left (hopefully this is legal), we get
∂
∂τ
∫ vT
−∞ρ(x′, τ |x, 0)dx′ = (vT − µ)ρ(vT , τ |x, 0) +
σ2
2ρx′(vT , τ |x, 0) + νS . (3.160)
Using the facts that∫ vT
−∞ ρ(x′, τ |x, 0)dx′ = 1 and ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0, this reduces to
0 =σ2
2ρx′(vT , τ |x, 0) + νS ∀τ ≥ 0, (3.161)
hence
ρx′(vT , τ |x, 0) = −2νS
σ2∀τ ≥ 0. (3.162)
Hopefully, this allows us to conclude that
ρx′τ (vT , τ |x, 0) = 0 ∀τ ≥ 0. (3.163)
Next, a differentiation of (3.154) gives the second order equation
R′′X(τ) = −R′
X(τ) +σ2
2
∫ vT
−∞
∫ vT
−∞x′
∂2
∂x′2ρτ (x′, τ |x, 0)xρS(x)dx′dx. (3.164)
47
An integration by parts reduces this equation to
R′′X(τ) = −R′
X(τ) +σ2
2
∫ vT
−∞vTρx′τ (vT , τ |x, 0)xρS(x)dx − σ2
2
∫ vT
−∞
∫ vT
−∞ρx′τ (x′, τ |x, 0)xρS(x)dx′dx.
(3.165)
Another integration by parts gives us (hopefully correct)
∫ vT
−∞x′ρx′τ (x′, τ |x, 0)dx′ = vT ρτ (vT , τ |x, 0) −
∫ vT
−∞ρτ (x′, τ |x, 0)dx′ (3.166)
We should be able to conclude that ρτ (vT , τ |x, 0) = 0 ∀τ ≥ 0 since ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0. Using
this fact, and substituting (3.166) into (3.165) gives
R′′X(τ) = −R′
X(τ) +σ2
2
∫ vT
−∞vT ρx′τ (vT , τ |x, 0)xρS(x)dx
σ2
2
∫ vT
−∞
∫ vT
−∞ρτ (x′, τ |x, 0)xρS(x)dx′dx.
(3.167)
48
(k.) The linear integrate and fire model.
This follows Mattia and Giudic [20]. and Doiron, Rinzel and Reyes [6]. The linear integrate and fire
model consists of the SDEdV
dt= µ(t) + σ(t)
dζ
dt, (3.168)
with initial condition
V (0) =
v ∈ (vL, vT ) if vL > −∞,
v ∈ (−∞, vT ) if vL = −∞.(3.169)
A ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immediately reset
to V = vR, where vR ∈ [vL, vT ). Thus, we impose the reset condition
V (t−) = vT and V (t+) = vR. (3.170)
It is important to note that either
vL < vR < vT , (3.171)
or
vL = vR < vT . (3.172)
We first assumme that (3.171) holds in order to properly derive the boundary conditons satisfied by
solutions of the Fokker-Planck equation associated with (3.168). This will allow us to use a limiting
argument to obtain the correct boundary conditions when (3.172) holds. Later we follow [6, 20] and
assume that
vR = vL = 0 and vT > 0. (3.173)
Remarks.
(i) We follow [6, 20] and assume for simplicity that σ(t) = σ > 0 is a constant. However, recent
experiments show that interesting results can occur when σ(t) is a step function [7].
(ii) Suppose that µ(t) = µ is a constant. If µ < 0, σ > 0 and vL = −∞ then
E(V (t)) = v + µt→ −∞ as t→ ∞. (3.174)
Thus, the probability of a spike occuring decreases as t increses. If µ > 0, σ > 0 and vL ≥ −∞, then
spikes occur. If µ > 0, vL ≥ −∞ and σ = 0 then a deterministic train of equally spaced spikes occurs.
(iii) An interesting phenomenon occurs when µ(t) is a step function as in [6, 20]. This causes oscil-
lations in the ’emission rate” function ν(t), which appears below in the Fokker-Planck equation. Our
main goal is to understand the analytical techniques developed in [6, 20] which show how oscillations
in ν(t) arise when µ(t) is a step function.
The Fokker-Planck equation associated with equation (3.168) consists of the pde
∂
∂tρ(V, t) = −µ(t)
∂
∂Vρ(V, t) +
σ2(t)
2
∂2
∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.175)
49
with the ’absorbing’ boundary condition
ρ(vT , t) = 0 ∀t ≥ 0, (3.176)
and normalizing condition∫ vT
vL
ρ(V, t)dV = 1 ∀t ≥ 0. (3.177)
The ’probability current’ is the function
Sρ(V, t) = µ(t)ρ(V, t) − σ2(t)
2
∂
∂Vρ(V, t). (3.178)
When vL < vR < vT solutions satisfy the reflctive boundary condition Sρ(vL, t) = 0, i. e.
µ(t)ρ(vL, t) −σ2(t)
2
∂
∂Vρ(vL, t) = 0 when vL < vR < vT . (3.179)
We assume that the functions µ(t) and σ(t) are piecewise continuous in t and V so that
ρ(V, t) is continuous in t and V, (3.180)
and∂
∂tρ(V, t) and
∂
∂Vρ(V, t) are piecewise continuous in t and V. (3.181)
Remark (iv) The specific solution ρ(V, t|v, 0) of (3.175) which is associated with the SDE initial
value problem (3.168)-(3.170) satisfies the additional condition
ρ(V, 0|vR, 0) = δ(V − vR). (3.182)
The emission rate function. We noted above in Remark (iii) that the function ν(t) in (3.175) is
called the ’emission rate’ function. Our goal is to analyze ν(t) in two distinct parameter regimes. The
first regime is when vL < vR < vT . We will derive the identities
ν(t) = −σ2(t)
2
∂
∂Vρ(vT , t) ∀t ≥ 0 when vL < vR < vT . (3.183)
and∂
∂Vρ(vT , t) =
∂
∂Vρ(v+
R , t) −∂
∂Vρ(v−R , t) ∀t ≥ 0 when vL < vR < vT . (3.184)
The second regime is vL = vR < vT . Here we let vL → v−R and show how (3.184) reduces to
∂
∂Vρ(vT , t) =
∂
∂Vρ(v+
R , t) −∂
∂Vρ(v−R , t) ∀t ≥ 0 when vL = vR < vT . (3.185)
Case I: vL < vR < vT . The first step is to integrate (3.175) from v = vL to v = vT and obtain
∂
∂t
∫ vT
vL
ρ(V, t)dV = −µ(t)
∫ vT
vL
∂
∂Vρ(V, t)dV +
∫ vT
vL
σ2(t)
2
∂2
∂V 2ρ(V, t)dv + ν(t)
∫ vT
vL
δ(V − vR)dV.
(3.186)
50
This reduces to
∂
∂t
∫ vT
vL
ρ(V, t)dV = µ(t)ρ(vL, t)− µ(t)ρ(vT , t) +σ2(t)
2
∂
∂Vρ(vT , t)−
σ2(t)
2
∂
∂Vρ(vL, t) + ν(t). (3.187)
Combining (3.187) with conditions (3.176), (3.177) and (3.179) reduces (3.187) to (3.183).
Next, we derive a second identity satisfied by ν(t). For this we integrate (3.175) from v = vR − ǫ to
v = vR + ǫ, and obtain
∂
∂t
∫ vR+ǫ
vR−ǫ
ρ(V, t)dV = −µ(t)
∫ vR+ǫ
vR−ǫ
∂
∂Vρ(V, t)dV+
∫ vR+ǫ
vR−ǫ
σ2(t)
2
∂2
∂V 2ρ(V, t)dv+ν(t)
∫ vR+ǫ
vR−ǫ
δ(V−vR)dV.
(3.188)
This reduces to
∂
∂t
∫ vR+ǫ
vR−ǫ
ρ(V, t)dV = −µ(t) (ρ(vR + ǫ, t) − ρ(vR − ǫ, t))+σ2(t)
2
(
∂
∂Vρ(vR + ǫ, t) − ∂
∂Vρ(vR − ǫ, t)
)
+ν(t).
(3.189)
Letting ǫ→ 0+ in (3.189), and using the continuity conditions (3.180) and (3.181), we conclude that
ν(t) = −σ2(t)
2
(
∂
∂Vρ(v+
R , t) −∂
∂Vρ(v−R , t)
)
∀t ≥ 0 when vL < vR < vT . (3.190)
Combining (3.183) and (3.190) gives the identity
∂
∂Vρ(vT , t) =
∂
∂Vρ(v+
R , t) −∂
∂Vρ(v−R , t) ∀t ≥ 0 when vL < vR < vT . (3.191)
Case II: vL = vR < vT . Recall from (3.179) that
µ(t)ρ(vL, t) −σ2(t)
2
∂
∂Vρ(vL, t) = 0 ∀t ≥ 0 when vL < vR < vT . (3.192)
Letting vL → v−R reduces this to
µ(t)ρ(v−R , t) −σ2(t)
2ρV (v−R , t) = 0 ∀t ≥ 0 when vL = vR < vT . (3.193)
The continuity property (3.180) implies that ρ(v−R , t) = ρ(vR, t) ∀t ≥ 0. This, together with (3.193),
imply that∂
∂Vρ(v−R , t) =
2µ(t)
σ2(t)ρ(vR, t) ∀t ≥ 0 when vL = vR < vT . (3.194)
Finally, we substitute (3.194) into (3.191) and obtain the identity
∂
∂Vρ(vT , t) =
∂
∂Vρ(v+
R , t) −2µ(t)
σ2(t)ρ(vR, t) ∀t ≥ 0 when vL = vR < vT . (3.195)
Remark (v). We can remove the term ν(t)δ(V −vR) from (3.175) when vL = vR, and instead impose
the additional condition (3.195). We do this below where we restrict our attention to the parameter
regime vL = vR = 0 and θ = vT > 0.
51
Eigenfunctions: Basic Theory. Following [20], we assume that
vL = vR = 0 and vT = θ > 0. (3.196)
Our goals are to answer the following questions:
(I.) What is the ODE problem satisfied by eigenfunctions φ(V, t) of the Fokker-Planck equation?
(II.) What is the related ’adjoint’ ODE problem?
(III.) How to find eigenvalues λ and eigenfunctions φλ(V, t) and ψλ(V, t)?
(IV.) How to compute the eigenvalues λ and eigenfunctions φλ(V, t)?
(V.) How to find ψλ(V, t) and ∂∂µψλ(V, t) for the adjoint problem?
(VI.) Stationary solution: how to find φ0(V, t) and ψ0(V, t) when λ = 0?
(I.) We look for a solution of the Fokker-Planck equation (3.175) which has the form
ρ(v, t) = φ(V, t)eλt. (3.197)
The eigenfunction φ(V, t) satisfies
λφ(V, t) = −µ(t)∂
∂Vφ(V, t) +
σ2(t)
2
∂2
∂V 2φ(V, t), (3.198)
with absorbing condition
φ(θ, t) = 0. (3.199)
Because of (3.196), we need to impose boundary condition (3.195), namely
∂
∂Vφ(θ, t) =
∂
∂Vφ(0, t) − 2µ(t)
σ2(t)φ(0, t) ∀t ≥ 0. (3.200)
We will see below that the eigenvalues λ are complex when µ(t) is a positive constant, and they are
real and negative when µ(t) ≤ 0 is zero or a negative constant
(II.) To understand how the adjoint ODE problem arises, we define two linear operators. The first is
the operaor L defined by
Lφ(V, t) = −µ(t)∂
∂Vφ(V, t) +
σ2(t)
2
∂2
∂V 2φ(V, t). (3.201)
Thus, an eigenfunction φ(V, t), and its eigenvalue λ, satisfy
Lφ(V, t) = λφ(V, t). (3.202)
Define the inner product
(ψ(V, t), φ(V, t)) =
∫ θ
0
ψ(V, t), φ(V, t)dV. (3.203)
52
The second linear operator, the ’adjoint operator’ L+, is defined by
(
L+ψ(V, t), φ(V, t))
= (ψ(V, t), Lφ(V, t)) , (3.204)
and satisfies
L+ψ(V, t) = µ(t)∂
∂Vψ(V, t) +
σ2(t)
2
∂2
∂V 2ψ(V, t). (3.205)
For each eigenfunction φ, and an associated eigenvalue λ, of L, there is a corresponding eigenfunction
ψ(V, t) of L+ which satisfies
λψ(V, t) = µ(t)∂
∂Vψ(V, t) +
σ2(t)
2
∂2
∂V 2ψ(V, t). (3.206)
It follows from (3.204) applied to φ and ψ, and conditions (3.199) and (3.200), that
ψ(0) = ψ(θ) and∂
∂Vψ(0, t) = 0. (3.207)
It also follows from these properties that if ψ(V, t) and φ(V, t) are eigenfunctions with eigenalues λ′
and λ, respectively, then ψ(V, t) and φ(V, t) satisfy the orthogonality condition
(ψ(V, t), φ(V, t)) = 0 if λ 6= λ′, (3.208)
and the normalizing property
(ψ(V, t), φ(V, t)) = 1 if λ = λ′. (3.209)
(III.) How to find the eigenvalues λ and eigenfunctions φλ(V, t) and ψλ(V, t)
The first step is to substitute
φλ(V, t) = exp
(
mV
θ
)
(3.210)
into
λφλ(V, t) = −µ(t)∂
∂Vφλ(V, t) +
σ2(t)
2
∂2
∂V 2φλ(V, t), (3.211)
and obtain the algebra problem
m2 − 2µθ
σ2m− 2λθ2
σ2= 0. (3.212)
Note that the variable t appears in m through µ(t) or σ(t). Solving (3.212) gives
m = ζ ± γ, (3.213)
where
ζ =µθ
σ2and γ =
θ
σ2
√
µ2 + 2λσ2 (3.214)
Thus, the general solution of (3.198) is
φλ(V, t) = exp
(
ζV
θ
)(
c1 exp
(
γV
θ
)
+ c2 exp
(
−γVθ
))
. (3.215)
53
It follows from (3.215), the boundary condition φλ(θ, t) = 0, and an algebraic manipulation, that
φλ(V, t) = cλ exp
(
ζV
θ
)
sinh
(
γ(θ − V )
θ
)
, (3.216)
where cλ is a constant which is possibly complex, and is determined below in (3.245)-(3.246).
The next step is to substitute (3.243) into the second boundary condition
∂
∂Vφλ(θ, t) =
∂
∂Vφλ(0, t) − 2µ(t)
σ2(t)φλ(0, t) ∀t ≥ 0. (3.217)
This gives the ’characteristic equation’ from which the eigenvalues are computed, namely
γ exp(ζ) = γ cosh(γ) + ζ sinh(γ), (3.218)
where we recall that
where ζ =µθ
σ2and γ =
θ
σ2
√
µ2 + 2λσ2.
Finally, it follows from an integration of (3.211) from V = 0 to V = θ, and the boundary conditions
φλ(θ, t) = 0,∂
∂Vφλ(θ, t) =
∂
∂Vφλ(0, t) − 2µ(t)
σ2(t)φλ(0, t),
that∫ θ
0
φλ(V, t)dV = 0 when λ 6= 0. (3.219)
Note. We will show below that, if λ = 0, then
∫ θ
0
φ0(V, t)dV = 1. (3.220)
54
(IV.) How to compute the eigenvalues λ and eigenfunctions φλ(V, t)
Recall that the eigenvalues λ satisfy
ζ =µθ
σ2and γ =
θ
σ2
√
µ2 + 2λσ2. (3.221)
and that γ satsifies
γ exp(ζ) = γ cosh(γ) + ζ sinh(γ), (3.222)
Substituting γ = γ1 + iγ2 into (3.221) and separating real and imaginary parts gives the system
γ1 exp(ζ) − γ1 cosh(γ1) cos(γ2) + γ2 sinh(γ1) sin(γ2) − ζ sinh(γ1) cos(γ2) = 0,
γ2 exp(ζ) − γ2 cosh(γ1) cos(γ2) − γ1 sinh(γ1) sin(γ2) − ζ cosh(γ1) sin(γ2) = 0.(3.223)
We asssume that λ is a function of µ, hence γ = γ1 + iγ2 is also a function of µ. Our goal is to derive
ODE’s satisfied by γ1(µ) and γ2(µ). We then use (3.221) to find the values of λ.
The first step is to find the initial values for γ1(µ) and γ2(µ). Mattia et al [20] show that
λN = −σ2
θ22N2π2, N ≥ 1 when µ = 0. (3.224)
Thus, from (3.221) and (3.224) it follows that
γ1(0) = 0 and γ2(0) = 2Nπ, N ≥ 1 when µ = 0. (3.225)
Next, differentiate (3.221) with respect to µ and obtain
dγ
dµ
(
eζ − (1 + ζ) cosh(γ) − γ sinh(γ))
=θ
σ2
(
sinh(γ) − γeζ)
. (3.226)
We will make use of the identities
sinh(γ) = sinh(γ1) cos(γ2) + i cosh(γ1) sin(γ2), (3.227)
cosh(γ) = cosh(γ1) cos(γ2) + i sinh(γ1) sin(γ2), (3.228)
andγ sinh(γ) = γ1 sinh(γ1) cos(γ2) − γ2 cosh(γ1) sin(γ2)
+ i (γ2 sinh(γ1) cos(γ2) + γ1 cosh(γ2) sin(γ2)) .(3.229)
We write
eζ − (1 + ζ) cosh(γ) − γ sinh(γ) = A + Bi, (3.230)
where
A = eζ − (1 + ζ) cosh(γ1) cos(γ2) − γ1 sinh(γ1) cos(γ2) + γ2 cosh(γ1) sin(γ2),
B = −(1 + ζ) sinh(γ1) sin(γ2) − γ2 sinh(γ1) cos(γ2) − γ1 cosh(γ1) sin(γ2).(3.231)
55
Next, we writeθ
σ2
(
sinh(γ1) − γeζ)
= C + Di, (3.232)
whereC = θ
σ2
(
sinh(γ1) cos(γ2) − eζγ1
)
,
D = θσ2
(
cosh(γ1) sin(γ2) − eζγ2
)
.(3.233)
Combining (3.230), (3.230), (3.231), (3.232) and (3.233), we obtain
γ′1(µ) = CA+DBA2+B2 , γ1(0) ≈ constant
õ,
γ′2(µ) = DA−CBA2+B2 , γ2(0) ≈ 2Nπ.
(3.234)
After solving (3.234), we substitute γ1 and γ2 into (3.221) and get
λN =σ2
2θ2(
γ21 − γ2
2
)
− µ2
2σ2− i
σ2
2θ2γ1γ2. (3.235)
To find the γ and λ values numerically, and reproduce Figures 1 and 2 of Mattia et al, we use the
programs described below.
The programs lifeigenvalues.ode, twodnewton.m, f1.m, f2.m These can be found at
www.math.pitt.edu/∼troy/lif/
Goal: use lifeigenvalues.m, twodnewton.m, f1.m and f2.m to find the eigenvalues, and to reproduce
Figure 1 in Mattia et al. The results are shown below in Figure 1. In the xpp program lifeigenvalues.ode
we set
µ = start = .1001, σ = 1, θ = 1, N = 1, dt = .0005
and continue the solution forward to µ=5, and backwards to µ = 0. The the variable t represents
µ. The intial conditions are found by the two dimensional Newton iteration program twodnewton.m
which finds the gamma values to 12 decimal accuracy.
The matlab programs mattifig2eigenvalues.m, f1a., f2a.m These can be found at
www.math.pitt.edu/∼troy/lif/mattiafigure2/
Goal: use mattiafig2eigenvalues.m, f1a.m, f2a.m to find the eigenvalues as a first step in reproducing
the deterministic firing rate diagram in Figure 2 of Mattia et al (see Figures 2 and 4). The program
mattiafig2eigenvalues.m solves eqs (3.223) by two dimensional newton iteration when
µ = 25, σ = 1, θ = 1.
These are the values used by Mattia et al. The left sides of eqs. (3.223) are defined in f1a.m and f2a.m
Further details are given in a later section.
We now continue the analysis and show how to find the eigenfunction ψλ(V, t). Substitute
ψλ(V, t) = exp
(
mV
θ
)
(3.236)
56
-100
-50
0
50
100
0 1 2 3 4 5
Figure 1: Imaginary part of the eigenvalues vs. µ The programs for these computations are are
lifeigenvalues.m, twodnewton.m, f1.m, f2.m
into
λψλ(V, t) = µ(t)∂
∂Vψλ(V, t) +
σ2(t)
2
∂2
∂V 2ψλ(V, t), (3.237)
and obtain the algebra equation
m2 +2µθ
σ2m− 2λθ2
σ2= 0. (3.238)
Solving (3.238) gives
m = −ζ ± γ, (3.239)
where
ζ =µθ
σ2and γ =
θ
σ2
√
µ2 + 2λσ2. (3.240)
Thus, the general solution of (3.237) is
ψλ(V, t) = exp
(
−ζVθ
)(
c1 exp
(
γV
θ
)
+ c2 exp
(
−γVθ
))
. (3.241)
Substituting the boundary condtions
∂
∂Vψλ(0, t) = 0 and ψλ(0, t) = ψλ(θ, t) (3.242)
57
into (3.241), we find that the eigenfunction ψλ(V, t) is given by
ψλ(V, t) = exp
(
−ζVθ
)(
γ cosh
(
γV
θ
)
+ ζ sinh
(
γV )
θ
))
. (3.243)
It remains to find the value cλ such that the ’normalizing’ condition holds, i.e
∫ θ
0
cλφλ(V, t)ψλ(V, t)dV = 1. (3.244)
Substitution of (3.243) and (3.243) into (3.244) gives
cλ
∫ θ
0
sinh
(
γ(θ − V )
θ
)(
γ cosh
(
γV
θ
)
+ ζ sinh
(
γV )
θ
))
dV = 1. (3.245)
An integration shows that
cλ =2γ
θ [ζγ cosh(γ) + (γ2 − ζ2) sinh(γ)](3.246)
Finally, we have
∂
∂µψλ(V, t) = exp
(
−ζVθ
)[
γ cosh
(
γV
θ
)
+ ζ sinh
(
γV
θ
)]
. (3.247)
(VI.) Stationary solution: how to find φ0(V, t) and ψ0(V, t) when λ = 0?
To find a stationary solution we set λ = 0 in (3.198), and look for a solution φ0(V, t) of
0 = −µ(t)∂
∂Vφ0(V, t) +
σ2(t)
2
∂2
∂V 2φ0(V, t) ∀t ≥ 0, (3.248)
with boundary conditions
φ0(θ, t) = 0 (3.249)
and∂
∂Vφ0(θ, t) =
∂
∂Vφ0(0, t) −
2µ(t)
σ2(t)φ0(0, t) ∀t ≥ 0, (3.250)
and normalizing condition∫ θ
0
φ0(V, t)dV = 1 ∀t ≥ 0, (3.251)
Note that φ0(V, t) candepend on t since µ(t) in (3.250) may be a function of t.
The eigenfunction ψ0(V, t) is given by
ψ0(V, t) = 1 ∀ V ∈ [0, θ], (3.252)
hence∫ θ
0
φ0(V, t)ψ0(V, t)dV =
∫ θ
0
φ0(V, t)dV = 1 ∀ V ∈ [0, θ]. (3.253)
Thus, φ0 satisfies the normalizing condition
∫ θ
0
φ0(V, t)dV = 1 ∀t ≥ 0. (3.254)
58
Note. The exact formula for φ0 depends on whether µ = 0 or µ 6= 0.
Case 1: the Eigenfunctions and eigenvalues when µ(t) = 0 ∀t ≥ 0.
Our goals are to address the following issues:
(I.) When µ(t) = 0 ∀t ≥ 0 prove that the solutions φ0(V ) and ψ0(V ) are given by
φ0(V ) =2
θ2(θ − V ) and ψ0(V ) = 1, 0 ≤ V ≤ θ. (3.255)
(II.) When µ(t) = 0 ∀t ≥ 0 prove that
λN = −σ2
θ22N2π2, φN (V, t) = sin
(
2NπV
θ
)
, ψN = cos
(
2NπV
θ
)
, N ≥ 1. (3.256)
(III.) When µ(t) = 0 ∀t ≥ 0 prove that
∫ θ
0
φN (V, t)dV = 0 and
∫ θ
0
φN (V, t)ψN (V, t)dV = 0, N ≥ 1. (3.257)
(IV.) When µ(t) = 0 ∀t ≥ 0 is there a formula for ρ(V, t|0, 0)? In particular we will show that there
is a major problem with getting the property ρ(V, 0|0, 0) = δ(V ) to hole when µ = 0. The difficulty is
that ∈θ0 φN (V )ψN (V )dV = 0 when µ = 0.
(V.) When µ(t) = 0 ∀t ≥ 0 is there a formula for the firing rate ν(t)?
Proof of (I.) We set µ(t) = 0 in (3.248) and investigate the existence of a solution of
∂2
∂V 2φ0(V ) = 0, (3.258)
with boundary conditions
φ0(θ) = 0 and∂
∂Vφ0(θ) =
∂
∂Vφ0(0), (3.259)
and normalizing condition∫ θ
0
φ0(V )dV = 1. (3.260)
Note that φ0(V ) is independent of t since µ(t) = 0 ∀t ≥ 0.
The general solution of (3.259) is
φ0(V ) = c1V + c2, 0 ≤ V ≤ θ. (3.261)
Combining (3.261) with (3.259) and (3.260 shows that
φ0(V ) =2
θ2(θ − V ) , 0 ≤ V ≤ θ. (3.262)
As shown earlier the eigenfunction ψ0(V ) is given by
ψ0(V ) = 1, 0 ≤ V ≤ θ. (3.263)
59
Proof of (II.) To find the non-zero eigenvalues and eigenfunctions when µ(t) = 0, we solve
∂2
∂V 2φ(V ) − 2λ
σ2φ(V ) = 0, (3.264)
with boundary conditions
φ(θ) = 0 and∂
∂Vφ(θ) =
∂
∂Vφ(0) (3.265)
The eigenvalues and eigenfunctions are
λN = −σ2
θ22N2π2 and φN (V ) = sin
(
2NπV
θ
)
, N ≥ 1. (3.266)
Next, we solve the related adjoint boundary value problem
∂2
∂V 2ψ(V ) − 2λN
σ2ψ(V ) = 0, (3.267)
with boundary conditions∂
∂Vψ(θ) = 0 and ψ(0) = ψ(θ) (3.268)
The solution of (3.267)-(3.268) is
ψN (V ) = cos
(
2Nπ
θV
)
. (3.269)
Proof of (III.) When µ(t) = 0 ∀t ≥ 0 the eigenfunctions satisfy
∫ θ
0
φN (V )dV =
∫ θ
0
sin
(
2NπV
θ
)
dV = 0, N ≥ 1, (3.270)
and∫ θ
0
φN (V )ψN (V )dV =
∫ θ
0
sin
(
2NπV
θ
)
cos
(
2Nπ
θV
)
dV = 0, N ≥ 1. (3.271)
Proof of (IV.) Our goal is to derive a formula for ρ(V, t|0, 0) when µ(t) = 0 ∀t ≥ 0.
We follow the methods in Gardner [11] and assume that ρ(V, t|0, 0) exists, and that there are values
AN |N ≥ 1 such that
ρ(V, t|0, 0) = φ0(V ) +
∞∑
N=1
ANφN (V )e−λN t, (3.272)
where
φ0(V ) =2
θ2(θ − V ) , λN = −σ
2
θ22N2π2 and φN (V ) = sin
(
2NπV
θ
)
, N ≥ 1 (3.273)
Recall that ρ(V, 0|0, 0) = δ(V ). Thus, setting t = 0 in (3.272) gives
δ(V ) =2
θ2(θ − V ) +
∞∑
N=1
AN sin
(
2NπV
θ
)
. (3.274)
60
Note that δ(V ) is required to satisfy
δ(V ) = 0, 0 < V ≤ θ. (3.275)
From (3.274) it follows that
δ(0) =2
θand δ
(
θ
2
)
=1
θ. (3.276)
But this seems to contradict (3.275). If so, then do we conclude there the pdf ρ(V, t|0, 0) does not
satisfy ρ(V, 0|0, 0) = δ(V )? One possibility is that ρ(V, t|0, 0) satisfies
ρ(V, t|0, 0) ≡ φ0(V ) =2
θ2(θ − V ) , 0 ≤ V ≤ θ and t ≥ 0. (3.277)
(V.) To find the firing rate ν(t) when µ(t) = 0 ∀t ≥ 0, we begin with the definition
ν(t) = −σ2(t)
2
∂
∂Vρ(θ, t|0, 0). (3.278)
Suppose that (3.277) holds. Then combining (3.277) with (3.278), we obtain
ν(t) =σ2(t)
θ2, t ≥ 0. (3.279)
This all seems illegitimate.
Case 2: the eigenvalues and eigenfunctions when µ(t) 6= 0 for all t ≥ 0.
Our goal here is to prove the following:
(I.) When µ(t) 6= 0 prove that the solutions φ0(V ) and ψ0(V ) exist and are given by
φ0(V, t) =c
µ(t)
(
1 − exp(−2ζ(θ − V )
θ)
)
, (3.280)
where
c =
[
σ2(t)
2µ2(t)
(
2µ(t)θ
σ2(t)− 1 + exp
(
−2µ(t)θ
σ2(t)
))]−1
and ζ =µ(t)θ
σ2(t). (3.281)
(II.) When µ(t) 6= 0 the non-zero eigenvalues and their eigenfunctions are
(III.) When µ(t) 6= 0 the eigenfunctions satisfy
∫ θ
0
φN (V, t)ψN (V, t)dV = 1, N = 0,±1,±2.. (3.282)
Proof of (I.) Mattia et al [20] analyze the case µ(t) 6= 0 and show that the stationary solution
of (3.248)-(3.249)-(3.250)-(3.254) exists and is given by
φ0(V, t) =c
µ(t)
(
1 − exp(−2ζ(θ − V )
θ)
)
, (3.283)
where
c =
[
σ2(t)
2µ2(t)
(
2µ(t)θ
σ2(t)− 1 + exp
(
−2µ(t)θ
σ2(t)
))]−1
and ζ =µ(t)θ
σ2(t). (3.284)
61
Furthermore, it is easily verified that
limµ(t)→0+
φ0(V, t) =2
θ2(θ − V ) , 0 ≤ V ≤ theta, (3.285)
in agreeement with (3.280)
The firing rate ν(t) when µ(t) 6= 0 and ρ(V, t|0, 0) ≈ φ0(V, t).
Here we assume the Fokker-Planck solution reaches equilibrium, i.e. ρ(V, t) ≈ φ0(V, t). It follows
from (3.183) and (3.196) that ν(t) satisfies
ν(t) = −σ2(t)
2
∂
∂Vφ0(θ, t) ∀t ≥ 0. (3.286)
Substitution of (3.283)-(3.284) into (3.286) gives
ν(t) = c =
[
σ2(t)
2µ2(t)θ
(
2µ(t)θ
σ2(t)− 1 + exp
(
−2µ(t)θ
σ2(t)
))]−1
and ζ =µ(t)θ
σ2(t). (3.287)
An analysis of (3.287) shows that
ν(t) ≈ µ(t) when µ(t) is large. (3.288)
Interestingly, when µ(t) = 0 and no stationary solution exists, formula (3.286) indicates that the firing
rate is given by
ν(t) =σ2
θ2when µ(t) = 0. (3.289)
We compare this formula with that derived from first passage techniques (this assumes that the pdf
ρ(V, t|0, 0) exists), beginning with the definition
ν(t) =1
E(T (0)). (3.290)
Here E(T (0)) denotes the mean first passage time for V (t) to reach V (t) = θ from initial value
V (0) = vR = 0. To find E(T (0)) we set µ(t) = 0 and solve the ODE problem
σ2
2M ′′(V ) = −1, M ′(0) = M(θ) = 0. (3.291)
The solution of (3.291) is
M(V ) =θ2 − V 2
σ2. (3.292)
When V = 0 this reduces to
M(0) = E(T (0)) =θ2
σ2. (3.293)
Substituting (3.293) into (3.290) gives ν(t) = σ2
θ2 , in agreement with (3.289).
How to find the firing rate ν(t) when µ(t) 6= 0.
62
Recall from (3.183) that the firing rate ν(t) satisfies by
ν(t) = −σ2(t)
2
∂
∂Vρ(vT , t) ∀t ≥ 0. (3.294)
We assume that
ρ(V, t) =∑
N
aN(t)φN (V, t), −∞ < N <∞. (3.295)
To find each ak(t) multiply (3.295) by ψk(V, t) and integrate from V = 0 to V = θ to get
∫ θ
0
ψk(V, t)ρ(V, t)dV =
∫ θ
0
ψk(V, t)
(
∑
N
aN (t)φN (V, t)
)
dV, −∞ < N <∞ (3.296)
This reduces to
ak(t) = (ψk(V, t), ρ(V, t)) =
∫ θ
0
ψk(V, t)ρ(V, t)dV (3.297)
since
(ψk(V, t), φj(V, t)) = 0 when k 6= j, and (ψk(V, t), φk(V, t)) = 1. (3.298)
Thus, for each N we have
aN (t) = (ψN (V, t), ρ(V, t)) =
∫ θ
0
ψN (V, t)ρ(V, t)dV. (3.299)
Differentiating both sides of (3.299) with respect to t gives
d
dtaN (t) =
(
ψN (V, t),∂
∂tρ(V, t)
)
+dµ(t)
dt
(
∂
∂µψN (V, t), ρ(V, t)
)
. (3.300)
This reduces to
d
dtaN(t) = (ψN (V, t), Lρ(V, t)) +
dµ(t)
dt
(
∂
∂µψN (V, t), ρ(V, t)
)
, (3.301)
henced
dtaN (t) =
(
L+ψN (V, t), ρ(V, t))
+dµ(t)
dt
(
∂
∂µψN (V, t), ρ(V, t)
)
, (3.302)
and therefored
dtaN (t) = (λNψN (V, t), ρ(V, t)) +
dµ(t)
dt
(
∂
∂µψN (V, t), ρ(V, t)
)
, (3.303)
Combining (3.304) with (3.295) and (3.299) gives
d
dtaN (t) = λNaN (t) +
dµ(t)
dt
∑
m
am(t)
(
∂
∂µψN (V, t), φm(V, t)
)
, (3.304)
The hard part is to find all the eigenvalues λN ’s and eigenfunctions φN ’s.
63
How to compute the ‘deterministic’ and population firing rate functyions.
Our goal here is to complete the computation of Figure 2 (left panel) of Mattia et al.
The deterministic firing rate. The non-noisy deterministic firing rate is shown in the upper left
panel of Figure 2. For this computation we use several matlab programs, all of which can be found at
www.math.pitt.edu/∼troy/lif/mattiafigure2/
(i) mattiafig2eigenvalues.m, f1a., f2a.m are used to compute the eigenvalues by means of a two dimen-
sional Newton iteration scheme when
µ = 25, σ = 1, θ = 1.
(ii) mattiafigure2driver.m, mattiafig2psi.m, mattiafig2odes.m, mattiafig2phiprime.m, are used to-
gether with the matlab ODE45 program to find the aN (t) functions, and subsequently to construct
the deterministic firing rate function. The program mattiafig2odes.m uses the command load mat-
tiafig2gammavalues.dat to construct the lambda values which appear on the right side of the aN (t)
ODES.
The population (i.e. noisy) firing rate. We consider a population of N uncoupled (i.e. in-
dependent) cells and obtain the ”noisy” firing rate function shown in Figure 2 (upper right panel).
Each cell is modelled by the linear integrate and fire SDE
dV
dt= µ(t) + σ(t)
dζ
dt, V (0) = 0. (3.305)
We require (see Figure 4) that the solution V (t) satisfy
0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.306)
Recall that a ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immedi-
ately reset to V = vR = 0. The reset condition
V (t−) = vT and V (t+) = vR. (3.307)
The procedure to obtain the firing rate is as follows: for each i ∈ [1, N ] solve (3.305)-(3.306)-(3.307)
and obtain the corresponding spike train
yi(t) =∑
j
δ(t− tij), (3.308)
where the values tij are the spike times. The firing rate ν(t) is defined by
ν(t) = limN→∞
1
N
N∑
i=1
∑
j
δ(t− tij)
. (3.309)
64
0 100 200 4000
30
60
t (msec)
V
0 100 200 4000
30
60
t (msec)
V
0 100 200 4000
30
60
t (msec)
V
0 100 200 4000
30
60
t (msec)
V
Figure 2: Upper left: “deterministic ” firing rate function computed from solving the Fokker-Planck
equation. Upper right: noisy firing rate function. Lower left: deterministic and noisy firing rate
functions superimposed. Lower right: the dashed curves denote the deterministic firing rate plus or
minus 2.5, where the value 2.5 is estimated from the finite size effect formula (??). Parameters are
µ = 25 and θ = σ = 1.
We let ∆ > 0 be small and approximate ν(t) by
ν(t) ≈ 1
N
N∑
i=1
∑
j
σij(t, t+ ∆)
∆
, (3.310)
where
σij(t, t+ ∆) =
1 if tij ∈ (t, t+ ∆),
0 otherwise.(3.311)
Matlab program mattiafigure2noisyfiringrate.m This program can be found at
65
0 100 200 4000
30
60
t (msec)
V
0 0.1 0.2 0.3 0.40
10
20
30
40
50
60
Figure 3: Left: deterministic firing rate function computed with matlab program ODE45. Right:
computation of the deterministic firing rate using the explicit Euler method. The ODE45 method gives
a better fit to the SDE generated population firing rate shown in Figure 2. The matlab driver programs
for these computations are www.math.pitt.edu/∼troy/lif/mattiafigure2/mattiafig2drivernew.m and
mattiafig2eulerdriver.m
0 500 10000
0.5
1
t (msec)
V
Figure 4: A solution of (3.305)-(3.306)-(3.307) for parameter values µ = 25, θ = 1, σ = 1. The problem
is solved for 0 ≤ t ≤ 1 second. The t values are multiplied by 1000 to illustrate milliseconds.
www.math.pitt.edu/∼troy/lif/mattiafigure2/mattiafig2noisyfiringrate.m
The main goal of mattiafigure2noisyfiringrate.m is to obtain the approximation for ν(t) given by (3.310)-
(3.311) (see Figure 2).
Step I. In the first part of the program we create a set of spike times. For this define N (the number
of uncoupled cells in the population) and ∆ by
N = 10000 and ∆ = .001 (3.312)
66
Define the define the step size for the solution of (3.305)-(3.306)-(3.307) by
dt =∆
100=.001
100= .000001 (3.313)
The number of time steps used to solve (3.305)-(3.306)-(3.307) for one cell is
K =1
dt= 100000. (3.314)
Next, set V (0) = 0, and initialize a row containing the spike times by the commands
V(0)=0; spiketimes=[]
Use the two for loops described below to solve (3.305)-(3.306)-(3.307) N=10000 times using Euler’s
method. During each run save the spike times in the row spiketimes.
for i=1:N (i.e. the number of cells is N)
for k=1:K (i.e. the number of time steps per solution is K)
V ((k + 1) ∗ dt) = V (k ∗ dt) + µ ∗ dt+ σ ∗√dt ∗ randn
Check for a spike: if V ((k + 1) ∗ dt) = 1 then V ((k + 1) ∗ dt) = 0 (reset condition) and add the
number t=(k+1)*dt to the row containing spike times. To do this use the command
spiketimes=[spiketimes (k+1)*dt];
end; end;
Step II. Sort through the row of spiketimes and create the function
n(t, t+ ∆t) = the number of spikes in (t, t+ ∆t), (3.315)
This function will then be used to obtain
ν(t) =n(t, t+ ∆t)
N∆t. (3.316)
To obtain ν(t, t+ ∆) we subdivide [0,1] into 1/∆ subintervals of (0,1]. We define the vector envalues,
which is the approximation to n(t, t+ ∆), by
envalues(kk) = the number of spikes in ((kk − 1)∆, kk∆] , 1 ≤ kk ≤ 1
∆= 1000. (3.317)
It takes 20 minutes to compute envalues. We then construct the noisy firing rate with the command
firingrate = envalues/(numberofcells ∗ ∆),
where numberofcells = N = 10000, ∆ = .001 and save it externally with the commands
tt = linspace(0, 1, 1/delta);
aa = [tt firingrate];
67
save mattiafig2noisyfiringrate.dat − ascii aa
Matlab program sortingexample.m This program can be found at
www.math.pitt.edu/∼troy/lif/mattiafigure2/sortingexample.m
The program sortingexample.m gives a simple example showing the algorithm used to create the vector
envalues by sorting a row of 12 spiketimes according to the number of spikes in 1/∆ subintervals of
[0,1] of length ∆.
Step III. In the third part of the program mattiafigure2noisyfiringrate.m we draw the four firing rate
pictures in Figure 2. To do this use the load command to get the externally saved data for both the
deterministic non noisy firing rate, and the noisy firing rate.
To load the detereministic firing rate data use the two commands
loadmattiafig2drivernewfiringrate.dat;
nonnoisyfiringrate = mattiafig2drivernewfiringrate;
To load the noisy firing rate data use the two commands
loadnoisyfiringrate.dat;
noisyfiringratedata = noisyfiringrate;
We use these data sets to obtain Figure 2.
68
How to solve the basic firing rate step problem.
We consider a population of N statistically identical, independent neuronal cells. Our goal is to
compute the deterministic and population firing rates ν∞(t) and νN (t) in “layer I”in response to a
step input from “layer 0.” Our step input example is
µ(t) =
0 if 0 ≤ t < 1,
25 if t ≥ 1.(3.318)
Each cell is modelled by the linear integrate and fire SDE
dV
dt= µ(t) + σ
dζ
dt, V (0) = 0, (3.319)
with the constraint that
0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.320)
Recall that a ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immedi-
ately reset to V = vR = 0. The reset condition is
V (t−) = vT and V (t+) = vR. (3.321)
In our computations we set
vR = 0, vT = θ = 1 and σ = 1. (3.322)
The Fokker-Planck problem associated with (3.318)-(3.322) consists of the pde
∂
∂tρ(V, t) = −µ(t)
∂
∂Vρ(V, t) +
σ2(t)
2
∂2
∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.323)
with the ’absorbing’ boundary condition
ρ(vT , t) = 0 ∀t ≥ 0, (3.324)
and normalizing condition∫ vT
vL
ρ(V, t)dV = 1 ∀t ≥ 0. (3.325)
We let ρ(V, t|0, 0) denote the particular solution of (3.323)-(3.324)-(3.325) such that
ρ(V, 0|0, 0) = δ(V ). (3.326)
The first theoretical difficulty is due to the fact that when µ(t) = 0 over [0, 1) we could not find a
solution ρ(V, t|0, 0) of the full problem (3.323)-(3.324)-(3.325)-(3.326). However, there is a stationary
solution which we might be able to use, namely
ρS(V ) =2
θ2(θ − V ). (3.327)
69
0 500 1000 15000
0.5
1
t (msec)
V
0 500 10000
30
60
t (msec)
νN
0 500 10000
1
3
5
t (msec)
νN
1000 14000
30
60
t (msec)
νN
Figure 5: Upper left: solution of the SDE problem (3.319)-(3.322). Parameters: σ = θ = 1,
µ(t) = 0 when t ∈ [0, 1), and µ(t) = 25 when t ∈ [1, 2]. Time has been translated to msec.
Upper right: population firing rate νN (t). Lower left: blowup of the firing rate diagram over
the first 1000 msec. shows that νN (t) quickly reaches the stationary firing rate νs = 1 as pre-
dicted by the theory in (3.329). Lower right: blowup of the firing rate diagram over the sec-
ond 1000 msec. shows the ringing effect. Computations were done with the matlab program
www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemfiringrate.m
The deterministic firing rate function ν∞(t) is defined by
ν∞(t) = −σ2
2
∂
∂Vρ(vT , t|0, 0) ∀t ≥ 0. (3.328)
Since there is only a stationary pdf when 0 ≤ t < 1 and µ(t) = 0, we conjecture that ρ(V, t|0, 0) =
ρS(V ) when 0 ≤ t < 1, and that
ν∞(t) = −σ2
2ρ′S(θ) =
σ2
θ2= 1 ∀t ∈ [0.1) (3.329)
70
0 500 10000
30
60
t (msec)
ν∞
0 500 10000
30
60
t (msec)
νN
0 500 10000
1
3
5
t (msec)
νN
1000 14000
30
60
t (msec)
νN
Figure 6: Upper left: infinite size firing rate ν∞(t). Parameters: σ = θ = 1, µ(t) = 0
when t ∈ [0, 1), and µ(t) = 25 when t ∈ [1, 2]. Time has been translated to msec. Up-
per right: population firing rate νN (t) and deterministic firing rate ν∞(t). Lower left: blowup
of the firing rate diagram over the first 1000 msec. Lower right: blowup over the second
1000 msec. shows the ringing effect. Computations were done with the matlab program
www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemnoisyfiringrate.m
Our numerical experiments illustrated in Figure 5 (upper right and lower left panels) indicate that
the prediction given in (3.329) does hold.
Next, we determine ρ(V, t) and ν∞(t) when t ≥ 1. We assume that ρ(V, t) has the form
ρ(V, t) =
∞∑
n=−∞an(t)φn(V, t), t ≥ 1 (3.330)
where φn(V, t) is the eigenfunction associated with the eigenvalue λn when µ = 25. Thus, we get
ν∞(t) = −σ2
2
∂
∂Vρ(θ, t) ∀t ≥ 1. (3.331)
71
0 500 10000
1
3
5
t (msec)
νN
1000 1500 200024
26
29
t (msec)
νN
Figure 7: Left panel: blowup of the first 1000 msec of the firing rate diagram shown in Figure 5.
Right panel: blowup of the second 1000 msec. The same vertical and horizontal scales in each panel
illustrate the significance of the finite size effect due to ν∞(t) changing from ν∞(t) = 1 (left panel) to
ν∞(t) = 25 (right panel) in the formula νN (t) = ν∞(t) +√
ν∞(t)N ζ(t).
Combining (3.330) and (3.331) gives
ν∞(t) = −σ2
2
∞∑
n=−∞an(t)
∂
∂Vφn(θ, t) ∀t ≥ 1. (3.332)
To find each ak(t) multiply (3.295) by ψk(V, t) and integrate from V = 0 to V = θ to get
∫ θ
0
ψk(V, t)ρ(V, t)dV =
∫ θ
0
ψk(V, t)
( ∞∑
n=−∞an(t)φn(V )
)
dV, t ≥ 1. (3.333)
This reduces to
ak(t) = (ψk(V, t), ρ(V, t)) =
∫ θ
0
ψk(V, t)ρ(V, t)dV (3.334)
since
(ψk(V, t), φj(V, t)) = 0 when k 6= j, and (ψk(V, t), φk(V, t)) = 1. (3.335)
Thus, for each n we have
an(t) = (ψn(n, t), ρ(V, t)) =
∫ θ
0
ψn(V, t)ρ(V, t)dV, t ≥ 1. (3.336)
When t ≥ 1 the function an(t) satisfies the initial value ODE problem
a′n(t) = λNan, t > 1, (3.337)
and
an(1) =
∫ θ
0
ψN (V, 1+)ρ(V, 1−|0, 0)dV. (3.338)
72
Substituting ρ(V, 1−|0, 0) = ρS(V ) = 2θ2 (θ − V ) reduces (3.338) to
an(1) =
∫ θ
0
ψN (V, 1+)2
θ2(θ − V ) dV (3.339)
I. The computation of ν∞(t). The first step in computing ν∞(t) is to generate the gamma values
and the eigenvalues. For this use the matlab programs f1a.m, f2a.m amd mattiafig2eigenvalues.m in
www.math.pitt.edu/∼ troy/lif/stepproblem/
In all three programs the parameters are set to µ = 25, σ = 1 and θ = 1. Remove the per-
cent sign in front of the two save commands in mattiafig2eigenvalues.m and give the names mat-
tiafig2gammavaluesnew.dat and mattiafig2eigenvaluesnew.dat to the two data sets that contain the
gamma values and the eigenvalues. Type mattiafig2eigenvalues at the matlab prompt and the gamma
values and eigenvalues will be computed and saved in the files mattiafig2gammavaluesnew.dat and
mattiafig2eigenvaluesnew.dat
Next, to find ν∞(t) itself use the programs stepproblemdriver.m which is on the website
www.math.pitt.edu/∼ troy/lif/stepproblem/
Type stepproblemdriver at the matlab prompt ν∞(t) will be computed and drawn. The matlab
programs called by stepproblemdriver.m are also on this website.
II. The computation of νN (t). The procedure to obtain νN (t) is similar to that of the last subsection.
First, for each i ∈ [1, N ] solve (3.318)-(3.322) and obtain the corresponding spike train
yi(t) =∑
j
δ(t− tij), (3.340)
where the values tij are the spike times. The deterministic firing rate ν(t) satisfies
ν∞(t) = limN→∞
1
N
N∑
i=1
∑
j
δ(t− tij)
. (3.341)
We let ∆ > 0 be small and approximate ν∞(t) by the population firing rate
νN (t) =1
N
N∑
i=1
∑
j
σij(t, t+ ∆)
∆
, (3.342)
where
σij(t, t+ ∆) =
1 if tij ∈ (t, t+ ∆),
0 otherwise.(3.343)
Matlab program www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemnoisyfiringrate.m
73
This program computes the approximation for νN (t) given by (3.342)-(3.343) (see Figure 5).
The first part of the program creates the spike times. For this define N (the number uncoupled cells
in the population) and ∆ by
N = 10000 and ∆ = .001 (3.344)
Define the the step size for the solution of (3.318)-(3.322) by
dt =∆
100=.001
100= .000001 (3.345)
The number of time steps used to solve (3.318)-(3.322) for one cell (Figure 5, upper left panel) is
K =2
dt= 200000. (3.346)
Next, set V (0) = 0, and initialize a row containing the spike times by the commands
V(0)=0; spiketimes=[]
Use the two for loops described below to solve the SDE=10000 times using Euler’s method. During
each run save the spike times in the row spiketimes.
for i=1:N (i.e. the number of cells is N)
for k=1:K (i.e. the number of time steps per solution is K)
V ((k + 1) ∗ dt) = V (k ∗ dt) + µ ∗ dt+ σ ∗√dt ∗ randn
Check for a spike: if V ((k + 1) ∗ dt) = 1 then V ((k + 1) ∗ dt) = 0 (reset condition) and add the
number t=(k+1)*dt to the row containing spike times. To do this use the command
spiketimes=[spiketimes (k+1)*dt];
end; end;
Next, sort through the row of spiketimes and create the function
n(t, t+ ∆t) = the number of spikes in (t, t+ ∆t), (3.347)
This function is then used to obtain
νN (t) =n(t, t+ ∆t)
N∆t. (3.348)
To obtain νN (t, t+∆) we subdivide [0,2] into 2/∆ subintervals of (0,2]. We define the vector envalues,
which is the approximation to n(t, t+ ∆), by
envalues(kk) = the number of spikes in ((kk − 1)∆, kk∆] , 1 ≤ kk ≤ 2
∆= 2000. (3.349)
It takes 20 minutes to compute envalues. We then construct νN (t) with the command
firingrate = envalues/(numberofcells ∗ ∆),
74
where numberofcells = N = 10000, ∆ = .001 and save it externally with the commands
tt = linspace(0, 1, 1/delta);
aa = [tt firingrate];
save steproblemfiringrate.dat − ascii aa
In the final part of stepproblemnoisyfiringrate.m we draw the firing rate diagrams in Figure 5. To do
this use the load command to get the externally saved data.
75
How to solve the basic feed forward problem.
This follows Doiron et [6]. We consider M populations of N cells which, in layer 1, are statistically
independent and identical. Our goals are to compute (i) the ‘finite size effect’ population firing rates
ν1,N (t), ν2,N (t), ν3,N (t), and (ii) the ‘infinite size limit’ firing rates ν1,∞, ν2,∞, ν3,∞ in layers 1, 2
and 3 in response to noise, and a step input from layer 0 of the form
ν0(t) =
0 if 0 ≤ t < t∗ (msec),
µ if t ≥ t∗.(3.350)
Each cell in layer m (m ≥ 1) receives input from exactly CEx excitatory cells and CIn inhibitory cells
in layer m-1, and also from a bath of noise present in layer m. The transmembrane potential Vi,m(t)
of cell i in layer m satisfies the ODE
dVi,m
dt= −β + Ii,m(t), Vj(0) = 0, (3.351)
where β > 0 is a constant, and Ii,m(t) is the input to cell i in layer m from layer m-1, and also from
the bath of noise in layer m. The positive constant β guarantees that (3.351) has a ‘stable rest state’
in the absence of the input Ii,m(t). To see this, remove Ii,m(t) so that (3.351) reduces to
dVi,m
dt= −β. (3.352)
If vL = vR = 0 and 0 ≤ V (0) ≤ vT then
Vi,m(t) → vL = vR = 0 as t→ tmin for some first tmin ≥ 0. (3.353)
Thus, vL = 0 is a stable rest state for (3.351) when β > 0. If β = 0 and Vi,m(0) > 0, then Vi,m(t) ≡constant, hence (3.353) does not hold and there is no stable rest state. Doiron et al [6] set β = .5
The input Ii,m(t) has the form
Ii,m(t) = JEx
N∑
j=1
KEx,m(i, j)yj(t) + JIn
N∑
j=1
KIn,m(i, j)yj(t) + σζi,m(t), (3.354)
where yj(t) is a spike train generated by cell j in layer m-1, KEx,m and KIn,m are N by N matrices,
σ2 is the variation of the noise in each layer, JEx ≥ 0 is the magnitude of postsynaptic current due to
a single excitatory presynaptic input, and JIn ≤ 0 is the magnitude of postsynaptic current due to a
single inhibitory presynaptic input. We assume that JEx and JIn have the form
JEx =JEx
CExand JIn =
JIn
CIn, (3.355)
where
JEx ∈ 0, 1, 2, 3 and JIn ∈ 0,−1,−2,−3. (3.356)
76
Thus, (3.354) becomes
Ii,m(t) =JEx
CEx
N∑
j=1
KEx,m(i, j)yj(t) +JIn
CIn
N∑
j=1
KIn,m(i, j)yj(t) + σζi,m(t). (3.357)
To understand how CEx and CIn are determined, define
NEx = the number of excitatory cells that can potentially connect to another ‘downstream’ cell,
pEx = the probability that an excitatory cell will connect to a downstream cell,
and
CEx = the number of excitatory cells that can potentially connect to another ‘downstream’ cell times
the probability that an excitatory cell will connect to a downstream cell.
Thus,
CEx = NEx ∗ pEx. (3.358)
Similarly,
NIn = the number of inhibitory cells that can potentially connect to another ‘downstream’ cell,
pIn = the probability that an inhibitory cell will connect to a downstream cell,
and
CEIn = the number of excitatory cells that can potentially connect to another ‘downstream’ cell times
the probability that an excitatory cell will connect to a downstream cell.
Thus,
CIn = NIn ∗ pEx. (3.359)
To determine the values of NEx and NIn we combine two observations. First,
NEx +NIn = N = the total number of cells in the population. (3.360)
Secondly, experimental evidence indicates that
NEx = 4NIn. (3.361)
Thus, if N = 1000, we conclude that
NEx = .8 ∗N = 800 and NIn = .2 ∗N = 200. (3.362)
Experimental evidence suggests that pIn = 4pEx, and that a reasonable value for pEx is pEx = .125 [6].
Therefore, pIn = .5 and we conclude that
CEx = NEx ∗ pEx = 800 ∗ .125 = 100 and CIn = 200 ∗ pIn = 200 ∗ .5 = 100. (3.363)
Next, we need to understand the role of the N by N matrices KEx,m(i, j) and KIn,m(i, j).
77
The matrix KEx,m(i, j) consists of 0’s and 1’s, with exactly CEx randomly distributed 1’s in each row.
The interpretation of 0’s and 1’s in row i of KEx,m(i, j) is that
ProbKEx,m(i, j) = 1 = pEx and Prob KEx,m(i, j) = 0 = 1 − pEx. (3.364)
The definition of the N by N matrix KIn(i, j) is similar and omitted for the sake of brevity.
How to estimate values for JEx and JIn in the LIF model.
Let V If (t) and V Exp(t) denote the theoretical (i.e LIF) transmembrane potential and experimental
transmembrane potential, repectively. The experimental rest state and threshold values are
V ExpR = −65 mv and V Exp
T = −55 mv. (3.365)
The theoretical rest state and threshold values are
V IfR = 0 mv and V If
T = 1 mv. (3.366)
We assume that V If (t) and V Exp(t) are related linearly by an equation of the form
V IfR (t) = aV Exp(t) + b. (3.367)
Substituting (3.365) and (3.366) gives
V IfR (t) =
1
10
(
V Exp(t) + 65)
. (3.368)
Next, let JEx denote the height of the response of a synapse to an excitatory change in voltage, and
let JIn denote the height of the response of a synapse to an inhibitory change in voltage. We believe
the experimental observation that an excitatory kick causes an increase of .05 mv. in transmembrane
potential from −65 mv. to −64.5 mv. Substituting this into (3.368) gives
JEx =1
10(−64.5 + 65) = .05 mv. (3.369)
Similarly, we believe the experimental observation that an inhibitory kick causes a dcrease in trans-
membrane potential from −65 mv. to −66 mv. Substituting this into (3.368) gives
JIn =1
10(−66 + 65) = −.1 mv. (3.370)
Note that |JIn| = 2|JEx|. That is, an inhibitory kick has double the strength of an excitatory kick.
One possible case to study is the ’balanced’ setting, in which it is required that
JExCEx + JInCIn = 0. (3.371)
Substituting (3.369) and (3.369) into (3.369) gives
CEx = 2CIn. (3.372)
78
The ubalanced case requires that
JExCEx + JInCIn 6= 0. (3.373)
Properties of layer 0. In layer 0 let Xi(t) denote the number of spikes generated by cell number i
in time interval (0, t). It is assumed that each Xi(t) is a Poisson random variable with the same firing
rate
ν0(t) =
0 Hz if 0 ≤ t < t∗ (msec),
40 Hz if t ≥ t∗.(3.374)
Thus, each cell generates spike trains which are statisitically identical.
Properties of layer 1, 2 and 3. The transmembrane potential Vi,m(t) of cell i in layer m (m ≥ 1)
satisfies the SDEdVi,m
dt= −β + Ii,m(t), Vj(0) = 0, (3.375)
where the input Ii,m(t) has the form
Ii,m(t) = JEx
N∑
j=1
KEx,m(i, j)yj(t) + JIn
N∑
j=1
KIn,m(i, j)yj(t) + σζi,m(t). (3.376)
We assume that
JEx =JEx
CExand JIn =
JIn
CIn, (3.377)
hence (3.375) becomes
dVi,m
dt= −β +
JEx
CEx
N∑
j=1
KEx,m(i, j)yj(t) +JIn
CIn
N∑
j=1
KIn,m(i, j)yj(t) + σζi,m(t), Vj(0) = 0. (3.378)
The spike trains yj(t) are generated by cell j in layer 0, and are all statically identical. Because of
this, we follow Doiron et al [6] and assume that the cells in layer 1 are statistically independent and
identical. To understand what ‘statistically independent’ means let Vk(t) denote the transmembrane
potential of cell no. k, and let ρm(Vk, t) denote its pdf (probability distribution function). If i 6= j let
ρ(Vi, Vj , t) denote the joint pdf of Vi(t) and Vj(t). Then ρm(Vi, t) is obtained from ρ(Vi, Vj , t) by the
formula
ρm(Vi, t) =
∫ vT
vL
ρ(Vi, Vj , t)dVj . (3.379)
Cells number i and j are statistically independent if
ρ(Vi, Vj , t) = ρm(Vi, t)ρm(Vj , t), i 6= j. (3.380)
The population of cells is statistically independent if (3.380) extends to any finite combination of cells
in the population. The cells are ‘identical’ if they are identically distributed, i.e.
ρm(Vi, t) = ρm(Vj , t), i 6= j. (3.381)
79
Because the cells in layer 1 are statistically independent and identical, each cell in layer 1 is modelled
by diffusion approximation to (3.375), namely
dV
dt= −β +
(
JEx + JIn
)
ν0(t) +
√
(
J2Ex
CEx+J2
In
CEx
)
ν0(t) + σ2ζ(t), V (0) = 0, (3.382)
where
0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.383)
If a ’spike’ occurs if V (t) = vT at some finite t value then the solution is immediately reset to
V = vR = 0. The reset condition is
V (t−) = vT and V (t+) = vR. (3.384)
In our computations we set
vR = 0 and vT = θ = 1. (3.385)
The computations for Figures 8, 9 and 12 We follow Doiron et al and use parameter values
N = 1000, β = −.5, σ = .63, CEx = 100, JEx = 2, CIn = 48, JIn = −.96 (3.386)
For the values given in (3.386) the SDE (3.382) becomes
dV
dt= −.5 + 1.04ν0(t) +
√
.00592ν0(t) + .632ζ(t), V (0) = 0, (3.387)
where
ν0(t) =
0 Hz if 0 ≤ t < 500 (msec),
40 Hz if 500 (msec) ≤ t ≤ 900 (msec).(3.388)
In layer 1 we solve (3.382)-(3.388) 1000 times.
For layers 2 and 3 we substitute the values given in (3.386) into (3.378) and obtain
dVi,m
dt= −.5 + .02
N∑
j=1
KEx,m(i, j)yj(t) − .02
N∑
j=1
KIn,m(i, j)yj(t) + .63ζi,m(t), Vj(0) = 0, (3.389)
We solve (3.389) 1000 times for m=2 (layer 2) and m=3 (layer 3). When N=1000 note that KEx(i, j)
and KIn(i, j) are 1000 by 1000 matrices with 1 million entries. To simplify the computations we take
advantage of their sparse nature by generating smaller N by CEx and N by CIn (i.e. 1000 by 100
and 1000 by 48) matrices: the rows of the bigger matrix consist of 100 different randomly distributed
integers between 1 and 1000, and the rows of the smaller consist consist of 48 different randomly
distributed integers between 1 and 1000, This is done with the program /lif/fforward2/ kmatrix.m
All three figures exhibit oscillations during the ‘transient phase.’ During the transient phase the
functions ν1,N(t), ν2,N (t), ν3,N (t) oscillate with period ≈ 24.33 msec. This result is obtained with
the formula
Period ≈ 1 sec.
(−β + (JEx + JIn)ν0(t)) cycles=
1000 msec.
41.1 cycles= 24.33
msec
cycle. (3.390)
80
700 750 8000
50
100
200
t (msec)
Layer 3ν3,N
layer3noisyfiringrate4a.eps
1000 10750
50
100
t (msec)
νN
Layer 3
700 750 8000
50
100
200
t (msec)
Layer 2ν2,N
layer2noisyfiringrate4a.eps
1000 10750
50
100
t (msec)
νN
Layer 2
0 500 10000
50
100
t (msec)
νN
Layer 1
1000 10750
50
100
t (msec)
νN
Layer 1
Figure 8: Propagation of ringing. Computations for layer 1 were done by solving (3.382)-(3.388) 1000
times. Computations for for layers 2 and 3 were done by solving (3.389) 1000 times for m=1 (layer 1)
and m=2 (layer 2). Error in computaions for layer 1: in [0,500] we used σ = 1.66 instead of the cor-
rect value σ = .63 The computations were done in /lif/feedforweard3/ with layer1noisyfiringrate4.m,
layer2noisyfiringrate2.m, and layer3noisyfiringrate4.m.
Next, we descrivbe how to find ν1,∞(t) from the Fokker-Planck equation
∂
∂tρ(V, t) = (β − µ(t))
∂
∂Vρ(V, t) +
σ2(t)
2
∂2
∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.391)
where
µ(t) = (JEx + JIn)ν0(t), (3.392)
81
700 7500
50
100
t (msec)
Layer 3ν
3,N
700 750 8000
50
100
t (msec)
Layer 2ν2,N
700 750 8000
50
100
t (msec)
Layer 1ν1,N
Figure 9: Propagation of ringing as in Figure 8 but now we use the correct for creating spike-
trains in layer 1. The computations were done in /lif/feedforweard3/ with layer1noisyfiringrate5.m,
layer2noisyfiringrate5.m, and layer3noisyfiringrate5.m.
and
σ2(t) =
√
(
J2Ex
CEx+J2
In
CEx
)
ν0(t) + σ2 (3.393)
82
700 750 8000
50
100
t (msec)
Layer 1 N=1000Cex=100 Cin=48ν
1,N
700 750 8000
50
100
t (msec)
Layer 1 N=10000Cex=100 Cin=48ν
1,N
700 750 8000
50
100
t (msec)
Layer 1 N=10000Cex=1000 Cin=480ν
1,N
Figure 10: Layer 1 ringing. Top row: N=1000 CEx = 100 and CIn = 48. Row 2: N=1000, CEx = 100
and CIn = 48 repeat the experiment 10 times. Row 3: N=10000 and CEx = 1000, CIn = 480 Layer 0
input: ν0(t) = 0 Hz for t ∈ [0, 500), and ν0(t) = 40 Hz for t ∈ [500, 900]. Parameters: σ = .63, JEx =
2, JIn = −.96, CIn = 48. (upper and middle panels) Computations done in /lif/feedforweard3/ with
layer1noisyfiringrate5a.m, layer1noisyfiringrate5b.m, layer1noisyfiringrate5bb.m.
83
ρ(V, t) satisfies the ’absorbing’ boundary condition
ρ(vT , t) = 0 ∀t ≥ 0, (3.394)
and normalizing condition∫ vT
vL
ρ(V, t)dV = 1 ∀t ≥ 0. (3.395)
We let ρ(V, t|0, 0) denote the particular solution of (3.323)-(3.324)-(3.325) such that
ρ(V, 0|0, 0) = δ(V ). (3.396)
As in the last section, because µ(t) = 0 over [0, .5), we could not find ρ(V, t|0, 0). However, there is a
stationary solution, namely
ρS(V ) =2
θ2(θ − V ). (3.397)
The ‘infinite size limit’ firing rate ν∞(t) is defined by
ν∞(t) = −σ2
2
∂
∂Vρ(vT , t|0, 0) ∀t ≥ 0. (3.398)
Since there is only a stationary pdf when 0 ≤ t < t∗ and µ(t) = 0, we conjecture that ρ(V, t|0, 0) =
ρS(V ) when 0 ≤ t < t∗, and that
ν∞(t) = −σ2
2ρ′S(θ) =
σ2
θ2∀t ∈ [0, t∗) (3.399)
Our numerical experiments described below indicate that the prediction given in (3.399) does hold.
To compute ν2,∞(t) we need to solve the Fokker-Planck equation corresponding to the SDE
dV
dt= µ2(t) + σ2(t)ζ(t), V (0) = 0, (3.400)
where
µ2(t) = µ2 + (JExCEx + JInCIn) ν1,∞(t) and σ22,∞(t) = σ2 +
(
J2ExCEx + J2
InCIn
)
ν1,∞(t). (3.401)
In Figure ?? we assume that
µ2 = 0, JEx =JEx
CEx=
1
CExand JIn =
JIn
CIn= 0, (3.402)
and therefore (3.401) reduces to
µ2(t) = ν1,∞(t) and σ22,∞(t) = σ2 +
ν1,∞(t)
CEx= σ2 +
ν1,∞(t)
100(3.403)
Substituting (3.403) into (3.400) gives the SDE
dV
dt= ν1,∞(t) +
(
σ2 +ν1,∞(t)
100
)
ζ(t), V (0) = 0. (3.404)
84
Feedforward propagation of finite width pulses.
In the unbalanced setting we assume that
JEx =JEx
CExand JIn =
JIn
CIn. (3.405)
The parameters for our balanced setting experiments are
dt = .00001, β = −.5, σ = .63, JEx = 2, CEx = 100, JIn = −.96, CIn = 48. (3.406)
The firing rate in Figures 11 and 12 is
ν0(t) =
0 Hz if 0 ≤ t < 750 (msec),
41.1 Hz if 750 (msec) ≤ t < 800 (msec),
0 Hz if 800 (msec) ≤ t < 900 (msec).
(3.407)
700 750 8000
50
100
t (msec)
Layer 1 N=1000Cex=100 Cin=48ν
1,N
700 750 8000
50
100
t (msec)
Layer 1 N=10000Cex=100 Cin=48ν
1,N
700 750 8000
50
100
t (msec)
Layer 1 N=10000Cex=1000 Cin=480ν
1,N
Figure 11: Upper right: repeat the experiment in upper left 10 times. Lower: increase N to N=10000
and preserve CEx
N = .1 and CIn
N = .048. /lif/feedforward4/layer1noisyfiringratetwobump4a.m,
layer1noisyfiringratetwobump4aa.m and layer1noisyfiringratetwobumpaaa.m
In the balanced setting We assume that
JEx =JEx√CEx
and, JIn =JIn√CIn
and JEx
√
CEx = JIn
√
CIn (3.408)
Question. In the figures do we see memory in the background noise after the input?
85
700 750 8000
50
100
200
t (msec)
Layer 3ν3,N
700 750 8000
50
100
200
t (msec)
Layer 2ν2,N
700 750 8000
50
100
200
t (msec)
Layer 1ν1,N
Figure 12: Unbalanced setting propagation of one-bump ring. Layer 0 input: ν0(t) = 0 Hz for t ∈[0, 750), ν0(t) = 41.1 Hz for t ∈ [750, 775], ν0(t) = 0 Hz for t ∈ [775, 900].Parameters: dt = .00001, β =
−.5, σ = .63, JEx = 2, JIn = −.96, CIn = 48, CEx = 100. The ‘background’ noise does not propa-
gate to layers 2 and 3. Computations in /lif/feedforweard4/ with layer1noisyfiringrateonebump4a.m,
layer2noisyfiringrateonebumpb.m, and layer3noisyfiringrateonebump4a.m.
86
525
50
100
t (msec)
Layer 2
95 spikes
300 525 8000
50
100
t (msec)
Layer 1
Cex=Cin=50
Jex=−Jin=.35
N=1000
914 spikes
β=.5, σ=.5
ν1,N
Figure 13: Propagation failure in the balanced setting . Parameters: JEx = −JIn = −.35, CEx =
CIn = 50 dt = .00001, β = .5, σ = .63, Dashed curve is the layer 0 input: ν0(t) = 0 Hz for
t ∈ [0, 500), ν0(t) = 80 Hz for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900] Computations done in
/lif/feedforweard4/ with layer1fff.m and layer2fff.m
87
300 500 6000
15
50
100
t (msec)
Layer 3
886 spikesν
3,N
300 500 6000
10
20
Blowupν3,N
300 500 6000
50
100
t (msec)
Layer 2
1122 spikesν
2,N
300 500 6000
10
20
Blowupν2,N
300 500 6000
15
50
100
t (msec)
Layer 1
Cex=Cin=50
Jex=−Jin=.7
N=1000
1418 spikes
β=.5, σ=.63
ν1,N
300 500 6000
10
20
Blowupν
1,N
Figure 14: Balanced setting. Propagation failure. Parameters: N = 1000, β = .5, σ = .63, CEx =
CIn = 50, JEx = −JIn = .7 Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz
for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. Computations done in /lif/feedforweard4/
with layer1mmm1.m, lyer1mmm1blowup.m, layer2mmm1, layer2mmm1blowup.m, layer3mmm1.m,
layer3mmmblowup.m
88
300 525 800
50
100
t (msec)
Layer 2
739 spikes
300 525 8000
50
100
t (msec)
Layer 1
Cex=CIn
=50
JEx
=−JIn
=1
N=1000
453 spikes
ν1,N
Figure 15: Propagation success in the balanced case. Dashed curve is the layer 0 input: ν0(t) = 0 Hz
for t ∈ [0, 500), ν0(t) = 4 Hz for t ∈ [500, 550] and ν0(t) = 0 Hz for t ∈ [550, 900]. Parameters: dt =
.00001, β = .5, σ = .63, JEx = 1, JIn = −1. Computations done in /lif/feedforward4/layer1ccc.m,
layer2ccc.m, layer2ddd.m
89
300 525 8000
50
100
t (msec)
Layer 1
Cex=CIn
=50
JEx
=−JIn
=1
N=1000
453 spikes
ν1,N
300 500 8000
50
100
t (msec)
Layer 1
N=20000
3800 spikes
ν1,N
300 500 8000.5
4
8
Blowup
N=20000
ν1,N
Figure 16: Balanced setting. Parameters: dt = .00001, β = .5, σ = .63, CEx = CIn = 50 and
JEx = 1, JIn = −1. Dashed curve is the layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 4 Hz for
t ∈ [500, 550] and ν0(t) = 0 Hz for t ∈ [550, 900] Middle row: repeat the experiment in top row 20
times. Bottom row: Blowup of figure in middle row. Computations done in /lif/feedforweard4/ with
layer1ccc.m and layer1ccca.m
90
500 600
50
100
t (msec)
Layer 3
6410 spikesν
3,N
500 6000
5
10
Blowupν3,N
500 600
50
100
t (msec)
Layer 2
4357 spikesν
2,N
500 6000
5
10
Blowupν2,N
300 500 6000
15
50
100
t (msec)
Layer 1
Cex=Cin=50
Jex=−Jin=1
N=1000
2922 spikes
β=.5, σ=.63
ν1,N
500 6000
5
10
Blowupν1,N
Figure 17: Balanced setting. Progation success. Parameters: N = 1000, β = .5, σ = .63, CEx =
CIn = 50, JEx = −JIn = 1. Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz
for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. The layer 0 firing rate successefully propagates
to higher layers. Computations done in /lif/feedforweard4/ with matlab programs layer1mmm.m,
lyer1mmmblowup.m, layer2mmm.m, layer2mmmblowup.m, layer3mmm.m, layer3mmmblowup.m
91
5000 100000
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50
Jex=−Jin=1
4229 spikes
β=.5, σ=.63
ν1,N
500 6000
2
5
t (msec)
Blowup 1ν1,N
2000 2400 28000
2
5
t (msec)
Blowup 2ν1,N
9200 9600 100000
2
5
t (msec)
Blowup 3ν1,N
Figure 18: Balanced setting layer 1 computations to illustrate what happens to the post-stimulus
noise shown in third row of Figure 17. Layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz
for t ∈ [500, 600], ν0(t) = 0 Hz for t ∈ [600, 10000]. Parameters CEx = CIn = 50, JEx = −JIn = 1
and β = .1, σ = .1 Upper left: layer 1 firing rate curve. Upper right: blowup of the upper left panel
indicates that the post-stimulus noise in the firing rate in the time interval [600,800] has increased
when compared with the pre-stimulus noise in [0,500]. Lower panels: blowups over [2000,2800] and
[9200,10000] seems to indicate that the evolves into the stationary state. Comparison with the next
figure gives further credence to this conjecture. Computations done in /lif/feedforweard4/ with matlab
prograam layer1mmm5.m,
In Figure 18 we investigate what happens to the post-stimulus noise that is observed in the third row
of Figure 17. For this we solve the SDE on the extended time interval [0,10000]. It seems that the
post-stimulus immediastely enteres the stationary state.
92
5000 100000
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50
Jex=−Jin=1
4229 spikes
β=.5, σ=.63
ν1,N
500 6000
2
5
t (msec)
Blowup 1ν1,N
2000 2400 28000
2
5
t (msec)
Blowup 2ν1,N
9200 9600 100000
2
5
t (msec)
Blowup 3ν1,N
Figure 19: Balanced setting layer 1 computations to extend those in Figure 18 and illustrate what
happens to the noise when no stimulus is present. Layer 0 input is set equal to zero: ν0(t) = 0 Hz ∀t ≥0. ν0(t) = 15 Hz Parameters CEx = CIn = 50, JEx = −JIn = 1 and β = .1, σ = .1 Upper left: layer 1
firing rate curve. Upper right: blowup of the upper left panel shows that the noise in the firing rate
increases. Lower panels: blowups over [2000,2800] and [9200,10000] indicates that the noise evolves
into the stationary state. The firing rate over [9200,10000] apopears to be the same as in Figure 18.
Computations done in /lif/feedforweard4/ with matlab prograam layer1mmm5a.m,
In Figure 19 we remove the stimulus and de,onstrate how the noise evolves into the stationary state.
93
300 500 600
50
100
t (msec)
Layer 3
552 spikes
ν1,N
300 500 6000
5
10
Blowupν1,N
500 600
50
100
t (msec)
Layer 2
352 spikes
ν1,N
500 6000
5
10
Blowupν1,N
300 500 6000
50
100
t (msec)
Layer 1
Cex=Cin=50
Jex=−Jin=1
N=1000
221 spikes
β=.5, σ=.63
ν1,N
500 6000
5
10
Blowupν1,N
Figure 20: Balanced setting. Sucess of propagation. Parameters: N = 1000, β = .5, σ = .63,
CEx = CIn = 50, JEx = −JIn = .35 Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500),
ν0(t) = 1 Hz for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. The small layer 0 firing rate successe-
fully propagates to higher layers. Computations done in /lif/feedforweard4/ with matlab programs
layer1ppp.m, layerpppblowup.m, layer2ppp.m, layer2pppblowup.m, layer3ppp.m, layer3pppblowup.m
94
200 300 550 6500
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50 Jex=−Jin=1
β=.5, σ=.63
ν1,N
/lif/feedforward4/layer1mmm3.m layer1mmm31.eps
200 300 550 6500
15
50
100
t (msec)
Layer 1 N=125000
Cex=Cin=50 Jex=−Jin=1
β=.5, σ=.63
spikes: .26 2779 64.2 2852 66.2spikes: .26 2779 64.2 2852 66.2
Diff=73 Rel. Change=2.7%
ν1,N
/lif/feedforward4/layermmm3.m layer1mmm331.eps
200 300 550 6500
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50 Jex=−Jin=1
β=.1, σ=.1
ν1,N
/lif/feedforward4/layermmm4.m layer1mmm314.eps
200 300 550 6500
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50 Jex=−Jin=1
spikes: 0 2672 3 2832 4
Diff=160 Rel. Change=6%
β=.1, σ=.1
ν1,N
Figure 21: Balanced setting. Layer 1 computations. Dashed curve is layer 0 input: ν0(t) = 0 Hz
for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ [300, 550], ν0(t) = 15 Hz for
t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ [650, 900]. Parameters: CEx = CIn = 50, JEx = −JIn = 1.
Upper row: β = .5, σ = .63 gives an increase of 74 more spikes (2.76 percent) in [550,650] than in
[200,300]. Lower row: β = .1, σ = .1 gives an increase of 153 more spikes (5.6 percent) Computations
done in /lif/feedforweard4/ with matlab prograams layer1mmm3.m, layer1mmm4.m
Basic assumptions on a the pool of cells.
We consider a collection, of size N, of neuronal cells which are stastically identical and sparsely
connectd. For fixed i ∈ [1,N] define the counting process
Xi(t) = the number of spikes emitted by cell i in (0, t). (3.409)
We assume that the cells are statistically identical (Figure 24) and sparsely connected (Figure 25). To
say that they are statistically identical means that the pdf (probability distribution function) of Xi(t)
95
0 200 300 550 650
50
100
t (msec)
Layer 2 N=1000
spikes: 0 3427 18 3787 24
Diff=360 Rel. Change=10.5%
ν2,N
200 300 550 650 9000
1
2
3
4
t (msec)
Layer 2
Blowup
ν2,N
200 300 550 6500
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50 Jex=−Jin=1
spikes: 0 2672 3 2832 4
Diff=160 Rel. Change=6%
β=.1, σ=.1
ν1,N
0 200 300 550 6500
0.5
1
V
t (msec)
Layer 2
Figure 22: Balanced setting. Layer 1 and layer 2 computations. Dashed curve is layer 0 input:
ν0(t) = 0 Hz for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ [300, 550],
ν0(t) = 15 Hz for t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ [650, 900]. Parameters: CEx = CIn = 50,
JEx = −JIn = 1 and β = .1, σ = .1 Lower left: layer 1 firing rate shows that there is an increase
of 160 more spikes (6 percent) in [550,650] than in [200,300]. Upper left: layer 2 shows an increase
of 360 more spikes (10.5 percent) in [550,650] than in [200,300]. Upper right: blowup of layer 2 to
see the noise between stimulus regions. Lower right: typical layer 2 solution. Computations done in
/lif/feedforweard4/ with matlab prograams layer1mmm4.m, layer2mmm4.m
is the same for 1 ≤ i ≤ N. To say that the cells are sparsely connected means that the probability of
two cells receiving input from the same third cell is small. To develop a mathematical formualtion of
the concept of ‘sparse connectivity’ we define
Probi→ k = The probability that cell i is connected to cell k (3.410)
96
250 600 9500
15
50
100
t (msec)
Layer 1 N=1000
Cex=Cin=50 Jex=−Jin=1
β=.1, σ=.1
ν1,N
250 600 9500
15
50
100
t (msec)
Layer 1 N=25000
Cex=Cin=50 Jex=−Jin=1
spikes: 0 2717 1.6 2872 1.9 2867 1
Diff1=155 Rel. Change=5.7%
β=.1, σ=.1
ν1,N
Figure 23: Balnced setting 3 stimulus computations. Dashed curve is the layer 0 input: ν0(t) = 0 Hz
for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ (300, 550), ν0(t) = 15 Hz for
t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ (650, 900] Parameters: dt = .00001, β = .1, σ = .1, JEx = 1,
JIn = −1. Lower panel repeats the N=1000 experiment 25 times and shows an average increase of
152 spikes from 1st to 2nd stimulus, no increase from 2nd to 3rd stimulus. Computations done in
/lif/feedforward4/layer1mmm6.m
97
ν(t)
Figure 24: Schematic of a population of cells that are statistically identical. This diagram was provided
by Brent Doiron.
and
Probi→ k, j → k = The probability that cells i and j are both connected to cell k. (3.411)
If the cells are independent then
Probi→ k, j → k = Probi→ kProbj → k (3.412)
Set
Probi→ k = p ∈ (0, 1). (3.413)
The cells sparsely connected if they are independent and
Probi→ k, j → k = p2 << 1. (3.414)
Figure 25 illustrates the results of Oswald et al [24] who give experimental evidence for the sparse
connectivity of a pool of pyramidal cells. The upper panel graphs p, the probability that two cells are
connected, vs. the distance between cells. Note that 0 < p < .2 so that p2 < .04 is small, as required
for sparse connectivity.
98
A
B
0.3
0.2
0.1
0.0 Prob
abilit
y of
Con
nect
ion
140 120 100 80 60 40 20
Distance between Somas (μm)
x y
-150 -100 -50 0 50 100 150
Distance X (μm)
-100
-50
50
100
Dis
tanc
e Y
(μm
)
0
-150
150
Tested Connected Presynaptic Cell
L1
L2/3
Figure 25: Upper: probability p that two cell pyramidal cells are connected vs. distance (µm). Note
that 0 < p < .2 so that p2 < .04 Lower panel: This diagram was provided by Brent Doiron.
Assumptions on the current entering a cell.
I. We consider a population of N neuronal cells which are stastically identical and indpependent. For
each i ∈ [1,N] and t > 0 let
Xi(t) = the number of spikes emitted by cell i in [0, t). (3.415)
We assume that the Xi(t) satisfy Xi(0) = 0, and are independent Poisson processes with parameter
ν∞. The firing rate dXi
dt of cell i is the spike train yi(t) defined by
yi(t) =dXi
dt=∑
j
δ(t− tij). (3.416)
From (3.416) it follows that
E (yi(t)) = ν∞ ∀i ∈ [1, N ]. (3.417)
Since the Xi(t) are independent then yi(t) and yk(t′) are independent when i 6= k, hence
E (yi(t)yk(t′)) = E (yi(t))E (yk(t′)) = ν2∞ ∀i 6= k ∈ [1, N ], and t, t′ ≥ 0. (3.418)
99
Because each Xi(t) is Poisson process, we conclude that
E (yi(t)yi(t′)) = ν∞δ(t− t′) + ν2
∞ ∀i ∈ [1, N ], and t ≥ t′ ≥ 0. (3.419)
II. The current Ij(t) which enters cell j is assumed to have the form
Ij(t) = µ+ JN∑
i=1
yi(t) + σζj(t), (3.420)
where µ ∈ R and σ > 0 are constants, and ζj(t) is white noise. The cells are excitatory if J > 0, and
inhibitory of J < 0.
III. Goal. Our goal is to find constants µ ∈ R and σ > 0 such that Ij(t) ≈ Ij(t) where
Ij(t) = µ+ σζj(t), (3.421)
and ζj(t) is white noise. In particular, we show below that
µ = µ+ JNν∞ and σ2 = σ2 + J2Nν∞, (3.422)
and therefore
Ij(t) = µ+ JNν∞ +√
σ2 + J2Nν∞ζj(t). (3.423)
Remark. It is interesting to observe that Var(Ij)(t) = Var(Ij)(t) = ∞ for every value of N.
IV. To derive the values of µ and σ2 given in (3.422) we require that two conditions hold, namely
E (Ij(t)) = E(Ij(t)) and E (Ij(t)Ij(t′)) = E(Ij(t)Ij(t
′)). (3.424)
The first step is to conclude from (3.421) that
E(
Ij(t))
= µ, (3.425)
and
E(
Ij(t)Ij(t′))
= µ2 + σ2δ(t− t′). (3.426)
Next, repeat these calculations for the form of Ij(t) assumed in (3.420). First, we get
E (Ij(t)) = µ+ J
N∑
i=1
E (yi(t)) = µ+ JNν∞(t). (3.427)
Equating the right sides of (3.425) and (3.427) gives
µ = µ+ JNν∞. (3.428)
Substituting (3.428) into (3.426) gives
E(
Ij(t)Ij(t′))
= µ2 + 2µJNν∞ + J2N2ν2∞ + σ2δ(t− t′). (3.429)
100
Next, observe that
E (Ij(t)Ij(t′)) = E
([
µ+ J
N∑
i=1
yi(t) + σζj(t)
][
µ+ J
N∑
k=1
yk(t′) + σζj(t′)
])
. (3.430)
We need to evaluate the right side of (3.430). First, note that
E(
µ2)
= µ2, E(
µσζj(t))
= E(
µσζj(t′))
= 0, (3.431)
and
E
(
µJN∑
i=1
yi(t)
)
= E
(
µJN∑
i=1
yi(t′)
)
= µJNν∞. (3.432)
Since the spike trains are independent of the white noise terms, it follows that
E(
µJyiζj(t))
= E (µJyi)E(
ζj(t))
= 0 and E(
µJζj(t′))
= E (µJyi)E(
ζj(t′))
= 0. (3.433)
Because ζj(t) is white noise we conclude that
E(
σ2ζj(t)ζj(t′))
= σ2δ(t− t′). (3.434)
Combining (3.430) with (3.431), (3.432), (3.433) and (3.434) gives
E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + σ2δ(t− t′) + J2
N∑
i=1
N∑
k=1
E(yi(t)yk(t′)). (3.435)
Next, writeN∑
i=1
N∑
k=1
E(yi(t)yk(t′)) =
N∑
i=1
E(yi(t)yi(t′)) +
∑
i6=k
E(yi(t)yk(t′)). (3.436)
It follows from (3.419) that
N∑
i=1
E(yi(t)yi(t′)) = Nν∞δ(t− t′) +Nν2
∞. (3.437)
From (3.418) we conclude that
∑
i6=k
E(yi(t)yk(t′)) = N (N − 1) ν2∞. (3.438)
Combining (3.437) and (3.438) gives
N∑
i=1
N∑
k=1
E(yi(t)yk(t′)) = Nν∞δ(t− t′) +N2ν2∞. (3.439)
Substituting (3.439) into (3.435), we obtain
E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + σ2δ(t− t′) + J2
(
Nν∞δ(t− t′) +N2ν2∞)
. (3.440)
101
This reduces to
E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + ν2
∞J2N2 + σ2δ(t− t′) + J2Nν∞δ(t− t′) (3.441)
Finally, we equate the right sides of (3.429) and (3.441), integrate from t− ǫ to t+ ǫ, and get
σ2 = σ2 + J2Nν∞. (3.442)
This completes the proof of (3.422).
102
The relationship between the population firing rate and the determinitic firing rate.
Our first goal in this section is to show that
νN (t) ≈ ν(t) +
√
ν(t)
Nζ(t) when N >> 1. (3.443)
where νN (t) denotes the population firing rate, ν(t) = ν∞(t) is the deterministic firing rate and ζ(t)
is white noise.
Our second goal is to prove that
Prob 1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t when ∆t > 0 is small. (3.444)
I. To prove (3.443) we use the Central Limit Theorem. Let Y1, Y2, .., YN be identically distributed
random variables ( i.e. they all have the same probability distribution function), with
E(Yi) = µ and V ar(Yi) = σ2, 1 ≤ i ≤ N. (3.445)
The Central Limit Theorem states that∑N
i=1 Yi −Nµ
σ√N
≈ N(0, 1) when N >> 1. (3.446)
Next, recall that for fixed i ∈ [1,N],
Xi(t) = the number of spikes emitted by cell i in (0, t). (3.447)
Thus,
∆Xi(t) = Xi(t+ ∆t) −Xi(t) = the number of spikes emitted by cell i in (t, t+ ∆t). (3.448)
For fixed t and small ∆t we assume that ∆Xi(t) is a Poisson random variable parameter ν(t). Then
E (∆Xi(t)) = V ar (∆Xi(t)) = ν(t)∆t, 1 ≤ i ≤ N. (3.449)
To use the Central Limit Theorem we set Yi = ∆Xi(t), µ = ν(t)∆t and σ =√
ν(t)∆t in (3.446) and
conclude that∑N
i (∆Xi) −Nν(t)∆t√
ν(t)∆t√N
≈ N(0, 1) when N >> 1. (3.450)
It follows from (3.450) that
N∑
i
(∆Xi) −Nν(t)∆t ≈√
ν(t)∆t√NN(0, 1) when N >> 1. (3.451)
Dividing both sides of (3.452) by N∆t gives
1
N
N∑
i
(
∆Xi
∆t
)
≈ ν(t) +
√
ν(t)
N∆tN(0, 1) when N >> 1. (3.452)
103
The population firing rate νN (t) satisfies
νN (t) =1
N
N∑
i=1
dXi
dt≈ 1
N
N∑
i
(
∆Xi
∆t
)
. (3.453)
Substituting (3.453) into (3.452), we obtain
νN (t) ≈ ν(t) +
√
ν(t)
N∆tN(0, 1) when N >> 1. (3.454)
To link this with white noise ζ(t) we make use of the property
ζ(t) ≈ ∆W
∆t=N(0, 1)√
∆t(3.455)
Combining( 3.454) and (3.455) gives
νN (t) ≈ ν(t) +
√
ν(t)
Nζ(t) when N >> 1. (3.456)
II. Our second goal is to show that
Prob1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t. (3.457)
First, observe that
Prob1 spike occurs in(t, t+ ∆t) = Prob Xi(t+ ∆t) −Xi(t) = 1 . (3.458)
From the assumption that ∆Xi(t) is a Poisson random variable with parameter ν(t) it follows that
Prob 1 spike occurs in(t, t+ ∆t) = Prob∆Xi(t) = 1 = e−ν(t)∆tν(t)∆t. (3.459)
Since e−ν(t)∆tν(t)∆t ≈ ν(t)∆t when ∆t > 0 is small, we conclude that
Prob1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t. (3.460)
104
How to solve the SDE problem dVdt = f(V ) + Jδ(t− T ) + σζ(t), V (0) = V0 ?
Goal: develop an Euler numerical scheme to solve
dV
dt= f(V ) + Jδ(t− T ) + σζ(t), V (0) = V0, (3.461)
where f(V ) is a function of V, J is a real parameter which can be positive, negative or zero, σ ≥ 0 is
the standard deviation of the white noise term, and V0 is the initial value of the solution.
The numerical Euler scheme to solve (3.461) is
V (tk+1) = V (tk) + f(V (tk))dt + Jδ(tk − T )dt+ σrandn√dt, V (t1) = V0. (3.462)
To implement (3.462) the first step is to define the total integration time and step size, e.g.
totaltime = 10 and dt = .001 (3.463)
Next, define K, the number of integration steps, by
K = totaltime/dt = 10/.001 = 10000 (3.464)
Define the integation times by
tk+1 = tk + dt, k = 1, 2, ...,K − 1 (3.465)
In the theoretical solution we have
∫ tk+1
tk
δ(t− T )dt =
1 if T ∈ [tk, tk+1),
0 if T /∈ [tk, tk+1).(3.466)
In order that the numerical solution be consistent with this property we set
δ(tk − T ) =
1dt if T ∈ [tk, tk+1),
0 if T /∈ [tk, tk+1).(3.467)
From this we obtain the approximation
∫ tk+1
tk
δ(t− T )dt ≈ δ(tk − T )dt =
(
1dt
)
dt = 1 if T ∈ [tk, tk+1),
0 if T /∈ [tk, tk+1).(3.468)
Combining (3.462) with (3.468) gives V (t1) = V0 and
V (tk+1) =
V (tk) + f(V (tk))dt+ J + σrandn√dt, if T ∈ [tk, tk+1),
V (tk) + f(V (tk))dt+ σrandn√dt if T /∈ [tk, tk+1).
(3.469)
Remark. It follows from (3.469) that the term δ(t− T ) in the model provides a ‘kick’ of magnitude
J (positive, negative or zero) to the solution at t = T, i.e when T ∈ [tk, tk+1].
105
How to solve the SDE problem dVdt = f(V ) + J
∑
j δ(t− Tj) + σζ(t), V (0) = V0 ?
Goal: develop an Euler numerical scheme to solve
dV
dt= f(V ) + J
∑
j
δ(t− Tj) + σζ(t), V (0) = V0, (3.470)
where f(V ) is a function of V, J is a real parameter, σ ≥ 0 is the standard deviation of the white
noise term, and V0 is the initial value of the solution. The term
∑
j
δ(t− Tj) (3.471)
is a spike train. In the simplest setting it is assumed that the spike train is externally generated, with
Tj > 0 ∀j ≥ 1. To accurately describe the numerical solution we define the set of spike times
S = T1, T2, ... (3.472)
The numerical Euler scheme to solve (3.461) is
V (tk+1) = V (tk) + f(V (tk))dt+ J
∑
j
δ(tk − Tj)
dt+ σrandn√dt, V (t1) = V0. (3.473)
To implement (3.473) the first step is to define the total integration time, the step size dt, and K, the
total number of time steps. For example,
totaltime = 10, dt = .001, K = totaltime/dt = 10000 (3.474)
Define the integation times by
tk+1 = tk + dt, k = 1, 2, ...,K − 1 (3.475)
In the theoretical solution we have
∫ tk+1
tk
δ(t− Tj)dt =
1 if Tj ∈ [tk, tk+1),
0 if Tj /∈ [tk, tk+1).(3.476)
In order that the numerical solution (3.473) be consistent with this property we set
δ(Tk − T ) =
1dt if Tj ∈ [tk, tk+1),
0 if Tj /∈ [tk, tk+1).(3.477)
From this we obtain the approximation
∫ tk+1
tk
δ(t− Tj)dt ≈ δ(tk − Tj)dt =
(
1dt
)
dt = 1 if Tj ∈ [tk, tk+1),
0 if Tj /∈ [tk, tk+1).(3.478)
106
Observe that over a given subinterval [tk, tk+1) it is possible that several spikes lie in [tk, tk+1), and
that each such spike will provide an extra ‘kick’ to the solution. Thus, we define
γk = the number of spikes in S which lie in [tk, tk+1), k ≥ 1. (3.479)
Note that either γk = 0 (i.e. no spikes lie in [tk, tk+1)), or else γk > 0 (i.e. γk > 0 spikes lie in
[tk, tk+1)). To account for these possibilities we set
∑
j
δ(tk − Tj)
dt = γk, k ≥ 1. (3.480)
Combining (3.473) with (3.480) gives V (t1) = V0 and
V (tk+1) =
V (tk) + f(V (tk))dt + Jγk + σrandn√dt, if γk > 0,
V (tk) + f(V (tk))dt + σrandn√dt if γk = 0.
(3.481)
Remarks.
(i) It follows from (3.481) that the term J∑
j δ(t−Tj) in the model provides a ‘kick’ of magnitude Jγk
to the solution. Note that the kick can be either positive (J > 0), negative (J < 0), or zero (J = 0).
(ii) At the k-th step it is necessary to do a search of the set S of spike times to precisely determine
γk, the number of spikes that lie in [tk, tk+1).
107
The feed forward model of Doiren et al [6]
In layer I it is assumed that there N cells, they re statistically identical and uncorellated (i.e. indep-
nedent). The tansmembrane potential of each cell is modelled by the same linear integrate and fire
SDEdV
dt= µ(t) + σ(t)
dζ
dt, V (0) = 0, (3.482)
0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞), (3.483)
with reset condition
V (t−) = vT = 0 and V (t+) = vR = 1. (3.484)
The system receives a step input after a time t∗ when the firing rate has become stationary. Each cell
generates a spike train
yi(t) =∑
j
δ(t− tij). (3.485)
They compute the population firing rate νN (t), and also the deterministic firing rate ν∞.
In layer II it is assumed that the cells are uncorellated (i.e. independent). They are assumed to receive
input from the cells in layer I, and also from noise in their surroundings. The tansmembrane potential
of cell j in layer II is modelled by
dVj
dt= µ+ J
N∑
i=1
Kijyi(t) + σdζ
dt, V (0) = 0, (3.486)
where yi(t) denotes the spike train generated by cell i in layer I, J > 0 if the cells in layer I are
excitatory, and J < 0 if the cells in layer I are inhibitory. A typical values of J is
J =J
Nwhere J = 1, 2 or 3. (3.487)
The connectivity matrix is assumed to be sparse and is defined by
Kij =
1 if cell i is connected to cell j,
0 if cell i is not connected to cell j.(3.488)
Anne Marie’s data [24] indicates that
Prob(Kij = 1) = .15,
Prob(Kij = 0) = .85(3.489)
Therefore .15 of the entries of Kij are ones and .85 of the entries of Kij are zeros. Define
p = Prob (cell i is connected to cell j) , (3.490)
108
and
Kj =N∑
i=1
Kij . (3.491)
Then
E(Kj) = Np = the expected number of cells connected to cell j (3.492)
Define
C = Np = the expected number of cells connected to cell j (3.493)
For example, p = .15 according to Anne Marie’s data. This, if a N=5000 then
C = .15 ∗ 5000 = 750 = the expected number of cells connected to cell j (3.494)
Note that Kij is a 5000 by 5000 sparse matrix a 750 by 750 submatrix of ones, and the rest of the
entries are zeros.
109
How to compute the spectrum in Figure 3 of Mattia et al.
We solve the linear integrate and fire SDE
dV
dt= µ(t) + σ(t)
dζ
dt, V (0) = 0, (3.495)
0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞), (3.496)
with reset condition
V (t−) = vT = 0 and V (t+) = vR = 1. (3.497)
The parameters are
µ = 20, σ = θ = 1. (3.498)
To compute the spectrum we consider N=1000 cells, and for each of these cells solve (3.496)-(3.496)-
(3.497) over 0 ≤ t ≤ 60 seconds. Since this is the drift dominant setting we should get about 20
spikes per second times 20 seconds, which gives approximately 400 spikes per cell. Because we have
1000 cells we should end up with about 400,000spikes. During each run only keep the spikes after a
transient period of about 500 msec during which ρ(V, t|0, 0) settles into the stationary state ρS(V ),
and the the firing rate function ν(t) ≈ νS ≈ 20.
First example. Figure 26 demonstrates the ringing phenomenon when we do this numerical experiment
over a 1 second time interval, for two values of σ, namely σ = 1( left panel) and σ = .5 (right panel).
Here we keep all spikes bcause we want to see the transient period behavior.
0 100 200 4000
30
60
t (msec)
νN µ=20, θ
N=1000 cells
=1, σ=1
0 100 200 4000
30
60
t (msec)
νN
N=1000 cells
µ=20, θ=1, σ=.5
Figure 26: Left: ringing (i.e. oscillations) in the population firing rate function for parameter
values µ = 20, σ = 1, θ = 1 and N = 1000 cells. Right: the ringing phenomenon becomes
more pronounced when σ is lowered to σ = .5 The matlab code to obtain these diagrams is at
www.math.pitt.edu/∼troy/lif/mattiafigure3/mattiafigure3ring.m.m
110
4 The Wiener Process W (t).
(a.) The conditional, single and joint probability distributions of W (t).
The random process W (t), t ≥ 0 is a Wiener process (Schaum’s, p.172) if
(1.) W (0) = 0, E(W (t)) = 0 ∀t ≥ 0, and W(t) has stationary independent increments.
(2.) When t > s the increment W (t) −W (s) is normally distributed.
Implications. The assumption that W (t) has stationary, independent increments means that if t > s
then W (t)−W (s) has the same cdf and pdf as W (t+h)−W (s+h) ∀h (see [26], pp. 163-164). Setting
h = −s, we conclude, since W (0) = 0, that W (t) −W (s) and W (t− s) have the same cdf, i.e.
P (W (t) −W (s) ≤ w) = P (W (t− s) ≤ w) =
∫ w
−∞
1√
2π(t− s)e−
η2
2(t−s)dη, (4.1)
and a differentiaiton of both sides of (4.1) with respect to w gives the pdf
ρ(w, t− s) =1
√
2π(t− s)e−
w2
2(t−s) . (4.2)
Our goal here is to show how to derive the distribution (4.2) as well as the joint pdf ρ(w, t;w′, t′).
Step I. Assume that ρ(w, 0) = δ(w), and that the conditional probability density function is
ρ(w, t|w′, s) =1
√
2π(t− s)e−
(w−w′)22(t−s) , t > s. (4.3)
Step II. To derive the single time distribution ρ(w, t) we follow the fundamental principle of beginning
with an integral equation representation for ρ(w, t) :
ρ(w, t) =
∫ ∞
−∞ρ(w, t;w′, 0)dw′. (4.4)
Next, substitute ρ(w, t;w′, 0) = ρ(w, t|w′, 0)ρ(w′, 0) into the right side of (4.4) and get
ρ(w, t) =
∫ ∞
−∞ρ(w, t|w′, 0)ρ(w′, 0)dw′ =
∫ ∞
−∞ρ(w, t|w′, 0)δ(w′)dw′. (4.5)
From (4.3), the definition of the delta function and (4.5) we obtain
ρ(w, t) =
∫ ∞
−∞
1√2πt
e−(w−w′)2
2t δ(w′)dw′ =1√2πt
e−w2
2t . (4.6)
Replace t with t − s in (4.6) to get (4.2). Finally, we obtain the joint probability distribution
ρ(w, t;w′, t′) by setting
ρ(w, t;w′, t′) = ρ(w, t|w′, t′)ρ(w′, t′) =1
√
2π(t− t′)e− (w−w′)2
2(t−t′)1√2πt′
e−w′22t′ (4.7)
Note: W (t) is not stationary since ρ(w, t, w′, s) 6= ρ(w, t− s;w′, 0).
For the N-variable case, suppose that t1 < t2.. < tN . Then the n-variable joint distribution is
ρ(wN , tN , wN−1, tN−1, .., w1, t1) =
(
2∏
i=1
1√
2π(ti − ti−1)e− (wi−wi−1)2
2(ti−ti−1)
)
1√2πt1
e−w2
12t1 (4.8)
111
Mean and variance. From (4.6) it follows that the mean and variance of W (t) are
E(W (t)) =
∫ ∞
−∞
w√2πt
e−w2
2t dw = 0 and E(W 2(t)) =
∫ ∞
−∞
w2
√2πt
e−w2
2t dw = t ∀t ≥ 0. (4.9)
(b.) Important properties: ρ(w, 0) = δ(w) and ρ(w′, t|w, t) = δ(w′ − w).
Both of these properties are used later in the derivation of the Fokker-Planck equation.
Let g(w) be an L1 function on (−∞,∞). Then the delta function δ(x) satisfies
∫ ∞
−∞g(w)δ(w)dw = g(0). (4.10)
Thus, to show that ρ(w, 0) = δ(w), we need to prove that
∫ ∞
−∞g(w)ρ(w, 0)dw = g(0), (4.11)
First, observe that∫ ∞
−∞g(w)ρ(w, t)dw =
∫ ∞
−∞g(w)
1√2πt
e−w2
2t dw. (4.12)
Let w = t1/2y. Then (4.12) becomes
∫ ∞
−∞g(w)ρ(w, t)dw =
1√2π
∫ ∞
−∞g(√ty)e−
y2
2 dy. (4.13)
Next, we let t→ 0+ on both sides of (4.13) and conclude that
∫ ∞
−∞g(w)ρ(w, 0)dw =
1√2π
∫ ∞
−∞g(0)e−
y2
2 dy = g(0)1√2π
∫ ∞
−∞e−
y2
2 dy = g(0), (4.14)
since 1√2π
∫∞−∞ e−
y2
2 dy = 1. Thus, we have shown that
∫ ∞
−∞g(w)ρ(w, 0)dw = g(0), (4.15)
From this and (4.10) it follows that ρ(w, 0) = δ(w).
Proof that ρ(w′, t|w, t) = δ(w′ − w). The function δ(w′ − w) satisfies
∫ ∞
−∞g(w′)δ(w′ − w)dw′ = g(w). (4.16)
Thus, to show that ρ(w′, t|w, t) = δ(w′ − w), it suffices to prove that
∫ ∞
−∞g(w′)ρ(w′, t|w, t)dw′ = g(w). (4.17)
Proceeding as in part (b.), we observe that
∫ ∞
−∞g(w′)ρ(w′, t|w, s)dw′ =
∫ ∞
−∞g(w′)
1√
2π(t− s)e−
(w′−w)2
2(t−s) dw′, t > s. (4.18)
112
Let w′ = w + y√t− s. Then (4.18) becomes
∫ ∞
−∞g(w′)ρ(w′, t|w, s)dw′ =
1√2π
∫ ∞
−∞g(w + y
√t− s)e−
y2
2 dy. (4.19)
Next, we let s→ t− on both sides of (4.19), and conclude that
∫ ∞
−∞g(w′)ρ(w′, t|w, t)dw′ =
1√2π
∫ ∞
−∞g(w)e−
y2
2 dy = g(w)1√2π
∫ ∞
−∞e−
y2
2 dy = g(w), (4.20)
since 1√2π
∫∞−∞ e−
y2
2 dy = 1. Thus, we have shown that (4.17) holds, as required.
Examples. Doering makes use (p. 20, eq. (4.8)) of ρ(x′, t|x, t) = δ(x′ − x) to obtain
∫ ∞
−∞x′ρ(x′, t|x, t)dx′ =
∫ ∞
−∞x′δ(x′ − x)dx′ = x, (4.21)
∫ ∞
−∞f(x′, t)∆tρ(x′, t|x, t)dx′ =
∫ ∞
−∞f(x′, t)∆tδ(x′ − x)dx′ = f(x, t)∆t. (4.22)
(c.) Proof that ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ wFirst we recall from above how to find the pdf for general Z = g(X,Y ). For a given z ∈ R define
DZ(z) = (x, y)|g(x, y) ≤ z. (4.23)
Let f(x, y) be the joint pdf for X and Y. Then the cdf FZ(z) is defined by
FZ(z) =
∫ ∫
DZ(z)
f(x, y)dydx. (4.24)
Example I. If Z = Y +X then
DZ(z) = (x, y)|y ≤ z − x (4.25)
and
FZ(z) =
∫ ∫
DZ (z)
f(x, y)dydx =
∫ ∞
−∞
∫ z−x
−∞f(x, y)dydx. (4.26)
The corresponding pdf for Z is the convolution
fZ(z) =d
dzFZ(z) =
∫ ∞
−∞f(x, z − x)dydx. (4.27)
Example II. If Z = Y −X then the pdf for Z is
fZ(z) =d
dzFZ(z) =
∫ ∞
−∞f(x, z + x)dydx. (4.28)
Example III. Suppose that Y and X are functions of t and that Z = Y (t2) −X(t1). Then
fZ(z) =d
dz
∫ ∞
−∞
∫ z+x
−∞f(x, t1, y, t2)dydx =
∫ ∞
−∞f(x, t1, z + x, t2)dx. (4.29)
113
Now let z = w, Y (t) = W (t+ h) and X(t) = W (t). Then t2 = t+ h, t1 = t, and we get
fZ(w) =d
dw
∫ ∞
−∞
∫ w+w1
−∞ρ(w1, t+ h,w2, t)dw2dw1 =
∫ ∞
−∞ρ(w1, t, w + w1, t+ h)dw1. (4.30)
Thus, since ρ(w1, t, w + w1, t+ h) = ρ(w + w1, t+ h,w1, t), we have
fZ(w) =
∫ ∞
−∞ρ(w + w1, t+ h,w1, t)dw1. (4.31)
where ρ(w + w1, t+ h,w1, t) is the joint pdf for the Wiener process, and
ρ(w + w1, t+ h,w1, t) = ρ(w + w1, t+ h|w1, t)ρ(w1, t). (4.32)
Recall that, if t′ > t′′, then
ρ(w′, t′|w′′, t′′) =1
√
2π(t′ − t′′)e− (w′′−w′)2
2(t′−t′′) , and ρ(w′, t′) =1√2πt′
e−(w′)22t′ . (4.33)
Combining (4.33) with (4.32) gives
ρ(w + w1, t+ h,w1, t) =1√2πh
e−(w)2
2h1√2πt
e−(w1)2
2t . (4.34)
Substituting (4.34) into (4.30), we obtain
fZ(w) =d
dwProbW (t+ h) −W (t) ≤ w =
∫ ∞
−∞
1√2πh
e−(w)2
2h1√2πt
e−(w1)2
2t =1√2πh
e−(w)2
2h . (4.35)
Finally, since W (0) = 0, we note that
d
dwProbW (h) −W (0) ≤ w =
d
dwProbW (h) ≤ w =
d
dw
∫ w
−∞
1√2πh
e−(w)2
2h =1√2πh
e−(w)2
2h .
(4.36)
Thus, we conclude that
ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ w, (4.37)
and the proof is complete.
(d.) Proof that E(W (t)W (s)) = min(t, s).
First, recall that if X is a normal random variable then its probability density is
fX(x) =1√2πσ
e−(x−µ)2
2σ2 dx, (4.38)
where the mean, µ, and variance σ2, satisfy
E(X) = µ and E(X2) − [E(X)]2 = σ2. (4.39)
Let t > s. It follows from (4.7) that the autocorellation E(W (t)W (s)) is
E (W (t)W (s)) =
∫ ∞
−∞
∫ ∞
−∞ww′ 1
√
2π(t− s)e−
(w−w′)22(t−s)
1√2πs
e−w′22s dwdw′. (4.40)
114
Rearranging terms gives
E (W (t)W (s)) =
∫ ∞
−∞
1√2πs
w′e−w′22s
(
∫ ∞
−∞w
1√
2π(t− s)e−
(w−w′)22(t−s) dw
)
dw′. (4.41)
Using (4.38)-(4.39), we see that the inner integral reduces to
∫ ∞
−∞w
1√
2π(t− s)e−
(w−w′)22(t−s) dw = w′. (4.42)
Combining (4.41), (4.42) and a second application of (4.38)-(4.39) gives
E (W (t)W (s)) =1√2πs
∫ ∞
−∞w′2e−
w′22s dw′ = s = min(t, s). (4.43)
(e.) Proof that W (t) has independent increments.
We need to prove that
E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = E (W (t2) −W (t1))E (W (t1) −W (t0)) , 0 ≤ t0 < t1 < t2.
(4.44)
Expanding the left side of (4.44), and using the property E(W (t)W (s)) = min(t, s), we obtain
E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = E(W (t2)W (t1))−E(W 2(t1))−E(W (t2)W (t0))+E(W (t1)W (t0))
(4.45)
This reduces to
E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = t1 − t1 + t0 − t0 = 0. (4.46)
Similarly, an expansion of the right side of (4.44) gives
E (W (t2) −W (t1))E (W (t1) −W (t0)) = (E(W (t2)) − E(W (t1))) (E(W (t1)) − E(W (t0))) = 0,
(4.47)
since E(W (t)) = 0 ∀t ≥ 0.
(f.) Proof that W (t) −W (s) ∼√t− sN(0, 1) when t > s.
This property is at the core of why matlab uses dW = (√
∆t)(randn) in the solution of stochastic
ode’s. We need to prove that the cdf’s P (W (t) −W (s) ≤ w) and P (√t− sX ≤ w) have the same
pdf, where X = N(0, 1) denotes the normal random variable. These cdf’s have the same pdf if
d
dwP (
√t− sX ≤ w) =
d
dwP (
√t− sX ≤ w) =
1√
2π(t− s)e−
w2
2(t−s) . (4.48)
First, we know from (4.1) that
P (W (t) −W (s) ≤ w) =
∫ w
−∞
1√
2π(t− s)e−
η2
2(t−s) dη, (4.49)
115
henced
dwP (W (t) −W (s) ≤ w) =
1√
2π(t− s)e−
w2
2(t−s). (4.50)
Next,
P (√t− sX ≤ w) = P
(
X ≤ w√t− s
)
=
∫ w√t−s
−∞
1√2πe−
η2
2 dη, (4.51)
and therefored
dwP (
√t− sX ≤ w) =
1√
2π(t− s)e−
w2
2(t−s). (4.52)
Thus, (4.48) now follows from (4.50) and (4.52) and the proof is complete.
(g.) W (t) is m.s. continuous, and continuous in probability at t.
Definition A random process X(t) is mean square continuous at t if
limh→0
E([X(t+ h) −X(t)]2) = 0. (4.53)
We now show that W (t) is mean square continuous at t. First, because of the linearity of E, we have
E(
[W (t+ ∆t) −W (t)]2)
= E(
W 2(t+ ∆t))
− 2E (W (t+ ∆t)W (t)) + E(
W 2(t))
. (4.54)
This, and (4.43), imply that
E(
[W (t+ ∆t) −W (t)]2)
= t+ ∆t− 2t+ t = ∆t. (4.55)
From (4.55) we conclude that
lim∆t→0
E(
[W (t+ ∆t) −W (t)]2)
= 0, (4.56)
hence W (t) is continuous in mean square at t.
In Section 1 (part h) it was proved that a randon process that is mean square continuous at t is also
continuous in probability at t. Thus, W (t) is continuous in probability at t.
(h.) Proof that W (t) does not have a mean square first derivative.
Recall from part (j.), property 4 of the previous section that if a random process X(t) has a mean
square first derivative then RX′(t, s) = ∂2RX (t,s)∂t∂s ∀t, s ≥ 0. Therefore, to prove that W (t) does not
have a mean square derivative, we’ll prove that
∂2RW (t, s)
∂t∂s|s=t does not exist. (4.57)
First, recall that the autocorrelation function for the Wiener process satisfies For this start by recalling
that
RW (t, s) = σ2min(t, s) (4.58)
116
From this it follows that
lim∆s→0RW (t, s+ ∆s) −RW (t, s)
∆s= H(t− s), (4.59)
Where H is the heaviside function. Therefore
∂2
∂s∂tRW (t, s) does not exist. (4.60)
This completes the proof that W (t) does not have a mean square first derivative.
(i.) The generalized first derivative of W satisfies RW ′ (t, s) = δ(t− s)
In the last section it was proved that
lim∆s→0RW (t, s+ ∆s) −RW (t, s)
∆s= H(t− s), (4.61)
Thus (handwaving, admittedly), if W is a Wiener process with parameter σ2, then
RW ′(t, s) = E(W ′(t)W ′(s)) =∂2
∂s∂tE(W (t)W (s)) =
∂2
∂s∂tRW (t, s) = σ2δ(t− s). (4.62)
(j.) Properties of the mean square integral Y =∫ t
t0W (α)dα.
Definition. Recall from Section 1, part (k) that the mean square integral
∫ t
t0
X(α)dα exists ⇐⇒∫ t
t0
∫ t
t0
RX(α, β)dαdβ exists. (4.63)
The Wiener process with parameter σ has autocorellation function RW (t, s) = σ2min(t, s), and it
follows that∫ t
0
∫ t
0RW (α, β)dαdβ exists. Thus, Y (t) =
∫ t
0W (α)dα exists.
Property 1. The mean of Y (t) is
µY (t) = E (Y (t)) =
∫ t
0
E (W (α)) dα = 0. (4.64)
Property 2. The variance of Y (t) is
V ar(Y (t)) = E(
Y 2(t))
=∫ t
0
∫ t
0 E (W (α)W (β)) dαdβ =∫ t
0
∫ t
0 (min(α, β) dαdβ
=∫ t
0
(
∫ β
0αdα+
∫ t
ββdα
)
dβ =∫ t
0β2
2 dβ.+∫ t
ββ(t− β)dβ = σ2t3
3 .(4.65)
Property 3. Let t > s. Then the autocorellation function RY (t)(t, s) satisfies
RY (t)(t, s) =1
6σ2s2(3t− s). (4.66)
The proof follows Problem 6.12, p. 224 of Schaum’s.The big insight is to write Y (t) as
Y (t) =
∫ s
0
W (α)dα +
∫ t
s
W (α)dα = Y (s) +
∫ t
s
(W (α) −W (s)) dα+ (t− s)W (s). (4.67)
117
He does this to make use of the ”independent increment ” property of W (t). Next,
RY (t)(t, s) = E(
Y 2(s))
+∫ t
sE (W (α) −W (s))Y (s)dα + (t− s)E(W (s)Y (s))
= σ2s3
3 +∫ t
s E(
[W (α) −W (s)]∫ s
0 W (β))
dβdα+ (t− s)∫ s
0 E(W (s)W (β))dβ
= σ2s3
3 +∫ t
s
∫ s
0E ([W (α) −W (s)]W (β)) dβdα + σ2(t− s)
∫ s
0min(s, β)dβ
= σ2s3
3 +∫ t
s
∫ s
0 E ([W (α) −W (s)][W (β) −W (0)]) dβdα+ σ2(t− s)∫ s
0 βdβ
= σ2s3
3 +∫ t
s
∫ s
0E (W (α) −W (s))E (W (β) −W (0)) dβdα+ σ2(t− s) s2
2 ,
(4.68)
since 0 < β < s < α implies that W (α)−W (s) and W (β) −W (0) are independent. The definition of
W (t) guarantees that E(W (α) −W (s)) = E(W (β) −W (0)) = 0, hence (4.68) reduces to
RY (t)(t, s) =σ2s3
3+ σ2(t− s)
s2
2=σ2s2
6(3t− s). (4.69)
(k.) Y (t) =∫ t
0 X(η)dη is a Wiener process if X(t) is normal white noise.
To prove this result we follow Problem 6.22, p. 230 of Schaum’s. Let Y (t) =∫ t
0X(η)dη, where
X(t) is a normal white noise process. Then Y (t) is also a normal process (see, Problem 6.10, p. 223
of Schaum’s). Therefore, to conclude that Y (t) is a Wiener process we need to verify that their
autocorellation functions are equal, i.e. RY (t, s) = RW (t, s) = min(t, s). We start with
RY (t, s) = σ2
∫ t
0
∫ s
0
RX(α, β)dβdα = σ2
∫ t
0
∫ s
0
δ(α− β)dαdβ. (4.70)
Note that if z > 0 then
σ2
∫ u
0
δ(α− z)dα = σ2H(u− z), (4.71)
where H is the Heaviside function. Combining (4.70) and (4.71) shows that (4.70) reduces to
RY (t, s) = σ2
∫ t
0
H(s− β)dβ. (4.72)
Now consider two cases: if s > t then s > β, H(s− β) = 1 and (4.70) becomes
RY (t, s) = σ2
∫ t
0
dβ = σ2t = min(t, s). (4.73)
Next, suppose that s < t. Then
RY (t, s) = σ2
∫ s
0
H(s− β)dβ + σ2
∫ t
s
H(s− β)dβ = σ2
∫ s
0
H(s− β)dβ = σ2s = min(t, s). (4.74)
The case s = t is similar. Thus, we conclude that RY (t, s) = min(t, s) = RW (t, s), as required.
(l.) Scaling: dWτ =√ǫdWt if τ = ǫt. (i.e. they have the same pdf’s)
Proof: First, define
dWτ = W (τ + ∆τ) −W (τ) and dWt = W (t+ ∆t) −W (t). (4.75)
118
Then
P (dWτ ≤ w) = P (W (τ + ∆τ) −W (τ) ≤ w) (4.76)
Substitute τ = ǫt anad ∆t = ǫ∆t into (4.76) and get
P (W (τ + ∆τ) −W (τ) ≤ w) = P (W (ǫt+ ǫ∆t) −W (ǫt) ≤ w) (4.77)
The stationary property implies that the right side reduces to
P (W (ǫt+ ǫ∆t) −W (ǫt) ≤ w) = P (W (ǫ∆t) −W (0) ≤ w) = P (W (ǫ∆t) ≤ w) (4.78)
Next, prove that
P (W (ǫ∆t) ≤ w) = P(√ǫW (∆t) ≤ w
)
(4.79)
First, note that
P (W (ǫ∆t) ≤ w) =
∫ w
−∞
1√2πǫ∆t
exp(− x2
2ǫ∆t)dx (4.80)
Now let x = u√ǫ and transform (4.80) into
P (W (ǫ∆t) ≤ w) =
∫ w√ǫ
−∞
1√2π∆t
exp(− u2
2∆t)du (4.81)
Thus,
P (W (ǫ∆t) ≤ w) = P
(
W (∆t) ≤ w√ǫ
)
= P(√ǫW (∆t) ≤ w
)
(4.82)
Now work backwards from the right side. That is,
P(√ǫW (∆t) ≤ w
)
= P(√ǫ [W (t+ ∆t) −W (t)] ≤ w
)
= P(√ǫdWt ≤ w
)
(4.83)
This imnplies that dWτ =√ǫdWt (i.e. they have the same pdf’s).
(m.) dXt = f(Xt)dt+B(Xt)dWt becomes ǫdXτ = f(Xt)dτ +√ǫB(Xt)dWτ if τ = ǫt.
Proof. Multiply both sides of
dXt = f(Xt)dt+B(Xt)dWt (4.84)
by ǫ and get
ǫdXt = f(Xt)ǫdt+B(Xt)ǫdWt. (4.85)
From the previous section we have dWτ =√ǫdWt. Also, dτ = ǫdt. Substitute these into (4.85) and
obtain the required equation, i.e.
ǫdXτ = f(Xτ )dτ +B(Xτ )√ǫdWτ (4.86)
119
5 Doering’s derivation of the Fokker-Planck equation
(a.) The Stochastic ODE.
The Langevin stochastic differential equation for a first order Markov process is
dX(t)
dt= f(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (5.1)
where ζ(t) is white noise. Our goal is to derive the Fokker-Planck equation (i.e. Kolmogorov’s forward
equation) for the associated conditional probability density function, namely
∂
∂tρ(x, t|y, s) = − ∂
∂x(f(x, t)ρ(x, t|y, s)) +
1
2
∂2
∂x2
(
g(x, t))2ρ(x, t|y, s))
. (5.2)
At s = t the solution must satisfy
ρ(x, s|y, s) = δ(x− y) (5.3)
Remarks. (i) The interpretation of the product ρ(x, t|y, s)dx is
ρ(x, t|y, s)dx = P (X(t) ∈ (x, x+ dx)|X(s) = y) . (5.4)
(ii) If f = 0 and g = 1 then the pde (5.2) becomes
∂
∂tρ(x, t|y, s) =
1
2
∂2
∂x2ρ(x, t|y, s), (5.5)
which is satified by the conditional probability ρ(w, t|w′, s) = 1√2π(t−s)
e−(w−w′)22(t−s) , t > s, defined above
in (4.3) for the Wiener process W (t). The transition probability density function ρ(w, t|w′, s) is the
pdf for the solution W (t) of the stochastic ODE problem
W ′ = ζ(t), t > s, such that W (s) = w′. (5.6)
(iii) Let t > s. Then the mean of g(X(t)), given that X(s) = y, is
E (g(X(t))|X(s) = y) =
∫ ∞
−∞g(x′)ρ(x′, t|y, s)dx′. (5.7)
(iv) Important example: if s = t and y = x then “the mean of g(X(t)), given thatX(s) = x, ” is
E (g(X(t))|X(t) = x) =
∫ ∞
−∞g(x′)ρ(x′, t|x, t)dx′ =
∫ ∞
−∞g(x′)δ(x′ − y)dx′ = g(x). (5.8)
(v) More generally, for any continuous random variable X(t) it seems that it ought to be true that
E (g(X(t))|X(t) = x) = g(x). (5.9)
One way to get this is
E (g(X(t))|X(t) = x) =
∫ ∞
−∞g(x)fX(x′, t)dx′ = g(x)
∫ ∞
−∞fX(x′, t)dx′ = g(x). (5.10)
120
(vi) Even more generally, it seems that if we have two random variables X(t) and Y (t) then
E (g(X(t))Y (s)|X(t) = x) = E(g(x)Y (s)) = g(x)E(Y (s)). (5.11)
(b.) Evaluation of E(∆X(t)|X(t) = x) and E(
(∆X(t))2|X(t) = x)
.
To derive (5.2) Doering’s method is to let t > s and analyze the behavior of ρ on a small interval
(t, t + ∆t). For this he assumes that X(t) = x and analyzes the behavior of solutions of the related
discrete SDE
∆X(t) = X(t+ ∆t) −X(t) = f(X(t), t)∆t+ g(X(t), t)∆W (t), X(t) = x, (5.12)
where
∆W (t) = W (t+ ∆t) −W (t). (5.13)
Goal I. Prove that
E (∆X(t)|X(t) = x) =
∫ ∞
−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′ = f(x, t)∆t. (5.14)
This result will be used in the derivation of the Fokker-Planck equation. The first step in the proof is
to use linearity to conclude that
E (∆X(t)|X(t) = x) = E (X(t+ ∆t)|X(t) = x) − E (X(t)|X(t) = x) . (5.15)
Note that
E (X(t+ ∆t)|X(t) = x) =
∫ ∞
−∞x′ρ(x′, t+ ∆t|x, t)dx′ (5.16)
and
E (X(t)|X(t) = x) =
∫ ∞
−∞x′ρ(x′, t|x, t)dx′ =
∫ ∞
−∞x′δ(x′ − x)dx′ = x. (5.17)
Now combine property (5.17) with the requirement that∫∞−∞ ρ(x′, t+ ∆t|x, t)dx′ = 1 and get
E (X(t)|X(t) = x) = x = x
∫ ∞
−∞ρ(x′, t+ ∆t|x, t)dx′ =
∫ ∞
−∞xρ(x′, t+ ∆t|x, t)dx′. (5.18)
Finally, from (5.15), (5.16) and (5.18) it follows that
E (∆X(t)|X(t) = x) =
∫ ∞
−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′. (5.19)
The next step is to substitute the right side of (5.12) into E (∆X(t)|X(t) = x) for ∆X(t), which gives
E (∆X(t)|X(t) = x) = E (f(X(t), t)∆t|X(t) = x) + E (g(X(t), t)∆W (t)|X(t) = x) . (5.20)
The first term on the right side of (5.20) becomes
E (f(X(t), t)∆t|X(t) = x) =
∫ ∞
−∞f(x′, t)∆tρ(x′, t|x, t)dx′ =
∫ ∞
−∞f(x′, t)∆tδ(x′ − x)dx′ = f(x, t)∆t,
(5.21)
121
again because of (5.8). Next, since we are given that X(t) has the fixed value X(t) = x, then it seems
to make sense to conclude that X(t) and g(X(t)) do not depend on any other random variable, in
particular ∆W = W (t+ ∆t)−W (t). Also, ∆W is computed without the use of knowledge of X(t) or
g(X(t)). Therefore, it seems to make sense that ∆W and g(X(t)) are independent and that
E (g(X(t), t)∆W (t)|X(t) = x) = E (g(X(t), t)|X(t) = x)E (∆W (t)|X(t) = x) = 0. (5.22)
Here, we use the property
E (∆W |X(t) = x) = E (∆W ) = E(W (t+ ∆t)) − E(W (t)) = 0, (5.23)
since E(W (t)) = E(W (t+ ∆t)) = 0. Thus, it follows from (5.20), (5.21) and (5.22) that
E (∆X(t)|X(t) = x) = f(x, t)∆t. (5.24)
Finally, we combine (5.19) with (5.24) and obtain∫ ∞
−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′ = f(x, t)∆t. (5.25)
This completes the proof of (5.14).
Goal II. Prove that
E(
(∆X(t))2|X(t) = x)
=
∫ ∞
−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′ = g2(x, t)∆t + f2(x, t) (∆t)
2. (5.26)
This result will be used in the derivation of the Fokker-Planck equation.
The first step in the proof is to observe that
E(
(∆X(t))2|X(t) = x)
= E(
X2(t+ ∆t)|X(t) = x)
−2E (X(t+ ∆t)X(t)|X(t) = x)+E(
X2(t)|X(t) = x)
.
(5.27)
This translates into the integral form
E(
(∆X(t))2|X(t) = x)
=
∫ ∞
−∞(x′)2ρ(x′, t+∆t|x, t)dx′−2
∫ ∞
−∞xx′ρ(x′, t+∆t|x, t)dx′+
∫ ∞
−∞(x′)2ρ(x′, t|x, t)dx′
(5.28)
The second integral has x in the integrand since X(t) = x is given. Using (5.8), we conclude that
∫ ∞
−∞(x′)2ρ(x′, t|x, t)dx′ =
∫ ∞
−∞(x′)2δ(x′ − x)dx′ = x2 (5.29)
Because∫∞−∞ ρ(x′, t+ ∆t|x, t)dx′ = 1, it follows that
x2 = x2
∫ ∞
−∞ρ(x′, t+ ∆t|x, t)dx′ =
∫ ∞
−∞x2ρ(x′, t+ ∆t|x, t)dx′. (5.30)
We combine (5.28), (5.29) and (5.30) and obtain
E(
(∆X(t))2|X(t) = x)
=
∫ ∞
−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′. (5.31)
122
Next, we proceed as in the previous case and use the ode to conclude that
E(
(∆X(t))2|X(t) = x)
= E(
[f(X(t), t)∆t+ g(X(t), t)∆W ]2 |X(t) = x
)
(5.32)
The right side of (5.32) expands to
E(
[f(X(t), t)∆t]2 |X(t) = x
)
+2E (f(X(t), t)∆tg(X(t), t)∆W |X(t) = x)+E(
[g(X(t), t)∆W ]2 |X(t) = x
)
.
(5.33)
Translating the first term to integral form, and using (5.8) gives
E(
[f(X(t), t)∆t]2 |X(t) = x
)
=
∫ ∞
−∞(f(x′, t)∆t)2ρ(x′, t|x, t)dx′ =
∫ ∞
−∞(f(x′, t)∆t)2δ(x′−x)dx′ = f2(x, t)(∆t)2.
(5.34)
The second term on the right side of (5.35) becomes
2E (f(X(t), t)∆tg(X(t), t)∆W |X(t) = x) = 2E (f(X(t), t)∆tg(X(t), t)|X(t) = x)E (∆W |X(t) = x) = 0
(5.35)
Similarly, the third term on the right side of (5.35) becomes
E(
[g(X(t), t)∆W ]2 |X(t) = x
)
= E(
[g(X(t), t)]2 |X(t) = x
)
E(
[∆W ]2)
= g2(x, t)E(
[∆W ]2)
,
(5.36)
where
E([∆W ]2) = E([W (t+ ∆t)]2) − 2E(W (t+ ∆t)W (t)) + E(W 2(t)) = t+ ∆t− 2t+ t = ∆t. (5.37)
Combining (5.320, (5.35), (5.34), (5.36) and (5.37), we obtain
E(
(∆X(t))2|X(t) = x)
= g2(x, t)∆t+ f2(x, t)[∆t]2. (5.38)
Finally, we combine (5.31) with (5.38) and obtain∫ ∞
−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′ = g2(x, t)∆t + f2(x, t)[∆t]2. (5.39)
We make use of this identity below in part (e.) where the Fokker-Planck equation is derived.
(c.) The Chapman-Kolmogorov equation
To derive the Chapman-Kolmogorov equation we start with
ρ(x1, t1|x3, t3) =ρ(x1, t1, x3, t3)
ρ(x3, t3)=
1
ρ(x3, t3)
∫ ∞
−∞ρ(x1, t1, x2, t2, x3, t3)dx2 (5.40)
An algebraic manipulation gives
ρ(x1, t1, x2, t2, x3, t3)
ρ(x3, t3)=ρ(x1, t1, x2, t2, x3, t3)
ρ(x2, t2, x3, t3)
ρ(x2, t2, x3, t3)
ρ(x3, t3). (5.41)
Using conditional probability notation, we write (5.41) in the form
ρ(x1, t1, x2, t2, x3, t3)
ρ(x3, t3)= ρ(x1, t1|x2, t2, x3, t3)ρ(x2, t2|x3, t3). (5.42)
123
The Markov property implies that the first term on the right side of (5.42) reduces to
ρ(x1, t1|x2, t2, x3, t3) = ρ(x1, t1|x2, t2). (5.43)
Combining (5.43) with (5.42) gives the important identity
ρ(x1, t1, x2, t2, x3, t3)
ρ(x3, t3)= ρ(x1, t1|x2, t2)ρ(x2, t2|x3, t3). (5.44)
Finally, we substitute (5.44) into (5.40) and obtain the Chapman-Kolmogorov equation
ρ(x1, t1|x3, t3) =
∫ ∞
−∞ρ(x1, t1|x2, t2)ρ(x2, t2|x3, t3)dx2. (5.45)
(d.) Use of the Chapman-Kolmogorov eq. to derive the Fokker-Planck eq.
We make use the following version of (5.45):
ρ(x, t+ ∆t|y, s) =
∫ ∞
−∞ρ(x, t+ ∆t|z, t)ρ(z, t|y, s)dz. (5.46)
Our goal is to show how to use (5.46) to derive the Fokker-Planck pde
∂
∂tρ(x, t|y, s) +
∂
∂x[f(x, t)ρ(x, t|y, s)] − ∂2
∂x2
[
g2(x, t)ρ(x, t|y, s)]
= 0. (5.47)
First, it follows from an integration of (5.46) that
∫ ∞
−∞R(x)ρ(x, t + ∆t|y, s)dx =
∫ ∞
−∞R(x)
∫ ∞
−∞ρ(x, t+ ∆t|z, t)ρ(z, t|y, s)dzdx, (5.48)
where R(x) is a C∞ function. Changing the order of integration gives∫ ∞
−∞R(x)ρ(x, t + ∆t|y, s)dx =
∫ ∞
−∞ρ(z, t|y, s)
(∫ ∞
−∞R(x)ρ(x, t + ∆t|z, t)dx
)
dz. (5.49)
Next, expanding R(x) in a power Taylor series centered at x = z gives∫ ∞
−∞ρ(x, t+∆t|y, s)dx =
∫ ∞
−∞ρ(z, t|y, s)
∫ ∞
−∞
[
R(z) +R′(z)(x− z) +R′′(z)
2(x− z)2 + ...
]
ρ(x, t+∆t|z, t)dxdz.(5.50)
Keep terms up to second order (this is WLOG), and regroup them to obtain
∫∞−∞R(x)ρ(x, t+ ∆t|y, s)dx =
∫∞−∞ ρ(z, t|y, s)R(z)dz
+∫∞−∞ ρ(z, t|y, s)R′(z)
∫∞−∞(x− z)ρ(x, t+ ∆t|z, t)dxdz
+∫∞−∞ ρ(z, t|y, s)R′′(z)
2
∫∞−∞(x − z)2ρ(x, t+ ∆t|z, t)dxdz.
(5.51)
Subtituting (5.25) and (5.39) into the right side of (5.51) reduces (5.51) to
∫∞−∞R(x)ρ(x, t + ∆t|y, s)dx =
∫∞−∞ ρ(z, t|y, s)R(z)dz
+∫∞−∞ ρ(z, t|y, s)R′(z)f(z, t)∆tdz
+∫∞−∞ ρ(z, t|y, s)R′′(z)
2
[
g2(z, t)∆t+ f2(z, t)[∆t]2]
dz.
(5.52)
124
Next, replace z with x on the right side of (5.53) and obtain
∫∞−∞R(x)ρ(x, t + ∆t|y, s)dx =
∫∞−∞ ρ(x, t|y, s)R(x)dx
+∫∞−∞ ρ(x, t|y, s)R′(x)f(x, t)∆tdx
+∫∞−∞ ρ(x, t|y, s)R′′(x)
2
[
g2(x, t)∆t+ f2(x, t)[∆t]2]
dx.
(5.53)
Put all terms on one side of the equation, divide by ∆t, and get
0 =∫∞−∞R(x)ρ(x,t+∆t|y,s)−ρ(x,t|y,s)
∆t dx−∫∞−∞ ρ(x, t|y, s)R′(x)f(x, t)dx
−∫∞−∞ ρ(x, t|y, s)R′′(x)
2
[
g2(x, t) + f2(x, t)∆t]
dx.(5.54)
Let ∆t→ 0 and this equation reduces to
0 =
∫ ∞
−∞R(x)
∂
∂tρ(x, t|y, s)dx−
∫ ∞
−∞R′(x)ρ(x, t|y, s)f(x, t)dx −
∫ ∞
−∞R′′(x)
1
2ρ(x, t|y, s)g2(x, t)dx.
(5.55)
Integrate the second and third integrals by parts and obtain
0 =
∫ ∞
−∞R(x)
∂
∂tρ(x, t|y, s)dx+
∫ ∞
−∞R(x)
∂
∂x[f(x, t)ρ(x, t|y, s)] dx−
∫ ∞
−∞R(x)
1
2
∂2
∂x2
[
g2(x, t)ρ(x, t|y, s)]
dx.
(5.56)
Combining all three integrals together gives
0 =
∫ ∞
−∞R(x)
(
∂
∂tρ(x, t|y, s) +
∂
∂x[f(x, t)ρ(x, t|y, s)] − 1
2
∂2
∂x2
[
g2(x, t)ρ(x, t|y, s)]
)
dx. (5.57)
Finally, since R(x) is arbitrary, it follows that ρ(x, t|y, s) satisfies the Fokker-Planck pde
∂
∂tρ(x, t|y, s) +
∂
∂x[f(x, t)ρ(x, t|y, s)] − 1
2
∂2
∂x2
[
g2(x, t)ρ(x, t|y, s)]
= 0 (5.58)
This completes the derivation of the Fokker-Planck equation.
(e.) Use of Ito’s Lemma to derive the Fokker-Planck eq.
Assume that the SDE for a first order Markov process X(t) satisfying X(s) = y is
dX(t)
dt= a(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (5.59)
To keep with the notation in various books, we changed f(X(t), t) to a(X(t), t) in the SDE. Our goal
is to derive the associated the conditional pdf ρ(x, t|y, s). To use Ito’s Lemma we let f ∈ C∞(−∞,∞)
be arbitrarily chosen and assume that f has compact support. Then, by Ito’s Lemma, f(X(t)) satisfies
df(X(t)) =
(
a(X(t), t)fx(X(t)) +g2(X(t), t)
2fxx(X(t))
)
dt+ g(X(t), t)fx(X(t))dW (t). (5.60)
Integrate both sides of (5.60) from s to t and get
f(X(t)) = f(x) +
∫ t
s
(
a(X(η), η)fx +g2(X(η), η)
2fxx
)
dη +
∫ t
s
g(X(η), η)fxdW (η). (5.61)
125
Next, integrate both sides of (5.61) against the conditional pdf ρ(x, t|y, s) and get
E(f(X(t)) = f(x) +
∫ t
s
E
(
a(X(η), η)fx(X(η)) +g2(X(η), η)
2fxx(X(η))
)
dη, (5.62)
where E(h(X(τ), τ))) is defined by
E(h(X(τ), τ)) =
∫ ∞
−∞h(x′, τ)ρ(x′, τ |y, s)dx′. (5.63)
Take the first derivative of both sides of (5.62) with respect to t and get
d
dtE(f(X(t))) = E(a(X(t), t)fx(X(t))) +E(
g2(X(t), t)
2fxx(X(t))). (5.64)
The left side of (5.64) reduces to
d
dtE(f(X(t))) =
d
dt
∫ ∞
−∞f(x′)ρ(x′, t|y, s)dx′ =
∫ ∞
−∞f(x′)ρt(x
′, t|y, s)dx′ (5.65)
Integration by parts shows that
E(a(X(t), t)fx(X(t))) =∫∞−∞ a(x′, t)fx(x′)ρ(x′, t|y, s)dx′
= −∫∞−∞ f(x′) ∂
∂x′ (a(x′, t)ρ(x′, t|y, s))dx′
(5.66)
andE(g2(X(t),t)
2 fxx(X(t))) =∫∞−∞
g2(x′,t)2 fxx(x′)ρ(x′, t|y, s)dx′
=∫∞−∞ f(x′) ∂2
∂x′2 (g2(x′,t)2 ρ(x′, t|y, s))dx′
(5.67)
Finally, equate the right sides of (5.65), (5.66) and (5.67) and get
∫ ∞
−∞f(x′)ρt(x
′, t|y, s)dx′ =
∫ ∞
−∞f(x′)(− ∂
∂x′(a(x′, t)ρ(x′, t|y, s)) +
∂2
∂x′2(g2(x′, t)
2ρ(x′, t|y, s)))dx′
(5.68)
Since f(x) is arbitrary then the integrands are equal, which shows that the transition density ρ(x, t|y, s)satisfies the Fokker-Planck equation
ρt(x, t|y, s) = − ∂
∂x(a(x′, t)ρ(x, t|y, s)) +
∂2
∂x2(g2(x, t)
2ρ(x, t|y, s)) (5.69)
126
6 Derivation of the Backwards Kolmogorov Equation.
The stochastic differential equation is
dX(t)
dt= f(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (6.1)
Our goal is to derive the Backwards Kolmogorov Equation
∂
∂sρ(x, t|y, s) = −f(y, s)
∂
∂yρ(x, t|y, s) − g2(y, s)
2
∂2
∂y2ρ(x, t|y, s), ρ(x, s|y, s) = δ(x− y). (6.2)
The starting point is
∂
∂sρ(x, t|y, s) = lim
∆s→0
1
∆s(ρ(x, t|y, s+ ∆s) − ρ(x, t|y, s)) . (6.3)
We use the following version of the Chapman-Kolmogorov equation (5.45):
ρ(x, t|y, s) =
∫ ∞
−∞ρ(x, t|z, s+ ∆s)ρ(z, s+ ∆s|y, s)dz. (6.4)
Substitution of (6.4) into (6.3) gives
∂
∂sρ(x, t|y, s) = lim
∆s→0
1
∆s
(
ρ(x, t|y, s+ ∆s) −∫ ∞
−∞ρ(x, t|z, s+ ∆s)ρ(z, s+ ∆s|y, s)dz
)
. (6.5)
Next, observe that∫ ∞
−∞ρ(z, s+ ∆s|y, s)dz = 1. (6.6)
Combining (6.5) and (6.6 ), we obtain
∂
∂sρ(x, t|y, s) = lim
∆s→0
∫ ∞
−∞
ρ(z, s+ ∆s|y, s)∆s
(ρ(x, t|y, s+ ∆s) − ρ(x, t|z, s+ ∆s)) dz. (6.7)
Expanding (be careful here) ρ(x, t|z, s+ ∆s) around z = y gives
ρ(x, t|z, s+ ∆s) ≈ ρ(x, t|y, s+ ∆s) + ρy(x, t|y, s+ ∆s)(z − y) +1
2ρyy(x, t|y, s+ ∆s)(z − y)2. (6.8)
Substitute (6.8) into the right side of (6.7) and get
∂
∂sρ(x, t|y, s) = − lim
∆s→0
∫ ∞
−∞
ρ(z, s+ ∆s|y, s)∆s
(
∂
∂yρ(x, t|y, s+ ∆s)(z − y) +
1
2
∂2
∂y2ρ(x, t|y, s+ ∆s)(z − y)2
)
dz.
(6.9)
We need to evaluate the right side of (6.9). First, observe that
∫ ∞
−∞ρ(z, s+ ∆s|y, s) ∂
∂yρ(x, t|y, s+ ∆s)(z − y)dz =
∂
∂yρ(x, t|y, s+ ∆s)
∫ ∞
−∞ρ(z, s+ ∆s|y, s)(z − y)dz
(6.10)
It follows exactly as in the derivation of (5.25) that
∂
∂yρ(x, t|y, s+ ∆s)
∫ ∞
−∞ρ(z, s+ ∆s|y, s)(z − y)dz =
∂
∂yρ(x, t|y, s+ ∆s)f(y, s)∆s. (6.11)
127
Combining (6.10) and (6.11) gives
∫ ∞
−∞ρ(z, s+ ∆s|y, s) ∂
∂yρ(x, t|y, s+ ∆s)(z − y)dz =
∂
∂yρ(x, t|y, s+ ∆s)f(y, s)∆s. (6.12)
Likewise, following the same steps as in the derivation of (5.39), we obtain
∫ ∞
−∞ρ(z, s+∆s|y, s)1
2
∂2
∂y2ρ(x, t|y, s+∆s)(z−y)2dz =
1
2
∂2
∂y2ρ(x, t|y, s+∆s)
(
g2(y, s)∆s+ f2(y, s)[∆s]2)
.
(6.13)
Combining (6.9), (6.12) and (6.13), we conclude that
∂
∂sρ(x, t|y, s) = − lim
∆s→0
(
f(y, s)∂
∂yρ(x, t|y, s+ ∆s) + (g2(y, s) + f2(y, s)∆s)
1
2
∂2
∂y2ρ(x, t|y, s+ ∆s)
)
.
(6.14)
Finally, taking the limit, we obtain the backwards Kolmogorov equation
∂
∂sρ(x, t|y, s) = −f(y, s)
∂
∂yρ(x, t|y, s) − g2(y, s)
1
2
∂2
∂y2ρ(x, t|y, s). (6.15)
Important special case. When f and g are independent of t, the backwards Kolmogorov equation
becomes∂
∂sρ(x, t|y, s) = −f(y)
∂
∂yρ(x, t|y, s) − g2(y)
1
2
∂2
∂y2ρ(x, t|y, s). (6.16)
Suppose that s = 0 and that we want to derive the associated backwards equation satisfied by
ρ(x, t|y, 0). Gardner ([11], p. 137) notes that
ρ(x, t|y, 0) = ρ(x, 0|y,−t). (6.17)
Then ρ(x, t|y, 0) satisfies
∂
∂τρ(x, t|y, 0) = f(y)
∂
∂yρ(x, t|y, 0) + g2(y)
1
2
∂2
∂y2ρ(x, t|y, 0), ρ(x, 0|y, 0) = δ(x− y). (6.18)
Remarks: (i) We’ll use this form of the backwards equation in the analysis of exit times.
(ii) It is easily verified that the Wiener process conditional pdf (4.7) and the Ornstein-Uhlenbeck
process pdf (7.24) both satisfy (6.17).
128
7 Examples of the Fokker - Planck equation.
(a.) wienerSDEcalar.m solves the Wiener process SDE W ′ = σζ(t).
We show how to obtain a numerical “realization” of the Wiener process. The SDE is
dW (t)
dt= σζ(t), (7.1)
where ζ(t) is white noise. The site http://www.engineering.ucsb.edu/ moehlis/APC591/tutorials/tutorial7/node2.html
has a description of the matlab programs wienerSDEscalar.m (scalar version) and wienerSDEvec-
tor.m (vectorized version) which solve this equation. On google type wiener process in matlab and
click on the entry “ A Standard Wiener Process”. These programs are also the 1st two programs in
Higham’s article [12]. The “scalar” discretization of (7.1) is
W (ti+1) = W (ti) + σ(√
∆t)
randn, (7.2)
where√
∆t = ti+1 − ti, and randn is the command to obtain an element of N(0, 1). Here
(∆t)ζ(t) = (∆t)∆W
∆t= ∆W =
(√∆t)
randn. (7.3)
To see where the square root term comes we recall from (4.55) that
E(
[W (t+ ∆t) −W (t)]2)
= t+ ∆t− 2t+ t = ∆t. (7.4)
Thus E((∆W )2) = ∆t, and from this we get the average value ∆W =√
∆t in (12.58) and (7.3).
The program wienerSDEscalar.m below gives the “scalar” solution W ′(t) = σζ(t)
T = 1; N = 500; dt = T/N; sigma=2
dW=zeros(1,N); W=zeros(1,N); dW(1) = sqrt(dt)*randn; W(1) = dw(1);
sum=zeros(1,N); sum(1)=W(1); meanvalue=zeros(1,N); meanvalue(1)=W(1);
for j = 2:N
dW(j) = sigma*sqrt(dt)*randn; W(j) = W(j-1) + sigma*dW(j); sum(j)=W(j)+sum(j-1); end
for j=2:N
meanvalue(j)=sum(j)/j; end;
plot([0:dt:T],[0,meanvalue],[0:dt:T],[0,W],’r-’); xlabel(’t’,’FontSize’,16); ylabel(’W(t)’,’FontSize’,16,’Rotation’,0)
legend(’blue curve is the mean’,’red jagged vurve is W(t)’,2)
The program wienerSDEvector.m below gives the vectorized solution of W ′(t) = σζ(t).
T = 1; N = 500; dt = T/N; sigma=2; dW = sqrt(dt)*randn(1,N); W =σ cumsum(dW);
Note: the matlab command cumsum computes the integral∫ t
0dW (t)
plot([0:dt:T],[0,W],’r-’); xlabel(’t’,’FontSize’,16); ylabel(’W(t)’,’FontSize’,16,’Rotation’,0)
(b.) The Fokker-Planck equation for the Wiener process SDE
129
Recall that the Langevin stochastic differential equation for a first order Markov process X(t) is
dX(t)
dt= f(X(t), t) + g(X(t), t)ζ(t), (7.5)
where ζ(t) is white noise. The associated Fokker-Planck pde for the conditional probability density is
∂
∂tρ(x, t|y, s) =
[
− ∂
∂xf(x, t) +
1
2
∂2
∂x2(g(x, t))2
]
ρ(x, t|y, s). (7.6)
The intial condition is
ρ(x, s|y, s) = δ(x− y) (7.7)
The stochastic differential equation for the Wiener process is
dX(t)
dt= σζ(t), (7.8)
where ζ(t) is white noise. Thus, f(x, t) = 0 and g(x, t) = σ, and the Fokker-Planck equation (7.6) is
∂
∂tρ(x, t|y, s) =
σ2
2
∂2
∂x2ρ(x, t|y, s). (7.9)
Solution by the Fourier transform method. We let σ = 1 and solve the single time pde
∂
∂tu(x, t) =
1
2
∂2
∂x2u(x, t), (7.10)
with initial condition
u(x, 0) = δ(x). (7.11)
Taking the Fourier transform of both sides of (7.10), we obtain
∂
∂th(ω, t) = −ω
2
2h(ω, t), (7.12)
where
h(ω, t) =
∫ ∞
−∞e−iωxρ(x, t, y, s)dx. (7.13)
The Fourier transform of (7.11) gives
h(ω, 0) =
∫ ∞
−∞e−iωxδ(0)dx = 1. (7.14)
The solution of (7.12) which satisfies (7.14) is
h(ω, t) = e−ω2
2 t. (7.15)
Now consider the function v = be−ax2
. The Fourier transform of v is
v =
∫ ∞
∞
e−iωxbe−ax2
dx = be−ω2
4a
∫ ∞
−∞e−a(x−i ω2
2a )2dx =b√ae−
ω2
4a√π (7.16)
130
Let a = 12t and b = 1√
2πt. Then v = h(ω, t) and the corresponding functions u(x, t) and v(x, t) satisfy
u(x, t) = v(x, t) =1√2πt
e−x2
2t (7.17)
To obtain the conditional probability solution we replace the t with t− s and x with x− y to obtain
ρ(x, t|y, s) =1
√
2π(t− s)e−
(x−y)2
2(t−s) . (7.18)
Finally, the joint probability distribution ρ(x, t, y, s) is given by the product
ρ(x, t, y, s) =1
√
2π(t− s)e−
(x−y)2
2(t−s)1√2πs
e−y2
2s . (7.19)
(c.) ornstein.m solves the Ornstein-Uhlenbeck SDE dX(t)dt = −γX + σζ(t).
The matlab program ornstein.m computes the solution of the SDE dX(t)dt = −γX+σζ(t). The method
is to select points t1, t2, ..., tN all equally spaced, with ∆t = ti+1 − ti. The discretization is
X(ti+1) −X(ti) = −γX(ti) + σ(√
∆t)
randn. (7.20)
Here the lst term on the right is due to the observation that
(∆t)ζ(t) = (∆t)∆W
∆t= ∆W =
(√∆t)
randn, (7.21)
where W denotes the Wiener process. Note: Section 5 (d.) shows how to apply the Ito Lemma
to the Ornstein process. The program ItoExample2.m computes the true solution, solves the SDE
numerically using Euler’s method, and then compares the true and numerical solution.
The Focker-Planck equation. Substituting f(x, t) = −γx and g(x, t) = σ into
∂
∂tρ(x, t|y, s) =
[
− ∂
∂xf(x, t) +
1
2
∂2
∂x2(g(x, t))2
]
ρ(x, t|y, s) (7.22)
gives the pde
∂
∂tρ(x, t|y, s) = γρ(x, t|y, s) + γx
∂
∂xρ(x, t|y, s) +
σ2
2
∂2
∂x2ρ(x, t|y, s). (7.23)
Below, in part (d.) Monahan [16] shows how to use the SDE to derive the conditional probability
ρ(x, t|y, s) =1
√
2πg(t− s)e−
(x−ye−γ(t−s))2
2(g(t−s) , (7.24)
where
g(t) =σ2
2γ
(
1 − e−2γt)
. (7.25)
Letting t→ ∞ gives the steady state probability function
ρstat(x) =
√
γ
πσ2e−
γx2
σ2 (7.26)
131
Doering claims that “the covariance in the steady state is”
E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞ρ(x, t|y, s)ρstat(y)dxdy =
σ2
2γe−γ|t−s|. (7.27)
To verify (7.27) we begin with
E (X(t)X(s)) =
∫ ∞
−∞
∫ ∞
−∞
xy√
2πg(t− s)e−
(x−ye−γ(t−s))2
2(g(t−s)
√
γ
πσ2e−
γy2
σ2 dxdy (7.28)
Assume that t > s. Reordering the terms in the integration gives
E (X(t)X(s)) =
∫ ∞
−∞y
√
γ
πσ2e−
γy2
σ2
(
∫ ∞
−∞
x√
2πg(t− s)e−
(x−ye−γ(t−s))2
2(g(t−s) dx
)
dy (7.29)
This immeditately reduces to
E (X(t)X(s)) =
∫ ∞
−∞y
√
γ
πσ2e−
γy2
σ2
(
ye−γ(t−s))
dy. (7.30)
This further reduces to
E (X(t)X(s)) = e−γ(t−s)
∫ ∞
−∞y2
√
γ
πσ2e−
γy2
σ2 dy =σ2
2γe−γ|t−s|. (7.31)
Substituting s = t into (7.31) gives the “fluctuation-dissipation” equation
E([X(t)]2) =σ2
2γ. (7.32)
Note that (7.32) is independent of t. A second way to derive (7.32)is to use the steady state pdf
ρstat(x) and compute
E([X ]2) =
∫ ∞
−∞x2ρstat(x)dx =
σ2
2γ. (7.33)
The fluctuation-dissipation equation appears in the analysis when red noise is used to model cli-
mate [26].
(d.) Autocorellation and power functions for the Ornstein-Uhlenbeck SDE.
Doering does’nt say if ρ(x, t|y, s)ρstat(y) in (7.28) is the joint probability function ρ(x, t, y, s). In
Monahana’s and Doering’s notes it looks like the joint probability function is
ρ(x, t, y, s) =1
√
2πg(t− s)e−
(x−ye−γ(t−s))2
2(g(t−s)
√
γ
πσ2e−
γy2
σ2 , (7.34)
where g(t) = σ2
2γ
(
1 − e−2γt)
. Recall from (7.31) that
E (X(t)X(s)) =σ2
2γe−γ|t−s|. (7.35)
From this we obtain the autocorellation function
RX(τ) = E (X(t)X(t+ τ)) =σ2
2γe−γ|τ |. (7.36)
132
The “power” is given by
Power = RX(0) =σ2
2γ, (7.37)
and the “power function” is given by
S(ω) =1
2π
∫ ∞
−∞e−iωτ σ
2
2γe−γ|τ |dτ =
σ2
2π
1
ω2 + γ2. (7.38)
(e.) How to analyze the Ornstein-Uhlenbeck SDE to obtain ρ(x, t|y, s).
This follows the monahan.pdf notes and shows how to derive the pdf ρ(x, t|y, s) from the SDE
dX(t)
dt= −γX + σζ(t). (7.39)
The first step is to multiply (7.39) by eγt, integrate the result from 0 to t, and obtain
X(t) = e−γtX(0) + σ
∫ t
0
e−γ(t−t′)ζ(t′)dt′, ∀t ≥ 0. (7.40)
Monahan remarks (p. 4, top line of monahan.pdf ) that X(t) is a gaussian process (presumably since
the integral of white noise is gaussian - problem 6.22 of Schaum’s gives part of the proof). Therefore,
we only need to figure out the mean and variance for X(t), then substitute them into the right spot
in the formula for the pdf of a gaussian process. The mean is given by
E(X(t)) = E(e−γtX(0)) + σ
∫ t
0
e−γ(t−t′)E (ζ(t′)) dt′ = e−γtX(0) ∀t ≥ 0, (7.41)
since E(ζ(t)) = 0 ∀t ≥ 0. Next, to find the variance, start with the covariance
E ([X(t) − E(X(t))][X(t+ s) − E(X(t+ s))]) = σ2
∫ t
0
∫ t+s
0
e−γ(t−t′)eγ(t+s−t′)E (ζ(t′)ζ(t′′)) dt′′dt′.
(7.42)
There were four other terms in this computaion, but two cancel each other and two must be zero since
E(ζ(t)) = 0 ∀t ≥ 0. Because E(ζ(t′)ζ(t′′)) = δ(t′ − t′′), then (7.42) becomes
E ([X(t) − E(X(t))][X(t+ τ) − E(X(t+ τ))]) = σ2
∫ t
0
e−γ(t−t′)
(∫ t+τ
0
e−γ(t+s−t′′)δ(t′ − t′′)dt′′)
dt′.
(7.43)
Evaluation of the inner integral using the deltal function reduces (7.43) to
E ([X(t) − E(X(t))][X(t+ τ) − E(X(t+ τ))]) = σ2
∫ t
0
e−γ(2t+τ−2t′)dt′ =σ2
2γe−γ(2t+τ)
(
e2γt − 1)
.
(7.44)
Finally, letting τ = 0 reduces this to the variance formula
V ar(X(t)) = E(
[X(t) − E(X(t))]2)
=σ2
2γ
(
1 − e−2γt)
. (7.45)
(f.) The Fokker-Planck pde for systems of SDE’s
133
Systems (p.34 of Doering’s paper) of SDE’e have the form
dXi(t)
dt= fi(X1, .., XN) +
N∑
j=1
gi,j(X1, .., XN )ζj(t), i = 1, .., N (7.46)
Let G be the N by N matrix G = (gi,j). The matrix D = GGT satisfies
Di,j =
N∑
k=1
gi,kgj,k for 1 ≤ i, j ≤ N. (7.47)
The Fokker-Planck equation associated with the system (7.46) is
∂
∂tρ =
(
− ∂
∂xifi +
1
2
∂
∂xi
∂
∂xjDi,j
)
ρ. (7.48)
Here the subscript notation implies summation. That is, equation (7.48) is really
∂
∂tρ =
−N∑
i=1
∂
∂xifi +
1
2
N∑
i,j=1
∂
∂xi
∂
∂xjDi,j
ρ. (7.49)
Doering notes that the Fokker-Planck equation (7.48) can be written in the form
∂
∂tρ+
∂
∂xiJi = 0, (7.50)
where J is the current vector field with components
Ji(x, t) =
(
fi −1
2
∂
∂xiDi,j
)
ρ(x, t) (7.51)
The summation forms of (7.50) and (7.51) are
∂
∂tρ+
N∑
i=1
∂
∂xiJi = 0, (7.52)
Ji(x, t) =
fi −N∑
j=1
1
2
∂
∂xjDi,j
ρ(x, t) (7.53)
The current vector J gives the flow of probability in state space.
Open problem. Doering claims that finding ways to solve ∇ · J = 0 is an unresolved open problem.
134
8 Ito’s Lemma and applications
(a.) Informal proof of Ito’s Lemma
In google type Ito’s Lemma - Wikipedia to find all of this. Consider the stochastic ODE
dx(t)
dt= a(x, t) + b(x, t)ζ(t), (8.1)
where ζ(t) is normal white noise. Multiply both sides by dt and get the Ito process
dx(t) = a(x, t)dt + b(x, t)dW (t). (8.2)
Ito’s Lemma. Let f(x, t) be a C2 function. Then f(x(t), t) is an Ito process and
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.3)
The first step in proving (8.3) is to consider the expansion
f(x+ ∆x, t+ ∆t) = f(x, t) +∂f
∂x∆x+
∂f
∂t∆t+
1
2
∂2f
∂x2(∆x)2 + · · · (8.4)
Next, in (8.2) replace dx(t), dt and dW (t) with ∆x, ∆t and ∆W and get
∆x = a(x, t)∆t + b(x, t)∆W. (8.5)
Substitute (8.5) into (8.4), set (∆W )2 = ∆t (i.e. repalce (∆W )2 with its average value E(∆W )2 = ∆t)
and keep the lowest order terms. This gives
∆f =
(
a(x, t)∂f
∂x+∂f
∂t+
(b(x, t))2
2
∂2f
∂x2
)
∆t+ b(x, t)∂f
∂x∆W. (8.6)
where ∆f = f(x+ ∆x, t+ ∆t)− f(x, t). Replace ∆f, ∆t and ∆W with df, dt and dW, and get (8.3).
(b.) Example 1. Solve dx(t)dt = λx+ µxζ(t) using Ito’s Lemma.
The solution of dx(t)dt = λx + µxζ(t) using Ito’s Lemma is on pp. 59-61 of [23]. First, recall the SDE
formsdx(t)
dt= a(x, t) + b(x, t)ζ(t), (8.7)
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.8)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.9)
To put dx(t)dt = λx + µxζ(t) into the right form multiply by dt and get
dx(t) = λxdt + µxdW (t). (8.10)
Here a(x, t) = λx and b(x, t) = µx, and therefore (8.9) becomes
df(x(t), t) =
[
λx∂f
∂x+∂f
∂t+
1
2(µx)2
∂2f
∂x2
]
dt+ µx∂f
∂xdW (t). (8.11)
135
To apply Ito’s Lemma we let f(x, t) = ln(x), x > 0. Substituting ∂f∂x = 1
x ,∂f∂t = 0 and ∂2f
∂x2 = − 1x2
into (8.11) gives
df =
(
λ− µ2
2
)
dt+ µdW (t). (8.12)
Integration from 0 to t gives
ln(x(t)) = ln(x(0)) +
(
λ− µ2
2
)
t+ +µW (t) t ≥ 0. (8.13)
From this we obtain
x(t)) = x(0)e
((
λ−µ2
2
)
t+µW (t)
)
, t ≥ 0. (8.14)
Ocksendal[23], p. 62, points out that W (t) obeys the bound
lim supt→∞W (t)
√
2tlog(logt)= 1 a.s. (8.15)
From (8.14) and (8.15) Ocksendal., on p.60, makes the following conclusions:
if λ > µ2
2 then x(t) → ∞ as t→ ∞,
if λ < µ2
2 then x(t) → 0 as t→ ∞,
if λ = µ2
2 then x(t) flucutates between arbitrarily large and small values as t→ ∞.
(8.16)
Conclusions. When µ = 0 and λ > 0 the solution grows exponentially fast without bound. When
we add the multiplicative white noise term µxζ(t) to the equation the effect is to thwart the growth
because of the extra term −µ2
2 t that appears in the answer. This term won’t be present in section
(d.) where we solve dx(t)dt = λx + µζ(t). The difference is that the multiplicative white noise term is
replaced with the “aditive” white noise term µζ(t) in the SDE.
(c.) ItoExample1.m solves dx(t)dt = λx + µxζ(t) and compares with the true sol.
The program ItoExample1.m solves the stochastic ODE numerically and compares the numerical solu-
tion with the exact solution - www.engineering.ucsb.edu/ moehlis/APC591/tutorials/tutorial7/node3.html
describes the numerical method - in google type wiener process in matlab, and click on white noise.
From (8.14) above we have the exact solution x(t) = x(0)e
((
λ−µ2
2
)
t+µW (t)
)
, t ≥ 0.
The program ItoExample1.m is below.
lambda = 2; mu = 1; Xzero = 1; T = 1; N = 28; dt = 1/N ;
dW = sqrt(dt)*randn(1,N); W = cumsum(dW); Here W (t) =∫ t
0ζ(t′)dt′.
Xtrue = Xzero*exp((lambda-0.5*mu2)*([dt:dt:T])+mu*W);
plot([0:dt:T],[Xzero,Xtrue],’m-’), hold on
Xem = zeros(1,N); Xem(1) = Xzero + dt*lambda*Xzero + mu*Xzero*dW(1);
for j=2:N
136
Xem(j) = Xem(j-1) + dt*lambda*Xem(j-1) + mu*Xem(j-1)*dW(j); end
plot([0:dt:T],[Xzero,Xem],’b–*’), hold off
xlabel(’t’,’FontSize’,12); ylabel(’x’,’FontSize’,16,’Rotation’,0,’HorizontalAlignment’,’right’)
emerr = abs(Xem(end)-Xtrue(end))
(d.) Linear stability for dx(t)dt = λx+ µxζ(t)
According to Higham [12], pp. 539-541, the three kinds of linear stability are deterministic stability,
asymptotic stability and mean square stability. First, to compute the asymptotic stability fo the SDE,
we recall that the exact solution is x(t) = x(0)e
((
λ−µ2
2
)
t+µW (t)
)
, t ≥ 0. From this it follows that if
µ = 0 then x = 0 is deterministically stable if Re(λ) < 0. When µ 6= 0 we have the solution x = 0 is
asymtotically stable if Re(λ) < µ2
2 . This is because
|x(t)| → 0 as t→ ∞ ⇐⇒ Re(λ) <µ2
2. (8.17)
The solution x = 0 is mean square stable if E(x2(t)) → 0 as t→ ∞. For this we compute
E(x2(t)) = E(
x2(0)e((2λ−µ2)t+2µW (t)))
= x2(0)e(2λ−µ2)tE(
e2µW (t))
. (8.18)
Note that
E(
e2µW (t))
=
∫ ∞
−∞
e2µw
√2πt
e−w2
2t dt =
∫ ∞
−∞
e2µ2t
√2πt
e−(w−2µt)2
2t dt = e2µ2t. (8.19)
Combining (8.18) and (8.19), we conclude that
E(x2(t)) = x2(0)e(2λ+µ2)t → 0 as t→ ∞ ⇐⇒ Re(λ) < −µ2
2. (8.20)
Note that mean square stability implies asymptotic stability, but asymptotic stability does not neces-
sarily imply mean square stability. More details are in Higham’s article [12].
(e.) higham3.m solves dx(t)dt = x+ .5xζ(t) 1000 times and graphs 5 solutions
This is a very impiortant exercise. The program solves dx(t)dt = x + .5xζ(t) 1000 times, computes the
sample mean for all 1000 solutions, gropahs the first 5 solutions, and also graphs the mena of the 1000
solutions. The program higham3.m is below.
randn(’state’,100); clf; T=1; N=500; dt=T/N; t=[dt:dt:1];M=1000; dW=sqrt(dt)*randn(M,N); W=cumsum(dW,2);
dW is an M ( no. of W paths) row by N ( t1, .., tN values) column matrix of randn nos.
cumsum(dW,2) computes the cumulative sum across each row of dW.
U=exp(repmat(t,[M,1]) + 0.5*W);
The command repmat(t,[M,1]) creates an M by 1 matrix M rows. Each element of M is the row
t = [t1, t2, .., tN ] It’s easy to see if we let M=7, N=10, dt=T/N; t=[dt:dt:T] Then repmat(t,[M,1])
creates a 7 by 1 matrix - a column matrix, each entry of which is the row [.1, .2, .., .9, 1]
137
Umean=mean(U);
Umean is a single row, with N elements, corresponding to t1, .., t10 The first entry of Umean = (sum
of column 1 of U)/M the j-th entry is (sum of col j of U)/M, etc
figure(1); plot([0,t],[1,Umean],’b-’), hold on; plot([0,t],[ones(5,1),U(1:5,:)],’r–’); axis([0 1 0 5.5])
averr=norm( (Umean-exp(9*t/8)),’inf’)
(f.) ItoExample2.m solves dx(t)dt = λx+ µζ(t) and compares with the true sol.
First, recall the general SDE forms
dx(t)
dt= a(x, t) + b(x, t)ζ(t), (8.21)
dx(t) = a(x, t)dt + b(x, t)dW (t). (8.22)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.23)
To put dx(t)dt = λx + µζ(t) into the right form multiply by dt and get
dx(t) = λxdt + µdW (t). (8.24)
Here a(x, t) = λx and b(x, t) = µ, and therefore (8.23) becomes
df(x(t), t) =
[
λx∂f
∂x+∂f
∂t+
1
2µ2 ∂
2f
∂x2
]
dt+ µ∂f
∂xdW (t). (8.25)
Let f(x, t) = xe−λt. Substituting ∂f∂x = e−λt, ∂f
∂t = −λxe−λt and ∂2f∂x2 = 0 into (8.25) gives
df = µeλtdW (t). (8.26)
Integration from 0 to t gives
x(t)e−λt = x(0) + µ
∫ t
0
e−λsdW (s) t ≥ 0. (8.27)
From this we obtain the “true” solution
x(t) = x(0)eλt + µ
∫ t
0
eλ(t−s)dW (s) t ≥ 0. (8.28)
ItoExample2.m matlab program is below.
clf; lambda = 1 ;mu =2; Xzero = 1;T = 1; N = 28; dt = 1/N;
dW = sqrt(dt)*randn(1,N); These are the Brownian increments
The command to compute∫ t
0e−λsdW
integral =cumsum(exp(-lambda*[dt:dt:T]).*dW);
Next, compute the true solution x(t) = x(0)eλt + µ∫ t
0eλ(t−s)dW (s) is on the next line
Xtrue = Xzero*exp(lambda*([dt:dt:T]))+ mu*exp(lambda*[dt:dt:T]).* integral;
Plot the true solutions
138
figure(1); plot([0:dt:T],[Xzero,Xtrue],’m-’), hold on
Below, compute the ” numerical euler method solution” and plot it.
Xem = zeros(1,N); Xem(1) = Xzero + dt*lambda*Xzero + mu*dW(1);
for j=2:N Xem(j) = Xem(j-1) + dt*lambda*Xem(j-1) + mu*dW(j); end
plot([0:dt:T],[Xzero,Xem],’b–*’), hold off
xlabel(’t’,’FontSize’,12)
ylabel(’x’,’FontSize’,16,’Rotation’,0,’HorizontalAlignment’,’right’)
emerr = abs(Xem(end)-Xtrue(end))
(g.) Solve dx(t)dt = µx− x2 + σxζ(t). and find the Fokker-Planck eq.
First, recall the general SDE forms
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.29)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.30)
To put dx(t)dt = µx− x2 + σxζ(t) into the right form multiply by dt and get
dx(t) = (µx− x2)dt+ σxdW (t). (8.31)
Here a(x, t) = µx− x2 and b(x, t) = σx, and therefore (8.30) becomes
df(x(t), t) =
[
(µx − x2)∂f
∂x+∂f
∂t+
1
2σ2x2 ∂
2f
∂x2
]
dt+ σx∂f
∂xdW (t). (8.32)
Let f(x, t) = ln(x). Substituting ∂f∂x = 1
x∂f∂t = 0 and ∂2f
∂x2 = − 1x2 into (8.32) gives
df =
(
µ− σ2
2− x
)
dt+ σdW. (8.33)
Integration from 0 to t gives
x(t) = x(0)e
((
µ−σ2
2
)
t−∫
t
0x(η)dη+σW (t)
)
. (8.34)
To obtain the “exact” solution let v =∫ t
0x(η)dη. Then (8.34) becomes
v′ev = x(0)e
((
µ−σ2
2
)
t+σW (t))
. (8.35)
Solve this equation and get
e
∫ t
0x(η)dη
= 1 + x(0)
∫ t
0
e
((
µ−σ2
2
)
s+σW (s))
ds. (8.36)
Finally, substitute (8.36) into (8.34) and obtain the exact solution
x(t) =x(0)e
((
µ−σ2
2
)
t+σW (t))
1 + x(0)∫ t
0e
((
µ−σ2
2
)
s+σW (s))
ds. (8.37)
139
Remarks. The program verhulst.m solves this eq. and shows that solutions go extinct at finite t,
possibly because x(t) = 0 at a finte t. It seems that the solution in (8.37) won’t allow x(t) = 0 at a
finite value of t. A proof is needed.
The Fokker - Planck equation. Let f(x, t) = µx− x2 and g(x, t) = σx in
∂
∂tρ(x, t|y, s) =
[
− ∂
∂xf(x, t) +
1
2
∂2
∂x2(g(x, t))2
]
ρ(x, t|y, s) (8.38)
and get the Fokker-Planck pde
∂
∂tρ(x, t|y, s) =
∂
∂x
(
(x2 − µx)ρ(x, t|y, s))
+σ2
2
∂2
∂x2
(
x2ρ(x, t|y, s))
. (8.39)
Steady state, time independent solutions satisfy
d
dx
(
(x2 − µx)ρ(x))
+σ2
2
d2
dx2
(
x2ρ(x))
= 0. (8.40)
This reduces to the first order ODE
(x2 − µx)ρ(x) +σ2
2
d
dx
(
x2ρ(x))
= 0. (8.41)
This further reduces to2
σ2+
2(σ2 − µ)
σ2x+ρ′
ρ= 0. (8.42)
Because X(t) > 0 ∀t ≥ 0, we assume that ρ(x) = 0 ∀x ∈ (−∞, 0]. An integration of (8.42) gives
ρ(x) = Nx2( µ
σ2 −1)e−2x
σ2 ∀x ≥ 0. (8.43)
The requirement 1 =∫∞−∞ ρ(x)dx =
∫∞0ρ(x)dx shows that
N =
(∫ ∞
0
x2( µ
σ2 −1)e−2xσ2 dx
)−1
(8.44)
The substitution t = 2xσ2 leads to
N =
(
σ2
2
)
2µ
σ2 −1(∫ ∞
0
t2(µ
σ2 −1)e−tdt
)−1
. (8.45)
Finally, observe that∫ ∞
0
t2(µ
σ2 −1)e−tdt <∞ if 0 < σ < 2µ, (8.46)
∫ ∞
0
t2(µ
σ2 −1)e−tdt = ∞ if σ ≥ 2µ. (8.47)
Thus, when σ ≥ 2µ it follows that N = 0, hence ρ(x) = 0 ∀x and the most probable state is X = 0.
This is the basis of Doering’s explanation of why solutions eventually become extinct when σ ≥ 2µ.
(h.) The Brownian bridge SDE dx = k−x1−t + dW (t), x(0) = A
140
Our goals are (i) solve the SDE, and (ii) prove that E(x(t)) = A(1−t)+kt ∀t, and E((x(t)−k)2) → 0 as
t→ 1− The numerical solution is in BrownianBridge.m The websites en.wikipedia.org/wiki/Brownianbridge
and http://math.la.asu.edu/ kawski/MATLAB/other/wiener.m also discuss the brownian bridge. This
problem is from [23], p. 72. First, recall the SDE forms
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.48)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.49)
Here a(x, t) = k−x1−t and b(x, t) = 1. Let f(x, t) = x
1−t . Substitute fx = 11−t , ft = x
(1−t))2 and fxx = 0
into (8.49) and get
df(x(t), t) =
(
k − x
(1 − t)2+
x
(1 − t)2+ 0
)
dt+1
1 − tdW (t). (8.50)
This further reduces to
df(x(t), t) =
(
k
(1 − t)2
)
dt+1
1 − tdW (t). (8.51)
Integrate both sides from 0 to t, substitute f(x, t) = x1−t , and obtain
x
(1 − t)= A+
(
kt
(1 − t)
)
dt+
∫ t
0
1
1 − sdW (s). (8.52)
Multiply both sides by (1 − t) and arrive at the author’s answer
x(t) = A(1 − t) + kt+ (1 − t)
∫ t
0
1
1 − sdW (s), (8.53)
From (8.53) it follows that E(x(t)) = A(1 − t) + kt.
Next, we show that E((x(t) − k)2) → 0 as t→ 1− it follows from (8.53) that
E(
(x(t) − k)2)
= (A+ k)2(1 − t)2 + 2(A+ k)(1 − t)∫ t
01
1−sE(ζ(s))ds
+ (1 − t)2∫ t
0
∫ t
01
1−t′1
1−t′′E(ζ(t′)ζ(t′′))dt′dt′′.(8.54)
Since E(W (s)) = 0 and E(ζ(t′)ζ(t′′)) = δ(|t′ − t′′|), this reduces to
E(
(x(t) − k)2)
= (A+ k)2(1 − t)2 + (1 − t)2∫ t
0
1
(1 − t′)2dt′, (8.55)
hence
E(
(x(t) − k)2)
= (A+ k)2(1 − t)2 + (1 − t)t→ 0 as t→ 1− (8.56)
From (12.1) it follows that if A = k = 0 then the maximum uncertainty of x(t) is at t = 12 .
(i.) Numerical sol. and Fokker-Planck pde for dx(t)dt = α− λx+ σ
√xζ(t).
This equation models price assets and is discussed in [12], pp. 544-546. First, recall the SDE forms
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.57)
141
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.58)
To put dx(t)dt = α− λx + σ
√xζ(t) into the right form multiply by dt and get
dx(t) = (α− λx)dt + σ√xdW (t). (8.59)
Thus, a(x, t) = α− λx and b(x, t) = σ√x, and (8.58) becomes
df(x(t), t) =
[
(α− λx)∂f
∂x+∂f
∂t+
1
2σ2x
∂2f
∂x2
]
dt+ σ√x∂f
∂xdW (t). (8.60)
Let f(x, t) =√x. Substituting ∂f
∂x = 12√
x∂f∂t = 0 and ∂2f
∂x2 = − 14x3/2 into (8.60) gives
df =
(
4α− σ2
8√x
− λ
2
√x
)
dt+ σdW, (8.61)
or equivalently,
df =
(
4α− σ2
8f− λ
2f
)
dt+ σdW, (8.62)
Integrate (8.61) from 0 to t. let f =√x and get the exact solution
√
x(t) =√
x(0) +
∫ t
0
(
4α− σ2
8√
x(s)− λ
2
√
x(s)
)
ds+ σW (t). (8.63)
Numerical solutions: chain.m and chain1.m solve (8.59) and (8.62) and compare the solutions.
The Fokker-Planck equation. To derive the the Fokker-Planck pde we recall the basic forms
dx(t)
dt= f(x(t), t) + g(x(t), t)ζ(t), (8.64)
where ζ(t) is white noise. The associated Fokker-Planck pde for the conditional probability density is
∂
∂tρ(x, t|y, s) =
[
− ∂
∂xf(x, t) +
1
2
∂2
∂x2(g(x, t))2
]
ρ(x, t|y, s). (8.65)
For our equation f = α− λx and g = σ√x. Substituting these functions into (8.65), gives
∂
∂tρ(x, t|y, s) =
[
− ∂
∂x(α− λx) +
1
2
∂2
∂x2σ2x
]
ρ(x, t|y, s). (8.66)
We look for a steady state solution of (8.66) by assuming that ∂∂tρ(x, t|y, s). This leads to the ODE
[
− d
dx(α− λx) +
1
2
d2
dx2σ2x
]
ρ(x) = 0. (8.67)
This further simplifies toσ2
2
d
dx(xρ) − (α − λx)ρ = 0. (8.68)
We assume that ρ(x) = 0 ∀x < 0. When x ≥ 0 separation of variables gives the solution
ρ(x) = Mx2α
σ2 −1e−2λx
σ2 , x ≥ 0, (8.69)
142
where M =(
∫∞0 x
2α
σ2 −1e−2λx
σ2 dx)−1
Note that M = 0 if α ≤ 0 or λ ≤ 0. Thus, is there is no steady
state when α ≤ 0 or λ ≤ 0 and the solution x(t) = 0 becomes the most probable state. There is
bizarre behavior when we let α = 0 or λ ≤ 0. Use the program chain1.m to see this behavior.
(j.) Proof that∫ t
0WNdW (s) = W N+1
N+1 − N2
∫ t
0WN−1(s)ds and E
(
∫ t
0WNdW (s)
)
= 0.
First, recall the SDE forms
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.70)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.71)
The idea is to recognize that W (t) solves dx(t)dt = ζ(t). Multiply by dt and get dx(t) = (0)dt+ dW (t).
Here a(x, t) = 0 and b(x, t) = 1, and therefore (8.71) becomes
df(x(t), t) =
[
(0)∂f
∂x+∂f
∂t+
1
2
∂2f
∂x2
]
dt+∂f
∂xdW (t). (8.72)
Let f(x, t) = xN+1
N+1 . Substituting ∂f∂x = xN , ∂f
∂t = 0 and ∂2f∂x2 = NxN−1 into (8.72) gives
df =N
2xN−1 + xNdW (t). (8.73)
Finally, integrate from 0 to t, set x = W (t), f = W N+1(t)N+1 and get
∫ t
0
WNdW (s) =WN+1(t)
N + 1− N
2
∫ t
0
WN−1(s)ds. (8.74)
For example, if N = 1 this reduces to the basic identity
∫ t
0
WdW (s) =W 2(t)
2− 1
2
∫ t
0
ds =W 2
2− t
2. (8.75)
If N = 2 then (8.74) reduces to
∫ t
0
W 2dW (s) =W 3(t)
3−∫ t
0
W (s)ds. (8.76)
If N = 3 we get∫ t
0
W 3(s)dW (s) =W 4(t)
4− 3
2
∫ t
0
W 2(s)ds. (8.77)
To prove the second identity take E of both sides of (8.74) and get
E
(∫ t
0
WN (s)dW (s)
)
=1
N + 1E(
WN+1(t))
− N
2
∫ t
0
E(
WN−1(s))
ds. (8.78)
This becomes
E
(∫ t
0
WN (s)dW (s)
)
=1
N + 1
∫ ∞
−∞wN+1 1√
2πte−
w2
2t dw − N
2
∫ ∞
−∞wN−1 1√
2πte−
w2
2t dw. (8.79)
Using the Gamma function, evaluate the two terms on the right and see that they add up to zero.
143
(k.) Proof that∫ t
0 g(s)dW (s) = g(t)W (t) −∫ t
0 W (s)dg(s) and E(
∫ t
0 g(s)dW (s))
= 0.
First, recall the SDE
dx(t) = a(x, t)dt + b(x, t)dW (t), (8.80)
df(x(t), t) =
[
a(x, t)∂f
∂x+∂f
∂t+
1
2(b(x, t))2
∂2f
∂x2
]
dt+ b(x, t)∂f
∂xdW (t). (8.81)
As in the last example, the idea is to recognize that W (t) solves dx(t)dt = ζ(t). Multiply by dt and get
dx(t) = (0)dt+ dW (t). Here a(x, t) = 0 and b(x, t) = 1, and therefore (8.81) becomes
df(x(t), t) =
[
(0)∂f
∂x+∂f
∂t+
1
2
∂2f
∂x2
]
dt+∂f
∂xdW (t). (8.82)
Let f(x, t) = g(t)x. Substituting ∂f∂x = g(t), ∂f
∂t = g′(t)x and ∂2f∂x2 = 0 into (8.82 ) gives
df = g′(t)x(t)dt + g(t)dW (t). (8.83)
Finally, integrate from 0 to t, set x = W (t), f = g(t)W (t) and get∫ t
0
g(s)dW (s) = g(t)W (t) −∫ t
0
W (s)dg(s)ds. (8.84)
To prove the second identity take E of both sides of (8.84 ) and get
E
(∫ t
0
g(s)dW (s)
)
= g′(t)E (W (t)) −∫ t
0
E (W (s)) dg(s) = 0, (8.85)
since E(W (t)) = 0 ∀t.
(l.) How to find ρ(V, t|v, 0) numerically from the SDE
The goal is to obtain the pdf for the Fokker-Planck solution ρ(V, t|v, 0), at a given t, from the SDE
dV = f(V )dt+ σdW, V (0) = v. (8.86)
The steps to do this are the following:
Step I. Select a given t, e.g. t=10. Use a for loop to solve the SDE with Euler’s method on [0,10] a
large number of times, e.g. 100000 times, and save the solution value at t=10 in a row vector callled
solutionvalues. Initialize
solutionvalues=[]
and at the end of each run put
solutionvalues=[solutionsvalues V(t=10)];
Step II. Create 1000 bins for ρ(V, t|v, 0) For example, if we want ρ(V, t|v, 0) for V ∈ [−20, 20] then let
∆ = (20 + 20)/1000 = 40/1000. Create a vector
envalues=zeros(1,1000);
For kk=1:1000, sort through the vector solutionvalues and put the no. of solutionvalues that lie in
((kk− 1) ∗∆, kk ∗∆] into envalues(kk). After this procedure is done divide envalues by size(envalues)
and plot envalues vs 1000 V values in[-20,20]. This should give the graph of ρ(V, t|v, 0) vs V.
144
9 A power law associated with dX = µXdt + σXdW, X(0) = x0
This follows the derivation given by Reed [25].
(a.). Transform dX = µXdt+ σXdW,X(0) = x0 into dYdt = µ+ σζ(t), Y (0) = y0.
Consider the SDE
dX = µXdt+ σXdW, X(0) = x0. (9.1)
Let Y = ln(X), apply Ito’s Lemma with f(X, t) = ln(X) and get the SDE
dY = (µ− σ2
2)dt+ σdW (t), Y (0) = y0 = ln(x0). (9.2)
The term (µ− σ2
2 )dt is a ’drift’ term.
(b.) Find ρ(y, t|y0, 0) for dYdt = (µ− σ2
2 )dt+ σζ(t), Y (0) = y0.
Method I. Solve the SDE
dY = (µ− σ2
2)dt+ σdW (t), Y (0) = y0. (9.3)
Integrating both sides of (9.3) from 0 to t gives the solution
Y = (µ− σ2
2)t+ y0 + σW (t). (9.4)
The function Y (t) is a random process. To find the pdf ρ(y, t|y0, 0) = fY (t)(y) we observe that
Prob[Y ≤ y] = Prob[(µ− σ2
2 )t+ y0 + σW (t) ≤ y]
= Prob
[
W (t) ≤ (y−(µ−σ2
2 )t−y0)
σ
]
=∫ (y−(µ−σ2
2 )t−y0)/σ
−∞1√2πt
eη2
2t dη.
(9.5)
Differentiating with respect to y gives
ρ(y, t|y0, 0) = fY (t)(y) =d
dyProb[Y ≤ y] =
1√2πtσ
exp
(
(y − (µ− σ2
2 )t− y0)2
2σ2t
)
. (9.6)
Method II. Find the solution ρ(y, t|y0, 0) of the associated Fokker-Planck equation
∂ρ
∂t= − ∂
∂y
(
(µ− σ2
2)ρ
)
+1
2
∂2ρ
∂y2, (9.7)
such that
ρ(y, 0|y0.0) = δ(y − y0) and
∫ ∞
−∞ρ(y, t|y0, 0)dy = 1. (9.8)
The solution of (9.7)-(9.8) is the function defined in (9.6). More detials later as to how to derive this
solution from the Fokker-Planck equation.
145
(c.) Use moment generating functions to derive the power law.
Given that t = T, where T > 0 is a constant, the moment generating function for Y (T ) is
MY (T )(s) = E(eY (T )s|T ) =
∫ ∞
0
exp(ys)1√
2πtσexp
(
(y − (µ− σ2
2 )T − y0)2
2σ2T
)
dy (9.9)
An evaluation of the right side gives
MY (T )(s) = E(eY (T )s|T ) = exp
(
y0s+ [(µ− σ2
2)s+
σ2
2s2]T
)
. (9.10)
Reed [25] assumes that T > 0 is an exponential random variable with pdf
fT (t) = e−λt, (9.11)
for some λ > 0. This assumption, together with (9.4), leads us to define the random variable
Y = Y (T ) = (µ− σ2
2)T + y0 + σW (T ), −∞ < Y <∞. (9.12)
Associated with Y is the random variable
X = eY , 0 < X <∞. (9.13)
Our goal is to find the pdf ’s fY (y) and fX(x), and to show that fX(x) has the form of a power law.
To do this we begin by finding a formula for the moment generating function
MY (s) = E(
eY s)
. (9.14)
Since Y is a combination of two random variables, T and W (T ), the evaluation of E(
eY s)
involves
a two variable pdf. In this setting Reed shows that
MY (s) = E(
eY s)
= ET
(
E(eY (T )s|T ))
= ET
(
MY (T )(s))
=∫∞0MY (t)(s)fT (t)dt
=∫∞0 exp
(
y0s+ [(µ− σ2
2 )s+ σ2
2 s2]t)
λexp−λtdt.
(9.15)
Aside. I think that the identity
E(
eY s)
= ET
(
E(eY (T )s|T ))
(9.16)
can be derived by the same method that we use to derived a similar integral representation to prove
that z2 = T2 − T1 is an exponential random variable in our analysis of the Poisson process.
146
Next, an integration of the right side of the last line in (9.15) gives
MY (s) =ey0sλ
λ− [(µ− σ2
2 )s+ σ2
2 s2]. (9.17)
An algebraically more amenable form of (9.17) is
MY (s) =−(2/σ2)ey0sλ
s2 + 2σ2 (µ− σ2
2 )s− 2λσ2
. (9.18)
The zeros of the denominator are s1 = α > 0 and s2 = −β < 0, where
α = − 1
σ2
(
µ− σ2
2
)
+1
σ2
√
(
µ− σ2
2
)2
+ 2λσ2 and β =1
σ2
(
µ− σ2
2
)
+1
σ2
√
(
µ− σ2
2
)2
+ 2λσ2
(9.19)
Note that α > 0 and β > 0, and that
αβ =2λ
σ2. (9.20)
Combining (9.18), (9.19) and (9.20), we obtain
MY (s) =ey0sαβ
(α− s)(s+ β). (9.21)
The pdf fY (y, y0) for Y satisfies
MY (s) =
∫ ∞
−∞esyfY (y)dy =
ey0sαβ
(α− s)(s+ β), − β < s < α, (9.22)
and is given by
fY (y, y0) =
αβα+β exp (β(y − y0)) , y < y0,
αβα+β exp (−α(y − y0)) , y ≥ y0.
(9.23)
To verify this claim we substitute (9.24) into (9.22) and obtain
MY (s) = αβα+β
(
∫ y0
−∞ esyeβ(y−y0)dy +∫∞
y0esye−α(y−y0)dy
)
= αβα+β
(
e−βy0∫ y0
−∞ e(β+s)dy + eαy0∫∞
y0e(s−α)dy
)
= αβα+β
(
esy0
β+s + esy0
α−s
)
= ey0sαβ(α−s)(s+β) − β < s < α.
(9.24)
Note that we have used the identities
esy0
β + s=
∫ y0
−∞esyeβ(y−y0)dy and
esy0
α− s=
∫ ∞
y0
esye−α(y−y0)dy. (9.25)
147
These can also be written as
esy0
β + s=
∫ ∞
−∞esyeβ(y−y0)H(y0 − y)dy and
esy0
α− s=
∫ ∞
−∞esye−α(y−y0)H(y − y0)dy, (9.26)
where H is the heaviside function. An integration shows that fY (y) satisfies the requirement
∫ ∞
−∞fY (y, y0)dy = 1. (9.27)
Finally, we need to determine the pdf fX(x). For this we start with
Prob[X ≤ x] = Prob[eY ≤ x]
= Prob[Y ≤ ln(x)]
=∫ ln(x)
−∞ fY (y)dy.
(9.28)
Differentiaion with respect to x gives
fX(x, x0) =d
dxProb[X ≤ x] =
1
xfY (ln(x)). (9.29)
Combining this with (9.24), we find that the pdf for X is the power law
fX(x, x0) =
αβα+β
1x0
(
xx0
)β−1
, 0 < x < x0,
αβα+β
1x0
(
xx0
)−α−1
, x ≥ x0.(9.30)
An integration shows that fX(x, x0) satisfies the requirement
∫ ∞
0
fX(x, x0)dx = 1. (9.31)
A calculation shows that the mean of X is
µX =
∫ ∞
0
xfX(x, x0)dx =
αβ(β+1)(α−1) if α > 1,
0, otherwise.(9.32)
(d.) The generalized power law.
Suppose that T is the Gamma random variable whose pdf is
fT (t) = λeλt (λt)N−1
(N − 1)!, N ≥ 1. (9.33)
Substitution of (9.33) into the last line of (9.15), followed by an integration, gives
MY (s) = ey0s
(
λ
λ− [(µ− σ2
2 )s+ σ2
2 s2]
)N
(9.34)
148
An algebraically more amenable form of (9.34) is
MY (s) = ey0s
(
−(2/σ2)λ
s2 + 2σ2 (µ− σ2
2 )s− 2λσ2
)N
(9.35)
Then
MY (s) = ey0s
(
αβ
(α− s)(s+ β)
)N
, (9.36)
where, as we observed earlier, α > 0 and β > 0 satisfy
α = − 1
σ2
(
µ− σ2
2
)
+1
σ2
√
(
µ− σ2
2
)2
+ 2λσ2 and β =1
σ2
(
µ− σ2
2
)
+1
σ2
√
(
µ− σ2
2
)2
+ 2λσ2
(9.37)
Again we note that α > 0 and β > 0, and that
αβ =2λ
σ2. (9.38)
Our immediate goal is to find the pdf gY (y,N, y0) such that
MY (s) =
∫ ∞
−∞esygY (y)dy = ey0s
(
αβ
(α− s)(s+ β)
)N
(9.39)
For this we make use of the identity
(
αβ
(α− s)(s+ β)
)N
=
(
1
Γ(N)
)2
(αβ)N ∂2(N−1)
∂αN−1∂βN−1
(
1
(α− s)(β + s)
)
. (9.40)
It folllows from (9.22) and (9.24) that
1
(α− s)(β + s)=
∫ y0
−∞
esy
α+ βeβ(y−y0)dy +
∫ ∞
y0
esy
α+ βe−α(y−y0)dy. (9.41)
Combining (9.39), (9.40 ) and (9.41 ), we obtain
∫ ∞
−∞esygY (y,N, y0)dy = ey0s
(
αβ
(α− s)(s+ β)
)N
(9.42)
where gY (y,N, y0), the pdf of Y , is given by
gY (y,N, y0) =
(
1Γ(N)
)2
(αβ)N ∂2(N−1)
∂αN−1∂βN−1
(
1α+β e
β(y−y0))
, y < y0,
(
1Γ(N)
)2
(αβ)N ∂2(N−1)
∂αN−1∂βN−1
(
1α+β e
−α(y−y0))
, y ≥ y0.(9.43)
The correspomding pdf for X is the generalized power law
fX(x,N, x0) =
(
1Γ(N)
)2(αβ)N
x0
∂2(N−1)
∂αN−1∂βN−1
(
1α+β
(
xx0
)β−1)
, 0 < x < x0, ,
(
1Γ(N)
)2(αβ)N
x0
∂2(N−1)
∂αN−1∂βN−1
(
1α+β
(
xx0
)−α−1)
, x ≥ x0, .(9.44)
149
(e.) The generalized power law when N=2
Substitution of N=2 into (9.43) and (9.44) shows that the pdf’s for Y and X are given by
gY (y, 2, y0) =
(αβ)2(
2(α+β)3 − y−y0
(α+β)2
)
eβ(y−y0), y < y0,
(αβ)2(
2(α+β)3 + y−y0
(α+β)2
)
e−α(y−y0), y ≥ y0,(9.45)
and
gX(x, 2, x0) =
(αβ)2
x0
(
2(α+β)3 − ln(x/x0)
(α+β)2
)(
xx0
)β−1
, 0 < x < x0,
(αβ)2
x0
(
2(α+β)3 + ln(x/x0)
(α+β)2
)(
xx0
)−α−1
, x ≥ x0
(9.46)
(f.) The generalized power law when N=3
Substitution of N=3 into (9.43) and (9.44) shows that the pdf’s for Y and X are given by
gY (y, 3, y0) =
(αβ)3(
4!(α+β)5 − 2(3!)
(α+β)4 (y − y0) + 2(y−y0)2
(α+β)3
)
eβ(y−y0), y < y0,
(αβ)3(
4!(α+β)5 + 2(3!)
(α+β)4 (y − y0) + 2(y−y0)2
(α+β)3
)
e−α(y−y0), y ≥ y0,(9.47)
and
gX(x, 3, x0) =
(αβ)3
x0
(
4!(α+β)5 − 2(3!)
(α+β)4 ln(x/x0) + 2(ln(x/x0))2
(α+β)3
)(
xx0
)β−1
, 0 < x < x0,
(αβ)3
x0
(
4!(α+β)5 − 2(3!)
(α+β)4 ln(x/x0) + 2(ln(x/x0))2
(α+β)3
)(
xx0
)−α−1
, x ≥ x0
(9.48)
Remark. It appears that there is at most one relative maximum for each fX(x,N, x0).
(f.) Large time behavior properties of X(t).
Here our goal is to understand the large time behavior of the original random variable X(t). First, let
K2 > K1 > 0 be arbitratily chosen. Then
Prob[K1 < X(t) < K2] = Prob[ln(K1) < ln(X(t)) < ln(K2)]
= Prob[ln(K1) < −(σ2
2 − µ)t+ y0 + σW (t) < ln(K2)]
= Prob[ 1σ
(
ln(K1
x0) + (σ2
2 − µ)t)
< W (t) < 1σ
(
(σ2
2 − µ)t+ ln(K2
x0
)
]
(9.49)
Define
τ1 =1
σ
(
ln(K1
x0) + (
σ2
2− µ)t
)
and τ2 =1
σ
(
ln(K2
x0) + (
σ2
2− µ)t
)
(9.50)
150
Then
Prob[K1 < X(t) < K2] = Prob[τ1 < W (t) < τ2] =
∫ τ2
τ1
1√2πe−x2/2dx (9.51)
Note that
limt→∞
τ1 = limt→∞
τ2 = ∞ if σ2 > 2µ. (9.52)
This and (9.51) imply that
limt→∞
Prob[K1 < X(t) < K2] = limt→∞
∫ τ2
τ1
1√2πe−x2/2dx = 0 if σ2 > 2µ. (9.53)
Suppose that K2 = ∞. Then
limt→∞
Prob[X(t) > K1] = limt→∞
∫ ∞
τ1
1√2πe−x2/2dx = 0 if σ2 > 2µ. (9.54)
This means that, for each K1 > 0,
limt→∞
Prob[X(t) ≤ K1] = limt→∞
(
1 −∫ ∞
τ1
1√2πe−x2/2dx
)
= 1 if σ2 > 2µ. (9.55)
Since K1 > 0 is arbitrary, this gives evidence that X(t) goes to extinction as t → ∞ when σ2 > 2µ.
What seems contradictory is that simultaneously we could have that the mean of X is finite when
σ2 > 2µ. This can happen, at least for σ2 just above 2µ, if λµ > 1.
(f.) A power law associated with the Gompertz equation.
The Gompertz equation consists of the system
dNdt = K1NG,
dGdt = −K2G,
(9.56)
where K1 > 0, K2 > 0. The initial conditions are
N(0) = N0 > 0, G(0) = G0 > 0. (9.57)
The solution of the second equation in (9.56) is
G(t) = G0e−K2t. (9.58)
Substituting this into the first equation of (9.56), we get
dN
dt= K1G0e
−K2tN, N(0) = N0. (9.59)
Solving this equation gives
N(t) = N0exp
(
K1G0
K2
(
1 − e−K2t)
)
. (9.60)
To introduce noise into the Gompertz equation we assume that K2 is noisy, i.e.
K2 = K2 + σζ(t), (9.61)
151
where K2 is the mean value of K2 and ζ(t) denotes white noise. Then system (9.56) becomes
dNdt = K1NG,
dGdt = −K2G+ σGζ(t).
(9.62)
Multiplying each equation by dt puts the system into the standard SDE form
dN = K1NGdt,
dG = −K2Gdt+ σGdW (t),
(9.63)
where dW (t) denotes gausssian noise. For simplicity we drop the bar on K2 and study
dN = K1NGdt, N(0) = N0
dG = −K2Gdt+ σGdW (t), G(0) = G0
(9.64)
Following the steps in (a.)-(e.) above, we let Y = ln(G) and y0 = ln(G0), and let T be a Gamma
random variable of order N. If N=1 then T is an exponential random variable. Thus, Y = Y (T ) and
G = G(T ) are random variables The pdf for Y is
fY (y, y0) =
αβα+β exp (β(y − y0)) , y < y0,
αβα+β exp (−α(y − y0)) , y ≥ y0,
(9.65)
and the pdf for G is the power law
fG(G,G0) =
αβα+β
1G0
(
GG0
)β−1
, 0 < G < G0,
αβα+β
1G0
(
GG0
)−α−1
, G ≥ G0,(9.66)
where
α =1
σ2
(
µ+σ2
2
)
+1
σ2
√
(
µ+σ2
2
)2
+ 2λσ2 and β = − 1
σ2
(
µ+σ2
2
)
+1
σ2
√
(
µ+σ2
2
)2
+ 2λσ2
(9.67)
152
(h.) The Gompertz equation with a delay term.
The Gompertz delay equation consists of the system
dN(t)dt = K1N(t)G(t),
dG(t)dt = −K2G(t− τ),
(9.68)
where K1 > 0, K2 > 0 and τ ∈ (−∞,∞). Our goal here is to analyze solutions of
dG(t)
dt= −K2G(t− τ). (9.69)
The simplest solution of (9.69) has the form
G(t) = eλt. (9.70)
Substituting (9.70) into (9.69) gives the eigenvalue equation
λ = −K2e−λτ . (9.71)
Let λ = α+ iβ. Then (9.71) becomes
α+ iβ = −K2e−(α+iβ)τ . (9.72)
Separating real and imaginary parts, we obtain the algebraic system
f(α, β, τ) = α+K2e−ατcos(βτ) = 0,
g(α, β, τ) = β −K2e−ατsin(βτ) = 0.
(9.73)
It follows from (9.73) that
cos(βτ) = − αK2eατ and sin(βτ) = β
K2eατ . (9.74)
Therefore 1 = sin2(βτ) + cos2(βτ) = α2+β2
K22e2ατ . Solving for τ gives the function
τ = − 1
2αln
(
α2 + β2
K22
)
. (9.75)
We use (9.75) in our investigation of (9.73) when λ is real, imaginary or strictly complex.
I. Real eigenvalues.
Let β = 0 so that λ = α+ iβ = α is real. In this case (9.73) reduces to
α+K2e−ατ = 0. (9.76)
If α ≥ 0 then (9.76) has no solution. Assuming that α < 0, we solve (9.76) for τ, and obtain
τ = − 1
αln
(−αK2
)
, −∞ < α < 0, (9.77)
153
which is equivalent to (9.75). The corresponding branch Λ0 of solutions of (9.73) is defined by
Λ0 =
(α, β, τ)|α < 0, β = 0, τ = − 1
αln
(−αK2
)
(9.78)
Remarks.
(i) The function τ(α) = − 1α ln
(
−αK2
)
has a unique relative maximum value τ∗ = 1eK2
at α = −eK2,
no relative minimum, a unique zero at α = −K2,
limα→−∞
τ(α) = 0 and limα→0−
τ(α) = −∞. (9.79)
(ii) For each τ ∈ (0, 1eK2
) there are exactly two values α1 < −eK2 < α2 < 0 such that (α1, 0, τ) ∈ Λ0
and (α1, 0, τ) ∈ Λ0. Corresponding to α1 and α2 is the two parameter family
G = c1eα1t + c2e
α2t (9.80)
of solutions of the delay equationd
dtG(t) = −K2G (t− τ) . (9.81)
The function G in (9.80) either monotonically approaches zero as t → ∞, or else G has one relative
maximum or minimum at some finite t, and subsequently approaches zero monotonically.
(iii) For each α ∈ (−∞, 0) there is a unique τ(α) = − 1α ln
(
−αK2
)
such that (α, 0, τ(α)) ∈ Λ0. It follows
by direct substitution that for each α ∈ (−∞, 0), the function
G = c1eαt (9.82)
solves the delay equaiond
dtG(t) = −K2G
(
t+1
αln
(−αK2
))
. (9.83)
II. Imaginary eigenvalues. Let α = 0 so that λ = iβ is purely imaginary, and (9.73) reduces to
K2cos(βτ) = 0,
β −K2sin(βτ) = 0.
(9.84)
It follows from (9.84) that, when α = 0, the solutions of (9.73) are
(α, β, τ) =
(
0,±K2,(4N+1)
2K2
)
, N = 0, 1, 2, ..(
0,±K2,− (3N+1)2K2
)
, N = 0, 1, 2, ..(9.85)
For each integer N ≥ 0, the point (α, β, τ) =(
0,±K2,(4N+1)
2K2
)
generates the two parameter family
G(t) = c1sin(K2t) + c2cos(K2t) (9.86)
154
of periodic solutions of the delay equation
d
dtG(t) = −K2G
(
t− (4N + 1)
2K2
)
. (9.87)
Likewise, for each N ≥ 0, the point (α, β, τ) =(
0,±K2,− (3N+1)2K2
)
generates the two parameter family
G(t) = d1sin(K2t) + d2cos(K2t) (9.88)
of periodic solutions of the delay equation
d
dtG(t) = −K2G
(
t+(3N + 1)
2K2
)
. (9.89)
III. Complex eigenvalues. When λ = α + iβ is complex our goal is to show that infintely many
branches solutions of (9.73) exist. Our main result is
Theorem 9.1 For each integer N there is a branch ΓN of solutions (αN (τ), βN (τ)) of (9.73) which
bifurcate from
(α, β) = (0,K2) (9.90)
as τ passes through the critical value
τN =(2N + 1)π
2. (9.91)
The solutions (αN (τ), βN (τ)) exist for all τ ∈(
1eK2
,∞)
when N ≥ 0, and for all τ ∈ (−∞, 0) when
N < 0. Corresponding to each (αN (τ), βN (τ)) is a two family
G = c1eαN (τ)tcos(βN (τ)t) + c2e
αN (τ)tsin(βN (τ)t) (9.92)
of solutions of the delay equationd
dtG(t) = −K2G (t− τ) . (9.93)
Proof. The proof uses the Implicit Function Theorem. It follows from (9.73) that the matrix of partial
derivatives with respect to α and β of f and g is
A =
(
1 −K2τe−ατ cos(βτ) −K2τe
−ατsin(βτ)
K2τe−ατsin(βτ) 1 −K2τe
−ατ cos(βτ)
)
. (9.94)
Note that
det(A) = 1 when (α, β, τ) =
(
0,K2,(2N + 1)π
2K2
)
. (9.95)
Thus, for each integer N the Implicit Function Theorem implies that there is an ǫN > 0 such that if
τ ∈(
(2N+1)π2K2
− ǫN ,(2N+1)π
2K2+ ǫN
)
then there are unique C2 functions αN (τ), βN (τ) which satisfy
αN (τ) +K2e−αN (τ)τcos(βN (τ)τ) = 0,
βN (τ) −K2e−αN (τ)τsin(βN(τ)τ) = 0.
(9.96)
155
Let (τmin(N), τmax(N)) denote the the maximal interval of existence of (αN (τ), βN (τ)). The branch
ΓN of solutions of (9.73) which bifurcate from (α, β) = (0,K2) as τ passes through τN is defined by
ΓN = (αN (τ), βN (τ), τ)|τ ∈ (τmin(N), τmax(N)) . (9.97)
We need to show that
(τmin(N), τmax(N)) =
(
π
eK2,∞)
∀N ≥ 0, (9.98)
and
(τmin(N), τmax(N)) = (−∞, 0) ∀N ≤ −1. (9.99)
For simplicity, we prove (9.98) for the case N = 0. The details for all other N values are the same.
When N = 0 let (α(τ), β(τ)) = (α0(τ), β0(τ)), and (9.96) becomes
α(τ) +K2e−α(τ)τcos(β(τ)τ) = 0,
β(τ) −K2e−α(τ)τsin(β(τ)τ) = 0.
(9.100)
The maximal interval of existence of (α(τ), β(τ)) is denoted by (τmin, τmax) , where
τmin < τ0 =π
2K2< τmax. (9.101)
Differentiating (9.100) with respect to τ gives the ode’s
(1 + ατ)α′ − βτβ′ = β2 − α2, α( π2K2
) = 0,
βτα′ + (1 + ατ)β′ = −2αβ, β( π2K2
) = K2.
(9.102)
An algebraic manipulation transforms (9.102) into
α′ = 1(1+ατ)2+(βτ)2
(
β2 − α2 − αβ2τ − α3τ)
, α( π2K2
) = 0,
β′ = − β(1+ατ)2+(βτ)2
(
2α+ α2τ + β2τ)
, β( π2K2
) = K2.
(9.103)
To analyze the behavior of solutions of (9.103) we will use the auxiliary variables
Q(τ) = β(τ)τ and H(τ) = α(τ)τ + 1. (9.104)
It follows from (9.104) and (9.103) that Q and H satisfy
Q′ = Qτ(H2+Q2) , Q( π
2K2) = π
2 ,
H ′ = α(τ)HH2+Q2 +Q′Q, H( π
2K2) = 1.
(9.105)
Analysis of solutions of (9.103) on the interval π2K2
≤ τ < τmin.
Our goal here is to prove that
τmax = ∞, α(τ) > 0 and β(τ) > 0 ∀τ > π
2K2, and (α(τ), β(τ)) → (0, 0) as τ → ∞. (9.106)
156
We begin by proving that
τmax = ∞. (9.107)
First, note that the right side of both equations in (9.103) are continuously differentiable with respect
to α and β. Thus, if τmin <∞ then standard theory implies that
lim supτ→τ−
min
(|α′| + β′|) = ∞. (9.108)
The first step in showing that (9.108) cannot occur is to make use of Q. It follows from (9.105) that
Q′(τ) > 0 and Q(τ) ≥ π
2∀τ ∈
[
π
2K2, τmax
)
. (9.109)
From this we obtain the upper bound
1
(1 + ατ)2 + (βτ)2≤ 4
π2∀τ ∈
[
π
2K2, τmax
)
. (9.110)
It follows from (9.75) that
τ = − 1
2αln
(
α2(τ) + β2(τ)
K22
)
> 0 ∀τ ∈(
π
2K2, τmax
)
. (9.111)
This implies that
α2 + β2 ≤ K22 ∀τ ∈
[
π
2K2, τmax
)
. (9.112)
It follows from (9.110), (9.112) and the first equation in (9.103) that
|α′| ≤ 4
π2
(
α2 + β2)
(1 + |α|τ) ≤ 4K22
π2(1 +K2τmax) . (9.113)
Similarly, it follows from (9.112) and the second equation in (9.103) that
|β′| ≤ 4|β|π2
(
|α| (2 + |α|τ) + β2τ)
≤ 4K22
π2(2 +K2τmax) . (9.114)
Thus, α′(τ) and β′(τ) are uniformly bounded on[
π2K2
, τmax
)
, hence (9.108) cannot hold. Therefore,
it must be the case that τmax = ∞, as claimed.
Estimates on the behavior of β(τ) and α(τ) when τ ≥ π2K2
.
From (9.103) we find that
α′(π
2K2) =
4K22
π2> 0 and β′(
π
2K2) = −2K2
2
π< 0. (9.115)
Thus, α(τ) > 0 and 0 < β(τ) < K2 for small τ − π2K2
> 0. Uniqueness of the solution β = 0 of the
second equation in (9.103) implies that
β(τ) > 0 ∀τ ∈ [π
2K2,∞). (9.116)
157
To obtain a sharper estimate for β(τ) we make use of the functional Q(τ). For this, we multiply the
first equation in (9.105) by Q and get
QQ′ =Q2
τ(H2 +Q2)≤ 1
τ. (9.117)
Integrating (9.117) from π2K2
to τ gives
K2 ≤ Q(τ)) ≤√
K22 + 2
(
ln(τ) − ln(π
2K2)
)
. (9.118)
Substituting Q(τ) = β(τ)τ into (9.118) gives the following upper and lower bounds on β(τ) :
K2
τ≤ β(τ)) ≤ 1
τ
√
K22 + 2
(
ln(τ) − ln(π
2K2)
)
∀τ ≥ π
2K2. (9.119)
We now determine the behavior of α(τ) when τ ≥ π2K2
. First, we claim that α(τ) > 0 ∀τ > π2K2
. If
not, there is a value τ > π2K2
such that
α(τ) > 0 ∀τ ∈ [π
2K2, τ ) and α(τ ) = 0. (9.120)
This and the second equation in (9.103) imply that
β′(τ) < 0 ∀τ ∈ [π
2K2, τ) and β(τ ) ∈ (0,K2). (9.121)
Substituting (α(τ), β(τ), τ) = (0, β(τ ), τ ) into (9.100) reduces (9.100) to
cos(β(τ )τ ) = 0,
β(τ ) = K2sin(β(τ )τ ).
(9.122)
From (9.122) it follows that β(τ )τ = (2N+1)π2K2
for some N > 0, and
β(τ ) = K2 or β(τ ) = −K2, (9.123)
which contradicts (9.121). Thus, we conclude that
α(τ) > 0 ∀τ ∈ [π
2K2,∞). (9.124)
Next, we use the functional H(τ) to obtain a precise upper bound estimate on α(τ). Combining the
two equations inin (9.105) leads to
H ′ =1
τ− H
τ(H2 +Q2)), H(0) = 1. (9.125)
From this it easily follows that suppose that α′(τ) = 0 at some τ > 0. The first equation in (9.103)
implies that
β2(1 − ατ) − α2(1 + ατ) = 0 when α′ = 0. (9.126)
158
Since α > 0, we conclude from (9.126) that
1 − ατ > 0 when α′ = 0. (9.127)
From this and the first equation in (9.103) it follows that
α′′ = 1(1+ατ)2+(βτ)2
(
2ββ′(1 − ατ) − α3)
< 0 when α′ = 0. (9.128)
Thus, there can be at most one zero of α′ on ( π2K2
,∞).
Analysis of solutions on the interval τmin < τ ≤ π2K2
.
Our goal here is to prove that
τmin =1
eK2and (α(τ), β(τ)) → (−eK2, 0) as τ → 1
eK2
+
. (9.129)
To prove (9.129) we make use of the auxiliary variables Qand H. From (9.109) and the first equation
in (9.100 ) we conclude that
α(τ) = −K2e−α(τ)τcos(Q(τ)) < 0 ∀τ ∈ (τmin,
π
2K2). (9.130)
It follows from (9.100) and (9.109) that
− 1
α(τ)τ=tan(β(τ)τ)
β(τ)τ=tan(Q(τ))
Q(τ)> 1 ∀τ ∈
(
τmin,π
2
)
. (9.131)
From this and (9.130) we conclude that
H(τ) = α(τ)τ + 1 ∈ (0, 1) ∀τ ∈(
τmin,π
2
)
. (9.132)
The estimates in (9.109) and (9.132) imply that
1
H2 +Q2≥ 4
π2 + 4∀τ ∈
(
τmin,π
2
)
. (9.133)
From this and the first equation in (9.105) we obtain
Q′
Q=
1
τ(H2 +Q2)≥ 4
τ(π2 + 4)∀τ ∈
(
τmin,π
2
)
. (9.134)
Integrating (9.134) from τ to π2K2
gives
ln(Q(τ)) ≤ ln
(
Q
(
π
2K2
))
+4
π2 + 4
(
ln(τ) − ln
(
π
2K2
))
∀τ ∈(
τmin,π
2
)
. (9.135)
We use (9.135) to prove that τmin ≥ 0. Suppose that τmin < 0. Since ln(τ) → −∞ as τ → 0+, it
follows from (9.135) that
ln(Q(τ)) → −∞ as τ → 0+, (9.136)
hence Q(0) = 0. However, this contradicts (9.109). Therefore it must be the case that τmin ≥ 0. Next,
we show that τmin > 0. Suppose, on the contrary, that
τmin = 0. (9.137)
159
We need to obtain a contradiction of (9.137). First, it follows from (9.109) and (9.135) that
ln(τ) → −∞ and Q(τ) → 0+ as τ → 0+. (9.138)
Combining (9.138) and (9.131) gives
−α(τ)τ =Q(τ)
tan(Q(τ))→ 1 as τ → 0+. (9.139)
Thus,
α(τ) → −∞ as τ → 0+. (9.140)
However, from (9.140) and (9.75) we obtain
−α(τ)τ =1
2ln
(
α2 + β2
K22
)
→ ∞ as τ → 0+, (9.141)
contradicting (9.139). Therefore, it must be the case that
τmin > 0. (9.142)
Recall from () that
α(τ)τ → −1 and β(τ)τ → 0 as τ → τ+min. (9.143)
From this and (9.100) it follows that
α(τ) → −eK2 and β(τ) → 0 as τ → τ+min. (9.144)
Combining these properties with (9.75) gives
τmin = − 1
2α(τmin)ln
(
α2(τmin) + β2(τmin)
K22
)
=ln(e2)
2eK2=
1
eK2. (9.145)
160
10 Ito’s lemma for systems
(a.) Stament of Ito’s Lemma for systems.
We develop a two variable version following [23], pp. 45-46. Consider the system of SDE’s
dx1 = u1dt+ v11dB1 + v12dB2,
dx2 = u2dt+ v21dB1 + v22dB2.(10.1)
Here Bi denotes the Wiener process. In matrix notation system (22.75) is written as
dx(t) = udt+ vdB(t), (10.2)
where
x =
(
x1(t)
x2(t)
)
, u =
(
u1
u2
)
, v =
(
v11 v1,2
v2,1 v2,2
)
and dB =
(
dB1
dB1
)
. (10.3)
Let g(t, x) = (g1(t, x), g2(t, x)) be a C2 function of (t, x). Then
dg =
( ∂g1
∂t∂g2
∂t
)
dt+
( ∂g1
∂x1dx1 + ∂g1
∂x2dx2
∂g2
∂x1dx1 + ∂g2
∂x2dx2
)
+1
2
(
∑
i,j∂2g1
∂xi∂xjdxidxj
∑
i,j∂2g2
∂xi∂xjdxidxj
)
, (10.4)
where the rules for evaluating dxidxj are dtdt = dtdBk = dBkdt = 0, and dBidBj = δi,jdt.
(b.) Example 1.∫ t
0etAdB(s) = etAB(t) −
∫ t
0AetAB(s)ds
To prove this integration by parts formula we let g(t, x) = etAx, where A is a two by two matrix, and
x =
(
x1(t)
x2(t)
)
satisfies dx = dB =
(
dB1
dB1
)
. Thus, x is an Ito process and can use the Ito Lemma for
systems as follows: a computation of each term in (22.78) shows that
dg = AetAx(t)dt + etAdx(t) (10.5)
Integrate both sides of (10.5) with respect to t and substitute x(t) = B(t), dx(t) = dB(t) and
g = etAB(t) into the result. This gives∫ t
0etAdB(s) = etAB(t) −
∫ t
0AetAB(s)ds as required.
(c.) Example 2. Solve LQ′′ + RQ′ + 1CQ = G+ αζ(t)
We follow the appoach in [23], pp. 62-63 and transform the equation by setting
x =
(
x1(t)
x2(t)
)
=
(
Q
Q′
)
, A =
(
0 1
− 1CL −R
L
)
, H =
(
01LG
)
, K =
(
0αL
)
. (10.6)
Then x(t) satisfies
dx(t) = Ax+Hdt+KdB(t). (10.7)
Thus, (10.7) is an Ito system. To solve (10.7) multiply both sides by e−tA and rewrite it in the form
e−tAdx(t) − e−tAAx = e−tAHdt+ e−tAKdB(t). (10.8)
161
Now define g = e−tAx = e−tA
(
x1
x2
)
. Substitute g into
dg =
( ∂g1
∂t∂g2
∂t
)
dt+
( ∂g1
∂x1dx1 + ∂g1
∂x2dx2
∂g2
∂x1dx1 + ∂g2
∂x2dx2
)
+
(
∑
i,j∂2g1
∂xi∂xjdxidxj
∑
i,j∂2g2
∂xi∂xjdxidxj
)
, (10.9)
and obtain
dg = AetAx(t)dt+ etAdx(t). (10.10)
Substitute (10.10) into the left side of (10.8) and get
dg = e−tAHdt+ e−tAKdB(t). (10.11)
Integrate both sides of (10.11) from 0 to t and obtain
e−tAx(t) = x0 +
∫ t
0
e−sAHds+
∫ t
0
e−sAKdB(s). (10.12)
Finally, multiply both sides of (10.12) by etA, integrate∫ t
0 e−sAKdB(s) by parts using the formula
derived above in part (a.), and obtain
x(t) = etA
[
x0 + e−tAKB(s) +
∫ t
0
e−sA (H +AKB(s)) ds
]
. (10.13)
This differs from the answer in [23], p. 63, whre a minus sign replaces the plus sign in the integral
term on the right side of (10.13).
(d.) Find the SDE satisfied by Y = (cos(x), sin(x)) where dx(t) = dB(t).
This follows [23], p. 63. Substitute Y = (Y1, Y2) = g(t, x) = (g1, g2) = (cos(x), sin(x)) into
dg =
( ∂g1
∂t∂g2
∂t
)
dt+
( ∂g1
∂x1dx1 + ∂g1
∂x2dx2
∂g2
∂x1dx1 + ∂g2
∂x2dx2
)
+1
2
(
∑
i,j∂2g1
∂xi∂xjdxidxj
∑
i,j∂2g2
∂xi∂xjdxidxj
)
, (10.14)
and get(
dY1
dY2
)
=
(−sin(x)dx
cos(x)dx
)
+1
2
(−cos(x)−sin(x)
)
(dx)2 (10.15)
Substitute Y1 = cos(x), Y2 = sin(x), dx = dB and (dx)2 = dt, and get the Ito form
(
dY1
dY2
)
= −1
2
(
Y1
Y2
)
dt+
(−Y2
Y1
)
dB (10.16)
Dividing by dt gives the SDE form
( dY1
dt
dY2
dt
)
= −1
2
(
Y1
Y2
)
+
(−Y2
Y1
)
ζ(t) (10.17)
162
11 Exit time problems following Oksendal
Goal The goal of this first section on exit time problems is to understand techniques to figure out
the average time it takes for an event to occur. We follow Oksendal [23] and uses his notation.
(a.) Use of Ito’s Lemma to derive the generator A of X(t).
Assume that Xt (i.e Xt = X(t) in the notation given above) is a time homogeneous N-dimensional
Ito diffusion. This means ([23], Ch. VII, p. 104) that Xt ∈ RN satisfies a stochastic ode initial value
problem of the form
dXt = b(Xt)dt+ σ(Xt)dW, t ≥ s, Xs = x, (11.1)
where x ∈ RN , b(x) ∈ RN and σ(x) ∈ RN depend only on x, and satisfy
|b(x) − b(y)| + |σ(x) − σ(y)| ≤ D|x− y|, for some D > 0. (11.2)
Note: when N = 1 the associated probability transition density function ρ(x′, t|x, s) satisfies
ρ(x′, t|x, s)dx′ = PrXt ∈ (x′, x′ + dx′)|X(s) = x and ρ(x′, 0|x, 0) = δ(x′ − x). (11.3)
Inequality (11.2) guarantees uniqueness of the N-dimensional solution of (11.1). Oksendal lets this
unique solution be denoted by Xt = Xx,st When s = 0 he sets Xx
t = X0,xt . In particular, when s = 0
we have Xt = Xxt , and the generator ([23], p. 111) A of Xt is defined by
Af(x) = limt→0+
Ex[f(Xt)] − f(x)
t, x ∈ R and Xt ∈ RN , (11.4)
where f ∈ C20 (RN ). If N=1 we have
Ex[f(Xt)] = E[f(Xxt )] =
∫ ∞
−∞f(x′)ρ(x′, t|x, 0)dx′. (11.5)
The set of functions for which the limit in (11.4) exists for a specific x is denoted by DA(x), and DA
denotes the set of functions for which the limit exists for all x. Our goal is to prove that ([?], p. 113)
Af(x) =
N∑
i=1
bi(x)∂f
∂xi+
1
2
N∑
i=1
(σσT )i,j∂2f
∂xi∂xj, where f ∈ C2
0
(
RN)
. (11.6)
Remarks (i) When σσT is a diagonal matrix with the same entries σ2 then
Af(x) =
N∑
i=1
bi(x)∂f
∂xi+
1
2
N∑
i=1
σ2 ∂2f
∂x2i
. (11.7)
(ii) The big trick in proving (11.6) is to develop an integral representation for Af(x) inn which the
delta function is of use when t→ 0. Below this happens in (11.13)-(11.14)-(11.15).
Eq. (11.7) is the setting for the example in the next subsection When N = 1 Eq. (11.6) reduces to
Af(x) = b(x)fx(x) +σ2(x)
2fxx(x), where f ∈ C2
0
(
RN)
. (11.8)
163
We assume that N = 1 and derive (11.8 ). The general case is similar. Ito’s Lemma implies that
df(Xt) =
(
b(Xt)fx(Xt) +σ2(Xt)
2fxx(Xt)
)
dt+ σ(Xt, t)fx(Xt)dW (t). (11.9)
Integrate both sides of (11.9) from 0 to t and get
f(Xt) = f(x) +
∫ t
0
(
b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη)
)
dη +
∫ t
0
σ(Xη)fx(Xη)dW (η). (11.10)
Substitue (11.10) into the right side of (11.4) and get
Af(x) = limt→0+
Ex[f(Xt)] − f(x)
t= lim
t→0+
1
t
∫ t
0
Ex
[
b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη)
]
dη (11.11)
Now use L’hopital’s rule and obtain
Af(x) = limt→0+
Ex
[
b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη)
]
. (11.12)
Thus,
Af(x) = limt→0+
∫ ∞
−∞
(
b(x′)fx(x′) +σ2(x′)
2fxx(x′)
)
ρ(x′, t|x, 0)dx′. (11.13)
Recall from (11.3) that ρ(x′, 0|x, 0) = δ(x′ − x). Then (11.13) becomes
Af(x) =
∫ ∞
−∞
(
b(x′)fx(x′) +σ2(x′)
2fxx(x′)
)
δ(x′ − x)dx′, (11.14)
and this reduces to the formula (11.8), namely
Af(x) = b(x)fx(x) +σ2(x)
2fxx(x). (11.15)
Remark. Below he makes use of the generalization of (11.15) to
Af(Xη) = b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη). (11.16)
(b.) Dynkin’s formula.
Let f ∈ C20 (RN ). Assume that X(t) satisfies SDE (11.1) with s = 0, that τ is the first time that X(t)
leaves a set, and that Ex(τ) <∞. Then Dynkin’s formula is
Ex[f(Xτ )] = f(x) + Ex
(∫ τ
0
Af(Xη)dη
)
, (11.17)
where
Af(Xη) =
N∑
i=1
bi(Xη)∂f
∂xi+
1
2
N∑
i=1
(σσT )i,j∂2f
∂xi∂xj, (11.18)
Priniciples guiding the use of Dynkin’s formula. In every situation we need f to have compact
support. In the two examples that follow this section the function f is chosen so that (i) f has
compact support, and (ii) Af(x) ≡ constant. In the 1st problem we compute an average exit time,
164
and for this the f is chosen so that the resulting constant is non-zero. In the 2nd problem where we
compute Probthe exit time τ <∞, f is chosen so that Af ≡ 0.
To derive (11.17) when N = 1 start with the formula
f(Xτ ) = f(x) +
∫ τ
0
(
b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη)
)
dη +
∫ t
0
σ(Xη)fx(Xη)dW (η). (11.19)
Note that
Ex
(∫ t
0
σ(Xη)fx(Xη)dW (η)
)
=
∫ t
0
Ex[σ(Xη)fx(Xη)dW (η)]dη = 0. (11.20)
Now take Ex of both sides of (11.19), use (11.20) and obtain (11.17), as required.
(c.) The exit time from a ball in RN for Brownian motion (exit1.m).
This is from [26], p. 115. Let Bt be N-dimensional Brownian motion such that
B0 = a = (a1, a2, ..., aN ) ∈ RN . (11.21)
Assume that there is a first time τK where BτK hits the boundary of the ball
K = KR = x ∈ RN | |x| < R. (11.22)
To find Ea(τK) follow the following three step principle:
I. Define the related, monotone increasing sequence
σk = mink, τK. (11.23)
II. Use Dynkin’s formula, with Af ≡ constant 6= 0, to develop a corresponding sequence of related
equations, and then obtain an upper bound for Ea(σk).
III. Use the monotonicity of σk to obtain Ea(τK).
To accomplish all of this apply Dynkin’s formula (11.17) with f chosen such that Af ≡ constant 6= 0.
For this we let
X = B, x = a, τ = σk, f ∈ C20
(
RN)
such that f(x) = |x|2 =
N∑
i=1
x2i if |x| ≤ R. (11.24)
Define ∆f(x) =∑N
i=1∂2f∂x2
i
= 2N when f(x) = |x|2. Dynkin’s formula (11.17) for our f gives
Ea[f(Bσk)] = |a|2 + Ea
(∫ σk
0
1
2∆f(Bη)dη
)
. (11.25)
Equation (11.25) is the “related” equation. To make use of (11.25) note that f(Bη) = |Bη| since Bt
is in the ball K when 0 ≤ η ≤ σk ≤ τ. Thus,
Ea
(∫ σk
0
1
2∆f(Bη)dη
)
= Ea
(∫ σk
0
Ndη
)
= NEa(σk). (11.26)
165
From this and (11.25) we conclude that
Ea(σk) =Ea[f(Bσk
)] − |a|2N
. (11.27)
Since |Bσk|2 ≤ R2, it follows that
Ea(σk) =Ea(
|Bσk|2)
− |a|2N
≤ Ea(
|R|2)
− |a|2N
=|R|2 − |a|2
N(11.28)
The value of (11.28) is that it implies that the monotone increasing sequence Ea(σk) has a limit as
k → ∞. Also, we hopefully can prove
limk→∞
σk = τK . (11.29)
Note that f(BτK ) = R2. Combine this, (11.27) and (11.29) (i.e replace σk with τK in (11.27)), and
get
Ea(τK) =Ea[f(BτK )] − |a|2
N=Ea(R2) − |a|2
N=
R2 − |a|2N
. (11.30)
Remark. When N = 1 and a = 0 formula (11.30) predicts that E0(τK) = R2. It is interesting to
compare (11.30) with the following integration by parts formula developed in Sect 5, part (j):
∫ t
0
WdW (s) =W 2(t)
2− t
2. (11.31)
Setting t = τK and taking E0 of both sides gives
E0
(∫ τK
0
WdW (s)
)
= E0
(
W 2(τK)
2− τK
2
)
= E0
(
W 2(τK)
2
)
− E0(τK
2
)
. (11.32)
Since the term on the left is zero we conclude that
E0
(
W 2(τK)
2
)
− E0(τK
2
)
= 0. (11.33)
Since W 2(τK) = R2 and E0(
R2
2
)
= R2
2 , this further reduces to
E0 (τK) = R2, (11.34)
as predicted by (11.30). The matlab program exit1.m verifies this formula.
(d.) If |b| > R what is ProbBt ever hits the ball |x| = R ?This is from [26], p. 115. Let Bt be N-dimensional Brownian motion, let R > 0 be fixed, and let αk
denote the exit time from the annulus
Ak = x ∈ RN : R < |x| < 2kR ∀k ≥ 1. (11.35)
There are two cases to consider, N = 2 and N > 2. The method also seems to work when N = 1.
Case (i) N = 2 : When N = 2 we define f ∈ C20 (R2) by
f(x) = ln(|x|) if x ∈ Ak. (11.36)
166
The function f has been chosen so that Af(x) = ∆f(x) ≡ 0 when x ∈ Ak, and the domain of f gets
bigger as k increases. This reduces Dynkin’s formula (11.17) to
Eb (f(Bαk)) = ln(|b|) ∀k ≥ 1. (11.37)
Note that f(Bαk) = ln(R) or f(Bαk
) = ln(2kR). Thus,
Eb (f(Bαk)) = ln(R) OR Eb (f(Bαk
)) = ln(2kR). (11.38)
Let pk be the probability that |Bαk| = R, and let qk be the probability that |Bαk
| = 2kR, where
pk + qk = 1 ∀k ≥ 1.Combining these probabilities with (11.37) with (11.38), we conclude that
ln(R)pk + ln(2kR)qk = ln(|b|) ∀k ≥ 1. (11.39)
Solving for pk and qk gives
pk = 1 +ln(R) − ln(|b|)
kln(2)and qk =
ln(|b|) − ln(R)
kln(2)∀k ≥ 1. (11.40)
Thus (pk, qk) → (1, 0) as k → ∞. Also, when k = ∞ the outer part of the annulus is at R = ∞. From
these observations we conlclude that
ProbBt ever hits the ball |x| = R = 1. (11.41)
Remark. When N = 1 the same proof, with f(x) = x2, seems to give the same result.
Case (ii) N > 2 : When N > 2 we define f ∈ C20 (R2) by
f(x) = |x|2−N if x ∈ Ak. (11.42)
Again, f has been chosen so that Af(x) = ∆f(x) = 0 when x ∈ Ak, and the domain of f gets bigger
as k increases. Dynkin’s formula (11.17) gives
Eb (f(Bαk)) = |b|2−N ∀k ≥ 1. (11.43)
Note that f(Bαk) = R2−N or f(Bαk
) =(
2kR)2−N
. Thus,
Eb (f(Bαk)) = R2−N OR Eb (f(Bαk
)) =(
2kR)2−N
. (11.44)
Let pk be the probability that |Bαk| = R, and let qk be the probability that |Bαk
| = 2kR, where
pk + qk = 1 ∀k ≥ 1.Combining these probabilities with (11.43) with (11.44), we conclude that
R2−Npk +(
2kR)2−N
qk = |b|2−N ∀k ≥ 1. (11.45)
Since qk lies in [0, 1], it follows from (11.45) that
pk →( |b|R
)2−N
< 1 as k → ∞. (11.46)
167
Also, when k = ∞ the outer part of the annulus is at R = ∞. This, and (11.46) imply that
ProbBt ever hits the ball |x| = R =
( |b|R
)2−N
< 1. (11.47)
This result is qualitatively different from the cases N = 1 and N = 2.
(e.) The Backward Kolmogorov Equation - Oksendal version.
Assume that Xt satisfies a stochastic ode initial value problem of the form
dXt = b(Xt)dt+ σ(Xt)dW, t ≥ 0, X0 = x, (11.48)
Let f ∈ C20 (RN ). Then
f(Xτ ) = f(x) +
∫ τ
0
(
b(Xη)fx(Xη) +σ2(Xη)
2fxx(Xη)
)
dη +
∫ t
0
σ(Xη)fx(Xη)dW (η). (11.49)
Define
u(t, x) = Ex[f(Xt)] = E[f(Xt)|Xt = x] (11.50)
Our goal is to prove that u(t, x) satisfies the backward Kolmogorov equation
∂u
∂t= Au, t > 0, u(0, x) = f(x), (11.51)
where A is the generator defined by
Af(x) = limt→0+
Ex[f(Xt)] − f(x)
t, x ∈ R, (11.52)
and satisfies
Af(x) = b(x)fx(x) +σ2(x)
2fxx(x). (11.53)
Remark. The next section shows a different version of the backward Kolmogorov equation, where a
minus sign appears on the left side of (11.51), and the variable t decreases.
We will make use of the key identity
EXr [f(Xt)] = Ex[f(Xt+r)]. (11.54)
Property (11.54) follows from a uniqueness argument. We now follow ([23], p. 124), and let
g(x) = u(x, t) = Ex[f(Xt)]. (11.55)
From (11.52) and (11.55) it follows that
Ag(x) = limr→0+
Ex[g(Xr] − g(x)
r= lim
r→0+
Ex[EXr (f(Xt)] − Ex[f(Xt)])
r. (11.56)
Substitutting (11.54) into (11.56) gives
Ag(x) = limr→0+
Ex[Ex(f(Xt+r)] − Ex[f(Xt)]
r= lim
r→0+
Ex[f(Xt+r)] − Ex[f(Xt)]
r. (11.57)
168
This further reduces to
Au = limr→0+
u(t+ r, x) − u(t, x)
r=∂u
∂t, (11.58)
and the proof is complete. Thus, if f ∈ C20 (RN ) then u(t, x) = Ex[f(Xt)] satisfies
∂
∂tEx[f(Xt)] = a(x)
∂
∂xEx[f(Xt] +
σ2(x)
2
∂2
∂x2Ex[f(Xt], (11.59)
and initial condition
Ex[f(X0)] = f(x). (11.60)
(f.) The Backward Kolmogorov Equation - Wikipedia version.
Assume that Xt satisfies a stochastic ode initial value problem of the form
dXt = b(Xt)dt+ σ(Xt)dW. (11.61)
According to Wikipedia the Bacward Kologorov equation associated with (11.61) is
− ∂
∂tp(x, t) = b(x)
∂
∂xp(x, t) +
1
2
∂2
∂x2σ2(x)p(x, t) (11.62)
where
t ≤ s and p(x, s) = us(x). (11.63)
(g.) The Feynman-Kac formula - Wikepedia version
This is from Wikipedia. Assume that Xt satisfies a stochastic ode initial value problem of the form
dXt = b(Xt)dt+ σ(Xt)dW. (11.64)
The starting point for the Fenman-Kac formula is the associated pde problem
− ∂
∂tf(x, t) = b(x)
∂
∂xf(x, t) +
1
2
∂2
∂x2σ2(x)f(x, t), f(x, T ) = ψ(x). (11.65)
Here, the pde is the same as the backwards Kolmogorov equation in the previous section since the
solution f(x, t) satisfies the ”terminal condition” f(x, T ) = ψ(x) instead of an intitial condition. The
Feynman-Kac formula for the solution of (11.65) is
f(x, t) = E[ψ(XT )|XT = x]. (11.66)
Does this make sense? Yes, when we use the identity
f(x, t) =
∫ ∞
−∞f(x′, T )ρ(x′, T |x, t)dx′ = E[f(x, T )|Xt = x]E[ψ(XT )|Xt = x]. (11.67)
Note that f(x, t) can also be written as
f(x, t) =
∫ ∞
−∞f(x′, t)δ(x′ − x)dx′ =
∫ ∞
−∞f(x′, t)ρ(x′, t|x, t)dx′ = E[f(x, t)|Xt = x] (11.68)
169
(h.) The Feynman-Kac formula - Oksendal version
This follows [23], p. 128. Let Xt satisfies the stochastic ode
dXt = b(Xt)dt+ σ(Xt)dW. (11.69)
Assume that f ∈ C20 (RN ) and q ∈ C(RN ), and define
v(x, t) = Ex[exp
(
−∫ t
0
q(Xs)ds
)
f(Xt)]. (11.70)
The claim in [23] is that v(x, t) is the Feynman-Kac formula for the solution of
∂v
∂t= Av − q(x)v, t > 0, x ∈ RN , (11.71)
v(x, 0) = f(x), x ∈ RN . (11.72)
The first step in proving (11.70) is to note that there are two random variables,
Yt = f(Xt) and Zt = exp(−∫ t
0
q(Xs)ds), (11.73)
such that
v(x, t) = Ex[ZtYt] = Ex[Ztf(Xt)], (11.74)
where Zt satisfies the SDE
dZt = −Ztq(Xt)dt. (11.75)
Now follow the steps of the proof of part (e.) above, beginning with
Av(x, t) = limr→0+
1
r(Ex[v(Xr, t)] − v(x, t)) . (11.76)
Combining (11.70), (11.73) and (11.76), we get
Av(x, t) = limr→0+
1
r
(
Ex[EXr [Ztf(Xt)] − Ex[Ztf(Xt))
. (11.77)
The first big trick here is to note that
Ex[EXr [Ztf(Xt)]] = Ex[Ex[f(Xt+r)exp
(
−∫ t
0
q(Xs+r)ds
)
]] (11.78)
This further reduces to
Ex[EXr [Ztf(Xt)]] = Ex[f(Xt+r)exp
(
−∫ t
0
q(Xs+r
)
ds)]. (11.79)
We need to write the right side of (11.79) in a useful form. For this note that
−∫ t
0
q(Xs+r)ds = −∫ t+r
r
q(Xη)dη =
∫ r
0
q(Xs)ds−∫ t+r
0
q(Xs)ds. (11.80)
From this we conclude that
exp
(
−∫ t
0
q(Xs+r)ds
)
= exp
(∫ r
0
q(Xs)ds
)
exp
(
−∫ t+r
0
q(Xs)ds
)
. (11.81)
170
Thus,(11.79) becomes
Ex[EXr [Ztf(Xt)]] = Ex[exp
(∫ r
0
q(Xs)ds
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)]. (11.82)
Combine (11.82) with (11.77) and the definition of v(x, t) to get
Av(x, t) = limr→0+
1
r(v(x, t + r) − v(x, t))+
1
rEx[
(
exp
(∫ r
0
q(Xs)ds
)
− 1
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)].
(11.83)
Thus,
Av(x, t) =∂v
∂t+ lim
r→0+
1
rEx[
(
exp
(∫ r
0
q(Xs)ds
)
− 1
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)]. (11.84)
To obtain (11.70) it remains to prove that
limr→0+
1
rEx[
(
exp
(∫ r
0
q(Xs)ds
)
− 1
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)] = q(x)v(x, t). (11.85)
Applying L’hopital’s rule shows that the left side of (11.85 ) reduces to
limr→0+
Ex[q(Xr)exp
(∫ r
0
q(Xs)ds
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)] (11.86)
Since X0 = x it makes sense to conclude that
limr→0+
Ex[q(Xr)exp
(∫ r
0
q(Xs)ds
)
f(Xt+r)exp
(
−∫ t+r
0
q(Xs
)
ds)] = q(x)v(x, t) (11.87)
Another way to see if this makes sense is to take the following approach. First, note that
ρ(x1, t+ r, x2, r|x, 0) = ρ(x1, t+ r|x2, r)ρ(x2, r|x, 0). (11.88)
The derivation of (11.88 ) follows from combining the two identities
ρ(x1, t+r, x2, r, x, 0) = ρ(x1, t+r|x2, r, x, 0)ρ(x2, r, x, 0) = ρ(x1, t+r|x2, r)ρ(x2, r|x, 0)ρ(x, 0) (11.89)
and
ρ(x1, t+ r, x2, r, x, 0) = ρ(x1, t+ r, x2, r|x, 0)ρ(x, 0). (11.90)
From (11.88 ) we make the general conclusion
limr→0+
Ex[q(Xr)h(Xt + r)] = limr→0+
∫ ∞
−∞
∫ ∞
−∞h(x1)q(x2)ρ(x1, t+ r|x2, r)ρ(x1, r|x, 0)dx1dx2. (11.91)
Thus,
limr→0+
Ex[q(Xr)h(Xt+r)] =
∫ ∞
−∞
∫ ∞
−∞h(x1)q(x2)ρ(x1, t|x2, 0)δ(x−x1)dx1dx2 =
∫ ∞
−∞h(x1q(x)ρ(x1, t|x, 0)dx1.
(11.92)
Finally, we get
limr→0+
Ex[q(Xr)h(Xt + r)] =
∫ ∞
−∞h(x1)q(x)ρ(x1, t|x, 0)dx1 = q(x)Ex[h(Xt)]. (11.93)
Note that (11.93 ) gives a result like (11.87 ) as desired.
171
12 First passage time - Gardner version
(a.) Basic ODE for first passage time. Exercises
This follows Gardner ([11], pp.136-139). Assume that Xt satisfies
dXt = A(Xt)dt+ σ(Xt)dW, X0 = x ∈ [a, b], 0 < t < T, (12.1)
where σ(x) > 0, a < b are finite, and T = T (x) denotes the exit time from [a, b]. We restrict our
attention only to the interval [0, T ]. Following Gardner, we assume that the particle is removed if
X(t) reaches x = a or x = b, i.e. there is an ’absorbing’ barrier at x = a and x = b. Thus, if the exit
time is reached then the problem ends. Note that T ≥ 0 is a random variable, and T (a) = T (b) = 0.
However, T (x) > 0 when a < x < b, and in this case we observe that
a < Xt < b ∀t ∈ [0, T ), XT = a or XT = b. (12.2)
Gardner assumes that absorbing boundaries exist at x = a and x = b so that the particle is removed
from the system once it reaches either of these two points. In this case the solution of the corresponding
Fokker-Planck equation satisfies the boundary condition
ρ(x′, t|x, 0) = 0 if x′ = a or x′ = b, ∀t ≥ 0. (12.3)
This means (see the remarks in Gardner [11] following eq. (5.2.67) on p. 129) that the probability of
renetering (a, b) is zero once the boundary is reached.
Goals. Our goals are the following:
(i) Develop the pdf p(x, t) for T.
(ii) Use the pdf to find the ODE satisfied by M(x) = E[T (x)], namely
σ2(x)
2
d2M
dx2+A(x)
dM
dx= −1, M(a) = M(b) = 0. (12.4)
The first step is to link the cdf for T with the cdf for Xt by observing that
ProbT > t = Proba < Xt < b = G(x, t) =
∫ b
a
ρ(x′, t|x, 0)dx′, (12.5)
where ρ(x′, t|x, 0) satisfies the Fokker-Planck equation (i.e the forward Kolmogorov equation)
∂
∂tρ = − ∂
∂x′(A(x′)ρ) +
1
2
∂2
∂x′2(σ2(x′)ρ), ρ(x′, 0|x, 0) = δ(x′ − x), (12.6)
and also the backword Kolmogorov equation (Gardner [11], Eq. (5.2.143), p. 137)
∂
∂tρ = A(x)
∂
∂xρ+
σ2(x)
2
∂2
∂x2ρ, ρ(x′, 0|x, 0) = δ(x′ − x), (12.7)
The function
G(x, t) =
∫ b
a
ρ(x′, t|x, 0)dx′ (12.8)
172
is called the survival function. Below, we will make use of the property
G(x, 0) =
∫ b
a
ρ(x′, 0|x, 0)dx′ =
∫ b
a
δ(x′ − x)dx′ = 1. (12.9)
The cdf for T is defined by
ProbT ≤ t = 1 − ProbT > t = 1 −G(x, t) = 1 −∫ b
a
ρ(x′, t|x, 0)dx′. (12.10)
Let p(x, t) be the pdf for T. It is obtained by differentiating the cdf with respect to t, giving
p(x, t) =∂
∂tProbT ≤ t = −Gt(x, t) = −
∫ b
a
ρt(x′, t|x, 0)dx′. (12.11)
Therefore, the mean value of T is given by
E(T (x)) =
∫ ∞
0
t′p(x, t′)dt′ = −∫ ∞
0
t′Gt(x, t′)dt′ =
∫ ∞
0
G(x, t′)dt′ =
∫ ∞
0
∫ b
a
ρ(x′, t|x, 0)dx′dt′.
(12.12)
To find the ODE satisfied by E(T (x)), we note from (12.8)-(12.9) that
1 =
∫ ∞
0
−Gt(x, t′)dt′ = −
∫ ∞
0
∫ b
a
ρt(x′, t|x, 0)dx′dt′. (12.13)
Substituting the right side of (12.7) into the right side of (12.14) gives
1 = −∫ ∞
0
∫ b
a
[
A(x)∂
∂xρ(x′, t′|x, 0) +
σ2(x)
2
∂2
∂x2ρ(x′, t′|x, 0)
]
dx′dt′. (12.14)
This further reduces to
1 = −A(x)d
dx
∫ ∞
0
∫ b
a
ρ(x′, t′|x, 0)dx′dt′ +σ2(x)
2
d2
dx2
∫ ∞
0
∫ b
a
ρ(x′, t′|x, 0)dx′dt′. (12.15)
Finally, define the function
M(x) = E(T (x)) =
∫ ∞
0
∫ b
a
ρ(x′, t′|x, 0)dx′dt′. (12.16)
It follows from (12.15) that M(x) satisfies the ODE
σ2(x)
2
d2M
dx2+A(x)
dM
dx= −1 (12.17)
If we have absorption at x = a and x = b then M satisfies the boundary conditions
M(a) = M(b) = 0. (12.18)
Another possibility is absorption at x = a and reflection at x = b, i.e.
M(a) = M ′(b) = 0. (12.19)
Exercises. First passage time exercises are in www.math.pitt.edu/∼ troy/stochastic/firstpassageexercises.pdf
173
(b.) Basic ODE for Nth moment.
This follows Gardner ([11], p. 138). The Nth moment of T = T (x) is defined by
MN (x) = E(TN(x)) =
∫ ∞
0
(t′)Np(x, t′)dt′ = −∫ ∞
0
(t′)NGt(x, t′)dt′. (12.20)
Recall that G(x, t) =∫ b
aρ(x′, t|x, 0)dx′. Then
MN (x) = −∫ ∞
0
(t′)NGt(x, t′)dt′ = −
∫ ∞
0
(t′)N
(
∫ b
a
ρt(x′, t|x, 0)dx′
)
dt′. (12.21)
If the N th moment MN (x) exists then it is reasonable to assume that
limt′→∞
(t′)NG(x, t′) = limt′→∞
(t′)NGt(x, t′) = 0. (12.22)
Goal. Our goal is to show that if conditions (12.22) hold then MN (x) satisfies the ODE
σ2(x)
2
d2MN
dx2+A(x)
dMN
dx= −NMN−1(x) ∀N ≥ 0. (12.23)
The first step is to note that
M0(x) = −∫ ∞
0
Gt(x, t′)dt′ = G(x, 0) =
∫ b
a
ρ(x′, 0|x, 0)dx′ =
∫ b
a
δ(x′ − x)dx′ = 1. (12.24)
Next, let k ≥ 1. An integration by parts shows that
Mk(x) = −∫ ∞
0
(t′)kGt(x, t′)dt′ =
∫ ∞
0
k(t′)k−1G(x, t′)dt′. (12.25)
Thus, we have the general identity
∫ ∞
0
(t′)k−1G(x, t′)dt′ =Mk(x)
k. (12.26)
Proceeding with the derivation of (12.23), we let N ≥ 1 and begin with an expression for MN−1(x).
It follows from (12.21 ) and the fact that G(x, t) =∫ b
aρ(x′, t|x, 0)dx′ that
MN−1(x) = −∫ ∞
0
(t′)N−1Gt(x, t′)dt′ = −
∫ ∞
0
(t′)N−1
∫ b
a
ρt(x′, t|x, 0)dx′ (12.27)
Substitute for ρt from (12.7) and get
MN−1(x) = −∫ ∞
0
(t′)N−1
∫ b
a
A(x)∂
∂xρ(x′, t′|x, 0)dx′dt′−
∫ ∞
0
(t′)N−1
∫ b
a
σ2(x)
2
∂2
∂x2ρ(x′, t′|x, 0)dx′dt′.
(12.28)
Therefore,
MN−1(x) = −A(x)d
dx
∫ ∞
0
(t′)N−1
∫ b
a
ρ(x′, t′|x, 0)dx′dt′−σ2(x)
2
d2
dx2
∫ ∞
0
(t′)N−1
∫ b
a
ρ(x′, t′|x, 0)dx′dt′.
(12.29)
174
Substituting∫ b
aρ(x′, t′|x, 0)dx′ = G(x, t′) reduces (12.29) to
MN−1(x) = −A(x)d
dx
∫ ∞
0
(t′)N−1G(x, t′)dt′ − σ2(x)
2
d2
dx2
∫ ∞
0
(t′)N−1G(x, t′)dt′. (12.30)
From (12.26) it follows that∫ ∞
0
(t′)N−1G(x, t′)dt′ =MN (x)
N. (12.31)
Finally, substitute (12.31) into the right side of (12.30) and obtain the desired ODE
NMN−1(x) = −A(x)dMN
dx− σ2(x)
2
d2MN
dx2. (12.32)
Remark. When N = 1 we combine (12.24) with (12.32) and obtain the ODE from the previous
section, namelyσ2(x)
2
d2M1
dx2+A(x)
dM1
dx= −1. (12.33)
(c.) Example I. 1st and 2nd moments for Brownian motion.
Our goal here is to compute the first and 2nd moments for Brownian motion with either absorbing or
reflecting boundary conditions. The ODE initial value problem is
dXt = σdW, X0 = x, (12.34)
Let L > 0 be fixed, and let T = T (x) denote the first time where XT = ±L.Absorbing boundary conditions. If we impose absorbing boundary conditions then the first
moment M1(x) = E(T (x)) satisfies
−1 =σ2
2
d2M1
dx2, M1(−L) = M1(L) = 0. (12.35)
The solution of (12.35) is
M1(x) =L2 − x2
σ2∀x ∈ [−L,L]. (12.36)
Note that
M1(0) =L2
σ2, (12.37)
which agrees with (11.34) when σ = 1. The second moment M2(x) = E(T 2(x)) satisfies
−2M1(x) =σ2
2
d2M2
dx2, M2(L) = M2(−L) = 0. (12.38)
The solution of (12.38) is
M2(x) =1
3σ4
(
x4 + 5L4 − L2x2)
(12.39)
For comparison we see that
M1(0) =L2
σ2and M2(0) =
5L4
6σ4. (12.40)
175
Reflecting and absorbing boundary conditions. If we impose an absorbing boundary condition
at x = −L and a reflecting conditon at x = L then the first moment M1(x) = E(T (x)) satisfies
−1 =σ2
2
d2M1
dx2, M1(−L) = M ′
1(L) = 0. (12.41)
The solution of (12.41) is
M1(x) =1
σ2
(
3L2 − x2 + 2Lx)
∀x ∈ [−L,L]. (12.42)
Note that M1(0) = 3L2
σ2 . This is three times the size of M1(0) in (12.37).
176
(d.) Estimating integrals by the method of steepest descent.
In the next subsection we figure out the first passage time over a barrier. For this we need to know
how to estimate integrals of the form
∫ d
c
eMf(x′)dx′, M >> 1. (12.43)
We folllow the method of steepest descent outlined in Wikipedia. Assume that f ∈ C0(c, d), that f(x)
attains a global maximum (possible negative) at a unique x = x ∈ (c, d), and that
f(x) ≈ f(x) +f ′′(x)
2(x− x)2 when x ≈ x. (12.44)
Note that f ′′(x) < 0. Then
∫ d
c
eMf(x′)dx′ ≈∫ d
c
eM [f(x)+ f′′(x)2 (x′−x)2)]dx′ = eMf(x)
∫ d
c
e−M|f′′(x)|
2 (x′−x)2dx′. (12.45)
Now make the substitution v2 = M|f ′′(x)|2 (x′ − x)2 and obtain
∫ d
c
eMf(x′)dx′ ≈ eMf(x)
√
2
M |f ′′(x)|
∫ ∞
−∞e−v2
dv, M >> 1. (12.46)
This reduces to the final estimate
∫ d
c
eMf(x′)dx′ ≈ eMf(x)
√
2π
M |f ′′(x)| , M >> 1. (12.47)
Stirling’s formula. As an example, we use (12.47) to estimate
Γ(N + 1) =
∫ ∞
0
tNe−tdt, N >> 1 (12.48)
To put (12.48) into the form of (12.43) we let t = Nx. Then (12.48) becomes
Γ(N + 1) = NN+1
∫ ∞
0
xNe−Nxdx = NN+1
∫ ∞
0
eN(ln(x)−x)dx. (12.49)
Let
M = N, f(x) = ln(x) − x, f ′(x) =1
x− 1, f ′′(x) = − 1
x2. (12.50)
Thus, f ′(x) = 0 at x = 1 and (12.47) gives Stirling’s formula
Γ(N + 1) ≈ NN+1e−N
√
2π
N= NNe−N
√2πN, N >> 1. (12.51)
177
(e.) Escape over a double-well potential barrier.
This follows Gardner ([11], pp. 139-142). The SDE initial value problem is
dXt = −U ′(Xt)dt+√
2DdW, X0 = x. (12.52)
Assumptions: The “potential” function U(X) has exactly two relative minima, at X = a and X = c,
where a < c, a unique relative maximum at X = b ∈ (a, c), and U(X) → ∞ as |X | → ∞.
From these assumptions it follows that the deterministic equation
dXt = −U ′(Xt)dt (12.53)
has two stable constant soulutions X = a and X = c, which are separated by the stable solution
X = b. Thus, X = b acts as a “barrier” for solutions of (12.52).
Examples. One example is
dXt = (X −X3)dt+√
2DdW, X0 = x. (12.54)
Here
U ′(X) = X3 −X and U(X) =1
4
(
X2 − 1)2. (12.55)
Thus, a = −1, b = 0, c = 1 and
U(−1) = U(1) = 0 and U(0) =1
4. (12.56)
A second example is
dXt = −((X + 1)(X +1
2)(X − 1))dt+
√2DdW, X0 = x. (12.57)
Here
U ′(X) = (X + 1)(X +1
2)(X − 1) and U(X) =
X4
4+X3
6− X2
2− X
2. (12.58)
Thus, a = −1, b = − 12 , c = 1 and
U(−1) =1
12, U(1) = −13
12and U(0) =
23
8(24). (12.59)
For this example U(X) is similar to the function in Gardner ([11], Fig. 5.3, p. 140).
Goals. Our goals are the following:
(i) Let X0 = x ∈ (−∞, x0) in the stochastic equation (12.52), where x0 ∈ (b, c), and derive a formula
for the mean exit time for the solution to reach the barrier X = x0 when D is small.
(ii) Obtain an asymptotic estimate, as D → 0+, for escape time when x = a, the value where the
deterministic equation (12.53) has a stable solution.
178
Remarks. The probability density function ρ(x′, t|x, 0) associated with the solution of (12.52) satisfies
the Fokker-Planck equation
∂
∂tρ =
∂
∂x′(U ′(x′)ρ) +D
∂2ρ
∂x′2, ρ(x′, 0|x, 0) = δ(x′ − x). (12.60)
A stationary solution of (12.60) satisfies
0 =∂
∂x′(U ′(x′)ρ) +D
∂2ρ
∂x′2. (12.61)
The relevant stationary solution of (12.61) is
ρs(x′) = Nexp
(
−U(x′)
D
)
, (12.62)
where N is the constant value such that
N
∫ ∞
−∞exp
(
−U(η)
D
)
dη = 1. (12.63)
The function ρs(x′) has relative maxima at x = a and x = c, and a relative minimum at x′ = b. This
means that there is a relatively high probability of being to the left or right of b (i.e. the barrier) but
not near b.
The mean exit time is the solution of the problem
DM ′′ − U ′(x)M ′ = −1, −∞ < x < b, and M ′(−∞) = M(x0) = 0, (12.64)
with boundary conditions
M(x0) = 0 and limx→−∞
M ′(x)exp(−U(x)/D) = 0. (12.65)
The unique solution of (12.64)-(12.65) is
M(x) =1
D
∫ x0
x
exp
(
U(y)
D
)(∫ y
−∞exp
(
−U(η)
D
)
dη
)
dy. (12.66)
When x = a it follows from (12.66) that the mean exit time
M(a) =1
D
∫ x0
a
exp
(
U(y)
D
)(∫ y
−∞exp
(
−U(η)
D
)
dη
)
dy. (12.67)
Our goal is to show that
M(a) ≈ 2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
, 0 < D << 1. (12.68)
The first step in the derivation of (12.68) is to analyze the integral
∫ y
−∞exp
(−U(η)
D
)
dη, when y ≈ x0 and 0 < D << 1. (12.69)
179
The integrand f(η) = exp(−U(η)/D) has a unique maximum at η = a. Substitute
U(η) = U(a) +U ′′(a)
2D(η − a)2 (12.70)
into (12.69) and get
∫ y
−∞exp
(−U(η)
D
)
dη ≈ exp
(
−U(a)
D
)∫ y
−∞exp
(
−U′′(a)
2D(η − a)2
)
dη (12.71)
The substitution u2 = U ′′(a)2D (η − a)2 leads us to the approximation
∫ y
−∞exp
(−U(η)
D
)
dη ≈√
2D
U ′′(a)exp
(
−U(a)
D
)∫ ∞
−∞exp(−u2)du. (12.72)
Thus,∫ y
−∞exp
(−U(η)
D
)
dη ≈√
2πD
U ′′(a)exp
(
−U(a)
D
)
when 0 < D << 1. (12.73)
The second step in the derivation of (12.68) is to analyze the integral
∫ x0
a
exp
(
U(y)
D
)
dy, where 0 < D << 1. (12.74)
The integrand h(y) = exp(U(y)/D) has a unique maximum at x = b. Substitute
U(y) = U(b) +U ′′(b)
2D(η − b)2 (12.75)
into (12.74) and get
∫ x0
a
exp
(
U(y)
D
)
dy ≈ exp
(
U(b)
D
)∫ y
−∞exp
(
−|U ′′(b)|2D
(η − b)2)
dη. (12.76)
Thus,∫ x0
a
exp
(
U(y)
D
)
dy ≈√
2D
|U ′′(b)|exp(
U(b)
D
)∫ ∞
−∞exp(−u2)du, (12.77)
and therefore
∫ x0
a
exp
(
U(y)
D
)
dy ≈√
2πD
|U ′′(b)|exp(
U(b)
D
)
when 0 < D << 1. (12.78)
Finally, we multiply (12.73) and (12.78), and conclude from (12.67 ) that
M(a) ∼ 2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
, 0 < D << 1. (12.79)
Remarks.
(i) We did not have to restrict X0 to X0 = a in order to get the approximation (12.79). Just let x < b
be arbitrarily chosen so that we have formula (12.66), i.e.
M(x) =1
D
∫ x0
x
exp
(
U(y)
D
)(∫ y
−∞exp
(
−U(η)
D
)
dη
)
dy. (12.80)
180
Since x < b all of the estimates work and we get the asymptotic result
M(x) ≈ 2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
, 0 < D << 1. (12.81)
It is this “uniformity” in the asymptotics of M(x) that is used to prove that T is an exponential
random variable in the next section
(ii) Gardner ([11], p. 140) notes that (12.79) is the Arrhenius formula from chemical reaction theory.
(iii) A lower bound on M(a) it terms of x) is found by differentialing (12.67) twice with respect to
x0. This gives∂
∂x0M(a) =
1
Dexp
(
U(x0)
D
)(∫ x0
−∞exp
(
−U(η)
D
)
dη
)
. (12.82)
and∂2
∂x20
M(a) =1
D+
1
D2exp
(
U(x0)
D
)
U ′(x0)
(∫ x0
−∞exp
(
−U(η)
D
)
dη
)
. (12.83)
Eq. (12.83) reduces to∂2
∂x20
M(a) ≥ 1
D∀x0 ∈ [a, b], (12.84)
since U ′(x0) ≥ 0 when a ≤ x0 ≤ b. Integrating (12.84) gives
∂
∂x0M(a) ≥ x0 − a
Dand M(a) ≥ (x0 − a)2
D∀x0 ∈ [a, b]. (12.85)
In particular
M(a)|x0=b ≥(b− a)2
D. (12.86)
This shows that the mean passage time from X = a to X = b is large when D > 0 is small.
(iii) When x0 > c there is a contribution to mean passage time due to the relative minimum at x = c.
In this case a repitition of the arguments given above lead to
M(a) ≈ 2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
+2π
√
U ′′(c)|U ′′(b)|exp
(
U(b) − U(c)
D
)
, 0 < D << 1.
(12.87)
(f.) Why escape time is asymptotically an exponential random variable.
Our goal is to prove that, for the double-well problem, the escape time T fromX = a toX = x0 ∈ (b, c)
is a continuous random variable which satisfes the asymptotic estimates
Prob (T ≤ t) ∼ 1 − e−λt and fT (t) ∼ λe−λt as D → 0, (12.88)
where
λ =
(
Cexp
(
∆U
D
))−1
=
(
2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
)−1
. (12.89)
Remark. The first two moments of the exit time in Benzi’s paper on stochastic resonance match up
correctly with the moments for an exponential random variable.
181
I. The first step in the derivation of (12.88)-(12.89) is to use mathematical induction to prove that,
for each N ≥ 0, the moment MN (x) = E(TN (x)) satisfies
MN (x) = E(TN(x)) ∼ N !
(
Cexp
(
∆U
D
))N
as D → 0, (12.90)
where
C =2π
√
U ′′(a)|U ′′(b)|and ∆U = U(b) − U(a). (12.91)
Previously we showed that M0(x) = E(1) = 1. Also, at the end of the previous we showed that if
X0 = x ∈ [a, b) then
M1(x) ∼ Cexp
(
∆U
D
)
as D → 0. (12.92)
Induction hypothesis. Let N ≥ 1 and assume that
MN−1(x) = E(TN−1(x)) ∼ (N − 1)!
(
Cexp
(
∆U
D
))N−1
as D → 0. (12.93)
To determine the asymptotic formula for MN(x) = E(TN(x)), we solve the BVP
−NMN−1(x) = −U ′(x)dMN
dx+D
d2MN
dx2, M ′
N (∞) = MN(x0) = 0. (12.94)
Proceeding as in part (g.) above, we integrate twice and get
MN(x) =N
D
∫ x0
x
exp
(
U(y)
D
)(∫ y
−∞MN−1(η)exp
(
−U(η)
D
)
dη
)
dy. (12.95)
Susbstitute the estimate
MN−1(η) ∼ (N − 1)!
(
Cexp
(
∆U
D
))N−1
(12.96)
into the right side of (12.95) and get
MN (x) ∼ (N)!
(
Cexp
(
∆U
D
))N−11
D
∫ x0
x
exp
(
U(y)
D
)(∫ y
−∞exp
(
−U(η)
D
)
dη
)
dy. (12.97)
In the analysis of M1(x) we previously proved that
1
D
∫ x0
x
exp
(
U(y)
D
)(∫ y
−∞exp
(
−U(η)
D
)
dη
)
dy ∼ Cexp
(
∆U
D
)
as D → 0. (12.98)
Finally, combining (12.97) and (12.98) gives the desired estimate, namely
MN (x) ∼ (N)!
(
Cexp
(
∆U
D
))N
as D → 0. (12.99)
II. The second step in the derivation of (12.88)-(12.89) is to use the generating function method to
identify the what type of random variable the exit time T is. Recall (see [26], Ch. 4 - p. 126) that if
X is a random variable then
E(
etX)
= E
( ∞∑
N=0
1
N !XN tN
)
=
∞∑
N=0
1
N !E(XN)tN (12.100)
182
Let X be an expential random variable with parameter λ > 0. Then
E(
etX)
=
∫ ∞
−∞etxλe−λxdx =
∫ ∞
−∞λe(t−λ)xdx, 0 < t < λ. (12.101)
Thus, since 0 < t < λ, we get
E(
etX)
=λ
λ− t=
1
1 − tλ
=
∞∑
N=0
tN
λN(12.102)
It follows from uniqueness of generating functions, and equating the coefficients of tN on the right
sides of (12.100) and (12.102), that if X satisfies
E(
XN)
=N !
λN∀N ≥ 0, (12.103)
then X is an exponential random variable with parameter λ > 0. We have shown that the escape
time random variable T satisfies
E(
TN)
=N !
λN∀N ≥ 0, (12.104)
where
λ =
(
Cexp
(
∆U
D
))−1
=
(
2π√
U ′′(a)|U ′′(b)|exp
(
U(b) − U(a)
D
)
)−1
. (12.105)
Thus T must be an exponential random variable with parameter λ =(
Cexp(
∆UD
))−1.
183
(g.) The probability, and mean time, of exit thru a particular end of [a, b].
This follows Gardner ([11], pp.143-144). Assume that Xt satisfies
dXt = A(Xt)dt+√
B(Xt)dW, X0 = x ∈ (a, b). (12.106)
Let a < b be finite, and let T ≥ 0 be the random variable for the exit time from (a, b). Thus,
a < Xt < b ∀t ∈ [0, T ), XT = a or XT = b. (12.107)
Note that T is a function of x ∈ (a, b), and T (a) = T (b) = 0. Our goals are the following:
I. Determine πa(x) and πb(x), the probabilities that the particle exits (a, b) thru x = a or x = b.
II. Derive the the pdf’s for Ta and Tb, the times to exit (a, b) thru x = a and x = b,
III. Derive and solve the the ODE’s satisfied by E[Ta(x)] and E[Ta(x)].
I. The functions πa(x) and πb(x) are defined by
πa(x) = ProbT > 0 and XT (x) = a = ProbXT (x) = a (12.108)
and
πb(x) = ProbT > 0 and XT (x) = b = ProbXT (x) = b. (12.109)
The first step in finding πa(x) and πb(x) is to observe that
ProbT > t = Proba < Xt < b = G(x, t) =
∫ b
a
ρ(x′, t|x, 0)dx′, (12.110)
where ρ(x′, t|x, 0) satisfies the Fokker-Planck equation (i.e the forward Kolmogorov equation)
∂
∂tρ(x′, t|x, 0) = −
(
∂
∂x′(A(x′)ρ(x′, t|x, 0)) − 1
2
∂2
∂x′2(B(x′)ρ(x′, t|x, 0))
)
, ρ(x′, 0|x, 0) = δ(x′ − x).
(12.111)
and also the backword Kolmogorov equation
∂
∂tρ(x′, t|x, 0) = A(x)
∂
∂xρ(x′, t|x, 0) +
B(x)
2
∂2
∂x2ρ(x′, t|x, 0), ρ(x′, 0|x, 0) = δ(x′ − x). (12.112)
It follows from (12.111) that ρ(x′, t|x, 0)) satisfies
∂
∂tρ(x′, t|x, 0)) = − ∂
∂x′J(x′, t|x, 0), (12.113)
where
J(x′, t|x, 0) = A(x′)ρ(x′, t|x, 0) − 1
2
∂
∂x′(B(x′)ρ(x′, t|x, 0)), (12.114)
At t = 0 this reduces to
J(x′, 0|x, 0) = A(x′)δ(x − x′) − 1
2
∂
∂x′(B(x′)δ(x − x′)) = 0 ∀x ∈ (a, b). (12.115)
In particular,
J(a, 0|x, 0) = J(b, 0|x, 0) = 0 ∀x ∈ (a, b). (12.116)
184
In part III we make use of (12.116). Next, integration of (12.113) from x′ = a to x′ = b gives
∫ b
a
∂
∂tρ(x′, t|x, 0))dx′ = −J(b, t|x, 0) + J(a, t|x, 0). (12.117)
Integration of (12.117) from t′ = t to t′ = ∞ gives
∫ b
a
ρ(x′, t|x, 0)dx′ =
∫ ∞
t
J(b, t′|x, 0)dt′ −∫ ∞
t
J(a, t′|x, 0)dt′. (12.118)
Define
ga(x, t) = −∫ ∞
t
J(a, t′|x, 0)dt′ and gb(x, t) =
∫ ∞
t
J(b, t′|x, 0)dt′. (12.119)
Remark. The interpretation of ga(x, t), ga(x, 0), gb(x, t), gb(x, 0) are that
ga(x, t) = ProbT > t and XT = a, (12.120)
ga(x, 0) = ProbT > 0 and XT = a = ProbXT = a, (12.121)
gb(x, t) = ProbT > t and XT = b, (12.122)
gb(x, 0) = ProbT > 0 and XT = b = ProbXT = b. (12.123)
Thus,
πa(x) = ga(x, 0) = ProbXT = a and πb(x) = gb(x, 0) = ProbXT = b. (12.124)
Next, combine (12.118) and (12.119) and get
∫ b
a
ρ(x′, t|x, 0)dx′ = ga(x, t) + gb(x, t), x ∈ (a, b). (12.125)
Next, integrate the backward pde (12.112) from x′ = a to x′ = b and get
∫ b
a
∂
∂tρ(x′, t|x, 0))dx′ = A(x)
∂
∂x
∫ b
a
ρ(x′, t|x, 0)dx′ +B(x)
2
∂2
∂x2
∫ b
a
ρ(x′, t|x, 0)dx′. (12.126)
Substituting (12.125) into the right side of (12.126) gives
∫ b
a
∂
∂tρ(x′, t|x, 0))dx′ = A(x)
∂
∂x(ga(x, t) + gb(x, t)) +
B(x)
2
∂2
∂x2(ga(x, t) + gb(x, t)). (12.127)
Now substitute the right side of (12.118) into the left side of (12.127) and get
A(x)∂
∂x(ga(x, t) + gb(x, t)) +
B(x)
2
∂2
∂x2(ga(x, t) + gb(x, t)) = J(a, t|x, 0) − J(b, t|x, 0). (12.128)
We break this equation into two distinct equations by setting
A(x)∂
∂xga(x, t) +
B(x)
2
∂2
∂x2ga(x, t) = J(a, t|x, 0), (12.129)
185
and
A(x)∂
∂xgb(x, t) +
B(x)
2
∂2
∂x2gb(x, t) = J(b, t|x, 0). (12.130)
It follows from (12.129), (12.130) and (12.124) that πa(x) = ga(x, 0) and πb(x) = gb(x, 0) satisfy
A(x)∂
∂xπa(x) +
B(x)
2
∂2
∂x2πa(x) = 0, πa(a) = 1, πa(b) = 0. (12.131)
and
A(x)∂
∂xπb(x) +
B(x)
2
∂2
∂x2πb(x) = 0, πa(a) = 0, πa(b) = 1. (12.132)
By solving (12.131) and (12.132) we determine the probabilities that the particle leaves (a, b) thru
x = a or x = b. This completes goal I stated above.
II. To achieve goal II we need to determine the pdf’s for Ta and Tb, the exit tmes thru x = a and
x = b. The first step is to conclude from (12.110) that
ProbT ≤ t = 1 −∫ b
a
ρ(x′, t|x, 0)dx′. (12.133)
Let p(x, t) be the pdf for T. Then
p(x, t) =∂
∂tProbT ≤ t = −
∫ b
a
ρt(x′, t|x, 0)dx′ =
∫ b
a
∂
∂x′J(x′, t|x, 0). (12.134)
Completing the integration gives
p(x, t) = J(b, t|x, 0) − J(a, t|x, 0). (12.135)
It follows from (12.119) that
∂
∂tga(x, t) = J(a, t|x, 0) and
∂
∂tgb(x, t) = J(b, t|x, 0). (12.136)
Before proceeding further we determine the pde’s satisfied by ga(x, t) and gb(x, t). These pde’s are
used in part II to find E(Ta(x)) and E(Tb(x)). Substituting (12.136) into (12.129) and (12.130) gives
A(x)∂
∂xga(x, t) +
B(x)
2
∂2
∂x2ga(x, t) =
∂
∂tga(x, t), (12.137)
and
A(x)∂
∂xgb(x, t) +
B(x)
2
∂2
∂x2gb(x, t) =
∂
∂tgb(x, t). (12.138)
We now continue with the derivation of the pdf’s for Ta and Tb. From (12.110) it follows that
ProbT > t = −∫ ∞
t
[J(a, t′|x, 0) − J(b, t|x, 0)dt′] . (12.139)
It from this equation that
ProbT > t and XT = a = ga(x, t) = −∫ ∞
t
J(a, t′|x, 0)dt′. (12.140)
186
Gardner ([11], p. 142) notes that
ProbT > 0 and XT = a = ProbXT = a = ga(x, 0). (12.141)
Thus,
πa(x) = ProbXT = a = ga(x, 0) = −∫ ∞
0
J(a, t′|x, 0)dt′, (12.142)
Equation (12.140) can be written as
ProbT > t|XT = aProbXT = a = ga(x, t) = −∫ ∞
t
J(a, t′|x, 0)dt′. (12.143)
Finally, define
ProbTa > t = ProbT > t|XT = a. (12.144)
Then (12.140)-(12.144) imply that
ProbTa > t =ga(x, t)
ga(x, 0), (12.145)
and therefore
ProbTa ≤ t = 1 − ga(x, t)
ga(x, 0). (12.146)
Thus the pdf for Ta is
pa(x, t) = − 1
ga(x, 0)
∂
∂tga(x, t). (12.147)
Likewise the pdf for Tb is
pb(x, t) = − 1
gb(x, 0)
∂
∂tgb(x, t). (12.148)
Remark. We have now determine the pdf’s for Ta and Tb which completes goal II.
III. Our third goal is to determine the mean values of Ta and Tb are given by
E(Ta(x)) =
∫ ∞
0
t′pa(x, t′)dt′ = −∫ ∞
0
t′
ga(x, 0)
∂
∂tga(x, t′)dt′ =
1
ga(x, 0)
∫ ∞
0
ga(x,′ )dt′ (12.149)
and
E(Tb(x)) =
∫ ∞
0
t′pb(x, t′)dt′ = −
∫ ∞
0
t′
gb(x, 0)
∂
∂tgb(x, t
′)dt′ =1
gb(x, 0)
∫ ∞
0
gb(x,′ )dt′. (12.150)
Since ga(x, 0) = πa(x) and gb(x, 0) = πb(x), we conclude from (12.149) and (12.150) that
∫ ∞
0
ga(x, t′)dt′ = E(Ta(x))πa(x) and
∫ ∞
0
gb(x, t′)dt′ = E(Tb(x))πa(x). (12.151)
Integration of (12.137) and (12.137) from t = 0 to t = ∞ gives
A(x)∂
∂x
∫ ∞
0
ga(x, t′)dt′ +B(x)
2
∂2
∂x2
∫ ∞
0
ga(x, t′)dt′ = −ga(x, 0) = −πa(x), (12.152)
and
A(x)∂
∂x
∫ ∞
0
gb(x, t′)dt′ +
B(x)
2
∂2
∂x2
∫ ∞
0
gb(x, t′)dt′ = −gb(x, 0) = −πb(x). (12.153)
187
Substitution of (12.151) into (12.152) and (12.153) gives
A(x)∂
∂x(E(Ta(x))πa(x))dt′ +
B(x)
2
∂2
∂x2(E(Ta(x))πa(x))dt′ = −πa(x), (12.154)
and
A(x)∂
∂x(E(Tb(x))πb(x))dt
′ +B(x)
2
∂2
∂x2(E(Tb(x))πb(x))dt
′ = −πb(x). (12.155)
These are the ODE’s to find E(Ta(x))πa(x) and E(Tb(x))πb(x). Thus, once we solve (12.131-(12.132)
and find πa(x) and πb(x), we can then find E(Ta(x)) and E(Tb(x)).
Example. Consider the Wiener process ODE
dXt = σdW, X0 = x ∈ (a, b). (12.156)
Thus A(x) = 0 and B(x) = σ2, and the function πa(x) satisfies
σ2
2
d2
dx2πa(x) = 0, πa(b) = 0, πa(a) = 1. (12.157)
The solution of this problem is
πa(x) =x− b
a− b∀x ∈ [a, b]. (12.158)
Note that πa(a+b2 ) = 1
2 as we would expect. To find E(Ta(x) we need to solve
σ2
2
d2
dx2(E(Ta(x))πa(x)) = −πa(x), E(Ta(a))πa(a) = E(Ta(b))πa(b) = 0. (12.159)
Substituting (12.158) into the right side of (12.159) reduces (12.159) to
σ2
2
d2
dx2(E(Ta(x))πa(x)) = −x− b
a− b, E(Ta(a))πa(a) = E(Ta(b))πa(b) = 0. (12.160)
The solution of (12.160) is
E(Ta(x))πa(x) =(x− b)(a− b)2 − (x− b)3
3σ2(a− b), a ≤ x ≤ b. (12.161)
Finally, we conclude that
E(Ta(x)) =(a− b)2 − (x− b)2
3σ2, a ≤ x ≤ b. (12.162)
Note that
E(Ta(a+ b
2)) =
(a− b)2
4σ2. (12.163)
If a = L and b = −L then we would have
E(TL(0)) =L2
σ2. (12.164)
as we would expect.
(h.) Example: exit thru x = 0 or x = 1 in dU = γU(1 − U)dt+ σ√
U(1 − UdW.
188
We consider the SDE (see [5], p.246) initial value problem
dU = γU(1 − U)dt+ σ√
U(1 − UdW U0 = x ∈ (a, b). (12.165)
Recall from the last section that for an SDE of the form
dXt = A(Xt)dt+√
B(Xt)dW, X0 = x ∈ (a, b), (12.166)
the probabilities πa(x) and πb(x) of exiting thru x = a or x = b, respectively, satisfy
A(x)∂
∂xπa(x) +
B(x)
2
∂2
∂x2πa(x) = 0, πa(a) = 1, πa(b) = 0. (12.167)
and
A(x)∂
∂xπb(x) +
B(x)
2
∂2
∂x2πb(x) = 0, πa(a) = 0, πa(b) = 1. (12.168)
For equation (12.165) we observe that
A(x) = γx(1 − x) and B(x) = σ2x(1 − x), x ∈ (a, b) = (0, 1). (12.169)
Thus, the probability π0(x) pf passing thru x = 0 satisfies
γx(1 − x)∂
∂xπ0(x) +
σ2x(1 − x)
2
∂2
∂x2π0(x) = 0, π0(0) = 1, π0(1) = 0. (12.170)
In agreement with Doering et al [5], we find that the solution of this BVP is
π0(x) =e
2γ
σ2 (1−x) − 1
e2γ
σ2 − 1. (12.171)
Remarks. (i) The probability π1(x) of exit thru x = 1 is π1(x) = 1 − π0(x).
(ii) Thus, a solution must pass thru x = 0 or x = 1 no matter what the value of the ratio 2γσ2 is,
whereas in the extinction problem in part (f.) above this is not the case. That is, solutions of
dx(t)
dt= µx− x2 + σxpζ(t), X0 = x > 0. (12.172)
satisfy Xt = 0 for some finite time t > 0 if and only if 0 < p ≤ 12 .
(i.) Example: finite time extinction in dx(t)dt = µx− x2 + σxpζ(t).
We let p > 0 and consider the SDE
dx(t)
dt= µx− x2 + σxpζ(t), X0 = x > 0. (12.173)
The solution of (12.173) becomes extinct at a finite time T > 0 if XT = 0. Note that T is a random
variable. We develop analytical methods which are useful for the following:
I. Show that the mean exit time (i.e. mean extinction time) is infinite if 1 ≥ p > 12 and σ2 > 2µ.
II. Develop a formula for the mean exit time E(T (x)) when 0 < p ≤ 12 .
189
Part I. There are two cases: (i) p = 1, and (ii) 0 < p < 1.
Case (i.) p=1. Our method is to obtain the expression for mean exit time M(x) = E(T (x)), and
show that M(x) must be infinite when σ2 > 2µ. The function M(x) solves
(
µx− x2)
M ′(x) +σ2
2x2M ′′(x) = −1, M(0) = M ′(∞) = 0. (12.174)
An algebraic manipulation shows that (12.174) is equivalent to
M ′′(x) +2
σ2(µx−1 − 1)M ′(x) = − 2
σ2x−2, M(0) = M ′(∞) = 0. (12.175)
This equation can be written in the form
(
M ′(x)x2µ
σ2 e−2x
σ2
)′= − 2
σ2x
2µ
σ2 −2e−2x
σ2 , M(0) = M ′(∞) = 0. (12.176)
An integration from x to ∞ gives
M ′(x) =2
σ2x−
2µ
σ2 e2xσ2
∫ ∞
x
η2µ
σ2 −2e−2η
σ2 dη. (12.177)
Finally, we integrate (12.177) from 0 to x and get
M(x) =2
σ2
∫ x
0
t−2µ
σ2 e2tσ2
(∫ ∞
t
η2µ
σ2 −2e−2η
σ2 dη
)
dt. (12.178)
We need to show that the right side of (12.178) is infinite. First, note that 0 < t < x, hence
M(x) ≥ 2
σ2
∫ x
0
t−2µ
σ2 e2tσ2
(∫ x
t
η2µ
σ2 −2e−2η
σ2 dη
)
dt ≥ 2K
σ2
∫ x
0
t−2µ
σ2 e2tσ2
(∫ x
t
η2µ
σ2 −2dη
)
dt, (12.179)
where K = e−2x
σ2 . When σ2 > 2µ this reduces to
M(x) ≥ 2K
2µ− σ2
∫ x
0
t−2µ
σ2 e2t
σ2
(
x2µ
σ2 −1 − t2µ
σ2 −1)
dt = ∞ when σ2 > 2µ. (12.180)
When σ2 = 2µ inequality (12.179) reduces to
M(x) ≥ 2
σ2
∫ x
0
t−1e2t
σ2
(∫ x
t
η−1e−2η
σ2 dη
)
dt ≥ 2K
σ2
∫ x
0
t−1e2t
σ2
(∫ x
t
η−1dη
)
dt = ∞. (12.181)
We conclude that M(x) = E (T (x)) = ∞ ∀x > 0, hence finite time extinction does not happen.
Case (i) p 6= 1. The function M(x) solves
(
µx− x2)
M ′(x) +σ2
2x2pM ′′(x) = −1, M(0) = M ′(∞) = 0, (12.182)
or equivalently
M ′′(x) +2
σ2(µx1−2p − x2−2p)M ′(x) = − 2
σ2x−2p, M(0) = M ′(∞) = 0. (12.183)
Define the integrating factor
ψ(x) = exp
(
µ
σ2(1 − p)x2−2p − 2
σ2(3 − 2p)x3−2p
)
. (12.184)
190
Then (12.183) becomes
(ψ(x)M ′(x))′= − 2
σ2x−2pψ(x), M(0) = M ′(∞) = 0. (12.185)
Two integrations of this equation give
M(x) =2
σ2
∫ x
0
1
ψ(t)
(∫ ∞
t
η−2pψ(η)dη
)
dt. (12.186)
It follows from (12.186) that M(x) <∞ when 0 < p ≤ 12 .
(k.) Stochastic Resonance - Benzi paper ideas (benzipaper1.pdf).
This follows the stochastic resonance ideas in [1]. Can all this be extended to give period doubling,or
even more complicated behavior of the Acos(Ωt) is replaced with A1cos(Ω1t) +A2cos(Ω2t)?
Benzi considers the SDE
dXt = −(
X3t − aXt −Acos(Ωt)
)
dt+ ǫdWt X0 = x > 0. (12.187)
where a > 0 is fixed. When A = ǫ = 0 there are two stable fixed points
x1 = −√a and x2 =
√a (stable fixed points). (12.188)
These are separated by the unstable fixed point
x0 = 0 (unstable fixed point). (12.189)
Goal. The goal is to show that if 0 < A << a3/2 there is a range (ǫ1, ǫ2) of positive ǫ values such
that if ǫ1 < ǫ < ǫ2 then the system has a large peak in the power spectrum corresponding to a nearly
periodic behavior of Xt, with period nearly 2π/Ω, and amplitude 2√a. To show how this stochastic
resonance phenomenon occurs, Benzi begins by analyzing the special case
Case (i): A = 0. When A = 0 he lets τ1 and τ2 denote the exit times from the basin of attraction of
x1 = −√a and x2 =
√a, i.e.
τ1 = τ1(x) = inft > 0 : Xt = 0 and X0 = x ∈ (−∞, 0), (12.190)
τ2 = τ2(x) = inft > 0 : Xt = 0 and X0 = x ∈ (0,∞). (12.191)
Associated with these exit times are the mean values of their respective moments, i.e.
M iN = E
(
τNi
)
, i = 1, 2 and M i0 = 1. (12.192)
Below we show that the mean values of τ1 and τ2 satisfy
M11 (x) = M1
1 (−√a) ∼ π
a√
2exp
(
a2
2ǫ2
)
as ǫ→ 0+, (12.193)
191
M21 (x) = M2
1 (√a) ∼ π
a√
2exp
(
a2
2ǫ2
)
as ǫ→ 0+. (12.194)
Note that each of these estimates is uniform with respect to x. By analyzing the exit time ODE
ǫ2
2
d2MN
dx2+ (ax− x3)
dMN
dx= −NMN−1(x) (with appropriate B.C.′s), (12.195)
it is easy to prove that the second moments M i2 satisfy
M12 (x) = M1
2 (−√a) = 2
(
M11 (−
√a))2 ∼ π2
a2exp
(
a2
ǫ2
)
as ǫ→ 0+, (12.196)
M22 (x) = M2
2 (√a) = 2
(
M21 (√a))2 ∼ π2
a2exp
(
a2
ǫ2
)
as ǫ→ 0+. (12.197)
Each of these estimates is uniform with respect to x. This completes the analysis of the case A = 0.
Case (ii): 0 < A << a3/2. Benzi begins the analysis of this case by setting t = 0 and t = π/Ω
in (12.187), and studying the the behavior of solutions of the associated sub-systems
(t = 0) dXt = −(
X3t − aXt −A
)
dt+ ǫdWt, X0 = x, (12.198)
and
(t = π/Ω) dXt = −(
X3t − aXt +A
)
dt+ ǫdWt, X0 = x. (12.199)
Sub-system (12.198). Newton’s method shows that the sub-system
dXt = −(
X3t − aXt −A
)
dt+ ǫdWt, A > 0, (12.200)
has three constant solutions
x′1 ∼ −√a+
A
2a, x′0 ∼ −A
a, x′2 ∼
√a+
A
2aas A→ 0+. (12.201)
The associated potential function V (x) satisfies
V (x) =x4
4− a
2x2 −Ax, V ′(x) = x3 − ax−A. (12.202)
Then
V (x′1) ∼1
4
(
−√a+
A
2a
)4
− a
2
(
−√a+
A
2a
)2
−A
(
−√a+
A
2a
)
as A→ 0+. (12.203)
To first order in A this reduces to
V (x′1) ∼a2
4
(
1 − 2A
a3/2
)
− a2
2
(
1 − A
a3/2
)
+A√a as, (12.204)
hence
V (x′1) ∼ −a2
4
(
1 − 4A
a3/2
)
as A→ 0+. (12.205)
192
Also,
V (x′0) ∼A2
aas A→ 0+. (12.206)
For x′2 we find that
V (x′2) ∼1
4
(√a+
A
2a
)4
− a
2
(√a+
A
2a
)2
−A
(√a+
A
2a
)
as A→ 0+, (12.207)
V (x′2) ∼a2
4
(
1 +2A
a3/2
)
− a2
2
(
1 +A
a3/2
)
−A√a as A→ 0+, (12.208)
hence
V (x′2) ∼ −a2
4
(
1 +4A
a3/2
)
as A→ 0+. (12.209)
Therefore,
V (x′0) − V (x′1) ∼a2
4
(
1 − 4A
a3/2
)
and V (x′0) − V (x′2) ∼a2
4
(
1 +4A
a3/2
)
as A→ 0+. (12.210)
Next, observe that
V ′′(x′1) ∼ 2a, V ′′(x′0) ∼ −a and V ′′(x′2) ∼ +2a as A→ 0+. (12.211)
This implies that
√
|V ′′(x′0)V′′(x′1)| ∼ 2
√a and
√
|V ′′(x′0)V′′(x′2)| ∼ 2
√a as A→ 0+. (12.212)
The escape time from the left well is given by
µ(x′1) ∼2π
√
|V ′′(x0)V ′′(x1)|exp
(
2
ǫ2(V (x′0) − V (x′1))
)
as (ǫ, A) → (0, 0), (12.213)
and the escape time from the right well is given by
µ(x′2) ∼2π
√
|V ′′(x0)V ′′(x2)|exp
(
2
ǫ2(V (x′0) − V (x′2))
)
as (ǫ, A) → (0, 0). (12.214)
Our estimates show that escape time from the left well of
dXt = −(
X3t − aXt −A
)
dt+ ǫdWt, A > 0, (12.215)
is given by
µ(x′1) ∼π
a√
2exp
(
a2
2ǫ2
(
1 − 4A
a3/2
))
as (ǫ, A) → (0, 0), (12.216)
and the escape time from the right well is given by
µ(x′2) ∼π
a√
2exp
(
a2
2ǫ2
(
1 +4A
a3/2
))
as (ǫ, A) → (0, 0), (12.217)
When A = 0 the exit time from the left well, i.e. the basin of attraction of x1 = −√a, is
E(τ1) ∼π
a√
2exp
(
a2
2ǫ2
)
as ǫ→ 0+. (12.218)
193
Sub-system (12.199). Newton’s method shows that the sub-system
dXt = −(
X3t − aXt +A
)
dt+ ǫdWt, A > 0, (12.219)
has three constant solutions
x′′1 ∼ −√a− A
2a, x′′0 ∼ A
a, x′′2 ∼
√a− A
2aas A→ 0+. (12.220)
The associated potential function V (x) satisfies
V (x) =x4
4− a
2x2 +Ax, V ′(x) = x3 − ax+A. (12.221)
Then
V (x′′1 ) ∼ 1
4
(
−√a− A
2a
)4
− a
2
(
−√a− A
2a
)2
+A
(
−√a− A
2a
)
as A→ 0+. (12.222)
To first order in A this reduces to
V (x′′1 ) ∼ a2
4
(
1 +2A
a3/2
)
− a2
2
(
1 +A
a3/2
)
−A√a as, (12.223)
hence
V (x′′1 ) ∼ −a2
4
(
1 +4A
a3/2
)
as A→ 0+. (12.224)
Also,
V (x′′0 ) ∼ +A2
aas A→ 0+. (12.225)
For x′′2 we find that
V (x′′2 ) ∼ 1
4
(√a− A
2a
)4
− a
2
(√a− A
2a
)2
+A
(√a− A
2a
)
as A→ 0+, (12.226)
V (x′′2 ) ∼ a2
4
(
1 − 2A
a3/2
)
− a2
2
(
1 − A
a3/2
)
+A√a as A→ 0+, (12.227)
hence
V (x′′2 ) ∼ −a2
4
(
1 − 4A
a3/2
)
as A→ 0+. (12.228)
Therefore,
V (x′′0 ) − V (x′′1 ) ∼ a2
4
(
1 +4A
a3/2
)
and V (x′′0 ) − V (x′′2 ) ∼ a2
4
(
1 − 4A
a3/2
)
as A→ 0+. (12.229)
Next, observe that
V ′′(x′′1 ) ∼ 2a, V ′′(x′′0 ) ∼ −a and V ′′(x′′2 ) ∼ +2a as A→ 0+. (12.230)
This implies that
√
|V ′′(x′′0 )V ′′(x′′1 )| ∼ 2√a and
√
|V ′′(x′′0 )V ′′(x′′2 )| ∼ 2√a as A→ 0+. (12.231)
194
13 Large deviation theory - Freidlin-Wentzell version
(a.) Maier’s notes on exit time asymptotics
This part follows Maier’s notes [17, 18] (see santafewentzellnotes.pdf amd maier2.pdf). Consider the
system (see [17], pp. 6-7)
dxi = bi(x(t)) +∑
j
σi,j(x(t))√N
dwj(t), where x(0) = x0 = an attractor. (13.1)
Remark. Be sure that the attractor that the solution starts on at t = 0 does not satisfy σ(attractor) =
b(attractor) = 0, or else uniqueness prevents the solution from leaving the attractor even when noise
is present.
Let D = σσT and define (see [18], p.12, and also part (b.) below) the Lagrangian
L(x, x) =1
2[x− b(x)]TD−1(x)[x − b(x)]. (13.2)
For example, consider the systemdx1
dt = b1 + σ1√N
dW1
dt
dx2
dt = b2dt+σ1√N
dW2
dt .
(13.3)
Then D−1 is a square, diagonal matrix with diagonal elements 1σ21
and 1σ22, and
L(x, x) =1
2
(
1
σ21
(x1 − b1)2 +
1
σ21
(x2 − b1)2
)
. (13.4)
The mean value E(T ) of the time to exit a region U containing x(0) = x0 is ([17], pp. 6-7) is given by
E(T ) ∼ exp(NS0) as N → ∞, (13.5)
where
S0 = inf
∫
L(x(t), x(t))dt. (13.6)
The infinum is taken over all trajectories x(t) which begin at x0 and terminate on U. The ”action” S0
is defined in (13.13) in part (b.) below. The computation of S0 is motivated by Hamilton’s principle
which says that “the system undergoes the trajectory between t0 and t1 whose action is stationary.
Remarks. (I) First, one possible interpretaion of the minimization procedure described above is that
the mean time is the exponential of the minimal kinetic energy gernerated by white noise being fed
into the system, i.e.
Kinetic Energy ≡ 1
2
(
W 21 + W 2
2
)
= N1
2
(
1
σ21
(x1 − b1)2 +
1
σ21
(x2 − b1)2
)
. (13.7)
(II) Maier([17], p. 8) notes that in general one expects subdominant large-N asymptotics
E(T ) ∼ CNαexp(NS0) as N → ∞, (13.8)
195
where C and α are constants [19, 21, 22]. In the Section (c.) below we obtain similar formulas for a
one-dimensional problem where we can figure out the numbers exactly.
(b.) Links between the Hamiltonian, Lagrangian and Action integral.
The link between the Lagrangian L(x, x) and and Hamiltonian H(x, p) (i.e. type Lagrangian Mechan-
ics in google) is
H(qj , pj , t) =∑
i
qipi − L(qj , qj , t), (13.9)
where∂H
∂qj= −pj,
∂H
∂pj= qj ,
∂H
∂t= −∂L
∂t. (13.10)
For example, if we consider the equation
d2x
dt2+ x = 0, (13.11)
then p = x, and therefore
H =1
2
(
x2 + x2)
and L(x, x) =1
2
(
x2 − x2)
. (13.12)
The action is defined to be the integral
S(x(t)) =
∫ t1
t0
L(x, x)dη (13.13)
The computation of S0 in (13.8) in the previous section is motivated by Hamilton’s principle which
says that “the system undergoes the trajectory between t0 and t1 whose action is stationary.
(c.) Example: dXt = 2λXt +√
2N b(X)dWt
Our goal is to determine the large time behavior of the solution of the SDE initial value problem
dXt = 2λXt +
√
2
NdWt, X0 = 0. (13.14)
The 2’s in the equation are for only convenience. We assume that b(0) 6= 0 so that the solution will
not stay at X = 0 for all t > 0. For simplicity we let b(X) = 1 and consider the cases λ < 0 and λ = 0
separately.
Case (i) λ < 0.
In this case observe that Xt ≡ 0 is a globally stable solution of the deterministic equation
dXt = 2λXt. (13.15)
Our goal is to determine, when N >> 1, how long it will take the solution of the stochastic prob-
lem (13.14) to overcome the deterministic stability properties of the attractor X = 0, and leave the
interval (−∞, 1) thru X = 1. When N >> 1 the effect of the noise term is small, and therefore we ex-
pect that the exit time should be large. This is the type of problem that falls within Freidlin-Wentzell
196
theory of large deviations [10]. Fortunately, we can apply our one-dimensional method (see Section
10, part (a.)) and obtain an estimate for the exit time thru X = 1. In particular we show below that
the mean time, i.e. the exit time, for a solution to start at X = 0 and leave (−∞, 1) across X = 1 is
given by
Mean Exit Time ∼√
π
4|λ|3(
N−1/2e|λ|N)
, as N → ∞. (13.16)
Remark. This is the type of ”subdominant large-N asymptotic growh estimate” given above in (13.8)
by Maier [17] (see formula (9), p. 8 of santafewentzell.pdf), i.e.
Mean Exit time ∼ CNαeS0N , as N → ∞, (13.17)
where α = 12 , and S0 = 0.
The first step in the derivation of (13.16) is to solve
1
NM ′′(x) + 2λxM ′(x) = −1,−∞ < x < 1, (13.18)
with boundary conditions
M ′(−∞) = M(1) = 0. (13.19)
Write (13.18) as(
M ′(x)eλNx2)′
= −NeλNx2
. (13.20)
Integrating (13.20) from −∞ to x, and using the condition M ′(−∞) = 0, gives
M ′(x)eλNx2
= −N∫ x
−∞eλNt2dt, (13.21)
or equivalently,
M ′(x) = −Ne−λNx2
∫ x
−∞eλNt2dt. (13.22)
Next, integrate (13.22) from x to 1, using the condition M(1) = 0, and obtain
M(x) = N
∫ 1
x
e−λNs2
(∫ s
−∞eλNt2dt
)
ds. (13.23)
Thus, the exit time for the solution to start at X = 0 and leave the interval (−∞, 1) thru X = 1 is
given by
M(0) = N
∫ 1
0
e−λNs2
(∫ s
−∞eλNt2dt
)
ds. (13.24)
We now obtain an aymptotic estimate for the value of M(0) when N >> 1. First, estimate the inner
integral∫ s
−∞ eλNt2dt. The substitution u2 = |λ|Nt2 leads to
∫ s
−∞eλNt2dt =
√
1
|λ|N
∫
√|λ|Ns
−∞e−u2
du ∼√
π
|λ|N , N >> 1. (13.25)
197
Substituting (13.25) into (13.24) leads to
M(0) ∼√
πN
|λ|
∫ 1
0
e−λNs2
ds, N >> 1. (13.26)
Note that −λ = |λ|. The substitution v2 = |λ|Ns2 transforms (13.26) into
M(0) ∼√π
|λ|
∫
√|λ|N
0
ev2
dv, N >> 1. (13.27)
L’Hopital’s rule gives the desired asymptotic estimate
M(0) ∼√
π
4|λ|3(
N−1/2e|λ|N)
as N → ∞. (13.28)
Remarks. The associated action (see (13.8) and (13.13)) along a path from X = 0 to X = 1 is
S(x(t)) =
∫ T
0
L(x, x)dη =1
4
∫ T
0
(
Xt − 2λXt
)2
dη. (13.29)
We conclude from the above analysis that
S0 = inf
∫ T
0
L(x, x)dη = inf1
4
∫ T
0
(
Xt − 2λXt
)2
dη = |λ|. (13.30)
In general, is it possible that S0 = the value of the largest, or maybe the smallest, of the absolute
values of the eigenvalues? Try working it out for two systems that are independent so that the exit
time analysis is doable.
Case (ii) λ = 0.
In this case the deterministic equation becomes
dXt = 0. (13.31)
Note that X = 0 is no longer an attractor for the deterministic equation. Our goal is to determine,
when N >> 1, how long it will take the solution of the related stochastic problem
dXt =
√
2
NdWt, X0 = 0. (13.32)
to leave the interval (−L,L) thru X = ±L. When N >> 1 the effect of the noise term is small, and
therefore we expect that the exit time should be large. Again, this is the type of problem that falls
within Freidlin-Wentzell theory of large deviations [10], and we can apply our one-dimensional method
and obtain an estimate for the exit time thru X = ±L. In particular we show below that the mean
time, i.e. the exit time, for a solution to start at X = 0 and leave (−L,L) is given by
Mean Exit Time ∼ NL2
2(13.33)
198
Remark. This is the type of ”subdominant large-N asymptotic growh estimate” given above in (13.8),
with α = 1 and S0 = 0. Note that there is no attarctor to start from in (13.32). The papers always
seem to want to start the solution of the SDE on an attractor.
The first step in the derivation of (13.33) is to solve
1
NM ′′(x) = −1,−∞ < x < 1, (13.34)
with boundary conditions
M(−L) = M(L) = 0. (13.35)
Write the solution of (13.34) as
M(x) = −N2x2 + c1x+ c2. (13.36)
Conditions (13.35) imply that
M(x) = −N2x2 +N
L2
2. (13.37)
In particular, when x = 0 this gives
Mean Exit Time = M(0) = NL2
2. (13.38)
(d.) Muratov - 2 emails, and paper (Deville.pdf
This follows Deville, Vanden-Eijinden and Muratov [3] (see Deville.pdf). Consider the system
ǫdxdt = x− 1
3x3 − y,
dydt = x+ a+ δ2dW2.
(13.39)
They assume that
a >√
1 + 2ǫ. (13.40)
This causes the constant solution (x, y) = (−a,−a+3 /3) to be a stable node, and therefore solutions
won’t oscillate when they are close to the node. This gives solutions a greater chance to jump away
from the node and enter large amplitude oscillation mode when noise is present. To understand how
long it takes a solution to leave the vicinity of the stable node they rescale by setting
ζ = b−1(x+ a), η = b−2(y + a− a3/3), t = bs. (13.41)
Then (13.39) becomes, to “leading order,”
dζ = b2(−2ζ − η + ζ2)ds
dη = ζds+ δ2b−3/2dWs.
(13.42)
199
The constant solution (x, y) = (−a,−a+3 /3) rescales to
(ζ, η) = (0, 0). (13.43)
There is a unique trajectory of solutions of (13.42) which pass thru (ζ, η) = (2, 0). Let Γ denote the
component of this trajectory which lies in the region η < 0, ζ < 2. Note that if there is no noise then
a solution which lies to the right of Γ must satisfy ζ → ∞ in finite time. When noise is present we
need to estimate the meant time to reach Γ if a solution satisfies the initial condition
(ζ(0), η(0)) = (0, 0). (13.44)
To obtain the mean time estimate they use Freidlin-Wentzell theory of large deviations [10]. Suppose
that δ2b−3/2 satisfies
δ2b−3/2 → 0. (13.45)
Then they claim that τ, the mean value of the time it takes to reach Γ, satisfies
τ ∼ becb3δ−22 → ∞ as b→ 0, (13.46)
where c > 0 is a constant. The factor b indicates that τ is in terms of the original time t. According
to Muratov, ” τ = bexp(Smin/delta22) >> b(time is measured in the original units, hence a factor of
b). This does not seem quite accurate. I think they should have said that the mean time to reach Γ
satisfies
Mean time ∼ ecb3δ−22 → ∞ as δ2b
−3/2 → 0. (13.47)
First, he defines
Smin =1
2minpaths
(
b3∫ T
0
(ηs − ξ)2ds |ξ=b2ǫ−1(−2ξ − η + ξ2)
)
. (13.48)
Note that the integrand is identically zero whenever ηs = ξ (i.e. when η follows a deterministic
trajectory), so one essentially needs to find a minimizer going from the fixed point to the separatrix
of the deterministic system.
Is there a guiding principle to conclude that (ηs − ξ)2 is the integrand? His answer: Yes, it’s from
the Wiener measure of the noise. The easiest way to see how this comes about is by writing the path
integral formulation for the transition probability:
P (x(T ) ∈ A|x(0) = x0) =
∫
Dx(t)x(0)=x0,x(T )∈A)exp(−1/(2δ2)
∫ T
0
(ηs − ξ)2ds), (13.49)
which you can get by discretizing the SDE and taking into account that Delta W’s are independent
Gaussian random variables. Note that the large deviation principle exists also for more general
processes, even jump processes (see our recent JCP paper).
200
14 Continuum limits of Master equations.
(a.) Derivation of Ct = DCxx from C(x, t + τ) = λrC(x− ∆x, t) + λlC(x + ∆x, t).
This follows the derivation on p. 405 of [8]. Assume that particles move randomly with average step
length ∆t over unit time period τ. Let C(x, t)∆x = the number of particles in [x, x + ∆x] at time t.
Let λr =the probability that a particle moves to the right ∆x distance over the time period τ, and
λl =the probability that a particle moves to the left ∆x distance over the time period τ. The random
walk model describing the movement of the particles is
C(x, t+ τ) = λrC(x− ∆x, t) + λlC(x+ ∆x, t). (14.1)
To derive the Fokker-Planck equation Ct = DCxx from (14.1) we use the expansions
C(x, t + τ) = C(x, t) + Ct(x, t)τ, (14.2)
C(x + ∆x, t) = C(x, t) + Cx(x, t)∆x +1
2Cxx(x, t) (∆x)
2, (14.3)
C(x − ∆x, t) = C(x, t) − Cx(x, t)∆x +1
2Cxx(x, t) (∆x)2 . (14.4)
Substitute (14.2)-(14.4) into (14.1), assume that λl = λr = 12 , and obtain
Ct = DCxx where D =(∆x)
2
2τ= constant as τ → 0 and ∆x→ 0. (14.5)
That D is a constant implies that the average distance a particle moves during time τ is ∆x =√
2τD.
The SDE associated with Ct(x, t) = DCxx is
dX
dt=
√2Dζ(t). (14.6)
(b.) Derivation of the Fokker-Planck Eq. for dU = γU(1 − U)dt+ σ√
U(1 − UdW.
This follows [5]. We analyze (see p. 245 in the pdf file doeringphysica.pdf) the master equation
dpn
dt = −k1
Ω n(N − n)pn + k1
Ω (n− 1)(N − n+ 1)pn−1
− k2
Ω n(N − n)pn + k2
Ω (n+ 1)(N − n− 1)pn+1, n = 0, 1, . . . , N,
(14.7)
with boundary conditions given by
p−1 = pN+1 = 0. (14.8)
Define
u =n
Nand f(u, t) = pn(t), n = 0, 1, . . . , N. (14.9)
Our goal is to show that the Fokker-Planck equation associated with (14.7) is
∂f(u, t)
∂t=
∂
∂u
[
− (k1 − k2)N
Ωu(1 − u) +
1
2
(k1 + k2)
Ω
∂
∂uu(1 − u)
]
f(u, t), (14.10)
201
with boundary conditions
f(0, t) = f(1, t) = 0 ∀t ≥ 0. (14.11)
Remark (I). The function f(u, t) is the probability density function for the stochastic ode
dU = γU(1 − U)dt+ σ√
U(1 − UdW, (14.12)
where
γ =(k1 − k2)N
Ωand σ2 =
(k1 + k2)
Ω. (14.13)
The first step in deriving (14.10) is to define ∆u = 1N , substitute (14.9) into (14.7) and get
∂f(u,t)∂t = − k1
Ω(∆u)2u(1 − u)f(u, t) + k1
Ω(∆u)2 (u − ∆u)(1 − u+ ∆u)f(u− ∆u, t)
− k2
Ω(∆u)2 u(1 − u)f(u, t) + k2
Ω(∆u)2 (u+ ∆u)(1 − u− ∆u)f(u+ ∆u, t).
(14.14)
Next, define
g(u) = u(1 − u). (14.15)
Substitution of (14.15) into (14.14) gives
∂f(u,t)∂t = − k1
Ω(∆u)2 g(u)f(u, t) + k1
Ω(∆u)2 g(u− ∆u)f(u− ∆u, t)
− k2
Ω(∆u)2 g(u)f(u, t) + k2
Ω(∆u)2 g(u+ ∆u)f(u+ ∆u, t).
(14.16)
Remark (I). A random walk equation which is analagous to (14.1) in part (a.) above, and which
which approximates (14.16) is
f(u, t+ ∆t) = (1 − g(u))f(u, t) + λRg(u− ∆u)f(u− ∆u, t) + λLg(u+ ∆u)f(u+ ∆u, t), (14.17)
where
λR =k1∆t
Ω(∆u)2, λL =
k2∆t
Ω(∆u)2and
(∆u)2
∆t=k1 + k2
Ω. (14.18)
To see this we substitute (14.18) into (14.17) and obtain the approximation of (14.16) given by
f(u,t+∆t)−f(u,t)∆t = − k1
Ω(∆u)2 g(u)f(u, t) + k1
Ω(∆u)2 g(u− ∆u)f(u− ∆u, t)
− k2
Ω(∆u)2 g(u)f(u, t) + k2
Ω(∆u)2 g(u+ ∆u)f(u+ ∆u, t).
(14.19)
Continuing with the derivation of (14.10), we expand g(u−∆u)f(u−∆u, t) and g(u+∆u)f(u+∆u, t)
in Taylor series centered at (u, t) and keep terms up to order (∆u)2, and obtain
g(u− ∆u)f(u− ∆u, t) = (g(u) − g′(u)∆u+ g′′(u)2 )(∆u)2(f(u, t) − fu(u, t)∆u+ fuu(u,t)
2 (∆u)2
g(u+ ∆u)f(u+ ∆u, t) = (g(u) + g′(u)∆u+ g′′(u)2 )(∆u)2(f(u, t) + fu(u, t)∆u+ fuu(u,t)
2 (∆u)2.
(14.20)
Now substitue (14.20) into (14.16) and only keep terms up to order (∆u)2. This gives
∂f(u, t)
∂t=
∂
∂u
[
− (k1 − k2)
Ω∆ug(u) +
1
2
(k1 + k2)
Ω
∂
∂ug(u)
]
f(u, t). (14.21)
Setting g(u) = u(1 − u) and N = 1∆u transforms (14.21) into (14.10) and the derivation is complete.
202
15 Waiting time (i.e. time of first passage) problems
Goal The goal of this section is to understand techniques to figure out the average time it takes for
an event to occur, i.e. the waiting time. We’ll examine waiting times for problems involving
(i) Random walk ([26], Ch. 5): two difference equations
(ii) Poisson counting processes (e.g. the neuron firing problem in poissonneuron.pdf)
(iii) Example: the A→ B → C protein degradation process (see hnd1stPassage.pdf).
(iv) birth-death processes ([26], Ch. 9).
(v) The SDE Benzi et al stochastic resonance problem (see benzipaper.pdf).
(i.) Random walk. ([26], Ch. 5, p.173). For each N ≥ 1 let the random variable ZN satisfy
P (ZN = −1) = q and P (ZN = 1) = p where p > 0, q > 0 and p+ q = 1, (15.1)
The mean for ZN is
E(ZN ) = (1)P (ZN = 1) + (−1)P (ZN = −1) = (1)p+ (−1)q = p− q. (15.2)
Note that
E(Z2N ) = (1)2P (ZN = 1) + (−1)2P (ZN + −1) = p+ q = 1. (15.3)
Therefore the variance for ZN is
V ar(ZN ) = E(Z2N ) − (E(ZN ))
2= 1 − (p− q)2 = (p+ q)2 − (p− q)2 = 4pq. (15.4)
The random walk XN is defined by XN =∑N
k=1 Zk ∀N ≥ 1. The probability distriution function for
XN is (see [26], Ch. 5, p.176) given by
P (XN = m) =N !
(N − .5(N +m))!(.5(N +m)!p
(N+m)2 q
(N−m)2 ∀m ∈ [−N,N ]. (15.5)
To derive (15.5) we let N+ be the number of +1′s in the first N steps, and let N− denote the number
of −1′s. Then we have the two equations
N+ +N− = N and N+ −N− = XN . (15.6)
These eqations give
XN = m ⇐⇒ N+ =N +m
2. (15.7)
Since the number of +1’s is binomially distributed we conclude that
P (XN = k) = P (N+ =N + k
2) =
N !
(N − .5(N +m))!(.5(N +m)!p
(N+m)2 q
(N−m)2 . (15.8)
The mean and variance for XN are (see [26], p. 177)
E (XN ) =N∑
k=1
E (Zk) = N(p− q) ∀N ≥ 1, (15.9)
203
V ar (XN ) = V ar
(
N∑
k=1
Zk
)
=
N∑
k=1
V ar (Zk) = 4Npq ∀N ≥ 1, (15.10)
From (15.2)-(15.3) it follows([26], p. 178) that the autocorrelation is
E (XxXm) = min(n,m) + (nm−min(n,m))(p− q)2. (15.11)
Example 1. Let X0 = k ∈ 0, .., N. Find p(k) = P (Xm = 0 for some m ≥ 1|X0 = k). Assume that
absorption takes place at k = 0, and that Xi = 0 before Xi = N. Then p(k) satisies
p(k) = p(k + 1)p+ p(k − 1)q, p(0) = 1 and p(N) = 0. (15.12)
Let p(k) = λk. If p 6= 12 then λ1 = 1, λ2 = r = q
p and the solution is
p(k) =rk − rN
1 − rN. (15.13)
If p = q = 12 then λ1 = λ2 = 1 and
p(k) = 1 − k
N. (15.14)
Remark. We have only figured out the probability that absorption takes place, not how many steps
it takes to do so. To We figure out how many steps it takes in the next example.
Example 2: the waiting time problem. Suppose that absorption takes place at k = 0 or k = N
and let Tk denote the minimum number of steps m such that Xm = N or Xm = 0. Then T0 = TN = 0.
For each k ∈ 1, .., N − 1 find the average value of Tk. First, let
p(k,m) = Prob(Xm = N or Xm = 0, Xi 6= 0, Xi 6= N if i ≤ m− 1|X0 = k). (15.15)
Then p(k,m) satisfies the 2nd order difference equation
p(k,m) = p(k + 1,m− 1)p+ p(k − 1,m− 1)q (15.16)
p(k,m) = P (Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m|X0 = k)
= P (X1 = k + 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)P (X1 = k + 1|X0 = k)
+ P (X1 = k − 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)P (X1 = k − 1|X0 = k)
= P (X1 = k + 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)p
+ P (X1 = k − 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)q
= P (Xm−1 = N or Xm−1 = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m− 1|X0 = k + 1)p
+ P (Xm−1 = N or Xm−1 = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m− 1|X0 = k − 1)q
= p(k + 1,m− 1)p+ p(k − 1,m− 1)q.
(15.17)
204
Observe that p(k,m) = P (Tk = m) Let Y (k) denote the average value of Tk. Then
Y (k) = E(Tk) =
∞∑
m=1
mp(k,m). (15.18)
Note that after one step we have the two-dimensional difference equation
p(k,m) = p(k + 1,m− 1)p+ p(k − 1,m− 1)q. (15.19)
Substitute (15.19) into (15.20) and get
Y (k) =
∞∑
m=1
m (p(k + 1,m− 1)p+ p(k − 1,m− 1)q) . (15.20)
Let i = m− 1. After some manipulation this equation reduces to ([26], pp. 193-194)
Y (k) = Y (k + 1)p+ Y (k − 1)q + 1, Y (0) = Y (1) = 0. (15.21)
The boundary conditions Y (0) = Y (N) = 0 means absorption is immedate. Proceeding as in Example
1, we let r = qp and get
Y (k) =1
q − p
(
k −N
[
1 − rk
1 − rN
])
if p 6= 1
2. (15.22)
and
Y (k) = k(N − k) if p = q =1
2. (15.23)
(ii.) Poisson processes
A random process X(t)|t ≥ 0 is a counting process if
(a) X(0) = 0, X(t) ≥ 0, X(t) is integer valued, and
(b) if s < t then X(s) ≤ X(t) and X(t) −X(s) = the number of events that have occured on (s, t).
A counting process X(t) is a Poisson process with rate λ > 0 if
(c) X(t) has stationary and independent increments (see Sec. I, part (g.) above), and
(c) the number of events in an interval of length t is Poisson distributed with mean λt, i.e. ∀s, t > 0
P X(t+ s) −X(s) = k = e−λt (λt)k
k!∀k ≥ 0. (15.24)
These properties imply that ([26], pp. 200-201) E (X(t)) = λt, V ar(X(t)) = λt, and
P X(t+ ∆t) −X(t) = 1 = λ∆t+ o(∆t),
P X(t+ ∆t) −X(t) ≥ 2] = o(δt).
A first passage time process.
The variable X(t) gives the number of events that occur on (0, t). Let T1 be the random variable that
denotes the first time that X = 1. It follows from this definiton and (20.1) that
P T1 > t = P X(t) = 0 = e−λt. (15.25)
205
Thus, the cumulative distribution function for T1 is
FT1(t) = P 0 < T1 < t = 1 − P T1 > t = 1 − e−λt. (15.26)
From this it follows that the probability density function for T1 is
fT1(t) =d
dtFT1(t) = λe−λt. (15.27)
Neuron example. Assume that a nerve impulse arrives at a synapse, and subsequently a corre-
sponding signal crosses over the synapse to contribute to the excitation of a neuron. Assume that
the arrival rate of the impulses is Posson distributed with rate λ = 1/10. The units of λ are 1/msec,
hence on average one impulse arrives at the synapse every 10 msec. Let X(t) denote the number of
impulses that have arrived by time t. Thus,
P (X(t+ τ) −X(t) = N) = exp(−λτ) (λτ)N
N !∀τ > 0 and N ≥ 0.
.
Problem 1. What is the average value of the arrival time of the N th impulse?
Solution: We need to compute E(TN), where TN denotes the time of arrival of the N th impluse.
The first method is to use the fact that the probability distribution for TN is ([26], p. 201)
fTN (t) =
λe−λt (λt)N−1
(N−1)! ∀t ≥ 0.
0 t < 0.(15.28)
Here is an alternative derivation of (15.28). Note that
P (TN > t) = P (X(t) ≤ N − 1) =
N−1∑
k=0
P (X(t) = k) =
N−1∑
k=0
e−λt (λt)k
k!, (15.29)
Thus, the cdf is
P (TN ≤ t) = 1 − P (X(t) ≤ N − 1) = 1 −N−1∑
k=0
e−λt (λt)k
k!. (15.30)
From this we get the pdf
fTN (t) =d
dt
(
1 −N−1∑
k=0
e−λt (λt)k
k!
)
= e−λt (λt)N−1
(N − 1)!. (15.31)
The average value of TN is
E(TN) =
∫ ∞
0
tλe−λt (λt)N−1
(N − 1)!dt =
N
λ. (15.32)
Problem 2. Assume that the neuron cannot fire until at least three impulses have arrived at the
synapse. What is the probability that the neuron fires in 60 msec.?
206
Solution: we need to compute the probability that at most two impulses arrive by 60 msec. since
the neuron cannot fire until three or more arrive. Thus, we compute
P (X(60) −X(0) ≤ 2)
Using the fact that X(0) = 0, we obtain
P (X(60) −X(0) ≤ 2) =
2∑
i=0
(P (X(60) −X(0) = i) =
2∑
i=0
exp(−λ60)(λ60)i
i!
Since λ = 110 , this reduces to
P (X(60)−X(0) ≤ 2) =
2∑
i=0
exp(−6)(6)i
i!= exp(−6)(1 + 6 + 18) = .0619
Problem 3. This is the “delay’ problem. How long does it take so that the probability the neuron
does not fire is equal to 12? Note that if t is small then the probability that the neuron does not fire
is pretty high. The amount of time needed is the dalay and is denoted by τ.
Solution: we need to compute the time t = τ > 0 such that probability that at most two impulses
arrive by τ msec is equal to 12 , i.e.
P (X(τ) −X(0) ≤ 2) =1
2.
Setting λ = 110 gives
P (X(τ) −X(0) ≤ 2) =
2∑
i=0
exp(−λτ) (λτ)i
i!= exp(− τ
10)(1 +
τ
10+
τ2
200) =
1
2
Using the matlab program delay.m, we find that τ ≈ 26.66
(c.) The cdf and pdf for the random variable Zi = Ti − Ti−1
Let X(t) be a Poisson process ∀t ≥ 0. Then Zi = Ti − Ti−1 represents the length of the ”inter-spike
interval” (Ti−1, Ti). Our first goal is to derive an integral representation for the cdf of the non-negative
Zi. After that we find the pdf. We will make use of the expansion
P (A) =
∞∑
i=1
P (A ∩Ai) =
∞∑
i=1
P (A|Ai)P (Ai), (15.33)
Let ∆τ > 0 be small and fixed, set τ0 = 0 and define the partition points τk = τk−1 + ∆τ and Ik =
[τk−1, τk−1 + ∆τ ] ∀k ≥ 1. Then [0,∞) = ∪k≥1Ik and we conclude that
P (Zi > z) ≈∑∞k=1 P (Zi > z and Ti−1 ∈ Ik)
=∑∞
k=1 P (Zi > z|Ti−1 ∈ Ik)P (Ti−1 ∈ Ik)
≈∑∞k=1 P (Zi > z|Ti−1 = τk)fTi−1(τk)∆τ.
(15.34)
207
Letting ∆τ → 0, we conclude from (15.34) that
P (Zi > z) =
∫ ∞
0
P (Zi > z|Ti−1 = τ)fTi−1 (τ)dτ. (15.35)
To evaluate the right side of (15.35) note that
P (Zi > z|Ti = τ) = P (Ti > Ti−1 + z = τ + z ≥ τ = Ti−1)
= P (X(τ + z) −X(τ) = 0)
= P (X(z) = 0) = e−λz.
(15.36)
Substituting (15.36) into (15.35) gives
P (Zi > z) =
∫ ∞
0
e−λzfTi−1(τ)dτ = e−λz
∫ ∞
0
fZ1(τ)dτ = e−λz, (15.37)
since∫∞0 fZ1(τ)dτ = 1. It follows from (15.37) that
P (Zi ≤ z) = 1 − e−λz and fZi(z) =d
dzP (Zi ≤ z) = λe−λz. (15.38)
Remark. The pdf fZi(z) = λe−λz decreases. The pdf in the next example is not strictly decreasing.
(d.) The first passage time for the reaction A→ B → C.
This follows hnd1stPassage.pdf. Assume that A, B, and C are proteins, that A decays at poisson
rate α into B, and that B decays at poisson rate β into C. This means that the lifetimes τA and τB
are exponentially distributed, e.g.
FA(t) = P0 < τA < t = 1 − e−αt and pA(t) = F ′A(t) = αe−αt.
The waiting time, or ”first passage time” for C is the time it takes C to form. The goal is to find the
average waiting time for C. For this we first need to compute pC(t)dt using
pC(t)dt = Pwaiting time for C falls in (t, t+ dt)
= Pcombined lifetime of A and B falls in (t, t+ dt)
= PτA + τB ∈ (t, t+ dt).
(15.39)
Note that
(1.) A disappears at an unknown intermediate time τ. We have to take into account every possiblity.
(2.) if B disppears (i.e. C appears) in (t, t+ dt) then the lifetime of B is t− τ.
These two events are independent since the protein has no memory. Therefore, to take into account
all possible values of τ we construct the intermediate probability
P (t, τ) = PτA ∈ (τ, τ + dτ) AND τB ∈ (t− τ, t− τ + dt)
= PτA ∈ (τ, τ + dτ)PτB ∈ (t− τ, t− τ + dt)
= (pA(τ)dτ) (pB(t− τ)dt)
= αe−ατdτβe−β(t−τ)dt.
(15.40)
208
Now add up the intermediate contribution to pC(t)dt from each τ and get
pC(t)dt =
(∫ t
0
αe−ατβe−β(t−τ)dτ
)
dt (15.41)
After division by dt we get
pC(t) =
∫ t
0
αe−ατβe−β(t−τ)dτ =αβ
α− β
(
e−βt − e−αt)
if α 6= β, (15.42)
pC(t)dt = α2te−αt if α = β. (15.43)
Here pc(t) is the distrubiton of first passage times. The average first passage time is
E(τC) =
∫ ∞
0
tpC(t)dt =1
α+
1
βif α 6= β, (15.44)
E(τC) =
∫ ∞
0
tpC(t)dt =2
αif α = β. (15.45)
We now use a second method, following part (c) above, to develop the equation (15.41) for pC(t).
Here we use the finite expansion
P (A) =
N∑
i=1
P (A ∩Ai) =
N∑
i=1
P (A|Ai)P (Ai), (15.46)
Let ∆τ = tN > 0, set τ0 = 0 and define the partition points τk = τk−1 + h and Ik = [τk−1, τk−1 +
h] ∀k ≥ 1. Then [0, t) = ∪k≥1Ik and we conclude that
PC(t)dt ≈∑Nk=1 P (τA ∈ [τk−1, τk] and τB ∈ [t− τA, t− τA + dt])
=∑N
k=1 P (τA ∈ [τk−1, τk])P (τB ∈ [t− τA, t− τA + dt])
≈∑Nk=1 pA(τk−1)∆τpB(t− τk−1)dt.
(15.47)
Let N → ∞ so that ∆τ → 0 and, as required, the approximation (15.47) converges to
PC(t)dt =
∫ t
0
pA(τ)pB(t− τ)dτdt, (15.48)
Remark. Suppose that the reaction is A→ B → G→ C. Then pC(t) satisfies
pC(t) =
∫ t
τ=0
∫ t−τ
ζ=0
αe−ατβe−βζγe−γ(t−τ−ζ)dζdτ. (15.49)
(d.) Birth-death processes This follows [26] (Ch. 9). Let X(t) be a queuing system which is
a Markov process. Define pN (t) = P (X(t) = N) . Assume the birth rate and death rate probabilities
P (X (t+ ∆t = N |X(t) = N − 1) =
∫ t+∆t
t
aN−1dt, P (X(t+ ∆t) = N |X(t) = N+1) =
∫ t+∆t
t
dN+1dt
(15.50)
209
The difference equations satisfied by pN are
pN (t+ ∆t) = aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 − aN−1∆t)(1 − dN+1)pN (t) ∀N ≥ 1, (15.51)
p0(t+ ∆t) = dN+1∆tp1(t) + (1 − d0)p0(t). (15.52)
A brief derivation of (15.51) is
pN(t+ ∆t) = P (X(t+ ∆t) = N)
= P (X(t+ ∆t) = N |X(t) = N + 1)P (X(t) = N + 1)
+ P (X(t+ ∆t) = N |X(t) = N − 1)P (X(t) = N + 1)
+ P (X(t+ ∆t) = N |X(t) = N)P (X(t) = N)
=(
∫ t+∆t
t aN−1dt)
pN−1(t) +(
∫ t+∆t
t dN+1dt)
pN+1(t) + P (X(t+ ∆t) = N |X(t) = N − 1)pN (t)
= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + P (X(t+ ∆t) = N |X(t) = N − 1)pN (t)
= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 −∫ t+∆t
taN−1dt)(1 −
∫ t+∆t
tdN+1dt)pN (t)
= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 − aN−1∆t)(1 − dN+1∆t)pN (t)
(15.53)
Subtratc pN(t)∆t from both sides of (15.51), divide both sides by ∆t, let ∆t → 0, and get the ODE
p′N(t) = aN−1pN−1(t) + dN+1pN+1(t) − (aN−1 + dN+1)pN (t) ∀N ≥ 1. (15.54)
Likewise,
p′0(t) = d1p1(t) − (a0 + d0)p0(t). (15.55)
16 Time of first passage for discrete problems
(a.) Random walk: how to find probability that XN = 0 for some N > 0.
The first step in understanding random walks in one diemsnion is to follow [26], Ch. 5, Problem 5.2, p.
73. Define the set Zi, i = 1, ..., N of independent and identically distributed random variables which
have only two values, ±1, as follows: P (Zi = 1) = p and P (Zi = −1) = q = 1 − p, where 0 < p < 1.
17 White noise
(a.) Definition of white noise. A continuous time WSS random variable X(t) is called white
noise if
µX(t) = 0 and RX(τ) = σ2δ(τ) ∀τ ∈ R. (17.1)
In this case the frequency power spectrum SX(ω) satisfies
SX(ω) =
∫ ∞
−∞σ2δ(τ)dτ = σ2 ∀ω ∈ R. (17.2)
210
Because SX(ω) ≡ σ2, all frequencies contribute equally to power, hence X(t) is white noise.
(b.) Example: uniform white noise. Let X be a uniform random variable on [a, b]. The
probability density function and cumulative distribution are (see p. 45 of Schaum’s)
fX(x) =
0 for x < a,1
b−a for a ≤ x ≤ b,
0 for x > b
(17.3)
FX(x) =
0 for x < a,x−ab−a for a ≤ x ≤ b,
1 for x > b
(17.4)
The mean µX and variance σ2 of X satisfy
µX = E(X) =
∫ ∞
−∞xfX(x)dx
∫ b
a
x
b− adx =
1
2(b+ a), (17.5)
To define uniform white noise we extend all of this to the bivariate case. Let X and Y be independent
uniform random variables on [−1, 1] with joint probability distribution
fXY (x, y) =
14 for − 1 ≤ x, y ≤ 1,
0 otherwise.(17.6)
We claim that X and Y are independent. This is true if (definition on p. 80 of Schaum’s) FXY (x, y) =
FX(x)FY (y). To see this we first compute
FXY (x, y) =
∫ x
−∞
∫ y
−∞fXY (x, y)dxdy =
0 if x < −1 or y < −1,14 (x+ 1)(y + 1) if (x, y) ∈ [−1, 1]x[−1, 1],12 (x+ 1) if − 1 < x < 1 and y > 1,12 (y + 1) if − x > 1 and − 1 < y < 1,
1 if x > 1 and y > 1,
(17.7)
From (17.4) it follows that FX(x)FY (y) satisfy
FX(x)FY (y) =
∫ x
−∞fX(x)dx
∫ y
−∞fY (y)dy =
0 if x < −1 or y < −1,14 (x+ 1)(y + 1) if (x, y) ∈ [−1, 1]x[−1, 1],12 (x+ 1) if − 1 < x < 1 and y > 1,12 (y + 1) if − x > 1 and − 1 < y < 1,
1 if x > 1 and y > 1,
(17.8)
Alternatively, it is easily verified that
fXY (x, y) = fX(x)fY (y) ∀(x, y) ∈ R2, (17.9)
211
also implies that X and Y are independent.
Next, from (??) we conclude that E(X) = E(Y ) = 0. From (17.6) it follows that
E(XY ) =
∫ ∞
−∞
∫ ∞
−∞fXY (x, y)dxdy =
∫ 1
−1
∫ 1
−1
xy
4dxdy = 0 (17.10)
Thus, it must be the case that
E(XY ) = E(X)E(Y ) = 0. (17.11)
To define uniform white noise, let X(t) denote a uniform random variable on [−1, 1] such that X(t)
and X(s) are independent when t 6= s. It follows from (??) that
E(X(t)) = µX(t) = 0 and E(X2(t)) =1
3= σ2 ∀t. (17.12)
Since X(t) and X(s) are independent when t 6= s, then the autocorellation function RX(t, s) satisfies
RX(t, t+ τ) =
E(X(t)X(t+ τ)) = E(X(t))E(X(t+ τ)) = 0 if τ 6= 0,
E(X2(t)) = σ2 if τ = 0.(17.13)
That is
RX(t, t+ τ) = σ2δ(τ) ∀τ ∈ R. (17.14)
From (5.5), (17.12) and (17.14) follows that X(t) satisfies the definition of white noise.
(c.) Computing RX(τ) and SX(ω) for uniform white noise.
To compute RX(τ) and SX(ω) for uniform white noise use the matlab programs white.m and Rxest.m
18 Research ideas and papers to get
(a.) Feynman extension Could we make use of Feynman’s bm probability method to see self
organization evolve in a system of single-cells as the number of cells increases? His ideas are shownn
in ([9], see the diagram on p. 120). Would his cancelling out idea somehow work? How to set it up
so an anlaogy to his cancelling “angles” occurs? Wouyld some kind of multiplicative noise help?
(b.) Oscillations in predator-prey systems. Start with a dead system and give it multiplicative
noise to bring it to life. The key is to add on an ode for the σ term. Same for a system of genes with
a combination of multiplicative/additive noise? How to get multiplicative noise to produce a steady
state that was not there before - speciation example?
(c.) Neuro papers: Brunel-Hakim, Amit-Brunel, Brunel
Brunel & Hakim Neuro comp 1999
Amit & Brunel Cerebral cortex 1997
Brunel J. Comp. Neuroscience 1997
212
(d.) alternans. Start with a benzi system and hook it to a second system where the ǫ obeys an
equation putting it at one level for the regulat oscillations, and at a second level it gives the period
doubling oscillation, e.g.
ǫ′ = ǫ(1 − ǫ)(1 + ǫ) + σζ
19 Use of the steady state pde
The following principles lead to information about X(t) by analyzing the pde:
(1) find the time dependent pde
(2) solve the time independent steady state pde
(3) Evaluate ”average value of X” by figuring out E(X) for the steady state pde
(4) Evaluate ”average value of X2” by figuring out E(X2) for the steadystate pde.
(5) Evaluate “ average displacement” as Langevin did for Einstein’s result did by computing√
E(X2).
213
20 Population growth in a randomly varying environement.
(a.) The problem. This follows [14]. Consider the discrete population growth model
Nt+1 = ltNt, t = 1, 2, ... (20.1)
where the li are identically distributed, independent random variables (this will allow the central
limit theorem to apply), whose pdf is denoted by f(l). The solution of (20.1) is
Nt = N0
t∏
i=1
li. (20.2)
The mean and variance of l are given by
λ = µl = E(l) =
∫ ∞
−∞lf(l)dl and σl
2 = E(l2) − (E(l))2 (20.3)
Then
E (Nt) = N0
t∏
i=1
E(li) = N0λt, (20.4)
and from this we conclude that
E (Nt) → ∞ as t→ ∞ if λ > 1, (20.5)
E (Nt) → 0 as t→ ∞ if 0 < λ < 1. (20.6)
Extinction occurs if, for each K1 > 0,
Prob Nt > K1 → 0 as t→ ∞. (20.7)
When λ > 1 we know from (20.5) that the mean of Nt satisfies E (Nt) → ∞ as t → ∞. The problem
is to determine whether extinction can occur when λ > 1.
(b.) Put the extinction problem into the Central Limit Theorem setting.
To determine whether extinction occurs, derive a formula for the probability that Nt lies between two
given numbers K1 > 0 and K2 > 0. First, because ln(x) is an increasing function it is easy to prove
the following important connection between Nt and ln(Nt) :
Prob K1 ≤ Nt ≤ K2 = Probln(K1) ≤ ln(Nt) ≤ ln(K3) . (20.8)
Combining this equation with the fact that
ln(Nt) = ln(N0) +
t∑
i=1
ln(li), (20.9)
we obtain
ProbK1 < Nt < K2 = Prob
ln
(
K1
N0
)
≤t∑
i=1
ln(li) ≤ ln
(
K2
N0
)
. (20.10)
214
Division by t gives
ProbK1 < Nt < K2 = Prob
1
tln
(
K1
N0
)
≤ 1
t
t∑
i=1
ln(li) ≤1
tln
(
K2
N0
)
. (20.11)
Subtract µln(l) from each quantity, divide by σln(l)/√t. This leads to
ProbK1 < Nt < K2 = Prob
τ1(t) <1t
∑ti=1 ln(li) − µln(l)
σln(l)/√t
< τ2(t)
, (20.12)
where
τ1(t) =
1t ln(
K1
N0
)
− µln(l)
σln(l)/√t
and τ2(t) =
1t ln(
K2
N0
)
− µln(l)
σln(l)/√t
. (20.13)
The right side of (20.12) is now set up to apply the Cental Limit Theorem, which gives
Prob K1 < Nt < K2 ≈ Prob τ1(t) < τ < τ2(t) when t >> 1. (20.14)
where τ = N(0, 1).
(c.) Conditions which imply that E(Xt) → ∞, yet extinction occurs.
Let K1 > 0 and K2 = ∞, and assume that the positive random variable l satisfies
λ = µl = E(l) > 1 and µln(l) = E(ln(l)) < 0. (20.15)
Then τ2 = ∞ and (20.14) becomes
Prob K1 < Nt <∞ ≈ Prob τ1(t) < τ <∞ when t >> 1. (20.16)
It follows from (20.13) and (20.15) that
τ1(t) → ∞ as t→ ∞. (20.17)
This and (20.16) imply that
E(Xt) = X0λt as as t→ ∞, (20.18)
and
Prob K1 < Nt <∞ ≈ Probτ1(t) < τ <∞ → 0 as t→ ∞. (20.19)
Thus, the mean of Xt tends to +∞, yet extinction occurs when conditions (20.15) hold. It remains to
determine conditions under which µln(l) = E(ln(l)) < 0. For this we make use of the approximation
ln(l) ≈ ln(λ) +(l − λ)
λ− 1
2λ2(l − λ)2 when l ≈ λ. (20.20)
Note that
E(l − λ) = E(l) − λ = 0 and E(l − λ)2 = E(l2) − λ2 = σ2l . (20.21)
215
Combining (20.20) and (20.21), we obtain
E(ln(l)) ≈ ln(λ) − σ2l
2λ2< 0 when σl > λ
√
2ln(λ) (20.22)
Important observation. Note that CV, the coefficient of variation, is given by
CV =σ
λ. (20.23)
Thus, when λ1 > is small, we have
E(ln(l)) ≈ ln(λ) − (CV)2
2< 0 when CV =
σ
λ>√
2ln(λ) (20.24)
Thus, when λ − 1 > 0 is small and CV >√
2ln(λ) then X(t) goes extinct and E(X(t)) → ∞ as
t→ ∞.
(d.) Proof that ρ =(
∏ti=1 li
)1/t
≤ λ = 1t
∑ti=1 li when li ≥ 0.
Our goal is to prove that the geometric mean over a cycle of l′is of length t is is less than or equal to
their arithmetic mean. These are defined by
ρ =
(
t∏
i=1
li
)1/t
and λ =1
t
t∑
i=1
li. (20.25)
Note that1
λ
t∑
i=1
li = t. (20.26)
Thus,
1 = exp
(
1
λ
t∑
i=1
li − t
)
=t∏
i=1
exp
(
liλ− 1
)
. (20.27)
Use the inequaltiy exp(x) ≥ x+ 1 and obtain
1 =t∏
i=1
exp
(
liλ− 1
)
≥t∏
i=1
liλ
=1
λt
t∏
i=1
li. (20.28)
From this it follows that
λt ≥t∏
i=1
li, (20.29)
hence
λ ≥ ρ =
(
t∏
i=1
li
)1/t
. (20.30)
This completes the proof.
(e.) A periodic sequence li such that ρ < 1 < λ and (Xt, Xt) → (0, 0) and as t→ ∞.
Define a periodic sequence li, of period 10, by
l1 = l1 = .. = l9 = 1.1, l10 = .3 (20.31)
216
Over a cycle of length 10 we have
ρ =
(
10∏
i=1
li
)1/10
= .71/10 = .96 and λ =1
10
10∑
i=1
li = 1.02 (20.32)
At t = 10N we have
X10N = .710NX0 → 0 as N → ∞, (20.33)
and also
Xt → ∞ as t→ ∞. (20.34)
Furthermore, we find that
σ2l = E(l2) − E2(l) =
1
10(l21 + ..+ l210) − λ2 = 1.098 − 1.022 = .0576 (20.35)
and therefore,
E(ln(λ)) ≈ ln(λ) − σ2l
2λ= .0198− .0576
2.02= −.0087 (20.36)
According to the theory developed above, this implies that the solution goes to extinction in spite of
the fact that the mean value of the l′is is greater than 1.
Figure 27: Left panel: X vs. t Right panel: Arithmetic mean of X vs. t. ρ = .96 and λ = 1.02
(f.) A periodic sequence li such that ρ = 1 < λ, Xt is periodic, Xt → constant as t→ ∞.
Define a periodic sequence li, of period 10, by
l1 = l1 = .. = l9 = 1.1, l10 = 1.1−9 (20.37)
Over a cycle of length 10 we have
ρ =
(
10∏
i=1
li
)1/10
= 1 and λ =1
10
10∑
i=1
li = 1.032 (20.38)
The solution is periodic (see Figure 28), and its mean approaches a constant at t→ ∞, again in spite
of the fact that the mean of the l′is is greater than 1.
217
Figure 28: Left panel: X vs. t Right panel: Arithmetic mean of X vs. t. Here ρ = 1 and λ = 1.032
21 A noisy genetic swith model
This follows [28]. The model is
x′ = −ax+ y + σ1ζ1(t)x, x(0) > 0,
y′ = x2
1+x2 − by, y(0) = 0
b′ = (b− b1)(b − b2)(b3 − b) + σ2ζ2(t), b(0) = b3.
(21.1)
where x is protein concentration, y is messenger RNA concentration, and a, b are parameters. Here
I’ve assumed that b is variable and subject to noise, 0 < b1 < b2 < b3, and b3 is chosen so that if
b ≡ b3 then (0, 0) is the only constant solution of the (x, y) system. Also, b1 > 0 is small so that if
b ≡ b1 then there are three constant solutions, and the (x, y) system can be switched on by the noise
in the x eq. Strogatz ([28], p. 245) gives further details of the (x, y) system with no noise present. To
turn the system on the idea is to let b2 < b3 and close to b3, and letσ2 > be “just large enough“ so
that b quickly jumps all the way down to b)1 and stays there with low probability of jumpimng back
up and swithing off the system
218
22 Coherence resonance in SIR (Kuske et al [13])
(a.) The problem This follows [13]. Assume that a population has three types of individuals,
S (susceptibles), I (infectives) and R (recovered), and that there is an integer N with
S1 + It + Rt = N ∀t ≥ 0. (22.1)
It is also assumed that S1, It, Rt Markov processes, nd that over a small interval [t, t + ∆t] the
increments ∆S = St+∆t − St and ∆I = It+∆t − It satisfy
∆S =(
µ(N − S) − β SIN
)
∆t+ ∆Z1 − ∆Z2,
∆I =(
β SIN − (γ + µ)I
)
∆t+ ∆Z2 − ∆Z3.(22.2)
They approximate the discrete model (22.2) with the continuous SDE system
dS =(
µ(N − S) − β SIN
)
dt+G1dW1 −G2dW2,
dI =(
β SIN − (γ + µ)I
)
dt+G2dW2 −G3dW3,(22.3)
where
G1 =√
µ(N + S), G2 =
√
β
NSI, G3 =
√
(γ + µ)I. (22.4)
Figure 29: Solution of (22.3) for parameters R0 = 15, µ = 1/55, γ = 20, N = 20000000, and initial
conditions (S(0), I(0)) = (133330, 2695). The solid curve is the stochastic solution. The dashed curve
is the deterministic solution.
(b.) What are ∆Z1,∆Z2,∆Z3, and how is (22.3) derived from (22.2)?
219
To understand what the variables ∆Z1,∆Z2 and ∆Z3 represent we consider the interval [t, t+∆t] where
increments ∆S and ∆I take place. Kuske et al assume that ∆S can be expressed as a combination
of independent Poisson processes ∆S1, ∆S2 and ∆S3, which denote the contributions to ∆S due to
birth rate, death rate and the rate of removal of susceptibles to I, the infected population. Thus,
∆S = ∆S1 − ∆S2 − ∆S3. (22.5)
Over [t, t+ ∆t] it is also assumed that the means of ∆S1, ∆S2 and ∆S3 satisfy
E(∆S1) = µN∆t, E(∆S2) = µSt∆t, and E(∆S3) = βStItN
. (22.6)
Since ∆S1, ∆S2 and ∆S3 are Poisson processes, then
V ar(∆S1) = µN∆t, V ar(∆S2) = µSt∆t, and V ar(∆S3) = βStItN
∆t. (22.7)
The difference ∆S1 − ∆S2 is also a Poisson process, with mean
E(∆S1 − ∆S2) = E(∆S1) − E(∆S2) = µN∆t− µSt∆t. (22.8)
Because ∆S1 and ∆S2 are independent, then
V ar(∆S1 − ∆S2) = V ar(∆S1) + V ar(∆S2) = µN∆t+ µSt∆t. (22.9)
To apply the Central Limit Theorem we define the variables ∆Z1 and ∆Z2 by
∆Z1 = ∆S1 − ∆S2 − [E(∆S1) − E(∆S2)] and ∆Z2 = ∆S3 − E(∆S3). (22.10)
Note that
E [∆Z1] = E [∆Z2] = 0. (22.11)
From (22.10), (22.7) and the independence of ∆S1 and ∆S2 it follows that
V ar [∆Z1] = V ar[∆S1] + V ar[∆S2] = µN∆t+ µSt∆t. (22.12)
By the Central Limit Theorem there exists c1 such that
∆Z1 = ∆S1 − ∆S2 − [E(∆S1) − E(∆S2)] ≈ c1∆X1, (22.13)
whereX1 denotes a normal random variable with E(X1) = 0 andE(X21 ) = V ar(X1) = 1.To determine
value of c1 square both sides of (22.13 ) and get
V ar(∆Z1) = E(
∆Z21
)
= V ar (∆S1 − ∆S2) ≈ c21E(
X21
)
= c21. (22.14)
Combining (22.14 ) and (22.9) gives
µN∆t+ µSt∆t ≈ c21 (22.15)
220
Thus, c1 =√
µ(N + St)∆t, and it follows from (22.6) and (22.13) that
∆S1 − ∆S2 ≈ µ(N − St)∆t+√
µ(N + St)∆tX1. (22.16)
Let W1 be a standard Wiener process, and recall that ∆W1 = W1(t+ ∆t) −W1(t) and√
∆tX1 have
the same pdf. This allows us to replace√
∆tX1 with ∆W1 in (22.16) and conclude that
∆S1 − ∆S2 ≈ µ(N − St)∆t+√
µ(N + St)∆W1. (22.17)
For the variable ∆Z2 = ∆S3 − E(∆S3) we have
V ar [∆Z2] = V ar[∆S3] = βStItN
∆t. (22.18)
As above, the Central Limit Theorem implies that there exists c2 and a Wiener process W2 such that
∆Z2 = ∆S3 − E(∆S3) ≈ c2∆W2. (22.19)
Note that ∆W1 and ∆W2 are independent since ∆S1,∆S2 and ∆S3 are independent. Squaring both
sides of (22.19), and using (22.18), we get
E(∆Z22 ) = E[(∆S3 − E(∆S3))
2] = V ar(∆S3) = βStItN
∆t ≈ c22V ar(∆W22 ) = c22∆t. (22.20)
Thus, c2 =√
β StIt
N and it follows from (22.6) and (22.18) that
∆S3 ≈ βStItN
∆t+
√
βStItN
∆W2. (22.21)
It follows from (22.5), (22.16) and (22.21) that
∆S ≈(
µ(N − St) − βStItN
)
∆t+√
µ(N + St)∆W1 −√
βStItN
∆W2. (22.22)
Replacing ∆S,∆t,∆W1 and ∆W2 with dS, dt, dW1 and dW2 gives the final approximation
dS =
(
µ(N − St) − βStItN
)
dt+√
µ(N + St)dW1 −√
βStItN
dW2. (22.23)
Next, we give a similar derivation for the second equation in (22.3). It is assumed that ∆I can be
expressed as the difference of independent Poisson processes ∆I1 and ∆I2, which denote the increase
in ∆I due to the rate of removal of susceptibles to I, and the decrease in ∆I due to the removal of
infected individuals to the recoverd population. Thus,
∆I = ∆I1 − ∆I2, (22.24)
and over [t, t+ ∆t] we assume that
∆I1 − E(∆I1) = ∆S3 − E(∆S3). (22.25)
221
Thus,
∆I1 ≈ βStItN
+
√
βStItN
dW2. (22.26)
Next, assume that that the mean of ∆I2 satisfies
E(∆I2) = (γ + µ)It∆t. (22.27)
As above, there is a Wiener process W3, independent of W1 and W2 such that
∆I2 ≈ (γ + µ)It∆t+√
(γ + µ)It∆W3. (22.28)
From (22.24), (22.26) and (22.28) we conclude that
∆I ≈(
βStItN
+ (γ + µ)It
)
∆t+
√
βStItN
∆W2 +√
(γ + µ)It∆W3. (22.29)
Finally, replace ∆I,∆t,∆W2 and ∆W3 with dI, dt, dW2 and dW3 and get the second SDE
dI =
(
βStItN
+ (γ + µ)It
)
dt+
√
βStItN
dW2 +√
(γ + µ)ItdW3. (22.30)
(c.) How to rescale and identify the small parameter?
Step I. Determine the steady state:
Seq =N
R0and Ieq =
Nµ
β(R0 − 1) . (22.31)
Step II. Non-dimenionalize and focus the solutions around (Seq , Ieq) by defining
u =S − Seq
Seqand v =
I − Ieq
Ieq. (22.32)
Step III. Identify the small parameter. For this, they first linearize the (u, v) system around the (u, v)
steady state (0, 0). At this point they also rescale time by setting η = Ωt where Ω is chosen so that
the eigenvalues λ of the linearized matrix are complex, with Im(λ) ≈ 1. Here,
λ = −ǫ2 ±√
ǫ4 − 1, (22.33)
where
ǫ2 =µR0
2Ω<< 1 and Ω =
√
βµ
R0(R0 − 1) (22.34)
(d.) Other key parameters.
Key parameters are
R0 =β
γ + µ> 1 and b2 =
Ieq
Seq=µR0(R0 − 1)
β(22.35)
Very important properties which they use below are
βb
R0Ω= 1 and b =
µ(R0 − 1)
Ω. (22.36)
222
Numbers used in Figure 29 (i.e. Kuske et al first diagram) are
R0 = 15, µ = 1/55, γ = 20, N = 20000000 (22.37)
hence
β ≈ 300.27273, Ω ≈ 2.2573, ǫ2 ≈ .0604, b ≈ .11269 (22.38)
(e.) How to derive the key linear system deterministic solution?
The big picture: To get stochastic resonance to work they work entirely with linear approximations.
For this they linearize the non-dimensionalized system around (0, 0) and get the resultant linear system
to produce stochastic resonance. Hopefully this causes the full nonlinear system to exhibit stochastic
resonance. The all important thing to do first is to derive the key approximation to the linear system.
Step I. Write the linearized system (8) in the matrix form
U ′ = MU +G(dW1, dW2.dW3)T , (22.39)
where U = (u, v)T , and M and G are the constant matrices
M =
[ −µR0
Ω−µ(R0−1)
Ωβ
ΩR00
]
=
[
−2ǫ2 −b1b 0
]
and G =
[
g1 −b2g2 0
0 g2 −g2
]
,
and
g1 =
√
µ
ΩS2eq
(N + Seq) =
√
µ
ΩN(R2
0 +R0) and g2 =
√
βSeq
ΩNIeq=
√
R0Ω3
µ3(R0 − 1)3N.
Here 0 ≤ g1, g2 << 1 and are non-constant due to the linearization around (0, 0).
The eigenvalues of M are complex and are given by
λ = −ǫ2 ± i√
1 − ǫ4.
Next, solve U ′ = MU of (22.39) and get the fundamental matrix solution
eMt = e−ǫ2tcos(√
1 − ǫ4t)E1 + e−ǫ2tsin(√
1 − ǫ4t)E2. (22.40)
Using the fact that ǫ2 = µR0
2Ω , we find that E1 and E2 are
E1 = I and E2 =1√
1 − ǫ4
[ −µR0
2Ω−µ(R0−1)
Ωβ
ΩR0
µR0
2Ω
]
.
Thus, if√
1 − ǫ4 ≈ 1 and D = (d1, d2)T then the general solution U = (u, v)T of the deterministic
part U ′ = MU of the linear system is
U ≈ e−ǫ2t
[
d1cos(t) − µR0
2Ω d1sin(t) −µ(R0−1)2Ω d2sin(t)
d2cos(t) + βΩR0
d1sin(t) +µR0
2Ω d2sin(t)
]
.
223
This reduces to the key approximation[
u
v
]
≈ e−ǫ2tC1
[
bcos(t)
sin(t)
]
+ e−ǫ2tC2
[
bsin(t)
−cos(t)
]
, (22.41)
(f.) How to derive the SDE’s which use the slow time scale T = ǫ2t?
Guided by (22.41), Kuske et al follow [28], set T = ǫt, and look for a solution
[
u
v
]
=
[
u0
v0
]
+O(ǫ2), (22.42)
where[
u0
v0
]
= A(T )
[
bcos(t)
sin(t)
]
+B(T )
[
bsin(t)
−cos(t)
]
. (22.43)
Note that (u0, v0)T satisfies
d
(
u0
v0
)
=
[
0 −b1/b 0
](
u0
v0
)
. (22.44)
Kuske et all assume that A(T ) and B(T ) are stochastic and satisfy
[
dA
dB
]
=
[
f1(A,B)
f2(A,B)
]
dT +
[
dN1(T )
dN2(T )
]
, (22.45)
where (dN1, dN2) has the general form
[
dN1
dN2
]
= Σ1
[
dζ11(T )
dζ12(T )
]
+ Σ2
[
dζ21(T )
dζ22(T )
]
+ Σ3
[
dζ31(T )
dζ32(T )
]
, (22.46)
where the dζij are standard Brownian motions on the slow time scale T. To figure out f1, f2, dN1 and
dN2 they use a Fredholm alternative type criterion which is described below.
First, note that if two functions u and v satisfy u = u(A,B, t) and v = v(A,B, t), then Ito’s Lemma
for systems (next section) gives
du = uAdA+ uBdB + (.5 ∗Q1uAA +Q2uAB + .5 ∗Q3uBB) dt+ utdt,
dv = vAdA+ vBdB + (.5 ∗Q4vAA +Q5vAB + .5 ∗Q6vBB) dt+ vtdt.(22.47)
where the subscripts mean partial derivatives. Set u = u0, v = v0. The second partials of u0 and v0
are zero because of the linear form of (22.43). Using (22.43) reduces the right side of (22.47) to
du0 = bcos(t)dA+ bsin(t)dB + (u0)tdt,
dv0 = sin(t)dA− cos(t)dB + (v0)tdt.(22.48)
Substitute dA and dB from (22.45) into (22.48) and get
du0 = bcos(t)(f1dT + dN1) + bsin(t)(f2dT + dN2) + (u0)tdt,
dv0 = sin(t)(f1dT + dN1) − cos(t)(f2dT + dN2) + (v0)tdt.(22.49)
224
In matrix form this system becomes
d
(
u0
v0
)
=
(
bcos(t) bsin(t)
sin(t) −cos(t)
)(
f1dT + dN1
f2dT + dN2
)
dT +
(
0 −b1/b 0
)(
u0
v0
)
dt (22.50)
Recall that (u, v)T satisfies
d
(
u
v
)
dt =
(
−2ǫ2 −b1/b 0
)(
u
v
)
dt+
(
g1dW1 − b2g2dW2
g2dW2 − g2dW3
)
. (22.51)
Subtract (22.50) from(22.51) and get
d
(
u− u0
v − v0
)
−(
0 −b1/b 0
)(
u− u0
v − v0
)
dt = −(
bcos(t) bsin(t)
sin(t) −cos(t)
)(
f1dT + dN1
f2dt+ dN2
)
+
(
−2ǫ2 0
0 0
)(
u
v
)
dt+
(
g1dW1 − b2g2dW2
g2dW2 − g2dW3
)
.
(22.52)
Replace (u, v)T on the right side with (22.42), set dT = ǫ2dt, ignore the O(ǫ2) term, and get
d
(
u− u0
v − v0
)
−(
0 −b1/b 0
)(
u− u0
v − v0
)
dt = −(
bcos(t) bsin(t)
sin(t) −cos(t)
)(
f1dT + dN1
f2dt+ dN2
)
+
(
−2 0
0 0
)(
A(T )bcos(t) +B(T )bsin(t)
A(T )sin(t) −B(T )cos(t)
)
dT +
(
g1dW1 − b2g2dW2
g2dW2 − g2dW3
)
.
(22.53)
(f.) How to use the adjoint system to find the fi + dNi(T ) terms
System (22.53) has a solution if its right side is othogonal to all solutions of the adjoint system
V ′ = −MTV, (22.54)
where
−MT =
[
0 − 1b
b 0
]
.
The eigenvalues of −MT are λ = ±i. The fundamental matrix solution of (22.54) is
e−MT t = cos(t)I − sin(t)MT (22.55)
Thus, the general solution of V ′ = −MTV is
V = d1
[
cos(t)
bsin(t)
]
+d2
b
[
−sin(t)
bcos(t)
]
. (22.56)
225
Let D1 = d1 and D2 = d2
b . Then
V = D1
[
cos(t)
bsin(t)
]
+D2
[
−sin(t)
bcos(t)
]
. (22.57)
To evaluate f1, f2, dN1 and dN2 we use the inner product criterion
∫ 2π
0 (u∗, v∗)
((
bcos(t) bsin(t)
sin(t) −cos(t)
)(
f1
f2
)
dT +
(
bcos(t) bsin(t)
sin(t) −cos(t)
)(
dN1(T )
dN2(T )
))
dt
+∫ 2π
0 (u∗, v∗)
((
2 0
0 0
)(
A(T )bcos(t) +B(T )bsin(t)
A(T )sin(t) −B(T )cos(t)
)
dT −(
g1dW1 − b2g2dW2
g2dW2 − g2dW3
))
dt =
(
0
0
)
(22.58)
where (u∗, v∗) = (cos(t), bsin(t)) and (u∗, v∗) = (−sin(t), bscos(t)).
Very important insight: we must expand each dWi in terms of sin(t) and cos(t) in order to apply the
inner product criterion (22.58).
Kuske et al claim that each gjdWk(t) can be written as
gjdWk(t) =gj
ǫ[cos(t)dwk1(T ) + sin(t)dwk2(T )] , (22.59)
where the dwki are independent standard Brownian motions. Here is an outline of a proof. Start with
standard normal random variables X , X1 and X2 and set
Y = cos(α)X1 + cos(α)X2. (22.60)
A computation shows that X and Y have the same moment generating function MX(t) = et2/2.
Therefore ∆tX and ∆tY have the same pdf so we write
∆tX = cos(α)∆tX1 + cos(α)∆tX2, (22.61)
or equivalently
dW = dtX = cos(α)dtX1 + sin(α)dtX2 = cos(α)dw1(t) + sin(α)dw2(t). (22.62)
Set T = ǫ2t and get
dW =1
ǫ[cos(α)dw1(T ) + sin(α)dw2(T )] . (22.63)
Replace α with t and get
dW =1
ǫ[cos(t)dw1(T ) + sin(t)dw2(T )] . (22.64)
Set for dW = dWk, dw1 = dwk1, dw2 = dwk2 and multiply by gj to get (22.59).
Finally, we evaluate (22.58) and get
f1dT + dN1(T ) = −AdT + 12bǫ
(
g1dw11(T ) − b2g2dw21(T ) + bg2dw22(T ) − bg2dw32(T ))
f2dT + dN2(T ) = −BdT + 12bǫ
(
−g1dw12(T ) + b2g2dw22(T ) + bg2dw21(T ) − bg2dw31(T ))
(22.65)
226
Thus, the (A,B)T SDE system is
d
(
A
B
)
= −(
1 0
0 1
)(
A
B
)
dT + ΓdW (T ), (22.66)
where
Γ =1
2bǫ
(
g1 −b2g2 bg2 −bg2 0 0
0 −bg2 −b2g2 0 g1 bg2
)
(22.67)
and
dW (T ) = (dw11, dw21, dw22, dw32, dw12, dw31)T (22.68)
Note that
ΓΓT =g21 + (1 + b2)b2g2
2
4b2ǫ2
(
1 0
0 1
)
. (22.69)
The SDE system (22.66) is an Ornstein-Uhlenbeck process with a statonary distribution. The mean
of the stationary porocess is zero. To see this take E of both sides of (22.66) and get
E(A(T ))′ = −E(A(T )) and E(B(T ))′ = −E(B(T )). (22.70)
Solving these equations gives
E(A(T )) = C1e−T → 0 and E(B(T )) = C2e
−T → 0 as T → ∞. (22.71)
Thus, the mean of the stationary process is E(A(∞)) = E(A(∞)) = 0.
Our next goal is to show that A and B are independent.
Step I
First, according to Arnold, p. 133, we have the following theorem (I think Kuske et al are referring
to on p. 9 of their paper):
Theorem. The solution of the multivariate equation
dXt = (A(t)Xt + a(t)) dt+B(t)dWt, X0 = c (22.72)
is a stationary gaussian process if A(t) ≡ A, a(t) ≡ 0, B(t) ≡ B, the eignevalues of A have negative
real parts, c is N(0,K) (i.e. c is n-variate normally distributed), where
K =
∫ ∞
0
eAtBBT eAT tdt. (22.73)
Step II. The normal n-variate random variable has joint pdf
fX(x) =1
(2π)N/2|detK|1/2exp
[
−1
2(x− µ)TK−1(x − µ)
]
, (22.74)
227
where µ is a vector representing normal mean, and the matrix K = (σij) = (cov(XiXj) is the
covariance matrix. In 2 dimensions, if X = (X1, X2) then
K =
[
σ2x ρσxσy
ρσxσy σ2y
]
and K−1 =1
1 − ρ2
1σ2
x
−ρσxσy
−ρσxσy
1σ2
y
,
If K is a diagonal matrix then X1 and X2 are independent. From (22.71) it follows that µ = (0, 0)
for the (A,B) system stationary distribution. Furthermore, we have K = ΓΓT , which is a diagonal
matrix.
(g.) Statement of Ito’s Lemma for systems.
We develop a two variable version following [23], pp. 45-46. Consider the system of SDE’s
dx1 = u1dt+ v11dB1 + v12dB2,
dx2 = u2dt+ v21dB1 + v22dB2.(22.75)
Here Bi denotes the Wiener process. In matrix notation system (22.75) is written as
dx(t) = udt+ vdB(t), (22.76)
where
x =
(
x1(t)
x2(t)
)
, u =
(
u1
u2
)
, v =
(
v11 v1,2
v2,1 v2,2
)
and dB =
(
dB1
dB1
)
. (22.77)
Let g(t, x) = (g1(t, x), g2(t, x)) be a C2 function of (t, x). Then
dg =
( ∂g1
∂t∂g2
∂t
)
dt+
( ∂g1
∂x1dx1 + ∂g1
∂x2dx2
∂g2
∂x1dx1 + ∂g2
∂x2dx2
)
+1
2
(
∑
i,j∂2g1
∂xi∂xjdxidxj
∑
i,j∂2g2
∂xi∂xjdxidxj
)
, (22.78)
where the rules for evaluating dxidxj are dtdt = dtdBk = dBkdt = 0, and dBidBj = δi,jdt.
(b.) Properties of the Bivariate normal distribution. This follows Schaum’s outline ([26],
pp. 88-89). A bivariate random variable (X,Y ) has pdf
fXY (x, y) =1
2πσxσy
√
1 − ρ2exp
(
−1
2q(x, y)
)
, (22.79)
where
q(x, y) =1
1 − ρ2
(
(
x− µx
σx
)2
− 2ρ
(
x− µx
σx
)(
y − µy
σy
)
+
(
y − µy
σy
)2)
, (22.80)
and ρ is the coefficient of correlation
ρ =Cov(X,Y )
σxσy=E(XY ) − E(X)E(Y )
σxσy. (22.81)
Kuske et al use the following properties in their analysis of the (A,B) amplitude sytem:
228
• (i) If ρ = 0 then X and Y are uncorellated.
• (ii) If ρ = 0 then X and Y are independent.
The N-variate normal distribution. Let The random variable X = (X1, .., XN )′ is an “n-variate
normal random variable” if its joint pdf is
fX(x) =1
(2π)N/2|detK|1/2exp
[
−1
2(x− µ)TK−1(x − µ)
]
, (22.82)
where µ is a vector representing normal mean. The matrix K = (σij) = (cov(XiXj) is the covariance
matrix. In 2 dimensions, if X = (x, y), then
K =
[
σ2x ρσxσy
ρσxσy σ2y
]
and K−1 =1
1 − ρ2
1σ2
x
−ρσxσy
−ρσxσy
1σ2
y
,
and (22.82) reduces to (22.79)-(22.80).
(c.) Properties of the Ornstein-Uhlenbeck process.
Type Ornstein-Uhlenbeck process in google. The wikepedia site gives the following: The Ornstein-
Uhlenbeck process is a stochastic process Xt given by the SDE
dXt = −θ(Xt − µ)dt+ σdWt, (22.83)
where θ, µ, σ are parameters and Wt is the Wiener process. According to Doering ([4], p. 23) the
function dXt
dt represents the acceleration of a particle in some medium subject to a frictional retarding
force proportional to its velocity, −θXt, and a rapidly fluctuating force due to collisions with the
’fluid” particles in the medium.
Let f(X, t) = Xte−θt and use Ito’s lemma to get
df = eθtθµdt+ σeθtdWt. (22.84)
Integration gives
Xteθt = X0 +
∫ t
0
eθtθµds+ σ
∫ t
0
eθsdWs, (22.85)
hence
Xt = X0e−θt + µ(1 − e−θt) + σ
∫ t
0
eθ(s−t)dWs, (22.86)
This implies that the first moment of Xt is
E(Xt) = X0e−θt + µ(1 − e−θt) → µ as t→ ∞. (22.87)
The covariance is (Arnold, p.134)
cov(XsXt) = E [(Xt − E(Xt))(Xs − E(Xs))] = e−θ(t+s)
(
var(X0) +σ2
2θ
(
e2θmint,s − 1)
)
,
(22.88)
229
which reduces to
cov(XsXt) = e−θ(t+s)var(X0) +σ2
2θ
(
e−θ|t−s| − e−θ(t+s))
(22.89)
Gardner ([11], p. 77) notes that Xt and Xs are maximally corellated when |t− s| = 1θ , this difference
being called the ”corellation time.”
The variance of Xt is bounded, i.e.
var(Xt) = cov(XtXt) = e−2θtvar(X0) +σ2
2θ
(
1 − e−θ(2t))
→ σ2
2θas t→ ∞. (22.90)
By way of comparison, the Wiener process Wt has unbounded variance
var(Wt) = σ2t→ ∞ as t→ ∞. (22.91)
If we assume that Xs = y and µ = 0 then Doering ([4], pp. 23-24) notes that the associated Fokker-
Planck equation is
∂ρ(x, t|y, s)∂t
=
[
θ∂ρ(x, t|y, s)
∂xx+
σ2
2
∂2ρ(x, t|y, s)∂x2
]
ρ(x, t|y, s), (22.92)
and ρ(x, t|y, s) is given by
ρ(x, t|y, s) =1
√
2πΣ(t− s)exp
(
− (x− ye−θ(t−s))2
2Σ(t− s)
)
, (22.93)
where
Σ(t− s) =σ2
2θ
(
1 − e−2θt)
. (22.94)
The “stationary state solution” ρs(x) can be obtained from the following limit:
ρstat(x) = lims→−∞
ρ(x, t|y, s) =
√
θ
πσ2exp
(
−θx2
σ2
)
. (22.95)
This solution can be obtained by setting ∂ρ(x,t|y,s)∂t = 0 in (22.93) and solving the resultant ODE.
The “covariance in the stationary state” is given by
cov(XsXt) =σ2
2θ
(
e−θ|t−s|)
(22.96)
Systems. According to Arnold, p. 133, we have the following theorem which is what I think Kuske
et al are referring to on p. 9 of their paper:
Theorem. The solution of the multivariate equation
dXt = (A(t)Xt + a(t)) dt+B(t)dWt, X0 = c (22.97)
is a stationary gaussian process if A(t) ≡ A, a(t) ≡ 0, B(t) ≡ B, the eignevalues of A have negative
real parts, c is N(0,K) (i.e. c is n-variate normally distributed), where
K =
∫ ∞
0
eAtBB′eA′tdt. (22.98)
230
Question 2. Kuske et al state that ” the one-time-point stationary laws of the Ornstein-
Uhlenbeck process can be identified as a bivariate normal distribution, N(0,K) with mean
zero and covariance matrix
K = cov(A(T ), B(T ))′
where
K =
∫ ∞
0
exp
[(
−1 0
0 −1
)
s
]
ΓΓ′
(
−1 0
0 −1
)′
s
ds
How do they know this? Are they using Ludwig Arnold’s theorem? The outcome of
this computatiuon gives a diagonal matrix for K, which I think means that A and B are
uncorlellated, and also independent because they have the special property that they are
bivariate normally distributed. I think that they are using Arnold’s theorem to get the covariant
matrix K, then they show that K is diagonal, hence the coefficient of variation ρ = 0. This means
that A and B are uncorellated. Then use the theorem (Rachel says it is on p. 110 of Gardner [11])
that says that the pdf of A and B ais the binormal distribution. Once we know this, and the fact that
ρ = 0, we can conclude that the bivariate normal pdf, becomes a product of normal pdf’s, hence A
and B are independent.
(e.) Computing Power Spectral Density (PSD). First, an example ( see rxoftau1.m) - compute
the PSD if the autocorellation function is
y = .5 ∗ exp(−|tau|), (22.99)
The PSD of y is the Fourier transform
S(f) =
∫ ∞
−∞.5 ∗ exp(−|tau|)e−2πifτdτ. (22.100)
Note that
S(0) =
∫ ∞
−∞.5 ∗ exp(−|tau|)dτ = 1. (22.101)
The key seems to be to let the nimber of t values be about the same as n, the number of frequency
points The program rxoftau1.m figures out the PSD. In matlab type rxoftau1
- see powerspectrum.m for the basic matlab example
An idea for Rachel’s paper is to use dt=.6, like is used in powerspectrum.m and integrate for 512
points past t=100 out to t=300.
231
References
[1] R. Benzi, A. Sultera & A. Vulpani The mechanism of stochastic resonance. J. Phys. A:
Math. Gen. Vol. 14 (1981), L453-L457 (see benzipaper1.pdf)
[2] J. de la Rocha, B. Doiron, E. S. Brown, K. Jpsoc & A. Reyes Correlation between
neural spike trains increrases ith firing rate systems. Nature (2007)
[3] R. Deville, E. Vanden-Eijinden & C. Muratov Two distinct mechanisms of coherence in
randomly perturbed dynamical systems. PRE Vol. 72 (2005), No. 031105 (see Deville.pdf)
[4] C. R. Doering Modelling complex systems: stochastic processes, stochastic differential equations,
and Fokker-Planck equations. 1990 Lectures in Complex Systems, SFI Studies in the Sciences of
Complexity, Leture Vol. III, L. Nadel and D. Stein eds., Addison Wesley 1991
[5] C. Doering, C. Mueller & P. Smerka Interacting particles, the stochastic Fisher-
Kolmogorov-Piscounov equation, and duality. Physica A 325 (2003), pp. 243-259
[6] B. Doiron, J. Rinzel & A. D. Reyes Stochastic synchronization in finite size spiking networks.
and duality. Phys Rev. E 74 (2006), pp. 243-259
[7] B. Doiron Private communication.
[8] L. Edelstein-Keshet Mathematical Models in Biology. Random House 1987
[9] I. Ekeland The best of all worlds. U. of Chicago Press 2006
[10] W. I. Freidlin & A. D. Wentzell Random Perturbations of Dynamical Systems Springer-
Verlag 1984
[11] G. W. Gardner handbook of Stochastic Methods. Springer 2003
[12] D. J. Higham An algorithmic approach to numerical simulation of Stochastic Differential equa-
tions. SIAM Review Vol. 43, no. 3 (2001), pp. 525-546
[13] R. Kuske, L. F. Cordilla & P. Greenwood Sustained oscillations via coherence in SIR
Jour. Theor. Biology, Vol. 245, No. 3 (2007), pp. 459-469
[14] R. C. Lewontin & D. Cohen On population growth in a randomly varying environemnt.
PNAS Vol. 62, No. 4 (1969), pp. 1056-1060
[15] R. Maier Large Fluctuations in stochastically nonlinear systems: applications in computing.
Santa fe lecture - santafeweVol. 63, no. 4 (2003), pp. 498-504
[16] P. D. Monahan A primer for SAD. Europhysics Letters Vol. 63, no. 4 (2003), pp. 498-504
232
[17] R. S. Maier Large fluctuations in stochastically perturbed nonliear systems: applications in
computing. Notes for a talk given at the Santa Fe Institute 1999 (see santafewentzellnotes.pdf)
[18] R. S. Maier LExponentially rare large fluctuatiosn: computing their frequency. Notes for a talk
given at U. texas (see mier2.pdf)
[19] R. Maier & D. Stein Transition-rate theory for non-gradient drift fields. Phys. Rev. Lett. 69
(1992), pp. 3691-3695
[20] M. Matia & P. D. Giudic Population dynamics of interacting spiking neurons. Phys. Rev. E.
66 (2002), pp. 051917-1 to 051917-19
[21] B. Matkowsky, Z. Schuss, C. Knessl, C. Tier & M. Mangel Asymptotic solution of the
Kramers-Moyal equation and first passage times for markov jump processes. Phys. Rev. A. 29
(1984), pp. 3359-3369
[22] T. Naeh, M. Klosek, B. Matkowsky & Z. Schuss A direct approach to the exit problem.
SIAM Appl. Math. 50 (1990), pp. 595-627
[23] B. Oksendal Stochastic Differential Equations. Springer, 4th edition
[24] Oswald Spasre connectivity of pyramidal cells
[25] W. J. Reed The Pareto, Zipf and other power laws Economics Letters Vol. 74 (2001), pp. 15-19
[26] P. D. Sardeshmukh, C. Penland & M. Newman Drifts induced by multiplicative red noise
with application to climnate. Europhysics Letters Vol. 63, no. 4 (2003), pp. 498-504
[27] Schaum’s Outline: Probability, Random Variables and random Processes
[28] S. Strogatz Nonlinear Dynamics and Chaos, Westview Press (1994)
233