Poisson Convergence - ULisboa · Poisson Convergence Jo~ao Brazuna Weak Law of Small Numbers Basic Limit Theorem - A Simple Approach Basic Limit Theorem - A Formal Approach Examples

PoissonConvergence

Joao Brazuna

Weak Law ofSmallNumbers

Basic LimitTheorem - ASimple Approach

Basic LimitTheorem - AFormalApproach

Examples

Generalizationto PoissonProcess

Weak Law ofSmall Numbers -Generalization

Poisson Process

Poisson Convergence

Joao Brazuna

Probability TheoryInstituto Superior Tecnico

December 19, 2016

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents

1 Weak Law of Small NumbersBasic Limit Theorem - A Simple ApproachBasic Limit Theorem - A Formal ApproachExamples

2 Generalization to Poisson ProcessWeak Law of Small Numbers - GeneralizationPoisson Process

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents



PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Weak Law of Small Numbers

Theorem - Weak Law of Small Numbers

Events with low frequency in a large population follow aPoisson distribution even when the probabilities of the eventsvaried.

Also known as law of rare events, it was proposed by LadislausJosephovich Bortkiewicz in 1898.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

What do we know?

Bernoulli Distribution

X ∼ Bernoulli(p)⇔ P(X = x) =

{p, if x = 1

1− p, if x = 0

where p is the probability of success of a Bernoulli trial.

Binomial Distribution

If X1, · · · ,Xn:

are independent Bernoulli trials;

have a constant probability of success p,

then

S =n∑

j=1

Xj ∼ Binomial(n, p)

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

What are we going to see?

Bernoulli Distribution

X ∼ Bernoulli(p)⇔ P(X = x) =

{p, if x = 1

1− p, if x = 0

where p is the probability of success of a Bernoulli trial.

Binomial Distribution

If X1, · · · ,Xn:

are independent Bernoulli trials;

have eventually different small probabilities of success pj ,

then

S =n∑

j=1

Xja∼ Poisson

n∑j=1

pj

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Bernoulli, Binomial and Poisson Distributions

Bernoulli and Binomial Distributions

Xj ∼i .i .d .

Bernoulli(p)⇒ S =n∑

j=1Xj ∼ Binomial(n, p)

Bernoulli and Poisson Distributions

Xj ∼indep.

Bernoulli(pj)⇒ S =n∑

j=1Xj

a∼ Poisson

(n∑

j=1pj

)

Corollary

S ∼ Binomial(n, p)⇒ Sa∼ Poisson(np)

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents



PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Basic Limit Theorem - A Formal Approach

Notation - Random Variables

Let us consider a sequence of random variables{Xn,j

}such

that n ∈ N and j ∈ {1, · · · , n}.This means that for every n ∈ N we have n random variables

Xn,1, Xn,2, · · · ,Xn,n

since the index j is a natural number between 1 and n.

Notation - Probability of Success

pn,j = P(Xn,j = 1)

Notation - Sum of Interest (also a sequence)

Sn = Xn,1 + Xn,2 + · · ·+ Xn,n =n∑

j=1Xn,j

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Basic Limit Theorem

Weak Law of Small Numbers

For each n ∈ N, let Xn,j , with j ∈ {1, · · · , n} be independentrandom variables such that

P(Xn,j = 1) = pn,j ;P(Xn,j = 0) = 1− pn,j .

which means that Xn,j ∼ Bernoulli(pn,j).Suppose that, as n→∞:

1∑n

j=1 pn,j → λ ∈ R+ ;

2 maxj∈{1,··· ,n} pn,j → 0 .

Then,

Sn = Xn,1 + · · ·+ Xn,nd→ Z

whereZ ∼ Poisson(λ).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Auxiliary Lemma 1

Lemma

Let z1, · · · , zn,w1, · · · ,wn ∈ C with modulus not greater thanρ. Then, ∣∣∣∣∣ n∏

j=1zj −

n∏j=1

wj

∣∣∣∣∣ ≤ ρn−1n∑

j=1

∣∣zj − wj

∣∣ .Proof

By induction over n.

For n = 1:|z1 − w1| = ρ1−1|z1 − w1| .

For n > 1:

Induction Hypothesis:

∣∣∣∣∣ n∏j=1

zj −n∏

j=1wj

∣∣∣∣∣ ≤ ρn−1n∑

j=1

∣∣zj − wj

∣∣for a certain n ∈ N.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Auxiliary Lemma 1 - Proof

Proof (cont.)

To prove:

∣∣∣∣∣n+1∏j=1

zj −n+1∏j=1

wj

∣∣∣∣∣ ≤ ρn n+1∑j=1

∣∣zj − wj

∣∣ for the same n.

∣∣∣∣∣∣n+1∏j=1

zj −n+1∏j=1

wj

∣∣∣∣∣∣ =

∣∣∣∣∣∣zn+1

n∏j=1

zj − wn+1

n∏j=1

wj

∣∣∣∣∣∣ =

=

∣∣∣∣∣∣zn+1

n∏j=1

zj − zn+1

n∏j=1

wj + zn+1

n∏j=1

wj − wn+1

n∏j=1

wj

∣∣∣∣∣∣ ≤≤

∣∣∣∣∣∣zn+1

n∏j=1

zj − zn+1

n∏j=1

wj

∣∣∣∣∣∣+

∣∣∣∣∣∣zn+1

n∏j=1

wj − wn+1

n∏j=1

wj

∣∣∣∣∣∣ =

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Proof (cont.)

=|zn+1|

∣∣∣∣∣∣n∏

j=1

zj −n∏

j=1

wj

∣∣∣∣∣∣+|zn+1 − wn+1|n∏

j=1

∣∣wj

∣∣ I .H.≤≤ρ ρn−1

n∑j=1

∣∣zj − wj

∣∣+|zn+1 − wn+1| ρn = ρnn+1∑j=1

∣∣zj − wj

∣∣�

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Auxiliary Lemma 2

Lemma

If z ∈ C with |z | ≤ 1 then∣∣ez − (1 + z)∣∣ ≤ |z |2.

Proof

Using Taylor series,

ez = 1 + z +z2

2!+

z3

3!+

z4

4!+ · · · =

∞∑n=0

zn

n!⇔

⇔ez − (1 + z) =z2

2!+

z3

3!+

z4

4!+ · · · =

∞∑n=2

zn

n!.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Proof (cont.)

Since |z | ≤ 1, we have|z |n ≤|z |2 , ∀n ∈ {2, · · · } .

So, ∣∣ez − (1 + z)∣∣ =

∣∣∣∣∣z2

2+

z3

3!+

z4

4!+ · · ·

∣∣∣∣∣ =

=

∣∣∣∣∣z2

2

∣∣∣∣∣∣∣∣∣∣1 +|z |3

+|z |2

4× 3+ · · ·

∣∣∣∣∣ ≤≤|z |

2

2

(1 +

1

2+

1

22+ · · ·

)=|z |2

2× 2 =

=|z |2 .

�

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Basic Limit Theorem - Proof

We are going to prove the result using Levy’s ConvergenceTheorem, which states that if

ϕSn(t) →n→∞

ϕZ (t)

for all t ∈ R and ϕZ is continuous at t = 0 then

Snd→ Z .

For each r.v. Xn,j let ϕn,j be its characteristic function.

ϕn,j(t) = E(e itXn,j

)=

= e it×1P(Xn,j = 1) + e it×0P(Xn,j = 0) =

= e itpn,j + 1× (1− pn,j) =

= (1− pn,j) + pn,jeit

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


So, Sn’s characteristic function is

ϕSn(t) = E(e itSn

)= E

(e it

∑nj=1 Xn,j

)=

= E

n∏j=1

e itXn,j

=indep.

n∏j=1

E(e itXn,j

)=

=n∏

j=1

ϕn,j(t) =n∏

j=1

[(1− pn,j) + pn,je

it]

=

=n∏

j=1

[1 + pn,j

(e it − 1

)].

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Remember that if Z ∼ Poisson(λ), its probability massfunction is given by

P(Z = z) =

{e−λ λ

z

z! , if z ∈ N0

0, otherwise

so its characteristic function is

ϕZ (t) = E(e itZ

)=∞∑z=0

e itzP(Z = z) =

=∞∑z=0

e itze−λλz

z!= e−λ

∞∑z=0

(λ e it

)zz!

=

= e−λ × eλ eit

= eλ(e it−1).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


To prove thatϕSn(t) →

n→∞ϕZ (t)

we are going to show that∣∣ϕSn(t)− ϕZ (t)∣∣ →n→∞

0.

Consider a sequence of r.v.

Yn ∼ Poisson

n∑j=1

pn,j

so that

ϕYn(t) = e∑n

j=1 pn,j(e it−1).

Since, by assumption,∑n

j=1 pn,j →n→∞ λ, we have

ϕYn(t)→ ϕZ (t)

so, by Levy’s Convergence Theorem,

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Ynd→ Z .

∣∣ϕSn(t)− ϕYn(t)∣∣ =

∣∣∣∣∣∣n∏

j=1

[1 + pn,j

(e it − 1

)]− e

∑nj=1 pn,j(e it−1)

∣∣∣∣∣∣ =

=

∣∣∣∣∣∣n∏

j=1

[1 + pn,j

(e it − 1

)]−

n∏j=1

epn,j(eit−1)

∣∣∣∣∣∣Note that

0 ≤ pn,j ≤ 1 (it’s a probability);∣∣∣e it∣∣∣ = 1 (it defines a unitary circle);∣∣∣∣1 + pn,j

(e it − 1

)∣∣∣∣ ≤ 1;∣∣∣epn,j(e it−1)∣∣∣ = e

pn,j

[Re(e it)−1

]≤ 1 (exponent ≤ 0).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Using auxiliary lemmas 1 and 2, with maxj pn,j ≤ 12 and∣∣∣e it − 1

∣∣∣ ≤ ∣∣∣e it∣∣∣+ 1 ≤ 1 + 1 = 2,∣∣∣∣∣∣n∏

j=1

[1 + pn,j

(e it − 1

)]−

n∏j=1

epn,j(eit−1)

∣∣∣∣∣∣ L1≤

≤n∑

j=1

∣∣∣∣1 + pn,j

(e it − 1

)− epn,j(e

it−1)∣∣∣∣ =

=n∑

j=1

∣∣∣∣∣epn,j(e it−1) −[

1 + pn,j

(e it − 1

)]∣∣∣∣∣ L2≤

≤n∑

j=1

∣∣∣∣pn,j (e it − 1)∣∣∣∣2 =

n∑j=1

p2n,j

∣∣∣e it − 1∣∣∣2 ≤

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


≤n∑

j=1

pn,j maxj

{pn,j}

22 = 4 maxj

{pn,j} n∑

j=1

pn,j → 0.

Sincen∑

j=1pn,j → λ and ϕYn(t)→ ϕZ (t) as n→∞,

∣∣ϕSn(t)− ϕZ (t)∣∣ =∣∣ϕSn(t)− ϕYn(t) + ϕYn(t)− ϕZ (t)

∣∣ ≤≤∣∣ϕSn(t)− ϕYn(t)

∣∣+∣∣ϕYn(t)− ϕZ (t)

∣∣→ 0 + 0 = 0

we conclude thatϕSn → ϕZ

so, by Levy’s Convergence Theorem,

Snd→ Z .

�

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents



PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Some Examples

Poisson Convergence.nb

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

World War II Example

Let Un,j ∼indep.

Unif [−n, n],

Xn,j =

{1, if Un,j ∈]a, b[⊆ [−n, n]

0, otherwise

and Sn =n∑

j=1Xn,j the number of points that land on ]a, b[.

pn,j = P(Xn,j = 1) = P(Un,j ∈]a, b[) =

∫ b

a

1

2ndu =

b − a

2n∑nj=1 pn,j = n × b−a

2n = b−a2 ∈ R+ does not depend on n;

maxj∈{1,··· ,n} pn,j = maxj∈{1,··· ,n}b−a2n = b−a

2n → 0.

So

Snd→ Z ∼ Poisson

(b − a

2

),

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

World War II Example

The flying bomb hits in South London during World War II fita Poisson distribution.The area was divided on 576 regions with the same area. Thetotal number of hits was 537. Adapting our last result for R2,let S be the number of regions with k hits.

Sa∼ Poisson

(537

576

)k 0 1 2 3 4 ≥ 5

576× P(S = k) 226.74 211.39 98.54 30.62 7.14 1.57Real Values 229 211 93 35 7 1

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents



PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Weak Law of Small Numbers - Generalization

Theorem - Weak Law of Small Numbers

For each n ∈ N, let Xn,j , with j ∈ {1, · · · , n} be independentnon-negative integer valued random variables such that

P(Xn,j = 1) = pn,j ;P(Xn,j ≥ 2) = εn,j .

Suppose that, as n→∞:

1∑n

j=1 pn,j → λ ∈ R+ ;

2 maxj∈{1,··· ,n} pn,j → 0 ;

3∑n

j=1 εn,j → 0 .

Then,

Sn = Xn,1 + · · ·+ Xn,nd→ Z

whereZ ∼ Poisson(λ).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Proof

Let

X ′n,j =

{1, if Xn,j = 1

0, otherwise

and

S ′n = X ′n,1 + · · ·+ X ′n,n =n∑

j=1

X ′n,j .

By the first two assumptions and using the Weak Law of SmallNumbers, we conclude that

S ′nd→ Z .

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Proof (cont.)

By the third assumption, we know that

P(Sn 6= S ′n)→ 0.

Using the Converging Together Lemma,

Snd→ Z .

�

PoissonConvergence

Joao Brazuna




Examples



Poisson Process


Converging Together Lemma

If

Xnd→ X ;

Ynd→ c where c ∈ R is a constant,

thenXn + Yn

d→ X + c .

Corollary

If

S ′nd→ Z ;

Sn − S ′nd→ 0 ,

thenSn

d→ Z .

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Contents



PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Definition - Counting Process (in Continuous Time)

A family of r.v.{N(t) : t ≥ 0

}is said to be a counting process

if it represents the total number of events that have occurredup to time t. It must satisfy:

N(t) ∈ N0, ∀t ≥ 0 ;

N(s) ≤ N(t), ∀0 ≤ s < t ;

N(t)− N(s) corresponds to the number of events thatoccurred in the interval ]s, t], ∀0 ≤ s < t.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Definition - Counting Process with Independent Increments

The counting process{N(t) : t ≥ 0

}is said to have

independent increments if the number of events that occur indisjoint intervals are independent r.v., that is,if 0 = t0 < t1 < · · · < tn thenN(t1), N(t2)− N(t1),..., N(tn)− N(tn−1) are independent r.v.

Definition - Counting Process with Stationary Increments


}is said to have stationary

increments if the distribution of the number of events thatoccur in any interval depends only on the length of the interval,that is,

N(t2 + s)− N(t1 + s)d= N(t2)− N(t1) , ∀s ≥ 0, 0 ≤ t1 < t2.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Definition 1 - Poisson Process


}is said to be a Poisson

process with rate λ > 0 if:{N(t) : t ≥ 0

}has independent and stationary

increments;

N(t) ∼ Poisson(λt).

This is the usual definition, but it is redundant since the secondcondition follows from the first one.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Definition 2 - Poisson Process


}is said to be a Poisson

process with rate λ > 0 if:

N(0) = 0;{N(t) : t ≥ 0


increments;

P[N(h) = 1] = λh + o(h);

P[N(h) ≥ 2] = o(h)

where o(h) stands for functions g1(h) and g2(h) such that

limh→0

gi (h)h = 0, ∀i ∈ {1, 2}.

Let us check that this definition implies that

N(t) ∼ Poisson(λt).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Theorem

Let{N(t) : t ≥ 0

}be a counting process in continuous time.

Suppose that:

N(0) = 0;{N(t) : t ≥ 0


increments;

P[N(h) = 1] = λh + o(h);

P[N(h) ≥ 2] = o(h)

where o(h) stands for functions g1(h) and g2(h) such that

limh→0

gi (h)h = 0, ∀i ∈ {1, 2}.

Then,N(0, t) ∼ Poisson(λt).

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Proof

LetXn,j = N

(jn t)− N

(j−1n t).

pn,j = P(Xn,j = 1) = P

[N

(j

nt

)− N

(j − 1

nt

)= 1

]=

= P

[N

(t

n

)− N(0) = 1

]= P

[N

(t

n

)= 1

]=

= λ× t

n+ o

(t

n

)n∑

j=1

pn,j =n∑

j=1

P(Xn,j = 1) =n∑

j=1

[λ× t

n+ o

(t

n

)]→ λt.

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Proof (cont.)

maxj∈{1,··· ,n}

pn,j = maxj∈{1,··· ,n}

{λ× t

n + o(tn

)}= pn,j

because it does not depend on j . Therefore, as n→∞,

maxj∈{1,··· ,n}

pn,j = λ× t

n+ o

(t

n

)→ 0.

Finally,

εn,j = P(Xn,j ≥ 2) = P

[N

(j − 1

nt

)− N

(j

nt

)≥ 2

]=

= P

[N

(t

n

)− N(0) ≥ 2

]= o

(t

n

)→ 0

PoissonConvergence

Joao Brazuna




Examples



Poisson Process

Poisson Process

Proof (cont.)

thusn∑

j=1εn,j → 0.

We are on the conditions of the last theorem, since Xn,j arenon-negative integer valued r.v. with pn,j = P(Xn,j = 1),εn,j = P(Xn,j ≥ 2) and

1∑n

j=1 pn,j → λt ∈ R+ ;

2 maxj∈{1,··· ,n} pn,j → 0 ;

3∑n

j=1 εn,j → 0 .

Therefore,n∑

j=1

Xn,jd→ N(t) ∼ Poisson(λt).

�

PoissonConvergence

Joao Brazuna

Appendix

Bibliography

Bibliography I

Rick Durrett.Probability Theory and Examples.CUP, 2013.

Isabel Rodrigues.Apontamentos de Complementos de Probabilidades eEstatıstica.Instituto Superior Tecnico, 2015.

Manuel Cabral Morais.Lecture Notes - Probability Theory .Instituto Superior Tecnico, 2014.

Manuel Cabral Morais.Lecture Notes - Stochastic Processes.Instituto Superior Tecnico, 2014.

https://fenix.tecnico.ulisboa.pt/downloadFile/1974104408195718/2014-12-22-LectureNotesOnProbabilityTheoryA4(110).pdf

https://fenix.tecnico.ulisboa.pt/downloadFile/3779580682389/2014-06-10-LectureNotesStochasticProcesses-ENG(chaps0-4)110.pdf

Documents

Poisson Convergence - ULisboa · Poisson Convergence Jo~ao Brazuna Weak Law of Small Numbers Basic Limit Theorem - A Simple Approach Basic Limit Theorem - A Formal Approach Examples