View
20
Download
0
Category
Preview:
Citation preview
Mathematical Statistics
MAS 713
Chapter 3.2
Previous lectures
1 Random variables2 Discrete Random Variables3 Continuous Random Variables4 Expectation of a random variable5 Variance of a random variable
Questions?
Mathematical Statistics (MAS713) Ariel Neufeld 2 / 66
This lecture
3.2. Special random variables3.2.1 Introduction3.2.2 Binomial random variables3.2.3 Hypergeometric random variables3.2.4 Poisson random variables3.2.5 Uniform random variables3.2.6 Exponential random variables
Additional reading : Chapter 3 in the textbook
Mathematical Statistics (MAS713) Ariel Neufeld 3 / 66
3.2. Special random variables 3.2.1 Introduction
Introduction
In practice, we are often running same kind of (random) experiments,from which we are often interested in the same kind of results
; it turns out that certain “types” of random variables come up overand over again in applications
In this chapter, we’ll study a variety of those special random variables
You can also go to
http://www.socr.ucla.edu/htmls/dist/
and have a look at the numerous ‘special’ distributions there
Mathematical Statistics (MAS713) Ariel Neufeld 4 / 66
3.2. Special random variables 3.2.1 Introduction
Exponential Family
Mathematical Statistics (MAS713) Ariel Neufeld 5 / 66
3.2. Special random variables 3.2.1 Introduction
Exponential Family - BackgroundDefinition
A family of discrete probabilities (Pθ)θ is an exponential family if eachof its probability mass function p(x ;θ) can be represented in thefollowing General Form:
p(x ;θ) = h(x)g(θ) exp(θT u(x)
)A family of continuous probabilities (Pθ)θ is an exponential family if
each of its probability density function f (x ;θ) can be represented in thefollowing General Form:
f (x ;θ) = h(x)g(θ) exp(θT u(x)
)θ - vector of the natural parameters
u(x),h(x) - some function of x .
g(θ) - normalizes the distribution g(θ)∫
h(x) exp(θT u(x)
)dx = 1
Mathematical Statistics (MAS713) Ariel Neufeld 6 / 66
3.2. Special random variables 3.2.1 Introduction
Example: Bernoulli Distribution -
Bernoulli(x |π) := pBer (x ;π) = πx(1− π)1−x
= exp (x lnπ + (1− x) ln(1− π))= exp (x (lnπ − ln(1− π)) + ln(1− π))
= exp
(x ln
(π
1− π
))exp (ln(1− π))
= (1− π) exp(ln
(π
1− π
)x)
Setting θ := ln(
π1−π
)which gives the logistic sigmoid function
π = σ(θ) := 11+exp(−θ)
u(x) = x ; h(x) = 1; g(θ) = σ(−π).
=⇒ pBer (x ; θ) = h(x)g(θ) exp(θT u(x)
)Mathematical Statistics (MAS713) Ariel Neufeld 7 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF DISCRETE MEMBERS:
Bernoulli: p(x ;π) = πx (1− π)1−x x ∈ {0,1}Binomial: p(x ;n, π) = Cn
r πx(1− π)n−x x ∈ {0,1, . . . ,n}
Multinomial: p(x ;π) = n!x1!x2!...xn!
∏ni=1 π
xii xi ∈ {0,1, . . . ,n}
Poisson: p(x ;λ) = λx
x! e−λ x ∈ N0
Dirichlet: p(x ;π) =Γ(
∑i πi)∏
i Γ(πi )
∏i xπi−1
i x ∈ [0,1],∑
i xi = 1
Mathematical Statistics (MAS713) Ariel Neufeld 8 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF DISCRETE MEMBERS:
Binomial Model
- used to model the number of successes in a sequence of nindependent Bernoulli yes/no events.
Mathematical Statistics (MAS713) Ariel Neufeld 9 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF DISCRETE MEMBERS:
Multinomial Model
- analog of the Bernoulli distribution for the categorical distribution- each trial results in exactly one of some fixed finite number k ofpossible outcomes, with probabilities p1, . . . ,pk .- There are still n - independent trials.
Mathematical Statistics (MAS713) Ariel Neufeld 10 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF DISCRETE MEMBERS:
Poisson Model
- captures the probability of the number of events occurring in a fixedperiod of time.- It assumes these events occur with a known average rate andindependently of the time since the last event.- It can also be used for the number of events in intervals such asdistance, area or volume.
Mathematical Statistics (MAS713) Ariel Neufeld 11 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF CONTINUOUS MEMBERS:
Gaussian: p(x ;µ, σ) = 1√2πσ2 exp
(− (x−µ)2
2σ2
)x ∈ R
Inverse Normal: p(x ;λ, µ) =√
λ2πx3 exp
(− λ
2µ2x (x − µ)2)
x ∈ R+
Exponential: p(x ;λ) = λ exp (−λx) x ∈ R+
Laplace: p(x ;a,b) = 12b exp(− |x−µ|b ) x ∈ R
Beta: p(x ;π, β) = yπ−1(1−x)β−1
B(π,β) x ∈ [0,1]
Gamma: p(x ; k , θ) = xk−1 exp(− xθ )
Γ(k)θk x ∈ R+
Mathematical Statistics (MAS713) Ariel Neufeld 12 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF CONTINUOUS MEMBERS:
Exponential Model
- describes the inter arrival time of events in a homogeneous Poissonprocess.
Used in practical models to approximate:
time it takes until a radio active particle decays.
time taken until received next phone call at exchange.
time until a default occurs in credit modeling, or loss is recorded inOperational Risk.
Mathematical Statistics (MAS713) Ariel Neufeld 13 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF CONTINUOUS MEMBERS:
Beta Model
- often used as a prior distribution on a random variable on the interval[0,1] - ie. a probability.
Mathematical Statistics (MAS713) Ariel Neufeld 14 / 66
3.2. Special random variables 3.2.1 Introduction
EXAMPLES OF CONTINUOUS MEMBERS:
Gamma Model
- often used as a distribution of waiting times or failure times.
Mathematical Statistics (MAS713) Ariel Neufeld 15 / 66
3.2. Special random variables 3.2.1 Introduction
Independent random variables
DefinitionRandom variables X and Y are said to be independent if and only iffor all (x , y) ∈ R× R,
P(X ≤ x ,Y ≤ y) = P(X ≤ x)× P(Y ≤ y)
X , Y indep. ⇐⇒ all couples of events (X ≤ x), (Y ≤ y) indep.
PropertyRandom variables X and Y are said to be independent if and only iffor all functions h, g
E(h(X )g(Y )) = E(h(X ))× E(g(Y ))
Example: X and Y independent =⇒ E(XY ) = E(X )E(Y ).
Mathematical Statistics (MAS713) Ariel Neufeld 16 / 66
3.2. Special random variables 3.2.2 Binomial random variables
ExamplesConsider the following random experiments and random variables :
flip a coin 10 times. Let X = number of heads obtained.a worn machine tool produces 1% defective parts.Let X = number of defective parts in the next 25 parts produced.each sample of air has a 10% chance of containing a particularmolecule.Let X = the number of air samples that contain the molecule inthe next 18 samples analysed.a multiple-choice test contains 10 questions, each with 4 choices,and you guess at each question.Let X = the number of questions answered correctly.
; similar experiments, similar random variables
; a general framework would be very useful
Mathematical Statistics (MAS713) Ariel Neufeld 17 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Binomial distribution
Mathematical Statistics (MAS713) Ariel Neufeld 18 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Binomial distributionAssume:
the outcome of a random experiment can be classified aseither a “Success” (S) or a “Failure” (F)(; S = {Success,Failure})we observe a Success with probability πn independent repetitions of this experiment are performed
Define X = number of successes observed over the n repetitions.We say that X is a binomial random variable with parameters n and π :
X ∼ Bin(n, π)
See that SX = {0,1,2, . . . ,n} and the binomial probability mass is
pBin(x) =(
nx
)πx(1− π)n−x for x ∈ SX
where(n
x
)is the number of different groups of x objects that can be
chosen from a set of n objectsMathematical Statistics (MAS713) Ariel Neufeld 19 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Binomial distributionNote : the coefficients
(nx
)= n!/(x!(n − x)!) are called the binomial
coefficients, they are the coefficients arising in the famous Newton’sbinomial expansion
(a + b)n =n∑
k=0
(nk
)akbn−k
These coefficients are often represented in the Pascal’s triangle,named after the French mathematician Blaise Pascal (1623-1662)
Mathematical Statistics (MAS713) Ariel Neufeld 20 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Binomial distribution : pmf and cdfBinomial pmf and cdf, for n = 5 and π = {0.1,0.2,0.5,0.8,0.9}
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
p = 0.1
x
p X(x
)
●
●
●
● ● ●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
p = 0.2
x
p X(x
)
●
●
●
●
● ●
−1 0 1 2 3 4 5 60.
00.
20.
40.
60.
81.
0
p = 0.5
x
p X(x
)●
●
● ●
●
●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
p = 0.8
x
p X(x
)
● ●
●
●
●
●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
p = 0.9
x
p X(x
)
● ● ●
●
●
●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
● ● ● ●
●
●
●
● ● ●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
●
● ● ●
●
●
●
●
● ●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
●
●
●
●
●
●
●
●
●
●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
● ●
●
●
●
●
● ● ●
●
●
●
−1 0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
● ● ●
●
●
●
● ● ● ●
●
●
Mathematical Statistics (MAS713) Ariel Neufeld 21 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Bernoulli distribution
Mathematical Statistics (MAS713) Ariel Neufeld 22 / 66
3.2. Special random variables 3.2.2 Binomial random variables
The Bernoulli distributionSpecial case Bin(1, π) ; the Bernoulli distribution,
X ∼ Ber(π)
pmf :
pBer (x) =
1− π if x = 0π if x = 10 otherwise
Note : if X ∼ Bin(n, π), we can represent it as
X =n∑
i=1
Xi
where Xi ’s are n independent Bernoulli r.v. with parameters π
; each repetition of the experiment in the Binomial framework iscalled a Bernoulli trial
Mathematical Statistics (MAS713) Ariel Neufeld 23 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : properties
First note that
∑x∈SX
p(x) =n∑
x=0
(nx
)πx(1− π)n−x = (π + (1− π))n = 1
using the binomial expansion
Second, it is easy to see thatif X1 ∼ Bin(n1, π), X2 ∼ Bin(n2, π) and X1 is independent of X2, then
X1 + X2 ∼ Bin(n1 + n2, π)
Mathematical Statistics (MAS713) Ariel Neufeld 24 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : expectation and variance
Remind the representation X =∑n
i=1 Xi , with Xi ∼ Bern(π)
We know that
E(Xi) = π and Var(Xi) = π(1− π)
It follows
E(X ) = E
(n∑
i=1
Xi
)=
n∑i=1
E(Xi) =n∑
i=1
π = nπ,
-
Var(X ) = Var
(n∑
i=1
Xi
)ind.=
n∑i=1
Var(Xi) =n∑
i=1
π(1− π) = nπ(1− π)
Mathematical Statistics (MAS713) Ariel Neufeld 25 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : expectation and variance
Mean and variance of the binomial distributionIf X ∼ Bin(n, π),
µ = E(X ) = nπ and σ2 = Var(X ) = nπ(1− π)
Mathematical Statistics (MAS713) Ariel Neufeld 26 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : examples
ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) In the long-run, what proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?
a) Let X be the number of defective disks in a package. Then, it is clear that
X ∼ Bin(10,0.01)
Hence P(X > 1) = 1− P(X = 0)− P(X = 1)
= 1−(
100
)0.0100.9910 −
(101
)0.0110.999 ' 0.005
; in the long-run, 0.5 percent of the packages will have to be replacedMathematical Statistics (MAS713) Ariel Neufeld 27 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : examples
ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) In the long-run, what proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?
a) Let X be the number of defective disks in a package. Then, it is clear that
X ∼ Bin(10,0.01)
Hence P(X > 1) = 1− P(X = 0)− P(X = 1)
= 1−(
100
)0.0100.9910 −
(101
)0.0110.999 ' 0.005
; in the long-run, 0.5 percent of the packages will have to be replacedMathematical Statistics (MAS713) Ariel Neufeld 27 / 66
3.2. Special random variables 3.2.2 Binomial random variables
ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) What proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?
b) Let Y be the number of packages that the person has to return. We have
Y ∼ Bin(3, π)
where π is the probability that a package is returned, that is, contains morethan 1 defective disk.
From a), we know that π = 0.005
Thus, the probability that exactly one of the three packages is returned equals
P(Y = 1) =(
31
)0.00510.9952 = 0.015
Mathematical Statistics (MAS713) Ariel Neufeld 28 / 66
3.2. Special random variables 3.2.2 Binomial random variables
ExampleIt is known that disks produced by a certain company will be defective withprobability 0.01 independently of each other. The company sells the disk inpackages of 10 and offers a money-back guarantee that at most 1 of thedisks is defective.a) What proportion of packages is returned?b) If someone buys three packages, what is the probability that exactly one ofthem will be returned?
b) Let Y be the number of packages that the person has to return. We have
Y ∼ Bin(3, π)
where π is the probability that a package is returned, that is, contains morethan 1 defective disk.
From a), we know that π = 0.005
Thus, the probability that exactly one of the three packages is returned equals
P(Y = 1) =(
31
)0.00510.9952 = 0.015
Mathematical Statistics (MAS713) Ariel Neufeld 28 / 66
3.2. Special random variables 3.2.2 Binomial random variables
ExampleSuppose that 10% of all bits transmitted through a digital communicationchannel are erroneously received and that whether any is erroneouslyreceived is independent of whether any other bit is erroneously received.Consider sending a large number of messages, each consisting of 20 bits.a) What proportion of these messages will have exactly 2 erroneously bits?b) What proportion of these messages will have at least 5 erroneously bits?c) For what proportion of these messages will more than half the bits beerroneously?
Let X be the number of erroneously received bits in a message of 20 bits.Clearly, we have X ∼ Bin(20,0.1). Thus we have
a) P(X = 2) =(20
2
)0.120.918 = 0.2852
b) P(X ≥ 5) = 1− P(X = 0)− P(X = 1)− P(X = 2)− P(X =3)− P(X − 4) = . . .
c) P(X > 10) = P(X = 11) + P(X = 12) + . . .+ P(X = 20) = . . .
; very tedious ! ; use statistical tables or a statistical software
Mathematical Statistics (MAS713) Ariel Neufeld 29 / 66
3.2. Special random variables 3.2.2 Binomial random variables
ExampleSuppose that 10% of all bits transmitted through a digital communicationchannel are erroneously received and that whether any is erroneouslyreceived is independent of whether any other bit is erroneously received.Consider sending a large number of messages, each consisting of 20 bits.a) What proportion of these messages will have exactly 2 erroneously bits?b) What proportion of these messages will have at least 5 erroneously bits?c) For what proportion of these messages will more than half the bits beerroneously?
Let X be the number of erroneously received bits in a message of 20 bits.Clearly, we have X ∼ Bin(20,0.1). Thus we have
a) P(X = 2) =(20
2
)0.120.918 = 0.2852
b) P(X ≥ 5) = 1− P(X = 0)− P(X = 1)− P(X = 2)− P(X =3)− P(X − 4) = . . .
c) P(X > 10) = P(X = 11) + P(X = 12) + . . .+ P(X = 20) = . . .
; very tedious ! ; use statistical tables or a statistical software
Mathematical Statistics (MAS713) Ariel Neufeld 29 / 66
3.2. Special random variables 3.2.2 Binomial random variables
Binomial distribution : examples
Mathematical Statistics (MAS713) Ariel Neufeld 30 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
The Hypergeometric distribution
Mathematical Statistics (MAS713) Ariel Neufeld 31 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
The Hypergeometric distributionConsider the following situation :
We are interested in the number of defectives in a sample of n unitsdrawn from a lot containing N units, of which k are defective
Careful: It can be tempting to regard X , the number of defectives inthe sample, as a Binomial random variable : n units are repeatedlyclassified as defective (‘Success’) or not defective (‘Failure’)
However, if the first drawing yields a defective with probability k/N, thesecond drawing does with probability (k − 1)/(N − 1) or k/(N − 1)(depending on the first drawing)
; the ‘trials’ are not independent, and the probability of successis not constant ; violation of the Binomial distribution assumptions
; X actually follows the so-called Hypergeometric distribution
X ∼ Hyp(N, k ,n)
Mathematical Statistics (MAS713) Ariel Neufeld 32 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric distribution : properties
See that SX = {max(0,n + k − N), . . . ,min(k ,n)} and thehypergeometric probability mass function is given by
p(x) =
(kx
)(N−kn−x
)(Nn
) for x ∈ SX
The mean and variance of the Hypergeometric distribution can bedetermined from the trials that comprise the experiment
However, the trials are not independent, and so the calculations amore difficult than for the Binomial.
Mathematical Statistics (MAS713) Ariel Neufeld 33 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric distribution : properties
Mean and variance of the hypergeometric distributionIf X ∼ Hyp(N, k ,n),
µ = E(X ) = nkN
and σ2 = Var(X ) = nkN
(1− k
N
)(N − nN − 1
)
Mathematical Statistics (MAS713) Ariel Neufeld 34 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric probability mass function
●
●
● ●
●
●
0 1 2 3 4 5
0.0
0.2
0.4
0.6
0.8
1.0
Hypergeometric pmf
x
p(x)
●
●
● ●
●
●
●
●
●● ● ●
●
●
●
(N,k,n)(10,5,5)(50,25,5)(50,3,5)
Mathematical Statistics (MAS713) Ariel Neufeld 35 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric distribution : remarksLet π = k
N , the initial proportion of ‘defectives’ in the full lot
Then,E(X ) = nπ and Var(X ) = nπ(1− π)N − n
N − 1Compared to the Binomial mean and variance, we see that the
dependence in successive trials affects the variance but not the mean
The trials that compromise the experiment would be independent ifwe did sampling with replacement, namely, if each unit selected forthe sample is replaced before the next one is drawn
Similarly, the trials would be independent if we were sampling froman infinite set (N =∞), as the proportion of defectives would remainconstant for every trial (k/(N − 1) ∼ k/N, (N − n)/(N − 1) ∼ 1)
; Binomial random variable
; when N is ‘large’, the Binomial distribution is a good approximationto the Hypergeometric distribution
Mathematical Statistics (MAS713) Ariel Neufeld 36 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric distribution : examples
ExampleThe components of a 6-component system are to be randomly chosen from abin of 20 used components. The resulting system will be functional if at least4 of its 6 components are in working condition.If 15 of the 20 components in the bin are in working condition, what is theprobability that the resulting system will be functional?
Let X be number of working components chosen. Then X ∼ Hyp(20,15,6)
The probability that the system will be functional is
P(X ≥ 4) = P(X = 4) + P(X = 5) + P(X = 6) =
(154
)(52
)+(15
5
)(51
)+(15
6
)(50
)(206
)' 0.8687
Mathematical Statistics (MAS713) Ariel Neufeld 37 / 66
3.2. Special random variables 3.2.3 Hypergeometric random variables
Hypergeometric distribution : examples
ExampleThe components of a 6-component system are to be randomly chosen from abin of 20 used components. The resulting system will be functional if at least4 of its 6 components are in working condition.If 15 of the 20 components in the bin are in working condition, what is theprobability that the resulting system will be functional?
Let X be number of working components chosen. Then X ∼ Hyp(20,15,6)
The probability that the system will be functional is
P(X ≥ 4) = P(X = 4) + P(X = 5) + P(X = 6) =
(154
)(52
)+(15
5
)(51
)+(15
6
)(50
)(206
)' 0.8687
Mathematical Statistics (MAS713) Ariel Neufeld 37 / 66
3.2. Special random variables 3.2.4 Poisson random variables
The Poisson distribution
Mathematical Statistics (MAS713) Ariel Neufeld 38 / 66
3.2. Special random variables 3.2.4 Poisson random variables
The Poisson distributionAssume you are interested in the number of occurrences of somerandom phenomenon in a fixed period of time
Define X = number of occurrences. We say that X is a Poissonrandom variable with parameter λ, i.e.
X ∼ P(λ) or X ∼ Pois(λ),
ifpPois(x) = e−λ
λx
x!for x ∈ SX = {0,1,2, . . .}
Note : Simeon-Denis Poisson(1781-1840) was a French
mathematician
Mathematical Statistics (MAS713) Ariel Neufeld 39 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : how does it arise?think of the time period of interest as being split up into a large number,say n, of subperiods
assume that the phenomenon could occur at most one time in each ofthose subperiods, with some common probability π
if what happens within one interval is independent to others,
X ∼ Bin(n, π)
now, as n increases, π should decrease (the shorter the period, the lesslikely the occurrence of the phenomenon) ; let π = λ/n for some λ > 0
then, for any x ∈ {0,1, . . . ,n},
P(X = x) =n!
x!(n − x)!
(λ
n
)x (1− λ
n
)n−x
=n!
nx(n − x)!(1− λ/n)xλx
x!
(1− λ
n
)n
Mathematical Statistics (MAS713) Ariel Neufeld 40 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : how does it arise?finally, as
n!nx(n − x)!(1− λ/n)x → 1 and
(1− λ
n
)n
→ e−λ
as n→∞, it remains
P(X = x) = e−λλx
x!for x ∈ {0,1, . . .}
which is the Poisson distribution
the Poisson distribution is thus suitable for modelling the number ofoccurrences of a random phenomenon satisfying some assumptions ofcontinuity, stationarity, independence and non-simultaneity
λ is called the intensity of the phenomenon
Note : we defined the P(λ) distribution by partitioning a time period, howeverthe same reasoning can be applied to any interval, area or volume
; the number of flaws in a new windshield might also be Poisson distributedMathematical Statistics (MAS713) Ariel Neufeld 41 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : pmf and cdf
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
lambda = 0.1
x
p X(x
)
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
lambda = 0.5
x
p X(x
)
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
lambda = 1
x
p X(x
)
● ●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
lambda = 2
x
p X(x
)
●
● ●
●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
lambda = 10
x
p X(x
)
● ● ● ●●
●
●
●●
● ●●
●●
●●
● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
●
●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●● ● ● ● ● ● ● ● ● ● ● ● ● ●
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
x
FX(x
)
● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●●
● ● ● ●
● ● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●
Poisson pmf and cdf, for λ = {0.1,0.5,1,2,10}
Mathematical Statistics (MAS713) Ariel Neufeld 42 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : properties
First we have∑x∈SX
p(x) =∞∑
x=0
e−λλx
x!= e−λ
∞∑x=0
λx
x!= e−λeλ = 1
Similarly,
E(X ) =∑
x∈SX
xp(x) =∞∑
x=0
xe−λλx
x!= λ
∞∑x=1
e−λλx−1
(x − 1)!= λ
E(X 2) =∑
x∈SX
x2p(x) = . . . = λ2 + λ
; Var(X ) = E(X 2)− (E(X ))2 = λ2 + λ− λ2 = λ
Mathematical Statistics (MAS713) Ariel Neufeld 43 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : properties
Mean and variance of the Poisson distributionIf X ∼ P(λ),
E(X ) = λ and Var(X ) = λ
Mathematical Statistics (MAS713) Ariel Neufeld 44 / 66
3.2. Special random variables 3.2.4 Poisson random variables
ExampleOver a 10-minute period, a counter records an average of 1.3 gammaparticles per millisecond coming from a radioactive substance. To a goodapproximation, the distribution of the count, X , of gamma particles during thenext millisecond is Poisson distributed. Determine:a) λ,b) the probability of oberving one or more gamma particles during the nextmillisecond andc) the variance of this number
a) The mean of the Poisson distribution is λ, so we can approximate λ by thelong-run average of the number of particles per millisecond, that is, λ ' 1.3.So we have
X ∼ P(1.3)b) Thus,
P(X ≥ 1) = 1− P(X = 0) = 1− e−1.3 1.30
0!= 0.727
c) The variance of the Poisson distribution is also equal to λ, hence
Var(X ) = 1.3 (particles2)Mathematical Statistics (MAS713) Ariel Neufeld 45 / 66
3.2. Special random variables 3.2.4 Poisson random variables
ExampleOver a 10-minute period, a counter records an average of 1.3 gammaparticles per millisecond coming from a radioactive substance. To a goodapproximation, the distribution of the count, X , of gamma particles during thenext millisecond is Poisson distributed. Determine:a) λ,b) the probability of oberving one or more gamma particles during the nextmillisecond andc) the variance of this number
a) The mean of the Poisson distribution is λ, so we can approximate λ by thelong-run average of the number of particles per millisecond, that is, λ ' 1.3.So we have
X ∼ P(1.3)b) Thus,
P(X ≥ 1) = 1− P(X = 0) = 1− e−1.3 1.30
0!= 0.727
c) The variance of the Poisson distribution is also equal to λ, hence
Var(X ) = 1.3 (particles2)Mathematical Statistics (MAS713) Ariel Neufeld 45 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : examplesExampleSuppose that the number of drivers who travel between a particular originand destination during a designated time period has Poisson distribution withparameter λ = 20.In the long-run, in what proportion of time periods will the number of driversa) be at most 10? b) exceed 20? c) be between 10 and 20, inclusive? Strictlybetween 10 and 20?
Let X be the number of drivers. It is given that X ∼ P(20)
a)
P(X ≤ 10) = P(X = 0) + P(X = 1) + . . .+ P(X = 10)
= e−20 + e−20 × 20 + e−20 202
2+ . . .+ e−20 2010
10!= . . .
; tedious ! ; use statistical tables or a statistical softwareMathematical Statistics (MAS713) Ariel Neufeld 46 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : examplesExampleSuppose that the number of drivers who travel between a particular originand destination during a designated time period has Poisson distribution withparameter λ = 20.In the long-run, in what proportion of time periods will the number of driversa) be at most 10? b) exceed 20? c) be between 10 and 20, inclusive? Strictlybetween 10 and 20?
Let X be the number of drivers. It is given that X ∼ P(20)
a)
P(X ≤ 10) = P(X = 0) + P(X = 1) + . . .+ P(X = 10)
= e−20 + e−20 × 20 + e−20 202
2+ . . .+ e−20 2010
10!= . . .
; tedious ! ; use statistical tables or a statistical softwareMathematical Statistics (MAS713) Ariel Neufeld 46 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : examples
Mathematical Statistics (MAS713) Ariel Neufeld 47 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson approximation to the Binomial distribution
Since it was derived as a limit case of the Binomial distribution whenn is ‘large’ and π is ‘small’, one can expect the Poisson distribution tobe a good approximation to Bin(n, π) in that case ; true
As it involves only one parameter, the Poisson pmf or the Poissontables are usually easier to handle than the corresponding Binomialpmf and tables
Mathematical Statistics (MAS713) Ariel Neufeld 48 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson approximation to the Binomial distribution
ExampleIt is known that 1% of the books at a certain bindery have defectivebindings.Compare the probabilities that x (x = 0,1,2, . . .) of 100 books will havedefective bindings using the (exact) formula for the binomialdistribution and its Poisson approximation
The exact Binomial pmf is pBin(x) =(100
x
)× 0.01x × 0.99100−x ,
while its Poisson approximation is
pPois(x) = e−λλx
x!
with λ = n × π = 100× 0.01 = 1
Mathematical Statistics (MAS713) Ariel Neufeld 49 / 66
3.2. Special random variables 3.2.4 Poisson random variables
Poisson distribution : examplesMatlab computations give :
We see that the error we would make by using the Poissonapproximation instead of the true distribution is only of order 10−3
; very good approximationMathematical Statistics (MAS713) Ariel Neufeld 50 / 66
3.2. Special random variables 3.2.5 Uniform random variables
The Uniform distribution
Mathematical Statistics (MAS713) Ariel Neufeld 51 / 66
3.2. Special random variables 3.2.5 Uniform random variables
The Uniform distributionThere are also numerous continuous distributions which are of greatinterest. The simplest one is certainly the uniform distribution
A random variable is said to be uniformly distributed over a finiteinterval [α, β], i.e.
X ∼ U[α,β]
if its probability density function is given by
f (x) ={ 1β−α if x ∈ [α, β]
0 otherwise(; SX = [α, β])
Constant density ; X is just as likely to be “close” to any value in SX
By integration, it is easy to show that
F (x) =
0 if x < αx−αβ−α if α ≤ x ≤ β1 if x > β
Mathematical Statistics (MAS713) Ariel Neufeld 52 / 66
3.2. Special random variables 3.2.5 Uniform random variables
The Uniform distribution
x
FX(x
)
α β
0.0
0.2
0.4
0.6
0.8
1.0
cdf F (x)
x
f X(x
)
α β
1
β − α
pdf f (x) = F ′(x)
Mathematical Statistics (MAS713) Ariel Neufeld 53 / 66
3.2. Special random variables 3.2.5 Uniform random variables
Uniform distribution : properties
Note that the constant Uniform density is set to 1/(β −α) on [α, β] soas to ensure that
∫ βα f (x)dx = 1
Now,
E(X ) =
∫ β
αx
1β − α
dx =1
β − α
[x2
2
]βα
=β2 − α2
2(β − α)=α+ β
2
Similarly,
E(X 2) =
∫ β
αx2 1β − α
dx =1
β − α
[x3
3
]βα
=β3 − α3
3(β − α)=β2 + αβ + α2
3
which implies Var(X ) = E(X 2)− (E(X ))2 = . . . = (β−α)2
12
Mathematical Statistics (MAS713) Ariel Neufeld 54 / 66
3.2. Special random variables 3.2.5 Uniform random variables
Uniform distribution : properties
Mean and variance of the Uniform distributionIf X ∼ U[α,β],
E(X ) =α+ β
2and Var(X ) =
(β − α)2
12
Mathematical Statistics (MAS713) Ariel Neufeld 55 / 66
3.2. Special random variables 3.2.5 Uniform random variables
Uniform distribution : example
Note:
The probability that X lies in any subinterval [a,b] of [α, β] is simplyequal to the length of that interval, i.e. b − a, divided by the length ofthe interval [α, β], i.e. β − α :
P(a < X < b) = b−aβ−α
.
Mathematical Statistics (MAS713) Ariel Neufeld 56 / 66
3.2. Special random variables 3.2.5 Uniform random variables
Uniform distribution : exampleExampleBuses arrive at a specified stop at 15-minute intervals starting at 7A.M. That is, they arrive at 7, 7:15, 7:30, 7:45, etc.If a passenger arrives at the stop at a time this uniformly distributedbetween 7 and 7:30, find the probability that he waits less than 5minutes for a bus
Let X denote the time (in minutes) past 7 A.M. that the passengerarrives at the stop. We have X ∼ U[0,30]
The passenger will have to wait less than 5 min if he arrives between7:10 and 7:15 or between 7:25 and 7:30. This happens with probability
P((10 < X < 15) ∪ (25 < X < 30)) = P(10 < X < 15) + P(25 < X < 30)
=5
30+
530
=13
Mathematical Statistics (MAS713) Ariel Neufeld 57 / 66
3.2. Special random variables 3.2.5 Uniform random variables
Uniform distribution : exampleExampleBuses arrive at a specified stop at 15-minute intervals starting at 7A.M. That is, they arrive at 7, 7:15, 7:30, 7:45, etc.If a passenger arrives at the stop at a time this uniformly distributedbetween 7 and 7:30, find the probability that he waits less than 5minutes for a bus
Let X denote the time (in minutes) past 7 A.M. that the passengerarrives at the stop. We have X ∼ U[0,30]
The passenger will have to wait less than 5 min if he arrives between7:10 and 7:15 or between 7:25 and 7:30. This happens with probability
P((10 < X < 15) ∪ (25 < X < 30)) = P(10 < X < 15) + P(25 < X < 30)
=5
30+
530
=13
Mathematical Statistics (MAS713) Ariel Neufeld 57 / 66
3.2. Special random variables 3.2.6 Exponential random variables
The Exponential distribution
Mathematical Statistics (MAS713) Ariel Neufeld 58 / 66
3.2. Special random variables 3.2.6 Exponential random variables
The Exponential distributionRemind that a Poisson distributed r.v. counts the number of
occurrences of a given phenomenon over a unit period of time
The (random) amount of time before the first occurrence of thatphenomenon is often of interest as well
If N ∼ P(λ) denote the number of occurrences over a unit period oftime, then it can be shown that the number of occurrences of thephenomenon by a time x , say Nx , is ∼ P(λx) (“Poisson process”)
Denote X the amount of time before the first occurrence
This time will exceed x (x ≥ 0) if and only if there have been nooccurrences of the phenomenon by time x , that is, Nx = 0
As Nx ∼ P(λx), it followsP(X > x) = P(Nx = 0) = e−λx (λx)0
0! = e−λx , which yields the cdf of X :
F (x) = 1− e−λx for x ≥ 0
This particular distribution is called the Exponential distributionMathematical Statistics (MAS713) Ariel Neufeld 59 / 66
3.2. Special random variables 3.2.6 Exponential random variables
The Exponential distributionA random variable is said to be an Exponential random variable with
parameter λ (λ > 0), i.e.X ∼ Exp(λ),
if its probability density function is given by
f (x) ={λe−λx if x ≥ 0
0 otherwise(; SX = R+)
By integration, it is easy to show that
F (x) ={
0 if x < 01− e−λx if x ≥ 0
From the above argument, it is easy to understand why thisdistribution is often useful for representing random amounts of time,like the amount of time required to complete a specified task, thewaiting time at a counter, the amount of time until you receive a phonecall, the amount of time until an earthquake occurs, etc.
Mathematical Statistics (MAS713) Ariel Neufeld 60 / 66
3.2. Special random variables 3.2.6 Exponential random variables
The Exponential distribution
x
FX(x
)
0
ln 2
λ
0
1/2
1
●
cdf F (x)
x
f X(x
)
0
λ
pdf f (x) = F ′(x)
Mathematical Statistics (MAS713) Ariel Neufeld 61 / 66
3.2. Special random variables 3.2.6 Exponential random variables
Exponential distribution : properties
One can check that∫ +∞
−∞f (x)dx =
∫ +∞
0λe−λx dx = λ
[e−λx
(−λ)
]+∞
0= 1
Moreover,
E(X ) =
∫ +∞
0x λe−λxdx =
[−x e−λx]+∞
0 +
∫ +∞
0e−λxdx (by Int.-by-parts)
= 0 +
[−e−λx
λ
]+∞
0=
1λ
Similarly, E(X 2) =∫ +∞
0 x2e−λx dx = . . . = 2λ2 , so that Var(X ) = 1
λ2 .
Mathematical Statistics (MAS713) Ariel Neufeld 62 / 66
3.2. Special random variables 3.2.6 Exponential random variables
Exponential distribution : properties
Mean and variance of the Exponential distributionIf X ∼ Exp(λ),
E(X ) =1λ
and Var(X ) =1λ2
Mathematical Statistics (MAS713) Ariel Neufeld 63 / 66
3.2. Special random variables 3.2.6 Exponential random variables
Exponential distribution : example
ExampleSuppose that, on the average, 3 trucks arrive per hour to be unloaded at awarehouse.What is the probability that the time between the arrivals of two successivetrucks will be a) less than 5 minutes? b) at least 45 minutes?
Assuming the number of trucks arriving during one hour is Poissondistributed (with parameter λ = 3). Then the amount of time X between twotruck arrivals follows the Exp(3) distribution
Hence,
a) P(X ≤ 1/12) =∫ 1/12
0 3e−3x dx = 1− e−1/4 = 0.221
b) P(X > 3/4) =∫∞
3/4 = e−9/4 = 0.105
Mathematical Statistics (MAS713) Ariel Neufeld 64 / 66
3.2. Special random variables 3.2.6 Exponential random variables
Exponential distribution : example
ExampleSuppose that, on the average, 3 trucks arrive per hour to be unloaded at awarehouse.What is the probability that the time between the arrivals of two successivetrucks will be a) less than 5 minutes? b) at least 45 minutes?
Assuming the number of trucks arriving during one hour is Poissondistributed (with parameter λ = 3). Then the amount of time X between twotruck arrivals follows the Exp(3) distribution
Hence,
a) P(X ≤ 1/12) =∫ 1/12
0 3e−3x dx = 1− e−1/4 = 0.221
b) P(X > 3/4) =∫∞
3/4 = e−9/4 = 0.105
Mathematical Statistics (MAS713) Ariel Neufeld 64 / 66
3.2. Special random variables 3.2.6 Exponential random variables
Other useful distributionsIn the remainder of this we will also encounter some other continuousdistributions, among these are
the Student-t distribution, X ∼ tν ;the χ2 distribution, X ∼ χ2
ν ;the Fisher (or just F ) distribution, X ∼ Fd1,d2
We will return to them later when we will need them
- The several distributions that we have introduced so far are veryuseful in the application of statistics to problems of engineering andphysical science
-However, the most important distribution is certainly the
Normal distribution
which we will introduce in the next subchapter
Mathematical Statistics (MAS713) Ariel Neufeld 65 / 66
3.2. Special random variables 3.2.6 Exponential random variables
ObjectivesNow you should be able to :
Understand the assumptions for some common discreteprobability distributionsSelect an appropriate discrete probability distribution to calculateprobabilities in specific applicationsCalculate probabilities, determine means and variances for somecommon discrete probability distributionsUnderstand the assumptions for some common continuousprobability distributionsSelect an appropriate continuous probability distribution tocalculate probabilities in specific applicationsCalculate probabilities, determine means and variances for somecommon continuous probability distributions
Put yourself to the test ! ; Q9 p.128, Q13 p.130, Q20 p.131, Q33p.133
Mathematical Statistics (MAS713) Ariel Neufeld 66 / 66
Recommended