Discrete random variables and probability distributionssayan/113/lectures/lec3.pdfRandom variables Distributions for discrete random variables Expectation and variance Discrete distributions

Random variablesDistributions for discrete random variables

Expectation and varianceDiscrete distributions

Discrete random variables and probability

distributions

Artin Armagan

Sta. 113 Chapter 3 of Devore

January 16, 2009

Artin Armagan Discrete random variables and probability distributions



Table of contents

1 Random variables

2 Distributions for discrete random variables

3 Expectation and variance

4 Discrete distributionsBernoulliBinomialHypergeometricPoisson




Mathematical definition

Definition

A random variable is a function that maps an event from thesample space S to a real number:

X : ω → R,

where ω ∈ S.




Intuition

Think of a function as a machine. It has inputs and outputs:f : x → y .




Intuition

Think of a function as a machine. It has inputs and outputs:f : x → y .A catapult




Intuition

The catapult takes as inputs: a rock and tension cord.The output is the distance the rock flies.




Intuition

The machine/function is now the flipping of a quarter by mythumb. The output is one of two possibilities: {H,T}. Let us callH = 1 and T = 0.




Intuition

The machine/function is now the flipping of a quarter by mythumb. The output is one of two possibilities: {H,T}. Let us callH = 1 and T = 0.This function is a (discrete) random variable it maps {H,T} intoreal numbers {0, 1},




Intuition

The machine/function is now the flipping of a quarter by mythumb. The output is one of two possibilities: {H,T}. Let us callH = 1 and T = 0.This function is a (discrete) random variable it maps {H,T} intoreal numbers {0, 1},Why is this function random ? Why is it discrete ?




Intuition

Back to the catapult. Even if we know exactly the size and shapeof the rock as well as the tension of the cord, the distance the rockflies may not always be the same due to variation in wind andmany other factors.




Intuition

Back to the catapult. Even if we know exactly the size and shapeof the rock as well as the tension of the cord, the distance the rockflies may not always be the same due to variation in wind andmany other factors.The catapult is a (continuous) random variable it maps the stateof the catapult to real numbers [0,∞).




Intuition

Back to the catapult. Even if we know exactly the size and shapeof the rock as well as the tension of the cord, the distance the rockflies may not always be the same due to variation in wind andmany other factors.The catapult is a (continuous) random variable it maps the stateof the catapult to real numbers [0,∞).Why is this function random ? Why is it continuous ?




Discrete versus continuous rv

Definition

A discrete random variable is a rv which takes a finite orcountable number of values.A continuous random variable is a rv which takes values in aninterval of the real line or all of the real line.




Bernoulli random variable

Definition

A random variable that takes values 0 or 1 is called a Bernoulli

random variable.




Bernoulli random variable

Definition

A random variable that takes values 0 or 1 is called a Bernoulli

random variable.




Distribution function

Definition

The probability distribution function or probability mass

function of a discrete random variable is defined for every possiblex by

p(x) = IP(X = x) = IP(X (s) = x : for all s ∈ S).




An example

0 5 10 15 200

0.05

0.1

0.15

0.2

x

p(x)




Matlab code

x= 1:20;y = poisspdf(x,4);plot(x,y,’*’)




Some properties

1 p(x) ≥ 0 for all X = x

2∑

x p(x) = 1




Parameter of a pdf

Definition

If p(x) is parameterized by a quantity α then α is the parameter ofthe pdf and the set of pdfs characterized by varying α is called afamily of distribution functions.




The Bernoulli family





X = {0, 1}

p(x ;α) =

{

1 − α if x = 0α if x = 1.





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)





0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

x

p(x)




Matlab code

for i = 0:10figure(i+1);alpha = i*.1;x=[0,1];y=[alpha,1-alpha];plot(x,y,’*’,’LineWidth’,8);h=gca;set(h,’FontSize’,[20]);set(h,’YLim’,[0 1]);set(h,’XLim’,[0 1]);xlabel(’x’);ylabel(’p(x)’);

end




Pregnancy

Suppose a couple is trying to get pregnant.




Pregnancy

Suppose a couple is trying to get pregnant.Let the number of months be the random variable X and let theprobability of conception be p.




Pregnancy


IP(x = 1) = IP(S) = p




Pregnancy


IP(x = 1) = IP(S) = p

IP(x = 2) = IP(FS) = (1 − p)p




Pregnancy


IP(x = 1) = IP(S) = p

IP(x = 2) = IP(FS) = (1 − p)p

IP(x = 3) = IP(FFS) = (1 − p)2p




Pregnancy


IP(x = 1) = IP(S) = p

IP(x = 2) = IP(FS) = (1 − p)p

IP(x = 3) = IP(FFS) = (1 − p)2p

orp(x) = (1 − p)x−1p, x = 1, 2, 3, ...




Cumulative distribution function

Definition

The cumulative distribution function (cdf) F (x) of a rv X withpdf p(x) is defined as

F (x) = IP(X ≤ x) =

x∑

i=1

p(i).

So for any number x, F (x) is the probability that the observedvalue of X is at most x.




An example

0 5 10 15 200

0.05

0.1

0.15

0.2

x

p(x)




An example

0 5 10 15 200

0.2

0.4

0.6

0.8

1

x

F(x

)




Matlab code

x= 1:20;y = poisscdf(x,4);i=1:.001:20;[dum,ind] = size(i);vals = zeros(1,ind);for j=1:ind

vals(1,j) = y(floor(i(j)));endplot(i,vals,’-’)hold onplot(x,y,’r*’)hold off;h=gca;set(h,’FontSize’,[20]);xlabel(’x’);ylabel(’F(x)’);




Pregnancy

The pdf of X is

p(x) = (1 − p)x−1p, x = 1, 2, 3, ...




Pregnancy

The pdf of X is

p(x) = (1 − p)x−1p, x = 1, 2, 3, ...

Compute the cdf

F (x) =

x∑

i=1

(1 − p)i−1p = p

x∑

i=1

(1 − p)i−1 = p

x−1∑

i=0

(1 − p)i .

k∑

i=0

ai =1 − ak+1

1 − a

which implies

F (x) = p1 − (1 − p)x

1 − (1 − p)= 1 − (1 − p)x .




Expectation of a discrete rv

Definition

Let X be a discrete rv with a set of possible values D and pdfp(x). The expected or mean value of X is

E[X ] = µX

=∑

x∈D

x · p(x).




Examples

Bernoulli:X = {0, 1}

p(x ;α) =

{

1 − α if x = 0α if x = 1.




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

µX

= 0 · (1 − α) + 1 · α = α.




Pregnancy

The pdf of X is

p(x) = p(1 − p)x−1, x = 1, 2, 3, ...




Pregnancy

The pdf of X is

p(x) = p(1 − p)x−1, x = 1, 2, 3, ...

µX

=

∞∑

x=1

x · p(x) =

∞∑

x=1

x · (1 − p)x−1p =1

p.




Another example

µX

= 0.0491

0 5 10 15 200

0.05

0.1

0.15

0.2

x

p(x)




Heavy tails

The discrete rv X takes values x = 1, 2, 3, .... and has pdf

p(x) =6

π2

1

x2.




Heavy tails

The discrete rv X takes values x = 1, 2, 3, .... and has pdf

p(x) =6

π2

1

x2.

Why 6π2 ?

What is the mean

µX

=6

π2

∞∑

x=1

x ·1

x2=

6

π2

∞∑

x=1

1

x= ∞.




Heavy tail

0 20 40 60 80 1000

0.5

1

x

p(x)

0 20 40 60 80 1000

0.5

1

x

p(x)




Heavy tail: semilog

0 20 40 60 80 10010

−5

100

x

p(x)

0 20 40 60 80 10010

−10

10−5

100

x

p(x)




Matlab code: page 1

figure(1)

x= 1:100;

y = (6/pi^2)*(1./x.^2);

subplot(2,1,1);

plot(x,y,’r*’);

h=gca;

set(h,’FontSize’,[20]);

xlabel(’x’);

ylabel(’p(x)’);

y = (6/pi^2)*(1./x.^3);

subplot(2,1,2);

plot(x,y,’r*’);

h=gca;


xlabel(’x’);

ylabel(’p(x)’);




Matlab code: page 2

figure(2)

x= 1:100;

y = (6/pi^2)*(1./x.^2);

subplot(2,1,1);

semilogy(x,y,’r*’);

h=gca;


xlabel(’x’);

ylabel(’p(x)’);

y = (6/pi^2)*(1./x.^3);

subplot(2,1,2);

semilogy(x,y,’r*’);

h=gca;


xlabel(’x’);

ylabel(’p(x)’);




Expectation of a function of a discrete rv

Proposition

Let X be a discrete rv with a set of possible values D and pdfp(x). The expectation of a function h(X ) is

E[h(X )] =∑

x∈D

h(x) · p(x).




Linearity of expectation

Proposition

E[aX + b] =∑

x∈D

(ax + b) · p(x),





Proposition

E[aX + b] =∑

x∈D

(ax + b) · p(x),

=∑

x∈D

ax · p(x) +∑

x∈D

b · p(x),





Proposition

E[aX + b] =∑

x∈D

(ax + b) · p(x),

=∑

x∈D

ax · p(x) +∑

x∈D

b · p(x),

= a∑

x∈D

x · p(x) + b∑

x∈D

p(x),





Proposition

E[aX + b] =∑

x∈D

(ax + b) · p(x),

=∑

x∈D

ax · p(x) +∑

x∈D

b · p(x),

= a∑

x∈D

x · p(x) + b∑

x∈D

p(x),

= a E[X ] + b,





Proposition

E[aX + b] =∑

x∈D

(ax + b) · p(x),

=∑

x∈D

ax · p(x) +∑

x∈D

b · p(x),

= a∑

x∈D

x · p(x) + b∑

x∈D

p(x),

= a E[X ] + b,

= aµX

+ b.




Variance of a discrete rv

Definition

Let X be a discrete rv with a set of possible values D and pdfp(x). The variance of X is

V[X ] = σ2X





Definition


V[X ] = σ2X

= E[(X − µX)2]





Definition


V[X ] = σ2X

= E[(X − µX)2]

=∑

x∈D

(x − µX)2 · p(x).





Definition


V[X ] = σ2X

= E[(X − µX)2]

=∑

x∈D

(x − µX)2 · p(x).

The standard deviation σX

=√

σ2X.




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,

= α2 − α3 + (1 − 2α + α2)α




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,

= α2 − α3 + (1 − 2α + α2)α

= α2 − α3 + α − 2α2 + α3




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,

= α2 − α3 + (1 − 2α + α2)α

= α2 − α3 + α − 2α2 + α3

= α − α2




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,

= α2 − α3 + (1 − 2α + α2)α

= α2 − α3 + α − 2α2 + α3

= α − α2

= α(1 − α)




Examples


p(x ;α) =

{

1 − α if x = 0α if x = 1.

σ2X

= (0 − α)2 · (1 − α) + (1 − α)2 · α,

= α2 − α3 + (1 − 2α + α2)α

= α2 − α3 + α − 2α2 + α3

= α − α2

= α(1 − α)




Properties of variance

V[X ] = E[(X − µX)2]





V[X ] = E[(X − µX)2]

= E[(X 2 − 2XµX

+ µ2X)],





V[X ] = E[(X − µX)2]

= E[(X 2 − 2XµX

+ µ2X)],

= E[X 2] − 2 E[X ]µX

+ µ2X,





V[X ] = E[(X − µX)2]

= E[(X 2 − 2XµX

+ µ2X)],

= E[X 2] − 2 E[X ]µX

+ µ2X,

= E[X 2] − 2µ2X

+ µ2X,





V[X ] = E[(X − µX)2]

= E[(X 2 − 2XµX

+ µ2X)],

= E[X 2] − 2 E[X ]µX

+ µ2X,

= E[X 2] − 2µ2X

+ µ2X,

= E[X 2] − µ2X,





V[X ] = E[(X − µX)2]

= E[(X 2 − 2XµX

+ µ2X)],

= E[X 2] − 2 E[X ]µX

+ µ2X,

= E[X 2] − 2µ2X

+ µ2X,

= E[X 2] − µ2X,

= E[X 2] − (E[X ])2.




Variance of a function of a discrete rv

Proposition

Let X be a discrete rv with a set of possible values D and pdfp(x). The variance of a function h(X ) is

V[h(X )] = E[(h(X ) − E[h(x)])2].




More properties of variance

V[aX + b] = E[(aX + b)2] − (E[aX + b])2.





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.

= E[a2X 2] + E[2abX ] + b2 − a2µ2 − b2 + 2µab.





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.

= E[a2X 2] + E[2abX ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2ab E[X ] + b2 − a2µ2 − b2 + 2µab.





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.

= E[a2X 2] + E[2abX ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2ab E[X ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2abµ + b2 − a2µ2 − b2 + 2µab.





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.

= E[a2X 2] + E[2abX ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2ab E[X ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2abµ + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] − a2µ2,





V[aX + b] = E[(aX + b)2] − (E[aX + b])2.

= E[a2X 2 + 2abX + b2] − (aµ + b)2.

= E[a2X 2] + E[2abX ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2ab E[X ] + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] + 2abµ + b2 − a2µ2 − b2 + 2µab.

= a2E[X 2] − a2µ2,

= a2V[X ].




BernoulliBinomialHypergeometricPoisson

Two ways

There are at least two ways to think about distributions:





Two ways


1 the distribution function p(x)





Two ways



2 an experiment that generates the distribution.





Two ways



2 an experiment that generates the distribution.

It is good to have an idea of both because one or the other may bemore useful at times.





Bernoulli

X = {0, 1}

p(x ; p) =

{

1 − p if x = 0p if x = 1.





Bernoulli

X = {0, 1}

p(x ; p) =

{

1 − p if x = 0p if x = 1.

What is the corresponding experiment ?





Bernoulli

X = {0, 1}

p(x ; p) =

{

1 − p if x = 0p if x = 1.

What is the corresponding experiment ?Flip a coin once with IP(H) = p. This is a Bernoulli trial.





Binomial

The experiment: run the Bernoulli trial n times with each trialindependent of the other and count the number of 1’s. This countis the random variable.





Binomial

The experiment: run the Bernoulli trial n times with each trialindependent of the other and count the number of 1’s. This countis the random variable.So the possible values of the rv X are 0, 1, 2, ..., n.





Binomial

The experiment: run the Bernoulli trial n times with each trialindependent of the other and count the number of 1’s. This countis the random variable.So the possible values of the rv X are 0, 1, 2, ..., n.Say n = 3, the outcomes are

000, 001, 010, 011, 100, 101, 110, 111

this corresponds to X taking

0, 1, 1, 2, 1, 2, 2, 3.





Binomial


000, 001, 010, 011, 100, 101, 110, 111


0, 1, 1, 2, 1, 2, 2, 3.

There are two parameters for this experiment: n the number oftrials and p the probability of a 1 for each trial.





Binomial


000, 001, 010, 011, 100, 101, 110, 111


0, 1, 1, 2, 1, 2, 2, 3.

There are two parameters for this experiment: n the number oftrials and p the probability of a 1 for each trial.What is the pdf ?





Binomial pdf

Theorem

The binomial probability distribution function is

IP(X = x) = bin(x ; n, p) =

(

n

x

)

px(1 − p)n−x x = 0, 1, ..., n.





Binomial pdf

Theorem


IP(X = x) = bin(x ; n, p) =

(

n

x

)

px(1 − p)n−x x = 0, 1, ..., n.

IP(getting x 1’s) = px(1 − p)n−x .





Binomial pdf

Theorem


IP(X = x) = bin(x ; n, p) =

(

n

x

)

px(1 − p)n−x x = 0, 1, ..., n.


{number of ways of getting x 1’s} =

(

n

x

)

.





Binomial pdf

Theorem


IP(X = x) = bin(x ; n, p) =

(

n

x

)

px(1 − p)n−x x = 0, 1, ..., n.


{number of ways of getting x 1’s} =

(

n

x

)

.

http://www.stat.berkeley.edu/~stark/Java/Html/BinHist.htm


http://www.stat.berkeley.edu/~stark/Java/Html/BinHist.htm




Sampling with replacement

An urn full of red and yellow m&ms are given to STA 113 students.






The students are told that there are 500 m&ms in the urn and 200are red.






The students are told that there are 500 m&ms in the urn and 200are red.Every minute they are allowed to randomly remove one m&m fromthe urn, record the color, and return the m&m to urn.






The students are told that there are 500 m&ms in the urn and 200are red.Every minute they are allowed to randomly remove one m&m fromthe urn, record the color, and return the m&m to urn.

This procedure is sampling with replacement and the distributionof the number of red m&ms drawn after 500 minutes is a binomialdistribution with n = 500 and p = 200

500 = 25 .





The binomial pdf

Fix n = 4 and vary p.





The binomial pdf

0 1 2 3 40

0.2

0.4

0.6

0.8

1

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

x

p(x)





The binomial pdf

0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

p(x)





The binomial pdf

0 1 2 3 40.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

p(x)





The binomial pdf

0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

x

p(x)





The binomial pdf

0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

p(x)





The binomial pdf

0 1 2 3 40

0.2

0.4

0.6

0.8

1

x

p(x)





Matlab code

n=4;

x=0:n;

for i=1:11

p = (i-1)*.1;

y=binopdf(x,n,p);

figure(i);

plot(x,y,’*’);

h=gca;


xlabel(’x’);

ylabel(’p(x)’);

filename = sprintf(’bin4%d.eps’,i);

saveas(h,filename,’psc2’)

end





The binomial pdf

Fix p = .4 and vary n.





The binomial pdf

0 1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

x

p(x)





The binomial pdf

0 2 4 6 8 100

0.05

0.1

0.15

0.2

0.25

x

p(x)





The binomial pdf

0 5 10 150

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 5 10 15 200

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

x

p(x)





The binomial pdf

0 5 10 15 20 250

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

x

p(x)





The binomial pdf

0 5 10 15 20 25 300

0.02

0.04

0.06

0.08

0.1

0.12

0.14

x

p(x)





The binomial pdf

0 5 10 15 20 25 30 350

0.02

0.04

0.06

0.08

0.1

0.12

x

p(x)





The binomial pdf

0 10 20 30 400

0.02

0.04

0.06

0.08

0.1

0.12

x

p(x)





The binomial pdf

0 10 20 30 400

0.02

0.04

0.06

0.08

0.1

0.12

x

p(x)





The binomial pdf

0 10 20 30 40 500

0.02

0.04

0.06

0.08

0.1

x

p(x)





The binomial pdf

0 10 20 30 40 500

0.02

0.04

0.06

0.08

0.1

x

p(x)





The binomial pdf

0 10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

x

p(x)





The binomial pdf

0 10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

x

p(x)





The binomial pdf

0 10 20 30 40 50 60 700

0.02

0.04

0.06

0.08

x

p(x)





The binomial pdf

0 20 40 600

0.02

0.04

0.06

0.08

x

p(x)





The binomial pdf

0 20 40 60 800

0.02

0.04

0.06

0.08

x

p(x)





The binomial pdf

0 20 40 60 800

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

x

p(x)





The binomial pdf

0 20 40 60 800

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

x

p(x)





The binomial pdf

0 20 40 60 800

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

x

p(x)





The binomial pdf

0 20 40 60 80 1000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

x

p(x)





Matlab code

p=.4;

for i=1:20

n = i*5;

x= 0:n;

y=binopdf(x,n,p);

ym = max(y);

figure(i);

plot(x,y,’*’);

h=gca;


set(h,’XLim’,[0 n]);

set(h,’YLim’,[0 ym]);

xlabel(’x’);

ylabel(’p(x)’);

filename = sprintf(’vpbin%d.eps’,i);


end





The binomial pdf

Fix pn = 4 and vary n.





The binomial pdf

0 1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

p(x)





The binomial pdf

0 2 4 6 8 100

0.05

0.1

0.15

0.2

0.25

x

p(x)





The binomial pdf

0 5 10 150

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 5 10 15 200

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 5 10 15 20 250

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 5 10 15 20 25 300

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 5 10 15 20 25 30 350

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 400

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 400

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 40 500

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 40 500

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 40 50 600

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 40 50 600

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 10 20 30 40 50 60 700

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 20 40 600

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 20 40 60 800

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 20 40 60 800

0.05

0.1

0.15

0.2

x

p(x)





The binomial pdf

0 20 40 60 800

0.05

0.1

0.15

x

p(x)





The binomial pdf

0 20 40 60 800

0.05

0.1

0.15

x

p(x)





The binomial pdf

0 20 40 60 80 1000

0.05

0.1

0.15

x

p(x)





Matlab code

val=4;

for i=1:20

n = i*5;

p = val/n;

x= 0:n;

y=binopdf(x,n,p);

ym = max(y);

figure(i);

plot(x,y,’*’);

h=gca;




xlabel(’x’);

ylabel(’p(x)’);

filename = sprintf(’vbbin%d.eps’,i);


end





Binomial cdf

Theorem

If X ∼ Bin(n, p) the cdf is denoted

IP(X ≤ x) = Bin(x ; n, p) =x

∑

i=0

(

n

i

)

pi(1 − p)n−i x = 0, 1, ..., n.





Binomial cdf

Example:I give you an exam and there are n = 120 of you. The probabilitysomeone fails is p = .1.





Binomial cdf

Example:I give you an exam and there are n = 120 of you. The probabilitysomeone fails is p = .1.What is the probability that at most 12 fail the test ?





Binomial cdf

Example:I give you an exam and there are n = 120 of you. The probabilitysomeone fails is p = .1.What is the probability that at most 12 fail the test ?

Bin(12; 120, .1) =

12∑

i=0

(

120

i

)

× (.1)i × (.9)120−i .





Binomial pdf

If X ∼ Bin(n, p) what is E[X ] ?





Binomial pdf


E[X ] = E[X1] + E[X2] + E[X3] + ... + E[Xn],

where Xi is a Bernoulli random variable.





Binomial pdf


E[X ] = E[X1] + E[X2] + E[X3] + ... + E[Xn],


E[Xi ] = p,

soE[X ] = np.





Binomial pdf

If X ∼ Bin(n, p) what is V[X ] ?





Binomial pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

= V[X1] + V[X2] + V[X3] + ... + V[Xn],

where Xi is an independent Bernoulli random variable.





Binomial pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

= V[X1] + V[X2] + V[X3] + ... + V[Xn],


V[Xi ] = p(1 − p),

soV[X ] = np(1 − p).





Binomial pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

= V[X1] + V[X2] + V[X3] + ... + V[Xn],


V[Xi ] = p(1 − p),

soV[X ] = np(1 − p).





Sampling without replacement

This time the urn full of m&ms are put in a room with someextremely intelligent and highly civilized people.






There are N = 500 m&ms in the urn and M = 200 are red.






There are N = 500 m&ms in the urn and M = 200 are red.Each day for n = 40 days, whenever they find some time fromdrama, they remove one m&m from the urn (having fought overit).






There are N = 500 m&ms in the urn and M = 200 are red.Each day for n = 40 days, whenever they find some time fromdrama, they remove one m&m from the urn (having fought overit).Since we don’t trust them, we interfere and record the color of them&m picked and then leave them alone.






There are N = 500 m&ms in the urn and M = 200 are red.Each day for n = 40 days, whenever they find some time fromdrama, they remove one m&m from the urn (having fought overit).Since we don’t trust them, we interfere and record the color of them&m picked and then leave them alone.Of course they cannot resist the temptation and eat that pickedm&m (having again fought over it).






This procedure is sampling without replacement and thedistribution of the number of red m&ms drawn after 40 days is ahypergeometric distribution parameterized as hyp(x ;N,M, n).






1 The population to be sampled consists of N objects.







2 Each object is either a 0 or a 1 and there are M 1’s. Eachtrial is Bernoulli. The probability of selecting a 1 is p = M

N.








N.

3 A sample of n objects is selected without replacement suchthat each subset of size n of the N objects is equally likely.








N.


The rv X is the number of 1’s in the n objects drawn andX ∼ Hyp(n,M,N).








N.


The rv X is the number of 1’s in the n objects drawn andX ∼ Hyp(n,M,N).What is the probability density function p(x) ?





Pdf of the hypergeometric

IP(X = x) = hyp(x ; n,M,N)







=number of outcomes with x 1’s

total number of outcomes









=

(

Mx

)(

N−Mn−x

)

(

Nn

) .









=

(

Mx

)(

N−Mn−x

)

(

Nn

) .

The denominator is the total number of outcomes or ways tochoose n out of N objects without considering order.









=

(

Mx

)(

N−Mn−x

)

(

Nn

) .

The denominator is the total number of outcomes or ways tochoose n out of N objects without considering order.The numerator has two terms:(

Mx

)

is the number of ways of selecting x 1’s out of M 1’s withoutconsidering order.









=

(

Mx

)(

N−Mn−x

)

(

Nn

) .

The denominator is the total number of outcomes or ways tochoose n out of N objects without considering order.The numerator has two terms:(

Mx

)

is the number of ways of selecting x 1’s out of M 1’s withoutconsidering order.(

N−Mn−x

)

is the number of ways of selecting n − x 0’s out of N − M0’s without considering order.






Proposition

If X is the number of 1’s in a random sample of size n drawn froma population of M 1’s and N − M 0’s then the probabilitydistribution of X called the hypergeometric distribution is

hyp(x ; n,M,N) =

(

Mx

)(

N−Mn−x

)

(

Nn

) ,

where max(0, n − N + M) ≤ x ≤ min(n,M).





Example: a genetic screen

A genetic screen of 1000 genes is run to check for association withdiabetes and it is found that 40 genes of the 1000 are associatedwith the occurance of diabetes.






A genetic screen of 1000 genes is run to check for association withdiabetes and it is found that 40 genes of the 1000 are associatedwith the occurance of diabetes.Genes in the oxidative phosphorylation (OXPHOS) pathway arethought to be implicated in diabetes due to their effect onmetabolism. There are 60 genes in this pathway all of which wereincluded in the initial screen.






A genetic screen of 1000 genes is run to check for association withdiabetes and it is found that 40 genes of the 1000 are associatedwith the occurance of diabetes.Genes in the oxidative phosphorylation (OXPHOS) pathway arethought to be implicated in diabetes due to their effect onmetabolism. There are 60 genes in this pathway all of which wereincluded in the initial screen.The overlap between genes in the OXPHOS pathway and the genesthat associate with diabetes is 35. Does this imply that genes inthe OXPHOS pathway are enriched or associated with diabetes ?






N = 1000 objects






N = 1000 objectsx = 35 genes associated with diabetes and OXPHOS






N = 1000 objectsx = 35 genes associated with diabetes and OXPHOSn = 40 genes randomly sampled (found to be associated withdiabetes)






N = 1000 objectsx = 35 genes associated with diabetes and OXPHOSn = 40 genes randomly sampled (found to be associated withdiabetes)M = 60 genes associated with OXPHOS






N = 1000 objectsx = 35 genes associated with diabetes and OXPHOSn = 40 genes randomly sampled (found to be associated withdiabetes)M = 60 genes associated with OXPHOSOut of 1000 objects of which 60 belong to OXPHOS if I randomlysample 40 will 35 of them be OXPHOS ?






N = 1000 objectsx = 35 genes associated with diabetes and OXPHOSn = 40 genes randomly sampled (found to be associated withdiabetes)M = 60 genes associated with OXPHOSOut of 1000 objects of which 60 belong to OXPHOS if I randomlysample 40 will 35 of them be OXPHOS ?

hyp(35; 40, 60, 1000) = 5.6503 · 10−43.

So very unlikely by chance so OXPHOS and diabetes might beassociated.





Hypergeometric pdf

If X ∼ Hyp(n,M,N) what is E[X ] ?





Hypergeometric pdf


E[X ] = E[X1] + E[X2] + E[X3] + ... + E[Xn],






Hypergeometric pdf


E[X ] = E[X1] + E[X2] + E[X3] + ... + E[Xn],


E[Xi ] = p =M

N,

so

E[X ] = n ·M

N.





Hypergeometric pdf

If X ∼ Hyp(n,M,N) what is V[X ] ?





Hypergeometric pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

< V[X1] + V[X2] + V[X3] + ... + V[Xn],

because Xi are not independent Bernoulli random variables.





Hypergeometric pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

< V[X1] + V[X2] + V[X3] + ... + V[Xn],


V[X ] =N − n

N − 1· np(1 − p),





Hypergeometric pdf


V[X ] = V[X1 + X2 + X3 + ... + Xn],

< V[X1] + V[X2] + V[X3] + ... + V[Xn],


V[X ] =N − n

N − 1· np(1 − p),

where N−nN−1 < 1 is the correction factor and approaches 1 when

N ≫ n.





Simeon Poisson





Poisson distribution

Definition

A random variable X is said to have a Poisson distribution withparameter λ > 0 if the pdf of X is

Pois(x ;λ) =e−λλx

x!, x = 0, 1, 2, 3, ...





Properties of the Poisson pdf

∞∑

x=0

e−λλx

x!= 1

By a series expansion

eλ = 1 + λ +λ2

2!+

λ3

3!+ ... =

∞∑

x=0

λx

x!.

Therefore

e−λ

∞∑

x=0

λx

x!= 1.





Properties of the Poisson pdf

The mean and variance of the Poisson distribution is

E[X ] = V[X ] = λ.





Parameter λ

λ is a positive real number, equal to the expected number ofoccurrences that occur during the given interval.





Parameter λ

λ is a positive real number, equal to the expected number ofoccurrences that occur during the given interval.For instance, if the events occur on average every 4 minutes, andyou are interested in the number of events occurring in a 10minute interval, you would use as model a Poisson distributionwith λ = 10/4 = 2.5.





Things modeled using Poisson distribution

Examples

1 The number of soldiers killed by horse-kicks each year in eachcorps in the Prussian cavalry.






Examples


2 The number of spelling mistakes one makes while typing asingle page.






Examples



3 The number of phone calls at a call center per minute.






Examples




4 The number of times a web server is accessed per minute.






Examples





5 The number of roadkill (animals killed) found per unit lengthof road.






Examples





5 The number of roadkill (animals killed) found per unit lengthof road.

6 The number of mutations in a given stretch of DNA after acertain amount of radiation.





Poisson distribution as binomial limit

Theorem

If we take the binomial pdf bin(x ; n, p) and take the limitlimn→∞,p→0 np = λ > 0 then





Poisson distribution as binomial limit

Theorem

If we take the binomial pdf bin(x ; n, p) and take the limitlimn→∞,p→0 np = λ > 0 then

bin(x ; n, p) → Pois(x ;λ).





Some pictures

0 50 100 150 200 250 3000

0.02

0.04

0.06

0.08

0.1

0.12

x

p(x)





Some pictures

0 50 100 150 200 250 3000

0.02

0.04

0.06

0.08

x

p(x)





Some pictures

0 50 100 150 200 250 3000

0.01

0.02

0.03

0.04

0.05

0.06

0.07

x

p(x)





Some pictures

0 50 100 150 200 250 3000

0.01

0.02

0.03

0.04

0.05

0.06

x

p(x)





Some pictures

0 50 100 150 200 250 3000

0.01

0.02

0.03

0.04

0.05

0.06

x

p(x)





Some pictures

0 50 100 150 200 250 3000

0.01

0.02

0.03

0.04

0.05

x

p(x)





Matlab code

for i=1:6

figure(i)

n=300;

x= 0:n;

lambda = i*10;

y=poisspdf(x,lambda);

y1 = max(y);

plot(x,y,’*’);

hold on

y=binopdf(x,n,lambda/n);

y2 = max(y);

ym=max(y1,y2);

plot(x,y,’r*’);

h=gca;


xlabel(’x’);

ylabel(’p(x)’);



hold off

filename = sprintf(’binapprox%d.eps’,i);

saveas(h,filename,’psc2’);

end





Poisson process

The Poisson process is defined in terms of occurence of events andis a function of the counts of events as a function of time, N(t).





Poisson process

The Poisson process is defined in terms of occurence of events andis a function of the counts of events as a function of time, N(t).The basic idea is that there is a rate parameter λ and the numberof events in the time (t, t + τ ] follows a Poisson distribution withparameter λτ





Poisson process


IP[(N(t + τ) − N(t)) = k] =e−λτ (λτ)k

k!k = 0, 1, 2, ...,

where N(t + τ) − N(t) describes the number of events in the timeinterval (t, t + τ ].





Poisson process


IP[(N(t + τ) − N(t)) = k] =e−λτ (λτ)k

k!k = 0, 1, 2, ...,

where N(t + τ) − N(t) describes the number of events in the timeinterval (t, t + τ ].The above is a Poisson process, not a density or distributionfunction.





Poisson process

The general properties of a (homogenous) Poisson process are

1 Memorylessness: The number of arrivals occurring during thetime interval t + τ is independent of the number of arrivalsoccurring before time t;





Poisson process



2 Orderliness: Arrivals do not occur simultaneously

limτ→0

P [N(t + τ) − N(t) > 1|N(t + τ) − N(t) ≥ 1] = 0;





Poisson process



2 Orderliness: Arrivals do not occur simultaneously

limτ→0

P [N(t + τ) − N(t) > 1|N(t + τ) − N(t) ≥ 1] = 0;

3 Homogeniety: The probability that exactly one event happensin the time interval τ is λτ

IP[N(t + τ) − N(t) = 1] = λτ + o(τ).





Example

Telephone call arrivals:Call requests arrive at times T1,T2, ...What we have access to is N(t) or the number of that arrive intime t.





Example

Telephone call arrivals:Call requests arrive at times T1,T2, ...What we have access to is N(t) or the number of that arrive intime t.Suppose that the counts are distributed as a Poisson process withrate parameter λ = 4 per min.





Example


1 What is the probability of exactly 140 arrivals in 30 minutes ?





Example



2 What is the expected value of the number of calls in a 30minute interval ?





Example



2 What is the expected value of the number of calls in a 30minute interval ?





Example

What is the probability of exactly 10 arrivals in 30 minutes ?





Example

What is the probability of exactly 10 arrivals in 30 minutes ?

IP[(N(t + τ) − N(t)) = k] =e−λτ (λτ)k

k!k = 0, 1, 2, ...,

IP[N(30) = 140] =e−4·30(4 · 30)140

140!.

= 0.0069.





Example

What is the expected value of the number of calls in a 30 minuteinterval ?





Example


E[N(30)] =∞

∑

k=0

k IP[N(30) = k]





Example


E[N(30)] =∞

∑

k=0

k IP[N(30) = k]

=∞

∑

k=0

ke−120120k

k!.





Example


E[N(30)] =∞

∑

k=0

k IP[N(30) = k]

=∞

∑

k=0

ke−120120k

k!.

= 120.





Example


E[N(30)] =∞

∑

k=0

k IP[N(30) = k]

=∞

∑

k=0

ke−120120k

k!.

= 120.

val = 0;

for k=0:10000

val = val + k*poisspdf(k,120);

end


Documents

Discrete random variables and probability distributionssayan/113/lectures/lec3.pdfRandom variables Distributions for discrete random variables Expectation and variance Discrete distributions