Negative Binomial Distribution - cknudson.comcknudson.com/StudentWork/NegBin.pdf · Negative Binomial Distribution Negative Binomial Distribution in R Relationship with Geometric

Introduction to theNegative BinomialDistribution

Negative BinomialDistribution in R

Relationship withGeometric distribution

MGF, Expected Valueand Variance

Relationship with otherdistributions

Thanks!

Negative Binomial Distribution

Andre Archer, Ayoub Belemlih, Peace MadimutsaMacalester College

November 30, 2016

1/??






Thanks!

Outline1 Introduction to the Negative Binomial Distribution

Defining the Negative Binomial DistributionExample 1Example 2: The Banach Match ProblemTransformation of PdfWhy so Negative?CDF of X

2 Negative Binomial Distribution in RR CodeExample 3

3 Relationship with Geometric distribution4 MGF, Expected Value and Variance

Moment Generating FunctionExpected Value and Variance

5 Relationship with other distributionsPossion Distribution

6 Thanks!2/??






Thanks!

Introduction to the Negative Binomial Distribution

Introduction to the NegativeBinomial Distribution

3/??






Thanks!

Defining the Negative Binomial Distribution

X ∼ NB(r , p)

Given a sequence of r Bernoulli trials with probability of success p,X follows a negative binomial distribution if X = k is the numberof trials needed to get to the rth success.

Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·

4/??






Thanks!


Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·

P(X = k) = P(rth on kth trial)

5/??






Thanks!


Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·


= P(r-1th on k-1 trials) · P(success on kth trial)

6/??






Thanks!


Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·



=

(k − 1

r − 1

)pr−1(1− p)k−1−(r−1) · p

7/??






Thanks!


Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·



=

(k − 1

r − 1

)pr−1(1− p)k−1−(r−1) · p

=

(k − 1

r − 1

)pr−1(1− p)k−r · p

8/??






Thanks!


Pdf of X

P(X = k) =

(k − 1

r − 1

)pr (1− p)k−r

where X = r , r + 1, · · ·



=

(k − 1

r − 1

)pr−1(1− p)k−1−(r−1) · p

=

(k − 1

r − 1

)pr−1(1− p)k−r · p

=

(k − 1

r − 1

)pr (1− p)k−r

9/??






Thanks!

Example 1

A door-to-door encyclopediasalesperson is required to doc-ument five in-home visits eachday. Suppose that she has a 30%chance of being invited into anygiven home, with each addressrepresenting an independent trial.What is the probability that sherequires fewer than eight housesto achieve her fifth success? (pg.269)

10/??






Thanks!

Example 2: The Banach Match Problem

Suppose that an absent-mindedprofessor (is there any otherkind?) has m matches in his rightpocket and m matches in his leftpocket. When he needs a matchto light his pipe, he is equallylikely to choose a match from ei-ther pocket. We want to computethe probability density function ofthe random variable W that givesthe number of matches remainingwhen the professor first discoversthat one of the pockets is empty.- math.utah.edu Steven Banach

1892 - 1945

11/??






Thanks!

Transformation of Pdf

Pdf of X

We can also express the pdf in terms a discrete rv, Y = thenumber of failures.

P(Y = k) =

(k + r − 1

k

)pr (1− p)k

where Y = 0, 1, · · ·

12/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

13/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

Let x = r + y

P(X = r + y) =

(y + r − 1

r − 1

)pr (1− p)y+r−r

14/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

Let x = r + y

P(X = r + y) =

(y + r − 1

r − 1

)pr (1− p)y+r−r

P(X − r = y) =

(y + r − 1

r − 1

)pr (1− p)y

15/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

Let x = r + y

P(X = r + y) =

(y + r − 1

r − 1

)pr (1− p)y+r−r

P(X − r = y) =

(y + r − 1

r − 1

)pr (1− p)y

P(Y = y) =

(y + r − 1

r − 1

)pr (1− p)y

16/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

Let x = r + y

P(X = r + y) =

(y + r − 1

r − 1

)pr (1− p)y+r−r

P(X − r = y) =

(y + r − 1

r − 1

)pr (1− p)y

P(Y = y) =

(y + r − 1

r − 1

)pr (1− p)y

We can use(y+r−1

r−1)

=(y+r−1

y

).

17/??






Thanks!


P(X = x) =

(x − 1

r − 1

)pr (1− p)x−r

Let x = r + y

P(X = r + y) =

(y + r − 1

r − 1

)pr (1− p)y+r−r

P(X − r = y) =

(y + r − 1

r − 1

)pr (1− p)y

P(Y = y) =

(y + r − 1

r − 1

)pr (1− p)y

Since(y+r−1

r−1)

=(y+r−1

y

),

P(Y = y) =

(y + r − 1

y

)pr (1− p)y

18/??






Thanks!

Why so Negative?

Pdf of Y

P(Y = k) =

(k + r − 1

k

)pr (1− p)k

where Y = 1, 2, · · ·

The binomial coefficient in the pdf may be rearranged as follows:(k + r − 1

k

)=

(k + r − 1) · (k + r − 2) · · · rk!

= (−1)k(−r − (k − 1)) · (r − (k − 2)) · · · (−r)

k!

= (−1)k(−r) · · · (r − (k − 2)) · (−r − (k − 1))

k!

= (−1)k(−rk

)19/??






Thanks!

CDF of X

Cdf of X

F (X ≤ k) =k∑

j=r

(j − 1

r − 1

)pr (1− p)j−r

20/??






Thanks!

Negative Binomial Distribution in R

Negative Binomial Distributionin R

21/??






Thanks!

R Code

Probability Density of NB(r=size,p=prob)

1 dnbinom ( x , s i z e , prob , mu, l o g = FALSE)

22/??






Thanks!

R Code

Probability Density of NB(r=size,p=prob)

1 dnbinom ( x , s i z e , prob , mu, l o g = FALSE)

Using Example 1,

1 dnbinom (7−5 , s i z e =5, prob =0.3)2 ## [ 1 ] 0 .017860534 dnbinom (5−5 , s i z e =5, prob =0.3) + dnbinom (6−5 , s i z e =5,

prob =0.3) + dnbinom (7−5 , s i z e =5, prob =0.3)5 ## [ 1 ] 0 .0287955

23/??






Thanks!

R Code

Cumulative Density of NB(r=size,p=prob)

1 pnbinom ( q , s i z e , prob , mu, l o w e r . t a i l = TRUE, l o g . p =FALSE)

24/??






Thanks!

R Code

Cumulative Density of NB(r=size,p=prob)

1 pnbinom ( q , s i z e , prob , mu, l o w e r . t a i l = TRUE, l o g . p =FALSE)

Using Example 1,

1 dpnbinom ( 2 , s i z e =5, prob =0.3)2 ## [ 1 ] 0 .0287955

25/??






Thanks!

Example 3

Darryl statistics homework lastnight was to flip a fair coin andrecord the toss, X, when headsappeared for the second time.The experiment was to berepeated a total of one hundredtimes. The following are the onehundred values for X that Darrylturned in this morning. Do youthink that he actually did theassignment? Explain. (pg. 269)

26/??






Thanks!

Example 3

1 #Second Example2 k <− 2 : 1 03 p r o b a b i l i t i e s <− c ( )4 f o r ( i i n k ) {5 p r o b a b i l i t i e s <− c ( p r o b a b i l i t i e s , dnbinom ( i −2, s i z e =2,

prob =0.5) )6 }7 e x p e c t e d v a l u e s <− c ( )8 f o r ( j i n p r o b a b i l i t i e s ) {9 e x p e c t e d v a l u e s <− c ( e x p e c t e d v a l u e s , round (100 ∗ j ,

d i g i t s =0) )10 }11 o b s e r v e d v a l u e s <−c ( 2 4 , 2 6 , 1 9 , 1 3 , 8 , 5 , 3 , 1 , 1 )1213 t b l <− data . f rame ( k , p r o b a b i l i t i e s , o b s e r v e d v a l u e s ,

e x p e c t e d v a l u e s )14 t b l

27/??






Thanks!

Example 3

1 > t b l2 ## k p r o b a b i l i t i e s o b s e r v e d v a l u e s e x p e c t e d v a l u e s3 ## 1 2 0.250000000 24 254 ## 2 3 0.250000000 26 255 ## 3 4 0.187500000 19 196 ## 4 5 0.125000000 13 127 ## 5 6 0.078125000 8 88 ## 6 7 0.046875000 5 59 ## 7 8 0.027343750 3 3

10 ## 8 9 0.015625000 1 211 ## 9 10 0.008789062 1 1

28/??






Thanks!

Relationship with Geometric distribution

Relationship with Geometricdistribution

29/??






Thanks!

We can also interpret X ∼ NB(r , p) as the sum of rindependent geometric distributions.

X = X1 + X2 + · · ·+ Xr

P(X ) = P(X1 + X2 + · · ·+ Xr ) = P(X1) · P(X2) · · · · · P(Xr )

30/??






Thanks!

Consider the case where r = 3 and P(X = 3),

31/??






Thanks!


P(X = r) = P(X1 = 1) · P(X2 = 1) · P(X3 = 1)

32/??






Thanks!


P(X = r) = P(X1 = 1) · P(X2 = 1) · P(X3 = 1)

= p · p · p

33/??






Thanks!


P(X = r) = P(X1 = 1) · P(X2 = 1) · P(X3 = 1)

= p · p · p= p3

34/??






Thanks!


P(X = r) = P(X1 = 1) · P(X2 = 1) · P(X3 = 1)

= p · p · · · · · p= p3

From the pdf of X,

P(X = 3) = p3

35/??






Thanks!


We have to figure out a way to distribute 3 trials to each randomvariable.

36/??






Thanks!



P(X = 4) = P(X1 = 2) · P(X2 = 1) · P(X3 = 1)

37/??






Thanks!



P(X = 4) = P(X1 = 2) · P(X2 = 1) · P(X3 = 1) = (1− p)p · p · p

38/??






Thanks!



P(X = 4) = P(X1 = 2) · P(X2 = 1) · P(X3 = 1) = (1− p)p · p · p= p3(1− p)

39/??






Thanks!



P(X = 4) = P(X1 = 2) · P(X2 = 1) · P(X3 = 1) = (1− p)p · p · p= p3(1− p)

From the pdf of X,

P(X = 4) =

(3

2

)p3(1− p)

Soo... we did something wrong!

40/??






Thanks!

Remain calm, we only missed something minor!

We can have P(X1 = 2) · P(X2 = 1) · P(X3 = 1) orP(X1 = 1) · P(X2 = 2) · P(X3 = 1) orP(X1 = 1) · P(X2 = 1) · P(X3 = 2).

41/??






Thanks!



All possible cases are weak compositions of 4 into 3 parts.

42/??






Thanks!



All possible cases are weak compositions of 4 into 3 parts.(3

2

)

43/??






Thanks!


THEREFORE!

P(X = 4) =

(3

2

)P(X1 = 2) · P(X2 = 1) · P(X3 = 1)

=

(3

2

)(1− p)p · p · p

=

(3

2

)p3(1− p)

From the pdf of X,

P(X = 4) =

(3

2

)p3(1− p)

44/??






Thanks!

MGF, Expected Value and Variance

MGF, Expected Value andVariance

45/??






Thanks!

Moment Generating Function

MGF of X

M(t) =

[etp

1− et(1− p)

]r

46/??






Thanks!


M(t) = E [etX ] =∞∑k=r

etk(k − 1

r − 1

)(1− p)k−rpr

=∞∑k=r

etk(k − 1

r − 1

)(1− p)k−rpr · e

tr

etr

=∞∑k=r

et(k−r)(k − 1

r − 1

)(1− p)k−r (etp)r

= (etp)r∞∑k=r

(k − 1

r − 1

)(et(1− p))k−r

Setting j = k − r ,

= (etp)r∞∑j=0

(j + r − 1

r − 1

)(et(1− p))j

47/??






Thanks!


To finish this, we need something else ...

48/??






Thanks!


Lemma

(1− x)−r =∞∑k=0

(r + k − 1

k

)xk

Using Taylor Expansion around x = 0,

(1− x)−r =∞∑k=0

f (k)(0)

k!xk where f (x) = (1− x)−r

f (k)(0)

k!=

1

k!

dk

dxk(1− x)−r

∣∣∣∣x=0

=r · (r + 1) · · · (r + k − 1)

k!(1− 0)−r−k

=(r − 1)!

(r − 1)!

r · (r + 1) · · · · · (r + k − 1)

k!=

(r + k − 1)!

(r − 1)!k!

=

(r + k − 1

k

)49/??






Thanks!


Continuing where we left off,

M(t) = (etp)r∞∑j=0

(r + j − 1

k

)(et(1− p))j

=(etp)r

(1− et(1− p))r=

[etp

1− et(1− p)

]rwhere t < − log(1− p).

50/??






Thanks!


We could just use this theorem instead.

Thereom 3.12.3b

Let W1,W2, · · · ,Wn be independent random variables withmoment-generating functions MW1(t), MW2(t), · · · ,and MWn(t),respectively. Let W = W1 + W2 + · · ·+ Wn. Then

MW (t) = MW1(t) ·MW2(t) · · ·MWn(t)

51/??






Thanks!


Since X is sum of r independent geometric random variables,X = X1 + · · ·+ Xr ,

MX (t) = MX1(t) · · ·MXr (t)

52/??






Thanks!



MX (t) = MX1(t) · · ·MXr (t)

If we recall, MGF of a geometric rv is etp1−et(1−p) .

53/??






Thanks!



MX (t) = MX1(t) · · ·MXr (t)

If we recall, MGF of a geometric rv is etp1−et(1−p) ,

MX (t) =etp

1− et(1− p)· · · · · etp

1− et(1− p)=

[etp

1− et(1− p)

]r

54/??






Thanks!

Expected Value and Variance

Expected Value and Variance of Negative Binomial Variable

If X ∼ NB(r , p), then

E(X ) = r/p

Var(X ) = r(1− p)/p

55/??






Thanks!




E(X ) = r/p

Var(X ) = r(1− p)/p

Since X = X1 + X2 + · · ·Xr , where Xi is a geometric randomvariable,

56/??






Thanks!




E(X ) = r/p

Var(X ) = r(1− p)/p


E(X ) = E(X1) + E(X2) + · · ·E(Xr )

57/??






Thanks!




E(X ) = r/p

Var(X ) = r(1− p)/p


E(X ) = E(X1) + E(X2) + · · ·E(Xr )

Since E(Xi ) = 1/p,

58/??






Thanks!




E(X ) = r/p

Var(X ) = r(1− p)/p


E(X ) = E(X1) + E(X2) + · · ·E(Xr )

Since E(Xi ) = 1/p

E(X ) = r/p

59/??






Thanks!




E(X ) = r/p

Var(X ) = r(1− p)/p


E(X ) = E(X1) + E(X2) + · · ·E(Xr )

Since E(Xi ) = 1/p

E(X ) = r/p

Similarly, Var(X ) = r(1− p)/p.

60/??






Thanks!

Relationship with other distributions


61/??






Thanks!

Possion Distribution

We can use Possion distribution to approximate a negativebinomial distribution.

limr→∞

NB(r , p) = Pois(λ)

where λ = rp.

62/??






Thanks!


limr→∞

NB(r , p) = Pois(λ) where λ = rp

limr→∞

P(X = k) = limr→∞

(n + k − 1

k

)pk(1− p)n−k

63/??






Thanks!


limr→∞

NB(r , p) = Pois(λ) where λ = rp

limr→∞


(r + k − 1

k

)pk(1− p)r−k

Let λ = rp =⇒ p = λr ,

64/??






Thanks!


limr→∞

NB(r , p) = Pois(λ), where λ = rp

limr→∞


(r + k − 1

k

)pk(1− p)r−k


= limr→∞

(r + k − 1

k

)(λ

r

)k(1− λ

r

)r−k

65/??






Thanks!


limr→∞


limr→∞


(r + k − 1

k

)pk(1− p)n−k


= limr→∞

(r + k − 1

k

)(λ

r

)k(1− λ

r

)r−k

= limr→∞

(r + k − 1)!

k!(r − 1)!

(λ

r

)k(1− λ

r

)r−k

66/??






Thanks!


limr→∞


limr→∞


(r + k − 1

k

)pk(1− p)n−k


= limr→∞

(r + k − 1

k

)(λ

r

)k(1− λ

r

)r−k

= limr→∞

(r + k − 1)!

k!(r − 1)!

(λ

r

)k(1− λ

r

)r−k

=λk

k!limr→∞

(r + k − 1)!

(r − 1)!

1

rk

(1− λ

r

)r

·(

1− λ

r

)−k67/??






Thanks!


limr→∞

(r + k − 1)!

(r − 1)!

1

rk

(1− λ

r

)r

=(r − k + 1)!

r r+k· (r − λ)r

(r − 1)!= 1

68/??






Thanks!


limr→∞

(r + k − 1)!

(r − 1)!

1

rk

(1− λ

r

)r

=(r − k + 1)!

r r+k· (r − λ)r

(r − 1)!= 1

limr→∞

(1− λ

r

)−k= e−λ

69/??






Thanks!


λk

k!limr→∞

(r + k − 1)!

(r − 1)!

1

rk

(1− λ

r

)r

·(

1− λ

r

)−k=λk

k!e−λ

70/??






Thanks!

THANKS!

71/??

Documents

Negative Binomial Distribution - cknudson.comcknudson.com/StudentWork/NegBin.pdf · Negative Binomial Distribution Negative Binomial Distribution in R Relationship with Geometric