Lecture 6: Special Probability Distributionsweb.iku.edu.tr/~eyavuz/dersler/probability/6-nopause.pdf · The Discrete Uniform Distribution The Bernoulli Distribution The Binomial Distribution

The Discrete Uniform Distribution The Bernoulli Distribution The Binomial Distribution The Negative Binomial and Geometric Di

Lecture 6: Special Probability Distributions

Assist. Prof. Dr. Emel YAVUZ DUMAN

MCB1007 Introduction to Probability and StatisticsIstanbul Kultur University


Outline

1 The Discrete Uniform Distribution

2 The Bernoulli Distribution

3 The Binomial Distribution

4 The Negative Binomial and Geometric Distribution


Outline






The Discrete Uniform Distribution

If a random variable can take on k different values with equalprobability, we say that it has a discrete uniform distribution;symbolically,

Definition 1

A random variable X has a discrete uniform distribution and itis referred to as a discrete uniform random variable if and only ifits probability distribution is given by

f (x) =1

kfor x = x1, x2, · · · , xk

where xi �= xj when i �= j .


The mean and the variance of this distribution are

Mean:

μ = E [X ] =k∑

i=1

xi f (xi )︸︷︷︸1/k

=k∑

i=1

xi · 1k=

x1 + x2 + · · ·+ xkk

,

Variance:

σ2 = E [(X − μ)2] =k∑

i=1

(xi − μ)2 · 1k

=(x1 − μ)2 + (x2 − μ)2 + · · ·+ (xk − μ)2

k


In the special case where xi = i , the discrete uniform distributionbecomes f (x) = 1

k for x = 1, 2, · · · , k , and in this from it applies,for example, to the number of points we roll with a balanced die.

Example 2

If X has the discrete uniform distribution f (x) = 1/k forx = 1, 2, · · · , k , show that

(a) its mean is μ = k+12 ;

(b) its variance is σ2 = k2−112 .


Solution.

(a) Mean:

μ = E (X ) =k∑

i=1

if (x) =k∑

i=1

i · 1k=

1 + 2 + · · ·+ k

k

=k(k+1)

2

k=

k + 1

2.

(b) Variance: σ2 = μ′2 − μ2 = E (X 2)− [E (X )]2

μ′2 = E (X 2) =

k∑i=1

i2f (x) =k∑

i=1

i2 · 1k=

12 + 22 + · · ·+ k2

k

=k(k+1)(2k+1)

6

k=

(k + 1)(2k + 1)

6,

σ2 =(k + 1)(2k + 1)

6−(k + 1

2

)2

=k2 − 1

12.


Example 3

If X has the discrete uniform distribution f (x) = 1/k forx = 1, 2, · · · , k , show that its moment-generating function is givenby

MX (t) =et(1− ekt)

k(1− et).

Also find the mean of this distribution by evaluating limt→0M′X (t),

compare the results with that obtained in Example 2.


Solution. We know that

MX (t) = E (etX ) =

k∑i=1

e it1

k=

et + e2t + · · · + ekt

k.

On the other hand, since

S = et + e2t + e3t + · · ·+ e(k−1)t + ekt

−etS = −e2t − e3t − e4t − · · · − ekt − e(k+1)t ⇒S(1− et) = S − etS = et − e(k+1)t = et(1− ekt) ⇒S(1− et)

1− et= S = et + e2t + e3t + · · ·+ e(k−1)t + ekt =

et(1− ekt)

1− et

thus

MX (t) =et + e2t + · · ·+ ekt

k=

et(1− ekt)

k(1− et).


MX (t) =et + e2t + · · ·+ ekt

k⇒

M ′X (t) =

et + 2e2t + · · · + kekt

k⇒

μ = limt→0

M ′X (t) =

1 + 2 + · · ·+ k

k=

k(k+1)2

k=

k + 1

2.


Outline






The Bernoulli Distribution

If an experiment has two possible outcomes, “success” and“failure” and their probabilities are, respectively, θ and 1− θ, thenthe number of successes, 0 or 1, has a Bernoulli distribution;symbolically,

Definition 4

A random variable X has a Bernoulli distribution and it isreferred to as a Bernoulli random variable if and only if itsprobability distribution is given by

f (x ; θ) = θx(1− θ)1−x for x = 0, 1.

Thus, f (0; θ) = 1− θ and f (1; θ) = θ are combined into a singleformula. Since the Bernoulli distribution is a special case of theBinomial distribution, we shall not discuss it here in any detail.


In connection with the Bernoulli distribution, a success may begetting heads with a balanced coin, it may be catching pneumonia,it may be passing (or failing) an examination, and it may be losinga race. We refer to an experiment to which the Bernoullidistribution applies as a Bernoulli trial, or simply a trial, and tosequences of such experiments as repeated trials.Examples.

Toss a coin: S = {H,T}.Throw a fair die: S = {face value is a six, face value is not asix }.Sent a message through a network and record whether or notit is received: S = {successful transmission, unsuccessfultransmission}.Draw a part from an assembly line and record whether or notit is defective: S = {defective, good}.


Example 5

If X has the Bernoulli distribution, show that

(a) its mean is μ = θ;

(b) its variance is σ2 = θ(1− θ).

Solution.

(a)

μ = E (X ) =1∑

x=0

xf (x ; θ) =1∑

x=0

xθx(1− θ)1−x

= 0 · θ0(1− θ)1−0 + 1 · θ1(1− θ)1−1 = θ

(b)

μ′2 = E (X 2) =

1∑x=0

x2f (x ; θ) =1∑

x=0

x2θx (1− θ)1−x

= 02 · θ0(1− θ)1−0 + 12 · θ1(1− θ)1−1 = θ ⇒σ2 = E (X 2)− [E (X )]2 = μ′

2 − μ2 = θ − θ2 = θ(1 − θ)


Example 6

Show that for the Bernoulli distribution μ′r = θ for r = 1, 2, 3, · · · .

Solution.

MX (t) = E (etX ) =1∑

x=0

etx f (x ; θ) =1∑

x=0

etxθx(1− θ)1−x

= et·0θ0(1− θ)1−0 + et·1θ1(1− θ)1−1 = 1− θ + θ · et

= 1 + θ(et − 1) = 1 + θ

(1 + t +

t2

2!+

t3

3!+ · · ·+ tr

r !+ · · · − 1

)

= 1 + θt + θt2

2!+ θ

t3

3!+ · · ·+ θ

tr

r !+ · · · ⇒ μ′

r = θ.


Outline






The Binomial Distribution

Repeated trials play a very important role in the probability andstatistics, especially when

the number of trial is fixed,

the parameter θ (the probability of a success) is the same foreach trial, and

the trials are all independent.

The theory that we shall discuss in this section has manyapplications; for instance, it applies if we want to know theprobability of getting 5 heads in 12 flips of a coin, the probabilitythat 7 of 10 persons will recover from a tropical disease, or theprobability that 35 of 80 persons will respond to a mail-ordersolicitation. However, this is the case only if each of the 10persons has the same chance of recovering from the disease andand their recoveries are independent.


To derive a formula for the probability of getting “x successes in ntrials” under the stated conditions, observe that the probability ofgetting x successes and n − x failures in a specific order isθx(1− θ)n−x . There is one factor θ for each success, one factor1− θ for each failure , and the x factors θ and n− x factors 1− θare all multiplied together by virtue of the assumption ofindependence. Since this probability applies to any sequence of ntrials in which there are x successes and n− x failures, we haveonly to count how many sequences of this kind there are and thenmultiply θx(1− θ)n−x by that number. Clearly, number of ways inwhich we can select the x trials on which there is to be a success is(nx

), and it follows that the desired probability for “x successes in n

trials” is(nx

)θx(1− θ)n−x .


Let θ be the probability that an event will happen in any singleBernoulli trial (called the probability of success). Then 1− θ is theprobability that the event will fail to happen in any single trial(called the probability of failure). The probability that the eventwill happen exactly x times in n trials (i.e., successes and n − xfailures will occur) is given by the probability function

b(x ; n, θ) =

(n

x

)θx(1− θ)n−x

where the random variable X denotes the number of successes in ntrials and x = 0, 1, · · · , n.Definition 7

A random variable X has a binomial distribution and it is referredto as a binomial random variable if and only if its probabilitydistribution is given by

b(x ; n, θ) =

(n

x

)θx(1− θ)n−x for x = 0, 1, 2 · · · , n.


Example 8

Find the probability of getting five heads and seven tails in 12 flipsof a balanced coin.

Solution. Substituting x = 5, n = 12, and θ = 12 into the formula

for the binomial distribution, we get

b

(5; 12,

1

2

)=

(12

5

)(1

2

)5(1− 1

2

)12−5

≈ 0.19334.


Example 9

Find the probability that 7 of 10 persons will recover form atropical disease if we can assume independence and the probabilityis 0.80 that any one of them will recover from the disease.

Solution. Substituting x = 7, n = 10, and θ = 0.80 into theformula for the binomial distribution, we get

b (7; 10, 0.80) =

(10

7

)(0.8)7 (1− 0.8)10−7 ≈ 0.20133.


Example 10

Find the probability that in a family of 4 children there will be

1 at least one boy

2 at least one boy and at least one girl.

Solution.1 Probability of male birth is θ = 1/2 = 0.5. Then

P(at least one boy) = 1− P(no boy) = 1− b(0; 4, 0.5)

= 1−(4

0

)0.500.54−0 = 1− 1

24=

15

16.

2

P(at least one boy and girl) = 1− [P(no boy) + P(no girl)]

= 1− [2 · b(0; 4, 0.5)] = 1−(2 ·

(4

0

)0.500.54−0

)

= 1−(2 · 1

24

)=

7

8.


Example 11

Out of 2000 families with 4 children each how many would youexpect to have

1 at least one boy

2 2 boys

3 1 or 2 girls

4 no girls.

Solution. 1) 2000 · 1516 = 1875.

2) 2000 · (42) 122

122

= 750.3)P(1 or 2 girls) = P(1 girl) + P(2 girls) =

(41

)12

123

+(42

)122

122

= 58 ⇒

200058 = 1250.

4) P(no girls) =(40

)120

124

= 116 ⇒ 2000 1

16 = 125.


Theorem 12

The moment-generating function of the binomial distribution isgiven by

MX (t) = [1 + θ(et − 1)]n.

Proof. For a sequence of n binomial trials, define

Xj =

{0 if failure in jth trial

1 if success in jth trial.

Then the Xj are independent and X = X1 + X2 + · · ·+Xn. For themoment generating function of Xj , we have

MXj(t) = E (etXj ) = et·0(1− θ) + et·1θ = (1− θ) + etθ.

Since Xj are independent then

MX (t) = MX1(t) ·MX2(t) · · ·MXn(t) = ((1− θ) + etθ)n

orMX (t) = [1 + θ(et − 1)]n.


Theorem 13

The mean and the variance of the binomial distribution are

μ = nθ and σ2 = nθ(1− θ).

Proof. Proceeding as in Theorem 12, we have for j = 1, 2, · · · , n,

E (Xj) =1∑

x=0

xb(x ; n, θ) = 0 · (1− θ) + 1 · θ = θ.

Var(Xj) = E [(Xj − θ)2] = (0− θ)2(1− θ) + (1− θ)2θ

= θ2 − θ3 + θ − 2θ2 + θ3 = θ(1− θ).

μ = E (X ) = E (X1) + E (X2) + · · ·+ E (Xn) = nθ

σ2 = Var(X ) = Var(X1) + Var(X2) + · · ·+ Var(Xn) = nθ(1− θ).


Example 14

In 100 tosses of a fair coin the expected (or mean) number ofheads is

E (X ) = nθ = 100 · 12= 50.

while the standard deviation is

σ =√

nθ(1− θ) =

√100 · 1

2

(1− 1

2

)= 5.


Example 15

If the probability of a defective bolt is 0.1, find (a) the mean, (b)the standard deviation, for the number of defective bolts in a totalof 400 bolts.

Solution. (a) Mean: μ = nθ = 400(0.1) = 40, i.e., we can expect40 bolts to be defective.(b) Variance: σ2 = nθ(1− θ) = 400(0.1)(0.9) = 36. Hence, thestandard deviation σ =

√36 = 6.


Outline






The Negative Binomial and Geometric Distribution

In connection with repeated Bernoulli trials, we are sometimesinterested in the number of the trial on which the kth successoccurs.The negative binomial random variable and distribution are basedon an experiment satisfying the following conditions:

The experiment consists of x repeated trials.

Each trial can result in just two possible outcomes. We callone of these outcomes a success and the other, a failure.

The probability of success, denoted by θ, is the same on everytrial.

The trials are independent.

The experiment continues until k successes are observed,where k is specified in advance.


In the game of craps, you decide to play until you lose 5 games.You wonder how many games you will play with this terminationrule. The probability of losing any one game is 0.5071. The gamesare a series of independent Bernoulli trials, and the randomvariable is the number of wins until the fifth loss. This is asituation described by the negative binomial distribution.For the negative binomial distribution the random variable is thenumber of failures before the kth success is observed. Thedistribution has two parameters. The parameter θ is the probabilityof success on any one trial and the parameter k is the number ofsuccesses to be observed before the experiment is complete.The geometric distribution is a special case with k equal to 1.


If the kth success is to occur on the xth trial, there must be k − 1successes on the first x − 1 trials, and the probability for this is

b(k − 1; x − 1, θ) =

(x − 1

k − 1

)θk−1(1− θ)(x−1)−(k−1)

=

(x − 1

k − 1

)θk−1(1− θ)x−k .

The probability of a success on the xth trial is θ, and theprobability that the kth success occurs on the xth trial is, therefore

θ · b(k − 1; x − 1, θ) = θ

(x − 1

k − 1

)θk−1(1− θ)x−k

=

(x − 1

k − 1

)θk(1− θ)x−k .


Definition 16

A random variable X has negative binomial distribution and it isreferred to as a negative random variable if and only if

b∗(x ; k , θ) =(x − 1

k − 1

)θk(1− θ)x−k

for x = k , k + 1, k + 2, · · · .Thus, the number of the trial on which the kth success occurs is arandom variable having a negative binomial distribution with theparameter k and θ. In the literature of statistics, negative binomialdistributions are also referred to as binomial waiting-timedistributions or as Pascal distributions.


Example 17

If the probability is 0.40 that a child exposed to a certaincontagious disease will catch it, what is the probability that thetent child exposed to the disease will be third to catch it?

Solution. Substituting x = 10, k = 3, and θ = 0.40 into theformula for the negative binomial distribution, we get

b∗(10; 3, 0.40) =(10− 1

3− 1

)0.43(1− 0.4)10−3 =

(9

2

)0.43 · 0.67

=9!

2! · 7!0.43 · 0.67 ≈ 0.0645.


Example 18

Bob is a high school basketball player. He is a 70% free throwshooter. That means his probability of making a free throw is 0.70.During the season, what is the probability that Bob makes his thirdfree throw on his fifth shot?

Solution. Substituting x = 5, k = 3, and θ = 0.70 into theformula for the negative binomial distribution, we get

b∗(5; 3, 0.40) =(5− 1

3− 1

)0.73(1− 0.7)5−3 =

(4

2

)0.73 · 0.32

=4!

2! · 2!0.73 · 0.32 = 0.18522.


Theorem 19

b∗(x ; k , θ) =k

x· b(k ; x , θ).

Proof.

b∗(x ; k , θ) =(x − 1

k − 1

)θk (1− θ)x−k

=(x − 1)!

(k − 1)![(x − 1)− (k − 1)]!θk (1− θ)x−k

=(x − 1)!

(k − 1)!(x − k)!θk(1 − θ)x−k

=k

x· x(x − 1)!

k(k − 1)!(x − k)!θk(1 − θ)x−k

=k

x· x!

k!(x − k)!θk(1− θ)x−k

=k

x·(x

k

)θk(1− θ)x−k =

k

x· b(k ; x , θ).


Example 20

Use Theorem 19 to rework Example 17.

Solution. Substituting x = 10, k = 3, and θ = 0.40 into theformula of Theorem 19, we get

b∗(10; 3, 0.40) =3

10· b(3; 10, 0.40)

=3

10·(10

3

)0.43(1− 0.4)10−3

=3

10·(10

3

)0.43 · 0.67

=3

10· 10!

3! · 7!0.43 · 0.67 ≈ 0.0645.


Theorem 21

The mean and the variance of the negative binomial distributionare

μ =k

θand σ2 =

k

θ

(1

θ− 1

).


Example 22

A web site contains three identical computer servers. Only one isused to operate the site, and the other two are spares that can beactivated in case the primary system fails. The probability of afailure in the primary computer (or any activated spare system)from a request for service is 0.0005. Assuming that each requestrepresents an independent trial, what is the mean number ofrequests until of all three servers fail? What is the probability thatall three servers fail within five requests?


Solution.What is the mean number of requests until of all three servers fail?

Let X denote the number of requests until all three servers fail,and let X1, X2 and X3 denote the number of requests before afailure of the first, second, and third servers used, respectively.Now, X = X1 + X2 + X3. Also, the requests are assumed tocomprise independent trials with constant probability θ = 0.0005.Furthermore, a spare server is not effected by the number ofrequests before it is activated. Therefore, X has a negativebinomial distribution with θ = 0.0005 and k = 3. Consequently,

μX = E (X ) =k

θ=

3

0.0005= 6000 requests.


What is the probability that all three servers fail within five requests?

The probability is P(X ≤ 5) and

P(X ≤ 5) = P(X = 3) + P(X = 4) + P(X = 5)

= b∗(3; 3, 0.0005) + b∗(4; 3, 0.0005) + b∗(5; 3, 0.0005)

=

(2

2

)0.00053(1− 0.0005)3−3

+

(3

2

)0.00053(1− 0.0005)4−3

+

(4

2

)0.00053(1− 0.0005)5−3

= 1.25 × 10−10 + 3.75 × 10−10 + 7.49 × 10−10

= 1.249 × 10−9.


Example 23

Pat is required to sell candy bars to raise money for the 6th gradefield trip. There are thirty houses in the neighborhood, and Pat isnot supposed to return home until five candy bars have been sold.So the child goes door to door, selling candy bars. At each house,there is a 0.4 probability of selling one candy bar. What’s theprobability of selling the last candy bar at the 11th house? What’sthe probability of Pat finishing on or before the 8th house?


Solution.What’s the probability of selling the last candy bar at the 11th house?

Substituting x = 11, k = 5, and θ = 0.40 into the formula for thenegative binomial distribution, we get

P(X = 11) = b∗(11; 5, 0.4)

=

(11− 1

5− 1

)0.45(1− 0.4)11−5

=

(10

4

)0.450.66

= 0.1003


What’s the probability of Pat finishing on or before the 8th house?

P(X ≤ 8) = P(X = 5) + P(X = 6) + P(X = 7) + P(X = 8)

= b∗(5; 5, 0.4) + b∗(6; 5, 0.4) + b∗(7; 5, 0.4) + b∗(8; 5, 0.4)

=

(5− 1

5− 1

)0.45(1− 0.4)5−5 +

(6− 1

5− 1

)0.45(1− 0.4)6−5

+

(7− 1

5− 1

)0.45(1− 0.4)7−5 +

(8− 1

5− 1

)0.45(1− 0.4)8−5

= 0.01024 + 0.03072 + 0.055296 + 0.0774144

= 0.1736704.


Since the negative binomial distribution with k = 1 has manyimportant applications, it is given a spacial name; it is called thegeometric distribution.

Definition 24

A random variable X has a geometric distribution and it isreferred to as a geometric random variable if and only if itsprobability distribution is given by

g(x ; θ) = θ(1− θ)x−1 for x = 1, 2, 3, · · · .


Example 25

If the probability is 0.75 that an applicant for a driver’s license willpass the road test on any given try, what is the probability that anapplicant will finally pass the test on the fourth try?

Solution. Substituting x = 4, and θ = 0.75 into the formula forthe geometric distribution, we get

g(4; 0.75) = 0.75(1 − 0.75)4−1 = 0.75 · 0.253 = 0.01171875.

Of course, this result is based on the assumption that the trials areall independent, and there may be some question here about itsvalidity.


Example 26

From past experience it is known that 3% of accounts in a largeaccounting population are in error. (a) What is the probabilitythat 5 accounts are audited before an account in error is found?(b) What is the probability that the first account in error occurs inthe first five accounts audited?

Solution. (a) Substituting x = 5, and θ = 0.03 into the formulafor the geometric distribution, we get

P(X = 5) = P(5th in error)P(1st four correctly stated)

= g(5; 0.03) = 0.03(1 − 0.03)5−1 = 0.02656

(b)

P(X ≤ 5) = 1− P(first five are correctly stated)

= 1− b(5; 5, 0.97) = 1−(5

5

)0.975(1− 0.97)5−5 = 0.14127.


Theorem 27

The mean and the variance of the geometric distribution are

μ =1

θand σ2 =

1

θ

(1

θ− 1

)=

1− θ

θ2.


Example 28

A rat must choose between five doors, one of which contains achocolate. If the rat chooses the wrong door, it is returned to thestarting point and chooses again, and continues until it gets thechocolate. Let X be the trial on which the chocolate is found. (a)What is the probability of the rat getting chocolate on the firstattempt? 2nd attempt? (b) Find the expected value and varianceof X .

Solution. (a) Since there are 5 doors θ = 1/5 = 0.20.

P(X = 1) = g(1; 0.2) = 0.2(1 − 0.2)1−1 = 0.2.

P(X = 2) = g(2; 0.2) = 0.2(1 − 0.2)2−1 = 0.16.

(b)

μ = E (X ) =1

θ=

1

0.2= 5.

σ2 ==1− θ

θ2=

1− 0.2

0.22= 20.


Thank You!!!

Documents

Lecture 6: Special Probability Distributionsweb.iku.edu.tr/~eyavuz/dersler/probability/6-nopause.pdf · The Discrete Uniform Distribution The Bernoulli Distribution The Binomial Distribution