Lecture XI

The Bernoulli distribution characterizes the coin toss. Specifically, there are two events X=0,1 with X=1 occurring with probability p. The probability distribution function P[X] can be written as:

Fall 2005 Lecture X 2

1[ ] 1xxP X p p

Next, we need to develop the probability of X+Y where both X and Y are identically distributed. If the two events are independent, the probability becomes:

Fall 2005 Lecture X 3

1 1 2

,

1 1 1x y x yx y x y

P X Y P X P Y

p p p p p p

Now, this density function is only concerned with three outcomes Z=X+Y={0,1,2}. There is only one way each for Z=0 or Z=2. Specifically for Z=0, X=0 and Y=0. Similarly, for Z=2, X=1 and Y=1. However, for Z=1 either X=1 and Y=0 or X=0 or Y=1. Thus, we can derive:

Fall 2005 Lecture X 4

Fall 2005 Lecture X 5

2 00

2 1 0 2 0 11 0 0 1

11

02

[ 0] 1

1 1, 0 0, 1

1 1

2 1

2 1

P Z p p

P Z P X Y P X Y

p p p p

p p

P Z p p

Next we expand the distribution to three independent Bernoulli events where Z=W+X+Y={0,1,2,3}.

Fall 2005 Lecture X 6

zz

yxwyxw

yxwyxw

pp

pp

pppppp

YPXPWP

YXWPZP

3

3

111

1

1

111

,,

Again, there is only one way for Z=0 and Z=3. However, there are now three ways for Z=1 or Z=2. Specifically, Z=1 if W=1, X=1 or Y=1. In addition, Z=2 if W=1 and X=1, W=1 and Y=1, and X=1 and Y=1. Thus the general distribution function for Z can now be written as:

Fall 2005 Lecture X 7

Fall 2005 Lecture X 8

3 00

3 1 0 0 3 0 1 01 0 0 0 1 0

3 0 0 1 20 0 1 1

3 1 1 0 3 1 0 11 1 0 1 0 1

3 0 1 1 10 1 1 2

03

0 1

1 1 1

1 3 1

2 1 1

1 3 1

3 1

P Z p p

P Z p p p p

p p p p

P Z p p p p

p p p p

P Z p p

Based on this development, the binomial distribution can be generalized as the sum of n Bernoulli events. For the case above, n=3. The distribution

function (ignoring the constants) can be written as:

Fall 2005 Lecture X 9

Fall 2005 Lecture X 10

3333

3

23232

13131

03030

13

12

11

10

ppCZP

ppCZP

ppCZP

ppCZP

rnrnr ppCrZP 1

The next challenge is to explain the Crn

term. To develop this consider the polynomial expression:

Fall 2005 Lecture X 11

32233

222

1

33

2

babbaaba

bababa

baba

This sequence can be linked to our discussion of the Bernoulli system by letting a=p and b=(1-p). What is of primary interest is the sequence of constants. This sequence is usually referred to as Pascal’s Triangle:

Fall 2005 Lecture X 12

Fall 2005 Lecture X 13

1

1 1

1 2 1

1 3 3 1

1 4 6 4 1

This series of numbers can be written as the combinatorial of n and r, or

Thus, any quadratic can be written as:

Fall 2005 Lecture X 14

!

! !nr

nC

r n r

n

r

rnrnr

n baCba1

As an aside, the quadratic form (a-b)n can be written as:

Fall 2005 Lecture X 15

n

r

rnrnr

rn

n

r

rnrnrnr

nn

baC

baC

baba

1

1

1

Thus, the binomial distribution X~B(n,p) is then written as:

Fall 2005 Lecture X 16

knknk ppCkXP 1

Next recalling Theorem 4.1.6: E[aX+bY]=aE[X]+bE[Y], the expectation of the binomial distribution function can be recovered from the Bernoulli distributions:

Fall 2005 Lecture X 17

Fall 2005 Lecture X 18

npp

pppp

Xpp

kppCXE

n

i

n

i

n

iX

iXX

n

k

knknk

i

ii

1

1

1001

1

1

0

0111

1

1

In addition, by Theorem 4.3.3:

Thus, variance of the binomial is simply the sum of the variances of the Bernoulli distributions or n times the variance of a single Bernoulli distribution

Fall 2005 Lecture X 19

n

i i

n

i i XVXV11

Fall 2005 Lecture X 20

pppp

ppppp

XEXEXV

1

01112

2210201

22

pnpXVn

i i 1

1