Chapter 4: DISCRETE PROBABILITIES AND RANDOM ...bazuinb/ECE3800/B_Notes04.pdfChapter 4: DISCRETE PROBABILITIES AND RANDOM VARIABLES Sections 4.1 Discrete Random Variable and Probability

Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,

Probability, Statistics, and Random Signals, Oxford University Press, February 2016.

B.J. Bazuin, Fall 2020 1 of 51 ECE 3800

Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0

Chapter 4: DISCRETE PROBABILITIES AND RANDOM VARIABLES

Sections 4.1 Discrete Random Variable and Probability Mass Functions 4.2 Cumulative Distribution Functions 4.3 Expected Values 4.4 Moment Generating Functions 4.5 Several Important Discrete PMFs 4.5.1 Uniform PMF 4.5.2 Geometric PMF 4.5.3 The Poisson Distribution 4.6 Gambling and Financial Decision Making Summary Problems




Probability Mass Functions (pmf)

From: http://en.wikipedia.org/wiki/Probability_mass_function

In probability theory, a probability mass function (abbreviated pmf) gives the probability that a discrete random variable is exactly equal to some value.

Cumulative Distribution Function (CDF)

Probability mass functions are related discrete countable outcomes of an experiment and the probability each outcomes has at the discrete values.

A pmf for rolling a six sided dice would be

1 2 3 4 5 6

xf X1.0

0.0x

1/6

The probability of each discrete value is 1 in 6. Therefore, you would define “delta functions” of magnitude 1/6 at each of the six discrete values.

In addition, there is function based on performing a summation from – infinite to +infinite that would be a “cumulative function” of the probability called the Cumulative Distribution Function.




Properties of the pmf include

1. 𝑝 𝑘 𝑃𝑟 𝑋 𝑥 0 (all probabilities are positive)

2. The summation of the pmf for all k is equal to 1.

𝑝 𝑘 1.0

The pmf function may be defined based on a table or other form. For example, a possible pmf would be

𝑝 𝑘

0.4, 𝑘 0 0.3, 𝑘 10.2, 𝑘 20.1, 𝑘 3

where

0 𝑝 𝑘 1

𝑝 𝑘 0.4 0.3 0.2 0.1 1.0

A pmf for flipping a coin

𝑝 𝑘1 𝑝, 𝑘 0𝑝, 𝑘 1




This is also referred to as the Bernoulli Distribution, see

https://en.wikipedia.org/wiki/Bernoulli_distribution

The Bernoulli Distribution is a special case of the Binomial Distribution for (n=1) to be discussed later.




Generalized properties of probability mass function (pmf) (from Stark and Wood)

Properties of the pmf include

1. xforxf X ,0

2. 1

u

X uf

3.

x

u

XX ufxF The CDF is the sum of the pmf from –∞ to x

4.

2

1

21Prx

xuX ufxXx

The probability is the pmf sum of the region(s) of interest

Generalized Properties of CDF (from Stark and Wood)

Cumulative Distribution Function (CDF):The probability of the event that the observed random variable X is less than or equal to the allowed value x.

xXxFX Pr

The defined function can be discrete or continuous along the x-axis. Constraints on the cumulative distribution function are:

xforxFX ,10

0XF and 1XF (property #1 and #3 in the textbook)

XF is non-decreasing as x increases (property #2 in the textbook)

1221Pr xFxFxXx XX Notice that the “inequalities” are important for discrete random variables!




For discrete events, the cumulative density function, on the x-axis, consists of discrete steps “climbing” towards 1 at the appropriate points.

For a six-sided die,

6

161,Pr intint egereger aaX

The cumulative density function can be defined as:

For discrete events, 061,Pr intint egereger aaX or

061,Pr intintintint egerXegerXegereger aFaFaaX

There will be a difference for continuous events … coming soon.

Examples (watch the definition of the inequalities):

6

111Pr XFX

2

133Pr XFX

6

555Pr XFX

0.177Pr XFX

6

2

6

41414Pr XFX

6

3

6

2

6

52552Pr XX FFX

From: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press.




Important Discrete Random Variables

The Uniform Random Variable

The Bernoulli Random Variable

The Binomial Random Variable

The Geometric Random Variable

The Poisson Random Variable

The Zipf Random Variable

Definitions and examples available on homework solution/password web site.




UpdatingPreviousExamples

Experiment: Flip two Coins and count the number of heads

HHHTTHTTSPair ,,, 2,1,0S

For xXxFX Pr

xfor

xfor

xfor

xfor

xFX

2,1

21,43

10,41

0,0

And the probability mass function, xXxf X Pr , is then

else

xfor

xfor

xfor

xf X

,0

2,41

1,42

0,41

1 2 3 4

xFX

1.0

0.0x

0-1 1 2 3 4

xf X

1.0

0.0x1/4

0-1

1/2

Cumulative Distribution Function (CDF)

Probability Mass Function (pmf)

Note: The pmf corresponds to multiple Bernoulli trials resulting in a Binomial Probability of n=2 trials with a probability of p=50%.

𝑃𝑟 𝐴 𝑜𝑐𝑐𝑢𝑟𝑖𝑛𝑔 𝑘 𝑡𝑖𝑚𝑒𝑠 𝑖𝑛 𝑛 𝑡𝑟𝑖𝑎𝑙𝑠 𝑝 𝑘𝑛𝑘 ∙ 𝑝 ∙ 1 𝑝

𝑝 0 20

∙ 𝑝 ∙ 1 𝑝 𝑝 1 21

∙ 𝑝 ∙ 1 𝑝 𝑝 2 22

∙ 𝑝 ∙ 1 𝑝

𝑝 0 1 ∙ 0.5 𝑝 1 2 ∙ 0.5 𝑝 2 1 ∙ 0.5




4.3 Expected Values, Moments, Central Moments and Variance

For random variables, the expected value operation produces a “probabilistic average” of the particular probability based function of interest.

𝐸 𝑔 𝑋 𝑔 𝑥 ∙ 𝑝 𝑘

https://en.wikipedia.org/wiki/Expected_value

where g(X) is a function of the random variable X and p(k) is the pmf. (Chapter 3 ROI calculations are expected values!)

Meanor1stMoment

For example, the 1st moment or mean value of the random variable is defined by

𝜇 ≡ 𝐸 𝑋 𝑋 𝑥 ∙ 𝑝 𝑘

Themeansquarevalueorsecondmomentis

𝐸 𝑋 𝑋 𝑥 ∙ 𝑝 𝑘

Other“moments”(thenthmoment)aredefinedas

𝐸 𝑋 𝑋 𝑥 ∙ 𝑝 𝑘

CentralMoments

𝐸 𝑋 𝜇 𝑋 𝜇 ∙ 𝑝 𝑘

The2ndcentralmomentorvariance

𝜎 ≡ 𝐸 𝑋 𝜇 𝑋 𝜇 ∙ 𝑝 𝑘

The standard deviation is defined in terms of the 2nd central moment (or variance) as

𝜎 𝜎




Defined PMF functions: mean and standard deviation

Importance of mean and standard deviation

Often when we talk about values we say the “mean +/- 1 standard deviation”

𝜇 𝜎

That is to say that we expect the experimental result x to be

𝜇 𝜎 𝑥 𝜇 𝜎




A useful variance formula – moments proof (Theorem 4.1)

𝜎 𝐸 𝑋 𝜇 𝑋 𝜇 ∙ 𝑝 𝑘

𝜎 𝐸 𝑋 𝜇 𝑋 2 ∙ 𝜇 ∙ 𝑋 𝜇 ∙ 𝑝 𝑘

𝜎 𝐸 𝑋 𝜇 𝑋 ∙ 𝑝 𝑘 2 ∙ 𝜇 ∙ 𝑋 ∙ 𝑝 𝑘 𝜇 ∙ 𝑝 𝑘

𝜎 𝐸 𝑋 𝜇 𝐸 𝑋 2 ∙ 𝜇 𝜇

𝜎 𝐸 𝑋 𝜇 𝐸 𝑋 𝜇

or

𝐸 𝑋 𝜇 𝐸 𝑋 𝐸 𝑋

You need only compute the 1st and 2nd moment to derive the variance.

Alternate expected value operator proof

𝜎 𝐸 𝑋 𝜇 𝐸 𝑋 2 ∙ 𝜇 ∙ 𝑋 𝜇

𝜎 𝐸 𝑋 2 ∙ 𝜇 ∙ 𝐸 𝑋 𝐸 𝜇

𝜎 𝐸 𝑋 2 ∙ 𝜇 ∙ 𝜇 𝜇

𝜎 𝐸 𝑋 𝜇

If you know the mean and standard deviation, you can compute the 2nd moment. Note that the second moment is related to signal power/energy!

𝐸 𝑋 𝜇 𝜎




More basics math related to “the expected value operator”

𝐸 𝑔 𝑋 𝑔 𝑥 ∙ 𝑝 𝑘

A constant, non-random variable

𝐸 𝑐 𝑐 ∙ 𝑝 𝑘 𝑐 ∙ 𝑝 𝑘 𝑐

A constant multiplication

𝐸 𝑐 ∙ 𝑔 𝑋 𝑐 ∙ 𝑔 𝑋 ∙ 𝑝 𝑘 𝑐 ∙ 𝑔 𝑋 ∙ 𝑝 𝑘 𝑐 ∙ 𝐸 𝑔 𝑋

Therefore

𝐸 𝑎 ∙ 𝑋 𝑏 𝑎 ∙ 𝑋 𝑏 ∙ 𝑝 𝑘 𝑏 𝑎 ∙ 𝑋 ∙ 𝑝 𝑘 𝑎 ∙ 𝜇 𝑏

Note: 𝐸 𝑌 𝐸 𝑎 ∙ 𝑋 𝑏 𝑎 ∙ 𝜇 𝑏 𝜇

Summations (superposition)

𝐸 𝑔 𝑋 𝑔 𝑋 𝑔 𝑋 𝑔 𝑋 ∙ 𝑝 𝑘 𝑔 𝑋 ∙ 𝑝 𝑘 𝑔 𝑋 ∙ 𝑝 𝑘

𝐸 𝑔 𝑋 𝑔 𝑋 𝐸 𝑔 𝑋 𝐸 𝑔 𝑋

Note that multiplication of two functions generally does not work!




The expected value operator is a linear operator … therefore: the integration and differentiation of a function of random variables can be performed as.

𝑑𝑑𝑣

𝐸 𝑔 𝑋, 𝑣𝑑𝑑𝑣

𝑔 𝑋, 𝑣 ∙ 𝑝 𝑘𝑑𝑑𝑣

𝑔 𝑋, 𝑣 ∙ 𝑝 𝑘 𝐸𝑑𝑑𝑣

𝑔 𝑋, 𝑣

𝐸 𝑔 𝑋, 𝑣 ∙ 𝑑𝑣 𝑔 𝑋, 𝑣 ∙ 𝑝 𝑘 ∙ 𝑑𝑣 𝑔 𝑋, 𝑣 ∙ 𝑑𝑣 ∙ 𝑝 𝑘

𝐸 𝑔 𝑋, 𝑣 ∙ 𝑑𝑣 𝐸 𝑔 𝑋, 𝑣 ∙ 𝑑𝑣




Some Common Discrete Random Variables

1. Bernoulli – (flipping coins, one of two results)

1,0XS

qpp 10 and pp 1 , for 10 p

else

kp

kq

kpmfkPB

,0

1,

0,

1 kpkqkpmfkPB

k

kq

k

kFB

1,1

10,

0,0

1 kupkuqkpmfkFB

Mean, 2nd moment and variance

𝜇 ≡ 𝐸 𝑋 𝑋 𝑘 ∙ 𝑝 𝑘

𝜇 ≡ 𝐸 𝑋 𝑋 0 ∙ 𝑞 1 ∙ 𝑝 𝑝

𝐸 𝑋 𝑋 𝑘 ∙ 𝑝 𝑘

𝐸 𝑋 𝑋 0 ∙ 𝑞 1 ∙ 𝑝 𝑝

𝜎 𝐸 𝑋 𝜇

𝜎 𝑝 𝑝 𝑝 ∙ 1 𝑝 𝑝 ∙ 𝑞




2. Binomial – (S a sequence of Bernoulli trials)

nS X ,,2,1,0

knkk pp

k

np

1 , for nk ,,2,1,0

else

nkppk

n

kpmfkPknk

B

,0

,,1,0,1

kn

nkppj

n

k

kFk

j

jnjB

,1

0,1

0,0

0

Mean, 2nd moment and variance

See Example 4.3-1




3. Geometric

First Version

,2,1,0XS

𝑃 𝑘 𝑝𝑚𝑓 𝑘 𝑝 ⋅ 1 𝑝 , 𝑘 0,1,⋯ ,∞0, 𝑒𝑙𝑠𝑒

𝐹 𝑘

0,𝑘 0

𝑝 ⋅ 1 𝑝 , 0 𝑘 ∞

Math Tricks ….

1,1

11

1

00

qfor

q

qpqppp

kk

j

jk

j

j

1

1

0

111

11

k

kk

j

j qp

qppp

Therefore, it is commonly stated as

𝐹 𝑘0,𝑘 0

𝑝 ⋅1 𝑞

1 𝑞, 0 𝑘 ∞

Alternate version xaaxpmfxP xXX 0,1

Second Version

,2,1XS

𝑃 𝑘 𝑝𝑚𝑓 𝑘 𝑝 ⋅ 1 𝑝 , 𝑘 1,2,⋯ ,∞0, 𝑒𝑙𝑠𝑒

𝐹 𝑘0,𝑘 1

𝑝 ⋅1 𝑞1 𝑞

, 1 𝑘 ∞




Determine the expected value of the 1st version (caution different notations!)

0x

X nPxXE

𝐸 𝑋 𝑥 ⋅ 1 𝑎 ⋅ 𝑎

∞

1 𝑎 ⋅ 𝑥 ⋅ 𝑎

∞

Note: 1,1

1

0

afora

ax

x

and 2

0

1

0 1

1

1

1

aada

daxa

da

d

x

x

x

x

2

0

1

1

111

aaaaxaaXE

x

x

𝐸 𝑋 𝜇𝑎

1 𝑎

This allows the mean value to be quickly found once “a” is known. Determine the 2nd moment

0

22

xX nPxXE

𝐸 𝑋 𝑥 ⋅ 1 𝑎 ⋅ 𝑎

∞

Note: 32

2

0

2

02

2

1

2

1

11

aada

daxxa

da

d

x

x

x

x

0

2222 11x

xx axaxxaaXE

0

1

0

222 111x

x

x

x axaaaxxaaXE

23

22

1

11

1

21

aaa

aaaXE

𝐸 𝑋2 ⋅ 𝑎

1 𝑎𝑎

1 𝑎2 ⋅ 𝜇 𝜇

Determine the variance

𝐸 𝑋 𝜇 𝐸 𝑋 𝐸 𝑋

2

2

22

111

2

a

a

a

a

a

aXE

𝐸 𝑋 𝜇𝑎

1 𝑎𝑎

1 𝑎𝜇 𝜇




A six-sided die example (uniform pmf)

𝑝 𝑘

0, 𝑘 116

, 𝑘 1,2,3,4,5,6

0, 6 𝑘

Mean value

𝜇 ≡ 𝐸 𝑋 𝑥 ∙ 𝑝 𝑘 𝑘 ∙16

16∙ 𝑘

16∙

7 ∙ 62

72

3.5

and

𝜎 𝐸 𝑋 𝜇 𝑋 𝜇 ∙ 𝑝 𝑘 𝑘 𝜇 ∙16

𝜎 𝑘 2 ∙ 𝑘 ∙ 𝜇 𝜇 ∙16

16∙ 𝑘 2 ∙ 𝜇 ∙

16∙ 𝑘

16∙ 𝜇 1

𝜎16∙

13 ∙ 7 ∙ 66

2 ∙ 𝜇 ∙ 𝜇 𝜇916

𝜇

𝜎916

494

182 14712

3512

2.927

The standard deviation is

𝜎 𝜎 =1.708

Note that a generalized form for a uniform discrete random variable exists!

*Discrete Math Hints

𝑘𝑚 1 ∙ 𝑚

2

𝑘2 ∙ 𝑚 1 ∙ 𝑚 1 ∙ 𝑚

6

𝑘𝑚 1 ∙ 𝑚

4




Bernoulli Trials

A repeated trial can take the form of:

1. Repeated experiments where the relative frequency of occurrence is of interest

2. The creation of a new experiment that consists of a defined number of elementary events

Bernoulli Trials: Determining the probability that an event occurs k times in n independent trials of an experiment.

For some experiment let: pA Pr and qA Pr

where 1 qp

Then for an experiment where we get 2 event “A”s followed by 2 “not A” (i.e., AAAAB ) …

knk qpAAAAB PrPrPrPrPr

But what about the other ways to have 2 event A’s in 4 trials? Note that for each instance, the probability of occurring will be the same as just defined … so how many of them are there?

AAAAAAAAAAAAAAAAAAAAAAAA ,,,,,

The number of occurrences can be defined using binomial coefficients and the Binomial Theorem.

The number of instances is defined by the binomial coefficient, kn C or

k

n.

the number of ways to select k elements out of a set of n elements ...

Where !!

!

knk

n

k

n

Therefore, to describe the desired outcome of 2 A’s in 4 trials, the probability is

242242

4 !24!2

4

2

4242Pr

qpqpptrialsintimesoccuringA

Therefore …




Binomial Probability

The probability that an event occurs k times in n independent trials of an experiment can be defined as

knkn qp

k

nkptrialsnintimeskoccuringA

Pr

ExampleFlippingCoins

The probability for each outcome of flipping a coin 4 times, where Pr(H)= p and Pr(T)=q with

2

1 qp

4 H : 16

1

16

11

2

1

2

1

4

4

4

44Pr

04444

4

qppHHHH

3 H & 1 T: 16

4

16

14

2

1

2

1

3

4

3

43Pr

13343

4

qppHHHT

2 H & 2 T: 16

6

16

16

2

1

2

1

2

4

2

42Pr

22242

4

qppHHTT

1 H & 3 T: 16

4

16

14

2

1

2

1

1

4

1

41Pr

31141

4

qppHTTT

4 T: 16

1

16

11

2

1

2

1

0

4

0

40Pr

40040

4

qppTTTT

What if p = 0.6 and q=0.4? An “unfair coin”!!




ExampleBinaryCommunications

Example 1: For a bit-error-rate (BER) of 310 in a binary data stream, what is the probability of exactly 1 error in a 32-bit word?

3131332 10110

1

321

p

313332 10110321 p

0310.09695.010321 332 p

Example 2: For a bit-error-rate (BER) of 310 in a binary data stream, what is the probability of 0 errors in a 32-bit word?

3230332 10110

0

320

p

32332 101110 p

𝑝 0 0.9685

Example 3: What is that probability of having one or more errors in 32 bits?

32

1

323332

132 10110

32

i

ii

i iip

or 0315.09685.0101 32

32

132

pipi

Notice that 1 bit error dominates the computation …




ExampleBaseball/SoftballStatistics

Example 1: A batter has a 0.250 batting average. What is the probability that the batter gets 1 hit in 4 at bats?

knkn qp

k


Pr

314 25.0125.0

1

41

p

422.064

27

444

333

4

1475.025.0

!14!1

!41 31

4

p

Example 2: A batter has a 0.250 batting average. What is the probability that the batter gets 2 hit in 4 at bats?

224 25.0125.0

2

42

p

211.0128

27

44

33

44

11

2

3475.025.0

!24!2

!42 22

4

p

Example 3: A batter has a 0.250 batting average. What is the probability that the batter gets at least 1 hit in 4 at bats?

014321 44444 ppppp

684.0256

175

256

811

4444

33331175.025.0

!04!0

!4101 40

4

p

Example 4: A batter has a 0.250 batting average. What is the probability that the batter gets at most 1 hit in 4 at bats?

314044 75.025.0

!14!1

!475.025.0

!04!0

!410

pp

738.0256

189

256

108

256

81

444

333

4

14

4444

3333110 44

pp

Defining a player having a hitting slump … how many at bats until it is a slump?

How many at bats would the batter need to take … to have a 90% (or 99%) probability of getting at least one hit.

Miguel Cabrera’s career BA ….was 0.315 ? (see Excel Spread Sheet

1 𝑝 0 0.900 or 1 𝑝 0 0.990

1 𝑝 0 0.8949 or 1 𝑝 0 0.9889

See Baseball Spread Sheet




Example 1-10.2 from Cooper-McGillem

In playing an opponent of equal ability, which is more probable:

knkn qp

k


Pr

a) To win 4 games out of 7, or to win 5 games out of 9?

3744747 5.05.0

!3!4

!7

4

74

qpp

2734.0128

1

6

2105.0

23

5674 7

7

p

5955959 5.05.0

!4!5

!9

5

95

qpp

2461.0512

1

24

30245.0

234

67895 9

9

p

Therefore, winning 4 out of 7 is more probable.

b) To win at least 4 games out of 7, or to win at least 5 games out of 9.

77777 5.0

7

7

6

7

5

7

4

77654

pppp

128

1

1

1

1

7

2

42

6

2107654 7777

pppp

50.0128

64

128

11721357654 7777 pppp

999999 5.09

9

8

9

7

9

6

9

5

998765

ppppp

512

11

1

9

2

72

6

504

24

302498765 99999

ppppp

50.0512

256

512

119368412698765 99999 ppppp

The probabilities are the same! (You should have a 50-50 chance of winning or losing) !




ECE Applications of Bernoulli Trials

(1) Bit errors in binary transmissions:

Degree of error detection and correction needed. The theoretical validation of performance of the system after “extra bits” for error correction have been added.

bit-error-rate may also increase if a greater bandwidth is needed because of the “extra bits”

(2) Radar (or similar) signal detection:

After setting a signal detection threshold, the expected signal should be above the threshold when being received for a fixed number of sample times. If the signal is above the threshold for m (or more) of n sample periods, one may also say the signal has been detected.

n

mk

kns

ks

n

mks pp

k

nknpDetection 1,Pr

One can also define a noise threshold where the noise should not be above a particularly level more than m (or more) of n time samples.

n

mk

kna

ka

n

mka pp

k

nknpAlarmFalse 1,_Pr

(3) System reliability improvement using redundancy.

If a unit has a known failure rate, by incorporating redundant units, the system will have a longer expected lifetime.

Important when dealing with systems that cannot be serviced, systems that may be very expensive to service, systems that require very high reliability, system with components with high failure rates, etc. . (e.g. satellites, computer hard-disk farms, internet order entry servers).

Defining the probability that one of the redundant elements is still working …

FailedAllFunctional _Pr(1Pr




Moments of Random Variables

Example4.3‐1Binomial/BernoulliR.V.StarkandWoods

xnxXX pp

x

nxpmfxP

1

Determine the expected value

n

xX nPxXE

0

n

x

xnx ppx

nxXE

0

1

pnXE

Proof based on Wikipedia https://en.wikipedia.org/wiki/Binomial_distribution

n

x

xnxn

x

xnx ppxxn

nxpp

xxn

nxXE

10

1!!

!1

!!

!

Subtle change since you are multiplying by x, the x=0 term is always zero. As the math trick, can the “combinatorial” and probability be restructure to sum to one based on the “n-1” term summation that remains?

n

x

xnx ppxxn

nXE

1

1!1!

!

n

x

xnx pppxxn

nnXE

1

111 1!1!11

!1

n

x

xnx ppxxn

npnXE

1

111 1!1!11

!1

Let 1 xy

1

0

11!!1

!1n

y

yny ppyyn

npnXE

But the 1.0 desired is 11!!

!

0

m

y

ymy ppyym

m

Therefore pnXE




Determine the 2nd moment

0

22

xX nPxXE

pnpnnXE 22 1

Proof

𝐸 𝑋 𝑥 ∙𝑛!

𝑛 𝑥 ! ∙ 𝑥!∙ 𝑝 ∙ 1 𝑝 𝑥 ∙

𝑛!𝑛 𝑥 ! ∙ 𝑥!

∙ 𝑝 ∙ 1 𝑝

Cancelling one of the x and adjusting the summation. Now make two terms …

𝐸 𝑋 𝑥 1 1 ∙𝑛!

𝑛 𝑥 ! ∙ 𝑥 1 !∙ 𝑝 ∙ 1 𝑝

n

x

xnxn

x

xnx ppxxn

npp

xxn

nxXE

11

2 1!1!

!1

!1!

!1

The second term was previously computed (math trick … make summation = 1.0). The first term can now “cancel” an (x-1) … and look for a summation in x-2 terms.

pnppxxn

nXE

n

x

xnx

2

2 1!2!

!

n

x

xnx pppxxn

nnnpnXE

2

22222 1!2!22

1!2

n

x

xnx ppxxn

npnnpnXE

2

22222 1!2!22

!21

11 22 pnnpnXE

pnpnnXE 22 1

Next determine the variance





222 XEXEXE

222 1 pnpnpnnXE

pnpnpnpnpnpnXE 2222222

qpnppnXE 12

We can evaluate the variance form

𝐸 𝑋 𝜇𝑛

𝜎𝑛

𝑝 ∙ 𝑞

The maximum variance occurs for p=0.5 with minimal variances near p=0 or p=1.0.

Figure 4.3-1 Variance of a binomial RV versus p.

The variances is the largest when the individual event probability is p=q=0.5 !

If you were building a communications system, this can be used to define what the desired bit-wise probability should be to send the most information per bit !




Example Geometric Distribution (not identical to textbook index is offset by 1!)

Geometric Distribution:

In probability theory and statistics, the geometric distribution is either of two discrete probability distributions:

The probability distribution of the number Y of Bernoulli trials needed to get one success, supported on the set { 1, 2, 3, ...}

The probability distribution of the number X = Y − 1 of failures before the first success, supported on the set { 0, 1, 2, 3, ... }

https://en.wikipedia.org/wiki/Geometric_distribution

The number of heads before the first tail or the number of failures before the first success is:

𝑃 𝑥 𝑝𝑚𝑓 𝑥 𝑎 ∙ 1 𝑎 , 0 𝑥

𝑃 𝑥 𝑝𝑚𝑓 𝑥 1 𝑝 ∙ 𝑝, 0 𝑥


0x

X nPxXE

00

11x

x

x

x axaaaxXE

Note: 1,1

1

0

afora

ax

x

and 2

0

1

0 1

1

1

1

aada

daxa

da

d

x

x

x

x

2

0

1

1

111

aaaaxaaXE

x

x

𝐸 𝑋 𝜇𝑎

1 𝑎

𝐸 𝑋 𝜇1 𝑝𝑝

This allows the mean value to be quickly found once “a” is known.





0

22

xX nPxXE

0

22 1x

xaaxXE

Note: 32

2

0

2

02

2

1

2

1

11

aada

daxxa

da

d

x

x

x

x

0

2222 11x

xx axaxxaaXE

0

1

0

222 111x

x

x

x axaaaxxaaXE

23

22

1

11

1

21

aaa

aaaXE

22

22 2

11

2

a

a

a

aXE


222 XEXEXE

2

2

22

111

2

a

a

a

a

a

aXE

22

22

11 a

a

a

aXE

𝐸 𝑋 𝜇𝑎

1 𝑎𝑎

1 𝑎𝜇 𝜇

𝐸 𝑋 𝜇1 𝑝𝑝

1 𝑝𝑝

1 𝑝 𝑝 ∙ 1 𝑝𝑝

1 𝑝𝑝

𝐸 𝑋 𝜇 𝜎1 𝑝𝑝




Now can you repeat the computations for the other form of the geometric probability?

𝑃 𝑘 𝑝𝑚𝑓 𝑘 1 𝑎 ∙ 𝑎 ∙, 1 𝑘

𝑃 𝑘 𝑝𝑚𝑓 𝑘 𝑝 ∙ 1 𝑝 , 1 𝑘

resulting in (Assuming Y=X+1 based on the 𝑋 𝑓𝑜𝑟 0 𝑥 and 𝑌 𝑓𝑜𝑟 1 𝑦 )

𝐸 𝑌 𝜇 𝐸 𝑋 1 𝜇1 𝑝𝑝

11𝑝

𝐸 𝑌 𝜇 𝜎 𝐸 𝑋 𝜇1 𝑝𝑝

Note that the shape of the CDF is the same for the two distributions. Therefore, the means are different (shifted), but the variances must be identical.

[class derivation of equivalence of variance ?]

The geometric random variable arises in applications where one is interested in the time (i.e., number of trials) that elapses between the occurrences of events in a sequence of independent experiments. Examples where the modified geometric random variable arises are:

number of customers awaiting service in a queueing system (line at grocery store or DMV);

number of white dots between successive black dots in a scan of a black-and-white document.

https://en.wikipedia.org/wiki/Geometric_distribution




Chebyshev Inequality (Stark and Woods – continuous derivation)

There are a number of probability relationships that bound aspects of engineering problems. They are typically based on moments, particularly the mean and variance. This is the first.

The Chebyshev inequality furnishes a bound on the probability of how much an R.V. can deviate from its mean value.

ChebyshevinequalityTheorem4.4‐1

Let X be an arbitrary R.V. with known mean and variance. Then for any 0

2

2

XXXP

Derivation

dxxfXxXXEXX X

2222

Then

Xx

XX dxxfXxdxxfXx222

and

XxPdxxfdxxfXxXx

X

Xx

X2222

Results #1:

XxP2

2

If we also consider the complement of the probability described,

1 XxPXxP

and using the complement

XxP12

2

Therefore

Results #2: 2

2

1 XxP




It may be convenient to define the delta function in terms of a multiple of the standard deviation.

k

Then the Chebyshev inequality becomes

2

1

kkXxP

2

11

kkXxP




Chebyshev Inequality Textbook description

Defining a bound based on the variance of a random variable.

𝑃𝑟 |𝑋 𝜇 | 𝜖𝑉𝑎𝑟 𝑋𝜖

Define an indicator factor

𝐼 𝑥1, 𝑥 ∈ 𝐴0, 𝑥 ∉ 𝐴

Taking the expected value

𝐸 𝐼 𝑥 𝐼 𝑥 ∙ 𝑝 𝑘 𝐼 𝑥 ∙ 𝑃𝑟 𝑋 𝑥

𝐸 𝐼 𝑥 1 ∙ 𝑃𝑟 𝑋 𝑥: ∈

𝑃𝑟 𝑋 ∈ 𝐴

For the Chebyshev Inequality the set A is defined based on the equation

𝐴 |𝑋 𝜇 | 𝜖

which can be interpreted as

𝐼| |𝑋 𝜇𝜖

which can be graphically depicted as

Notice that the two functions are equal for

𝑋 𝜇𝜖

1




or

𝑋 𝜇 𝜖

Based on previous derivations, we know that

𝐸𝑋 𝜇𝜖

𝐸 𝑋 𝜇𝜖

𝑉𝑎𝑟 𝑋𝜖

The taking an expected value of the previous interpreted set

𝐸 𝐼| | 𝐸𝑋 𝜇𝜖


or based on the set

𝐸 𝐼| | 𝑃𝑟 𝑋 ∈ 𝐴 𝑃𝑟 |𝑋 𝜇 | 𝜖 𝐸𝑋 𝜇𝜖


Note that the Chebyshev Inequality is a bound that applies in all cases. There is no judgment or determination if it is a good or even useful bound!

Note that for any R.V. where the variance tends to zero, you would have

𝑃𝑟 |𝑋 𝜇 | 𝜖 → 0

and the “zero variance random variable must equal the mean value!

Extension, relating epsilon to sigma …

𝜖 𝑘 ∙ 𝜎 with 𝑉𝑎𝑟 𝑋 𝜎

𝑃𝑟 |𝑋 𝜇 | 𝑘 ∙ 𝜎𝜎𝑘 ∙ 𝜎

𝑃𝑟 |𝑋 𝜇 | 𝑘 ∙ 𝜎1𝑘

or considering the negative of the probability function

11𝑘

1 𝑃𝑟 |𝑋 𝜇 | 𝑘 ∙ 𝜎

or

11𝑘

𝑃𝑟 |𝑋 𝜇 | 𝑘 ∙ 𝜎

Note that this is developed here, but will be used and discussed much later in the course.




Moment-Generating Functions (Stark & Woods)

The text is now moving into some advanced concepts that support mathematical derivation of higher order moments.

I have been exposed to problems where the 4th moment of a R.V. is required as part of a solution. If you really like and are comfortable with Laplace and Fourier Transforms these approach provide solutions faster and more easily than more brute force summation or integral approaches.

The moment generation function (MGF) is the two sided Laplace transform of the probability mass function (pmf) or the probability density function (pdf). If the MGF exists, there is a forward and inverse relationship between the MGF and the pmf/pdf. The MGF is defined based on the expected value as

𝑀 𝑢 𝐸 𝑒𝑥𝑝 𝑢 ∙ 𝑋

𝑀 𝑢 𝑒𝑥𝑝 𝑢 ∙ 𝑋 ∙ 𝑝 𝑘

For continuous pdf, we would have

dxxtxftM XX exp

If you like s better than t in your Laplace transforms …

dxxsxfsM XX exp

For discrete R.V. we perform a discrete Laplace transform

i

iiXi

iiXX xsxPxsxpmfsM expexp

Why do we do this?

1. It enables a convenient computation of the higher order moments

2. It can be used to estimate fx(x) from experimental measurements of the moments

3. It can be used to solve problems involving the computation of the sums of R.V.

4. It is an important analytical instrument that can be used to demonstrate results and establish additional bounds (the Chernoff Bound and the Central Limit Theorem).




TheMGFenablesaconvenientcomputationofthehigherordermoments

Based on the definition

XtEtM X exp

Perform the Taylor series expansion of the exponential

𝑒𝑥𝑝 𝑥 1𝑥1!

𝑥2!

⋯𝑥𝑛!

⋯

𝑀 𝑡 𝐸 𝑒𝑥𝑝 𝑡 ∙ 𝑋 𝐸 1𝑡 ∙ 𝑋

1!𝑡 ∙ 𝑋

2!⋯

𝑡 ∙ 𝑋𝑛!

⋯

or

𝑀 𝑡 𝐸 𝑒𝑥𝑝 𝑡 ∙ 𝑋 1𝑡 ∙ 𝑚

1!𝑡 ∙ 𝑚

2!⋯

𝑡 ∙ 𝑚𝑛!

⋯

The mi are the ith moments of the density function!

So how would we solve for the moments? By taking the derivatives and setting t=0!

Taking the 1st derivative …

𝜕𝜕𝑡

𝑀 𝑡𝜕𝜕𝑡𝐸 𝑒𝑥𝑝 𝑡 ∙ 𝑋 0

𝑚1!

2 ∙ 𝑡 ∙ 𝑚2!

⋯𝑛 ∙ 𝑡 ∙ 𝑚

𝑛!⋯

Setting t=0

𝜕𝜕𝑡

𝑀 𝑡 0𝑚1!

0 ⋯ 0 ⋯𝑚1!

𝑚

Taking the nth derivative …

𝜕𝜕𝑡

𝑀 𝑡 0 ⋯ 0𝑛! ∙ 𝑚𝑛!

𝑛! ∙ 𝑡 ∙ 𝑚𝑛 1!

⋯

Setting t=0

𝜕𝜕𝑡

𝑀 𝑡 0 ⋯ 0𝑛! ∙ 𝑚𝑛!

0 ⋯𝑛! ∙ 𝑚𝑛!

𝑚

Therefore, all moments can be determined if the moment generation function exists!




Additionalusefulexamples:

Example 4.5-2 MGF of Binomial

knkX qp

k

nkpmf

MGF:

dxxtxftM XX exp

n

n

knkX ktqp

k

ntM

0

exp

n

n

knkX qtp

k

ntM

0

exp

Magical math steps … not really, but I haven’t done the derivation myself …

nX qtptM exp

The 1st Moment

0

1

0

expexpexp

t

nn

tX tpqtpnqtp

ttM

t

pnpqpntMt

n

tX

1

0

The 2nd Moment

0

1

02

2

expexp

t

n

t

X qtptpnt

tMt

0

2

0

1

02

2

expexp1exp

expexp

t

n

t

n

t

X

tpqtpntpn

qtptpntMt

221

02

2

1

nn

t

X qpnpnqppntMt




ppnpnnpnpntMt

t

X

11 22

02

2

qpnpntMt

t

X

2

02

2




Example 4.5-3 MGF of Geometric Distribution

10,1 fornpmf nX

MGF:

dxxtxftM XX exp

0

exp1n

nX nttM

Note infinite sum: 1,1

1

0

afora

ax

x

ttM X exp1

1

The 1st Moment

0

20

expexp1

1

exp1

1

ttX t

ttttM

t

111

11

exp1

exp12

0

20 tt

Xt

ttM

t

The 2nd Moment

2

02

2

exp1

exp1

t

t

ttM

tt

X

tt

t

t

ttM

tt

X expexp1

exp12

exp1

exp132

02

2

0

3

2

0

20

2

2

exp1

2exp12

exp1

exp1

ttt

Xt

t

t

ttM

t

2

2

02

2

1

2

1

t

X tMt




Discrete Uniform pmf

The same probability for m different values from 1 to m.

𝑃𝑟 𝑋 𝑘

0, 𝑘 11𝑚

, 1 𝑘 𝑚

0, 𝑚 𝑘

Computing the mean or first moment

𝐸 𝑋 𝜇 𝑘 ∙1𝑚

𝐸 𝑋 𝜇1𝑚∙ 𝑘

1𝑚∙𝑚 ∙ 𝑚 1

2𝑚 1

2

Computing the second moment

𝐸 𝑋 𝑘 ∙1𝑚

𝐸 𝑋1𝑚∙ 𝑘

1𝑚∙

2 ∙ 𝑚 1 ∙ 𝑚 1 ∙ 𝑚3 ∙ 2

2 ∙ 𝑚 1 ∙ 𝑚 16

𝐸 𝑋2 ∙ 𝑚 3 ∙ 𝑚 1

6

Computing the variance or second central moment

𝐸 𝑋 𝜇 𝑉𝑎𝑟 𝑋 𝜎 𝐸 𝑋 𝜇

𝑉𝑎𝑟 𝑋 𝜎2 ∙ 𝑚 3 ∙ 𝑚 1

6𝑚 1

2

𝑉𝑎𝑟 𝑋 𝜎2 ∙ 𝑚 3 ∙ 𝑚 1

6𝑚 2 ∙ 𝑚 1

4

𝑉𝑎𝑟 𝑋 𝜎4 ∙ 𝑚 6 ∙ 𝑚 2 3 ∙ 𝑚 6 ∙ 𝑚 3

12𝑚 1

12

𝑉𝑎𝑟 𝑋 𝜎𝑚 1

12𝑚 1 ∙ 𝑚 1

12

Computing the Moment Generating Function




𝑀 𝑢 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙ 𝑝 𝑘 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙1𝑚

𝑀 𝑢1𝑚∙ 𝑒𝑥𝑝 𝑢

1𝑚∙ 𝑒𝑥𝑝 𝑢 ∙ 𝑒𝑥𝑝 𝑢

𝑀 𝑢1𝑚∙ 𝑒𝑥𝑝 𝑢 ∙

1 𝑒𝑥𝑝 𝑢 ∙ 𝑚1 𝑒𝑥𝑝 𝑢

1𝑚∙𝑒𝑥𝑝 𝑢 ∙ 𝑚 1 𝑒𝑥𝑝 𝑢

𝑒𝑥𝑝 𝑢 1

Or example 4.5 on p. 86-87

Note that the computation of the moments is not straight forward and requires L’Hospital’s rule instead of direct derivation! (Note that at u=0 you always get “0/0”.)




Binomial pmf review

xnxXX pp

x

nxpmfxP

1


n

xX nPxXE

0

n

x

xnx ppx

nxXE

0

1

n

x

xnxn

x

xnx ppxxn

nxpp

xxn

nxXE

10

1!!

!1

!!

!

n

x

xnx ppxxn

nXE

1

1!1!

!

n

x

xnx pppxxn

nnXE

1

111 1!1!11

!1

n

x

xnx ppxxn

npnXE

1

111 1!1!11

!1

Let 1 xy

1

0

11!!1

!1n

y

yny ppyyn

npnXE

Therefore pnXE


0

22

xX nPxXE

𝐸 𝑋 𝑥 ∙𝑛!

𝑛 𝑥 ! ∙ 𝑥!∙ 𝑝 ∙ 1 𝑝 𝑥 ∙

𝑛!𝑛 𝑥 ! ∙ 𝑥!

∙ 𝑝 ∙ 1 𝑝

Cancelling one of the x and adjusting the summation. Now make two terms …

𝐸 𝑋 𝑥 1 1 ∙𝑛!

𝑛 𝑥 ! ∙ 𝑥 1 !∙ 𝑝 ∙ 1 𝑝

n

x

xnxn

x

xnx ppxxn

npp

xxn

nxXE

11

2 1!1!

!1

!1!

!1




The second term was previously computed (math trick … make summation = 1.0). The first term can now “cancel” an (x-1) … and look for a summation in x-2 terms.

pnppxxn

nXE

n

x

xnx

2

2 1!2!

!

n

x

xnx pppxxn

nnnpnXE

2

22222 1!2!22

1!2

n

x

xnx ppxxn

npnnpnXE

2

22222 1!2!22

!21

11 22 pnnpnXE

pnpnnXE 22 1


222 XEXEXE

222 1 pnpnpnnXE

pnpnpnpnpnpnXE 2222222

qpnppnXE 12

Determine the Moment Generating Function

n

n

knkX ktqp

k

ntM

0

exp

n

n

knkX qtp

k

ntM

0

exp

Which can be determined to be …

nX qtptM exp




Textbook Geometric Distribution (4.5.2)

𝑃𝑟 𝑋 𝑘0, 𝑘 0

𝑝 ∙ 1 𝑝 , 𝑘 1,2,⋯

The 1st Moment

𝐸 𝑋 𝜇 𝑘 ∙ 𝑝 ∙ 1 𝑝

𝐸 𝑋 𝜇 𝑝 ∙ 𝑘 ∙ 1 𝑝

using

20

1

0 1

1

1

1

aada

daxa

da

d

x

x

x

x

𝐸 𝑋 𝜇 𝑝 ∙1

1 1 𝑝𝑝 ∙

1𝑝

1𝑝

The 2nd Moment

𝐸 𝑋 𝑘 ∙ 𝑝 ∙ 1 𝑝

𝐸 𝑋 𝑝 ∙ 𝑘 ∙ 1 𝑝

using

32

2

0

2

02

2

1

2

1

11

aada

daxxa

da

d

x

x

x

x

𝐸 𝑋 𝑝 ∙ 𝑘 ∙ 𝑘 1 1 ∙ 1 𝑝

𝐸 𝑋 𝑝 ∙ 1 𝑝 ∙ 𝑘 ∙ 𝑘 1 ∙ 1 𝑝 𝑝 ∙ 𝑘 ∙ 1 𝑝




𝐸 𝑋 𝑝 ∙ 1 𝑝 ∙2

1 1 𝑝𝐸 𝑋

𝐸 𝑋2 ∙ 𝑝 ∙ 1 𝑝

𝑝1𝑝

𝐸 𝑋2 2 ∙ 𝑝

𝑝1𝑝

2 𝑝𝑝

The Variance


𝑉𝑎𝑟 𝑋 𝜎2 𝑝𝑝

1𝑝

1 𝑝𝑝

The Moment Generating Function (MGF)

𝑀 𝑢 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙ 𝑝 𝑘 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙ 𝑝 ∙ 1 𝑝

𝑀 𝑢 𝑝 ∙ 𝑒𝑥𝑝 𝑢 ∙ 𝑒𝑥𝑝 𝑢 ∙ 1 𝑝

using

1,1

1

0

afora

ax

x

𝑀 𝑢 𝑝 ∙ 𝑒𝑥𝑝 𝑢 ∙ 𝑒𝑥𝑝 𝑢 ∙ 1 𝑝

𝑀 𝑢 𝑝 ∙ 𝑒𝑥𝑝 𝑢 ∙1

1 𝑒𝑥𝑝 𝑢 ∙ 1 𝑝

𝑀 𝑢𝑝 ∙ 𝑒𝑥𝑝 𝑢

1 𝑒𝑥𝑝 𝑢 ∙ 1 𝑝𝑝

𝑒𝑥𝑝 𝑢 1 𝑝




Textbook Poisson Distribution (4.5.2)

In probability theory and statistics, the Poisson distribution, named after French mathematician Siméon Denis Poisson, is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant rate and independently of the time since the last event.[1] The Poisson distribution can also be used for the number of events in other specified intervals such as distance, area or volume.

https://en.wikipedia.org/wiki/Poisson_distribution

𝑃𝑟 𝑋 𝑘0, 𝑘 0

𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆 , 𝑘 0,1,2,⋯

Examples:

Wikipedia – pieces of mail received each day arrival time of new individuals in a line the number of photons striking a pixel of a camera

The 1st Moment

𝐸 𝑋 𝜇 𝑘 ∙𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆

𝐸 𝑋 𝜇 𝜆 ∙ 𝑒𝑥𝑝 𝜆𝜆𝑘 1 !

∙ 𝜆 ∙ 𝑒𝑥𝑝 𝜆𝜆𝑘!∙

𝐸 𝑋 𝜇 𝜆𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆 𝜆

The 2nd Moment

𝐸 𝑋 𝑘 ∙𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆

𝐸 𝑋 𝑘 ∙ 𝑘 1 1 ∙𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆




𝐸 𝑋 𝑘 ∙ 𝑘 1 ∙𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆 𝑘 ∙

𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆

𝐸 𝑋 𝜆 ∙𝜆𝑘 2!

∙ 𝑒𝑥𝑝 𝜆 𝐸 𝑋

𝐸 𝑋 𝜆 𝜆

The Variance


𝑉𝑎𝑟 𝑋 𝜎 𝜆 𝜆 𝜆 𝜆

The mean and the variance are both equal to lambda!

The Moment Generating Function (MGF)

𝑀 𝑢 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙ 𝑝 𝑘 𝑒𝑥𝑝 𝑢 ∙ 𝑘 ∙𝜆𝑘!∙ 𝑒𝑥𝑝 𝜆

𝑀 𝑢 𝑒𝑥𝑝 𝜆 ∙𝜆 ∙ 𝑒𝑥𝑝 𝑢

𝑘!

Recognizing an exponential sequence

𝑒𝑥𝑝 𝑥 1𝑥1!

𝑥2!

⋯𝑥𝑛!

⋯

𝑀 𝑢 𝑒𝑥𝑝 𝜆 ∙ 𝑒𝑥𝑝 𝜆 ∙ 𝑒𝑥𝑝 𝑢 𝑒𝑥𝑝 𝜆 ∙ 𝑒𝑥𝑝 𝑢 1

Interesting factors

𝑝 𝑘𝑝 𝑘 1

𝜆𝑘! ∙ 𝑒𝑥𝑝 𝜆

𝜆𝑘 1 ! ∙ 𝑒𝑥𝑝 𝜆

𝑝 𝑘𝑝 𝑘 1

𝜆𝑘

This implies that the distribution rises to a maximum at 𝜆 𝑘.




Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics, and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN:

013-147122-8.




Textbook Risk Taking Decision Making (4.6)

𝑌 𝑔 𝑋1, 𝑃𝑟 𝑋 0 1 𝑝𝑤, 𝑃𝑟 𝑋 1 𝑝

Examples:

Win-lose propositions (X=1 win, X=0 lose) Gambling bets with odds Buying a stock as an investment

The 1st Moment

𝐸 𝑌 𝜇 𝑔 𝑋 ∙ 𝑃𝑟 𝑋

𝐸 𝑌 𝜇 1 ∙ 1 𝑝 𝑤 ∙ 𝑝

𝐸 𝑌 𝜇 𝑤 ∙ 𝑝 𝑝 1

The 2nd Moment

𝐸 𝑌 𝑔 𝑋 ∙ 𝑃𝑟 𝑋

𝐸 𝑌 𝑤 ∙ 𝑝 1 𝑝

The Variance

𝐸 𝑌 𝜇 𝑉𝑎𝑟 𝑌 𝜎 𝐸 𝑌 𝜇

𝑉𝑎𝑟 𝑌 𝜎 𝑝 ∙ 1 𝑝 ∙ 𝑤 1

To have a chance at winning ….

𝐸 𝑌 𝜇 0

𝑤 or 𝑝

Notice that the variance goes up by the square of the amount wagered!




The Variance – alternate derivation

𝐸 𝑌 𝜇 𝑉𝑎𝑟 𝑌 𝜎 𝐸 𝑌 𝜇

𝑉𝑎𝑟 𝑌 𝜎 𝑤 ∙ 𝑝 1 𝑝 𝑤 ∙ 𝑝 𝑝 1

𝑉𝑎𝑟 𝑌 𝜎 𝑤 ∙ 𝑝 1 𝑝 𝑤 ∙ 𝑝 2 ∙ 𝑤 ∙ 𝑝 ∙ 𝑝 1 𝑝 2 ∙ 𝑝 1

𝑉𝑎𝑟 𝑌 𝜎 𝑤 ∙ 𝑝 𝑤 ∙ 𝑝 2 ∙ 𝑤 ∙ 𝑝 2 ∙ 𝑤 ∙ 𝑝 𝑝 𝑝

𝑉𝑎𝑟 𝑌 𝜎 𝑝 ∙ 𝑤 ∙ 1 𝑝 2 ∙ 𝑤 ∙ 1 𝑝 1 𝑝

𝑉𝑎𝑟 𝑌 𝜎 𝑝 ∙ 1 𝑝 ∙ 𝑤 2 ∙ 𝑤 1

𝑉𝑎𝑟 𝑌 𝜎 𝑝 ∙ 1 𝑝 ∙ 𝑤 1

To have a chance at winning ….

𝐸 𝑌 𝜇 0

𝑤 ∙ 𝑝 𝑝 1 0

𝑤1 𝑝𝑝

or

𝑝1

𝑤 1

Notice that the variance goes up by the square of the amount wagered!

Playing Dice ….

https://en.wikipedia.org/wiki/Craps

House percentages in winning, estimated income per hour!




Matlab Sims

Sec4_5_Coins

Geometric_Example

Binomial_hist

bon_nchoosek

Uniform_hist

Documents

Chapter 4: DISCRETE PROBABILITIES AND RANDOM ...bazuinb/ECE3800/B_Notes04.pdfChapter 4: DISCRETE PROBABILITIES AND RANDOM VARIABLES Sections 4.1 Discrete Random Variable and Probability