PROBABILITY DISTRIBUTIONS - Vrije Universiteit...

PROBABILITY DISTRIBUTIONS

Business Statistics

Probability distribution functions (discrete)

Characteristics of a discrete distribution

Example: uniform (discrete) distribution

Example: Bernoulli distribution

Example: binomial distribution

Probability density functions (continuous)

Characteristics of a continuous distribution

Example: uniform (continuous) distribution

Example: normal (or Gaussian) distribution

Example: standard normal distribution

Back to the normal distribution

Approximations to distributions

Old exam question

Further study

CONTENTS

Today we want to speed up. We will skip some slides or postpone a few. Prepare

well, we want to start the statistical topics as soon as

possible.

▪ A sample space is called discrete when its elements can be

counted

▪ We will code the elements of a discrete sample space 𝑆 as

1,2,3, … , 𝑛 or 0,1,2, … , 𝑛 − 1▪ Examples

▪ die 𝑥 ∈ 1,2,3,4,5,6 , so 𝑆 = 1,2,3,4,5,6▪ coin 𝑥 ∈ 0,1▪ number of broken TV sets 𝑥 ∈ 0,1,2,…

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

Distribution function

𝑃 𝑥 = 𝑃 𝑋 = 𝑥

▪ the probability that the (discrete) random variable 𝑋assumes the value 𝑥

▪ alternative notation: 𝑃𝑋 𝑥

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

Note our convention:

capital letters (𝑋) for random variables

lowercase letters (𝑥) for values

Example

▪ die: 𝑃 𝑥 =

1

6if 𝑥 = 1

1

6if 𝑥 = 2

1

6if 𝑥 = 3

1

6if 𝑥 = 4

1

6if 𝑥 = 5

1

6if 𝑥 = 6

0 otherwise

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

Example: flipping a coin 3 times

▪ sample space 𝑆 = 𝐻𝐻𝐻,𝐻𝐻𝑇,𝐻𝑇𝐻, 𝑇𝐻𝐻,…▪ define the random variable 𝑋 = number of heads

▪ distribution function 𝑃 𝑥 =

1

8if 𝑥 = 0

3

8if 𝑥 = 1

3

8if 𝑥 = 2

1

8if 𝑥 = 3

0 otherwise

▪ or: 𝑃𝑋 0 =1

8, 𝑃𝑋 1 =

3

8, 𝑃𝑋 2 =

3

8, 𝑃𝑋 3 =

1

8

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

▪ 𝑃 𝑥 is a (discrete) probability distribution function (pdf or

PDF)

▪ 𝑃 𝑥 = 𝑃 𝑋 = 𝑥 expresses the probability that 𝑋 = 𝑥▪ A random variable 𝑋 that is distributed with pdf 𝑃 is written

as

𝑋~𝑃

▪ Some properties of the pdf:▪ 0 ≤ 𝑃 𝑥 ≤ 1

▪ a probability is always between 0 and 1▪ σ𝑥∈𝑆𝑃 𝑥 = 1

▪ the probabilities of all elementary outcomes add up to 1

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

▪ A pdf may have one or more parameters to denote a

collection of different but “similar”pdfs

▪ Example: a regular die with 𝑚 faces

▪ 𝑃 𝑋 = 𝑥;𝑚 = 𝑃𝑋 𝑥;𝑚 = 𝑃 𝑥;𝑚 =1

𝑚(for 𝑥 = 1,… ,𝑚)

▪ 𝑋~𝑃 𝑚

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

𝑚 = 4 𝑚 = 6 𝑚 = 8 𝑚 = 12 𝑚 = 20

In addition to the (discrete) probability distribution function

(pdf)

▪ 𝑃 𝑋 = 𝑥 = 𝑃𝑋 𝑥 = 𝑃 𝑥we define the (discrete) cumulative distribution function (cdf or

CDF)

𝐹 𝑥 = 𝐹𝑋 𝑥 = 𝑃 𝑋 ≤ 𝑥

and therefore

𝐹 𝑥 =

𝑘=−∞

𝑥

𝑃 𝑋 = 𝑘 =

𝑘=−∞

𝑥

𝑃 𝑘

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

Depending on how we

count, you may also start

at 𝑘 = 0 or 𝑘 = 1

Example

▪ die: 𝑃 𝑋 = 2 =1

6, but 𝑃 𝑋 ≤ 2 = 𝑃 𝑋 = 1 +

𝑃 𝑋 = 2 =1

3

▪ Some properties of the cdf:▪ 𝐹 −∞ = 0 and 𝐹 ∞ = 1▪ monotonously increasing

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

▪ pdf

▪ cdf

PROBABILITY DISTRIBUTION FUNCTIONS (DISCRETE)

Expected value of 𝑋

𝐸 𝑋 =

𝑖=1

𝑁

𝑥𝑖𝑃 𝑋 = 𝑥𝑖 =

𝑖=1

𝑁

𝑥𝑖𝑃 𝑥𝑖

▪ Example▪ die with 𝑃 1 = 𝑃 2 = ⋯ = 𝑃 6 =

1

6▪ 𝐸 𝑋 = 1 ×

1

6+ 2 ×

1

6+ 3 ×

1

6+ 4 ×

1

6+ 5 ×

1

6+ 6 ×

1

6=

7

2= 3

1

2▪ Interpretation: mean (average)

▪ alternative notation: 𝜇 or 𝜇𝑋▪ so 𝐸 𝑋 = 𝜇𝑋

▪ Note difference between 𝜇 and the sample mean ҧ𝑥▪ e.g., rolling a specific die 𝑛 = 100 times may return a mean ҧ𝑥 = 3.72 or

3.43▪ while 𝜇 = 7/2, always (property of die, property of “population”)

CHARACTERISTICS OF A DISCRETE DISTRIBUTION

Variance

var 𝑋 =

𝑖=1

𝑁

𝑥𝑖 − 𝐸 𝑋2𝑃 𝑥𝑖

▪ Interpretation: dispersion▪ alternative notation: 𝜎2 or 𝜎𝑋

2 or 𝑉 𝑋

▪ so var 𝑋 = 𝜎𝑋2

▪ Note difference between 𝜎2 and the sample variance 𝑠2

▪ e.g., rolling a specific die 100 times may return a variance 𝑠2 = 2.86 or 3.04

▪ while 𝜎2 =35

12, always (property of die, property of “population”)

▪ And of course: standard deviation 𝜎𝑋 = var 𝑋

CHARACTERISTICS OF A DISCRETE DISTRIBUTION

Transformation rules of random variable 𝑋 and 𝑌▪ For means:

▪ 𝐸 𝑘 + 𝑋 = 𝑘 + 𝐸 𝑋▪ 𝐸 𝑎𝑋 = 𝑎𝐸 𝑋▪ 𝐸 𝑋 + 𝑌 = 𝐸 𝑋 + 𝐸 𝑌

▪ For variances:▪ var 𝑘 + 𝑋 = var 𝑋▪ var 𝑎𝑋 = 𝑎2var 𝑋▪ if 𝑋 and 𝑌 independent (so if cov 𝑋, `𝑌 ):

▪ var 𝑋 + 𝑌 = var 𝑋 + var 𝑌▪ if 𝑋 and 𝑌 dependent:

▪ var 𝑋 + 𝑌 = var 𝑋 + 2cov 𝑋, 𝑌 + var 𝑌

CHARACTERISTICS OF A DISCRETE DISTRIBUTION

▪ Generalization of fair die:▪ equal probability of integer outcomes from 𝑎 through 𝑏

▪ conditions: 𝑎, 𝑏 ∈ ℤ, 𝑎 < 𝑏▪ zero probability elsewhere

▪ uniform discrete distribution

▪ pdf: 𝑃 𝑥; 𝑎, 𝑏 = ൝1

𝑏−𝑎+1𝑥 ∈ ℤ and 𝑥 ∈ 𝑎, 𝑏

0 otherwise▪ Examples:

▪ coin: 𝑎 = 0, 𝑏 = 1▪ die: 𝑎 = 1, 𝑏 = 6

▪ Random variable:▪ 𝑋~𝑈 𝑎, 𝑏

EXAMPLE: UNIFORM DISTRIBUTION

EXAMPLE: UNIFORM DISTRIBUTION

No need to memorize or even

discuss this sheet. Most

information is either on the

formula sheet or unimportant.

▪ Example: choose a random number from 1 through 100with equal probability and denote it by 𝑋▪ random variable: 𝑋~𝑈 1,100

▪ pdf: 𝑃 𝑥 = 𝑃 𝑋 = 𝑥 =1

100(𝑥 ∈ 1,2,… , 100 )

▪ cdf: 𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥 =𝑥

100(𝑥 ∈ 1,2,… , 100 )

▪ expected value: 𝐸 𝑋 = 501

2

▪ variance: var 𝑋 =9999

12≈ 833.25

▪ Sample (𝑛 = 1000): ▪ values (e.g.): 45, 96, 33, 7, 44, 96, 20, …▪ mean: ҧ𝑥 = 50.92 (e.g.)

▪ variance: 𝑠𝑥2 = 823.25 (e.g.)

EXAMPLE: UNIFORM DISTRIBUTION

Given are two dice, with outcomes 𝑋 and 𝑌.

a. Find 𝐸 𝑋 + 𝑌b. Find var 𝑋 + 𝑌

EXERCISE 1

▪ Bernoulli experiment▪ random experiment with 2 discrete outcomes (coin type)

▪ head, true, “success”, female: 𝑋 = 1▪ tail, false, “fail”, male: 𝑋 = 0▪ Bernoulli distribution

▪ Examples:▪ winning a price in a lottery (buying one ticket)

▪ your luggage arrives in time at a destination

▪ Probability of success is parameter 𝜋 (with 0 ≤ 𝜋 ≤ 1)▪ 𝑃 1 = 𝑃 𝑋 = 1 = 𝜋▪ 𝑃 0 = 𝑃 𝑋 = 0 = 1 − 𝜋

▪ Random variable▪ 𝑋~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖 𝜋 or 𝑋~𝑎𝑙𝑡 𝜋

EXAMPLE: BERNOULLI DISTRIBUTION

▪ Expected value▪ 𝐸 𝑋 = 𝜋 (obviously!)

▪ Variance▪ var 𝑋 = 𝜋 1 − 𝜋▪ variance zero when 𝜋 = 0 or 𝜋 = 1 (obviously!)

▪ variance maximal when 𝜋 = 1 − 𝜋 =1

2(obviously!)

▪ pdf: 𝑝 𝑥; 𝜋 = ቐ𝜋 if 𝑥 = 1

1 − 𝜋 if 𝑥 = 00 otherwise

▪ cdf: (not so interesting)

EXAMPLE: BERNOULLI DISTRIBUTION

▪ Repeating a Bernoulli experiment 𝑛 times▪ 𝑋 is total number of “successes”

▪ 𝑃 𝑋 = 𝑥 is probality of 𝑥 “successes” in sample

▪ 𝑋 = 𝑋1 + 𝑋2 +⋯+ 𝑋𝑛▪ where 𝑋𝑖 is the outcome of Bernoulli experiment number 𝑖 =1,2,… , 𝑛

▪ 𝑋 has a binomial distribution

EXAMPLE: BINOMIAL DISTRIBUTION

▪ Example▪ flip a coin 10 times:𝑋 is number of “heads up”

▪ roll 100 dice: 𝑋 is number of “sixes”

▪ produce 1000 TV sets: 𝑋 is number of broken sets

▪ What is important?▪ the number of repitions (𝑛)

▪ the probability of success (𝜋) per item

▪ the constancy of 𝜋▪ the independence of the “experiments”

EXAMPLE: BINOMIAL DISTRIBUTION

▪ Expected value▪ 𝐸 𝑋 = 𝑛𝜋 (obviously!)

▪ Variance▪ var 𝑋 = 𝑛𝜋 1 − 𝜋▪ minimum (0) when 𝜋 = 0 or 𝜋 = 1 (obviously!)

▪ maximum for given 𝑛 when 𝜋 = 1 − 𝜋 =1

2(obviously!)

▪ pdf:

▪ 𝑝 𝑥; 𝑛, 𝜋 =𝑛!

𝑥! 𝑛−𝑥 !𝜋𝑥 1 − 𝜋 𝑛−𝑥 (𝑥 ∈ 0,1,2,… , 𝑛 )

▪ cdf:▪ 𝐹 𝑥; 𝑛, 𝜋 = σ𝑘=0

𝑥 𝑝 𝑥; 𝑛, 𝜋

▪ Random variable:▪ 𝑋~𝑏𝑖𝑛 𝑛, 𝜋 or 𝑋~𝑏𝑖𝑛𝑜𝑚 𝑛, 𝜋

EXAMPLE: BINOMIAL DISTRIBUTION

Recall the factorial function:

5! = 5 × 4 × 3 × 2 × 1

▪ Example:▪ roll 10 dice: what is the distribution of 𝑋 = number of “sixes”?

▪ What is the probability model?▪ you repeat an experiment 10 times (𝑛 = 10)

▪ with a probability 𝜋 =1

6of success and a probability 1 − 𝜋 =

5

6of failure per

experiment

▪ What is the probability distribution?

▪ 𝑋~𝑏𝑖𝑛 10,1

6

▪ where the random variable 𝑋 represents the total number of sixes

▪ so 𝑋 is not the outcome of a roll of the die!

▪ 𝐸 𝑋 = 10 ×1

6= 1

2

3

▪ so we expect on average 12

3sixes in 10 rolls

▪ var 𝑋 = 10 ×1

6×

5

6=

25

18

EXAMPLE: BINOMIAL DISTRIBUTION

EXAMPLE: BINOMIAL DISTRIBUTION

No need to memorize or even discuss this

sheet. Most information is either on the

formula sheet or unimportant.

▪ Calculating pdf and cdf values

▪ Example: binomial distrbution with 𝑛 = 8, 𝜋 = 0.5▪ what is 𝑃 3 = 𝑃 𝑋 = 3 (pdf)?

▪ what is 𝐹 3 = 𝑃 𝑋 ≤ 3 (cdf)?

▪ Different methods:▪ using a graphical calculator (not at the exam)

▪ using the formula (see next slides)

▪ using a table (see next slides)

▪ using Excel (see the computer tutorials)

▪ using online calculators (figure out for yourself)

EXAMPLE: BINOMIAL DISTRIBUTION

▪ pdf using the formula

▪ 𝑃 3; 8,0.5 =8!

3! 8−3 !0.53 1 − 0.5 8−3 = 0.2188

▪ or

▪ 𝑃 3; 8,0.5 = 830.53 1 − 0.5 8−3 = 0.2188

▪ using the binomial coefficient 𝑛𝑘

= 𝑛𝐶𝑘 =𝑛!

𝑘! 𝑛−𝑘 !

EXAMPLE: BINOMIAL DISTRIBUTION

At the exam, you can just use the tables.

Much easier!

▪ pdf using the table in Appendix A▪ 𝑃 3; 8,0.50 = 0.2188

EXAMPLE: BINOMIAL DISTRIBUTION

▪ At the exam: non-cumulative table only

▪ Problem: how to do the cdf?

▪ Use the definition:

𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥 =

𝑘=0

𝑥

𝑃 𝑋 = 𝑘

▪ 𝑃 𝑋 ≤ 3 = 𝑃 𝑋 = 0 + 𝑃 𝑋 = 1 + 𝑃 𝑋 = 2 +𝑃 𝑋 = 3

▪ use table, four times

EXAMPLE: BINOMIAL DISTRIBUTION

▪ Example▪ 𝐹 3; 8,0.50 = 0.0039 + 0.0313 + 0.1094 + 0.2188

EXAMPLE: BINOMIAL DISTRIBUTION

Note that this table gives a

pdf, not a cdf

▪ Note that cdf is 𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥▪ How to find 𝑃 𝑋 < 𝑥 ?

▪ use 𝑃 𝑋 ≤ 𝑥 = 𝑃 𝑋 ≤ 𝑥 − 1▪ How to find 𝑃 𝑋 > 𝑥 ?

▪ use 𝑃 X > x = 1 − 𝑃 𝑋 ≤ 𝑥▪ How to find 𝑃 𝑥1 < 𝑋 < 𝑥2 ?

▪ use 𝑃 𝑥1 < 𝑋 < 𝑥2 = 𝑃 𝑋 < 𝑥2 − 𝑃 𝑋 ≤ 𝑥1▪ Etc.

EXAMPLE: BINOMIAL DISTRIBUTION

▪ Use such rules to efficiently use the (pdf) table (𝑛 = 8)▪ 𝑃 𝑋 ≤ 7 = 𝑃 0 + 𝑃 1 +⋯+ 𝑃 7

▪ Much easier:▪ 𝑃 𝑋 ≤ 7 = 1 − 𝑃 8

EXAMPLE: BINOMIAL DISTRIBUTION

Example:

▪ Context:▪ on average, 20% of the emergency room patients at Greenwood

General Hospital lack health insurance

▪ In a random sample of 4 patients, what is the probability

that at least 2 will be uninsured?

EXAMPLE: BINOMIAL DISTRIBUTION

▪ Binomial model (patient is uninsured or not, 𝜋uninsured =0.20)▪ 𝑋 is number of uninsured patients in sample

▪ 𝑃 𝑋 ≥ 2 = 𝑃 𝑋 = 2 + 𝑃 𝑋 = 3 + 𝑃 𝑋 = 4 =0.1536 + 0.0256 + 0.0016 = 0.1808

EXAMPLE: BINOMIAL DISTRIBUTION

Note that this table gives a

pdf, not a cdf

Discrete distributions▪ probability distribution function (pdf): 𝑃 𝑥 = 𝑃 𝑋 = 𝑥▪ probability of obtaining the value 𝑥

Continuous distributions▪ the probability of obtaining the value 𝑥 is 0▪ define probability density function (pdf): 𝑓 𝑥

▪ 𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = 𝑎𝑏𝑓 𝑥 𝑑𝑥

▪ probability of obtaining a value between 𝑎 and 𝑏

PROBABILITY DENSITY FUNCTION (CONTINUOUS)

Compare with the

probability distribution

function (pdf) 𝑃 𝑋 = 𝑥for the discrete case

The red curve is the pdf, 𝑓 𝑥The integral is the grey area

under the pdf

So pdf refers to two distinct but related things:▪ probability distribution function 𝑃 𝑥 (discrete case)

▪ probability density function 𝑓 𝑥 (continuous case)

Note also that the dimensions are different▪ 𝑃 is a dimensionless probability

▪ example:

▪ if 𝑋 is in kg, the discrete pdf 𝑃 𝑋 is dimensionless

▪ while the continuous pdf 𝑓 𝑥 is in 1/kg

PROBABILITY DENSITY FUNCTION (CONTINUOUS)

Because 𝑓 𝑥 𝑑𝑥 should be

dimensionless, and 𝑑𝑥 is in in kg

In addition to the probability density function ...▪ 𝑃 𝑥 = 𝑃𝑋 𝑥

... we define the cumulative distribution function (cdf or CDF)

𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥 = න

−∞

𝑥

𝑓 𝑦 𝑑𝑦

Some properties of the cdf:▪ 𝐹 −∞ = 0 and 𝐹 ∞ = 1▪ monotonously increasing

PROBABILITY DENSITY FUNCTION (CONTINUOUS)

Compare with

𝐹 𝑥 = 𝑃 𝑋 ≤ 𝑥 =

𝑘=−∞

𝑥

𝑃 𝑋 = 𝑘

for the discrete case

𝑥

𝐹 𝑥

▪ pdf

▪ cdf

PROBABILITY DENSITY FUNCTION (CONTINUOUS)

𝑃 70 ≤ 𝑋 ≤ 75

= න

70

75

𝑓 𝑥 𝑑𝑥

𝑃 70 ≤ 𝑋 ≤ 75= 𝐹 75 − 𝐹 70

▪ Expected value

𝐸 𝑋 = න

−∞

∞

𝑥𝑓 𝑥 𝑑𝑥

▪ Example: let 𝑓 𝑥 = 1 for 𝑥 ∈ 0,1

▪ 𝐸 𝑋 = 01𝑥𝑑𝑥 = ቃ

1

2𝑥2

0

1=

1

2

▪ Interpretation: mean (average)▪ alternative notation for 𝐸 𝑋 : 𝜇 or 𝜇𝑋

CHARACTERISTICS OF A CONTINUOUS DISTRIBUTION

Compare with

𝐸 𝑋 =

𝑖=1

𝑛

𝑥𝑖𝑃 𝑥

for the discrete case

▪ Variance

var 𝑋 = න

−∞

∞

𝑥 − 𝐸 𝑋2𝑓 𝑥 𝑑𝑥

▪ Interpretation: dispersion▪ alternative notation for var 𝑋 : 𝜎2 or 𝜎𝑋

2 or V(𝑋)

CHARACTERISTICS OF A CONTINUOUS DISTRIBUTION

Compare with

var 𝑋 =

𝑖=1

𝑛

𝑥𝑖 − 𝐸 𝑋2𝑃 𝑥𝑖

for the discrete case

▪ Analogy with uniform discrete distribution▪ equal density for all outcomes between 𝑎 and 𝑏

▪ condition: 𝑎 < 𝑏▪ zero probability elsewhere

▪ uniform continuous distribution

▪ pdf: 𝑓 𝑥; 𝑎, 𝑏 = ൝1

𝑏−𝑎𝑥 ∈ 𝑎, 𝑏

0 otherwise

▪ or easier: 𝑓 𝑥; 𝑎, 𝑏 =1

𝑏−𝑎(𝑥 ∈ 𝑎, 𝑏 )

▪ Examples:▪ “standard” uniform deviate: 𝑎 = 0, 𝑏 = 1

EXAMPLE: UNIFORM (CONTINUOUS) DISTRIBUTION

Example: let 𝑋 be exam grade of randomly selected student▪ assume uniform distribution: 𝑋~𝑈 1,10▪ what is 𝑃 𝑋 ≥ 6.5 ?

Solution▪ use 𝑃 𝑋 ≥ 6.5 = 1 − 𝑃 𝑋 < 6.5 = 1 − 𝑃 𝑋 ≤ 6.5

▪ cdf: 𝑃 𝑋 ≤ 𝑥 = 𝐹 𝑥 = ∞−𝑥𝑓 𝑦 𝑑𝑦

▪ uniform continuous with 𝑎 = 1 and 𝑏 = 10

▪ pdf: 𝑓 𝑥 =1

9(𝑥 ∈ 1,10 )

▪ cdf: 𝑃 𝑋 ≤ 𝑥 = 1𝑥 1

9𝑑𝑦 =

1

9𝑥 − 1

▪ answer: 𝑃 𝑋 ≥ 6.5 = 1 −1

96.5 − 1

▪ or: area of black rectangle

EXAMPLE: UNIFORM (CONTINUOUS) DISTRIBUTION

For a continuous distribution

𝑃 𝑋 < 𝑥 = 𝑃 𝑋 ≤ 𝑥because 𝑃 𝑋 = 𝑥 = 0

1 6.5 10

1

9

𝑃 𝑋 ≥ 6.5 is the black area

▪ Expected value

▪ 𝐸 𝑋 =𝑎+𝑏

2

▪ Variance

▪ var 𝑋 =𝑏−𝑎 2

12𝑎)𝑏𝑥 −

𝑎+𝑏

2

2×

1

𝑏−𝑎𝑑𝑥 =

𝑏−𝑎 2

12)

▪ pdf

▪ 𝑓 𝑥 =1

𝑏−𝑎

▪ cdf▪ 𝐹 𝑥 =

𝑥−𝑎

𝑏−𝑎

▪ Random variable▪ 𝑋~𝑈 𝑎, 𝑏 or 𝑋~ℎ𝑜𝑚 0, 𝜃 or 𝑋~ℎ𝑜𝑚 𝜃 etc.

EXAMPLE: UNIFORM (CONTINUOUS) DISTRIBUTION

▪ pdf

▪ 𝑓 𝑥; 𝜇, 𝜎 =1

𝜎 2𝜋𝑒−1

2

𝑥−𝜇

𝜎

2

▪ cdf

▪ 𝐹 𝑥 = ∞−𝑥𝑓 𝑦; 𝜇, 𝜎 𝑑𝑦 =? ? ?

▪ Expected value▪ 𝐸 𝑋 = 𝜇

▪ Variance▪ var 𝑋 = 𝜎2

▪ Random variable▪ 𝑋~𝑁 𝜇, 𝜎 or 𝑋~𝑁 𝜇, 𝜎2

EXAMPLE: NORMAL (OR GAUSSIAN) DISTRIBUTION

In a concrete case indicate the

parameter’s symbol:

𝑁 12, 𝜎 = 2 or 𝑁 12, 𝜎2 = 4

Remember notation 𝜇𝑋 for expected

value and 𝜎𝑋2 for variance.

So here 𝜇𝑋 = 𝜇 and 𝜎𝑋2 = 𝜎2.

This is no coincedence!

Now, 𝜋 = 3.1415 ...

▪ Some characteristics▪ range: 𝑥 ∈ −∞,∞▪ pdf has maximum at 𝑥 = 𝜇▪ pdf is symmetric around 𝑥 = 𝜇▪ not too interesting for 𝑥 < 𝜇 − 3𝜎 and for 𝑥 > 𝜇 + 3𝜎

EXAMPLE: NORMAL (OR GAUSSIAN) DISTRIBUTION

▪ Normal distribution with 𝜇 = 0 and 𝜎 = 1▪ so a 0-parameter distribution: standard normal

▪ pdf

▪ 𝑓 𝑥 =1

2𝜋𝑒−

1

2𝑥2

▪ cdf▪ 𝐹 𝑥 = ∞−

𝑥𝑓 𝑦 𝑑𝑦 =? ? ?= Φ 𝑥

▪ with Φ −∞ = 0, Φ ∞ = 1, Φ 0 = 0.5, 𝑑Φ

𝑑𝑥= 𝑓 𝑥

▪ Expected value▪ 𝐸 𝑋 = 0

▪ Variance▪ var 𝑋 = 1

▪ Random variable▪ 𝑋~𝑁 0,1 , we often write 𝑍~𝑁 0,1

EXAMPLE: STANDARD NORMAL DISTRIBUTION

Remember the trick:

if you don’t know

something, just give it

a name

▪ Important because any normally distributed variable can be

“standardized” to standard normal distribution

▪ Methods for determing the values of Φ 𝑥 :▪ using a graphical calculator (not at the exam)

▪ not using a formula (no formula available for Φ 𝑥 )

▪ using a table (see next slides)

▪ using Excel (see the computer tutorials)

▪ using online calculators (figure out for yourself)

EXAMPLE: STANDARD NORMAL DISTRIBUTION

▪ Calculating the value of the cdf with a table▪ 𝑃 𝑍 ≤ 1.36 = Φ 1.36▪ table C-2 (p.768): 𝑃 𝑍 ≤ 1.36 = 0.9131

EXAMPLE: STANDARD NORMAL DISTRIBUTION

Note that cdf is 𝑃 𝑍 ≤ 𝑥▪ How to find 𝑃 𝑍 < 𝑥 ?

▪ use 𝑃 𝑍 ≤ 𝑥 (why?)

▪ How to find 𝑃 𝑍 > 𝑥 ?▪ use 1 − 𝑃 𝑍 ≤ 𝑥 (why?)

▪ or use 𝑃 𝑍 > 𝑥 = 𝑃 𝑍 < −𝑥 (why?)

▪ How to find 𝑃 𝑍 ≥ 𝑥 ?▪ is easy now ...

▪ How to find 𝑃 𝑥 ≤ 𝑍 ≤ 𝑦 ?▪ use 𝑃 𝑍 ≤ 𝑦 − 𝑃 𝑍 ≤ 𝑥

▪ Etc.

EXAMPLE: STANDARD NORMAL DISTRIBUTION

= −

Scale for standard normal,

but this applies to any

continuous distribution

▪ Inverse lookup▪ 𝑃 𝑋 ≤ 𝑥 = Φ 𝑥 = 0.90▪ table C-2 (p.768): 𝑥 ≈ 1.28

EXAMPLE: STANDARD NORMAL DISTRIBUTION

No need to know this table by heart...

but two values can be convenient to know

▪ 𝑃 𝑍 ≤ 1.96 = 0.95, a 𝑧-value as large as 1.96 or

larger occurs only with 5% probability

▪ 𝑃 −1.645 ≤ 𝑍 ≤ 1.645 = 0.95, a 𝑧-value as large as

1.96 or larger or as small as −1.645 or smaller occurs

only with 5% probability

▪ so remember 1.96 and 1.645▪ (you can always look them up if you forgot or are unsure)

EXAMPLE: STANDARD NORMAL DISTRIBUTION

Note: 𝑋~𝑁 𝜇, 𝜎2 ⇔ 𝑋 − 𝜇~𝑁 0, 𝜎2 ⇔𝑋−𝜇

𝜎~𝑁 0,1

▪ Standardization

▪ 𝑥 → 𝑧 =𝑥−𝜇

𝜎and 𝑋 → 𝑍 =

𝑋−𝜇

𝜎

▪ If 𝑋~𝑁 𝜇, 𝜎2 , how to determine 𝑃 𝑋 ≤ 𝑥 ?

▪ 𝑃 𝑋 ≤ 𝑥 = 𝑃 𝑋 − 𝜇 ≤ 𝑥 − 𝜇 = 𝑃𝑋−𝜇

𝜎≤

𝑥−𝜇

𝜎= 𝑃 𝑍 ≤

𝑥−𝜇

𝜎

▪ Example▪ suppose 𝑋~𝑁 180, 𝜎2 = 25

▪ 𝑃 𝑋 ≤ 190 = 𝑃 𝑍 ≤190−180

5= 𝑃 𝑍 ≤ 2 = 0.9772

▪ 𝑃 𝑋 ≤ 𝑥 = 0.90 = 𝑃 𝑍 ≤𝑥−180

5⇒

𝑥−180

5= 1.28 ⇒ 𝑥 = 186.4

BACK TO THE NORMAL DISTRIBUTION

This is our way of doing

normalcdf and invnorm if you

don’t have a graphical calculator!

▪ What is “normal” about the normal distribution?▪ it has quite a weird pdf formula

▪ and an even weirder cdf formula

▪ But▪ it is unimodal

▪ it is symmetric

▪ very often empirical distributions “look” normal

▪ a quantity is approximately normal if it is influenced by many

additive factors, none of which is dominating

▪ several statistics (mean, sum, ...) are normally distributed

▪ You’ll learn that soon▪ when we discuss the Central Limit Theorem (CLT)

BACK TO THE NORMAL DISTRIBUTION

▪ Scaling▪ If 𝑋~𝑁 𝜇𝑋, 𝜎𝑋

2 then 𝑎𝑋 + 𝑏~𝑁 𝑎𝜇𝑋 + 𝑏, 𝑎2𝜎𝑋2

▪ Additivity▪ If 𝑋~𝑁 𝜇𝑋, 𝜎𝑋

2 and 𝑌~𝑁 𝜇𝑌, 𝜎𝑌2 and 𝑋, 𝑌 independent, then

𝑋 + 𝑌~𝑁 𝜇𝑋 + 𝜇𝑌, 𝜎𝑋2 + 𝜎𝑌

2

PROPERTIES OF THE NORMAL DISTRIBUTION

pdf of 0.825𝑋 + 11

pdf of 𝑋

Sometimes, we can approximate a “difficult” distribution by a

“simpler” one

▪ Important case: binomial normal▪ example 1: flipping a coin (𝜋 = 0.50, 𝑋 = #heads) very often

APPROXIMATIONS TO DISTRIBUTIONS

▪ But also when 𝜋 ≠ 0.50▪ example 2: flipping a biased coin (𝜋 = 0.30, 𝑋 = #heads) very

often

APPROXIMATIONS TO DISTRIBUTIONS

𝑛 = 10; 𝜋 = .30 𝑛 = 20; 𝜋 = .30 𝑛 = 40; 𝜋 = .30

▪ binomial normal▪ 𝑏𝑖𝑛 𝑛, 𝜋 𝑁 𝜇, 𝜎2

▪ using 𝜇 =? ? ? and 𝜎2 =? ? ?

We know that when 𝑋~𝑏𝑖𝑛 𝑛, 𝜋▪ 𝐸 𝑋 = 𝑛𝜋▪ var 𝑋 = 𝑛𝜋 1 − 𝜋

So, replace▪ 𝜇 = 𝑛𝜋▪ 𝜎2 = 𝑛𝜋 1 − 𝜋

So,▪ 𝑏𝑖𝑛 𝑛, 𝜋 𝑁 𝑛𝜋, 𝑛𝜋 1 − 𝜋

▪ rule: allowed when 𝑛𝜋 ≥ 5 and 𝑛 1 − 𝜋 ≥ 5

APPROXIMATIONS TO DISTRIBUTIONS

The book says ≥ 10instead of ≥ 5

▪ Example binomial normal▪ roll a die 𝑛 = 900 times

▪ study the occurrence of “sixes” (so 𝜋 =1

6)

▪ what is the probability of no more then 170 “sixes”?

▪ Exact: 𝑃𝑏𝑖𝑛 𝑛=900;𝜋=1/6 X ≤ 170 =?

▪ Two problems:▪ need to add 171 pdf-terms (𝑃 𝑋 = 0 until 𝑃 𝑋 = 170 )

▪ 900! gives an ERROR

▪ Approximation: 𝑃𝑁 𝜇=150;𝜎2=125 𝑋 ≤ 170 =

𝑃𝑍 𝑍 ≤170−150

125= Φ𝑍 1.7888 ≈ 0.9631

APPROXIMATIONS TO DISTRIBUTIONS

900 ×1

6= 150

900 ×1

6× 1 −

1

6= 125

▪ Now take 𝑋~𝑏𝑖𝑛 18,0.5▪ In a “binomial” context 𝑃 𝑋 ≤ 11 = 𝑃 𝑋 < 12▪ But in a “normal” context 𝑃 𝑋 ≤ 11 = 𝑃 𝑋 < 11

▪ So, take care about using integers

▪ Safest: go half-way: 𝑃 𝑋 ≤ 11.5 = 𝑃 𝑋 < 11.5▪ This is the continuity correction

APPROXIMATIONS TO DISTRIBUTIONS

The intuitive notion of the continuity correction▪ when approximating a discrete distribution by a continuous

distribution

APPROXIMATIONS TO DISTRIBUTIONS

𝑃𝑏𝑖𝑛 𝑋 ≤ 7 ≈ 𝑃𝑁 𝑋 ≤ 71

2𝑃𝑏𝑖𝑛 𝑋 ≥ 7 ≈ 𝑃𝑁 𝑋 ≥ 6

1

2

Improving previous result

▪ without continuity correction▪ 𝑃𝑏𝑖𝑛 𝑛=900;𝜋=1/6 X ≤ 170 = 𝑃𝑁 𝜇=150;𝜎2=125 (

)

𝑋 ≤

170 = 𝑃𝑍 𝑍 ≤170−150

125= Φ𝑍 1.788 ≈ 0.9631

▪ with continuity correction▪ 𝑃𝑏𝑖𝑛 𝑛=900;𝜋=1/6 X ≤ 170 = 𝑃𝑁 𝜇=150;𝜎2=125 (

)

𝑋 ≤

170.5 = 𝑃𝑍 𝑍 ≤170.5−150

125= Φ𝑍 1.833 ≈ 0.9664

APPROXIMATIONS TO DISTRIBUTIONS

30 June 2014, Q1d

OLD EXAM QUESTION

30 June 2014, Q1f

OLD EXAM QUESTION

Doane & Seward 5/E 6.1-6.4, 6.8, 7.1-7.5

Tutorial exercises week 1

discrete probability distributions

continuous probability distributions

expectation and variance

FURTHER STUDY