Continuous Distributions

Continuous random variables

For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following properties :

1. f(x) ≥ 0

2. 1.f x dx

3. .

b

a

P a X b f x dx

Graph: Continuous Random Variableprobability density function, f(x)

1.f x dx

.b

a

P a X b f x dx

The Uniform distribution from a to b

Definition: A random variable , X, is said to have a Uniform distribution from a to b if X is a continuous random variable with probability density function f(x):

1

0 otherwise

a x bf x b a

Graph: the Uniform Distribution(from a to b)

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x

The Cumulative Distribution function, F(x) (Uniform Distribution from a to b)

0

0.1

0.2

0.3

0.4

0 5 10 15

0

1

x a

x aF x P X x a x b

b ax b

a b

f x

x

F x

Cumulative Distribution function, F(x)

0

1

x a

x aF x P X x a x b

b ax b

0

0.5

1

0 5 10 15 a b x

F x

The Normal distribution

Definition: A random variable , X, is said to have a Normal distribution with mean and standard deviation if X is a continuous random variable with probability density function f(x):

2

221

2

x

f x e

Graph: the Normal Distribution(mean , standard deviation )

Note: 2

221

2

x

f x e

2

222

11 0

2

x

f x e x

if 0 or x x

Thus the point is an extremum point of f(x). (In this case a maximum)

Thus the points – , + are points of inflection of f(x)

2

22

3

1

2

xdf x e x

dx

2 2

2 22 22

5 3

1 1

2 2

x x

e x e

2

22

2

23

1 1 0

2

x xe

2

2if 1 or 1

x x

i.e. x

Also 2

221

12

x

f x dx e dx

xz

To evaluate

Proof: 2

221

2

x

e dx

Make the substitution1

dz dx

When , and when , . x z x z

2 22

22 2 21 1 1

2 2 2

x z z

e dx e dz e dz

Consider evaluating

2

2

z

e dz c

Note:2 2 2

22 2 2 2

z z uzc e dz e dz e dz e du

2 2

2 2

22 2

z uz u

e e dudz e dudz

Make the change to polar coordinates (R, )

z = R sin() and u = R cos()

Using

or

2 2 2R z u tanz

u and

2 2R z u 1tan

z

u

and

2 2

22

z u

c e dudz

2

0 0

, sin , cosf z u dzdu f R R RdRd

Hence

2

2

2

2 2 22

00 0 0 0

1 2R

R

e RdRd e d d

and

2

2 2z

c e dz

or 22

2221 1

12 2

xz

e dz e dz

The Exponential distribution

Consider a continuous random variable, X with the following properties:

1. P[X ≥ 0] = 1, and

2. P[X ≥ a + b] = P[X ≥ a] P[X ≥ b] for all a > 0, b > 0.

These two properties are reasonable to assume if X = lifetime of an object that doesn’t age.

The second property implies:

P X a b

P X a b X a P X bP X a

The property:

models the non-aging property

P X a b X a P X b

i.e. Given the object has lived to age a, the probability that is lives a further b units is the same as if it was born at age a.

Let F(x) = P[X ≤ x] and G(x) = P[X ≥ x] .Since X is a continuous RV then G(x) = 1 – F(x)(P[X ≥ x] = 0 for all x.)

1. G(0) = 1, and

2. G(a + b) = G(a) G(b) for all a > 0, b > 0.

The two properties can be written in terms of G(x):

We can show that the only continuous function, G(x), that satisfies 1. and 2. is a exponential function

From property 2 we can conclude

22G a G a a G a G a G a

Using induction

nG na G a a G a G a G a

Hence putting a = 1.

1n

G n G

Also putting a = 1/n.

11n

nG G 1

1 or 1 nnG G

Finally putting a = 1/m.

1

1 1 1n

m mnn

nm mG G G G

Since G(x) is continuous

1x

G x G for all x ≥ 0.

Now 0 1 1G

If G(1) = 0 then G(x) = 0 for all x > 0 and G(0) = 0 if G is continuous. (a contradiction)

Also 0 1 and 0.G G

If G(1) = 1 then G(x) = 1 for all x > 0 and G() = 1 if G is continuous. (a contradiction)

Thus G(1) ≠ 0, 1 and 0 < G(1) < 1

Let = - ln(G(1)) then G(1) = e-

Thus 1xx xG x G e e

0

0 0

xe xf x F x

x

To find the density of X we use:

A continuous random variable with this density function is said to have the exponential distribution with parameter .

0 0Finally 1

1 0x

xF x G x

e x

0

0.5

1

-2 0 2 4 6

Graphs of f(x) and F(x)

0

0.5

1

-2 0 2 4 6

f(x)

F(x)

Consider a continuous random variable, X with the following properties:

1. P[X ≥ 0] = 1, and

2. P[x ≤ X ≤ x + dx|X ≥ x] = dx

for all x > 0 and small dx.

These two properties are reasonable to assume if X = lifetime of an object that doesn’t age.

The second property implies that if the object has lived up to time x, the chance that it dies in the small interval x to x + dx depends only on the length of that interval, dx, and not on its age x.

Another derivation of the Exponential distribution

Let F (x ) = P[X ≤ x] = the cumulative distribution function of the random variable, X .

Then P[X ≥ 0] = 1 implies that F(0) = 0.

Determination of the distribution of X

Also P[x ≤ X ≤ x + dx|X ≥ x] = dx implies

P x X x dxP x X x dx X x

P X x

1

F x dx F xdx

F x

or 1

F x dx F x dF xF x

dx dx

We can now solve the differential equation

for the unknown F.

1

dFF

dx

1 1

dF dxF

1 or

1dF dx

F

and ln 1 F x c

ln 1 F x c

and 1 or 1x c x cF e F F x e

Now using the fact that F(0) = 0.

0 1 0 implies 1 and 0c cF e e c

Thus 1 and x xF x e f x F x e

This shows that X has an exponential distribution with parameter .

The Weibull distribution

A model for the lifetime of objects that do age.

Recall the properties of continuous random variable, X, that lead to the Exponential distribution. namely

1. P[X ≥ 0] = 1, and

2. P[x ≤ X ≤ x + dx|X ≥ x] = dx

for all x > 0 and small dx.

Suppose that the second property is replaced by:

The second property now implies that object does age. If it has lived up to time x, the chance that it dies in the small interval x to x + dx depends both on the length of that interval, dx, and its age x.

2. P[x ≤ X ≤ x + dx|X ≥ x] = x)dx = x – 1dx

for all x > 0 and small dx.

A continuous random variable, X satisfies the following properties:

1. P[X ≥ 0] = 1, and

2. P[x ≤ X ≤ x + dx|X ≥ x] = x-1dx

for all x > 0 and small dx.

Derivation of the Weibulll distribution

Let F (x ) = P[X ≤ x] = the cumulative distribution function of the random variable, X .

Then P[X ≥ 0] = 1 implies that F(0) = 0.

Also P[x ≤ X ≤ x + dx|X ≥ x] = x-1dx implies

P x X x dxP x X x dx X x

P X x

1

1

F x dx F xx dx

F x

1 or 1F x dx F x dF x

x F xdx dx

We can now solve the differential equation

for the unknown F.

1 1

dFx F

dx

11 1

dF x dxF

11 or

1dF x dx

F

and ln 1x

F c

ln 1 F x c

1 or 1x c x c

F e F x e

again F(0) = 0 implies c = 0.

The distribution of X is called the Weibull distribution with parameters and.

Thus 1x

F x e

1and 0x

f x F x x e x

The Weibull density, f(x)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5

( = 0.5, = 2)

( = 0.7, = 2)

( = 0.9, = 2)

The Gamma distribution

An important family of distibutions

The Gamma Function, (x)

An important function in mathematics

The Gamma function is defined for x ≥ 0 by

1

0

x ux u e du

1x uu e

x

u

Properties of Gamma Function, (x)

1. (1) = 1

0

00

1 1u ue du e e e

2. (x + 1) = x(x)

0

0

1 x u x ux u e du u e du

We will use integration by parts to evaluate x uu e du

x uu e du wdv wv vdw 1where and , hence ,x u x uw u dv e du dw xu dx v e

1Thus x u x u x uu e du u e x u e du

1

00 0

and x u x u x uu e du u e x u e du

or 1 0x x x

3. (n) = (n - 1)! for any positive integer n

using 1 and 1 1x x x

2 1 1 1 1!

3 2 2 2 2!

4 3 3 3 2 3!

121 1

20 0

thus x u ux u e du u e du

4. 12

Recall the density for the normal distribution

2

221

2

x

f x e

If and 1 then

2

21

2

x

f x e

2 2 2

2 2 2

0 0

1 1 2Now 1= 2

2 2

x x x

e dx e dx e dx

2

2

0

Thus =2

x

e dx

Make the substitution 2

2 .xu 121

21

Hence 2 and or .2

ux u du xdx dx du du

x

Also when 0, then 0,x u

122

2 12

0 0

1=

2 2 2

x u ue dx e dx

12and = .

3 1 1 12 2 2 2= .

Using 1 .x x x

5 3 3 3 12 2 2 2 2= .

7 5 5 5 3 12 2 2 2 2 2= .

2 1 2 1 5 31 12 2 2 2 2 2=n nn

12 2

2 !5.

!2 n

nn

n

2

2 ! 2 !

2 !2 !2n n n

n n

n n

Graph: The Gamma Function

0

5

10

15

20

25

0 1 2 3 4 5

x

(x)

The Gamma distribution

Let the continuous random variable X have density function:

1 0

0 0

xx e xf x

x

Then X is said to have a Gamma distribution with parameters and .

Graph: The gamma distribution

0

0.1

0.2

0.3

0.4

0 2 4 6 8 10

( = 2, = 0.9)

( = 2, = 0.6)

( = 3, = 0.6)

Comments

1. The set of gamma distributions is a family of distributions (parameterized by and ).

2. Contained within this family are other distributionsa. The Exponential distribution – in this case = 1, the

gamma distribution becomes the exponential distribution with parameter . The exponential distribution arises if we are measuring the lifetime, X, of an object that does not age. It is also used a distribution for waiting times between events occurring uniformly in time.

b. The Chi-square distribution – in the case = /2 and = ½, the gamma distribution becomes the chi- square (2) distribution with n degrees of freedom. Later we will see that sum of squares of independent standard normal variates have a chi-square distribution, degrees of freedom = the number of independent terms in the sum of squares.

The Exponential distribution

0

0 0

xe xf x

x

The Chi-square (2) distribution with d.f.

21

2 2

112

2

0

0 0

xx e xf x

x

2

2 2

22

10

2

0 0

x

x e x

x

0

0.1

0.2

0 4 8 12 16

Graph: The 2 distribution

( = 4)

( = 5)

( = 6)

Summary

Important continuous distributions

Uniform distribution from a to b

1

0 otherwise

a x bf x b a

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x0

0.5

1

0 5 10 15 a b x

F x

0

1

x a

x aF x P X x a x b

b ax b

Normal distribution mean and standard deviation

2

221

2

x

f x e

0

0 0

xe xf x F x

x

the exponential distribution with parameter

f(x) =

0

0.5

1

-2 0 2 4 6

f(x)

The Weibull distribution - parameters and.

1and 0x

f x F x x e x

f(x) =

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5

( = 0.5, = 2)

( = 0.7, = 2)

( = 0.9, = 2)

The gamma distribution - parameters and

0

0.1

0.2

0.3

0.4

0 2 4 6 8 10

( = 2, = 0.9)

( = 2, = 0.6)

( = 3, = 0.6)

1 0

0 0

xx e xf x

x

The 2 distribution with degrees of freedom

0

0.1

0.2

0 4 8 12 16

( = 4)

( = 5)

( = 6)

21

2 2

112

2

0

0 0

xx e xf x

x