Chapter 6: Functions of Random Variablesfaculty.nps.edu/rdfricke/OA3101/Chapter 6.pdf · Chapter 6: Functions of Random Variables ... • Solution: 3/15/15 9 f (y)= ⇢ 2y ... Mathematical

Chapter 6:Functions of Random Variables

Professor Ron FrickerNaval Postgraduate School

Monterey, California

3/15/15 1Reading Assignment: Sections 6.1 – 6.4 & 6.8

Goals for this Chapter

•  Introduce methods useful for finding the probability distributions of functions of random variables–  Method of distribution functions–  Method of transformations

3/15/15 2

Sections 6.1 & 6.2:Functions of Random Variables

•  In OA3102 you’re going to learn to do inference–  Using a sample of data, you’re going to make

quantitative statements about the population–  For example, you might want to estimate the mean

of the population from a sample of data Y1,…, Yn

•  To estimate the mean of the population (distribution) you will use the sample average

3/15/15 3

µµ

Y =nX

i=1

Yi

Functions of Random Variables

•  But in doing statistics, you will quantify the “goodness” of the estimate–  This error of estimation will have to be quantified

probabilistically since the estimates themselves are random

–  That is, the observations (the Yis) are random observations from the population, so they are random variables, and so are any statistics that are functions of the random variables

–  So, is a function of other random variables and the question then is how to find its probability distribution?

3/15/15 4

Y

But Functions of Random Variables Arise in Other Situations As Well

•  You’re analyzing a subsurface search and detection operation–  Data shows that the time to detect one target is uniformly

distributed on the interval (0, 2 hrs)–  But you’re interested in analyzing the total time to

sequentially detect n targets, so you need to know the distribution of the sum of n detection times

•  You’re analyzing a depot-level repair activity–  It’s reasonable to assume that the time to receipt of one

repair part follows an exponential distribution with days

–  But the end-item is not repaired until the last part is received, so you need to know the distribution of the maximum receipt time of n parts

3/15/15 5

� = 5

Functions of Random Variables

•  To determine the probability distribution for a function of n random variables, Y1, Y2, …, Yn, we must find the joint probability distribution of the r.v.s

•  To do so, we will assume observations are:–  Obtained via (simple) random sampling–  Independent and identically distributed

•  Thus:or

3/15/15 6

p(y1, y2, . . . , yn) = p(y1)p(y2) · · · p(yn)

f(y1, y2, . . . , yn) = f(y1)f(y2) · · · f(yn)

Methods for Finding Joint Distributions

•  Three methods:–  Method of distribution functions–  Method of transformations–  Method of moment-generating functions

•  We’ll do the first two–  Method of moment-generating functions follows

from Chapter 3.9 (which we skipped)•  If two random variables have the same

moment-generating functions, then they also have the same probability distributions

–  Beyond the scope of this class (unless you’re an OR Ph.D. student)

3/15/15 7

Section 6.3:Method of Distribution Functions

•  The idea: You want to determine the distribution function of a random variable U –  U is a function of another random variable Y –  You know the pdf of Y, f (y)

•  The methodology “by the numbers”:1.  For Y with pdf f (y), specify the function U of Y2.  Then find by directly

integrating f (y) over the region for which 3.  Then the pdf for U is found by differentiating

•  Let’s look at an example…

3/15/15 8

FU (u)

U uFU (u) = P (U u)

Textbook Example 6.1

•  For clarity, let’s dispense with the story and get right to the problem: Y has pdf

Find the pdf of U, fU(u), where U = 3Y-1. •  Solution:

3/15/15 9

f(y) =

⇢2y, 0 y 10, elsewhere

Textbook Example 6.1 Solution (cont’d)

3/15/15 10

Can Also Apply the Method to Bivariate (and Multivariate) Distributions

•  The idea:–  In the bivariate case, there are random variables

Y1 and Y2 with joint density f (y1,y2) –  There is some function U = h(Y1,Y2) of Y1 and Y2

•  I.e., for every point (y1,y2) there is one and only one value of U

–  If there exists a region of values (y1,y2) such that , then integrating f (y1,y2) over this region gives

–  Then get by differentiation•  Again, let’s do some examples…

3/15/15 11

U uP (U u) = FU (u)

fU (u)


•  Returning to Example 5.4, we had the following joint density of Y1 and Y2:

Find the pdf of U, fU(u), where U = Y1-Y2. Also find E(U).

•  Solution:

3/15/15 12

f(y1, y2) =

⇢3y1, 0 y2 y1 10, elsewhere

Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7th edition, Thomson Brooks/Cole.


3/15/15 13


3/15/15 14


3/15/15 15

•  Graphs of the cdf and pdf of U:



•  Let (Y1,Y2) denote a random sample of size n = 2 from the uniform distribution on (0,1). Find the pdf for U = Y1+Y2.

•  Solution:

3/15/15 16



3/15/15 17



•  Using a geometry-based approach…

3/15/15 18


3/15/15 19



3/15/15 20


Empirically Demonstrating the Result

•  Let’s use R to verify the Example 6.3 result:

3/15/15 21

Histogram of u

u

Density

0.0 0.5 1.0 1.5 2.0

0.0

0.2

0.4

0.6

0.8

1.0

Very close. Of course, there’s “noise” related to the stochastic nature of the demonstration.

Distribution Function Method Summary

Let U be a function of the random variables Y1, Y2, …, Yn. Then:

1.  Find the region U = u in the (y1, y2, …, yn) space

2.  Find the region3.  Find by integrating

over the region4.  Find the density function by

differentiating :

3/15/15 22

U u

FU (u) = P (U u)f(y1, y2, . . . , yn) U u

fU (u)FU (u) fU (u) = dFU (u)/du

Illustrating the Method

3/15/15 23

•  Consider the case , where Y is continuous with cdf and pdf

1.  Find the region U = u in the (y1, y2, …, yn) spaceSolution: As Figure 6.7 shows, or

U = h(Y ) = Y 2

FY (y) fY (y)

Y 2 = uY = ±

pu


Illustrating the Method Continued

2.  Find the region whereSolution: By inspection of Figure 6.7, it should be clear that whenever

3.  Find by integrating over the regionSolution: There are two cases to consider. First, if then it must be (see Fig. 6.7)

Second, if … 3/15/15 24

U u

U u�pu Y

pu

FU (u) = P (U u)f(y1, y2, . . . , yn) U u

u 0

FU (u) = P (U u) = P (Y 2 u) = 0

u > 0

Illustrating the Method Continued

So, we can write the cdf of U as

4.  Finally, find the pdf by differentiating :

3/15/15 25

FU (u) =

Z pu

�puf(y)dy = FY (

pu)� FY (�

pu)

FU (u) =

⇢FY (

pu)� FY (�

pu), u > 0

0, otherwise

fU (u) =

⇢ 12pu[fY (

pu) + fY (�

pu)] , u > 0

0, otherwise

FU (u)


•  Let Y have pdf

Find the pdf for U = Y 2

. •  Solution:

3/15/15 26

fY (y) =

⇢(y + 1)/2, �1 y 10, elsewhere


•  Let U be a uniform r.v. on (0,1). Find a transformation G(U) such that G(U) has an exponential distribution with mean .

•  Solution:

3/15/15 27

�


3/15/15 28

Empirically Demonstrating the Result

•  Let’s use R to verify the Example 6.5 result:

3/15/15 29

Histogram of y

y

Density

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

Histogram of exp_rvs

exp_rvs

Density

0 2 4 6 8 10 12

0.0

0.2

0.4

0.6

0.8

Simulating Exponential R.V.s Using U(0,1)s

•  What the last example shows is that we can use computer subroutines designed to generate U(0,1) random variables to generate other types of random variables–  It works as long as the cdf F(y) has a unique

inverse–  Not always the case; for those there are other

methods–  And, the beauty of R and other modern software

packages is that most or all of this work is already done for you

3/15/15 30

F�1(·)

Section 6.3 Homework

•  Do problems 6.1, 6.4, 6.11

3/15/15 31

Section 6.4:Method of Transformations

•  A (sometimes simpler) method to use, if is an increasing / decreasing function–  Remember, an increasing function is one where,

if y1 < y2, then h(y1) < h(y2) for all y1 and y2:

–  Similarly, a decreasing function is one where, if y1 > y2, then h(y1) > h(y2) for all y1 and y2

3/15/15 32

U = h(y)


h�1(u)

Method of Transformations

3/15/15 33

•  Note that if h(y) is an increasing function of y then is an increasing function of u –  If u1 < u2, then

•  So, the set of points y such that is the same as the set of points y such that

•  This allows us to derive the following result…

h�1(u1) = y1 < y2 = h�1(u2)

h(y) u1

y h�1(u1)


Method of Transformations

•  And, from this result, it follows that

3/15/15 34

FU (u) = P (U u) = P [h(Y ) u]

= P�h�1[h(Y )] h�1(u)

= P [Y h�1(u)]

= FY [h�1(u)]

fU (u) =dFU (u)

du

=dFY [h�1(u)]

du

= fY�h�1(u)

� d[h�1(u)]

du


•  Returning to Example 6.1, solve via the Method of Transformations, where Y has pdf

I.e., find the pdf of U, fU(u), where U = 3Y-1. •  Solution:

3/15/15 35

f(y) =



3/15/15 36

Now, for Decreasing Functions…

•  Now, if h is a decreasing function (e.g., Figure 6.9)then if u1 < u2, we have

•  From this, it follows that

•  Differentiating gives…

3/15/15 37

h�1(u1) = y1 > y2 = h�1(u2)

FU (u) = P (U u) = P [Y � h�1(u)]

= 1� FY [h�1(u)]


Putting It All Together

3/15/15 38

•  Theorem: Let Y have pdf . If h(y) is either increasing or decreasing for all y such that ,* then U = h(Y) has pdf

fU (u) = �fY [h�1(u)]

d[h�1(u)]

du

= fY [h�1(u)]

��d[h�1(u)]

du

��

fU (u) = fY [h�1(u)]

��d[h�1(u)]

du

��

fY (y)

fY (y) > 0

* Note that the function only has to be increasing or decreasing over the support of the pdf of y: .{y : fY (y) > 0}


•  Let Y have pdf

Find the pdf of U = -4Y+3. •  Solution:

3/15/15 39

f(y) =



3/15/15 40


•  Let Y1 and Y2 have the joint pdf

Find the pdf for U = Y1 + Y2 . •  Solution:

3/15/15 41

f(y1, y2) =

⇢e�(y1+y2), y1 � 0, y2 � 0

0, otherwise


3/15/15 42


•  Example 5.19 had the joint pdf

Find the pdf for U = Y1Y2 as well as E(U). •  Solution:

3/15/15 43

f(y1, y2) =

⇢2(1� y1), 0 y1 1, 0 y2 1

0, otherwise


3/15/15 44


3/15/15 45

Distribution Function Method Summary

Let U = h(Y ), where h(Y ) is either an increasing or decreasing function of y for all y such that .

1.  Find the inverse function

2.  Evaluate

3.  Find by

3/15/15 46

fY (y) > 0

y = h�1(u)

d[h�1(u)]

du

fU (u) = fY [h�1(u)]

��d[h�1(u)]

du

��

fU (u)

Section 6.4 Homework

•  Do problems 6.23, 6.24, 6.25

3/15/15 47

Section 6.8:Summary

•  We often need to know the probability distribution for functions of random variables–  This is a common requirement for statistics and

will arise throughout OA3102 and OA3103–  It also comes up in lots of other ORSA probability

problems•  Today we can often simulate and get an

empirical estimate of a distribution, as we did here for a couple of examples, but a closed-form analytical result is better

3/15/15 48

Summary Continued

•  We covered two methods –  Distribution function method (chpt. 6.3)–  Transformation method (chpt. 6.4)

and we skipped over two methods–  Moment-generating function method (chpt. 6.5)–  Multivariable transforms w/ Jacobians (chpt. 6.6)

•  Which is better depends on the problem–  Sometimes one or more are inapplicable –  Sometimes one is easier than the other–  Sometimes one or more works and others don’t

3/15/15 49

Summary Continued

•  The sections we skipped prove some results you need to know for OA3102 and beyond:

1. 

2. 

3. 

3/15/15 50

If Y ⇠ N(µ,�2) then Z =Y � µ

�⇠ N(0, 1)

If Z ⇠ N(0, 1) then Z2 ⇠ �2(⌫ = 1)

If Zi ⇠ N(0, 1), i = 1, . . . , n thennX

i=1

Z2i ⇠ �2(n)

Summary Continued

4.  Let Y1, Y2, …, Yn be independent normally distributed random variables with and for i = 1, 2, …, n, and let a1, a2, …, an be constants. Then is a normally distributed random variable with and

3/15/15 51

E(Yi) = µi

V (Yi) = �2i

U =nX

i=1

aiYi = a1Y1 + a2Y2 + · · ·+ anYn

E(U) =nX

i=1

aiµi V (U) =nX

i=1

a2i�2i

What We Have Just Learned

•  Introduced methods useful for finding the probability distributions of functions of random variables–  Method of distribution functions–  Method of transformations

3/15/15 52

Documents

Chapter 6: Functions of Random Variablesfaculty.nps.edu/rdfricke/OA3101/Chapter 6.pdf · Chapter 6: Functions of Random Variables ... • Solution: 3/15/15 9 f (y)= ⇢ 2y ... Mathematical