Upload
shawn-sparks
View
218
Download
1
Embed Size (px)
Citation preview
week 7 1
Joint Distribution of two or More Random Variables
• Sometimes more than one measurement (r.v.) is taken on each member of the sample space. In cases like this there will be a few random variables defined on the same probability space and we would like to explore their joint distribution.
• Joint behavior of 2 random variable (continuous or discrete), X and Y determined by their joint cumulative distribution function
• n – dimensional case
.,,, yYxXPyxF YX
.,,,..., 111,...,1 nnnXX xXxXPxxFn
week 7 2
Discrete case • Suppose X, Y are discrete random variables defined on the same probability
space.
• The joint probability mass function of 2 discrete random variables X and Y is the function pX,Y(x,y) defined for all pairs of real numbers x and y by
• For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and
yYandxXPyxp YX ,,
1,, x y
YX yxp
week 7 3
Example for illustration • Toss a coin 3 times. Define,
X: number of heads on 1st toss, Y: total number of heads.
• The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}.
• We display the joint distribution of X and Y in the following table
• Can we recover the probability mass function for X and Y from the joint table?
• To find the probability mass function of X we sum the appropriate rows of the table of the joint probability function.
• Similarly, to find the mass function for Y we sum the appropriate columns.
week 7 4
Marginal Probability Function • The marginal probability mass function for X is
• The marginal probability mass function for Y is
• Case of several discrete random variables is analogous.• If X1,…,Xm are discrete random variables on the same sample space with
joint probability function
The marginal probability function for X1 is
• The 2-dimentional marginal probability function for X1 and X2 is
y
YXX yxpxp ,,
x
YXY yxpyp ,,
mmmXX xXxXPxxpn
,...,,... 111,...1
m
nxx
mXXX xxpxp,...,
1,...1
2
11,...
m
nxx
mXXXX xxxxpxxp,...,
321,...21
3
121,...,,,,
week 7 5
Example • Roll a die twice. Let X: number of 1’s and Y: total of the 2 die.
There is no available form of the joint mass function for X, Y. We display the joint distribution of X and Y with the following table.
• The marginal probability mass function of X and Y are
• Find P(X ≤ 1 and Y ≤ 4)
week 7 6
Independence of random variables
• Recall the definition of random variable X: mapping from Ω to R such that
for .
• By definition of F, this implies that (X > 1.4) is and event and for the discrete case (X = 2) is an event.
• In general is an event for any set A that is formed by taking unions / complements / intersections of intervals from R.
• DefinitionRandom variables X and Y are independent if the events and are independent.
FxX :
AXAX :
Rx
AX BY
week 7 7
Theorem
• Two discrete random variables X and Y with joint pmf pX,Y(x,y) and marginal mass function pX(x) and pY(y), are independent if and only if
• Proof:
• Question: Back to the rolling die 2 times example, are X and Y independent?
ypxpyxp YXYX ,,
week 7 8
Conditional Probability on a joint discrete distribution
• Given the joint pmf of X and Y, we want to find
and
• Back to the example on slide 5,
• These 11 probabilities give the conditional pmf of Y given X = 1.
yYP
yYandxXPyYxXP
|
xXP
yYandxXPxXyYP
|
01|2 XYP
5
1
3610
362
1|3 XYP
01|12 XYP
week 7 9
Definition• For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then theconditional pmf of Y given X = x is
• Is this a valid pmf?
• Similarly, the conditional pmf of X given Y = y is
• Note, from the above conditional pmf we get
Summing both sides over all possible values of Y we get
This is an extremely useful application of the law of total probability.
• Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
xp
yxpxXypxyp
X
YXXYXY
,|| ,
||
yp
yxpyYxpyxp
Y
YXYXYX
,|| ,
||
ypyxpyxp YYXYX |, |,
y
YYXy
YXX ypyxpyxpxp |, |,
week 7 10
Example
• Suppose we roll a fair die; whatever number comes up we toss a coin that many times. What is the distribution of the number of heads?Let X = number of heads, Y = number on die.We know that
Want to find pX(x).
• The conditional probability function of X given Y = y is given by
for x = 0, 1, …, y.
• By the Law of Total Probability we have
Possible values of x: 0,1,2,…,6.
6,5,4,3,2,1,6
1 yypY
2
| 2
1|
x
yyxp YX
6
| 6
1
2
1|
xy
y
yYYXX x
yypyxpxp
week 7 11
Example • Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of
success p on any trail. Let X = number of failures before 1st success andY = number of failures before 2nd success (including those before the 1st success).
• The marginal pmf of X and Y are for x = 0, 1, 2, ….
• andfor y = 0, 1, 2, ….
• The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is the probability that the 1st success occurred after x failure and the 2nd success after y – x more failures (after 1st success), e.g.
• The joint pmf is then given by
• Find the marginal probability functions of X, Y. Are X, Y independent? • What is the conditional probability function of X given Y = 5
xX pqxXPxp
yY qpyyYPyp 21
52, 5,2 qpFFSFFFSPp YX
otherwise
yxqpyxp
y
YX0
0,
2
,
week 7 12
Some Notes on Multiple Integration
• Recall: simple integral cab be regarded as the limit as N ∞ ofthe Riemann sum of the form
where ∆xi is the length of the ith subinterval of some partition of [a, b].
• By analogy, a double integral
where D is a finite region in the xy-plane, can be regarded as the limit as N ∞ of the Riemann sum of the form
where Ai is the area of the ith rectangle in a decomposition of D into rectangles.
b
adxxf
N
iii xxf
1
D
dAyxf ,
N
iiiii
N
iiii yxyxfAyxf
11
,,
week 7 13
Evaluation of Double Integrals • Idea – evaluate as an iterated integral. Restrict D now to a region such that
any vertical line cuts its boundary in 2 points. So we can describe D by
yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
• Theorem
Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
If f(x,y) is continuous on D then
• Comment: When evaluating a double integral, the shape of the region or the form of f (x,y) may dictate that you interchange the role of x and y and integrate first w.r.t x and then y.
U
L
U
L
x
x
y
yD
dxdyyxfdAyxf ,,
week 7 14
Examples
1) Evaluate where D is the triangle with vertices (0,0), (2,0) and (0,1).
2) Evaluate where D is the region in the 1st quadrate bounded
by y = 0, y = x and x2 + y2 = 1.
3) Integrate over the set of points in the positive quadrate for
which y > 2x.
D
ydAx
D
dAyxxy 22
yxxeyxf 2,
week 7 15
The Joint Distribution of two Continuous R.V’s
• DefinitionRandom variables X and Y are (jointly) continuous if there is a non-negative function fX,Y(x,y) such that
for any “reasonable” 2-dimensional set A.
• fX,Y(x,y) is called a joint density function for (X, Y).
• In particular , if A = {(X, Y): X ≤ x, Y ≤ x}, the joint CDF of X,Y is
• From Fundamental Theorem of Calculus we have
A
YX dxdyyxfAYXP ,, ,
x y
YXYX dvduvufyxF ,,, ,,
yxFxy
yxFyx
yxf YXYXYX ,,, ,
2
,
2
.
week 7 16
Properties of joint density function
• for all
• It’s integral over R2 is
0,, yxf YX
1,, dxdyyxf YX
Ryx ,
week 7 17
Example• Consider the following bivariate density function
• Check if it’s a valid density function.
• Compute P(X > Y).
otherwise
yxxyxyxf YX
0
10,107
12,
2
,
week 7 18
Properties of Joint Distribution Function
For random variables X, Y , FX,Y : R2 [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)
• FX,Y (x,y) is non-decreasing in each variable i.e.
if x1 ≤ x2 and y1 ≤ y2 .
• and
0,lim ,
yxF YX
yx
1,lim ,
yxF YX
yx
22,11, ,, yxFyxF YXYX
yFyxF YYXx
,lim , xFyxF XYXy
,lim ,
week 7 19
Marginal Densities and Distribution Functions
• The marginal (cumulative) distribution function of X is
• The marginal density of X is then
• Similarly the marginal density of Y is
x
YXX dyduyufxXPxF ,,
dyyxfxFxf YXXX ,,
'
dxyxfyf YXY ,,
week 7 20
Example
• Consider the following bivariate density function
• Check if it is a valid density function.
• Find the joint CDF of (X, Y) and compute P(X ≤ ½ ,Y ≤ ½ ).
• Compute P(X ≤ 2 ,Y ≤ ½ ).
• Find the marginal densities of X and Y.
otherwise
yxxyyxf YX
0
10,106,
2
,
week 7 21
Generalization to higher dimensions
Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then
• Marginal density of X is given by:
• Marginal density of X, Y is given by :
dydzzyxfxf X ,,
dzzyxfyxf YX ,,,,
week 7 22
Example
Given the following joint CDF of X, Y
• Find the joint density of X, Y.
• Find the marginal densities of X and Y and identify them.
10,10, 2222, yxyxxyyxyxF YX
week 7 23
Example
Consider the joint density
where λ is a positive parameter.
• Check if it is a valid density.
• Find the marginal densities of X and Y and identify them.
otherwise
yxeyxf
y
YX0
0,
2
,