Upload
kazunsanjaya
View
214
Download
0
Embed Size (px)
Citation preview
Review of Random Variables
for Communications
EN 2072
Semester 4 May 2011
Prof. Dileeka Dias
Department of Electronic & Telecommunication Engineering
University of Moratuwa
Contents
Introduction Random Variables
Discrete Continuous
Joint Random Variables Discrete Continuous
Functions of Random Variables Single Random Variable
Mean Variance Moments
Two Random variables Correlation and Covariance
Random Variables: Definition
Outcome of a random experiment may be A numerical value (1, 2, 3, 4, 5, 6) Described by a phrase (Heads, Tails)
From a mathematical point of view it is preferable to have a numerical value (real number) assigned to each sample point according to some rule.
If there are m sample points , we assign a real number x( ) to each sample point.
X( ) is the function that maps sample points into real numbers x1, x2, .. xm
X is a random variable which takes on values x1, x2, .. xm.
Random Variables: Definition
Random Variables: Definition
Example 1 Heads 1
Tails 0
Example 2 1 10
2 20
3 30
4 40
5 50
6 60
X = 1 or 0 , each with probability 1/2
X = 10, 20, 30 .60, each with
probability 1/6
iix 10
"" ,0
"" ,1
Tails
Headsx
i
ii
Random Variables: Discrete
A discrete random variable maps events to values of a countable set (e.g., the integers), with each value in the range having probability greater than or equal to zero.
A discrete random variable is described by a discrete proability density function (probablity mass function)
1)(where
....... ,2 ,1),(
i
iX
iX
xP
mixP
Random Variables: Discrete
Discrete Probability Density Function (PDF) or Probability Mass Function (PMF) and Discrete Cumulative Distribution Function (CDF)
Discrete PDF: Discrete CDF:
4 ,6
2
2 ,6
3
1 ,6
1
)(
x
x
x
xP iX
4 ,1
42 ,6
4
21 ,6
11 ,0
)()(
x
x
x
x
xXPxFX
Random Variables: Discrete
Example: The Poisson random variable
A random variable X is said to have a Poisson distribution if
Where is called average or the expected value
...... 3 ,2 ,1 ,0 ,!
)(
kk
ekXP
k
PDF CDF
Example: Telephone calls arriving at a
switch, page view requests to a website are examples of Poisson processes
The probability that there are k incoming calls during the time t and t+ is given by
Where is the average call arrival rate
!
)()()(
k
ektNtNP
k
Random Variables: Discrete
Random Variables: Continuous
A random variable is called continuous if it can assume all possible values in the possible range of the random variable.
The continuous random variable X is described by the (continuous) probability density function.
Random Variables: Continuous
It is denoted by , the probability that the random variable X takes the value between x and x+ x where is a very small change in X. )()( xxXxPxfX
)(xfX
Random Variables: Continuous
If there are two points a and b then the probability that the random variable will take the value between a and b is given by
1)(
dxxf X
Random Variables: Continuous
The CDF of a continuous random variable is given by:
x
UX duufxF )()(
)()( xFxf Xdxd
X
xexf
x
X ,2
1)(
2
2
2
)(
2
),( 2N
onDistributi Normal Standard )1,0( N
Source: Wikipedia
Mean: Variance: 2
Random Variables: Continuous
The Gaussian Random Variable
Random Variables: Continuous
2
2/
0
0
2
)(
2
0
2
)(
2
2
)(
2
22
1
2
1
1
2
1
2
1
2
1
2
1
)()(
2
2
2
2
2
2
2
2
xerf
dte
duedue
due
duufxF
x
t
x uu
x u
x
XX
The Gaussian Random Variable
x
t dtexerf
0
22
2
ut
Random Variables: Continuous
xQ
xerfxFX
1
22
1
2
1)(
2
The Gaussian Random Variable
x
t dtexerf
0
22
21
2
1 xerfxQ
Random Variables: Continuous
Example:
Transmitted pulses
Received signal
1 p(t) 0 -p(t)
The received signal is Sampled at the peak point And compared with a threshold of 0 The sample consists of a contribution from the signal and a contribution from the AWGN that corrupts the channel. The sample value can be Ap + n -Ap + n
n is a sample value of the random variable with zero mean
Random Variables: Continuous
Probability of error
P(e|0) = P(-AP + n) > 0
= P(n > Ap )
P(e|1) = P(AP + n) < 0
= (n< - AP)
P(n > Ap ) P(n< - AP)
Random Variables: Continuous
For a Gaussian Random variable
)(1)( xXP
xQxFX
nQ
nQnXP 11)(
PP
PP
AQAnPeP
AQAnPeP
)()1|(
)()0|(
Two (Joint) Random Variables: Discrete
If X and Y are two discrete random variables, the conditional probability of xi and yj is given by
)|(| jiYX yxP
1)|()|( || ijXYj
jiYX
i
xyPyxP
)()|()()|(),( || iXijXYjYjiYXjiXY xPxyPyPyxPyxP
Using
Conditional Probability
Joint Probability
Joint Random Variables : Discrete
)(
)|()(
)()|(),(
|
|
jY
jiYX
i
jY
jYjiYX
i
jiXY
i
yP
yxPyP
yPyxPyxP
)()|()()|(),( || iXijXYjYjiYXjiXY xPxyPyPyxPyxP
)(),( iXjiXY
j
xPyxP Similarly Marginal Probabilities
Joint Random Variables : Discrete
)()|()()|(),( || jXijXYjYjiYXjiXY xPxyPyPyxPyxP
)()(),(
Hence,
)()|(
or )()|(
ift independen are Y and X
|
|
jYiXjiXY
jYijXY
iXjiYX
yPxPyxP
yPxyP
xPyxP
Joint Random Variables : Discrete
Example: A BSC has error probability p. The probability of transmitting a 1 is a and the probability of transmitting a 0 is 1-a.
Determine the probabilities of receiving a 1 and a 0 at the receiver.
X: RV indicating the input Y: RV indicating the output x1 = 1, x2 = 0 PX (1 ) = a, PX (0 ) = 1-a
pPP
pPP
XYXY
XYXY
1)1|1()0|0(
)0|1()1|0(
||
||
Joint Random Variables : Discrete
y1 = 1
PY (1 ) = ??
y2 = 0
PY (0 ) = ??
PX (1 ) = a x1 = 1
PX (0 ) = 1-a
x2 = 0
Probability of error
Probability of correct reception
paap
apapPPPPP
j
XYXYY
2)(
)1()1()0()0|1()1()1|1()1(
1for
||
)()|(),()( | iXijXY
i
jiXY
i
jY xPxyPyxPyP
)1(12)(1
)1)(1()0()0|0()1()1|0()0(
2for
||
Y
XYXYY
Ppaap
appaPPPPP
j
Joint CDF of continuous random variables
Joint PDF of continuous random variables
Joint Random Variables: Continuous
1),(
dxdyyxf XY
) and (),( yYxXPyxFXY
),(
),(2
yxFyx
yxf XYXY
dxdyyxfyYyxXxP
x
x
y
y
XY ),(),(
2
1
2
1
2121
Joint Random Variables : Continuous
Conditional Densities for continuous random variables
X and Y are independent if
)(
)()|()|(
Hence,
)()|(),(
)()|(),(
||
|
|
yf
xfxyfyxf
xfxyfyxf
yfyxfyxf
Y
XXYYX
XXYXY
YYXXY
Bayes Rule for Continuous RVs
Conditional PDFs
)()|(
or )()|(
|
|
yfxyf
xfyxf
YXY
XYX
Joint PDFs
)()(),( yfxfyxf YXXY Which means that
Functions of a Random Variable
The Mean (Expected Value)
Example: for a Gaussian RV,
i
ii
X
xPx
dxxxfXXE
)(
or
)(][__
dyedyye
dyeyXE
dxxeXE
yy
y
x
2
2
2
2
2
2
2
2
2
)(
2
)(
2
)(
2
)(
2
2
1
)(2
1][
y Let x
2
1][
Odd function of y 2
DC value of a signal
Functions of a Random Variable
The Mean (Expected Value)
Let Y = g(X)
Therefore:
Mean Square Value
A function of a RV
i
ii
X
xPxg
dxxfxgYYE
)()(
or
)()(][__
dxxfxXE X )(][22
Functions of a Random Variable
The Mean (Expected Value)
Let Y = g(X)
Therefore:
A function of a RV
i
ii
X
xPxg
dxxfxgYYE
)()(
or
)()(][__
dxxfxXE X )(][22
Mean Square Value of a signal
Root Mean Square (RMS) value of a signal
][ 2XE
Functions of a Random Variable
Example
A sinusoidal signal is given by . This is sampled at random time instants. The sampled output is a random variable X.
Find E[X] and E[X2]
The sampling instant is a random variable T. Let be another rv.
tA cos
2,0~ U
q
P (q
1/2
20
Uniform Distribution
cosAX
Functions of a Random Variable
Example contd.
22
2
1cos][
02
1cos][
cos
22
2
0
222
2
0
AA
dAXE
dAXE
AX
q
q
q
q
Functions of a Random Variable
Variance
2__
2_
2__
2
)(Deviation Std.
)()( Variance
XXEX
dxxfXxXXEX XX
Average AC power of a signal
Functions of a Random Variable
Variance
22
2__2
2____2
2____2
2__
2
][][
][
][2][
2
XEXE
XXE
XEXXEXE
XXXXE
XXEX
Functions of a Random Variable
Moments
The nth moment of X is given by:
The nth central moment of X is given by:
dxxfxXXE Xnnn )(][
__
dxxfxxXXE X
n
n )(])[(___
Functions of two Random Variables
Correlation and Covariance Estimate the nature of dependence between two
random variables
Correlation
If X and Y are independent:
X and Y are said to be orthogonal if
dxdyyxxyfXYER XYXY ),(][
][][)()( YEXEdyyyfdxxxfR YXXY
0][ XYERXY X Y
Functions of two Random Variables
Correlation and Covariance
Covariance
dxdyyxfYyXxYYXXEC XYXYXY ),(
________
____
________
____
][][][
][][][
YXRYEXEXYE
YXXEYYEXXYE
YYXXEC
XY
XYXY
For Independent RVs
0
][][____
XY
XY
C
YXYEXER
Correlation can be Positive Zero or Negative
Functions of two Random Variables
Correlation and Covariance
Covariance
For Independent RVs
0
][][____
XY
XY
C
YXYEXER
Correlation can be Positive Zero or Negative
____
YXRC XYXY
X and Y are uncorrelated
Functions of two Random Variables
Correlation and Covariance
X and Y are positively correlated
X and Z are negatively correlated X and W are uncorrelated
Scatter Plots
Relationship between Independence and Correlatedness
Correlation Coefficient
Functions of two Random Variables
11
XY
YX
XYXY
C