II. Characterization of Random Variables. © Tallal Elshabrawy 2 Random Variable Characterizes a random experiment in terms of real numbers Discrete Random

II. Characterization of Random Variables

© Tallal Elshabrawy 2

Random Variable

Characterizes a random experiment in terms of real numbers

Discrete Random Variables The random variable can only take a finite number of

values

Continuous Random Variables The random variable can take a continuum of values


Probability Mass Function

Only Suitable to characterize discrete random variables

P X x


Cumulative Distribution Function

1 2 1 2

1 2 1 2 2 1

F 0 F 1 0 F 1

F is monotone non-decreasing function

For < F F

For < P =F F

X X X

X

X X

X X

x

x

x x x x

x x x X x x x

F P X x X x

Properties of FX x


Probability Density Function

Used to characterize Continuous Random Variables

2

1

1 2 2 1

1 2 2 1

1 2

df F

df 0

F f d f d 1

P P P

P F F

P f d

X X

X

x

X X X

X X

x

X

x

x xx

x

x y y x x

x X x X x X x

x X x x x

x X x x x


Uniform Random Variable

xf x

a b

a b

1

0

1f

0

0

F

1

X

X

x a

x a x bb a

x b

x a

x ax a x b

b ax b

fX x

FX x

1

b a


Gaussian Random VariableMany physical phenomenon can be modeled as Gaussian Random Variables most popular to communication engineers is … AWGN Channels

2

22 2

2

1~N , f

2

x m

XX m x e

mean

standard deviation

2

2

2

2

2

2

1F d 1 Q

2where

1Q d

2

y mx

X

t

x

x mx e y

x e t


Exponential Random Variable

Commonly encountered in the study of queuing systems

1f 0

x

bX x e x

b

F 1 0

x

bX x e x


How to Characterize a Distribution

Client: Tell me how good is your network?

Salesman: Well, P(Delay<1)=0.1, P(Delay<2)=0.3,P(Delay<3)=0.2,……

Client: HmmmSo what does this really mean?

Salesman: How can I explain this?


Mean of Random Variables

Client: Tell me how good is your network?

Salesman: Well, The average delay per packet is 1 sec

Client: HmmmSo what does this really mean?

Salesman: If you need to send 100 packets, they will most likely take 100 seconds


Mean of a Random Variable

E P X x X x

E f d

XX x x x

Discrete Random Variable

Continuous Random Variable


Consider a Network where the delay ‘D’ is either 1 or 5 secondsi.e., P[D = 1] = 0.3, P[D =5] = 0.7

P[D = 0, 2, 3, 4] = 0, P[D = 6, 7, 8, 9, …] = 0

What is the mean delay? Let assume 100 packets, then most likely

30 packets will be delayed for 1 sec

70 packets will be delayed for 5 sec Therefore 100 packets will most likely take 30x1+70x5 = 380 sec Average Delay = 380/100 = 3.8 sec E[D] = 1xP[D=1]+5xP[D=5] = 3.8 sec

Example


Moments of a Random Variable



E P n nX x X x

E f d

n nXX x x x


Central Moments

E E E P n n

X X x X X x

E E E f d

n n

XX X X X x x




Variance

Variance is a measure of random variable’s randomness around its mean value

2

22

Var =E E

Var =E E

X X X

X X X

Std = VarX X


Conditional CDF

Define FX|A[x] as the conditional cumulative distribution function of the random variable X conditioned on the occurrence of the event A, then

P ,F P

P

X A

X x Ax X x A

A

P ,P

P

A BB A

A

Remember Bayes’s Rule

1 2 1 2

1 2 1 2 2 1

F 0 F 1 0 F 1

F is monotone non-decreasing function

For < F F

For < P =F F

X A X A X A

X A

X A X A

X A X A

x

x

x x x x

x x x X x A x x

Properties of FX A x


Conditional CDF: Example

Consider a uniformly distributed random variable X with CDF

0 0

F 0 1

1 1

X

x

x x x

x

Calculate the conditional CDF of X given that X<1/2. In other words we would like to compute FX|X<1/2[x]

x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

0 P , 1 2 0

F 0 0

X X

x X x X

x x



0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

0 1 2 P , 1 2 P

PF 2 0 1 2

P 1 2

X X

x X x X X x

X xx x x

X




0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

P , 1 2F P 1 2

P 1 2

X X

X x Xx X x X

X

1 2

1 2 P , 1 2 P 1 2

P 1 2F 1 1 2

P 1 2

X X

x X x X X

Xx x

X




0 0

F 0 1

1 1

X

x

x x x

x


x

FX[x]

0 1

1

1 2

0 0

F 2 0 1 2

1 1 2

X X

x

x x x

x x

FX[x]

0 1/2

1



Exercise

For some random variable X and given constants a, b such that a<b

0

F FF

F F

1

X XX a X b

X X

x a

x ax a x b

b a

x b

P ,F P

P

X a X b

X x a X bx X x a X b

a X b


Conditional PDF

Define fX|A[x] as the conditional probability density function of the random variable X conditioned on the occurrence of the event A, then

df F

dX A X Ax x

x

2

1

1 2

f 0

F f d f d 1

P f d

X A

x

X A X A X A

x

X Ax

x

x y y x x

x X x A x x

Properties of fX A x


Conditional PDF: Example


0 0

f 1 0 1

0 1

X

x

x x

x

Calculate the conditional PDF of X given that X<1/2. In other words we would like to compute fX|X<1/2[x]

x

fX[x]

0 1

1

x

fX|X<1/2[x]

1

2

1 2

0 0

f 2 0 1 2

0 1 2

X X

x

x x

x

1/2


Exercise

For some random variable X and given constants a, b such that a<b

0

F FF

F F

1

X XX a X b

X X

x a

x ax a x b

b a

x b

df F

d X a X b X a X bx xx

0

f ff

F F P

0

X XX a X b

X X

x a

x xx a x b

b a a x b

x b


Conditioning on a Characteristic of Experiment

Conditioning does not necessarily have to be on the numerical outcome of an experiment

It is possible to have qualitative conditioning based on a characteristic of an experiment

Example: Consider a random variable X that represents the score of students in a given course Conditioning based on experiment outcome

The distribution of grades given it is greater than 80% (i.e., FX|X>80[x])

Conditioning based on experiment characteristic The distribution of grades given the gender of students (i.e., FX|M[x])



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then

1

1

1

F P P P

F P F P

f f P

n

n

N

X n nn

N

X nX An

N

X nX An

x X x X x A A

x X x x A

x x A

The unconditional CDF/PDF is basically the conditioned CDF averaged across the probability of occurrence of conditioning events

Example:

For a bit b sent over a communication channel and the received voltage r

P[r<0]=P[r<0|b=1]*P[b=1]+P[r<0|b=0]*P[b=0]



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then

P PP

P

n n

n

X x A AA X x

X xDiscrete Random Variable

For a continuous random variable P [X=x|An]=0, P [X=x]=0 resulting in an undetermined expression



Consider a set of N mutually exclusive events A1, A2,…, AN. Suppose we know FX|An[x] for n=1, 2, …, N. Then for a continuous random variable

P PP

P

0

P f P f

f P f PP

f f

0

f PP

f

n

n n

n

n nn

n XX A

n nX A X A

nX X

nX A

nX

x X x A AA x X x

x X x

x X x A x x X x x

x A x AA x X x

x x

x AA X x

x


Conditional Expected Value

The expected value of a random variable X conditioned on some event A

E P X AX A x x Discrete Random Variable

E f d

X AX A x x x Continuous Random Variable

Documents

II. Characterization of Random Variables. © Tallal Elshabrawy 2 Random Variable Characterizes a random experiment in terms of real numbers Discrete Random