7/27/2019 5 into to RV

1/6

ECE270: Handout 5

Introduction to Random Variables (RVs)

Outline:

1. informal definition of a RV,

2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV,

3. a general rule to find probability of events concerning a RV,

4. cumulative distribution function (CDF) of a RV,

5. formal definition of a RV using CDF,

6. discrete RV: probability mass function (pmf) and CDF,7. continuous RV: probability density function (pdf) and CDF,

8. basic properties of the CDF.

The outcome of a random experiment need not be a numbers. Examples are: coding theincoming patients in a hospital according to their insurance and health status, randomlyselecting a committee from a group of people, randomly selecting balls from an urn, randomlyselecting cards from a deck.

Usually we are interested in some measurement or numerical attribute of the outcome. Ex-amples are: counting the number of heads when tossing a coin 10 times, the number ofre-transmission needed until the receiver receives the data packet correctly, the number of er-rors in erroneous received data packets, the lifetime of a memory chip, the number of packetsarriving in t sec at a server, the number of queries arriving in t sec at a call center, the numberof particles emitted by a radioactive mass during a fixed time period, the random thermalnoise being added to the signal at the receiver of a communication system at a specific time.

In these examples, we assign a real number to the outcome of the random experiment throughmeasurement. Since the outcomes are random, the results of the measurements will be randomtoo. Hence it makes sense to talk about the probabilities of the resulting numerical values.

Informal Definition of a RV

A RV X is a function that assigns a real valued number x = X() to each outcome S(Recall: a function is a rule for assigning a numerical value to each element of a set).

Sample space S is the domain of the RV X and the set of all real numbers taken on by X,Sx, Sx R, is the range of the RV.

Three Types of a RV

Three types of RVs: i) discrete, ii) continuous, iii) mixed.

7/27/2019 5 into to RV

2/6

i) The range of a discrete RV X is a countable set (either finite or infinite) Sx ={x1, x2,...} or Sx = {x1, x2,...,xn}.

ii) The range of a continuous RV X is an uncountable set, e.g., Sx = [0, ), Sx =(, ), Sx = [0, 1], Sx = [a, b] for < a < b < .

iii) The range of a mixed RV X is the union of an uncountable and a countable sets.

Notation: capital letters X,Y,Z,V,U,... denote RVs, lowercase letters x,y,z,v,u,... de-note possible values of RVs.

Consider the RV X, the function or rule that assigns a real number x = X() to the outcome S is fixed and deterministic, e.g., the rule of count the number of heads when we toss acoin 3 times. The randomness in the experiment is complete as soon as we toss the coin 3times. The process of counting is deterministic.

In some random experiments the outcome is already the numerical value we are interested in,e.g., measure the lifetime of a chip under certain conditions, X() = (identity function).

The distribution of the values of a RV X is determined by the probabilities of the basic eventsof the underlying random experiment, i.e., we should be able to compute the probability ofthe observed value ofX in terms of the probability of the underlying event.

The specification of the measurements on the outcomes of a random experiment defines afunction on S and hence a RV.

A function of a RV is another RV.

EXAMPLE 1

A fair coin is tossed 3 times and the sequence of heads and tails is noted. The sample spaceis S = {H H H , H H T , H T H , H T T , T H H , T H T , T T H , T T T } and the outcomes are equallyprobable.

a) Let X be the number of heads in the 3 tosses. Sx = {0, 1, 2, 3}.

b) A player pays 1.5$ to play the following game: the player receives 1$ ifX = 2 and8$ ifX = 3, but nothing otherwise. Let Y be the reward to the player. Sy = {0, 1, 8}.

c) Let Z be a function ofX such that Z = 0 ifX {0, 1, 2} and Z = 1 ifX = 3.Sz = {0, 1}.

HHH HHT HTH THH HTT THT TTH TTT

X

() 3 2 2 2 1 1 1 0Y() 8 1 1 1 0 0 0 0Z() 1 0 0 0 0 0 0 0

P(X= 0) = P({T T T}) = 1/8

P(X= 1) = P({T T H , T H T , H T T }) = 3/8

P(X= 2) = P({T H H , H H T , H T H }) = 3/8

Page 2

7/27/2019 5 into to RV

3/6

P(X= 3) = P({HHH}) = 1/8

P(Y = 0) = P({T T T , T T H , T H T , H T T }) = 4/8 = 1/2

P(Y = 1) = P({THH,HTH,HHT}) = 3/8

P(Y = 8) = P({HHH}) = 1/8P(Z= 0) = P(X {0, 1, 2}) = P(X = 0) + P(X = 1) + P(X= 2) = 1/8 + 3/8 + 3/8 = 7/8P(Z= 1) = P(X = 3) = 1/8

or alternatively:P(Z= 0) = P({T T T , T T H , T H T , H T T , H H T , H T H , T H H }) = 7/8P(Z= 1) = P({HHH}) = 1/8

A General Rule to Find Probabilities of Events Concerning a RV X

The example shows a general technique to find the probability of events involving RVs. Tofind the probability ofX B (where B R) we need to find the set of outcomes A, A Sthat are mapped to B, i.e., the set A = { : X() B}.

If the experiment outcome A then event A occurs. Hence X() B event B occurs.If event B occurs then X() B implies A event A occurs. So: P(X B) = P(A) =P({ : X() B}). We refer to A and B as equivalent events.

In some random experiments the outcome is already the numerical value we are interestedin. In such cases we simply let X() = , i.e., the identity function is used to obtain a randomvariable. Example: measure the received signal at a receive antenna.

CDF of a RVX

Regardless ofX being discrete, continuous, or mixed, the cumulative distribution function(CDF) of a RV X is defined as the probability of the event B = {X x}:

FX(x) = P(B) = P({X x}) P(X x) for < x <

The event B = {X x} and its probability vary as x is varied. Hence, FX(x) is a functionof the variable x.

In terms of the underlying random experiment FX(x) = P({ : X() x}

an event).

Formal Definition of a RV Using CDF

We are ready for a formal definition of a RV: Consider a random experiment with samplespace S. The RV X is a function from S to real line R with the property that the setA = { : X() b} is an event for every b R. This definition simply requires that everyset A has a well defined probability in the underlying random experiment.

Page 3

7/27/2019 5 into to RV

4/6

Discrete RV: pmf and CDF

A discrete RV X has the CDF FX(x) that is right continuous, non-decreasing staircasefunction of x, with jumps at points x1, x2,... where xi Sx, and it grows from 0 to 1:

FX(x) = i=1

P(X = xi) pmf

u(x xi)

pX(x) = P(X = x) = P({ : X() = x}), for x a real number, is called the probability massfunction (pmf) ofX, and is the magnitude of the jump in CDF at the point x.

pX(x) is a function of x over the real line and is nonzero only at the points x1, x2,... Sx.

pmf pX(x) for all x Sx provides all information required to calculate probabilities of anyevent involving a discrete RV X. We can obtain CDF from pmf and vice versa.

We can forget about the underlying random experiment, its sample space S and its associated

probability law and just work with Sx and the pmf ofX

, or equivalently the CDF ofX

. The pmf ofX is the probability of all elementary events from Sx. The pmf pX(x) satisfies

three properties: 1) 0 pX(x) 1, 2)

xSxpX(x) = 1, 3) P(X B) =

xB pX(x) where

B R.

EXAMPLE 2 (CDF of a discrete RV, back to example 1)

Find and plot FX(x) in example 1.

Continuous RV: pdf and CDF

A continuous RV X has the CDF FX(x) that is continuous everywhere and is a non-decreasingfunction of x that grows from 0 to 1 as x ranges from its minimum to its maximum values.Also, FX(x) is smooth enough that it can be written as an integral of some nonnegativefunction fX(x):

FX(x) =

x

fX(x) pdf

dx

Considering FX(x) we calculate probabilities as integral of probability density function (pdf)over an interval of the real line (, x]. We can obtain CDF from pdf and vice versa.

For a continuous RV X we have P(X = x) = 0.EXAMPLE 3 (CDF of a continuous RV)

We spin an arrow attached to the center of a circular board. Let be the angle of the arrowwhere 0 < 2. The probability that falls in a subinterval of (0, 2] is equal to the lengthof the subinterval divided by 2. We define the RV X() = /2. What is FX(x)?

Page 4

7/27/2019 5 into to RV

5/6

EXAMPLE 4 (CDF of a mixed RV)

The waiting time X of a customer at a taxi stand is zero if the customer finds a taxi parkedat the stand, and a uniformly distributed random length of time in the interval [0, 1] (in hour)if no taxi is found upon arrival. The probability that a taxi is at the stand when the customerarrives is p. What is the CDF ofX?

SOLUTION: using the total probability theorem we have

FX(x) = P(X x) = P(X x|find taxi)p + P(X x|no taxi)(1 p)

P(X x|find taxi) =

1 x 00 x < 0

P(X x|no taxi) =

1 x > 1x 0 x 10 x < 0

FX(x) = P(X x) =

1 p + 1 (1 p) = 1 x > 11 p + x(1 p) = p + x(1 p) 0 x 1

0 x < 0

Basic Properties of the CDF

The three axioms of probability and the corresponding properties imply that the CDF hasthe following properties:

(i) 0 FX(x) 1

(ii) limx FX(x) = 1

(iii) limx FX(x) = 0

(iv) FX(x) is a nondecreasing function of x, that is, if a < b then FX(a) FX(b)

(v) FX(x) is continuous from the right, i.e., for h > 0, FX(b) = limh0 FX(b + h) =FX(b

+). This property implies that at points of discontinuity, the CDF is equal to thelimit from the right.

These five properties confirm that, in general, CDF is a nondecreasing function ofx thatgrows from 0 to 1 as x increases from to .

More properties follow:

(vi) P(a < X b) = FX(b) FX(a)

(vii) P(X = b) = FX(b) FX(b) = FX(b

+) FX(b). This property says that the

probability that X = b is equal to the magnitude of the jump of the CDF at the pointb. So if the CDF is continuous at a point b, then P(X= b) = 0.

(viii) P(X > x) = 1 FX(x)

Page 5

7/27/2019 5 into to RV

6/6

Based on above properties we can compute P(a X b), P(a X < b), P(a < X < b).Since {a X b} = {a} {a < X b}, we have:

P(a X b) = P(X = a) + P(a < X b)

= FX(a) FX(a) + FX(b) FX(a)

= FX(b) FX(a)

Since {a X b} = {b} {a X< b}, we have:

P(a X < b) = P(a X b) P(X = b) = FX(b) FX(a

)

Since {a X < b} = {a} {a < X < b}, we have:

P(a < X< b) = P(a X < b) P(X = a) = FX(b) FX(a)

NOTE: if the CDF is continuous at the endpoints of an interval, then the endpoints have zero

probabilities, and therefore, they can be included in, or excluded from, the interval withoutaffecting the probabilities.

EXAMPLE 5 (back to example 1)Consider example 1 and let A = {1 < X 2}, B = {0.5 X < 2.5}, C = {1 X < 2}. Find

P(A), P(B), P(C).

EXAMPLE 6 (back to example 3)

Find P(0.5