Lecture 6: Random Variable

Xuexin Wang

WISE

October 2012

1 / 24

Random Variables

A random variable is a number whose value depends upon theoutcome of a random experiment. Mathematically, a random variableX is a real-valued function on Ω, the space of outcomes:

X : Ω → R

In tossing dice, we are often interested in the sum of the two diceand are not really concerned about the separate values of each die

In flipping a coin, we may be interested in the total number ofheads that occur and not care at all about the actual head-tailsequence.

Choose a random point in the unit square {(x, y) : 0 ≤ x, y ≤ 1}and let X be its distance from the origin.

2 / 24

Random Variables: Example

EXAMPLE 1: Suppose that our experiment consists of tossing 3fair coins. If we let X denote the number of heads that appear,then X is a random variable taking on one of the values 0, 1, 2,and 3 with respective probabilities

P{X = 0} = P{(T, T, T )} =1

8

P{X = 1} = P{(T, T,H), (T,H, T ), (H,T, T )} =3

8

P{X = 2} = P{(T,H,H), (H,T,H), (H,H, T )} =3

8

P{X = 3} = P{(H,H,H)} =1

8

3∑

i=0

P{X = i} = 1.

3 / 24

Random Variables: Example

Example 2: Independent trials consisting of the flipping of a coinhaving probability p of coming up heads are continually performeduntil either a head occurs or a total of n flips is made. If we let Xdenote the number of times the coin is flipped, then X is a randomvariable taking on one of the values 1, 2, 3, ..., n with respectiveprobabilities

P{X = 1} = P{H = p

P{X = 2} = P{(T,H)} = (1− p)p

...

P{X = n− 1} = P{(T, T, · · · , T,H)} = (1− p)n−2p

P{X = n} = P{(T, T, · · · , T, T ), (T, T, · · · , T,H)} = (1− p)n−1

P{∪ni=1{X = i}} = 1

4 / 24

Cumulative Distribution Function

For a random variable X, the function F defined by

F (x) = P{X ≤ x}, −∞ < x < ∞

is called the cumulative distribution function (cdf), or, more simply,the distribution function,of X.

A note on notation:random variables will always be denoted withuppercase letters and the realized values of the variable will bedenoted by the corresponding lowcase letters.

Example 1 continued: The cdf of X is

F (x) =

0 if −∞ < x < 018 if 0 ≤ x < 112 if 1 ≤ x < 278 if 2 ≤ x < 31 if 3 ≤ x < ∞

5 / 24

Properties of CDF

1. F is a nondecreasing function; that is, if a < b, thenF (a) ≤ F (b).

2. limx→∞ F (x) = 1.

3. limx→−∞ F (x) = 0.

4. F is right continuous. That is, for any b and any decreasingsequence bn, n ≥ 1, that converges to b, limn→−∞ F (bn) = F (b).

5. P (a < X ≤ b) = F (b)− F (a).

6 / 24

Discrete Random Variables

A random variable that can take on at most a countable numberof possible values is said to be discrete.

Define probability mass function p(x) of X by

p(x) = P (X = x),

such that, if X must assume one of the values x1, x2, · · · , then

p(xi) ≥ 0, for i = 1, 2, · · ·

p(x) = 0, for all other values of x

∞∑

i=1

p(xi) = 1.

Example 3: The probability mass function of a random variableX is given by p(i) = cλi/i!, i = 0, 1, 2, · · · , where is some positivevalue. Find (a) P{X = 0} and (b) P{X > 2}.

7 / 24

Discrete Random Variables

Solution: since∑

∞

i=0 p(xi) = 1,

c

∞∑

i=0

λi

i!= 1

which, because eλ =∑

∞

i=0λi

i! , implies that ceλ = 1, or c = e−λ

Hence,

(a)

P (X = 0) =e−λ

λ0

0!= e−λ

(b)

P (X > 2) = 1− P (X ≤ 2)

= 1− P (X = 0)− P (X = 1)− P (X = 2)

= 1− e−λ − λe−λ −λ2e−λ

2

8 / 24

Expected Values

If X is a discrete random variable having a probability massfunction p(x), then the expectation, or the expected value, of X,denoted by E[X], is defined by

E[X] =∑

x:p(x)>0

xp(x)

Example 1 continued: Find E(X)

E(X) = 0×1

8+ 1×

3

8+ 2×

3

8+ 3×

1

8

=3

2

9 / 24

Expection of a Function of a Random Variable

Proposition

If X is a discrete random variable that takes on one of the values xi,i1, with respective probabilities p(xi), then, for any real-valuedfunction g,

E[g(X)] =∑

i

g(xi)p(xi).

Proof.by grouping together all the terms in

∑

i g(xi)p(xi) having the samevalue of g(xi). Specifically, suppose that yj , j1, represent the differentvalues of g(xi), i1. Then, grouping all the g(xi) having the same valuegives ∑

i

g(xi)p(xi) =∑

j

∑

i:g(xi)=yi

g(xi)p(xi)

=∑

j

∑

i:g(xi)=yi

yjp(xi)

=∑

j

yj∑

i:g(xi)=yi

p(xi)

=∑

j

yjP{g(X) = yj}10 / 24

Expection of a Function of a Random Variable

Corollary

If a and b are constants, then

E(aX + b) = aE(X) + b

E[X], is also referred to as the mean or the first moment of X

The quantity E[Xn], n ≥ 1, is called the nth moment of X.

11 / 24

Variance

DefinitionIf X is a random variable with mean µ, then the variance of X,denoted by V ar(X), is defined by

V ar(X) = E[(X − µ)2]

V ar(X) = E(X2)− [E(X)]2.

For any constants a and b,

V ar(aX + b) = a2V ar(X).

The square root of the V ar(X) is called the standard deviationof X, and we denote it by SD(X). That is,

SD(X) =√

V ar(X).

12 / 24

The Bernoulli and Binormial Random VariablesSuppose that a trial, or an experiment, whose outcome can beclassified as either a success or a failure is performed. If we letX = 1 when the outcome is a success and X = 0 when it is afailure, then the probability mass function of X is given by

p(0) = P{X = 0} = 1− p

p(1) = P{X = 1} = p,

where where p, 0 ≤ p ≤ 1, is the probability that the trial is asuccess. The random variable X is said to be a Bernoulli randomvariable.Suppose now that n independent trials, each of which results in asuccess with probability p and in a failure with probability 1− p,are to be performed. If X represents the number of successesthat occur in the n trials, then X is said to be a binomial randomvariable with parameters (n, p), such that the probability massfunction is

p(i) =

(

ni

)

pi(1− p)n−i, i = 0, 1, · · · , n

Note∑

∞

i=0 p(i) = 113 / 24

The Bernoulli and Binormial Random Variables:

ExamplesExample: It is known that screws produced by a certain company willbe defective with probability .01, independently of each other. Thecompany sells the screws in packages of 10 and offers a money-backguarantee that at most 1 of the 10 screws is defective. Whatproportion of packages sold must the company replace?

Solution: If X is the number of defective screws in a package,then X is a binomial random variable with parameters (10, .01).Hence, the probability that a package will have to be replaced is

1− P (X = 0)− P (X = 1) = 0.004.

Example: The following gambling game, known as the wheel offortune (or chuck-a-luck), is quite popular at many carnivals andgambling casinos: A player bets on one of the numbers 1 through 6.Three dice are then rolled, and if the number bet by the playerappears i times, i = 1, 2, 3, then the player wins i units; if the numberbet by the player does not appear on any of the dice, then the playerloses 1 unit. Is this game fair to the player?

14 / 24

The Bernoulli and Binormial Random Variables:

ExamplesSolution: If we assume that the dice are fair and actindependently of each other, then the number of times that thenumber bet appears is a binomial random variable withparameters (3, 16 ).

P (X = −1) =

(

30

)(

1

6

)0 (5

6

)3

=125

216

P (X = 1) =

(

31

)(

1

6

)1 (5

6

)2

=75

216

P (X = 2) =

(

32

)(

1

6

)2 (5

6

)1

=15

216

P (X = 3) =

(

33

)(

1

6

)3 (5

6

)0

=1

216

In order to determine whether or not this is a fair game for theplayer, let us calculate E[X]

E[X] =−125 + 75 + 30 + 3

216=

−17

216 15 / 24

Properties of The Bernoulli and Binormial

Random Variables

E(Xk) =n∑

i=0

ik(

ni

)

pi(1− p)n−i

=

n∑

i=1

ik(

ni

)

pi(1− p)n−i

i

(

ni

)

= n

(

n− 1i− 1

)

E(Xk) = np

n∑

i=0

ik−1(

n− 1i− 1

)

pi−1(1− p)n−i

= npn−1∑

j=0

(j + 1)k−1(

n− 1j

)

pj(1− p)n−1−j

= npE[(Y + 1)k−1]

where Y is Binomial(n− 1, p).16 / 24

Properties of The Bernoulli and Binormial

Random Variables

E[X] = np.

E(X2) = npE[Y + 1] = np((n− 1)p+ 1)

V ar(X) = E(X2)− (E[X])2 = np((n− 1)p+ 1)− (np)2 = np(1− p).

Proposition

If X is a binomial random variable with parameters (n, p), where0 < p < 1, then as k goes from 0 to n, PX = k first increasesmonotonically and then decreases monotonically, reaching its largestvalue when k is the largest integer less than or equal to (n+ 1)p.

P{X = k + 1} =p

1− p

n− k

k + 1P{X = k}

17 / 24

The Poisson Random Variables

A random variable X that takes on one of the values 0, 1, 2, · · · is saidto be a Poisson random variable with parameter λ if, for some λ > 0,

p(i) = P{X = i} = e−λλi

i!, i = 0, 1, 2, · · ·

Remark: The Poisson probability distribution was introduced bySiméon Denis Poisson.

Remark: It may be used as an approximation for a binomialrandom variable with parameters (n, p) when n is large and p issmall enough.

18 / 24

The Poisson Random Variables

X Binomial(n, p), λ = np

P{X = i} =n!

(n− i)!i!pi(1− p)n−i

=n!

(n− i)!i!(λ

n)i(1−

λ

n)n−i

=n(n− 1) · · · (n− i+ 1)

niλi

i!

(1− λn)n

(1− λn)i

Now, for n large and λ moderate,

(1−λ

n)n ≈ e−λ,

n(n− 1) · · · (n− i+ 1)

ni≈ 1, (1−

λ

n)i ≈ 1

So,

P{X = i} ≈ e−λλi

i!.

19 / 24

Properties of The Poisson Random Variables

E[X] =

∞∑

i=0

ie−λλi

i!

= λ∞∑

i=1

e−λλi−1

(i− 1)!

= λ

E[X2] =

∞∑

i=0

i2e−λλi

i!

= λ

∞∑

i=1

ie−λλi−1

(i− 1)!

= λ

∞∑

j=0

(j + 1)e−λλj

j!by letting j = i− 1

= λ(λ+ 1).

V ar(X) = E[X2]− (E[X])2 = λ

20 / 24

The Geometric Random Variable

Suppose that independent trials, each having a probability p,0 < p < 1, of being a success, are performed until a success occurs. Ifwe let X equal the number of trials required, then

P{X = n} = (1− p)n−1p, n = 1, 2, · · ·

Any random variable X whose probability mass function is given byEquation above is said to be a geometric random variable withparameter p.

E[X] =1

p

E[X2] =2− p

p2

V ar(X) =1− p

p2

21 / 24

The Negative Binomial Random Variable

Suppose that independent trials, each having probability p, 0 < p < 1,of being a success are performed until a total of r successes isaccumulated. If we let X equal the number of trials required, then

P{X = n} =

(

n− 1r − 1

)

pr(1− p)n−r, n = r, r + 1, · · ·

Any random variable X whose probability mass function is given byEquation above is said to be a negative binomial random variablewith parameters (r, p).

E[X] =r

p

E[X2] =r

p(r + 1

p− 1)

V ar(X) =r(1− p)

p2.

22 / 24

Expected Value of Sums of Random Variables

For a random variable X, let X(ω) denote the value of X when ω ∈ Ωis the outcome of the experiment. Now, if X and Y are both randomvariables, then so is their sum. That is, Z = X + Y is also a randomvariable. Moreover, Z(ω) = X(ω) + Y (ω).

Suppose that the experiment consists of flipping a coin 5 times,with the outcome being the resulting sequence of heads and tails.Suppose X is the number of heads in the first 3 flips and Y is thenumber of heads in the final 2 flips. Let Z = X + Y . Then, forinstance, for the outcome ω = (h, t, h, t, h),

X(ω) = 2

Y (ω) = 1

Z(ω) = X(ω) + Y (ω) = 3

We prove results under the assumption that the set of possible valuesof the probability experiment-that is, the sample space Ω is eitherfinite or countably infinite.

23 / 24

Expected Value of Sums of Random Variables

Let p(ω) = P (ω) be the probability that ω is the outcome of theexperiment.

Proposition

E[X]=∑

ω∈Ω p(ω)X(ω)

Corollary

For random variable X1, X2, · · · , Xn

E

[

n∑

i=1

Xi

]

=n∑

i=1

E[Xi]

24 / 24