1 Variance of RVs Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1

Variance of RVs

Supplementary Notes

Prepared by Raymond WongPresented by Raymond Wong

2

e.g.1 (Page 3)

The distribution function D of a random variable X with finitely many values is the function on the values of X defined by D(x) = P(X=x)

The distribution function of the random variable X assigns each value x of the random variable the probability that X achieves that value.

3

Visualize the distribution function using a diagram called a histogram. Graphs that show, for each integer

value x of X, a rectangle of width 1 centered at x, whose height (and thus area) is proportional to the probability P(X=x)

4

X = 0 X = 1 X = 2 X = 3Sample Space

Let X be a random variable denoting a number equal to 0, 1, 2, or 3.The sample space where we consider random variable X is

1/8 3/8 3/8 1/8

x

P(X = x)

The histogram is

X

P(X = x)

0 1 2 3

1/8

2/8

3/8

5

Expected Value Consider that we flip a coin n times. We flip the coin 100 times. The expected number of heads is 50.

To what extend do we expect to see 50 heads? Is it surprising to see 55, 60 or 65 heads instead?

General Question: How much do we expect a random variable to deviate from its expected value.

6

10 flipsArea of rectangles with bases from x = a to x = b is probability that X is between a and b

Cumulative distribution function D: D(a, b) = P(a X b)

e.g.,D(2, 3) = P(2 X 3)

= P(X = 2) + P(X = 3)

7

10 flips 25 flips

12.55 205

~7~7

We observe that the results are not spreadas broadly (relatively speaking) for 25 flips(compared with 10 flips)

Virtually, all results lie between 5 and 20.

8

25 flips

12.55 20

~7~7

100 flips

50 65

~15~15

35

We observe that the spread has only doubledeven though the number of trials has quadrupled.

= 4 x 25

2 x 7

9

100 flips 400 flips

50 65

~15~15

35 200 230

~30~30

170


= 4 x 100

= 2 x 30

10

400 flips

200 230

~30~30

170

The curve is quite similar to the bell-shaped curve, called normal curve.

We want to study the “spread”mathematically.

11

We have seen the scenario of flipping a fair coin n times.

Consider another scenario that a student answers n questions in an exam where he can answer a question correctly with probability 0.8.

12

10 question 25 questions

2014 25

~5~5

8

We observe that the results are not spreadas broadly (relatively speaking) for 25 questions(compared with 10 questions)

Virtually, all results lie between 14 and 25.

13

25 questions

2014 25

~5~5

100 questions

80 91

~11~11

69


= 4 x 25

2 x 5

14

100 questions 400 questions

80 91

~11~11

69 320 342

~22~22

298


= 4 x 100

= 2 x 22

15

400 questions

320 342

~22~22

298

The curve is quite similar to the bell-shaped curve, called normal curve.

We want to study the “spread”mathematically.

However, the curves might showasymmetry.

10 question

16

e.g.2 (Page 12)

Illustration of Lemma 5.26Lemma 5.26If X is a random variable that always takes on the value 5, then E(X) = 5

X = 5Sample Space

P(X=5)=1

Lemma 5.26If X is a random variable that always takes on the value c, then E(X) = c

Why is it correct?

17

e.g.2

Lemma 5.26If X is a random variable that always takes on the value c, then E(X) = c

Why is it correct?

E(X)

X = 5Sample Space

P(X=5)=1

= c P(X = c)

= c . 1

= c

18

e.g.3 (Page 12)

Corollary 5.27Let X be a random variable on a sample space.Then E(E(X)) = E(X).

Why is it correct?Note that E(X) is a value (or a constant).

e.g., expected number of heads when flipping coins

Let E(X) be where is a value.

Consider E(E(X))= E()

= (By Lemma 5.26)

= E(X)

19

e.g.4 (Page 13) We would like to have some way of

measuring the deviation of X from E(X). That is, how does Y = X – E(X) behave? Our first attempt is to look at the

expectation of Y.

Consider E(X – E(X)) = E(X) – E(E(X))= E(X) – E(X)= 0

Thus, E(Y) is identically zero, and is not a useful measure of how close a random variable is to its expectation.

How about E(Y2)?

20

e.g.5 (Page 14) Variance

Let X be a random variable. We define the variance of X denoted by V(X) to

be the expected value of (X – E(X))2

(i.e., E( (X-E(X))2 )

)))((()( 2XEXEXV

XSx

XExxXP 2))()((

That is,

Sx

XEsXsP 2))()()((

The sample space for random variable X

The sample space for the original application

21

e.g.5 Variance of a Random Variable X

a. V(X) = E( ( X – E(X) )2 )

x1 x3

x2 …xnSample Space

s1 s3

s2 …snSample Space

Based on outcome

Based on X

Number (e.g., 2)

Outcome (e.g., HTH)

X(s3) (e.g., X(HTH) = 2)

n

iii xXPXExXV

1

2 )())(()(

n

iii sPXEsXXV

1

2 )())()(()(

b.

c.

22

TTTT TTTH TTHT THTT

HTTT HTTH HTHT HHTTSample Space

TTHH THTH THHT THHH

HTHH HHTH HHHT HHHH

e.g.6 (Page 14) Suppose that we want to flipping 4 coins. Let X be the random variable denoting the number of heads.

The sample space of flipping 4 coins is

4 tails

1/16

3 tails1/16

3 tails

3 tails

3 tails

2 tails

2 tails

2 tails

1 tail

2 tails 2 tails

2 tails

1 tail 1 tail 1 tail 0 tail1/16

1/16 1/16 1/16 1/16 1/16 1/16

1/16 1/16 1/16 1/16 1/16 1/16 1/16

We are interested in the sample space where each blue point is in the format of “X = ?”

X: random variable denoting the number of heads when we flip 4 coins.

(a) What is E(X)?(b) What is V(X)?

(a)What is E(X)?(b) What is V(X)?

23

TTTT TTTH TTHT THTT


TTHH THTH THHT THHH

HTHH HHTH HHHT HHHH

e.g.6

4 tails

1/16

3 tails1/16

3 tails

3 tails

3 tails

2 tails

2 tails

2 tails

1 tail

2 tails 2 tails

2 tails


1/16 1/16 1/16 1/16 1/16 1/16

1/16 1/16 1/16 1/16 1/16 1/16 1/16





24X = 0 X = 1 X = 2 X = 3Sample Space X = 4

TTTT TTTH TTHT THTT


TTHH THTH THHT THHH

HTHH HHTH HHHT HHHH

e.g.6

4 tails

1/16

3 tails1/16

3 tails

3 tails

3 tails

2 tails

2 tails

2 tails

1 tail

2 tails 2 tails

2 tails


1/16 1/16 1/16 1/16 1/16 1/16

1/16 1/16 1/16 1/16 1/16 1/16 1/16


The sample space where we are interested in X is

1/16 4/16 6/16 4/16 1/16

(or ¼) (or 3/8) (or ¼)



25X = 0 X = 1 X = 2 X = 3Sample Space X = 4

e.g.6


1/16 4/16 6/16 4/16 1/16

(or ¼) (or 3/8) (or ¼)



26

X = 0 X = 1 X = 2 X = 3Sample Space X = 4

e.g.6 The sample space where we are interested in X is

1/16 4/16 6/16 4/16 1/16

(or ¼) (or 3/8) (or ¼)


(a) E(X) = 0x1/16 + 1x1/4 + 2x3/8 + 3x1/4 + 4x1/16= 2

(b) V(X) = E((X-E(X))2)= E((X-2)2)

= (0-2)2 x 1/16 + (1-2)2 x ¼ + (2-2)2 x 3/8 + (3-2)2 x ¼ + (4-2)2 x 1/16

= 1



V(Y) = 1 where Y:random variable denoting the number of heads when we flip 4 coins

27

e.g.7 (Page 15) Calculating variances from scratch is

very time-consuming. Many random variables X such as the

binomial distribution can actually be built as the sum of simpler random variables (e.g., X = X1 + X2 + X3).

We know that E(X) = E(X1) + E(X2) + E(E3)

Do you think that V(X) = V(X1) + V(X2) + V(X3)?

We will answer this question in some slides later?


28

T HSample Space

e.g.8 (Page 16) Suppose that we want to flip ONE coin. Let X be the random variable denoting the number of heads.

The sample space of flipping ONE coin is0 head

1/2

1 head

1/2


X: random variable denoting the number of heads when we flip ONE coin.




29

T HSample Space

e.g.8

The sample space of flipping ONE coin is0 head

1/2

1 head

1/2





30

T HSample Space

e.g.8 The sample space of flipping ONE coin is

0 head

1/2

1 head

1/2


X = 0 X = 1Sample Space 1/2 1/2




31

e.g.8






32

e.g.8 The sample space where we are interested in X is



(a) E(X) = 0x1/2 + 1x1/2= 1/2

(b)V(X) = E((X-E(X))2)= E((X-1/2)2)

= (0-1/2)2 x 1/2 + (1-1/2)2 x 1/2

= 1/4



Consider we flip FOUR coins

Let Xi be the random variabledenoting the number of headsfor the i-th flip

Y = X1 + X2 + X3 + X4

From this example, we observe that

V(Y)=V(X1)+V(X2)+V(X3)+V(X4)(i.e., V(Y)=4V(X))


33

RRRRR RWRRRSample Space

RRWRR …RRRWR

e.g.9 (Page 17) Suppose that a student answers 5 questions in an exam. Suppose that he can answer a question correctly with probability =

0.8. Let X be the random variable denoting the number of questions he

answers correctly.

The sample space of answering 5 questions is

5 correct

0.85

4 correct0.840.2


X: random variable denoting the number of questions he answers correctly for 5 Qs.

4 correct0.840.2

4 correct0.840.2



34


RRWRR …RRRWR

e.g.9

The sample space of answering 5 questions is

5 correct

0.85

4 correct0.840.2


4 correct0.840.2

4 correct0.840.2



35

X = 0 X = 1Sample Space

X = 2

X = 3 X = 4 X = 5


RRWRR …RRRWR

e.g.9 The sample space of answering 5 questions is

5 correct

0.85

4 correct0.840.2

4 correct0.840.2

4 correct0.840.2



Note that answering 5 questions is a Bernoulli trial process with success probability = 0.8

55

0.85 . 0.20

54

0.84 . 0.21

53

0.83 . 0.22

52

0.82 . 0.23 51

0.81 . 0.24 50

0.80 . 0.25


36


X = 2

X = 3 X = 4 X = 5

e.g.9



55

0.85 . 0.20

54

0.84 . 0.21

53

0.83 . 0.22

52

0.82 . 0.23 51

0.81 . 0.24 50

0.80 . 0.250.00032

0.0064

0.0512

0.2048

0.4096

0.32768


37


X = 2

X = 3 X = 4 X = 5

e.g.9



50

0.85 . 0.20

54

0.84 . 0.21

53

0.83 . 0.22

52

0.82 . 0.23 51

0.81 . 0.24 50

0.80 . 0.250.00032

0.0064

0.0512

0.2048

0.4096

0.32768



(a)E(X) = 0.00032 x 0 + 0.0064 x 1 + 0.0512 x 2 + 0.2048 x 3 + 0.4096 x 4 + 0.32768 x 5

= 4

(b)V(X) = (0 – 4)2x0.00032 + (1 – 4)2x0.0064 + (2 – 4)2x0.0512 + (3 – 4)2x0.2048 + (4 – 4)2x0.4096 + (5 – 4)2x0.32768

ORWe know that this is a Bernoulli trial process with 5 trials and success prob. = 0.8

E(X) = 5 x 0.8 = 4

= 0.8

V(X) = 0.8

38

e.g.9


V(X) = 0.8

Xi: random variable denoting the number of questions he answers correctly for the i-th question.

(a) What is E(Xi)?(b) What is V(Xi)?

W RSample Space

The sample space of answering the i-th question is0

question0.2

1 question

0.8


X = 0 X = 1Sample Space 0.2 0.8

(a) E(Xi) = 0.2 x 0 + 0.8 x 1= 0.8OR

We know that this is a Bernoulli trial process with 1 trial and success prob. = 0.8

E(Xi) = 1x0.8 = 0.8 (b) V(Xi) = (0-0.8)2x0.2 +

(1-0.8)2x0.8= 0.16

V(Xi) = 0.16

From this example, we observe that V(X)=V(X1)+V(X2)+V(X3)+V(X4)+V(X5)(i.e., V(X)=5V(Xi))

39

e.g.10 (Page 18) Suppose that the bag contains two

coins (a $1 coin and a $5 coin) (a) Suppose that we withdraw one

coin from the bag.Let X1 be the random variable denoting the amount of money we obtain. (i) What is E(X1)? (ii) What is V(X1)?

Bag:{$1, $5}X1: random variable denoting the amount of money we obtain

(a) (i) What is E(X1)? (ii) What is V(X1)?

40

e.g.10

$1 $5Sample Space

1

0.5


X = 1 X = 5Sample Space 0.5 0.5

The sample space of withdrawing a coin is5

0.5

(a)(i)E(Xi) = 0.5 x 1 + 0.5 x 5= 3

(ii) V(Xi) = (1-3)2x0.5 + (5-3)2x0.5

= 4


(a) (i) What is E(X1)? (ii) What is V(X1)?

E(X1) = 3V(X1) = 4

41

e.g.10


E(X1) = 3V(X1) = 4

(b) Suppose that we withdraw two coins from the bag (one after the other), without replacement.

Let X1 be the random variable denoting the amount of money we obtain for the 1st draw.Let X2 be the random variable denoting the amount of money we obtain for the 2nd draw.

Let X be the random variable denoting the amount of money we obtain after these two draws. (NOTE: X = X1 + X2)

(i) What is E(X1)? (ii) What is V(X1)? (iii) What is E(X2)? (iv) What is V(X2)? (v) What is E(X)? (vi) What is V(X)?

X1: random variable denoting the amount of money we obtain for the first draw

X2: random variable denoting the amount of money we obtain for the second draw

X: random variable denoting the amount of money we obtain after these two draws (NOTE: X = X1 + X2)

(i) What is E(X1)?(ii) What is V(X1)?(iii) What is E(X2)?(iv) What is V(X2)?(v) What is E(X)?(vi) What is V(X)?

42

e.g.10


E(X1) = 3V(X1) = 4

X1: random variable denoting the amount of money we obtain for the first draw

X2: random variable denoting the amount of money we obtain for the second draw

X: random variable denoting the amount of money we obtain after these two draws (NOTE: X = X1 + X2)

(i) What is E(X1)?(ii) What is V(X1)?(iii) What is E(X2)?(iv) What is V(X2)?(v) What is E(X)?(vi) What is V(X)?

(i) E(X1) = 3

(ii) V(X1) = 4

(iii) E(X2) = 3

(iv) V(X2) = 4

(v)There are two cases.

Case 1: $1 $5

Case 2: $5 $1

X = 1 + 5 = 6 X = 5 + 1 = 6

P(Case 1) = 0.5 P(Case 2) = 0.5 E(X) = 6x0.5 + 6x0.5 = 6

(vi) V(X) = (6-6)2x0.5 + (6-6)2x0.5 = 0

From this example, we observe that V(X)V(X1)+V(X2)(i.e., V(X) 2V(Xi))

43

e.g.11 (Page 20)

We know that E(X1+X2) = E(X1) + E(X2)

In previous examples, we observe that In some cases,

V(X1+X2) = V(X1) + V(X2) In other cases,

V(X1+X2) V(X1) + V(X2)

X1 and X2 are “independent”.

X1 and X2 are not “independent”.

44

e.g.12 (Page 21) Let X be a random variable. Let Y be a random variable. X and Y are independent when “X

has value x” is independent of “Y has value y”, regardless of choice of x and y.

Formally, X and Y are independent if and only if for all values x, y, P((X=x)(Y=y)) = P(X = x).P(Y=y)

45

e.g.13 (Page 21)

E.g., Suppose that we roll two dice. X is the random variable denoting

the amount rolled on the first dice. Y is the random variable denoting

the amount rolled on the second dice.

X and Y are independent because for every 1 i, j 6, P((X=i)(Y=j)) = P(X = i).P(Y=j)

46

e.g.14 (Page 22) We want to show that:

Let X1 be a random variable Let X2 be a random variable. If X1 and X2 are independent, then

V(X1+X2) = V(X1) + V(X2)

Before we show the above statement, we need to illustrate some other statements.

47

e.g.14

Lemma 5.28If X and Y are independent random variables on sample space S withvalues x1, x2, …, xk and y1, y2, …, ym, respectively, then E(XY) = E(X)E(Y)

Why is it correct?

The answer can be found in the appendix.

48

e.g.22 (Page 25) Suppose that we flip two fair coins. We have two random variables X and Y.

X = 1 if head for coin 1

0 if tail for coin 1Y =

0 if head for coin 2

1 if tail for coin 2 (a) What is E(X)? (b) What is E(Y)? (c) What is E(XY)? (d) Is “E(XY) = E(X)E(Y)”?

(a)E(X) = 1x1/2 + 0x1/2 = 1/2(b)E(Y) = 0x1/2 + 1x1/2 = 1/2(c)There are 4 cases.

Case 1: HHCase 2: HTCase 3: TH

Case 4: TT

X = 1 Y = 0 XY = 0X = 1 Y = 1 XY = 1X = 0 Y = 0 XY = 0X = 0 Y = 1 XY = 0

1/41/41/41/4

E(XY) = 0x1/4 + 1x1/4 + 0x1/4 + 0x1/4

= 1/4(d) Yes. E(X)E(Y) =1/4

49

e.g.22 Suppose that we flip two fair coins. We have two random variables X and Y.

X = 1 if head for coin 1

0 if tail for coin 1Y =

0 if head for coin 2

1 if tail for coin 2 (a) What is E(X)? (b) What is E(Y)? (c) What is E(XY)? (d) Is “E(XY) = E(X)E(Y)”?

Conclusion: If X and Y are independent, E(XY) = E(X)E(Y)

50

e.g.23 (Page 25) Suppose that we flip one fair coin. We have two random variables X and Z.

X = 1 if head

0 if tailZ = 1- X

(a) What is E(X)? (b) What is E(Z)? (c) What is E(XZ)? (d) Is “E(XZ) = E(X)E(Z)”?

(a)E(X) = 1x1/2 + 0x1/2 = 1/2(b)

E(Z) = 0x1/2 + 1x1/2 = 1/2

(c)There are 2 cases.Case 1: HCase 2: T

X = 1 Z = 0 XZ = 0X = 0 Z = 1 XZ = 0

1/21/2

E(XZ) = 0x1/2 + 0x1/2

(d) No. E(X)E(Z) =1/4 E(XZ) = 0 Thus, E(X)E(Z) E(XZ)

Z = 0 if X = 1 (head)

1 if X = 0 (tail)

= 0

51

e.g.23 Suppose that we flip one fair coin. We have two random variables X and Z.

X = 1 if head

0 if tailZ = 1- X

(a) What is E(X)? (b) What is E(Z)? (c) What is E(XZ)? (d) Is “E(XZ) = E(X)E(Z)”?

Conclusion: If X and Z are NOT independent, E(XZ) E(X)E(Z)

52

e.g.24 (Page 26)Theorem 5.29If X and Y are independent random variables, then V(X+Y) = V(X) + V(Y)

Why is it correct?V(X+Y)

V(X) = E( (X – E(X))2 )

= E( ( (X+Y) – [E(X+Y)] )2 )

= E( ( (X+Y) – [E(X)+E(Y)] )2 ) (By Linearity of Expectation)

= E( ( X+Y – E(X) – E(Y) )2 )= E( ( X – E(X) + Y – E(Y) )2 )= E( ( [X – E(X)] + [Y – E(Y)] )2 )

= E( [X – E(X)]2 + 2 [X – E(X)] [Y – E(Y)] + [Y – E(Y)]2 )

= E( ( (X+Y) – E(X+Y) )2 )

= E( [X – E(X)]2 ) + E(2 [X – E(X)] [Y – E(Y)]) + E( [Y – E(Y)]2 )(By Linearity of Expectation)

= E( [X – E(X)]2 ) + 2E([X – E(X)] [Y – E(Y)]) + E( [Y – E(Y)]2 )= V(X) + 2E([X – E(X)] [Y – E(Y)]) + V(Y)= V(X) + V(Y) + 2E([X – E(X)] [Y – E(Y)])

If I can prove that “E([X – E(X)] [Y – E(Y)]) = 0”, then “V(X+Y) = V(X) + V(Y)”In the following, we will prove “E([X – E(X)] [Y – E(Y)]) = 0”

53

e.g.24

In the following, we will prove “E([X – E(X)] [Y – E(Y)]) = 0”Consider E([X – E(X)] [Y – E(Y)])

= E( XY – X.E(Y) – E(X).Y + E(X)E(Y) )

= E(XY) – E(X.E(Y)) – E(E(X).Y) + E( E(X)E(Y) )

(By Linearity of Expectation)

= E(XY) – E(Y).E(X) – E(X).E(Y) + E( E(X)E(Y) )

If c is a constant and X is a random variable, E(X . c) = cE(X)

= E(XY) – E(Y).E(X) – E(X).E(Y) + E(X)E(Y)

If c1 and c2 are constant, E(c1 . c2) = c1

.c2

= E(X)E(Y) – E(Y).E(X) – E(X).E(Y) + E(X)E(Y)

(By Lemma 5.28 (i.e., E(XY) = E(X)E(Y)) since X and Y are independent)

= 0

Done!

From the previous slide, V(X+Y) = V(X) + V(Y) + 2E([X – E(X)] [Y – E(Y)])

Thus, V(X+Y) = V(X) + V(Y)

54

e.g.25 (Page 29) Suppose that we flip one fair coin. We have a random variable X.

X = 1 if head

0 if tail From some examples we illustrated before, we know that

V(X) = 1/4 Suppose that we flip one fair coin 10 times. We have random variables X1, X2, …, X10.

Xi = 1 if head for the i-th flip

0 if tail for the i-th flip (a) According to Theorem 5.29, what is V(X1+ X2 + … + X10)? (b) According to Theorem 5.29, what is V(X1+ X2 + … + X100)? (c) According to Theorem 5.29, what is V(X1+ X2 + … + X400)?

(a)V(X1+ X2 + … + X10) = V(X1) + V(X2) + … + V(X10)= ¼.10 = 5/2(b)V(X1+ X2 + … + X100) = V(X1) + V(X2) + … + V(X100)= ¼.100= 25

(c)V(X1+ X2 + … + X400) = V(X1) + V(X2) + … + V(X400)= ¼.400= 100

55

e.g.25

(a)V(X1+ X2 + … + X10) = V(X1) + V(X2) + … + V(X10)= ¼.10 = 5/2(b)V(X1+ X2 + … + X100) = V(X1) + V(X2) + … + V(X100)= ¼.100= 25

(c)V(X1+ X2 + … + X400) = V(X1) + V(X2) + … + V(X400)= ¼.400= 100

Conclusion: Flipping the coins 10 times variance = 5/2 Flipping the coins 100 times variance = 25 Flipping the coins 400 times variance = 100

56

e.g.26 (Page 30)Theorem 5.xIn a Bernoulli trials process with n trials in which each experiment has twooutcomes and probability p of success, the Variance of the outcome is np(1-p)

Why is it correct?

Xi = 1 if success for the i-th trial

0 if fail for the i-th trial

p

1-p

E(Xi) V(Xi) = (1-E(Xi))2xp + (0-E(Xi))2x(1-p)

= 1xp + 0x(1-p) = p

Xi = 1 Xi = 0Sample Space p 1-p

= (1-p)2xp + (0-p)2x(1-p)= (1-p)2xp + p2x(1-p)= (1-p)x(1-p)xp + pxpx(1-p)= (1-p)xpx((1-p) + p)= (1-p)xp x 1= (1-p)xp

By Theorem 5.29, we know V(X) = V(X1+X2+…+Xn)

X: number of successes (i.e., X = X1 + X2 + … + Xn)

=V(X1)+V(X2)+…+V(Xn)=(1-p)xp + (1-p)xp + …+ (1-p)xp

=n(1-p)xp=np(1-p) Done!

57

e.g.27 (Page 31) Let X be a random variable.

Standard derivation of X, denoted by (X), is defined to be )(XV

Sometimes, we write only.

58

e.g.28 (Page 32)

Conclusion: Flipping the coins 100 times variance = 25 Flipping the coins 400 times variance = 100

525 10100

100 flips 400 flips

15 15 30 30

3 3 3 3

This means that “most” data can be found with 3 standard deviations from the expected value.

59

e.g.28

Conclusion: Flipping the coins 100 times variance = 25 Flipping the coins 400 times variance = 100

525 10100

This means that “most” data can be found with 3 standard deviations from the expected value.

How about flipping the coins 25 times?

Since flipping the coin 25 times (with success prob. = 0.5) is a Bernoulli trial process, we know that the variance is np(1-p)

= 25x1/2x(1-1/2)

= 25/42/54/25

3 3(i.e., 15/2) (i.e., 15/2)

We know that “most” data can be found with 15/2 from the expected value.

60

e.g.29 (Page 33)

Central Limit Theorem

Within :

About 68% data can be found with 1 standard deviation from the expected value.

Within 2:

About 95.5% data can be found with 2 standard deviations from the expected value.

2 2

P(X = x)

P(X = x)

P(X = x)

X

X

61

e.g.29

Central Limit TheoremWithin 3:


3 3

P(X = x)

X

62

e.g.29 Central Limit Theorem

a b

P(a X b) =

P(X = x)

X

b

a

x

dxe 2

2

2

1

This is called a normal distribution.

63

e.g.30 (Page 36) Suppose that we want to be

95% sure that the number of heads in n coin flips is within 1% of the expected value, how big does n have to be?

X:number of heads in n coin flips

1% ofE(X)

P(X = x)

X1% ofE(X)

E(X)

This is a Bernoulli trial process with n trials and 0.5 success probability

Variance = np(1-p)= n.1/2.(1-1/2)= n.1/2.1/2= n.1/4= n/4

2/4/ nn

E(X) = np = n.1/2 = n/2

Within 2:

2 2

P(X = x)

X

By Central Limit Theorem, we know this graph:


Thus, we have 2 =0.01 x E(X)

2 x =0.01 x n/22/n

64

e.g.30 Suppose that we want to be



1% ofE(X)

P(X = x)

X1% ofE(X)

E(X)

2 x =0.01 x n/22/n

65

e.g.30 Suppose that we want to be



1% ofE(X)

P(X = x)

X1% ofE(X)

E(X)

2 x =0.01 x n/22/n=0.005 x nn

1 =0.005 x n/ n1 =0.005 x n

1/0.005 = nn =(1/0.005)2

n =40000

Therefore, if we flip the coin 40000 times, then we are 95% sure that the number of heads is within 1% of the expected value

66

Appendix

Appendix

67

e.g.14

Outline: In the following, we want to prove the

correctness of the following lemma with some explanations

Lemma 5.28If X and Y are independent random variables on sample space S withvalues x1, x2, …, xk and y1, y2, …, ym, respectively, then E(XY) = E(X)E(Y)

68

e.g.15 (Page 23) Let X be a set = {4, 5, 6}

Let Y be a set = {6, 9}

Given a particular xi, what does mean?

x1 x2 x3

y1 y2

2

1jji yx

2

1jji yx 21 yxyx ii

96 ii xx

69



What does mean?

x1 x2 x3

y1 y2

3

1

2

1i jji yx

3

1

2

1i jji yx

3

121 )(

iii yxyx

)()()( 231322122111 yxyxyxyxyxyx )9666()9565()9464(

70



What is the set H of all possible values of x.y where x X and y Y ?

x1 x2 x3

y1 y2

x y x.y

4 6 24

4 9 36

5 6 30

5 9 45

6 6 36

6 9 54

The set of all possible values is{24, 36, 30, 45, 54}

H: all possible values of x.y where x X and y Y H = {24, 36, 30, 45, 54}

71



What does mean?

x1 x2 x3

y1 y2

)(3

1

2

1ji

i jji yxyx

)(3

1

2

1ji

i jji yxyx

3

12211 ))()((

iiiii yxyxyxyx

))()((

))()(())()((

23231313

2222121221211111

yxyxyxyx

yxyxyxyxyxyxyxyx

))(96)(66(

))(95)(65())(94)(64(

2313

22122111

yxyx

yxyxyxyx

)(54)(63

)(45)(30)(36)(24

2313

22122111

yxyx

yxyxyxyx

)(54

)(45)(30)(63)(36)(24

23

2212132111

yx

yxyxyxyxyx


Show that it equals

zyxji

jiHz ji

yxz:),(

)(

72

e.g.18 Let X be a set = {4, 5, 6}


What does mean?

x1 x2 x3

y1 y2

)(3

1

2

1ji

i jji yxyx

)(3

1

2

1ji

i jji yxyx

)(54

)(45)(30)(63)(36)(24

23

2212132111

yx

yxyxyxyxyx


Show that it equals

zyxji

jiHz ji

yxz:),(

)(

73

e.g.18 Let X be a set = {4, 5, 6}


What does mean?

x1 x2 x3

y1 y2

)(3

1

2

1ji

i jji yxyx

)(3

1

2

1ji

i jji yxyx

)(54

)(45)(30)(63)(36)(24

23

2212132111

yx

yxyxyxyxyx


)(54

)(45)(30)]()(36[)(24

23

2212132111

yx

yxyxyxyxyx

)(54)(45)(30])(36[)(24 23221236:),(

11 yxyxyxyxyxji yxji

ji

Note that x1y2 = 4.9=36

Note that x3y1 = 6.6=36

Show that it equals

zyxji

jiHz ji

yxz:),(

)(

74

e.g.18 Let X be a set = {4, 5, 6}


What does mean?

x1 x2 x3

y1 y2

)(3

1

2

1ji

i jji yxyx

)(3

1

2

1ji

i jji yxyx


)(54)(45)(30])(36[)(24 23221236:),(


ji

Show that it equals

zyxji

jiHz ji

yxz:),(

)(

75

e.g.18 Let X be a set = {4, 5, 6}


What does mean?

x1 x2 x3

y1 y2

)(3

1

2

1ji

i jji yxyx

)(3

1

2

1ji

i jji yxyx


)(54)(45)(30])(36[)(24 23221236:),(


ji

54:),(23

45:),(22

30:),(12

36:),(24:),(

)(54)(45

)(30])(36[)(24

jiji

jijiji

yxjiyxji

yxjiyxjiji

yxjiji

yxyx

yxyxyx

zyxji

jiHz ji

yxz:),(

)(

Show that it equals

zyxji

jiHz ji

yxz:),(

)(

z is a value of XY

76

e.g.19 (Page 23)

We know that)(

3

1

2

1ji

i jji yxyx

zyxji

jiHz ji

yxz:),(

)(

z is a value of XY


Let X be a random variable and Y be a random variable. Do you think that the following is correct?

)()(3

1

2

1ji

i jji yYPxXPyx

zyxji

jiHz ji

yYPxXPz:),(

)()(

z is a value of XYYes.

77

e.g.20 (Page 23)

Let X be a random variable with values x1, x2, …, xk and Y be a random variable with values y1, y2, …, ym. Do you think that the following is correct?

)()(1 1

ji

k

i

m

jji yYPxXPyx

zyxji

jizz ji

yYPxXPz:),( XY of valuea is :

)()(

Yes.

78

e.g.21 (Page 23)Lemma 5.28If X and Y are independent random variables on sample space S withvalues x1, x2, …, xk and y1, y2, …, ym, respectively, then E(XY) = E(X)E(Y)

Why is it correct?Consider E(X)E(Y)

])(][)([11

m

jjj

k

iii yYPyxXPx

m

jjj

k

iii yYPyxXPx

1 1

)(])([

m

j

k

ijjii yYPyxXPx

1 1

])()([

m

j

k

ijiji yYPxXPyx

1 1

])()([

m

j

k

ijiji yYPxXPyx

1 1

)()(

zyxji

jizz ji

yYPxXPz:),( XY of valuea is :

)()(

(By the equation on the previous slide)

zyxji

jizz ji

yYxXPz:),( XY of valuea is :

)(

(because X and Y are independent)

zyxji

jizz ji

yxXYPz:),( XY of valuea is :

)(

)( XY of valuea is :

zXYPzzz

)(XYE Done!

Documents

1 Variance of RVs Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong