Linear Combinations, Covariance andCorrelation

Rob McCulloch

1. Mean and Variance of a Linear Function2. Covariance and Correlation for RVs3. Mean and Variance of a Linear Combination

1. Mean and Variance of a Linear Function

Previously we looked at

Y = c0 + c1X

where Y and X were discrete.

Our formulas for the mean and variance work the same way fordiscrete and continuous random variables.

1

11. Mean and Variance of a Linear Function

Var Y c Var X( ) ( ) 12

E Y c c E X( ) ( ) �0 1

Y c c X �0 1

X1Y |c| V V

Let Y and X be random variables such that

Then,

2

Example:

Suppose you plan to play the game with winnings W withE (W ) = 0 and Var(W ) = 100.

There is a $3 cover charge to get in to play so your total winningsT is represented by:

T = −3 +W

Then,,

E (T ) = −3 + E (W ) = −3.

Var(T ) = Var(W ) = 100.

3

Suppose sneak in, don’t pay the cover and ”double your money”:

T = 2W

E (T ) = 2E (W ) = 2(0) = 0.

Var(T ) = 4Var(W ) = 400.

We don’t have to know the distribution of W .We don’t have to know if W is discrete or continuous.

4

Example:

Suppose you think returns on the market next period can berepresented by

M ∼ N(6, 225),

where M represents the uncertain market return (in percent).

You also could invest in a “risk-free” asset which pays rf for sure.

Suppose you put fraction w1 into the risk free asset and w2 intothe market. (w1 + w2 = 1, wi ≥ 0).

Then the return on your portfolio is:

P = w1 rf + w2M

5

Suppose rf = 2% and w2 = .6.

Then,

P = .4(2) + .6M= .8 + .6M

What are the mean and standard deviation of P?

E (P) = .8 + .6E (M) = .8 + .6(6) = 4.4.

σP = .6σM = .6 (15) = 9.

6

2. Covariance and Correlation for RVs

Suppose we have a pair of random variables (X ,Y ).

We would like to have a measure of how dependent the randomvariables are.

7

The covariance between bivariate discrete randomvariables X and Y is

cov(X ,Y ) = σXY =∑

p(x , y)(x − µx)(y − µy )

where the sum is over all possible (x , y) pairs.

For continuous random variables there is a formula using calculus.

8

Example:

µX = .1, µY = .1.

σX = .05, σY = .05.

X.05 .15

.05 .4 .1Y

.15 .1 .4

x y prob x-E(X) y-E(Y) prod

0.05 0.05 0.4 -0.05 -0.05 0.0025

0.15 0.05 0.1 0.05 -0.05 -0.0025

0.05 0.15 0.1 -0.05 0.05 -0.0025

0.15 0.15 0.4 0.05 0.05 0.0025

Cov(X ,Y ) = σXY

= .4 ∗ .0025 + .1 ∗ (−.0025) + .1 ∗ (−.0025) + .4 ∗ .0025 = .0015.

Intuition:There is an 80% chance X and Y move in the same direction.

9

Example:

µX = .1, µY = .1.

σX = .05, σY = .05.

X.05 .15

.05 .1 .4Y

.15 .4 .1

x y prob x-E(X) y-E(Y) prod

0.05 0.05 0.1 -0.05 -0.05 0.0025

0.15 0.05 0.4 0.05 -0.05 -0.0025

0.05 0.15 0.4 -0.05 0.05 -0.0025

0.15 0.15 0.1 0.05 0.05 0.0025

Cov(X ,Y ) = σXY

= .1 ∗ .0025+ .4 ∗ (−.0025) + .4 ∗ (−.0025) + .1 ∗ .0025 = −.0015.

Intuition:There is an 80% chance X and Y move in opposite directions.

10

ρσ

σ σXYXY

X Y

=

The correlation between random variables(discrete or continuous) is

11

ρ: the basic facts

− ≤ ≤1 1ρ

If ρ is close to 1, then it means there is a line, with positive slope, such that (X,Y) is likely to fall close to it.

If ρ is close to -1, same thing, but the line has a negative slope.

12

Example:

X.05 .15

.05 .4 .1Y

.15 .1 .4

The correlation is:

ρXY =.0015

.05 ∗ .05= 0.6.

13

Example:

X.05 .15

.05 .1 .4Y

.15 .4 .1

The correlation is:

ρXY =−.0015.05 ∗ .05

= −0.6.

14

Example X

0 1

Y

0 .25 .25

1 .25 .25

Let us compute the covariance:.25(-.5)(-.5) + .25(-.5)(.5) + .25(.5)(-.5) +.25(.5)(.5)=0

The covariance is 0 and so is the correlation: not surprising, right?

15

Independence and correlation

Suppose two RV’s are independent.

That means they have nothing to do with each other.

That means they have nothing to do with each other linearly.

That means the correlation is 0.

If X and Y are independent, then

XY 0 cov(X,Y)ρ = =

Note: the converse is not necessarily true. Cov=0 does not necessarily mean they are independent.

16

3. Mean and Variance of a Linear Combination

We looked at the returns on a portfolio which mixed a risklessasset and a risky asset.

How about mixing two risky assets?

Let X be the return on one risky asset and Y be the return onanother.

Suppose you put fraction w1 of your wealth into X and w2 into Y .

Let P denote the return on your portfolio.

ThenP = w1 X + w2 Y

The random variable P is a linear combination of the randomvariables X and Y . 17

Example:

Suppose X and Y have the joint distribution given below andw1 = w2 = .5.

P = .5X + .5Y

µX = .1, µY = .1.

σX = .05, σY = .05.

X.05 .15

.05 .1 .4Y

.15 .4 .1

What is the distribution of P?

If X turns out to be .15and Y turns out to be .05the P turns out to be.5 * .15 + .5 * .05 = .1

x y prob p

1 0.05 0.05 0.1 0.05

2 0.15 0.05 0.4 0.10

3 0.05 0.15 0.4 0.10

4 0.15 0.15 0.1 0.1518

P = .5X + .5Y

P:

prob p

1 0.1 0.05

2 0.4 0.10

3 0.4 0.10

4 0.1 0.15

or P:

prob p

1 0.1 0.05

2 0.8 0.10

3 0.1 0.15

What are the mean and variance of P?

µP = E(P) = .1 ∗ .05 + .4 ∗ .1 + .4 ∗ .1 + .1 ∗ .15 = 0.1

σ2P = Var(P) = .1 ∗ (.05− .1)2 + .8 ∗ (.1− .1)2 + .1 ∗ (.15− .1)2=.0005

σP =√.0005 = .0224.

The mean makes sensebut why is the standard deviation so low ??!!

19

P = .5X + .5Y

x y prob p

1 0.05 0.05 0.1 0.05

2 0.15 0.05 0.4 0.10

3 0.05 0.15 0.4 0.10

4 0.15 0.15 0.1 0.15

P has a way lower variance because 80% of the time if one of Xand Y are above average, the other is below and the two cancelout in P which is just the average.

80% of the time, P is equal to µP = .1.

Apparently, covariance matters when we linearly combine RVs !!

20

Mean and Variance of a Combination of two RVs

Let Y , X1, and X2 be random variables such that

Y = c0 + c1 X1 + c2X2.

then,

µy = c0 + c1 µX1 + c2 µX2 .

σ2Y = c21 σ2X1

+ c22 σ2X2

+ 2 c1 c2 σX1X2 .

21

Example:

In the example we just did we have

µX = .1, µY = .1.

σX = .05, σY = .05.

σXY = −.0015.

P = .5X + .5Y

µP = .5 ∗ .1 + .5 ∗ .1 = .1

σ2P = .52 ∗ .052 + .52 ∗ .052 + 2 ∗ .5 ∗ .5 ∗ (−.0015) = .0005.√.0005 = .0224.

22

Example:

Let’s use the formulas on our positive covariance example.

µX = .1, µY = .1. σX = .05, σY = .05.σXY = .0015.

P = .5X + .5Y

µP = .5 ∗ .1 + .5 ∗ .1 = .1

σ2P = .52 ∗ .052 + .52 ∗ .052 + 2 ∗ .5 ∗ .5 ∗ (.0015) = .002√.002 = .0447.

With a positive covariance, X and Y tend to go up and downtogether, so you don’t get much variance reduction by averaging.

23

In general, with a combination of k random variables we have:

Let Y , X1, X2, X3, ...Xk (have k X ’s) be randomvariables such that

Y = c0 + c1 X1 + c2X2 + . . .+ ckXk

then,

µy = c0 + c1 µX1 + c2 µX2 + . . .+ ck µXk.

σ2Y = c21 σ2X1

+ c22 σ2X2

+ . . .+ c2k σ2Xk

+ 2∑

all pairs

ci cj σXiXj

24

Example:

With k = 3 we have:

Y = c0 + c1 X1 + c2X2 + c3X3

µy = c0 + c1 µX1 + c2 µX2 + c3 µX3 .

σ2Y = c21 σ2X1

+ c22 σ2X2

+ c23 σ2X3

+ 2 (c1 c2 σX1X2 + c2 c3 σX2X3 + c1 c3 σX1X3)

25

Note:

The return on a portfolio in general.

Suppose we put wealth fraction wi into asset with return ri ,i = 1, 2, . . . , k .

Then, the return on the portfolio is

P = w1 r1 + w2 r2 + . . .+ wk rk =∑

wi ri .

26

Example

μ

μ

μ

σ

σ

σ

ρ

ρ

ρ

σ

σ

σ

X

X

X

X

X

X

X X

X X

X X

X X

X X

X X

1

2

3

1

2

3

1 2

1 3

2 3

1 2

1 3

2 3

05

1

15

01

009

008

3

2

2

002846

001789

001697

2

2

2

=

=

=

=

=

=

=

= −

=

=

= −

=

.

.

.

.

.

.

.

.

.

.

.

.

Y X X X= + +. . .2 5 31 2 3

μy = .2*(.05) + .5*(.1) + .3*(.15 )= 0.105000

.2*.2*(.01) + .5*.5*(.009) + .3*.3*(.008) + 2*.2*.5*.002846 + 2*.2*.3*(-.001789) + 2*.5*.3*.001697 = 0.00423362

σY2 =

Think in terms of financial assetsand portfolios.

27

Special Cases

Here are some important special cases of our formulae(remember, these work for any RV’s, continuous or discrete)

E X X E X E X( ) ( ) ( )1 2 1 2+ = +

Var X X Var X Var X X X( ) ( ) ( ) cov( , )1 2 1 2 1 22+ = + +

E X X E X E X( ) ( ) ( )1 2 1 2− = −

If the correlation is 0, then:

Var X X Var X Var X X X( ) ( ) ( ) cov( , )1 2 1 2 1 22− = + −

Var X X Var X Var X( ) ( ) ( )1 2 1 2+ = +

Var X X Var X Var X( ) ( ) ( )1 2 1 2− = +

28

For example:

Y = X1 + X2

= 0 + (1) (X1) + (1) (X2)

E (Y ) = 0 + (1)E (X1) + (1)E (X2) =E (X1) + E (X2).

Var(Y ) = (12)Var(X1) + (12)Var(X2) + 2(1)(1) cov(X1,X2) =Var(X1) + Var(X2) + 2 cov(X1,X2).

For example:

Y = X1 − X2

= 0 + (1) (X1) + (−1) (X2)

E (Y ) = 0 + (1)E (X1) + (−1)E (X2) =E (X1)− E (X2).

Var(Y ) = (12)Var(X1)+ (−1)2 Var(X2)+2(1)(−1) cov(X1,X2) =Var(X1) + Var(X2)− 2 cov(X1,X2).

29

Example:

Suppose the wining on a game are represented by the randomvariable W with

E (W ) = −1, Var(W ) = 100

You pay a dollar to play and after that you are just as likely to loseas win.

30

Suppose you double the bet so that

T1 = 2W .

E (T1) =

Var(T1) =

31

Suppose you double the bet so that your total winnings are

T1 = 2W .

E (T1) = 2E (W ) = 2(−1) = −2.

Var(T1) = 22Var(W ) = 4Var(W ) = 400.

32

Suppose you play twice so that your total winnings are

T2 = W1 +W2

E (T2) =

Var(T2) =

33

Suppose you play twice so that your total winnings are

T2 = W1 +W2

E (T2) = E (W1) + E (W2) = 2E (W ) = 2(−1) = −2.

Var(T2) = Var(W1) + Var(W2) = 2Var(W ) = 200.

34

Suppose you play the game twice and are interested in the averagewinnings

A =W1 +W2

2=

1

2W1 +

1

2W2

E (A) =

Var(A) =

35

Suppose you play the game twice and are interested in the averagewinnings

A =W1 +W2

2=

1

2W1 +

1

2W2

E (A) = 12E (W1) +

12E (W2) =

22E (W ) = E (W ) = −1.

Var(A) = 122Var(W1) +

122Var(W2) =

24Var(W ) = Var(W )

2 = 50.

36

Double:

I mean doubles.

I variance goes up by 4.

Sum two:

I mean doubles.

I variance goes up by 2.

Average two

I mean is the same.

I variance goes down by 12 .

37