8/10/2019 1415215404_395__835ps7

1/4

Problem Set #7: OLS

Economics 835: Econometrics

Fall 2014

1 A preliminary result

Suppose our data set Dn is a random sample of size n on the scalar random variables (xi, yi) with finitemeans, variances, and covariance. Let:

cov(xi, yi) = 1

n

ni=1

(xi x)(yi y)

Prove that plim cov(xi, yi) = cov(xi, yi).

We will use this result repeatedly in this problem set and in the future. So once you have proved this result,please feel free to take it as given for the remainder of this course. You can also take as given that if youdefine var(xi) = cov(xi, xi) then plim var(xi) = var(xi).

2 OLS with a single explanatory variable

In many cases, the best way to understand various issues in regression analysis - measurement errors, proxyvariables, omitted variables bias, etc. - is to work through the issue in the special case of a single explanatoryvariable. That way, we can develop intuition without getting lost in the linear algebra. Once we have thebasics down, we can then look at the multivariate case to see if anything changes. This problem goes throughthe main starting results.

Suppose our regression model has an intercept and a single explanatory variable, i.e.:

yi= 0+1xi+ui

where (yi, xi, ui) are scalar random variables. To keep things fairly general, we will assume this is a modelof the best linear predictor, i.e. E(ui) = E(xiui) = cov(xi, ui) = 0.

Our data Dn consists of a random sample of size n on (xi, yi), arranged into the matrices:

X=

1 x11 x2...

...1 xn

y=

y1y2...yn

Let:

=

01

= (XX)1Xy (1)

1

8/10/2019 1415215404_395__835ps7

2/4

ECON 835, Fall 2014 2

be the usual OLS regression coefficients.

a) Show that:

1 = cov(xi, yi)

var(xi)

0 = E(yi) 1E(xi)

b) Show that equation (1) implies that:

1 =1

n

ni=1(xi x)(yi y)1

n

ni=1(xi x)

2 =

cov(xi, yi)

var(xi)

0 = y 1x

The idea for this problem is that you get a little practice translating between different ways of writing thesame model, so even if you know another way to get these results please start with equation ( 1).

c) Without using linear algebra (i.e., just apply Slutskys theorem and the Law of Large Numbers to theresult from part (b) of this question), prove that

plim 1 = 1

plim 0 = 0

3 OLS with measurement error

Often variables are measured with error. Let (yi, xi, ui) be scalar random variables such that

yi = 0+1xi+ui where cov(x, u) = 0

Unfortunately, we do not have data on yi andxi; instead we have data on yi and xi where:

yi = yi+vi

xi = xi+wi

where wi and vi are scalar random variables representing measurement error. We assume classical mea-surement error1:

cov(vi, xi) = cov(vi, ui) = cov(vi, wi) = 0

cov(wi, xi) = cov(wi, ui) = cov(wi, vi) = 0

Letx = var(wi)/var(xi) and lety = var(vi)/var(yi).

a) Let 1be the OLS regression coefficient from the regression of yon x. Find plim 1in terms of (1, x, y)

b) What is the effect of (classical) measurement error inx on the sign and magnitude of plim 1?

c) What is the effect of (classical) measurement error in y on the sign and magnitude of plim 1?

1Strictly speaking the classical model of measurement error also assumes independence and normality, but we wont need

those for our results

8/10/2019 1415215404_395__835ps7

3/4

ECON 835, Fall 2014 3

4 Choice of units: The simple version

In applied work one is often faced with choosing units for our variables. Should we express proportions asdecimals or percentages? Miles or kilometers? etc. The short answer is that it doesnt matter if we arecomparing across linearly related scales; the OLS coefficients will scale accordingly, so one can choose unitsaccording to convenience.

Suppose our data set Dn is a sample (random or otherwise; this question is about an algebraic propertyof OLS and not a statistical property) of size n on the scalar random variables (yi, xi). Let the regressioncoefficients for the OLS regression ofyi on xi be:

1 = cov(xi, yi)

var(xi)

0 = y 1x

Now lets suppose we take a linear transformation of our data. That is, let:

xi = axi+b

yi = cyi+d

where (a,b,c,d) are a set of scalars (both a and c must be nonzero), and let the regression coefficients forthe OLS regression of yi on xi be:

1 = cov(xi,yi)

var(xi)

0 = y 1 x

where y= 1n

n

i=1yi and x= 1

nn

i=1xi.

a) Find1 in terms of (0,1,a,b,c,d).

b) Find0 in terms of (0,1,a,b,c,d).

5 Choice of units: The general version

The results from the previous question carry over when there is more than one explanatory variable: addinga constant to either yi or xi only changes the intercept, multiplying yi by a constant c ends up multiplyingall coefficients by c, and multiplying any variable in xi by a constant a ends up multiplying the coefficienton that variable by 1/a. In this question we will prove that result. In working through this question,please remember that our results and arguments will be analogous to those in the previous (much easier)question, and that the purpose of this question is to get some practice with more advanced tools like the

Frisch-Waugh-Lovell theorem.Let y be ann 1 matrix of outcomes, and Xbe an n K matrix of explanatory variables. Let:

= (XX)1Xy

be the vector of coefficients from the OLS regression ofy on X. We are interested in what will happen if weapply some linear transformation to our variables.

a) We start by seeing what happens if we take some multiplicative transformation. Let:

X = XA

y = cy

8/10/2019 1415215404_395__835ps7

4/4

ECON 835, Fall 2014 4

whereA is a KKmatrix2 with full rank (i.e., A1 exists) and c is a nonzero scalar. Let:

= (X

X)1

X

y

be the vector of coefficients from the OLS regression ofy on X.

Show that = cA1.

b) Suppose that the covariance matrix ofis . What is the covariance matrix of?

c) Using this result, what happens to our OLS coefficients if we multiply one of the explanatory variablesby 10 and leave everything else unchanged?

d) Using this result, what happens to our OLS coefficients if we multiply the dependent variable by 10 andleave everything else unchanged?

e) Next we consider an additive transformation. For this we suppose we have an intercept, and we changethe notation slightly. Let:

X=

1 x11 x2

1...

1 xn

where xi is a 1 (K 1) matrix and let

=

01

= (XX)1Xy

where y is an n 1 matrix of outcomes. Our transformed data are:

X = X +nb

y = y+dn

whereb is a 1Kmatrix whose first element is zero, d is a scalar, and n is ann 1 matrix of ones. Let:

=

01

= (XX)1Xy

be the vector of coefficients from the OLS regression of yi onxi.

Show that 1 = 1. The Frisch-Waugh-Lovell theorem might be useful here.

f) Suppose that the covariance matrix of1 is 1. What is the covariance matrix of1?

g) What happens to our OLS coefficients other than the intercept when we add 5 to the dependent variablefor all observations? When we add 5 to one of the explanatory variables?

2We are mostly interested in the case where A is diagonal, i.e., we are multiplying each column in X by some number. But

notice that this setup includes a lot of other redefinitions of variables.