Violations of Assumptions In Least Squares Regression

Standard Assumptions in Regression

• Errors are Normally Distributed with mean 0

• Errors have constant variance• Errors are independent• X is Measured without error

2

2

2

00

00

00

εV

Example Xs and OLS Estimators

t X0(t) X1(t)1 1 02 1 13 1 24 1 35 1 46 1 57 1 68 1 79 1 810 1 911 1 1012 1 013 1 114 1 215 1 316 1 417 1 518 1 619 1 720 1 821 1 922 1 10

“t” is used to imply time ordering

12^

1

2

2

20

^

1

2

2

1

^

1^

1

^

0

^

11

1

21

^

1010

'

1

'

1050,...,1

XXβ

YX'XXβ

εXβY

V

XX

X

nV

XXV

XYYaYXX

XX

ntXY

n

tt

n

tt

t

n

ttt

n

tn

tt

t

ttt

Non-Normal Errors (Centered Gamma)

Errors = (Gamma(2,3.7672)-7.3485)

• Yt = 50 + 10Xt + (t-7.35) = 0*+1Xt+t*

• E(t*) = 0 V(t*) = 27

0.350320.1227273

0.12272736136364.0

6136364.0295455.4

770110

1102227'

1

^

1

12^

SE

V XXβ

Based on 100,000 simulations, the 95% CI for 1 contained 10 in 95.05% of the samples.Average=9.99887, SD=0.3502Average s2(b1) = 0.1224804

Non-Constant Error Variance (Heteroscedasticity)

• Mean: E(Y|X) = X = 50 + 10X

• Standard Deviation: Y|X = X + 0.5

• Distribution: NormalX E(Y|X) 0 50 0.51 60 1.52 70 2.53 80 3.54 90 4.55 100 5.56 110 6.57 120 7.58 130 8.59 140 9.510 150 10.5

Non-Constant Error Variance (Heteroscedasticity)

4763.00.218409

0.2184090.59205-

0.59205-2.289773

5.10000

05.000

005.100

0005.0

1

^

2

29191

1919199919

912

91

912

91

1999191919

91912

SE

V

VVV

EEE

VE

11

1111^

111^

XX'XYX'XX'

XX'XX'XYX'XX'YX'XX'β

βXβX'XX'YX'XX'YX'XX'β

00

00000

00

00

00000

00

YXβY

Based on 100,000 simulations, the 95% CI for 1 contained 10 in 92.62% of the samples.Average=9.998828, SD = 0.467182Average s2(b1) = 0.1813113 < 0.2184

Correlated Errors

Vε 2

321

32

2

12

2

2

t1

2

2

1

211

1

1

1

1

1

:1 iii)

2 oft independen ii)

1

,0~ i)

:Assuming

,0~,...,

nnn

n

n

n

nttt

V

t

iid

Example: 2= 9, =0.5, n=22

Correlated Errors

0.3035730.092157

0.0921570.46078-

0.46078-3.841111

15.05.05.0

5.015.05.0

5.05.015.0

5.05.05.01

5.01

9

1

^

2

322222122

3222

222

1222

2

SE

V

VVV

EEE

VE

11

1111^

111^

XX'XYX'XX'

XX'XX'XYX'XX'YX'XX'β

βXβX'XX'YX'XX'YX'XX'β

YXβY

Based on 100,000 simulations, the 95% CI for 1 contained 10 in 84.56% of the samples.Average=10.00216, SD=0.3039251Average s2(b1) = 0.0476444 < 0.092157

Measurement Error in X• Z=True Value of Independent Variable (Unobserved)

• X=Observed Value of Independent Variable• X=Z+U (Z can be fixed or random)• Z, U independent (assumed) when Z is random• V(U) is independent of Z when Z is fixed

22

11

^

22122

2

11

^2

2

1

1:example in this e.g. fixed

1

1,~ :random

,0~

ZZnEZ

ENZZ

NU

u

zuuz

zzZii

u

Measurement Error in X

21

^

2211

^

2222

1

2

20226.0,10~

Error, MeasurmentWithout

09.91.1

10

220)1(22

1

110

1

1

91220

N

ZZnE

UZXZZ

u

ui

i

Based on 100,000 simulations, the 95% CI for 1 contained 10 in 76.72% of the samples.Average=9.197568, SD=0.6076758Average s2(b1) = 0.4283653 >> (0.20226)2