1

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Suppose that you have alternative estimators of a population characteristic , one unbiased, the other biased but with a smaller variance. How do you choose between them?

0

0

pro

bab

ilit

y d

ensi

ty

estimator B

estimator A

One way is to define a loss function which reflects the cost to you of making errors, positive or negative, of different sizes.

2

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

0

-3 0 3

loss

error (negative) error (positive)

04

pro

bab

ilit

y d

ensi

ty

estimator B

3

A widely-used loss function is the mean square error of the estimator, defined as the expected value of the square of the deviation of the estimator about the true value of the population characteristic.

222MSE ZZZEZ

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Z

bias

04

pro

bab

ilit

y d

ensi

ty

estimator B

4

222MSE ZZZEZ

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Z

bias

The mean square error involves a trade-off between the variance of the estimator and its bias. Suppose you have a biased estimator like estimator B above, with expected value Z.

04

pro

bab

ilit

y d

ensi

ty

estimator B

5

222MSE ZZZEZ

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Z

bias

The mean square error can be shown to be equal to the sum of the variance of the estimator and the square of the bias.

6

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

To demonstrate this, we start by subtracting and adding Z .

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

7

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

We expand the quadratic using the rule (a + b)2 = a2 + b2 + 2ab, where a = Z – Z and b = Z – .

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

8

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

We use the first expected value rule to break up the expectation into its three components.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

9

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

The first term in the expression is by definition the variance of Z.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

10

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

(Z – ) is a constant, so the second term is a constant.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

11

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

In the third term, (Z – ) may be brought out of the expectation, again because it is a constant, using the second expected value rule.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

12

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

Now E(Z) is Z, and E(–Z) is –Z.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

13

Hence the third term is zero and the mean square error of Z is shown be the sum of the variance of Z and the bias squared.

22

22

22

22

22

2

2

2

2

2

2

MSE

ZZ

ZZZZZ

ZZZZ

ZZZZ

ZZZZ

ZZ

ZE

ZEEZE

ZZE

ZE

ZEZ

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared

14

In the case of the estimators shown, estimator B is probably a little better than estimator A according to the MSE criterion.

CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

0

0

pro

bab

ilit

y d

ensi

ty

estimator B

estimator A

Copyright Christopher Dougherty 2012.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be

used as a resource for teaching an econometrics course. There is no need to

refer to the author.

The content of this slideshow comes from Section R.6 of C. Dougherty,

Introduction to Econometrics, fourth edition 2011, Oxford University Press.

Additional (free) resources for both students and instructors may be

downloaded from the OUP Online Resource Centre

http://www.oup.com/uk/orc/bin/9780199567089/.

Individuals studying econometrics on their own who feel that they might benefit

from participation in a formal course should consider the London School of

Economics summer school course

EC212 Introduction to Econometrics

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

or the University of London International Programmes distance learning course

EC2020 Elements of Econometrics

www.londoninternational.ac.uk/lse.

2012.10.31