23 Estimation

Hadley Wickham

Stat310Estimation

Saturday, 11 April 2009

1. What’s next

2. Recap

3. Example: speed of sound

4. T distribution

5. Comparing estimators

6. Interval estimators


Today: finish of estimation

Thursday & Tuesday: testing

Last class: where next? Other stats courses and why you should bother

Help session tomorrow.

What’s next


Recap

What is the distribution of the average of n iid normal random variables with the same mean and variance?

How can you form a 95% ci for a random variable with that distribution?


ExampleWe want to figure out what the speed of sound is. We do this by performing an experiment with our velocitometer. A velocitometer can measure the speed of anything, but has normally distributed error with standard deviation 10 meters per second.

How can we decrease this error? How can we frame this problem statistically?


Your turnWe perform the experiment 10 times and get the following 10 speeds:

340 333 334 332 333 336 350 348 331 344 (mean: 338, sd: 7.01)

What is our estimate of the speed of sound? What is the error (sd) of this estimate? Give an interval that we’re 95% certain the true speed of sound lies in.


Hat notation

Usually write the estimate of a parameter with a little hat over it. Subscript identifies type of estimator used.

µ̂ !̂2 !̂2µ̂MM

µ̂ML


expt

upper

325

330

335

340

345

350

355

50 100 150 200


expt

upper

325

330

335

340

345

350

355

50 100 150 200


expt

lowe

r

325

330

335

340

345

350

355

50 100 150 200


Example

We want to figure out what the speed of sound is. We do this by performing an experiment with our velocitometer. A velocitometer can measure the speed of anything, but has normally distributed error with standard deviation 10 meters per second.

Why is this example not realistic?


Some reasons

No such thing as a velocitometer! Scientific experiments usually much more complicated

Don’t normally know the errors are normally distributed.

Don’t normally know the standard deviation of the errors.


Resolution

Possible to overcome all of these problems, but we’re going to focus on just one.

What happens if we don’t know the standard deviation, but have to estimate it?


What is an estimate for the standard deviation of a normal distribution?

When we have to estimate the sd, what do you think happens to the distribution of our estimate of the mean? (Would it get more or less accurate? What will happen to the confidence interval?)

What about as n gets bigger?

Your turn


X̄n ! µ

!/"

n# Z

X̄n ! µ

s/"

n# tn!1

t-distribution

Xi ! Normal(µ, !2)

Parameter called degrees of freedom


x

dens

0.1

0.2

0.3

−3 −2 −1 0 1 2 3

df1215Inf


Properties of the t-dist

Heavier tails compared to the normal distribution.

Practically, if n > 30, the t distribution is practically equivalent to the normal.

limn!"

tn = Z


t-tablesBasically the same as the standard normal. But one table for each value of degrees of freedom.

Easiest to use calculator or computer: http://www.stat.tamu.edu/~west/applets/tdemo.html

(For homework, use this applet, for final, I’ll give you a small table, if necessary)


http://www.stat.tamu.edu/~west/applets/tdemo.html




Example

Back to the example.

340 333 334 332 333 336 350 348 331 344 (mean: 338, sd: 7.01)

If sd is known: (332, 344)

If not known: (333, 342) (2.23)


Constructing interval

X̄n ! µ

!/"

n# Z

X̄n ! µ

s/"

n# tn!1


Steps

Form confidence interval for standardised distribution.

Write as probability statement.

Back transform.

Write as interval.


More complicated case

Find 95% confidence interval for standard deviation in previous case (sd = 7.01, n = 10)

(n! 1)S2

!2" "2(n! 1)


Standard deviation

Find confidence interval for χ2(9). Generally want the shortest confidence interval, but hard to find when not symmetric.

Any of the following are correct. The best has the smallest interval.


0.00

0.02

0.04

0.06

0.08

0.10

0 5 10 15 20 25 30


0.00

0.02

0.04

0.06

0.08

0.10 (0.05, 1)(3.33,Inf)

Length: Inf

0 5 10 15 20 25 30


0.00

0.02

0.04

0.06

0.08

0.10 (0.03, 0.99)(2.85,21.67)Length: 18.8

0 5 10 15 20 25 30


0.00

0.02

0.04

0.06

0.08

0.10 (0.025, 0.975)(2.7,19.0)

Length: 16.3

0 5 10 15 20 25 30


0.00

0.02

0.04

0.06

0.08

0.10 (0.01, 0.96)(2.09,17.61)Length: 15.5

0 5 10 15 20 25 30


0.00

0.02

0.04

0.06

0.08

0.10 (0, 0.95)(0.0,16.9)

Length: 16.9

0 5 10 15 20 25 30


Steps

Form confidence interval for standardised distribution.

Write as probability statement.

Back transform.

Write as interval.


Comparing estimators

We know two ways of estimating the standard deviation of the normal distribution: the usual standard deviation of the data, or the estimate from maximum likelihood or method of moments

How can we compare the two?


What are other estimators of the

standard deviation?


Your turn

How can we compare different point estimators?

Say I have u1 and u2 which are functions of the X's, trying to estimate some θ. Based on what properties could I choose between u1 and u2? (probability, mean and variance)


E(!̂n) = ! Unbiased

V ar(!̂1) < V ar(!̂2)

Minimum variance

Common problem is to find UMVE (unbiased minimum variance estimator) across all possible estimators


Low bias, low variance Low bias, high variance

High bias, low variance High bias, high variance


Technology

23 Estimation