Maximum-likelihood estimation

A general method for estimating parameters in a model

The method of least-squares

ii xYE 10][

Model for the expectation (fixed part of the model):

][ iii YEyr Residuals:

The method of least-squares: Find the values for the parameters (β0 and β1) that makes the sum of the squared residuals (Σrj2) as small as possible.

Can only be used when the error term is normal (residuals are assumed to be drawn from a normal distribution)

),0(~ where,10 NxY iiii

ii xYE 10][

Model for the expectation (fixed part of the model):

][ iii YEyr Residuals:

The maximum likelihood method is more general!- Can be applied to models with any probability distribution

The maximum likelihoodExample: We want to estimate the probability, p, that individuals are infected with a certain kind of parasite.

Ind.: Infected:

1

2

4

5

6

3

1

0

1

0

1

1

7

8

9

1

0

0

10 1

The maximum likelihood method (discrete distribution): 1.Write down the probability of each observation by using the model parameters2.Write down the probability of all the data

3.Find the value parameter(s) that maximize this probability

Probability of observation:

p

1-p

p

p

p

p

p

1-p

1-p

1-p

46 )1()|DataPr( ppp

The maximum likelihoodExample: We want to estimate the probability, p, that individuals are infected with a certain kind of parasite.

Ind.: Infected:

1

2

4

5

6

3

1

0

1

0

1

1

7

8

9

1

0

0

10 1

- Find the value parameter(s) that maximize this probability

Probability of observation:

p

1-p

p

p

p

p

p

1-p

1-p

1-p

46 )1()|DataPr()( ppppL

Likelihood function:

0.0 0.2 0.4 0.6 0.8 1.0

0.0

00

00

.00

04

0.0

00

80

.00

12

p

L(p

, K, N

)

Exercises!