sampling_distributions_and_point_estimation_of_parameters.pdf

8/18/2019 sampling_distributions_and_point_estimation_of_parameters.pdf

1/48

1

CH.7 Sampling Distributions and

Point Estimation of Parameters• Introduction

– Parameter estimation, sampling distribution, statistic, point estimator, point estimate

• Sampling distribution and the Central Limit Theorem

• General Concepts of Point Estimation

– Unbiased estimators

– Variance of a point estimator

– Standard error of an estimator

– Mean squared error of an estimator

• Methods of point estimation

– Method of moments

– Method of maximum likelihood


2/48

2

7-1 Introduction

•The field of statistical inference consists of those

methods used to make decisions or to draw conclusionsabout a population.

• These methods utilize the information contained in a

sample from the population in drawing conclusions.

• Statistical inference may be divided into two major

areas:

• Parameter estimation

• Hypothesis testing


3/48

3

7-1 Introduction

Parameter estimation examples

•Estimate the mean fill volume of soft drink cans: Soft drink cans

are filled by automated filling machines. The fill volume may

vary because of differences in soft drinks, automated machines,and measurement procedures.

• Estimate the mean diameter of bottle openings of a specific bottle manufactured in a plant: The diameter of bottle openings

may vary because of machines, molds and measurements.


4/48

4

7-1 Introduction

Hypothesis testing example

•2 machine types are used for filling soft drink cans: m1 and m2

• You have a hypothesis that m1 results in larger fill volume of

soft drink cans than does m2.

•Construct the statistical hypothesis as follows:

• the mean fill volume using machime m1 is larger than the

mean fill volume using machine m2.

• Try to draw conclusions about a stated hypothesis.


5/48

5

Definition

7-1 Introduction

• Since a statistic is a random variable, it has a probability distribution.

• The probability distribution of a statistic is called a sampling distribution


6/48

6

7-1 Introduction

xμ

X x1=25, x2=30, x3=29, x4=31

25 30 29 3128.75

4

x + + +

= = Point estimate of μ

point

estimate

point

estimator


7/48

7

7-1 Introduction

2s2σ

2S

x1=25, x2=30, x3=29, x4=31

2

2 1

( )

6.91

n

i

i

x x

s n

=

−

= =−

∑ Point estimate of 2σ

point

estimate

point

estimator


8/48

8

7-1 Introduction


9/48

9

7-1 Introduction


10/48

10

7.2 Sampling Distributions and the

Central Limit TheoremStatistical inference is concerned with making decisions about a

population based on the information contained in a random

sample from that population.

Definitions:


11/48

11


Central Limit Theorem• The probability distribution of is called the sampling distribution of mean.

• Suppose that a random sample of size n is taken from a normal population with mean andvariance .

• Each observation X1, X2,…,Xn is normally and independently distributed with mean andvariance

• Linear functions of independent normally distributed random variables are also normallydistributed. (Reproductive property of Normal Distr.)

• The sample mean has a normal distribution

• with mean

• and variance

X

2σ

μ 2σ

1 2 ... n X X X X n

+ + +=

... x

n

μ μ μ μ μ

+ + += =

2 2 2 22

2

... X

n n

σ σ σ σ σ

+ + += =

2

, X N n

σ μ ⎛ ⎞⎜ ⎟

⎝ ⎠∼


12/48

12


Central Limit Theorem


13/48

13


Central Limit TheoremDistributions of average scores

from throwing dice.

Practically

if n ≥30, the normal approximation

will be satisfactory regardless of theshape of the population.

İf n


14/48

14


Central Limit TheoremExample 7-1


15/48

15



Figure 7-2 Probability for Example 7-1


16/48

16



X has a continuous distribution:Find the distributon of the sample

mean of a random sample of size

n=40?

Example 7-2

1/ 2, 4 6( )0,

x f xotherwise

≤ ≤⎧= ⎨⎩

5μ = 2 2(6 4) /12 1/3σ = − =

By C.L.T, is approximately normally

distributed with

X

25 1/[3(40)] 1/120 X X

μ σ = = =


17/48

17


Central Limit Theorem• Two independent populations

– 1st population has mean and variance

– 2nd population has mean and variance

• The statistic has the following mean and variance :

• mean

• and variance

1 2 X X −

1 2 1 21 2 X X X X

μ μ μ μ μ −

= − = −

1 2 1 2

2 22 2 2 1 2

1 2

X X X X n n

σ σ σ σ σ

− = + = +

1μ

2μ

2

1σ

22σ

7 2 S li Di ib i d h


18/48

18


Central Limit TheoremApproximate Sampling Distribution of a

Difference in Sample Means


19/48

19


Central Limit Theorem Ex.7-3

• Two independent and approximately normal populations

– X1: life of a component with old process =5000 =40

– X2: life of a component with improved process =5050 =30• n1=16 , n2=25

• What is the probability that the difference in the two sample means

is at least 25 hours?2 1 X X −

2 1 2 150 X X X X μ μ μ − = − =

2 1 2 1

2 2 2 136 X X X X

σ σ σ −

= + =

1μ

2μ

1σ

2σ

1 1

2 2

2 11

1

405000 10016

X X n

σ μ μ σ = = = = =

2 2

2 22 2

2

2

305050 36

25 X X

n

σ μ μ σ = = = = =

( )2 1

2 1

2 1

25 25 50( 25) 2.14 0.9838

136

X X

X X

P X X P Z P Z P Z μ

σ

−

−

⎛ ⎞− −⎛ ⎞− ≥ = ≥ = ≥ = ≥ − =⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎝ ⎠

⎝ ⎠


20/48

20

7-3 General Concepts of Point Estimation

• We may have several different choices for the point

estimator of a parameter. Ex: to estimate the mean of a

population – Sample mean

– Sample median

– The average of the smallest and largest observations in the sample• Which point estimator is the best one?

• Need to examine their statistical properties and develop

some criteria for comparing estimators• For instance, an estimator should be close to the true value

of the unknown parameter


21/48

21


7-3.1 Unbiased Estimators

Definition

When an estimator is unbiased, the bias is zero.

bias =


22/48

22


1 2

1 1 1 1( ) ... n E X E X X X n

n n n n

μ μ ⎛ ⎞

= + + + = =⎜ ⎟

⎝ ⎠

2 2

Are and unbiased estimators of and ? X S μ σ

Unbiased !

( )

( )

( )

( ) ( )

2

22 1

1

2 2 2 2

1 1 1 1

2 2 2 2 2

1 1

2 2

1

1

( ) 1 1

1 12 2

1 1

1 12

1 1

1

1

n

i ni

ii

n n n n

i i i ii i i i

n n

i i

i i

n

i

i

X X

E S E E X X n n

E X XX X E X XX X n n

E X nX nX E X nX n n

E X nE X n

=

=

= = = =

= =

=

⎛ ⎞−⎜ ⎟ ⎛ ⎞

⎜ ⎟= = −⎜ ⎟− −⎜ ⎟ ⎝ ⎠

⎜ ⎟⎝ ⎠

⎛ ⎞ ⎛ ⎞= − + = − +

⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠⎛ ⎞ ⎛ ⎞

= − + = −⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠

⎛ ⎞= −⎜ ⎟− ⎝ ⎠

∑

∑

∑ ∑ ∑ ∑∑ ∑

∑


23/48

23


2 2 2 2 2 2 2( ) ( ) ( )i i i

V X E X E X σ μ σ μ = = − ⇒ = +

Example 7-1 (continued)

( )2 2

22 2 2 2 2( ) ( ) ( ) ( ) ( )V X E X E X E X E X n n

σ σ μ μ = = − = − ⇒ = +

Unbiased !


24/48

24


• There is not a unique unbiased estimator.

• n=10 data 12.8 9.4 8.7 11.6 13.1 9.8 14.1 8.5 12.1 10.3

• There are several unbiased estimators of µ – Sample mean (11.04)

– Sample median (10.95) – The average of the smallest and largest observations in the sample (11.3)

– A single observation from the population (12.8)

• Cannot rely on the property of unbiasedness alone to select the

estimator.

• Need a method to select among unbiased estimators.


25/48

25


7-3.2 Variance of a Point Estimator

Definition

The sampling distributions of two

unbiased estimators

.ˆˆ 21 ΘΘ and


26/48

26


7-3.2 Variance of a Point Estimator

2

2

( )

( )

( ) ( ) 2

i

i

V X

V X

nV X V X for n

σ

σ

=

=

< ≥

The sample mean is better estimator of µ than a single observation Xi


27/48

27


7-3.3 Standard Error: Reporting a Point Estimate

Definition


28/48

28


7-3.3 Standard Error: Reporting a Point Estimate


29/48

29


Example 7-5


30/48

30



ˆ2 X X σ ±Probability that true mean is within is 0.9545


31/48

31


7-3.4 Mean Square Error of an Estimator

There may be cases where we may need to use a biased estimator.

So we need another comparison measure:

Definition

7 3 G l C f P i E i i


32/48

32


Θ̂

The MSE of is equal to the variance of the estimator plus the squared bias.

If is an unbiased estimator of θ, MSE of is equal to the variance of

2

2 2

2

ˆ ˆ( ) ( )

ˆ ˆ ˆ( ) ( )

ˆ( ) ( )

MSE E

E E E

V bias

θ

θ Θ = Θ −

⎡ ⎤ ⎡ ⎤= Θ − Θ + − Θ⎣ ⎦ ⎣ ⎦

= Θ +

Θ̂

Θ̂ ˆ


Θ

7 3 G l C t f P i t E ti ti


33/48

33


2ˆ( ) ( )V biasΘ +2 2

2 2 2 2

2 2 2 2

2 2 2 2

2 2

2

ˆ ˆ ˆ( ) ( )

ˆ ˆ ˆ ˆ ˆ ˆ2 ( ) ( ) 2 ( ) ( )

ˆ ˆ ˆ ˆ ˆ ˆ( ) 2 ( ) ( ) ( ) 2 ( ) ( )

ˆ ˆ ˆ ˆ( ) ( ) 2 ( ) ( )

ˆ ˆ( ) 2 ( )ˆ( )

E E E

E E E E E

E E E E E E

E E E E

E E

E

θ

θ θ

θ θ

θ θ

θ θ

θ

⎡ ⎤ ⎡ ⎤Θ − Θ + − Θ⎣ ⎦ ⎣ ⎦⎡ ⎤ ⎡ ⎤= Θ − Θ Θ + Θ + − Θ + Θ⎣ ⎦ ⎣ ⎦

= Θ − Θ Θ + Θ + − Θ + Θ

= Θ − Θ + − Θ + Θ

= Θ + − Θ= Θ −


ˆ( ) MSE Θ

7 3 G l C f P i E i i


34/48

34



7 3 G l C t f P i t E ti ti


35/48

35


1Θ̂

.ˆ 2Θ

1Θ̂


1Θ̂

An estimate based on would more likely be close to the true value of

than would an estimate based on ..ˆ2

Θ

biased but has

small variance

2Θ̂unbiased but has

large variance

θ

7 3 General Concepts of Point Estimation


36/48

36



Exercise: Calculate the MSE of the following estimators.

1

2

ˆ

ˆ

X

X

Θ =

Θ =

7 4 M th d f P i t E ti ti


37/48

37

7-4 Methods of Point Estimation

How can good estimators be obtained?

7-4.1 Method of Moments

7 4 M th d f P i t E ti ti


38/48

38


Example 7-7

n

E(X)=

7 4 Methods of Point Estimation


39/48

39


7-4.2 Method of Maximum Likelihood



40/48

40


Example 7-9



41/48

41



ln ( )

( )

( )

y f x

dy f x

dx f x

=′

=



42/48

42


Example 7-12

1

( )

( )

m

m

y ax b

dyma ax b

dx

−

= +

= +



43/48

43





44/48

44


Properties of the Maximum Likelihood Estimator



45/48

45


2σ

Properties of the Maximum Likelihood Estimator 2

2 2

1

2 2

22 2

1ˆ ( )

1ˆ( )

ˆ( )

n

i

i

MLE of is

X X n

n E n

bias E

n

σ

σ

σ σ

σ σ σ

=

= −

−

=

−= − =

∑

bias is negative. MLE for tends to underestimate

The bias approaches zero as n increases.

MLE for is an asymptotically unbiased estimator for

2σ

2σ

2σ



46/48

46


The Invariance Property



47/48

47


Example 7-13



48/48

48


Complications in Using Maximum Likelihood Estimation

• It is not always easy to maximize the likelihood

function because the equation(s) obtained from

d L(θ)/d θ = 0 may be difficult to solve.

• It may not always be possible to use calculus

methods directly to determine the maximum of L(θ).

Documents

sampling_distributions_and_point_estimation_of_parameters.pdf