1 1 Slide

Slide

Statistical InferenceStatistical Inference

We have used probability to We have used probability to model model the uncertainty the uncertainty observed in real life situations.observed in real life situations.

We can also the tools of probability in We can also the tools of probability in making making inferencesinferences about these situations about these situations

In addition we can also In addition we can also assess the reliabilityassess the reliability of these of these inferences.inferences.

Population and SamplePopulation and Sample

Part of our uncertainty is often caused because we Part of our uncertainty is often caused because we cannot access all of the information in which we are cannot access all of the information in which we are interested.interested.

The The populationpopulation is the set of all elements of interest in a is the set of all elements of interest in a particular study.particular study.

A A samplesample is a selection of some of the members of the is a selection of some of the members of the population.population.

2 2 Slide

Slide

Statistical InferenceStatistical Inference

We will try to model the variation in some quantity We will try to model the variation in some quantity measured on the members of a population by using measured on the members of a population by using an appropriate probability distribution.an appropriate probability distribution.

Remember, we are often interested in summaries Remember, we are often interested in summaries of such variation – we use measures such as means of such variation – we use measures such as means and standard deviations. When these are applied to and standard deviations. When these are applied to populations we call them populations we call them parametersparameters and when and when they are applied to samples we call them they are applied to samples we call them statisticsstatistics..

A A statisticstatistic is a numerical characteristic of a sample. is a numerical characteristic of a sample. A A parameterparameter is a numerical characteristic of a is a numerical characteristic of a

population.population.

3 3 Slide

Slide

Example 1.Example 1.

Suppose that we want to know the proportion Suppose that we want to know the proportion pp, of , of students banking at Bath University Barclays, that is in students banking at Bath University Barclays, that is in favour of extended opening hours.favour of extended opening hours.

We don’t have resources to contact everyone in the We don’t have resources to contact everyone in the population, so we select a small sample and ask each population, so we select a small sample and ask each member of it if they are in favour or not.member of it if they are in favour or not.

IfIf this sample is representative, it seems reasonable to this sample is representative, it seems reasonable to use the proportion of the sample in favour as an use the proportion of the sample in favour as an indication of the value of indication of the value of pp..

QuestionsQuestions What makes a sample representativeWhat makes a sample representative?? Is this a good procedure – what about reliability of Is this a good procedure – what about reliability of

guess?guess?

N.B. N.B. finite populationfinite population..

4 4 Slide

Slide

Example 2.Example 2.

A piece of new software is claimed to handle electronic A piece of new software is claimed to handle electronic transactions more efficiently than existing software.transactions more efficiently than existing software.

A broker tries out the new software for a week and is A broker tries out the new software for a week and is anxious to see if the the average time to complete the anxious to see if the the average time to complete the deals improves on the value using the current software deals improves on the value using the current software which is 2.75 hours.which is 2.75 hours.

Population: all the future deals that the software might Population: all the future deals that the software might handle.handle.

Sample: deals processed during trial week.Sample: deals processed during trial week. Parameter: average time to process all future deals.Parameter: average time to process all future deals. Statistics: average time to process deals during trial Statistics: average time to process deals during trial

week.week. Probability model is …?Probability model is …?

5 5 Slide

Slide

Choosing and using a Probability Model.Choosing and using a Probability Model.

Typically we use a probability distribution to Typically we use a probability distribution to model the variation in the characteristic of model the variation in the characteristic of interest interest both within the population and the both within the population and the sample which is drawn from itsample which is drawn from it..

Example: Electronic transaction timesExample: Electronic transaction times..

After examining the sample data After examining the sample data via via histograms histograms etcetc., we might decide that a ., we might decide that a Normal distribution adequately describes the Normal distribution adequately describes the pattern of variation in deal completion times.pattern of variation in deal completion times.

Specifically, if X is a typical completion time, Specifically, if X is a typical completion time, X~N(μ,σ²) – assumed or adopted model.X~N(μ,σ²) – assumed or adopted model.

6 6 Slide

Slide

Statistical Inference.Statistical Inference.

But this “model” is But this “model” is incomplete!incomplete! We do not know the value of μ ( or the value of σ).We do not know the value of μ ( or the value of σ). Can we Can we inferinfer something about these population something about these population

parameters from say, the equivalent sample statistics?parameters from say, the equivalent sample statistics?

Typical tasks of Statistical InferenceTypical tasks of Statistical Inference.. POINT ESTIMATION – guess a single numerical value for POINT ESTIMATION – guess a single numerical value for

a parameter.a parameter. INTERVAL ESTIMATION – guess a set of ‘likely’ values for INTERVAL ESTIMATION – guess a set of ‘likely’ values for

a parameter.a parameter. HYPOTHESIS TESTING – use data to decide whether or HYPOTHESIS TESTING – use data to decide whether or

not some assertion about the unknown value of a not some assertion about the unknown value of a parameter is true.parameter is true.

7 7 Slide

Slide

Point EstimationPoint Estimation

Consider the electronic transaction times example.Consider the electronic transaction times example. MODEL: Typical transaction processing time, X.MODEL: Typical transaction processing time, X. X~N(μ,σ²)X~N(μ,σ²) Over the trial week we will obtain a sample of say, n Over the trial week we will obtain a sample of say, n

observations, observations, Let us assume that these observations are independent.Let us assume that these observations are independent. At the end of the week we will have the actual values,At the end of the week we will have the actual values,

Interest is in the average transaction time in the Interest is in the average transaction time in the population of future deals – this corresponds to the population of future deals – this corresponds to the parameter μ in our modelparameter μ in our model

We use the data to guess the value of the parameter.We use the data to guess the value of the parameter.

nXXX ,...,, 21

nxxx ,...,, 21

8 8 Slide

Slide

Point EstimationPoint Estimation

Definition: A statistic used to estimate a parameter Definition: A statistic used to estimate a parameter value is called a value is called a (point) estimator(point) estimator..

Think of the estimator as a ‘recipe’ that tells you what Think of the estimator as a ‘recipe’ that tells you what to do with your observations in order to obtain the to do with your observations in order to obtain the actual numerical guess which is known as the actual numerical guess which is known as the estimateestimate..

In ideal circumstances, i.e. when the assumptions about In ideal circumstances, i.e. when the assumptions about the model are correct, the best estimator for the mean the model are correct, the best estimator for the mean of a normal distribution, μ, is the sample meanof a normal distribution, μ, is the sample mean

If actual data values are , the corresponding estimate is If actual data values are , the corresponding estimate is

n

i

in

XX1

n

i

in

xx1

9 9 Slide

Slide

Sampling Variation and Sampling Sampling Variation and Sampling DistributionsDistributions

An estimator is composed of random variables and so it too, An estimator is composed of random variables and so it too, is a random variable. is a random variable.

When we carry out the process of estimation as described, When we carry out the process of estimation as described, we do not really notice this random behaviour since we we do not really notice this random behaviour since we obtain just a single value of the estimate.obtain just a single value of the estimate.

If we repeated the whole estimation process, drawing a If we repeated the whole estimation process, drawing a different sample, we would most likely obtain a different different sample, we would most likely obtain a different value of the estimate. Different possible samples lead to value of the estimate. Different possible samples lead to different possible values of the estimate. This is different possible values of the estimate. This is sampling sampling variationvariation..

The nature of the variation in possible values of the The nature of the variation in possible values of the estimates is described by the probability distribution of the estimates is described by the probability distribution of the corresponding estimator.corresponding estimator.

This probability distribution is commonly known as the This probability distribution is commonly known as the sampling distributionsampling distribution of the estimator. of the estimator.

10 10 Slide

Slide

Sampling Distributions: ExampleSampling Distributions: Example

If ~N(μ,σ²), independently, thenIf ~N(μ,σ²), independently, then

N. B. distribution of isN. B. distribution of is Still normal,Still normal, Has the same mean as individual X’s,Has the same mean as individual X’s, Has a variance reduced by a factor of n.Has a variance reduced by a factor of n.

So is likely to be closer to μ than the individual X’s.So is likely to be closer to μ than the individual X’s.

nXXX ,...,, 21

),(~2

nNX

X

X

11 11 Slide

Slide

Indicating Precision of EstimationIndicating Precision of Estimation

Having obtained the data values , we Having obtained the data values , we can quote the value of the estimate .can quote the value of the estimate .

But we should also indicate the likely extent of the But we should also indicate the likely extent of the remaining uncertainty, i.e. the reliability or remaining uncertainty, i.e. the reliability or variability of the estimate.variability of the estimate.

One way to do this is to quote the standard One way to do this is to quote the standard deviation of the estimate which is known as its deviation of the estimate which is known as its standard errorstandard error..

Example: ~N(μ,σ²) then using the Example: ~N(μ,σ²) then using the sample mean to estimate μ, SE( ) =σ/sample mean to estimate μ, SE( ) =σ/nn

PROBLEM: In most cases we won’t know value of σ!PROBLEM: In most cases we won’t know value of σ!

XnXXX ,...,, 21

nxxx ,...,, 21

x

12 12 Slide

Slide

An Estimator for σ²An Estimator for σ²

The usual estimator is the The usual estimator is the sample variance, sample variance, s²s²

The corresponding estimator forThe corresponding estimator for σ is the σ is the sample standard deviation, s= sample standard deviation, s= s²s²

We can now quote the estimated standard We can now quote the estimated standard error of the sample mean, ESE( )= s/error of the sample mean, ESE( )= s/n.n.

2

1

2 )(1

1XX

ns

n

ii

X

13 13 Slide

Slide

Numerical ExampleNumerical Example

Electronic transaction timesElectronic transaction times.. Suppose n=100, =2.62 and s²= 0.81,Suppose n=100, =2.62 and s²= 0.81, ESE( )= s/ESE( )= s/n= n= (0.81/100)= 0.9/10 = 0.09.(0.81/100)= 0.9/10 = 0.09.

We would report that the estimate of the mean We would report that the estimate of the mean transaction time is 2.62 hours with an transaction time is 2.62 hours with an estimated standard error of 0.09 hours.estimated standard error of 0.09 hours.

x

x

14 14 Slide

Slide

Sampling Distribution ofSampling Distribution of

Process of Statistical InferenceProcess of Statistical Inference

Population Population with meanwith mean

= ?= ?

Population Population with meanwith mean

= ?= ?

A sampleA sampleof of nn elements is selected elements is selected

from the population.from the population.

XX

The sample data The sample data provide a value forprovide a value for

the sample meanthe sample mean . .

The sample data The sample data provide a value forprovide a value for

the sample meanthe sample mean . .xx

The value of is used toThe value of is used tomake inferences aboutmake inferences about

the value of the value of ..

The value of is used toThe value of is used tomake inferences aboutmake inferences about

the value of the value of ..

xx

15 15 Slide

Slide

Another example of point estimationAnother example of point estimation

The ideas of estimator and estimate, sampling The ideas of estimator and estimate, sampling distribution and standard error are quite distribution and standard error are quite general.general.

We describe another - very common – We describe another - very common – situation which illustrates this.situation which illustrates this.

16 16 Slide

Slide

Opinion PollsOpinion Polls

Suppose that we are interested in estimating Suppose that we are interested in estimating the proportion of the voting population which the proportion of the voting population which is in favour of adopting the Euro.is in favour of adopting the Euro.

We take a sample of n independent individuals We take a sample of n independent individuals from the voting population and ask each from the voting population and ask each member of this sample if they are pro- or anti-member of this sample if they are pro- or anti-euro.euro.

Suppose r are in favour.Suppose r are in favour.

The parameter of interest is the proportion of The parameter of interest is the proportion of the population that is pro-euro. Call this Π.the population that is pro-euro. Call this Π.

17 17 Slide

Slide

Opinion PollsOpinion Polls

How do we model this situation?How do we model this situation? Let = 1 if i’th person is pro-euro and Let = 1 if i’th person is pro-euro and

= 0 if i’th person is anti-euro.= 0 if i’th person is anti-euro. P( =1) = П and P( =0) = 1- П.P( =1) = П and P( =0) = 1- П. So R= ~Binomial(n, П)So R= ~Binomial(n, П) EstimatorEstimator for П is for П is proportion of the sampleproportion of the sample

that is pro-euro, i.e. S=R/n.that is pro-euro, i.e. S=R/n. Applying this to the data we obtain the Applying this to the data we obtain the

estimateestimate of П, r/n. of П, r/n.

iX

iX

iX iX

X

nXXX ...21

18 18 Slide

Slide

Sampling Distribution of S=R/nSampling Distribution of S=R/n

P(S=(r/n)) = for r=0,1,…,n.P(S=(r/n)) = for r=0,1,…,n.

E(S)=E(R)/n = n Π/n = Π E(S)=E(R)/n = n Π/n = Π Var(S) = Var(R/n)= var(R)/ n² = n Π (1- Π)/n²Var(S) = Var(R/n)= var(R)/ n² = n Π (1- Π)/n²

= Π (1- Π)/n= Π (1- Π)/n and SE(S)= and SE(S)= (Π (1- Π)/n)(Π (1- Π)/n) ESE(S) = ESE(S) = ( (1/n) (r/n) (1-(r/n)) ( (1/n) (r/n) (1-(r/n))

)(1 rnr

r

n

19 19 Slide

Slide

Opinion PollsOpinion Polls

Numerical example.Numerical example. Suppose n = 400 and r = 240.Suppose n = 400 and r = 240. Then s=240/400 = 0.6Then s=240/400 = 0.6 ESE(S)=ESE(S)=(0.6x0.4/400) = 0.0245.(0.6x0.4/400) = 0.0245. So the estimate of the proportion in favour of So the estimate of the proportion in favour of

adopting the Euro is 0.6 with an estimated adopting the Euro is 0.6 with an estimated standard error of 0.0245.standard error of 0.0245.