Hypothesis testing Hypothesis testing

• Want to know something about apopulation

• Take a sample from that population

• Measure the sample

• What would you expect the sample tolook like under the null hypothesis?

• Compare the actual sample to thisexpectation

population

sample

Y = 2675.4

Hypothesis testing

• Hypotheses are about populations

• Tested with data from samples

• Usually assume that sampling is

random

Types of hypotheses

• Null hypothesis - a specific statement

about a population parameter made for

the purposes of argument

• Alternate hypothesis - includes other

possible values for the population

parameter besides the value states in

the null hypothesis

The null hypothesis is usuallythe simplest statement,whereas the alternativehypothesis is usually the

statement of greatest interest.

A good null hypothesis would

be interesting if proven wrong.

A null hypothesis is specific;

an alternate hypothesis is not.

Hypothesis testing: exampleCan sheep recognize each other?

The experiment and the

results

• Sheep were trained to get a reward

near a certain other sheep’s picture

• Then placed in a Y-shaped maze

You must choose…

Stating the hypotheses

H0: Sheep go to each face with

equal probability (p = 0.5).

HA: Sheep choose one face over

the other (p ! 0.5).

Estimating the value

• 16 of 20 is a proportion of p = 0.8

• This is a discrepancy of 0.3 from the

proportion proposed by the null

hypothesis, p =0.5

Null distribution

• The null distribution is the samplingdistribution of outcomes for a teststatistic under the assumption that thenull hypothesis is true

0.00000120

0.0000219

0.0001818

0.001117

0.004616

0.01515

0.03714

0.07413

0.1212

0.1611

0.1810

0.169

0.128

0.0747

0.0376

0.0155

0.00464

0.00113

0.000182

0.000021

0.0000010

ProbabilityProportion of correctchoices

• Test statistic = a quantity calculated

from the data that is used to evaluate

how compatable the data are with the

expectation under the null hypothesis

The null distribution of p

Test statistic = 16

The null distribution of p

Values at least

as extreme as

the test statistic

• P-value - the probability of obtaining the

data* if the null hypothesis were true

*as great or greater difference from the null

hypothesis

P =

0.012

P-value

P-value calculation

P

=2*(Pr[16]+Pr[17]+Pr[18]+Pr[19]+Pr[20])

=2*(0.005+0.001+0.0002+0.00002+0.000001)

= 0.012

How to find P-values

• Get test statistic

• Compare with null distribution from:

– Simulation

– Parametric tests

– Non-parametric tests

– Re-sampling

Statistical significance

The significance level, !, is a probability used as a criterion

for rejecting the null hypothesis. If the P-value for a test is

less than or equal to !, then the null hypothesis is rejected.

! is often 0.05

Significance for the sheep

example

• P = 0.012

• P < !, so we can reject the null

hypothesis

Larger samples give more

information

• A larger sample will tend to give and

estimate with a smaller confidence

interval

• A larger sample will give more power to

reject a false null hypothesis

Hypothesis testing: another example

The genetics of symmetry in flowers

Heteranthera - Mud plantain

Stigma and anthers are asymmetric

in different genotypes

Can the pattern of inheritance

be explained by a single locus

with simple dominance?

Model predicts a 3:1 ratio of right-handed

flowers

H0: Right- and left-handed offspring occur at a

3:1 ratio (the proportion of right-handed

individuals in the offspring population is p = 3/4)

HA: Right- and left-handed offspring do not occur

at a 3:1 ratio (p ! 3/4)

Data

Of 27 offspring, 21 were “right-

handed” and 6 were “left-handed.”

Estimating the proportion

!

ˆ p =21

27= 0.778

* The “hat” notation denotes an estimate for a

population parameter from a sample

Number of right-handed flowers in a random sample

of 27Probability

0-8 0

9 0.000005

10 0.000031

11 0.000124

12 0.000492

13 0.001752

14 0.005301

15 0.013830

16 0.031094

17 0.060673

18 0.100891

19 0.143051

20 0.172339

21 0.171782

22 0.141034

23 0.091477

24 0.045499

25 0.016409

26 0.003759

27 0.000457

Sampling distribution of null

hypothesis

P = 0.83.

The P-value:

Rock-paper-scissors battle

Jargon

Significance level

• A probability used as a criterion for

rejecting the null hypothesis

• Called !

• If p < !, reject the null hypothesis

• For most purposes, ! = 0.05 is

acceptable

Type I error

• Rejecting a true null hypothesis

• Probability of Type I error is ! (the

significance level)

Type II error

• Not rejecting a false null hypothesis

• The probability of a Type II error is "

• The smaller ", the more power a test

has

Power

• The probability that a random sample of

a particular size will lead to rejection of

a false null hypothesis

• Power = 1- "

Reality

Result

Ho true Ho false

Reject Ho

Do not reject Ho correct

correctType I error

Type II error

One- and two-tailed tests

• Most tests are two-tailed tests

• This means that a deviation in eitherdirection would reject the nullhypothesis

• Normally ! is divided into !/2 on oneside and !/2 on the other Test statistic

2.5%2.5%

One-sided tests

• Also called one-tailed tests

• Only used when one side of the null

distribution is nonsensical

• For example, comparing grades on a

multiple choice test to that expected by

random guessing

Critical value

• The value of a test statistic beyond

which the null hypothesis can be

rejected

“Statistically significant”

• P < !

• We can “reject the null hypothesis”

We never “accept the null

hypothesis”