Testing Hypotheses

Tuesday, October 28

Objectives: Understand the logic of hypothesis testing

and following related concepts Sidedness of a test (left-, right- or two-

sided) Test statistic p-value Level of significance Rejection region Type I and II errors Power of a test

Introduction

Examples of questions that can be answered by a hypothesis test are Is the proportion of people born in February

equal to the proportion of days that February contains out of a year (i.e. 28/365)?

Are the proportions of teenagers that favor the death penalty and adults that favor the death penalty equal?

Introduction

• There are five steps that should be followed when performing hypothesis tests:– Determine the null and alternative hypotheses– Verify necessary data conditions, and if they are

met, summarize the data with the appropriate test statistic

– Assuming the null hypothesis is true, find the p-value

– Decide if the results are statistically significant based on the p-value

– State your conclusion in the context of the situation

Forming Hypotheses

• Consider the following questions:– Does the majority of the population favor a new

legal standard for the blood alcohol level that constitutes drunk driving?

– Do female students study, on average, more than men do?

– Will side effects be experienced by fewer than 20% of people who take some new medication?

• All of these questions can be answered with a “yes” or a “no”, and each makes a specific statement about a situation.

Forming Hypotheses

• For instance, each leads to two competing hypothesis statements:– Hypothesis 1: The proportion favoring the new

blood alcohol statement is not a majority– Hypothesis 2: The proportion favoring the new

blood alcohol statement is a majority• Or

– Hypothesis 1: On average, women do not study more than men do

– Hypothesis 2: On average, women do study more than men do

Forming Hypotheses

In statistical situations, these hypothesis statements are split into two categories: null hypotheses and alternative hypotheses The null hypothesis, represented by H0, is a

statement that nothing is happening. It varies from situation to situation, but it is generally thought of as the status quo, no relationship, no difference, etc. Most of the time, a researcher is trying to disprove or reject the null hypothesis.

Forming Hypotheses

The alternative hypothesis, represented by Ha, is a statement that something is happening. Usually, the researcher is trying to prove that the alternative hypothesis is true. It is typically a statement that goes against the status quo, or that says there is a relationship, or that there is a difference, etc.

Forming Hypotheses

• Examples of null hypotheses:– There is no such thing as ESP.– There is no difference in pulse rates for men and

women.– There is no relationship between exercise intensity

and the resulting aerobic benefit• Examples of alternative hypotheses:

– There is ESP– Men have a lower pulse rate than women do– Increasing exercise intensity increases aerobic

benefit

Forming Hypotheses

It may be easy to remember the logic of hypothesis testing and the assignment of null and alternative hypotheses as Innocent until proven guilty

As in the U.S. Judicial system We assume the null hypothesis is true until

we can conclusively say otherwise.

Forming Hypotheses

• Continuing with the pharmaceutical example, the appropriate hypotheses are:– At least 20% of people who take this new drug will

experience side effects.– Less than 20% of people who take this new drug

will experience side effects• If we say p is the proportion of people taking the

new drug who experience side effects, the null and alternative hypotheses can be rewritten as:– H0: p ≥ .20– Ha: p < .20

Forming Hypotheses

This brings us to another classification of hypotheses: one-sided or two-sided

If we want to show that a proportion is less than or greater than some value, then the hypothesis test is one-sided

If we want to show that a proportion is different from some value, then the hypothesis test is two-sided

Notation for Hypotheses

Typically, the null hypothesis for proportions is written in one of three ways H0: p = null value

H0: p ≤ null value

H0: p ≥ null value

You may notice that, in all three cases, there is always the possibility that the proportion is equal to the null value.

Notation for Hypotheses

Similar to the null hypothesis, there are three ways to write the alternative hypotheses for tests about proportions Ha: p ≠ null value

Ha: p > null value

Ha: p < null value

In all three cases, there is no possibility that the proportion is equal to the null value.

Notation for Hypotheses

• In fact, the three ways to write the null and alternative hypotheses relate directly to each other– H0: p = null value & Ha: p ≠ null value

– H0: p ≤ null value & Ha: p > null value

– H0: p ≥ null value & Ha: p < null value

• In each case, the null hypothesis and the alternative hypothesis state the exact opposite thing

Notation for Hypotheses

• Further still, each set of null and alternative hypotheses corresponds to either a one-sided, or a two-sided hypothesis test:– H0: p = null value & Ha: p ≠ null value (two-sided)

– H0: p ≤ null value & Ha: p > null value (one-sided)

– H0: p ≥ null value & Ha: p < null value (one-sided)

• The sidedness of the test depends on how many sides of the null value the alternative hypothesis describes.

Logic of Hypothesis Testing

Although it would be ideal if a statement could be made about whether the null hypothesis or the alternative hypothesis is true, that’s not exactly what hypothesis testing does

Conclusions from hypothesis tests are based on the following question: If the null hypothesis is true about the

population, what is the probability of observing the sample data that was collected?

Logic of Hypothesis Testing

• In fact, the correct technical interpretation of the p-value of a test is:– The probability that the data in the sample was

collected, given that the null hypothesis is true• This is why we typically compare the p-value to

0.05.• If the p-value is less than 0.05, then the

corresponding interpretation is that there is a less than 5% chance that we observed our sample, or a sample with a test statistic more extreme, if the null hypothesis is true.

Logic of Hypothesis Testing

• Due to our logic of hypothesis testing and the interpretation of the p-value, we have to be careful when stating our conclusion for hypothesis tests

• The two possible conclusions should be written:– We fail to reject the null hypothesis.– We reject the null hypothesis in favor of the

alternative hypothesis.

Reaching a Conclusion

The process of reaching a conclusion about a hypothesis can be broken into three parts. Compute the appropriate test statistic Find the corresponding p-value for the test

statistic Make a decision based on your selected level

of significance (also called α level)

Computing a Test Statistic

• For the Big 5 parameters, the test statistics will be either t or z

• Each test statistic is made based on the null value.

• If the population parameter is equal to the null value, then the null hypothesis is true.

Computing a Test Statistic

The generic format for find the test statistic for any of the Big 5 is as follows:

Sometimes, the standard error depends on the null value.

When this happens, the standard error is referred to as the null standard error

TestStatistic SampleStatistic NullValue

NullStdError

Incorrect Conclusions

Sometimes, a sample that was randomly selected will produce an odd result. It may produce data that cause you to fail to

reject the null hypothesis when the null hypothesis is actually not true.

It may produce data that cause you to reject the null hypothesis when the null hypothesis is actually true

Cases like these are errors

Incorrect Conclusions

When the null hypothesis is actually true, but you reject it, this is called Type 1 Error, or a false positive.

When the null hypothesis is actually not true, but you fail to reject it, this is called Type 2 Error, or a false negative.

Incorrect Conclusions

• The probability of making a Type 1 Error is actually the probability that you conclude significance when there is no significance.

• The probability that this happens is equal to the level of significance for your hypothesis test

• The level of significance is the value for the p-value at which you declare a test is statistically significant, typically 0.05.

Incorrect Conclusions

• The probability of a Type 2 Error is often of great interest to researchers

• Another way of viewing Type 2 Error is to think of it as the chance that you will not find significance if, in fact, the alternative hypothesis is actually true

• The probability of finding significance if the alternative hypothesis is true is called the power of a test, thus the probability of a Type 2 error is 1 – power.

Incorrect Conclusions

• The calculation for power is rather complicated, and it won’t be covered in this course, but there are three factors that effect power– The larger a sample size is, the higher the

power will be– The larger the level of significance is, the

higher the power will be– The farther the actual value for the population

parameter falls from the null value, the higher the power will be

Review: If you understood today’s lecture, you should be able to solve

12.1, 12.5, 12.9, 12.17, 12.19, 12.23, 12.25, 12.27, 12.29, 12.31