I’ve got the Power!

Hypothesis Tests

• When we perform a hypothesis test, we make a decision to either

Reject the Null Hypothesis or Fail to Reject the Null Hypothesis.

• There is always the possibility that we made an incorrect decision.

• We can make an incorrect decision in two ways:

1. The null hypothesis is true, but we mistakenly reject it.

2. The null hypothesis is false, but we fail to reject it.

Type I Error

• A Type I error is the mistake of rejecting the null hypothesis when it is true.

• In testing for a medical disease, the null hypothesis is usually the assumption that a person is healthy. A Type I error is a false positive; a healthy person is diagnosed with the disease.

• The probability of rejecting the null hypothesis when it is true is equal to the alpha-level of the test.

• If alpha = .05, in the long-run, we will incorrectly reject the null hypothesis when it is really true about 5% of the time.

Type II Error

• A type II error is made when we fail to reject the null hypothesis when it is false and the alternative is true. Prob(Type II Error) =

• In medical testing for a disease, this would be equivalent to a person who has the disease being diagnosed as disease free. This is called a false negative.

• Our ability to detect a false hypothesis is called the power of the test. The Power of a Test is the probability that it correctly rejects a false null hypothesis. Power = 1 - Prob (Type II Error) or

Type I and Type II Errors

Null Hypothesis True

Null Hypothesis False

Reject Null

Type I Error Probability= alpha

Correct Decision

Fail to Reject Null

Correct Decision

Type II Error Probability = beta

Suppose that a baseball player who has always been a .250 career hitter suddenly improves over one winter to the point where the probability of getting a hit during an at-bat is .333.

He asks management to renegotiate his contract, since he is a more valuable player now.

Management has no reason to believe that he is better than a .250 hitter.

Suppose they decide to give the player 20 at-bats to show that his true batting average is greater than .250.

Hypothesis Test

• H(o): p = .250 H(a): p > .250

• If alpha is set at 0.05, and the player is given 20 at-bats to show that he has improved, what does his average need to be after those 20 at-bats in order to reject the null hypothesis?

Power of the Test

If he is really a .333 hitter, how often will he have the necessary average of about .410 with n=20 at-bats, necessary to reject the null hypothesis?

Alpha = 0.05

Null Hypothesisp = 0.250

Alternative HypothesisThe Truthp= 0.333

Retain NullType II Error

Reject NullPower

The value 0.225 is the power of the test to detect the change. In other words, approximately 22.5% of the time, we will be able to correctly reject the null hypothesis in favor of the alternative hypothesis.

• A type II error is the probability of making the incorrect decision to fail to reject the null hypothesis when it is false. In this case the probability of a type II error is 0.775.

• Power = 1 - Prob(Type II Error) Power = 1 - beta

How do you increase Power?• Open the Fathom document entitled power

05.ftm.• Read About this Demo and when you are finished,

resize it so that all the sliders are visible.• You can change the values of p_null, p_alternate,

n, and alpha by dragging on the slider or by clicking on the value in blue, typing a new value and pressing return.

• The probability of a Type II Error and the power will automatically update. These sliders are not true sliders but are controlled by a formula to calculate their values.

Some questions to think about?

• How can you increase the power of the test? Explore as many different ways as possible.

• If the player wants the decision to be made after 20 at bats, is there any way to increase the power of the test?

• How are alpha and beta related?• Is there a way to reduce the probability of

both types of error?

Type I and Type II Errors

Which type of error is worse?

In assessing the weather prior to leaving home on a spring morning, we make an informal test of the hypothesis “The weather is fair today.” Making our decision based on the “best” information available to us, we complete the test and dress accordingly. What would be the consequences of a Type I and Type II error?

A)Type I error - inconvenience in carrying needless rain equipment.

Type II error - clothes get soaked.

B) Type I error - clothes get soaked

Type II error - inconvenience in carrying needless rain equipment.

Which type of error is worse?

The court is deciding a death penalty case. We work on the assumption that a person is innocent until proven guilty.

What is the consequence of a Type I error?

What is the consequence of a Type II error?

Which is worse?

We are testing parachutes. If our assumption is that the parachutes work,

What is the consequences of a Type I error?

What is the consequences of a Type II error?

Which is worse?

A drug company tests random samples of a pain killer for the level of the active ingredient. They will reject the lot if the level is above a specified value.

What is the consequences of a Type I error?

What is the consequences of a Type II error?

Which is worse?

Type I and Type II Errors and the Justice System

A web article about the U.S. Justice system and the consequences of a Type I and Type II error.

http://www.intuitor.com/statistics/T1T2Errors.html