View
215
Download
2
Category
Preview:
Citation preview
Part 13: Statistical Tests – Part 113-1/37
Statistics and Data Analysis
Professor William Greene
Stern School of Business
IOMS Department
Department of Economics
Part 13: Statistical Tests – Part 113-2/37
Statistics and Data Analysis
Part 13 – Statistical Tests: 1
Part 13: Statistical Tests – Part 113-3/37
Statistical Testing
Methodology: The scientific method and statistical testing
Classical hypothesis testing Setting up the test Test of a hypothesis about a mean Other kinds of statistical tests
Mechanics of hypothesis testing Applications
Part 13: Statistical Tests – Part 113-4/37
Classical Hypothesis Testing
The scientific method applied to statistical hypothesis testing Hypothesis: The world works according to my hypothesis Testing or supporting the hypothesis
Data gathering Rejection of the hypothesis if the data are inconsistent with it Retention and exposure to further investigation if the data are
consistent with the hypothesis Failure to reject is not equivalent to acceptance.
Part 13: Statistical Tests – Part 113-5/37
http://query.nytimes.com/gst/fullpage.html?res=9C00E4DF113BF935A3575BC0A9649C8B63
Part 13: Statistical Tests – Part 113-6/37
Methodology
The standard approach would be to hypothesize that there is no link and seek data (evidence) that are (is) inconsistent with the hypothesis.
That is the way the NCI usually carries out an investigation.
This one was different.
Part 13: Statistical Tests – Part 113-7/37
Errors in Testing
Correct Decision
Type II Error
Type I ErrorCorrect
Decision
Hypothesis is Hypothesis is True False
I Do Not Reject the Hypothesis
I Reject the Hypothesis
Part 13: Statistical Tests – Part 113-8/37
A Legal Analogy: The Null Hypothesis is INNOCENT
Correct DecisionType II Error
Guilty defendant goes free
Type I ErrorInnocent defendant is
convictedCorrect Decision
Null Hypothesis Alternative Hypothesis Not Guilty Guilty
Finding: Verdict Not Guilty
Finding: VerdictGuilty
The errors are not symmetric. Most thinkers consider Type I errors to be more serious than Type II in this setting.
Part 13: Statistical Tests – Part 113-9/37
(Worldwide) Standard Methodology
“Statistical” testing Methodology
Formulate the “null” hypothesis Decide (in advance) what kinds of “evidence”
(data) will lead to rejection of the null hypothesis. I.e., define the rejection region)
Gather the data Carry out the test.
Part 13: Statistical Tests – Part 113-10/37
Formulating the Hypothesis
Stating the hypothesis: A belief about the “state of nature” A parameter takes a particular value There is a relationship between variables And so on…
The null vs. the alternative By induction: If we wish to find evidence of
something, first assume it is not true. Look for evidence that leads to rejection of
the assumed hypothesis.
Part 13: Statistical Tests – Part 113-11/37
Terms of Art
Null Hypothesis: The proposed state of nature
Alternative hypothesis: The state of nature that is believed to prevail if the null is rejected.
Part 13: Statistical Tests – Part 113-12/37
Example: Credit Rule
Investigation: I believe that Fair Isaacs relies on home ownership in deciding whether to “accept” an application. Null hypothesis: There is no relationship Alternative hypothesis: They do use
homeownership data. What decision rule should I use?
Part 13: Statistical Tests – Part 113-13/37
Some Evidence
= Homeowners
48% of cardholders are homeowners.
38% of nonholders are homeowners.
Part 13: Statistical Tests – Part 113-14/37
The Rejection Region
What is the “rejection region?” Data (evidence) that are
inconsistent with my hypothesis Evidence is divided into two types:
Data that are inconsistent with my hypothesis (the rejection region)
Everything else
Part 13: Statistical Tests – Part 113-15/37
Application: Breast Cancer On Long Island
Null Hypothesis: There is no link between the high cancer rate on LI and the use of pesticides and toxic chemicals in dry cleaning, farming, etc.
Neyman-Pearson Procedure Examine the physical and statistical evidence If there is convincing covariation, reject the null hypothesis What is the rejection region?
The NCI study: Working hypothesis: There is a link: We will find the
evidence. How do you reject this hypothesis?
Part 13: Statistical Tests – Part 113-16/37
Formulating the Testing Procedure
Usually: What kind of data will lead me to reject the hypothesis?
Thinking scientifically: If you want to “prove” a hypothesis is true (or you want to support one) begin by assuming your hypothesis is not true, and look for plausible evidence that contradicts the assumption.
Part 13: Statistical Tests – Part 113-17/37
Hypothesis Testing Strategy
Formulate the null hypothesis Gather the evidence Question: If my null hypothesis were
true, how likely is it that I would have observed this evidence? Very unlikely: Reject the hypothesis Not unlikely: Do not reject. (Retain the
hypothesis for continued scrutiny.)
Part 13: Statistical Tests – Part 113-18/37
Hypothesis About a Mean
I believe that the average income of individuals in a population is (about) $30,000. H0 : μ = $30,000 (The null)
H1: μ ≠ $30,000 (The alternative)
I will draw the sample and examine the data. The rejection region is data for which the
sample mean is far from $30,000. How far is far????? That is the test.
Part 13: Statistical Tests – Part 113-19/37
Application
The mean of a population takes a specific value:
Null hypothesis: H0: μ = $30,000H1: μ ≠ $30,000
Test: Sample mean close to hypothesized population mean?
Rejection region: Sample means that are far from $30,000
Part 13: Statistical Tests – Part 113-20/37
Deciding on the Rejection Region
If the sample mean is far from $30,000, I will reject the hypothesis. I choose, the region, for example, < 29,500 or > 30,500
The probability that the mean falls in the rejection region even though the hypothesis is true (should not be rejected) is the probability of a type 1 error. Even if the true mean really is $30,000, the sample mean could fall in the rejection region.
29,500 30,000 30,500
Rejection Rejection
Part 13: Statistical Tests – Part 113-21/37
Reduce the Probability of a Type I Error by Making the Rejection Region Smaller
28,500 29,500 30,000 30,500 31,500
Reduce the probability of a type I error by moving the boundaries of the rejection region farther out.
You can make a type I error impossible by making the rejection region very far from the null. Then you would never make a type I error because you would never reject H0.
Probability outside this interval is large.
Probability outside this interval is much smaller.
Part 13: Statistical Tests – Part 113-22/37
Setting the α Level
“α” is the probability of a type I error Choose the width of the interval by choosing the
desired probability of a type I error, based on the t or normal distribution. (How confident do I want to be?)
Multiply the corresponding z or t value by the standard error of the mean.
Part 13: Statistical Tests – Part 113-23/37
Testing Procedure The rejection region will be the range of
values greater than μ0 + zσ/√N orless than μ0 - zσ/√N
Use z = 1.96 for 1 - α = 95% Use z = 2.576 for 1 - α = 99% Use the t table if small sample and
sampling from a normal distribution.
Part 13: Statistical Tests – Part 113-24/37
Deciding on the Rejection Region
If the sample mean is far from $30,000, reject the hypothesis.
Choose, the region, say,
Rejection Rejection
$30,000 1.96N
$30,000 1.96
N
I am 95% certain that I will not commit a type I error (reject the hypothesis in error). (I cannot be 100% certain.)
Part 13: Statistical Tests – Part 113-25/37
The Testing Procedure (For a Mean)
0
0
0
Reject if x > 1.96N
or x - > 1.96N
x - or > 1.96
/ N or z > 1.96
0
0
0
Reject if x < -1.96N
or x - < -1.96N
x - or < -1.96
/ N or z < -1.96
x - 30,000Reject if 1.96
/ N
Part 13: Statistical Tests – Part 113-26/37
The Test Procedure
Choosing z = 1.96 makes the probability of a Type I error 0.05.
Choosing z = 2.576 would reduce the probability of a Type I error to 0.01.
Part 13: Statistical Tests – Part 113-27/37
What to use for σ?
The known value if there is one The sample estimate if random sampling.
Part 13: Statistical Tests – Part 113-28/37
Application
0H : = $30,000
N = 13,444 (Huge sample. t is the same as normal)
x = $30,144.3 (Is this far from $30,000?)
s = $15035.5
$30114.3 - $30,000t = = 0.881
$15035.5/ 13,444
The rejection region is |t| > 1.96.
Do not reject the hypothesis.
Part 13: Statistical Tests – Part 113-29/37
Part 13: Statistical Tests – Part 113-30/37
If you choose 1-Sample Z… to use the normal distribution, Minitab assumes you know σ and asks for the value.
Part 13: Statistical Tests – Part 113-31/37
Specify the Hypothesis Test
Minitab assumes 95%. You can choose some other value.
Part 13: Statistical Tests – Part 113-32/37
The Test Results (Are In)
2NNi 1 ii 1 i
x xx sMean x , StDev=s= , SE Mean=
N N 1 N
sx 1.96
N
Part 13: Statistical Tests – Part 113-33/37
An Intuitive Approach
Using the confidence interval The confidence interval gives the range of plausible values.
If this range does not include the null hypothesis, reject the hypothesis.If the confidence interval contains the hypothesized value, retain the hypothesis.
Includes $30,000.
Part 13: Statistical Tests – Part 113-34/37
The P value
The “P value” is the probability that you would have observed the evidence that you did observe if the null hypothesis were true.
If the P value is less than the Type I error probability (usually 0.05) you have chosen, you will reject the hypothesis.
Part 13: Statistical Tests – Part 113-35/37
Insignificant Results
The test results are “significant” if the P value is less than α.
These test results are “insignificant” at the 5% level.
This is 1 – α.
Part 13: Statistical Tests – Part 113-36/37
Application: One sided test of a mean
Hypothesis: The mean is greater than some value Business application: Does a new machine that we
might buy produce grommets faster than the one we have now? H0: μ ≤ M (where M is the mean for the old machine.)
H1: μ > M Rejection region: Mean of a sample of production rates
from the new machine is far above M. Buy the new machine,
Academic Application: Do SAT Test Courses work? Null hypothesis: The mean grade on the do-overs is less
than the mean on the original test. Reject means the do-over appears to be better.
Part 13: Statistical Tests – Part 113-37/37
Summary
Methodological issues: Science and hypothesis tests
Standard methods: Formulating a testing procedure Determining the “rejection region”
Many different kinds of applications
Recommended