Upload
florence-griffith
View
216
Download
0
Embed Size (px)
DESCRIPTION
Today, 3 – 4 pm (Prof Morgan) Today, 4 – 6 pm (Christine) Tuesday, 3 – 6 pm (Prof Morgan) Tuesday, 6 – 8 pm (Yue) Tuesday, 8 – 9 pm (Michael) (My office hours this week have been moved to Monday and Tuesday to answer questions before the exam) Office Hours this Week
Citation preview
Synthesis and Review 2/20/12
• Hypothesis Tests: the big picture• Randomization distributions• Connecting intervals and tests• Review of major topics• Open Q+A
Section 4.4, 4.5, ES 2 Professor Kari Lock MorganDuke University
• Make one double-sided page of notes for in-class exam
• WORK PRACTICE PROBLEMS! (old exams and solutions to review questions
under Documents on course website)
• Read sections corresponding to anything you are still confused about
• Practice using technology to summarize, visualize, and perform inference on data
To Do
• Today, 3 – 4 pm (Prof Morgan) • Today, 4 – 6 pm (Christine)• Tuesday, 3 – 6 pm (Prof Morgan)• Tuesday, 6 – 8 pm (Yue)• Tuesday, 8 – 9 pm (Michael)
(My office hours this week have been moved to Monday and Tuesday to answer questions before the exam)
Office Hours this Week
Hypothesis Testing1. Define the parameter(s) of interest2. State your hypotheses3. Set significance level, (usually 0.05 if unspecified)4. (Collect your data)5. Plot your data6. Calculate the observed sample statistic7. Create a randomization distribution8. Calculate the p-value9. Assess the strength of evidence against H0
10. Make a formal decision based on the significance level11. Interpret the conclusion in context
Exercise and GenderAmong college students, does one gender exercise more than the other?
Mean number of exercise hours per week for femalesMean number of exercise hours per week for males
F
M
0 : 0: 0
F M
a F M
HH
Exercise and Gender
3.00M FX X
Exercise and Gender
www.lock5stat.com/statkey
p-value = 0.218 Little evidence against H0
Do not reject H0
This study does not provide evidence that there is any association between gender and exercise times among college students
Conclusion:
Results this extreme would happen about 22% of the time just by random chance if H0 were true, so this study does not provide adequate evidence against H0
Think:
• A randomization distribution is the distribution of statistics that would be observed, just by random chance, if the null hypothesis were true
1. Simulate randomizations assuming the null hypothesis is true
2. Calculate the statistic for each simulated randomization
Randomization Distribution
• In a randomized experiment the “randomness” is the random allocation of cases to treatment groups
• If the null hypothesis is true, it doesn’t make any difference which treatment group you get placed in
• Simulate randomizations assuming H0 is true by reallocate units to treatment groups, and keeping the response values the same
Randomized Experiments
• In observational studies, there is no random allocation to treatment groups
• In observational studies, what does “by random chance” even mean? What is random???
• How could we generate a randomization distribution for observational studies?
Observational Studies
• When data is collected by random sampling, without random allocation between groups, we can bootstrap to see what would happen by random chance
• Bootstrapping (resampling with replacement) simulates the distribution of the sample statistic that we would observe when taking many random samples of the population
Bootstrapping
• For a randomization distribution however, we need to know the distribution of the sample statistic, when the null hypothesis is true
• How could we bootstrap assuming the null hypothesis is true?
• Add/subtract values to each unit first to make the null hypothesis true (“shift the distribution”)
Bootstrapping
Reallocating versus Resampling
What is random? How do we simulate “random chance”?
Randomized Experiments
Random assignment to treatment groups
Reallocate (rerandomize)
Observational Studies
Random sampling from the population
Resample (bootstrap)
• In both cases, we need to make the null hypothesis true for a randomization distribution
Was the exercise by gender data collected via a randomized experiment?
(a)Yes(b) No(c) There is no way to tell
Exercise by Gender
• The randomness is not who is which gender (as with randomized experiments), but who is selected to be a part of the study
• Male sample mean: 12.4 hours• Female sample mean: 9.4 hours
• Add 3 hours to all the females, and then resample using bootstrapping
• www.lock5stat.com/statkey
Exercise by Gender
• Reallocating and resampling usually give similar answers in terms of a p-value
• For this class, it is fine to just use reallocating for tests, even if it is not actually a randomized experiment
• The point is to understand the reason for generating a randomization distribution
Method of Randomization
• Let’s return to the body temperature data
• Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98.05, 98.47 )
• Let’s do a hypothesis test to see how much evidence this data provides against = 98.6
H0 : = 98.6Ha : ≠ 98.6
Body Temperatures
• How would we create a randomization distribution?
• The sample mean is 98.26. Add 0.34 to each unit so we can sample with replacement mimicking sampling from a population with mean 98.6
• Take many bootstrap samples to create a randomization distribution
Body Temperatures
Randomization Distribution
p-value = 0.002
Two Distributions
• If a (1 – α)% confidence interval does not contain the value of the null hypothesis, then a two-sided hypothesis test will reject the null hypothesis using significance level α• Intervals provide a range of plausible values for the population parameter, tests are designed to assess evidence against a null hypothesis
Intervals and Tests
• Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98.05, 98.47 )
H0 : = 98.6Ha : ≠ 98.6
• At α = 0.05, we would reject H0
Body Temperatures
REVIEW
Population
Sample
Sampling
Statistical Inference
The Big Picture
Exploratory Data Analysis
Was the sample randomly selected?
Possible to generalize to
the population
Yes
Should not generalize to
the population
No
Was the explanatory variable randomly
assigned?
Possible to make
conclusions about causality
Yes
Can not make conclusions
about causality
No
Data Collection
Variable(s) Visualization Summary StatisticsCategorical bar chart,
pie chartfrequency table,
relative frequency table, proportion
Quantitative dotplot, histogram,
boxplot
mean, median, max, min, standard deviation,
z-score, range, IQR,five number summary
Categorical vs Categorical
side-by-side bar chart, segmented bar chart,
mosaic plot
two-way table, proportions
Quantitative vs Categorical
side-by-side boxplots statistics by group
Quantitative vs Quantitative
scatterplot correlation
Descriptive StatisticsThink of a topic or question you would like to use data to help you answer.
– What would the cases be? – What would the variables be? (Limit to one or two variables)
Descriptive StatisticsHow would you visualize and summarize the variable or relationship between variables?
a) bar chart/pie chart, proportions, frequency table/relative frequency table
b) dotplot/histogram/boxplot, mean/median, sd/range/IQR, five number summary
c) side-by-side or segmented bar plots/mosaic plots, difference in proportions, two-way table
d) side-by-side boxplot, stats by groupe) scatterplot, correlation
Statistic vs Parameter
• A sample statistic is a number computed from sample data.
• A population parameter is a number that describes some aspect of a population
Sampling Distribution
• A sampling distribution is the distribution of statistics computed for different samples of the same size taken from the same population
• The spread of the sampling distribution helps us to assess the uncertainty in the sample statistic
• In real life, we rarely get to see the sampling distribution – we usually only have one sample
• A bootstrap sample is a random sample taken with replacement from the original sample, of the same size as the original sample
• A bootstrap statistic is the statistic computed on the bootstrap sample
• A bootstrap distribution is the distribution of many bootstrap statistics
Bootstrap
Original Sample
BootstrapSample
BootstrapSample
BootstrapSample
.
.
.
Bootstrap Statistic
Sample Statistic
Bootstrap Statistic
Bootstrap Statistic
.
.
.
Bootstrap Distribution
Confidence Interval
• A confidence interval for a parameter is an interval computed from sample data by a method that will capture the parameter for a specified proportion of all samples
• A 95% confidence interval will contain the true parameter for 95% of all samples
Standard Error
• The standard error (SE) is the standard deviation of the sample statistic
• The SE can be estimated by the standard deviation of the bootstrap distribution
• For symmetric, bell-shaped distributions, a 95% confidence interval is
statistic 2 SE
Percentile Method
• If the bootstrap distribution is approximately symmetric, a P% confidence interval can be gotten by taking the middle P% of a bootstrap distribution
Bootstrap DistributionBest Guess at Sampling Distribution
Statistic
2 3 4 5 6 7 8
Best Guess at Sampling Distribution
Statistic
2 3 4 5 6 7 8
Observed Statistic
Best Guess at Sampling Distribution
Statistic
2 3 4 5 6 7 8
Observed Statistic
P%
Best Guess at Sampling Distribution
Statistic
2 3 4 5 6 7 8
Observed Statistic
P%P%P%
Upper BoundUpper Bound
Lower Bound
• How unusual would it be to get results as extreme (or more extreme) than those observed, if the null hypothesis is true?
• If it would be very unusual, then the null hypothesis is probably not true!
• If it would not be very unusual, then there is not evidence against the null hypothesis
Hypothesis Testing
• The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true
• The p-value measures evidence against the null hypothesis
p-value
Hypothesis TestingDistribution of Statistic Assuming Null
Statistic
-3 -2 -1 0 1 2 3
Observed Statistic
Distribution of Statistic Assuming Null
Statistic
-3 -2 -1 0 1 2 3
Distribution of Statistic Assuming Null
Statistic
-3 -2 -1 0 1 2 3
Observed Statistic
p-value
• A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true
• The p-value is calculated by finding the proportion of statistics in the randomization distribution that fall beyond the observed statistic
Randomization Distribution
Statistical ConclusionsStrength of evidence against H0:
Formal decision of hypothesis test, based on = 0.05 :
statistically significant
not statistically significant
Formal Decisions
For a given significance level, ,
p-value < Reject Ho
p-value > Do not Reject Ho
Errors
Reject H0 Do not reject H0
H0 true
H0 false
TYPE I ERROR
TYPE II ERRORTrut
h
Decision
0If true, probability = H
0If true,
probability = 1H
If true, probability =
ap
Hower
If true, probability = 1
ap r
Howe
QUESTIONS???