Upload
francine-roberts
View
214
Download
1
Embed Size (px)
Citation preview
Statistics (and SoTL Projects)
Jim SmithMichigan State UniversityLyman Briggs College
It’s as much who you know, as what you know!
“Lies, Damned Lies, and Statistics”
A Case Study in Statistics
Use of a Stats Analytical Framework
Statistics 101
Some Basics
Which Statistical Test Should You Use in Your SoTL Work?
Some Examples
Learning Goals
By the end of this session, scholars will (have):
• Understand the need for and the role(s) of statistical tests in science education research;
• Be able to explain some basic tenets of statistics;
• Explored some literature examples of the application of statistics to SoTL work; and
• Begin to formulate a statistical framework for their Biology Scholars research project.
Statistics and SoTL Projects
Did student learning increase as a result of my cell model assignment?
Do representations in comic book form improve student understanding of photosynthetic processes?
Are there differences in the preparedness and motivation levels of students who W/D/F in Introductory Biology?
Does increased metacognition decrease the W/D/F rate in Introductory Biology?
Statistics are important components of strategies to answer these questions!
With respect to Statistics:
Where would you place yourself on the following scale?
1. From my perspective, there’s “Lies, Damned Lies, and Statistics”
2. I agree with the above statement somewhat, but not strongly.
3. I am halfway between “Damned Lies” and “Comfort”.
4. I agree with the below statement, but not strongly.
5. I am completely comfortable with statistics as they apply to SoTL work.
The Problem – Young and Fry (2008)
“While the results are reported to be statistically significant, I am not familiar enough with the Spearman’s Rho nonparametric correlation analysis to know whether the data are truly convincing. ”
ASM Bio Scholar Summer 2012
Young and Fry (2008)
Framework for Analysis of Statistical Inference
Identify Variables of Interest
Formulate
Hypotheses Design
Sampling Strategy
Collect Raw Data
Perform
Statistical Test(s)
Inference(Level 1)
Inference (Level 2+)
Tasks
• Identify the appropriate elements in the article of interest and map them onto the boxes
• Decide if the statistical test was appropriately chosen and carried out
• Decide if the claims (inferences) are supportable at each level
• Tie the framework to your project
Smith, Valles & Zeleke, in prep.
An Application – Young and Fry (2008)
What is the MAI, and how was it used to measure aspects of metacognition?
An Application – Young and Fry (2008)
What is the difference between metacognitive knowledge and metacognitive regulation?
Framework for Analysis of Statistical Inference
Identify Variables of Interest
Formulate
Hypotheses Design
Sampling Strategy
Collect Raw Data
Perform
Statistical Test(s)
Inference(Level 1)
Inference (Level 2+)
Tasks
• Identify the appropriate elements in the article of interest and map them onto the boxes
• Decide if the statistical test was appropriately chosen and carried out
• Decide if the claims (inferences) are supportable at each level
• Tie the framework to your project
Smith, Valles & Zeleke, in prep.
ASM BSP S12: STATS 101
What do I really need to know about Statistics?
ASM BSP S12: STATS 101
What do I really need to know about Statistics?
If you are going to do SoTL research, you really do need a basic working knowledge of statistical inference
Fun, but no help for today
Extremely helpful!
What the difference between:
Experimental (or Research) Hypothesis
and
Statistical Hypothesis
What the difference between:
Experimental (or Research) Hypothesis
HA: Metacognition improves student learning
and
Statistical Hypothesis
What the difference between:
Experimental (or Research) Hypothesis
and
Statistical Hypothesis
HA: Course grade is correlated with MAI total
What the difference between:
Experimental (or Research) Hypothesis
and
Statistical Hypothesis
H0: No correlation of Course grade with MAI total
The Problem – Young and Fry (2008)
The data allow us to reject the hypothesis that there is no correlation between Course Grade and MAI total.
What is the difference between significance level and effect size?
What the difference between:
Experimental (or Research) Hypothesis
HA: Metacognition improves student learning
and
Statistical Hypothesis
What the difference between:
Discrete Numbers
and
Continuous Numbers
With respect to Statistics:
Where would you place yourself on the following scale?
1. From my perspective, there’s “Lies, Damned Lies, and Statistics”
2. I agree with the above statement somewhat, but not strongly.
3. I am halfway between “Damned Lies” and “Comfort”.
4. I agree with the below statement, but not strongly.
5. I am completely comfortable with statistics as they apply to SoTL work.
What the difference between:
Discrete Numbers
Categorical; membership known without error
and
Continuous Numbers
What the difference between:
Discrete Numbers
and
Continuous Numbers
Not known without error; have a distribution
Overall: Mean = 74.70 (74.70%) +/- 11.45 SD TH: Mean = 24.47 (81.57%) +/- 3.91 SD ER: Mean = 28.96 (72.40%) +/- 5.52 SD MC: Mean = 21.27 (70.90%) +/- 4.45 SD
Discrete or Continuous?
What the difference between:
Non-parametric Statistics
For discrete data; for data that violate parametric assumptions
and
Parametric Statistics
What the difference between:
Non-parametric Statistics
e.g., Chi-squared tests, Mann-Whitney U test, Wilcoxon Rank Sums test
and
Parametric Statistics
What the difference between:
Non-parametric Statistics
and
Parametric Statistics
What the difference between:
Non-parametric Statistics
and
Parametric Statistics
e.g., t-test, ANOVA, Pearson’s r
Assumptions of Parametric Statistics(for continuous data)
Observations are unbiased and independent
Data are normally distributed
Equal variances across groups
Sta
tist
ical
Dec
isio
n T
ree
From: Robert Gerwien, “A Painless Guide to Statistics: Online Resources for Biology”, http://abacus.bates.edu/~ganderso/biology/resources/statistics.html
Sta
tist
ical
Dec
isio
n T
ree
Was the use of Spearman’s Rank Correlation test appropriate?
Work with your group
Yes/No, and why
Young and Fry: Revisited
Young and Fry: Was theirs the appropriate statistical test?
REM - You will find disagreement in the ranks (including among manuscript reviewers)
Test Time: What kind of data? Test?
Encouragingly, using t test analysis, we found that in four of our courses (both BSCI 223 courses, BSCI 424, and BSCI 422) there was significant improvement on the concept inventory scores from presurvey to postsurvey.
From Marbach-Ad et al. 2009
Test Time: What kind of data? Test?
From Drew & Triplett 2008
Test Time: What kind of data? Test?
From Steck et al. 2012
Test Time: What kind of data? Test?
From Steck et al. 2012
Test Time: What kind of data? Test?
From Steck et al. 2012
“I’m baffled…”
You may (probably) need help!!
Become friends/colleagues with a really smart biologist or social scientist who uses a statistical approach in their own work
• A lot of these folks have strong experimental design skills
• Ecologists and Evolutionary Biologists seem to be particularly strong
On-campus statistical consulting service?