Upload
michel
View
19
Download
0
Embed Size (px)
DESCRIPTION
Moving Away From Aggregate Statistics: Observation Oriented Modeling and Assessement. Lisa D. Cota, MS Statistical Analyst Office of University Assessment & Testing Oklahoma State University. Why not NHST?. - PowerPoint PPT Presentation
Citation preview
Moving Away From Aggregate Statistics: Observation Oriented Modeling and
Assessement
Lisa D. Cota, MSStatistical Analyst
Office of University Assessment & TestingOklahoma State University
Why not NHST?• Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.
Retrieved from http://psycnet.apa.org/journals/amp/49/12/997/• Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students
share with their teachers. Methods of Psychological Research, 7(1), 1–20. Retrieved from http://www2.uni-jena.de/svw/metheval/lehre/0405-ws/evaluationuebung/haller.pdf
• Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review. doi:10.3758/s13423-013-0572-3
• Lamiell, J. T. (2013). Statisticism in personality psychologists’ use of trait constructs: What is it? How was it contracted? Is there a cure? New Ideas in Psychology, 31(1), 65–71. doi:10.1016/j.newideapsych.2011.02.009
• Lykken, D. What’s wrong with psychology anyway? (1991). In Thinking Clearly About Psychology. University of Minnesota Press.
• Toomela, A. (2010). Quantitative methods in psychology: Inevitable and useless. Frontiers in Psychology, 1(July), 29. doi:10.3389/fpsyg.2010.00029
Why not NHST?
• Assessment should be understandable– NHST: Not easily understood…in fact easily
misunderstood• Oakes, 1986; Gigerenzer, 2004• Hoekstra, Morey, Rouder, & Wagonmaker, 2014
• One problem (of many) with NHST:– Variance Explained ≠ Accuracy
y = 0.5x + 3, R2 = 0.663
y = 0.5x + 3, R2 = 0.663
y = 0.5x + 3, R2 = 0.663
y = 0.5x + 3, R2 = 0.663
Assumptions
• Assumptions for independent samples t-test under NHST:– Null hypothesis is true– Observations are independent between and
within groups– Population variances are exactly equal– Random sampling / random assignment (???)* Continuous DV* DV has a normal population distribution
Observation Oriented Modeling
• No meaningless aggregates• Person-centered • No assumption-laden p-values
Observation Oriented Modeling: Ordinal Pattern Analysis
Results
Randomization Test
Results, with chance-value
VALUE Rubric Data
Results
Significant ≠ Meaningful
Significant ≠ Meaningful
Significant ≠ Meaningful
Transparency, Honesty, and Progress
Raw Numbers
Proportions
Compare w. SPSS….
SPSS Legacy Dialog, k Independent Samples
Proportions
Advantages of OOM
• Assumption-free• Transparent• Honest
OOM website
• “We’ve never actually looked at the data!” …before now!
• www.idiogrid.com/OOM