Upload
aria
View
40
Download
0
Tags:
Embed Size (px)
DESCRIPTION
A Causal Rasch Model for Understanding Comprehension in the Context of Reader-Text-Task. AERA/NCME April 13-17, 2012 Vancouver, Canada A. Jackson Stenner Donald S. Burdick Mark H. Stone. - PowerPoint PPT Presentation
Citation preview
1
A Causal Rasch Model for Understanding Comprehension in the Context of Reader-
Text-TaskAERA/NCME April 13-17, 2012
Vancouver, Canada
A. Jackson StennerDonald S. Burdick
Mark H. Stone
2
Reading is a process in which information from the text and the
knowledge possessed by the reader act together to
produce meaning as measured by a particular task.
Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985) Becoming a nation of readers: The report of the Commission on Reading Urbana, IL: University of Illinois
3
Data Structure for Testing the Lexile Theory
a b c d e f g h i j k l m … N123456789
10111213…N
Reader-text-task transaction produces an outcome which can be viewed as reading comprehension and/or text comprehensibleness usefully presented as a percent.
Texts (Native Task)
Read
ers
Empirical Text Complexity Measures
New Task Types
Read
er A
bilit
y M
easu
res
4
A Causal Rasch Model
= Reader Ability
Text Complexity
Comprehension
-Conceptual
Statistical
RawScore
=i
e (RA – TC - TD)i
1 + e (RA – TC i - TD)
RA = Reading AbilityTC = Text ComplexityTD = Task Difficulty
Task Difficulty
-
5
200L
1700L 200L
1700L
-300L 300L
Reader AbilityDial
Text ComplexityDial Task Difficulty
Dial
0
ComprehensionDisplay
72%
The Measurement Trade-off Property
6
Imagine a world where assessment items are generated
and scored in real-time.
7
Theoretical versus Empirical Text Complexity for
719 Articles*
Reliability = 0.997
SEM = 12.8L
r = 0.968
r” = 0.969
R2” = 0.938
RMSE” = 89.6L
* Inclusion criteria: 50 encounters and 1,000 items
Mean Theoretical = 884.4L (356.2)
Mean Empirical = 884.4L (355.0)
8
Artifactual Sources of Variance in Empirical Text complexity Measures1. Random measurement error2. Sampling error3. Range restriction4. Systematic error in empirical complexity measures5. Wrong function form (not linear)6. Variation in empirical text complexities across
estimation algorithms
We have estimated that the first three of these artifactual sources of variance account for no more than 4% of the total variance in the system – leaving 2% still unexplained. Sources 4-6 may account for this remaining 2%.
9
May 2007 – April 2011347 Encounters138,695 Words3,342 Items983 Minutes
Student 15287th GradeMaleHispanicPaid Lunch
Text Demands forCollege and Career
1200
1000
1400
1600
May 2016(12th Grade)
10
Conclusions1. There is only a very small proportion of
variation in empirical text complexity left unexplained.
2. None of the hypotheses about genre (expository vs. narrative), coherence, cohesion, grade dependence, gender dependence, second language dependence have been supported.
3. It is possible that the small amount of unexplained variance is due to artifacts in the estimation of empirical text complexity.
11
A. Jackson Stenner Chairman & CEO, MetaMetrics
University of North Carolina, Chapel [email protected]
Contact Info: