Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
5-talk symposium on DRDP
Symposium title: Supporting Children’s Kindergarten and Early Learning Goals with a Standards-Aligned Assessment -Effective Practices in Statewide Aggregate Reporting
1. Policy implications
2. The correspondence between the foundations/standards and DRDP
3. This talk– use of DRDP data by states4. the Criterion Zone Boundary process
5. DRDP reports
The Use of DRDP Data for Statewide
Aggregate Reporting and Data-Based
Decision Making
Josh Sussman and Perman Gochyyev
BEAR Center
UC Berkeley
Objective
Demonstrate how the DRDP – designed as a formative assessment --also produces valid and reliable information that supports the needs of states for
1. Information about child development and kindergarten readiness (using Criterion Zone Boundaries)
2. Disaggregated data
3. Actionable evidence to support data based policymaking
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
• Technical infrastructure & data fidelity (Ackerman, 2018)
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
Analysis &
Interpretation
• Technical infrastructure & data fidelity (Ackerman, 2018)
• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)
• Technical skills
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
Analysis &
Interpretation
Data Use
• Technical infrastructure & data fidelity (Ackerman, 2018)
• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)
• Technical skills
• Clarity on what the data can be used for (Shen & Cooley, 2008)
• Data leadership (Roegman et al., 2018)
Remainder of the presentation
I. Features of DRDP measurement that support data use
II. DRDP data for understanding, reporting, and accountability
III. Additional possibilities for DRDP data to support state-level DBDM
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Designed as a formative
assessment (e.g., Akers,
2014; Black & Wiliam,
1998)
Criterion Zone Boundaries:
(1) Information about
students’ readiness for kindergarten
(2) Information to support
state-level policymaking
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Designed as a formative
assessment (e.g., Akers,
2014; Black & Wiliam,
1998)
Reporting
• What percent of eligible kindergarteners are ready for kindergarten?• What about disaggregated groups
Data analysis
• 1.5 million cases of simulated DRDP data “based on a true story”
wletc’ = wletc + r + s, where
r = N(0.4, 1)
s = 0.4
• Panel data, unbalanced, cross-nested, complex (e.g., students join and leave)
Statewide information and reporting
What percent of eligible
kindergarteners are ready for
kindergarten?
• Use of criterion zone boundaries to define K-
readiness
III. More advanced analytics has the
possibility to support state-level DBDM
1. At what age does inequality begin and in which domains is it greatest?
2. Which types of preschool programs best support student development?
3. How much longer, on average, does it take for students classified as ELLs to become K-ready than those classified as proficient?
Average development of K-eligible children
Do
ma
in S
core
When does equity begin to be a concern?
When does equity begin to be a concern?
When does equity begin to be a concern?
800
300
Lan
gu
ag
e a
nd
Lit
era
cy D
ev
elo
pm
en
t
Infancy
End of K
570
i1
i2
i3
i4
When does equity begin to be a concern?
800
300
Lan
gu
ag
e a
nd
Lit
era
cy D
ev
elo
pm
en
t
Infancy
End of K
570
i1
i2
i3
i4
Which types of preschool programs best support
child development?
Advanced analytics can help contextualize
gaps in an actionable metric
• Survival models: how much longer do ELLs, a group who are known to struggle, take to become K-ready?
• The answers have the promise to inform policy decisions about the timing and intensity of interventions
Survival models in education
• It is about time• From whether to when• How many months does it take for children to become kindergarten-ready?
• Why?• The observations are censored
• Some kids may not be categorized as K-ready within the period of data collection• If a child reached K-ready level after the data collection, researchers will not know that
• Singer & Willett (1991):• We believe that researchers avoid asking questions about time-to-event ("When?") because of
methodological difficulties introduced when members of the sample do not experience the target events during the data collection period.
• One approach: dichotomize at a particular time and model the binary outcome• Does not eliminate censoring• Does not answer: how long does it take for an average child to reach• How to incorporate time-varying covariates
Survival models: K-readiness
• Time to K-readiness (instead of K-readiness itself)• We can also identify factors predicting students’ K-readiness
• ELL vs. non-ELL: differences in proportions of K-ready children• if ELL status is varies across time, survival models will accommodate that • Investigate gaps between ELLs and non-ELLs across ages as a function of probability of being K-
ready?
• Hazard probability: conditional probability that a child will become K-ready in that month, given that he or she stayed “not K-ready” up to that point.
• A chronological summary of the probability of becoming K-ready• States (“K-ready”) can be absorbing or recurring• Accommodate ”delayed entry:” kids becoming parts of the “snapshot” observation at different
ages
• “Of all the survival methods available, we believe that discrete-time survival analysis offers the most promise for exploring educational transitions.” (Willett and Singer, 1991)