Upload
baldwin-bruce
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Instruction, Teacher Evaluation and Value-Added Student Learning
Minneapolis Public SchoolsNovember, 2002
www.mpls.k12.mn.us/REA/
Caution! Do not use the Federal Framework
for designating Adequate Yearly Progress to evaluate the effectiveness of Instruction: It won’t work!
Clinical studies have shown that AYP can be injurious to school, teacher and student long-term health.
Minneapolis Public SchoolsMeasuring Up Against High Standards
MPS developed standards-based reading and math tests - Northwest Achievement Levels Tests (NALT).
All students participate Sensitive to growth Predictive of high stakes outcomes
Developed indicators in concert with stakeholders who are internal and external to the district.
Indicators of school performance are based on multiple indicators. 33 for elementary and middle schools 24 for high schools
Linking Assessments to High Standards:
Choose items for the assessment bank which directly link to the standards. Use your teacher experts, curriculum specialists, and assessment personnel to create the bank
“All Means ALL” - create levels of the assessment which allow for 98% of students being assessed
Find out where the State benchmarks locate on the scale (linking studies)
Measure continuous progress towards the Standard
Elementary/Middle QPA 33 Indicators
37%
33%
6%
6%
6%
6%6%
Reading
Math
Writing
Behavior
Climate
Attendance
Programs
Criteria for Indicators Meyer (1996)
Indicators must measure things that matter or are valued by society (MCA, MBST).
Indicators must be able to avoid being “corrupted.”
inclusion v. exclusion criteria Indicators must be able to measure the
unique contribution schools “add” for each measured outcome.
Linkage with State Standards
NALT Scale Score Which Predicts Passage of theMinnesota Basic Standards Test (MBST) in 8th Grade
Grade Reading Scale Score Reading Scale Score
3 190 1994 197 2065 204 2146 209 2187 213 2258 218 232
True gain model: Growth Curves
Class of 2005 Longitudinal Reading Gains
190
200
210
220
230
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11
Grade 12
NA
LT
Sc
ale
Sc
ore
(R
as
ch
Mo
de
l)
High Standard
Basic Standard
Class of 2005 (n=2006)
Improving Student Performance One Student at a Time
180
190
200
210
220
230
240
250
Grade 2
Grade 4
Grade 6
Grade 8
Grade 10
Grade 12
Josh (AI)
Fahim (AA)
High Standard
Basic Standard
Figure 22. Longitudinal NALT Reading GainsClass of 2007
Figure 22. Longitudinal NALT Reading GainsClass of 2007
180
190
200
210
220
230
240
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Me
an
Sca
le S
core
High Standard
Basic Standard
Class of 2007(n = 2058)
Correlation of Poverty and Math NALT ScoresR2 = 0.7901
30
40
50
60
70
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percent free or reduced price lunch
NA
LT M
ath
Ave
rage
NC
E
School P = 44.4 nce
Distribution of school average mathematics gain scores on the 1999 NALT by free or reduced price
lunch percentages.
R2 = 0.0074
-3-2-10123456789
10
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Percent free or reduced price lunch
NA
LT
Mat
h N
CE
Gai
n
School P = 6.3 nce gain
Percent of Students Making About One Year Growth or More on NALT
Math by Racial/Ethnic Category
51% 48%53% 53%
60%58% 56%
66% 63% 64%
0%
20%
40%
60%
80%
100%
AmericanIndian
AfricanAmerican
AsianAmerican
HispanicAmerican
WhiteAmerican
Per
cent
age
of S
tude
nts
2000
2001National Norm = 57%
Characteristics of “Needs Improvement Schools” in
MPS One school in this category two
years in a row: Student Stability 78% Staff Stability 33% (14/21 new staff) Three principals in three years Low parent choice Higher Poverty (84%) Large ELL Population (32%)
Characteristics of “Distinguished School” in
MPS One school in this category two
years in a row: Student Stability 94% Staff Stability 87% (3/23 new staff) One principals in three years High parent choice (waiting list) Lower Poverty (40%) Moderate ELL Population (20%)
Value-added Model
Primary question is this, “How can an equitable accountability system be put in place when there is a large degree of variability in the way student characteristics are distributed among our schools?”
English language learners Special education programs Race/ethnicity Poverty
MPS relies on an empirical- and literature-based model to make predictions, not set expectations!
Schools are acknowledged/rewarded for “Beating the Odds.”
Value-added Teacher Effects
Post-test reading score= Pretest reading score + Free or reduced price lunch + Racial/ethnic code + Neighborhood poverty concentration + Lives with both parents code + Limited English Proficiency status + Special Education status + Teacher effects
Teachers who “beat the odds” in second grade reading reported:
more use of small group instruction more development of word attack
skills more individual student oral reading more guidance during initial practice more explicit and direct phonics
instruction more use of systematic motivation
strategies
Measurement Recommendations Measure students on standards-based
assessments as soon as they enter school Continuously monitor student progress
towards the standards Produce individual student growth curves Use data to monitor the effectiveness of
interventions Identify schools and teachers who make
exceptional progress with “at risk” students
Intervention Recommendations
Involve exceptional teachers in staff development (modeling,video tape, etc.)
Give incentives for exceptional teachers to work in the highest need schools
Place staff development in the schools with “hands on” modeling and support
Identify teaching practices that correlate with success and replicate them
Reward groups of teachers (e.g. schools) who “beat the odds”
Number of Words Read Correctly per minute End of Kindergarten 2002
0
5
10
15
Percent of Students
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95
Number of Words Correct per Minute
Note: 53% of all Kindergarten students read10 or more words correctly in Spring 2002
Letter Sounds Gain from Fall to Spring for Half Day vs. Full Day Kindergarten
05
10152025303540
Number of Letter Sounds
Gain
AfricanAmerican
AmericanIndian
Asian Hispanic White Total
Half Day Full Day
Kindergarten Letter Sounds Gains from Fall to Spring by Length of Day and Intervention Group
05
1015202530354045
Number of Letter Sounds
Gain
AfricanAmerican
AmericanIndian
Asian Hispanic White Total
Half Day - Partial All Day - Full Intervention
Reading Fluency Value-Added
0
0.2
0.4
0.6
0.8
1
Effect Size (in standard
deviation units)
Full DayKindergarten
Full Intervention Full Day + FullIntervention
All Students African American White Hispanic Asian