Joint Meeting of the Commissioner’s and AYP Task Force
October 14, 2010NH DOE
1Joint Task Force Meeting: October 14, 2010
Overview of Key Policy Decisions• Subgroups—SWD, Low SES, “other”, & Whole School • Minimum n—5 • ELL performance
– Reading—progress towards English language proficiency as determined by ACCESS scores (AMAO 1)
– Math—use NECAP scores• Participation rate—rated as “met” or “not met” for every
subgroup (above) and every test. “Met” = 95% • K, K-1,K-2 Schools—Must participate in Level 2• High school indicators• Content areas for inclusion in the performance system—
reading, math, writing, science• Proposed cutscores for growth, achievement, and total
system
2Joint Task Force Meeting: October 14, 2010
Participation• The distributions we looked at two weeks ago indicate
that essentially all schools/subgroups meet the 95% threshold
• However, both groups felt strongly that it still be included as both a signal and reward
• Therefore, we suggest awarding each school points for each subgroup that meets the participation target and not giving any points for subgroups not meeting the target.
• Proposal: • For each subgroup meeting 95% criterion = 1 point• For each subgroup not meeting 95%=0 points
Joint Task Force Meeting: October 14, 2010 3
Review of Subgroup Performance
• Switch to PDF slides/ bubble charts
Joint Task Force Meeting: October 14, 2010 4
Individual Targets• As we discussed in May, individual targets
should (must) be created, evaluated, and reported– The group decided to establish individual student
targets for students currently below proficient, to reach proficient in 3 years or less or by 8th grade (whichever is first), while proficient/advanced students stay above proficient
– The target is based on a defined and meaningful criterion (proficient) and can be used in the aggregate to establish school and subgroup targets
Joint Task Force Meeting: October 14, 2010 5
Aggregate Criterion Targets• Similar to aggregating the observed student
growth percentiles, we can aggregate the targets for all of the students in the school/subgroup and find the median– We can then compare the median of all of the
observed growth percentiles with the median of the targets
Joint Task Force Meeting: October 14, 2010 6
Norm-referenced growth still counts
• Schools with a lot of high achieving students will have relatively low aggregate targets so that low observed median growth percentiles could still allow schools to meet targets
• Colorado required schools, in order to be classified in one of the higher rubric categories, to still have a relatively modest median growth percentile
Joint Task Force Meeting: October 14, 2010 7
Joint Task Force Meeting: October 14, 2010 8
A rubric-based approach• As seen on the following slide, a rubric is used to
“score” growth• We would also establish rubrics for the other
indicators, such as status, attendance, graduation, etc.– We would also do these rubric ratings for subgroups
• We could then aggregate these rubric scores into the major classifications of inclusion, status, “gaps”, and “readiness”
• We could, but not sure if we would want to, aggregate across all rubric scores into a single composite– Or we could make adequacy decisions without creating a
single composite
Joint Task Force Meeting: October 14, 2010 9
Growth Rubric with Cut Scores for Median SGPs (based on CO, but slightly different)
4 (rubric score)
3
2
1
Yes No
55-69
Did median SGP exceed target SGP?
45-55
56-9970-99
40-54
30-44
1-391-29
10Joint Task Force Meeting: October 14, 2010
Checking the values
• But, how do we know the growth percentiles demarking each rubric category are the right ones?
• Well, there is no “right”, but it needs to make sense and appear fair….
Joint Task Force Meeting: October 14, 2010 11
Percentages of schools in each rubric category: Math
Joint Task Force Meeting: October 14, 2010 12
Whole School Low SES SWD All Other
Rubric Score
Did Not Meet
Met Target
Did Not Meet
Met Target
Did Not Meet
Met Target
Did Not Meet
Met Target
4 2.94 32.81 0.00 34.16 0.79 51.28 0.00 39.76
3 0.00 39.38 2.78 33.66 12.70 35.90 0.00 38.28
2 32.35 26.56 33.33 29.70 38.89 12.82 0.00 21.36
1 64.71 1.25 63.89 2.48 47.62 0.00 100.00 0.59
Percentages of schools in each rubric category: Reading
Joint Task Force Meeting: October 14, 2010 13
Whole School Low SES SWD All Other
Rubric Score
Did Not Meet
Met Target
Did Not Meet
Met Target
Did Not Meet
Met Target
Did Not Meet
Met Target
4 0.00 27.17 0.00 27.49 0.00 50.00 0.00 39.48
3 0.00 45.09 0.00 28.29 6.36 36.11 0.00 37.46
2 9.09 26.88 5.00 38.65 39.09 13.89 0.00 21.04
1 90.91 0.87 95.00 5.58 54.55 0.00 100.00 2.02
How about Science & Writing?
• We’ll look at some data and proposed cutscores for science and writing for elementary-middle school
Joint Task Force Meeting: October 14, 2010 14
Elementary-Middle School Science
Joint Task Force Meeting: October 14, 2010 15
Percent Proficient & Advanced in Science
Whole School ELL SWD Low SES All Other
N Valid 369 146 355 331 348
Missing 16 239 30 54 37
Mean 40.49 18.53 20.45 34.15 51.11
Median 41.00 0.00 14.30 33.30 52.55
Std. Deviation 20.93 32.90 21.68 24.76 21.38
Minimum 0.00 0.00 0.00 0.00 0.00
Maximum 100.00 100.00 100.00 100.00 100.00
Percentiles 25 24.45 0.00 2.50 14.30 33.30
50 41.00 0.00 14.30 33.30 52.55
75 55.80 29.78 33.30 50.00 67.45
Elementary-Middle School Science• What’s the lowest percent
proficient/advanced we can tolerate for “level 3”? 40%, 50%?
• Suggest as a starting point:– Level 1- 0-20% – Level 2 - 21-40%– Level 3 - 41-65%– Level 4 - 66-100%
Joint Task Force Meeting: October 14, 2010 16
Elementary-Middle School Science Rubric Score Distributions
Joint Task Force Meeting: October 14, 2010 17
Rubric Score
Whole School ELL Low SES SWD All Other
4 12.5 4.6 10.0 4.1 27.1
3 39.2 3.3 22.2 9.8 37.4
2 29.8 3.0 27.4 24.7 22.7
1 18.5 89.2 40.4 61.4 12.7
Elementary-Middle School Writing
Joint Task Force Meeting: October 14, 2010 18
Percent Proficient & Advanced in Writing
Whole School ELL SWD Low SES All Other
N Valid 350 121 337 283 305
Missing 35 264 48 102 80
Mean 50.68 26.94 16.89 53.02 66.46
Median 53.45 0.00 11.40 50.00 68.40
Std. Deviation 21.42 37.32 19.33 24.51 16.42
Minimum 0.00 0.00 0.00 0.00 14.30
Maximum 100.00 100.00 100.00 100.00 100.00
Percentile 25 39.03 0.00 0.00 39.20 57.00
50 53.45 0.00 11.40 50.00 68.40
75 65.53 50.00 25.00 66.70 77.80
Elementary-Middle School Writing• Suggest as a starting point:
– Level 1- 0-30% – Level 2 - 31-50%– Level 3 - 51-65%– Level 4 - 66-100%
Joint Task Force Meeting: October 14, 2010 19
Elementary-Middle School Writing Rubric Score Distributions
Joint Task Force Meeting: October 14, 2010 20
Rubric Score
Whole School ELL Low SES SWD All Other
4 14.4 5.7 16.9 2.3 39.2
3 41.2 1.4 21.8 1.7 33.4
2 28.2 4.6 29.8 12.4 11.8
1 16.2 88.3 31.5 83.6 15.6
Let’s look at some actual school results to evaluate the rubric cutscores
(see Word doc—an excerpt below)
Joint Task Force Meeting: October 14, 2010 21
School NameBakersville School
Bicentennial Elementary School
Campton Elementary School
Dr. Norman W. Crisp School
Math_Whole Sch MEDIAN_SGP 59.5 61.0 61.5 28.0Math_Whole Sch Target MED_SGP 51.0 16.0 37.0 41.5Math_”Met Target?”WS 1 1 1 0Math_Growth_Level_WS 4 4 4 1Math Whole School %prof& adv 53.5 90.3 63.0 51.2Read_Whole Sch MEDIAN_SGP 50.0 58.0 55.0 26.0Read_Whole Sch Target MED_SGP 43.0 16.0 23.0 32.0Read_”Met Target?”WS 1 1 1 0Read_Growth_Level_WS 3 4 3 1Read Whole Sch %prof& adv 58.3 90.8 77.5 54.8
Points• Reading “growth”
– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4
• Math “growth”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4
• Science “status”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4
• Writing “status”– Whole school = 4– Econ Disadvantaged = 4– SWD = 4– All Others = 4
• Participation rate• Attendance
Joint Task Force Meeting: October 14, 2010 22
Weights• Reading and math indicators include both growth
and status– Should they then count twice as much as science &
writing?• Reading and math are tested 3x more than
science and writing– Should they count 3x more than science & writing?
• So should reading and math count 6x more than science and writing?– Should growth count more than status?
• What about attendance and participation?
Joint Task Force Meeting: October 14, 2010 23