The Transition from MPG to MAP

Transitioning Students from MPG to MAP

© 2011 Northwest Evaluation Association. All Rights Reserved. 2

• This session addresses transitioning students from MPG to MAP.

• It examines the point at which most partners transition students and the nature of the effect on scores.

• Findings from a research study will be presented.

• Next steps.

What will be covered?


Most of our partners transition their students from MPG to MAP

during the fall of 2nd grade for both reading and mathematics.

When do most partners transition students?


What is the impact ?

Total# lower in fall (RIT)

%lower in fall (RIT)

Mean RIT diff

MPG to MAP SP10 to FA10 (1st-2nd) 113,234 69,662 62% 10

MPG to MPG SP10 to FA10 (1st-2nd) 46,357 22,029 48% 6

MPG to MAP SP10 to FA10 (2nd -3rd) 49,940 26,685 53% 9

MAP to MAP SP10 to FA10 (2nd -3rd) 286,956 130,514 45% 7

For reading an additional 9% of students experience a drop in scores when transitioning from MPG to MAP at

the beginning of 2nd grade vs. the beginning of 3rd grade


What is the impact ?

Total # lower in fall (RIT)

%lower in fall (RIT)

Mean RIT diff

MPG to MAP SP10 to FA10 (1st-2nd) 125,238 76,318 61% 8

MPG to MPG SP10 to FA10 (1st-2nd) 45,988 20,985 47% 5

MPG to MAP SP10 to FA10 (2nd -3rd) 47,432 28,515 60% 7

MAP to MAP SP10 to FA10 (2nd -3rd) 282,306 130,647 46% 6

For mathematics the point of transition made little difference


There is something about the 2 forms of the tests that is different.• Item type• Audio support for MPG• Content

What does this mean?


• The study was conducted with second graders who took MPG as first graders.

• These students were administered items from the 2-5 reading pool both with and without audio assistance.

• Items in the field test were re-calibrated under both the non-audio & audio conditions and evaluated

• Field study tests were rescored

Field Study (Fall 2011)


Results of Fall 2011 Study

Studentsthe sample was similar to the population

MPG (spring grade 1)to MAP 2-5 (fall grade 2) Total# lower in

fall%lower in fall

Mean RIT diff

2010 - population 113,234 69,662 62% 10

2011 - study sample 1,161 814 70% 12


Spring Grade 1 to Fall Grade 2Transition

FA11 Grade2Q1 (142-165)N = 177

SP11 Grade1Q1 (145-166)N = 215

FA11Q1 N = 6

FA11Q2 N = 30

FA11 Grade2<142N = 2

SP11 Grade1<145N = 8

FA11 Grade2Q2 (166-175)N = 108

SP11 Grade1Q2 (167-176)N = 318

FA11Q3 N = 5

FA11Q4 N = 3

FA11Q3 N = 36

FA11Q4 N = 8

FA11<142 N = 2

FA11Q1 N = 164

FA11 Grade2Q3 (176-186)N = 94

SP11 Grade1Q3 (177-186)N = 323

FA11Q4 N = 50

FA11Q1 N = 94

FA11Q2 N = 85

FA11 Grade2Q4 (187-215)N = 173

SP11 Grade1Q4 (187-214)N = 290

FA11>215 N = 6

FA11Q2 N = 30

FA11Q3 N = 64

FA11Q1 N = 17

FA11 Grade2>215N = 0

SP11 Grade1>214N = 7

FA11Q4 N = 7


ItemsCalibrations varied for 55-90% of the items across all goals


+20 to 24 +15 to 19 +10 to 14 +5 to 9 +0 to 4 -1 to -4 -5 to -9 -10

Number of Items 5 8 17 11 12 4 3 1

Baseline RIT Range 160-187 158-182 152-213 162-216 155-213 183-207 177-211 209

GoalTotal Number

of Items

Easier with Audio by 5+ RIT

More Difficult with Audio by 5+ RIT

Essentially the Same - within 4 RIT

PercentageRIT Range

Dif PercentageRIT Range

Dif PercentageInformational Comprehension 20 55.0% 5 to 17 0.0% NA 45.0%

Literary Comprehension 20 75.0% 6 to 24 15.0% 5 to 10 10.0%

Word Meaning & Analysis 21 71.4% 6 to 22 4.8% 7 23.8%


Items (cont.)



Calibrations by Goal - Informational Comprehension



Calibrations by Goal - Literary Comprehension


1916


Calibrations by Goal - Word Meaning & Analysis


9


Field tests raw scoresThe majority of students score higher on audio across both forms of the test (72.1% form A & 78.5% form B) with only 9.4% & 7.2% yielding the same score across forms respectively.



Field tests re-scored (scaled score)


Spring 2011 Grade 1(MPG Reading)Fall 2011 Grade 2 (MAP Reading 2-5)

MAP 2-5 Audio FT NonAudio FT

Quartile RIT Range N RIT Range N N N

<1 <145 8 <142 4 0 0

1 145-166 215 142-165 458 30 179

2 167-176 318 166-175 253 124 301

3 177-186 323 176-186 199 383 366

4 187-214 290 187-215 241 621 315

>4 >214 7 >215 6 3 0


Growth Target Spring to Spring

Growth Targets

Spring to Spring Growth Targets

QuartileOverall Drop in Score from SP to FA No Drop in Score SP to FA

N Met Didn't Meet % Met N Met Didn't Meet % Met N Met Didn't Meet % Met

<1 8 5 3 62.5% 1 0 1 0.0% 7 5 2 71.4%

1 208 94 114 45.2% 138 46 92 33.3% 70 48 22 68.6%

2 313 159 154 50.8% 232 97 135 41.8% 81 62 19 76.5%

3 314 160 154 51.0% 231 95 136 41.1% 83 65 18 78.3%

4 281 127 154 45.2% 182 53 129 29.1% 99 74 25 74.7%

>4 7 2 5 28.6% 7 2 5 28.6% 0 0 0 0.0%

Total 1131 547 584 48.4% 791 293 498 37.0% 340 254 86 74.7%


What does this mean?

• It is necessary to consider the purpose of the test– MPG Reading: Assesses pre-and emergent readers

in grades K-2– MAP 2-5 Reading: Assesses students in grades 2-5

who can read

• Tests administered to students for their intended purpose provide information on the construct of interest (e.g. performance on informational comprehension, etc.). When tests are administered otherwise, the likelihood of introducing construct irrelevance increases.


The strategy largely depends on what you plan to do/decisions that will be made based on the data in conjunction with what works best

for your students

Transition Strategies


Reading• When the student can read connected text

sufficiently to access the items• When the student has been exposed to grade

level content• When the student scores at the upper end of

the scale for 2 consecutive administrations



Mathematics• When the student has been exposed to grade

level content • When the student scores at the upper end of

the scale for 2 consecutive administrations



• Concentrating on item development at the ends of the scale

• Updated selection strategy for MPG Fall 2012

• Releasing Common Core aligned MPG for Fall 2012

• Additional studies

– Fall 2012 –MPG goal scores predict MAP scores

– Fall 2013 – Drop when transitioning from MPG Common Core to MAP Common Core

– Fall 2013/Winter 2014 - MPG Common Core goal scores predict MAP Common Core scores

• Research investigating use of a transition instrument

– To identify which test is more appropriate to students

Next Steps


Questions?

Contact information:

Transition Study : [email protected]

Product: [email protected]

Thank You

Education

The Transition from MPG to MAP