Linking the Texas STAAR Assessments to NWEA MAP …€¦ · Linking the Texas STAAR Assessments to NWEA MAP Tests August 2016 Page 2 of 23 Introduction Northwest Evaluation Association™

LinkingtheTexasSTAARAssessmentstoNWEAMAPTests

August2016

Page2of23

Introduction

Northwest EvaluationAssociation™ (NWEA™) is committed to providing partnerswith usefultools to help make inferences from the Measures of Academic Progress® (MAP®) interimassessment scores. One important tool is the concordance table between MAP and statesummativeassessments.Concordance tableshavebeenused fordecades to relate scoresondifferent testsmeasuring similar but distinct constructs. These tables, typically derived fromstatisticallinkingprocedures,provideadirectlinkbetweenscoresondifferenttestsandservevarious purposes. Aside fromdescribing how a score on one test relates to performance onanothertest,theycanalsobeusedtoidentifybenchmarkscoresononetestcorrespondingtoperformancecategoriesonanothertest,ortomaintaincontinuityofscoresonatestafterthetest is redesigned or changed. Concordance tables are helpful for educators, parents,administrators,researchers,andpolicymakerstoevaluateandformulateacademicstandingandgrowth.

Recently,NWEA completed a concordance study to connect the scales of the State of TexasAssessments of Academic Readiness™ (STAAR™) reading and math with those of the MAP

ReadingandMAPforMathematicsassessments.Inthisreport,wepresentthe3rdthrough8thgrade cut scores onMAP reading andmathematics scales that correspond to the 2016-2017benchmarksontheSTAARreadingandmathtests. InformationabouttheconsistencyrateofclassificationbasedontheestimatedMAPcutscoresisalsoprovided,alongwithaseriesoftablesthat predict the probability of receiving a Level II (i.e., “Satisfactory”) or higher performancedesignationon theSTAARassessments,basedon theobservedMAP scores takenduring thesameschoolyear.AdetaileddescriptionofthedataandanalysismethodusedinthisstudyisprovidedintheAppendix.

OverviewofAssessments

STAAR includes a series of vertically scaled achievement tests aligned to the Texas statecurriculum,theTexasEssentialKnowledgeandSkills(TEKS)inmathandreadingforgrades3-8,writingforgrades4and7,scienceforgrades5and8,socialscienceforgrades5and8andend-of-courseassessmentsforEnglishI,EnglishII,AlgebraI,biology,andU.Shistory.STAARtestscanbedeliveredonlineorinthepaper-and-pencilform.Foreachgradeandsubject,therearetwocut scores that distinguish between performance levels: Level I: Unsatisfactory AcademicPerformance, Level II: Satisfactory Academic Performance and Level III: Advanced AcademicPerformance.TheLevelIIcutscoredemarkstheminimumlevelofperformanceconsideredtobe“Proficient”foraccountabilitypurposes.

Page3of23

MAP tests are vertically scaled interim assessments that are administered in the form of acomputerizedadaptivetest(CAT).MAPtestsareconstructedtomeasurestudentachievementfromGradesKto12inmath,reading,languageusage,andscienceandarealignedtotheTEKSstandards.MAPscoresarereportedonaRaschUnit(RIT)scalewitharangefrom100to350.EachsubjecthasitsownRITscale.

ToaidinterpretationofMAPscores,NWEAperiodicallyconductsnormingstudiesofstudentandschoolperformanceonMAP.Forexample,the2015RITScalenormingstudy(Thum&Hauser,2015)employedmulti-levelgrowthmodelsonnearly500,000longitudinaltestscoresfromover100,000studentsthatwereweightedtocreatelarge,nationallyrepresentativenormsformath,reading,languageusage,andgeneralscience.

EstimatedMAPCutScoresAssociatedwithSTAARReadinessLevels

Tables1to4reporttheSTAARscaledscoresassociatedwitheachofthethreeperformancelevels,aswellastheestimatedcutscoresontheMAPtestsassociatedwiththeSTAARperformancelevels.Specifically,Tables1and2applytoMAPscoresobtainedduringthespringtestingseasonforreadingandmath,respectively.Tables3and4applytoMAPteststakeninapriortestingseason(fallorwinter)forreadingandmath,respectively.Thetablesalsoreportthepercentilerank(basedontheNWEA2015MAPNorms)associatedwitheachestimatedMAPcutscore.TheMAPcutscorescanbeusedtopredictstudents’mostprobableSTAARperformancelevel,basedontheirobservedMAPscores.Forexample,a6thgradestudentwhoobtainedaMAPmathscoreof240inthespringtestingseasonislikelytobeattheveryhighendofLevelII(Satisfactory)ontheSTAARtakenduringthatsametestingseason(seeTable2).Similarly,a3rdgradestudentwhoobtainedaMAPreadingscoreof210inthefalltestingseasonislikelytobeatLevelIII(Advanced)ontheSTAARtakeninthespringof3rdgrade(seeTable3).

Page4of23

TABLE1.CONCORDANCEOFPERFORMANCELEVELSCORERANGESBETWEENSTAAR

ANDMAPREADING(WHENMAPISTAKENINSPRING)

Grade

STAAR

LevelIUnsatisfactory

LevelIISatisfactory

LevelIIIAdvanced

3 700-1364 1365-1554 1555-2300

4 700-1453 1454-1632 1633-2300

5 700-1488 1489-1666 1667-2300

6 700-1535 1536-1717 1718-2300

7 700-1584 1585-1752 1753-2300

8 700-1605 1606-1782 1783-2300

Grade

MAP


LevelIISatisfactory

LevelIIIAdvanced

RIT %ile RIT %ile RIT %ile

3 100-195 1-41 196-213 42-83 214-350 84-99

4 100-203 1-43 204-220 44-83 221-350 84-99

5 100-208 1-41 209-225 42-82 226-350 83-99

6 100-210 1-36 211-229 37-82 230-350 83-99

7 100-212 1-35 213-232 36-82 233-350 83-99

8 100-213 1-33 214-234 34-82 235-350 83-99Notes.1.%ile=percentile.2.Boldednumbersindicatethecutscoresconsideredtobeatleast“proficient”foraccountabilitypurposes.

Page5of23


ANDMAPMATH(WHENMAPISTAKENINSPRING)

Grade

STAAR


LevelIISatisfactory

LevelIIIAdvanced

3 700-1380 1381-1595 1596-2300

4 700-1486 1487-1669 1670-2300

5 700-1520 1521-1723 1724-2300

6 700-1555 1556-1771 1772-2300

7 700-1593 1594-1797 1798-2300

8 700-1611 1612-1853 1854-2300

Grade

MAP


LevelIISatisfactory

LevelIIIAdvanced


3 100-201 1-44 202-219 45-87 220-350 88-99

4 100-213 1-50 214-232 51-89 233-350 90-99

5 100-216 1-38 217-239 39-86 240-350 87-99

6 100-220 1-38 221-243 39-86 244-350 87-99

7 100-224 1-40 225-249 41-88 250-350 89-99

8 100-225 1-38 226-257 39-91 258-350 92-99Notes.1.%ile=percentile.2.Boldednumbersindicatethecutscoresconsideredtobeatleast“proficient”foraccountabilitypurposes.

Page6of23


ANDMAPREADING(WHENMAPISTAKENINFALLORWINTERPRIORTOSPRING

STAARTESTS)

Notes.1.%ile=percentile.2.Boldednumbersindicatethecutscoresconsideredtobeatleast“proficient”foraccountabilitypurposes.

GradeSTAAR


LevelIISatisfactory

LevelIIIAdvanced

3 700-1364 1365-1554 1555-2300

4 700-1453 1454-1632 1633-2300

5 700-1488 1489-1666 1667-2300

6 700-1535 1536-1717 1718-2300

7 700-1584 1585-1752 1753-2300

8 700-1605 1606-1782 1783-2300 MAPFALL

GradeLevelI

UnsatisfactoryLevelII

SatisfactoryLevelIIIAdvanced


3 100-184 1-40 185-206 41-87 207-350 88-99

4 100-195 1-43 196-215 44-86 216-350 87-99

5 100-201 1-39 202-221 40-85 222-350 86-99

6 100-204 1-33 205-226 34-85 227-350 86-99

7 100-208 1-34 209-230 35-85 231-350 86-99

8 100-209 1-31 210-232 32-83 233-350 84-99 MAPWINTER

GradeLevelI




3 100-192 1-41 193-211 42-85 212-350 86-99

4 100-201 1-44 202-219 45-85 220-350 86-99

5 100-206 1-41 207-224 42-84 225-350 85-99

6 100-208 1-34 209-228 35-83 229-350 84-99

7 100-210 1-33 211-231 34-83 232-350 84-99

8 100-212 1-33 213-233 34-82 234-350 83-99

Page7of23


ANDMAPMATH(WHENMAPISTAKENINFALLORWINTERPRIORTOSPRINGSTAAR

TESTS)

Notes.1.%ile=percentile.2.Boldednumbersindicatethecutscoresconsideredtobeatleast“proficient”foraccountabilitypurposes.

GradeSTAAR


LevelIISatisfactory

LevelIIIAdvanced

3 700-1380 1381-1595 1596-2300

4 700-1486 1487-1669 1670-2300

5 700-1520 1521-1723 1724-2300

6 700-1555 1556-1771 1772-2300

7 700-1593 1594-1797 1798-2300

8 700-1611 1612-1853 1854-2300 MAPFALL

GradeLevelI




3 100-188 1-44 189-207 45-90 208-350 91-99

4 100-201 1-48 202-221 49-92 222-350 93-99

5 100-206 1-36 207-229 37-89 230-350 90-99

6 100-212 1-37 213-236 38-88 237-350 89-99

7 100-218 1-40 219-243 41-89 244-350 90-99

8 100-220 1-37 221-253 38-93 254-350 94-99 MAPWINTER

GradeLevelI




3 100-196 1-44 197-214 45-88 215-350 89-99

4 100-208 1-49 209-227 50-90 228-350 91-99

5 100-212 1-37 213-235 38-88 236-350 89-99

6 100-217 1-38 218-240 39-87 241-350 88-99

7 100-222 1-41 223-247 42-89 248-350 90-99

8 100-223 1-37 224-255 38-92 256-350 93-99

Page8of23

ConsistencyRateofClassification

Consistencyrateofclassification(Pommerich,Hanson,Harris,&Sconing,2004),expressedintheformofaratebetween0and1,providesameanstomeasurethedeparturefromequity forconcordances(Hansonetal.,2001).ThisindexcanalsobeusedasanindicatorforthepredictivevalidityoftheMAPtests,i.e.,howaccuratelytheMAPscorescanpredictastudent’sproficiencystatus in the STAAR test. For each pair of concordant scores, a classification is consideredconsistentiftheexamineeisclassifiedintothesameperformancecategoryregardlessofthetestusedformakingadecision.Consistencyrateprovidedinthisreportcanbecalculatedas,forthe“proficient”performancecategoryconcordantscores,thepercentageofexamineeswhoscoreatorabovebothconcordantscoresplusthepercentageofexamineeswhoscorebelowbothconcordantscoresoneachtest.HigherconsistencyrateindicatesstrongercongruencebetweenSTAARandMAPcutscores.Theresults inTable5demonstratethatonaverageMAPreadingscorescanconsistentlyclassifystudents’proficiency(LevelIIorhigher)statusonSTAARreadingtest86%ofthetimeandMAPmathscorescanconsistentlyclassifystudentsonSTAARmathtest86%ofthetime.ThosenumbersarehighsuggestingthatbothMAPreadingandmathtestsaregreatpredictorsofthestudents’proficiencystatusontheSTAARtests.

TABLE5.CONSISTENCYRATEOFCLASSIFICATIONFORMAPANDSTAARLEVELII

EQUIPERCENTILECONCORDANCES

GradeReading Math

ConsistencyRate

False ConsistencyRate

FalsePositives Negatives Positives Negatives

3 0.88 0.06 0.06 0.90 0.07 0.034 0.88 0.07 0.05 0.88 0.08 0.045 0.86 0.08 0.06 0.89 0.06 0.056 0.86 0.07 0.07 0.87 0.08 0.057 0.85 0.08 0.07 0.86 0.09 0.058 0.84 0.08 0.08 0.78 0.13 0.09

ProficiencyProjection

Proficiencyprojectiontellshowlikelyastudentisclassifiedas“proficient”onSTAARtestsbasedonhis/herobservedMAPscores.Theconditionalgrowthnormsprovidedinthe2015MAPNormswere used to calculate this information (Thum & Hauser, 2015). The results of proficiencyprojection and corresponding probability of achieving “proficient” on the STAAR tests are

Page9of23

presentedinTables6to8.ThesetablesestimatetheprobabilityofscoringatLevelIIoraboveonSTAAR inthespringandtheprior fallorwintertestingseason.Forexample, ifa3rdgradestudentobtainedaMAPmathscoreof190inthefall,theprobabilityofobtainingaLevelIIorhigherSTAARscoreinthespringof3rdgradeis56%.Table6presentstheestimatedprobabilityofmeetingLevelIIbenchmarkwhenMAPistakeninthespring,whereasTables7and8presenttheestimatedprobabilityofmeetingLevelIIbenchmarkwhenMAPistakeninthefallorwinterpriortotakingtheSTAARtests.

Page10of23

TABLE6.PROFICIENCYPROJECTIONANDPROBABILITYFORPASSINGSTAARLEVELII(SATISFACTORY)WHENMAPISTAKENINTHESPRING

Grade

Reading Math

Start%ile

RITSpring

ProjectedProficiency Start%ile

RITSpring

ProjectedProficiency

CutScore LevelII Prob. CutScore LevelII Prob.

3

5 174 196 No <0.01 5 181 202 No <0.0110 179 196 No <0.01 10 186 202 No <0.0115 183 196 No <0.01 15 189 202 No <0.0120 186 196 No <0.01 20 192 202 No <0.0125 188 196 No 0.01 25 194 202 No <0.0130 191 196 No 0.06 30 196 202 No 0.0235 193 196 No 0.17 35 198 202 No 0.0840 195 196 No 0.38 40 200 202 No 0.2545 197 196 Yes 0.62 45 202 202 Yes 0.5050 199 196 Yes 0.83 50 203 202 Yes 0.6355 201 196 Yes 0.94 55 205 202 Yes 0.8560 202 196 Yes 0.97 60 207 202 Yes 0.9665 204 196 Yes 0.99 65 209 202 Yes 0.9970 207 196 Yes >0.99 70 211 202 Yes >0.9975 209 196 Yes >0.99 75 213 202 Yes >0.9980 211 196 Yes >0.99 80 215 202 Yes >0.9985 214 196 Yes >0.99 85 218 202 Yes >0.9990 218 196 Yes >0.99 90 221 202 Yes >0.9995 223 196 Yes >0.99 95 226 202 Yes >0.99

4

5 181 204 No <0.01 5 189 214 No <0.0110 187 204 No <0.01 10 194 214 No <0.0115 190 204 No <0.01 15 198 214 No <0.0120 193 204 No <0.01 20 201 214 No <0.0125 196 204 No 0.01 25 203 214 No <0.0130 198 204 No 0.03 30 206 214 No <0.0135 200 204 No 0.11 35 208 214 No 0.0240 202 204 No 0.27 40 210 214 No 0.0845 204 204 Yes 0.50 45 212 214 No 0.2550 206 204 Yes 0.73 50 213 214 No 0.3755 208 204 Yes 0.89 55 215 214 Yes 0.6360 210 204 Yes 0.97 60 217 214 Yes 0.8565 212 204 Yes 0.99 65 219 214 Yes 0.9670 214 204 Yes >0.99 70 221 214 Yes 0.9975 216 204 Yes >0.99 75 224 214 Yes >0.9980 218 204 Yes >0.99 80 226 214 Yes >0.9985 221 204 Yes >0.99 85 229 214 Yes >0.9990 225 204 Yes >0.99 90 233 214 Yes >0.9995 230 204 Yes >0.99 95 238 214 Yes >0.99

Page11of23

TABLE6.(CONTINUED)

Grade

Reading Math

Start%ile

RITSpring


RITSpring



5

5 188 209 No <0.01 5 195 217 No <0.0110 193 209 No <0.01 10 201 217 No <0.0115 197 209 No <0.01 15 205 217 No <0.0120 199 209 No <0.01 20 208 217 No <0.0125 202 209 No 0.01 25 210 217 No 0.0130 204 209 No 0.06 30 213 217 No 0.0835 206 209 No 0.17 35 215 217 No 0.2540 208 209 No 0.38 40 217 217 Yes 0.5045 210 209 Yes 0.62 45 219 217 Yes 0.7550 212 209 Yes 0.83 50 221 217 Yes 0.9255 214 209 Yes 0.94 55 223 217 Yes 0.9860 216 209 Yes 0.99 60 225 217 Yes >0.9965 217 209 Yes 0.99 65 228 217 Yes >0.9970 220 209 Yes >0.99 70 230 217 Yes >0.9975 222 209 Yes >0.99 75 232 217 Yes >0.9980 224 209 Yes >0.99 80 235 217 Yes >0.9985 227 209 Yes >0.99 85 238 217 Yes >0.9990 231 209 Yes >0.99 90 242 217 Yes >0.9995 236 209 Yes >0.99 95 248 217 Yes >0.99

6

5 192 211 No <0.01 5 198 221 No <0.0110 197 211 No <0.01 10 204 221 No <0.0115 201 211 No <0.01 15 208 221 No <0.0120 203 211 No 0.01 20 211 221 No <0.0125 206 211 No 0.06 25 214 221 No 0.0130 208 211 No 0.17 30 217 221 No 0.0835 210 211 No 0.38 35 219 221 No 0.2540 212 211 Yes 0.62 40 221 221 Yes 0.5045 214 211 Yes 0.83 45 223 221 Yes 0.7550 216 211 Yes 0.94 50 225 221 Yes 0.9255 218 211 Yes 0.99 55 227 221 Yes 0.9860 219 211 Yes 0.99 60 230 221 Yes >0.9965 221 211 Yes >0.99 65 232 221 Yes >0.9970 223 211 Yes >0.99 70 234 221 Yes >0.9975 226 211 Yes >0.99 75 237 221 Yes >0.9980 228 211 Yes >0.99 80 239 221 Yes >0.9985 231 211 Yes >0.99 85 243 221 Yes >0.9990 235 211 Yes >0.99 90 247 221 Yes >0.9995 240 211 Yes >0.99 95 253 221 Yes >0.99

Page12of23

TABLE6.(CONTINUED)

Grade

Reading Math

Start%ile

RITSpring


RITSpring



7

5 193 213 No <0.01 5 199 225 No <0.0110 199 213 No <0.01 10 206 225 No <0.0115 202 213 No <0.01 15 210 225 No <0.0120 205 213 No 0.01 20 214 225 No <0.0125 208 213 No 0.06 25 217 225 No <0.0130 210 213 No 0.17 30 219 225 No 0.0235 212 213 No 0.38 35 222 225 No 0.1540 214 213 Yes 0.62 40 224 225 No 0.3745 216 213 Yes 0.83 45 226 225 Yes 0.6350 218 213 Yes 0.94 50 229 225 Yes 0.9255 220 213 Yes 0.99 55 231 225 Yes 0.9860 222 213 Yes >0.99 60 233 225 Yes >0.9965 224 213 Yes >0.99 65 235 225 Yes >0.9970 226 213 Yes >0.99 70 238 225 Yes >0.9975 228 213 Yes >0.99 75 241 225 Yes >0.9980 231 213 Yes >0.99 80 244 225 Yes >0.9985 234 213 Yes >0.99 85 247 225 Yes >0.9990 238 213 Yes >0.99 90 251 225 Yes >0.9995 243 213 Yes >0.99 95 258 225 Yes >0.99

8

5 194 214 No <0.01 5 199 226 No <0.0110 200 214 No <0.01 10 206 226 No <0.0115 204 214 No <0.01 15 211 226 No <0.0120 207 214 No 0.01 20 215 226 No <0.0125 209 214 No 0.06 25 218 226 No <0.0130 212 214 No 0.27 30 221 226 No 0.0435 214 214 Yes 0.50 35 224 226 No 0.2540 216 214 Yes 0.73 40 226 226 Yes 0.5045 218 214 Yes 0.89 45 229 226 Yes 0.8550 220 214 Yes 0.97 50 231 226 Yes 0.9655 222 214 Yes 0.99 55 233 226 Yes 0.9960 224 214 Yes >0.99 60 236 226 Yes >0.9965 226 214 Yes >0.99 65 238 226 Yes >0.9970 228 214 Yes >0.99 70 241 226 Yes >0.9975 231 214 Yes >0.99 75 244 226 Yes >0.9980 233 214 Yes >0.99 80 247 226 Yes >0.9985 236 214 Yes >0.99 85 251 226 Yes >0.9990 240 214 Yes >0.99 90 255 226 Yes >0.9995 246 214 Yes >0.99 95 262 226 Yes >0.99

Note.%ile=percentile

Page13of23

ABLE7.PROFICIENCYPROJECTIONANDPROBABILITYFORPASSINGSTAARREADINGLEVELII(SATISFACTORY)WHENMAPISTAKENINTHEFALLORWINTERPRIORTOSPRINGSTAARTESTS

GradeStart%ile

RITFall


RITWinter

ProjectedProficiencyCutScore LevelII Prob. CutScore LevelII Prob.

3

5 162 196 No <0.01 5 171 196 No <0.0110 168 196 No 0.03 10 176 196 No <0.0115 172 196 No 0.06 15 180 196 No 0.0120 175 196 No 0.10 20 183 196 No 0.0425 178 196 No 0.20 25 185 196 No 0.0930 180 196 No 0.29 30 188 196 No 0.2235 182 196 No 0.34 35 190 196 No 0.2840 184 196 No 0.44 40 192 196 No 0.4245 186 196 Yes 0.56 45 194 196 Yes 0.5850 188 196 Yes 0.61 50 196 196 Yes 0.7255 190 196 Yes 0.71 55 198 196 Yes 0.8360 192 196 Yes 0.80 60 199 196 Yes 0.8765 194 196 Yes 0.84 65 201 196 Yes 0.9470 197 196 Yes 0.92 70 204 196 Yes 0.9875 199 196 Yes 0.95 75 206 196 Yes 0.9980 202 196 Yes 0.97 80 208 196 Yes >0.9985 205 196 Yes 0.99 85 211 196 Yes >0.9990 209 196 Yes >0.99 90 215 196 Yes >0.9995 214 196 Yes >0.99 95 221 196 Yes >0.99

4

5 173 204 No <0.01 5 179 204 No <0.0110 178 204 No 0.01 10 184 204 No <0.0115 182 204 No 0.04 15 188 204 No 0.0120 185 204 No 0.09 20 191 204 No 0.0225 188 204 No 0.15 25 194 204 No 0.0830 190 204 No 0.23 30 196 204 No 0.1635 192 204 No 0.33 35 198 204 No 0.2840 194 204 No 0.38 40 200 204 No 0.4245 196 204 Yes 0.50 45 202 204 Yes 0.5050 198 204 Yes 0.62 50 204 204 Yes 0.6555 200 204 Yes 0.67 55 205 204 Yes 0.7260 202 204 Yes 0.77 60 207 204 Yes 0.8465 204 204 Yes 0.85 65 209 204 Yes 0.9270 206 204 Yes 0.91 70 211 204 Yes 0.9675 209 204 Yes 0.95 75 214 204 Yes 0.9980 211 204 Yes 0.97 80 216 204 Yes >0.9985 214 204 Yes 0.99 85 219 204 Yes >0.9990 218 204 Yes >0.99 90 223 204 Yes >0.9995 224 204 Yes >0.99 95 228 204 Yes >0.99

Page14of23

TABLE7.(CONTINUED)

GradeStart%ile

RITFall


RITWinter

ProjectedProficiencyCut-Score LevelII Prob. Cut-Score LevelII Prob.

5

5 181 209 No <0.01 5 186 209 No <0.0110 186 209 No 0.02 10 191 209 No <0.0115 190 209 No 0.05 15 195 209 No 0.0220 193 209 No 0.12 20 197 209 No 0.0425 195 209 No 0.19 25 200 209 No 0.1230 198 209 No 0.28 30 202 209 No 0.1735 200 209 No 0.38 35 204 209 No 0.2840 202 209 Yes 0.50 40 206 209 No 0.4245 204 209 Yes 0.56 45 208 209 Yes 0.5850 206 209 Yes 0.67 50 210 209 Yes 0.7255 208 209 Yes 0.77 55 212 209 Yes 0.8360 210 209 Yes 0.85 60 214 209 Yes 0.9165 212 209 Yes 0.88 65 215 209 Yes 0.9470 214 209 Yes 0.93 70 218 209 Yes 0.9875 216 209 Yes 0.96 75 220 209 Yes 0.9980 218 209 Yes 0.97 80 222 209 Yes >0.9985 221 209 Yes 0.99 85 225 209 Yes >0.9990 225 209 Yes >0.99 90 229 209 Yes >0.9995 231 209 Yes >0.99 95 234 209 Yes >0.99

6

5 186 211 No 0.01 5 190 211 No <0.0110 192 211 No 0.04 10 196 211 No 0.0115 196 211 No 0.12 15 199 211 No 0.0320 198 211 No 0.16 20 202 211 No 0.0925 201 211 No 0.28 25 204 211 No 0.1730 203 211 No 0.39 30 207 211 No 0.3535 205 211 Yes 0.50 35 209 211 Yes 0.5040 207 211 Yes 0.56 40 211 211 Yes 0.6545 209 211 Yes 0.67 45 212 211 Yes 0.7250 211 211 Yes 0.77 50 214 211 Yes 0.8355 213 211 Yes 0.84 55 216 211 Yes 0.8860 215 211 Yes 0.88 60 218 211 Yes 0.9465 217 211 Yes 0.93 65 220 211 Yes 0.9770 219 211 Yes 0.96 70 222 211 Yes 0.9975 221 211 Yes 0.97 75 224 211 Yes >0.9980 224 211 Yes 0.99 80 226 211 Yes >0.9985 226 211 Yes >0.99 85 229 211 Yes >0.9990 230 211 Yes >0.99 90 233 211 Yes >0.9995 236 211 Yes >0.99 95 238 211 Yes >0.99

Page15of23

TABLE7.(CONTINUED)

GradeStart%ile

RITFall


RITWinter


7


8



Page16of23

TABLE8.PROFICIENCYPROJECTIONANDPROBABILITYFORPASSINGSTAARMATHLEVELII(SATISFACTORY)WHENMAPISTAKENINTHEFALLORWINTERPRIORTOSPRINGSTAARTESTS

GradeStart%ile

RITFall


RITWinter

ProjectedProficiencyCutScore LevelII Prob. CutScore LevelII Prob.

3

5 169 202 No <0.01 5 176 202 No <0.0110 174 202 No 0.01 10 181 202 No <0.0115 177 202 No 0.04 15 184 202 No 0.0120 179 202 No 0.08 20 187 202 No 0.0225 182 202 No 0.17 25 189 202 No 0.0530 184 202 No 0.22 30 191 202 No 0.1035 185 202 No 0.27 35 193 202 No 0.2040 187 202 No 0.38 40 195 202 No 0.3445 189 202 Yes 0.50 45 197 202 Yes 0.5050 190 202 Yes 0.56 50 198 202 Yes 0.5855 192 202 Yes 0.68 55 200 202 Yes 0.7460 194 202 Yes 0.78 60 202 202 Yes 0.8665 195 202 Yes 0.83 65 203 202 Yes 0.9070 197 202 Yes 0.89 70 205 202 Yes 0.9575 199 202 Yes 0.92 75 207 202 Yes 0.9880 201 202 Yes 0.96 80 209 202 Yes 0.9985 204 202 Yes 0.99 85 212 202 Yes >0.9990 207 202 Yes >0.99 90 215 202 Yes >0.9995 212 202 Yes >0.99 95 220 202 Yes >0.99

4

5 179 214 No <0.01 5 185 214 No <0.0110 184 214 No <0.01 10 190 214 No <0.0115 188 214 No 0.01 15 194 214 No <0.0120 190 214 No 0.03 20 197 214 No 0.0125 193 214 No 0.08 25 199 214 No 0.0230 195 214 No 0.14 30 201 214 No 0.0535 197 214 No 0.22 35 203 214 No 0.1040 198 214 No 0.27 40 205 214 No 0.2045 200 214 No 0.38 45 207 214 No 0.3450 202 214 Yes 0.50 50 209 214 Yes 0.5055 204 214 Yes 0.62 55 211 214 Yes 0.6660 205 214 Yes 0.62 60 212 214 Yes 0.7465 207 214 Yes 0.73 65 214 214 Yes 0.8670 209 214 Yes 0.83 70 216 214 Yes 0.9375 211 214 Yes 0.89 75 218 214 Yes 0.9780 214 214 Yes 0.96 80 221 214 Yes 0.9985 216 214 Yes 0.98 85 223 214 Yes >0.9990 220 214 Yes >0.99 90 227 214 Yes >0.9995 225 214 Yes >0.99 95 232 214 Yes >0.99

Page17of23

TABLE8.(CONTINUED)

GradeStart%ile

RITFall


RITWinter


5

5 187 217 No <0.01 5 192 217 No <0.0110 193 217 No 0.02 10 198 217 No <0.0115 196 217 No 0.05 15 201 217 No 0.0120 199 217 No 0.12 20 204 217 No 0.0325 202 217 No 0.23 25 207 217 No 0.1130 204 217 No 0.33 30 209 217 No 0.2035 206 217 No 0.44 35 211 217 No 0.3440 208 217 Yes 0.56 40 213 217 Yes 0.5045 210 217 Yes 0.67 45 215 217 Yes 0.6650 211 217 Yes 0.72 50 217 217 Yes 0.8055 213 217 Yes 0.81 55 219 217 Yes 0.8960 215 217 Yes 0.88 60 221 217 Yes 0.9565 217 217 Yes 0.93 65 223 217 Yes 0.9870 219 217 Yes 0.96 70 225 217 Yes 0.9975 221 217 Yes 0.98 75 228 217 Yes >0.9980 224 217 Yes 0.99 80 230 217 Yes >0.9985 227 217 Yes >0.99 85 233 217 Yes >0.9990 230 217 Yes >0.99 90 237 217 Yes >0.9995 236 217 Yes >0.99 95 242 217 Yes >0.99

6

5 192 221 No <0.01 5 196 221 No <0.0110 198 221 No 0.01 10 202 221 No <0.0115 202 221 No 0.05 15 205 221 No <0.0120 205 221 No 0.12 20 209 221 No 0.0325 207 221 No 0.19 25 211 221 No 0.0730 209 221 No 0.28 30 214 221 No 0.2035 212 221 No 0.44 35 216 221 No 0.3440 214 221 Yes 0.56 40 218 221 Yes 0.5045 216 221 Yes 0.67 45 220 221 Yes 0.6650 218 221 Yes 0.77 50 222 221 Yes 0.8055 220 221 Yes 0.85 55 224 221 Yes 0.8960 222 221 Yes 0.91 60 226 221 Yes 0.9565 224 221 Yes 0.95 65 228 221 Yes 0.9870 226 221 Yes 0.97 70 230 221 Yes 0.9975 228 221 Yes 0.99 75 233 221 Yes >0.9980 231 221 Yes >0.99 80 236 221 Yes >0.9985 234 221 Yes >0.99 85 239 221 Yes >0.9990 238 221 Yes >0.99 90 243 221 Yes >0.9995 243 221 Yes >0.99 95 248 221 Yes >0.99

Page18of23

TABLE8.(CONTINUED)

GradeStart%ile

RITFall


RITWinter


7

5 195 225 No <0.01 5 198 225 No <0.0110 201 225 No <0.01 10 204 225 No <0.0115 205 225 No 0.02 15 208 225 No <0.0120 209 225 No 0.06 20 212 225 No 0.0125 211 225 No 0.11 25 215 225 No 0.0530 214 225 No 0.22 30 217 225 No 0.1035 216 225 No 0.32 35 220 225 No 0.2640 218 225 No 0.44 40 222 225 No 0.4245 221 225 Yes 0.62 45 224 225 Yes 0.5850 223 225 Yes 0.73 50 226 225 Yes 0.7455 225 225 Yes 0.82 55 228 225 Yes 0.8560 227 225 Yes 0.89 60 230 225 Yes 0.9365 229 225 Yes 0.94 65 233 225 Yes 0.9870 231 225 Yes 0.97 70 235 225 Yes 0.9975 234 225 Yes 0.99 75 238 225 Yes >0.9980 237 225 Yes >0.99 80 240 225 Yes >0.9985 240 225 Yes >0.99 85 244 225 Yes >0.9990 244 225 Yes >0.99 90 248 225 Yes >0.9995 250 225 Yes >0.99 95 254 225 Yes >0.99

8

5 197 226 No <0.01 5 199 226 No <0.0110 203 226 No 0.01 10 206 226 No <0.0115 208 226 No 0.04 15 210 226 No <0.0120 211 226 No 0.10 20 214 226 No 0.0225 214 226 No 0.18 25 217 226 No 0.0830 217 226 No 0.30 30 220 226 No 0.2135 219 226 No 0.40 35 222 226 No 0.3540 222 226 Yes 0.55 40 225 226 Yes 0.5845 224 226 Yes 0.65 45 227 226 Yes 0.7250 226 226 Yes 0.74 50 229 226 Yes 0.8455 229 226 Yes 0.85 55 231 226 Yes 0.9260 231 226 Yes 0.90 60 234 226 Yes 0.9865 233 226 Yes 0.94 65 236 226 Yes 0.9970 236 226 Yes 0.97 70 239 226 Yes >0.9975 238 226 Yes 0.98 75 241 226 Yes >0.9980 241 226 Yes 0.99 80 245 226 Yes >0.9985 245 226 Yes >0.99 85 248 226 Yes >0.9990 249 226 Yes >0.99 90 253 226 Yes >0.9995 256 226 Yes >0.99 95 259 226 Yes >0.99


Page19of23

SummaryandDiscussion

ThisstudyproducedasetofcutscoresonMAPreadingandmathtestsforGrades3to8thatcorrespondtoeachSTAARperformance level.Byusingmatchedscoredata fromasampleofstudentsfromTexas,thestudydemonstratesthatMAPscorescanaccuratelypredictwhetherastudentcouldbeproficientoraboveonthebasisofhis/herMAPscores.Thisstudyalsousedthe2015NWEAnormingstudyresultstoprojectastudent’sprobabilitytomeetproficiencybasedonthatstudent’spriorMAPscoresinfallandwinter.TheseresultswillhelpeducatorspredictstudentperformanceinSTAARtestsasearlyaspossibleandidentifythosestudentswhoareatrisk of failing tomeet required standards so that they can receive necessary resources andassistancetomeettheirgoals.

Whileconcordancetablescanbehelpfulandinformative,theyhavegenerallimitations.First,theconcordance tablesprovide informationaboutscorecomparabilityondifferent tests,but thescorescannotbeassumedtobeinterchangeable.InthecaseforSTAARandMAPtests,astheyarenotparallelincontent,scoresfromthesetwotestsshouldnotbedirectlycompared.Second,while the sampledataused in this studywere collected from147 schools in Texas, cautionsshouldbeexercisedwhengeneralizingtheresultstotesttakerswhodiffersignificantlyfromthissample. Finally, cautions should also be exercised if the concorded scores are used for asubpopulation.NWEAwillcontinuetogatherinformationaboutSTAARperformancefromotherschoolsinTexastoenhancethequalityandgeneralizabilityofthestudy.

Page20of23

References

Hanson,B.A.,Harris,D.J.,Pommerich,M.,Sconing,J.A.,&Yi,Q.(2001).Suggestionsfortheevaluationanduseofconcordanceresults.(ACTResearchReportNo.2001-1).IowaCity,IA:ACT,Inc.

Kolen,M.J.,&Brennan,R.L.(2004).Testequating,scaling,andlinking.NewYork:Springer.

Pommerich,M.,Hanson,B.,Harris,D.,&Sconing,J.(2004).Issuesinconductinglinkagebetweendistincttests.AppliedPsychologicalMeasurement,28(4),247-273.

ThumY.M.,&Hauser,C.H.(2015).NWEA2015MAPNormsforStudentandSchoolAchievementStatusandGrowth.NWEAResearchReport.Portland,OR:NWEA.

Page21of23

Appendix

DataandAnalysis

Data

Datausedinthisstudywerecollectedfrom147schoolsinTexas.ThesamplecontainedmatchedSTAARandMAPreadingscoresof50,108studentsinGrades3to8andmatchedSTAARandMAPmathscoresof46,987students inGrades3to8whocompletedbothMAPandSTAARinthespringof2015.

Tounderstandthestatisticalcharacteristicsofthetestscores,descriptivestatisticsareprovidedinTableA1below.AsTableA1indicates,thecorrelationcoefficientsbetweenMAPandSTAARreadingscoresrangefrom0.75to0.82,andthecorrelationcoefficientsbetweenMAPandSTAARmath scores range from 0.76 to 0.87. In general, all these correlations indicate a strongrelationshipbetweenMAPandSTAARtestscores.

TABLEA1.DESCRIPTIVESTATISTICSOFTHESAMPLEDATA

STAAR MAPSubject Grade N r Mean SD Min Max Mean SD Min Max

Reading

3 8,750 0.82 1464 148.26 1026 1911 204 15.16 143 2484 8,736 0.82 1548 150.91 1047 1995 212 15.14 134 2595 8,627 0.78 1595 145.94 1110 2025 218 14.10 140 2706 8,451 0.79 1623 139.78 1178 2074 218 16.50 143 2617 8,403 0.78 1664 133.51 1295 2140 221 17.45 140 2648 7,141 0.75 1696 133.39 1178 2156 223 17.26 143 283

Math

3 8,838 0.84 1486 154.28 1033 1921 210 13.36 145 2834 8,844 0.87 1569 153.21 1191 2065 222 15.47 138 2905 8,634 0.87 1639 151.22 1189 2091 230 16.45 155 2986 8,160 0.84 1647 145.49 1024 2189 229 16.00 146 2787 8,256 0.86 1678 140.65 1015 2229 234 18.19 145 3128 4,255 0.76 1633 97.65 1315 2103 228 15.88 164 293

Page22of23

EquipercentileLinkingProcedure

The equipercentile procedure (e.g., Kolen & Brennan, 2004) was used to establish theconcordancerelationshipbetweenSTAARandMAPscoresforgrades3to8inreadingandmath.Thisprocedurematchesscoresonthetwoscalesthathavethesamepercentilerank(i.e.,theproportionofscoresatorbeloweachscore).

Supposeweneedtoestablishtheconcordedscoresbetweentwotests.𝑥isascoreonTest𝑋(e.g.,STAAR).ItsequipercentileequivalentscoreonTest𝑌(e.g.,MAP),𝑒& 𝑥 ,canbeobtainedthroughacumulative-distribution-basedlinkingfunctiondefinedinEquation(A1):

𝑒& 𝑥 = 𝐺*+[𝑃 𝑥 ] (A1)

where𝑒& 𝑥 istheequipercentileequivalentofscoresonSTAARonthescaleofMAP,𝑃 𝑥 isthepercentilerankofagivenscoreonTest𝑋.𝐺*+istheinverseofthepercentilerankfunctionforscores on Test𝑌 which indicates the scores on Test𝑌 corresponding to a given percentile.Polynomial loglinear pre-smoothing was applied to reduce irregularities of the frequencydistributionsaswellasequipercentilelinkingcurve.

ConsistencyrateofClassification

Consistency rateof classificationaccuracy,expressed in the formofa ratebetween0and1,measures the extent to which MAP scores (and the estimated MAP cut scores) accuratelypredictedwhetherstudentsinthesamplewouldpass(i.e.,LevelIIorhigher)onSTAARtests.

Tocalculateconsistencyrateofclassification,samplestudentsweredesignated“BelowSTAARcut”or“AtoraboveSTAARcut”basedontheiractualSTAARscores.Similarly,theywerealsodesignatedas“BelowMAPcut”or“AtoraboveMAPcut”basedontheiractualMAPscores.A2-waycontingencytablewasthentabulated(seeTableA2),classifyingstudentsas“Proficient”onthe basis of STAAR cut score and concordantMAP cut score. Students classified in the truepositive(TP)categorywerethosepredictedtobeProficientbasedontheMAPcutscoresandwerealsoclassifiedasProficientbasedontheSTAARcutscores.Studentsclassifiedinthetruenegative(TN)categorywerethosepredictedtobeNotProficientbasedontheMAPcutscoresandwerealsoclassifiedasNotProficientbasedontheSTAARcutscores.Studentsclassifiedinthefalsepositive(FP)categorywerethosepredictedtobeProficientbasedontheMAPcutscoresbutwereclassifiedasNotProficientbasedontheSTAARcutscores.Studentsclassifiedinthefalsenegative(FN)categorywerethosepredicatedtobeNotProficientbasedontheMAPcutscoresbutwereclassifiedasProficientbasedontheSTAARcutscores.Theoverallconsistencyrateofclassificationwascomputedastheproportionofcorrectclassificationsamongtheentiresampleby(TP+TN)/(TP+TN+FP+FN).

Page23of23

TABLEA2.DEFINITIONOFCONSISTENCYRATEFORSTAARTOMAPCONCORDANCE

STAARScore

BelowSTAARcut AtorAboveSTAARcut

MAPScoreBelowMAPcut TrueNegapve FalsePosipve

AtorAboveMAPcut FalseNegapve TruePosipveNote.Shadedcellsaresummedtocomputetheconsistencyrate.

ProficiencyProjection

MAPconditionalgrowthnormsprovidestudent’sexpectedgainscoresacrosstestingseasons(Thum&Hauser,2015).Thisinformationisutilizedtopredictastudent’sperformanceontheSTAARbasedonthatstudent’sMAPscoresinpriorseasons(e.g.fallandwinter).TheprobabilityofastudentachievingLevelII(Proficient)onSTAAR,basedonhis/herfallorwinterMAPscoreisgiveninEquation(A2):

𝑃𝑟 𝐴𝑐ℎ𝑖𝑒𝑣𝑒𝑖𝑛𝑔𝐿𝑒𝑣𝑒𝑙𝐼𝐼𝑖𝑛𝑠𝑝𝑟𝑖𝑛𝑔 𝑎𝑅𝐼𝑇𝑠𝑐𝑜𝑟𝑒𝑜𝑓𝑥) = 1 − Φ 𝑥 + 𝑔 − 𝑐

𝑆𝐷 (𝐴2)

where,Φisastandardizednormalcumulativedistribution,𝑥isthestudent’sRITscoreinfallorwinter,𝑔istheexpectedgrowthfromfallorwintertospringcorrespondingto𝑥,𝑐istheMAPcut-scoreforspring,and𝑆𝐷istheconditionalstandarddeviationofgrowthfromfallorwintertospring.

FortheprobabilityofastudentachievingLevelIIontheSTAARtests,basedonhis/herspringscore𝑠,itcanbecalculatedbyEquation(A3):

𝑃𝑟 𝐴𝑐ℎ𝑖𝑒𝑣𝑒𝑖𝑛𝑔𝐿𝑒𝑣𝑒𝑙𝐼𝐼𝑖𝑛𝑠𝑝𝑟𝑖𝑛𝑔 𝑎𝑅𝐼𝑇𝑠𝑐𝑜𝑟𝑒𝑜𝑓𝑠𝑖𝑛𝑠𝑝𝑟𝑖𝑛𝑔) = 1 − Φ 𝑠 − 𝑐𝑆𝐸 (𝐴3)

whereSEisthestandarderrorofmeasurementforMAPreadingormathtest.

Founded by educators nearly 40 years ago, Northwest Evaluation Association (NWEA) is a global not-for-profit educationalservices organization known for our flagship interim assessment, Measures of Academic Progress (MAP). More than 7,800partnersinU.S.schools,schooldistricts,educationagencies,andinternationalschoolstrustustoofferpre-kindergartenthroughgrade 12 assessments that accurately measure student growth and learning needs, professional development that fosterseducators’abilitytoacceleratestudentlearning,andresearchthatsupportsassessmentvalidityanddatainterpretation.Tobetterinforminstructionandmaximizeeverylearner’sacademicgrowth,educatorscurrentlyuseNWEAassessmentswithnearlyeightmillionstudents.

©NorthwestEvaluationAssociation2016.MeasuresofAcademicProgress,MAP,andPartneringtohelpallkidslearnareregisteredtrademarksofNorthwestEvaluationAssociationintheU.S.andinothercountries.NorthwestEvaluationAssociationandNWEAaretrademarksofNorthwestEvaluationAssociationintheU.S.andinothercountries.Thenamesofothercompaniesandtheirproductsmentionedarethetrademarksoftheirrespectiveowners.

Documents

Linking the Texas STAAR Assessments to NWEA MAP …€¦ · Linking the Texas STAAR Assessments to NWEA MAP Tests August 2016 Page 2 of 23 Introduction Northwest Evaluation Association™