Maryland School Assessment- Mathematicsarchives.marylandpublicschools.org/NR/rdonlyres/734553DF-3BE3-4E8C...Maryland School Assessment-Mathematics: Grades 3 through 8 2011 Administration

Maryland School Assessment-

Mathematics: Grades 3 through 8

Technical Report:

2011 Administration

Maryland School Assessment-Mathematics: Grades 3 through 8 2011 Administration

ii

TABLE OF CONTENTS

Table of Contents ......................................................................................................................... ii

List of Tables ............................................................................................................................... iii

List of Figures ............................................................................................................................ viii

Introduction .................................................................................................................................. 1

1. Overview of the 2011 Maryland School Assessment-Mathematics ..................................... 3

1.1 Purposes/Uses of the 2011 MSA-Math .............................................................................. 3

1.2 The State Curriculum ......................................................................................................... 4

1.3 Development and Review of the 2011 MSA-Math Items and Test ................................... 5

1.4 Test Form Design, Specifications, Item Types, and Item Roles ........................................ 7

1.5 Operational Test Form Construction Using the Rasch Model ......................................... 19

1.6 Test Administration of the 2011 MSA-Math ................................................................... 23

1.7 Hand Scoring Procedures of the 2011 MSA-Math .......................................................... 28

1.8 The 2011 MSA-Math Operational Item Analyses ........................................................... 36

1.9 Linking, Equating, and Scaling Procedures ................................................................... 110

1.10 Score Interpretation ...................................................................................................... 132

1.11 Test Validity of the 2011 MSA-Math .......................................................................... 133

1.12 Unidimensionality Analyses of the 2011 MSA-Math .................................................. 135

1.13 Field Test Analyses and Item Bank Construction ........................................................ 137

1.14 Quality Control Procedures .......................................................................................... 139

2. Current Performance Results of the 2011 MSA-Math ..................................................... 141

3. Overview of Statistical Summaries .................................................................................. 151

3.1 Classical Descriptive Statistics ....................................................................................... 151

3.2 Scale Score Descriptive Statistics .................................................................................. 153

3.3 Classical and Rasch (1-Parameter Logistic IRT) Item Parameters ................................ 153

3.4 Inter-Rater Reliability ..................................................................................................... 155

3.5 Correlations among Mathematics Standards .................................................................. 155

3.6 Decision Accuracy and Consistency at the Cut Scores .................................................. 156

3.7 Differential Item Functioning ......................................................................................... 156

3.8 Equating and Scaling ...................................................................................................... 161

4. The 2011 MSA-Math Statistical Summary ...................................................................... 165

References ................................................................................................................................ 227

Appendix A: The 2011 MSA-Math Stratified Random Sampling ........................................... 230

Appendix B: Scale Score Histograms and Tukey Charts ......................................................... 243

Appendix C: The 2011 MSA-Math Classical and Rasch Item Parameters .............................. 274

Appendix D: The 2011 MSA-Math Blueprints ........................................................................ 299


iii

LIST OF TABLES

Table 1.1. The 2011 MSA-Math Responsibility for Test Development .................................................... 6 Table 1.2. The 2011 MSA-Math Test Form Design: Grades 3 through 8 .................................................. 7 Table 1.3. Item Type of Content Standard for the 2011 MSA-Math: Grades 3 and 4 ............................. 10 Table 1.4. Item Type of Content Standard for the 2011 MSA-Math: Grades 5 and 6 ............................. 11 Table 1.5. Item Type of Content Standard for the 2011 MSA-Math: Grades 7 and 8 ............................. 12 Table 1.6. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 3 .................... 13 Table 1.7. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 3 .................. 13 Table 1.8. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 3 ........................... 13 Table 1.9. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 4 .................... 14 Table 1.10. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 4 ................ 14 Table 1.11. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 4 ......................... 14 Table 1.12. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 5 .................. 15 Table 1.13. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 5 ................ 15 Table 1.14. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 5 ......................... 15 Table 1.15. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 6 .................. 16 Table 1.16. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 6 ................ 16 Table 1.17. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 6 ......................... 16 Table 1.18. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 7 .................. 17 Table 1.19. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 7 ................ 17 Table 1.20. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 7 ......................... 17 Table 1.21. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 8 .................. 18 Table 1.22. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 8 ................ 18 Table 1.23. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 8 ......................... 18 Table 1.24. The 2011 Math Operational Test Construction Using the Rasch Model: Grade 5 Form A .. 20 Table 1.25. Descriptive Statistics for Form-to-Form Common Items ...................................................... 36 Table 1.26. P-Value Comparisons of Core Linking Items for Previous Year vs. Year 2011: Grade 3

Form A ............................................................................................................................................. 37 Table 1.27. P-Value Comparisons of Core Linking Items for Previous Year vs. Year 2011: Grade 3

Form F ............................................................................................................................................. 38 Table 1.28. P-Value Comparisons of Core Linking Items for Previous Year vs. Year 2011: Grade 4










Form F ............................................................................................................................................. 48 Table 1.38. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 3 Form A ..... 50


iv

Table 1.39. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous Year vs. Year 2011: Grade 3 Form A .............................................................................................. 51

Table 1.40. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year 2011: Grade 3 Form A ..................................................................................................................... 52

Table 1.41. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 3 Form F ...... 55 Table 1.42. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous

Year vs. Year 2011: Grade 3 Form F .............................................................................................. 56 Table 1.43. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year

2011: Grade 3 Form F ..................................................................................................................... 57 Table 1.44. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 4 Form A ..... 60 Table 1.45. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous

Year vs. Year 2011: Grade 4 Form A .............................................................................................. 61 Table 1.46. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year

2011: Grade 4 Form A ..................................................................................................................... 62 Table 1.47. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 4 Form F ...... 65 Table 1.48. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous














2011: Grade 7 Form F ..................................................................................................................... 97 Table 1.68. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 8 Form A ... 100 Table 1.69. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous

Year vs. Year 2011: Grade 8 Form A ............................................................................................ 101


v

Table 1.70. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year 2011: Grade 8 Form A ................................................................................................................... 102

Table 1.71. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 8 Form F .... 105 Table 1.72. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous

Year vs. Year 2011: Grade 8 Form F ............................................................................................ 106 Table 1.73. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year

2011: Grade 8 Form F ................................................................................................................... 107 Table 1.74. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 3 .......................... 113 Table 1.75. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 4 .......................... 116 Table 1.76. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 5 .......................... 119 Table 1.77. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 6 .......................... 122 Table 1.78. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 7 .......................... 125 Table 1.79. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 8 .......................... 128 Table 1.80. The 2011 MSA-Mathematic Slope and Intercept: Grades 3 through 8 ............................... 131 Table 1.81. The 2011 MSA-Math Eigenvalues between the First and Second Components ................. 136 Table 2.1. MSA-Math Cut Scores: Grades 3 through 8 ......................................................................... 142 Table 2.2. The 2011 MSA-Math Pass Rates: Grade 3 through Grade 8 ................................................ 142 Table 2.3. The 2011 MSA-Math Pass Rates by Gender: Grade 3 through Grade 8 ............................... 143 Table 2.4. The 2011 MSA-Math Pass Rates by Ethnicity: Grade 3 through Grade 8 ............................ 144 Table 2.5. The 2011 MSA-Math Pass Rates by LEA: Grade 3 .............................................................. 145 Table 2.6. The 2011 MSA-Math Pass Rates by LEA: Grade 4 .............................................................. 146 Table 2.7. The 2011 MSA-Math Pass Rates by LEA: Grade 5 .............................................................. 147 Table 2.8. The 2011 MSA-Math Pass Rates by LEA: Grade 6 .............................................................. 148 Table 2.9. The 2011 MSA-Math Pass Rates by LEA: Grade 7 .............................................................. 149 Table 2.10. The 2011 MSA-Math Pass Rates by LEA: Grade 8 ............................................................ 150 Table 3.1. Criteria to Evaluate Mean-Square Fit Statistics .................................................................... 155

Table 3.2. 2 T Contingency Table at the thk level .............................................................................. 157 Table 3.3. DIF Classification for BCR and ECR Items .......................................................................... 159 Table 3.4. DIF Classification for SR and SPR Items .............................................................................. 160 Table 4.1. The 2011 MSA-Math Classical Descriptive Statistics: Grades 3 through 8 ......................... 166 Table 4.2. The 2011 MSA-Math Scale Score Descriptive Statistics: Grades 3 through 8 ..................... 167 Table 4.3. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 3 .............................................. 168 Table 4.4. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 4 .............................................. 168 Table 4.5. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 5 .............................................. 169 Table 4.6. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 6 .............................................. 169 Table 4.7. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 7 .............................................. 170 Table 4.8. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 8 .............................................. 170 Table 4.9. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 3 ............................ 171 Table 4.10. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 4 .......................... 171 Table 4.11. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 5 .......................... 171 Table 4.12. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 6 .......................... 172 Table 4.13. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 7 .......................... 172 Table 4.14. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 8 .......................... 172 Table 4.15. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 3 ................. 173 Table 4.16. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 4 ................. 174 Table 4.17. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 5 ................. 175 Table 4.18. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 6 ................. 176 Table 4.19. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 7 ................. 177 Table 4.20. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 8 ................. 178 Table 4.21. The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 3 Form A

....................................................................................................................................................... 179


vi

Table 4.22. The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 3 Form F ....................................................................................................................................................... 181

Table 4.23. The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 3 Form A ....................................................................................................................................................... 183


















Table 4.41.The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 8 Form A ....................................................................................................................................................... 219




Table A.1. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 LEA ....................................................................................................................................................... 231

Table A.2. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 Ethnicity......................................................................................................................................... 232

Table A.3. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 Gender ........................................................................................................................................... 232


vii







Table A.10. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 6 LEA ............................................................................................................................................... 237









Table C.1. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 3 Form A ................... 275 Table C.2. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 3 Form F .................... 277 Table C.3. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 4 Form A ................... 279 Table C.4. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 4 Form F .................... 281 Table C.5. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 5 Form A ................... 283 Table C.6. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 5 Form F .................... 285 Table C.7. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 6 Form A ................... 287 Table C.8. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 6 Form F .................... 289 Table C.9. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 7 Form A ................... 291 Table C.10. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 7 Form F .................. 293 Table C.11. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 8 Form A ................. 295 Table C.12. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 8 Form F .................. 297 Table D.1. The 2011 MSA-Math Blueprint: Grade 3 ............................................................................ 300 Table D.2. The 2011 MSA-Math Blueprint: Grade 4 .............................. Error! Bookmark not defined. Table D.3. The 2011 MSA-Math Blueprint: Grade 5 .............................. Error! Bookmark not defined. Table D.4. The 2011 MSA-Math Blueprint: Grade 6 .............................. Error! Bookmark not defined. Table D.5. The 2011 MSA-Math Blueprint: Grade 7 .............................. Error! Bookmark not defined. Table D.6. The 2011 MSA-Math Blueprint: Grade 8 .............................. Error! Bookmark not defined.


viii

LIST OF FIGURES

Figure 1.1. Test Information Curves of Target Form vs. Current Year’s Math Operational Test Form .. 22 Figure 1.2. Standard Errors of Target Form vs. Current Year’s Math Operational Test Form ................ 22 Figure 1.3. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 3 Form A 54 Figure 1.4. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 3 Form F. 59 Figure 1.5. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 4 Form A 64 Figure 1.6. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 4 Form F. 69 Figure 1.7. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 5 Form A 74 Figure 1.8. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 5 Form F. 79 Figure 1.9. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 6 Form A 84 Figure 1.10. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 6 Form F

......................................................................................................................................................... 89 Figure 1.11. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 7 Form As

......................................................................................................................................................... 94 Figure 1.12. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 7 Form F

......................................................................................................................................................... 99 Figure 1.13. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 8 Form A

....................................................................................................................................................... 104 Figure 1.14. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 8 Form F

....................................................................................................................................................... 109 Figure 1.15. Anchor Evaluation Steps Chart for MSA-Math ................................................................. 112 Figure 1.16. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:

Grade 3 Form A ............................................................................................................................. 115 Figure 1.17. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:

Grade 3 Form F .............................................................................................................................. 115 Figure 1.18.Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:

Grade 4 .......................................................................................................................................... 118 Figure 1.19. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:

Grade 4 Form F .............................................................................................................................. 118 Figure 1.20. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:



Grade 6 .......................................................................................................................................... 124 Figure 1.23. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form:





Grade 8 Form F .............................................................................................................................. 130 Figure 3.1. Item Characteristic Curve .................................................................................................... 161 Figure 3.2. Category Response Curves for a One-Step Item.................................................................. 162 Figure 3.3. Category Response Curves for a Two-Step Item ................................................................. 163 Figure B.1. Year 2006 Scale Score Distribution: Grade 3 ..................................................................... 244 Figure B.2. Year 2011 Scale Score Distribution: Grade 3 Form A ........................................................ 245


ix

Figure B.3. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores with the Percent Differences between CDFs: Grade 3 Form A ..................................................... 246

Figure B.4. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores with the Cumulative Percent Differences between CDFs: Grade 3 Form A ................................. 246

Figure B.5.Year 2011 Scale Score Distribution: Grade 3 Form F ......................................................... 247 Figure B.6. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2009 Scale Scores

with the Percent Differences between CDFs: Grade 3 Form F ..................................................... 248 Figure B.7. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2009 Scale Scores

with the Cumulative Percent Differences between CDFs: Grade 3 Form F .................................. 248 Figure B.8. Year 2006 Scale Score Distribution: Grade 4 ..................................................................... 249 Figure B.9. Year 2011 Scale Score Distribution: Grade 4 Form A ........................................................ 250 Figure B.10. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores

with the Percent Differences between CDFs: Grade 4 Form A ..................................................... 251 Figure B.11. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores

with the Cumulative Percent Differences between CDFs: Grade 4 Form A ................................. 251 Figure B.12. Year 2011 Scale Score Distribution: Grade 4 Form F ...................................................... 252 Figure B.13. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores


with the Cumulative Percent Differences between CDFs: Grade 4 Form F .................................. 253 Figure B.15. Year 2006 Scale Score Distribution: Grade 5 ................................................................... 254 Figure B.16. Year 2011 Scale Score Distribution: Grade 5 Form A ...................................................... 255 Figure B.17. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores











with the Percent Differences between CDFs: Grade 7 Form F ..................................................... 268


x

Figure B.35. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores with the Cumulative Percent Differences between CDFs: Grade 7 Form F .................................. 268

Figure B.36. Year 2006 Scale Score Distribution: Grade 8 ................................................................... 269 Figure B.37. Year 2011 Scale Score Distribution: Grade 8 Form A ...................................................... 270 Figure B.38. Cumulative Distribution Functions (CDFs) for the Year 2006 vs. Year 2011 Scale Scores




with the Cumulative Percent Differences between CDFs: Grade 8 Form F .................................. 273


1

INTRODUCTION

The Maryland School Assessment (MSA) is a measure of students’ reading and mathematics comprehension. The MSA fulfills recommendations of the Visionary Panel for Better Schools and meets the federal testing requirements of the No Child Left Behind Act (NCLB) of 2001.

New academic standards were designed to inform parents, teachers, and educators of what students actually learned in schools and to make schools accountable for teaching contents measured by the MSA. To this end, the Maryland State Department of Education (MSDE), in collaboration with hundreds of educators across the state, developed a series of mathematics tests to measure students’ achievement against the new academic standards.

In 2003, the MSA-Math was introduced in grades 3, 5, and 8; grades 4, 6, and 7 were added to the program in 2004.

The purpose of the 2011 MSA-Math Technical Report is to provide users and other interested parties with a general overview and statistical analysis results of the MSA-Math.

The 2011 Technical Report is composed of four sections and four appendices.

The first section contains the following information:

General overview and purposes of the MSA-Math

Development and review of the MSA-Math items and test

Test form design, test form specifications, item type, and item roles

Operational form construction using the Rasch model

Test administration

Scoring procedures

Operational item analyses

Linking, equating, and scaling procedures

Score interpretation

Test validity and unidimensionality analyses

Field test analysis and item bank construction

Quality assurance

The second section provides the current year’s academic achievement results for grades 3 through 8. It contains information about the cutoff score and pass rate at each performance level for the 2011 math assessment. In addition, students’ performance levels were analyzed by key student subgroups such as gender, ethnicity, and LEA (Local Education Agency).

In the third section, we summarized detailed statistical procedures used for the 2011 math test. This section provides psychometric information in detail to those who might be interested in learning psychometric characteristics and procedures applied in the MSA-Math.

The last section contains statistical results of the 2011 MSA-Math. It includes descriptive statistics for the 2011 math test based on raw scores and scale scores, accuracy and consistency


2

of the 2011 math test, rater agreement rates, correlation coefficients among substrands, and total and substrand RS/SS conversion tables. Accordingly, this section provides the statistical and psychometric characteristics of the 2011 MSA-Math.

Four appendices provide additional statistical results for the 2011 MSA-Math: Appendix A contains stratified random sampling results; Appendix B contains 2011 MSA-Math scale score histograms and Tukey charts; Appendix C contains both classical and Rasch (one-parameter logistic item response theory) item parameters. Appendix D contains test blueprints for grades 3 through 8.


3

1. OVERVIEW OF THE 2011 MARYLAND SCHOOL ASSESSMENT-MATHEMATICS

In 2002, the Maryland State Department of Education (MSDE), in order to conform to the requirements of the new Federal program “No Child Left Behind,” retired its award-winning Maryland School Performance Assessment Program and adopted a testing program known as the Maryland School Assessment (MSA). The new program, like its predecessor, was based on the State Curriculum, which set reasonable academic standards for what teachers were expected to teach and for what students were expected to learn in schools.

In 2003, the MSA-Math was introduced in grades 3, 5, and 8, with grades 4, 6, and 7 being added to the program in 2004. A Bookmark standard setting was conducted in 2003 to set proficiency level cut scores for grades 3, 5, and 8. Because 2004 was the first testing year for grades 4, 6, and 7, a second Bookmark standard setting was held in summer 2004 to set cut scores for these additional grades. The performance level cut scores were used to assign students to three proficiency levels (Basic, Proficient, and Advanced) for AYP reporting under the “No Child Left Behind” act. Information about the Bookmark procedures and results can be obtained from MSDE. The same cut scores have been applied since 2003 (grades 3, 5, and 8) and 2004 (grades 4, 6, and 7).

It should be noted that in 2007, the MSA-Math was administered using a new vendor and applying a different IRT method (e.g., the Rasch model); therefore, a transformation of scale scores using the equipercentile method was conducted with the 2006 population data. Detailed information on the scale score transformation and its results can be found in Appendix C, Year 2006 MSA-Math Recalibration Results from 3-PL IRT to the Rasch Model Using the Equipercentile Method in the 2007 MSA-Math Technical Report.

In 2007, MSDE implemented an important action plan on the MSA-Math test: dropping all of the Stanford Achievement Test Series, Tenth Edition (Stanford 10) items from the 2008 assessment. Consequently, several Stanford 10 items which contributed to the 2007 criterion-referenced test (CRT) were replaced by Maryland-specific items in 2008.

For the purposes of year-to-year linking and equating, we first constructed a 2011 linking pool; only operational selected-response (SR) items (i.e., multiple-choice items) were selected in the linking pool. It should be noted that these SR items appeared both in current and previous years’ assessments and were used as either core or core link items in previous years’ assessments (i.e., in any assessment before 2011). After setting up the linking pool, we conducted a stability check of linking items and decided which items should be excluded from or which items should remain in the linking pool. During the calibration and equating processes, we kept the original operational Rasch item difficulty parameters of any linking items that remained through the stability check to put the 2011 assessment on a common scale. Accordingly, scale scores of the 2011 assessment were linked back to the 2006 assessment and all the scale scores of different years were comparable within each content and grade.

1.1 Purposes/Uses of the 2011 MSA-Math

By measuring students’ achievement against the new academic standards, the 2011 MSA-Math fulfills two main purposes. First, the MSA-Math was designed to inform parents, teachers, and educators of what students actually learned in schools by providing specific feedback that can be


4

used to improve the quality of schools, classrooms, and individualized instructional programs and to model effective assessment approaches that can be used in classrooms. Second, the MSA-Math serves as an accountability tool to measure performance levels of individual students, schools, and districts against the new academic standards.

1.2 The State Curriculum

Federal law requires that states align their tests with their state content standards. MSDE worked carefully and rigorously to construct new tests to provide a strong alignment as defined by the U.S. Department of Education.

The State Curriculum (SC), which defined what students should know and be able to do at each grade level, helped schools understand the standards more clearly, and included more specificity with indicators and objectives. The format of the SC specified standards statements, indicators, and objectives. Standards are broad, measurable statements of what students should know and be able to do. Indicators and objectives provide more specific content knowledge and skills that are unique at each grade level.

The objectives assessed by the MSA at each grade level are embedded in the SC. In addition, they are identified with the notation assessment limit. Assessment limits provide clarification about the specific skills and content that students are expected to have learned for each assessed objective. Even though some objectives in the SC may not have an assessment limit at a given grade-level, these non-assessed objectives still must be included in instruction. They introduce important concepts in preparation for assessed skills and content at subsequent grade levels.

The following provides one example of assessment limit of Grade 3 MSA-Math:

STANDARD 1.0

Knowledge of Algebra, Patterns, and Functions

TOPIC:

A. PATTERNS AND FUNCTIONS

INDICATOR:

1. Identify, describe, extend, and create numeric patterns and functions

OBJECTIVES:

a. Represent and analyze numeric patterns using skip counting

Assessment limits:

Use 2, 5, 10, or 100 starting with any whole number (0 – 1000)

It should be noted that it was not the case that every indicator would necessarily be tested each year even if 100% of the standards should be tested. Consequently, the SC specified curricular indicators and objectives that contributed directly to measuring content standards, which were aligned to the MSA. More information on assessment limits and standards can be found in Appendix D, The 2011 MSA-Math Blueprint.


5

1.3 Development and Review of the 2011 MSA-Math Items and Test

As seen in Table 1.1, the development of the 2011 MSA-Math test required the involvement of four groups in addition to MSDE and Pearson. These groups are as follows:

National Psychometric Council

The National Psychometric Council (NPC) took a major role in reviewing and making recommendations to MSDE on the development and implementation of the 2011 MSA-Math program. For example, they made recommendations to MSDE on issues, such as test blueprints, operational form construction, field test design, item analysis, item selection for scoring purposes, linking, equating and scaling issues, and other relevant statistical and psychometric issues.

Content Review Committee

Content Review Committee members ensured that the MSA-Math was appropriately difficult and fair. Committee members were either specialists in math for test items or experts in test construction and measurement. They represented all levels of education as well as the ethnic and social diversity of Maryland students. Committee members were from different areas of the state.

The educators’ understanding of Maryland curriculum and extensive classroom experience made them a valuable source of information. They reviewed test items and forms and took a holistic approach to ensure that tests were fair and balanced across reporting categories.

Bias Review Committee

In addition to the Content Review Committee, a separate Bias Review Committee examined each item on math tests. They looked for indications of bias that would impact the performance of an identifiable group of students. Committee members discussed and, if necessary, rejected items based on gender, ethnic, religious, or geographical bias.

Vision Review Committee

A Vision Review Committee reviewed the items and any associated art for bias to the visually impaired. The committee made their recommendations to exclude from Form A any item they had a concern with since this form is usually used for large print and Braille forms.


6

Table 1.1. The 2011 MSA-Math Responsibility for Test Development

Development of the 2011 MSA-Math Primary Responsibility

Development of Preliminary Blueprints and Item Specifications

Pearson; MSDE; NPC

Development of Operational Form Requirement and Session Blueprint

MSDE

Item Writing MSDE; Pearson

Item Review Pearson; MSDE; NPC; Content Review Committee

Bias Review Pearson; MSDE; Bias Review Committee

Vision Review Pearson: MSDE; Vision Review Committee

Modification of Special Forms Pearson; MSDE

Review of Special Forms MSDE

Construction of Operational Test Forms Pearson; MSDE; NPC

Construction of Field Test Forms Pearson; MSDE

Review of Operational Test Forms MSDE; NPC

Final Construction of Test Forms Pearson; MSDE


7

1.4 Test Form Design, Specifications, Item Types, and Item Roles

Test Form Design

The MSA-Math test had two forms of operational items at each grade. Field test items were embedded within the operational items resulting in a total of 10 test forms at each grade. As can be seen from Table 1.2, Forms A, B, C, D and E are identical with respect to operational items (designated as operational Form A) and differ only with respect to field test items. This is also true for Forms F, G, H, J, and K (designated as operational Form F).

Table 1.2. The 2011 MSA-Math Test Form Design: Grades 3 through 8

Operational Item Sets Field test Item Sets

A F A B C D E F G H J K

Form A X X

Form B X X

Form C X X

Form D X X

Form E X X

Form F X X

Form G X X

Form H X X

Form J X X

Form K X X

Note. Forms A, B, C, D, and E (Form A) are identical, and Forms F, G, H, J, and K (Form F) are identical in terms of operational test items.

Test Form Specifications and Reporting Category

Tables 1.3, 1.4, and 1.5 provide information on the total number of operational items included in each operational test form and how these items were broken down based on each content standard. It should be noted that the test specifications in these tables represent the targeted test design for each grade and show the targeted distribution of each content standard.

Mathematics has a total of seven content standards (Algebra, Geometry, Measurement, Statistics, Probability, Numbers and Computation, and Process). Some standards have been combined for purposes of reporting subscale. Specifically, Geometry and Measurement standards and Statistics and Probability standards were combined to produce a total of five subscale reporting categories. Tables 1.6 through 1.23 provide information on the actual distribution of score points by standard and reporting category. The number of items and score points for each reporting standard were identical across forms within each grade.


8

Item Types The 2011 MSA-Math included four types of items: selected response (SR), student-produced response (SPR), brief constructed response (BCR), and extended constructed response (ECR).

SR items require students to select a correct answer from several alternatives. For the 2011 MSA-Math, students selected an answer from four options. Each SR item was scored dichotomously (i.e., 0 or 1).

SPR items require students to record their answers on a grid by shading in circles corresponding to the numbers in their answer. For the 2011 MSA-Math, only grade 7 and 8 tests included SPR items. Each SPR item was scored dichotomously.

BCR items require students to provide a short answer using words, numbers, and/or symbols, while ECR items require students to write an answer that consists of more information than is required for a brief constructed response item.

Both BCR and ECR items consist of Step A and Step B. Step A contributes to the content score while Step B contributes to the process score. Each step was considered as an independent item and separately scored;

All BCR and ECR Step A items received a 0-1 score point range from two independent scorers; all BCR Step B items received a 0-2 score point range; all ECR Step B items received a 0-3 score point range from two independent scorers. The score given was the higher of the first and the second Reader’s scores, provided they were adjacent. A resolution reader’s score was used if two non-adjacent initial scores were received. That is, the resolution reader’s score was used in place of both the first and second Reader’s scores. It should be noted that grade 3 and 4 tests did not include ECR items.

The Role of Operational SR Item

Most SR items were used for both form-to-form and year-to-year calibration and linking. As a result, operational SR items fell into one of the following four categories: unique core, common core, unique core linking, and common core linking items. Form-to-form linking was conduced with both the common core and the common core linking items. The form-to-form calibration and linking procedures can be found in section 1.9, Form-to-Form Linking Procedures. More importantly, however, year-to-year linking was conduced with only the core linking items and year-to-year calibration and linking procedures can be found in section 1.9, Year-to-Year Linking Procedures.

While unique core items appeared on either operational form A or F, common core items appeared on both forms. As a result, only the common core items were used for form-to-form linking. Because the core items were not included into the possible 2011 linking pool, on the other hand, item parameters of these items were recalibrated with the 2011 live operational data (i.e., stratified random sample) and then reserved in the 2011 Maryland item bank for the possible use as core linking items in the future. Classical and Rasch analyses on these core items can be found in section 1.8, Validation Check with the 2011 Core Items.

While a few core linking items appeared only on either operational form A or F (i.e., unique core linking), most core linking items (i.e., common core linking) appeared on both operational forms. As a result, the common core linking items appearing on both forms were used for both form-to-


9

form and year-to-year linking. The unique core linking items were used only for year-to-year linking.

The role of the core linking items was to place the 2011 scale on the 2006 scale. Because these core linking items carried their operational item parameters on the 2006 scale, they were included in the 2011 year-to-year linking pool. Classical analysis on these items can be found in section 1.8, P-Value Check with Year-to-Year Core Linking Items, and calibration, linking and equating procedures on these core linking items can be found in section 1.9, Linking, Equating, and Scaling Procedures of the 2011 MSA-Math.

The Role of Operational SPR, BCR, and ECR Items

SPR, BCR, and ECR items were divided into one of the following two categories: unique core or common core items. Only the common core items appearing on both operational forms were used for form-to-form calibration and linking. Because these items were not included in the 2011 year-to-year linking pool, new Rasch item and step difficulty parameters were estimated with the 2011 live, operational data set (i.e., stratified random sample). These new item and step difficulty parameters were used to produce each student’s theta estimate. More detailed information about how much these items changed across years in terms of classical and Rasch item difficulty can be found in section 1.8, Validation Check with the 2011 Core Items.


10

Table 1.3. Item Type of Content Standard for the 2011 MSA-Math: Grades 3 and 4

Grade Standard Item Type No. of Items of Each Form

A F

3 65 65

Algebra SR, BCR 13 13

Geometry SR, BCR 8 8

Measurement SR, BCR 7 7

Statistics SR, BCR 12 12

Probability SR 2 2

Number Computation SR, BCR 16 16

Process BCR 7 7

4 64 64

Algebra SR, BCR 14 14




Probability SR, BCR 7 7


Process BCR 7 7

Note. SR items are selected response items, and BCR items are brief constructed response items. Form A designates the forms A, B, C, D, and E. Form F designates the forms F, G, H, J, and K.


11



A F

5 65 65

Algebra SR, BCR, ECR 15 15






Process BCR, ECR 8 8

6 62 62

Algebra SR, BCR, ECR 14 14







Note. SR items are selected response items, BCR items are brief constructed response items, and ECR items are Extended Constructed Response. Form A designates the forms A, B, C, D, and E. Form F designates the forms F, G, H, J, and K.


12



A F

7 62 62

Algebra SR,SPR, BCR, ECR 14 14

Geometry SR, SPR, ECR 7 7

Measurement SR, SPR, BCR 6 6

Statistics SR, SPR, BCR, ECR 9 9

Probability SR, SPR, BCR 5 5

Number Computation SR, SPR 14 14


8 62 62

Algebra SR,SPR, BCR, ECR 15 15

Geometry SR, SPR, ECR 8 8

Measurement SR, SPR, BCR 5 5

Statistics SR, SPR, BCR, ECR 9 9

Probability SR, SPR, BCR 5 5

Number Computation SR, SPR 12 12


Note. SR items are selected response items, SPR items are student-produced response, BCR items are brief constructed response items, and ECR items are extended constructed response. Form A designates the forms A, B, C, D, and E. Form F designates the forms F, G, H, J, and K.


13

Table 1.6. Item Distribution of Each Content Standard for the 2011 MSA-Math: Grade 3

Form Total Item Number of Each Standard

Total # of Item 1* 2* 3* 4* 5* 6* 7*

A 13 8 7 12 2 16 7 65

F 13 8 7 12 2 16 7 65

Note. 1*. Algebra; 2*. Geometry; 3*. Measurement; 4*. Statistics; 5*. Probability; 6*. Numbers and Computation; 7*. Process

Table 1.7. Total and Reporting Content Standard Scores for the 2011 MSA-Math: Grade 3

Form Total and Reporting Standard Scores

1 2&3 4&5 6 7 Total Score

A 13 15 14 16 14 72

F 13 15 14 16 14 72

Table 1.8. Item Type and Score Point Distribution for the 2011 MSA-Math: Grade 3

Form # of

SR Item

# of BCR Item Total # of Item

Scores of SR Scores of BCR Total

Score Step A Step B Step A Step B

A 51 7 7 65 51 7 14 72

F 51 7 7 65 51 7 14 72


14


Form Total Item Number of Each Standard Total # of

Item 1* 2* 3* 4* 5* 6* 7*

A 14 7 7 8 7 14 7 64

F 14 7 7 8 7 14 7 64





A 14 14 15 14 14 71

F 14 14 15 14 14 71


Form # of

SR Item

# of BCR item Total # of Item

Scores of SR Item

Scores of BCR

Total Score Step A Step B Step A Step B

A 50 7 7 64 50 7 14 71

F 50 7 7 64 50 7 14 71


15



Item 1* 2* 3* 4* 5* 6* 7*

A 15 6 8 9 4 15 8 65

F 15 6 8 9 4 15 8 65





A 15 14 13 15 17 74

F 15 14 13 15 17 74


Form

# of

SR Item

# of BCR Item # of ECR Item Total # of Item

Scores of SR

Scores of BCR Scores of ECR Total

Score Step A Step B Step A Step B Step A Step B Step A Step B

A 49 7 7 1 1 65 49 7 14 1 3 74

F 49 7 7 1 1 65 49 7 14 1 3 74


16



Item 1* 2* 3* 4* 5* 6* 7*

A 14 8 6 9 4 14 7 62

F 14 8 6 9 4 14 7 62





A 14 14 13 14 15 70

F 14 14 13 14 15 70


Form # of

SR Item

# of BCR Item # of ECR Item Total # of Item

Scores of SR

Scores of BCR Scores of ECR Total

Score Step A Step B Step A Step B Step A Step B Step A Step B

A 48 6 6 1 1 62 48 6 12 1 3 70

F 48 6 6 1 1 62 48 6 12 1 3 70


17



Item 1* 2* 3* 4* 5* 6* 7*

A 14 7 6 9 5 14 7 62

F 14 7 6 9 5 14 7 62





A 14 13 14 14 17 72

F 14 13 14 14 17 72


Form # of SR

Item

# of SPR Item

# of BCR Item

# of ECR Item Total # of Item

Scores of SR

Scores of

SPR

Scores of BCR Scores of

ECR Total

Score Step

A Step

B Step

A Step

B Step

A Step

B Step

A Step

B

A 36 12 4 4 3 3 62 36 12 4 8 3 9 72

F 36 12 4 4 3 3 62 36 12 4 8 3 9 72


18



Item 1* 2* 3* 4* 5* 6* 7*

A 15 8 5 9 5 12 8 62

F 15 8 5 9 5 12 8 62





A 15 13 14 12 19 73

F 15 13 14 12 19 73


Form # of SR

Item

# of SPR Item

# of BCR Item

# of ECR Item Total # of Item

Scores of SR

Scores of

SPR

Scores of BCR Scores of

ECR Total

Score Step

A Step

B Step

A Step

B Step

A Step

B Step

A Step

B

A 34 12 5 5 3 3 62 34 12 5 10 3 9 73

F 34 12 5 5 3 3 62 34 12 5 10 3 9 73


19

1.5 Operational Test Form Construction Using the Rasch Model

The selection of items to be included in the final operational test forms of the 2011 MSA-Math required a careful consideration based on test blueprints, classical item analyses, DIF analyses, and IRT analyses. Specifically, the Rasch model (i.e., one-parameter logistic IRT) played a major role in constructing the 2011 operational forms. First, Pearson suggested the following guidelines:

Do not include items that are too easy or too hard.

Do not include BCR items with score distributions that do not elicit the full range of rubric scores.

Do not include items with DIF classifications “C” for the SR items and “CC” for the BCR items unless they have been deemed acceptable by the external review of content experts.

Finally, do not include items which have Rasch Infit and Outfit mean-squares lower than 0.5 or higher than 1.5. More specific information on Rasch Infit and Outfit mean-squares can be found in the third part of the 2011 technical report, Overview of Statistical Summaries.

A procedure for using IRT methods to build tests that meet any desired set of test specifications was outlined by Lord (1977). The procedure utilizes an item bank with item parameter estimates available for the IRT model of choice, with accompanying information functions. The steps in the procedure suggested by Lord (1977) are as follows:

First, the shape of desired test information needs to be decided. This was termed as the “target information function” by Lord (1977).

Second, specific items need to be selected from the item bank with item information functions that will fill up hard-to-fill areas under the target information function.

Third, the test information function after test items are added needs to be recalculated. Fourth, until the test information function approximates the target information function to

a satisfactory degree, test items need to keep on being selected.

It should be noted that these steps were implemented within a framework defined by the content specification of the test. In addition, math content specialists from MSDE reviewed the final test forms of the 2011 MSA-Math. The following table and figure show an example of the 2011 MSA-Math operational test form construction using the Rasch (i.e., 1-PL IRT) method. Detailed information about constructing operational forms using the Rasch method can be obtained from either MSDE or Pearson.


20

Table 1.24. The 2011 Math Operational Test Construction Using the Rasch Model: Grade 5 Form A

Item Type P-value A 1iD 2iD 3iD

SR 0.90 1.00 -1.2053

SR 0.55 1.00 0.7908

SR 0.59 1.00 0.5836

SR 0.58 1.00 0.6958

SR 0.88 1.00 -1.2818

SR 0.75 1.00 -0.2844

ECR_A 0.94 1.00 -2.4767

ECR_B 0.68 1.00 -2.1132 0.0348 1.4706

SR 0.66 1.00 0.2299

SR 0.92 1.00 -1.4432

SR 0.78 1.00 -0.4703

SR 0.64 1.00 0.2578

SR 0.63 1.00 0.3916

SR 0.80 1.00 -0.6463

SR 0.62 1.00 0.4042

SR 0.77 1.00 -0.6135

BCR_A 0.30 1.00 2.0661

BCR_B 0.36 1.00 0.6112 2.7674

SR 0.86 1.00 -1.1173

SR 0.64 1.00 0.3609

SR 0.73 1.00 -0.1396

SR 0.66 1.00 0.2101

SR 0.53 1.00 0.8950

SR 0.67 1.00 -0.0202

SR 0.78 1.00 -0.6658

BCR_A 0.38 1.00 1.6930

BCR_B 0.54 1.00 0.4994 1.1810

SR 0.63 1.00 0.2597

SR 0.53 1.00 1.1378

SR 0.86 1.00 -1.4702

SR 0.82 1.00 -0.8511

SR 0.62 1.00 0.5144

SR 0.48 1.00 1.1803

SR 0.61 1.00 0.5095

SR 0.83 1.00 -0.9261

SR 0.67 1.00 -0.0107

BCR_A 0.70 1.00 -0.0954

BCR_B 0.68 1.00 -0.1225 0.1813

SR 0.42 1.00 1.4294


21

Table 1.24 (Continued)

Item Type P-value A 1iD 2iD 3iD

SR 0.53 1.00 0.8786 SR 0.69 1.00 -0.0834

SR 0.70 1.00 0.0311

SR 0.73 1.00 -0.3249

SR 0.62 1.00 0.2864

SR 0.55 1.00 0.8127

SR 0.45 1.00 1.0083

SR 0.53 1.00 0.8821

SR 0.75 1.00 -0.2639

BCR_A 0.45 1.00 1.3128

BCR_B 0.78 1.00 -2.0263 0.3587

SR 0.66 1.00 0.2830

SR 0.85 1.00 -1.1480

BCR_A 0.41 1.00 1.3489

BCR_B 0.48 1.00 -0.0256 2.0230

SR 0.55 1.00 0.4071

SR 0.80 1.00 -0.9339

SR 0.78 1.00 -0.5602

SR 0.49 1.00 1.0878

BCR_A 0.43 1.00 1.3038

BCR_B 0.55 1.00 -1.1435 2.3563

SR 0.55 1.00 0.4897

SR 0.60 1.00 0.5523

Note. A: item discrimination estimate; 1iD : first structure measure estimate; 2iD : second structure measure

estimate; and 3iD : third structure measure estimate

Note. Please refer to section 3.3 of this technical report to get detailed information about how to estimate structure

measure estimate ( ijD = iD + ijF )

Note. BCR_A and ECR_A: Step A of BCR and ECR; BCR_B and ECR_B: Step B of BCR and ECR item.

Note. SR: Selected Response item; BCR: Brief Constructed Response item; ECR: Extended Constructed Response item.


22

Figure 1.1. Test Information Curves of Target Form vs. Current Year’s Math Operational Test Form

Figure 1.2. Standard Errors of Target Form vs. Current Year’s Math Operational Test Form

Grade 5 Test Information Curve

0

2

4

6

8

10

12

14

16

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Theta

Info Target

FormA

Grade 5 Conditional Standard Error Curve

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Theta

SE

M Target

FormA


23

1.6 Test Administration of the 2011 MSA-Math

The 2011 MSA-Math test was administered to all students in grades 3 through 8 except for students taking the Alt-MSA-Math or the Mod-MSA-Math. Pearson coordinated the test administration procedures with MSDE prior to implementation. This chapter was prepared to provide general information about the 2011 test administration. Detailed information about the 2011 test administration can be obtained from the 2011 Test Administration and Coordination Manual (TACM) and Examiner’s Manual (EM), which are available from either MSDE or Pearson.

Test Materials

All test materials had to be stored in a secure location prior to test administration. The School Test Coordinator (STC) provided test administration training and test materials to the test examiners. The Daily Testing Materials Tracking Record (or an equivalent form designed by the LEA) was used to track the distribution and return of Test Books.

Before testing began, the Test Examiners (TEs) carefully inventoried all test materials given to them, as they were accountable for the return of all secure materials at the end of testing. The TEs checked to ensure they have all the materials they needed for testing.

For the Test Examiner, Pearson provided the following materials:

MSA Examiner’s Manual for grades 3 through 8-Math

Pre-printed labels

For each student, the following materials were provided by Pearson:

Test/Answer Book

Special accommodations testing materials, if necessary

For each student, the following additional materials were provided by school or student:

Two No. 2 pencils with erasers

Blank scratch paper

Classroom Calculator for Day 1 (all grades)

Classroom ruler with both U.S. customary and metric measurements (all grades)

Classroom protractor for grades 5 through 8

Classroom compass for grades 7 and 8 only

Each classroom used for the assessment also needed the following additional materials:

A sign for the door that reads "Testing: Do not Disturb"

A digital clock or a watch, or clock with a second hand

Two test-related Examiner’s Manuals (EMs) were developed for the 2011 MSA: one version for reading and the other for mathematics for use in all grades 3-8. Developed in partnership with MSDE, the EMs contained instructions for preparation and administration of the test. In addition to the EMs, one Test Administration and Coordination Manual (TACM) was developed for use


24

by the Local Accountability Coordinators (LAC) and building-level School Test Coordinators (STC). Included in this manual were instructions for preparation of materials for testing, monitoring of testing, and packaging of materials for return to Pearson for scoring. The TACM was distributed and reviewed during a workshop in January for STCs and LACs, with duplicates sent to each school along with its testing materials.

Test Administration Schedule

The primary test window for MSA was established by MSDE (March 7-16, 2011, with make-up testing held March 17-23, 2011). However, each LEA (Local Education Agency) set a specific schedule for administration of the MSA within that window for their district. For a given test, grade, content area, and test format, all testing (with the exception of the make-up administration) had to take place on the same schedule. Each LEA schedule was submitted to MSDE in advance and approved for each district by the state. For example, all Grade 3 MSA-Math must be administered on the same days throughout the LEA. In addition, each content area in each grade was tested on two days during the window. In any given grade, one content area’s primary testing window was completed before beginning the second content area’s primary testing window.

The MSA-Math testing schedule allowed approximately 2 hours and 30 minutes for testing on Day 1 and 1 hour and 45 minutes on Day 2 (including preparation time and breaks).

For the 2011 MSA-Math, the primary testing days were as follows:

Test materials delivered to schools

(Examiner’s Manuals, Test/Answer Books,

and Test Coordinator’s Kits)

February 16-21, 2011

Mathematics Primary Testing Window March 7 – March 16, 2011

Make-up Testing Window March 17 – March 23, 2011

If a student was absent on the testing days, a make-up test was administered on any two consecutive days within the testing window. If a school had an unscheduled closing or delayed opening that prohibited the administration from occurring on the scheduled testing dates, the STCs were consulted by LACs to determine the testing schedule to be followed.

During the administration of the 2011 MSA-Math, MSDE had testing monitors in selected schools observing administration procedures and testing conditions. All monitors had identification cards for security purposes. There was no prior notification of which schools would be monitored, but monitors followed local procedures for reporting to the school’s main office and giving proper notification that an MSDE monitor was in the building.

Student Participation

MSDE calculates actual participation of students who took the test. This means that the schools are held accountable not only for student achievement on MSA or Mod-MSA testing, but also they are accountable to ensure that at least 95% of students participate in testing. Accordingly, schools should do all they can to test all students on MSA, Mod-MSA, or Alt-MSA, as applicable.


25

All students in grades 3 through 8 had to participate in either the 2011 MSA-Math or, if determined to be eligible by the student’s IEP, the Mod-MSA-Math. The only exception was that students with severe cognitive disabilities were assessed by the Alternate Maryland School Assessment (Alt-MSA) instead of the MSA-Math or Mod-MSA-Math. The criteria that students need in order to be tested in the Alt-MSA program instead of the MSA-Math or Mod-MSA-Math can be viewed in section 5, Appendix A of the TACM.

Participation of English Language Learners (ELLs) in the MSA-Math or the Mod-MSA-Math

There are special rules that apply to the participation of English Language Learners (ELLs) in the MSA-Math and the Mod-MSA-Math, as follows:

For the MSA-Math and Mod-MSA-Math, ELL students must participate in MSA-Math or Mod-MSA-Math regardless of how recently they entered the U.S. educational system. For ELL students in their first year of enrollment in a U.S. school, “participation” in the MSA-Math or the Mod-MSA-Math is defined as allowing the student to attempt the test for at least 20 minutes. If, after 20 minutes, the TE determines in his or her professional judgment that the student does not possess sufficient English fluency to be able to continue testing, the test administration for that student may be concluded at that time.

Accommodations for Assessment

Accommodations for assessment of students with disabilities (i.e., students having an Individualized Education Program or a Section 504 Plan) and students for English Language Learners (ELL) had to be approved and documented according to the procedures and requirements outlined in the document entitled Maryland Accommodations Manual: A Guide to Selecting, Administrating, and Evaluating the Use of Accommodations for Instruction and Assessment (MAM). A copy of the most recent edition of this document is available electronically on the LAC and STC web pages at https://docushare.msde.state.md.us/docushare.

No accommodations could be made for students merely because they were members of an instructional group. Any accommodation had to be based on individual needs and not on a category of disability area, level of instruction, environment, or other group characteristics. Responsibility for confirming the need and appropriateness of an accommodation rested with the LAC and school-based staff involved with each student’s instructional program. A master list of all students and their accommodations had to be maintained by the principal and submitted to the LAC, who provided a copy to MSDE upon request. Please refer to Section 1 of the 2011 TACM for further information regarding testing accommodations.

Large-Print and Braille Test Books and KurzweilTM Test Forms on CD

The MSA-Math was administered to those requiring (1) large-print Student Test/Answer Books or (2) Braille Test Books, or (3) KurzweilTM Test Forms on CD for a verbatim reading accommodation. For large-print Test/Answer Books, Braille Test Books, and KurzweilTM Test Forms on CD, student responses were transcribed into the standard-size Test/Answer Book following testing.

The student’s name, LEA number, and school number were written on the large-print Test/Answer Book for proper transcription into the standard-size Test/Answer Book.


26

The pre-printed student ID label was affixed to the standard-size Test/Answer Book containing the transcribed responses, and not to the large-print Test/Answer Book or Braille books. The bubbles on the demographic page of the standard-size Test/Answer Book were not filled in if there was a pre-printed student ID label for the student.

A certified Test Examiner (TE) transcribed the student responses into a standard-size Test/Answer Book exactly as given by the student. The standard-size Test/Answer Book with the pre-printed label attached was returned to Pearson with all other Test/Answer Books.

Large-Print Test/Answer Books and Braille Test/Answer Books containing the original student responses prior to transcription are to be returned with Non-Scorable materials. Any Test/Answer Books which were used as source documents for transcription were invalidated by drawing a large slash across the student demographic page with a black permanent marker.

Once the student responses had been transcribed, the transcribed Test/Answer Book was returned for scoring with the standard-size materials. Specific packing instructions are provided in the 2011 TACM in sections 2 and 3.

Verbatim Reading Accommodation and KurzweilTM Test Form on CD

Students who had a verbatim reading accommodation documented in their Individual Education Plan (IEP), ELL Plan, or Section 504 Plan, and who received that accommodation in regular instruction, received the accommodation on the 2011 MSA-Math. The accommodation was provided by a live reader or through technology. Appendix L of the 2011 TACM provided information on verbatim reading instruction. Technology used to provide the verbatim reading accommodation was KurzweilTM reading software. Official, secure electronic copies of the test were ordered through the LAC. MSDE encouraged (but did not require) the use of the KurzweilTM software to ensure uniformity in the delivery of the verbatim reading accommodation throughout the state.

Students using KurzweilTM software had to familiarize themselves with its operation prior to the test administration. When there were technical difficulties with KurzweilTM a certified staff member was used instead. KurzweilTM Test Form CDs were shipped by Pearson. After testing, schools returned the CDs to Pearson with the non-scorable secure materials.

Administration Procedures for Students with IEP, 504 Plan, or ELL Plan Permitting a Dictated Responses or Use of Word Processor

A student whose IEP, 504 Plan, or ELL Plan permitted a dictated response had his/her responses transcribed at the school level by an eligible TE, or by a staff member working under the direct supervision of a certified TE, into the student’s Test/Answer Book with a pre-printed ID label attached.

A student whose IEP, 504 Plan, or ELL plan permitted the use of a word processor had his/her responses transcribed by hand or under the direct supervision of an eligible TE or STC exactly as the student entered his/her responses on the word processor. The student’s responses were always transcribed at the school level into the student’s Test/Answer Book with the pre-printed ID label attached. After the student’s responses had been transcribed, the memory of the word processor was cleared. The original word-processed print-out was returned to Pearson with the non-scorable materials.


27

Test Format

All grade levels of the MSA-Math used a Test Book format in which students wrote their answers directly in the Test Book. There were 10 forms of MSA-Math. Different test forms were administered to students in each classroom participating in math tests, and each test form was identified by color and form number/letter. All forms of the MSA Test/Answer Books for each grade had the same grade designation and picture on the front cover. The Test/Answer Books were spiraled within a classroom, and each student used a combined Test/Answer Book.

Since the Test/Answer Books were scanned for scoring, students were encouraged not to use highlights in any part of the book. Although students might be accustomed to using highlighters in daily instruction, highlighting in the Test/Answer Book could obliterate information in a student’s book when it was scanned for scoring. As an alternative to highlighting, students were allowed to lightly circle or underline information in test items or perform calculations to help them in responding, as long as markings did not interfere with the bubbled answer choice area and/or the track marks along the outside margins of each page.

Security of Test Materials

The following code of ethics conforms to the Standards for Educational and Psychological Testing developed by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education (Pearson, 2011):

It is breach of professional ethics for school personnel to provide verbal or nonverbal clues or answers, teach items on the test, share writing prompts, coach, hint, or in any way influence a student’s performance during the testing situation. A breach of ethics may result in invalidation of test results and local education agency (LEA) or MSDE disciplinary action. (p. 11)

The Test/Answer Books for the 2011 MSA-Math were confidential and kept secure at all times. Unauthorized use, duplication, or reproduction of any or all portions of the assessment was prohibited, which is reflected by the following statement (Pearson, 2011):

Violation of security can result in prosecution and/or penalties as imposed by the Maryland State Board of Education and/or State Superintendent of Schools in accordance with the COMAR 13A.03.04 and 13A.12.05. (p. 11)

All materials were treated as confidential and placed in locked areas. Secure and non-secure test materials were as follows:

Secure materials: Test/Answer Books (including large-print and Braille), KurzweilTM test forms on CD, and used scratch paper

Non-secure materials: TACM, Examiner’s Manuals, unused pre-printed student ID labels, unused bill of ladings, and unused green/orange shipping labels


28

1.7 Hand Scoring Procedures of the 2011 MSA-Math

Students’ responses to SR and SPR items were machine-scored, and their responses to BCR and ECR items were individually read and scored by Pearson.

Hand Scoring Staff

The PSC Project Manager (PSC PM), Content Specialist (CS), and Scoring Directors (SD) participated in the rangefinding sessions in Maryland. (Detailed information about rangefinding procedures can be found in the following portion of this section: Development Procedures for Rangefinding.) The SD was responsible for maintaining annotations and meeting minutes from all sessions. These notes were a record of the comments and decisions made by the MSDE personnel and members of the Maryland teacher committee. These notes were utilized by the SD responsible for training the Scoring Supervisors and Scorers for the respective Maryland prompts.

1) Scorer

A graduate of a four-year accredited college or university who had completed the Maryland-specific domain training. The scorers were eligible to score items for which they had been trained and successfully qualified.

2) Scoring Supervisor

A reader who directly monitored the scoring of a team of Scorers and retrained as needed. The reader had successfully completed the PSC Scoring Supervisor training.

3) Scoring Director (SD)

An experienced and knowledgeable PSC team leader who was responsible for selecting a wide variety of student responses for such activities as rangefinding and building training materials. Selected papers were then submitted to MSDE for comment and approval. Scoring Directors remained on the project as rangefinding participants and trainers. Scoring Directors worked with scoring supervisors and the Content Specialist to oversee the scoring of several items. An SD’s main duty during scoring was to rule on validity of questionable papers and to maintain consistency in scoring decisions.

4) Content Specialist (CS)

Experienced content/training personnel who had served as SDs and were selected by the Scoring Resources staff and Project Manager to train and support Scoring Directors for Maryland.

Scorer Recruitment and Qualifications

All Scorers for MSDE had to provide Pearson their résumé and documentation of a four-year college degree. Human Resources made every effort to recruit Scorers with a teaching background and to match Scorers to projects which suited their educational background and previous scoring experience. Regardless of educational background, applicants then participated in a one-day general introductory training workshop presented by a PSC staff member. These workshops allowed Pearson to introduce potential Scorers and Scoring Supervisors to large-scale scoring in general and to the Maryland rubric specifically. The PSC staff member who presented the workshop evaluated potential Supervisors and submitted these evaluations to the PSC Site Manager with his/her recommendations. Those who successfully completed the workshop were


29

added to Pearson’s general pool of potential Scorers and Supervisors of MSA Math. This addition to the scoring pool did not qualify these Scorers for scoring the MSDE program.

Scoring Supervisor Selection

The training for new Scoring Supervisors consisted of a two-day course focusing on the duties and responsibilities necessary to successfully manage a team of Scorers. The workshop was led by the PSC Site Manager and Scoring Directors. The instruction included a review of PSC policies and procedures, sessions on use of ePEN and the monitoring reports to track a Scorer’s speed and accuracy, role playing activities which explored various situations that could occur with Scorers during the scoring of a project, and Scorer counseling and retraining guidelines. Upon completion of the workshop, the PSC Site Manager and Scoring Directors in conjunction with the Content Specialist reviewed each participant’s performance, making sure that each had a complete understanding of the Scoring Supervisor role and its responsibilities. Any participant they found who did not perform to their satisfaction was not added to the qualified Supervisor list.

Scoring Supervisor Project Training and Qualification

Project-specific Supervisor training for MSDE was conducted in the days immediately preceding Scorer training. This training began with the SD reading the rubrics aloud and answering any questions the Supervisor might have regarding the rubric. The SD then read each anchor paper aloud to the Supervisors. Each response in the anchor set was thoroughly explained, including the notes and comments of the rangefinding committee. Practice Set 1 was reviewed next. The Supervisors scored the practice set individually in the electronic scoring system (ePEN) as well as recorded their scores on a paper copy of the practice set, and then waited for all Supervisors to complete scoring the set. When everyone had completed scoring the training set, the SD discussed the responses one by one, focusing on why each received that score and not another. The SD reviewed with the group the reason for assigning each score point and discussed each paper in its entirety. The Supervisors were then ready to score Practice Set 2. Practice Set 2 was scored and reviewed exactly as Practice Set 1.

Having thoroughly discussed both practice sets with the group, the SD explained that in order for a participant to qualify as a Scoring Supervisor, it was required that the Supervisor should score at least 90% perfect agreement on Step A and 80% perfect agreement on Step B on two of three qualifying sets or one of two qualifying sets, depending on the number of sets available for each item. The Supervisors scored the first qualifying set individually and recorded their scores in ePEN. As each Supervisor finished scoring, the SD reviewed the qualifying reports before allowing the Supervisor to proceed to the next qualifying set. Each response was reviewed and any questions the Supervisor had were addressed before the Supervisor attempted the next qualifying set. The Supervisor followed the same procedure with Qualifying set 2 (and set 3 if available). Supervisors had to pass one of two or two of three sets (depending on the number of qualifying sets available per item) with 80% agreement for Math Step B and 90% in Math Step A as specified in the qualification rules or they would be released from the MSDE project.

Scoring Supervisor Duties

Scoring Supervisors were responsible for monitoring the training and qualifying of the Scorers assigned to their team. The Supervisors assisted the SD, if requested, during the training of the Scorers. The Supervisor was responsible for monitoring Scorers’ progress through the qualifying sets. The Supervisor was also responsible for monitoring each Scorer’s assignment of scores to


30

the responses. Additionally, the Supervisor reviewed the statistical reports with each individual on the team. The Supervisor consulted the SD regarding variations by the team members from the acceptable standards (95% for Math Step A, and 85% for Math Step B). The Supervisor had the initial responsibility to see that the Scorer maintained the set standards through individual retraining. The SD monitored the Supervisor by reviewing team statistics and working one-on-one with the Supervisor.

Scoring Director Selection and Qualification

The candidates for Scoring Director had been recommended by the Content Specialist, PSC Resource Staffing Managers or Site Manager. The recommendations were based upon the evaluations the candidates received as Scorers and Supervisors and were part of their personnel file. The candidates generally had been Supervisors on large-scale projects for multiple teams, and/or they had served as Supervisors on small-scale projects where Supervisors trained their individual teams. They had been evaluated on their ability to train Scorers as well as their ability to monitor the scoring accuracy and consistency of Scorers. These evaluations were submitted in writing at the end of each scoring project by the Site Managers and SDs that had observed the work of the SD candidates.

Scoring Director Project Training

The SDs familiarized themselves with the rubric. Any questions regarding the rubric were addressed by the PSC Content Specialist or MSDE. The next step was for the SD to become familiar with all their items and all training materials and scoring decisions/issues associated with their items prior to Supervisor training.

Scoring Director Duties

The SD’s job was to conduct the training of the Supervisors and Scorers, oversee the actual scoring of the papers, monitor the work of the Supervisor, and act as the decision-maker for situations or questions that may arise during the scoring process. For example, all condition code (foreign language, off-topic, off-mode, etc.) responses were reviewed by the SD, who had to confirm any such decision and ensure consistency of decisions. (Blank condition codes were assigned at the Scorer level and did not require SD confirmation.) Additionally the SD and Supervisor conducted all resolution readings. The resolution score became the reported score.

The SD also reviewed any potential questionable content responses and forwarded those to the Content Specialist to consult with MSDE before processing.

The SD was also responsible for daily statistical review and analysis of all monitoring reports to ensure the quality of the scoring. Review of the data allowed the SD not only to monitor the Scorer but also to provide the Supervisor with additional input. Available data included 1) individual Scorer agreement rates between two independent scorings; 2) score point distributions by Scorer and trend review; 3) prompt statistics for agreement rates and score point distributions; 4) Resolution data; 5) scorer-level and item-level agreement on validity papers pre-scored by MSDE.

Scorer Training Scorer training was conducted either via online training modules approved by MSDE in advance of scoring or via stand-up training led by the SD. For items requiring stand-up training, each SD was responsible for training the items he/she monitored throughout scoring. After sufficient student responses were scored for equating purposes for the first item, the SD reconvened the group and


31

trained the second item. Training began with the definition and an overview of holistic scoring. Training continued with a reading and discussion of the generic rubric and item, and then the student responses in the anchor set were read and discussed. In the anchor set the scores had been recorded on the student responses and were arranged in ascending point-scale order. Each annotated anchor response was read aloud and discussed thoroughly. Emphasis was placed on the Scorers’ understanding of how the responses differed from one another in incremental quality, how each response reflected the description of its score point as generalized in the scoring rubric, and how each reflected the MSDE’s standard for application of each score point.

Once Scorers had all their questions answered and the discussion of the anchor set was finished, the Scorers began to assign scores to the first practice set. Each Scorer independently read and scored the responses in the practice set in the electronic scoring system (ePEN). The correct scores were then read to the group when everyone had completed the scoring. In addition, each practice paper was discussed as to reasons for applying each given score. At this point, Scorers interacted with the SD in discussing the characteristics of each response that earned the assigned score point. The same format was followed for each practice set. During this process, the job of the Scorer was to internalize the scoring scale and adjust his or her individual scoring to conform to that scale. Once all practice papers had been scored and fully discussed, Scorers began the qualifying process.

For MSA Math, there were two or three qualifying sets, depending on the particular item. MSDE informed PSC in writing for each specific administration how many qualifying sets were approved and were available to the Scorers. Scorers had to achieve at least 90% perfect agreement on Step A and 80% perfect agreement on Step B on two of three qualifying sets or one of two qualifying sets, depending on the number of sets available for each item.

Scoring Rules for MSA-Math

The following scoring rules were applied to MSA-Math BCR and ECR items:

Math BCR (Brief Constructed Response) items were scored: Step A: 0, 1 with two readings Step B: 0, 1, 2 with two readings

Math ECR (Extended Constructed Response) items were scored: Step A: 0, 1 with two readings Step B: 0,1,2,3 with two readings

Scores given were the higher of the 1st and 2nd Scorer’s scores provided they were adjacent.

For example:

1st Scorer 2nd Scorer Final Score

1 2 2

2 3 3


32

A resolution scorer was used if two non-adjacent initial scores were received.

The resolution scorer’s score was used in place of both the 1st and 2nd Scorers’ scores.

For example:

1st Scorer 2nd Scorer Resolution Scorer Final Score

0 2 1 1

0 3 2 2

1 3 3 3

2 0 1 1

3 0 2 2

Inter-Rater Agreement

Pearson’s scoring system generated many kinds of internal monitoring reports that enabled the project leadership to monitor the accuracy and consistency of scoring. These reports were compiled by prompt, listed the entire prompt’s Scorers, and provided the results of their scoring for each day. Information on these reports included the number of responses read by the Scorers during the period, the number and percent of condition code responses, and the number of responses for which there had been a second reading. The number of responses with second readings provided data that allowed for reporting of the number and percent of responses with perfect agreement; the number and percent of responses on which the first Scorer was a point lower than the second Scorer; the number and percent of responses on which the first Scorer was a point higher than the second Scorer (Adjacent); and the number and percent of responses differing by more than one score point (Non-Adjacent). The Scoring Director also reviewed the daily statistical reports to identify individuals or teams who might need retraining in order to provide continuous scoring consistency on the project. MSDE received data summary reports. Statistical summaries of inter-rater reliability can be found in section 3.4, Inter-Rater Reliability.

Scorer Retraining

When a Scorer’s performance fell below acceptable parameters for a project, the Scorer was retrained. Retraining was the process by which the SD or Supervisor utilized a number of methods such as individual tutoring on problem score points, individual review of selected responses, and anchor and rubric review to get a Scorer back on track with the guidelines provided by a specific program. Group retraining was conducted by the SD every Monday (or following any extended break) during the scoring project. In addition, daily retraining occurred as deemed necessary by the MSDE representative and CS.

Backreading

Pearson’s ePEN system allowed Supervisors and/or SDs to conduct backreads as an additional monitoring method. When conducting backreads, the Supervisor or SD received images of student responses and the scores assigned by the Scorer. Responses selected for backreads might be randomly selected or might be targeted backreads (e.g., responses receiving specific scores,


33

etc.). These backreads were very useful in tracking specific areas of confusion for a given Scorer or group of Scorers and assisted the Supervisor and SD in knowing just how to direct retraining activities for individual Scorers or teams. The initial backreading percentage was set at 3%. This percentage might be adjusted either higher or lower by the Supervisor based upon the performance of the Scorer.

Development Procedures for Rangefinding

Scoring Directors were selected by the PSC Scoring Resource Manager and Content Specialist to prepare sets of papers for client approval. These experienced SDs were judged by the CS for their ability to recognize and assemble a wide variety of responses. The SD also participated with the clients as a facilitator during the rangefinding session in order to make notes and be prepared to assemble the finished sets to the client’s specifications. For a given math prompt, the SD had the following responsibilities:

1) To know the prompt and the rubric thoroughly 2) To read responses

Looked for responses that seemed to represent the full range of quality as described in the rubric.

Searched all orders for responses, with particular emphasis on the state’s high-performing districts.

Included not only papers that were homogeneous in their level of quality but also papers that differed in quality from variable to variable but which could be given an overall classification of High, Medium, or Low.

Marked High, Medium, and Low papers—marked especially good ones that might potentially receive top scores.

3) To sort copies

Copies were sorted into piles, reflecting the nature of the flag—all potential high papers were together, all potential medium papers were together, etc., with all problem papers grouped together.

For problem or decision papers, duplicates of types of problems were culled. The best example of each problem type was retained; the rest were set aside for possible future use.

4) To develop sets for rangefinding

Decided which particular papers from the sorted piles should go into sets for rangefinding. Each paper selected went into a rangefinding set arranged in performance from low to high performance.

Rangefinding Procedures

The objective of rangefinding sessions was for the team members to arrive at a consensus as to the score of each paper in the proposed training materials. These sessions were attended by Maryland educators, MSDE, and PSC Project Manager, Content Specialists, and Scoring Directors, who selected and prepared all of the papers that would be reviewed. These papers and their corresponding scores formed the basis of selecting final Anchor Sets, Practice Sets, and Qualifying Sets. Discussions among the team members were important, as they revealed what kinds of qualities characterized certain score points. The most difficult aspects involved


34

balancing widely discrepant qualities found in the same paper and defining the line between adjacent scores.

Prior to rangefinding, MSDE Scoring staff reviewed all rangefinding items and categorized each item into one of three categories.

1) An item that very closely mirrors a historical item that has been through rangefinding and has been scored by committee,

2) An item that is very similar but not a precise match to an item that has been through rangefinding and has been scored by committee, and

3) A new type of item for 2011 requiring the committee to review and score the item.

During formal rangefinding MSDE shared Category 1 and Category 2 items with the committee to prompt discussion and gain committee agreement as to the correct category for the item. Category 2 items were reclassified to either a Category 1 or a Category 3 item by the committee.

Items the committee agreed were Category 1 items utilized existing business rules for scoring this type of item and were scored by a combination of the following staff: MSDE Scoring staff, MSDE Content staff, Pearson Scoring Directors and Pearson Content Specialists, and at least one individual from the rangefinding committee. MSDE Scoring and MSDE Content reviewed non-agreement scores from the group and determined the final score for the paper.

The procedure for assigning scores to the papers in each set for Category 3 items was as follows:

The item was reviewed by the committee and criteria were discussed for receiving full credit.

Selected “grounding” papers that represented the full range of scores were read aloud and discussed by the rangefinding panel. Reading aloud focused attention on the ideas presented—or what the student had to say—allowing the panel members to divorce themselves from how the paper looked or how well it had been edited.

After each response was read, each panel member independently assigned a score. An overall tentative score was assigned to each response on which there seemed to be consensus. However, all assigned scores at this point, even those on responses for which there were complete agreement, were provisional and subject to change based on later considerations.

All subsequent responses were read and scored by each panel member independently, using the tentative scores on the previous sets as guidelines. After each set had been read, the results were recorded on a consensus sheet and discussed after each committee member had already recorded tentative scoring decisions. There might be frequent reference to previous responses to make sure that decisions on score points were consistent.


35

This iterative process of reading, charting, and discussing successive responses had three results:

It established scores for papers for which there was virtually unanimous agreement.

It identified papers that were on the line between two adjacent scores, necessitating the clarification of that line.

It contributed to understanding the rationale behind scoring decisions.

During this process, the tentative scores assigned to earlier responses became firm.


36

1.8 The 2011 MSA-Math Operational Item Analyses

Classical Analysis with Common Items Used for Form-to-Form Linking

As mentioned in section 1.4, two operational forms were randomly distributed to students and linked using common items appearing on both forms (i.e., operational forms A and F). As a result, classical analysis of these common items was conducted to check if the two groups taking different operational forms were equivalent. The following descriptive statistics were calculated based on a raw, number-right score of the common items: mean (M) and standard deviation (SD). The results indicated that the students taking the two operational forms were statistically close across all grades, as seen from Table 1.25.

Table 1.25. Descriptive Statistics for Form-to-Form Common Items

Grade Form No. of Items N M SD

3 A 39 31,082 30.61 7.06

F 39 30,211 31.01 6.80

4 A 34 30,037 25.43 6.54

F 34 29,027 25.92 6.29

5 A 39 30,354 28.58 6.93

F 39 29,293 31.27 7.75

6 A 35 29,778 21.97 7.08

F 35 28,531 22.35 6.93

7 A 35 30,180 21.46 8.17

F 35 29,100 22.03 8.01

8 A 34 30,089 20.74 8.09

F 34 29,114 21.31 7.97

Note. Form A designates the identical operational portion of Forms A, B, C, D, and E. Form F designates the identical operational portion of Forms F, G, H, J, and K. Note. Analysis was conducted with a statewide population.


37

P-Value Check with Year-to-Year Core Linking Items

As mentioned in section 1.4, different years’ assessments were linked using core linking items. This section was prepared to provide information about how much p-values (i.e., classical item difficulty) of the 2011 core linking items varied from previous years.

First, only SR items were used for the purpose of year-to-year linking. Second, classical analysis (e.g., p-value) on these items was conducted with a statewide population, and item sequence numbers on the tables were assigned based on the 2011 assessment. Finally it should be noted that detailed information about Rasch analysis on these core linking items can be found in section 1.9, Linking, Equating, Scaling Procedures.

As seen in Tables 1.26 through 1.37, we could conclude that most of the 2011 p-values were almost the same or slightly increased compared to those of previous years across all grades.

Table 1.26. P-Value Comparisons of Core Linking Items for Previous Year vs. Year 2011: Grade 3 Form A

Item Seq.

No.

Item

CID Previous

Year

Y11

FA

Item Seq.

No.

Item

CID Previous

Year

Y11

FA

001 3488069 0.89 0.90 048 3510065 0.96 0.95

002 3548059 0.71 0.78 049 3510005 0.61 0.62

005 100000067832 0.58 0.59 051 3510027 0.87 0.90

006 100000044159 0.57 0.62 052 3510035 0.87 0.89

007 100000004268 0.54 0.52 056 3510058 0.87 0.90

013 100000366020 0.59 0.68 062 3510347 0.68 0.71

014 3510017 0.91 0.94 063 100000025212 0.89 0.89

015 3509926 0.40 0.50 064 100000044162 0.83 0.88

016 3509960 0.76 0.82 065 100000018395 0.91 0.89

017 100000044152 0.86 0.88 066 3510036 0.85 0.88

018 100000025225 0.85 0.85 067 3510329 0.55 0.69

019 3510022 0.47 0.57 068 3510033 0.79 0.84

020 3509983 0.91 0.92 069 100000067837 0.66 0.67

023 3509955 0.49 0.62 071 100000025211 0.78 0.80

031 3548507 0.85 0.89 072 3509950 0.72 0.78

032 3509988 0.73 0.75 081 3510176 0.67 0.68

033 3488123 0.60 0.57 082 3509929 0.52 0.58

041 3488171 0.74 0.74

Note. Analysis was conducted with a statewide population. Note. Item sequence numbers were assigned based on the 2011 assessment.

Descriptive Statistics for Year-Year Linking Common Items: Grade 3 Form A

Form Year No. of Items M SD

A Previous Year 33 0.73 0.15

2011 33 0.76 0.14


38

Table 1.27. P-Value Comparisons of Core Linking Items for Previous Year vs. Year 2011: Grade 3 Form F

Item Seq. No. Item CID Previous

Year Y11 FF Item Seq. No. Item CID Previous

Year Y11 FF

001 3488069 0.89 0.90 048 3510065 0.96 0.96

002 3548059 0.71 0.80 049 3510005 0.61 0.63

005 100000067832 0.58 0.61 051 3510027 0.87 0.90

006 100000044159 0.57 0.63 052 3510035 0.87 0.90

007 100000004268 0.54 0.54 056 3510058 0.87 0.91

013 100000366020 0.59 0.70 062 3510347 0.68 0.73

014 3510017 0.91 0.94 063 100000025212 0.89 0.89

015 3509926 0.40 0.49 064 100000044162 0.83 0.87

016 3509960 0.76 0.83 065 100000018395 0.91 0.91

017 100000044152 0.86 0.88 066 3510036 0.85 0.89

018 100000025225 0.85 0.86 067 3510329 0.55 0.70

019 3510022 0.47 0.57 068 3510033 0.79 0.85

020 3509983 0.91 0.93 069 100000067837 0.66 0.69

023 3509955 0.49 0.64 071 100000025211 0.78 0.79

031 3548507 0.85 0.89 072 3509950 0.72 0.78

032 3509988 0.73 0.78 081 3510176 0.67 0.69

033 3488123 0.60 0.57 082 3509929 0.52 0.59

041 3488171 0.74 0.76

Note. Analysis was conducted with a statewide population. Note. Item sequence numbers were assigned based on the 2011 assessment. Descriptive Statistics for Year-Year Linking Common Items: Grade 3 Form F


F Previous Year 33 0.73 0.15

2011 33 0.77 0.13


39



Year Y11 FA Item Seq. No. Item CID Previous

Year Y11 FA

002 100000069344 0.86 0.84 047 3515575 0.71 0.88

003 100000069387 0.67 0.68 049 100000011499 0.91 0.92

006 3515840 0.65 0.72 050 3515630 0.51 0.58

007 3515641 0.81 0.83 054 3515635 0.56 0.60

008 100000007115 0.89 0.91 055 3515631 0.76 0.78

010 3515605 0.50 0.66 062 100000201857 0.48 0.47

018 3488166 0.79 0.80 063 3515634 0.75 0.81

019 3515447 0.41 0.59 064 3515853 0.71 0.79

022 3497865 0.67 0.71 066 3515836 0.58 0.60

023 3515643 0.36 0.51 067 3515933 0.76 0.81

024 3488190 0.56 0.41 068 3548079 0.94 0.96

025 100000069423 0.57 0.61 069 100000201852 0.85 0.84

026 3490562 0.53 0.54 071 3515592 0.82 0.88

030 100000018336 0.79 0.83 078 3515506 0.86 0.91

032 3551599 0.82 0.86 080 3515632 0.69 0.72

033 3515428 0.90 0.96 081 3548088 0.74 0.79





2011 33 0.74 0.15


40




Year Y11 FF

002 100000069344 0.86 0.87 047 3515575 0.71 0.90

003 100000069387 0.67 0.69 049 100000011499 0.91 0.93

006 3515840 0.65 0.72 050 3515630 0.51 0.63

007 3515641 0.81 0.85 054 3515635 0.56 0.64

008 100000007115 0.89 0.93 055 3515631 0.76 0.80

010 3515605 0.50 0.67 062 100000201857 0.48 0.49

018 3488166 0.79 0.82 063 3515634 0.75 0.82

019 3515447 0.41 0.61 064 3515853 0.71 0.84

022 3497865 0.67 0.72 066 3515836 0.58 0.63

023 3515643 0.36 0.52 067 3515933 0.76 0.81

024 3488190 0.56 0.40 068 3548079 0.94 0.96

025 100000069423 0.57 0.59 069 100000201852 0.85 0.86

026 3490562 0.53 0.55 071 3515592 0.82 0.88

030 100000018336 0.79 0.82 078 3515506 0.86 0.94

032 3551599 0.82 0.87 080 3515632 0.69 0.72

033 3515428 0.90 0.97 081 3548088 0.74 0.80


Descriptive Statistics for Year-Year Linking Common Items: Grade 4 Form F



2011 33 0.76 0.15


41




Year Y11 FA

002 3488503 0.80 0.79 043 3511513 0.85 0.87

008 3512527 0.68 0.67 047 3488391 0.89 0.87

016 3511458 0.89 0.91 048 3488431 0.74 0.76

018 3488373 0.66 0.70 049 3492140 0.91 0.91

019 3512698 0.92 0.88 055 3488485 0.89 0.83

020 3512529 0.56 0.59 056 3488509 0.82 0.78

021 3512606 0.63 0.66 058 3512637 0.80 0.77

023 100000043853 0.67 0.70 060 100000065196 0.81 0.80

026 3488370 0.88 0.83 061 3511429 0.75 0.77

027 3488376 0.81 0.80 064 3511626 0.81 0.87

028 3492126 0.88 0.88 070 3488326 0.68 0.67

037 100000366318 0.75 0.87 071 3488251 0.61 0.65

038 100000143144 0.53 0.53 072 100000022545 0.76 0.77

039 3488443 0.43 0.47 082 3488328 0.71 0.71

042 3511566 0.66 0.69 083 3511448 0.76 0.76





2011 33 0.76 0.11


42




Year Y11 FF

002 3488503 0.80 0.80 045 100000028200 0.64 0.70

008 3512527 0.68 0.68 046 3595464 0.55 0.56

016 3511458 0.89 0.91 047 3488391 0.89 0.88

018 3488373 0.66 0.71 048 3488431 0.74 0.76

019 3512698 0.92 0.88 049 3492140 0.91 0.91

020 3512529 0.56 0.60 055 3488485 0.89 0.85

021 3512606 0.63 0.66 056 3488509 0.82 0.82

023 100000043853 0.67 0.70 058 3512637 0.80 0.77

026 3488370 0.88 0.83 060 100000065196 0.81 0.81

027 3488376 0.81 0.81 061 3511429 0.75 0.78

028 3492126 0.88 0.89 064 3511626 0.81 0.88

037 100000366318 0.75 0.87 070 3488326 0.68 0.67

038 100000143144 0.53 0.53 071 3488251 0.61 0.66

039 3488443 0.43 0.47 072 100000022545 0.76 0.77

042 3511566 0.66 0.70 082 3488328 0.71 0.73

043 3511513 0.85 0.86 083 3511448 0.76 0.81





2011 33 0.76 0.11


43




Year Y11 FA

001 3516257 0.79 0.89 035 3516241 0.82 0.86

003 100000028429 0.59 0.59 036 100000065061 0.82 0.84

004 100000022483 0.55 0.55 037 3516329 0.50 0.64

006 3516243 0.64 0.76 038 100000094472 0.48 0.49

009 3492071 0.66 0.67 045 3492095 0.80 0.79

010 3516559 0.84 0.90 050 3516565 0.42 0.52

011 3516255 0.70 0.77 051 3488358 0.70 0.69

012 100000022476 0.64 0.55 054 3516906 0.60 0.59

019 100000064614 0.80 0.78 055 100000004453 0.55 0.53

020 3516909 0.59 0.61 056 3516293 0.39 0.50

025 100000064637 0.64 0.61 057 100000022470 0.54 0.53

026 3516290 0.60 0.72 058 3488489 0.73 0.75

027 100000043862 0.61 0.65 061 3492120 0.65 0.66

030 3492143 0.77 0.77 068 3516613 0.51 0.57

034 3516331 0.38 0.55 080 100000065148 0.60 0.57





2011 33 0.66 0.12


44




Year Y11 FF

001 3516257 0.79 0.90 035 3516241 0.82 0.88

003 100000028429 0.59 0.61 036 100000065061 0.82 0.85

004 100000022483 0.55 0.57 037 3516329 0.50 0.67

006 3516243 0.64 0.77 038 100000094472 0.48 0.49

009 3492071 0.66 0.68 045 3492095 0.80 0.81

010 3516559 0.84 0.91 050 3516565 0.42 0.57

011 3516255 0.70 0.79 051 3488358 0.70 0.71

012 100000022476 0.64 0.57 054 3516906 0.60 0.60

019 100000064614 0.80 0.79 055 100000004453 0.55 0.53

020 3516909 0.59 0.62 056 3516293 0.39 0.51

025 100000064637 0.64 0.61 057 100000022470 0.54 0.55

026 3516290 0.60 0.73 058 3488489 0.73 0.75

027 100000043862 0.61 0.66 061 3492120 0.65 0.64

030 3492143 0.77 0.79 068 3516613 0.51 0.57

034 3516331 0.38 0.58 080 100000065148 0.60 0.58





2011 33 0.68 0.12


45




Year Y11 FA

001 3517650 0.60 0.70 049 100000012796 0.58 0.52

002 3517601 0.44 0.55 050 3517602 0.39 0.52

003 3517678 0.88 0.95 051 3517687 0.56 0.58

007 3487535 0.63 0.60 063 100000012794 0.43 0.42

008 3517613 0.63 0.73 064 3517714 0.50 0.59

010 3517677 0.70 0.65 065 3517885 0.35 0.52

012 100000026796 0.85 0.83 066 100000018106 0.61 0.62

018 3517652 0.63 0.74 069 3487898 0.51 0.53

019 3547473 0.77 0.85 070 3517691 0.61 0.67

020 3517739 0.85 0.86 072 3555858 0.39 0.49

030 3517639 0.28 0.36 079 3555859 0.74 0.77

031 3517665 0.35 0.41 080 100000363463 0.44 0.57

032 3517609 0.50 0.56 081 3488830 0.58 0.58

043 3517656 0.59 0.68





2011 33 0.62 0.14


46




Year Y11 FF

001 3517650 0.60 0.72 049 100000012796 0.58 0.53

002 3517601 0.44 0.57 050 3517602 0.39 0.53

003 3517678 0.88 0.96 051 3517687 0.56 0.61

007 3487535 0.63 0.61 063 100000012794 0.43 0.39

008 3517613 0.63 0.75 064 3517714 0.50 0.60

010 3517677 0.70 0.68 065 3517885 0.35 0.52

012 100000026796 0.85 0.84 066 100000018106 0.61 0.61

018 3517652 0.63 0.75 069 3487898 0.51 0.51

019 3547473 0.77 0.87 070 3517691 0.61 0.72

020 3517739 0.85 0.87 072 3555858 0.39 0.48

030 3517639 0.28 0.37 079 3555859 0.74 0.79

031 3517665 0.35 0.42 080 100000363463 0.44 0.60

032 3517609 0.50 0.58 081 3488830 0.58 0.60

043 3517656 0.59 0.69





2011 33 0.64 0.15


47




Year Y11 FA

001 100000064981 0.63 0.63 047 3487901 0.85 0.86

002 100000026755 0.66 0.69 048 3514056 0.74 0.83

005 100000018156 0.68 0.73 050 3487525 0.50 0.52

007 3514053 0.70 0.76 051 100000018153 0.66 0.66

008 100000043330 0.45 0.49 052 3514103 0.56 0.72

014 3500150 0.47 0.47 053 3514076 0.46 0.52

022 3514595 0.68 0.72 062 3514710 0.53 0.61

027 100000043320 0.47 0.49 065 3487902 0.82 0.83

032 3500154 0.73 0.73 066 100000018151 0.56 0.57

033 3514062 0.39 0.45 072 3487712 0.63 0.62

038 100000049037 0.67 0.67 073 100000004114 0.34 0.32

041 100000043323 0.36 0.51 078 3487912 0.53 0.56

042 3487718 0.65 0.62 079 3547536 0.49 0.49

046 3487566 0.54 0.50





2011 33 0.61 0.13


48




Year Y11 FF

001 100000064981 0.63 0.65 047 3487901 0.85 0.88

002 100000026755 0.66 0.70 048 3514056 0.74 0.85

005 100000018156 0.68 0.74 050 3487525 0.50 0.53

007 3514053 0.70 0.78 051 100000018153 0.66 0.66

008 100000043330 0.45 0.50 052 3514103 0.56 0.74

014 3500150 0.47 0.49 053 3514076 0.46 0.52

022 3514595 0.68 0.73 062 3514710 0.53 0.63

027 100000043320 0.47 0.53 065 3487902 0.82 0.84

032 3500154 0.73 0.75 066 100000018151 0.56 0.58

033 3514062 0.39 0.47 072 3487712 0.63 0.62

038 100000049037 0.67 0.65 073 100000004114 0.34 0.33

041 100000043323 0.36 0.51 078 3487912 0.53 0.57

042 3487718 0.65 0.64 079 3547536 0.49 0.51

046 3487566 0.54 0.54





2011 33 0.63 0.13


49

Validation Check with the 2011 MSA-Math Core Items

As mentioned in section 1.4, operational items fell into one of two categories: core and core linking items. Because the core items were not included into the 2011 year-to-year linking pool, Rasch item and step difficulty parameters of the core items were reestimated with the 2011 stratified random samples during calibration and equating. (Please see section 1.9 and Appendix A for stratified random sampling procedures) As a result, this section was prepared to provide detailed information about how much the core items changed in terms of item difficulty, both classical item p-value and Rasch item difficulty. Detailed information about the roles of the 2011 core and core linking items can be found in section 1.4, Test Form Design, Specifications, Item Type, and Item Roles.

First of all, it should be noted that a smaller number of cases (i.e., about 2,500) in the table indicates that it is a field-test item. P-values of both BCR and ECR items were calculated by dividing the item mean score by the item score range (i.e., score point 2 for BCR and 3 for ECR). The percentage of “Omits” for each CR item was low and indicated that a small number of students did not respond at all. In general, item p-value analysis results indicated that most of the 2011 p-values were almost the same or somewhat increased compared to those in previous years across all grades.

With respect to the Rasch item calibration and equating, it should be noted that we coded “Omit” of each item as “missing” before we ran the data with the Rasch model. In general, the level of the 2011 item difficulties stayed almost the same or became a little lower compared to that of previous years across all grades. It should be noted that all of the Rasch item and step difficulty parameters were on a common scale (i.e., linked to the 2006 assessment).

In conclusion, both p-value and Rasch item difficulty results reflected the same phenomenon, indicating that the level of item difficulty stayed the same or became a little lower.


50

Table 1.38. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 3 Form A

Item CID Previous Year Y11 FA Item CID Previous Year Y11 FA

100000011186 0.74 0.74 584928 0.60 0.61

3595529 0.62 0.61 100000063519 0.86 0.85

100000018407 0.95 0.96 100000067805 0.85 0.86

3509941 0.64 0.65 100000080018 0.79 0.76

3595501 0.56 0.56 100000247326 0.93 0.86

100000246319 0.75 0.72 100000233317 0.99 0.98

100000004272 0.89 0.90 3509978 0.67 0.62

3509957 0.80 0.79 3985610 0.50 0.50

3564081 0.43 0.46 100000025208 0.87 0.90

3497891 0.42 0.43 100000122405 0.84 0.86

100000246267 0.90 0.88 3547998 0.89 0.88

100000246315 0.88 0.86 3564094 0.56 0.54

3510067 0.85 0.83 100000246294 0.87 0.85

3564083 0.83 0.81 100000018404 0.88 0.86

3509940 0.73 0.73 3488201 0.91 0.94

Note. Bold-faced number indicates that it is Brief Constructed Response (BCR) item.


51

Table 1.39. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous Year vs. Year 2011: Grade 3 Form A

Score-Point Distribution (%)

Year Item CID Item Type N Mean SD 0 1 2 Omit

2010 100000011186 BCR 25,967 0.74 0.44 26.24 73.35 0.38

2010 3595529 BCR 25,879 1.25 0.52 4.18 66.14 28.87 0.72

2010 3509941 BCR 25,879 0.64 0.48 35.23 64.00 0.72

2010 3595501 BCR 25,782 1.12 0.57 11.13 64.96 22.72 1.09

2008 3509957 BCR 29,364 0.80 0.40 18.50 80.16 1.34

2008 3564081 BCR 29,364 0.85 0.62 25.91 59.61 12.79 1.68

2009 3510067 BCR 10,712 0.85 0.35 14.28 85.47 0.24

2009 3564083 BCR 10,712 1.65 0.60 6.22 21.54 71.76 0.49

2006 3509940 BCR 24,120 0.73 0.44 22.16 72.78 4.98

2006 584928 BCR 24,120 1.19 0.40 7.44 63.16 28.16 1.03

2009 3509978 BCR 10,712 0.67 0.47 32.77 66.92 0.32

2009 3985610 BCR 10,712 1.00 0.44 9.02 81.02 9.30 0.66

2009 3547998 BCR 8,808 0.89 0.32 11.06 88.51 0.43

2009 3564094 BCR 8,808 1.13 0.57 10.10 65.59 23.63 0.68

2011 100000011186 BCR 31,082 0.74 0.44 25.30 74.08 0.62

2011 3595529 BCR 31,082 1.23 0.56 5.83 64.03 29.33 0.79

2011 3509941 BCR 31,082 0.65 0.48 34.39 64.66 0.86

2011 3595501 BCR 31,082 1.11 0.58 10.13 65.65 22.83 1.20

2011 3509957 BCR 31,082 0.79 0.41 20.23 78.72 1.00

2011 3564081 BCR 31,082 0.93 0.62 22.20 60.56 16.00 1.11

2011 3510067 BCR 31,082 0.83 0.37 16.15 83.35 0.45

2011 3564083 BCR 31,082 1.61 0.65 8.85 19.95 70.59 0.54

2011 3509940 BCR 31,082 0.73 0.44 22.91 73.07 3.99

2011 584928 BCR 31,082 1.22 0.54 4.51 65.85 28.22 1.23

2011 3509978 BCR 31,082 0.62 0.49 37.47 61.93 0.59

2011 3985610 BCR 31,082 1.00 0.52 12.55 73.04 13.59 0.76

2011 3547998 BCR 31,082 0.88 0.32 11.28 88.12 0.59

2011 3564094 BCR 31,082 1.08 0.59 12.98 64.59 21.55 0.83


52

Table 1.40. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year 2011: Grade 3 Form A

Year Item Seq.

No. Item CID Item Type Item Difficulty Step 0-1

Step 1-2

2010 3 100000011186 BCR_A 0.8552

2010 4 3595529 BCR_B 0.8995 -2.3179 2.3179

2008 8 100000018407 SR -1.6709

2010 21 3509941 BCR_A 1.3031

2010 22 3595501 BCR_B 1.6084 -2.0125 2.0125

2010 24 100000246319 SR 0.7899

2008 25 100000004272 SR -0.6989

2008 26 3509957 BCR_A 0.1115

2008 27 3564081 BCR_B 2.3975 -1.7280 1.7280

2008 28 3497891 SR 2.3499

2010 29 100000246267 SR -0.4910

2010 30 100000246315 SR -0.2795

2009 36 3510067 BCR_A -0.0129

2009 37 3564083 BCR_B 0.2571 -0.4253 0.4253

2006 42 3509940 BCR_A 0.4317

2006 43 584928 BCR_B 1.0061 -2.0871 2.0871

2009 44 100000063519 SR -0.2397

2009 45 100000067805 SR -0.1887

2009 46 100000080018 SR 0.3073

2010 47 100000247326 SR -0.8600

2009 50 100000233317 SR -3.1944

2009 53 3509978 BCR_A 1.1411

2009 54 3985610 BCR_B 2.0758 -2.6995 2.6995

2008 55 100000025208 SR -0.3393

2009 70 100000122405 SR -0.1119

2009 73 3547998 BCR_A -0.4669

2009 74 3564094 BCR_B 1.6096 -2.0080 2.0080

2010 75 100000246294 SR -0.1621

2008 76 100000018404 SR -0.3870

2007 80 3488201 SR -1.1266

2011 3 100000011186 BCR_A 0.8674

2011 4 3595529 BCR_B 1.1352 -2.2067 2.2067

2011 8 100000018407 SR -1.7879

2011 21 3509941 BCR_A 1.4057

2011 22 3595501 BCR_B 1.7804 -1.9419 1.9419


53

Table 1.40 (continued)

Year Item Seq.


Step 1-2

2011 24 100000246319 SR 1.0140

2011 25 100000004272 SR -0.7315

2011 26 3509957 BCR_A 0.4850

2011 27 3564081 BCR_B 2.4555 -1.7392 1.7392

2011 28 3497891 SR 2.6221

2011 29 100000246267 SR -0.4145

2011 30 100000246315 SR -0.1400

2011 36 3510067 BCR_A 0.1409

2011 37 3564083 BCR_B 0.4472 -0.3188 0.3188

2011 42 3509940 BCR_A 0.7796

2011 43 584928 BCR_B 1.0594 -2.3505 2.3505

2011 44 100000063519 SR -0.0768

2011 45 100000067805 SR -0.0370

2011 46 100000080018 SR 0.6848

2011 47 100000247326 SR -0.1847

2011 50 100000233317 SR -3.1024

2011 53 3509978 BCR_A 1.4993

2011 54 3985610 BCR_B 2.1074 -2.3088 2.3088

2011 55 100000025208 SR -0.6075

2011 70 100000122405 SR -0.2221

2011 73 3547998 BCR_A -0.3395

2011 74 3564094 BCR_B 1.8206 -1.9347 1.9347

2011 75 100000246294 SR -0.1274

2011 76 100000018404 SR -0.1555

2011 80 3488201 SR -1.2151


54

Figure 1.3. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 3 Form A


55

Table 1.41. P-Value Comparison of Core Items for Previous Year vs. Year 2011: Grade 3 Form F

Item CID Previous Year Y11 FF Item CID Previous Year Y11 FF

100000011186 0.74 0.76 3564099 0.41 0.41

3595529 0.62 0.62 100000025223 0.86 0.90

100000044155 0.95 0.95 100000067776 0.92 0.93

3509918 0.82 0.81 3488199 0.95 0.96

3564076 0.54 0.51 3492707 0.93 0.94

100000259291 0.85 0.83 100000063516 0.94 0.93

100000247327 0.90 0.90 3509949 0.78 0.78

3509922 0.67 0.68 3985609 0.71 0.71

3564085 0.36 0.38 100000246321 0.91 0.92

100000246320 0.92 0.93 100000011212 0.81 0.84

3488191 0.83 0.86 3547998 0.89 0.89

3511859 0.87 0.91 3564094 0.56 0.55

3510073 0.79 0.80 100000246291 0.90 0.88

3595503 0.58 0.56 100000018403 0.90 0.89

3488087 0.44 0.44 100000011197 0.73 0.79



56

Table 1.42. Score-Point Distribution Comparisons of Constructed Response Core Items for Previous Year vs. Year 2011: Grade 3 Form F



2010 100000011186 BCR 25,967 0.74 0.44 26.24 73.35 0.38

2010 3595529 BCR 25,879 1.25 0.52 4.18 66.14 28.87 0.72

2008 3509918 BCR 29,364 0.82 0.38 17.44 82.14 0.41

2008 3564076 BCR 29,364 1.08 0.59 12.73 64.81 21.51 0.95

2008 3509922 BCR 29,253 0.67 0.47 30.34 67.24 2.42

2008 3564085 BCR 29,253 0.71 0.56 32.05 60.86 5.26 1.83

2008 3510073 BCR 29,364 0.79 0.41 20.72 78.64 0.65

2008 3595503 BCR 29,364 1.17 0.57 8.16 64.41 26.25 1.17

2010 3488087 BCR 25,577 0.44 0.50 55.17 43.61 1.18

2010 3564099 BCR 25,407 0.82 0.65 30.82 53.83 13.31 1.84

2010 3509949 BCR 25,572 0.78 0.42 22.18 76.61 1.20

2010 3985609 BCR 25,735 1.42 0.68 10.57 36.56 52.17 0.57

2009 3547998 BCR 8,808 0.89 0.32 11.06 88.51 0.43

2009 3564094 BCR 8,808 1.13 0.57 10.10 65.59 23.63 0.68

2011 100000011186 BCR 30,211 0.76 0.43 23.56 75.95 0.49

2011 3595529 BCR 30,211 1.25 0.55 5.38 63.26 30.68 0.68

2011 3509918 BCR 30,211 0.81 0.39 17.55 81.40 0.99

2011 3564076 BCR 30,211 1.02 0.62 16.70 61.76 19.94 1.32

2011 3509922 BCR 30,211 0.68 0.47 30.34 67.69 1.96

2011 3564085 BCR 30,211 0.76 0.56 28.41 63.50 6.47 1.46

2011 3510073 BCR 30,211 0.80 0.40 19.35 80.18 0.46

2011 3595503 BCR 30,211 1.12 0.58 11.24 64.55 23.60 0.57

2011 3488087 BCR 30,211 0.44 0.50 54.83 43.62 1.50

2011 3564099 BCR 30,211 0.81 0.63 28.94 56.34 12.36 2.07

2011 3509949 BCR 30,211 0.78 0.42 20.93 77.87 1.18

2011 3985609 BCR 30,211 1.42 0.68 10.29 36.62 52.54 0.50

2011 3547998 BCR 30,211 0.89 0.31 10.02 89.37 0.60

2011 3564094 BCR 30,211 1.10 0.58 11.35 65.69 22.06 0.87


57

Table 1.43. Rasch Item and Step Difficulty Comparisons of Core Items for Previous Year vs. Year 2011: Grade 3 Form F

Year Item Seq.


Step 1-2

2010 3 100000011186 BCR_A 0.8552

2010 4 3595529 BCR_B 0.8995 -2.3179 2.3179

2010 8 100000044155 SR -1.2851

2008 21 3509918 BCR_A -0.1008

2008 22 3564076 BCR_B 1.5690 -1.8761 1.8761

2010 24 100000259291 SR 0.0725

2010 25 100000247327 SR -0.4907

2008 26 3509922 BCR_A 1.0913

2008 27 3564085 BCR_B 3.2375 -2.1210 2.1210

2010 28 100000246320 SR -0.8076

2007 29 3488191 SR -0.2146

2008 30 3511859 SR -0.4072

2008 36 3510073 BCR_A 0.2315

2008 37 3595503 BCR_B 1.1948 -1.9764 1.9764

2010 42 3488087 BCR_A 2.5974

2010 43 3564099 BCR_B 2.8239 -1.4658 1.4658

2008 44 100000025223 SR -0.4276

2009 45 100000067776 SR -1.1465

2007 46 3488199 SR -1.8387

2007 47 3492707 SR -1.2271

2009 50 100000063516 SR -1.3338

2010 53 3509949 BCR_A 0.7528

2010 54 3985609 BCR_B 1.1340 -0.7901 0.7901

2010 55 100000246321 SR -0.6330

2008 70 100000011212 SR 0.1068

2009 73 3547998 BCR_A -0.4669

2009 74 3564094 BCR_B 1.6096 -2.0080 2.0080

2010 75 100000246291 SR -0.4586

2008 76 100000018403 SR -0.6989

2008 80 100000011197 SR 0.6199

2011 3 100000011186 BCR_A 0.8674

2011 4 3595529 BCR_B 1.1352 -2.2067 2.2067

2011 8 100000044155 SR -1.5257

2011 21 3509918 BCR_A 0.3533

2011 22 3564076 BCR_B 2.2253 -1.8356 1.8356


58


Year Item Seq. No.

Item CID Item Type Item Difficulty Step 0-1

Step 1-2

2011 24 100000259291 SR 0.2321

2011 25 100000247327 SR -0.3971

2011 26 3509922 BCR_A 1.3410

2011 27 3564085 BCR_B 3.3584 -2.1421 2.1421

2011 28 100000246320 SR -0.9848

2011 29 3488191 SR -0.2744

2011 30 3511859 SR -0.8099

2011 36 3510073 BCR_A 0.4877

2011 37 3595503 BCR_B 1.7429 -2.0126 2.0126

2011 42 3488087 BCR_A 2.6798

2011 43 3564099 BCR_B 2.9814 -1.6470 1.6470

2011 44 100000025223 SR -0.4730

2011 45 100000067776 SR -0.9821

2011 46 3488199 SR -1.8514

2011 47 3492707 SR -1.5038

2011 50 100000063516 SR -0.9858

2011 53 3509949 BCR_A 0.6823

2011 54 3985609 BCR_B 1.1536 -0.9791 0.9791

2011 55 100000246321 SR -0.7877

2011 70 100000011212 SR 0.1537

2011 73 3547998 BCR_A -0.3395

2011 74 3564094 BCR_B 1.8206 -1.9347 1.9347

2011 75 100000246291 SR -0.2438

2011 76 100000018403 SR -0.4235

2011 80 100000011197 SR 0.5667


59

Figure 1.4. Rasch Item Difficulty Comparison Plot for Previous Year vs. Year 2011: Grade 3 Form F


60



100000284521 0.84 0.86 3564170 0.59 0.67

3487819 0.53 0.56 3515642 0.29 0.29

3564186 0.42 0.45 3985619 0.43 0.45

100000069392 0.55 0.60 100000069314 0.94 0.94

100000069398 0.79 0.80 3487992 0.90 0.93

3515451 0.70 0.80 3488145 0.71 0.70

3564161 0.67 0.73 3564189 0.49 0.48

100000069419 0.70 0.72 100000063664 0.59 0.51

3515807 0.75 0.78 100000011504 0.66 0.69

3564165 0.40 0.43 3488035 0.78 0.84

100000247264 0.81 0.79 100000012184 0.51 0.52

100000069354 0.57 0.62 100000011490 0.96 0.96

3515868 0.81 0.83 3515638 0.62 0.69

3515825 0.96 0.97 3564169 0.46 0.46

100000251989 0.81 0.79 3515848 0.83 0.86

3515862 0.49 0.57 100000421708 0.49 0.73



61




2010 3487819 BCR 26,588 0.53 0.50 46.69 52.71 0.56

2010 3564186 BCR 26,515 0.84 0.44 19.01 77.03 3.07 0.84

2007 3515451 BCR 30,402 0.70 0.46 27.68 70.46 1.87

2007 3564161 BCR 30,402 1.35 0.77 16.32 28.41 53.07 2.19

2009 3515807 BCR 23,861 0.75 0.43 20.88 75.36 3.76

2009 3564165 BCR 23,861 0.80 0.64 31.46 54.90 12.46 1.18

2007 3515862 BCR 30,103 0.49 0.50 49.21 48.76 2.02

2007 3564170 BCR 30,103 1.17 0.79 21.52 34.56 41.26 2.66

2009 3515642 BCR 23,861 0.29 0.45 70.32 28.60 1.08

2009 3985619 BCR 23,861 0.86 0.72 31.92 46.50 19.58 2.00

2010 3488145 BCR 26,576 0.71 0.45 28.72 70.65 0.61

2010 3564189 BCR 26,485 0.99 0.56 15.98 67.96 14.91 0.95

2007 3515638 BCR 30,103 0.62 0.49 35.31 62.15 2.54

2007 3564169 BCR 30,103 0.92 0.67 24.33 53.80 19.02 2.85

2011 3487819 BCR 30,037 0.56 0.50 43.47 56.07 0.46

2011 3564186 BCR 30,037 0.91 0.47 15.63 77.11 6.78 0.48

2011 3515451 BCR 30,037 0.80 0.40 18.78 80.31 0.89

2011 3564161 BCR 30,037 1.46 0.73 13.30 25.53 60.27 0.89

2011 3515807 BCR 30,037 0.78 0.41 15.93 78.46 5.61

2011 3564165 BCR 30,037 0.87 0.75 33.67 42.74 22.00 1.40

2011 3515862 BCR 30,037 0.57 0.49 41.66 57.37 0.94

2011 3564170 BCR 30,037 1.34 0.73 14.33 35.31 49.26 1.04

2011 3515642 BCR 30,037 0.29 0.45 69.85 29.06 1.07

2011 3985619 BCR 30,037 0.90 0.72 29.49 47.72 21.19 1.56

2011 3488145 BCR 30,037 0.70 0.46 29.46 69.81 0.67

2011 3564189 BCR 30,037 0.95 0.58 18.31 65.81 14.69 1.02

2011 3515638 BCR 30,037 0.69 0.46 29.19 69.18 1.62

2011 3564169 BCR 30,037 0.91 0.59 20.92 64.53 13.41 1.05


62


Year Item Seq.


Step 1-2

2010 1 100000284521 SR -0.5213

2010 4 3487819 BCR_A 1.4019

2010 5 3564186 BCR_B 2.7967 -3.0944 3.0944

2010 9 100000069392 SR 1.2801

2009 11 100000069398 SR -0.2657

2007 20 3515451 BCR_A 0.0472

2007 21 3564161 BCR_B 0.3581 -0.4204 0.4204

2009 27 100000069419 SR 0.2651

2009 28 3515807 BCR_A -0.2644

2009 29 3564165 BCR_B 2.1333 -1.6175 1.6175

2010 31 100000247264 SR -0.2536

2009 34 100000069354 SR 1.0565

2006 35 3515868 SR -0.7073

2006 36 3515825 SR -2.6807

2010 37 100000251989 SR -0.3545

2007 38 3515862 BCR_A 1.1734

2007 39 3564170 BCR_B 0.6541 -0.6701 0.6701

2009 44 3515642 BCR_B 2.6593

2009 45 3985619 BCR_B 1.8365 -1.1225 1.1225

2009 46 100000069314 SR -2.0806

2007 48 3487992 SR -1.5824

2010 51 3488145 BCR_A 0.3722

2010 52 3564189 BCR_B 1.5344 -2.0477 2.0477

2009 53 100000063664 SR 0.9156

2008 56 100000011504 SR 0.5235

2007 57 3488035 SR -0.5019

2008 65 100000012184 SR 1.3485

2008 70 100000011490 SR -2.6762

2007 73 3515638 BCR_A 0.3998

2007 74 3564169 BCR_B 1.3844 -1.5044 1.5044

2006 77 3515848 SR -0.9267

2004 79 100000421708 SR 0.5731

2011 1 100000284521 SR -0.8546

2011 4 3487819 BCR_A 1.1653

2011 5 3564186 BCR_B 2.0935 -2.6172 2.6172


63


Year Item Seq.


Step 1-2

2011 9 100000069392 SR 1.0618

2011 11 100000069398 SR -0.3307

2011 20 3515451 BCR_A -0.3252

2011 21 3564161 BCR_B 0.2543 -0.4216 0.4216

2011 27 100000069419 SR 0.3062

2011 28 3515807 BCR_A -0.4754

2011 29 3564165 BCR_B 1.8745 -0.9919 0.9919

2011 31 100000247264 SR -0.2580

2011 34 100000069354 SR 0.9852

2011 35 3515868 SR -0.5019

2011 36 3515825 SR -3.0366

2011 37 100000251989 SR -0.2409

2011 38 3515862 BCR_A 1.0359

2011 39 3564170 BCR_B 0.4705 -0.9068 0.9068

2011 44 3515642 BCR_B 2.6660

2011 45 3985619 BCR_B 1.8186 -1.1454 1.1454

2011 46 100000069314 SR -1.8735

2011 48 3487992 SR -1.6686

2011 51 3488145 BCR_A 0.4804

2011 52 3564189 BCR_B 1.6680 -2.0764 2.0764

2011 53 100000063664 SR 1.5151

2011 56 100000011504 SR 0.4124

2011 57 3488035 SR -0.5946

2011 65 100000012184 SR 1.4437

2011 70 100000011490 SR -2.6967

2011 73 3515638 BCR_A 0.5104

2011 74 3564169 BCR_B 1.8414 -1.8818 1.8818

2011 77 3515848 SR -0.9513

2011 79 100000421708 SR 0.1530


64



65



100000069333 0.62 0.63 3595537 0.60 0.70

100000069368 0.66 0.74 100000025172 0.61 0.64

3985632 0.47 0.51 3985613 0.47 0.55

3515833 0.96 0.98 100000069427 0.87 0.90

100000252072 0.51 0.54 100000252071 0.69 0.74

100000044142 0.85 0.88 3488145 0.71 0.71

3595499 0.48 0.49 3564189 0.49 0.48

100000247277 0.68 0.65 3515789 0.78 0.88

3515648 0.54 0.57 100000251996 0.90 0.88

3564163 0.56 0.65 100000069339 0.82 0.83

3515684 0.69 0.76 3488163 0.58 0.63

100000284524 0.70 0.71 3548698 0.95 0.97

3515581 0.63 0.67 100000007087 0.77 0.77

100000025152 0.71 0.78 3595563 0.71 0.72

100000069382 0.94 0.95 100000284503 0.43 0.43

3515646 0.63 0.69 3497868 0.52 0.57



66




2009 100000069368 BCR 2,491 0.66 0.47 32.80 65.68 1.53

2009 3985632 BCR 2,491 0.95 0.47 12.85 77.36 8.59 1.20

2010 100000044142 BCR 26,543 0.85 0.36 14.66 84.55 0.73

2010 3595499 BCR 26,498 0.96 0.53 15.86 70.92 12.16 0.90

2008 3515648 BCR 30,101 0.54 0.50 45.53 53.57 0.89

2008 3564163 BCR 30,101 1.13 0.78 23.19 37.31 37.84 1.66

2008 3515646 BCR 29,933 0.63 0.48 36.05 62.96 0.99

2008 3595537 BCR 29,933 1.20 0.88 28.87 18.89 50.70 1.55

2009 100000025172 BCR 15,855 0.61 0.49 38.37 61.26 0.37

2009 3985613 BCR 15,855 0.94 0.62 21.99 60.90 16.34 0.78

2010 3488145 BCR 26,576 0.71 0.45 28.72 70.65 0.61

2010 3564189 BCR 26,485 0.99 0.56 15.98 67.96 14.91 0.95

2010 100000007087 BCR 26,676 0.77 0.42 22.35 77.07 0.52

2010 3595563 BCR 26,634 1.43 0.61 6.09 44.70 48.43 0.68

2011 100000069368 BCR 29,027 0.74 0.44 25.92 73.63 0.40

2011 3985632 BCR 29,027 1.03 0.41 6.62 83.08 9.90 0.34

2011 100000044142 BCR 29,027 0.88 0.33 11.50 87.64 0.85

2011 3595499 BCR 29,027 0.98 0.55 15.52 69.55 13.98 0.94

2011 3515648 BCR 29,027 0.57 0.49 41.84 57.28 0.88

2011 3564163 BCR 29,027 1.30 0.71 13.64 39.94 45.25 1.16

2011 3515646 BCR 29,027 0.69 0.46 29.81 69.47 0.72

2011 3595537 BCR 29,027 1.39 0.77 16.63 26.11 56.52 0.72

2011 100000025172 BCR 29,027 0.64 0.48 35.09 64.48 0.43

2011 3985613 BCR 29,027 1.11 0.64 15.57 57.43 26.56 0.43

2011 3488145 BCR 29,027 0.71 0.45 28.07 71.36 0.51

2011 3564189 BCR 29,027 0.97 0.58 17.75 65.79 15.42 0.91

2011 100000007087 BCR 29,027 0.77 0.42 22.28 77.10 0.58

2011 3595563 BCR 29,027 1.43 0.64 7.18 40.78 51.24 0.72


67


Year Item Seq.


Step 1-2

2010 1 100000069333 SR 0.9476

2009 4 100000069368 BCR_A 0.4952

2009 5 3985632 BCR_B 1.6717 -2.6668 2.6668

2006 9 3515833 SR -2.9353

2010 11 100000252072 SR 1.5076

2010 20 100000044142 BCR_A -0.6437

2010 21 3595499 BCR_B 1.7679 -2.2015 2.2015

2010 27 100000247277 SR 0.5808

2008 28 3515648 BCR_A 1.2519

2008 29 3564163 BCR_B 0.9986 -0.6913 0.6913

2006 31 3515684 SR 0.0333

2010 34 100000284524 SR 0.4064

2006 35 3515581 SR 0.3351

2008 36 100000025152 SR 0.0922

2009 37 100000069382 SR -1.9720

2008 38 3515646 BCR_A 0.6734

2008 39 3595537 BCR_B 0.8984 0.1686 -0.1686

2009 44 100000025172 BCR_B 0.7480

2009 45 3985613 BCR_B 1.5908 -1.7399 1.7399

2009 46 100000069427 SR -1.1395

2010 48 100000252071 SR 0.4944

2010 51 3488145 BCR_A 0.3722

2010 52 3564189 BCR_B 1.5344 -2.0477 2.0477

2006 53 3515789 SR -0.5378

2010 56 100000251996 SR -1.2336

2009 57 100000069339 SR -0.5694

2008 65 3488163 SR 0.9709

2007 70 3548698 SR -2.5803

2010 73 100000007087 BCR_A -0.0833

2010 74 3595563 BCR_B -0.0165 -1.4651 1.4651

2010 77 100000284503 SR 1.8809

2008 79 3497868 SR 1.1893

2011 1 100000069333 SR 0.9168

2011 4 100000069368 BCR_A 0.2076

2011 5 3985632 BCR_B 1.3722 -3.0381 3.0381


68


Year Item Seq. No.


Step 1-2

2011 9 3515833 SR -3.1070

2011 11 100000252072 SR 1.4368

2011 20 100000044142 BCR_A -0.7849

2011 21 3595499 BCR_B 1.7241 -2.0968 2.0968

2011 27 100000247277 SR 0.8179

2011 28 3515648 BCR_A 1.2240

2011 29 3564163 BCR_B 0.7720 -0.8499 0.8499

2011 31 3515684 SR 0.1479

2011 34 100000284524 SR 0.4805

2011 35 3515581 SR 0.7823

2011 36 100000025152 SR -0.0672

2011 37 100000069382 SR -1.9537

2011 38 3515646 BCR_A 0.5279

2011 39 3595537 BCR_B 0.6303 -0.3046 0.3046

2011 44 100000025172 BCR_B 0.8670

2011 45 3985613 BCR_B 1.3256 -1.5587 1.5587

2011 46 100000069427 SR -1.1271

2011 48 100000252071 SR 0.1663

2011 51 3488145 BCR_A 0.4804

2011 52 3564189 BCR_B 1.6680 -2.0764 2.0764

2011 53 3515789 SR -0.7537

2011 56 100000251996 SR -0.7924

2011 57 100000069339 SR -0.4073

2011 65 3488163 SR 0.9311

2011 70 3548698 SR -2.5568

2011 73 100000007087 BCR_A 0.0561

2011 74 3595563 BCR_B 0.2628 -1.1643 1.1643

2011 77 100000284503 SR 2.1819

2011 79 3497868 SR 1.2038


69



70



100000028258 0.47 0.46 100000028202 0.67 0.64

100000245772 0.73 0.73 3985560 0.38 0.39

100000065205 0.55 0.55 100000063880 0.79 0.71

3985586 0.52 0.52 100000245581 0.77 0.82

100000065179 0.63 0.62 3488347 0.47 0.47

100000245597 0.52 0.53 3564046 0.44 0.45

100000009922 0.69 0.70 3488381 0.50 0.62

100000245783 0.97 0.97 3488394 0.45 0.49

100000245825 0.78 0.75 100000245580 0.62 0.67

100000297356 0.74 0.75 3548459 0.76 0.75

100000028212 0.69 0.73 3564051 0.69 0.68

3595468 0.48 0.45 3488316 0.56 0.57

100000078887 0.74 0.74 100000160221 0.43 0.47

100000009927 0.72 0.68 3985559 0.49 0.53

3985561 0.54 0.40 100000009962 0.69 0.68

100000063887 0.77 0.75 3488530 0.38 0.38

100000065237 0.58 0.61 3564054 0.34 0.34

100000063620 0.92 0.85

Note. Bold-faced number indicates that it is Brief Constructed Response (BCR) item or Extended Constructed Response (ECR) item.


71



Year Item CID Item Type N Mean SD 0 1 2 3 Omit

2009 100000065205 BCR 2,491 0.55 0.50 43.64 55.44 0.92

2009 3985586 BCR 2,491 1.04 0.38 4.54 85.23 9.59 0.64

2010 100000028212 BCR 26,077 0.69 0.46 30.60 68.67 0.71

2010 3595468 BCR 26,030 0.95 0.61 20.97 61.88 16.18 0.89

2009 100000009927 BCR 2,485 0.72 0.45 25.79 71.99 2.21

2009 3985561 BCR 2,485 1.08 0.74 21.81 44.75 31.59 1.85

2009 100000028202 ECR 2,488 0.67 0.47 31.15 67.04 1.81

2009 3985560 ECR 2,488 1.13 0.77 17.40 53.98 22.15 5.02 1.45

2009 3488347 BCR 6,870 0.47 0.50 51.50 47.16 1.34

2009 3564046 BCR 6,870 0.88 0.93 47.82 12.52 37.66 2.01

2009 3548459 BCR 8,508 0.76 0.43 22.06 75.83 2.10

2009 3564051 BCR 8,508 1.38 0.66 7.96 41.68 48.23 2.14

2009 100000160221 BCR 2,485 0.43 0.49 53.64 42.58 3.78

2009 3985559 BCR 2,485 0.98 0.93 40.80 13.68 42.01 3.50

2010 3488530 BCR 25,265 0.38 0.49 59.20 36.75 3.80

2010 3564054 BCR 25,226 0.68 0.75 47.12 31.81 16.91 3.95

2011 100000065205 BCR 30,354 0.55 0.50 44.37 55.21 0.42

2011 3985586 BCR 30,354 1.04 0.43 6.88 81.06 11.64 0.42

2011 100000028212 BCR 30,354 0.73 0.45 26.62 72.69 0.67

2011 3595468 BCR 30,354 0.89 0.61 23.38 61.96 13.74 0.88

2011 100000009927 BCR 30,354 0.68 0.47 29.69 68.22 2.06

2011 3985561 BCR 30,354 0.80 0.85 45.57 24.43 27.76 2.18

2011 100000028202 ECR 30,354 0.64 0.48 35.61 63.69 0.70

2011 3985560 ECR 30,354 1.16 0.75 16.36 54.55 23.62 4.62 0.73

2011 3488347 BCR 30,354 0.47 0.50 51.28 47.01 1.69

2011 3564046 BCR 30,354 0.89 0.93 46.68 12.84 38.21 2.25

2011 3548459 BCR 30,354 0.75 0.43 23.19 74.70 2.09

2011 3564051 BCR 30,354 1.35 0.71 11.60 37.39 48.90 2.10

2011 100000160221 BCR 30,354 0.47 0.50 50.20 47.49 2.23

2011 3985559 BCR 30,354 1.06 0.91 35.57 16.80 44.73 2.71

2011 3488530 BCR 30,354 0.38 0.48 59.23 37.69 2.97

2011 3564054 BCR 30,354 0.67 0.77 48.41 29.69 18.75 3.03


72


Year Item Seq.


Step 1-2

Step 2-3

2009 1 100000028258 SR 1.3712

2010 3 100000245772 SR 0.0923

2009 4 100000065205 BCR_A 0.9512

2009 5 3985586 BCR_B 0.8324 -3.1999 3.1999

2009 6 100000065179 SR 0.4628

2010 7 100000245597 SR 1.2028

2008 9 100000009922 SR 0.2022

2010 10 100000245783 SR -2.8869

2010 17 100000245825 SR -0.2130

2010 22 100000297356 SR 0.0470

2010 24 100000028212 BCR_A 0.3107

2010 25 3595468 BCR_B 1.4747 -1.6789 1.6789

2009 34 100000078887 SR -0.1752

2009 35 100000009927 BCR_A -0.0358

2009 36 3985561 BCR_B 0.9570 -1.0043 1.0043

2010 40 100000063887 SR -0.2621

2009 41 100000065237 SR 0.7796

2010 44 100000063620 SR -1.6123

2009 45 100000028202 ECR_A 0.3011

2009 46 3985560 ECR_B 1.9845 -2.4626 0.4892 1.9734

2010 50 100000063880 SR -0.3590

2004 51 100000245581 SR -0.9991

2009 52 3488347 BCR_A 1.4298

2009 53 3564046 BCR_B 1.5002 0.8247 -0.8247

2007 54 3488381 SR 0.9673

2007 57 3488394 SR 1.3515

2010 59 100000245580 SR 0.5942

2009 62 3548459 BCR_A -0.2124

2009 63 3564051 BCR_B 0.0385 -1.2689 1.2689

2008 65 3488316 SR 0.8605

2009 73 100000160221 BCR_A 1.5360

2009 74 3985559 BCR_B 1.1767 0.6668 -0.6668

2008 79 100000009962 SR 0.2214

2010 80 3488530 BCR_A 1.8673

2010 81 3564054 BCR_B 2.2051 -0.6519 0.6519

2011 1 100000028258 SR 1.3100

2011 3 100000245772 SR -0.0857

2011 4 100000065205 BCR_A 0.8718

2011 5 3985586 BCR_B 0.8350 -2.8194 2.8194


73


Year Item Seq.


Step 1-2

Step 2-3

2011 6 100000065179 SR 0.5228

2011 7 100000245597 SR 1.0934

2011 9 100000009922 SR 0.0894

2011 10 100000245783 SR -3.2761

2011 17 100000245825 SR -0.3649

2011 22 100000297356 SR -0.2154

2011 24 100000028212 BCR_A -0.1171

2011 25 3595468 BCR_B 1.5239 -1.7740 1.7740

2011 34 100000078887 SR -0.1507

2011 35 100000009927 BCR_A 0.1098

2011 36 3985561 BCR_B 1.5783 -0.0326 0.0326

2011 40 100000063887 SR -0.2299

2011 41 100000065237 SR 0.5956

2011 44 100000063620 SR -1.0976

2011 45 100000028202 ECR_A 0.4347

2011 46 3985560 ECR_B 1.8988 -2.6153 0.4082 2.2071

2011 50 100000063880 SR -0.0662

2011 51 100000245581 SR -0.8168

2011 52 3488347 BCR_A 1.2508

2011 53 3564046 BCR_B 1.4035 0.8746 -0.8746

2011 54 3488381 SR 0.4962

2011 57 3488394 SR 1.1666

2011 59 100000245580 SR 0.2255

2011 62 3548459 BCR_A -0.3130

2011 63 3564051 BCR_B 0.0803 -0.9205 0.9205

2011 65 3488316 SR 0.7693

2011 73 100000160221 BCR_A 1.2630

2011 74 3985559 BCR_B 1.0066 0.4049 -0.4049

2011 79 100000009962 SR 0.2212

2011 80 3488530 BCR_A 1.7777

2011 81 3564054 BCR_B 1.9885 -0.4800 0.4800


74



75



100000028258 0.47 0.47 100000063620 0.92 0.86

100000245515 0.43 0.50 3488445 0.92 0.92

100000065205 0.55 0.56 3488452 0.86 0.86

3985586 0.52 0.53 100000022530 0.78 0.76

100000065317 0.34 0.37 3985592 0.83 0.83

100000009923 0.71 0.75 3488381 0.50 0.62

3488268 0.59 0.65 3488394 0.45 0.49

3488245 0.83 0.86 100000143153 0.80 0.83

100000022537 0.53 0.58 100000028238 0.60 0.63

100000297356 0.74 0.77 3985553 0.56 0.60

3511336 0.43 0.46 100000009963 0.66 0.68

3563987 0.38 0.41 100000022532 0.43 0.45

100000022549 0.68 0.70 3595471 0.44 0.49

3512615 0.79 0.79 100000065216 0.59 0.64

3595439 0.59 0.55 3488530 0.38 0.37

100000065193 0.95 0.95 3564054 0.34 0.34

100000065233 0.71 0.70



76




2009 100000065205 BCR 2,491 0.55 0.50 43.64 55.44 0.92

2009 3985586 BCR 2,491 1.04 0.38 4.54 85.23 9.59 0.64

2008 3511336 BCR 30,537 0.43 0.49 52.23 42.61 5.16

2008 3563987 BCR 30,537 0.75 0.75 36.69 37.97 18.73 6.61

2009 3512615 BCR 8,508 0.79 0.40 19.19 79.47 1.34

2009 3595439 BCR 8,508 1.18 0.62 10.30 57.71 30.16 1.83

2009 100000022530 BCR 2,492 0.78 0.41 19.22 78.25 2.53

2009 3985592 BCR 2,492 1.66 0.61 5.58 18.26 74.12 2.05

2009 100000028238 BCR 2,488 0.60 0.49 36.09 59.69 4.22

2009 3985553 BCR 2,488 1.12 0.81 23.71 32.48 39.75 4.06

2010 100000022532 BCR 25,654 0.43 0.50 55.36 42.19 2.32

2010 3595471 BCR 25,523 0.87 0.94 49.29 10.31 37.30 2.82

2010 3488530 BCR 25,265 0.38 0.49 59.20 36.75 3.80

2010 3564054 BCR 25,226 0.68 0.75 47.12 31.81 16.91 3.95

2011 100000065205 BCR 29,293 0.56 0.50 43.72 55.89 0.39

2011 3985586 BCR 29,293 1.05 0.43 6.63 80.87 12.11 0.39

2011 3511336 BCR 29,293 0.46 0.50 50.67 46.08 3.18

2011 3563987 BCR 29,293 0.82 0.76 35.97 38.74 21.59 3.47

2011 3512615 BCR 29,293 0.79 0.41 19.26 78.72 2.00

2011 3595439 BCR 29,293 1.10 0.67 15.58 54.39 28.00 2.00

2011 100000022530 BCR 29,293 0.76 0.43 21.79 75.96 2.22

2011 3985592 BCR 29,293 1.66 0.61 5.22 19.02 73.74 1.67

2011 100000028238 BCR 29,293 0.63 0.48 34.38 62.85 2.70

2011 3985553 BCR 29,293 1.20 0.82 22.85 28.87 45.50 2.69

2011 100000022532 BCR 29,293 0.45 0.50 52.77 44.87 2.27

2011 3595471 BCR 29,293 0.98 0.92 40.13 15.76 41.29 2.60

2011 3488530 BCR 29,293 0.37 0.48 60.25 36.96 2.66

2011 3564054 BCR 29,293 0.68 0.78 48.21 29.61 19.35 2.74


77


Year Item Seq.


Step 1-2

Step 2-3

2009 1 100000028258 SR 1.3712

2010 3 100000245515 SR 1.6584

2009 4 100000065205 BCR_A 0.9512

2009 5 3985586 BCR_B 0.8324 -3.1999 3.1999

2009 6 100000065317 SR 2.1102

2008 7 100000009923 SR -0.0519

2008 9 3488268 SR 0.7163

2007 10 3488245 SR -0.8703

2008 17 100000022537 SR 1.0137

2010 22 100000297356 SR 0.0470

2008 24 3511336 BCR_A 1.4848

2008 25 3563987 BCR_B 1.7477 -0.8139 0.8139

2008 34 100000022549 SR 0.1410

2009 35 3512615 BCR_A -0.4877

2009 36 3595439 BCR_B 0.5673 -1.6948 1.6948

2010 40 100000065193 SR -2.1352

2009 41 100000065233 SR 0.0353

2010 44 100000063620 SR -1.6123

2008 50 3488445 SR -1.8688

2007 51 3488452 SR -1.2619

2009 52 100000022530 BCR_A -0.4562

2009 53 3985592 BCR_B -0.7297 -0.3422 0.3422

2007 54 3488381 SR 0.9673

2007 57 3488394 SR 1.3515

2009 59 100000143153 SR -0.5633

2009 62 100000028238 BCR_A 0.5541

2009 63 3985553 BCR_B 0.7737 -0.5023 0.5023

2008 65 100000009963 SR 0.2800

2010 73 100000022532 BCR_A 1.6379

2010 74 3595471 BCR_B 1.5127 0.8480 -0.8480

2009 79 100000065216 SR 0.7262

2010 80 3488530 BCR_A 1.8673

2010 81 3564054 BCR_B 2.2051 -0.6519 0.6519

2011 1 100000028258 SR 1.3100


78


Year Item Seq.


Step 1-2

Step 2-3

2011 3 100000245515 SR 1.2615

2011 4 100000065205 BCR_A 0.8718

2011 5 3985586 BCR_B 0.8350 -2.8194 2.8194

2011 6 100000065317 SR 1.8204

2011 7 100000009923 SR -0.1160

2011 9 3488268 SR 0.4218

2011 10 3488245 SR -1.0096

2011 17 100000022537 SR 0.7823

2011 22 100000297356 SR -0.2154

2011 24 3511336 BCR_A 1.2703

2011 25 3563987 BCR_B 1.5647 -0.7648 0.7648

2011 34 100000022549 SR 0.1350

2011 35 3512615 BCR_A -0.4468

2011 36 3595439 BCR_B 0.8260 -1.2888 1.2888

2011 40 100000065193 SR -2.3882

2011 41 100000065233 SR 0.1537

2011 44 100000063620 SR -1.0976

2011 50 3488445 SR -2.0029

2011 51 3488452 SR -1.0322

2011 52 100000022530 BCR_A -0.2865

2011 53 3985592 BCR_B -0.7452 -0.3694 0.3694

2011 54 3488381 SR 0.4962

2011 57 3488394 SR 1.1666

2011 59 100000143153 SR -0.8320

2011 62 100000028238 BCR_A 0.4456

2011 63 3985553 BCR_B 0.6481 -0.3350 0.3350

2011 65 100000009963 SR 0.2208

2011 73 100000022532 BCR_A 1.4206

2011 74 3595471 BCR_B 1.1783 0.5388 -0.5388

2011 79 100000065216 SR 0.5413

2011 80 3488530 BCR_A 1.7777

2011 81 3564054 BCR_B 1.9885 -0.4800 0.4800


79



80



100000028396 0.55 0.57 3516333 0.70 0.68

100000245469 0.88 0.89 3564008 0.68 0.67

3517004 0.94 0.92 100000245436 0.42 0.45

3564010 0.68 0.68 100000065162 0.70 0.63

100000064773 0.63 0.60 100000065116 0.73 0.72

100000065101 0.77 0.75 100000022501 0.45 0.45

100000094477 0.30 0.26 3595485 0.78 0.74

3985566 0.36 0.33 3492097 0.85 0.84

100000249208 0.86 0.77 100000064587 0.41 0.40

100000245938 0.53 0.52 3985580 0.48 0.48

100000245433 0.67 0.71 100000064586 0.80 0.80

100000028367 0.38 0.38 100000004461 0.78 0.78

3595489 0.54 0.54 100000022488 0.49 0.46

100000065159 0.63 0.63 3488404 0.43 0.43

100000245478 0.61 0.59 3985578 0.55 0.53

100000064628 0.67 0.67 100000064621 0.55 0.58



81




2009 3517004 ECR 18,303 0.94 0.23 5.28 94.38 0.33

2009 3564010 ECR 18,303 2.04 0.87 3.59 22.83 37.58 35.17 0.83

2009 100000094477 BCR 2,488 0.30 0.46 63.55 29.90 6.55

2009 3985566 BCR 2,488 0.73 0.72 34.81 40.23 16.28 8.68

2010 100000028367 BCR 24,723 0.38 0.48 61.36 37.04 1.58

2010 3595489 BCR 24,645 1.08 0.83 30.02 30.21 37.82 1.89

2009 3516333 BCR 18,303 0.70 0.46 29.33 69.95 0.72

2009 3564008 BCR 18,303 1.37 0.79 18.58 23.89 56.34 1.19

2010 100000022501 BCR 24,765 0.45 0.50 54.60 43.97 1.41

2010 3595485 BCR 24,728 1.57 0.58 4.30 33.87 60.25 1.56

2009 100000064587 BCR 2,491 0.41 0.49 54.96 41.43 3.61

2009 3985580 BCR 2,491 0.96 0.74 26.42 44.40 25.89 3.29

2009 3488404 BCR 2,485 0.43 0.50 54.57 42.86 2.58

2009 3985578 BCR 2,485 1.09 0.62 13.08 60.85 24.10 1.97

2011 3517004 ECR 29,778 0.92 0.27 6.99 92.32 0.69

2011 3564010 ECR 29,778 2.04 0.92 5.40 20.10 36.08 37.08 1.29

2011 100000094477 BCR 29,778 0.26 0.44 68.24 25.59 6.04

2011 3985566 BCR 29,778 0.65 0.74 41.58 32.98 16.20 8.68

2011 100000028367 BCR 29,778 0.38 0.49 60.24 38.44 1.28

2011 3595489 BCR 29,778 1.09 0.84 29.39 29.51 39.57 1.48

2011 3516333 BCR 29,778 0.68 0.47 30.12 68.04 1.80

2011 3564008 BCR 29,778 1.35 0.76 15.26 29.73 52.52 2.35

2011 100000022501 BCR 29,778 0.45 0.50 53.61 44.63 1.69

2011 3595485 BCR 29,778 1.48 0.71 10.86 26.97 60.62 1.45

2011 100000064587 BCR 29,778 0.40 0.49 57.21 40.08 2.70

2011 3985580 BCR 29,778 0.95 0.72 25.30 48.45 23.51 2.68

2011 3488404 BCR 29,778 0.43 0.50 55.85 42.98 1.16

2011 3985578 BCR 29,778 1.06 0.59 12.60 65.26 20.26 1.78


82


Year Item Seq.


Step 1-2

Step 2-3

2008 2 100000028396 SR 0.7908

2010 5 100000245469 SR -1.2818

2009 7 3517004 ECR_A -2.4767

2009 8 3564010 ECR_B -0.2026 -1.9106 0.2374 1.6732

2010 13 100000064773 SR 0.3916

2009 21 100000065101 SR -0.6135

2009 22 100000094477 BCR_A 2.0661

2009 23 3985566 BCR_B 1.6893 -1.0781 1.0781

2010 24 100000249208 SR -1.1173

2010 28 100000245938 SR 0.8950

2010 29 100000245433 SR -0.0202

2010 31 100000028367 BCR_A 1.6930

2010 32 3595489 BCR_B 0.8402 -0.3408 0.3408

2009 33 100000065159 SR 0.2597

2010 44 100000245478 SR 0.5095

2009 46 100000064628 SR -0.0107

2009 47 3516333 BCR_A -0.0954

2009 48 3564008 BCR_B 0.0294 -0.1519 0.1519

2010 49 100000245436 SR 1.4294

2010 52 100000065162 SR 0.0311

2009 53 100000065116 SR -0.3249

2010 59 100000022501 BCR_A 1.3128

2010 60 3595485 BCR_B -0.8338 -1.1925 1.1925

2008 62 3492097 SR -1.1480

2009 66 100000064587 BCR_A 1.3489

2009 67 3985580 BCR_B 0.9987 -1.0243 1.0243

2009 69 100000064586 SR -0.9339

2008 70 100000004461 SR -0.5602

2008 71 100000022488 SR 1.0878

2009 77 3488404 BCR_A 1.3038

2009 78 3985578 BCR_B 0.6064 -1.7499 1.7499

2009 79 100000064621 SR 0.4897

2011 2 100000028396 SR 0.6355

2011 5 100000245469 SR -1.5044


83


Year Item Seq.


Step 1-2

Step 2-3

2011 7 3517004 ECR_A -2.0736

2011 8 3564010 ECR_B -0.0664 -1.5314 0.0203 1.5111

2011 13 100000064773 SR 0.4499

2011 21 100000065101 SR -0.4765

2011 22 100000094477 BCR_A 2.4967

2011 23 3985566 BCR_B 1.8157 -0.7847 0.7847

2011 24 100000249208 SR -0.6865

2011 28 100000245938 SR 0.8852

2011 29 100000245433 SR -0.0427

2011 31 100000028367 BCR_A 1.6232

2011 32 3595489 BCR_B 0.7895 -0.3122 0.3122

2011 33 100000065159 SR 0.3832

2011 44 100000245478 SR 0.5646

2011 46 100000064628 SR 0.1509

2011 47 3516333 BCR_A -0.0151

2011 48 3564008 BCR_B 0.1055 -0.5646 0.5646

2011 49 100000245436 SR 1.2690

2011 52 100000065162 SR 0.3152

2011 53 100000065116 SR -0.1755

2011 59 100000022501 BCR_A 1.3886

2011 60 3595485 BCR_B -0.2922 -0.5149 0.5149

2011 62 3492097 SR -1.0963

2011 66 100000064587 BCR_A 1.5611

2011 67 3985580 BCR_B 1.1123 -1.2436 1.2436

2011 69 100000064586 SR -0.8095

2011 70 100000004461 SR -0.6262

2011 71 100000022488 SR 1.2743

2011 77 3488404 BCR_A 1.4066

2011 78 3985578 BCR_B 0.8273 -2.0342 2.0342

2011 79 100000064621 SR 0.6947


84



85



100000028439 0.75 0.75 100000022498 0.62 0.63

100000313187 0.86 0.91 3595494 0.56 0.58

3517004 0.94 0.94 3492148 0.72 0.76

3564010 0.68 0.70 3488386 0.62 0.68

100000173712 0.62 0.65 3488387 0.64 0.73

100000065108 0.73 0.68 100000022504 0.60 0.63

100000094481 0.83 0.81 3595488 0.50 0.50

3985581 0.55 0.44 100000028401 0.84 0.83

100000028423 0.57 0.51 100000064587 0.41 0.41

100000094468 0.78 0.74 3985580 0.48 0.48

100000245433 0.67 0.70 100000064632 0.67 0.69

100000028363 0.72 0.71 100000245474 0.61 0.58

3595491 0.66 0.66 100000144352 0.53 0.53

100000313284 0.54 0.56 100000065098 0.55 0.54

100000245444 0.79 0.78 3985582 0.23 0.24

100000162295 0.73 0.67 3503955 0.60 0.60



86




2009 3517004 ECR 18,303 0.94 0.23 5.28 94.38 0.33

2009 3564010 ECR 18,303 2.04 0.87 3.59 22.83 37.58 35.17 0.83

2009 100000094481 BCR 2,482 0.83 0.38 16.20 82.64 1.17

2009 3985581 BCR 2,482 1.10 0.73 20.83 45.45 32.31 1.41

2010 100000028363 BCR 24,497 0.72 0.45 27.83 70.61 1.54

2010 3595491 BCR 24,471 1.32 0.78 19.07 28.39 50.85 1.65

2010 100000022498 BCR 24,607 0.62 0.48 36.87 61.04 2.04

2010 3595494 BCR 24,411 1.11 0.77 24.07 37.71 35.23 2.82

2008 100000022504 BCR 2,286 0.60 0.49 36.31 59.89 3.81

2008 3595488 BCR 2,286 0.99 0.91 36.79 17.06 41.08 5.07

2009 100000064587 BCR 2,491 0.41 0.49 54.96 41.43 3.61

2009 3985580 BCR 2,491 0.96 0.74 26.42 44.40 25.89 3.29

2009 100000065098 BCR 2,488 0.55 0.50 42.16 54.82 3.01

2009 3985582 BCR 2,488 0.47 0.63 56.95 32.64 7.15 3.26

2011 3517004 ECR 28,531 0.94 0.25 5.87 93.56 0.56

2011 3564010 ECR 28,531 2.09 0.89 4.33 18.97 36.52 38.94 1.20

2011 100000094481 BCR 28,531 0.81 0.39 16.10 80.98 2.89

2011 3985581 BCR 28,531 0.89 0.79 33.25 36.64 26.16 3.88

2011 100000028363 BCR 28,531 0.71 0.45 27.70 70.81 1.44

2011 3595491 BCR 28,531 1.32 0.76 16.56 31.69 49.97 1.71

2011 100000022498 BCR 28,531 0.63 0.48 34.26 62.90 2.75

2011 3595494 BCR 28,531 1.16 0.78 20.09 37.25 39.27 3.21

2011 100000022504 BCR 28,531 0.63 0.48 30.87 63.37 5.64

2011 3595488 BCR 28,531 1.01 0.92 37.12 14.72 42.92 5.00

2011 100000064587 BCR 28,531 0.41 0.49 55.26 40.68 4.06

2011 3985580 BCR 28,531 0.96 0.72 23.65 48.54 23.72 4.04

2011 100000065098 BCR 28,531 0.54 0.50 43.54 53.60 2.77

2011 3985582 BCR 28,531 0.47 0.64 56.57 30.96 8.13 4.18


87


Year Item Seq.


Step 1-2

Step 2-3

2008 2 100000028439 SR -0.2839

2010 5 100000313187 SR -1.3494

2009 7 3517004 ECR_A -2.4767

2009 8 3564010 ECR_B -0.2026 -1.9106 0.2374 1.6732

2009 13 100000173712 SR 0.3014

2010 21 100000065108 SR -0.1755

2009 22 100000094481 BCR_A -1.0875

2009 23 3985581 BCR_B 0.5992 -1.0400 1.0400

2009 24 100000028423 SR 0.5311

2009 28 100000094468 SR -0.6524

2010 29 100000245433 SR -0.0202

2010 31 100000028363 BCR_A -0.0837

2010 32 3595491 BCR_B 0.3055 -0.3163 0.3163

2010 33 100000313284 SR 0.8525

2010 44 100000245444 SR -0.5801

2010 46 100000162295 SR -0.1908

2010 47 100000022498 BCR_A 0.3903

2010 48 3595494 BCR_B 0.7504 -0.7203 0.7203

2007 49 3492148 SR -0.5390

2007 52 3488386 SR -0.0256

2007 53 3488387 SR -0.0325

2008 59 100000022504 BCR_A 0.3219

2008 60 3595488 BCR_B 0.8855 0.3642 -0.3642

2009 62 100000028401 SR -1.1408

2009 66 100000064587 BCR_A 1.3489

2009 67 3985580 BCR_B 0.9987 -1.0243 1.0243

2009 69 100000064632 SR 0.0037

2010 70 100000245474 SR 0.3428

2009 71 100000144352 SR 0.7768

2009 77 100000065098 BCR_A 0.5895

2009 78 3985582 BCR_B 2.5300 -1.0218 1.0218

2008 79 3503955 SR 0.5396

2011 2 100000028439 SR -0.2890

2011 5 100000313187 SR -1.7132


88


Year Item Seq.


Step 1-2

Step 2-3

2011 7 3517004 ECR_A -2.0736

2011 8 3564010 ECR_B -0.0664 -1.5314 0.0203 1.5111

2011 13 100000173712 SR 0.2612

2011 21 100000065108 SR 0.0225

2011 22 100000094481 BCR_A -0.8429

2011 23 3985581 BCR_B 1.3207 -0.7635 0.7635

2011 24 100000028423 SR 0.9446

2011 28 100000094468 SR -0.3704

2011 29 100000245433 SR -0.0427

2011 31 100000028363 BCR_A -0.1109

2011 32 3595491 BCR_B 0.2308 -0.3923 0.3923

2011 33 100000313284 SR 0.8487

2011 44 100000245444 SR -0.5459

2011 46 100000162295 SR 0.1805

2011 47 100000022498 BCR_A 0.3100

2011 48 3595494 BCR_B 0.6207 -0.7964 0.7964

2011 49 3492148 SR -0.3460

2011 52 3488386 SR 0.0671

2011 53 3488387 SR -0.1907

2011 59 100000022504 BCR_A 0.1906

2011 60 3595488 BCR_B 1.0505 0.4844 -0.4844

2011 62 100000028401 SR -0.9836

2011 66 100000064587 BCR_A 1.5611

2011 67 3985580 BCR_B 1.1123 -1.2436 1.2436

2011 69 100000064632 SR 0.0598

2011 70 100000245474 SR 0.7050

2011 71 100000144352 SR 0.9085

2011 77 100000065098 BCR_A 0.9643

2011 78 3985582 BCR_B 2.8227 -1.0241 1.0241

2011 79 3503955 SR 0.5628


89



90



100000263280 0.79 0.75 3564025 0.34 0.34

100000064017 0.32 0.34 100000063911 0.67 0.65

3985664 0.39 0.42 100000012791 0.37 0.35

100000063936 0.86 0.86 100000043347 0.74 0.73

3487551 0.60 0.61 3595366 0.43 0.30

100000282586 0.53 0.48 100000250878 0.54 0.51

100000263250 0.63 0.58 100000353725 0.60 0.56

3491692 0.48 0.47 100000282585 0.59 0.64

3564159 0.50 0.47 100000064068 0.68 0.67

100000063975 0.25 0.20 3985667 0.40 0.39

100000063997 0.72 0.65 100000071528 0.51 0.57

100000063933 0.76 0.72 100000353345 0.54 0.51

3517818 0.34 0.32 100000353336 0.50 0.51

3564023 0.36 0.34 100000063909 0.61 0.69

100000063914 0.47 0.45 3985662 0.35 0.40

100000063971 0.31 0.38 100000018108 0.65 0.65

100000282608 0.26 0.23 100000064018 0.57 0.59

3517706 0.55 0.50



91




2009 100000064017 BCR 2,484 0.32 0.47 63.73 32.17 4.11

2009 3985664 BCR 2,484 0.78 0.74 35.91 40.78 18.68 4.63

2010 3491692 ECR 21,935 0.48 0.50 49.65 46.03 4.20

2010 3564159 ECR 21,488 1.49 1.06 19.13 30.60 22.42 21.53 6.15

2010 3517818 BCR 20,906 0.34 0.48 58.28 30.64 11.07

2010 3564023 BCR 20,539 0.72 0.53 28.00 55.90 3.45 12.63

2010 3517706 BCR 22,490 0.55 0.50 42.91 52.52 4.33

2010 3564025 BCR 22,266 0.69 0.74 44.95 33.67 15.67 5.28

2009 100000043347 ECR 10,238 0.74 0.44 24.67 73.81 1.51

2009 3595366 ECR 10,238 1.28 0.72 11.97 44.23 40.01 1.11 2.69

2009 100000064068 ECR 2,486 0.68 0.47 30.65 67.66 1.69

2009 3985667 ECR 2,486 1.19 0.62 9.17 61.18 27.51 0.89 1.25

2009 100000063909 BCR 2,485 0.61 0.49 28.17 61.37 10.46

2009 3985662 BCR 2,485 0.70 0.48 20.28 68.49 0.93 10.30

2011 100000064017 BCR 30,180 0.34 0.47 63.02 34.33 2.57

2011 3985664 BCR 30,180 0.84 0.74 33.10 42.69 20.81 3.16

2011 3491692 ECR 30,180 0.47 0.50 49.15 46.71 4.06

2011 3564159 ECR 30,180 1.40 1.09 18.46 30.48 22.60 21.60 6.29

2011 3517818 BCR 30,180 0.32 0.47 57.00 31.82 10.90

2011 3564023 BCR 30,180 0.67 0.59 25.79 54.00 6.71 12.57

2011 3517706 BCR 30,180 0.50 0.50 45.92 49.72 4.17

2011 3564025 BCR 30,180 0.67 0.70 40.24 40.77 13.26 5.27

2011 100000043347 ECR 30,180 0.73 0.44 24.81 72.85 2.26

2011 3595366 ECR 30,180 0.90 0.51 14.91 73.04 8.38 0.14 3.27

2011 100000064068 ECR 30,180 0.67 0.47 31.88 66.91 1.16

2011 3985667 ECR 30,180 1.18 0.63 10.80 57.80 29.98 0.20 1.14

2011 100000063909 BCR 30,180 0.69 0.46 26.34 68.96 4.54

2011 3985662 BCR 30,180 0.79 0.46 17.75 74.85 2.17 4.83


92


Year Item Seq.


Step 1-2

Step 2-3

2010 4 100000263280 SR -1.1346

2009 5 100000064017 BCR_A 1.5621

2009 6 3985664 BCR_B 1.1803 -0.9952 0.9952

2009 9 100000063936 SR -1.7463

2007 11 3487551 SR -0.3270

2010 17 100000282586 SR 0.4481

2010 21 100000263250 SPR -0.0536

2010 22 3491692 ECR_A 0.7610

2010 23 3564159 ECR_B 0.6508 -1.3030 0.3303 0.9727

2009 24 100000063975 SPR 1.9910

2009 25 100000063997 SPR -0.7019

2010 26 100000063933 SPR -0.8874

2010 27 3517818 BCR_A 1.5333

2010 28 3564023 BCR_B 2.1002 -2.4025 2.4025

2009 29 100000063914 SR 0.7734

2010 38 100000063971 SPR 1.7880

2010 39 100000282608 SPR 2.0604

2010 40 3517706 BCR_A 0.3815

2010 41 3564025 BCR_B 1.5390 -0.7818 0.7818

2009 42 100000063911 SR -0.3440

2009 44 100000012791 SR 1.2954

2009 45 100000043347 ECR_A -0.6646

2009 46 3595366 ECR_B 1.6353 -3.0103 -0.6940 3.7043

2010 47 100000250878 SPR 0.4131

2010 48 100000353725 SPR 0.0616

2010 52 100000282585 SR 0.1269

2009 53 100000064068 ECR_A -0.3662

2009 54 3985667 ECR_B 1.7808 -3.8074 0.0287 3.7787

2009 55 100000071528 SPR 0.5274

2010 56 100000353345 SPR 0.2476

2010 57 100000353336 SPR 0.6775

2009 67 100000063909 BCR_A -0.4142

2009 68 3985662 BCR_B 2.5311 -3.5077 3.5077

2008 71 100000018108 SR -0.2796

2009 78 100000064018 SPR 0.1902

2011 4 100000263280 SR -0.8945

2011 5 100000064017 BCR_A 1.5009

2011 6 3985664 BCR_B 1.0367 -1.1360 1.1360


93


Year Item Seq. No.


Step 1-2

Step 2-3

2011 9 100000063936 SR -1.8067

2011 11 3487551 SR -0.0492

2011 17 100000282586 SR 0.6502

2011 21 100000263250 SPR 0.0904

2011 22 3491692 ECR_A 0.6758

2011 23 3564159 ECR_B 0.6840 -1.3053 0.1967 1.1086

2011 24 100000063975 SPR 2.3619

2011 25 100000063997 SPR -0.4241

2011 26 100000063933 SPR -1.0892

2011 27 3517818 BCR_A 1.4905

2011 28 3564023 BCR_B 1.7459 -2.1226 2.1226

2011 29 100000063914 SR 0.7756

2011 38 100000063971 SPR 1.2654

2011 39 100000282608 SPR 2.2465

2011 40 3517706 BCR_A 0.5804

2011 41 3564025 BCR_B 1.5909 -1.2226 1.2226

2011 42 100000063911 SR -0.2430

2011 44 100000012791 SR 1.5594

2011 45 100000043347 ECR_A -0.8028

2011 46 3595366 ECR_B 2.9325 -4.5203 0.7355 3.7848

2011 47 100000250878 SPR 0.4892

2011 48 100000353725 SPR 0.3091

2011 52 100000282585 SR -0.1181

2011 53 100000064068 ECR_A -0.3270

2011 54 3985667 ECR_B 2.8505 -4.7413 -1.0980 5.8392

2011 55 100000071528 SPR 0.2080

2011 56 100000353345 SPR 0.5279

2011 57 100000353336 SPR 0.5603

2011 67 100000063909 BCR_A -0.6136

2011 68 3985662 BCR_B 2.2752 -3.4808 3.4808

2011 71 100000018108 SR -0.2386

2011 78 100000064018 SPR 0.0153


94



95



100000263280 0.79 0.78 3564025 0.34 0.36

100000064058 0.60 0.62 100000250201 0.71 0.70

3985652 0.44 0.45 100000012791 0.37 0.36

100000063936 0.86 0.88 100000064101 0.64 0.61

100000026812 0.40 0.41 3985665 0.39 0.37

100000282586 0.53 0.52 100000250877 0.51 0.52

100000263247 0.37 0.43 100000353340 0.41 0.41

100000026808 0.35 0.38 3487727 0.59 0.61

3595374 0.38 0.40 3517648 0.68 0.66

100000064079 0.33 0.32 3564027 0.75 0.63

100000141392 0.30 0.31 100000353334 0.67 0.71

100000064098 0.80 0.77 100000353344 0.71 0.76

3517818 0.34 0.35 100000012801 0.35 0.38

3564023 0.36 0.36 100000012815 0.41 0.44

100000063979 0.38 0.41 3595391 0.51 0.53

100000043353 0.63 0.69 3513570 0.57 0.53

3517732 0.39 0.36 100000063939 0.35 0.36

3517706 0.55 0.53



96




2009 100000064058 BCR 2,483 0.60 0.49 38.18 59.85 1.97

2009 3985652 BCR 2,483 0.88 0.65 25.53 56.18 15.71 2.58

2010 100000026808 ECR 22,647 0.35 0.48 62.46 33.83 3.66

2010 3595374 ECR 22,378 1.15 1.14 40.34 15.17 24.28 15.17 4.81

2010 3517818 BCR 20,906 0.34 0.48 58.28 30.64 11.07

2010 3564023 BCR 20,539 0.72 0.53 28.00 55.90 3.45 12.63

2010 3517706 BCR 22,490 0.55 0.50 42.91 52.52 4.33

2010 3564025 BCR 22,266 0.69 0.74 44.95 33.67 15.67 5.28

2009 100000064101 ECR 2,487 0.64 0.48 33.61 63.93 2.45

2009 3985665 ECR 2,487 1.16 0.75 13.35 61.92 16.85 6.71 1.17

2009 3517648 ECR 10,238 0.68 0.47 30.82 68.36 0.82

2009 3564027 ECR 10,238 2.24 0.86 4.46 10.18 38.20 45.66 1.49

2008 100000012815 BCR 2,569 0.41 0.49 53.44 41.11 5.45

2008 3595391 BCR 2,569 1.02 0.59 9.42 65.01 18.72 6.85

2011 100000064058 BCR 29,100 0.62 0.49 36.49 61.72 1.76

2011 3985652 BCR 29,100 0.89 0.64 23.99 58.03 15.62 2.09

2011 100000026808 ECR 29,100 0.38 0.48 58.61 37.77 3.56

2011 3595374 ECR 29,100 1.19 1.14 36.93 14.23 27.93 16.24 4.41

2011 3517818 BCR 29,100 0.35 0.48 55.76 34.70 9.22

2011 3564023 BCR 29,100 0.73 0.60 23.58 57.24 7.88 10.60

2011 3517706 BCR 29,100 0.53 0.50 43.22 52.71 3.87

2011 3564025 BCR 29,100 0.72 0.70 37.00 43.40 14.47 4.74

2011 100000064101 ECR 29,100 0.61 0.49 37.46 61.23 1.31

2011 3985665 ECR 29,100 1.12 0.71 12.42 66.34 14.07 5.86 1.29

2011 3517648 ECR 29,100 0.66 0.47 33.08 65.70 1.17

2011 3564027 ECR 29,100 1.89 0.82 8.13 12.19 58.88 19.58 1.17

2011 100000012815 BCR 29,100 0.44 0.50 52.81 43.85 3.16

2011 3595391 BCR 29,100 1.05 0.50 6.93 74.49 15.32 3.06


97


Year Item Seq.


Step 1-2

Step 2-3

2010 4 100000263280 SR -1.1346

2009 5 100000064058 BCR_A -0.0425

2009 6 3985652 BCR_B 0.9437 -1.6435 1.6435

2009 9 100000063936 SR -1.7463

2008 11 100000026812 SR 1.1110

2010 17 100000282586 SR 0.4481

2010 21 100000263247 SPR 1.3556

2010 22 100000026808 ECR_A 1.4517

2010 23 3595374 ECR_B 1.2670 -0.1693 -0.8031 0.9724

2009 24 100000064079 SPR 1.5282

2010 25 100000141392 SPR 1.7706

2010 26 100000064098 SPR -1.1179

2010 27 3517818 BCR_A 1.5333

2010 28 3564023 BCR_B 2.1002 -2.4025 2.4025

2009 29 100000063979 SR 1.2313

2008 38 100000043353 SPR -0.4017

2009 39 3517732 SPR 1.2098

2010 40 3517706 BCR_A 0.3815

2010 41 3564025 BCR_B 1.5390 -0.7818 0.7818

2010 42 100000250201 SR -0.4898

2009 44 100000012791 SR 1.2954

2009 45 100000064101 ECR_A -0.1422

2009 46 3985665 ECR_B 1.2418 -2.8313 1.1189 1.7124

2010 47 100000250877 SPR 0.5541

2010 48 100000353340 SPR 1.1221

2008 52 3487727 SR 0.1334

2009 53 3517648 ECR_A -0.4422

2009 54 3564027 ECR_B -0.8657 -0.9282 -0.5852 1.5135

2010 55 100000353334 SPR -0.2599

2010 56 100000353344 SPR -0.5226

2008 57 100000012801 SPR 1.3565

2008 67 100000012815 BCR_A 0.8567

2008 68 3595391 BCR_B 0.0819 -2.2244 2.2244

2008 71 3513570 SR 0.2170

2009 78 100000063939 SPR 1.3800

2011 4 100000263280 SR -0.8945

2011 5 100000064058 BCR_A 0.0981

2011 6 3985652 BCR_B 1.0582 -1.7073 1.7073

2011 9 100000063936 SR -1.8067



98

Year Item Seq.


Step 1-2

Step 2-3

2011 11 100000026812 SR 1.1191

2011 17 100000282586 SR 0.6502

2011 21 100000263247 SPR 1.0841

2011 22 100000026808 ECR_A 1.3798

2011 23 3595374 ECR_B 1.2316 -0.1991 -0.8977 1.0969

2011 24 100000064079 SPR 1.6259

2011 25 100000141392 SPR 1.7744

2011 26 100000064098 SPR -1.1885

2011 27 3517818 BCR_A 1.4905

2011 28 3564023 BCR_B 1.7459 -2.1226 2.1226

2011 29 100000063979 SR 0.9780

2011 38 100000043353 SPR -0.3836

2011 39 3517732 SPR 1.5349

2011 40 3517706 BCR_A 0.5804

2011 41 3564025 BCR_B 1.5909 -1.2226 1.2226

2011 42 100000250201 SR -0.4897

2011 44 100000012791 SR 1.5594

2011 45 100000064101 ECR_A 0.1237

2011 46 3985665 ECR_B 1.4624 -3.1958 1.3386 1.8572

2011 47 100000250877 SPR 0.5205

2011 48 100000353340 SPR 1.2214

2011 52 3487727 SR 0.1672

2011 53 3517648 ECR_A -0.1506

2011 54 3564027 ECR_B 0.0836 -1.1232 -1.3537 2.4768

2011 55 100000353334 SPR -0.5274

2011 56 100000353344 SPR -0.9404

2011 57 100000012801 SPR 1.3060

2011 67 100000012815 BCR_A 0.9721

2011 68 3595391 BCR_B 0.2798 -2.7171 2.7171

2011 71 3513570 SR 0.4944

2011 78 100000063939 SPR 1.4155


99



100



3514013 0.54 0.50 100000064837 0.46 0.48

3564107 0.71 0.66 3985702 0.40 0.39

100000264701 0.48 0.47 100000026766 0.26 0.36

100000250166 0.77 0.77 3487759 0.32 0.31

100000064847 0.20 0.21 3564128 0.54 0.51

3985701 0.20 0.20 100000064933 0.81 0.80

100000220610 0.41 0.44 3514078 0.22 0.25

100000264735 0.40 0.47 3564109 0.31 0.39

100000264753 0.34 0.34 100000064863 0.46 0.52

100000199104 0.43 0.43 100000043326 0.49 0.46

100000264698 0.50 0.48 100000065015 0.50 0.50

100000064945 0.27 0.30 100000264729 0.62 0.63

3985679 0.48 0.50 100000353872 0.36 0.34

3492049 0.63 0.63 100000264722 0.55 0.49

3487636 0.22 0.21 100000012739 0.45 0.42

3595428 0.35 0.34 3985697 0.50 0.45

100000026743 0.41 0.42 100000018144 0.69 0.63

3492059 0.44 0.52



101



Year Item CID Item N Mean SD 0 1 2 3 Omit

2009 3514013 BCR 11,185 0.54 0.50 44.28 54.39 1.32

2009 3564107 BCR 11,185 1.42 0.66 7.56 38.66 51.87 1.90

2009 100000064847 ECR 2,491 0.20 0.40 71.02 19.83 9.15

2009 3985701 ECR 2,491 0.59 0.91 55.80 18.47 10.60 6.34 8.79

2009 100000064945 BCR 2,483 0.27 0.44 67.70 26.74 5.56

2009 3985679 BCR 2,483 0.96 0.72 22.71 47.32 24.37 5.60

2010 3487636 BCR 22,348 0.22 0.41 76.60 21.21 2.12

2010 3595428 BCR 22,162 0.70 0.81 50.97 24.20 21.75 2.94

2009 100000064837 ECR 2,493 0.46 0.50 48.54 46.33 5.13

2009 3985702 ECR 2,493 1.20 0.82 16.93 41.40 32.93 4.29 4.45

2009 3487759 BCR 11,185 0.32 0.47 64.58 31.94 3.48

2009 3564128 BCR 11,185 1.07 0.75 20.78 43.25 31.92 4.06

2007 3514078 ECR 32,836 0.22 0.41 72.06 21.62 6.33

2007 3564109 ECR 32,836 0.94 1.00 33.90 32.10 14.85 10.60 8.55

2009 100000012739 BCR 2,487 0.45 0.50 49.94 45.11 4.95

2009 3985697 BCR 2,487 1.00 0.52 7.80 73.42 13.51 5.27

2011 3514013 BCR 30,089 0.50 0.50 47.21 50.47 2.28

2011 3564107 BCR 30,089 1.32 0.69 10.01 41.94 45.19 2.79

2011 100000064847 ECR 30,089 0.21 0.41 72.41 20.99 6.55

2011 3985701 ECR 30,089 0.61 0.95 57.20 18.43 9.89 7.65 6.29

2011 100000064945 BCR 30,089 0.30 0.46 66.07 29.68 4.15

2011 3985679 BCR 30,089 1.00 0.71 20.40 48.90 25.64 4.86

2011 3487636 BCR 30,089 0.21 0.41 75.60 21.44 2.87

2011 3595428 BCR 30,089 0.68 0.81 50.23 23.57 22.26 3.80

2011 100000064837 ECR 30,089 0.48 0.50 48.47 47.61 3.69

2011 3985702 ECR 30,089 1.17 0.85 21.67 35.47 35.36 3.56 3.62

2011 3487759 BCR 30,089 0.31 0.46 63.25 31.32 5.30

2011 3564128 BCR 30,089 1.03 0.77 21.64 41.32 30.69 6.09

2011 3514078 ECR 30,089 0.25 0.43 71.65 25.29 2.93

2011 3564109 ECR 30,089 1.17 1.01 27.00 37.49 18.25 14.19 2.93

2011 100000012739 BCR 30,089 0.42 0.49 48.89 42.13 8.73

2011 3985697 BCR 30,089 0.91 0.48 7.51 76.17 7.33 8.70


102


Year Item Seq.


Step 1-2

Step 2-3

2009 3 3514013 BCR_A 0.1810

2009 4 3564107 BCR_B -0.9848 -1.1835 1.1835

2010 6 100000264701 SR 0.4127

2010 15 100000250166 SPR -1.2100

2009 16 100000064847 ECR_A 1.9474

2009 17 3985701 ECR_B 1.7916 -0.4753 -0.1498 0.6252

2009 18 100000220610 SPR 0.5579

2010 19 100000264735 SPR 0.8626

2010 20 100000264753 SPR 1.1874

2009 21 100000199104 SPR 0.6409

2010 23 100000264698 SR 0.3191

2009 24 100000064945 BCR_A 1.4454

2009 25 3985679 BCR_B 0.1651 -1.2302 1.2302

2009 26 3492049 SPR -0.4824

2010 34 3487636 BCR_A 1.9670

2010 35 3595428 BCR_B 1.0288 -0.1750 0.1750

2008 36 100000026743 SPR 0.7689

2009 37 3492059 SPR 0.5743

2009 39 100000064837 ECR_A 0.3647

2009 40 3985702 ECR_B 1.0231 -2.2879 -0.2807 2.5685

2008 43 100000026766 SR 1.4866

2009 44 3487759 BCR_A 1.2982

2009 45 3564128 BCR_B 0.0944 -1.0009 1.0009

2009 49 100000064933 SR -1.7363

2007 54 3514078 ECR_A 1.6578

2007 55 3564109 ECR_B 0.9387 -1.3082 0.3860 0.9222

2009 56 100000064863 SPR 0.3265

2009 57 100000043326 SPR 0.3680

2009 63 100000065015 SR 0.2031

2010 64 100000264729 SR -0.3348

2010 74 100000353872 SPR 1.0984

2010 75 100000264722 SPR 0.0350

2009 76 100000012739 BCR_A 0.4406

2009 77 3985697 BCR_B 0.0477 -2.5343 2.5343

2009 80 100000018144 SR -0.7487

2011 3 3514013 BCR_A 0.1260

2011 4 3564107 BCR_B -0.7774 -1.1328 1.1328

2011 6 100000264701 SR 0.3594

2011 15 100000250166 SPR -1.3656



103

Year Item Seq.


Step 1-2

Step 2-3

2011 16 100000064847 ECR_A 1.9993

2011 17 3985701 ECR_B 1.9666 -0.5702 0.0165 0.5536

2011 18 100000220610 SPR 0.5501

2011 19 100000264735 SPR 0.3267

2011 20 100000264753 SPR 1.2529

2011 21 100000199104 SPR 0.6764

2011 23 100000264698 SR 0.4975

2011 24 100000064945 BCR_A 1.4164

2011 25 3985679 BCR_B 0.1442 -1.4317 1.4317

2011 26 3492049 SPR -0.5141

2011 34 3487636 BCR_A 2.0944

2011 35 3595428 BCR_B 1.0876 -0.2705 0.2705

2011 36 100000026743 SPR 0.7068

2011 37 3492059 SPR 0.0625

2011 39 100000064837 ECR_A 0.3005

2011 40 3985702 ECR_B 1.2534 -2.3018 -0.6177 2.9195

2011 43 100000026766 SR 1.0957

2011 44 3487759 BCR_A 1.3048

2011 45 3564128 BCR_B 0.0858 -1.0235 1.0235

2011 49 100000064933 SR -1.6848

2011 54 3514078 ECR_A 1.7694

2011 55 3564109 ECR_B 0.8846 -1.4575 0.3985 1.0590

2011 56 100000064863 SPR 0.0387

2011 57 100000043326 SPR 0.4990

2011 63 100000065015 SR 0.3312

2011 64 100000264729 SR -0.5427

2011 74 100000353872 SPR 0.9601

2011 75 100000264722 SPR 0.0891

2011 76 100000012739 BCR_A 0.5847

2011 77 3985697 BCR_B 0.6836 -3.1803 3.1803

2011 80 100000018144 SR -0.6572


104



105



100000043324 0.48 0.48 100000026770 0.30 0.36

3595434 0.47 0.47 3595425 0.42 0.44

100000026750 0.59 0.61 100000064895 0.43 0.43

100000250168 0.19 0.17 100000064972 0.24 0.26

3487937 0.27 0.30 3985691 0.20 0.25

3564125 0.33 0.34 3487556 0.75 0.78

100000065028 0.73 0.74 3487639 0.58 0.64

100000264754 0.30 0.28 3985692 0.34 0.38

100000264733 0.55 0.59 100000220615 0.29 0.29

100000012737 0.41 0.46 100000064900 0.40 0.42

100000250120 0.66 0.66 100000004116 0.49 0.52

100000064945 0.27 0.33 100000264729 0.62 0.65

3985679 0.48 0.55 100000353934 0.56 0.50

100000064989 0.57 0.61 3492050 0.75 0.72

3487636 0.22 0.22 100000012739 0.45 0.43

3595428 0.35 0.35 3985697 0.50 0.47

100000064980 0.26 0.32 100000064874 0.71 0.64

3514131 0.45 0.48



106




2010 100000043324 BCR 22,034 0.48 0.50 50.13 46.29 3.50

2010 3595434 BCR 21,741 0.94 0.92 43.46 13.69 37.93 4.78

2008 3487937 ECR 2,279 0.27 0.44 65.99 26.94 7.06

2008 3564125 ECR 2,279 0.98 1.15 39.97 19.26 13.91 16.89 9.96

2009 100000064945 BCR 2,483 0.27 0.44 67.70 26.74 5.56

2009 3985679 BCR 2,483 0.96 0.72 22.71 47.32 24.37 5.60

2010 3487636 BCR 22,348 0.22 0.41 76.60 21.21 2.12

2010 3595428 BCR 22,162 0.70 0.81 50.97 24.20 21.75 2.94

2008 100000026770 ECR 2,215 0.30 0.46 65.91 30.02 4.06

2008 3595425 ECR 2,215 1.27 1.13 24.65 35.80 10.38 23.57 5.60

2009 100000064972 BCR 2,488 0.24 0.43 64.75 24.12 11.13

2009 3985691 BCR 2,488 0.41 0.64 55.71 24.52 8.16 11.62

2009 3487639 ECR 2,489 0.58 0.49 33.87 57.65 8.48

2009 3985692 ECR 2,489 1.01 0.88 22.74 43.47 17.92 7.11 8.76

2009 100000012739 BCR 2,487 0.45 0.50 49.94 45.11 4.95

2009 3985697 BCR 2,487 1.00 0.52 7.80 73.42 13.51 5.27

2011 100000043324 BCR 29,114 0.48 0.50 47.49 47.97 4.51

2011 3595434 BCR 29,114 0.95 0.94 41.06 11.96 41.39 5.47

2011 3487937 ECR 29,114 0.30 0.46 64.28 29.81 5.86

2011 3564125 ECR 29,114 1.02 1.16 42.77 18.48 14.82 18.01 5.72

2011 100000064945 BCR 29,114 0.33 0.47 63.64 32.97 3.33

2011 3985679 BCR 29,114 1.10 0.65 12.94 56.70 26.44 3.81

2011 3487636 BCR 29,114 0.22 0.42 75.08 22.42 2.48

2011 3595428 BCR 29,114 0.69 0.82 50.67 22.69 23.23 3.34

2011 100000026770 ECR 29,114 0.36 0.48 59.40 36.23 4.22

2011 3595425 ECR 29,114 1.33 1.19 27.92 28.60 12.21 26.71 4.22

2011 100000064972 BCR 29,114 0.26 0.44 64.05 25.66 10.02

2011 3985691 BCR 29,114 0.49 0.65 47.60 31.66 8.68 11.43

2011 3487639 ECR 29,114 0.64 0.48 31.64 64.41 3.83

2011 3985692 ECR 29,114 1.14 0.85 20.24 43.51 26.12 6.12 3.82

2011 100000012739 BCR 29,114 0.43 0.49 48.50 42.85 8.39

2011 3985697 BCR 29,114 0.94 0.48 6.22 76.61 8.50 8.38


107


Year Item Seq.


Step 1-2

Step 2-3

2010 3 100000043324 BCR_A 0.3845

2010 4 3595434 BCR_B 0.3994 0.6339 -0.6339

2008 6 100000026750 SR -0.2948

2010 15 100000250168 SPR 2.2058

2008 16 3487937 ECR_A 1.4131

2008 17 3564125 ECR_B 0.9003 -0.3528 0.0446 0.3081

2010 18 100000065028 SPR -0.9745

2010 19 100000264754 SPR 1.4327

2010 20 100000264733 SPR 0.0230

2010 21 100000012737 SPR 0.7805

2010 23 100000250120 SR -0.5131

2009 24 100000064945 BCR_A 1.4454

2009 25 3985679 BCR_B 0.1651 -1.2302 1.2302

2009 26 100000064989 SPR -0.1047

2010 34 3487636 BCR_A 1.9670

2010 35 3595428 BCR_B 1.0288 -0.1750 0.1750

2009 36 100000064980 SPR 1.5450

2010 37 3514131 SPR 0.5542

2008 39 100000026770 ECR_A 1.3141

2008 40 3595425 ECR_B 0.4226 -1.2296 1.1910 0.0386

2009 43 100000064895 SR 0.6158

2009 44 100000064972 BCR_A 1.6652

2009 45 3985691 BCR_B 1.9739 -0.7417 0.7417

2007 49 3487556 SR -1.3716

2009 54 3487639 ECR_A -0.3680

2009 55 3985692 ECR_B 1.1219 -2.0393 0.4881 1.5512

2009 56 100000220615 SPR 1.4325

2010 57 100000064900 SPR 0.8337

2008 63 100000004116 SR 0.1652

2010 64 100000264729 SR -0.3348

2010 74 100000353934 SPR 0.0301

2010 75 3492050 SPR -1.1053

2009 76 100000012739 BCR_A 0.4406

2009 77 3985697 BCR_B 0.0477 -2.5343 2.5343

2009 80 100000064874 SR -0.8887

2011 3 100000043324 BCR_A 0.3396

2011 4 3595434 BCR_B 0.3357 0.6920 -0.6920

2011 6 100000026750 SR -0.2508


108


Year Item Seq.


Step 1-2

Step 2-3

2011 15 100000250168 SPR 2.4519

2011 16 3487937 ECR_A 1.4659

2011 17 3564125 ECR_B 1.1460 -0.2932 -0.1015 0.3948

2011 18 100000065028 SPR -1.1078

2011 19 100000264754 SPR 1.7771

2011 20 100000264733 SPR -0.0502

2011 21 100000012737 SPR 0.5104

2011 23 100000250120 SR -0.3904

2011 24 100000064945 BCR_A 1.4164

2011 25 3985679 BCR_B 0.1442 -1.4317 1.4317

2011 26 100000064989 SPR -0.2805

2011 34 3487636 BCR_A 2.0944

2011 35 3595428 BCR_B 1.0876 -0.2705 0.2705

2011 36 100000064980 SPR 1.3433

2011 37 3514131 SPR 0.4071

2011 39 100000026770 ECR_A 1.1258

2011 40 3595425 ECR_B 0.5402 -0.8780 0.8115 0.0665

2011 43 100000064895 SR 0.7846

2011 44 100000064972 BCR_A 1.6793

2011 45 3985691 BCR_B 1.9569 -1.0804 1.0804

2011 49 3487556 SR -1.3128

2011 54 3487639 ECR_A -0.5168

2011 55 3985692 ECR_B 1.2467 -2.2743 -0.0533 2.3275

2011 56 100000220615 SPR 1.4967

2011 57 100000064900 SPR 0.8124

2011 63 100000004116 SR 0.0898

2011 64 100000264729 SR -0.5427

2011 74 100000353934 SPR 0.2310

2011 75 3492050 SPR -1.0440

2011 76 100000012739 BCR_A 0.5847

2011 77 3985697 BCR_B 0.6836 -3.1803 3.1803

2011 80 100000064874 SR -0.6573


109



110

1.9 Linking, Equating, and Scaling Procedures

For the purpose of year-to-year linking and equating, we constructed a 2011 linking pool that included only operational selected-response (SR) items (i.e., multiple-choice items). These SR items appeared in both current and previous years’ assessments and were used as either core or core link item in previous years’ assessments (i.e., in any assessment before 2011). After setting up the linking pool, we conducted a stability check of linking items and decided which items should be excluded from or which items should remain in the linking pool. During the calibration and equating process, we kept and fixed the original operational Rasch item difficulty parameters of any linking items that remained through the stability check to put the 2011 assessment on a common scale. Accordingly, scale scores of the 2011 assessment were linked back to the 2006 assessment and all the scale scores of different years were comparable within each content and grade. Rasch recalibration was conducted using the 2006 MSA-Math data in 2007 due to the IRT model transition (i.e., from 3-PL to the Rasch). Detailed information on the 2006 Rasch recalibration and results can be obtained in the 2007 MSA-Math technical report.

Stratified Random Sampling Procedures

To select equating samples, a stratified random sampling method was applied to the 2011 state examinee population. To verify that the sample was representative of the statewide examinee population, the distributions of LEA, gender, and ethnicity of the 2011 sample were compared with those of the 2011 population. Appendix A, The 2011 MSA-Math Stratified Random Sampling, provides the results of 2011 sampling. The results indicated that the equating samples were well representative of the statewide examinee population in terms of LEA, gender, and ethnicity.

Robust Z Procedures

After selecting equating samples, each operational form was independently calibrated to estimate Rasch item difficulty of each item. Then Robust z values of all anchor items were calculated using the following calculations (South Carolina Department of Education, 2001):

The mean and standard deviation of the linking pool’s item difficulties for each operational form

The ratio of the standard deviations between operational Form A and Form F

The correlation between operational Form A and F item difficulties

The difference between operational Form A and F for each item in the linking pool

The mean of the differences calculated above

The median of the differences calculated above

The interquartile range of the differences calculated above

The robust z is defined as (the difference between the test form1 and other test form item difficulty minus the median of the differences) / (interquartile range multiplied by 0.74).


111

Guidelines for Selecting Form-to-Form or Year-to-Year Linking Items

Once the above calculations were made, the following guidelines were followed in determining form-to-form or year-to-year common items used for Rasch linking and equating:

Conform to the following protocol criteria: A correlation greater than 0.95 and a standard deviation ratio between 0.9 and 1.1. For example, use all the possible linking items as anchors if an original set of linking items meets these two criteria.

Try not to include items with an absolute value of robust z exceeding 1.645.

If one item difficulty on one form of the current year is eliminated from the linking pool, other item difficulties of the other forms should not be included.

Should not eliminate more than 20 percent of the linking pool items.

Figure 1.15 depicts how we applied the anchor stability guidelines into the 2011 MSA-Math equating.

Form-to-Form Linking Procedures

The stability of the common items appearing on both operational forms was verified at each grade level:

Calibrate the two operational test forms separately

Calculate robust z values of Rasch item difficulties for Forms A and F

Correlate Rasch item difficulties between Form A and Form F

Calculate standard deviation ratio between two forms

After examining the robust z values, correlation coefficient, and standard deviation ratio between the two separate calibrations, it was determined that the common item difficulties were consistent across the two forms for all items and could be included as form-to-form linking items in the fixed calibration of the two forms.

Year-to-Year Linking Procedures

The two 2011 operational forms included a set of year-to-year linking common items that appeared on both current and previous operational forms. We utilized the Rasch item fixed equating method for all of the operational items to be placed on a common scale within each grade.

The stability of the linking common items was evaluated using robust z values, correlation coefficients, and standard deviation ratios.

Tables 1.74 through 1.79 include Rasch item difficulties used for calculating robust z values, correlation coefficients, and standard deviations. Figures 1.16 through 1.27 depict item difficulty plots between current and previous years. It should be noted that the item difficulties of the 2011 operational forms were obtained from independent calibration, and those of previous assessments were on a common scale (i.e., linked to the 2006 assessment).


112

Figure 1.15. Anchor Evaluation Steps Chart for MSA-Math

Calculate

- Robust Z value

- Correlation Coefficient

- Standard Deviation Ratio

Is the correlation higher than 0.95 and the SD ratio between 0.9 and 1.1?

Are at least 80% of the original anchor items

still available?No

Yes

Re-compute the correlation and the SD ratio excluding the dropped

item (don’t recalculate Robust Z values).

Identify the item with the largest Robust Z value

and drop it.Yes

Stop! Keep the remaining items.

No


113

Table 1.74. Core Linking Item Difficulties of Previous Year vs. Year 2011: Grade 3

Item Seq No.

Previous Form A

Y2011 Form A Robust Z

Item Seq No.

Previous Form F

Y2011 Form F Robust Z

1 -0.561 -1.0434 -.2561 1 -0.561 -1.0721 -.7373

2 0.6288 0.216 .1830 2 0.6288 0.1983 -.2417

5 1.8287 1.2822 -.6605 5 1.8287 1.3332 -.6414

6 1.5712 1.0714 -.3659 6 1.5712 1.1685 -.0707

7 2.0124 1.6983 .8056 7 2.0124 1.7158 .5817

13 1.3184 0.7838 -.5855 13 1.3184 0.6349 -1.7975

14 -1.132 -1.5719 .0088 14 -1.1315 -1.9276 -2.4899

15 2.4187 1.7801 -1.2416 15 2.4187 1.8802 -.9058

16 0.3981 -0.149 -.6643 16 0.3981 -0.3359 -2.1080

17 -0.386 -0.8134 .0934 17 -0.3864 -0.772 .0344

18 -0.204 -0.5288 .7369 18 -0.2038 -0.522 .4489

19 2.0077 1.4598 -.6694 19 2.0077 1.4593 -.9667

20 -1.147 -1.3594 1.4472 20 -1.147 -1.4837 .3351

23 1.8411 1.1428 -1.6182 23 1.8411 1.2016 -1.5269

31 -0.278 -0.9738 -1.5999 31 -0.2784 -0.856 -1.1463

32 0.5005 0.241 1.1501 32 0.5005 0.1342 .1531

33 1.3765 1.3126 2.3841 33 1.3765 1.4753 3.0133

41 0.7136 0.416 .9097 41 0.7136 0.4992 1.0872

48 -2.182 -1.9527 4.2351 48 -2.1822 -2.1696 2.4832

49 1.3897 1.0496 .6416 49 1.3897 1.1076 .6709

51 -0.591 -1.0406 -.0517 51 -0.5906 -0.9616 .1242

52 -0.617 -0.9487 .6914 52 -0.6165 -0.8494 .9735

56 -0.606 -1.0477 .0000 56 -0.6059 -1.1217 -.7662

62 0.9229 0.5952 .7198 62 0.9229 0.5372 .0338

63 -0.499 -0.9436 -.0196 63 -0.4987 -0.7615 .7896

64 -0.008 -0.7632 -1.9803 64 -0.0075 -0.4097 -.0676

65 -0.85 -0.9037 2.4453 65 -0.8495 -1.0583 1.1217

66 -0.54 -0.7044 1.7482 66 -0.5397 -0.7023 1.4058

67 1.5719 0.7114 -2.6415 67 1.5719 0.7431 -2.6910

68 0.0473 -0.4803 -.5413 68 0.0473 -0.3529 -.0553

69 1.3024 0.8025 -.3665 69 1.3024 0.8938 -.1070

71 0.4822 -0.0263 -.4208 71 0.4822 0.1462 .3395

72 0.8475 0.1814 -1.4151 72 0.8475 0.1376 -1.9598

81 1.2021 0.8111 .3205 81 1.2021 0.8109 .0000

82 1.8021 1.4047 .2801 82 1.8021 1.4469 .2214


114

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean

SD

0.474

1.126

0.049

1.048

0.474

1.126

0.062

1.091

*Note: mean and standard deviation of Year 10 is calculated with freely calibrated estimates.

Correlation and Standard Deviation Ratio

Correlation Coefficient

SD Ratio

0.983

93%

0.983

97%

Values Used for Robust Z Statistics

Mean Diff

Median Diff

IQR Diff

-0.425

-0.442

0.214

-0.412

-0.391

0.220

Based on correlation coefficients and SD ratios, none of the linking common items were dropped from the linking pool.

Maryland S

Figure 1.1Form A

Figure 1.1Form F

School Assessmen

16. Item Diffic

17. Item Diffic

nt-Mathematics:

ulty Plot of Pr

ulty Plot of Pr

: Grades 3 throu

revious Year F

revious Year F

ugh 8

115

Form (Base F

Form (Base F

orm) vs. Curr

orm) vs. Curr

2011 Ad

rent Year (201

rent Year (201

dministration

11) Form: Gra

11) Form: Gra

ade 3

ade 3


116


Item Seq No.

Previous Form A


Item Seq No.

Previous Form F


2 -0.9015 -0.692 1.0616 2 -1 -0.867 .6656

3 0.6341 0.4383 -.0475 3 0.6 0.3592 -.4125

6 0.4436 0.265 -.0004 6 0.4 0.3315 .1549

7 -0.8522 -0.638 1.0733 7 -1 -0.649 1.2553

8 -1.3705 -1.488 .1662 8 -1 -1.55 -.0807

10 0.9009 0.5327 -.5192 10 0.9 0.6187 -.4380

18 -0.4437 -0.367 .6993 18 -0 -0.37 .8012

19 1.4979 1.0371 -.7726 19 1.5 0.9999 -1.1901

22 0.4883 0.2233 -.2368 22 0.5 0.265 -.2327

23 1.757 1.3964 -.4984 23 1.8 1.5092 -.3181

24 0.9747 2.0049 3.3073 24 1 2.0024 4.1278

25 1.2547 0.9051 -.4683 25 1.3 1.0908 -.0256

26 1.1949 1.2714 .6976 26 1.2 1.3272 1.0068

30 -0.2781 -0.634 -.4847 30 -0 -0.442 -.0246

32 -0.5617 -0.992 -.6900 32 -1 -1.1 -1.3317

33 -1.7288 -2.554 -1.7700 33 -2 -2.475 -2.0563

47 -0.1077 -1.136 -2.3255 47 -0 -1.296 -3.5976

49 -1.3258 -1.623 -.3241 49 -1 -1.735 -.8796

50 0.9291 0.9964 .6725 50 0.9 0.9504 .6199

54 0.6901 0.8529 .9338 54 0.7 0.8075 .9549

55 -0.4674 -0.242 1.1059 55 -0 -0.311 1.0912

62 1.0327 1.5959 2.0294 62 1 1.6143 2.5729

63 -0.2435 -0.463 -.1134 63 -0 -0.508 -.3770

64 -0.106 -0.255 .0814 64 -0 -0.484 -.7722

66 0.6346 0.9574 1.3716 66 0.6 0.8733 1.3777

67 -0.3619 -0.298 .6645 67 -0 -0.357 .5634

68 -2.3 -2.693 -.5879 68 -2 -2.667 -.7332

69 -0.9014 -0.707 1.0203 69 -1 -0.844 .7450

71 -0.8156 -1.047 -.1446 71 -1 -0.965 .0246

78 -1.2169 -1.689 -.8038 78 -1 -1.985 -2.1302

80 -0.0118 0.1166 .8397 80 -0 0.2704 1.5293

81 -0.1831 -0.361 .0004 81 -0 -0.265 .2619


117

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean

SD

-0.055

1.002

-0.165

1.160

-0.055

1.002

-0.183

1.195

*Note: mean and standard deviation of Year 10 is calculated with freely calibrated estimates.


Correlation Coefficient

SD Ratio

0.945

116%

0.947

119%

Values used fro Robust Z Statistics

Mean Diff

Median Diff

-0.111

-0.178

-0.128

-0.157

IQR Diff 0.494 0.388

Based on correlation coefficients, SD ratios, robust z, and item difficulty plot, item number 24, 47 and 62 appearing on both forms was dropped from the linking pool.

The following correlation coefficients and SD ratios were calculated after dropping that item:

Correlation with Base 0.967 0.971

SD Ratio 107% 110%


118

Figure 1.18.Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form: Grade 4

Figure 1.19. Item Difficulty Plot of Previous Year Form (Base Form) vs. Current Year (2011) Form: Grade 4 Form F

Rasch Item Diffculties of Common Items: Grade 4 Form A

-4

-3

-2

-1

0

1

2

3

4

-4 -3 -2 -1 0 1 2 3 4

Base Form

Form

A

Rasch Item Diffculties of Common Items: Grade 4 Form F

-4

-3

-2

-1

0

1

2

3

4

-4 -3 -2 -1 0 1 2 3 4

Base Form

Fo

rm F


119


Item Seq No.

Previous Form A


Item Seq No.

Previous Form F


2 -0.459 -0.4926 .3847 2 -0.4592 -0.4594 .1864

8 -0.05 0.201 1.7044 8 -0.05 0.2083 1.4751

16 -1.704 -1.9622 -.6575 16 -1.7042 -1.9939 -1.2568

18 0.179 -0.1141 -.8204 18 0.179 -0.0372 -.8903

19 -1.786 -1.7548 .6854 19 -1.7862 -1.6111 1.0603

20 0.4459 0.5546 1.0441 20 0.4459 0.547 .6914

21 0.3045 0.2977 .5081 21 0.3045 0.4593 .9591

23 0.335 0.0384 -.8366 23 0.335 0.0706 -1.1306

26 -1.098 -0.925 1.3424 26 -1.098 -0.6594 2.3739

27 -0.551 -0.6743 -.0325 27 -0.551 -0.6056 -.0847

28 -1.132 -1.4683 -1.0218 28 -1.1318 -1.356 -.9302

37 -0.818 -1.3506 -1.9340 37 -0.8175 -1.2274 -1.8560

38 1.0908 0.9199 -.2534 38 1.0908 1.0269 -.1311

39 1.6277 1.2826 -1.0617 39 1.6277 1.3715 -1.0898

42 0.1548 0.1513 .5234 42 0.1548 0.1342 .0847

43 -1.129 -1.3138 -.3165 43 -1.1293 -1.0678 .4940

47 -1.259 -1.2253 .6965 47 -1.2591 -1.3961 -.4955

48 -0.213 -0.3223 .0325 48 -0.213 -0.2103 .2009

49 -1.459 -1.6391 -.2965 49 -1.4589 -1.6168 -.5997

55 -1.28 -0.8816 2.3879 55 -1.2799 -0.9345 1.9093

56 -0.591 -0.5778 .6000 56 -0.5908 -0.7746 -.7288

58 -0.695 -0.4449 1.6988 58 -0.6947 -0.2726 2.2917

60 -0.537 -0.6952 -.1926 60 -0.5374 -0.5544 .1027

61 -0.503 -0.408 .9782 61 -0.5025 -0.3568 .9138

64 -0.926 -1.25 -.9638 64 -0.926 -1.2938 -1.6461

70 0.2463 0.1122 -.0826 70 0.2463 0.2328 .1201

71 0.5581 0.342 -.4631 71 0.5581 0.39 -.6506

72 -0.128 -0.4124 -.7796 72 -0.1281 -0.243 -.3854

82 -0.072 -0.0247 .7577 82 -0.0717 -0.0813 .1396

83 -0.684 -0.5074 1.3587 83 -0.6839 -0.7896 -.3395


120

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean -0.404 -0.485 -0.404 -0.437

SD 0.808 0.798 0.808 0.810


Correlation Coefficient 0.964 0.964

SD Ratio 99% 100%

Valued Used for Robust Z Statistics

Mean Diff -0.080 -0.032

Median Diff -0.116 -0.038

IQR Diff 0.291 0.271


Maryland S

Figure 1.2Form A

Figure 1.2Form F

School Assessmen

20. Item Diffic

21. Item Diffic

nt-Mathematics:

ulty Plot of Pr

ulty Plot of Pr

: Grades 3 throu

revious Year F

revious Year F

ugh 8

121

Form (Base F

Form (Base F

orm) vs. Curr

orm) vs. Curr

2011 Ad

rent Year (201

rent Year (201

dministration

11) Form: Gra

11) Form: Gra

ade 5

ade 5


122


Item Seq No.

Previous Form A


Item Seq No.

Previous Form F


1 -1.2053 -1.7141 -1.5239 1 -1.2053 -1.7077 -1.8397

3 0.5836 0.3997 .1351 3 0.5836 0.3426 -.3409

4 0.6958 0.5711 .4373 4 0.6958 0.5934 .4538

6 -0.2844 -0.7041 -1.0690 6 -0.2844 -0.5605 -.5421

9 0.2299 -0.1591 -.9122 9 0.2299 0.0193 -.1666

10 -1.4432 -1.9492 -1.5096 10 -1.4432 -1.961 -1.9280

11 -0.4703 -0.8525 -.8775 11 -0.4703 -0.7597 -.6184

12 0.2578 0.5286 2.4568 12 0.2578 0.701 3.5823

19 -0.6463 -0.863 -.0324 19 -0.6463 -0.6938 .7686

20 0.4042 0.236 .2152 20 0.4042 0.3126 .5158

25 0.3609 0.088 -.3194 25 0.3609 0.2371 .3311

26 -0.1396 -0.6535 -1.5500 26 -0.1396 -0.4655 -.8277

27 0.2101 -0.1009 -.5139 27 0.2101 -0.1007 -.7411

30 -0.6658 -0.693 .9352 30 -0.6658 -0.7015 .8363

34 1.1378 0.6119 -1.6113 34 1.1378 0.671 -1.6356

35 -1.4702 -1.5312 .7626 35 -1.4702 -1.5293 .7021

36 -0.8511 -1.2639 -1.0337 36 -0.8511 -1.2999 -1.5324

37 0.5144 -0.0128 -1.6179 37 0.5144 -0.0587 -2.2451

38 1.1803 0.8076 -.8290 38 1.1803 0.9493 -.2835

45 -0.9261 -0.7879 1.7798 45 -0.9261 -0.8513 1.4699

50 0.8786 0.7241 .2852 50 0.8786 0.6671 -.1717

51 -0.0834 -0.2249 .3516 51 -0.0834 -0.1796 .4894

54 0.2864 0.3895 1.6005 54 0.2864 0.4447 1.9487

55 0.8127 0.6175 .0774 55 0.8127 0.7429 .6408

56 1.0083 0.8043 .0324 56 1.0083 0.8357 .0513

57 0.8821 0.7081 .1856 57 0.8821 0.6916 -.0513

58 -0.2639 -0.7315 -1.3136 58 -0.2639 -0.4577 -.0702

61 0.283 -0.0668 -.7121 61 0.283 0.254 .8747

68 0.4071 0.4458 1.2717 68 0.4071 0.5354 1.7767


123

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean 0.075 -0.166 0.075 -0.091

SD 0.754 0.797 0.754 0.813



SD Ratio 106% 108%


Mean Diff -0.240 -0.166


IQR Diff 0.265 0.236


Maryland S

Figure 1.2

Figure 1.2Form F

School Assessmen

22. Item Diffic

23. Item Diffic

nt-Mathematics:

ulty Plot of Pr

ulty Plot of Pr

: Grades 3 throu

revious Year F

revious Year F

ugh 8

124

Form (Base F

Form (Base F

orm) vs. Curr

orm) vs. Curr

2011 Ad

rent Year (201

rent Year (201

dministration

11) Form: Gra

11) Form: Gra

ade 6

ade 6


125


Item Seq No.

Previous Form A


Item Seq No.

Previous Form F


1 -0.468 -0.7551 .0000 1 -0.4683 -0.8394 -.6807

2 0.4455 0.0357 -.6187 2 0.4455 0.0147 -.9590

3 -2.682 -3.4632 -2.4869 3 -2.682 -3.3312 -1.9773

7 -0.082 -0.1125 1.2912 7 -0.0824 -0.2863 .0988

8 -0.642 -1.0434 -.5765 8 -0.642 -0.9803 -.5278

10 -0.4 -0.5588 .6429 10 -0.3998 -0.5963 .1333

12 -1.476 -1.8478 -.4266 12 -1.4762 -1.743 -.1944

18 -0.636 -1.1148 -.9663 18 -0.6359 -1.1097 -1.1595

19 -1.124 -1.9542 -2.7319 19 -1.1243 -2.057 -3.2990

20 -1.918 -2.1757 .1479 20 -1.9183 -1.9976 .6798

30 1.4497 1.1147 -.2425 30 1.4497 1.1755 -.2289

31 0.9745 0.8979 1.0573 31 0.9745 0.8421 .4322

32 0.1508 -0.0955 .2037 32 0.1508 -0.0462 .1310

43 -0.409 -0.6826 .0684 43 -0.4094 -0.6825 -.2238

49 0.2502 0.1702 1.0402 49 0.2502 0.2861 1.2168

50 0.7735 0.2903 -.9879 50 0.7735 0.3088 -1.1171

51 -0.058 -0.0809 1.3290 51 -0.0583 -0.2208 .2919

63 0.9999 0.7863 .3682 63 0.9999 1.0445 1.2574

64 0.0092 -0.3211 -.2188 64 0.0092 -0.2159 .0000

65 1.0861 0.1247 -3.3933 65 1.0861 0.276 -2.7274

66 0.0819 -0.2843 -.3994 66 0.0819 -0.188 -.2089

69 0.3744 0.2142 .6368 69 0.3744 0.5037 1.6523

70 -0.278 -0.7221 -.7892 70 -0.2784 -0.845 -1.5922

72 0.6673 0.3798 -.0035 72 0.6673 0.514 .3347

79 -1.46 -1.2097 2.7032 79 -1.4603 -1.2692 1.9404

80 -0.02 -0.1351 .8647 80 -0.0202 -0.11 .6308

81 -0.039 -0.1245 1.0101 81 -0.0385 -0.1283 .6308


126

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean -0.164 -0.469 -0.164 -0.433

SD 0.958 1.017 0.958 1.029

Correlation and Standard Deviation


SD Ratio 106% 107%


Mean Diff -0.305 -0.269


IQR Diff 0.269 0.290


Maryland S

Figure 1.2Form A

Figure 1.2Form F

School Assessmen

24. Item Diffic

25. Item Diffic

nt-Mathematics:

ulty Plot of Pr

ulty Plot of Pr

: Grades 3 throu

revious Year F

revious Year F

ugh 8

127

Form (Base F

Form (Base F

orm) vs. Curr

orm) vs. Curr

2011 Ad

rent Year (201

rent Year (201

dministration

11) Form: Gra

11) Form: Gra

ade 7

ade 7


128


Item Seq No.

Previous Form A


Item Seq No.

Previous Form F


1 -0.427 -0.5515 .0000 1 -0.4269 -0.5782 -.0553

2 -0.572 -0.9134 -1.2857 2 -0.5724 -0.9183 -1.5098

5 -0.755 -1.1973 -1.8876 5 -0.755 -1.1954 -2.2161

7 -1.2 -1.365 -.2382 7 -1.2003 -1.4682 -.9268

8 0.3621 0.3424 .6232 8 0.3621 0.2182 .0000

14 0.3158 0.3823 1.1354 14 0.3158 0.3178 1.0905

22 -0.854 -1.1119 -.7920 22 -0.854 -1.0944 -.7213

27 0.2581 0.2696 .8086 27 0.2581 0.0664 -.3573

32 -1.16 -1.2338 .3018 32 -1.16 -1.1875 .8700

33 0.5139 0.449 .3547 33 0.5139 0.4023 .2414

38 -0.593 -0.9043 -1.1110 38 -0.5927 -0.7381 -.0112

41 0.5661 0.0493 -2.3302 41 0.5661 0.1765 -1.8364

42 -0.545 -0.5773 .5496 42 -0.5452 -0.6252 .4776

46 0.0785 0.2662 1.8555 46 0.0785 0.1056 1.2781

47 -2.156 -2.2317 .2876 47 -2.1555 -2.2384 .4559

48 -1.485 -1.9607 -2.0848 48 -1.4852 -2.1019 -3.5338

50 0.0551 -0.0564 .0778 50 0.0551 0.0696 1.1839

51 -0.515 -0.6657 -.1551 51 -0.515 -0.5985 .4514

52 -0.533 -1.0577 -2.3771 52 -0.533 -1.0403 -2.7162

53 0.1649 0.0392 -.0065 53 0.1649 0.0859 .4851

62 -0.142 -0.4321 -.9809 62 -0.1424 -0.4046 -.8842

65 -1.876 -1.9403 .3606 65 -1.8764 -1.9839 .2721

66 0.0346 -0.2266 -.8116 66 0.0346 -0.1242 -.1114

72 -0.381 -0.5126 -.0446 72 -0.3805 -0.4425 .6121

73 1.2076 1.3029 1.3065 73 1.2076 1.3111 1.8491

78 -0.093 -0.1956 .1331 78 -0.0934 -0.2588 -.1607

79 -0.049 0.2249 2.3658 79 -0.0487 0.0819 2.0517


129

Form Statistics

Form Statistics

Previous Year

Form

Y11

Form A

Previous Year

Form

Y11

Form F

Mean -0.362 -0.511 -0.362 -0.525

SD 0.764 0.840 0.764 0.837

Correlation and Standard Deviation


SD Ratio 110% 110%


Mean Diff -0.149 -0.162


IQR Diff 0.227 0.181


Maryland S

Figure 1.2Form A

Figure 1.2Form F

School Assessmen

26. Item Diffic

27. Item Diffic

nt-Mathematics:

ulty Plot of Pr

ulty Plot of Pr

: Grades 3 throu

revious Year F

revious Year F

ugh 8

130

Form (Base F

Form (Base F

orm) vs. Curr

orm) vs. Curr

2011 Ad

rent Year (201

rent Year (201

dministration

11) Form: Gra

11) Form: Gra

ade 8

ade 8


131

Reporting Scale Scores

In order to facilitate the use and interpretation of the results of the 2011 MSA-Math, the following formula was used to convert each student’s ability or theta to the reporting scale score:

2954.3808398.32 thetaeScorebilityScalReportingA

SEEReportingS 8398.32

where

theta = the Rasch (i.e., 1-PL IRT) ability estimate, and

SE = the conditional standard error of the ability estimate.

The following table contains information about the slopes and intercepts used to generate the 2011 scale scores. The slopes and intercepts were obtained during the 2006 recalibration. These same slopes and intercepts have been used since the 2006 assessment.

Table 1.80. The 2011 MSA-Mathematic Slope and Intercept: Grades 3 through 8

Grade Slope Intercept

3 32.6935 352.2959

4 32.8398 380.2954

5 30.7057 390.2866

6 29.6236 398.5595

7 28.1690 405.9549

8 28.3634 418.4843


132

1.10 Score Interpretation

To help provide appropriate interpretation of the 2011 MSA-Math test scores, two types of scores were created: 240-650 scale scores, and performance levels and descriptions.

240-650 Scale Scores

As explained in section 1.9, Linking, Equating, and Scaling Procedures, the 2011 scale scores were placed on a common scale (i.e., 2006 assessment) within the same grade and ranged from 240 to 260. As a result, these scale scores have the same meaning and are comparable across different years’ assessments. However, it should be noted that they are not comparable across grade levels.

For scale scores, a higher score simply means a higher performance on math tests. Thus, performance levels and descriptions can give a specific interpretation other than a simple interpretation because they were developed to bring meaning to those scale scores.

Performance Level Descriptors

As previously explained, performance level descriptors provide specific information about students’ performance levels and help interpret the 2011 MSA-Math scale scores. They describe what students at a particular level generally know and can be applicable to all students within each grade level.

Maryland standards are divided into three levels of achievement (www.marylandpublicshools.org):

Advanced is a highly challenging and exemplary level of achievement indicating outstanding accomplishment in meeting the needs of students.

Proficient is a realistic and rigorous level of achievement indicating proficiency in meeting the needs of students.

Basic is a level of achievement indicating that more work is needed to attain proficiency in meeting the needs of students.

Table 2.1 shows a range of scale scores at each performance level; for example, grade 4 math scale scores from 374 to 432 indicate the level of Proficient. Students in this level passed the MSA-Math standard. This level is considered a realistic and rigorous level of achievement. Further information about the 2011 MSA-Math score interpretation can be obtained from MSDE.


133

1.11 Test Validity of the 2011 MSA-Math

As noted in the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999), “validity is the most important consideration in test evaluation.”

Messick (1989) defined validity as follows:

Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment. (p.5)

This definition implies that test validation is the process of accumulating evidence to support intended use of test scores. Consequently, test validation is a series of ongoing and independent processes that are essential investigations of the appropriate use or interpretation of test scores from a particular measurement procedure (Suen, 1990).

In addition, test validation embraces all of the experimental, statistical, and philosophical means by which hypotheses and scientific theories can be evaluated. This is the reason that validity is now recognized as a unitary concept (Messick, 1989).

To investigate the validity evidence of the 2011 MSA-Math, content-related evidence, item development procedures, differential item functioning (DIF) analysis on gender and ethnicity, and evidence from internal structure were collected.

Content-Related Evidence

Content validity is frequently defined in terms of the sampling adequacy of test items. That is, content validity is the extent to which the items in a test adequately represent the domain of items or the construct of interest (Suen, 1990). Consequently, content validity provides judgmental evidence in support of the domain relevance and representativeness of the content in the test (Messick, 1989).

The 2011 MSA-Math blueprints provide extensive evidence regarding the alignment between the content in the 2011 MSA-Math and the SC. It should be noted that the 2011 MSA-Math operational test forms were built exclusively using a Maryland item bank program which contained both content and statistical information about both operational and field-tested items. Information on the item composition of the operational test forms can be obtained from section 1.4, Test Form Design, Specifications, Item Type, and Item Roles. In addition, the 2011 MSA-Math blueprints are presented in Appendix D.

Item Development

Test development for MSA-Math is ongoing and continuous. Content specialists, teachers from across Maryland, Pearson, and MSDE were greatly involved in developing and reviewing items. Committees such as content review, bias review, and vision review reviewed all of the items, which were finally stored in a Maryland item bank. Specifically, an internal review by MSDE and Pearson staff for content alignment and quality required a great deal of time and energy. More specific information on item (test) development and review can be obtained in section 1.3, Development and Review of the 2011 MSA-Math Items and Test.


134

Field test items were embedded and administered in one of ten test forms. Once these items were scored, MSDE and Pearson conducted additional item analysis and content review. Any field test items that exhibited statistical results that suggested potential problems were carefully reviewed by both MSDE and Pearson content specialists. A determination was then made as to whether an item should be eliminated, revised, or field-tested again. Information on statistical analyses for field test items can be obtained in section 1.13, Field Test Analyses and Item Bank Construction.

Differential Item Functioning (DIF)

1) Bias Review of Items

A separate Bias Review Committee examined each math item, with looking for indications of bias that could impact the performance of an identifiable group of students. They discussed or rejected items biased on gender, ethnic, religious, or geographical bias.

2) DIF Statistics

For DIF analyses, subgroups were first identified according to either reference or focal groups. For the 2011 MSA-Math, males and whites were assigned to the reference group and females and African-Americans were assigned to the focal group.

While the Mantel-Haenszel procedure was used for SR and SPR items, the standardized mean difference (SMD) and the standard deviation (SD), along with the Mantel statistic, were calculated for BCR and ECR items. All of the items were classified based on Educational Testing Service (ETS) guidelines. All DIF results were kept in the 2011 Maryland item bank. More information on DIF analyses can be obtained in section 3.7, Differential Item Functioning.

Evidence from Internal Structure

The 2011 MSA-Math has five reporting math standards: Algebra, Geometry and Measurement, Statistics and Probability, Numbers and Computations, and Process. Tables 4.3 through 4.8 show the correlations among the math standards.


135

1.12 Unidimensionality Analyses of the 2011 MSA-Math

Measurement implies order and magnitude along a single dimension (Andrich, 1989). Consequently, in the case of scholastic achievement, one-dimensional scale is required to reflect this idea of measurement (Andrich, 1988, 1989). However, unidimensionality cannot be strictly met in a real testing situation because students’ cognitive, personality, and test-taking factors usually have a unique influence on their test performance to some level (Andrich, 1988; Hambleton, Swaminathan, & Rogers, 1991). Consequently, what is required for unidimensionality to be met is an investigation of the presence of a dominant factor that influences test performance. This dominant factor is considered as the ability measured by the test (Andrich, 1988; Hambleton et al., 1991; Ryan, 1983).

To check the unidimensionality of the 2011 MSA-Math, we examined the relative sizes of the eigenvalues associated with a principal component analysis of the item set. First, polychoric correlation coefficients were computed with LISREL 8.5 (Jöreskog & Sörbom, 1993) because they were polytomously scored on math items. Principal component analysis was then applied to produce eigenvalues. The first and the second principal component eigenvalues were compared without rotation. Table 1.81 summarizes the results of the first and second principal component eigenvalues of the 2011 MSA-Math.

A general rule of thumb in exploratory factor analysis suggests that a set of items may represent as many factors as there are eigenvalues greater than 1 in this analysis because there is one unit of information per item and the eigenvalues sum to the total number of items. However, a set of items may have multiple eigenvalues greater than 1 and still be sufficiently unidimensional for analysis with IRT (Loehlin, 1987; Orlando, 2004). As seen from the following table, the first component extracted substantially larger eigenvalues across all grades: the size of the eigenvalue of the first component was over ten times greater than the second eigenvalue for each form at each grade. As a result, we could conclude that the assumption of unidimensionality for the 2011 MSA-Math was met.


136

Table 1.81. The 2011 MSA-Math Eigenvalues between the First and Second Components

Grade Form Number of

Items First

Eigenvalue Second

Eigenvalue

3 A 65 22.90 1.97

F 65 22.87 2.05

4 A 64 23.07 2.00

F 64 22.66 1.94

5 A 65 21.74 1.98

F 65 20.30 2.07

6 A 62 21.93 1.89

F 62 21.64 1.71

7 A 62 25.53 2.32

F 62 24.89 2.07

8 A 62 26.30 1.84

F 62 24.61 1.87

Note. Analysis was conducted with a statewide population.


137

1.13 Field Test Analyses and Item Bank Construction

All field test items embedded in operational forms were subjected to rigorous statistical analyses for their properties in order to provide information about which items may be included as operational items in the future. All statistical results concerning field test items were preserved in the 2011 Maryland item bank. The following field test analyses were conducted:

Classical item analyses for SR, SPR, BCR, and ECR items

Differential item functioning (DIF) analyses

IRT analyses

Classical Item Analyses for SR, SPR, BCR, and ECR items

Classical item analyses for SR, SPR, BCR, and ECR items were conducted within each field test form.

SR items were flagged for further scrutiny if:

An item distractor was not selected by any students (i.e., nonfunctional distractor)

An item was selected by a high proportion of high-ability students while being selected by a low proportion of low-ability students (i.e., ambiguous distractor)

An item p-value was less than .20 or greater than .90.

An item point-biserial was less than .10 (i.e., poorly discriminating). If an item point-biserial was close to zero or negative, the item was checked for a miskeyed answer.

SPR items were flagged for further scrutiny if:

An item p-value was less than .20 or greater than .90.

An item point-biserial was less than .10 (i.e., poorly discriminating). If an item point-biserial was close to zero or negative, the item was checked for a miskeyed answer.

BCR and ECR items were flagged for further scrutiny if:

An item did not elicit the full range of rubric scores.

The ratio of mean item score to maximum score (i.e.., adjusted p-value) was less than .20 or greater than .90.

An item-total correlation was less than .10.

All items required a careful decision. For example, an item that was flagged as being difficult (p-value less than .20) and poorly discriminating (point-biserial less than .10) was considered for being dropped as a possible operational item. However, if the item represented important content that had not been extensively taught, a justification could have been made for including it in an operational test form.

Differential Item Functioning Analyses

Analyses of Differential item functioning (DIF) are intended to compare the performance of different subgroups of the population on specific items, when the group have been statistically matched on their tested proficiency.


138

In present analyses, the gender reference group was males, and the ethnic reference group was Caucasians. The gender focal group was females and the ethic focal group was African-Americans. For each operational form, the student’s total score was used as the matching variable.

Any SR, SPR, BCR, and ECR items that were flagged as showing DIF were subjected to further examination. For each of these items, for example, math experts judged whether the differential difficulty of the item was unfairly related to group membership using the following criteria:

If the differential difficulty of the item is related to group membership, and the difference is deemed unfair, then the item should not be used at all.

If the differential difficulty of the item is related to group membership, but the difference is not deemed unfair, then the item should only be used if there is no other item matching the test blueprint.

It should be noted that DIF analysis results for all the field test items were archived in the 2011 Maryland item bank. Detailed information about the DIF procedures can be found in section 3.7, Differential Item Functioning.

Item Response Theory (IRT) Analyses

To put the 2011 field test items on a common scale (i.e., the 2006 scale), each field test item was freely calibrated after fixing Rasch item and step difficulty parameters of the 2011 operational items that had been already placed on the base scale during the 2011 operational calibration and equating. For example, each unique field test item appearing on one of five math test forms (i.e., A, B, C, D, and E) was independently calibrated after fixing the same operational items appearing across the field test forms with the same Rasch item and step difficulties because these unique field test forms all correspond to the same operational form (i.e., operational form A).

It should be noted that all the Rasch item difficulties, step difficulties, and fit statistics (i.e., Rasch Infit and Outfit indices) of the field test items were archived in the 2011 Maryland item bank. These field test items are eligible to be used as operational items in subsequent years.

Item Bank Construction

The number of test forms constructed each year and the need to replace items that are released to the public necessitates the availability of a large pool of items. The 2011 MSA-Math item bank continues to be maintained by Pearson in the form of computer files and paper copies. This enables the test items to be readily available to both Pearson and MSDE staff for reference, test construction, test book design, and printing.


139

1.14 Quality Control Procedures

A standard quality procedure at Pearson was to create a test deck for MSA programs. The test deck began when Quality Assurance entered mock data into the enrollment system, which was transferred to the materials requisition system; the order was packaged by our Distribution Center, and shipped to the Quality Assurance department. We then reviewed the packing list against the data entered, the materials algorithms applied, the materials packaged against the packing list, and the actual packaging of the documents. These documents were then used to create a test deck of mock data, along with advance copies of documents that were received from the printer. Advance printer copies were inclusive of documents throughout the print run to assure we were randomly testing printed documents. The Maryland test deck was a comprehensive set of all documents that:

Verified all scan positions for item responses and demographics to verify scanning setup and scan densities

Verified all constructed response score points, zoning of image, reader scoring, reader resolution, and reader check scores

Verified the handling of blank documents through the system

Test all demographic and item edits

Verified pre-id barcode read, match and no-match

Verified attemptedness rules applied by subtest

Verified duplicate student handling (same test duplicate, different test duplicate)

Verified duplicate student with different demographics rules applied

Verified the document counts to the enrollment, pre-id and actual document receipt

Verified pre-id matching and application to student record

Verified various raw score points and access to dummy and live scoring tables

Verified cut scores applied

Verified valid score on one subtest and invalid score on other subtest

Verified scoring applied to Braille and Large Print

Verified valid multiple choice and invalid constructed response

Verified valid constructed response and invalid multiple choice

Verified all special scoring rules

Verified all summary programs for rounding

Verified summary inclusion and exclusion (Braille, standard and non-standard student summarization)

Verified each scoring level for group reporting

Verified all reporting programs for accuracy in all text and data presented

Verified class, school, district, and state summary data on home reports

Verified all data file programs to assure valid information in every field


140

Verified data descriptions for accuracy against data file

Created compare programs to allow for update of files

The Maryland test deck was the first order processed through the Maryland system to verify all aspects of the materials packaging, scanning, editing, scoring, summary, and reporting. Pre-determined conditions were included in the test deck to assure the programs were processing all data to meet the requirements of the program with zero defects. Processing of live orders could not proceed until each phase of the test deck had been approved by our Quality Assurance department. An Issues Log with sign-off approvals was utilized to assure we were addressing any issues that arose in the review of the test deck data across all functional groups at Pearson.

Prior to release of any order for reporting we received a preliminary file from Scoring Operations to run a key check TRIAN to assure that all scoring keys had been determined and applied accurately. Any item that was not performing as expected was flagged and reviewed by our content specialist and psychometrician. Upon completion of the key check, we proceeded to run the pilot level reports.

We ran the pilot district utilizing live data. The pilot district included multiple buildings, all grades, and any unique accommodations. A formal pilot review process was conducted with Pearson staff experts prior to release of the information to MSDE.

Upon completion of the processing of all district-level data, Pearson Scoring Operations provided the Quality Assurance Department with one or more state-level data files, along with state data for review and approval. Pearson Quality Assurance programmers duplicated all data independently to ensure accurate interpretation of the expected results. A series of SAS programs were run on these files to ensure 100% accuracy. These included but were not limited to:

Statewide Duplicate Student

Statewide FD of Demographic Variables

District/Building/N-Count

Statewide RS/SS/Cut Score tables

Proc Means to verify summary statistics

Item Response listing to verify all constructed responses were scored and within the valid range

Normative data check for all raw scores

Reader Resolution report to verify all readings and resolution combinations

Upon complete review and approval by Quality Assurance, we posted the statewide student files to a secure FTP site for review by MSDE.


141

2. CURRENT PERFORMANCE RESULTS OF THE 2011 MSA-MATH

This section provides information about academic achievement results of Maryland students in grades 3 through 8. Table 2.1 contains information about the cutoff score of each performance level, and Table 2.2 contains information about the pass rate of each grade. The same cutoff scores have been applied since 2003 (for grades 3, 5, and 8) and 2004 (for grades 4, 6, and 7).

We also analyzed the performance rate of each grade based on key student subgroups such as gender, ethnicity, and LEA. Tables 2.3 though 2.10 contain information about the pass rate of each subgroup.


142

Table 2.1. MSA-Math Cut Scores: Grades 3 through 8

Grade Cut Score of Performance Level

Proficient Advanced

3 379 441

4 374 433

5 392 453

6 396 447

7 396 451

8 407 444

Note. Performance level cuts have been applied since 2003 (grades 3, 5, and 8) and 2004 (grades 4, 6, and 7).

Table 2.2. The 2011 MSA-Math Pass Rates: Grade 3 through Grade 8

Grade N Basic Proficient Advanced Proficient +Advanced

3 61,293 13.15 51.22 35.63 86.85

4 59,064 8.65 40.83 50.52 91.35

5 59,647 16.92 59.95 23.13 83.08

6 58,309 17.92 49.42 32.66 82.08

7 59,280 24.83 49.20 25.97 75.17

8 59,203 32.65 34.10 33.26 67.35

Note. Percentages may not add up to 100% due to rounding. Note. Analysis was conducted with a statewide population.


143

Table 2.3. The 2011 MSA-Math Pass Rates by Gender: Grade 3 through Grade 8

Grade Gender N Basic Proficient Advanced Proficient + Advanced

3 Male 31,246 13.91 51.68 34.41 86.09

Female 30,045 12.35 50.73 36.91 87.65

4 Male 29,883 9.40 41.30 49.30 90.60

Female 29,180 7.89 40.34 51.77 92.11

5 Male 30,343 18.00 58.96 23.04 82.00

Female 29,301 15.80 60.98 23.22 84.20

6 Male 29,746 19.72 47.33 32.95 80.28

Female 28,555 16.03 51.59 32.38 83.97

7 Male 30,053 26.56 48.12 25.32 73.44

Female 29,221 23.05 50.31 26.64 76.95

8 Male 30,047 33.59 33.58 32.83 66.41

Female 29,151 31.67 34.63 33.70 68.33



144

Table 2.4. The 2011 MSA-Math Pass Rates by Ethnicity: Grade 3 through Grade 8

Grade Ethnicity N Basic Proficient Advanced Proficient + Advanced

3

American Indian 171 17.54 57.89 24.56 82.46

Asian/Pacific Islander 3,799 4.47 35.51 60.02 95.53

African American 20,631 22.88 58.64 18.49 77.12

Native Hawaiian 63 12.70 53.97 33.33 87.30

White 26,349 6.19 45.27 48.53 93.81

Hispanic 7,707 16.02 60.17 23.81 83.98

Other 2,573 10.26 48.50 41.24 89.74

4

American Indian 176 8.52 49.43 42.05 91.48 Asian/Pacific Islander 3,573 2.71 22.22 75.06 97.29


Native Hawaiian 60 10.00 41.67 48.33 90.00

White 25,515 3.44 32.33 64.22 96.56

Hispanic 6,956 11.40 49.83 38.77 88.60

Other 2,369 5.28 36.85 57.87 94.72

5

American Indian 194 20.10 67.53 12.37 79.90



Native Hawaiian 67 22.39 56.72 20.90 77.61

White 26,158 8.64 58.88 32.48 91.36

Hispanic 6,705 21.98 64.18 13.84 78.02

Other 2,275 12.70 59.69 27.60 87.30

6



Native Hawaiian 47 19.15 51.06 29.79 80.85

White 25,739 8.90 45.67 45.42 91.10

Hispanic 6,281 23.98 55.21 20.81 76.02

Other 2,079 13.23 48.05 38.72 86.77

7



Native Hawaiian 64 25.00 57.81 17.19 75.00

White 26,383 13.17 49.83 37.00 86.83

Hispanic 6,145 32.25 53.54 14.21 67.75

Other 2,028 17.80 50.94 31.26 82.20

8

American Indian 162 32.72 46.30 20.99 67.28



Native Hawaiian 75 40.00 42.67 17.33 60.00

White 26,565 18.51 34.87 46.62 81.49

Hispanic 6,177 42.69 36.86 20.45 57.31

Other 1,916 24.16 36.27 39.56 75.84



145

Table 2.5. The 2011 MSA-Math Pass Rates by LEA: Grade 3

LEA N Basic Proficient Advanced Proficient +Advanced

01 635 11.18 47.09 41.73 88.82

02 5,629 7.37 43.45 49.17 92.63

03 7,553 10.05 50.89 39.06 89.95

04 1,191 6.30 47.27 46.43 93.70

05 403 14.64 50.12 35.24 85.36

06 2,014 7.60 48.86 43.55 92.40

07 1,175 14.55 59.49 25.96 85.45

08 1,773 17.54 52.12 30.34 82.46

09 300 22.33 52.00 25.67 77.67

10 2,865 8.76 56.44 34.80 91.24

11 274 13.14 61.31 25.55 86.86

12 2,860 11.08 55.94 32.97 88.92

13 3,644 6.01 44.40 49.59 93.99

14 138 24.64 57.97 17.39 75.36

15 10,393 10.58 47.55 41.86 89.42

16 8,794 21.32 59.02 19.66 78.68

17 599 7.18 46.24 46.58 92.82

18 1,282 9.20 48.67 42.12 90.80

19 214 15.42 65.42 19.16 84.58

20 355 14.08 56.34 29.58 85.92

21 1,667 13.08 45.47 41.45 86.92

22 1,128 9.66 54.08 36.26 90.34

23 434 0.69 24.42 74.88 99.31

24 94 76.60 21.28 2.13 23.40

30 5,879 25.51 56.57 17.91 74.49



146



01 623 4.98 35.79 59.23 95.02

02 5,382 3.88 30.38 65.74 96.12

03 7,272 5.47 39.47 55.06 94.53

04 1,198 4.67 29.38 65.94 95.33

05 412 6.31 41.02 52.67 93.69

06 1,953 4.25 38.50 57.25 95.75

07 1,101 10.81 50.86 38.33 89.19

08 1,749 9.03 42.42 48.54 90.97

09 343 13.12 44.90 41.98 86.88

10 2,818 4.58 43.29 52.13 95.42

11 277 6.50 45.49 48.01 93.50

12 2,626 6.13 39.64 54.23 93.87

13 3,591 4.07 33.28 62.66 95.93

14 131 12.98 48.85 38.17 87.02

15 10,026 8.04 39.18 52.78 91.96

16 8,340 14.69 50.02 35.29 85.31

17 514 4.09 33.07 62.84 95.91

18 1,212 5.86 29.87 64.27 94.14

19 206 9.71 45.63 44.66 90.29

20 297 12.12 42.09 45.79 87.88

21 1,595 7.21 37.37 55.42 92.79

22 1,139 6.32 40.83 52.85 93.68

23 499 2.40 27.86 69.74 97.60

24 94 70.21 29.79 0.00 29.79

30 5,666 18.88 51.78 29.33 81.12



147



01 649 14.48 62.56 22.96 85.52

02 5,539 10.71 57.57 31.72 89.29

03 7,340 14.21 61.05 24.74 85.79

04 1,210 9.01 58.93 32.07 90.99

05 395 14.68 63.80 21.52 85.32

06 1,994 8.88 67.05 24.07 91.12

07 1,138 18.10 68.10 13.80 81.90

08 1,862 19.55 65.41 15.04 80.45

09 311 23.47 63.34 13.18 76.53

10 2,850 13.65 63.02 23.33 86.35

11 309 13.27 70.23 16.50 86.73

12 2,831 12.47 65.74 21.79 87.53

13 3,635 8.80 52.35 38.84 91.20

14 158 11.39 68.99 19.62 88.61

15 10,122 13.24 53.27 33.49 86.76

16 8,417 26.49 63.63 9.87 73.51

17 543 6.81 65.38 27.81 93.19

18 1,199 14.51 55.30 30.19 85.49

19 197 9.14 75.13 15.74 90.86

20 312 17.31 67.95 14.74 82.69

21 1,666 14.95 64.11 20.95 85.05

22 985 17.26 64.97 17.77 82.74

23 422 9.24 59.72 31.04 90.76

24 109 86.24 13.76 0.00 13.76

30 5,454 33.90 58.76 7.33 66.10



148



01 601 15.14 53.08 31.78 84.86

02 5,423 13.33 44.99 41.67 86.67

03 7,173 19.20 52.80 28.01 80.80

04 1,216 8.39 48.93 42.68 91.61

05 351 15.95 53.85 30.20 84.05

06 1,937 6.87 47.13 46.00 93.13

07 1,135 19.47 52.42 28.11 80.53

08 1,808 15.65 51.88 32.47 84.35

09 310 24.84 48.39 26.77 75.16

10 2,770 9.35 52.13 38.52 90.65

11 307 16.61 53.42 29.97 83.39

12 2,710 13.10 51.55 35.35 86.90

13 3,547 7.98 42.29 49.73 92.02

14 145 17.93 60.00 22.07 82.07

15 9,768 15.06 46.15 38.79 84.94

16 8,169 26.66 53.41 19.93 73.34

17 544 5.51 50.37 44.12 94.49

18 1,216 12.01 43.67 44.33 87.99

19 198 10.61 54.55 34.85 89.39

20 292 15.41 51.37 33.22 84.59

21 1,670 13.11 51.14 35.75 86.89

22 988 16.80 54.45 28.74 83.20

23 466 6.65 44.21 49.14 93.35

24 169 79.29 18.34 2.37 20.71

30 5,315 36.31 50.84 12.85 63.69

32 81 51.85 38.27 9.88 48.15



149



01 640 21.41 57.34 21.25 78.59

02 5,444 20.52 49.52 29.96 79.48

03 7,192 26.07 50.83 23.10 73.93

04 1,266 14.61 53.95 31.44 85.39

05 351 14.53 48.15 37.32 85.47

06 2,103 18.64 49.93 31.43 81.36

07 1,196 22.66 54.52 22.83 77.34

08 2,023 19.33 54.57 26.10 80.67

09 291 28.87 47.42 23.71 71.13

10 2,916 13.17 54.29 32.54 86.83

11 308 10.39 64.29 25.32 89.61

12 2,814 20.15 53.16 26.69 79.85

13 3,841 11.14 48.37 40.48 88.86

14 132 26.52 49.24 24.24 73.48

15 9,965 18.70 46.19 35.11 81.30

16 8,274 39.70 47.66 12.64 60.30

17 508 19.49 55.91 24.61 80.51

18 1,197 15.12 51.55 33.33 84.88

19 171 16.37 60.82 22.81 83.63

20 302 25.83 52.65 21.52 74.17

21 1,630 17.36 51.23 31.41 82.64

22 942 24.73 52.65 22.61 75.27

23 468 7.48 43.38 49.15 92.52

24 168 82.14 16.67 1.19 17.86

30 5,060 49.92 42.06 8.02 50.08

32 78 29.49 65.38 5.13 70.51



150



01 660 33.33 41.06 25.61 66.67

02 5,361 27.44 35.24 37.33 72.56

03 7,047 32.92 36.43 30.65 67.08

04 1,288 17.31 43.25 39.44 82.69

05 394 17.01 46.70 36.29 82.99

06 2,060 23.50 35.19 41.31 76.50

07 1,144 33.30 37.76 28.93 66.70

08 2,128 25.00 35.48 39.52 75.00

09 353 38.24 39.66 22.10 61.76

10 2,902 21.19 37.59 41.21 78.81

11 286 19.58 45.80 34.62 80.42

12 2,818 25.55 38.72 35.73 74.45

13 3,762 14.73 31.58 53.69 85.27

14 164 31.71 36.59 31.71 68.29

15 10,087 24.19 32.06 43.75 75.81

16 8,485 55.12 31.15 13.73 44.88

17 568 16.90 42.96 40.14 83.10

18 1,251 19.98 39.09 40.93 80.02

19 186 34.41 48.92 16.67 65.59

20 290 26.90 41.03 32.07 73.10

21 1,677 22.30 33.69 44.01 77.70

22 973 38.23 34.64 27.13 61.77

23 476 8.19 27.31 64.50 91.81

24 207 90.34 9.66 0.00 9.66

30 4,573 63.11 26.63 10.26 36.89

32 63 55.56 25.40 19.05 44.44



151

3. OVERVIEW OF STATISTICAL SUMMARIES

This section provides general information about statistical and psychometric summaries used for the 2011 MSA-Math program. Actual statistical results described in this section appear in section 4 and appendices.

3.1 Classical Descriptive Statistics

Table 4.1 contains the classical descriptive statistics of each form for each grade and includes:

Form number

Number of items

Numbers of students (These numbers were based on a whole population.)

Means and standard deviations of raw scores

Stratified Cronbach’s Alpha

Standard error of measurement (SEM)

Stratified Cronbach’s Alpha

The 2011 MSA-Math tests included SR, SPR, BCR, and ECR items. Consequently, it was necessary to use an adequate reliability coefficient that addressed the important factor, different item type. The following formula depicts the reliability coefficient, Stratified Cronbach’s Alpha:

Stratified 2

22 ))1(()1(((1

t

CRCRSRSR

σ

ρσρσα

where

2

SR = variance of score on SR and SPR items

2

CR = variance of score on BCR and ECR items

2

t = variance of total score

SR

= reliability coefficient of score on SR and SPR items, and

CR

= reliability coefficient of score on BCR and ECR items.


152

Standard Error of Measurement (Based on Classical Test Theory)

The standard error of measurement (SEM) is commonly used in interpreting and reporting individual test scores and score differences on tests (Harvill, 1991).

Classical test theory is based on the following assumptions (Andrich & Luo, 2004):

Each person v has a true score on the construct, usually denoted by the variable Tv

The best overall indicator of the person’s true score is the sum of the scores on the items and is usually denoted by the variable Xv

This observed score will have an error for each person which is usually denoted by Ev

These errors are not correlated with the true score

Across a population of people, the errors sum to 0 and they are normally distributed.

From these assumptions, the following equations can be derived:

vvv ETX .

Therefore,

2x = 2

t + 2e

where 2x = the variance of the observed score in a population of persons,

2t = the variance of their true score variance, and

2e = the error variance.

The reliability coefficient of the test can be calculated by the following formula:

x

= 2

2

x

t

= 2

22

x

ex

.

Thus, the SEM is calculated by the following formula:

e = x x1 .

For example, consider a student with a score of 90 from a sample of students with a mean score of 60 and variance of 225 on a test with reliability of 0.80. According to the formulas provided above, the obtained score is 90, and its SEM is 6.71. Thus, an approximate 68% score band for estimating this student’s true score is from 83.29 (90 - 6.71) to 96.71 (90 + 6.71).

Note that this equation is only useful to estimate true score when the test reliability is reasonably high and the obtained score for the examinee is not an extreme deviate from the mean of the appropriate reference group. When we use this equation, consequently, we should be careful with statements so that they do not imply greater precision than is actually involved (Harvill, 1991).


153

Conditional Standard Error of Ability Estimate (Based on the Rasch Model)

Under the Rasch (i.e., 1-PL IRT) model, the SE for each person is as follows (Andrich & Luo, 2004):

ˆ =

L

ivivi pp

1)1(

1

where

v = subscript for a person,

i = subscript for an item,

L = length of the test,

= ability estimate, and

vip = the probability that a person answers an item correctly and defined as follows:

vip = iv

iv

e

e

1 where v is person’s ability and i is item’s difficulty.

A confidence band can be found for use in interpreting the ability estimate. For example, an approximate 68% confidence interval for is given by

± SE

3.2 Scale Score Descriptive Statistics

Table 4.2 provides information about scale score descriptive statistics of each form for each grade and includes:

Form number

Numbers of students

Mean and standard deviation of scale scores

10% quantile (P10), 25% quantile (Q1), median (P50), 75% quantile (Q3), 90% quantile, and IQR (Interquantile Range= Q3-Q1)

Conditional standard errors (SE) at the cut scores (i.e., B/P and P/A)

In addition, Appendix B provides frequency distributions and histograms of the scale scores of the 2011 MSA-Math as well as the 2006 MSA-Math (i.e., base year assessment).

3.3 Classical and Rasch (1-Parameter Logistic IRT) Item Parameters

Appendix C provides both classical and Rasch item parameters and includes:

Item type (SR, SPR, BCR, or ECR)


154

P-value: in order for p-values of BCR and ECR items to be comparable with p-values of the SR and SPR items they were calculated as modified proportions of the maximum obtainable domain scores.

Point-biserial correlation: a Pearson’s r between the scored item and the total score

Rasch item difficulty estimate ( iD )

Conditional standard error of Rasch item difficulty estimate

Rasch step difficulty estimate (or structure calibration estimate, ijF )

Mean-square infit

Mean-square outfit

First of all, it should be noted that all the Rasch item and step difficulty parameters were placed on a common scale (i.e., the 2006 assessment since the 2006 was recalibrated using the Rasch model).

Second, the following formula shows how structure measure estimate ( ijD ) is calculated from

both iD and ijF directly obtained from a run of Winsteps:

ijD = iD + ijF ,

where ijD = structure measure estimate

iD = item difficulty estimate,

ijF = structure calibration estimate (i.e., step difficulty estimate).

Finally, the following formulas show how conditional standard error (SE) of item difficulty

estimate ( iD ) and structure measure estimate ( ijF ) were driven (Wright & Masters, 1982):

SE( iD ) =

N

n

m

knik

m

knik

ii

kppk1

22 ])([/1

SE( ijF ) =

N

n

m

jknik

j

knik

i

pp1 1

2

0

))((/1

where

x

j

m

k

k

jijnijnnix

i

DDP0 0 0

exp/exp

x = 0, 1, ..., mi, and

k = 1, 2,…., mi.


155

Fit Statistics for the Rasch Model

Fit statistics are used for evaluating the goodness-of-fit of a model to the data. Fit statistics are calculated by comparing the observed and expected trace lines obtained for an item after parameter estimates are obtained using a particular model. Winsteps provides two kinds of fit statistics called mean-squares that show the size of the randomness or amount of distortion of the measurement system.

Outfit mean-squares are influenced by outliers and are usually easy to diagnose and remedy. Infit mean-squares, on the other hand, are influenced by response patterns and are harder to diagnose and remedy. Table 3.1 provides a guideline for evaluating mean-square fit statistics (Linacre & Wright, 2000).

In general, mean-squares near 1.0 indicate little distortion of the measurement system, while values less than 1.0 indicate observations are too predictable (redundancy, model overfit). Values greater than 1.0 indicate unpredictability (unmodeled noise, model underfit).

Table 3.1. Criteria to Evaluate Mean-Square Fit Statistics

Mean-Square Interpretation

> 2.0 Distorts or degrades the measurement system

1.5 – 2.0 Unproductive for construction of measurement, but not degraded

0.5 – 1.5 Productive for measurement

< 0.5 Unproductive for measurement, but not degrading. May produce misleadingly good reliabilities and separations

3.4 Inter-Rater Reliability

Tables 4.15 through 4.20 contain information about the scoring agreement between two ratings received for each item. When the two readers assigned the same score to a student’s answer, the scores were in perfect agreement. Scores differed by one score point were adjacent, and scores differing by two or more score points were in discrepancy. For further information about inter-rater agreement, please see section 1.7, Scoring Procedures of the 2011 MSA-Math. While the perfect agreement rates of Step A were above 95%, those of Step B were above 78% for all items. It should also be noted that the agreement rates including both perfect and adjacent rates were above 98% for Step B for all items across all grades.

3.5 Correlations among Mathematics Standards

The 2011 MSA-Math consisted of five subscore reporting standards: Algebra, Geometry and Measurement, Statistics and Probability, Numbers and Computations, and Process. Tables 4.3 through 4.8 contain correlation coefficients among these math standards.


156

3.6 Decision Accuracy and Consistency at the Cut Scores

Tables 4.9 through 4.14 contain the results of analyses performed to estimate the accuracy and consistency of the decisions for passing (proficient) on the 2011 MSA-Math. The analyses make use of the methods outlined and implemented in Livingston and Lewis (1995), Haertel (1996), and Young and Yoon (1998).

The accuracy of a decision is the extent to which it would agree with the decisions that would be made if each student could somehow be tested with all possible parallel forms of the assessments. The consistency of a decision is the extent to which it would agree with the decisions that would be made if the students had taken a different form of the examination, equal in difficulty and covering the same content as the form they actually took.

Students can be misclassified in one of two ways. Students who were below the proficiency cut score, but were classified (on the basis of the assessment) as being above a cut score, are considered to be false positives. Students who were above the proficiency cut score, but were classified as being below a cut score, are considered to be false negatives.

For the 2011 MSA-Math, Tables 4.9 through 4.14 include:

Performance level

Accuracy classifications

False positives

False negatives

Consistency classifications

The tables illustrate the general rule that decision consistency was less than decision accuracy.

3.7 Differential Item Functioning

This section provides information about differential item functioning (DIF) analyses used for the 2011 MSA-Math. While the reference group was either male or Caucasian students, the focal group was either female or African-American students. It should be noted that DIF analyses on the 2011 operational items indicated that all the items were satisfactory. All the DIF results were archived in the 2011 Maryland item bank.

Since the 2011 MSA-Math was a mixed-format examination, comprised of SR, SPR, BCR, and ECR items, the DIF procedure used consists of the Mantel Chi-square (Mantel, 1963) for the BCR and ECR items and the Mantel-Haenszel procedure (Mantel & Haenszel, 1959) for the SR and SPR items.

Brief Constructed Response (BCR) and Extended Constructed Response (ECR) Items

To help interpret the Mantel Chi-square (Mantel 2 ), the Educational Testing Service (ETS) DIF procedure uses the Mantel statistic in conjunction with the standardized mean difference (SMD).


157

Mantel Statistic

The Mantel 2 is simply a conditional mean comparison of the ordered response categories for reference and focal groups combined over values of the matching variable score. By “ordered” we mean that a response of 1 on an item is higher than 0, a response of 2 is higher than 1, and so on. “Conditional,” on the other hand, refers to the comparison of members from the two groups who received the same score on the matching variable, i.e., the total test score in our analysis.

Table 3.2 shows a 2 T K contingency table, where T is the number of response categories and K is the number of levels of the matching variable. The values,

1y ,

2y , …,

Ty are the T

scores that can be gained on the item. The values, Ftkn and Rtkn , represent the numbers of focal

and reference groups who are at the thk level of the matching variable and gain an item score of

ty . The “+” indicates total number over a particular index (Zwick, Donoghue, & Grima, 1993).

Table 3.2. 2 T Contingency Table at the thk level

Group Item Score

Total

1y

2y

Ty

Reference kRn 1 kRn 2 …

RTkn kRn

Focal kFn 1 kFn 2 …

FTkn kFn

Total kn 1 kn 2 …

Tkn kn

Note. This table was cited from Zwick, et al. (1993)

The Mantel statistic is defined as the following formula:

Mantel 2 =

kk

kk

kk

FVar

FEF

)(

))(( 2

where

kF = the sum of scores for the focal group at the thk level of the matching variable and

is defined as follows:

kF = ny Ftkt

t ,

The expectation of kF under the null hypothesis is

kFE = n

n

k

kF

ny tk

tt .

And, the variance of kF under the null hypothesis is as follows:


158

kFVar =

t ttkttktk

kk

kFkR nynynnn

nn )()()1(

22

2.

Under H0, the Mantel statistic has a chi-square distribution with one degree of freedom. In DIF applications, rejecting H0 suggests that the students of the reference and focal groups who are similar in overall test performance tend to differ in their mean performance. In the case of dichotomous items, on the other hand, the statistic is identical to the Mantel-Haenszel (1959) statistic without the continuity correction (Zwick, Donoghue, & Grima, 1993).

Standardized Mean Difference (SMD)

A summary statistic to accompany the Mantel approach is the standardized mean difference (SMD) between the reference and focal groups proposed by Dorans and Schmitt (1991). This statistic compares the means of the reference and focal groups, adjusting for differences in the distribution of the reference and focal group members across the values of the matching variable.

SMD = k k

RkFkFkFk mpmp

where

n

npF

kFFk

, the proportion of the focal group members who are at the thk level of the

matching variable,

t

FtktkF

RK nyn

m )(1

, the mean item score of the focal group members at the thk

level, and

mRk = the analogous value for the reference group.

As can be seen from the equation above, the SMD is the difference between the unweighted item mean of the focal group and the weighted item mean of the reference group. The weights for the reference group are applied to make the weighted number of the reference group students the same as in the focal group within the same ability. A negative SMD value implies that the focal group has a lower mean item score than the reference group, conditional on the matching variable.


159

DIF classification for BCR and ECR items

The SMD is divided by the total group item standard deviation to obtain an effect-size value for the SMD. This effect-size SMD is then examined in conjunction with the Mantel 2 to obtain DIF classifications that are depicted in Table 3.3 below.

Table 3.3. DIF Classification for BCR and ECR Items

Category Description Criterion

AA No DIF Non-significant Mantel 2 or

Significant Mantel 2 and |SMD/SD| ≤ .17

BB Weak DIF Significant Mantel 2 and .17 < |SMD/SD| ≤ .25

CC Strong DIF Significant Mantel 2 and .25 < |SMD/SD|

Note. SD is the total group standard deviation of the item score in its original metric.

Selected Response (SR) and Student-Produced Response (SPR) Items

For the SR and SPR items, the Mantel-Haenszel Chi-square (M-H 2 ) is used in conjunction with the M-H odds ratio transferred to what ETS calls the delta scale (D).

The Odds Ratio

The odds of a correct response are P/Q or P/(1-P). The odds ratio, on the other hand, is simply the odds of a correct response of the reference group divided by the odds of a correct response of the focal group.

For a given item, the odds ratio is defined as follows:

HM = QfP

QP

f

rr

/

/.

The corresponding null hypothesis is that the odds of getting the item correct are equal for the two groups. Thus, the odds ratio is equal to 1:

H0: HM = QfP

QP

f

rr

/

/ = 1.


160

The Delta Scale

In order to make the odds ratio symmetrical around zero with its range being in the interval to , the odds ratio is transformed into a log odds ratio as per the following:

HM = )ln( H-M .

The simple natural logarithm transformation of this odds ratio is symmetrical about zero in which zero has the interpretation of equal odds. This DIF measure is a signed index where a positive value signifies DIF in favor of the reference group while a negative value indicates DIF

in favor of the focal group. HM also has the advantage of being transformed linearly to other

interval scale metrics (Camilli & Shepard, 1994). This fact is utilized by ETS in creating their delta scale (D), which is defined as follows:

D = .35.2 HM

DIF classification for SR and SPR items

The following table depicts DIF classifications for SR items to examine the M-H 2 in conjunction with the delta scale (D):

Table 3.4. DIF Classification for SR and SPR Items

Category Description Criterion

A No DIF Non-significant M-H 2 or |D| < 1.0

C Strong DIF Significant M-H 2 and |D| ≥ 1.5

B Weak DIF Otherwise classified as B


161

3.8 Equating and Scaling

Tables 4.21 through 4.44 contain the 2011 MSA-Math total raw score to scale score (RS/SS) conversion tables. Conditional standard errors for the total scale scores are also included. It should be noted that each student’s total raw score for the strands was a summation of the individual item score within a strand level. The strand levels were classified as stated in section 1.4 and the item parameters within each strand was obtained using the Winsteps program in the same manner as those obtained for the total test. Once the item parameters were available, thetas (student proficiency scores) were calculated for each raw score point that could be obtained within each strand. The thetas were transferred to scale scores, using the same slope and intercept as that which were applied for the total math test score.

The Rasch and Partial Credit Models

The most basic expression of the Rasch model is in the item characteristic curve (ICC). It shows the probability of a correct response to an item as a function of the proficiency level. The probability of a correct response is bounded by 1 (certainty of a correct response) and 0 (certainty of an incorrect response).

Figure 3.1. Item Characteristic Curve

0.00

0.25

0.50

0.75

1.00

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Ability

Pro

bab

ilit

y o

f a

Co

rrec

t R

esp

on

se Probability


162

As an example, consider Figure 3.1 which depicts an item that falls at approximately 0.85 on the proficiency (horizontal) scale. When a person answers an item at the same level as their proficiency, then that person has a probability of roughly 50% of answering the item correctly. Another way of expressing this is that if we have a group of 100 people, all of whom have an proficiency of 0.85, we would expect about 50% of them to answer the item correctly. A person whose proficiency was above 0.85 would a higher probability of getting the item right, while a person whose proficiency is below 0.85 would have a lower probability of getting the item right. This makes intuitive sense and is the basic formulation of Rasch measurement for test items having only 2 possible categories (i.e., wrong or right).

Figure 3.2. Category Response Curves for a One-Step Item

Figure 3.2 extends this formulation to show the probabilities of obtaining a wrong answer or a right answer. The curve on the left (j = 0) shows the probability of getting a score of “0” while the curve on the right (j = 1) shows the probability of getting a score of “1”. The point at which the two curves cross indicates the transition point on the proficiency scale where the most likely response changes from a “0” to a “1”. Here, the probability of answering the item correctly is 50%.

The key step in the formulation, and the point at which the Rasch dichotomous model merges with the PCM, requires us to assume an additional response category. Suppose that, rather than scoring items as completely wrong or completely right, we add a category representing answers that, though not totally correct, are still clearly not totally incorrect. These relationships are shown in Figure 3.3.

0.00

0.25

0.50

0.75

1.00

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Ability

Pro

bab

ility

of

Cat

ego

ry j

Res

po

nse

(j =

0, 1

) j = 0 j = 1


163

The left-most curve (j = 0) in Figure 3.3 represents the probability for all examinees getting a score of “0” (completely incorrect) on the item, given their proficiency. Those of very low proficiency (i.e., below – 2) are very likely to be in this category and, in fact, are more likely to be in this category than the other two. Those receiving a “1” (partial credit) tend to fall in the middle range of abilities (the middle curve, j = 1). The final, right-most curve (j = 2) represents the probability for those receiving scores of “2” (completely correct). Very high-proficiency people are clearly more likely to be in this category than in any other, but there are still some of average and low proficiency that can get full credit for the item.

Figure 3.3. Category Response Curves for a Two-Step Item

Although the actual computations are quite complex, the points at which lines cross each other have a similar interpretation as for the dichotomous case. Consider the point at which the j = 0 line crosses the j = 1 line, indicated by the left arrow. For abilities to the left of (or less than) this point, the probability is greatest for a “0” response. To the right of (or above) this point, and up to the point at which the j = 1 and j = 2 lines cross (marked by the right arrow), the most likely response is a “1”. For abilities to the right of this point, the most likely response is a “2”.

Note that the probability of scoring a “1” response (j = 1) declines in both directions as proficiency decreases to the low extreme or increases to the high extreme. These points then may be thought of as the difficulties of crossing the thresholds between categories.

0.00

0.25

0.50

0.75

1.00

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Ability

Prob

ability

of

Cate

gory

j R

espo

nse

(j =

0, 1,

2)

j = 0 j = 1 j = 2

Threshold betweenj = 0 and j = 1

Threshold betweenj = 1 and j = 2


164

An important implication of the formulation can be summarized as follows: If the commonly used Rasch model applied to dichotomously (right/wrong) scored items can be thought of as simply a special case of the PCM, then the act of scaling multiple-choice items together with polytomous items, whether they have three or more response categories, is a straightforward process of applying the measurement model. The quality of the scaling can then be assessed in terms of known procedures.

One important property of the PCM is its proficiency to separate the estimation of item/task parameters from the person parameters. With the PCM, as with the Rasch model, the total score given by the sum of the categories in which a person responds is a sufficient statistic for estimating person proficiency (i.e., no additional information need be estimated). The total number of responses across examinees in a particular category is a sufficient statistic for estimating the step difficulty for that category. Thus with PCM, the same total score will yield the same proficiency estimate for different examinees.

The PCM is a direct extension of the dichotomous one-parameter logistic IRT model developed by Rasch (Rasch, 1980). For an item/task involving mi score categories, one general expression for the probability of scoring x on item/task i is given by

x

j

m

k

k

jijnijnnix

i

DDP0 0 0

exp/exp x = 0, 1, ..., mi,

where

0

0

0j

ijD and

0

0

1expj

ijD .

The above equation gives the probability of scoring x on the i-th test item as a function of proficiency ( ) and the difficulty of the mi steps of the task (Masters, 1982).

According to this model, the probability of an examinee scoring in a particular category (step) is the sum of the logit (log-odds) differences between and Dij of all the completed steps, divided by the sum of the differences of all the steps of a task. Thissen and Steinberg (1986) refers to this model as a divide-by-total model. The parameters estimated by this model are (1) an proficiency estimate for each person (or proficiency estimate at each raw score level) and (2) mi threshold (difficulty) estimates for each task with mi + 1 score categories.


165

4. THE 2011 MSA-MATH STATISTICAL SUMMARY

This chapter summarizes statistical results of the 2011 MSA-Math. It includes descriptive statistics of the 2011 math test based on raw scores and scale scores, accuracy and consistency of the 2011 math test, rater agreement rates, correlation coefficients among reporting standards, and total and substrand RS/SS conversion tables.


166

Table 4.1. The 2011 MSA-Math Classical Descriptive Statistics: Grades 3 through 8

Grade Form Total No. of Items

Total Score Point N Mean SD Reliability SEM

3 A 65 72 31,082 53.57 11.55 0.93 3.05

F 65 72 30,211 53.98 11.16 0.93 3.00

4 A 64 71 30,037 49.48 12.63 0.94 3.22

F 64 71 29,027 50.94 12.28 0.93 3.17

5 A 65 74 30,354 48.33 13.90 0.93 3.55

F 65 74 29,293 50.10 13.23 0.93 3.57

6 A 62 70 29,778 44.00 14.42 0.94 3.51

F 62 70 28,531 45.21 14.35 0.94 3.53

7 A 62 72 30,180 38.50 15.50 0.95 3.43

F 62 72 29,100 39.55 15.44 0.95 3.50

8 A 62 73 30,089 36.85 16.87 0.96 3.58

F 62 73 29,114 37.52 16.44 0.95 3.72



167

Table 4.2. The 2011 MSA-Math Scale Score Descriptive Statistics: Grades 3 through 8

Grade Form N M SD P10 Q1 Mdn Q3 P90 IQR

SE at cut-points

Prof. Adv.

3

A 31,082 423.7 42.0 370 395 424 450 476 55 9 12

F 30,211 427.2 43.1 372 399 426 459 481 60 9 12

Overall 61,293 425.4 42.6 372 396 426 453 476 57 N/A N/A

4

A 30,037 430.1 42.3 375 402 431 462 485 60 9 10

F 29,027 434.1 42.0 380 405 436 464 489 59 9 10

Overall 59,064 432.1 42.2 378 405 433 462 485 57 N/A N/A

5

A 30,354 425.7 37.1 380 399 425 452 476 53 8 10

F 29,293 426.5 35.9 383 402 426 451 471 49 8 10

Overall 59,647 426.1 36.5 381 401 425 451 476 50 N/A N/A

6

A 29,778 429.7 38.5 383 402 427 455 482 53 8 9

F 28,531 432.0 39.2 385 404 428 455 485 51 8 9

Overall 58,309 430.8 38.9 383 402 428 455 482 53 N/A N/A

7

A 30,180 422.5 39.5 372 394 421 451 475 57 8 9

F 29,100 425.3 38.0 376 397 424 454 475 57 8 8

Overall 59,280 423.9 38.8 375 396 422 451 475 55 N/A N/A

8

A 30,089 426.8 40.8 377 397 424 455 482 58 8 8

F 29,114 429.6 38.6 382 401 425 455 481 54 8 7

Overall 59,203 428.2 39.8 380 399 424 455 482 56 N/A N/A



168

Table 4.3. The 2011 MSA-Math Strand (Cluster) Correlations: Grade 3

Form N Mean SD 1 2 3 4 5

Form A

1. Algebra 31,082 20.12 5.28 1.00

2. Geometry and Measurement 31,082 24.33 4.88 0.67 1.00

3. Statistics and Probability 31,082 21.38 5.30 0.74 0.67 1.00

4. Numbers and Computation 31,082 24.95 6.35 0.76 0.70 0.76 1.00

5. Process 31,082 16.36 4.86 0.62 0.62 0.63 0.66 1.00

Form F

1. Algebra 30,211 20.87 4.93 1.00




5. Process 30,211 14.94 5.03 0.63 0.63 0.65 0.67 1.00




Form A

1. Algebra 30,037 21.74 5.27 1.00




5. Process 30,037 14.68 5.75 0.70 0.69 0.73 0.68 1.00

Form F

1. Algebra 29,027 21.88 4.94 1.00




5. Process 29,027 16.41 5.35 0.67 0.67 0.72 0.68 1.00



169



Form A

1. Algebra 30,354 22.59 6.15 1.00




5. Process 30,354 15.75 7.60 0.74 0.73 0.69 0.79 1.00

Form F

1. Algebra 29,293 22.22 5.94 1.00




5. Process 29,293 18.39 7.87 0.75 0.71 0.67 0.74 1.00




Form A

1. Algebra 29,778 18.92 6.48 1.00




5. Process 29,778 17.24 7.24 0.78 0.77 0.75 0.75 1.00

Form F

1. Algebra 28,531 19.85 6.37 1.00




5. Process 28,531 15.78 7.42 0.77 0.77 0.73 0.75 1.00



170



Form A

1. Algebra 30,180 17.17 7.02 1.00




5. Process 30,180 12.95 6.30 0.77 0.79 0.79 0.70 1.00

Form F

1. Algebra 29,100 16.65 6.67 1.00




5. Process 29,100 15.19 6.83 0.76 0.79 0.79 0.73 1.00




Form A

1. Algebra 30,089 15.13 7.73 1.00




5. Process 30,089 15.77 8.98 0.84 0.83 0.84 0.77 1.00

Form F

1. Algebra 29,114 16.41 7.86 1.00




5. Process 29,114 15.31 9.58 0.83 0.83 0.79 0.76 1.00



171

Table 4.9. The 2011 MSA-m Decision Accuracy and Consistency Indices: Grade 3

Form Performance Cut Accuracy False Positive False Negative Consistency

A B : PA 0.95 0.02 0.03 0.93

A BP : A 0.92 0.04 0.04 0.89

. . . .

F B : PA 0.95 0.02 0.03 0.93

F BP : A 0.92 0.04 0.04 0.89

Note. B: PA denotes the cut between Basic and Proficient, while BP:A denotes the cut between Proficient and Advanced.



A B : PA 0.97 0.01 0.02 0.95

A BP : A 0.92 0.04 0.04 0.89

. . . .

F B : PA 0.97 0.01 0.02 0.96

F BP : A 0.92 0.04 0.04 0.88




A B : PA 0.94 0.02 0.03 0.92

A BP : A 0.94 0.04 0.03 0.91

. . . .

F B : PA 0.94 0.03 0.03 0.92

F BP : A 0.94 0.04 0.03 0.91



172



A B : PA 0.93 0.03 0.04 0.91

A BP : A 0.94 0.03 0.03 0.91

. . . .

F B : PA 0.92 0.04 0.04 0.89

F BP : A 0.94 0.03 0.03 0.92




A B : PA 0.94 0.03 0.03 0.92

A BP : A 0.95 0.03 0.02 0.93

. . . .

F B : PA 0.94 0.03 0.03 0.92

F BP : A 0.94 0.03 0.03 0.92




A B : PA 0.93 0.03 0.03 0.91

A BP : A 0.94 0.03 0.03 0.92

. . . .

F B : PA 0.93 0.03 0.03 0.90

F BP : A 0.94 0.03 0.03 0.91



173

Table 4.15. The 2011 MSA-Math Score Difference between Rater 1 and Rater 2: Grade 3

Form Score Range Item CID

Perfect Adjacent Discrepancy Total

N % N % N % N %

A 0-1 100000011186 30,547 98.90 339 1.10 30,886 100.00

0-2 3595529 27,456 89.10 3,344 10.85 14 0.05 30,814 100.00

0-1 3509941 31,082 100.00 31,082 100.00

0-2 3595501 31,082 100.00 31,082 100.00

0-1 3509957 30,519 99.13 269 0.87 30,788 100.00

0-2 3564081 26,172 85.40 4,427 14.45 47 0.15 30,646 100.00

0-1 3510067 30,548 99.39 187 0.61 30,735 100.00

0-2 3564083 27,160 88.52 3,515 11.46 9 0.03 30,684 100.00

0-1 3509940 30,805 99.62 118 0.38 30,923 100.00

0-2 584928 26,859 86.99 3,891 12.60 125 0.40 30,875 100.00

0-1 3509978 29,710 99.65 104 0.35 29,814 100.00

0-2 3985610 29,128 95.07 1,512 4.93 30,640 100.00

0-1 3547998 30,056 97.30 834 2.70 30,890 100.00

0-2 3564094 25,682 83.37 5,104 16.57 20 0.06 30,806 100.00

F 0-1 100000011186 29,782 99.09 275 0.91 30,057 100.00

0-2 3595529 26,616 88.76 3,358 11.20 12 0.04 29,986 100.00

0-1 3509918 30,211 100.00 30,211 100.00

0-2 3564076 30,211 100.00 30,211 100.00

0-1 3509922 29,663 99.24 228 0.76 29,891 100.00

0-2 3564085 27,193 91.49 2,500 8.41 30 0.10 29,723 100.00

0-1 3510073 29,254 98.98 300 1.02 29,554 100.00

0-2 3595503 26,844 90.34 2,864 9.64 5 0.02 29,713 100.00

0-1 3488087 29,987 99.74 78 0.26 30,065 100.00

0-2 3564099 27,787 92.62 2,196 7.32 17 0.06 30,000 100.00

0-1 3509949 29,615 99.58 125 0.42 29,740 100.00

0-2 3985609 26,822 90.96 2,662 9.03 3 0.01 29,487 100.00

0-1 3547998 28,954 97.06 877 2.94 29,831 100.00

0-2 3564094 25,929 86.38 3,996 13.31 94 0.31 30,019 100.00



174




N % N % N % N %

A 0-1 3487819 29,284 97.95 612 2.05 29,896 100.00

0-2 3564186 26,230 87.79 3,635 12.17 14 0.05 29,879 100.00

0-1 3515451 30,037 100.00 30,037 100.00

0-2 3564161 30,037 100.00 30,037 100.00

0-1 3515807 29,405 98.81 355 1.19 29,760 100.00

0-2 3564165 24,408 82.03 5,202 17.48 145 0.49 29,755 100.00

0-1 3515862 28,200 99.72 78 0.28 28,278 100.00

0-2 3564170 22,828 77.26 6,564 22.22 154 0.52 29,546 100.00

0-1 3515642 29,485 99.15 252 0.85 29,737 100.00

0-2 3985619 25,255 85.10 4,390 14.79 32 0.11 29,677 100.00

0-1 3488145 29,441 99.20 238 0.80 29,679 100.00

0-2 3564189 24,426 82.71 5,004 16.94 101 0.34 29,531 100.00

0-1 3515638 29,465 98.83 348 1.17 29,813 100.00

0-2 3564169 25,787 86.93 3,861 13.02 17 0.06 29,665 100.00

F 0-1 100000069368 28,010 96.94 885 3.06 28,895 100.00

0-2 3985632 27,176 94.02 1,715 5.93 14 0.05 28,905 100.00

0-1 100000044142 29,027 100.00 29,027 100.00

0-2 3595499 29,027 100.00 29,027 100.00

0-1 3515648 28,434 98.87 324 1.13 28,758 100.00

0-2 3564163 25,085 87.30 3,579 12.46 69 0.24 28,733 100.00

0-1 3515646 28,482 99.03 280 0.97 28,762 100.00

0-2 3595537 23,453 81.79 5,072 17.69 151 0.53 28,676 100.00

0-1 100000025172 28,620 99.34 191 0.66 28,811 100.00

0-2 3985613 25,499 88.52 3,273 11.36 33 0.11 28,805 100.00

0-1 3488145 28,774 99.57 125 0.43 28,899 100.00

0-2 3564189 24,044 83.22 4,826 16.70 22 0.08 28,892 100.00

0-1 100000007087 28,549 98.93 309 1.07 28,858 100.00

0-2 3595563 24,878 86.65 3,819 13.30 13 0.05 28,710 100.00



175




N % N % N % N %

A 0-1 100000065205 29,748 98.44 472 1.56 30,220 100.00

0-2 3985586 27,738 91.79 2,472 8.18 9 0.03 30,219 100.00

0-1 100000028212 30,354 100.00 30,354 100.00

0-2 3595468 30,354 100.00 30,354 100.00

0-1 100000009927 29,918 99.35 195 0.65 30,113 100.00

0-2 3985561 27,095 90.16 2,956 9.84 1 0.00 30,052 100.00

0-1 100000028202 30,354 100.00 30,354 100.00

0-3 3985560 30,354 100.00 30,354 100.00

0-1 3488347 27,960 94.14 1,741 5.86 29,701 100.00

0-2 3564046 27,227 91.83 2,348 7.92 73 0.25 29,648 100.00

0-1 3548459 29,800 98.94 319 1.06 30,119 100.00

0-2 3564051 24,861 82.69 5,082 16.90 122 0.41 30,065 100.00

0-1 100000160221 29,269 98.14 554 1.86 29,823 100.00

0-2 3985559 27,314 92.11 2,179 7.35 161 0.54 29,654 100.00

0-1 3488530 29,536 99.48 154 0.52 29,690 100.00

0-2 3564054 27,777 93.70 1,839 6.20 30 0.10 29,646 100.00

F 0-1 100000065205 28,704 98.38 472 1.62 29,176 100.00

0-2 3985586 26,750 91.69 2,416 8.28 9 0.03 29,175 100.00

0-1 3511336 29,293 100.00 29,293 100.00

0-2 3563987 29,293 100.00 29,293 100.00

0-1 3512615 27,588 97.42 731 2.58 28,319 100.00

0-2 3595439 25,482 90.47 2,667 9.47 17 0.06 28,166 100.00

0-1 100000028200 29,293 100.00 29,293 100.00

0-3 3595464 29,293 100.00 29,293 100.00

0-1 100000022530 28,406 98.99 290 1.01 28,696 100.00

0-2 3985592 22,653 78.95 6,027 21.01 13 0.05 28,693 100.00

0-1 100000028238 27,913 97.22 797 2.78 28,710 100.00

0-2 3985553 24,671 86.34 3,649 12.77 254 0.89 28,574 100.00

0-1 100000022532 26,928 94.06 1,701 5.94 28,629 100.00

0-2 3595471 26,104 91.02 2,514 8.77 60 0.21 28,678 100.00

0-1 3488530 27,379 96.21 1,079 3.79 28,458 100.00

0-2 3564054 26,108 91.77 2,288 8.04 53 0.19 28,449 100.00

Note. Analysis was conducted with a statewide population. Note. Bold-faced item indicates an ECR item.


176




N % N % N % N %

A 0-1 3517004 29,402 99.44 166 0.56 29,568 100.00

0-3 3564010 22,416 76.31 6,622 22.54 336 1.14 29,374 100.00

0-1 100000094477 29,778 100.00 29,778 100.00

0-2 3985566 29,778 100.00 29,778 100.00

0-1 100000028367 27,482 98.51 416 1.49 27,898 100.00

0-2 3595489 24,251 89.77 2,704 10.01 59 0.22 27,014 100.00

0-1 3516333 29,207 99.55 133 0.45 29,340 100.00

0-2 3564008 26,937 91.95 2,322 7.93 36 0.12 29,295 100.00

0-1 100000022501 29,778 100.00 29,778 100.00

0-2 3595485 29,778 100.00 29,778 100.00

0-1 100000064587 28,617 97.92 607 2.08 29,224 100.00

0-2 3985580 26,188 90.21 2,799 9.64 42 0.14 29,029 100.00

0-1 3488404 28,961 99.05 278 0.95 29,239 100.00

0-2 3985578 26,230 89.51 3,030 10.34 44 0.15 29,304 100.00

F 0-1 3517004 28,202 99.43 162 0.57 28,364 100.00

0-3 3564010 21,542 76.47 6,304 22.38 325 1.15 28,171 100.00

0-1 100000094481 28,531 100.00 28,531 100.00

0-2 3985581 28,531 100.00 28,531 100.00

0-1 100000028363 27,601 99.68 89 0.32 27,690 100.00

0-2 3595491 24,007 87.65 3,342 12.20 42 0.15 27,391 100.00

0-1 100000022498 27,902 99.39 172 0.61 28,074 100.00

0-2 3595494 24,908 88.93 3,044 10.87 55 0.20 28,007 100.00

0-1 100000022504 28,531 100.00 28,531 100.00

0-2 3595488 28,531 100.00 28,531 100.00

0-1 100000064587 26,939 97.20 777 2.80 27,716 100.00

0-2 3985580 23,039 83.60 4,483 16.27 37 0.13 27,559 100.00

0-1 100000065098 26,675 99.27 197 0.73 26,872 100.00

0-2 3985582 24,475 90.57 2,273 8.41 275 1.02 27,023 100.00



177




N % N % N % N %

A 0-1 100000064017 29,272 99.68 94 0.32 29,366 100.00

0-2 3985664 24,802 85.10 4,308 14.78 33 0.11 29,143 100.00

0-1 3491692 30,180 100.00 30,180 100.00

0-3 3564159 30,180 100.00 30,180 100.00

0-1 3517818 28,769 99.50 145 0.50 28,914 100.00

0-2 3564023 24,347 86.73 3,667 13.06 58 0.21 28,072 100.00

0-1 3517706 26,730 99.76 65 0.24 26,795 100.00

0-2 3564025 22,339 85.66 3,737 14.33 2 0.01 26,078 100.00

0-1 100000043347 30,180 100.00 30,180 100.00

0-3 3595366 30,180 100.00 30,180 100.00

0-1 100000064068 28,257 97.92 601 2.08 28,858 100.00

0-3 3985667 24,086 84.70 4,314 15.17 38 0.13 28,438 100.00

0-1 100000063909 29,325 99.52 140 0.48 29,465 100.00

0-2 3985662 27,322 93.92 1,755 6.03 14 0.05 29,091 100.00

F 0-1 100000064058 28,459 99.62 109 0.38 28,568 100.00

0-2 3985652 26,765 94.22 1,639 5.77 2 0.01 28,406 100.00

0-1 100000026808 29,100 100.00 29,100 100.00

0-3 3595374 29,100 100.00 29,100 100.00

0-1 3517818 27,869 99.49 144 0.51 28,013 100.00

0-2 3564023 24,038 86.75 3,423 12.35 248 0.90 27,709 100.00

0-1 3517706 26,243 99.73 72 0.27 26,315 100.00

0-2 3564025 21,930 85.08 3,843 14.91 2 0.01 25,775 100.00

0-1 100000064101 29,100 100.00 29,100 100.00

0-3 3985665 29,100 100.00 29,100 100.00

0-1 3517648 27,328 97.91 582 2.09 27,910 100.00

0-3 3564027 23,006 83.36 4,556 16.51 35 0.13 27,597 100.00

0-1 100000012815 27,838 96.95 876 3.05 28,714 100.00

0-2 3595391 24,539 85.53 3,914 13.64 239 0.83 28,692 100.00



178




N % N % N % N %

A 0-1 3514013 29,188 99.35 191 0.65 29,379 100.00

0-2 3564107 25,550 87.50 3,585 12.28 64 0.22 29,199 100.00

0-1 100000064847 30,089 100.00 30,089 100.00

0-3 3985701 30,089 100.00 30,089 100.00

0-1 100000064945 27,747 99.03 272 0.97 28,019 100.00

0-2 3985679 23,833 85.28 3,745 13.40 369 1.32 27,947 100.00

0-1 3487636 28,312 98.32 483 1.68 28,795 100.00

0-2 3595428 24,744 86.73 3,773 13.23 12 0.04 28,529 100.00

0-1 100000064837 30,089 100.00 30,089 100.00

0-3 3985702 30,089 100.00 30,089 100.00

0-1 3487759 28,955 99.22 227 0.78 29,182 100.00

0-2 3564128 27,236 94.33 1,529 5.30 107 0.37 28,872 100.00

0-1 3514078 28,464 98.51 431 1.49 28,895 100.00

0-3 3564109 24,646 85.31 4,175 14.45 69 0.24 28,890 100.00

0-1 100000012739 28,275 99.45 157 0.55 28,432 100.00

0-2 3985697 26,084 92.64 2,067 7.34 5 0.02 28,156 100.00

F 0-1 100000043324 27,605 99.40 168 0.60 27,773 100.00

0-2 3595434 26,418 96.25 973 3.55 55 0.20 27,446 100.00

0-1 3487937 29,114 100.00 29,114 100.00

0-3 3564125 29,114 100.00 29,114 100.00

0-1 100000064945 27,192 99.36 174 0.64 27,366 100.00

0-2 3985679 22,816 83.38 4,290 15.68 258 0.94 27,364 100.00

0-1 3487636 27,759 98.74 355 1.26 28,114 100.00

0-2 3595428 24,886 89.07 3,047 10.91 8 0.03 27,941 100.00

0-1 100000026770 29,114 100.00 29,114 100.00

0-3 3595425 29,114 100.00 29,114 100.00

0-1 100000064972 28,142 99.18 232 0.82 28,374 100.00

0-2 3985691 26,504 94.32 1,497 5.33 100 0.36 28,101 100.00

0-1 3487639 27,753 99.72 79 0.28 27,832 100.00

0-3 3985692 23,533 84.70 4,131 14.87 119 0.43 27,783 100.00

0-1 100000012739 25,950 99.54 119 0.46 26,069 100.00

0-2 3985697 22,501 88.34 2,959 11.62 11 0.04 25,471 100.00



179

Table 4.21. The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 3 Form A

Raw Score Scale Score

(SS) Standard Error

(SE) 0 240a 47

1 240a 34

2 240a 24

3 243 20

4 254 18

5 263 16

6 270 15

7 276 14

8 282 13

9 287 13

10 292 12

11 296 12

12 300 11

13 304 11

14 308 11

15 311 10

16 315 10

17 318 10

18 321 10

19 324 10

20 327 10

21 329 10

22 332 9

23 335 9

24 337 9

25 340 9

26 343 9

27 345 9

28 348 9

29 350 9

30 353 9

31 355 9

32 357 9

33 360 9

34 362 9

35 365 9

36 367 9

37 370 9

38 372 9

39 375 9

40 377 9


180


Raw Score Scale Score (SS)

Standard Error (SE)

41 380 9

42 382 9

43 385 9

44 387 9

45 390 9

46 393 9

47 395 10

48 398 10

49 401 10

50 404 10

51 407 10

52 410 10

53 413 10

54 417 11

55 420 11

56 424 11

57 428 11

58 432 12

59 436 12

60 440 12

61 445 13

62 450 13

63 456 14

64 462 14

65 469 15

66 476 16

67 484 17

68 494 19

69 506 21

70 522 25

71 548 34

72 650b 47

Note. aLOSS was set to 240. Note. bHOSS was set to 650.


181

Table 4.22. The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 3 Form F


(SS) Standard Error

(SE) 0 240a 47

1 240a 33

2 240a 24

3 245 20

4 255 17

5 264 16

6 271 15

7 277 14

8 282 13

9 287 12

10 292 12

11 296 12

12 300 11

13 304 11

14 307 11

15 311 10

16 314 10

17 317 10

18 320 10

19 323 10

20 326 10

21 329 10

22 331 9

23 334 9

24 337 9

25 339 9

26 342 9

27 345 9

28 347 9

29 350 9

30 352 9

31 355 9

32 357 9

33 360 9

34 362 9

35 365 9

36 367 9

37 370 9

38 372 9

39 375 9

40 377 9


182



(SS) Standard Error

(SE) 41 380 9

42 382 9

43 385 9

44 388 9

45 390 9

46 393 10

47 396 10

48 399 10

49 402 10

50 405 10

51 408 10

52 411 10

53 415 11

54 418 11

55 422 11

56 426 11

57 429 11

58 434 12

59 438 12

60 443 13

61 448 13

62 453 14

63 459 14

64 465 15

65 472 16

66 481 17

67 490 18

68 501 20

69 514 22

70 532 26

71 559 35

72 650b 48



183


Strand Raw Score Scale Score

(SS) Standard Error

(SE)

AL 0 240a 48

AL 1 276 35

AL 2 303 26

AL 3 320 23

AL 4 335 21

AL 5 347 20

AL 6 359 20

AL 7 371 20

AL 8 383 20

AL 9 396 21

AL 10 411 23

AL 11 430 27

AL 12 458 35

AL 13 650b 48

GM 0 240a 48

GM 1 253 35

GM 2 281 27

GM 3 300 23

GM 4 315 21

GM 5 327 20

GM 6 339 19

GM 7 350 19

GM 8 361 19

GM 9 372 19

GM 10 383 20

GM 11 396 21

GM 12 411 23

GM 13 429 26

GM 14 457 35

GM 15 650b 48

SP 0 240a 50

SP 1 245 38

SP 2 279 30

SP 3 302 26

SP 4 320 24

SP 5 336 22

SP 6 351 21

SP 7 364 21

SP 8 377 21

SP 9 391 21

SP 10 405 22


184



(SS) Standard Error

(SE)

SP 11 421 24

SP 12 440 27

SP 13 469 35

SP 14 650b 48

NC 0 240a 47

NC 1 265 34

NC 2 291 26

NC 3 309 22

NC 4 322 20

NC 5 334 19

NC 6 344 18

NC 7 355 18

NC 8 365 18

NC 9 375 18

NC 10 385 19

NC 11 396 19

NC 12 409 21

NC 13 423 23

NC 14 441 26

NC 15 468 35

NC 16 650b 48

PR 0 240a 49

PR 1 279 36

PR 2 309 28

PR 3 329 25

PR 4 347 23

PR 5 363 23

PR 6 380 24

PR 7 398 25

PR 8 418 26

PR 9 439 26

PR 10 459 26

PR 11 480 26

PR 12 502 28

PR 13 532 36

PR 14 650b 48

Note. aLOSS was set to 240. Note. bHOSS was set to 650. Note. AL=Algebra; GM=Geometry and Measurement; SP=Statistics and Probability; NC=Numbers and Computation; PR=Process


185



(SS) Standard Error

(SE)

AL 0 240a 48

AL 1 260 36

AL 2 289 27

AL 3 308 24

AL 4 324 22

AL 5 338 21

AL 6 351 21

AL 7 364 21

AL 8 377 21

AL 9 391 22

AL 10 407 24

AL 11 426 27

AL 12 455 35

AL 13 650b 48

GM 0 240a 48

GM 1 253 35

GM 2 281 27

GM 3 300 23

GM 4 315 21

GM 5 327 20

GM 6 339 19

GM 7 351 19

GM 8 362 19

GM 9 374 20

GM 10 386 20

GM 11 399 21

GM 12 413 23

GM 13 432 26

GM 14 459 35

GM 15 650b 48

SP 0 240a 48

SP 1 256 35

SP 2 284 26

SP 3 302 23

SP 4 317 22

SP 5 331 21

SP 6 344 21

SP 7 357 21

SP 8 370 21

SP 9 384 22

SP 10 399 23


186



(SS) Standard Error

(SE)

SP 11 416 24

SP 12 436 28

SP 13 465 36

SP 14 650b 49

NC 0 240a 47

NC 1 266 34

NC 2 292 25

NC 3 309 22

NC 4 322 20

NC 5 333 19

NC 6 344 18

NC 7 353 18

NC 8 363 18

NC 9 373 18

NC 10 383 18

NC 11 393 19

NC 12 405 20

NC 13 419 22

NC 14 437 26

NC 15 464 35

NC 16 650b 48

PR 0 240a 49

PR 1 294 36

PR 2 325 28

PR 3 346 25

PR 4 365 24

PR 5 383 24

PR 6 400 24

PR 7 418 24

PR 8 436 25

PR 9 455 25

PR 10 473 25

PR 11 493 26

PR 12 516 29

PR 13 548 37

PR 14 650b 49



187



(SS) Standard Error

(SE)

0 240a 47

1 240a 34

2 246 24

3 261 20

4 272 18

5 281 16

6 288 15

7 295 14

8 301 13

9 306 13

10 311 12

11 315 12

12 319 12

13 323 11

14 327 11

15 331 11

16 334 10

17 337 10

18 340 10

19 343 10

20 346 10

21 349 10

22 352 10

23 355 9

24 357 9

25 360 9

26 363 9

27 365 9

28 368 9

29 370 9

30 373 9

31 375 9

32 378 9

33 380 9

34 383 9

35 385 9

36 387 9

37 390 9

38 392 9

39 395 9

40 397 9


188



(SS) Standard Error

(SE)

41 400 9

42 402 9

43 405 9

44 408 9

45 410 9

46 413 9

47 416 10

48 419 10

49 422 10

50 425 10

51 428 10

52 431 10

53 434 10

54 437 11

55 441 11

56 445 11

57 449 11

58 453 12

59 457 12

60 462 13

61 467 13

62 472 14

63 478 14

64 485 15

65 492 16

66 501 17

67 511 19

68 524 22

69 541 26

70 567 35

71 650b 48



189



(SS) Standard Error

(SE) 0 240a 47

1 240a 34

2 246 24

3 261 20

4 272 18

5 281 16

6 288 15

7 295 14

8 301 13

9 306 13

10 311 12

11 315 12

12 319 12

13 323 11

14 327 11

15 331 11

16 334 10

17 337 10

18 340 10

19 343 10

20 346 10

21 349 10

22 352 10

23 355 9

24 357 9

25 360 9

26 363 9

27 365 9

28 368 9

29 370 9

30 373 9

31 375 9

32 378 9

33 380 9

34 383 9

35 385 9

36 387 9

37 390 9

38 392 9

39 395 9

40 397 9


190



(SS) Standard Error

(SE) 41 400 9

42 402 9

43 405 9

44 407 9

45 410 9

46 413 9

47 415 9

48 418 10

49 421 10

50 424 10

51 427 10

52 430 10

53 433 10

54 436 10

55 439 11

56 443 11

57 447 11

58 451 12

59 455 12

60 460 12

61 464 13

62 470 13

63 476 14

64 482 15

65 489 16

66 498 17

67 508 19

68 520 22

69 537 26

70 564 35

71 650b 48



191



(SS) Standard Error

(SE)

AL 0 240a 48

AL 1 274 35

AL 2 301 26

AL 3 319 23

AL 4 334 21

AL 5 347 20

AL 6 359 20

AL 7 370 19

AL 8 382 20

AL 9 394 20

AL 10 407 21

AL 11 422 23

AL 12 441 27

AL 13 470 36

AL 14 650b 49

GM 0 240a 51

GM 1 280 39

GM 2 316 31

GM 3 340 26

GM 4 359 24

GM 5 375 22

GM 6 389 21

GM 7 402 20

GM 8 415 20

GM 9 427 21

GM 10 441 21

GM 11 456 23

GM 12 474 27

GM 13 502 35

GM 14 650b 48

SP 0 240a 48

SP 1 274 35

SP 2 302 26

SP 3 320 23

SP 4 334 21

SP 5 347 20

SP 6 358 19

SP 7 368 19

SP 8 379 19

SP 9 389 19

SP 10 401 20


192



(SS) Standard Error

(SE)

SP 11 413 21

SP 12 427 23

SP 13 445 26

SP 14 472 35

SP 15 650b 48

NC 0 240a 49

NC 1 273 36

NC 2 302 27

NC 3 322 24

NC 4 337 21

NC 5 350 20

NC 6 362 20

NC 7 374 19

NC 8 385 19

NC 9 397 20

NC 10 410 21

NC 11 424 23

NC 12 443 26

NC 13 470 35

NC 14 650b 48

PR 0 240a 48

PR 1 317 35

PR 2 345 27

PR 3 363 23

PR 4 379 22

PR 5 393 21

PR 6 407 21

PR 7 421 22

PR 8 437 23

PR 9 454 24

PR 10 472 25

PR 11 493 27

PR 12 517 30

PR 13 550 37

PR 14 650b 50



193



(SS) Standard Error

(SE)

AL 0 240a 48

AL 1 274 35

AL 2 301 26

AL 3 320 23

AL 4 335 21

AL 5 348 20

AL 6 360 20

AL 7 371 20

AL 8 383 20

AL 9 395 20

AL 10 408 21

AL 11 423 23

AL 12 443 27

AL 13 471 36

AL 14 650b 49

GM 0 240a 53

GM 1 288 41

GM 2 326 31

GM 3 350 26

GM 4 368 23

GM 5 382 21

GM 6 395 20

GM 7 407 20

GM 8 418 19

GM 9 430 20

GM 10 443 21

GM 11 457 22

GM 12 474 26

GM 13 501 35

GM 14 650b 48

SP 0 240a 48

SP 1 277 35

SP 2 305 27

SP 3 323 23

SP 4 338 21

SP 5 350 20

SP 6 362 19

SP 7 372 19

SP 8 383 19

SP 9 394 19

SP 10 405 19


194



(SS) Standard Error

(SE)

SP 11 417 21

SP 12 431 22

SP 13 448 26

SP 14 475 35

SP 15 650b 48

NC 0 240a 49

NC 1 273 36

NC 2 302 27

NC 3 321 23

NC 4 336 21

NC 5 349 20

NC 6 360 19

NC 7 372 19

NC 8 383 19

NC 9 394 20

NC 10 407 21

NC 11 421 23

NC 12 439 26

NC 13 466 35

NC 14 650b 48

PR 0 240a 49

PR 1 301 36

PR 2 330 28

PR 3 351 24

PR 4 367 23

PR 5 382 22

PR 6 396 22

PR 7 410 22

PR 8 426 23

PR 9 443 25

PR 10 463 26

PR 11 485 28

PR 12 511 30

PR 13 545 38

PR 14 650b 50



195



(SS) Standard Error

(SE)

0 240a 44

1 244 32

2 267 23

3 281 19

4 291 17

5 300 15

6 306 14

7 312 13

8 318 12

9 322 12

10 327 11

11 331 11

12 334 11

13 338 10

14 341 10

15 344 10

16 347 10

17 350 9

18 353 9

19 356 9

20 358 9

21 361 9

22 363 9

23 366 9

24 368 9

25 371 8

26 373 8

27 375 8

28 377 8

29 380 8

30 382 8

31 384 8

32 386 8

33 388 8

34 390 8

35 392 8

36 395 8

37 397 8

38 399 8

39 401 8


196



(SS) Standard Error

(SE)

40 403 8

41 405 8

42 407 8

43 409 8

44 412 8

45 414 8

46 416 8

47 418 8

48 420 8

49 423 8

50 425 8

51 427 9

52 430 9

53 432 9

54 435 9

55 437 9

56 440 9

57 443 9

58 446 10

59 449 10

60 452 10

61 455 10

62 459 11

63 462 11

64 467 11

65 471 12

66 476 13

67 482 13

68 488 14

69 495 16

70 504 17

71 515 20

72 530 23

73 554 32

74 650b 44



197



(SS) Standard Error

(SE) 0 240a 44

1 245 31

2 268 22

3 281 19

4 291 16

5 299 15

6 305 14

7 311 13

8 316 12

9 321 12

10 325 11

11 329 11

12 332 10

13 336 10

14 339 10

15 342 10

16 345 9

17 348 9

18 351 9

19 353 9

20 356 9

21 359 9

22 361 9

23 363 9

24 366 8

25 368 8

26 370 8

27 373 8

28 375 8

29 377 8

30 379 8

31 381 8

32 383 8

33 385 8

34 388 8

35 390 8

36 392 8

37 394 8

38 396 8

39 398 8


198



(SS) Standard Error

(SE) 40 400 8

41 402 8

42 404 8

43 406 8

44 408 8

45 410 8

46 413 8

47 415 8

48 417 8

49 419 8

50 421 8

51 424 8

52 426 9

53 428 9

54 431 9

55 433 9

56 436 9

57 439 9

58 442 9

59 445 10

60 448 10

61 451 10

62 454 10

63 458 11

64 462 11

65 466 12

66 471 12

67 476 13

68 482 14

69 489 15

70 497 17

71 507 19

72 521 23

73 544 32

74 650b 44



199



(SS) Standard Error

(SE)

AL 0 240a 45

AL 1 290 32

AL 2 314 24

AL 3 330 21

AL 4 343 19

AL 5 354 18

AL 6 363 17

AL 7 373 17

AL 8 382 17

AL 9 391 17

AL 10 401 18

AL 11 412 19

AL 12 425 21

AL 13 441 24

AL 14 466 33

AL 15 650b 45

GM 0 240a 45

GM 1 301 33

GM 2 327 25

GM 3 344 22

GM 4 358 20

GM 5 370 19

GM 6 381 18

GM 7 392 18

GM 8 403 18

GM 9 414 19

GM 10 426 20

GM 11 440 22

GM 12 457 25

GM 13 483 33

GM 14 650b 45

SP 0 240a 47

SP 1 283 35

SP 2 312 26

SP 3 332 22

SP 4 346 20

SP 5 359 19

SP 6 370 19

SP 7 381 18

SP 8 392 19

SP 9 404 19


200



(SS) Standard Error

(SE)

SP 10 418 21

SP 11 434 24

SP 12 459 33

SP 13 650b 45

NC 0 240a 44

NC 1 316 32

NC 2 340 24

NC 3 356 20

NC 4 368 19

NC 5 379 18

NC 6 389 17

NC 7 398 17

NC 8 407 17

NC 9 416 17

NC 10 425 17

NC 11 436 19

NC 12 448 20

NC 13 464 24

NC 14 488 32

NC 15 650b 44

PR 0 240a 47

PR 1 320 35

PR 2 351 27

PR 3 371 23

PR 4 387 21

PR 5 400 19

PR 6 410 17

PR 7 420 16

PR 8 428 16

PR 9 436 16

PR 10 445 16

PR 11 454 17

PR 12 464 18

PR 13 476 20

PR 14 491 22

PR 15 510 26

PR 16 537 34

PR 17 650b 46



201



(SS) Standard Error

(SE)

AL 0 240a 45

AL 1 290 32

AL 2 315 24

AL 3 331 21

AL 4 344 19

AL 5 355 18

AL 6 366 18

AL 7 375 17

AL 8 385 17

AL 9 395 18

AL 10 406 18

AL 11 417 19

AL 12 430 21

AL 13 447 25

AL 14 473 33

AL 15 650b 45

GM 0 240a 45

GM 1 291 33

GM 2 318 25

GM 3 336 22

GM 4 350 20

GM 5 363 20

GM 6 375 19

GM 7 387 19

GM 8 399 19

GM 9 411 20

GM 10 425 21

GM 11 439 22

GM 12 457 25

GM 13 484 33

GM 14 650b 45

SP 0 240a 45

SP 1 289 33

SP 2 316 25

SP 3 333 21

SP 4 346 20

SP 5 358 19

SP 6 369 18

SP 7 380 18

SP 8 391 18

SP 9 402 19


202



(SS) Standard Error

(SE)

SP 10 416 21

SP 11 432 24

SP 12 457 33

SP 13 650b 45

NC 0 240a 44

NC 1 315 32

NC 2 339 24

NC 3 355 20

NC 4 367 19

NC 5 378 18

NC 6 387 17

NC 7 397 17

NC 8 406 17

NC 9 415 17

NC 10 425 18

NC 11 436 19

NC 12 448 21

NC 13 464 24

NC 14 489 32

NC 15 650b 45

PR 0 240a 46

PR 1 317 34

PR 2 345 26

PR 3 363 22

PR 4 376 19

PR 5 388 18

PR 6 397 17

PR 7 406 16

PR 8 414 16

PR 9 423 16

PR 10 431 16

PR 11 440 17

PR 12 449 18

PR 13 460 19

PR 14 473 21

PR 15 491 25

PR 16 518 34

PR 17 650b 46



203



(SS) Standard Error

(SE) 0 240a 42

1 268 30

2 289 22

3 302 18

4 312 16

5 319 14

6 326 13

7 331 12

8 336 12

9 340 11

10 344 11

11 348 10

12 352 10

13 355 10

14 358 9

15 361 9

16 364 9

17 366 9

18 369 9

19 372 9

20 374 8

21 376 8

22 379 8

23 381 8

24 383 8

25 386 8

26 388 8

27 390 8

28 392 8

29 394 8

30 396 8

31 398 8

32 400 8

33 402 8

34 404 8

35 406 8

36 408 8

37 410 8

38 412 8

39 414 8

40 416 8


204



(SS) Standard Error

(SE) 41 418 8

42 420 8

43 423 8

44 425 8

45 427 8

46 429 8

47 431 8

48 434 8

49 436 8

50 438 8

51 441 9

52 443 9

53 446 9

54 449 9

55 452 9

56 455 10

57 458 10

58 461 10

59 465 10

60 468 11

61 472 11

62 477 12

63 482 12

64 487 13

65 494 14

66 501 16

67 511 18

68 524 22

69 546 30

70 650b 42



205



(SS) Standard Error

(SE) 0 240a 42

1 268 30

2 290 22

3 303 18

4 312 16

5 320 14

6 326 13

7 331 12

8 336 12

9 341 11

10 345 11

11 348 10

12 352 10

13 355 10

14 358 9

15 361 9

16 364 9

17 367 9

18 369 9

19 372 9

20 374 8

21 377 8

22 379 8

23 381 8

24 383 8

25 385 8

26 388 8

27 390 8

28 392 8

29 394 8

30 396 8

31 398 8

32 400 8

33 402 8

34 404 8

35 406 8

36 408 8

37 410 8

38 411 8

39 413 8

40 415 8


206



(SS) Standard Error

(SE) 41 417 8

42 419 8

43 421 8

44 423 8

45 426 8

46 428 8

47 430 8

48 432 8

49 434 8

50 437 8

51 439 8

52 441 9

53 444 9

54 447 9

55 449 9

56 452 9

57 455 10

58 458 10

59 462 10

60 466 11

61 470 11

62 474 12

63 479 12

64 485 13

65 491 15

66 499 16

67 509 18

68 523 22

69 545 31

70 650b 43



207



(SS) Standard Error

(SE)

AL 0 240a 44

AL 1 311 32

AL 2 336 24

AL 3 353 21

AL 4 366 19

AL 5 377 18

AL 6 388 17

AL 7 398 17

AL 8 408 17

AL 9 418 18

AL 10 429 19

AL 11 442 20

AL 12 458 24

AL 13 483 32

AL 14 650b 43

GM 0 240a 43

GM 1 332 31

GM 2 356 23

GM 3 371 20

GM 4 383 18

GM 5 394 17

GM 6 404 17

GM 7 413 17

GM 8 422 17

GM 9 432 17

GM 10 443 18

GM 11 455 20

GM 12 471 23

GM 13 494 31

GM 14 650b 43

SP 0 240a 43

SP 1 308 32

SP 2 333 24

SP 3 350 21

SP 4 363 20

SP 5 376 19

SP 6 388 19

SP 7 400 19

SP 8 412 19

SP 9 425 20

SP 10 441 22


208



(SS) Standard Error

(SE)

SP 11 459 25

SP 12 487 33

SP 13 650b 45

NC 0 240a 43

NC 1 327 32

NC 2 351 24

NC 3 367 20

NC 4 380 18

NC 5 391 17

NC 6 400 17

NC 7 410 17

NC 8 419 17

NC 9 428 17

NC 10 439 18

NC 11 451 20

NC 12 466 23

NC 13 490 31

NC 14 650b 43

PR 0 240a 43

PR 1 324 32

PR 2 349 24

PR 3 365 20

PR 4 378 19

PR 5 389 18

PR 6 399 17

PR 7 409 17

PR 8 419 17

PR 9 429 18

PR 10 440 18

PR 11 452 20

PR 12 466 21

PR 13 484 25

PR 14 510 32

PR 15 650b 44



209



(SS) Standard Error

(SE)

AL 0 240a 43

AL 1 309 32

AL 2 334 24

AL 3 350 20

AL 4 363 19

AL 5 374 18

AL 6 384 17

AL 7 393 17

AL 8 403 17

AL 9 413 18

AL 10 424 19

AL 11 437 20

AL 12 453 24

AL 13 478 32

AL 14 650b 43

GM 0 240a 43

GM 1 325 31

GM 2 349 23

GM 3 364 20

GM 4 376 18

GM 5 386 17

GM 6 396 17

GM 7 405 16

GM 8 414 17

GM 9 423 17

GM 10 434 18

GM 11 446 20

GM 12 461 23

GM 13 485 31

GM 14 650b 43

SP 0 240a 43

SP 1 309 32

SP 2 334 24

SP 3 351 21

SP 4 365 20

SP 5 378 19

SP 6 390 18

SP 7 401 18

SP 8 413 19

SP 9 425 19

SP 10 438 21


210



(SS) Standard Error

(SE)

SP 11 455 24

SP 12 479 31

SP 13 650b 43

NC 0 240a 43

NC 1 325 32

NC 2 349 23

NC 3 365 20

NC 4 377 18

NC 5 388 17

NC 6 397 17

NC 7 407 16

NC 8 416 16

NC 9 425 17

NC 10 435 18

NC 11 447 20

NC 12 462 23

NC 13 485 31

NC 14 650b 43

PR 0 240a 44

PR 1 337 32

PR 2 362 24

PR 3 379 20

PR 4 391 18

PR 5 402 17

PR 6 412 17

PR 7 421 16

PR 8 430 16

PR 9 439 17

PR 10 449 18

PR 11 460 19

PR 12 474 21

PR 13 492 25

PR 14 519 33

PR 15 650b 45



211



(SS) Standard Error

(SE) 0 240a 40

1 276 29

2 297 21

3 309 17

4 318 15

5 326 14

6 332 13

7 337 12

8 342 11

9 346 11

10 350 10

11 354 10

12 358 10

13 361 9

14 364 9

15 367 9

16 370 9

17 372 9

18 375 9

19 378 8

20 380 8

21 382 8

22 385 8

23 387 8

24 389 8

25 392 8

26 394 8

27 396 8

28 398 8

29 400 8

30 402 8

31 404 8

32 406 8

33 408 8

34 410 8

35 413 8

36 415 8

37 417 8

38 419 8

39 421 8

40 423 8


212



(SS) Standard Error

(SE) 41 425 8

42 427 8

43 429 8

44 432 8

45 434 8

46 436 8

47 439 8

48 441 8

49 443 8

50 446 9

51 449 9

52 451 9

53 454 9

54 457 9

55 460 9

56 464 10

57 467 10

58 471 10

59 475 11

60 479 11

61 484 12

62 489 12

63 494 13

64 501 14

65 509 15

66 518 17

67 529 19

68 544 22

69 564 26

70 592 31

71 634 39

72 650b 47



213



(SS) Standard Error

(SE) 0 240a 40

1 276 29

2 296 21

3 309 17

4 318 15

5 326 14

6 332 13

7 338 12

8 342 11

9 347 11

10 351 10

11 355 10

12 358 10

13 361 10

14 364 9

15 367 9

16 370 9

17 373 9

18 376 9

19 378 8

20 381 8

21 383 8

22 386 8

23 388 8

24 390 8

25 393 8

26 395 8

27 397 8

28 399 8

29 401 8

30 403 8

31 406 8

32 408 8

33 410 8

34 412 8

35 414 8

36 416 8

37 418 8

38 420 8

39 422 8

40 424 8


214



(SS) Standard Error

(SE) 41 426 8

42 428 8

43 431 8

44 433 8

45 435 8

46 437 8

47 439 8

48 442 8

49 444 8

50 446 8

51 449 8

52 451 9

53 454 9

54 457 9

55 459 9

56 462 9

57 465 9

58 469 10

59 472 10

60 475 10

61 479 10

62 483 11

63 487 11

64 492 12

65 497 12

66 503 13

67 509 14

68 517 15

69 526 17

70 539 21

71 560 29

72 650b 40



215



(SS) Standard Error

(SE)

AL 0 240a 41

AL 1 325 30

AL 2 348 22

AL 3 363 19

AL 4 374 17

AL 5 384 17

AL 6 394 16

AL 7 403 16

AL 8 412 16

AL 9 422 17

AL 10 432 18

AL 11 445 19

AL 12 460 22

AL 13 483 30

AL 14 650b 41

GM 0 240a 42

GM 1 326 31

GM 2 350 23

GM 3 367 20

GM 4 380 19

GM 5 392 18

GM 6 403 18

GM 7 414 18

GM 8 425 18

GM 9 437 19

GM 10 450 20

GM 11 466 23

GM 12 490 30

GM 13 650b 41

SP 0 240a 41

SP 1 325 30

SP 2 349 23

SP 3 365 20

SP 4 377 18

SP 5 388 17

SP 6 399 17

SP 7 408 17

SP 8 418 17

SP 9 428 17

SP 10 439 18

SP 11 452 20


216



(SS) Standard Error

(SE)

SP 12 468 23

SP 13 493 31

SP 14 650b 42

NC 0 240a 43

NC 1 326 32

NC 2 353 24

NC 3 370 20

NC 4 383 18

NC 5 394 17

NC 6 404 16

NC 7 413 16

NC 8 423 16

NC 9 433 17

NC 10 443 18

NC 11 456 20

NC 12 471 23

NC 13 495 30

NC 14 650b 42

PR 0 240a 42

PR 1 328 31

PR 2 354 24

PR 3 372 22

PR 4 388 20

PR 5 402 20

PR 6 415 19

PR 7 428 19

PR 8 441 19

PR 9 454 20

PR 10 469 21

PR 11 485 22

PR 12 503 24

PR 13 525 26

PR 14 553 29

PR 15 587 33

PR 16 632 39

PR 17 650b 48



217



(SS) Standard Error

(SE)

AL 0 240a 41

AL 1 328 30

AL 2 351 22

AL 3 366 19

AL 4 378 18

AL 5 389 17

AL 6 399 16

AL 7 408 16

AL 8 418 16

AL 9 428 17

AL 10 438 18

AL 11 450 19

AL 12 466 22

AL 13 489 30

AL 14 650b 41

GM 0 240a 42

GM 1 328 31

GM 2 353 24

GM 3 370 21

GM 4 384 19

GM 5 396 18

GM 6 408 18

GM 7 419 18

GM 8 431 18

GM 9 443 19

GM 10 456 20

GM 11 472 23

GM 12 495 30

GM 13 650b 41

SP 0 240a 41

SP 1 323 30

SP 2 347 23

SP 3 363 20

SP 4 376 18

SP 5 387 17

SP 6 397 17

SP 7 407 17

SP 8 417 17

SP 9 427 17

SP 10 438 18

SP 11 451 20


218



(SS) Standard Error

(SE)

SP 12 466 23

SP 13 490 30

SP 14 650b 41

NC 0 240a 44

NC 1 328 33

NC 2 356 24

NC 3 373 21

NC 4 387 18

NC 5 398 17

NC 6 408 16

NC 7 417 16

NC 8 427 16

NC 9 436 17

NC 10 446 18

NC 11 458 19

NC 12 473 22

NC 13 496 30

NC 14 650b 41

PR 0 240a 42

PR 1 323 31

PR 2 349 24

PR 3 366 21

PR 4 380 20

PR 5 394 19

PR 6 407 19

PR 7 419 18

PR 8 431 18

PR 9 442 18

PR 10 454 18

PR 11 466 18

PR 12 479 18

PR 13 491 19

PR 14 504 20

PR 15 520 23

PR 16 543 30

PR 17 650b 41



219

Table 4.41.The 2011 MSA-Math Total Raw Score to Scale Score Conversion Table: Grade 8 Form A


(SS) Standard Error

(SE) 0 240a 40

1 286 29

2 307 21

3 320 17

4 329 15

5 336 14

6 342 13

7 348 12

8 352 11

9 357 11

10 361 10

11 364 10

12 368 10

13 371 9

14 374 9

15 377 9

16 380 9

17 383 9

18 385 9

19 388 8

20 390 8

21 393 8

22 395 8

23 397 8

24 399 8

25 402 8

26 404 8

27 406 8

28 408 8

29 410 8

30 412 8

31 414 8

32 416 8

33 418 8

34 420 7

35 422 7

36 424 7

37 426 7

38 428 7

39 430 7

40 432 8


220



(SS) Standard Error

(SE) 41 434 8

42 436 8

43 438 8

44 440 8

45 442 8

46 444 8

47 446 8

48 448 8

49 450 8

50 453 8

51 455 8

52 457 8

53 460 8

54 462 8

55 464 8

56 467 9

57 470 9

58 473 9

59 475 9

60 479 9

61 482 10

62 485 10

63 489 11

64 493 11

65 498 12

66 503 12

67 508 13

68 515 14

69 523 16

70 533 18

71 547 22

72 570 30

73 650b 41



221



(SS) Standard Error

(SE) 0 240a 41

1 289 29

2 309 21

3 322 17

4 331 15

5 338 14

6 345 13

7 350 12

8 355 11

9 359 11

10 363 10

11 367 10

12 370 10

13 373 9

14 376 9

15 379 9

16 382 9

17 385 9

18 387 8

19 390 8

20 392 8

21 395 8

22 397 8

23 399 8

24 401 8

25 404 8

26 406 8

27 408 8

28 410 8

29 412 8

30 414 7

31 416 7

32 418 7

33 420 7

34 422 7

35 423 7

36 425 7

37 427 7

38 429 7

39 431 7

40 433 7


222



(SS) Standard Error

(SE) 41 435 7

42 437 7

43 439 7

44 440 7

45 442 7

46 444 7

47 446 8

48 448 8

49 450 8

50 453 8

51 455 8

52 457 8

53 459 8

54 462 8

55 464 8

56 466 9

57 469 9

58 472 9

59 475 9

60 478 10

61 481 10

62 485 10

63 489 11

64 493 11

65 497 12

66 502 12

67 508 13

68 515 14

69 523 16

70 533 18

71 546 21

72 568 29

73 650b 41



223



(SS) Standard Error

(SE)

AL 0 240a 42

AL 1 339 30

AL 2 363 23

AL 3 378 20

AL 4 391 18

AL 5 401 17

AL 6 411 16

AL 7 420 16

AL 8 429 16

AL 9 438 16

AL 10 448 17

AL 11 458 18

AL 12 470 19

AL 13 485 23

AL 14 509 30

AL 15 650b 41

GM 0 240a 41

GM 1 350 30

GM 2 373 23

GM 3 389 19

GM 4 401 18

GM 5 412 17

GM 6 422 17

GM 7 431 17

GM 8 441 17

GM 9 451 18

GM 10 463 19

GM 11 478 22

GM 12 501 30

GM 13 650b 41

SP 0 240a 42

SP 1 330 31

SP 2 355 24

SP 3 372 21

SP 4 386 19

SP 5 398 18

SP 6 409 18

SP 7 420 17

SP 8 431 18

SP 9 442 18

SP 10 454 19


224



(SS) Standard Error

(SE)

SP 11 467 20

SP 12 484 23

SP 13 509 31

SP 14 650b 42

NC 0 240a 41

NC 1 337 30

NC 2 360 22

NC 3 375 19

NC 4 387 18

NC 5 398 17

NC 6 408 17

NC 7 418 17

NC 8 429 18

NC 9 441 19

NC 10 456 22

NC 11 479 30

NC 12 650b 41

PR 0 240a 42

PR 1 328 31

PR 2 354 24

PR 3 372 21

PR 4 386 19

PR 5 399 18

PR 6 410 17

PR 7 420 17

PR 8 430 16

PR 9 439 16

PR 10 447 15

PR 11 455 15

PR 12 464 15

PR 13 472 16

PR 14 482 17

PR 15 492 18

PR 16 505 21

PR 17 523 25

PR 18 551 32

PR 19 650b 43



225



(SS) Standard Error

(SE)

AL 0 240a 41

AL 1 337 30

AL 2 360 22

AL 3 375 19

AL 4 387 18

AL 5 398 17

AL 6 407 16

AL 7 416 16

AL 8 424 16

AL 9 433 16

AL 10 442 17

AL 11 453 17

AL 12 464 19

AL 13 479 22

AL 14 502 30

AL 15 650b 41

GM 0 240a 41

GM 1 343 30

GM 2 367 23

GM 3 383 20

GM 4 395 18

GM 5 407 18

GM 6 417 17

GM 7 428 17

GM 8 438 18

GM 9 449 18

GM 10 462 20

GM 11 478 23

GM 12 502 30

GM 13 650b 41

SP 0 240a 42

SP 1 329 31

SP 2 354 23

SP 3 370 21

SP 4 384 19

SP 5 396 18

SP 6 407 18

SP 7 418 18

SP 8 429 18

SP 9 441 18

SP 10 453 19


226



(SS) Standard Error

(SE)

SP 11 467 21

SP 12 484 24

SP 13 509 31

SP 14 650b 42

NC 0 240a 41

NC 1 344 30

NC 2 367 22

NC 3 382 20

NC 4 395 18

NC 5 406 18

NC 6 417 17

NC 7 428 18

NC 8 440 19

NC 9 453 21

NC 10 471 24

NC 11 498 32

NC 12 650b 43

PR 0 240a 43

PR 1 341 33

PR 2 370 25

PR 3 388 21

PR 4 401 18

PR 5 412 16

PR 6 420 15

PR 7 428 14

PR 8 434 14

PR 9 441 13

PR 10 447 13

PR 11 454 14

PR 12 460 14

PR 13 468 15

PR 14 477 17

PR 15 488 19

PR 16 502 21

PR 17 520 25

PR 18 547 32

PR 19 650b 43



227

REFERENCES

AERA, APA, & NCME (1999). Standards for educational and psychological testing. Washington, D.C.: Author.

Allen, N. L., Donoghue, J. R., & Schoeps, T. L. (2001). The NAEP 1998 technical report (Technical Report). Washington, DC: National Center for Educational Statistics.

Andrich, A. (1988). Rasch models for measurement. Newbury Park, CA: SAGE Publications, Inc.

Andrich, A. (1989). Distinctions between assumptions and requirements in measurement in the social sciences. In J. A. Keats, R. Taft, R. A. Heath, & H. H. Lovibond (Eds.) Mathematical and theoretical systems. North-Holland: Elsevier Science Publisher B.V.

Andrich, A., & Luo, G. (2004). Modern measurement and analysis in social science. Murdoch University, Perth, Western Australia.

Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: SAGE Publications.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York, NY: Holt Rinehart Wilson.

Dorans, N. J., & Schmitt, A. P. (1991). Constructed-response and differential item functioning: A pragmatic approach (ETS Research Report No. 91-49). Princeton, NJ: Educational Testing Service.

Embretson, S., & Reise, S. (2000). Item response theory for psychologists. New Jersey: Lawrence Erlbaum Associates, Publishers.

Haertel, E. H. (1996). Estimating the decision consistency from a single administration of a performance assessment battery. A report on the National Board of Professional Teaching Standards McGEN Assessment. Palo Alto, CA: Stanford University.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: SAGE Publications, Inc.

Harvill, L. M. (1991). Standard error of measurement. Educational Measurement: Issues and Practice, 10, 181-189.

Huynh, H., Meyer III, J. P., & Barton, K. (2000). Technical documentation for the 1999 Palmetto achievement challenge tests of English language arts and mathematics, grades three through eight (Technical Report). Columbia: South Carolina Department of Education.

Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8 & PRELIS 2: User’s reference guide. Chicago: Scientific Software International.

Kolen, M. J., and Brennan, R. L. (1995). Test equating methods and practices. New York: Springer-Verlag.

Linacre, J. M., & Wright, B. D. (2000). A user’s guide to WINSTEPS: Rasch-model computer program. Chicago, IL: MESA Press.


228

Livingston, S. A., & Lewis, C. (1995). Estimating the consistency and accuracy of classifications based on test scores. Journal of Educational Measurement, 32, 179-197.

Loehlin, J. C. (1987). Latent variable models. NJ: Lawrence Erlbaum Associates, Publishers.

Lord, F. M., & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings.” Applied Psychological Measurement, 8, 452-461.

Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.

Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrica, 47, 149-174.

Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18, 5-11.

Mitzel, H. C., Lewis, D. M., Patz, R. J., & Green, D. R. (2001). The Bookmark procedure: Psychological perspectives. In G. J. Cizek (Ed.), Setting performance standards (pp. 249-282). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

Orlando, M. (2004, June). Critical issues to address when applying item response theory (IRT) models. Paper presented at the Drug Information Association, Bethesda, MD.

Pearson (2010, January). 2010 MSA/Mod-MSA-Reading and Mathematics: Test administration and coordination manual. San Antonio, TX: Pearson.

Qualls, A. L. (1995). Estimating the reliability of a test containing multiple item formats, Applied Measurement in Education, 8, 111-120.

Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: University of Chicago Press.

Ryan, J. P. (1983). Introduction to latent trait analysis and item response theory. In W. E. Hathaway (Ed.), Testing in the schools. New directions for testing and measurement, 19, San Francisco: Jossey-Bass.

South Carolina Department of Education. (2001). Technical documentation for the 2000 Palmetto achievement challenge tests of English language arts and mathematics (Technical Report). Columbia: South Carolina Department of Education.

Suen, H. K. (1990). Principles of test theories. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers.

Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrica, 51, 567-577.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis, MESA PRESS, Chicago.

Young, M. J., & Yoon, B. (1998, April). Estimating the consistency and accuracy of classifications in a standards-referenced assessment. (CSE Technical Report 475). Center for the Study of Evaluation, Standards, and Student Testing. Los Angeles, CA: University of California, Los Angeles.


229

Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.


230

A. APPENDIX A: THE 2011 MSA-MATH STRATIFIED RANDOM SAMPLING

Since reporting deadlines made it difficult to use the Maryland school population as calibration and equating data set, Pearson used equating samples instead of the population using stratified random sampling method across all the grades.

Pearson chose Local Education Agency (LEA) as one of the most important variables for stratification. Based on the population percentage of each LEA, Pearson randomly selected about 3,000 students for those grades from first-waved documents (i.e., 50% of the statewide population) which were randomly distributed and completely scored. It should be noted that this method has been applied since the 2006 assessment.

To verify that the sample was representative of the statewide examinee population in terms of gender and ethnicity, the distributions of gender and ethnicity of the 2011 samples were compared with the 2011 population. The results are shown in this Appendix. The percentages of students from the LEAs were all within 2 percentage point of the target values across all grades. The percentages of students from the five major ethnic groups were all within 6 percentage points of the target values across all grades. The percentages of male and female students were within 2 percentage points of the target values across all grades. Consequently, we concluded that the 2011 equating samples were representative of the 2011 statewide examinee population in terms of LEA, gender, and ethnicity.

.


231

Table A.1. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 LEA

Operational Form A Operational Form F

LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.03 31 1.06 -0.03 1.04 31 1.06 -0.02

02 9.23 275 9.39 -0.16 9.14 275 9.42 -0.28

03 12.39 369 12.60 -0.21 12.25 369 12.64 -0.39

04 1.91 58 1.98 -0.07 1.97 58 1.99 -0.01

05 0.65 0 0.00 0.00 0.66 0 0.00 0.00

06 3.26 98 3.35 -0.08 3.31 98 3.36 -0.05

07 1.91 57 1.95 -0.03 1.92 57 1.95 -0.03

08 2.90 87 2.97 -0.07 2.88 87 2.98 -0.10

09 0.49 15 0.51 -0.02 0.49 15 0.51 -0.02

10 4.69 140 4.78 -0.09 4.65 140 4.79 -0.14

11 0.47 13 0.44 0.03 0.42 13 0.45 -0.02

12 4.67 139 4.75 -0.08 4.66 139 4.76 -0.10

13 5.94 178 6.08 -0.14 5.95 178 6.10 -0.14

14 0.22 7 0.24 -0.02 0.24 7 0.24 0.00

15 16.79 508 17.34 -0.55 17.12 508 17.40 -0.27

16 14.29 431 14.71 -0.42 14.41 431 14.76 -0.35

17 0.95 29 0.99 -0.04 1.01 29 0.99 0.01

18 2.08 63 2.15 -0.07 2.10 63 2.16 -0.06

19 0.35 10 0.34 0.01 0.34 10 0.34 0.00

20 0.58 0 0.00 0.00 0.58 0 0.00 0.00

21 2.73 49 1.67 1.05 2.71 43 1.47 1.24

22 1.84 55 1.88 -0.03 1.84 55 1.88 -0.05

23 0.71 21 0.72 0.00 0.70 21 0.72 -0.02

24 0.18 5 0.17 0.01 0.13 2 0.07 0.06

30 9.72 291 9.94 -0.22 9.46 291 9.97 -0.50

Total 100.0 2929 100.0 0.00 100.0 2920 100.0 0.00

Note. 1. Allegany; 2. Anne Arundel; 3. Baltimore; 4. Calvert; 5. Caroline; 6. Carroll; 7. Cecil; 8. Charles; 9. Dorchester; 10. Frederick; 11. Garrett; 12. Harford; 13. Howard; 14. Kent; 15. Montgomery; 16. Prince George’s; 17. Queen Anne’s; 18. St. Mary’s; 19. Somerset; 20. Talbot; 21. Washington; 22. Wicomico; 23. Worcester; 24. LEA 24; 30. Baltimore City.


232

Table A.2. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 Ethnicity


Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.28 149 5.09 -4.81 0.28 150 5.14 -4.86

2 6.17 185 6.32 -0.15 6.23 193 6.61 -0.38

3 33.41 1042 35.58 -2.17 33.92 1051 35.99 -2.07

4 0.12 12 0.41 -0.29 0.08 6 0.21 -0.12

5 42.76 1346 45.95 -3.20 43.23 1306 44.73 -1.50

6 13.20 195 6.66 6.54 11.93 214 7.33 4.60

Miss 4.07 0 0.00 0.00 4.33 0 0.00 0.00

Total 100.0 2929 100.0 0.00 100.0 2920 100.0 0.00

Note. 1. American Indian; 2 Asian American; 3. African American; 4. Native Hawaiian 5. White; 6. Hispanic; Miss: Missing

Table A.3. 2011 MSA-Mathematics Population and Stratified Random Sampling (S.R.S): Grade 3 Gender


Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 51.46 1506 51.42 0.04 50.48 1406 48.15 2.33

2 48.54 1422 48.55 -0.01 49.51 1511 51.75 -2.23

Miss 0.00 1 0.03 -0.03 0.00 3 0.10 -0.10

Total 100.0 2929 100.0 0.00 100.0 2920 100.0 0.00

Note. 1. Male; 2. Female; Miss: Missing


233



LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.05 32 1.09 -0.03 1.06 32 1.09 -0.03

02 9.05 273 9.27 -0.22 9.17 273 9.29 -0.11

03 12.45 369 12.53 -0.08 12.17 369 12.56 -0.39

04 2.02 61 2.07 -0.05 2.04 61 2.08 -0.04

05 0.70 0 0.00 0.00 0.70 0 0.00 0.00

06 3.29 99 3.36 -0.07 3.32 99 3.37 -0.04

07 1.81 56 1.90 -0.09 1.92 56 1.91 0.02

08 2.96 90 3.06 -0.09 2.96 90 3.06 -0.10

09 0.59 18 0.61 -0.03 0.58 17 0.58 0.00

10 4.85 143 4.86 -0.01 4.69 143 4.87 -0.18

11 0.50 14 0.48 0.02 0.44 14 0.48 -0.04

12 4.42 133 4.52 -0.10 4.48 133 4.53 -0.05

13 6.07 182 6.18 -0.11 6.09 182 6.19 -0.11

14 0.22 7 0.24 -0.02 0.23 7 0.24 -0.01

15 16.71 508 17.26 -0.55 17.25 508 17.28 -0.04

16 14.07 423 14.37 -0.30 14.18 423 14.39 -0.22

17 0.86 26 0.88 -0.02 0.88 26 0.88 -0.01

18 2.07 61 2.07 0.00 2.03 61 2.08 -0.04

19 0.35 10 0.34 0.01 0.35 10 0.34 0.01

20 0.51 0 0.00 0.00 0.50 0 0.00 0.00

21 2.71 60 2.04 0.67 2.69 56 1.91 0.79

22 1.93 58 1.97 -0.04 1.93 58 1.97 -0.05

23 0.85 25 0.85 0.00 0.84 25 0.85 -0.01

24 0.19 5 0.17 0.02 0.13 5 0.17 -0.04

30 9.79 291 9.88 -0.10 9.39 291 9.90 -0.51

Total 100.0 2944 100.0 0.00 100.0 2939 100.0 0.00



234



Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.31 142 4.82 -4.51 0.28 128 4.36 -4.07

2 6.02 187 6.35 -0.34 6.08 203 6.91 -0.82

3 34.42 1048 35.60 -1.18 34.72 1086 36.95 -2.24

4 0.12 12 0.41 -0.29 0.09 12 0.41 -0.32

5 43.12 1370 46.54 -3.41 43.28 1325 45.08 -1.81

6 12.02 185 6.28 5.74 11.52 185 6.29 5.23

Miss 3.99 0 0.00 0.00 4.03 0 0.00 0.00

Total 100.0 2944 100.0 0.00 100.0 2939 100.0 0.00




Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 50.93 1499 50.92 0.01 50.25 1518 51.65 -1.40

2 49.07 1445 49.08 -0.01 49.75 1420 48.32 1.43

Miss 0.00 0 0.00 0.00 0.00 1 0.03 -0.03

Total 100.0 2944 100.0 0.00 100.0 2939 100.0 0.00



235



LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.09 33 1.12 -0.03 1.08 33 1.12 -0.04

02 9.23 278 9.43 -0.20 9.34 278 9.41 -0.07

03 12.48 369 12.52 -0.04 12.13 369 12.50 -0.37

04 2.00 61 2.07 -0.07 2.06 61 2.07 -0.01

05 0.66 20 0.68 -0.02 0.67 20 0.68 -0.01

06 3.34 100 3.39 -0.05 3.35 100 3.39 -0.04

07 1.90 57 1.93 -0.03 1.91 57 1.93 -0.02

08 3.18 94 3.19 -0.01 3.07 94 3.18 -0.12

09 0.51 16 0.54 -0.03 0.53 16 0.54 -0.01

10 4.81 143 4.85 -0.04 4.75 143 4.84 -0.10

11 0.55 15 0.51 0.04 0.48 15 0.51 -0.03

12 4.75 143 4.85 -0.10 4.74 143 4.84 -0.10

13 6.07 182 6.17 -0.11 6.12 182 6.16 -0.04

14 0.26 3 0.10 0.16 0.27 4 0.14 0.14

15 16.77 508 17.23 -0.46 17.18 508 17.20 -0.02

16 14.04 423 14.35 -0.31 14.18 423 14.32 -0.14

17 0.89 27 0.92 -0.03 0.94 27 0.91 0.02

18 1.98 60 2.04 -0.05 2.04 60 2.03 0.01

19 0.32 10 0.34 -0.02 0.34 10 0.34 0.00

20 0.53 0 0.00 0.00 0.52 0 0.00 0.00

21 2.74 51 1.73 1.01 2.84 56 1.90 0.95

22 1.63 50 1.70 -0.07 1.68 50 1.69 -0.02

23 0.71 21 0.71 -0.01 0.71 21 0.71 0.00

24 0.26 6 0.20 0.06 0.10 5 0.17 -0.07

30 9.30 278 9.43 -0.13 8.98 278 9.41 -0.44

Total 100.0 2948 100.0 0.00 100.0 2953 100.0 0.00



236



Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.36 120 4.07 -3.71 0.29 128 4.33 -4.05

2 6.03 169 5.73 0.30 6.11 200 6.77 -0.67

3 34.31 1088 36.91 -2.59 34.87 1062 35.96 -1.10

4 0.10 6 0.20 -0.10 0.12 8 0.27 -0.15

5 43.78 1378 46.74 -2.97 43.94 1379 46.70 -2.76

6 11.61 187 6.34 5.27 10.86 176 5.96 4.90

Miss 3.81 0 0.00 0.00 3.82 0 0.00 0.00

Total 100.0 2948 100.0 0.00 100.0 2953 100.0 0.00




Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 50.90 1493 50.64 0.25 50.84 1527 51.71 -0.87

2 49.10 1451 49.22 -0.12 49.15 1422 48.15 1.00

Miss 0.00 4 0.14 -0.13 0.01 4 0.14 -0.13

Total 100.0 2948 100.0 0.00 100.0 2953 100.0 0.00



237



LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.01 31 1.07 -0.06 1.05 31 1.07 -0.03

02 9.66 279 9.64 0.02 8.93 279 9.66 -0.74

03 12.11 369 12.75 -0.64 12.51 369 12.78 -0.28

04 2.13 62 2.14 -0.01 2.04 62 2.15 -0.11

05 0.58 0 0.00 0.00 0.62 0 0.00 0.00

06 3.24 99 3.42 -0.18 3.41 99 3.43 -0.02

07 1.88 58 2.00 -0.12 2.01 58 2.01 0.00

08 3.07 93 3.21 -0.14 3.13 93 3.22 -0.09

09 0.55 12 0.41 0.14 0.51 10 0.35 0.17

10 4.72 142 4.91 -0.19 4.78 142 4.92 -0.14

11 0.52 5 0.17 0.35 0.53 4 0.14 0.39

12 4.54 139 4.80 -0.26 4.76 139 4.81 -0.06

13 5.99 182 6.29 -0.29 6.18 182 6.30 -0.13

14 0.25 7 0.24 0.01 0.25 7 0.24 0.00

15 16.87 502 17.35 -0.47 16.62 502 17.39 -0.76

16 13.95 420 14.51 -0.56 14.07 420 14.55 -0.48

17 0.98 28 0.97 0.01 0.89 28 0.97 -0.08

18 2.12 62 2.14 -0.03 2.05 62 2.15 -0.09

19 0.34 10 0.35 -0.01 0.34 10 0.35 -0.01

20 0.52 0 0.00 0.00 0.48 0 0.00 0.00

21 2.86 29 1.00 1.86 2.86 25 0.87 2.00

22 1.65 51 1.76 -0.11 1.74 51 1.77 -0.02

23 0.80 24 0.83 -0.03 0.80 24 0.83 -0.03

24 0.37 9 0.31 0.06 0.20 9 0.31 -0.11

30 9.11 277 9.57 -0.46 9.12 277 9.59 -0.47

32 0.15 4 0.14 0.01 0.13 4 0.14 -0.01

Total 100.0 2894 100.0 0.00 100.0 2887 100.0 0.00

Note. 1. Allegany; 2. Anne Arundel; 3. Baltimore; 4. Calvert; 5. Caroline; 6. Carroll; 7. Cecil; 8. Charles; 9. Dorchester; 10. Frederick; 11. Garrett; 12. Harford; 13. Howard; 14. Kent; 15. Montgomery; 16. Prince George’s; 17. Queen Anne’s; 18. St. Mary’s; 19. Somerset; 20. Talbot; 21. Washington; 22. Wicomico; 23. Worcester; 24. LEA 24; 30. Baltimore City; 32. The Seed school


238



Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.28 93 3.21 -2.93 0.25 92 3.19 -2.93

2 5.52 169 5.84 -0.32 5.90 169 5.85 0.05

3 35.45 1055 36.45 -1.01 35.48 1044 36.16 -0.68

4 0.09 14 0.48 -0.39 0.07 7 0.24 -0.17

5 44.09 1380 47.68 -3.59 44.19 1394 48.29 -4.09

6 10.95 183 6.32 4.63 10.58 181 6.27 4.31

Miss 3.61 0 0.00 0.00 3.52 0 0.00 0.00

Total 100.0 2894 100.0 0.00 100.0 2887 100.0 0.00




Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 51.85 1475 50.97 0.88 50.14 1400 48.49 1.65

2 48.14 1416 48.93 -0.79 49.84 1483 51.37 -1.53

Miss 0.01 3 0.10 -0.10 0.02 4 0.14 -0.12

Total 100.0 2894 100.0 0.00 100.0 2887 100.0 0.00



239



LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.07 32 1.11 -0.04 1.09 32 1.11 -0.03

02 9.34 275 9.55 -0.21 9.02 275 9.56 -0.53

03 12.04 365 12.68 -0.63 12.22 365 12.68 -0.46

04 2.20 64 2.22 -0.02 2.07 64 2.22 -0.16

05 0.59 0 0.00 0.00 0.59 0 0.00 0.00

06 3.47 106 3.68 -0.21 3.63 106 3.68 -0.06

07 1.99 60 2.08 -0.09 2.04 60 2.08 -0.04

08 3.35 102 3.54 -0.20 3.48 102 3.54 -0.06

09 0.49 15 0.52 -0.03 0.49 15 0.52 -0.03

10 4.90 147 5.11 -0.21 4.94 147 5.11 -0.17

11 0.52 1 0.03 0.48 0.52 0 0.00 0.00

12 4.64 142 4.93 -0.29 4.86 142 4.93 -0.07

13 6.42 194 6.74 -0.31 6.54 194 6.74 -0.20

14 0.22 0 0.00 0.00 0.22 0 0.00 0.00

15 17.02 503 17.47 -0.45 16.59 503 17.48 -0.88

16 13.78 418 14.52 -0.74 14.14 418 14.52 -0.38

17 0.89 26 0.90 -0.02 0.82 26 0.90 -0.08

18 2.04 60 2.08 -0.04 2.00 60 2.08 -0.09

19 0.28 0 0.00 0.00 0.29 0 0.00 0.00

20 0.53 0 0.00 0.00 0.49 0 0.00 0.00

21 2.72 25 0.87 1.85 2.78 25 0.87 1.91

22 1.54 48 1.67 -0.13 1.64 48 1.67 -0.03

23 0.80 24 0.83 -0.03 0.78 24 0.83 -0.06

24 0.38 9 0.31 0.07 0.18 9 0.31 -0.13

30 8.64 259 9.00 -0.35 8.43 259 9.00 -0.57

32 0.13 4 0.14 -0.01 0.13 4 0.14 0.00

Total 100.0 2879 100.0 0.00 100.0 2878 100.0 0.00

Note. 1. Allegany; 2. Anne Arundel; 3. Baltimore; 4. Calvert; 5. Caroline; 6. Carroll; 7. Cecil; 8. Charles; 9. Dorchester; 10. Frederick; 11. Garrett; 12. Harford; 13. Howard; 14. Kent; 15. Montgomery; 16. Prince George’s; 17. Queen Anne’s; 18. St. Mary’s; 19. Somerset; 20. Talbot; 21. Washington; 22. Wicomico; 23. Worcester; 24. LEA 24; 30. Baltimore City; 32. The Seed school.


240



Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.34 79 2.74 -2.41 0.28 79 2.74 -2.46

2 5.73 215 7.47 -1.74 5.90 204 7.09 -1.18

3 35.49 1018 35.36 0.13 35.46 1017 35.34 0.12

4 0.08 10 0.35 -0.27 0.14 8 0.28 -0.14

5 44.29 1409 48.94 -4.65 44.73 1409 48.96 -4.23

6 10.63 148 5.14 5.49 10.10 161 5.59 4.50

Miss 3.45 0 0.00 0.00 3.40 0 0.00 0.00

Total 100.0 2879 100.0 0.00 100.0 2878 100.0 0.00




Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 51.53 1492 51.82 -0.29 49.83 1470 51.08 -1.25

2 48.45 1382 48.00 0.45 50.16 1403 48.75 1.42

Miss 0.01 5 0.17 -0.16 0.01 5 0.17 -0.17

Total 100.0 2879 100.0 0.00 100.0 2878 100.0 0.00



241



LEA 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

01 1.09 33 1.13 -0.05 1.14 33 1.14 0.01

02 9.35 272 9.35 0.01 8.75 272 9.36 -0.61

03 11.69 357 12.27 -0.58 12.12 357 12.28 -0.16

04 2.23 65 2.23 0.00 2.12 65 2.24 -0.12

05 0.66 20 0.69 -0.03 0.67 20 0.69 -0.02

06 3.42 104 3.57 -0.16 3.54 104 3.58 -0.03

07 1.93 58 1.99 -0.07 1.94 58 2.00 -0.06

08 3.56 108 3.71 -0.15 3.63 108 3.72 -0.09

09 0.61 18 0.62 0.00 0.58 18 0.62 -0.04

10 4.91 146 5.02 -0.11 4.90 146 5.02 -0.13

11 0.49 5 0.17 0.31 0.48 4 0.14 0.34

12 4.70 143 4.91 -0.22 4.83 143 4.92 -0.09

13 6.29 190 6.53 -0.24 6.42 190 6.54 -0.12

14 0.28 0 0.00 0.00 0.27 0 0.00 0.00

15 17.23 509 17.49 -0.26 16.84 509 17.52 -0.68

16 14.16 430 14.78 -0.62 14.51 430 14.80 -0.29

17 0.99 29 1.00 -0.01 0.93 29 1.00 -0.07

18 2.10 63 2.16 -0.06 2.13 63 2.17 -0.04

19 0.31 9 0.31 0.00 0.32 9 0.31 0.01

20 0.49 0 0.00 0.00 0.49 0 0.00 0.00

21 2.81 28 0.96 1.85 2.85 25 0.86 1.99

22 1.59 49 1.68 -0.10 1.70 49 1.69 0.01

23 0.80 24 0.82 -0.03 0.81 24 0.83 -0.02

24 0.42 11 0.38 0.04 0.28 11 0.38 -0.10

30 7.80 236 8.11 -0.31 7.65 236 8.12 -0.48

32 0.11 3 0.10 0.00 0.11 3 0.10 0.00

Total 100.0 2910 100.0 0.00 100.0 2906 100.0 0.00

Note. 1. Allegany; 2. Anne Arundel; 3. Baltimore; 4. Calvert; 5. Caroline; 6. Carroll; 7. Cecil; 8. Charles; 9. Dorchester; 10. Frederick; 11. Garrett; 12. Harford; 13. Howard; 14. Kent; 15. Montgomery; 16. Prince George’s; 17. Queen Anne’s; 18. St. Mary’s; 19. Somerset; 20. Talbot; 21. Washington; 22. Wicomico; 23. Worcester; 24. LEA 24; 30. Baltimore City; 32. The Seed school


242



Race 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 0.29 74 2.54 -2.26 0.26 81 2.79 -2.53

2 5.95 205 7.04 -1.10 5.85 187 6.43 -0.59

3 35.41 1014 34.85 0.57 34.90 999 34.38 0.52

4 0.13 9 0.31 -0.18 0.12 11 0.38 -0.25

5 44.44 1451 49.86 -5.42 45.31 1482 51.00 -5.69

6 10.58 157 5.40 5.18 10.29 146 5.02 5.26

Miss 3.20 0 0.00 0.00 3.27 0 0.00 0.00

Total 100.0 2910 100.0 0.00 100.0 2906 100.0 0.00




Gender 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ. 2011 Pop.

% 2011

S.R.S. % of 2011

S.R.S % of

Differ.

1 51.02 1442 49.55 1.47 50.47 1433 49.31 1.16

2 48.97 1467 50.41 -1.44 49.51 1472 50.65 -1.14

Miss 0.00 1 0.03 -0.03 0.01 1 0.03 -0.02

Total 100.0 2910 100.0 0.00 100.0 2906 100.0 0.00



243

B. APPENDIX B: SCALE SCORE HISTOGRAMS AND TUKEY CHARTS


244

Year 2006 Grade=3

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 95 95 0.16 0.16 250 ‚ 1 96 0.00 0.16 260 ‚ 4 100 0.01 0.17 270 ‚ 10 110 0.02 0.18 280 ‚ 41 151 0.07 0.25 290 ‚ 72 223 0.12 0.37 300 ‚* 145 368 0.24 0.61 310 ‚** 346 714 0.57 1.18 320 ‚*** 534 1248 0.88 2.07 330 ‚***** 916 2164 1.52 3.58 340 ‚****** 1280 3444 2.12 5.70 350 ‚********** 2048 5492 3.39 9.09 360 ‚************ 2468 7960 4.09 13.18 370 ‚****************** 3688 11648 6.11 19.29 380 ‚***************** 3427 15075 5.67 24.96 390 ‚***************************** 5747 20822 9.52 34.48 400 ‚*************************** 5315 26137 8.80 43.28 410 ‚*************************** 5395 31532 8.93 52.22 420 ‚*************************** 5367 36899 8.89 61.10 430 ‚************************* 5016 41915 8.31 69.41 440 ‚*************************** 5384 47299 8.92 78.33 450 ‚******************** 4014 51313 6.65 84.97 460 ‚***************** 3395 54708 5.62 90.59 470 ‚********** 1942 56650 3.22 93.81 480 ‚******** 1520 58170 2.52 96.33 490 ‚****** 1120 59290 1.85 98.18 500 ‚** 355 59645 0.59 98.77 510 ‚** 316 59961 0.52 99.29 520 ‚* 161 60122 0.27 99.56 530 ‚* 148 60270 0.25 99.80 540 ‚ 0 60270 0.00 99.80 550 ‚ 53 60323 0.09 99.89 560 ‚ 48 60371 0.08 99.97 570 ‚ 0 60371 0.00 99.97 580 ‚ 6 60377 0.01 99.98 590 ‚ 11 60388 0.02 100.00 600 ‚ 0 60388 0.00 100.00 610 ‚ 0 60388 0.00 100.00 620 ‚ 0 60388 0.00 100.00 630 ‚ 0 60388 0.00 100.00 640 ‚ 0 60388 0.00 100.00 650 ‚ 0 60388 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒ 1000 2000 3000 4000 5000 Frequency

Figure B.1. Year 2006 Scale Score Distribution: Grade 3


245

Year 2011 Grade=3 Form=A Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 10 10 0.03 0.03 250 ‚ 4 14 0.01 0.05 260 ‚ 8 22 0.03 0.07 270 ‚ 4 26 0.01 0.08 280 ‚ 5 31 0.02 0.10 290 ‚ 14 45 0.05 0.14 300 ‚* 42 87 0.14 0.28 310 ‚* 56 143 0.18 0.46 320 ‚*** 195 338 0.63 1.09 330 ‚*** 222 560 0.71 1.80 340 ‚***** 384 944 1.24 3.04 350 ‚******** 603 1547 1.94 4.98 360 ‚************ 895 2442 2.88 7.86 370 ‚*************** 1129 3571 3.63 11.49 380 ‚********************** 1621 5192 5.22 16.70 390 ‚***************************** 2161 7353 6.95 23.66 400 ‚************************************* 2777 10130 8.93 32.59 410 ‚*********************************** 2633 12763 8.47 41.06 420 ‚**************************************** 3036 15799 9.77 50.83 430 ‚******************************* 2351 18150 7.56 58.39 440 ‚********************************** 2556 20706 8.22 66.62 450 ‚************************************* 2748 23454 8.84 75.46 460 ‚************************************* 2812 26266 9.05 84.51 470 ‚***************** 1273 27539 4.10 88.60 480 ‚*************************** 1988 29527 6.40 95.00 490 ‚********* 709 30236 2.28 97.28 500 ‚ 0 30236 0.00 97.28 510 ‚****** 450 30686 1.45 98.73 520 ‚*** 255 30941 0.82 99.55 530 ‚ 0 30941 0.00 99.55 540 ‚ 0 30941 0.00 99.55 550 ‚* 111 31052 0.36 99.90 560 ‚ 0 31052 0.00 99.90 570 ‚ 0 31052 0.00 99.90 580 ‚ 0 31052 0.00 99.90 590 ‚ 0 31052 0.00 99.90 600 ‚ 0 31052 0.00 99.90 610 ‚ 0 31052 0.00 99.90 620 ‚ 0 31052 0.00 99.90 630 ‚ 0 31052 0.00 99.90 640 ‚ 0 31052 0.00 99.90 650 ‚ 30 31082 0.10 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆ 600 1200 1800 2400 3000 Frequency

Figure B.2. Year 2011 Scale Score Distribution: Grade 3 Form A

Maryland S

Figure B.3Percent D

Figure B.4Cumulativ

School Assessmen

3. Cumulativeifferences bet

4. Cumulativeve Percent Dif

nt-Mathematics:

e Distribution ween CDFs: G

e Distribution fferences betw

: Grades 3 throu

Functions (CDGrade 3 Form

Functions (CDween CDFs: G

ugh 8

246

DFs) for the Ym A

DFs) for the YGrade 3 Form A

Year 2006 vs. Y

Year 2006 vs. YA

2011 Ad

Year 2011 Sca

Year 2011 Sca

dministration

ale Scores with

ale Scores with

h the

h the


247

Year 2011 Grade=3 Form=F

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 8 8 0.03 0.03 250 ‚ 1 9 0.00 0.03 260 ‚ 7 16 0.02 0.05 270 ‚ 1 17 0.00 0.06 280 ‚ 7 24 0.02 0.08 290 ‚ 17 41 0.06 0.14 300 ‚ 32 73 0.11 0.24 310 ‚* 62 135 0.21 0.45 320 ‚** 127 262 0.42 0.87 330 ‚*** 250 512 0.83 1.69 340 ‚**** 271 783 0.90 2.59 350 ‚******* 528 1311 1.75 4.34 360 ‚********* 695 2006 2.30 6.64 370 ‚************** 1073 3079 3.55 10.19 380 ‚******************** 1498 4577 4.96 15.15 390 ‚**************************** 2132 6709 7.06 22.21 400 ‚*************************** 2044 8753 6.77 28.97 410 ‚********************************* 2455 11208 8.13 37.10 420 ‚*************************************** 2935 14143 9.72 46.81 430 ‚********************************************* 3401 17544 11.26 58.07 440 ‚********************************* 2460 20004 8.14 66.21 450 ‚*********************************** 2643 22647 8.75 74.96 460 ‚****************** 1325 23972 4.39 79.35 470 ‚*********************************** 2618 26590 8.67 88.01 480 ‚*************** 1143 27733 3.78 91.80 490 ‚************ 912 28645 3.02 94.82 500 ‚********* 706 29351 2.34 97.15 510 ‚****** 480 29831 1.59 98.74 520 ‚ 0 29831 0.00 98.74 530 ‚*** 247 30078 0.82 99.56 540 ‚ 0 30078 0.00 99.56 550 ‚ 0 30078 0.00 99.56 560 ‚* 94 30172 0.31 99.87 570 ‚ 0 30172 0.00 99.87 580 ‚ 0 30172 0.00 99.87 590 ‚ 0 30172 0.00 99.87 600 ‚ 0 30172 0.00 99.87 610 ‚ 0 30172 0.00 99.87 620 ‚ 0 30172 0.00 99.87 630 ‚ 0 30172 0.00 99.87 640 ‚ 0 30172 0.00 99.87 650 ‚* 39 30211 0.13 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒ 600 1200 1800 2400 3000 Frequency

Figure B.5.Year 2011 Scale Score Distribution: Grade 3 Form F

Maryland S

Figure B.6Percent D

Figure B.7Cumulativ

School Assessmen

6. Cumulativeifferences bet

7. Cumulativeve Percent Dif

nt-Mathematics:

e Distribution ween CDFs: G

e Distribution fferences betw

: Grades 3 throu

Functions (CDGrade 3 Form

Functions (CDween CDFs: G

ugh 8

248

DFs) for the Ym F

DFs) for the YGrade 3 Form F

Year 2006 vs. Y

Year 2006 vs. YF

2011 Ad

Year 2009 Sca

Year 2009 Sca

dministration

ale Scores with

ale Scores with

h the

h the


249

Year 2006 Grade=4

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 91 91 0.15 0.15 250 ‚ 3 94 0.00 0.15 260 ‚ 0 94 0.00 0.15 270 ‚ 6 100 0.01 0.16 280 ‚ 14 114 0.02 0.18 290 ‚ 13 127 0.02 0.21 300 ‚ 48 175 0.08 0.28 310 ‚* 107 282 0.17 0.46 320 ‚** 372 654 0.60 1.06 330 ‚**** 772 1426 1.25 2.31 340 ‚****** 1174 2600 1.90 4.21 350 ‚*********** 2280 4880 3.69 7.90 360 ‚*************** 2953 7833 4.78 12.68 370 ‚***************** 3412 11245 5.52 18.20 380 ‚******************** 4058 15303 6.57 24.77 390 ‚************************ 4779 20082 7.73 32.50 400 ‚************************** 5250 25332 8.50 41.00 410 ‚***************************** 5866 31198 9.49 50.49 420 ‚******************************** 6417 37615 10.39 60.88 430 ‚**************************** 5633 43248 9.12 70.00 440 ‚************************* 5088 48336 8.24 78.23 450 ‚***************** 3460 51796 5.60 83.83 460 ‚***************** 3329 55125 5.39 89.22 470 ‚************** 2812 57937 4.55 93.77 480 ‚******** 1618 59555 2.62 96.39 490 ‚****** 1293 60848 2.09 98.48 500 ‚* 246 61094 0.40 98.88 510 ‚** 409 61503 0.66 99.54 520 ‚* 202 61705 0.33 99.87 530 ‚ 0 61705 0.00 99.87 540 ‚ 0 61705 0.00 99.87 550 ‚ 68 61773 0.11 99.98 560 ‚ 0 61773 0.00 99.98 570 ‚ 12 61785 0.02 100.00 580 ‚ 0 61785 0.00 100.00 590 ‚ 0 61785 0.00 100.00 600 ‚ 0 61785 0.00 100.00 610 ‚ 0 61785 0.00 100.00 620 ‚ 0 61785 0.00 100.00 630 ‚ 0 61785 0.00 100.00 640 ‚ 0 61785 0.00 100.00 650 ‚ 0 61785 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒ 1000 2000 3000 4000 5000 6000 Frequency



250

Year 2011 Grade=4 Form=A

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 5 5 0.02 0.02 250 ‚ 8 13 0.03 0.04 260 ‚ 9 22 0.03 0.07 270 ‚ 8 30 0.03 0.10 280 ‚ 10 40 0.03 0.13 290 ‚ 5 45 0.02 0.15 300 ‚ 12 57 0.04 0.19 310 ‚ 14 71 0.05 0.24 320 ‚* 77 148 0.26 0.49 330 ‚*** 195 343 0.65 1.14 340 ‚**** 307 650 1.02 2.16 350 ‚****** 420 1070 1.40 3.56 360 ‚********** 775 1845 2.58 6.14 370 ‚************** 1030 2875 3.43 9.57 380 ‚****************** 1371 4246 4.56 14.14 390 ‚********************** 1630 5876 5.43 19.56 400 ‚**************************** 2106 7982 7.01 26.57 410 ‚*********************************** 2599 10581 8.65 35.23 420 ‚******************************* 2339 12920 7.79 43.01 430 ‚*********************************************** 3521 16441 11.72 54.74 440 ‚************************* 1900 18341 6.33 61.06 450 ‚****************************************** 3146 21487 10.47 71.54 460 ‚**************************** 2114 23601 7.04 78.57 470 ‚***************************** 2140 25741 7.12 85.70 480 ‚************* 989 26730 3.29 88.99 490 ‚*********************** 1710 28440 5.69 94.68 500 ‚******** 624 29064 2.08 96.76 510 ‚****** 483 29547 1.61 98.37 520 ‚**** 308 29855 1.03 99.39 530 ‚ 0 29855 0.00 99.39 540 ‚** 145 30000 0.48 99.88 550 ‚ 0 30000 0.00 99.88 560 ‚ 0 30000 0.00 99.88 570 ‚ 31 30031 0.10 99.98 580 ‚ 0 30031 0.00 99.98 590 ‚ 0 30031 0.00 99.98 600 ‚ 0 30031 0.00 99.98 610 ‚ 0 30031 0.00 99.98 620 ‚ 0 30031 0.00 99.98 630 ‚ 0 30031 0.00 99.98 640 ‚ 0 30031 0.00 99.98 650 ‚ 6 30037 0.02 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.1Percent D

Figure B.1Cumulativ

School Assessmen

10. Cumulativifferences bet

11. Cumulativve Percent Dif

nt-Mathematics:

ve Distributionween CDFs: G

ve Distributionfferences betw

: Grades 3 throu

n Functions (CGrade 4 Form

n Functions (Cween CDFs: G

ugh 8

251

CDFs) for the m A

CDFs) for the Grade 4 Form A

Year 2006 vs.

Year 2006 vs.A

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


252

Year 2011 Grade=4 Form=F Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 2 2 0.01 0.01 250 ‚ 7 9 0.02 0.03 260 ‚ 6 15 0.02 0.05 270 ‚ 4 19 0.01 0.07 280 ‚ 4 23 0.01 0.08 290 ‚ 2 25 0.01 0.09 300 ‚ 7 32 0.02 0.11 310 ‚ 16 48 0.06 0.17 320 ‚* 72 120 0.25 0.41 330 ‚** 121 241 0.42 0.83 340 ‚*** 233 474 0.80 1.63 350 ‚**** 332 806 1.14 2.78 360 ‚******** 576 1382 1.98 4.76 370 ‚*********** 853 2235 2.94 7.70 380 ‚************** 1078 3313 3.71 11.41 390 ‚******************** 1498 4811 5.16 16.57 400 ‚************************** 1940 6751 6.68 23.26 410 ‚******************************** 2365 9116 8.15 31.41 420 ‚*************************************** 2909 12025 10.02 41.43 430 ‚********************************* 2477 14502 8.53 49.96 440 ‚************************************** 2877 17379 9.91 59.87 450 ‚*************************** 2048 19427 7.06 66.93 460 ‚******************************************** 3304 22731 11.38 78.31 470 ‚************** 1083 23814 3.73 82.04 480 ‚***************************** 2142 25956 7.38 89.42 490 ‚************* 945 26901 3.26 92.68 500 ‚********** 771 27672 2.66 95.33 510 ‚********* 639 28311 2.20 97.53 520 ‚***** 407 28718 1.40 98.94 530 ‚ 0 28718 0.00 98.94 540 ‚*** 210 28928 0.72 99.66 550 ‚ 0 28928 0.00 99.66 560 ‚* 79 29007 0.27 99.93 570 ‚ 0 29007 0.00 99.93 580 ‚ 0 29007 0.00 99.93 590 ‚ 0 29007 0.00 99.93 600 ‚ 0 29007 0.00 99.93 610 ‚ 0 29007 0.00 99.93 620 ‚ 0 29007 0.00 99.93 630 ‚ 0 29007 0.00 99.93 640 ‚ 0 29007 0.00 99.93 650 ‚ 20 29027 0.07 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒ 600 1200 1800 2400 3000 Frequency

Figure B.12. Year 2011 Scale Score Distribution: Grade 4 Form F

Maryland S

Figure B.1Percent D

Figure B.1Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

253

CDFs) for the m F

CDFs) for the Grade 4 Form F

Year 2006 vs.

Year 2006 vs.F

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


254

Year 2006 Grade=5

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 78 78 0.12 0.12 250 ‚ 3 81 0.00 0.13 260 ‚ 2 83 0.00 0.13 270 ‚ 2 85 0.00 0.13 280 ‚ 4 89 0.01 0.14 290 ‚ 2 91 0.00 0.14 300 ‚ 19 110 0.03 0.17 310 ‚ 38 148 0.06 0.23 320 ‚ 94 242 0.15 0.38 330 ‚** 339 581 0.53 0.92 340 ‚**** 720 1301 1.13 2.05 350 ‚****** 1168 2469 1.84 3.89 360 ‚*********** 2214 4683 3.49 7.38 370 ‚******************** 3926 8609 6.19 13.56 380 ‚******************** 3956 12565 6.23 19.80 390 ‚***************************** 5734 18299 9.03 28.83 400 ‚************************* 5089 23388 8.02 36.85 410 ‚************************************ 7170 30558 11.30 48.14 420 ‚****************************** 6063 36621 9.55 57.70 430 ‚******************************** 6337 42958 9.98 67.68 440 ‚*********************** 4657 47615 7.34 75.02 450 ‚********************** 4491 52106 7.08 82.09 460 ‚***************** 3410 55516 5.37 87.46 470 ‚***************** 3366 58882 5.30 92.77 480 ‚******* 1493 60375 2.35 95.12 490 ‚********* 1749 62124 2.76 97.87 500 ‚*** 595 62719 0.94 98.81 510 ‚** 404 63123 0.64 99.45 520 ‚* 229 63352 0.36 99.81 530 ‚ 0 63352 0.00 99.81 540 ‚ 96 63448 0.15 99.96 550 ‚ 0 63448 0.00 99.96 560 ‚ 20 63468 0.03 99.99 570 ‚ 0 63468 0.00 99.99 580 ‚ 0 63468 0.00 99.99 590 ‚ 5 63473 0.01 100.00 600 ‚ 0 63473 0.00 100.00 610 ‚ 0 63473 0.00 100.00 620 ‚ 0 63473 0.00 100.00 630 ‚ 0 63473 0.00 100.00 640 ‚ 0 63473 0.00 100.00 650 ‚ 0 63473 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒ 1000 2000 3000 4000 5000 6000 7000 Frequency



255

Year 2011 Grade=5 Form=A Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 4 4 0.01 0.01 250 ‚ 0 4 0.00 0.01 260 ‚ 0 4 0.00 0.01 270 ‚ 5 9 0.02 0.03 280 ‚ 4 13 0.01 0.04 290 ‚ 5 18 0.02 0.06 300 ‚ 5 23 0.02 0.08 310 ‚ 10 33 0.03 0.11 320 ‚ 15 48 0.05 0.16 330 ‚* 75 123 0.25 0.41 340 ‚** 152 275 0.50 0.91 350 ‚**** 304 579 1.00 1.91 360 ‚********* 705 1284 2.32 4.23 370 ‚************** 1044 2328 3.44 7.67 380 ‚************************ 1828 4156 6.02 13.69 390 ‚************************** 1948 6104 6.42 20.11 400 ‚************************************* 2800 8904 9.22 29.33 410 ‚******************************************** 3303 12207 10.88 40.22 420 ‚************************************* 2760 14967 9.09 49.31 430 ‚*************************************** 2913 17880 9.60 58.90 440 ‚**************************************** 3005 20885 9.90 68.80 450 ‚******************************** 2369 23254 7.80 76.61 460 ‚******************************** 2377 25631 7.83 84.44 470 ‚********************* 1554 27185 5.12 89.56 480 ‚******************* 1444 28629 4.76 94.32 490 ‚******** 567 29196 1.87 96.19 500 ‚*********** 796 29992 2.62 98.81 510 ‚ 0 29992 0.00 98.81 520 ‚*** 214 30206 0.71 99.51 530 ‚* 102 30308 0.34 99.85 540 ‚ 0 30308 0.00 99.85 550 ‚* 40 30348 0.13 99.98 560 ‚ 0 30348 0.00 99.98 570 ‚ 0 30348 0.00 99.98 580 ‚ 0 30348 0.00 99.98 590 ‚ 0 30348 0.00 99.98 600 ‚ 0 30348 0.00 99.98 610 ‚ 0 30348 0.00 99.98 620 ‚ 0 30348 0.00 99.98 630 ‚ 0 30348 0.00 99.98 640 ‚ 0 30348 0.00 99.98 650 ‚ 6 30354 0.02 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.1Percent D

Figure B.1Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

256

CDFs) for the m A


Year 2006 vs.

Year 2006 vs.A

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


257

Year 2011 Grade=5 Form=F Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 3 3 0.01 0.01 250 ‚ 4 7 0.01 0.02 260 ‚ 0 7 0.00 0.02 270 ‚ 4 11 0.01 0.04 280 ‚ 5 16 0.02 0.05 290 ‚ 9 25 0.03 0.09 300 ‚ 3 28 0.01 0.10 310 ‚ 15 43 0.05 0.15 320 ‚ 13 56 0.04 0.19 330 ‚* 47 103 0.16 0.35 340 ‚* 98 201 0.33 0.69 350 ‚*** 256 457 0.87 1.56 360 ‚******* 522 979 1.78 3.34 370 ‚*********** 806 1785 2.75 6.09 380 ‚******************** 1475 3260 5.04 11.13 390 ‚***************************** 2190 5450 7.48 18.61 400 ‚************************************* 2772 8222 9.46 28.07 410 ‚*********************************** 2648 10870 9.04 37.11 420 ‚************************************************* 3641 14511 12.43 49.54 430 ‚***************************************** 3102 17613 10.59 60.13 440 ‚********************************* 2464 20077 8.41 68.54 450 ‚******************************************** 3315 23392 11.32 79.86 460 ‚********************* 1574 24966 5.37 85.23 470 ‚******************** 1514 26480 5.17 90.40 480 ‚****************** 1340 27820 4.57 94.97 490 ‚******* 497 28317 1.70 96.67 500 ‚****** 420 28737 1.43 98.10 510 ‚**** 280 29017 0.96 99.06 520 ‚** 172 29189 0.59 99.64 530 ‚ 0 29189 0.00 99.64 540 ‚* 85 29274 0.29 99.94 550 ‚ 0 29274 0.00 99.94 560 ‚ 0 29274 0.00 99.94 570 ‚ 0 29274 0.00 99.94 580 ‚ 0 29274 0.00 99.94 590 ‚ 0 29274 0.00 99.94 600 ‚ 0 29274 0.00 99.94 610 ‚ 0 29274 0.00 99.94 620 ‚ 0 29274 0.00 99.94 630 ‚ 0 29274 0.00 99.94 640 ‚ 0 29274 0.00 99.94 650 ‚ 19 29293 0.06 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒ 600 1200 1800 2400 3000 3600 Frequency


Maryland S

Figure B.2Percent D

Figure B.2Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

258

CDFs) for the m F


Year 2006 vs.

Year 2006 vs.F

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


259

Year 2006 Grade=6

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚** 304 304 0.47 0.47 250 ‚ 0 304 0.00 0.47 260 ‚ 16 320 0.02 0.49 270 ‚ 0 320 0.00 0.49 280 ‚ 6 326 0.01 0.50 290 ‚ 11 337 0.02 0.52 300 ‚ 34 371 0.05 0.57 310 ‚ 23 394 0.04 0.61 320 ‚* 138 532 0.21 0.82 330 ‚** 307 839 0.47 1.30 340 ‚***** 1012 1851 1.56 2.86 350 ‚********** 1915 3766 2.96 5.82 360 ‚***************** 3410 7176 5.27 11.08 370 ‚******************** 3945 11121 6.09 17.18 380 ‚********************* 4214 15335 6.51 23.68 390 ‚****************************** 5929 21264 9.16 32.84 400 ‚******************************** 6471 27735 9.99 42.83 410 ‚******************************** 6329 34064 9.77 52.61 420 ‚************************************ 7130 41194 11.01 63.62 430 ‚***************************** 5711 46905 8.82 72.44 440 ‚************************* 5036 51941 7.78 80.22 450 ‚*********************** 4587 56528 7.08 87.30 460 ‚**************** 3173 59701 4.90 92.20 470 ‚************* 2510 62211 3.88 96.08 480 ‚**** 829 63040 1.28 97.36 490 ‚****** 1147 64187 1.77 99.13 500 ‚** 321 64508 0.50 99.63 510 ‚* 152 64660 0.23 99.86 520 ‚ 29 64689 0.04 99.91 530 ‚ 42 64731 0.06 99.97 540 ‚ 0 64731 0.00 99.97 550 ‚ 8 64739 0.01 99.98 560 ‚ 10 64749 0.02 100.00 570 ‚ 0 64749 0.00 100.00 580 ‚ 1 64750 0.00 100.00 590 ‚ 0 64750 0.00 100.00 600 ‚ 0 64750 0.00 100.00 610 ‚ 0 64750 0.00 100.00 620 ‚ 0 64750 0.00 100.00 630 ‚ 0 64750 0.00 100.00 640 ‚ 0 64750 0.00 100.00 650 ‚ 0 64750 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒ 1000 2000 3000 4000 5000 6000 7000 Frequency



260

Year 2011 Grade=6 Form=A Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 2 2 0.01 0.01 250 ‚ 0 2 0.00 0.01 260 ‚ 0 2 0.00 0.01 270 ‚ 5 7 0.02 0.02 280 ‚ 0 7 0.00 0.02 290 ‚ 7 14 0.02 0.05 300 ‚ 2 16 0.01 0.05 310 ‚ 6 22 0.02 0.07 320 ‚ 3 25 0.01 0.08 330 ‚ 15 40 0.05 0.13 340 ‚* 60 100 0.20 0.34 350 ‚** 127 227 0.43 0.76 360 ‚******* 516 743 1.73 2.50 370 ‚************** 1017 1760 3.42 5.91 380 ‚******************** 1506 3266 5.06 10.97 390 ‚******************************** 2385 5651 8.01 18.98 400 ‚************************************* 2783 8434 9.35 28.32 410 ‚***************************************** 3084 11518 10.36 38.68 420 ‚********************************** 2542 14060 8.54 47.22 430 ‚****************************************** 3179 17239 10.68 57.89 440 ‚********************************** 2579 19818 8.66 66.55 450 ‚*************************** 2020 21838 6.78 73.34 460 ‚*************************** 2028 23866 6.81 80.15 470 ‚**************************** 2092 25958 7.03 87.17 480 ‚****************** 1333 27291 4.48 91.65 490 ‚**************** 1196 28487 4.02 95.66 500 ‚****** 466 28953 1.56 97.23 510 ‚***** 378 29331 1.27 98.50 520 ‚**** 264 29595 0.89 99.39 530 ‚ 0 29595 0.00 99.39 540 ‚ 0 29595 0.00 99.39 550 ‚** 139 29734 0.47 99.85 560 ‚ 0 29734 0.00 99.85 570 ‚ 0 29734 0.00 99.85 580 ‚ 0 29734 0.00 99.85 590 ‚ 0 29734 0.00 99.85 600 ‚ 0 29734 0.00 99.85 610 ‚ 0 29734 0.00 99.85 620 ‚ 0 29734 0.00 99.85 630 ‚ 0 29734 0.00 99.85 640 ‚ 0 29734 0.00 99.85 650 ‚* 44 29778 0.15 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.2Percent D

Figure B.2Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

261

CDFs) for the m A


Year 2006 vs.

Year 2006 vs.A

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


262


Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 2 2 0.01 0.01 250 ‚ 0 2 0.00 0.01 260 ‚ 0 2 0.00 0.01 270 ‚ 0 2 0.00 0.01 280 ‚ 0 2 0.00 0.01 290 ‚ 0 2 0.00 0.01 300 ‚ 4 6 0.01 0.02 310 ‚ 3 9 0.01 0.03 320 ‚ 6 15 0.02 0.05 330 ‚ 11 26 0.04 0.09 340 ‚ 30 56 0.11 0.20 350 ‚* 95 151 0.33 0.53 360 ‚***** 411 562 1.44 1.97 370 ‚*********** 803 1365 2.81 4.78 380 ‚***************** 1287 2652 4.51 9.30 390 ‚***************************** 2147 4799 7.53 16.82 400 ‚********************************** 2541 7340 8.91 25.73 410 ‚************************************** 2876 10216 10.08 35.81 420 ‚**************************************** 2998 13214 10.51 46.31 430 ‚***************************************** 3112 16326 10.91 57.22 440 ‚********************************* 2467 18793 8.65 65.87 450 ‚************************** 1947 20740 6.82 72.69 460 ‚*************************** 2051 22791 7.19 79.88 470 ‚*************************** 2057 24848 7.21 87.09 480 ‚********** 715 25563 2.51 89.60 490 ‚***************** 1302 26865 4.56 94.16 500 ‚******* 509 27374 1.78 95.94 510 ‚******* 488 27862 1.71 97.66 520 ‚***** 375 28237 1.31 98.97 530 ‚ 0 28237 0.00 98.97 540 ‚ 0 28237 0.00 98.97 550 ‚*** 218 28455 0.76 99.73 560 ‚ 0 28455 0.00 99.73 570 ‚ 0 28455 0.00 99.73 580 ‚ 0 28455 0.00 99.73 590 ‚ 0 28455 0.00 99.73 600 ‚ 0 28455 0.00 99.73 610 ‚ 0 28455 0.00 99.73 620 ‚ 0 28455 0.00 99.73 630 ‚ 0 28455 0.00 99.73 640 ‚ 0 28455 0.00 99.73 650 ‚* 76 28531 0.27 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.2Percent D

Figure B.2Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

263

CDFs) for the m F


Year 2006 vs.

Year 2006 vs.F

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


264

Year 2006 Grade=7

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 0 0 0.00 0.00 250 ‚* 259 259 0.39 0.39 260 ‚* 185 444 0.28 0.67 270 ‚ 23 467 0.03 0.71 280 ‚ 12 479 0.02 0.73 290 ‚ 28 507 0.04 0.77 300 ‚ 25 532 0.04 0.81 310 ‚ 61 593 0.09 0.90 320 ‚* 246 839 0.37 1.27 330 ‚** 409 1248 0.62 1.90 340 ‚******* 1306 2554 1.98 3.88 350 ‚************** 2765 5319 4.20 8.08 360 ‚******************** 3979 9298 6.04 14.12 370 ‚************************** 5116 14414 7.77 21.90 380 ‚***************************** 5818 20232 8.84 30.73 390 ‚***************************** 5891 26123 8.95 39.68 400 ‚***************************** 5703 31826 8.66 48.35 410 ‚***************************** 5760 37586 8.75 57.10 420 ‚**************************** 5606 43192 8.52 65.61 430 ‚********************** 4488 47680 6.82 72.43 440 ‚************************ 4852 52532 7.37 79.80 450 ‚******************** 3994 56526 6.07 85.87 460 ‚**************** 3202 59728 4.86 90.73 470 ‚************ 2372 62100 3.60 94.34 480 ‚******** 1665 63765 2.53 96.86 490 ‚***** 1008 64773 1.53 98.40 500 ‚** 436 65209 0.66 99.06 510 ‚** 327 65536 0.50 99.55 520 ‚* 174 65710 0.26 99.82 530 ‚ 82 65792 0.12 99.94 540 ‚ 0 65792 0.00 99.94 550 ‚ 21 65813 0.03 99.98 560 ‚ 10 65823 0.02 99.99 570 ‚ 5 65828 0.01 100.00 580 ‚ 1 65829 0.00 100.00 590 ‚ 0 65829 0.00 100.00 600 ‚ 0 65829 0.00 100.00 610 ‚ 0 65829 0.00 100.00 620 ‚ 0 65829 0.00 100.00 630 ‚ 0 65829 0.00 100.00 640 ‚ 0 65829 0.00 100.00 650 ‚ 0 65829 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒ 1000 2000 3000 4000 5000 Frequency



265


Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 6 6 0.02 0.02 250 ‚ 0 6 0.00 0.02 260 ‚ 0 6 0.00 0.02 270 ‚ 0 6 0.00 0.02 280 ‚ 10 16 0.03 0.05 290 ‚ 0 16 0.00 0.05 300 ‚ 8 24 0.03 0.08 310 ‚ 19 43 0.06 0.14 320 ‚ 14 57 0.05 0.19 330 ‚** 97 154 0.32 0.51 340 ‚**** 212 366 0.70 1.21 350 ‚************** 700 1066 2.32 3.53 360 ‚******************** 1000 2066 3.31 6.85 370 ‚************************ 1194 3260 3.96 10.80 380 ‚************************************* 1830 5090 6.06 16.87 390 ‚****************************************************** 2700 7790 8.95 25.81 400 ‚********************************************************* 2856 10646 9.46 35.28 410 ‚************************************************ 2396 13042 7.94 43.21 420 ‚*********************************************************** 2965 16007 9.82 53.04 430 ‚*********************************************************** 2967 18974 9.83 62.87 440 ‚************************************************* 2437 21411 8.07 70.94 450 ‚*********************************************** 2350 23761 7.79 78.73 460 ‚************************************* 1827 25588 6.05 84.78 470 ‚************************ 1204 26792 3.99 88.77 480 ‚******************************** 1586 28378 5.26 94.03 490 ‚****************** 912 29290 3.02 97.05 500 ‚******* 364 29654 1.21 98.26 510 ‚**** 221 29875 0.73 98.99 520 ‚*** 158 30033 0.52 99.51 530 ‚** 94 30127 0.31 99.82 540 ‚* 36 30163 0.12 99.94 550 ‚ 0 30163 0.00 99.94 560 ‚ 15 30178 0.05 99.99 570 ‚ 0 30178 0.00 99.99 580 ‚ 0 30178 0.00 99.99 590 ‚ 2 30180 0.01 100.00 600 ‚ 0 30180 0.00 100.00 610 ‚ 0 30180 0.00 100.00 620 ‚ 0 30180 0.00 100.00 630 ‚ 0 30180 0.00 100.00 640 ‚ 0 30180 0.00 100.00 650 ‚ 0 30180 0.00 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ 400 800 1200 1600 2000 2400 2800 Frequency


Maryland S

Figure B.3Percent D

Figure B.3Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

266

CDFs) for the m A


Year 2006 vs.

Year 2006 vs.A

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


267


Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 3 3 0.01 0.01 250 ‚ 0 3 0.00 0.01 260 ‚ 0 3 0.00 0.01 270 ‚ 0 3 0.00 0.01 280 ‚ 4 7 0.01 0.02 290 ‚ 0 7 0.00 0.02 300 ‚ 3 10 0.01 0.03 310 ‚ 6 16 0.02 0.05 320 ‚ 9 25 0.03 0.09 330 ‚* 41 66 0.14 0.23 340 ‚** 114 180 0.39 0.62 350 ‚***** 247 427 0.85 1.47 360 ‚********************* 1025 1452 3.52 4.99 370 ‚********************** 1094 2546 3.76 8.75 380 ‚*********************************** 1765 4311 6.07 14.81 390 ‚***************************************** 2055 6366 7.06 21.88 400 ‚********************************************************* 2873 9239 9.87 31.75 410 ‚*********************************************************** 2938 12177 10.10 41.85 420 ‚******************************************************* 2773 14950 9.53 51.37 430 ‚*********************************************** 2329 17279 8.00 59.38 440 ‚********************************************************** 2909 20188 10.00 69.37 450 ‚******************************************** 2219 22407 7.63 77.00 460 ‚********************************** 1686 24093 5.79 82.79 470 ‚******************************** 1624 25717 5.58 88.37 480 ‚****************************** 1492 27209 5.13 93.50 490 ‚**************** 814 28023 2.80 96.30 500 ‚************* 636 28659 2.19 98.48 510 ‚**** 214 28873 0.74 99.22 520 ‚** 105 28978 0.36 99.58 530 ‚* 73 29051 0.25 99.83 540 ‚* 35 29086 0.12 99.95 550 ‚ 0 29086 0.00 99.95 560 ‚ 13 29099 0.04 100.00 570 ‚ 0 29099 0.00 100.00 580 ‚ 0 29099 0.00 100.00 590 ‚ 0 29099 0.00 100.00 600 ‚ 0 29099 0.00 100.00 610 ‚ 0 29099 0.00 100.00 620 ‚ 0 29099 0.00 100.00 630 ‚ 0 29099 0.00 100.00 640 ‚ 0 29099 0.00 100.00 650 ‚ 1 29100 0.00 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ 400 800 1200 1600 2000 2400 2800 Frequency


Maryland S

Figure B.3Percent D

Figure B.3Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

268

CDFs) for the m F


Year 2006 vs.

Year 2006 vs.F

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


269

Grade=8

Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 0 0 0.00 0.00 250 ‚ 0 0 0.00 0.00 260 ‚** 452 452 0.67 0.67 270 ‚ 0 452 0.00 0.67 280 ‚ 22 474 0.03 0.70 290 ‚ 0 474 0.00 0.70 300 ‚ 23 497 0.03 0.73 310 ‚ 51 548 0.08 0.81 320 ‚ 59 607 0.09 0.90 330 ‚* 148 755 0.22 1.11 340 ‚** 391 1146 0.58 1.69 350 ‚**** 896 2042 1.32 3.01 360 ‚************* 2619 4661 3.87 6.88 370 ‚********************** 4472 9133 6.60 13.48 380 ‚****************************** 6082 15215 8.98 22.46 390 ‚******************************* 6178 21393 9.12 31.58 400 ‚************************************** 7593 28986 11.21 42.79 410 ‚*********************************** 6977 35963 10.30 53.09 420 ‚*********************************** 6905 42868 10.19 63.28 430 ‚************************* 5028 47896 7.42 70.71 440 ‚************************ 4862 52758 7.18 77.88 450 ‚******************* 3710 56468 5.48 83.36 460 ‚**************** 3275 59743 4.83 88.19 470 ‚*************** 3027 62770 4.47 92.66 480 ‚********** 2064 64834 3.05 95.71 490 ‚****** 1130 65964 1.67 97.38 500 ‚***** 935 66899 1.38 98.76 510 ‚** 327 67226 0.48 99.24 520 ‚* 232 67458 0.34 99.58 530 ‚* 166 67624 0.25 99.83 540 ‚ 35 67659 0.05 99.88 550 ‚ 39 67698 0.06 99.94 560 ‚ 18 67716 0.03 99.96 570 ‚ 16 67732 0.02 99.99 580 ‚ 4 67736 0.01 99.99 590 ‚ 0 67736 0.00 99.99 600 ‚ 4 67740 0.01 100.00 610 ‚ 0 67740 0.00 100.00 620 ‚ 0 67740 0.00 100.00 630 ‚ 0 67740 0.00 100.00 640 ‚ 0 67740 0.00 100.00 650 ‚ 0 67740 0.00 100.00 ‚ Šƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒƒˆƒƒƒ 1000 2000 3000 4000 5000 6000 7000 Frequency



270


Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 6 6 0.02 0.02 250 ‚ 0 6 0.00 0.02 260 ‚ 0 6 0.00 0.02 270 ‚ 0 6 0.00 0.02 280 ‚ 0 6 0.00 0.02 290 ‚ 13 19 0.04 0.06 300 ‚ 0 19 0.00 0.06 310 ‚ 22 41 0.07 0.14 320 ‚ 15 56 0.05 0.19 330 ‚ 31 87 0.10 0.29 340 ‚* 100 187 0.33 0.62 350 ‚**** 272 459 0.90 1.53 360 ‚************ 894 1353 2.97 4.50 370 ‚****************** 1350 2703 4.49 8.98 380 ‚*********************** 1695 4398 5.63 14.62 390 ‚******************************** 2407 6805 8.00 22.62 400 ‚***************************************** 3060 9865 10.17 32.79 410 ‚*************************************** 2896 12761 9.62 42.41 420 ‚************************************* 2787 15548 9.26 51.67 430 ‚********************************** 2563 18111 8.52 60.19 440 ‚********************************* 2494 20605 8.29 68.48 450 ‚************************* 1873 22478 6.22 74.71 460 ‚****************************** 2229 24707 7.41 82.11 470 ‚****************** 1325 26032 4.40 86.52 480 ‚***************** 1243 27275 4.13 90.65 490 ‚**************** 1180 28455 3.92 94.57 500 ‚******** 637 29092 2.12 96.69 510 ‚*** 258 29350 0.86 97.54 520 ‚****** 418 29768 1.39 98.93 530 ‚** 156 29924 0.52 99.45 540 ‚ 0 29924 0.00 99.45 550 ‚* 91 30015 0.30 99.75 560 ‚ 0 30015 0.00 99.75 570 ‚* 63 30078 0.21 99.96 580 ‚ 0 30078 0.00 99.96 590 ‚ 0 30078 0.00 99.96 600 ‚ 0 30078 0.00 99.96 610 ‚ 0 30078 0.00 99.96 620 ‚ 0 30078 0.00 99.96 630 ‚ 0 30078 0.00 99.96 640 ‚ 0 30078 0.00 99.96 650 ‚ 11 30089 0.04 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.3Percent D

Figure B.3Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

271

CDFs) for the m A


Year 2006 vs.

Year 2006 vs.A

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


272


Scale Score Cum. Cum. Midpoint Freq Freq Percent Percent ‚ 240 ‚ 0 0 0.00 0.00 250 ‚ 0 0 0.00 0.00 260 ‚ 0 0 0.00 0.00 270 ‚ 0 0 0.00 0.00 280 ‚ 0 0 0.00 0.00 290 ‚ 4 4 0.01 0.01 300 ‚ 0 4 0.00 0.01 310 ‚ 3 7 0.01 0.02 320 ‚ 9 16 0.03 0.05 330 ‚ 10 26 0.03 0.09 340 ‚ 19 45 0.07 0.15 350 ‚** 126 171 0.43 0.59 360 ‚******* 496 667 1.70 2.29 370 ‚************* 993 1660 3.41 5.70 380 ‚***************** 1312 2972 4.51 10.21 390 ‚***************************** 2158 5130 7.41 17.62 400 ‚***************************************** 3048 8178 10.47 28.09 410 ‚****************************************** 3182 11360 10.93 39.02 420 ‚*************************************** 2903 14263 9.97 48.99 430 ‚*********************************** 2627 16890 9.02 58.01 440 ‚**************************************** 2991 19881 10.27 68.29 450 ‚************************* 1838 21719 6.31 74.60 460 ‚**************************** 2108 23827 7.24 81.84 470 ‚***************** 1287 25114 4.42 86.26 480 ‚**************** 1213 26327 4.17 90.43 490 ‚************** 1047 27374 3.60 94.02 500 ‚********* 662 28036 2.27 96.30 510 ‚**** 305 28341 1.05 97.34 520 ‚****** 463 28804 1.59 98.94 530 ‚** 151 28955 0.52 99.45 540 ‚ 0 28955 0.00 99.45 550 ‚* 89 29044 0.31 99.76 560 ‚ 0 29044 0.00 99.76 570 ‚* 57 29101 0.20 99.96 580 ‚ 0 29101 0.00 99.96 590 ‚ 0 29101 0.00 99.96 600 ‚ 0 29101 0.00 99.96 610 ‚ 0 29101 0.00 99.96 620 ‚ 0 29101 0.00 99.96 630 ‚ 0 29101 0.00 99.96 640 ‚ 0 29101 0.00 99.96 650 ‚ 13 29114 0.04 100.00 ‚ Šƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒ 600 1200 1800 2400 3000 Frequency


Maryland S

Figure B.4Percent D

Figure B.4Cumulativ

School Assessmen



nt-Mathematics:



: Grades 3 throu



ugh 8

273

CDFs) for the m F


Year 2006 vs.

Year 2006 vs.F

2011 Ad

. Year 2011 Sc

. Year 2011 Sc

dministration

cale Scores wi

cale Scores wi

ith the

ith the


274

C. APPENDIX C: THE 2011 MSA-MATH CLASSICAL AND RASCH ITEM PARAMETERS


275

Table C.1. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 3 Form A

Item CID Item Type P-Value

Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

3488069 SR 0.90 0.26 -0.5610 0.0674 1.05 1.27

3548059 SR 0.78 0.54 0.6288 0.0501 0.84 0.70

100000011186 BCR 0.74 0.55 0.8674 0.0482 0.86 0.72

3595529 BCR 0.61 0.39 1.1352 0.0395 1.10 1.11 -2.2067 2.2067

100000067832 SR 0.59 0.54 1.8287 0.0432 0.83 0.78

100000044159 SR 0.62 0.25 1.5712 0.0440 1.24 1.34

100000004268 SR 0.52 0.40 2.0124 0.0428 1.02 1.06

100000018407 SR 0.96 0.22 -1.7879 0.1047 1.01 1.10

100000366020 SR 0.68 0.46 1.3184 0.0453 0.96 0.90

3510017 SR 0.94 0.33 -1.1315 0.0822 0.96 0.96

3509926 SR 0.50 0.44 2.4187 0.0429 0.98 0.99

3509960 SR 0.82 0.30 0.3981 0.0529 1.09 1.14

100000044152 SR 0.88 0.33 -0.3864 0.0649 1.05 0.98

100000025225 SR 0.85 0.36 -0.2038 0.0615 1.12 1.38

3510022 SR 0.57 0.31 2.0077 0.0428 1.13 1.18

3509983 SR 0.92 0.39 -1.1470 0.0824 1.08 1.25

3509941 BCR 0.65 0.44 1.4057 0.0448 1.00 1.10

3595501 BCR 0.56 0.49 1.7804 0.0376 0.96 0.95 -1.9419 1.9419

3509955 SR 0.62 0.34 1.8411 0.0432 1.11 1.18

100000246319 SR 0.72 0.49 1.0140 0.0471 0.92 0.83

100000004272 SR 0.90 0.39 -0.7315 0.0716 0.97 0.92

3509957 BCR 0.79 0.29 0.4850 0.0519 1.16 1.26

3564081 BCR 0.46 0.47 2.4555 0.0360 1.02 1.02 -1.7392 1.7392

3497891 SR 0.43 0.33 2.6221 0.0429 1.09 1.25

100000246267 SR 0.88 0.43 -0.4145 0.0652 0.93 0.73

100000246315 SR 0.86 0.53 -0.1400 0.0603 0.86 0.67

3548507 SR 0.89 0.32 -0.2784 0.0627 0.92 0.89

3509988 SR 0.75 0.41 0.5005 0.0518 1.08 1.13

3488123 SR 0.57 0.43 1.3765 0.0453 1.11 1.14

3510067 BCR 0.83 0.40 0.1409 0.0557 1.03 1.01

3564083 BCR 0.81 0.40 0.4472 0.0367 1.31 1.51 -0.3188 0.3188

3488171 SR 0.74 0.50 0.7136 0.0494 0.97 0.85

3509940 BCR 0.73 0.55 0.7796 0.0500 0.89 0.76

584928 BCR 0.61 0.49 1.0594 0.0406 0.92 0.92 -2.3505 2.3505

100000063519 SR 0.85 0.35 -0.0768 0.0589 1.08 1.23

100000067805 SR 0.86 0.47 -0.0370 0.0584 0.90 0.75

100000080018 SR 0.76 0.34 0.6848 0.0499 1.12 1.20


276

Table C.1 (continued)


Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

100000247326 SR 0.86 0.36 -0.1847 0.0610 1.01 1.17

3510065 SR 0.95 0.28 -2.1822 0.1247 1.74 1.91

3510005 SR 0.62 0.45 1.3897 0.0449 1.04 1.05

100000233317 SR 0.98 0.24 -3.1024 0.1866 0.96 0.60

3510027 SR 0.90 0.36 -0.5906 0.0683 1.00 0.95

3510035 SR 0.89 0.30 -0.6165 0.0689 1.10 1.65

3509978 BCR 0.62 0.44 1.4993 0.0442 1.00 1.03

3985610 BCR 0.50 0.35 2.1074 0.0412 1.12 1.14 -2.3088 2.3088

100000025208 SR 0.90 0.43 -0.6075 0.0687 0.91 0.83

3510058 SR 0.90 0.42 -0.6059 0.0687 0.94 1.17

3510347 SR 0.71 0.46 0.9229 0.0478 0.97 0.97

100000025212 SR 0.89 0.45 -0.4987 0.0668 0.87 0.74

100000044162 SR 0.88 0.39 -0.0075 0.0578 0.82 0.80

100000018395 SR 0.89 0.33 -0.8495 0.0742 1.29 1.28

3510036 SR 0.88 0.38 -0.5397 0.0674 1.18 1.41

3510329 SR 0.69 0.36 1.5719 0.0440 1.05 1.09

3510033 SR 0.84 0.38 0.0473 0.0571 1.00 1.00

100000067837 SR 0.67 0.44 1.3024 0.0454 0.99 0.97

100000122405 SR 0.86 0.49 -0.2221 0.0613 0.87 0.66

100000025211 SR 0.80 0.52 0.4822 0.0518 0.81 0.60

3509950 SR 0.78 0.37 0.8475 0.0484 1.01 0.97

3547998 BCR 0.88 0.43 -0.3395 0.0635 0.94 1.00

3564094 BCR 0.54 0.51 1.8206 0.0375 0.97 0.96 -1.9347 1.9347

100000246294 SR 0.85 0.48 -0.1274 0.0598 0.89 0.64

100000018404 SR 0.86 0.30 -0.1555 0.0604 1.08 1.28

3488201 SR 0.94 0.42 -1.2151 0.0845 0.90 0.56

3510176 SR 0.68 0.46 1.2021 0.0458 0.99 1.00

3509929 SR 0.58 0.47 1.8021 0.0434 0.93 0.91


277

Table C.2. Table C.2 The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 3 Form F


Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

3488069 SR 0.90 0.25 -0.5610 0.0695 1.01 1.17

3548059 SR 0.80 0.53 0.6288 0.0508 0.83 0.67

100000011186 BCR 0.76 0.54 0.8674 0.0487 0.86 0.69

3595529 BCR 0.62 0.37 1.1352 0.0397 1.19 1.21 -2.2067 2.2067

100000067832 SR 0.61 0.54 1.8287 0.0433 0.85 0.81

100000044159 SR 0.63 0.25 1.5712 0.0442 1.27 1.47

100000004268 SR 0.54 0.39 2.0124 0.0429 1.06 1.10

100000044155 SR 0.95 0.30 -1.5257 0.0983 0.99 0.78

100000366020 SR 0.70 0.46 1.3184 0.0456 0.92 0.85

3510017 SR 0.94 0.31 -1.1315 0.0853 0.76 0.79

3509926 SR 0.49 0.44 2.4187 0.0430 0.98 1.06

3509960 SR 0.83 0.30 0.3981 0.0540 0.96 1.04

100000044152 SR 0.88 0.33 -0.3864 0.0670 1.06 1.03

100000025225 SR 0.86 0.35 -0.2038 0.0634 1.10 1.01

3510022 SR 0.57 0.30 2.0077 0.0429 1.16 1.27

3509983 SR 0.93 0.38 -1.1470 0.0852 0.96 0.85

3509918 BCR 0.81 0.44 0.3533 0.0544 0.93 0.89

3564076 BCR 0.51 0.43 2.2253 0.0369 1.05 1.05 -1.8356 1.8356

3509955 SR 0.64 0.34 1.8411 0.0433 1.08 1.17

100000259291 SR 0.83 0.49 0.2321 0.0558 0.89 0.75

100000247327 SR 0.90 0.41 -0.3971 0.0664 0.95 0.79

3509922 BCR 0.68 0.42 1.3410 0.0458 1.06 1.10

3564085 BCR 0.38 0.49 3.3584 0.0395 0.99 0.99 -2.1421 2.1421

100000246320 SR 0.93 0.40 -0.9848 0.0807 0.89 0.66

3488191 SR 0.86 0.49 -0.2744 0.0642 0.86 0.59

3511859 SR 0.91 0.43 -0.8099 0.0761 0.89 0.72

3548507 SR 0.89 0.32 -0.2784 0.0645 0.96 1.01

3509988 SR 0.78 0.39 0.5005 0.0529 1.06 1.16

3488123 SR 0.57 0.44 1.3765 0.0456 1.16 1.19

3510073 BCR 0.80 0.37 0.4877 0.0525 1.09 1.14

3595503 BCR 0.56 0.38 1.7429 0.0383 1.15 1.17 -2.0126 2.0126

3488171 SR 0.76 0.49 0.7136 0.0501 1.01 0.83

3488087 BCR 0.44 0.49 2.6798 0.0431 0.88 0.89

3564099 BCR 0.41 0.44 2.9814 0.0359 1.03 1.07 -1.6470 1.6470

100000025223 SR 0.90 0.24 -0.4730 0.0678 1.12 1.32

100000067776 SR 0.93 0.41 -0.9821 0.0807 0.87 0.54

3488199 SR 0.96 0.28 -1.8514 0.1121 0.94 1.10


278



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

3492707 SR 0.94 0.34 -1.5038 0.0988 0.87 0.48

3510065 SR 0.96 0.25 -2.1822 0.1289 1.49 1.32

3510005 SR 0.63 0.45 1.3897 0.0453 1.02 0.97

100000063516 SR 0.93 0.45 -0.9858 0.0805 0.84 0.51

3510027 SR 0.90 0.33 -0.5906 0.0705 1.03 1.18

3510035 SR 0.90 0.27 -0.6165 0.0710 1.16 1.42

3509949 BCR 0.78 0.48 0.6823 0.0506 0.94 0.88

3985609 BCR 0.71 0.54 1.1536 0.0337 1.03 1.08 -0.9791 0.9791

100000246321 SR 0.92 0.41 -0.7877 0.0750 0.89 0.71

3510058 SR 0.91 0.39 -0.6059 0.0710 0.83 0.76

3510347 SR 0.73 0.44 0.9229 0.0484 1.02 1.03

100000025212 SR 0.89 0.43 -0.4987 0.0685 1.01 1.19

100000044162 SR 0.87 0.34 -0.0075 0.0592 1.05 1.05

100000018395 SR 0.91 0.31 -0.8495 0.0766 1.19 1.15

3510036 SR 0.89 0.37 -0.5397 0.0696 1.14 1.21

3510329 SR 0.70 0.34 1.5719 0.0443 1.01 1.01

3510033 SR 0.85 0.35 0.0473 0.0584 1.07 1.04

100000067837 SR 0.69 0.42 1.3024 0.0458 1.05 1.05

100000011212 SR 0.84 0.42 0.1537 0.0568 0.96 0.89

100000025211 SR 0.79 0.50 0.4822 0.0527 0.93 0.75

3509950 SR 0.78 0.36 0.8475 0.0490 0.98 0.90

3547998 BCR 0.89 0.41 -0.3395 0.0651 0.90 0.86

3564094 BCR 0.55 0.50 1.8206 0.0377 0.99 0.99 -1.9347 1.9347

100000246291 SR 0.88 0.41 -0.2438 0.0633 0.96 0.88

100000018403 SR 0.89 0.34 -0.4235 0.0669 0.98 0.95

100000011197 SR 0.79 0.41 0.5667 0.0518 1.00 1.10

3510176 SR 0.69 0.44 1.2021 0.0465 1.03 1.03

3509929 SR 0.59 0.48 1.8021 0.0436 0.94 0.94


279



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

100000284521 SR 0.86 0.27 -0.8546 0.0605 1.11 1.22

100000069344 SR 0.84 0.50 -0.9015 0.0613 1.04 0.79

100000069387 SR 0.68 0.39 0.6341 0.0449 1.05 1.02

3487819 BCR 0.56 0.44 1.1653 0.0429 1.01 1.03

3564186 BCR 0.45 0.38 2.0935 0.0444 1.06 1.09 -2.6172 2.6172

3515840 SR 0.72 0.37 0.4436 0.0461 1.06 1.02

3515641 SR 0.83 0.49 -0.8522 0.0605 1.03 0.83

100000007115 SR 0.91 0.40 -1.3705 0.0707 0.91 0.63

100000069392 SR 0.60 0.25 1.0618 0.0432 1.24 1.35

3515605 SR 0.66 0.50 0.9009 0.0438 0.89 0.85

100000069398 SR 0.80 0.40 -0.3307 0.0531 1.03 0.95

3488166 SR 0.80 0.42 -0.4437 0.0546 1.09 1.09

3515447 SR 0.59 0.47 1.4979 0.0425 0.97 0.96

3515451 BCR 0.80 0.56 -0.3252 0.0533 0.82 0.75

3564161 BCR 0.73 0.54 0.2543 0.0332 1.05 1.17 -0.4216 0.4216

3497865 SR 0.71 0.52 0.4883 0.0459 0.87 0.75

3515643 SR 0.51 0.47 1.7570 0.0425 0.93 0.94

3488190 SR 0.41 0.31 2.0885 0.0430 1.14 1.31

100000069423 SR 0.61 0.34 1.2547 0.0428 1.16 1.21

3490562 SR 0.54 0.41 1.1949 0.0430 1.06 1.07

100000069419 SR 0.72 0.44 0.3062 0.0471 1.03 1.08

3515807 BCR 0.78 0.43 -0.4754 0.0575 1.04 0.96

3564165 BCR 0.43 0.49 1.8745 0.0315 1.11 1.10 -0.9919 0.9919

100000018336 SR 0.83 0.48 -0.2781 0.0527 0.79 0.74

100000247264 SR 0.79 0.41 -0.2580 0.0524 1.03 0.93

3551599 SR 0.86 0.51 -0.5617 0.0563 0.69 0.48

3515428 SR 0.96 0.22 -1.7288 0.0802 0.64 0.72

100000069354 SR 0.62 0.42 0.9852 0.0435 1.05 1.09

3515868 SR 0.83 0.28 -0.5019 0.0553 1.15 1.43

3515825 SR 0.97 0.22 -3.0366 0.1360 0.96 0.72

100000251989 SR 0.79 0.54 -0.2409 0.0521 0.85 0.64

3515862 BCR 0.57 0.47 1.0359 0.0434 1.02 1.02

3564170 BCR 0.67 0.57 0.4705 0.0332 0.98 0.93 -0.9068 0.9068

3515642 BCR 0.29 0.49 2.6660 0.0454 0.83 0.77

3985619 BCR 0.45 0.61 1.8186 0.0323 0.85 0.84 -1.1454 1.1454

100000069314 SR 0.94 0.29 -1.8735 0.0846 0.99 1.20

3515575 SR 0.88 0.41 -1.0528 0.0644 0.91 0.75


280



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

3487992 SR 0.93 0.31 -1.6686 0.0786 0.98 0.88

100000011499 SR 0.92 0.36 -1.3258 0.0700 0.85 0.66

3515630 SR 0.58 0.47 0.9291 0.0437 1.00 0.97

3488145 BCR 0.70 0.50 0.4804 0.0460 0.97 0.94

3564189 BCR 0.48 0.43 1.6680 0.0388 1.08 1.08 -2.0764 2.0764

100000063664 SR 0.51 0.28 1.5151 0.0424 1.22 1.34

3515635 SR 0.60 0.41 0.6901 0.0448 1.13 1.32

3515631 SR 0.78 0.39 -0.4674 0.0551 1.17 1.23

100000011504 SR 0.69 0.47 0.4124 0.0464 0.98 0.91

3488035 SR 0.84 0.37 -0.5946 0.0567 1.02 1.07

100000201857 SR 0.47 0.39 1.6793 0.0425 1.04 1.11

3515634 SR 0.81 0.45 -0.2435 0.0524 0.90 1.07

3515853 SR 0.79 0.46 -0.1060 0.0507 0.95 0.84

100000012184 SR 0.52 0.40 1.4437 0.0425 1.01 1.03

3515836 SR 0.60 0.43 0.6346 0.0452 1.11 1.09

3515933 SR 0.81 0.31 -0.3619 0.0537 1.16 1.24

3548079 SR 0.96 0.34 -2.3000 0.0997 0.72 0.47

100000201852 SR 0.84 0.46 -0.9014 0.0616 1.07 1.02

100000011490 SR 0.96 0.26 -2.6967 0.1177 0.96 0.85

3515592 SR 0.88 0.43 -0.8156 0.0602 0.86 0.67

3515638 BCR 0.69 0.54 0.5104 0.0460 0.89 0.78

3564169 BCR 0.46 0.50 1.8414 0.0370 1.06 1.06 -1.8818 1.8818

3515848 SR 0.86 0.30 -0.9513 0.0625 1.13 1.64

3515506 SR 0.91 0.38 -1.2169 0.0678 0.74 0.61

100000421708 SR 0.73 0.47 0.1530 0.0484 0.94 0.82

3515632 SR 0.72 0.51 -0.0118 0.0501 0.97 0.84

3548088 SR 0.79 0.45 -0.1831 0.0517 0.92 0.89


281

Table C.4. The 2011 MSA-Math Classical and Rasch Item Parameters: Grade 4 Form F


Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

100000069333 SR 0.63 0.41 0.9168 0.0441 1.04 1.02

100000069344 SR 0.87 0.46 -0.9015 0.0634 1.00 0.75

100000069387 SR 0.69 0.37 0.6341 0.0455 1.05 1.02

100000069368 BCR 0.74 0.43 0.2076 0.0488 1.00 1.03

3985632 BCR 0.51 0.37 1.3722 0.0510 1.05 1.08 -3.0381 3.0381

3515840 SR 0.72 0.37 0.4436 0.0469 1.07 1.07

3515641 SR 0.85 0.47 -0.8522 0.0626 1.11 0.98

100000007115 SR 0.93 0.38 -1.3705 0.0736 0.86 0.54

3515833 SR 0.98 0.18 -3.1070 0.1471 1.00 1.15

3515605 SR 0.67 0.51 0.9009 0.0442 0.88 0.79

100000252072 SR 0.54 0.26 1.4368 0.0426 1.21 1.31

3488166 SR 0.82 0.40 -0.4437 0.0561 1.08 1.23

3515447 SR 0.61 0.47 1.4979 0.0426 0.96 0.95

100000044142 BCR 0.88 0.38 -0.7849 0.0615 0.97 0.86

3595499 BCR 0.49 0.48 1.7241 0.0390 1.01 1.01 -2.0968 2.0968

3497865 SR 0.72 0.53 0.4883 0.0465 0.86 0.77

3515643 SR 0.52 0.47 1.7570 0.0425 0.95 0.95

3488190 SR 0.40 0.29 2.0885 0.0428 1.14 1.28

100000069423 SR 0.59 0.32 1.2547 0.0430 1.16 1.22

3490562 SR 0.55 0.42 1.1949 0.0433 1.06 1.07

100000247277 SR 0.65 0.40 0.8179 0.0447 1.07 1.20

3515648 BCR 0.57 0.55 1.2240 0.0431 0.86 0.82

3564163 BCR 0.65 0.61 0.7720 0.0322 0.94 0.95 -0.8499 0.8499

100000018336 SR 0.82 0.45 -0.2781 0.0541 0.92 0.92

3515684 SR 0.76 0.31 0.1479 0.0495 1.13 1.18

3551599 SR 0.87 0.48 -0.5617 0.0582 0.70 0.52

3515428 SR 0.97 0.20 -1.7288 0.0834 0.65 0.74

100000284524 SR 0.71 0.50 0.4805 0.0465 0.93 0.91

3515581 SR 0.67 0.55 0.7823 0.0448 0.86 0.75

100000025152 SR 0.78 0.41 -0.0672 0.0516 0.98 0.92

100000069382 SR 0.95 0.29 -1.9537 0.0908 0.97 0.85

3515646 BCR 0.69 0.52 0.5279 0.0463 0.89 0.84

3595537 BCR 0.70 0.56 0.6303 0.0314 1.01 1.02 -0.3046 0.3046

100000025172 BCR 0.64 0.44 0.8670 0.0443 1.02 0.97

3985613 BCR 0.55 0.47 1.3256 0.0345 1.06 1.06 -1.5587 1.5587

100000069427 SR 0.90 0.28 -1.1271 0.0679 1.04 1.05

3515575 SR 0.90 0.39 -1.0528 0.0664 0.85 0.75


282



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

100000252071 SR 0.74 0.34 0.1663 0.0491 1.09 1.09

100000011499 SR 0.93 0.34 -1.3258 0.0725 0.77 0.68

3515630 SR 0.63 0.49 0.9291 0.0440 0.98 0.92

3488145 BCR 0.71 0.49 0.4804 0.0466 0.95 0.90

3564189 BCR 0.48 0.42 1.6680 0.0388 1.12 1.13 -2.0764 2.0764

3515789 SR 0.88 0.30 -0.7537 0.0608 1.04 1.22

3515635 SR 0.64 0.43 0.6901 0.0453 1.07 1.14

3515631 SR 0.80 0.38 -0.4674 0.0565 1.20 1.29

100000251996 SR 0.88 0.40 -0.7924 0.0615 0.96 0.90

100000069339 SR 0.83 0.38 -0.4073 0.0556 1.01 0.97

100000201857 SR 0.49 0.41 1.6793 0.0424 1.02 1.07

3515634 SR 0.82 0.42 -0.2435 0.0535 0.86 1.08

3515853 SR 0.84 0.45 -0.1060 0.0518 0.81 0.65

3488163 SR 0.63 0.49 0.9311 0.0440 0.96 0.91

3515836 SR 0.63 0.46 0.6346 0.0457 1.07 1.02

3515933 SR 0.81 0.28 -0.3619 0.0550 1.18 1.24

3548079 SR 0.96 0.32 -2.3000 0.1039 0.72 0.36

100000201852 SR 0.86 0.44 -0.9014 0.0635 1.01 0.80

3548698 SR 0.97 0.27 -2.5568 0.1156 0.94 1.10

3515592 SR 0.88 0.42 -0.8156 0.0620 0.92 0.71

100000007087 BCR 0.77 0.50 0.0561 0.0502 0.91 0.79

3595563 BCR 0.72 0.49 0.2628 0.0352 1.07 1.12 -1.1643 1.1643

100000284503 SR 0.43 0.32 2.1819 0.0429 1.09 1.30

3515506 SR 0.94 0.34 -1.2169 0.0700 0.62 0.44

3497868 SR 0.57 0.45 1.2038 0.0431 1.02 1.04

3515632 SR 0.72 0.49 -0.0118 0.0511 1.12 1.04

3548088 SR 0.80 0.44 -0.1831 0.0528 0.98 0.94


283



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000028258 SR 0.46 0.35 1.3100 0.0418 1.08 1.13

3488503 SR 0.79 0.24 -0.4592 0.0500 1.15 1.30

100000245772 SR 0.73 0.49 -0.0857 0.0466 0.92 0.83

100000065205 BCR 0.55 0.29 0.8718 0.0421 1.17 1.26

3985586 BCR 0.52 0.36 0.8350 0.0480 1.04 1.07 -2.8194 2.8194

100000065179 SR 0.62 0.47 0.5228 0.0432 0.94 0.86

100000245597 SR 0.53 0.37 1.0934 0.0420 1.10 1.13

3512527 SR 0.67 0.53 -0.0500 0.0466 0.97 0.90

100000009922 SR 0.70 0.43 0.0894 0.0454 0.99 0.96

100000245783 SR 0.97 0.24 -3.2761 0.1354 0.95 0.63

3511458 SR 0.91 0.30 -1.7042 0.0727 0.85 0.95

100000245825 SR 0.75 0.41 -0.3649 0.0502 0.99 0.92

3488373 SR 0.70 0.40 0.1790 0.0460 0.93 0.94

3512698 SR 0.88 0.23 -1.7862 0.0753 1.14 1.49

3512529 SR 0.59 0.25 0.4459 0.0448 1.26 1.36

3512606 SR 0.66 0.33 0.3045 0.0441 1.12 1.17

100000297356 SR 0.75 0.46 -0.2154 0.0477 0.95 0.87

100000043853 SR 0.70 0.42 0.3350 0.0441 0.93 0.88

100000028212 BCR 0.73 0.53 -0.1171 0.0469 0.88 0.77

3595468 BCR 0.45 0.47 1.5239 0.0360 1.03 1.02 -1.7740 1.7740

3488370 SR 0.83 0.41 -1.0980 0.0588 1.07 0.97

3488376 SR 0.80 0.43 -0.5510 0.0512 0.90 0.79

3492126 SR 0.88 0.45 -1.1318 0.0596 0.70 0.44

100000078887 SR 0.74 0.40 -0.1507 0.0472 1.01 0.95

100000009927 BCR 0.68 0.49 0.1098 0.0456 0.93 0.82

3985561 BCR 0.40 0.54 1.5783 0.0282 1.06 1.05 -0.0326 0.0326

100000366318 SR 0.87 0.27 -0.8175 0.0546 0.84 0.86

100000143144 SR 0.53 0.40 1.0908 0.0421 1.06 1.07

3488443 SR 0.47 0.32 1.6277 0.0425 1.19 1.31

100000063887 SR 0.75 0.42 -0.2299 0.0478 0.94 0.88

100000065237 SR 0.61 0.36 0.5956 0.0429 1.10 1.12

3511566 SR 0.69 0.40 0.1548 0.0450 1.05 1.10

3511513 SR 0.87 0.33 -1.1293 0.0595 0.90 1.06

100000063620 SR 0.85 0.30 -1.0976 0.0587 1.00 1.14

100000028202 ECR 0.64 0.45 0.4347 0.0436 0.97 0.94

3985560 ECR 0.39 0.57 1.8988 0.0306 0.94 0.93 -2.6153 0.4082 2.2071

3488391 SR 0.87 0.25 -1.2591 0.0618 1.12 1.45


284



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3488431 SR 0.76 0.27 -0.2130 0.0478 1.10 1.30

3492140 SR 0.91 0.38 -1.4589 0.0656 0.83 0.59

100000063880 SR 0.71 0.54 -0.0662 0.0465 0.88 0.75

100000245581 SR 0.82 0.32 -0.8168 0.0547 1.04 1.12

3488347 BCR 0.47 0.61 1.2508 0.0421 0.81 0.74

3564046 BCR 0.45 0.64 1.4035 0.0266 0.92 0.83 0.8746 -0.8746

3488381 SR 0.62 0.19 0.4962 0.0434 1.28 1.45

3488485 SR 0.83 0.41 -1.2799 0.0622 1.28 1.21

3488509 SR 0.78 0.52 -0.5908 0.0517 0.86 0.73

3488394 SR 0.49 0.43 1.1666 0.0420 1.01 1.00

3512637 SR 0.77 0.28 -0.6947 0.0531 1.29 1.52

100000245580 SR 0.67 0.50 0.2255 0.0445 0.95 0.87

100000065196 SR 0.80 0.36 -0.5374 0.0509 0.99 0.98

3511429 SR 0.77 0.43 -0.5025 0.0506 1.03 1.04

3548459 BCR 0.75 0.56 -0.3130 0.0492 0.81 0.71

3564051 BCR 0.68 0.45 0.0803 0.0332 1.12 1.33 -0.9205 0.9205

3511626 SR 0.87 0.30 -0.9260 0.0561 0.88 0.82

3488316 SR 0.57 0.56 0.7693 0.0425 0.88 0.81

3488326 SR 0.67 0.41 0.2463 0.0451 1.01 1.00

3488251 SR 0.65 0.35 0.5581 0.0430 1.05 1.02

100000022545 SR 0.77 0.38 -0.1281 0.0470 0.95 0.84

100000160221 BCR 0.47 0.63 1.2630 0.0421 0.76 0.69

3985559 BCR 0.53 0.66 1.0066 0.0271 0.88 0.80 0.4049 -0.4049

100000009962 SR 0.68 0.39 0.2212 0.0447 1.01 0.99

3488530 BCR 0.38 0.52 1.7777 0.0433 0.87 0.83

3564054 BCR 0.34 0.52 1.9885 0.0305 1.04 1.03 -0.4800 0.4800

3488328 SR 0.71 0.42 -0.0717 0.0471 1.00 0.99

3511448 SR 0.76 0.33 -0.6839 0.0535 1.16 1.35


285



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000028258 SR 0.47 0.36 1.3100 0.0414 1.01 1.04

3488503 SR 0.80 0.23 -0.4592 0.0500 1.11 1.21

100000245515 SR 0.50 0.40 1.2615 0.0414 1.00 1.02

100000065205 BCR 0.56 0.29 0.8718 0.0417 1.13 1.18

3985586 BCR 0.53 0.36 0.8350 0.0484 1.05 1.06 -2.8194 2.8194

100000065317 SR 0.37 0.42 1.8204 0.0425 0.96 0.99

100000009923 SR 0.75 0.33 -0.1160 0.0467 1.07 1.13

3512527 SR 0.68 0.52 -0.0500 0.0462 0.95 0.86

3488268 SR 0.65 0.28 0.4218 0.0432 1.18 1.25

3488245 SR 0.86 0.28 -1.0096 0.0577 1.03 1.11

3511458 SR 0.91 0.30 -1.7042 0.0734 0.77 0.79

100000022537 SR 0.58 0.30 0.7823 0.0425 1.12 1.14

3488373 SR 0.71 0.41 0.1790 0.0453 0.95 0.95

3512698 SR 0.88 0.23 -1.7862 0.0761 1.18 1.41

3512529 SR 0.60 0.24 0.4459 0.0441 1.21 1.31

3512606 SR 0.66 0.32 0.3045 0.0438 1.11 1.21

100000297356 SR 0.77 0.43 -0.2154 0.0476 0.90 0.83

100000043853 SR 0.70 0.42 0.3350 0.0438 0.92 0.90

3511336 BCR 0.46 0.53 1.2703 0.0419 0.86 0.82

3563987 BCR 0.41 0.61 1.5647 0.0302 0.88 0.86 -0.7648 0.7648

3488370 SR 0.83 0.41 -1.0980 0.0594 1.22 1.20

3488376 SR 0.81 0.42 -0.5510 0.0513 0.88 0.72

3492126 SR 0.89 0.44 -1.1318 0.0600 0.73 0.48

100000022549 SR 0.70 0.33 0.1350 0.0451 1.08 1.09

3512615 BCR 0.79 0.49 -0.4468 0.0505 0.88 0.86

3595439 BCR 0.55 0.50 0.8260 0.0328 1.01 1.01 -1.2888 1.2888

100000366318 SR 0.87 0.29 -0.8175 0.0550 0.85 0.90

100000143144 SR 0.53 0.39 1.0908 0.0418 1.05 1.08

3488443 SR 0.47 0.31 1.6277 0.0423 1.17 1.31

100000065193 SR 0.95 0.26 -2.3882 0.0941 0.98 1.24

100000065233 SR 0.70 0.21 0.1537 0.0448 1.24 1.41

3511566 SR 0.70 0.38 0.1548 0.0448 1.01 1.03

3511513 SR 0.86 0.34 -1.1293 0.0599 0.98 1.01

100000063620 SR 0.86 0.31 -1.0976 0.0593 0.94 1.01

100000028200 ECR 0.70 0.54 0.1245 0.0454 0.86 0.76

3595464 ECR 0.56 0.60 0.9312 0.0231 1.09 1.10 -0.2742 -0.6378 0.9120

3488391 SR 0.88 0.25 -1.2591 0.0624 0.92 1.01


286



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3488431 SR 0.76 0.29 -0.2130 0.0477 1.08 1.16

3492140 SR 0.91 0.40 -1.4589 0.0665 0.80 0.59

3488445 SR 0.92 0.26 -2.0029 0.0810 1.00 0.93

3488452 SR 0.86 0.31 -1.0322 0.0584 1.01 1.05

100000022530 BCR 0.76 0.34 -0.2865 0.0489 1.06 1.21

3985592 BCR 0.83 0.41 -0.7452 0.0386 1.05 1.34 -0.3694 0.3694

3488381 SR 0.62 0.20 0.4962 0.0430 1.24 1.39

3488485 SR 0.85 0.41 -1.2799 0.0631 1.17 1.04

3488509 SR 0.82 0.50 -0.5908 0.0521 0.77 0.66

3488394 SR 0.49 0.42 1.1666 0.0416 0.99 0.99

3512637 SR 0.77 0.30 -0.6947 0.0535 1.32 1.44

100000143153 SR 0.83 0.42 -0.8320 0.0549 0.96 0.91

100000065196 SR 0.81 0.33 -0.5374 0.0509 0.99 1.11

3511429 SR 0.78 0.43 -0.5025 0.0506 1.00 1.03

100000028238 BCR 0.63 0.59 0.4456 0.0435 0.83 0.75

3985553 BCR 0.60 0.57 0.6481 0.0290 0.99 1.01 -0.3350 0.3350

3511626 SR 0.88 0.28 -0.9260 0.0565 0.84 0.78

100000009963 SR 0.68 0.48 0.2208 0.0446 0.92 0.88

3488326 SR 0.67 0.37 0.2463 0.0449 1.01 1.03

3488251 SR 0.66 0.38 0.5581 0.0427 0.98 0.96

100000022545 SR 0.77 0.36 -0.1281 0.0469 0.97 0.88

100000022532 BCR 0.45 0.62 1.4206 0.0419 0.74 0.68

3595471 BCR 0.49 0.64 1.1783 0.0264 0.86 0.78 0.5388 -0.5388

100000065216 SR 0.64 0.48 0.5413 0.0427 0.92 0.86

3488530 BCR 0.37 0.51 1.7777 0.0426 0.83 0.78

3564054 BCR 0.34 0.52 1.9885 0.0301 1.00 0.99 -0.4800 0.4800

3488328 SR 0.73 0.42 -0.0717 0.0466 0.95 0.93

3511448 SR 0.81 0.36 -0.6839 0.0532 0.92 0.94


287



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3516257 SR 0.89 0.24 -1.2053 0.0582 0.91 1.07

100000028396 SR 0.57 0.46 0.6355 0.0431 0.98 1.01

100000028429 SR 0.59 0.37 0.5836 0.0433 1.08 1.07

100000022483 SR 0.55 0.27 0.6958 0.0432 1.27 1.41

100000245469 SR 0.89 0.22 -1.5044 0.0636 1.05 1.51

3516243 SR 0.76 0.41 -0.2844 0.0475 0.95 0.84

3517004 ECR 0.92 0.36 -2.0736 0.0775 0.92 0.65

3564010 ECR 0.68 0.57 -0.0664 0.0277 1.06 1.09 -1.5314 0.0203 1.5111

3492071 SR 0.67 0.48 0.2299 0.0445 0.93 0.84

3516559 SR 0.90 0.27 -1.4432 0.0625 0.84 0.90

3516255 SR 0.77 0.28 -0.4703 0.0491 1.06 1.16

100000022476 SR 0.55 0.27 0.2578 0.0444 1.34 1.61

100000064773 SR 0.60 0.42 0.4499 0.0439 1.06 1.06

100000064614 SR 0.78 0.46 -0.6463 0.0510 0.92 0.86

3516909 SR 0.61 0.45 0.4042 0.0440 0.99 0.96

100000065101 SR 0.75 0.47 -0.4765 0.0494 0.94 0.86

100000094477 BCR 0.26 0.46 2.4967 0.0495 0.95 0.91

3985566 BCR 0.33 0.52 1.8157 0.0329 1.17 1.15 -0.7847 0.7847

100000249208 SR 0.77 0.46 -0.6865 0.0523 0.93 0.76

100000064637 SR 0.61 0.47 0.3609 0.0450 0.95 0.91

3516290 SR 0.72 0.42 -0.1396 0.0477 0.90 0.84

100000043862 SR 0.65 0.43 0.2101 0.0457 0.99 0.93

100000245938 SR 0.52 0.26 0.8852 0.0440 1.32 1.47

100000245433 SR 0.71 0.43 -0.0427 0.0458 0.97 0.95

3492143 SR 0.77 0.45 -0.6658 0.0511 1.03 0.99

100000028367 BCR 0.38 0.61 1.6232 0.0439 0.78 0.69

3595489 BCR 0.54 0.68 0.7895 0.0293 0.81 0.76 -0.3122 0.3122

100000065159 SR 0.63 0.41 0.3832 0.0440 1.05 1.04

3516331 SR 0.55 0.41 1.1378 0.0429 1.08 1.11

3516241 SR 0.86 0.37 -1.4702 0.0631 1.04 0.85

100000065061 SR 0.84 0.43 -0.8511 0.0532 0.81 0.68

3516329 SR 0.64 0.39 0.5144 0.0436 1.03 1.05

100000094472 SR 0.49 0.47 1.1803 0.0432 0.98 1.00

100000245478 SR 0.59 0.47 0.5646 0.0435 0.93 0.88

3492095 SR 0.79 0.47 -0.9261 0.0542 1.08 0.97

100000064628 SR 0.67 0.46 0.1509 0.0449 0.99 1.00

3516333 BCR 0.68 0.62 -0.0151 0.0460 0.78 0.69


288



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3564008 BCR 0.67 0.64 0.1055 0.0318 0.85 0.93 -0.5646 0.5646

100000245436 SR 0.45 0.39 1.2690 0.0432 1.08 1.12

3516565 SR 0.52 0.53 0.8786 0.0430 0.89 0.88

3488358 SR 0.69 0.49 -0.0834 0.0463 0.97 1.04

100000065162 SR 0.63 0.36 0.3152 0.0446 1.11 1.46

100000065116 SR 0.72 0.36 -0.1755 0.0470 1.03 1.10

3516906 SR 0.59 0.42 0.2864 0.0446 1.09 1.11

100000004453 SR 0.53 0.53 0.8127 0.0433 0.92 0.89

3516293 SR 0.50 0.42 1.0083 0.0432 1.07 1.08

100000022470 SR 0.53 0.53 0.8821 0.0429 0.90 0.85

3488489 SR 0.75 0.39 -0.2639 0.0474 0.94 0.82

100000022501 BCR 0.45 0.47 1.3886 0.0433 0.98 1.00

3595485 BCR 0.74 0.54 -0.2922 0.0334 0.98 1.02 -0.5149 0.5149

3492120 SR 0.66 0.34 0.2830 0.0444 1.10 1.10

3492097 SR 0.84 0.41 -1.0963 0.0568 0.91 0.79

100000064587 BCR 0.40 0.59 1.5611 0.0440 0.84 0.76

3985580 BCR 0.48 0.65 1.1123 0.0331 0.82 0.82 -1.2436 1.2436

3516613 SR 0.57 0.25 0.4071 0.0442 1.28 1.39

100000064586 SR 0.80 0.35 -0.8095 0.0528 1.02 1.05

100000004461 SR 0.78 0.32 -0.6262 0.0507 1.04 1.27

100000022488 SR 0.46 0.53 1.2743 0.0431 0.89 0.85

3488404 BCR 0.43 0.49 1.4066 0.0434 0.98 0.99

3985578 BCR 0.53 0.56 0.8273 0.0388 0.91 0.90 -2.0342 2.0342

100000064621 SR 0.58 0.44 0.6947 0.0434 0.99 1.00

100000065148 SR 0.57 0.37 0.5523 0.0436 1.12 1.17


289



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3516257 SR 0.90 0.21 -1.2053 0.0591 0.86 1.34

100000028439 SR 0.75 0.43 -0.2890 0.0480 0.96 0.85

100000028429 SR 0.61 0.38 0.5836 0.0435 1.08 1.06

100000022483 SR 0.57 0.27 0.6958 0.0434 1.25 1.38

100000313187 SR 0.91 0.23 -1.7132 0.0695 1.01 1.12

3516243 SR 0.77 0.39 -0.2844 0.0481 0.96 0.89

3517004 ECR 0.94 0.32 -2.0736 0.0791 0.78 0.64

3564010 ECR 0.70 0.54 -0.0664 0.0280 1.05 1.07 -1.5314 0.0203 1.5111

3492071 SR 0.68 0.48 0.2299 0.0448 0.94 0.85

3516559 SR 0.91 0.24 -1.4432 0.0636 0.81 1.12

3516255 SR 0.79 0.27 -0.4703 0.0497 1.08 1.21

100000022476 SR 0.57 0.27 0.2578 0.0447 1.36 1.82

100000173712 SR 0.65 0.48 0.2612 0.0451 0.92 0.89

100000064614 SR 0.79 0.45 -0.6463 0.0517 0.95 0.90

3516909 SR 0.62 0.45 0.4042 0.0444 0.99 1.02

100000065108 SR 0.68 0.52 0.0225 0.0463 0.89 0.85

100000094481 BCR 0.81 0.38 -0.8429 0.0547 0.97 1.03

3985581 BCR 0.44 0.50 1.3207 0.0313 1.17 1.18 -0.7635 0.7635

100000028423 SR 0.51 0.55 0.9446 0.0440 0.90 0.87

100000064637 SR 0.61 0.47 0.3609 0.0454 0.98 0.94

3516290 SR 0.73 0.40 -0.1396 0.0483 0.94 0.97

100000043862 SR 0.66 0.43 0.2101 0.0462 0.92 0.84

100000094468 SR 0.74 0.42 -0.3704 0.0504 1.01 1.02

100000245433 SR 0.70 0.41 -0.0427 0.0462 1.07 1.05

3492143 SR 0.79 0.43 -0.6658 0.0517 0.99 0.89

100000028363 BCR 0.71 0.53 -0.1109 0.0471 0.86 0.74

3595491 BCR 0.66 0.59 0.2308 0.0310 0.97 0.94 -0.3923 0.3923

100000313284 SR 0.56 0.46 0.8487 0.0431 1.00 0.99

3516331 SR 0.58 0.39 1.1378 0.0431 1.11 1.14

3516241 SR 0.88 0.35 -1.4702 0.0644 1.02 0.90

100000065061 SR 0.85 0.40 -0.8511 0.0542 0.79 0.65

3516329 SR 0.67 0.32 0.5144 0.0438 1.07 1.13

100000094472 SR 0.49 0.45 1.1803 0.0432 0.99 1.00

100000245444 SR 0.78 0.50 -0.5459 0.0505 0.89 0.73

3492095 SR 0.81 0.45 -0.9261 0.0551 1.00 0.90

100000162295 SR 0.67 0.42 0.1805 0.0451 0.99 1.03

100000022498 BCR 0.63 0.40 0.3100 0.0451 1.05 1.09


290



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3595494 BCR 0.58 0.64 0.6207 0.0316 0.92 0.91 -0.7964 0.7964

3492148 SR 0.76 0.41 -0.3460 0.0486 0.99 0.93

3516565 SR 0.57 0.53 0.8786 0.0433 0.88 0.84

3488358 SR 0.71 0.48 -0.0834 0.0469 0.94 0.94

3488386 SR 0.68 0.39 0.0671 0.0461 1.01 0.94

3488387 SR 0.73 0.34 -0.1907 0.0477 1.08 1.33

3516906 SR 0.60 0.41 0.2864 0.0451 1.12 1.18

100000004453 SR 0.53 0.53 0.8127 0.0435 0.93 0.93

3516293 SR 0.51 0.42 1.0083 0.0435 1.07 1.13

100000022470 SR 0.55 0.54 0.8821 0.0432 0.88 0.85

3488489 SR 0.75 0.41 -0.2639 0.0479 0.98 0.88

100000022504 BCR 0.63 0.45 0.1906 0.0465 0.98 0.90

3595488 BCR 0.50 0.63 1.0505 0.0285 0.95 0.90 0.4844 -0.4844

3492120 SR 0.64 0.37 0.2830 0.0448 1.11 1.15

100000028401 SR 0.83 0.43 -0.9836 0.0561 0.90 0.75

100000064587 BCR 0.41 0.59 1.5611 0.0444 0.82 0.74

3985580 BCR 0.48 0.64 1.1123 0.0334 0.84 0.83 -1.2436 1.2436

3516613 SR 0.57 0.29 0.4071 0.0450 1.26 1.39

100000064632 SR 0.69 0.43 0.0598 0.0457 0.98 1.01

100000245474 SR 0.58 0.41 0.7050 0.0434 1.07 1.11

100000144352 SR 0.53 0.45 0.9085 0.0431 1.01 0.99

100000065098 BCR 0.54 0.48 0.9643 0.0436 0.98 0.94

3985582 BCR 0.24 0.51 2.8227 0.0372 1.01 1.01 -1.0241 1.0241

3503955 SR 0.60 0.52 0.5628 0.0438 0.91 0.86

100000065148 SR 0.58 0.38 0.5523 0.0438 1.09 1.13


291



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3517650 SR 0.70 0.49 -0.4683 0.0472 0.93 0.86

3517601 SR 0.55 0.54 0.4455 0.0441 0.90 0.88

3517678 SR 0.95 0.28 -2.6820 0.0818 0.63 0.41

100000263280 SR 0.75 0.46 -0.8945 0.0507 0.94 0.87

100000064017 BCR 0.34 0.49 1.5009 0.0458 0.97 0.91

3985664 BCR 0.42 0.50 1.0367 0.0333 1.19 1.17 -1.1360 1.1360

3487535 SR 0.60 0.31 -0.0824 0.0454 1.29 1.46

3517613 SR 0.73 0.56 -0.6420 0.0484 0.79 0.64

100000063936 SR 0.86 0.39 -1.8067 0.0620 0.95 0.84

3517677 SR 0.65 0.53 -0.3998 0.0469 0.97 0.94

3487551 SR 0.61 0.34 -0.0492 0.0453 1.22 1.37

100000026796 SR 0.83 0.41 -1.4762 0.0568 0.91 0.77

100000282586 SR 0.48 0.53 0.6502 0.0442 0.93 0.92

3517652 SR 0.74 0.46 -0.6359 0.0487 0.95 0.88

3547473 SR 0.85 0.42 -1.1243 0.0529 0.79 0.66

3517739 SR 0.86 0.36 -1.9183 0.0642 0.94 1.02

100000263250 SPR 0.58 0.61 0.0904 0.0453 0.83 0.74

3491692 ECR 0.47 0.67 0.6758 0.0445 0.77 0.69

3564159 ECR 0.47 0.73 0.6840 0.0263 0.88 0.88 -1.3053 0.1967 1.1086

100000063975 SPR 0.20 0.55 2.3619 0.0522 0.78 0.59

100000063997 SPR 0.65 0.52 -0.4241 0.0481 0.91 0.90

100000063933 SPR 0.72 0.53 -1.0892 0.0542 0.87 0.90

3517818 BCR 0.32 0.59 1.4905 0.0474 0.85 0.74

3564023 BCR 0.34 0.58 1.7459 0.0416 0.92 0.92 -2.1226 2.1226

100000063914 SR 0.45 0.26 0.7756 0.0457 1.36 1.57

3517639 SR 0.36 0.47 1.4497 0.0452 0.99 0.97

3517665 SR 0.41 0.43 0.9745 0.0442 1.02 1.09

3517609 SR 0.56 0.52 0.1508 0.0447 0.93 0.86

100000063971 SPR 0.38 0.64 1.2654 0.0451 0.81 0.70

100000282608 SPR 0.23 0.52 2.2465 0.0507 0.84 0.71

3517706 BCR 0.50 0.46 0.5804 0.0446 1.09 1.15

3564025 BCR 0.34 0.59 1.5909 0.0347 1.05 1.00 -1.2226 1.2226

100000063911 SR 0.65 0.43 -0.2430 0.0462 1.03 1.09

3517656 SR 0.68 0.40 -0.4094 0.0471 1.13 1.06

100000012791 SR 0.35 0.32 1.5594 0.0458 1.19 1.47

100000043347 ECR 0.73 0.58 -0.8028 0.0502 0.85 0.75

3595366 ECR 0.30 0.54 2.9325 0.0446 0.95 0.93 -4.5203 0.7355 3.7848


292



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000250878 SPR 0.51 0.63 0.4892 0.0443 0.81 0.73

100000353725 SPR 0.56 0.42 0.3091 0.0446 1.14 1.23

100000012796 SR 0.52 0.34 0.2502 0.0445 1.25 1.45

3517602 SR 0.52 0.46 0.7735 0.0441 1.02 1.06

3517687 SR 0.58 0.48 -0.0583 0.0453 1.04 0.98

100000282585 SR 0.64 0.38 -0.1181 0.0455 1.14 1.29

100000064068 ECR 0.67 0.43 -0.3270 0.0467 1.08 1.39

3985667 ECR 0.39 0.42 2.8505 0.0374 1.26 1.28 -4.7413 -1.0980 5.8392

100000071528 SPR 0.57 0.59 0.2080 0.0447 0.83 0.74

100000353345 SPR 0.51 0.50 0.5279 0.0445 0.99 1.00

100000353336 SPR 0.51 0.54 0.5603 0.0446 0.96 0.95

100000012794 SR 0.42 0.34 0.9999 0.0446 1.18 1.36

3517714 SR 0.59 0.60 0.0092 0.0458 0.80 0.69

3517885 SR 0.52 0.43 1.0861 0.0443 1.21 1.35

100000018106 SR 0.62 0.35 0.0819 0.0448 1.16 1.16

100000063909 BCR 0.69 0.53 -0.6136 0.0494 0.91 0.91

3985662 BCR 0.40 0.49 2.2752 0.0519 0.94 0.88 -3.4808 3.4808

3487898 SR 0.53 0.38 0.3744 0.0442 1.16 1.19

3517691 SR 0.67 0.58 -0.2784 0.0462 0.80 0.68

100000018108 SR 0.65 0.47 -0.2386 0.0461 1.02 1.00

3555858 SR 0.49 0.42 0.6673 0.0442 1.03 1.06

100000064018 SPR 0.59 0.43 0.0153 0.0452 1.10 1.07

3555859 SR 0.77 0.42 -1.4603 0.0567 1.34 1.54

100000363463 SR 0.57 0.51 -0.0202 0.0453 0.97 0.92

3488830 SR 0.58 0.54 -0.0385 0.0453 0.97 0.95


293



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3517650 SR 0.72 0.46 -0.4683 0.0474 0.92 0.87

3517601 SR 0.57 0.54 0.4455 0.0440 0.89 0.85

3517678 SR 0.96 0.24 -2.6820 0.0845 0.70 0.51

100000263280 SR 0.78 0.44 -0.8945 0.0510 0.95 0.89

100000064058 BCR 0.62 0.59 0.0981 0.0451 0.85 0.78

3985652 BCR 0.45 0.57 1.0582 0.0362 1.00 1.00 -1.7073 1.7073

3487535 SR 0.61 0.31 -0.0824 0.0456 1.23 1.32

3517613 SR 0.75 0.54 -0.6420 0.0487 0.79 0.64

100000063936 SR 0.88 0.36 -1.8067 0.0632 0.92 0.86

3517677 SR 0.68 0.51 -0.3998 0.0470 0.98 0.96

100000026812 SR 0.41 0.26 1.1191 0.0442 1.28 1.53

100000026796 SR 0.84 0.38 -1.4762 0.0577 0.97 0.96

100000282586 SR 0.52 0.53 0.6502 0.0440 0.94 0.91

3517652 SR 0.75 0.43 -0.6359 0.0490 0.93 0.80

3547473 SR 0.87 0.41 -1.1243 0.0533 0.70 0.62

3517739 SR 0.87 0.33 -1.9183 0.0654 1.08 1.18

100000263247 SPR 0.43 0.58 1.0841 0.0445 0.92 0.87

100000026808 ECR 0.38 0.53 1.3798 0.0452 0.93 0.87

3595374 ECR 0.40 0.64 1.2316 0.0249 1.17 1.17 -0.1991 -0.8977 1.0969

100000064079 SPR 0.32 0.64 1.6259 0.0464 0.71 0.60

100000141392 SPR 0.31 0.58 1.7744 0.0473 0.80 0.69

100000064098 SPR 0.77 0.46 -1.1885 0.0560 0.90 1.06

3517818 BCR 0.35 0.59 1.4905 0.0467 0.87 0.78

3564023 BCR 0.36 0.58 1.7459 0.0412 0.91 0.90 -2.1226 2.1226

100000063979 SR 0.41 0.29 0.9780 0.0453 1.31 1.48

3517639 SR 0.37 0.51 1.4497 0.0450 0.93 0.92

3517665 SR 0.42 0.47 0.9745 0.0440 0.97 0.99

3517609 SR 0.58 0.52 0.1508 0.0447 0.96 0.90

100000043353 SPR 0.69 0.38 -0.3836 0.0472 1.07 1.15

3517732 SPR 0.36 0.53 1.5349 0.0454 0.97 0.87

3517706 BCR 0.53 0.44 0.5804 0.0445 1.07 1.12

3564025 BCR 0.36 0.57 1.5909 0.0345 1.09 1.07 -1.2226 1.2226

100000250201 SR 0.70 0.35 -0.4897 0.0477 1.13 1.36

3517656 SR 0.69 0.39 -0.4094 0.0472 1.07 0.93

100000012791 SR 0.36 0.33 1.5594 0.0455 1.25 1.59

100000064101 ECR 0.61 0.42 0.1237 0.0449 1.10 1.28

3985665 ECR 0.37 0.52 1.4624 0.0341 1.13 1.17 -3.1958 1.3386 1.8572


294



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000250877 SPR 0.52 0.67 0.5205 0.0443 0.74 0.65

100000353340 SPR 0.41 0.57 1.2214 0.0447 0.91 0.84

100000012796 SR 0.53 0.32 0.2502 0.0445 1.26 1.44

3517602 SR 0.53 0.47 0.7735 0.0440 0.99 1.00

3517687 SR 0.61 0.47 -0.0583 0.0453 1.02 0.97

3487727 SR 0.61 0.51 0.1672 0.0446 0.96 0.94

3517648 ECR 0.66 0.38 -0.1506 0.0459 1.12 1.30

3564027 ECR 0.63 0.56 0.0836 0.0303 1.05 1.09 -1.1232 -1.3537 2.4768

100000353334 SPR 0.71 0.41 -0.5274 0.0481 1.02 1.18

100000353344 SPR 0.76 0.40 -0.9404 0.0515 1.01 1.40

100000012801 SPR 0.38 0.56 1.3060 0.0452 0.90 0.83

100000012794 SR 0.39 0.35 0.9999 0.0445 1.13 1.20

3517714 SR 0.60 0.58 0.0092 0.0459 0.87 0.78

3517885 SR 0.52 0.44 1.0861 0.0441 1.06 1.07

100000018106 SR 0.61 0.38 0.0819 0.0449 1.14 1.14

100000012815 BCR 0.44 0.64 0.9721 0.0444 0.81 0.76

3595391 BCR 0.53 0.52 0.2798 0.0461 0.93 0.91 -2.7171 2.7171

3487898 SR 0.51 0.37 0.3744 0.0441 1.22 1.28

3517691 SR 0.72 0.54 -0.2784 0.0463 0.81 0.73

3513570 SR 0.53 0.47 0.4944 0.0440 1.01 1.00

3555858 SR 0.48 0.47 0.6673 0.0440 1.05 1.07

100000063939 SPR 0.36 0.45 1.4155 0.0457 1.04 1.08

3555859 SR 0.79 0.40 -1.4603 0.0574 1.29 1.44

100000363463 SR 0.60 0.51 -0.0202 0.0454 0.98 0.93

3488830 SR 0.60 0.52 -0.0385 0.0453 0.96 0.92


295



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000064981 SR 0.63 0.47 -0.4269 0.0450 1.02 1.10

100000026755 SR 0.69 0.43 -0.5724 0.0457 0.98 0.91

3514013 BCR 0.50 0.67 0.1260 0.0445 0.76 0.69

3564107 BCR 0.66 0.63 -0.7774 0.0350 0.90 0.91 -1.1328 1.1328

100000018156 SR 0.73 0.32 -0.7550 0.0465 1.09 1.11

100000264701 SR 0.47 0.49 0.3594 0.0440 1.06 1.06

3514053 SR 0.76 0.37 -1.2003 0.0498 1.06 1.29

100000043330 SR 0.49 0.54 0.3621 0.0441 0.94 0.97

3500150 SR 0.47 0.42 0.3158 0.0440 1.04 1.07

100000250166 SPR 0.77 0.41 -1.3656 0.0516 0.99 1.16

100000064847 ECR 0.21 0.59 1.9993 0.0518 0.77 0.57

3985701 ECR 0.20 0.67 1.9666 0.0292 0.95 0.81 -0.5702 0.0165 0.5536

100000220610 SPR 0.44 0.56 0.5501 0.0450 0.93 0.88

100000264735 SPR 0.47 0.65 0.3267 0.0445 0.81 0.78

100000264753 SPR 0.34 0.46 1.2529 0.0467 1.10 1.08

100000199104 SPR 0.43 0.59 0.6764 0.0453 0.86 0.83

3514595 SR 0.72 0.30 -0.8540 0.0471 1.16 1.36

100000264698 SR 0.48 0.36 0.4975 0.0441 1.24 1.32

100000064945 BCR 0.30 0.61 1.4164 0.0474 0.82 0.70

3985679 BCR 0.50 0.66 0.1442 0.0352 0.88 0.91 -1.4317 1.4317

3492049 SPR 0.63 0.39 -0.5141 0.0461 1.11 1.54

100000043320 SR 0.49 0.37 0.2581 0.0440 1.23 1.37

3500154 SR 0.73 0.50 -1.1600 0.0494 0.92 0.85

3514062 SR 0.45 0.55 0.5139 0.0441 0.91 0.86

3487636 BCR 0.21 0.53 2.0944 0.0519 0.84 0.71

3595428 BCR 0.34 0.59 1.0876 0.0316 1.13 1.13 -0.2705 0.2705

100000026743 SPR 0.42 0.60 0.7068 0.0449 0.87 0.78

3492059 SPR 0.52 0.52 0.0625 0.0446 0.98 1.02

100000049037 SR 0.67 0.43 -0.5927 0.0459 1.01 0.96

100000064837 ECR 0.48 0.61 0.3005 0.0444 0.87 0.80

3985702 ECR 0.39 0.68 1.2534 0.0304 0.94 0.94 -2.3018 -0.6177 2.9195

100000043323 SR 0.51 0.52 0.5661 0.0443 0.98 0.96

3487718 SR 0.62 0.48 -0.5452 0.0456 0.99 0.96

100000026766 SR 0.36 0.52 1.0957 0.0457 0.91 0.96

3487759 BCR 0.31 0.67 1.3048 0.0473 0.72 0.59

3564128 BCR 0.51 0.73 0.0858 0.0334 0.72 0.70 -1.0235 1.0235

3487566 SR 0.50 0.48 0.0785 0.0443 1.05 1.08


296



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3487901 SR 0.86 0.38 -2.1555 0.0622 0.97 1.21

3514056 SR 0.83 0.28 -1.4852 0.0526 0.94 1.09

100000064933 SR 0.80 0.40 -1.6848 0.0550 0.93 0.99

3487525 SR 0.52 0.58 0.0551 0.0444 0.86 0.81

100000018153 SR 0.66 0.26 -0.5150 0.0454 1.31 1.66

3514103 SR 0.72 0.35 -0.5330 0.0455 1.02 1.11

3514076 SR 0.52 0.44 0.1649 0.0440 1.09 1.07

3514078 ECR 0.25 0.59 1.7694 0.0494 0.82 0.67

3564109 ECR 0.39 0.71 0.8846 0.0268 0.96 0.92 -1.4575 0.3985 1.0590

100000064863 SPR 0.52 0.52 0.0387 0.0446 0.98 0.93

100000043326 SPR 0.46 0.54 0.4990 0.0444 0.98 0.99

3514710 SR 0.61 0.37 -0.1424 0.0443 1.13 1.19

100000065015 SR 0.50 0.52 0.3312 0.0440 1.00 1.04

100000264729 SR 0.63 0.50 -0.5427 0.0456 0.95 0.88

3487902 SR 0.83 0.35 -1.8764 0.0575 1.02 1.10

100000018151 SR 0.57 0.43 0.0346 0.0441 1.09 1.10

3487712 SR 0.62 0.48 -0.3805 0.0451 0.99 0.94

100000004114 SR 0.32 0.29 1.2076 0.0461 1.28 1.47

100000353872 SPR 0.34 0.55 0.9601 0.0472 0.94 0.92

100000264722 SPR 0.49 0.41 0.0891 0.0457 1.10 1.23

100000012739 BCR 0.42 0.54 0.5847 0.0460 1.01 1.02

3985697 BCR 0.45 0.44 0.6836 0.0541 0.97 0.95 -3.1803 3.1803

3487912 SR 0.56 0.50 -0.0934 0.0450 0.99 0.94

3547536 SR 0.49 0.41 -0.0487 0.0451 1.20 1.26

100000018144 SR 0.63 0.40 -0.6572 0.0475 1.10 1.22


297



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

100000064981 SR 0.65 0.43 -0.4269 0.0449 0.98 1.06

100000026755 SR 0.70 0.42 -0.5724 0.0457 0.96 0.87

100000043324 BCR 0.48 0.59 0.3396 0.0444 0.84 0.77

3595434 BCR 0.47 0.61 0.3357 0.0287 1.09 1.07 0.6920 -0.6920

100000018156 SR 0.74 0.33 -0.7550 0.0466 1.01 0.95

100000026750 SR 0.61 0.27 -0.2508 0.0443 1.28 1.43

3514053 SR 0.78 0.33 -1.2003 0.0500 1.03 1.26

100000043330 SR 0.50 0.54 0.3621 0.0434 0.90 0.87

3500150 SR 0.49 0.40 0.3158 0.0435 1.09 1.16

100000250168 SPR 0.17 0.43 2.4519 0.0548 0.98 0.89

3487937 ECR 0.30 0.63 1.4659 0.0475 0.75 0.61

3564125 ECR 0.34 0.71 1.1460 0.0252 0.92 0.85 -0.2932 -0.1015 0.3948

100000065028 SPR 0.74 0.31 -1.1078 0.0497 1.10 1.46

100000264754 SPR 0.28 0.42 1.7771 0.0489 1.07 0.99

100000264733 SPR 0.59 0.31 -0.0502 0.0445 1.19 1.40

100000012737 SPR 0.46 0.58 0.5104 0.0443 0.86 0.82

3514595 SR 0.73 0.31 -0.8540 0.0472 1.12 1.15

100000250120 SR 0.66 0.48 -0.3904 0.0447 0.96 0.91

100000064945 BCR 0.33 0.61 1.4164 0.0466 0.83 0.69

3985679 BCR 0.55 0.63 0.1442 0.0348 0.77 0.79 -1.4317 1.4317

100000064989 SPR 0.61 0.55 -0.2805 0.0449 0.88 0.96

100000043320 SR 0.53 0.40 0.2581 0.0437 1.10 1.13

3500154 SR 0.75 0.47 -1.1600 0.0496 0.92 0.85

3514062 SR 0.47 0.54 0.5139 0.0436 0.91 0.90

3487636 BCR 0.22 0.55 2.0944 0.0513 0.88 0.74

3595428 BCR 0.35 0.60 1.0876 0.0310 1.08 1.02 -0.2705 0.2705

100000064980 SPR 0.32 0.63 1.3433 0.0466 0.81 0.70

3514131 SPR 0.48 0.50 0.4071 0.0439 0.96 0.97

100000049037 SR 0.65 0.39 -0.5927 0.0457 1.07 1.07

100000026770 ECR 0.36 0.64 1.1258 0.0457 0.80 0.69

3595425 ECR 0.44 0.70 0.5402 0.0245 1.03 1.01 -0.8780 0.8115 0.0665

100000043323 SR 0.51 0.51 0.5661 0.0437 0.96 0.93

3487718 SR 0.64 0.47 -0.5452 0.0455 1.01 0.97

100000064895 SR 0.43 0.35 0.7846 0.0444 1.17 1.35

100000064972 BCR 0.26 0.52 1.6793 0.0500 0.93 0.90

3985691 BCR 0.25 0.59 1.9569 0.0377 1.00 0.94 -1.0804 1.0804

3487566 SR 0.54 0.46 0.0785 0.0438 1.02 1.02


298



Point-Biserial

RaschDifficulty SEM

Ms. Infit

Ms. Outfit

Step 0-1

Step 1-2

Step 2-3

3487901 SR 0.88 0.35 -2.1555 0.0637 0.94 0.91

3514056 SR 0.85 0.27 -1.4852 0.0534 0.85 0.88

3487556 SR 0.78 0.44 -1.3128 0.0514 0.93 0.90

3487525 SR 0.53 0.55 0.0551 0.0440 0.92 0.88

100000018153 SR 0.66 0.25 -0.5150 0.0453 1.26 1.51

3514103 SR 0.74 0.33 -0.5330 0.0454 1.02 1.06

3514076 SR 0.52 0.47 0.1649 0.0436 1.04 1.01

3487639 ECR 0.64 0.49 -0.5168 0.0464 0.94 0.93

3985692 ECR 0.38 0.69 1.2467 0.0302 0.87 0.86 -2.2743 -0.0533 2.3275

100000220615 SPR 0.29 0.58 1.4967 0.0472 0.81 0.71

100000064900 SPR 0.42 0.47 0.8124 0.0446 1.03 1.03

3514710 SR 0.63 0.37 -0.1424 0.0441 1.13 1.28

100000004116 SR 0.52 0.31 0.0898 0.0441 1.21 1.21

100000264729 SR 0.65 0.48 -0.5427 0.0456 0.95 0.89

3487902 SR 0.84 0.35 -1.8764 0.0586 0.99 1.16

100000018151 SR 0.58 0.46 0.0346 0.0437 1.01 0.98

3487712 SR 0.62 0.46 -0.3805 0.0449 1.00 1.02

100000004114 SR 0.33 0.30 1.2076 0.0455 1.26 1.43

100000353934 SPR 0.50 0.57 0.2310 0.0450 0.89 0.90

3492050 SPR 0.72 0.43 -1.0440 0.0502 0.98 1.24

100000012739 BCR 0.43 0.55 0.5847 0.0454 0.94 0.88

3985697 BCR 0.47 0.47 0.6836 0.0550 1.01 1.04 -3.1803 3.1803

3487912 SR 0.57 0.51 -0.0934 0.0447 0.94 0.85

3547536 SR 0.51 0.45 -0.0487 0.0447 1.10 1.16

100000064874 SR 0.64 0.38 -0.6573 0.0475 1.13 1.21


299

D. APPENDIX D: THE 2011 MSA-MATH BLUEPRINTS


300

Table D.1 The 2011 MSA-Math Blueprint: Grade 3

Form A Form B Form C Form D Form E Form F Form G Form H Form J Form K

SR BCR SR BCR SR BCR SR BCR SR BCR SR BCR SR BCR

SR

BCR

SR

BCR

SR

BCR

1. Knowledge of Algebra, Patterns, or Functions - Students will algebraically represent, model, analyze, or solve mathematical or real-world problems involving patterns or functional relationships

Operational Items

(Field Test Items)

12

(4)

1

(1)

12

(1)

1

(0)

12

(2)

1

(0)

12

(2)

1

(1)

12

(2)

1

(1)

12

(2)

1

(1)

12

(2)

1

(1)

12

(3)

1

(1)

12

(1)

1

(0)

12

(3)

1

(1)

2. Knowledge of Geometry - Students will apply the properties of one, two, or three-dimensional geometric figures to describe, reason, or solve problems about shape, size, position, or motion of objects

Operational Items

(Field Test Items)

7

(1)

1

(1)

7

(2)

1

(0)

7

(2)

1

(0)

7

(2)

1

(0)

7

(2)

1

(0)

7

(1)

1

(0)

7

(1)

1

(1)

7

(1)

1

(0)

7

(2)

1

(0)

7

(0)

1

(1)

3. Knowledge of Measurement - Students will identify attributes, units, or systems of measurement or apply a variety of techniques, formulas, tools or technology for determining measurements.

Operational Items

(Field Test Items)

6

(2)

1

(0)

6

(2)

1

(1)

6

(2)

1

(1)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(1)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

4. Knowledge of Statistics - Students will collect, organize, display, analyze, or interpret data to make decisions or predictions

Operational Items

(Field Test Items)

11

(1)

1

(0)

11

(5)

1

(0)

11

(3)

1

(0)

11

(2)

1

(1)

11

(3)

1

(0)

11

(3)

1

(0)

11

(3)

1

(0)

11

(2)

1

(1)

11

(2)

1

(1)

11

(2)

1

(0)

5. Knowledge of Probability - Students will use experimental methods or theoretical reasoning to determine probabilities to make predictions or solve problems about events whose outcomes involve random variation.

Operational Items

(Field Test Items)

2

(1)

0

(0)

2

(1)

0

(0)

2

(1)

0

(0)

2

(2)

0

(0)

2

(1)

0

(0)

2

(1)

0

(0)

2

(1)

0

(0)

2

(1)

0

(0)

2

(1)

0

(0)

2

(2)

0

(0)

6. Knowledge of Number Relationships or Computation - Students will describe, represent, or apply numbers or their relationships or will estimate or compute using mental strategies, paper/pencil or technology.

Operational Items

(Field Test Items)

13

(4)

3

(0)

13

(2)

3

(1)

13

(3)

3

(1)

13

(3)

3

(0)

13

(3)

3

(1)

13

(4)

3

(0)

13

(4)

3

(0)

13

(4)

3

(0)

13

(4)

3

(0)

13

(3)

3

(0)


301



SR BCR SR BCR SR BCR SR BCR SR BCR SR BCR SR BCR

SR

BCR

SR

BCR

SR

BCR


Operational Items

(Field Test Items)

13

(4)

1

(0)

13

(4)

1

(0)

13

(5)

1

(0)

13

(4)

1

(0)

13

(5)

1

(0)

13

(5)

1

(0)

13

(3)

1

(0)

13

(5)

1

(0)

13

(5)

1

(0)

13

(4)

1

(0)


Operational Items

(Field Test Items)

6

(2)

1

(1)

6

(1)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)


Operational Items

(Field Test Items)

6

(2)

1

(0)

6

(3)

1

(0)

6

(2)

1

(0)

6

(3)

1

(0)

6

(2)

1

(0)

6

(3)

1

(0)

6

(2)

1

(0)

6

(1)

1

(0)

6

(2)

1

(0)

6

(2)

1

(0)


Operational Items

(Field Test Items)

7

(2)

1

(0)

7

(2)

1

(0)

7

(1)

1

(0)

7

(1)

1

(0)

7

(1)

1

(0)

7

(2)

1

(0)

7

(6)

1

(0)

7

(4)

1

(0)

7

(2)

1

(0)

7

(4)

1

(0)


Operational Items

(Field Test Items)

6

(1)

1

(0)

6

(1)

1

(0)

6

(1)

1

(0)

6

(0)

1

(1)

6

(1)

1

(1)

6

(0)

1

(1)

6

(1)

1

(1)

6

(1)

1

(1)

6

(0)

1

(1)

6

(1)

1

(1)


Operational Items

(Field Test Items)

12

(2)

2

(1)

12

(2)

2

(2)

12

(1)

2

(3)

12

(3)

2

(1)

12

(1)

2

(1)

12

(2)

2

(1)

12

(2)

2

(1)

12

(2)

2

(1)

12

(3)

2

(1)

12

(2)

2

(1)


302


Standard


SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR


Operational Items

(Field Test Items)

13

(3)

1

(1)

1

(1)

13

(2)

1

(0)

1

(1)

13

(2)

1

(0)

1

(1)

13

(0)

1

(1)

1

(1)

13

(0)

1

(0)

1

(1)

13

(2)

1

(0)

1

(0)

13

(4)

1

(0)

1

(1)

13

(2)

1

(1)

1

(1)

13

(2)

1

(1)

1

(1)

13

(3)

1

(0)

1

(2)


Operational Items

(Field Test Items)

5

(2)

1

(0)

0

(0)

5

(2)

1

(0)

0

(0)

5

(4)

1

(0)

0

(0)

5

(2)

1

(0)

0

(0)

5

(1)

1

(0)

0

(0)

5

(0)

1

(1)

0

(0)

5

(1)

1

(1)

0

(0)

5

(0)

1

(0)

0

(0)

5

(3)

1

(0)

0

(0)

5

(1)

1

(0)

0

(0)


Operational Items

(Field Test Items)

7

(1)

1

(1)

0

(0)

7

(0)

1

(1)

0

(0)

7

(0)

1

(0)

0

(0)

7

(4)

1

(0)

0

(0)

7

(5)

1

(0)

0

(0)

7

(1)

1

(1)

0

(0)

7

(2)

1

(0)

0

(0)

7

(1)

1

(0)

0

(0)

7

(1)

1

(0)

0

(0)

7

(2)

1

(0)

0

(0)


Operational Items

(Field Test Items)

8

(3)

1

(0)

0

(0)

8

(3)

1

(0)

0

(0)

8

(3)

1

(0)

0

(0)

8

(3)

1

(1)

0

(0)

8

(4)

1

(1)

0

(0)

8

(3)

1

(0)

0

(0)

8

(4)

1

(1)

0

(0)

8

(3)

1

(0)

0

(0)

8

(4)

1

(1)

0

(0)

8

(3)

1

(1)

0

(0)


Operational Items

(Field Test Items)

3

(1)

1

(0)

0

(0)

3

(1)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)

3

(2)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)

3

(3)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)

3

(0)

1

(0)

0

(0)


Operational Items

(Field Test Items)

13

(3)

2

(0)

0

(0)

13

(3)

2

(1)

0

(0)

13

(3)

2

(2)

0

(0)

13

(3)

2

(0)

0

(0)

13

(2)

2

(1)

0

(0)

13

(3)

2

(0)

0

(0)

13

(2)

2

(0)

0

(0)

13

(3)

2

(1)

0

(0)

13

(2)

2

(0)

0

(0)

13

(3)

2

(1)

0

(0)


303


Standard


SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR

SR

BCR

ECR


Operational Items

(Field Test Items)

12

(2)

1

(0)

1

(1)

12

(2)

1

(0)

1

(1)

12

(3)

1

(0)

1

(1)

12

(1)

1

(0)

1

(1)

12

(4)

1

(0)

1

(1)

12

(3)

1

(0)

1

(1)

12

(2)

1

(1)

1

(1)

12

(5)

1

(0)

1

(1)

12

(5)

1

(1)

1

(1)

12

(6)

1

(0)

1

(1)


Operational Items

(Field Test Items)

7

(1)

1

(0)

0

(0)

7

(2)

1

(2)

0

(0)

7

(1)

1

(1)

0

(0)

7

(1)

1

(2)

0

(0)

7

(1)

1

(1)

0

(0)

7

(1)

1

(1)

0

(0)

7

(1)

1

(0)

0

(0)

7

(1)

1

(0)

0

(0)

7

(1)

1

(0)

0

(0)

7

(3)

1

(1)

0

(0)


Operational Items

(Field Test Items)

5

(1)

1

(0)

0

(0)

5

(2)

1

(1)

0

(0)

5

(2)

1

(1)

0

(0)

5

(0)

1

(1)

0

(0)

5

(2)

1

(1)

0

(0)

5

(1)

1

(3)

0

(0)

5

(0)

1

(0)

0

(0)

5

(1)

1

(1)

0

(0)

5

(1)

1

(1)

0

(0)

5

(0)

1

(1)

0

(0)


Operational Items

(Field Test Items)

8

(3)

1

(1)

0

(0)

8

(4)

1

(0)

0

(0)

8

(3)

1

(0)

0

(0)

8

(5)

1

(0)

0

(0)

8

(2)

1

(0)

0

(0)

8

(2)

1

(0)

0

(0)

8

(2)

1

(0)

0

(0)

8

(1)

1

(0)

0

(0)

8

(1)

1

(0)

0

(0)

8

(0)

1

(0)

0

(0)


Operational Items

(Field Test Items)

4

(2)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(2)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(1)

0

(0)

0

(0)

4

(0)

0

(0)

0

(0)


Operational Items

(Field Test Items)

12

(3)

2

(1)

0

(0)

12

(1)

2

(0)

0

(0)

12

(2)

2

(0)

0

(0)

12

(2)

2

(0)

0

(0)

12

(1)

2

(0)

0

(0)

12

(4)

2

(0)

0

(0)

12

(4)

2

(1)

0

(0)

12

(3)

2

(1)

0

(0)

12

(3)

2

(0)

0

(0)

12

(3)

2

(0)

0

(0)


304

Table D.1 The 2011 MSA-Math Blueprint: Grade 7- Forms A-E

Standard

Form A Form B Form C Form D Form E

SR BCR ECR SPR SR BCRECR SPR SR BCR ECR SPR SR BCR ECR SPR SR BCRECR SPR


Operational Items

(Field Test Items)

9

(4)

1

(0)

1

(1)

3

(1)

9

(2)

1

(1)

1

(0)

3

(1)

9

(1)

1

(1)

1

(0)

3

(0)

9

(3)

1

(1)

1

(1)

3

(0)

9

(1)

1

(1)

1

(0)

3

(0)


Operational Items

(Field Test Items)

4

(1)

1

(0)

2

(1)

0

(1)

4

(0)

1

(0)

2

(1)

0

(1)

4

(0)

1

(1)

2

(0)

0

(1)

4

(1)

1

(1)

2

(0)

0

(1)

4

(0)

1

(1)

2

(0)

0

(1)


Operational Items

(Field Test Items)

4

(0)

0

(1)

1

(0)

1

(0)

4

(1)

0

(0)

1

(0)

1

(0)

4

(0)

0

(0)

1

(1)

1

(0)

4

(0)

0

(0)

1

(0)

1

(0)

4

(0)

0

(0)

1

(0)

1

(1)


Operational Items

(Field Test Items)

5

(0)

1

(0)

1

(1)

2

(1)

5

(0)

1

(1)

1

(1)

2

(1)

5

(0)

1

(0)

1

(1)

2

(0)

5

(0)

1

(0)

1

(1)

2

(0)

5

(1)

1

(0)

1

(1)

2

(0)


Operational Items

(Field Test Items)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)


Operational Items

(Field Test Items)

11

(1)

0

(0)

0

(0)

3

(0)

11

(3)

0

(0)

0

(0)

3

(0)

11

(5)

0

(0)

0

(0)

3

(2)

11

(2)

0

(0)

0

(0)

3

(2)

11

(4)

0

(0)

0

(0)

3

(2)


305

Table D.1 The 2011 MSA-Math Blueprint: Grade 7 – Forms F-K

Standard

Form F Form G Form H Form J Form K

SR BCR ECRSPR

SR BCRECR

SPR

SRBCR

ECR

SPR SR BCR ECR SPR SR BCR ECRSPR


Operational Items

(Field Test Items)

9

(1)

1

(1)

1

(0)

3

(1)

9

(3)

1

(1)

1

(0)

3

(1)

9

(1)

1

(1)

1

(0)

3

(1)

9

(0)

1

(1)

1

(1)

3

(0)

9

(3)

1

(1)

1

(0)

3

(0)


Operational Items

(Field Test Items)

4

(0)

0

(0)

1

(1)

2

(1)

4

(0)

0

(0)

1

(1)

2

(1)

4

(1)

0

(0)

1

(0)

2

(1)

4

(0)

0

(0)

1

(0)

2

(1)

4

(0)

0

(1)

1

(1)

2

(0)


Operational Items

(Field Test Items)

4

(0)

1

(1)

0

(0)

1

(1)

4

(0)

1

(0)

0

(0)

1

(1)

4

(0)

1

(0)

0

(1)

1

(1)

4

(1)

1

(0)

0

(0)

1

(1)

4

(0)

1

(0)

0

(0)

1

(1)


Operational Items

(Field Test Items)

5

(0)

1

(0)

1

(1)

2

(0)

5

(1)

1

(1)

1

(1)

2

(0)

5

(1)

1

(0)

1

(1)

2

(0)

5

(2)

1

(0)

1

(1)

2

(0)

5

(2)

1

(0)

1

(1)

2

(1)


Operational Items

(Field Test Items)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)

3

(1)

1

(1)

0

(0)

1

(1)

3

(0)

1

(1)

0

(0)

1

(1)

3

(1)

1

(0)

0

(0)

1

(1)


Operational Items

(Field Test Items)

11

(4)

0

(0)

0

(0)

3

(0)

11

(2)

0

(0)

0

(0)

3

(0)

11

(3)

0

(0)

0

(0)

3

(0)

11

(3)

0

(0)

0

(0)

3

(1)

11

(1)

0

(0)

0

(0)

3

(1)


306

Table D.1 The 2011 MSA-Math Blueprint: Grade 8- Forms A-E

Standard

Form A Form B Form C Form D Form E

SR BCR ECR SPR SR BCRECR

SPR

SR BCR ECR SPR SR BCR ECR SPR SR BCR ECR SPR


Operational Items

(Field Test Items)

8

(2)

2

(2)

1

(0)

4

(2)

8

(2)

2

(1)

1

(0)

4

(1)

8

(1)

2

(1)

1

(0)

4

(1)

8

(0)

2

(1)

1

(0)

4

(1)

8

(1)

2

(1)

1

(0)

4

(2)


Operational Items

(Field Test Items)

5

(1)

1

(0)

0

(0)

2

(0)

5

(1)

1

(1)

0

(1)

2

(1)

5

(1)

1

(1)

0

(1)

2

(0)

5

(1)

1

(1)

0

(0)

2

(1)

5

(0)

1

(0)

0

(1)

2

(0)


Operational Items

(Field Test Items)

3

(0)

0

(0)

1

(1)

1

(0)

3

(0)

0

(0)

1

(0)

1

(0)

3

(1)

0

(0)

1

(0)

1

(1)

3

(2)

0

(0)

1

(1)

1

(0)

3

(1)

0

(0)

1

(0)

1

(0)


Operational Items

(Field Test Items)

6

(2)

1

(0)

1

(1)

1

(0)

6

(1)

1

(0)

1

(1)

1

(0)

6

(0)

1

(0)

1

(1)

1

(0)

6

(1)

1

(0)

1

(1)

1

(1)

6

(2)

1

(1)

1

(0)

1

(1)


Operational Items

(Field Test Items)

2

(1)

1

(1)

0

(0)

2

(0)

2

(0)

1

(0)

0

(0)

2

(1)

2

(0)

1

(0)

0

(0)

2

(0)

2

(1)

1

(0)

0

(0)

2

(1)

2

(1)

1

(1)

0

(0)

2

(0)


Operational Items

(Field Test Items)

10

(0)

0

(0)

0

(0)

2

(1)

10

(2)

0

(0)

0

(0)

2

(1)

10

(3)

0

(0)

0

(0)

2

(2)

10

(1)

0

(0)

0

(0)

2

(0)

10

(1)

0

(0)

0

(0)

2

(1)


307

Table D.1 The 2011 MSA-Math Blueprint: Grade 8 – Forms F-K

Standard

Form F Form G Form H Form J Form K

SR BCR

ECR

SPR

SRBCR

ECRSPR

SR BCRECR SPR SR BCR ECR SPR

SR BCR ECRSPR


Operational Items

(Field Test Items)

8

(2)

2

(2)

1

(0)

4

(2)

8

(2)

2

(1)

1

(1)

4

(2)

8

(1)

2

(1)

1

(0)

4

(1)

8

(0)

2

(1)

1

(1)

4

(0)

8

(1)

2

(1)

1

(0)

4

(2)


Operational Items

(Field Test Items)

5

(1)

0

(0)

1

(0)

2

(0)

5

(1)

0

(1)

1

(0)

2

(0)

5

(1)

0

(1)

1

(1)

2

(0)

5

(1)

0

(1)

1

(0)

2

(1)

5

(0)

0

(0)

1

(1)

2

(1)


Operational Items

(Field Test Items)

3

(0)

1

(0)

0

(1)

1

(0)

3

(0)

1

(0)

0

(0)

1

(0)

3

(1)

1

(0)

0

(0)

1

(0)

3

(1)

1

(0)

0

(0)

1

(0)

3

(0)

1

(0)

0

(0)

1

(0)


Operational Items

(Field Test Items)

6

(2)

1

(0)

1

(1)

1

(0)

6

(1)

1

(0)

1

(1)

1

(0)

6

(0)

1

(0)

1

(1)

1

(1)

6

(1)

1

(0)

1

(1)

1

(1)

6

(2)

1

(0)

1

(1)

1

(0)


Operational Items

(Field Test Items)

2

(1)

1

(0)

0

(0)

2

(1)

2

(0)

1

(0)

0

(0)

2

(1)

2

(0)

1

(0)

0

(0)

2

(0)

2

(1)

1

(0)

0

(0)

2

(0)

2

(1)

1

(0)

0

(1)

2

(0)


Operational Items

(Field Test Items)

10

(0)

0

(0)

0

(0)

2

(1)

10

(2)

0

(0)

0

(0)

2

(1)

10

(3)

0

(0)

0

(0)

2

(2)

10

(2)

0

(0)

0

(0)

2

(2)

10

(2)

0

(0)

0

(0)

2

(0)


308

Documents

Maryland School Assessment- Mathematicsarchives.marylandpublicschools.org/NR/rdonlyres/734553DF-3BE3-4E8C...Maryland School Assessment-Mathematics: Grades 3 through 8 2011 Administration