Technical report page 558 560

• 1.California Department of EducationStatewide Assessment DivisionCalifornia Standards Tests Technical ReportSpring 2009 AdministrationSubmitted March 1, 2010 Educational Testing Service Contract No. 5417

2. STAR ProgramTable of Contents (Summarized)Chapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4 Limitations of the Assessment ............................................................................................................................................ 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8 Test Variations, Accommodations, and Modifications .................................................................................................... 11 Scores .................................................................................................................................................................................. 12 References ........................................................................................................................................................................... 20Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196 Selection of Item Writers .................................................................................................................................................. 198 Item Review Process ......................................................................................................................................................... 199 Field Testing ...................................................................................................................................................................... 203 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208 Reference ........................................................................................................................................................................... 213 Appendix 4.BCluster Targets ........................................................................................................................................ 223Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253 Procedures to Maintain Standardization ......................................................................................................................... 258 Accommodations for Students with Disabilities ............................................................................................................ 260 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269 Types of Scores and Subscores ...................................................................................................................................... 270 Score Verification Procedures ......................................................................................................................................... 271 Overview of Score Aggregation Procedures .................................................................................................................. 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373 Reliability Analyses ........................................................................................................................................................... 375 Decision Classification Analyses .................................................................................................................................... 379 Validity Evidence ............................................................................................................................................................... 382Page i 3. STAR Program IRT Analyses ...................................................................................................................................................................... 394 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611 Quality Control of Scanning ............................................................................................................................................. 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613 Quality Control of Psychometric Processes ................................................................................................................... 615 Quality Control of Reporting ............................................................................................................................................ 617Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Page ii 4. STAR ProgramTable of Contents (Full)Acronyms and Initialisms Used in the CST Technical Report.............................................................................. xivChapter 1: Introduction......................................................................................................................................... 1 Background ........................................................................................................................................................................... 1 Test Purpose.......................................................................................................................................................................... 1 Test Content .......................................................................................................................................................................... 2 Intended Population .............................................................................................................................................................. 2 Intended Use and Purpose of Test Scores.......................................................................................................................... 3 Testing Window ..................................................................................................................................................................... 3 Significant STAR Developments in 2009 ............................................................................................................................. 4Elimination of California Achievement Tests, Sixth Edition Survey (CAT/6 Survey) .............................................................. 4Release of CST Exemplars .................................................................................................................................................... 4 Limitations of the Assessment ............................................................................................................................................ 4Score Interpretation ............................................................................................................................................................... 4Out-of-Level Testing .............................................................................................................................................................. 4Score Comparison ................................................................................................................................................................. 4 Groups and Organizations Involved with the STAR Program ........................................................................................... 5State Board of Education (SBE)............................................................................................................................................. 5California Department of Education (CDE) ............................................................................................................................ 5Contractors ............................................................................................................................................................................ 5 Overview of the Technical Report ........................................................................................................................................ 5 References ............................................................................................................................................................................. 7Chapter 2: An Overview of CST Processes ........................................................................................................ 8 Item Development ................................................................................................................................................................. 8Item Formats .......................................................................................................................................................................... 8Item Development Specifications ........................................................................................................................................... 8Item Banking .......................................................................................................................................................................... 8Item Refresh Rate and Released Test Questions (RTQ) ....................................................................................................... 9Test Length ............................................................................................................................................................................ 9Test Blueprint ......................................................................................................................................................................... 9Content Rules and Item Selection.......................................................................................................................................... 9Psychometric Criteria ........................................................................................................................................................... 10Test Security and Confidentiality.......................................................................................................................................... 10Procedures to Maintain Standardization .............................................................................................................................. 11 Test Variations, Accommodations, and Modifications .................................................................................................... 11Test Variations, Accommodation and Modification Summaries ........................................................................................... 12 Scores .................................................................................................................................................................................. 12Individual Scores.................................................................................................................................................................. 13Group Scores ....................................................................................................................................................................... 13Calibration ............................................................................................................................................................................ 14Scaling ................................................................................................................................................................................. 15True Score Equating ............................................................................................................................................................ 15Equating Samples ................................................................................................................................................................ 17Equating the Braille Versions of the CSTs ........................................................................................................................... 18Scaling the CSTs for Integrated/Coordinated Science ......................................................................................................... 18 References ........................................................................................................................................................................... 20 Appendix 2.ACST Item and Estimated Time Chart ....................................................................................................... 21 Appendix 2.BReporting Clusters .................................................................................................................................... 22EnglishLanguage Arts ........................................................................................................................................................ 22HistorySocial Science ........................................................................................................................................................ 24Mathematics......................................................................................................................................................................... 24Science ................................................................................................................................................................................ 27 Appendix 2.C2009 Test Variations, Accommodations, and Modifications ................................................................. 32 Appendix 2.DSpecial Services Summary Tables .......................................................................................................... 35Chapter 3: Item Development........................................................................................................................... 196 Rules for Item Development ............................................................................................................................................. 196Item Development Specifications ....................................................................................................................................... 196Expected Item Ratio ........................................................................................................................................................... 197 Selection of Item Writers .................................................................................................................................................. 198Criteria for Selecting Item Writers ...................................................................................................................................... 198Item Writer Training ........................................................................................................................................................... 198 Page iii 5. STAR Program Item Review Process ......................................................................................................................................................... 199Contractor Review ............................................................................................................................................................. 199Content Expert Reviews .................................................................................................................................................... 200Statewide Pupil Assessment Review (SPAR) Panel .......................................................................................................... 203 Field Testing ...................................................................................................................................................................... 203Stand-Alone Field Testing .................................................................................................................................................. 203Embedded Field-Test Items ............................................................................................................................................... 204 Data Review Meetings ....................................................................................................................................................... 205 Item Banking ...................................................................................................................................................................... 206 References ......................................................................................................................................................................... 207Chapter 4: Test Assembly ................................................................................................................................ 208 Test Length ........................................................................................................................................................................ 208 Rules for Item Selection.................................................................................................................................................... 208Test Blueprint ..................................................................................................................................................................... 208Content Rules and Item Selection...................................................................................................................................... 208Psychometric Criteria ......................................................................................................................................................... 209Projected Psychometric Properties of the Assembled Tests .............................................................................................. 211Rules for Item Sequence and Layout ................................................................................................................................. 212 Reference ........................................................................................................................................................................... 213 Appendix 4.ATechnical Characteristics....................................................................................................................... 214 Appendix 4.BCluster Targets ........................................................................................................................................ 223Chapter 5: Test Administration ........................................................................................................................ 253 Test Security and Confidentiality ..................................................................................................................................... 253ETSs Office of Testing Integrity (OTI) ............................................................................................................................... 253Test Development .............................................................................................................................................................. 253Item Review ....................................................................................................................................................................... 253Item Bank ........................................................................................................................................................................... 254Transfer of Forms and Items to the CDE ........................................................................................................................... 254Firewall .............................................................................................................................................................................. 254Printing............................................................................................................................................................................... 255Test Administration ............................................................................................................................................................ 255Test Delivery ...................................................................................................................................................................... 255Processing and Scoring ..................................................................................................................................................... 256Data Management ............................................................................................................................................................. 256Transfer of Scores via Secure Data Exchange .................................................................................................................. 257Statistical Analysis ............................................................................................................................................................. 257Reporting and Posting Results........................................................................................................................................... 257Student Confidentiality ....................................................................................................................................................... 257Test Results ....................................................................................................................................................................... 257 Procedures to Maintain Standardization ......................................................................................................................... 258Test Administrators ............................................................................................................................................................ 258Directions for Administration (DFA) .................................................................................................................................... 259District and Test Site Coordinator Manual ......................................................................................................................... 259STAR Management System Manuals ................................................................................................................................ 259Test Booklets ..................................................................................................................................................................... 260 Accommodations for Students with Disabilities ............................................................................................................ 260Identification ....................................................................................................................................................................... 260Scoring............................................................................................................................................................................... 261 Demographic Data Corrections ........................................................................................................................................ 261 Testing Irregularities ......................................................................................................................................................... 261 Test Administration Incidents .......................................................................................................................................... 261 References ......................................................................................................................................................................... 262Chapter 6: Performance Standards ................................................................................................................. 263 Background ....................................................................................................................................................................... 263 Standard Setting Procedure ............................................................................................................................................. 263Standard Setting Methodologies ........................................................................................................................................ 264 Results ............................................................................................................................................................................... 265 References ......................................................................................................................................................................... 268Chapter 7: Scoring and Reporting ................................................................................................................... 269 Procedures for Maintaining and Retrieving Individual Scores ...................................................................................... 269Scoring and Reporting Specifications ................................................................................................................................ 269Scanning and Scoring ........................................................................................................................................................ 270 Types of Scores and Subscores ...................................................................................................................................... 270Raw Score ......................................................................................................................................................................... 270 Page iv 6. STAR ProgramSubscore............................................................................................................................................................................ 270Scale Score........................................................................................................................................................................ 270Performance Levels ........................................................................................................................................................... 270Writing Score ..................................................................................................................................................................... 271 Score Verification Procedures ......................................................................................................................................... 271Scoring Key Verification Process ....................................................................................................................................... 271Monitoring and Quality Control of Writing Scoring ............................................................................................................. 272Score Verification Process ................................................................................................................................................. 274 Overview of Score Aggregation Procedures .................................................................................................................. 274Individual Scores................................................................................................................................................................ 274 Reports to be Produced and Scores for Each Report .................................................................................................... 278Types of Score Reports ..................................................................................................................................................... 278Score Report Contents ...................................................................................................................................................... 278Score Report Applications.................................................................................................................................................. 279 Criteria for Interpreting Test Scores ................................................................................................................................ 280 Criteria for Interpreting Score Reports ............................................................................................................................ 280 Reference ........................................................................................................................................................................... 281 Appendix 7.AELA for Writing (Grades Four and Seven) Rubrics .............................................................................. 282Grade Four Scoring Rubric ................................................................................................................................................ 282Grade Seven Scoring Rubric ............................................................................................................................................. 285 Appendix 7.BScale Score Distributions....................................................................................................................... 288 Appendix 7.CDemographic Summaries ....................................................................................................................... 292 Appendix 7.DTypes of Score Reports .......................................................................................................................... 368Chapter 8: Analyses .......................................................................................................................................... 372 Samples Used for the Analyses ....................................................................................................................................... 372 Classical Item Analyses .................................................................................................................................................... 373Multiple-Choice Items ........................................................................................................................................................ 373Writing Tasks ..................................................................................................................................................................... 374 Reliability Analyses ........................................................................................................................................................... 375Intercorrelations, Reliabilities, and SEMs for Reporting Clusters ....................................................................................... 377Subgroup Reliabilities and SEMs ....................................................................................................................................... 377Conditional Standard Errors of Measurement .................................................................................................................... 377 Decision Classification Analyses .................................................................................................................................... 379Writing Score Reliability ..................................................................................................................................................... 380Prompt and Rater Agreement Summary ............................................................................................................................ 381 Validity Evidence ............................................................................................................................................................... 382Purpose of the CST ........................................................................................................................................................... 382The Constructs to Be Measured ........................................................................................................................................ 382The Scores Generated and the Interpretations and Uses of these Scores ........................................................................ 383Intended Test Population(s) ............................................................................................................................................... 383Validity Evidence Collected ................................................................................................................................................ 384Evidence Based on Response Processes ......................................................................................................................... 391Evidence Based on Internal Structure ................................................................................................................................ 392Evidence Based on Consequences of Testing................................................................................................................... 394 IRT Analyses ...................................................................................................................................................................... 394IRT Model-Data Fit Analyses ............................................................................................................................................. 394Model Fit Assessment Results ........................................................................................................................................... 396Evaluation of Scaling ......................................................................................................................................................... 396Summaries of Scaled IRT b-values.................................................................................................................................... 398Post-Equating Results ....................................................................................................................................................... 398 DIF Analyses ...................................................................................................................................................................... 399 References ......................................................................................................................................................................... 401 Appendix 8.AClassical Analyses .................................................................................................................................. 403 Appendix 8.BReliability Analyses ................................................................................................................................ 415 Appendix 8.CValidity Analyses .................................................................................................................................... 497 Appendix 8.DIRT Analyses ........................................................................................................................................... 541Scaling and Post-Equating Results .................................................................................................................................... 557 Appendix 8.EDIF Analyses............................................................................................................................................ 579Chapter 9: Quality Control Procedures ...........................................................................................................609 Quality Control of Item Development .............................................................................................................................. 609Item Specifications ............................................................................................................................................................. 609Item Writers........................................................................................................................................................................ 609Internal Contractor Reviews ............................................................................................................................................... 609Assessment Review Panel (ARP) Review ......................................................................................................................... 610Statewide Pupil Assessment Review (SPAR) Panel Review ............................................................................................. 610 Page v 7. STAR ProgramData Review of Field Tested Items .................................................................................................................................... 610 Quality Control of the Item Bank ...................................................................................................................................... 611 Quality Control of Test Materials ..................................................................................................................................... 611Collecting Test Materials .................................................................................................................................................... 611Processing Test Materials .................................................................................................................................................. 612 Quality Control of Scanning ............................................................................................................................................. 612Post-scanning Edits ........................................................................................................................................................... 612 Quality Control of Image Editing ...................................................................................................................................... 613 Quality Control of Answer Document Processing and Scoring .................................................................................... 613Accountability of Answer Documents ................................................................................................................................. 613Processing of Answer Documents ..................................................................................................................................... 613Scoring and Reporting Specifications ................................................................................................................................ 614Matching Information on CST Answer Documents ............................................................................................................ 614Matching Multiple-choice and Writing Scores for ELA Grades Four and Seven ................................................................ 614Storing Answer Documents................................................................................................................................................ 614 Quality Control of Psychometric Processes ................................................................................................................... 615Score Key Verification Procedures .................................................................................................................................... 615Quality Control of Item Analyses, DIF and Equating Process ............................................................................................ 615Score Verification Process ................................................................................................................................................. 616Offloads to Test Development............................................................................................................................................ 616 Quality Control of Reporting ............................................................................................................................................ 617Excluding Student Scores from Summary Reports ............................................................................................................ 617Chapter 10: Historical Comparisons ............................................................................................................... 618 Examinee Performance ..................................................................................................................................................... 619 Test Characteristics .......................................................................................................................................................... 620 References ......................................................................................................................................................................... 621 Appendix 10.AHistorical Comparisons Tables ........................................................................................................... 622 Appendix 10.BHistorical Comparisons Tables ........................................................................................................... 632Tables Table 2.1 Subgroup Definitions ........................................................................................................................................... 13Table 2.2 Scale Scores Ranges for Performance Levels .................................................................................................... 16Table 2.3 Sample Selected to Establish Reference Form ................................................................................................... 17Table 2.4 Proficiency-Level Cut Points for Science End-of-Course CSTs .......................................................................... 19Table 2.5 Computation of Proficiency-Level Cut Points on Integrated/Coordinated Science 1 ........................................... 19Table 2.C.1 Matrix of Test Variations, Accommodations, and Modifications ........................................................................ 32Table 2.C.2 Matrix of Test Variations for English Learners .................................................................................................. 34Table 2.D. 1 Special Services Summary for ELA, Grades Two and Three ......................................................................... 35Table 2.D.2 Special Services Summary for ELA, Grade Four............................................................................................. 40Table 2.D.3 Special Services Summary for ELA, Grades Five and Six .............................................................................. 45Table 2.D.4 Special Services Summary for ELA, Grade Seven .......................................................................................... 49Table 2.D.5 Special Services Summary for ELA, Grades Eight and Nine ........................................................................... 54Table 2.D.6 Special Services Summary for ELA, Grades Ten and Eleven ......................................................................... 58Table 2.D.7 Special Services Summary for HistorySocial Science, Grade Eight .............................................................. 62Table 2.D.8 Special Services Summary for World History .................................................................................................. 67Table 2.D.9 Special Services Summary for HistorySocial Science, Grade Eleven ........................................................... 72Table 2.D.10 Special Services Summary for Mathematics, Grades Two and Three........................................................... 77Table 2.D.11 Special Services Summary for Mathematics, Grades Four and Five............................................................. 82Table 2.D.12 Special Services Summary for Mathematics, Grades Six and Seven............................................................ 87Table 2.D.13 Special Services Summary for General Mathematics.................................................................................... 92Table 2.D.14 Special Services Summary for Algebra I ....................................................................................................... 97Table 2.D.15 Special Services Summary for Geometry .................................................................................................... 102Table 2.D.16 Special Services Summary for Algebra II .................................................................................................... 107Table 2.D.17 Special Services Summary for Summative High School Mathematics ........................................................ 112Table 2.D.18 Special Services Summary for Integrated Mathematics 1 ........................................................................... 118Table 2.D.19 Special Services Summary for Integrated Mathematics 2 ........................................................................... 123Table 2.D.20 Special Services Summary for Integrated Mathematics 3 ........................................................................... 128Table 2.D.21 Special Services Summary for Science, Grade Five ................................................................................... 133Table 2.D.22 Special Services Summary for Science, Grade Eight .................................................................................. 138Table 2.D.23 Special Services Summary for Grade Ten Life Science .............................................................................. 143Table 2.D.24 Special Services Summary for Biology ........................................................................................................ 148Table 2.D.25 Special Services Summary for Chemistry ................................................................................................... 154Table 2.D.26 Special Services Summary for Earth Science ............................................................................................. 160Table 2.D.27 Special Services Summary for Physics ....................................................................................................... 166Table 2.D.28 Special Services Summary for Integrated/Coordinated Science 1 .............................................................. 172 Page vi 8. STAR ProgramTable 2.D.29 Special Services Summary for Integrated/Coordinated Science 2 .............................................................. 178Table 2.D.37 Special Services Summary for Integrated/Coordinated Science 3 ............................................................... 184Table 2.D.30 Special Services Summary for Integrated/Coordinated Science 4 .............................................................. 190Table 3.1 Field-test Percentages for the CSTs ................................................................................................................. 198Table 3.2 CST ARP Member Qualifications, by subject and total ..................................................................................... 201Table 3.3 Summary of Items and Forms Presented in 2009 CSTs ................................................................................... 204Table 4.1 Target Statistical Specifications for the CSTs ................................................................................................... 210Table 4.A.1 Summary of 2009 CST Projected Technical Characteristics ......................................................................... 214Table 4.A.2 Summary of 2009 CST Projected Statistical Attributes .................................................................................. 215Table 6.1 Scale Scores Ranges for Performance Levels .................................................................................................. 265Table 6.2 Percentage of Examinees in Each Performance Level ..................................................................................... 266Table 7.1 Mean and Standard Deviation of Raw and Scale Scores for CSTs .................................................................. 275Table 7.2 Percentage of Examinees in Proficiency Levels for CSTs ................................................................................ 276Table 7.3 Subgroup Definitions ......................................................................................................................................... 277Table 7.B.1 Distribution of CST Scale Scores for ELA ...................................................................................................... 288Table 7.B.2 Distribution of CST Scale Scores for HistorySocial Science ........................................................................ 288Table 7.B.3 Distribution of CST Scale Scores for Mathematics, Grades Two through Seven .......................................... 289Table 7.B.4 Distribution of Scale Scores for Mathematics End-of-Course CSTs .............................................................. 289Table 7.B.5 Distribution of CST Scale Scores for Science ................................................................................................ 290Table 7.B.6 Distribution of CST Scale Scores for a Grade-Specific Population ................................................................ 291Table 7.C.1 Demographic Summary for ELA, Grade Two ................................................................................................ 292Table 7.C.2 Demographic Summary for ELA, Grade Three .............................................................................................. 294Table 7.C.3 Demographic Summary for ELA, Grade Four ................................................................................................ 296Table 7.C.4 Demographic Summary for ELA, Grade Five ................................................................................................ 298Table 7.C.5 Demographic Summary for ELA, Grade Six .................................................................................................. 300Table 7.C.6 Demographic Summary for ELA, Grade Seven ............................................................................................. 302Table 7.C.7 Demographic Summary for ELA, Grade Eight ............................................................................................... 304Table 7.C.8 Demographic Summary for ELA, Grade Nine ................................................................................................ 306Table 7.C.9 Demographic Summary for ELA, Grade Ten ................................................................................................. 308Table 7.C.10 Demographic Summary for ELA, Grade Eleven .......................................................................................... 310Table 7.C.11 Demographic Summary for HistorySocial Science, Grade Eight ............................................................... 312Table 7.C.12 Demographic Summary for World History ................................................................................................... 314Table 7.C.13 Demographic Summary for HistorySocial Science, Grade Eleven ............................................................ 316Table 7.C.14 Demographic Summary for Mathematics, Grade Two ................................................................................. 318Table 7.C.15 Demographic Summary for Mathematics, Grade Three .............................................................................. 320Table 7.C.16 Demographic Summary for Mathematics, Grade Four ................................................................................ 322Table 7.C.17 Demographic Summary for Mathematics, Grade Five ................................................................................. 324Table 7.C.18 Demographic Summary for Mathematics, Grade Six................................................................................... 326Table 7.C.19 Demographic Summary for Mathematics, Grade Seven ............................................................................. 328Table 7.C.20 Demographic Summary for General Mathematics ....................................................................................... 330Table 7.C.21 Demographic Summary for Algebra I .......................................................................................................... 332Table 7.C.22 Demographic Summary for Geometry ......................................................................................................... 334Table 7.C.23 Demographic Summary for Algebra II ......................................................................................................... 336Table 7.C.24 Demographic Summary for Summative High School Mathematics ............................................................. 338Table 7.C.25 Demographic Summary for Integrated Mathematics 1 ................................................................................ 340Table 7.C.26 Demographic Summary for Integrated Mathematics 2 ................................................................................ 342Table 7.C.27 Demographic Summary for Integrated Mathematics 3 ................................................................................ 344Table 7.C.28 Demographic Summary for Science, Grade Five ........................................................................................ 346Table 7.C.29 Demographic Summary for Science, Grade Eight ....................................................................................... 348Table 7.C.30 Demographic Summary for Grade Ten Life Science ................................................................................... 350Table 7.C.31 Demographic Summary for Biology ............................................................................................................. 352Table 7.C.32 Demographic Summary for Chemistry ......................................................................................................... 354Table 7.C.33 Demographic Summary for Earth Science .................................................................................................. 356Table 7.C.34 Demographic Summary for Physics ............................................................................................................ 358Table 7.C.35 Demographic Summary for Integrated/Coordinated Science 1 ................................................................... 360Table 7.C.36 Demographic Summary for Integrated/Coordinated Science 2 ................................................................... 362Table 7.C.37 Demographic Summary for Integrated/Coordinated Science 3 ................................................................... 364Table 7.C.38 Demographic Summary for Integrated/Coordinated Science 4 ................................................................... 366Table 7.D.1 Score Reports Reflecting CST Results .......................................................................................................... 368Table 8.1 Mean and Median Proportion Correct and Point-Biserial .................................................................................. 374Table 8.2 Reliabilities and Standard Errors of Measurement for the CSTs ....................................................................... 376Table 8.3 Scale Score CSEM at Cut Points ...................................................................................................................... 378Table 8.4 2009 CST Content Area Correlations (All Valid Scores) ................................................................................... 388Table 8.5 Evaluation of Common Items between New and Reference Test Forms .......................................................... 397Table 8.A.1 Item-by-item p-value and Point-Biserial for ELA ............................................................................................ 403Table 8.A.2 Item-by-item p-value and Point-Biserial for HistorySocial Science .............................................................. 405 Page vii 9. STAR Program Table 8.A.3 Item-by-item p-value and Point-Biserial for Mathematics ............................................................................... 406 Table 8.A.4 Item-by-item p-value and Point-Biserial for Mathematics [continued] ............................................................ 407 Table 8.A.5 Item-by-item p-value and Point-Biserial for Science ...................................................................................... 408 Table 8.A.6 Item-by-item p-value and Point-Biserial for Science [continued] .................................................................... 409 Table 8.A.7 Item-by-item p-value and Point-Biserial for Grade-Specific CSTs ................................................................. 411 Table 8.A.8 Distribution of Essay Scores for ELA (Grade Four)Overall and by Subgroup (all %) ................................. 412 Table 8.A.9 Distribution of Essay Scores for ELA (Grade Seven)Overall and by Subgroup (all %) .............................. 412 Table 8.A.10 Mean Scores for ELA (Grade Four Essay)Overall and by Subgroup ....................................................... 413 Table 8.A.11 Mean Scores for ELA (Grade Seven Essay)Overall and by Subgroup .................................................... 413 Table 8.A.12 Effect Sizes for ELA (Grades Four and Seven Essay)by Subgroup ......................................................... 414 Table 8.B.1 Reliabilities and Correlations for ELA ............................................................................................................ 415 Table 8.B.2 Reliabilities and Correlations for HistorySocial Science ............................................................................... 416 Table 8.B.3 Subscore Reliabilities and Correlations for Mathematics ............................................................................... 417 Table 8.B.4 Subscore Reliabilities and Correlations for Science ...................................................................................... 420 Table 8.B.5 Reliabilities and Correlations for Grade-Specific Tests .................................................................................. 422 Table 8.B.6 Reliabilities and SEMs for the CSTs by Gender ............................................................................................ 423 Table 8.B.7 Reliabilities and SEMs for the CSTs by Economic Status ............................................................................. 424 Table 8.B.8 Reliabilities and SEMs for the CSTs by Special Services .............................................................................. 425 Table 8.B.9 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................... 426 Table 8.B.10 Reliabilities and SEMs for the CSTs by English Fluency ............................................................................. 427 Table 8.B.11 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 428 Table 8.B.12 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 429 Table 8.B.13 Reliabilities and SEMs for the CSTs by Primary Ethnicity ........................................................................... 430 Table 8.B.14 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 431 Table 8.B.15 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 432 Table 8.B.16 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Not Economically Disadvantaged ................ 433 Table 8.B.17 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 434 Table 8.B.18 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 435 Table 8.B.19 Reliabilities and SEMs for the CSTs by Primary Ethnicity-for-Economically Disadvantaged ....................... 436 Table 8.B.20 Subscore Reliabilities and SEM for ELA by Gender/Economic Status ........................................................ 437 Table 8.B.21 Subscore Reliabilities and SEM for HistorySocial Science by Gender/Economic Status ........................... 438 Table 8.B.22 Subscore Reliabilities and SEM for Mathematics by Gender/Economic Status ........................................... 439 Table 8.B.23 Subscore Reliabilities and SEM for Science by Gender/Economic Status .................................................. 441 Table 8.B.24 Subscore Reliabilities and SEM for Grade-Specific CSTs by Gender/Economic Status.............................. 443 Table 8.B.25 Subscore Reliabilities and SEM for ELA by Special Services/English Fluency ........................................... 444 Table 8.B.26 Subscore Reliabilities and SEM for HistorySocial Science by Special Services/English Fluency .............. 446 Table 8.B.27 Subscore Reliabilities and SEM for Mathematics by Special Services/English Fluency .............................. 447 Table 8.B.28 Subscore Reliabilities and SEM for Science by Special Services/English Fluency...................................... 450 Table 8.B.29 Subscore Reliabilities and SEM for Grade-Specific CSTs by Special Services/English Fluency ................. 452 Table 8.B.30 Subscore Reliabilities and SEM for ELA by Primary Ethnicity ..................................................................... 453 Table 8.B.31 Subscore Reliabilities and SEM for HistorySocial Science by Primary Ethnicity........................................ 455 Table 8.B.32 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity ........................................................ 456 Table 8.B.33 Subscore Reliabilities and SEM for Science by Primary Ethnicity ............................................................... 459 Table 8.B.34 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity........................................... 461 Table 8.B.35 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Not Economically Disadvantaged .......... 462 Table 8.B.36 Subscore Reliabilities and SEM for HistorySocial Science by Primary Ethnicity-for-Not Economically Disadvantaged .......................................................................................................................................... 464 Table 8.B.37 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-Not EconomicallyDisadvantaged ................................................................................................................................................................ 465 Table 8.B.38 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Not EconomicallyDisadvantaged ................................................................................................................................................................ 468 Table 8.B.39 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for-Not EconomicallyDisadvantaged ................................................................................................................................................................ 470 Table 8.B.40 Subscore Reliabilities and SEM for ELA by Primary Ethnicity-for-Economically Disadvantaged ................. 471 Table 8.B.41 Subscore Reliabilities and SEM for HistorySocial Science by Primary Ethnicity-for-EconomicallyDisadvantaged ................................................................................................................................................................ 473 Table 8.B.42 Subscore Reliabilities and SEM for Mathematics by Primary Ethnicity-for-EconomicallyDisadvantaged ................................................................................................................................................................ 474 Table 8.B.43 Subscore Reliabilities and SEM for Science by Primary Ethnicity-for-Economically Disadvantaged ........... 477 Table 8.B.44 Subscore Reliabilities and SEM for Grade-Specific CSTs by Primary Ethnicity-for-Economically Disadvantaged .......................................................................................................................................... 479 Table 8.B.45 Reliability of Classification for ELA, Grade Two ........................................................................................... 480 Table 8.B.46 Reliability of Classification for ELA, Grade Three ........................................................................................ 480 Table 8.B.47 Reliability of Classification for ELA, Grade Four (Reading only) .................................................................. 480 Table 8.B.48 Reliability of Classification for ELA, Grade Four (Reading and Writing) ...................................................... 481 Table 8.B.49 Reliability of Classification for ELA, Grade Five ........................................................................................... 481 Page viii 10. STAR ProgramTable 8.B.50 Reliability of Classification for ELA, Grade Six............................................................................................. 481Table 8.B.51 Reliability of Classification for ELA, Grade Seven (Reading only) ............................................................... 482Table 8.B.52 Reliability of Classification for ELA, Grade Seven (Reading and Writing) .................................................... 482Table 8.B.53 Reliability of Classification for ELA, Grade Eight ......................................................................................... 482Table 8.B.54 Reliability of Classification for ELA, Grade Nine .......................................................................................... 483Table 8.B.55 Reliability of Classification for ELA, Grade Ten ........................................................................................... 483Table 8.B.56 Reliability of Classification for ELA, Grade Eleven....................................................................................... 483Table 8.B.57 Reliability of Classification for HistorySocial Science, Grade Eight ............................................................ 484Table 8.B.58 Reliability of Classification for World History ................................................................................................ 484Table 8.B.59 Reliability of Classification for HistorySocial Science, Grade Eleven ......................................................... 484Table 8.B.60 Reliability of Classification for Mathematics, Grade Two ............................................................................. 485Table 8.B.61 Reliability of Classification for Mathematics, Grade Three ........................................................................... 485Table 8.B.62 Reliability of Classification for Mathematics, Grade Four ............................................................................. 485Table 8.B.63 Reliability of Classification for Mathematics, Grade Five ............................................................................. 486Table 8.B.64 Reliability of Classification for Mathematics, Grade Six ............................................................................... 486Table 8.B.65 Reliability of Classification for Mathematics, Grade Seven .......................................................................... 486Table 8.B.66 Reliability of Classification for General Mathematics ................................................................................... 487Table 8.B.67 Reliability of Classification for Algebra I ....................................................................................................... 487Table 8.B.68 Reliability of Classification for Geometry ..................................................................................................... 487Table 8.B.69 Reliability of Classification for Algebra II ...................................................................................................... 488Table 8.B.70 Reliability of Classification for Summative High School Mathematics .......................................................... 488Table 8.B.71 Reliability of Classification for Integrated Mathematics 1 ............................................................................. 488Table 8.B.72 Reliability of Classification for Integrated Mathematics 2 ............................................................................. 489Table 8.B.73 Reliability of Classification for Integrated Mathematics 3 ............................................................................. 489Table 8.B.74 Reliability of Classification for Science, Grade Five ..................................................................................... 489Table 8.B.75 Reliability of Classification for Science, Grade Eight ................................................................................... 490Table 8.B.76 Reliability of Classification for Grade 10 Life Science .................................................................................. 490Table 8.B.77 Reliability of Classification for Biology ......................................................................................................... 490Table 8.B.78 Reliability of Classification for Chemistry ..................................................................................................... 491Table 8.B.79 Reliability of Classification for Earth Science ............................................................................................... 491Table 8.B.80 Reliability of Classification for Physics ......................................................................................................... 491Table 8.B.81 Reliability of Classification for Integrated/Coordinated Science 1 ................................................................ 492Table 8.B.82 Reliability of Classification for Integrated/Coordinated Science 2 ................................................................ 492Table 8.B.83 Reliability of Classification for Integrated/Coordinated Science 3 ................................................................ 492Table 8.B.84 Reliability of Classification for Integrated/Coordinated Science 4 ................................................................ 493Table 8.B.85 Reliability of Classification for Algebra I Grade 8 ...................................................................................... 493Table 8.B.86 Reliability of Classification for Geometry Grade 9..................................................................................... 493Table 8.B.87 Reliability of Classification for Algebra II Grade 10 ................................................................................... 494Table 8.B.88 Reliability of Classification for Summative High School Mathematics Grade 11 ....................................... 494Table 8.B.89 Reliability of Classification for World History-Grade 10 ................................................................................ 494Table 8.B.90 Inter-Rater Agreement Analyses for ELA (Grade Four) ............................................................................... 495Table 8.B.91 Inter-Rater Agreement Analyses for (ELA Grade Seven) ............................................................................ 495Table 8.B.92 Descriptive Statistics for the Ratings by the Two Raters ............................................................................. 495Table 8.B.93 Generalizability Analyses for Grade Four Essay[(Person: Essay) x Rater] .............................................. 496Table 8.B.94 Generalizability Analyses for Grade Seven Essay[(Person: Essay) x Rater] ........................................... 496Table 8.C.1 2009 CST Content Area Correlations (Male) ................................................................................................. 497Table 8.C.2 2009 CST Content Area Correlations (Female) ............................................................................................. 499Table 8.C.3 2009 CST Content Area Correlations (Gender unknown) ............................................................................. 501Table 8.C.4 2009 CST Content Area Correlations (American Indian) ............................................................................... 503Table 8.C.5 2009 CST Content Area Correlations (Asian American) ................................................................................ 505Table 8.C.6 2009 CST Content Area Correlations (Pacific Islander) ................................................................................ 507Table 8.C.7 2009 CST Content Area Correlations (Filipino) ............................................................................................. 509Table 8.C.8 2009 CST Content Area Correlations (Hispanic) ........................................................................................... 511Table 8.C.9 2009 CST Content Area Correlations (African American) ............................................................................. 513Table 8.C.10 2009 CST Content Area Correlations (White).............................................................................................. 515Table 8.C.11 2009 CST Content Area Correlations (Ethnicity unknown) .......................................................................... 517Table 8.C.12 2009 CST Content Area Correlations (English Only) ................................................................................... 519Table 8.C.13 2009 CST Content Area Correlations (Initially Fluent English Proficient) .................................................... 521Table 8.C.14 2009 CST Content Area Correlations (English Learner) .............................................................................. 523Table 8.C.15 2009 CST Content Area Correlations (Reclassified Fluent English Proficient) ............................................ 525Table 8.C.16 2009 CST Content Area Correlations (English Fluency unknown) .............................................................. 527Table 8.C.17 2009 CST Content Area Correlations (Not Economically Disadvantaged) .................................................. 529Table 8.C.18 2009 CST Content Area Correlations (Economically Disadvantaged) ......................................................... 531Table 8.C.19 2009 CST Content Area Correlations (Unknown Economic Status) ............................................................ 533Table 8.C.20 2009 CST Content Area Correlations (No Special Education Services) ...................................................... 535Table 8.C.21 2009 CST Content Area Correlations (Special Education Services) ........................................................... 537Page ix 11. STAR Program Table 8.C.22 2009 CST Content Area Correlations (Special Education unknown) ........................................................... 539 Table 8.D.1 IRT Model Data Fit Distribution for ELA ......................................................................................................... 541 Table 8.D.2 IRT Model Data Fit Distribution for HistorySocial Science ........................................................................... 541 Table 8.D.3 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven ............................................. 541 Table 8.D.4 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests ......................................................... 541 Table 8.D.5 IRT Model Data Fit Distribution for Science ................................................................................................... 542 Table 8.D.6 IRT Model Data Fit Distribution for ELA (field-test items) .............................................................................. 542 Table 8.D.7 IRT Model Data Fit Distribution for HistorySocial Science (field-test items) ................................................ 542 Table 8.D.8 IRT Model Data Fit Distribution for Mathematics, Grades Two through Seven (field-test items) ................... 542 Table 8.D.9 IRT Model Data Fit Distribution for Mathematics End-of-Course Tests (field-test items) ............................... 543 Table 8.D.10 IRT Model Data Fit Distribution for Science (field-test items) ...................................................................... 543 Table 8.D.11 IRT b-values for ELA, Grade Two................................................................................................................ 543 Table 8.D.12 IRT b-values for ELA, Grade Three ............................................................................................................. 543 Table 8.D.13 IRT b-values for ELA, Grade Four ............................................................................................................... 544 Table 8.D.14 IRT b-values for ELA, Grade Five................................................................................................................ 544 Table 8.D.15 IRT b-values for ELA, Grade Six ................................................................................................................. 544 Table 8.D.16 IRT b-values for ELA, Grade Seven ............................................................................................................ 544 Table 8.D.17 IRT b-values for ELA, Grade Eight .............................................................................................................. 545 Table 8.D.18 IRT b-values for ELA, Grade Nine ............................................................................................................... 545 Table 8.D.19 IRT b-values for ELA, Grade Ten ................................................................................................................ 545 Table 8.D.20 IRT b-values for ELA, Grade Eleven ........................................................................................................... 545 Table 8.D.21 IRT b-values for HistorySocial Science, Grade Eight ................................................................................ 546 Table 8.D.22 IRT b-values for World History..................................................................................................................... 546 Table 8.D.23 IRT b-values for HistorySocial Science Grade Eleven ............................................................................... 546 Table 8.D.24 IRT b-values for Mathematics, Grade Two .................................................................................................. 546 Table 8.D.25 IRT b-values for Mathematics, Grade Three ............................................................................................... 547 Table 8.D.26 IRT b-values for Mathematics, Grade Four ................................................................................................. 547 Table 8.D.27 IRT b-values for Mathematics, Grade Five .................................................................................................. 547 Table 8.D.28 IRT b-values for Mathematics, Grade Six .................................................................................................... 547 Table 8.D.29 IRT b-values for Mathematics, Grade Seven ............................................................................................... 548 Table 8.D.30 IRT b-values for General Mathematics ........................................................................................................ 548 Table 8.D.31 IRT b-values for Algebra I ............................................................................................................................ 548 Table 8.D.32 IRT b-values for Geometry .......................................................................................................................... 548 Table 8.D.33 IRT b-values for Algebra II .....................................................................