Evaluating individuals Evaluating classes Noting unusual patterns or trends Identifying gaps in learning USE OF SCORES To show overall classroom trends with respect to the learning progression To show the status of each individual student with respect to the learning progression To identify unusual patterns in a particular item or a particular student To identify remaining gaps in individual or class performance HOW TO RECORD AND INTERPRET SCORES GRADE LEVEL LEARNING PROGRESSION K is highest LP level in the specific LP of interest You might expect student performance by item to look like this (all dichotomous or right/wrong items) Students ordered by performance Items ordered by difficulty IN A PERFECT WORLD StudentEasiest item Hardest item Lowest performing Highest performing 11111 FOR RUBRIC SCORED ITEMS StudentEasiest item Hardest item Lowest performing Highest performing 55443 ARRANGE CLASS DATA ORDERED BY ITEM LEVEL AND STUDENT PERFORMANCE Right/wrong items code to the matching LP level (for LP level 2, right is 2, wrong is 0 see item 4) Rubric items should be scored by building block matching each rubric level Results might look like this CODE STUDENT DATA BY LP LEVEL ACHIEVED Pretest Posttest Class has gone from at least some mastery of Level 1 for most students (some students are doing better) to mastery of Levels 1 and 2, and partial mastery of Level 3 INTERPRETING CLASS TRENDS Student 1 is at the lowest level (lower anchor) with some partial mastery of LP level 1 Student 2 is at LP level 1; partial mastery of LP 2 Student 3 is near mastery of LP 3 Student 4 is at LP 3; perhaps some mastery of upper anchor INTERPRETING INDIVIDUAL STUDENT PERFORMANCE GAPS IN INDIVIDUAL PERFORMANCE Student 1 has several gaps in mastery of LP level 1 (items 2, and 7-9; note item 4 is dichotomous at LP 2) Student 2 has a gap in mastery of LP 2 (item 2, and items 6-9) Etc. Although the class is mostly at LP levels 2 and 3, notice that item 3 is unusually difficult May be due to a problem with the item May be due to a gap in class understanding of a concept UNUSUAL ITEM PATTERNS Note that Student 1 has an unusual pattern of responses. Although this student is doing somewhat poorly overall (and is the lowest scoring student), they also did very well on items 5, 6, and 9, achieving the maximum score on these items. It might be worth checking in with this student to learn more about their performance. UNUSUAL STUDENT PATTERNS Wilson, M. (2005). Constructing measures: An item response modeling approach. New York: Psychology Press, Taylor & Francis Group. Wilson, M., & Sloane, K. (2000). From principles to practice: An embedded assessment system. Applied Measurement in Education, 13 (2), pp Wright, B., & Masters, G. (1982). Rating Scale Analysis. Chicago: MESA Press. BIBLIOGRAPHY Use of Scores PPT by the Oregon Department of Education and Berkeley Evaluation and Assessment Research Center is licensed under a CC BY 4.0.Oregon Department of EducationBerkeley Evaluation and Assessment Research CenterCC BY 4.0 You are free to: Share copy and redistribute the material in any medium or format Adapt remix, transform, and build upon the material Under the following terms: Attribution You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.appropriate creditindicate if changes were made NonCommercial You may not use the material for commercial purposes.commercial purposes ShareAlike If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.same license Oregon Department of Education welcomes editing of these resources and would greatly appreciate being able to learn from the changes made. To share an edited version of this resource, please contact Cristen McLean, CREATIVE COMMONS LICENSE