Embed Size (px)
Citation preview
December 5 and 6, 2012
Formative Assessment Items and Tasks – in OAS
1
Phase I of Formative Item Bank
Background onRT3 Assessment Initiatives
• Purpose – To provide assessment resources that reflect the
rigor of the CCGPS– To balance the use of formative and summative
assessments in the classroom– To promote student learning
2
RT3 Assessment Initiatives• Development of a three-prong toolkit to support
teachers and districts and to promote student learning– An assessment literacy professional learning
opportunity that focuses on implementation of formative instruction – Initial Pilot January 2013
– A set of interim benchmarks in ELA, Math, and selected grades for Science and Social Studies plus additional items – Initial Pilot Spring 2013
– An expansive bank of formative instructional assessment items/tasks based on CCGPS in ELA and Mathematics as a teacher resource - Phase I Release Fall 2012; Phase II release Fall 2013
3
Formative Instructional Practices
4
• Design• Teach
• Re-Design• Teach
• Re-Design• Teach
• Re-Design• Teach Common Core
State Standards
Formative Instructional Practices• Conducted during instruction (lesson, unit, etc.)• Identifies student strengths and weaknesses• Helps teacher determine next steps
– Review– Differentiation– Continuation
• Supplies information to provide students with detailed feedback
• Assessment for the purpose of improving achievement• LOW stakes
5
Purpose of the Formative Item Bank
The purpose of the Formative Item Bank is to provide items and tasks for teachers to use in order to assess students’ knowledge while they are learning the curriculum. •To provide an opportunity for students to show what they know and show teachers what students do not understand•To provide teachers good examples of tasks/items that assess the rigor of the Common Core.
6
Formative Instructional Assessment Items
• Assess students’ conceptual and skill-level understanding
• Tasks require students to– Apply the content they know to real world problems– Express logical reasoning by writing, showing their
work and/or explaining their answer
7
Items/Tasks for Formative Instructional Use
• Aligned to CCGPS for Grades 3 - High School ELA & Math• Format similar to PARCC’s Common Assessments
prototypes - – In other words, items/tasks are more rigorous and require
students to show what they know and how they know it
• Item Development Process (same as used for state tests)– Written to align with curriculum – Piloted in RT3 classrooms in spring 2012– Reviewed with student data and approved by Georgia Educators
8
Formative Item Bank• Item types
– About 1700 Formative Classroom Assessment Items • About 750 Phase I items Phase I available in OAS as of Nov. 30, 2012
– Some multiple choice items– Most are open-ended or constructed-response format
• Holistic Rubric• Exemplar Paper (i.e.,“ideal response”)• Student Exemplars/Responses
– DOK targets• DOK 2; 19%• DOK 3; 41%• DOK 4; 39%
9
Anchors used in scoring
The Essence of the Common Core State Standards
10
The Common Core State Standards focus on students learning how to think, not just doing by rote memorization or quick scan-reading, and sets higher expectations on student comprehension and their ability to articulate their knowledge.
Capturing the Essence of ELA Common Core
• ELA CCSS require students to use evidence from what lies within the four corners of the text and make valid claims that can be proven with the text.
• Students must be more adept at explaining that evidence orally and in writing.
11
Capturing the Essence of ELA Common Core
• In order to be college-, career-, and life-ready, students need to be familiar and comfortable with a broad range of genres and formats and much greater emphasis on informational text. – In grades 3- 5: there is a call for more scientific,
technical, and historic texts.– In grades 6-8: more literary nonfiction including
essays, speeches, opinion pieces, literary essays, biographies, memoirs, journalism, and historical, scientific, technical, and economic accounts.
12
Capturing the Essence of Mathematics Common Core
• Math CC standards require students to understand why they are doing the things they are doing.
• Students should– be able to explain why they're doing what they're
doing and– justify why they are doing it, not just explain what
they are doing
13
Pilot• Conducted in spring 2012• Students assigned ELA forms could respond
within OAS• Students assigned Mathematics multiple-
choice forms could respond within OAS.• Students assigned Mathematics constructed
response items responded on a paper-pencil answer document.
14
Extended Response Items• Performance-based tasks• May address multiple standards, multiple
domains, and/or multiple areas of the curriculum
• May allow for multiple correct responses and/or varying methods of arriving at a correct answer
• Scored through use of a rubric and associated student exemplars
15
Grade 3 ELA: Reading Passage (Literary)
16
17
Grade 3 ELA: Reading Passage (Informational) and Item 1
Grade 3 ELA: Rubric for Item 1
18
Grade 3 ELA: Exemplar for Item 1
19
Grade 3 ELA: Sample Student Response for Item 1
20
Student Response
Teacher Feedback
Rating based upon rubric
Grade 3 ELA: Sample Student Response for Item 1
21
Grade 3: Sample Student Response for Item 1
22
Grade 3: Sample Student Response for ELA Item 1
23
Grade 3 ELA: Performance Data for Item 1
• Item Type: Extended Response• DOK: 3• Number of Students in Pilot: 90• Response Data
– 4’s: 1 for 1.11%– 3’s: 6 for 6.67%– 2’s: 27 for 30%– 1’s: 53 for 58.89%– 0’s: 3 for 3.33%
24
7.78% proficient
Grade 6 Math: Item
25
Grade 6 Math:Exemplar & Sample Student Response
26
Grade 6 Math: Sample Student Response
27
Grade 6 Math: Performance Data• Item Type: Extended Response• DOK: 3• Number of Students in Pilot: 87• Response Data
– 4’s: 0 for 0%– 3’s: 5 for 5.75%– 2’s: 34 for 39.08%– 1’s: 39 for 44.83%– 0’s: 9 for 10.34%
28
5.75% proficient
Scaffolded Items• Include a sequence of items or tasks• Designed to demonstrate deeper
understanding• May be multi-standard and multi-domain• May guide a student to mapping out a
response to a more extended task• Scored through use of a rubric and associated
student exemplars
29
Grade 3 Math: Scaffolded Item
30
Grade 3 Math: Performance Data• Item Type: Scaffolded • DOK: 3• Number of Students in Pilot: 44• Response Data
– 4’s: 5 for 11.36%– 3’s: 3 for 6.82%– 2’s: 18 for 40.19%– 1’s: 15 for 34.09%– 0’s: 3 for 6.82%
31
18.18% proficient
Overall ELA Pilot Summary Data
32
Grade
Number of students and percent falling into each score pointTotal student
N/ % 0 1 2 3 4
3475 1613 713 202 45 3048
15.60% 52.90% 23.40% 6.60% 1.50% 100%
4323 1518 814 199 83 2937
11.00% 51.70% 27.70% 6.80% 2.80% 100%
5367 1100 901 518 125 3011
12.20% 36.50% 29.90% 17.20% 4.20% 100%
6155 960 811 418 111 2455
6.30% 39.10% 33.00% 17.00% 4.50% 100%
7218 1387 1275 617 146 3643
6.00% 38.10% 35.00% 16.90% 4.00% 100%
8264 1140 1029 338 89 2860
9.20% 39.90% 36.00% 11.80% 3.10% 100%
9 - 10175 1016 783 361 81 2416
7.20% 42.10% 32.40% 14.90% 3.40% 100%11 - 12
376 1018 763 196 46 2399
15.70% 42.40% 31.80% 8.20% 1.90% 100%
Overall Math Pilot Summary Data
33
Grade Number of students and percent falling into each score point Total student N/ %
0 1 2 3 4
3771 667 373 81 36 1928
40.00% 34.60% 19.30% 4.20% 1.90% 100%
4795 800 360 87 58 2100
37.90% 38.10% 17.10% 4.10% 2.80% 100%
5548 513 252 124 44 1481
37.00% 34.60% 17.00% 8.40% 3.00% 100%
6927 768 269 65 14 2043
45.40% 37.60% 13.20% 3.20% 0.70% 100%
7896 632 243 62 11 1844
48.60% 34.30% 13.20% 3.40% 0.60% 100%
8984 791 314 100 51 2240
43.90% 35.30% 14.00% 4.50% 2.30% 100%
9-10798 697 186 45 27 1753
45.50% 39.80% 10.60% 2.60% 1.50% 100%
11-12 690 602 178 63 9 1542
44.70% 39.00% 11.50% 4.10% 0.60% 100%
Key Findings from Phase I Pilot• Students should be earning 3s or 4s to
demonstrate grade-level mastery of the standards
• Preponderance of score points 1 and 2– Incomplete responses – Responses hampered by writing skills– Students did not show work in mathematics and
could not explain why they did what they did
34
Implications for the Classroom
35
• Clearer directions for students so they understand the expectations of a good response– Complete sentences, good grammar and syntax– Connections– Explanations and rationales
• Student self-checklists to assist students in assessing their own responses working on tasks
• Reinforce instructional recommendations to teachers– Instruction aligned with CCGPS content and rigor– Classroom assessments designed with focus on students
articulating how they know what they know– Lessons and classroom assessments integrate knowledge; thus,
address multiple standards and domains
Resources to Support the Formative Items/Tasks in the OAS• About the Formative Assessment Items and
Tasks in the OAS provides a summary of information the items and development.
• Formative Instructional Resources – Released into OAS is a presentation with details on the items and how they were developed, how to access the items in the OAS, and how to use the items in the classroom.
36
Resources to Support the Formative Items/Tasks in the OAS
• Student Self-Checklist for Written Responses to ELA Constructed Response Items – is a rubric that students can use to evaluate
whether they have met the requirements of the task
– provides background information and how to use appropriately and when not to use.
37
Student Self-
Checklist for ELA
38
Resources to Support the Formative Items/Tasks in the OAS• Student Self-Checklist for Mathematics
Constructed Response Items – is a rubric that students can use to evaluate
whether they have met the requirements of the task.
– provides background information and how to use appropriately and when not to use.
39
Student Self-Checklist for Mathematics
40
Resources to Support the Formative Items/Tasks in the OAS• About the OAS Formative Instructional
Resources Item Search Tool provides information about how to use the tool to locate items that are tagged with multiple standards.
• Item Search Tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. The OAS shows only primary but this will show all standards associated with a single item/task.
41
OAS on GaDOE
42
http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS.aspx
Formative OAS Resources
43
http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS-Resources.aspx
Where Do You Find the Items?
44
Select the Tests tab
Select Create a new test
Give the test a name and unique identifier; choose any other desired options, then click Submit
Select an Item LevelNew CCGPS items are in Level 2
Select a Subject for the testNew CCGPS items are found under Language Arts or Math
Select a Grade levelNew CCGPS items are in grades 3-HS
Select a StandardNew CCGPS items are in grades 3-HS
Select a Sub-StandardNew CCGPS items are in grades 3-HS
Then click on “Show Items”
- OR -Use the “Advanced Search” Method
Copy the unique ID here from the Item Search Tool
Then click on “Search”
Put a check to the left of any and all items to be included in the test
Then click on “Add Selected to Test”
Make any needed changes or add more items (click Add Items to return to selecting items);
when finished, click Create Test
Assign Test to a Class
Click on the Training tab for more information about assigning tests to a class.
56
How Students Respond• Students can respond to multiple choice items
regardless of content area within the OAS.• Students can respond to constructed-response ELA
items within the OAS.• Students need to view constructed-response
Mathematics items in the OAS but provide written response on paper. – There is no math editor within OAS. – Some items require students to show work or draw a
graph, number line, or figure.
57
Scoring Open Ended Items in OASScoring Online
- All open ended ELA items can be scored online
- Some open ended math items can be scored online
- Great way to track students’ proficiency
- Students can track their own proficiency as well
- Prepare students for what’s to possibly come with PARCC
Not Scoring Online- A faux student must be created,
assigned the test, and take the test to access rubrics and exemplars
- All open ended ELA items can be scored off line
- All open ended math items can be scored off line
- Great idea if computers are not readily available
- Great for students that have not mastered keyboarding
58
Log in to the OAS
59
Click on Tests Tab; then click Score open ended items.
60
Highlight the test you want to score; then click Select.
61
Check the Select/De-Select All; then click Next.
62
63
Performance Scoring
Student Recorded Response
Question Number
with Dropdown
Feature
Show Question& View
Sample Paper
64
When the Show Question option is selected the
question appears.
65
When the View Sample Paper
option is Selected the student
exemplars will
appear.
66
When the View Sample Paper
option is selected the student
exemplars will appear
67
This drop down allows for the correct
performance score to be given
Here is where the teacher can give
commentary/feedback
Click Submit to continue to next
item .
After you have scored and submitted the last item, this screen will appear. Click the tab labeled Student.
68
Click the View teacher-assigned results.
69
Select the test for which want to see the report, and click Next.
70
Select a student whose report you want to see, then click next.
71
All tests that a student has taken will appear. Click the specific test for which you want to see the results.
72
Results page: this page contains analysis of entire test.
73
This test contained 6
items. 4 open ended that we
scored using the 4pt. rubric
(1,2,4,6) and 2 multiple choice
items (3,5).
You can click each question
and it will display the
item and how the student
responded or continue to scroll down this page.
Student recorded answers with questions
74
Finding an Item for a Specific Standard
• OAS only allows coding by a primary standard code.
• Many of the constructed response items are coded to multiple standards because content knowledge is not discrete and the Common Core expects students to assimilate content across standards and domains.
• A tool has been created that will allow teachers to search for a standard.
75
OAS Formative Instructional Resources Item Search Tool
• This tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. All items aligned to a standard can be located, whether it be the primary standard or a secondary standard.
• The Item Search Tool can be downloaded from the GaDOE website and installed on your desktop.
76
How to Use the Search Tool
77
• From the OAS Resources page on the GaDOE website, click “Download Item Search Tool”• Open the zip folder and then the folder named
“OASFormativeInstructionalResourcesItemSearchTool”
• Select and open the “setup.exe” file.
• If you do not have Microsoft Access 2007 installed on your computer, Access Run Time 2007 will be automatically installed. This application will allow you to view the database.
• Step by step instructions for installation are provided in the “About the Item Search Tool” document on the GaDOE website.
• A shortcut will be saved to your desktop. Double-click the shortcut to open the database.
78
How to Use the Search Tool
• Using the drop down menus select Subject, Grade, and Standard (in that order) and click RUN.
79
How to Use the Search Tool
How to Use the Search Tool• The report will include all items aligned to the selected
standard.• Copy and paste each Item ID into Advanced Search in OAS.
– Use Ctrl-C to copy
80
Formative Assessment InitiativesBringing a Balanced Assessment Focus to the Classroom
81
First initiative – Phase I items released into OAS; Phase II items will be piloted in early 2013
Another initiative to begin pilot in January 2013
Third initiative – Phase I item development begun with pilot in spring 2013
Georgia Department of Education Assessment and Accountability
Melissa FincherAssociate SuperintendentAssessment and [email protected]
Dr. Melodee Davis Director Assessment Research and [email protected]
Michael HunekeOAS [email protected]
Dr. Dawn SouterProject Manager. [email protected]
Robert Anthony Assessment Specialist, RT3Formative Item [email protected]
Dr. Jan ReyesAssessment Specialist, RT3Interim Benchmark [email protected]
82
