December 5 and 6, 2012 Formative Assessment Items and Tasks – in OAS 1 Phase I of Formative Item...

December 5 and 6, 2012

Formative Assessment Items and Tasks – in OAS

1

Phase I of Formative Item Bank

Background onRT3 Assessment Initiatives

• Purpose – To provide assessment resources that reflect the

rigor of the CCGPS– To balance the use of formative and summative

assessments in the classroom– To promote student learning

2

RT3 Assessment Initiatives• Development of a three-prong toolkit to support

teachers and districts and to promote student learning– An assessment literacy professional learning

opportunity that focuses on implementation of formative instruction – Initial Pilot January 2013

– A set of interim benchmarks in ELA, Math, and selected grades for Science and Social Studies plus additional items – Initial Pilot Spring 2013

– An expansive bank of formative instructional assessment items/tasks based on CCGPS in ELA and Mathematics as a teacher resource - Phase I Release Fall 2012; Phase II release Fall 2013

3

Formative Instructional Practices

4

• Design• Teach

• Re-Design• Teach

• Re-Design• Teach

• Re-Design• Teach Common Core

State Standards

Formative Instructional Practices• Conducted during instruction (lesson, unit, etc.)• Identifies student strengths and weaknesses• Helps teacher determine next steps

– Review– Differentiation– Continuation

• Supplies information to provide students with detailed feedback

• Assessment for the purpose of improving achievement• LOW stakes

5

Purpose of the Formative Item Bank

The purpose of the Formative Item Bank is to provide items and tasks for teachers to use in order to assess students’ knowledge while they are learning the curriculum. •To provide an opportunity for students to show what they know and show teachers what students do not understand•To provide teachers good examples of tasks/items that assess the rigor of the Common Core.

6

Formative Instructional Assessment Items

• Assess students’ conceptual and skill-level understanding

• Tasks require students to– Apply the content they know to real world problems– Express logical reasoning by writing, showing their

work and/or explaining their answer

7

Items/Tasks for Formative Instructional Use

• Aligned to CCGPS for Grades 3 - High School ELA & Math• Format similar to PARCC’s Common Assessments

prototypes - – In other words, items/tasks are more rigorous and require

students to show what they know and how they know it

• Item Development Process (same as used for state tests)– Written to align with curriculum – Piloted in RT3 classrooms in spring 2012– Reviewed with student data and approved by Georgia Educators

8

Formative Item Bank• Item types

– About 1700 Formative Classroom Assessment Items • About 750 Phase I items Phase I available in OAS as of Nov. 30, 2012

– Some multiple choice items– Most are open-ended or constructed-response format

• Holistic Rubric• Exemplar Paper (i.e.,“ideal response”)• Student Exemplars/Responses

– DOK targets• DOK 2; 19%• DOK 3; 41%• DOK 4; 39%

9

Anchors used in scoring

The Essence of the Common Core State Standards

10

The Common Core State Standards focus on students learning how to think, not just doing by rote memorization or quick scan-reading, and sets higher expectations on student comprehension and their ability to articulate their knowledge.

Capturing the Essence of ELA Common Core

• ELA CCSS require students to use evidence from what lies within the four corners of the text and make valid claims that can be proven with the text.

• Students must be more adept at explaining that evidence orally and in writing.

11

Capturing the Essence of ELA Common Core

• In order to be college-, career-, and life-ready, students need to be familiar and comfortable with a broad range of genres and formats and much greater emphasis on informational text. – In grades 3- 5: there is a call for more scientific,

technical, and historic texts.– In grades 6-8: more literary nonfiction including

essays, speeches, opinion pieces, literary essays, biographies, memoirs, journalism, and historical, scientific, technical, and economic accounts.

12

Capturing the Essence of Mathematics Common Core

• Math CC standards require students to understand why they are doing the things they are doing.

• Students should– be able to explain why they're doing what they're

doing and– justify why they are doing it, not just explain what

they are doing

13

Pilot• Conducted in spring 2012• Students assigned ELA forms could respond

within OAS• Students assigned Mathematics multiple-

choice forms could respond within OAS.• Students assigned Mathematics constructed

response items responded on a paper-pencil answer document.

14

Extended Response Items• Performance-based tasks• May address multiple standards, multiple

domains, and/or multiple areas of the curriculum

• May allow for multiple correct responses and/or varying methods of arriving at a correct answer

• Scored through use of a rubric and associated student exemplars

15

Grade 3 ELA: Reading Passage (Literary)

16

17

Grade 3 ELA: Reading Passage (Informational) and Item 1

Grade 3 ELA: Rubric for Item 1

18

Grade 3 ELA: Exemplar for Item 1

19

Grade 3 ELA: Sample Student Response for Item 1

20

Student Response

Teacher Feedback

Rating based upon rubric

Grade 3 ELA: Sample Student Response for Item 1

21

Grade 3: Sample Student Response for Item 1

22

Grade 3: Sample Student Response for ELA Item 1

23

Grade 3 ELA: Performance Data for Item 1

• Item Type: Extended Response• DOK: 3• Number of Students in Pilot: 90• Response Data

– 4’s: 1 for 1.11%– 3’s: 6 for 6.67%– 2’s: 27 for 30%– 1’s: 53 for 58.89%– 0’s: 3 for 3.33%

24

7.78% proficient

Grade 6 Math: Item

25

Grade 6 Math:Exemplar & Sample Student Response

26

Grade 6 Math: Sample Student Response

27

Grade 6 Math: Performance Data• Item Type: Extended Response• DOK: 3• Number of Students in Pilot: 87• Response Data

– 4’s: 0 for 0%– 3’s: 5 for 5.75%– 2’s: 34 for 39.08%– 1’s: 39 for 44.83%– 0’s: 9 for 10.34%

28

5.75% proficient

Scaffolded Items• Include a sequence of items or tasks• Designed to demonstrate deeper

understanding• May be multi-standard and multi-domain• May guide a student to mapping out a

response to a more extended task• Scored through use of a rubric and associated

student exemplars

29

Grade 3 Math: Scaffolded Item

30

Grade 3 Math: Performance Data• Item Type: Scaffolded • DOK: 3• Number of Students in Pilot: 44• Response Data

– 4’s: 5 for 11.36%– 3’s: 3 for 6.82%– 2’s: 18 for 40.19%– 1’s: 15 for 34.09%– 0’s: 3 for 6.82%

31

18.18% proficient

Overall ELA Pilot Summary Data

32

Grade

Number of students and percent falling into each score pointTotal student

N/ % 0 1 2 3 4

3475 1613 713 202 45 3048

15.60% 52.90% 23.40% 6.60% 1.50% 100%

4323 1518 814 199 83 2937

11.00% 51.70% 27.70% 6.80% 2.80% 100%

5367 1100 901 518 125 3011

12.20% 36.50% 29.90% 17.20% 4.20% 100%

6155 960 811 418 111 2455

6.30% 39.10% 33.00% 17.00% 4.50% 100%

7218 1387 1275 617 146 3643

6.00% 38.10% 35.00% 16.90% 4.00% 100%

8264 1140 1029 338 89 2860

9.20% 39.90% 36.00% 11.80% 3.10% 100%

9 - 10175 1016 783 361 81 2416

7.20% 42.10% 32.40% 14.90% 3.40% 100%11 - 12

376 1018 763 196 46 2399

15.70% 42.40% 31.80% 8.20% 1.90% 100%

Overall Math Pilot Summary Data

33

Grade Number of students and percent falling into each score point Total student N/ %

0 1 2 3 4

3771 667 373 81 36 1928

40.00% 34.60% 19.30% 4.20% 1.90% 100%

4795 800 360 87 58 2100

37.90% 38.10% 17.10% 4.10% 2.80% 100%

5548 513 252 124 44 1481

37.00% 34.60% 17.00% 8.40% 3.00% 100%

6927 768 269 65 14 2043

45.40% 37.60% 13.20% 3.20% 0.70% 100%

7896 632 243 62 11 1844

48.60% 34.30% 13.20% 3.40% 0.60% 100%

8984 791 314 100 51 2240

43.90% 35.30% 14.00% 4.50% 2.30% 100%

9-10798 697 186 45 27 1753

45.50% 39.80% 10.60% 2.60% 1.50% 100%

11-12 690 602 178 63 9 1542

44.70% 39.00% 11.50% 4.10% 0.60% 100%

Key Findings from Phase I Pilot• Students should be earning 3s or 4s to

demonstrate grade-level mastery of the standards

• Preponderance of score points 1 and 2– Incomplete responses – Responses hampered by writing skills– Students did not show work in mathematics and

could not explain why they did what they did

34

Implications for the Classroom

35

• Clearer directions for students so they understand the expectations of a good response– Complete sentences, good grammar and syntax– Connections– Explanations and rationales

• Student self-checklists to assist students in assessing their own responses working on tasks

• Reinforce instructional recommendations to teachers– Instruction aligned with CCGPS content and rigor– Classroom assessments designed with focus on students

articulating how they know what they know– Lessons and classroom assessments integrate knowledge; thus,

address multiple standards and domains

Resources to Support the Formative Items/Tasks in the OAS• About the Formative Assessment Items and

Tasks in the OAS provides a summary of information the items and development.

• Formative Instructional Resources – Released into OAS is a presentation with details on the items and how they were developed, how to access the items in the OAS, and how to use the items in the classroom.

36

Resources to Support the Formative Items/Tasks in the OAS

• Student Self-Checklist for Written Responses to ELA Constructed Response Items – is a rubric that students can use to evaluate

whether they have met the requirements of the task

– provides background information and how to use appropriately and when not to use.

37

Student Self-

Checklist for ELA

38

Resources to Support the Formative Items/Tasks in the OAS• Student Self-Checklist for Mathematics

Constructed Response Items – is a rubric that students can use to evaluate

whether they have met the requirements of the task.

– provides background information and how to use appropriately and when not to use.

39

Student Self-Checklist for Mathematics

40

Resources to Support the Formative Items/Tasks in the OAS• About the OAS Formative Instructional

Resources Item Search Tool provides information about how to use the tool to locate items that are tagged with multiple standards.

• Item Search Tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. The OAS shows only primary but this will show all standards associated with a single item/task.

41

OAS on GaDOE

42

http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS.aspx

Formative OAS Resources

43

http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS-Resources.aspx

Where Do You Find the Items?

44

Select the Tests tab

Select Create a new test

Give the test a name and unique identifier; choose any other desired options, then click Submit

Select an Item LevelNew CCGPS items are in Level 2

Select a Subject for the testNew CCGPS items are found under Language Arts or Math

Select a Grade levelNew CCGPS items are in grades 3-HS

Select a StandardNew CCGPS items are in grades 3-HS

Select a Sub-StandardNew CCGPS items are in grades 3-HS

Then click on “Show Items”

- OR -Use the “Advanced Search” Method

Copy the unique ID here from the Item Search Tool

Then click on “Search”

Put a check to the left of any and all items to be included in the test

Then click on “Add Selected to Test”

Make any needed changes or add more items (click Add Items to return to selecting items);

when finished, click Create Test

Assign Test to a Class

Click on the Training tab for more information about assigning tests to a class.

56

How Students Respond• Students can respond to multiple choice items

regardless of content area within the OAS.• Students can respond to constructed-response ELA

items within the OAS.• Students need to view constructed-response

Mathematics items in the OAS but provide written response on paper. – There is no math editor within OAS. – Some items require students to show work or draw a

graph, number line, or figure.

57

Scoring Open Ended Items in OASScoring Online

- All open ended ELA items can be scored online

- Some open ended math items can be scored online

- Great way to track students’ proficiency

- Students can track their own proficiency as well

- Prepare students for what’s to possibly come with PARCC

Not Scoring Online- A faux student must be created,

assigned the test, and take the test to access rubrics and exemplars

- All open ended ELA items can be scored off line

- All open ended math items can be scored off line

- Great idea if computers are not readily available

- Great for students that have not mastered keyboarding

58

Log in to the OAS

59

Click on Tests Tab; then click Score open ended items.

60

Highlight the test you want to score; then click Select.

61

Check the Select/De-Select All; then click Next.

62

63

Performance Scoring

Student Recorded Response

Question Number

with Dropdown

Feature

Show Question& View

Sample Paper

64

When the Show Question option is selected the

question appears.

65

When the View Sample Paper

option is Selected the student

exemplars will

appear.

66

When the View Sample Paper

option is selected the student

exemplars will appear

67

This drop down allows for the correct

performance score to be given

Here is where the teacher can give

commentary/feedback

Click Submit to continue to next

item .

After you have scored and submitted the last item, this screen will appear. Click the tab labeled Student.

68

Click the View teacher-assigned results.

69

Select the test for which want to see the report, and click Next.

70

Select a student whose report you want to see, then click next.

71

All tests that a student has taken will appear. Click the specific test for which you want to see the results.

72

Results page: this page contains analysis of entire test.

73

This test contained 6

items. 4 open ended that we

scored using the 4pt. rubric

(1,2,4,6) and 2 multiple choice

items (3,5).

You can click each question

and it will display the

item and how the student

responded or continue to scroll down this page.

Student recorded answers with questions

74

Finding an Item for a Specific Standard

• OAS only allows coding by a primary standard code.

• Many of the constructed response items are coded to multiple standards because content knowledge is not discrete and the Common Core expects students to assimilate content across standards and domains.

• A tool has been created that will allow teachers to search for a standard.

75

OAS Formative Instructional Resources Item Search Tool

• This tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. All items aligned to a standard can be located, whether it be the primary standard or a secondary standard.

• The Item Search Tool can be downloaded from the GaDOE website and installed on your desktop.

76

How to Use the Search Tool

77

• From the OAS Resources page on the GaDOE website, click “Download Item Search Tool”• Open the zip folder and then the folder named

“OASFormativeInstructionalResourcesItemSearchTool”

• Select and open the “setup.exe” file.

• If you do not have Microsoft Access 2007 installed on your computer, Access Run Time 2007 will be automatically installed. This application will allow you to view the database.

• Step by step instructions for installation are provided in the “About the Item Search Tool” document on the GaDOE website.

• A shortcut will be saved to your desktop. Double-click the shortcut to open the database.

78

How to Use the Search Tool

• Using the drop down menus select Subject, Grade, and Standard (in that order) and click RUN.

79

How to Use the Search Tool

How to Use the Search Tool• The report will include all items aligned to the selected

standard.• Copy and paste each Item ID into Advanced Search in OAS.

– Use Ctrl-C to copy

80

Formative Assessment InitiativesBringing a Balanced Assessment Focus to the Classroom

81

First initiative – Phase I items released into OAS; Phase II items will be piloted in early 2013

Another initiative to begin pilot in January 2013

Third initiative – Phase I item development begun with pilot in spring 2013

Georgia Department of Education Assessment and Accountability

Melissa FincherAssociate SuperintendentAssessment and Accountability404.651.9405mfincher@doe.k12.ga.us

Dr. Melodee Davis Director Assessment Research and Development404.657.0312medavis@doe.k12.ga.us

Michael HunekeOAS Manager404.232.1208mhuneke@doe.k12.ga.us

Dr. Dawn SouterProject Manager. RT3404.463.6667dsouter@doe.k12.ga.us

Robert Anthony Assessment Specialist, RT3Formative Item Bank404.463.6668ranthony@doe.k12.ga.us

Dr. Jan ReyesAssessment Specialist, RT3Interim Benchmark Assessments404.463.6665jreyes@doe.k12.ga.us

82