1 Illinois Alternate Assessment Directors Conference July 31, 2008

Preview:

Citation preview

1

Illinois Alternate Assessment

Directors Conference

July 31, 2008

2

Misc. Assessment Update

Vacancy

2% flexibility

2008-2009

Accommodations

Guidelines

3

IAA Agenda Overview

Where we’ve been What we’ve learned Adjustments Scoring Validation Calendar Resources

4

IAA “Life Cycle” to Date:

Fall 2006 Writing Pilot February 2007 Portfolios R, M, SSpring 2007 Writing OperationalFall 2007 Pilot R, M, SSpring 2008 Operational for R, M, S, W 2009 adds Gr. 3 Writing to complete

operational test

5

ISBE IAA 2008 Survey

May, 2008, an electronic survey was sent out to teachers and coordinators using contact information from the Pearson Assessment Network and Scoring System.

345 Teachers & Coordinators participated in the survey

6

Excellent Work, Coordinators!!

92% of teachers knew in advance when the testing window would fall.

94% of teachers reported the IAA coordinator had the online scoring set up and ready for scoring.

95% of teachers said they knew who to contact if there was a problem.

7

Time required for administration:

Subject Less than 15 minutes

15-30 minutes

11 Reading

tasks28% 58%

12 Math

tasks31% 55%

8 Science

tasks37% 52%

6 Writing

tasks40% 46%

8

Concern #1:

51% of teachers attended a training session

This raises questions regarding the use of the trainer of trainers model. The accuracy of test administration and scoring may have been impacted.

9

Concern #2:

Comparability to regular assessments in terms of content coverage.

“In states I have worked with, there tend to be more alternate assessment items than (ISBE) has developed.” (peer reviewer)

10

Response to the Two Concerns:Potential for Refinement

11

2009 IAA Training, Part 1:

More training sessions will be provided in 2009:

Northern Illinois including ChicagoJanuary 26-30, 2009

Central and Southern IllinoisFebruary 2-6, 2009

Watch the ISBE website for signup opportunities in your area of the state in Fall, 2008.

12

2009 Training, (Possible) Part 2:

To further ensure precise administration and scoring of the tasks, especially for those teachers who have not been trained:

Rubric language could be embedded in the teacher administration instructions.

13

Built-In Task Characteristics:

Plus: Task-specific texts, symbols, and artwork

embedded in the task. The “General Prompt” embedded in the task

(in the event it is needed). All tasks would follow the same rubric

“template” for teacher administration instructions.

Flexibility in presentation of tasks and how the student responds would be maintained.

14

Sample Science Task:

Primary Task:

a dog and a cat a cat and a plant three bushes

Which picture shows only mammals?

General Prompt:

Rabbits are mammals.

15

Sample Draft Teacher Instructions:

Level 4:Show the student the primary task and say: “Which picture shows only mammals?” Present the answer

options in order and say “a dog and a cat, a cat and a plant, three bushes.”

The student receives a score of 4 for a correct response. If the student does not respond or responds incorrectly, you may repeat the question exactly as it appearsone time only. If the student again chooses an incorrect answer or does not respond, move to Level 3.

Level 3: Show the student the general prompt and say: “Rabbits are mammals.” Re-present the primary task and say: “Which picture shows only mammals?” Re-present the answer

options in order and say “a dog and a cat, a cat and a plant, three bushes”.

The student receives a score of 3 for a correct response. If no response or incorrect response, move to Level 2.

Level 2: Indicate the specific prompt and say: “Cats and dogs are mammals.” Re-present the primary task and say: “Which picture shows only mammals?” Indicate the choices in order

and say: “a dog and a cat, a cat and a plant, three bushes”.

The student receives a score of 2 for a correct response. If no response or incorrect response, move to Level 1.

Level 1:If no response or an incorrect response after providing level 2 support, the student receives a score of 1.

Correct Answer: a dog and a cat

16

Build a Better IAA by:

Extending

content coverage

requires increasing

the quantity of items.

17

2008 IAA Operational Test

Reading Math Science Writing

9 tasks*

2 FT

Total: 11

10 tasks*

2 FT

Total: 12

6 tasks*

2 FT

Total: 8

5 tasks*

1 FT

Total: 6

* Required tasks covering identified priorities.

FT – field test task (pilot) for future tests

18

Sample Content Coverage Plan:

Each Standard assessed with:• At least 1 task for each objective• 2 tasks for each identified high priority

objective (2 priority objectives at each subject’s grade level).

• 4 objectives always assessed (tasks assessed annually as “anchor” items)

19

Using the previous content coverage plan, the 2009 Operational Test would look like:

Reading Math Science Writing

17 tasks*

4 FT

Total: 21

19 tasks*

4 FT

Total: 23

19 tasks*

4 FT

Total: 23

5 tasks

1 FT

Total: 6

* Required tasks covering all objectives in each subject.

FT – field test task (pilot) for future tests

20

Some Things Never Change

Assessment frameworks remain the same. IAA Participation Guidelines remain the same. Use of mode of communication specific to the

individual student remains the same. Options in types of student responses remains the

same. Score submission online remains the same. Number of Writing tasks remains the same. Content coverage is expanded, not changed, in

Reading, Math, and Science. Training will be provided on administration of the

tasks.

21

Potential Results?

Refinements should:

• Enhance content coverage.• Improve likelihood that the IAA

will pass peer review. • Avoid rejection of the IAA

by the federal agency.

22

August 4-5, 2008

To enhance the quality and quantity of the IAA performance tasks:

Item Writers will be working onReadingMath Science

Number of Writing tasks is sufficient at this time.

23

IAA Scoring Validation Has Not Changed

Recruiting more Coordinators to serve as:

Second scorersSuper raters

If interested, contact Pearl at 217/782-4823 or pschneid@isbe.net

24

IAA 2009 Proposed Calendar

• Jan. 26-30 & Feb. 2-6, 2009 IAA training sessions throughout the state

• Feb. 17, 2009 Tentative date for delivery of IAA test materials to schools/districts

• February, 2009 Online scoring system IDs & passwords distributed

March 9-27, 2009 – Regular test window

25

IAA Online Resources

Implementation ManualOnline Scoring User Guides FrameworksScoring RubricRCDTS codesAccommodations Guidelineswww.isbe.net/assessment/iaa.htm

26

Pearson Contact Information

Pearson’s Assessment Networkwww.pearsonaccess.com/il

Pearson Customer Support iaa_pearson@support.pearson.com 800/627-7990 State Code 814

Pearson Online Scoring Questionssupport@schoolsuccess.net888/234-4849

27

ISBE Contact Information

Pearl Schneider, IAA Coordinator

Email: pschneid@isbe.net

Phone: 217/782-4823

Recommended