Embed Size (px)
Citation preview
1
Illinois Alternate Assessment
Directors Conference
July 31, 2008
2
Misc. Assessment Update
Vacancy
2% flexibility
2008-2009
Accommodations
Guidelines
3
IAA Agenda Overview
Where we’ve been What we’ve learned Adjustments Scoring Validation Calendar Resources
4
IAA “Life Cycle” to Date:
Fall 2006 Writing Pilot February 2007 Portfolios R, M, SSpring 2007 Writing OperationalFall 2007 Pilot R, M, SSpring 2008 Operational for R, M, S, W 2009 adds Gr. 3 Writing to complete
operational test
5
ISBE IAA 2008 Survey
May, 2008, an electronic survey was sent out to teachers and coordinators using contact information from the Pearson Assessment Network and Scoring System.
345 Teachers & Coordinators participated in the survey
6
Excellent Work, Coordinators!!
92% of teachers knew in advance when the testing window would fall.
94% of teachers reported the IAA coordinator had the online scoring set up and ready for scoring.
95% of teachers said they knew who to contact if there was a problem.
7
Time required for administration:
Subject Less than 15 minutes
15-30 minutes
11 Reading
tasks28% 58%
12 Math
tasks31% 55%
8 Science
tasks37% 52%
6 Writing
tasks40% 46%
8
Concern #1:
51% of teachers attended a training session
This raises questions regarding the use of the trainer of trainers model. The accuracy of test administration and scoring may have been impacted.
9
Concern #2:
Comparability to regular assessments in terms of content coverage.
“In states I have worked with, there tend to be more alternate assessment items than (ISBE) has developed.” (peer reviewer)
10
Response to the Two Concerns:Potential for Refinement
11
2009 IAA Training, Part 1:
More training sessions will be provided in 2009:
Northern Illinois including ChicagoJanuary 26-30, 2009
Central and Southern IllinoisFebruary 2-6, 2009
Watch the ISBE website for signup opportunities in your area of the state in Fall, 2008.
12
2009 Training, (Possible) Part 2:
To further ensure precise administration and scoring of the tasks, especially for those teachers who have not been trained:
Rubric language could be embedded in the teacher administration instructions.
13
Built-In Task Characteristics:
Plus: Task-specific texts, symbols, and artwork
embedded in the task. The “General Prompt” embedded in the task
(in the event it is needed). All tasks would follow the same rubric
“template” for teacher administration instructions.
Flexibility in presentation of tasks and how the student responds would be maintained.
14
Sample Science Task:
Primary Task:
a dog and a cat a cat and a plant three bushes
Which picture shows only mammals?
General Prompt:
Rabbits are mammals.
15
Sample Draft Teacher Instructions:
Level 4:Show the student the primary task and say: “Which picture shows only mammals?” Present the answer
options in order and say “a dog and a cat, a cat and a plant, three bushes.”
The student receives a score of 4 for a correct response. If the student does not respond or responds incorrectly, you may repeat the question exactly as it appearsone time only. If the student again chooses an incorrect answer or does not respond, move to Level 3.
Level 3: Show the student the general prompt and say: “Rabbits are mammals.” Re-present the primary task and say: “Which picture shows only mammals?” Re-present the answer
options in order and say “a dog and a cat, a cat and a plant, three bushes”.
The student receives a score of 3 for a correct response. If no response or incorrect response, move to Level 2.
Level 2: Indicate the specific prompt and say: “Cats and dogs are mammals.” Re-present the primary task and say: “Which picture shows only mammals?” Indicate the choices in order
and say: “a dog and a cat, a cat and a plant, three bushes”.
The student receives a score of 2 for a correct response. If no response or incorrect response, move to Level 1.
Level 1:If no response or an incorrect response after providing level 2 support, the student receives a score of 1.
Correct Answer: a dog and a cat
16
Build a Better IAA by:
Extending
content coverage
requires increasing
the quantity of items.
17
2008 IAA Operational Test
Reading Math Science Writing
9 tasks*
2 FT
Total: 11
10 tasks*
2 FT
Total: 12
6 tasks*
2 FT
Total: 8
5 tasks*
1 FT
Total: 6
* Required tasks covering identified priorities.
FT – field test task (pilot) for future tests
18
Sample Content Coverage Plan:
Each Standard assessed with:• At least 1 task for each objective• 2 tasks for each identified high priority
objective (2 priority objectives at each subject’s grade level).
• 4 objectives always assessed (tasks assessed annually as “anchor” items)
19
Using the previous content coverage plan, the 2009 Operational Test would look like:
Reading Math Science Writing
17 tasks*
4 FT
Total: 21
19 tasks*
4 FT
Total: 23
19 tasks*
4 FT
Total: 23
5 tasks
1 FT
Total: 6
* Required tasks covering all objectives in each subject.
FT – field test task (pilot) for future tests
20
Some Things Never Change
Assessment frameworks remain the same. IAA Participation Guidelines remain the same. Use of mode of communication specific to the
individual student remains the same. Options in types of student responses remains the
same. Score submission online remains the same. Number of Writing tasks remains the same. Content coverage is expanded, not changed, in
Reading, Math, and Science. Training will be provided on administration of the
tasks.
21
Potential Results?
Refinements should:
• Enhance content coverage.• Improve likelihood that the IAA
will pass peer review. • Avoid rejection of the IAA
by the federal agency.
22
August 4-5, 2008
To enhance the quality and quantity of the IAA performance tasks:
Item Writers will be working onReadingMath Science
Number of Writing tasks is sufficient at this time.
23
IAA Scoring Validation Has Not Changed
Recruiting more Coordinators to serve as:
Second scorersSuper raters
If interested, contact Pearl at 217/782-4823 or [email protected]
24
IAA 2009 Proposed Calendar
• Jan. 26-30 & Feb. 2-6, 2009 IAA training sessions throughout the state
• Feb. 17, 2009 Tentative date for delivery of IAA test materials to schools/districts
• February, 2009 Online scoring system IDs & passwords distributed
March 9-27, 2009 – Regular test window
25
IAA Online Resources
Implementation ManualOnline Scoring User Guides FrameworksScoring RubricRCDTS codesAccommodations Guidelineswww.isbe.net/assessment/iaa.htm
26
Pearson Contact Information
Pearson’s Assessment Networkwww.pearsonaccess.com/il
Pearson Customer Support [email protected] 800/627-7990 State Code 814
Pearson Online Scoring [email protected]/234-4849
27
ISBE Contact Information
Pearl Schneider, IAA Coordinator
Email: [email protected]
Phone: 217/782-4823
