Training for ELPA and MI-Access Scoring and Administration

2008 ASSESSMENT & ACCOUNTABILITY

CONFERENCES

Training for ELPA and MI-Access Scoring and

Administration1

PU

RP

OS

E To help coordinators who train

assessment administrators better understand scoring procedures for MI-Access and the ELPA

To provide information on resources available to aid understanding

And, ultimately, to…

increase the reliability and validity of assessment results through better understanding, and

ensure that students’ efforts are accurately scored, reported, and counted

2

MI-Access Assessments

Participation — for students who have, or function as if they have, severe cognitive impairment

Supported Independence — for students who have, or function as if they have, moderate cognitive impairment

Functional Independence — for students who have, or function as if they have, mild cognitive impairment

Three Assessments

3

Content Areas Assessed

The MI-Access assessments cover three content areas: English language arts (ELA), mathematics, and science

The assessments reflect Michigan’s GLCEs, HSCEs, and/or Benchmarks, but they have been extended — or reduced in depth, breadth, and complexity — so they are appropriate for the student populations being assessed

4

MI-Access Participation & Supported Independence Assessments

The Participation and Supported Independence (P/SI) assessments use TWO item formats:

Activity-based observation — items are presented to students during familiar classroom activities or routines, which provide a performance context for assessing specific EGLCEs, EHSCEs, and/or EBs

Selected response — students are read an item stem, or question, and asked to select the correct response

5

Activity-Based Observation Item

ACTIVITY: The student will correctly indicate the difference between whole and part when engaged in a familiar food preparation activity, such as assembling ingredients for a snack. For example, the student could be shown a whole cup of milk and part of a cup of milk and then be asked, “Which one is a whole cup?”

SCORING FOCUS: Differentiating between a whole object and part of an object

Sample Participation Mathematics Item (Grades 6-8)

6

Selected-Response Item

Sample Participation Science Item (Grade 8)

7

P items have 2 picture answer choices

SI items have 3 picture answer choices

Picture Card Presentation Styles8

Picture 1 Picture 2 Picture 2 Picture 1

Show first Reverse and show again

Participation Selected-Response Item with 2 Answer Choices

Both picture cards must be presented at the same time in one order, then presented again with the positions reversed

ELA activity-based observation items that use words paired with pictures also have specific presentation styles (see Appendix C in the manual)

P/SI Assessment Administrators

Administered by TWO people—a Primary Assessment Administrator (PAA) and a Shadow Assessment Administrator (SAA)

The PAA and SAA work together prior to the assessment to determine the student’s anticipated response mode.

The PAA and SAA observe the student being assessed and simultaneously and independently score the student using a standardized scoring rubric (regardless of item format)

9

P/SI Scoring Rubrics

The scoring rubrics… are based on the student responding correctly, and take into consideration the amount of assistance required

to engage the student in the item

The scoring rubrics contain both… score points, and condition codes (which add meaning to zeroes)

10

P/SI Scoring Rubrics

MI-Access P/SI Scoring RubricsP Score Point/Condition Code

SI Score Point/Condition Code

Term

3 2Responds correctly with no assessment administrator assistance

2 1Responds correctly after assessment administrator provides verbal/physical cues

1 Not allowed in SIResponds correctly after assessment administrator provides modeling, short of hand-over-hand assistance

A A Incorrect

B B Resists/Refuses

C CAssessment administrator provides step-by-step directions and/or hand-over-hand assistance

11

P/SI Scoring Rubrics Online Learning Program

www.mi-access.info

Explains the P/SI rubrics in detail

Shows AAs using them to score students on sample items

Is being updated (still says “pilot”)

12

MI-Access Coordinator & Assessment Administrator Manual

The manual includes…

an entire section explaining how the P/SI assessments are designed and should be administered,

flow charts showing how to apply the scoring rubrics, and

score point and condition code definitions and examples

13

P/SI Item Analyses14

Compares how AAs scored released items

Can be used at district/building level to discuss issues of consistency

Produced only when 10 or more students in the same grade take the same test

Functional Independence

For Functional Independence, a scoring rubric is used only for ELA Expressing Ideas

Students are asked to respond to an open-ended prompt:

Tell about what you like to do outside when the weather is cold. Be sure to include details and examples in your response.

Students can write, draw, or use a combination to express their response

Students can also dictate their response if their disability prevents them from writing or drawing

15

ELA Expressing Ideas Rubric

Student responses are evaluated by external hand-scorers using a 4-point rubric that measures topic focus, organization, and use of language and visual conventions

4 is the highest score a student can receive

The rubric includes condition codes and comment codes

16

ELA Expressing Ideas Rubric

Condition codes add meaning to a score point of zero A = off topic B = illegible C = written in a language other than English D = blank/refused to respond

Comment codes elaborate on each score point Lacks clear focus on the prompt topic Shows limited development with insufficient details and/or examples Lacks clear organization of ideas and/or arrangement of figures Contains errors in language/visual conventions that interfere with understanding

17

FI ELA Item Analyses18

Shows how many students received each score point or condition code

Shows how many student received each comment code

RE

SO

UR

CE

SMI-Access P/SI Scoring Rubrics

Online Learning Program www.mi-access.info under the “Online

Learning” tab

2008/2009 MI-Access Coordinator and Assessment Administrator Manual www.mi.gov/mi-access under “Resources”

Sample Assessment Booklets and Released Item Booklets www.mi-access.info under the

“Functional Independence” and “Participation and Supported Independence” tabs

19

RE

SO

UR

CE

SExtended GLCEs, HSCEs, and

Benchmarks www.mi.gov/mi-access under the

“Participation and Supported Independence” and “Functional Independence” sections

Assessment Plans www.mi.gov/mi-access under the

“Participation and Supported Independence,” “Functional Independence,” and “IEP Team Information” sections

IEP Team Tools www.mi.gov/mi-access under the IEP

Team Information” section

20

CO

NTA

CT

SMI-Access

Linda [email protected](517)339-3940

Deb [email protected](517)699-3017

21

ELPA: Speaking Scoring

Item formats at all levels:

Sentence Repeat Short Conversation Guided Discussion Storytelling Story Retell

22


Sentence Repeat

Student hears and reads sentence before repeating it

One point maximum score

Standard focus is on S.6Demonstrate comprehensible pronunciation and intonation for clarity in oral communication

23


Sentence Repeat

Proctor listens for fluency and smoothness of speech, proper pronunciation, and appropriate intonation

24


Short Conversation

Student hears and reads prompt and responds to question(s)

Questions are as open-ended as possible, but still guide student toward focused answer

25


Short Conversation

Two points maximum score

Standard focus is on S.2Engage in conversations for personal expression and enjoyment

26


Short Conversation

Proctor listens for fluency and accuracy, but with an emphasis on the student’s ability to use appropriate vocabulary to express personal opinions, ideas, or points of view

27


Guided Discussion

Item format is a two-part set of questions with short prompt to lead into discussion

Student hears and reads prompt and responds to first question

28


Guided Discussion

Student then hears and reads follow-up question and responds

Two points maximum score for each prompt; total four points maximum

Proctor scores each portion individually

29


Guided Discussion

Standard focus is on S.4Use English to interact in the classroom

Benchmark S.4.2.fParticipate in guided discussions

30


Guided Discussion

Proctor listens for conversational flow and exchange of information; also for the student’s ability to focus the discussion on specific details or examples after the second prompt

31


Story Telling

Student sees sequence of three graphics and creates oral story or narrative about pictures

Sequence of graphics is marked First, Next, and Last

32


Story Telling

Four points maximum score

Standard focus is on S.7Present information, concepts, and ideas to an audience of listeners on a variety of topics

33


Story Telling

Proctor listens for complex sentence structures, varied and precise vocabulary, and logical presentation of ideas with appropriate transitions

34


Story Retell

Student hears and reads narrative or content-related text

Student responds by telling back to the proctor as much as possible, with allowance for added details

35


Story Retell

Student bases response on comprehensible English input appropriate to grade level

Four points maximum score

36


Story Retell

Standard focus is on S.7Present information, concepts, and ideas to an audience of listeners on a variety of topics

37


Story Retell

Proctor listens for authentic speech patterns of student, including use of grammatical constructions, descriptive vocabulary, and transitional phrases to retell recognizable story with fluency of speech

38


Story Retell

Speaking DVD available for training purposes

Contact OEAA for copies of DVD

39

CO

NTA

CT

S ELPA

Phil ChaseDepartment Specialist,

Assessment of English Language Learners

[email protected](517) 335-3967

40

Documents

Training for ELPA and MI-Access Scoring and Administration