ACCESS for ELLs ® Tier Placement, Test Security, and Score Reports Emily Evans, Center for Applied Linguistics January 2007 New Jersey Department of Education

ACCESS for ELLs® Tier Placement, Test Security, and Score Reports

Emily Evans, Center for Applied LinguisticsJanuary 2007New Jersey Department of Education

Developed by the Center for Applied Linguistics

2ACCESS for ELLs® Tiers, Ordering, Security, and Score Reports

Workshop GoalsWorkshop Goals

To review appropriate tier placement for students taking the ACCESS for ELLs® test

To be equipped to handle test operations, such as test security

To understand and begin to interpret ACCESS score reports


Workshop Objectives (1)Workshop Objectives (1)

To review roles of staff in administering the ACCESS for ELLs® test

To learn the guidance for placing students in tiers (A, B, C) of the ACCESS for ELLs® test

To understand test security procedures How to draft district and school security checklists


Workshop Objectives (2)Workshop Objectives (2)

To understand and interpret ACCESS for ELLs® score reports Domain, Raw, Scaled, Oral Language, Literacy,

Comprehension, and Overall (Composite) Scores Parent/Guardian Report Teacher Report Student Roster Report School Frequency Report District Frequency Report

To explore the WIDA Can Do Descriptors

To discuss programmatic implications of ACCESS results


Roles & Responsibilities


Roles and Responsibilities (1)Roles and Responsibilities (1)

District Assessment Coordinator Serves as main contact with MetriTech, Inc. Takes Inventory of materials immediately upon arrival and

places them in locked storage. Prepares list of grades to be tested in each school, list of testing

materials required by each school, and testing schedule of each school

Coordinates ordering and distribution of test materials to the schools and returning of test materials to MetriTech

Takes inventory of materials distributed and returned Responsible for training School Assessment Coordinators &

Test Administrators on test administration and security Anticipates and answers questions from school assessment

coordinators at your schools



Test Coordinator (at school-level) Takes inventory of boxes immediately upon their arrival from the

District Assessment Coordinator Verifies that there are enough testing materials Requests additional needed testing materials from MetriTech if

necessary Coordinates and distributes test materials in your school; Takes inventory of materials that are returned to District Test

Coordinators Reminds Test Administrators that all test materials are to be

kept secure and confidential



Test Administrators Complete online training course and certification through

Desire2Learn system (www.uwosh.edu/d2l) Become familiar with procedures in Test Administration Manual

for test accommodations for ELLs with disabilities Fill out the demographic student information on each test

booklet (if pre-ID labels were not ordered) Administer the components of ACCESS for ELLs® for which you

are certified (Kindergarten/Group Components/Speaking)

http://www.uwosh.edu/d2l


Tier Placement


Importance of Tier PlacementImportance of Tier Placement

Appropriate tier placement maximizes the accuracy and validity of the ACCESS for ELLS® test results.


Grade Leveland Tier

K

1-2

3-5

6-8

9-12

A (adaptive – no tiers)

A B C

A B C

A B C

A B C

100 (roll-out Spring 2005)



Listening — group admin, machine scored

Reading — group admin, machine scored

Speaking — individual admin, adaptive, TA scored

Writing — group admin, rater scored

Domains

Series

Structure of ACCESS for ELLs® Test Structure of ACCESS for ELLs® Test

1 Kindergarten form + (4 grade level clustersx 3 tiers) = 13 test forms

Each letter represents one tiered test form


Tier Alignment with Proficiency LevelsTier Alignment with Proficiency Levels

ENTERING BEGINNING DEVELOPINGEXPANDING BRIDGING

1 2 3 4 5

Annual ACCESS for ELLsACCESS for ELLs®®

Tier A

Tier B

Tier C


Tier Placement GuidelinesTier Placement Guidelines

Use previous test scores, if available

Use teacher judgment, if available

Keep in mind that 70-80% of students will take Tier B

Use reading/writing levels over oral skills

When in doubt, place the student in the higher tier

At intake use W-APT™ or equivalent English language proficiency test


Tier Placement “Rule of Thumb”Tier Placement “Rule of Thumb”

Assign an ELL the Tier B test unless you have compelling evidence that items in this tier, given their intended proficiency level, would be way too hard or way too easy for the student.

TIER B is appropriate for most ELLs - those who:have social language proficiency and some, but not

extensive, academic language proficiency in English OR

have acquired some literacy in English, though have not yet reached grade level literacy.


Profile 1: FatimaProfile 1: Fatima

Fatima shows developing language skills in most classroom subjects. Despite her academic interests, she is not yet approaching grade-level literacy in the core content areas.

Which tier is most appropriate for Fatima? Tier B


Profile 2: MohammedProfile 2: Mohammed

Mohammed is in his first year of instruction in English. He is comfortable with basic conversations outside the classroom, but struggles with even low-level reading tasks.

Which tier is most appropriate for Mohammed? Tier A


Profile 3: EstherProfile 3: Esther

Esther is approaching grade level literacy in the core academic content areas. Her teacher feels she will likely meet the state’s exit criteria for ELL support services by the end of the academic year.

Which tier is most appropriate for Esther? Tier C


Profile 4: LilyProfile 4: Lily

Lily shows developing proficiency in academic English, but does not yet reach grade level. She seems comfortable interacting with her monolingual English-speaking peers. Lily’s teacher characterizes her as a beginner.

Which tier is most appropriate for Lily?

A/B line Tier B


Profile 5: ByungProfile 5: Byung

Byung’s intake tests reveal expanding oral skills in English, as well as grade-level literacy in his native language. His English reading and writing skills are lower. A portfolio of his work provides evidence that his literacy skills are not yet on grade level.

Which tier is most appropriate for Byung?

B/C line Tier B


New Jersey’s Testing WindowNew Jersey’s Testing Window

TaskStart Date End Date

Duration (Days)

Test Ordering 02/01/07 02/28/07 28

Pre-ID Ordering 02/01/07 03/07/07 35

State Receives Test Materials 03/26/07 03/26/07 1

Test Window 04/16/07 06/01/07 47

Additional Materials Deadline 05/18/07

Districts Ship Completed Material to MT 06/11/07 06/11/07 1

Reports Shipped to State 08/10/07 08/10/07 1


Maintaining Test Security


Materials Received by the DistrictMaterials Received by the District

District Packing List

Each school’s Packing List

Your state’s schedule

Test Administration Manuals (1 per set of 20 test booklets)

Test Administration Scripts and Speaking Test Picture Cue Booklet/Script (1 per set of 8 booklets ordered per grade-level cluster)

10% overage of Listening, Reading and Writing test booklets

Pre-ID labels and return instructions


District Procedures Upon Receipt of Test MaterialsDistrict Procedures Upon Receipt of Test Materials

Locate and verify district and school packing lists (in Box 1).

Divide materials by school.

Note discrepancies between the Packing List and material received; record on the Documentation of Materials Not Returned form; fax this form to MetriTech (217-398-5798).

Deliver the test assessment materials to the School Assessment Coordinator.

Create sign-in and sign-out forms for test materials (see sample on next slide).

Order additional materials if necessary using the Additional Materials form; fax this form to MetriTech (217-398-5798).


Sample District Security Checklist (Part 1)Sample District Security Checklist (Part 1)

DIRECTIONS: The District Test Coordinator must be sure that the School Test Coordinator has signed the Agreement to Maintain Confidentiality before issuing secure test materials. The School Test Coordinator must sign the District Security Checklist form when secure test materials are issued. The School Test Coordinator’s signature confirms that he/she has received the materials listed below and that he/she will require test administrators to sign the school security checklist and the Agreement to Maintain Confidentiality before issuing secure test materials.

School Test Coordinator Signature: ___________________

Date: ___________________________________________

NOTE: The District Test Coordinator should keep a copy of all signed forms, including the agreements to maintain security, the District Security checklist, and the School Security checklist. The School Test Coordinator should keep one copy of all completed forms and return the original to the District Test Coordinator with the secure test materials.


General Security GuidelinesGeneral Security Guidelines

Only test facilitators, coordinators, and test administrators (or other authorized staff) may handle secure test materials.

Place all secure materials in locked storage.

Do not leave materials unattended before or after testing.

Do not share any specific test information with students prior to or after testing.

Do not copy any test booklets or other secure materials.


Sample District Security Checklist (Part 2)Sample District Security Checklist (Part 2)

Grade Materials # of copies

Kindergarten Test Booklets

Administrator’s scripts

1-2 Tier A Test Booklets


1-2 Tier B Test Booklets


1-2 Tier C Test Booklets


1 -2 Speaking Tests


Materials for Each SchoolMaterials for Each School

School Packing List

Grade/Tier Header Sheets

Documentation of Materials Not Returned form

District and School Test Administration Manuals (1 per set of 20 test booklets)

Speaking Tests for each grade-level cluster; Listening, Reading and Writing test booklets and scripts for each grade cluster & tier tested


School Procedures Upon Receipt of Test MaterialsSchool Procedures Upon Receipt of Test Materials

Divide test booklets into groups for each scheduled testing session. Test booklets may not be distributed prior to testing session Students must use Number 2 pencils; the scanning equipment

used to score answer documents will not read anything but Number 2 pencil marks.

Place pre-ID labels containing the student demographic information in the box on the front cover of the test booklet.

If pre-ID labels were not ordered or if any label contains incorrect information, the student demographic information must be filled in by hand; school test coordinator will give District Code number and School Code number to test administrators.


Sample School Security Checklist (Part 1) Sample School Security Checklist (Part 1)

School Security Checklist

ACCESS for ELLs® Check List

Spring 2007

District:________________ School:___________________

DIRECTIONS: The School Test Coordinator must make sure that the test administrator has signed the Agreement to Maintain Confidentiality before issuing secure test materials. The test administrator must date and sign this form when secure test materials are issued. The School Test Coordinator must sign this form when secure test materials are returned.

NOTE: The School Test Coordinator should keep one copy of all completed forms and return the original to the District Test Coordinator with the secure test materials.


School Security Checklist (Part 2)School Security Checklist (Part 2)

Grade Materials # of copies

Receiving Test Administrator’s Signature

Date & Time Received

School Test Coordinator’s Signature

Date and Time Returned

Kindergarten Test Booklets

Administrator scripts

1-2 Tier A Test Booklets


1-2 Tier B Test Booklets



Score Reports


ACCESS for ELLs® Interpretive GuideACCESS for ELLs® Interpretive Guide

The ACCESS for ELLs® Interpretive Guide for Score Reports (M. Gottlieb, April 2006) contains detailed information on the use of scores from this assessment.

Recommendation: Download the full document (34 pages) from www.wida.us.


ACCESS Component Test Weights as Percentage of Overall Composite ScoreACCESS Component Test Weights as Percentage of Overall Composite Score

Test Weights(Percent)Listening,

15%

Reading, 35%

Writing, 35%

Speaking, 15%

Listening

Reading

Writing

Speaking


Scores Received: Student LevelScores Received: Student Level

Each student receives a scale score and a proficiency level for: Listening Speaking Reading Writing

Scale scores are out of a possible 100 – 600 Scale scores are converted to proficiency scores

using the cuts established during standard setting Proficiency level scores range from 1.0 – 6.0


Scores Received: Student Level (cont’d.)Scores Received: Student Level (cont’d.)

Each student also receives a scale score and a proficiency level for Oral Language, Literacy, Comprehension, and Overall Composite.

Contribution of Language Domains to Composite Scores

Contribution of Language Domains (By Percent) Type of Composite Score

Listening Speaking Reading Writing

Oral Language 50% 50% -- -- Literacy -- -- 50% 50% Comprehension 30% -- 70% -- Overall 15% 15% 35% 35%


ACCESS for ELLs® Score ReportsACCESS for ELLs® Score Reports

There are 5 ACCESS Score Reports:

Teacher Report Parent/Guardian Report Student Roster Report School Frequency Report District Frequency Report


What does the Teacher Report tell us?What does the Teacher Report tell us?

Score Report

Audience or Stakeholder

Types of Information

Teacher

Teachers Administrators

Individual student’s scale scores and proficiency levels for each language domain, Oral Language, Literacy, Comprehension, and Overall Score; raw scores for Comprehension Tasks, Speaking, and Writing Tasks by English language proficiency standard

The Teacher Report contains individual data for one student.


TeacherReportTeacherReport

Report for one individual student


Teacher Report (cont’d.)


Teacher Report (cont’d.)

Raw Scores by Standard


What does the Parent Report tell us?What does the Parent Report tell us?

Score Report



Parent/Guardian Students Parents/ Guardians Teachers School Teams

Individual student’s Overall Score and levels of English language proficiency for language domains (Listening, Speaking, Reading, and Writing) and Comprehension

The Parent Report, like the Teacher Report, contains individual student data.


Interpretive Guide and Parent/Guardian Report Translations available on the WIDA website (www.wida.us)

Special thanks to Milwaukee and Wausau Public Schools for several of the translations!

Parent Report

http://www.wida.us/


Parent Report (cont’d.)

The Parent Report is currently available in:

Amharic

Bosnian-Croatian

Creole

French

Gujarati

Hmong

Korean

Lao

Polish

Portuguese

Russian

Serbian (Cyrillic)

Somali

Spanish

Swahili

Traditional Chinese

Urdu

More translations coming soon!


Parent Report (cont’d.)

May 2006

Dear Parent or Guardian,

This past winter, ELL students in grades kindergarten through twelfth grade participated in the administration of the ACCESS for ELLs® language proficiency test. ACCESS now provides a standardized measurement of academic language proficiency for English Language learners (ELL) students throughout the state of Wisconsin. With this information, we will also be able to monitor individual ELL student progress on an annual basis.

Enclosed you will find your child’s results on ACCESS. The Parent/Guardian Reports provides information about your child’s English Language Proficiency Level. This information is for you to review and keep.

If you have any questions regarding these tests or the information that is being sent to you about how your child did on these tests, please contact your child’s ELL teacher, building principal, or me.

Sincerely,

__________________________________(School ELL coordinator, principal, or teacher)

Parent letter template in English, Spanish, and Hmong that the school can customize - available at www.wida.us


What does the Student Roster Report tell us?

Score Report



Student Roster

Teachers Program

Coordinators/ Directors

Administrators

Scale scores and proficiency levels for each language domain, Oral Language, Literacy, Comprehension, and the Overall Score by school, grade, student, Tier, and grade level cluster

The Student Roster Report lists the scale scores and proficiency levels for a group (or class) of students.


Student Roster Report


What does the School Frequency Report tell us?

The School Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a school.


School Frequency Report


What does the District Frequency Report tell us?

The District Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a district.

Score Report



District Frequency

Program

Coordinators/ Directors

Administrators Boards of Education

Number of students and percent of total tested for each language domain, Oral Language, Literacy, Comprehension, and Overall Score by proficiency levels for grade levels within a district


District Frequency Report


Kindergarten Scores

The maximum Overall (Composite) English language proficiency level that a student taking the Kindergarten form of ACCESS for ELLs® can receive is 3.4.

Language Domain Scale Score

(Possible 100 - 600)*

English Language Proficiency Level

(Possible 1.0 - 6.0)* Listening 308 4.0 Speaking 400 6.0 Reading 284 3.0 Writing 255 2.4 Oral Language (Listening & Speaking) 354 5.4 Literacy (Reading & Writing) 270 2.7 Comprehension (Listening and Reading)

291 3.5

Overall Score (Composite—all language domains) 295 3.4


6-

Reaching

specialized or technical language reflective of the content area at grade level a variety of sentence lengths of varying linguistic complexity in extended oral or written

discourse as required at the specified grade level oral and written communication of English comparable to that of English proficient peers

5-

Bridging

the technical language of the content areas; a variety of sentence lengths of varying linguistic complexity in extended oral or written

discourse, including stories, essays, or reports; oral or written language approaching comparability to that of English proficient peers

when presented with grade level material

4-

Expanding

specific and some technical language of the content areas; a variety of sentence lengths of varying linguistic complexity in oral discourse or multiple,

related paragraphs; oral or written language with minimal phonological, syntactic, or semantic errors that do

not impede the overall meaning of the communication when presented with oral or written connected discourse with occasional visual and graphic support

3-

Developing

general and some specific language of the content areas; expanded sentences in oral interaction or written paragraphs; oral or written language with phonological, syntactic, or semantic errors that may impede

the communication but retain much of its meaning when presented with oral or written, narrative or expository descriptions with occasional visual and graphic support

2-

Beginning

general language related to the content areas; phrases or short sentences; oral or written language with phonological, syntactic, or semantic errors that often impede

the meaning of the communication when presented with one to multiple-step commands, directions, questions, or a series of statements with visual and graphic support

1-

Entering

pictorial or graphic representation of the language of the content areas; words, phrases, or chunks of language when presented with one-step commands/directions,

WH-questions, or statements with visual and graphic support

Performance Definitions

At this level, English language learners process, understand, produce or use:


Can Do Descriptors

Listening

For the given level of English language proficiency level, English language learners can:


Can Do Descriptors

Speaking



Can Do Descriptors

Reading



Can Do Descriptors

Writing



How do we use this information?How do we use this information?

Standards-based results help inform curriculum, instruction and assessment of ELLs

The Overall Composite Score summarizes student’s global language proficiency

Domain subscale scores allow for examination of strengths and weakness by domain

Raw scores by standards allow for examination of strengths and weakness by content area language

Individual report components offer a starting point for differentiating instruction and assessment

Writing and Speaking Rubrics in Interpretative Guide - criteria within rubrics scaffold across the levels of language proficiency and may be used in assessing classroom tasks and projects throughout the year


Programmatic Implications (1)Programmatic Implications (1)

If it’s appropriate to exit the student from ELL services? Does this student have the language skills necessary to access the content in the mainstream classroom without additional language support services? What additional evidence is needed to make a determination?

If the student’s English proficiency is weak in a particular language domain (e.g., Writing)?

If the student’s English proficiency is weak in a particular standard area (e.g., the language of Social Studies)? If so, consider additional content language support.

High scores (Levels 5-6) may indicate a need for Monitoring or Targeted Support. School teams should consider:


A balanced, long-term approach that focuses on grade-level academic standards and English proficiency standards, and utilizes strategies that increase comprehension and communication in English (e.g., sheltered instruction)

Enhancement of both oral language and literacy development

Providing L1 instruction (bilingual education) and/or support where feasible

Programmatic Implications (2) Programmatic Implications (2)

Mid-level scores (Levels 3-4) may indicate a need for 1-3 more years of ELL support services. School teams should consider:


Providing targeted communicative / social & instructional English briefly

Enrolling student in “newcomer” program if available and appropriate

Using content-based strategies (e.g., sheltered instruction) and L1 instruction, if possible

Scaffolding within programs and schoolGraphic supportPeer supportSupplemental and modified materials

Programmatic Implications (3) Programmatic Implications (3)

Beginner level scores (Levels 1-2) may need 5 or 6 more years of ELL support services. School teams should consider:

For more information, please contact the WIDA Hotline:1-866-276-7735 or www.wida.us/helpform

World Class Instructional Design and Assessment, www.wida.us

Center for Applied Linguistics, www.cal.org

Metritech, Inc., www.metritech.com

Questions

or

Comments?

Documents

ACCESS for ELLs ® Tier Placement, Test Security, and Score Reports Emily Evans, Center for Applied Linguistics January 2007 New Jersey Department of Education