Winter 2012 District Test Coordinator Training

Winter 2012 District Test Coordinator

Training

New Mexico Public Education Department

Assessment & Evaluation Bureau

PED – Assessment & Accountability2

Workshop Outline

A. Welcome & Resource Information

B. N.M. Statewide Assessment Program (NMSAP) Components & Calendar

C. NMSAP New Changes/Developments

Morning Break

D. Testing Staff Roles & Responsibilities

E. Test Security

Lunch Break

F. Special Situations and Accommodations

G. SBA-Specific Information (Measured Progress)


A1. Welcome and Introductions

THANK YOU TO OUR HOSTS!

Today’s presenters: Robert Romero,

Educational Administrator, NM PED Julie-ann Edwards

Program Manager, Measured Progress


A2. PED Assessment & Evaluation Bureau

Diana Jaramillo, Educational Administrator (Standards, SBA) 505-827-6570 [email protected]

Charles Trujillo, Educational Administrator (NMAPA) 505-827-6536 [email protected]

Robert Romero, Educational Administrator (SBA, WIDA, HSCE) 505-827-6524 [email protected]


A3. PED Academic Growth & Analysis Bureau

Dr. Cindy Gregory, Chief Statistician (School Accountability) 505-827-6508 [email protected]

SOAP Helpdesk 505-827-7950

STARS Helpdesk 505-827-7935

PED – Assessment & Accountability

A4. PED Assessment & Accountability Division

Dr. Tom Dauphinee, Deputy Director of Assessment & Accountability Division 505-827-6528 [email protected]

6


A5. Additional State Resources

NMPED General Helpdesk 505-827-4296 Submit questions or comments to PED regarding

a policy, rule, or statute: http://www.ped.state.nm.us/qab/contactForm.html

New Mexico Legislature Read bills, find legislators, and follow votes:

http://www.nmlegis.gov/lcs/default.aspx

7


A6. Test Manuals – See PED webpage

PED NMSAP Procedures Manual:General PED procedures for all NMSAP assessments

Test Security Manual Accommodations Manual SBA and NMHSCE

TCM: Test Coordinator’s ManualDFA: Directions for Administering

NMAPADTC: District Test Coordinator ManualTAM: Test Administrator’s Manual

ACCESS and W-APT Manuals: at www.wida.us 6.10.7 NMAC: State rules for Test Security &

Administration


A7. Testing Contractors/Helpdesks

Measured Progress SBA/HSGA & HSCE Helpdesk 877-676-6722 [email protected]

A.I.R. NMAPA Helpdesk 800-254-6130 [email protected]

WIDA 800-747-4868 [email protected]

W-APT 800-747-4868 [email protected]

9


B1. NMSAP Components

SBA/HSGA: Standards Based Assessment/High School Graduation Assessment (MP)

NMHSCE: New Mexico High School Competency Exam (MP)

NMAPA: New Mexico Alternate Performance Assessment (AIR)

ACCESS for ELLs: ELL language proficiency assessment (WIDA)

W-APT: ELL placement test (WIDA)

10


B2. NMSAP Calendar

At-A-Glance Format of all important dates chronologically by administration date & test family Test Family Grades Tested Initial Order Window Additional Order Window Test Materials in District Test Administration Window Test Materials Pickup Windows Reporting Dates


C. NMSAP Changes/Developments*

1. PED-Required SBA Testing Schedule

2. Grade/Cohort Terminology & Tracking

3. Participation in High School SBA/HSGA

4. HSGA Pass/Fail Cut & Reporting

*Information prepared December 20, 2011

12


C1. Required SBA Testing Schedule

13

3 weeks testing + 1 week make-ups only Two options for grades 3-8 Three options for high school All students at a grade tested on the same

schedule across the district Exceptions were submitted to PED and were

confirmed as approved or denied.


C2a. Cohort Terminology & Tracking

Grades 3-8, same as usual High school testing requirements now based

on cohort (years experienced in high school) High school terminology changing:

14

Current Terminology Former Terminology

High School Year 1 (HS1) Freshman or Grade 9

High School Year 2 (HS2) Sophomore or Grade 10

High School Year 3 (HS3) Junior or Grade 11

High School Year 4 (HS4) Senior or Grade 12


C2a. Cohort Terminology & Tracking

STARS contains high school year of record Opportunity to review/confirm high school

year in STARS prior to testing Final reporting file from PED to MP after

SOAP clean-up will inform high school year for reporting purposes

15


C3a. Participation in High School SBA/HSGA

HS2 All HS2 students (second year in high school) in

public school, charter school, and state educational institutions participate

No HS2 BIE students participate (as of 12/20/11) Student survey, reading and math sessions only;

no HS2 science allowed For growth model data and first opportunity for

graduation requirement Paid for by PED for 2012 only

16


C3b. Participation in High School SBA/HSGA

HS3 All HS3 students (third year in high school)

including public schools, charter schools, state educational institutions and BIE schools participate

Whole test: student survey, reading, math, and science sessions

For use with A-F School Grading (based on growth model) and opportunity for graduation requirement

Paid for by districts, as usual

17


C4. HSGA Pass/Fail Cut & Reporting

2012 first year HSGA requirement effective Reading and math only; science not required for

graduation Calculation change; pass/fail rate no longer

based on Proficient SBA cut. PED applying special calculation (composite score & confidence interval) to determine pass/fail

PED reporting HSGA pass/fail results separately from SBA results

18


C5. HSGA: 3 Chances to Pass then ADC

HS2 (Reading and Math) HS3 (Reading, Math, and Science) HS4 Fall Retest (if Reading and/or Math is

not passed) If Reading and/or Math is not passed after

taking the HSGA in HS4, the Alternate Demonstration of Standards Based Competencies (ADC) is required,

A new ADC process is being designed

19


Break

Please take a 15 minutes break at this time.

20


D. Testing Staff Roles & Responsibilities

1. Four Staff Roles

2. DTC Responsibilities

3. STC Responsibilities

4. TA and Proctor Responsibilities


D1. Four Staff Roles

DTC: District Test Coordinator (public schools)

STC: School Test Coordinator (public schools)

STC: School Test Coordinator (charter, special state supported, and BIE schools) : Acts as DTC

TA: Test Administrators

Proctors


D2a. DTC Leadership Responsibilities

Receives PED training 2x year

Be completely knowledgeable of 6.10.7 NMAC

Communicates assessment information to district, schools parents, and community

Develops District Test Security Policy or Checklist and Cell Phone policy

Responsible for test material orders, inventory, security, and protection at all times

Ensures consistency of test administration

Primary liaison with PED and testing contractors

Set standard for professionalism and motivational leader


D2b. DTC Training Responsibilities

Provides training to STCs and principals 2x year Disseminates 6.10.7 NMAC to all staff Disseminates and trains STC Checklist Provides clear instructions on what is allowed/not allowed Provides manuals in advance of training and reviews

manuals during training Assists STC with training materials Provides and reviews testing schedule calendar(s) Ensures availability of allowable support materials/tools

(calculators, rough draft paper, No. 2 lead pencils, etc.) Develops and trains local secure tracking procedures

(demonstration with substitute test materials is ideal)


D2c. DTC Test Administration Responsibilities

Inspects schools’ accommodations documents

Ensures schools have posted conspicuous signs near copy machines, entry doors, office and classroom doors

Ensures only certified & trained staff act as TAs and Proctors

Inspects schools to ensure each school securely stores assessment materials, follows tracking procedure

Reports irregularities to PED/conducts investigations


D2d. DTC Post-Test Responsibilities

Ensures all students were testedVerifies student labels/biogrid complete &

correctReturns all secure materials to vendor

according to directions


Ensures assessment records are organized and filed properly for 5 years

Maintains STC training agendas and TA signatures

Accounts for all confidentiality forms for individuals involved in handling assessment materials (5 years)

Submits completed Verification Forms to PED

27

D2e. DTC Documentation Responsibilities


D3a. STC Leadership Responsibilities

Receives PED training 2x year (charter, BIE, state school) Be completely knowledgeable of 6.10.7 NMAC Follows STC Checklist; documents completion of tasks Communicates assessment information to students Enforces district policies (test security, cell phone, etc.) Responsible for test material security, tracking, and

protection at all times within school Primary liaison with DTC (or PED and testing contractors) Develops student/teacher and room testing assignments Set standard for professionalism and motivational leader


D3b. STC Training Responsibilities

Provides training to TAs and Proctors 2x year Reviews 6.10.7 NMAC with TAs and Proctors Disseminates and trains TA/Proctor Checklist Provides manuals in advance of training and reviews manuals

during training Trains local secure tracking procedures (demonstration with

substitute test materials is ideal) Reviews student and room testing assignments, including

accommodations, with staff Best practice: spreadsheets with student names, type of

accommodation, location, and TA/Proctor is assigned Trains proper administration of accommodations

Demonstration using released items ideal Provides clear instructions on what is allowed/not allowed

29


D3c. STC Test Administration Responsibilities

Posts conspicuous signs near copy machines, entry doors, office and classroom doors

Actively monitors consistent, proper, and standardized administration across school and correct administration of accommodations (visit classrooms during test sessions)

Distributes allowable support materials/tools when appropriate by session and grade (calculators, rough draft paper, No. 2 lead pencils, etc.)

Securely stores assessment materials daily; follows tracking procedure at all times

Reports irregularities to DTC or PED/conducts investigations

Verify Student Pre-ID Labels & Roster are accurate


D3d. STC Post-Test Responsibilities

Ensures all students were tested Applies student labels Completes biogrid Returns all secure materials, including rough drafts,

to DTC Recycles manuals


D3e. STC Documentation Responsibilities

Develops STC Training Agenda and TA/Proctor sign-in sheets for training; captures all signatures and provides evidence of STC training to DTC

Captures all required signatures of Verification Forms and provides to DTC

Maintain local secure materials tracking system within school; provide to DTC upon request


D4. Test Administrator/Proctor Responsibilities

Administers assessments to assigned students according to directions; follow manuals, read scripts, and offer accommodations as directed

Maintains assessment materials security Protects confidentiality of students Reads the DFA script as written and reviews the

directions with students to ensure understanding Strictly monitors students during testing Maintains an appropriate & fair testing environment Reports irregularities immediately to STC and

principal

Test Security

Protect your students, your staff,

yourself and the test


E. Test Security

1. Four Principles of Test Security

2. Standardization

3. Prohibited Practices & Stray Mark Clean-Up

4. Administrative Forms

5. Materials Storage & Chain of Custody

6. Staff Qualifications

7. Handling Testing Irregularities

35


E1. Four Principles of Test Security

1. Protect the student

2. Protect your staff

3. Protect yourself

4. Protect the test


E2a. Standardization

About equity, fairness, and meaningful results Announce different from classroom work Standardized testing window Standardized testing schedule Training twice per year


E2b. Standardization

Manuals directions and scripts DTC & STCs use the Test Coordinator’s Manual DTC & STC read the DFAs for every grade level so

can answer TA and proctor questions TA reads the DFA directions carefully at least 2 weeks

before testing starts and administers test reading scripts EXACTLY as presented

TAs may not have test booklets more than 24 hrs. in advance of test administration

Maintain strict adherence to every element of the directions/instructions


E3a. Prohibited Practices

Photocopy or reproduce testing materials Paraphrasing questions or student answers Teach from, possess, or disseminate a photocopy of test

material Provide students with review of specific SBA test questions or

answers before, during, or after testing Permit test materials to remain unattended in an unlocked

room Coach or inappropriately assist students during testing Take test materials off-campus, unless PED authorized Disparage or diminish the significance, importance of use of

the standardized tests Cell phones/Smart phones prohibited Stray Mark Clean Up – detailed on next slide


E3b. Stray Mark Clean-up Forbidden

School or district personnel shall not erase or change the answer document in any way, except biogrid information.

Students directed to fully erase changed answers in session directions script

Answer documents with exceptional concerns regarding student work (answers in wrong answer spaces, etc.) should be returned in Special Handling Envelope with an explanatory note.

40


E4. Administrative Forms

Confidentiality Agreement – for all who come in contact with assessment materials: submitted to principal and kept on file for 5 years

Principal’s Verification Form – Verification of accommodations, proper storage, tracking of materials, trained staff; kept on file for 5 years

Superintendent’s Verification Form – sent to PED DTC must keep all forms on file at the district for 5

years The Superintendent’s Verification Form must be sent

to PED after the assessment window.


E5a. Materials Storage

Locked central storage – All testing materials are accounted for daily Materials must be signed out by TA daily

and be checked-in each day after testing Must be locked in a centralized room when

not in use Access to locked room must be limited


E5b. Materials Chain of Custody

Absolutely essential for keeping track of assessment materials

Is effective only if each booklet is counted and signatures document each transaction

Missing materials are immediately known Significantly decreases possibility of lost or

missing materials Significantly increases chances of finding

missing materials


E6a. TA Qualifications

Only trained, certified staff are permitted toadminister tests. Some examples:

Licensed school instructors & administrators

Teachers on waivers or intern licenses

Long-term substitutes with a valid teaching license

Coaches and contractors with a valid teaching license

Licensed counselors and related service providers


Other trained, non-certified staff members may be used to: Provide one-on-one accommodations to

Special Education, ELL, or IEP students

Supervised by a certified, trained staff member

Advance training is imperative.

Must sign Confidentiality Agreement

D6b. Non-certified Staff Support


E6c. Proctor Qualifications

Any staff or community member who has been trained in proper test administration and test security may serve as a proctor

Assist TA, but not act as test administrators

Hall monitoring/ restroom escort

No parents in child’s or relative’s classroom

Must sign Confidentiality Agreement


E7a. Testing Irregularities

A testing irregularity is: Any circumstance within or beyond the control of a school district that in the opinion of the PED or a school district, raises doubts about the propriety of standardized testing procedures, preparation materials, standardized test administration, standardized testing security, student scores attained from standardized testing, or teacher or student conduct observed during standardized testing. (6.10.7.7 (A) NMAC)


E7b. Reporting Testing Irregularities

TA immediately reports any testing irregularity to STC

STC immediately reports testing irregularity to principal & DTC

District/school shall conduct internal investigation immediately

DTC reports incident to PED within 24 hours Follow-up with written report


E7c. Invalidating Tests

The content area for a test must be invalidated if: there is evidence/observation of cheating; students move ahead to the next session

before being directed to do so; students take the test with a prohibited

modification; or students make no effort to complete the

test.


Break

Please break for lunch at this time.

50


F. Special Situations

1. NMHSCE Waivers

2. Parental Refusal

3. ELL Participation (3 years)

4. ELL Students New to the U.S.

5. Accommodations


With the approval of the local board of education, the local superintendent may request written approval from the Secretary of Education to award a diploma to a student who has not passed the Competency Examination and who is not covered under subparagraph (b) of paragraph (1) of subsection K of 6.2.30.10 regarding student with disabilities.

Waiver eligibility is evaluated on a case-by-case basis.(6.30.2 NMAC—Standards for Excellence)

F1. NMHSCE Waivers


E2. Parental Refusal Form

When parents refuse participation of their child in the New Mexico Statewide Assessment Program...

Parents take the refusal to the principal Principal notifies the district office personnel District office personnel should counsel the parent Parental Refusal Form must be completed & kept on file Child not excused from accountability for school or district


F3a. ELL (3 Years) Participation

After three years in U.S. schools, excluding kindergarten: Students must participate in English language

assessment with or without accommodations. Waiver to continue testing student in his/her

home language is available for up to two additional years.

Waiver request submitted by district superintendent to NM Secretary of Education

Waiver is effective only for current year


F3b. SBA Language Waiver Process

1. Complete student data on the form

2. Fill-in all fields – blank fields will cause rejections Rationale must be logical Students who scored ADVANCED on last year’s

NMELPA are no longer ELL

3. Burn on to a CD

4. Mail to the Secretary of Education by January 27th


F4. English Language Learners: New to U.S.

Option available first year in U.S. only May receive language exemption from Reading Language proficiency assessment substituted for

participation Must take Math and other assessments

Scores do not affect accountability performance results

Exchange students are exempt from participation in the NMSAP


F5a. Accommodations

Coded accommodations are allowable only for students on IEPs, 504 Plans or current ELL status.

Biogrid Codes for each accommodation Scribing directions are included in the Appendices All students are eligible for Test Administration Best

Practices. Must be assigned by a school team: SAT No biogrid coding required Examples: extended time, small group <5, or

individual setting

57


F5b. Accommodations Planning

Use assessment accommodation(s) in daily instruction prior to the SBA administration.

Inform students what assistance they will and will not be allowed to receive during testing. Try a practice session ahead of time.

Train TAs and Proctors in specific accommodations Reading passages cannot be read aloud to anyone Record what was actually used, not all available


F5c. Best Test Practice Accommodations

Must be assigned by a school team: SAT Does not require an IEP/504/ELL status No coding required in biogrid Examples:

Extended time Small Group <5 Individual setting

59


G. SBA-Specific Information (MP)

1. SBA Test Design & Subtests

2. SBA 2012 Key Dates

3. SBA 2012 Test Materials

4. Ordering Materials

5. Receiving Materials

6. Preparing Materials for Return

7. Return Packing Procedures

60


G1a. SBA Test Design

Grades 3-8, HS2, HS3 Reading & math all grades Writing grades 3, 5, 8 Science grades 4, 7, HS3 only (not HS2) 80% MC (1-point) ; 20% OE (2, 3, or 4-point) Same design as last year

61


G1b. SBA Subtests and Sessions

Grade Reading Math Writing Science

3, 5 & 8 2 3 3 0

6 & HS2 2 3 0 0

4, 7 & HS3 2 3 0 3

Test order: Student survey, reading, math,

writing or science.


G2. SBA 2012 Key Dates

63

Event Date(s)

Initial Ordering Window 1/3-1/20

Manuals Arrive 2/1

Initial Orders Arrive 2/27

Additional Ordering Window 2/15-4/11

Student Pre-ID Labels Arrive 3/23

Pick-up Date 4/10

Make-up Materials Pick-up Date 4/17

State Data File to PED 6/6

SBA Reports Posted Online (MP) 6/27

SBA Paper Reports in Districts 7/13


G3. SBA 2012 Test Materials

English, Spanish, large print, Braille 1 Form Grade 3 one integrated booklet High school one form across both years MRS, glossaries, rulers New student surveys (grade 3 integrated, grades 4-HS

separate; answer space in regular answer document) Answer space examples in folder; same as last year

64


G4. Ordering SBA Materials

Only PED-approved DTCs/STCs authorized to order Same ordering password for NMHSCE and SBA Automatic 10% overage shipped per initial order Order only tests (E,S,LP,B); all else calculated PO required; ok to use one PO for all orders No need to send PO to MP Save/change orders throughout initial window 2012 Price Grades 3-8 $32.62 2012 Price High School* $74.38

*PED paying HS2 costs this year only; shows $0 in OOS


G5. Receiving Test Materials

Inventory materials immediately upon receiving them. Packing Slips Security Checklist or the Local Inventory Tool

Store materials securely until the testing window; chain of custody starts with receipt of boxes

Report any discrepancies, shortages, or issues to the Measured Progress helpdesk right away

Keep the boxes until after materials are returned


G6a. Preparing SBA Materials for Return

All MC items must be transcribed into AD or they will not be scored!

After testing, label any typed responses papers and place papers for an individual student inside the student’s AD. Never staple, glue, or tape papers into an AD.

67


G6b. Sample Typed Response


G6c. Student Pre-ID Labels

Check Pre-ID Label Rosters; correct STARS/NASIS by end of testing window.

Do not use an SBA Pre-ID Label if it contains outdated or incorrect student information.

Just tear up and throw away unused Pre-ID Labels.

If no label, bubble in all biogrid boxes on cover

Critical Step! Accurately labeling answer documents is critical to accurate reporting. Include a quality control step after applying Pre-ID Labels.


G6d. Do Not Return These SBA Materials

Contaminated materials (blood, vomit, or other bodily output) Call the NM Helpdesk for instructions.

Manuals Math Reference Sheets Rulers Spanish Math & Science Glossaries Student Surveys Unused Pre-ID Labels or Pre-ID Label Rosters


G6e. Focus on Returning These SBA Materials

All Test Booklets (English, Spanish, LP, Braille)

All Answer Documents (used and unused)

All writing rough draft pages

Any student work pages used for accommodations containing student responses to items.


G7a. Non-Scoreable Boxes for SBA/HSGA

All Test Booklets Except Grade 3. Include all grade 4-8 and high school tests, including English, Spanish, Braille, and LP.

Writing Rough Draft Return envelope

Apply Pink Return

Label Over White

Delivery Label


G7b. Packing Scorable Boxes for SBA/HSGA

Return of Used Answer Documents and Grade 3 Test Booklet Envelope(s)

Special Handling Envelope

Return of Unused Answer Documents and Grade 3 Test Booklet Envelope(s)

Apply Blue Return

Label Over White

Delivery Label