California High School Exit Examination (CAHSEE)

California High School Exit Examination (CAHSEE)

Test Administration Training 2009-10 for District Coordinators

June 19, 2009-SacramentoJune 26, 2009-Santa Clara

June 30, 2009-WebCastJuly 22, 2009-Los Angeles

August 5, 2009-MaderaAugust 6, 2009-Monterey

September 29, 2009-San Diego

2

Presenters

•Mao Vang, CAHSEE Project Manager, Educational Testing Service (ETS)

•Rebecca Rabin-Reed, Product Administrator for Customer Services, ETS

•Bonnie Galloway, Consultant, California Department of Education

3

Presentation Objectives

•What’s New•Overview of CAHSEE •Maintaining Test Security•District Coordinator, Test Site Coordinator, Test Examiner, and Proctor Responsibilities

•Demographic Data Corrections•Resources•Questions and Answers

4

What’s New

•Answer document changes: Page 1

• Box 6 – grid “Yes” if Hispanic or Latino • Box 7 – grid all race/ethnicity that are applicable• “Decline to state” option is no longer available. • New Federal Standards for Collecting and Reporting

Race and Ethnicity Categories may be found at http://www.cde.ca.gov/ds/td/lo/refaq.asp#q14

Page 2 • Position of some boxes changed, but required data and

numbering remains the same. Page 12

• Box 24 – American Indian Education has been deleted.

http://www.cde.ca.gov/ds/td/lo/refaq.asp#q14

5

What’s New (Continued)

•Box 13 Directions

revised

6

What’s New (Continued)

•One set of manuals for entire year (July 2009–May 2010) District and Test Site Coordinator’s

Manual Directions for Administration Directions for Administration —

Special Test Versions

7

Overview of CAHSEE

8

CAHSEE Purpose

•Improve student achievement in public high schools and ensure that students who graduate from public high schools can demonstrate competency in reading, writing, and mathematics

9

CAHSEE Content

•The CAHSEE has two parts: English-language arts (ELA)

•State standards up to grade ten Mathematics

•State standards in grades six, seven, and Algebra I

10

CAHSEE Requirement

•All students must satisfy the CAHSEE requirement, as well as all other state and local requirements, to receive a California public high school diploma.

•The CAHSEE requirement can be satisfied by passing the examination or, for students with disabilities, receiving a local waiver according to California Education Code Section 60851(c).

11

Options to Participate in CAHSEE Administrations 2009-10

July 28-29

October 6-7

November 3-4

December 5 & 12

February 2-3

March 16-17

May 11-12

Grade 10(Census)

Grade 10(Census

or Make up)

Grade 10(Make up)

Grade 11

Grade 11 Grade 11 Grade 11

Grade 11 Grade 11

Grade 12

Grade 12

Grade 12 Grade 12 Grade 12

Grade 12 Grade 12

Adult Adult Adult Adult Adult Adult Adult

For testing schedules, go to: http://www.cde.ca.gov/ta/tg/hs/admin.asp

http://www.cde.ca.gov/ta/tg/hs/admin.asp

12

CAHSEE Testing Opportunities

•Grade ten: one time (February, March, or May)

•Grade eleven: *two times•Grade twelve: *at least three, but up to five times

•Adult students: *three times

* May participate in successive administrations.

13

TestDay

TestSession

ApproximateWorking Time

Approximate TotalAdministration Time

Day 1ELA

1 2 hours2 hours 30

minutes

21 hour 30

minutes2 hours

Day 2Math

11 hour 30

minutes2 hours

21 hour 30

minutes2 hours

Approximate Testing Times

14

Maintaining Test Security

15

Maintaining Test Security

•To ensure test security, district coordinators must: Read, agree, sign, and return the Test

Security Agreement (TSA) to ETS Ensure all test site coordinators read

and sign the TSA and all who have access to secure test materials read, sign and agree to the Test Security Affidavit

16

Maintaining Test Security (Continued)

17


18


Keep materials in a secure, locked storage

Establish and maintain procedures that will prevent cheating

19


Handle potential testing irregularities. Call ETS and CDE to report security breaches or test administration incidents

Cooperate with Global Compliance Services when they conduct audits•Before-testing•During-testing •After-testing

20

District Coordinator Responsibilities

DC

21

Before Testing

•Attend training •Send signed Test Security Agreement to ETS

•Read the District and Test Site Coordinator’s Manual (DTSCM)

DC

22

Before Testing (Continued)

•Go to http://www.ets.org/cahsee/ to obtain information and documents regarding the coordination of CAHSEE administrations

DC

http://www.ets.org/cahsee/

23

•Refer to CAHSEE Project Calendar

Before Testing (Continued)DC

24


• November 2009 CAHSEE Project Calendar Sample

DC

25


•Manuals are posted for reference.

DC

26

Go to thesecure CAHSEE Web site by clicking on CAHSEE Online from the ETS CAHSEE Web page or https://cahsee.ets.org/

Before Testing (Continued) DC

https://cahsee.ets.org/

27

•To obtain a username and password for CAHSEE Online, Contact ETS CAHSEE Support at 800-241-5687.

Before Testing (Continued)DC

28


•Use CAHSEE Online at https://cahsee.ets.org to do the following: Update your district contact information Order test materials Order supplemental shipments Submit a Pre-ID file Review/update an order Make student demographic corrections Download student results, roster reports,

and summary reports

DC


29


•Select the appropriate date option.Enter your Username and Password. Select the LOGIN button.

DC

30

CAHSEE Online main page


Update District Information and Testing Schedule

DC

31

•Schedule and set up test administrations

•Notify parents of district’s testing schedule


32


StandardOperationalTest Books

LargePrint

(Version 998)

AudioBooks

(Version 001)

AudioCD

Braille (Version

999)

LP-Braille (Version

999)

LP-BRCD

•Prepare to order materials

CAHSEE Test Books and Special Versions

DC


•Prevent excessive materials ordering•“Excessive materials” is the difference between the sum of the number of tests scored and 80 percent of the tests ordered by the school district for the entire school year. The difference must be greater than 100 to be “excessive materials.”

DC

Excessive Orders Examples

Scenario #1

# Ordered (80%) 20,000 16,000

# Scored 12,000 12,000

Difference 4,000

Cost per test booklet (2009-10)

$0.76

Excessive materials charge

$3,040.00

Scenario #2

# Ordered (80%) 10,000 8,000

# Scored 7,950 7,950

Difference 50

No excessive materials charge (Difference is less than 100)


35

•Order materials and select option to pre-ID


36

•Materials order and option to pre-ID warnings


37

•Materials order confirmation


38

•Go to “Pre-ID services” tab to obtain file layout


39


•Prepare data file according to pre-ID file layout

Upload pre-ID file

DC

40


• If submitting Pre-ID file, fill out the List of Demographic Fields Required to be Hand-gridded for Schools Participating in the Pre-ID Service. Provide the list as a handout for test site

coordinators during training•A Late pre-ID label service is available for a fee. You must have submitted a pre-ID file by the

deadline to use this additional service.

DC

41


42


•Designate and train test site coordinators•Emphasize the following in training:

Maintaining test security Reading Test Site Coordinator Responsibilities

Section of manual Hand-gridding fields not Pre-IDed Accommodations/Modifications/EL Variations

• Signature in Box 13 Checking in/out test materials Providing handout of important dates

DC

43


• Identify and coordinate the testing of students placed in nonpublic schools

•Go to California School Directory http://www.cde.ca.gov/re/sd/ to obtain NPS school code. Enter onto Box 12c.

DC

http://www.cde.ca.gov/re/sd/

44


•Receive test materials (starting about 2 weeks before test administration) Shipment 1: nonsecure testing materials Shipment 2: secure testing materials Shipment 3A: (optional) Pre-ID answer

documents Shipment 3B: (optional) late Pre-ID labels

DC

45


•Upon receipt of shipments, open boxes and conduct inventory

DC

46


47

• If more test materials are needed, contact the CAHSEE Support Center or go to CAHSEE Online to place supplemental orders

•Distribute test materials to sites


48

During Testing

•Be the liaison between test sites, CDE, and ETS

•Problem solve•Notify both CDE and ETS of potential testing irregularities

•Complete the Test Security Breach Report Form, if applicable

DC

During Testing (Continued) DC

50

After Testing

•Receive and inventory test materials from sites

•Account for all test booklets and CDs•Complete District Packing List and Security Tracking

•Ensure all answer documents are batched by grade with School and Grade Identification (SGID) Sheet and packed in scorable boxes

DC

51

School and Grade Identification (SGID) Sheet

PURPLEGrade 10 only

ORANGEGrade 11 only

BLUEGrade 12 only

GREENAdult education students only

SAMPLE

DC

52

Enter district scorable totals

Enter district non-scorable totals

After Testing (Continued) DC

53

•Arrange with courier (FedEx or land carrier) for pickup

After Testing (Continued)

For Scorable materials

For Non-scorable materials

DC

54


•Confirm that the designated CAHSEE color-coded label is in the lower right-hand corner on the top of each box being returned.

•Place the appropriate pre-addressed FedEx label to the immediate left of the designated color-coded label.

DC

55


•Late return of scorable materials may delay results and reports being delivered on time. “Late” = shipped from district after the

Wednesday following the test administration

•If scorable materials are received late, a late processing fee of $450 is charged to the district.

DC

56

Refer to CAHSEE Project Calendar for all pertinent deadlines

After Testing (Continued) DC

57

DCAfter Testing (Continued)

•Interpreting CAHSEE Scores Interpretation Guides (Top 11 languages) Score Conversion Tables

58


•Use CAHSEE Online to download student results

59


•Student detail files, rosters, and summary report

DC

60


•Send student and parent reports home

DC

61


•Send summary reports to schools (ELA Sample)

DC

62


•Send summary reports to schools (Math sample)

DC

63


•Send roster reports to schools

DC

64


•Additional resources and special services form

DC

65

Test Site Coordinator Responsibilities

TSC

66

Before Testing

•Attend training •Send signed Test Security Agreement and Affidavits to district coordinator

•Read the responsibilities and procedures in the “test site coordinator” section of the DTSCM

TSC

67


•Inform parents, students and teachers about testing schedule

•Inform teachers about their students’ attendance at the testing session

•Work with district coordinator to resolve problems

TSC

68


•Organize, implement, and supervise all activities necessary to conduct testing at school site

•Collect signed Test Security Affidavits from all who have access to test materials

•Control inventory. Use Test Site Security Tracking Form and

Inventory Form-Test Site

TSC

69

Before Testing (Continued)TSC

70

Before Testing (Continued)TSC

71


•Receive test materials

•Conduct an inventory of all materials within 24 hours of receipt

•Look at sequence numbers written on top of the boxes and make sure all boxes are accounted for Box 1 of 10, 2 of 10, etc.

TSC

72


•Follow the checklist for Inventorying Test Materials at the Test Site

•Save original boxes for return shipments

•Contact District Coordinator with additional material requests

TSC

73


•Keep test materials secure (Test booklets, Audio CDs, Answer Documents, etc.) in sealed boxes in a locked storage at all times when not in use for test administration

•Test booklets are shrink-wrapped in sets. Do not remove the shrink-wrap until absolutely necessary.

TSC

74


•Train test examiners and proctors•Include the following:

Responsibilities of test examiners and proctors

Security of test materials Testing schedules and locations Preparation of test locations for test

administration Arrangement for proctors, as required

TSC

75


•Train test examiners and proctors (continued) Information about students who have

accommodations/modifications in IEP or 504 plan•Directions for Administration-Special Test Versions manual

Verification of student’s identity Distribution of test materials Assistance with directions Administration of the test

TSC

76


•Train test examiners and proctors (continued) Procedures for completing demographic

information Completion of score codes Collection of and accounting for test

materials Scheduling and procedures for returning

test materials to test site coordinator

TSC

77


•If district submitted a Pre-ID file, provide the handout from the district coordinator for test examiners during training.

TSC

78

Before Testing (Continued)•Prepare answer documents

One answer document must be submitted for each student testing

If using Pre-ID service, prepare blank answer documents for recently enrolled students who have not passed one or both sections

VOID documents for students who are no longer enrolled or who were incorrectly included in the Pre-ID file

TSC

79


•Prepare and distribute test materials using Inventory Form-Test Site

Materials for Each Test Examiner Number to Include

Directions for Administration manual 1 per examiner

Test booklets 1 per student + 1 for examiner

Answer documents 1 per student + 1 blank

No. 2 pencils 1 per student + a few extras

List of Demographic Fields Required to be Hand-Gridded for Schools Participating in the Pre-ID Service

1 copy, if applicable

TSC

80

During Testing (Continued)

•Monitor test administration •Visit each testing room to make sure administration scripts are being read word-for-word

•Ensure students have sufficient time to test

•Ensure proper monitoring of breaks and extended breaks taken during testing, such as lunch

TSC

81


•Be prepared for emergency situations, restroom passes, etc.

•Document testing incidents and potential test security breaches If applicable, complete Test Administration

Incident Report Form and/or Test Security Breach Report Form.

•Ensure compliance with all examination procedures

TSC

82

During Testing (Continued)TSC

•Administration RemindersScratch paper is not allowed.Do not allow students to use extra paper to answer their ELA writing tasks.

All responses to math and ELA items, including writing task, must be recorded with No. 2 pencil on a single answer document.

Do not use a pre-ID answer document from one administration for another.

83


•Administration Reminders (Continued) Do not use correction fluid on any

answer documents. Do not use paper clips, tape, staples,

or glue on the answer documents. Do not attach removable “self-stick”

notes to the answer documents.

TSC

84


•Administration Reminders (Continued) Do not disassemble or pull pages out

of the multi-page answer document or insert additional pages.

If an answer document is damaged, the answers must be transcribed onto a new one and all student demographic fields must be hand-gridded.

TSC

85


•Provide accommodations and/or modifications to eligible students if specified in their IEP or 504 plan

•Provide identified English learners additional test variations if regularly used in the classroom

TSC

86

After Testing

•Collect all examination materials after each test session

•Verify that all test materials from test examiners are accounted for

•Put the scorable answer documents in a secure place between test sessions

TSC

87


•Prepare answer documents for return•Inspect for stray marks•If a student has a Pre-ID answer document, check to make sure the fields that were not Pre-ID are filled in.

•If a student is new or does not have a Pre-ID answer document, all demographic data must be hand-gridded.

TSC

88


•Check answer documents

•Check that Box 3 Score Code is completed on every scorable answer document.

TSC

89


•Check Box 13 Accommodations and Modifications Test examiner must grid Box 13

Accommodations and Modifications codes if any were actually used.

Test site coordinator’s or designee’s signature is required to confirm that accommodations and/or modifications were offered whether used or not.

TSC

90

• If used, correct code must be hand-gridded.

After Testing (Continued)TSC

91


•Transcribe student answers as required: If any student marks answers anywhere other

than in the answer document as a condition of his or her IEP or 504 plan, an authorized person must transfer the answers to the answer document exactly as produced by the student.

If a student uses more than one answer document, answers must be transcribed into a single document. Pages from two documents may not be physically combined.

TSC

92


•VOID an answer document if: The document has been damaged. It is a Pre-ID document for a student who

is no longer enrolled in the district. A student has used more than one answer

document and the answers have been transcribed onto another that has already been included in the scorable materials.

TSC

93


•Prepare scorable answer documents for return Locate the large plastic bag labeled “Test

Site Coordinator Packet” that contains all your return instructions and labels.

Separate and count answer documents by grade.

Do not mix scorable and non-scorable materials.

TSC

94


•Prepare scorable answer documents for return (Continued) Use appropriate SGID for each grade

• Hand-grid section B: Grade, Number of Answer Documents, Date of Exam

Place SGID on top of appropriate grade level documents

Wrap each bundle with paper bands provided in groups of 100

Place BLUE Shipping Labels on boxes with scorable answer documents

TSC

95

School and Grade Identification (SGID) Sheet

PURPLEGrade 10 only

ORANGEGrade 11 only

BLUEGrade 12 only

GREENAdult education students only

SAMPLE

TSC

96


•Prepare nonscorable materials for return Test Site Packing List and Security Tracking

Forms Unused SGIDs Test Booklets, including large type and Braille Audio CDs All original copies of transcribed essays

•Place PINK shipping labels on boxes with non-scorable materials

TSC

97

Enter school scorable totals

Enter school non-scorable totals

After Testing (Continued)TSC

98


•Return materials to district coordinator Follow specific information from district

coordinator and return materials accordingly and appropriately to the district office for consolidation of all test sites’ materials.

TSC

99

Test Examiner Responsibilities

TE

100

Before Testing

•Attend training•Read and sign CAHSEE Security Affidavit

•Read the Directions for Administration (DFA)

TE

101

Before TestingTE

102


•Set up the testing room Students face the same direction and have

at least four feet from center of desk to center of desk

•Cover or erase any information that could clue answers for students

•Place a ‘Testing, Do Not Disturb’ sign outside the door

TE

103


•Check out test materials from test site coordinator

TE

104

During Testing

•Verify identification of students (only students on the list that you received should be tested)

•Give pre-identified answer documents to the correct students

TE

105


•Make sure the students use only No. 2 pencils, including for essay writing.

•Students should clear their desks. Students are not allowed to use scratch

paper or correction fluid. Only No. 2 pencils may be used on the

answer document. Students should not have books, papers,

cell phones, or any electronic devices on their desks.

TE

106


•Hand out the test booklets in the order that you receive them.

•Follow and read the directions exactly as they are written in the DFA.

•Check that students are responding in the appropriate sections and are working independently.

TE

107


•Make sure that each student: Signs name in Box 1

Enters the version number and the barcode number from the back of the test booklet onto the answer document in each portion of the test.

TE

108


•If Pre-ID service is used, ensure students hand-grid additional fields required, if applicable.

TE

109


Approximate Testing Times

TE

TestDay

TestSession

ApproximateWorking Time

Approximate TotalAdministration

Time

Day 1

ELA

1 2 hours2 hours 30

minutes

21 hour 30

minutes2 hours

Day 2

Math

11 hour 30

minutes2 hours

21 hour 30

minutes2 hours


•If some students need more time and have to go to another testing room, the test examiner should collect the test booklets and answer documents from the students. A proctor or other school staff should escort the students to the designated testing room.

•No students should leave a testing room with the test booklets and answer documents by themselves.

TE

111

During Testing (Continued)•Document incidents on the Test Administration Incident Form, if applicable

TE

112

After Testing

•Account for all test booklets and answer documents Separate the answer documents from the

test booklets and make sure that no answer documents are inside test booklets

Separate answer documents by grade•You must account for ALL testing materials.

TE

113


•Code Box 13 & 18,

if applicable.

TE

114


•Code Box 3 (Score Code) for all students.

TE

115


•Return the test materials following the procedure your test site coordinator has set up.

TE

116

Proctor Responsibilities

•Attend training with the test site coordinator

•Read, agree to, sign, and return CAHSEE Test Security Affidavit to TSC

•Assist test examiner with the following: Distribute materials Monitor students during testing Collect materials

117

Demographic Data Corrections

118


•Use CAHSEE Online https://cahsee.ets.org for: Making student demographic

corrections

DC


119

Go to the secure CAHSEE Online site at https://cahsee.ets.org and select the CONTINUE button.

DC


120

Select the database in which you are making Demographic Corrections for. Enter your Username and Password and then select the LOGIN button.

DC

121

Select the drop down menu under the header “Demographic Corrections”.

DC

122

Select the administration in which you need to make demographic corrections to and select the SUBMIT button.

DC

123

There are several options to review your demographic data on this page. Select RETRIEVE MY DISTRICT’S PROFILE.

DC

124

You can view your data in a summary format. Listed here are critical fields.

DC

125

Below the Critical Fields are Optional Fields. Select one of the Optional Fields and you will see a summary of that data on your screen.

DC

126

For example, choose Primary Ethnicity and select the SUBMIT button.

DC

127

View the breakdown for Primary Ethnicity here. Select the Back Button to get to the Demographic Corrections page.

Notice the numbers for African American and American Indian, we will be working with them later.

DC

128

Option 1: To edit an individual student record, use any of the fields listed on the right to search for that student record. Then, select the SEARCH button. For example, the last name is used.

DC

129

All students with that last name are presented. If you select the last name highlighted in blue on the left, you will be able to access this student’s demographic information.

DC

130

You can change any demographic information listed on this page. You are not able to change anything in gray.

DC

131

Example: change this student’s Primary Ethnicity (African American to American Indian) and see how that affects your data. Select the SUBMIT button and you will return to the page with the student’s names.

DC

132

Select the Back to Search Page button and you will end up on the demographic corrections page. Repeat steps to retrieve your district profile and here is the change.

DC

133

African American changed from 83 to 82 and American Indian changed from 10 to 11.

DC

134

Option 2: Mass Student Demographic UpdateThis requires a file in either text format or excel format. The layout is exactly the same as your Pre-ID file except there is an added field for Census/Make-up, field 601.

DC

135

This PDF on the CAHSEE home page at http://www.ets.org/cahsee/ shows all the fields for a text file as well as what fields can be updated. To create a mass demographic correction file, use your Pre-ID file and add filler fields from positions 501-600.

DC


136

Here is a sample of an excel demographic correction file. Notice only key matching fields are needed. When you are ready to upload your file, remove all the headers from the file. Your file will not upload with headers included.

DC

137

Select the BROWSE button and select your demographic correction file from where you have saved it. Select Open.

DC

138

Your file has now been selected and you can select the CONTINUE button to update your district’s demographic data.

DC

139

Your file is processing. The larger the file the longer the processing time will be. Sometimes the files are too large and will need to be submitted as smaller files.

DC

140

You are now able to see where the corrections have been made. You can either CONTINUE with the changes or CANCEL them and start over.

DC

141

If you select CONTINUE, you will receive a confirmation page of your edits. At the bottom of the page is a Table of Errors; you will need to select CONTINUE here for the system to accept your edits.

DC

142

Once the system accepts your edits, you are taken back to your home page and you are able to see the edits you have made by going back into your District Profile page.

DC

143

Now that you have corrected your demographic data, you can see how it affects your District Reporting. Under the District Reporting section, select the CONTINUE button.

DC

144

Let’s look at your data prior to making demographic corrections by selecting District Demographic Summary Report for ELA within grade 11. Select the SUBMIT button and you will see your results.

DC

145

You can see that there are 5 American Indian students who were tested and 47 African American students tested. Now let’s look at this after demographic corrections are completed.

DC

146

You have submitted your Mass Demographic Corrections file and you can now see that there are 6 American Indian and 46 African American students who have tested.

DC

147

Final CAHSEE Data Corrections Window

•Deadline for 2008-09 final data corrections is October 30, 2009

•View reports with corrections immediately•Make corrections immediately after receiving results – don’t wait

•Districts are not charged for this service

148

Resources

149

Opportunities to Participate

•Scoring Opportunities [email protected]

• Item Writers’ Workshop October 27-28, 2009

•Reviewers Bias & Sensitivity and Content reviews in

March and April 2010 Differential Item Functioning (DIF) reviews

after each census administration Contact Usha Altieri 916-403-2406 or

[email protected].

150

Training Presentation Available

•Presentation materials posted at: http://www.ets.org/cahsee/ Look in the “About the Test” tab


151

CAHSEE Regulations

•California Code of Regulations, Title 5, sections 1200 to 1225 govern the administration of the CAHSEE in concert with state law (California Education Code sections 60850 to 60859).

•The CAHSEE regulations can be found at http://www.cde.ca.gov/ta/tg/hs/documents/cahseeregs.doc

http://www.cde.ca.gov/ta/tg/hs/documents/cahseeregs.doc

http://www.cde.ca.gov/ta/tg/hs/documents/cahseeregs.doc

152

Special Education and English Learner Resources

•Available at http://www.cde.ca.gov/ta/tg/hs/accmod.asp Q & A about Test Variations Request for Test Variations Scribe Guidelines Matrix 1. Matrix of Test Variations,

Accommodations, and Modifications for Administration of California Statewide Assessments

Matrix 2. Matrix of Test Variations for Administration of California Statewide Assessments for English Learners

http://www.cde.ca.gov/ta/tg/hs/accmod.asp

153

Student Testing General Guidance

•Available at Available at http://www.cde.ca.gov/ta/tg/sa/ CDE division updates and program notes Test administration dates Guidelines on Academic Preparation for

State Assessments Other guidance for California public

schools

http://www.cde.ca.gov/ta/tg/sa/

154

Contact Information

CAHSEE policy and general examination informationCDE CAHSEE Office

916-445-9449 http://www.cde.ca.gov/ta/tg/hs/

State and federal accountability informationCDE Academic Accountability Unit

916-319-0863 http://www.cde.ca.gov/ta/ac/

Test material ordering and logisticsETS CAHSEE Support

800-241-5687 http://www.ets.org/cahsee/

