Preparing for the 2012 HSTW Assessment

SouthernRegionalEducationBoard

1

Preparing for the

2012HSTW Assessment

Mike Ross Consultant, HSTW & MMGW S.W. Ohio

Susanne DavisHSTW Assessment Coordinator, Ohio


2

Topics Covered

What is the HSTW Assessment?

Administration Nuts & Bolts

Best Practices for Successful Assessments


3

What is the HSTW Assessment?

Required in even-numbered years Subject tests in reading,

mathematics and science based on NAEP frameworks

Student SurveyCourse-taking patternsSchool and classroom experiences

Teacher Survey Provides schools with

comprehensive school-level data breaking out student achievement by experiences


Examples of How Schools Use the Results

Identify improvement needs Gain support for school improvement Engage the entire faculty in school

improvement Set priorities and goals for school

improvement Improve classroom practices Lobby for state actions to support

school improvement

4


5

New in 2012

Teacher Survey will be online.

Constructed response items have been eliminated.

Overall testing time has been shortened by 20 minutes.


Testing Times

2010 2012Reading 90 min. 75 min.Mathematics 70 min. 80 min.Science 70 min. 55 min.

Total time 230 min. 210 min.

6


7

Test Content

Used 2009 NAEP frameworks as a starting pointModified to reflect college and career

readiness goals

Reading – focus on informational texts

Math – topics through Algebra II and some pre-calculus

Science – life, physical and earth science


8

Reading Test

75 minutes80% informational texts; 20%

literary nonfictionIncludes meaning vocabularyIncludes paired texts (to assess

critical analysis)Does not include literary fiction

or poetry


9

Reading Test

Passage length: 500-1500 words

Cognitive targets15% locate/recall55% integrate/interpret30% critique/evaluate


10

Mathematics Test

80 minutes

Scientific or graphing calculators permitted for one sectionType of calculator will neither advantage

nor disadvantage student

TopicsPre-algebra through Pre-calculus,

including Algebra II


11

Mathematics Test

Content10% - Number Properties and

Operations30% - Measurement/Geometry25% - Data Analysis, Statistics

and Probability35% - Algebra


12

Mathematics Test

Cognitive Targets50% - Low Complexity

recall routine procedures

45% - Moderate Complexitymake connections within

mathematics or between mathematics and real

world

5% - High Complexity

analyze a problem mathematically


13

Science Test

55 minutesContent

40% - Life science 40% - Physical science20% - Earth and space science


14

Science Test

Science Practices70%• Identifying science principles• Using science principles

30%• Using scientific inquiry• Using technological design


15

Student Survey Section 1 – Course Experience

Staff member completes this section for each student

Based on Course Matching and Transcript Analysis

Section 2 – School ExperienceStudents complete this sectionSchool and classroom experiencesBased on HSTW key practices

Section 3 – General InformationStudents complete this section


16

Teacher Survey

New for 2012: online survey! School and classroom experiences

School missionHigh expectations and extra helpGuiding and supporting studentsCurriculum content and engaging

students in learningTransitionsLeadership and professional growth


17

Results Reporting

1. School Report (late-June, 2012) PDF file on CD

2. Student Reports (late-April, 2012)

3. HSTW Award of Educational Achievement (late-April, 2012)


18

HSTW Assessment Report

Mailed to schools late-June, 2012 Executive Summary -“3-minute

report”One page summary of performance

and implementation Overview -“30-minute report”

Slightly longer summary Full report -“3-hour report”

Tables breaking out achievement by experiences

Teacher survey results


19

School Report

Executive SummaryOne-page snapshotGraphs and chartsIncludes:

• Implementation summary•Percentage of students

meeting readiness goals: 250, 257, 258

•Percentage of students at each proficiency level

•Blank quadrant for state data


21

School Report

The OverviewApproximately 15 pagesGraphs and chartsIncludes:

• Mean scores• Percent meeting readiness goals• Percent at each proficiency level• Performance on HSTW indices• More detailed implementation

summary


22


23

School Reports

Full report

All those tables!

Student achievement broken out by survey questions

Teacher survey results


25

Student Reports Mailed to schools late-April,

2012 Includes:

Performance level on reading, mathematics and science tests

Completion of recommended curriculum and concentration (C/T, humanities, math/science)

Award status Useful for resumes, college

applications, job applications


27

HSTW Award of Educational Achievement Award certificates mailed to schools

late-April, 2012 Awarded to students who:

Meet all three readiness goals (reading 250, math 257, science 258)

Complete at least 2 of 3 HSTW recommended curriculum goals•English/language arts, mathematics, science

Complete a concentration•Career/technical, mathematics/science, humanities


28

HSTW Award of Educational Achievement


29

Award of Educational Achievement

Opportunities for High Schools

Local community collegesLocal employersAdditional incentives for

students


30

HSTW Assessment Administration

Nuts & Bolts


31

2012 Student Assessment and Teacher Survey

Window

January 27– February 17(3 weeks)

Plan ahead to get the best results!


32

Conducting the Assessment Process

OrderingPreparingAdministeringReturning materials


33

Submit orders to your Regional Office by September 30, 2011

Do NOT order online! Do NOT send order to SREB!

The good news :

There is no cost to your school! HSTW SWO pays for the entire

Assessment!

Ordering the Assessment


34

Previous participation in an assessment Name, phone and email for the assessment

contact person (will be receiving materials for a year!)

Complete shipping address (no P.O. boxes) Principal’s name, phone and email Number of students to be tested and

sampling method SWO Region pays for 60 seniors at your

site. You must pay for any additional students beyond 60 or special composites.

Week you plan to administer the assessment

Number of teachers to be surveyed

Ordering Information Needed

from You


35

Unless there are special reasons, we encourage you to test no more than 60 students.

You will receive extra test sets. However, when submitting scan sheets to be scored, do not exceed the number ordered!

Schools testing more than 60 must pay for each additional student: $34 per student.

A cancellation fee of $13 per student may be charged if you cancel testing or miss the return deadline.

Other Ordering Information


36

Test Administration Guide Sampling Instructions Student Roster Form Checklist Test Description and Sample

Question Booklet Tasks and Timelines Course Description Guide Course Matching Chart

Administration Materials Online at www.sreb.org


Preparing for the Assessment

Administration materials are now available online at: www.sreb.org

Booklets & scan sheets will arrive by January 23, 2012

37


38

Sample Timeline


39

Identify Students for the Assessment Select one of two possible selection

methods:1. RANDOM SAMPLE of 60 (or more) seniors

- OR -

2. ALL SENIORS

Select a true random sample by using the instructions provided. You must also select five alternates at random.



40

Other Tips for Selecting Students

Only include currently enrolled seniors.

Only exclude students if their IEP exempts them from state assessments.

Same accommodations provided for state assessments should be provided for this assessment.


41

Other Tips for Selecting Students

Administrator Guide will contain suggested accommodations for English-language learners.

Contact ETS by October 31 to request large-print or audio CD versions .

Be reasonably assured that all students can complete ALL 4 components – reading, math, science, AND student survey.


42

Random Sampling Options

What is a “random sample”?

How do you draw a random sample?Two options available:

1.a random number table

OR

2.Excel (to generate random numbers)


Random Sampling

Simple random sample - every time a selection is made, every student on the list has an equal chance of being the one selected.

Selection Bias – anything that contaminates or distorts the process.


44

Group ActivityRandom Sampling

Procedures: Option #1 – Random Number

Table1. Develop a list of all eligible students

(delete names of students who moved or have IEP’s stating they may not participate).

2. Number the remaining names of students on the list

3. Use the random number sheet to select students, starting with column one and working down.

4. The table provided works for classes of up to 350. For larger classes, obtain a larger random number table from SREB.

5. Add student names to the roster until you have the appropriate number.


45

Random Sampling Demonstration: Option #2 – Excel

1. Develop a list of all eligible students (delete names of students who moved or have IEP’s stating they may not participate).

2. Number the remaining names of students on the list.

3. Copy your student list into Excel.4. Type =RAND() into each cell of the

column to the right of the student names. This command will insert a random number in that cell.

5. Use the Excel sort function to sort your list into random order.


OHIO CAREER CENTERS – Identifying Students When Home

Schools Are Also Participating

46

Career Center selects random sample.Home School selects random sample.Center and home school should compare lists

to see if any students were double-selected. If no students were double-selected, proceed

as usual. If students were double-selected, students

should assess at the school where they take their senior academic courses.

Call ETS (Lisa Rion – 609-734-5646) to obtain a special code.

Enter that special code on all double-selected students’ answer documents. Their results will go into both reports.


47


Complete the Student Roster - VERY IMPORTANT!!

Fully complete form including assessment answer document ID numbers.

File with school testing records for five years! Also give one copy to principal.

• You will need this fully completed roster form in Spring 2013 to conduct the follow-up survey of graduates

• Also needed to identify students who fail to include their names (for student reports and awards)


Student Roster

48


49


NotificationNotify students, parents and teachersProvide information on assessment

and/or schedule informational meetingRefer to your Board of Education policy

where applicable

SchedulingSchedule time and place (SREB

recommends administering over 2-3 days)

Include make-up sessionsIdentify assistants (follow recommended

ratios)


50


Course-Taking Preparation

Work with department chairs to complete course-matching chart (may update previous work)

Prepare student transcripts (including Fall grades and Spring schedule)


51


52

Other Tips for Course-Matching1. Get each department head for English,

math and science to help with this process.

2. Include any courses the seniors have taken from grades 9-12 within English, math, science or career-technical areas.

3. Using the course description guide, match each course from your school’s master schedule to the SREB course title on the course-matching chart. Example from the course-matching guide

Regular, Advanced, or College Prep Algebra I:“Algebra I courses include the study of properties and

operations of the real number system, evaluating rational algebraic expressions …”


53


Receive Assessment Materials

Booklets & scan sheets arrive at site- by January 23, 2012

Verify contents according to instructions

Keep materials secure (locked)


54

Administering the Assessment

Prepare Assessment MaterialsAssign a numbered answer

document to each student• Record the i.d. numbers on the

student roster

A staff person should complete student survey Section One – “Course Experience” for each senior

–Reduces testing time–More accurate


Administering the Teacher Survey

Detailed instructions will be provided, including:• Sample letter to teachers• Survey and log-on instructions

Teachers must enter the correct five-digit HSTW site code.

55


56


Review Test Arrangements

Confirm dates, times, locations and assistants

Remind students, teachers and parents

Prepare Students

Practice items and use of calculators


Idea Exchange #1

What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to prepare for the Assessment?

57


58


January 27– February 17, 2012

Schedule make-up sessions for absent students.

It is crucial that students complete ALL FOUR components!!


59

Administration Procedures

Assemble materials and set up the room.

Supervise student check-in. Supervise the session using the

administration scripts. You may answer questions about the

content of the student survey, but not the three subject tests.

Keep materials secure at all times. Schedule makeup sessions. Check i.d. information on scan sheets.


60


61

Administering the Teacher Survey

January 27– February 17, 2012Schedule early in the window!Include all full- and part-time classroom

teachers 9th-12th

RECOMMENDED – Administer during a staff meeting or other function to guarantee completion. Use computer lab.

Plan on one hour (although most teachers will finish in about 45 minutes).

Surveys are anonymous.Teachers shared by MMGW & HSTW sites

should take the HSTW version.


Idea Exchange #2

What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to administer the Assessment?

62


63

Returning Materials

Pack according to instructions!Use original boxes & mailing

envelope providedReturn items in their designated

box or envelopeInclude control formsUse the pre-paid shipping labelsKeep receipts and tracking

numbers


Returning Materials

VERY IMPORTANTDo not return more student tests

than you ordered!

Your school will be responsible for paying for all additional

answer documents sent to ETS for scoring ($34 each).

For example, if you ordered 60, do not return more than 60.

64


Returning Student Materials

1. Postmark deadline to mail materials to ETS: February 17, 2012

2. Use the prepaid labels, original boxes and the envelope provided. Include the completed survey control form.

3. If you misplace mailing materials, contact your Regional Office as soon as possible.

4. Return everything! Pack all test books AND extra scan sheets (used or unused) in original boxes and return as instructed.

5. Make sure your materials are picked up.

**KEEP YOUR UPS RECEIPT- this is the only method of tracking your materials.**

65


66

Teacher Surveys

All Teacher Surveys must be completed and submitted online by February 17,

2012.


67

Dates to Remember Orders to Regional Office

September 30, 2011 Administration materials available online

September 1, 2011 Requests for large print or CD versions to

Educational Testing Serviceby October 31, 2011

Testing materials arrive at sites by January 23, 2012

Student & Teacher administration window January 27 – February 17, 2012


68

Dates to Remember

Student materials postmark deadlineFebruary 17, 2012

Award certificates and student reports mailed to sitesby April 30, 2012

School reports mailed to siteslate-June, 2012


69

Common Mistakes to Avoid

1. Returning extra student scan sheets (i.e., more than ordered)

2. Missed deadlines• holding make-up tests late in window• shipping too close to deadline• forgetting to have materials picked up

3. Site code errors • wrong code or no code• site code zip code !

4. Incorrect shipping • shipping to wrong address• shipping in wrong carton or envelope• damaging scan sheets by using tape, rubber

bands, or paper clips


70

Common Mistakes to Avoid

5. Using incorrect test materials • using Xeroxed copies of scan sheets• using old scan sheets

6. Lost items • student roster• pre-paid mailing labels• original shipping cartons• UPS tracking numbers

7. Incomplete student information• names left blank on scan sheets• incorrect demographic information


71

Make the 2012 Assessment a

Success! Communication

Letters to students and their parentsAnnouncements to faculty and staff

CooperationImportance of the assessment to the

school’s improvement goalsImportance of students, administration

and faculty taking it seriouslyHandling make-up workCooperation from teachers whose

classes might be interrupted by the assessment


72

Make the 2012 Assessment a

Success! Encourage all students to do their best.

Build enthusiasmBe honest – explain how this helps

the school and future studentsMake students feel special for

participating in the assessment and thank them for their participation


Recommended Next Steps

to Get Started1. Place order.

2. Download and review materials from www.sreb.org .

3. Assemble a small planning team.

4. Schedule dates, times, locations.

5. Announce preliminary plans to faculty and staff.

73


74

Contact Information Susanne Davis, HSTW S.W. Ohio

[email protected](513) 677-5641

Missy Kramer, HSTW S.W. [email protected](513) 619-2330

Fran Cowart, [email protected](404) 879-5611

Lisa Rion, Educational Testing [email protected](609) 734-5646

Documents

Preparing for the 2012 HSTW Assessment