Grade 11 Grade 11 NECAP ReportNECAP Report

Rhode Island Department of Rhode Island Department of EducationEducation

February 27, 2008February 27, 2008

22

For the first time, performance results from the For the first time, performance results from the New England Common Assessment Program (NECAP) for New England Common Assessment Program (NECAP) for Grade 11 students are being released today by Governor Grade 11 students are being released today by Governor Carcieri and Commissioner McWalters. Carcieri and Commissioner McWalters.

Through a collaborative partnership believed to be the first Through a collaborative partnership believed to be the first of its kind in the nation, public school students from New of its kind in the nation, public school students from New Hampshire, Vermont, and Rhode Island participate in Hampshire, Vermont, and Rhode Island participate in common assessments to measure proficiency in reading, common assessments to measure proficiency in reading, mathematics and writing. mathematics and writing.

Students in grades 3-8 have participated in the NECAP Students in grades 3-8 have participated in the NECAP assessment program for the last 3 years. Those results assessment program for the last 3 years. Those results were released to the public on January 31.were released to the public on January 31.

First Release of NECAP Grade 11First Release of NECAP Grade 11

33

Rhode Island DataRhode Island Data61% proficient or better in reading at Grade 1161% proficient or better in reading at Grade 11

44

Rhode Island DataRhode Island Data

37% proficient or better in writing at Grade 1137% proficient or better in writing at Grade 11

55

Rhode Island DataRhode Island Data22% proficient or better in mathematics at 22% proficient or better in mathematics at

Grade 11Grade 11

66

When we look at When we look at the three subject areas…the three subject areas…

Reading and Writing grade 11 achievement Reading and Writing grade 11 achievement results mirror grade 8 in the first year of results mirror grade 8 in the first year of NECAPNECAP

Mathematics results do not.Mathematics results do not.

Why?Why?

What questions should we be asking?What questions should we be asking? Was it the test?Was it the test? Is the test too hard? too long? too different?Is the test too hard? too long? too different?

How did we get here?How did we get here?

77

The NECAP Test The NECAP Test Development ProcessDevelopment Process

First defined what students should know First defined what students should know

and be able to do - Grade Span Expectations (GSEs)and be able to do - Grade Span Expectations (GSEs)

TeachersTeachers developed the GSEs based on developed the GSEs based on

national standards and best researchnational standards and best research

Developed test specifications (items, length, etc..)Developed test specifications (items, length, etc..)

Developed and reviewed test itemsDeveloped and reviewed test items Teacher committees Teacher committees reviewed every item reviewed every item

for content and biasfor content and bias

andand thenthen …… ……

88

Gathered additional student data -Gathered additional student data - Teacher judgments; NHEIAP and NAEP historical dataTeacher judgments; NHEIAP and NAEP historical data

Considered the review of our test and our standards by Considered the review of our test and our standards by national content experts and professional organizationsnational content experts and professional organizations

Set Achievement Levels:Set Achievement Levels: Teachers in each subject area from the 3 NECAP states:Teachers in each subject area from the 3 NECAP states:

--Took the testTook the test

-Wrote Achievement Level Descriptors-Wrote Achievement Level Descriptors

-Developed Item Maps-Developed Item Maps

-Recommended cut scores for each achievement -Recommended cut scores for each achievement levellevel

The NECAP Standard Setting The NECAP Standard Setting ProcessProcess

99

What did the 2007 What did the 2007 Reading test look like?Reading test look like?

After responding to a small number of After responding to a small number of vocabulary questions, students were asked vocabulary questions, students were asked to read 3 long and 3 short literary and to read 3 long and 3 short literary and informational passages and answer informational passages and answer questions after each. questions after each.

There were 42 multiple choice (1 pt.) There were 42 multiple choice (1 pt.) questions and 9 constructed response (4 questions and 9 constructed response (4 pt.) questions. pt.) questions.

The test was given in 2 sessions, each The test was given in 2 sessions, each designed to take about an hour. Test designed to take about an hour. Test administrators were instructed to ensure administrators were instructed to ensure that 90 minutes be allowed for each that 90 minutes be allowed for each session.session.

1010

What did the 2007 What did the 2007 Writing test look like?Writing test look like?

The high school writing test was a single item test. The high school writing test was a single item test. This extended writing exercise required students to This extended writing exercise required students to respond to informational textrespond to informational text. All students were . All students were scored on the same task.scored on the same task.

Each student also responded to a second test item Each student also responded to a second test item that was not reported as part of their individual that was not reported as part of their individual student score. This second itemstudent score. This second item sampled one of sampled one of the other forms of writing in the high school GSEs the other forms of writing in the high school GSEs (persuasive, response to literary text, report or (persuasive, response to literary text, report or procedural writing). All forms of writing were procedural writing). All forms of writing were sampled within each school. Results from these sampled within each school. Results from these sampled items are reported at the school, district sampled items are reported at the school, district and state levels to provide and state levels to provide program-wide program-wide feedback. feedback.

1111

What did the 2007What did the 2007Mathematics test look like?Mathematics test look like?

There were 64 points on the testThere were 64 points on the test More than a third of the test points (24) More than a third of the test points (24)

came from came from multiple-choicemultiple-choice items. items. The remaining points (40) came from 1, 2 The remaining points (40) came from 1, 2

and 4 point and 4 point constructed responseconstructed response items. items. The test was given in 2 sessions, each The test was given in 2 sessions, each

designed to take about an hour. Test designed to take about an hour. Test administrators were instructed to ensure administrators were instructed to ensure that 90 minutes be allowed for each session.that 90 minutes be allowed for each session.

1212

What did 9,157 RI students What did 9,157 RI students say about Reading?say about Reading?

When asked how When asked how oftenoften they they read…read… 40.1%40.1% said they read a few times a week said they read a few times a week

or almost every day in their free time.or almost every day in their free time.

37%37% said they read a few times a month said they read a few times a month or almost never during their free time.or almost never during their free time.

How do they define “reading”?How do they define “reading”?

When asked how they get When asked how they get information about things that information about things that

interestinterest them… them…

15%15% said said …they use hard-copy printed …they use hard-copy printed materials to find information that interests them.materials to find information that interests them.

47%47% said said …they usually use …they usually use a computera computer to find to find information about things that interest them.information about things that interest them.

What did 9,157 RI students What did 9,157 RI students say about Reading? say about Reading? …cont’d.…cont’d.

Maybe they’re reading more Maybe they’re reading more

than they (and we) think they than they (and we) think they are.are.

Are we teaching 21Are we teaching 21stst century century reading skills?reading skills?

1515

What did 9,157 RI students sayWhat did 9,157 RI students sayabout Writing?about Writing?

How difficult was the writing test? How difficult was the writing test? 45%45% said the test was said the test was about the same as about the same as regular regular

workwork. . 20%20% said it was said it was harderharder than their writing school than their writing school

work. work. 14%14% of students said it was of students said it was easiereasier..

How hard did you try on the writing test?How hard did you try on the writing test? 46%46% said about said about the samethe same as I do on my regular as I do on my regular

work.work. 20%20% said they tried harder on this test. said they tried harder on this test. 13%13% said they did not try as hard as they do on said they did not try as hard as they do on

their regular work.their regular work.

1616

What did 9,157 RI students sayWhat did 9,157 RI students sayabout Writing?about Writing?

QQuestion:

How often are you asked to write…

Daily or More Often

A Few Times a Week

Less than Once a Week

...at least one paragraph

in Science class?

17% 30% 30%

…to explain your ideas

in Mathematics class?

25% 27% 26%

...at least one paragraph

in English class?

31% 38% 9%

1717

What did 9,157 RI students What did 9,157 RI students say about the Mathematics say about the Mathematics

test?test? 42%42% said the test was more difficult said the test was more difficult

than typical tests in schoolthan typical tests in school

Only 39%Only 39% said they have to explain said they have to explain how they solved a math problem how they solved a math problem almost every day in school (like on almost every day in school (like on the test)the test)

55%55% said they used a calculator on said they used a calculator on the test when allowedthe test when allowed

1818

What else did RI students say What else did RI students say about the Mathematics Test?about the Mathematics Test?

39%39% said they had enough time to said they had enough time to finish all test items and to check their finish all test items and to check their work.work.

19%19% said they had time to finish, but said they had time to finish, but not check their worknot check their work

13%13% said they felt rushed, but did said they felt rushed, but did finish.finish.

10%10% said they did not have enough said they did not have enough time.time.

1919

What did Rhode IslandWhat did Rhode Island Test Administrators say about Test Administrators say about

the Mathematics Test?the Mathematics Test? 84%84% of respondents said that most students of respondents said that most students

gave a good effort on the testgave a good effort on the test

29% 29% said students finished session 1 of the said students finished session 1 of the test in under an hour test in under an hour (the non-calculator (the non-calculator session).session).

28% 28% said students finished session 2 of the said students finished session 2 of the test in under an hourtest in under an hour

77% 77% of 631 respondents said to shorten the of 631 respondents said to shorten the time per session or keep the timing as istime per session or keep the timing as is

RI Test Administrators Respond: How much time did students require to complete each test session?

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Writing Reading Math Sess 1 Math Sess 2

Content Area Sessions

30-45 Mins 46-60 Mins

61-75 Mins 76-90 Mins

2121

What did Rhode IslandWhat did Rhode Island Test Coordinators say Test Coordinators say

about the GSEs?about the GSEs? OnlyOnly 44 44%% of respondents said the of respondents said the

GSEs have been GSEs have been fully integratedfully integrated into their curriculuminto their curriculum

37%37% said they have said they have just begunjust begun or or are workingare working on implementation and on implementation and integration of GSEsintegration of GSEs

19%19% said they have said they have not yet startednot yet started to integrate the GSEs into curriculum.to integrate the GSEs into curriculum.

Released Reading Test Item – Reading Test Item – Constructed Response (4 pts.)Constructed Response (4 pts.)

By Naomi Shihab Nye

Reading Test Item – Reading Test Item – Scoring GuideScoring Guide

Extended Writing PromptExtended Writing Prompt (12 pts.)(12 pts.)

Each Each student student response was response was scored by two scored by two independent independent scorers using scorers using a 6 pt. rubric. a 6 pt. rubric. (2 x 6 = 12 (2 x 6 = 12 maximum maximum points)points)

Scoring Rubric for the Writing Prompt

2626

Released Mathematics Test Released Mathematics Test Item – Item –

Short AnswerShort Answer(calculator allowed)(calculator allowed)

The manager of a music store ordered 20 The manager of a music store ordered 20 new violins. She ordered some of two new violins. She ordered some of two different models- the standard and the different models- the standard and the deluxe. Each standard violin costs $500, deluxe. Each standard violin costs $500, and each deluxe violin costs $800.and each deluxe violin costs $800.

If the manager spent exactly $11,500 on If the manager spent exactly $11,500 on these violins, how many deluxe violins did these violins, how many deluxe violins did she order? Show your work or explain how she order? Show your work or explain how you know.you know.

2727

2828

Released Mathematics Test Released Mathematics Test Item – Item –

performance informationperformance information

This item was worth 2 pointsThis item was worth 2 points The p value for this item is .33The p value for this item is .33 12.6% of students left the item blank12.6% of students left the item blank 48.9% of students attempted but 48.9% of students attempted but

earned no pointsearned no points 11.1% of students earned 1 point11.1% of students earned 1 point 27.3% of students earned 2 points27.3% of students earned 2 points

2929

4 point Constructed Response (CR) 4 point Constructed Response (CR) ItemItem

3030

3131

How did students do How did students do on this mathematics CR item?on this mathematics CR item?

Of 20 random Of 20 random students at students at proficiency (1140) :proficiency (1140) : 5 (25%) scored 4 pts5 (25%) scored 4 pts 0 scored 3 pts0 scored 3 pts 9 (45%) scored 2 pts9 (45%) scored 2 pts 1 (5%) earned 1 pt1 (5%) earned 1 pt 4 (20%) earned 0 pts4 (20%) earned 0 pts 1 left it blank1 left it blank

Of 20 random Of 20 random students at the students at the median (1135):median (1135): 1 (5%) scored 4 pts1 (5%) scored 4 pts 2 (10%) scored 3 pts2 (10%) scored 3 pts 3 (15%) scored 2 pts3 (15%) scored 2 pts 3 (15%) scored 1 pt3 (15%) scored 1 pt 9 (45%) earned 0 pts9 (45%) earned 0 pts 2 (10%) left it blank2 (10%) left it blank

3232

Some reflections on Some reflections on RI Standards…RI Standards…

It’s the first time we have assessed the It’s the first time we have assessed the Grade Span Expectations (GSEs) - our Grade Span Expectations (GSEs) - our current state standards.current state standards.

The standards are different than the former The standards are different than the former content standards used for the NSREcontent standards used for the NSRE

We We listenedlistened to community and post- to community and post-secondary feedback, looked at current secondary feedback, looked at current research, and updated our standards to research, and updated our standards to include what students need to know and be include what students need to know and be able to do to succeed in the 21able to do to succeed in the 21stst century century

More reflections on More reflections on RI Standards…RI Standards…

The Reading standards remain consistent with The Reading standards remain consistent with earlier standards and the 2007 scores reflect that.earlier standards and the 2007 scores reflect that.

Writing results are less than acceptable, Writing results are less than acceptable, but are but are consistentconsistent with feedback received from with feedback received from community employers, colleges and national community employers, colleges and national research.research.

Students’ writing skills need to improve. To do this, Students’ writing skills need to improve. To do this, students must write in students must write in all all content areas every day.content areas every day.

Our students did not meet our mathematics Our students did not meet our mathematics performance expectations.performance expectations.

These mathematics results These mathematics results areare also consistentalso consistent with feedback received from community with feedback received from community employers, colleges and international studies.employers, colleges and international studies.

3434

Some questions to be Some questions to be answered…answered…

What do the students say about the What do the students say about the results?results?

In what areas of the curriculum are In what areas of the curriculum are students struggling?students struggling?

Are students reading a variety of texts, Are students reading a variety of texts, including informational texts? including informational texts?

Are instructional materials and practices Are instructional materials and practices used in RI high schools aligned to the used in RI high schools aligned to the GSE’s?GSE’s?

3535

Some more questions to be Some more questions to be answered…answered…

Do Do allall students have access to the full students have access to the full curriculum?curriculum?

What is the impact of ‘tracking’?What is the impact of ‘tracking’?

How does instruction address student How does instruction address student preconceptions and misconceptions in each preconceptions and misconceptions in each content area?content area?

How does instruction create a balance between How does instruction create a balance between proceduralprocedural and and conceptualconceptual knowledge? knowledge?

3636

What is the RI Department of Education What is the RI Department of Education doing to support doing to support

RI schools, districts, and students?RI schools, districts, and students? The mathematics results are a call to action that The mathematics results are a call to action that

RIDE will lead over the next 12 months.RIDE will lead over the next 12 months. Personal Mathematics Plans, similar to RI’s Personal Mathematics Plans, similar to RI’s

Personal Literacy Plans, must be developed for Personal Literacy Plans, must be developed for students who are performing below grade level.students who are performing below grade level.

Supports to educators will be expand by Supports to educators will be expand by requiring prospective middle school teachers to requiring prospective middle school teachers to pass content examinations in order to be pass content examinations in order to be certified, focusing I-Plans on mathematics when certified, focusing I-Plans on mathematics when needed, and providing professional needed, and providing professional development.development.

3737

What is the What is the RI Department of Education doing?RI Department of Education doing?

….. continued….. continued

An expert task force will lead districts through a An expert task force will lead districts through a program alignment to ensure that mathematics program alignment to ensure that mathematics curriculum and materials are aligned to the GLE curriculum and materials are aligned to the GLE and GSE.and GSE.

RIDE will work with higher education to review the RIDE will work with higher education to review the mathematics preparation within all Teacher mathematics preparation within all Teacher Training Programs.Training Programs.

Work continues with the Pre K-16 Council and our 2 Work continues with the Pre K-16 Council and our 2 and 4 year colleges to define what is necessary for and 4 year colleges to define what is necessary for students to be ‘ready on day one’ for work, students to be ‘ready on day one’ for work, military, or post secondary education.military, or post secondary education.

3838

What is the What is the RI Department of Education doing?RI Department of Education doing?

….. continued….. continued

In addition to RI focus groups , New Hampshire, Vermont In addition to RI focus groups , New Hampshire, Vermont and Rhode Island will conduct a three-state conversation and Rhode Island will conduct a three-state conversation with mathematics teachers to review pedagogy and with mathematics teachers to review pedagogy and ‘habits of mind’ practiced in our schools.‘habits of mind’ practiced in our schools.

RI’s Statewide Curriculum include appendices with RI’s Statewide Curriculum include appendices with resources to support the implementation of the GLEs and resources to support the implementation of the GLEs and GSEs.GSEs.

Each year, NECAP test items, support materials and Each year, NECAP test items, support materials and student work samples are released for use and analysis student work samples are released for use and analysis by teachers and students.by teachers and students.

More data will be gathered and analyzed from the More data will be gathered and analyzed from the results, from teachers, and from students. results, from teachers, and from students.

3939

Final thoughts…Final thoughts… These results offer more questions than answers: These results offer more questions than answers:

let’s use them as a starting point, and see them let’s use them as a starting point, and see them both as a challenge and an opportunityboth as a challenge and an opportunity

Lowering our standards is Lowering our standards is notnot an option: we need an option: we need to make the changes necessary to help our to make the changes necessary to help our students rise to meet this challengestudents rise to meet this challenge

We must find out what our schools, students and We must find out what our schools, students and teachers need to help them meet the standards teachers need to help them meet the standards necessary for living and working in the 21necessary for living and working in the 21stst century.century.

Mathematics education must change.Mathematics education must change.