The New England Common Assessment Program (NECAP)

The New England Common Assessment Program (NECAP)

Mathematics Grades 3-5Including A Brief Introduction and Overview of the (now)

4-State Assessment Collaborative

Presented by Dan Hupp – April 15, 2009

This complete presentation, including audio, will be available as soon as possible at:

http://www.maine.gov/education/lsalt/necap/presentations.html

The PowerPoint portion of this presentation is currently posted.

What exactly is NECAP?

Which States Participate in NECAP?

No Child Left Behind Requirements

Each State Shall…

Develop and implement a statewide accountability system that will be effective in ensuring that all local educational agencies, public elementary schools and public secondary schools make adequate yearly progress.

NECAP is the assessment used by four New England states to meet the testing and accountability requirements of the No Child Left Behind Act.

What exactly is NECAP?

1. School Accountability

2. Public Reporting3. Program Evaluation and

Improvement4. Student Proficiency and Progress

Multiple Uses of NECAP Testing

Why is Maine Joining NECAP?

All Maine state departments were directed by the governor to achieve major cuts in future spending.

The Commissioner was directed to identify significant cost reductions across all MDOE programs.

The state assessment program had only two realistic options:

1) Adopt an all multiple-choice test format;

2) Find a less costly way to deliver a test that also includes student constructed responses.


In the fall of 2008, the MDOE conducted content comparisons and cost analyses associated with joining NECAP.

Comparison studies between NECAP and Maine standards showed adequate correlation (in reading and mathematics) to move forward with adoption of NECAP in Maine.


Savings were estimated at 1 million dollars per year.

The Maine Department of Education petitioned the 3-state collaborative for acceptance into the assessment group.

Acceptance was unanimously offered.


The economies of scale:

The MEA tests were developed, published, scored and reported for approximately 15,000

students /grade level.

The NECAP program provides the same services for approximately 50,000 students/grade level.

Where Do the Savings Come From?

To achieve the cost savings, Maine is adopting the following for the content areas of reading, writing, and mathematics for assessment purposes:

NECAP standards (GLEs) at all grades 2-high school,

NECAP tests at grades 3-8, NECAP cut scores at grades 3-8

to determine achievement levels, and

NECAP reports at grades 3-8.


The 2007 Maine Learning Results (MLRs) are still very much in effect and describe the breadth and depth of knowledge and skills expected to be taught and learned at all grades and in all content areas.

2007 MLRs and NECAP Standards

The MLRs will be modified to include the NECAP assessment standards (GLEs) in mathematics, reading and writing at grades 2-7 and high school, providing a consistent progression along the learning continuum.


Until the MLRs are modified, instruction should be informed by the current Parameters for Essential Instruction unless there is a discrepancy between them and the NECAP GLEs. In those instances, instruction should reflect the NECAP GLEs.


The Department is providing content-specific presentations that discuss the major differences between the NECAP GLEs and the MLRs, which will be posted at:

http://www.maine.gov/education/lsalt/necap/presentations.html


Can you supply a brief history of NECAP?

What are the major differences between the NECAP and the MEAs?

What are the major differences in reporting?

How will AYP be affected?

Some Questions We’ll Try to Answer

Developed through a collaborative effort by Vermont, New Hampshire and Rhode Island

Tests were administered for the first time in October 2005, assessing Reading and Math in grades 3 through 8 and Writing in grades 5 and 8

Grade 11 Reading, Math and Writing was added in Fall 2007

NECAP – A Brief History

Participating states share the same NECAP standards, tests accommodations, cut scores, and reports (for the most part).

Each state maintains its own alternate assessment program

Each state maintains its own accountability system.

NECAP – A Brief History

As in past years, Maine’s Personalized Alternate Assessment Portfolio (PAAP) will parallel the blueprint of the standard state assessment plan.

PAAP alternate standards and tasks in reading, writing, and mathematics are currently being revised.

Maine’s Alternate Assessment

PAAPs will now be administered in Grade 2 for the first time.

The Department is providing a presentation on PAAP that addresses revisions to Maine’s 2009-2010 Alternate Assessment Standards due to the adoption of NECAP standards that will be posted at: http://www.maine.gov/education/lsalt/necap/presentations.html

Maine’s Alternate Assessment

NECAP and the MEA use the same testing contractor, Measured Progress, and NECAP has acknowledged “borrowing” from the MEA when they were forming their program.

Consequently (fortunately), there are many similarities in policies, formats of test and answer booklets, administration manuals, reports, etc.

NECAP Overview

Maine educators will still be involved in reviewing and giving input on all test items by serving on NECAP Item Review and Bias/Sensitivity Committees.

Maine educators, working with colleagues from New Hampshire, Rhode Island and Vermont, have recently completed this work for the October 2009 test.

NECAP Overview - Committees

NECAP assesses the learning of one year (teaching year) at the beginning of the following year (testing year), so grades 2-7 are assessed at the beginning of grades 3-8 in reading and mathematics.

Fourth and seventh grade writing is assessed at the beginning of grades 5 and 8.

Alternate assessments are provided for students in grades 2-7.

NECAP Overview – Teaching/Testing Years

Reading, Mathematics, and Writing tests consist of multiple-choice (1 point) and constructed-response (4 points) items.

The mathematics sessions also include short-answer questions (1

or 2 points).

One writing session includes an extended response item (prompt - 12 points).

NECAP Overview - Test Design

There are 9 test forms at each grade which include 3 sessions in both reading and mathematics of 45 minutes each, with up to an additional 45 minutes allowed (100% extra time).

Forms contain common, field test and matrix items.

There are integrated test/answer booklets at grades 3 and 4.

NECAP Overview Test Design

Let’s Go Math Specific.

NECAP Test Design -Mathematics

Mathematics Test Design – Gr. 3 & 4

There are 65 common score points at grades 3-4 distributed among the following item types :

35-1 point Multiple Choice (MC), 10-1 point Short Answer (SA), 10-2 point Short Answer (SA),

54% MC and 46% Student Generated

NECAP Test Design -Mathematics

Mathematics Test Design – Gr. 5-8

There are 66 common score points at grades 5-8:

32-1 point MC 6-1 point SA 6-2 point SA 4-4 point Constructed Response

(CR)

48% MC and 52% Student Generated

NECAP Test Design - Mathematics

Illustrating Distribution of Emphasis

(PSSM, 2000)

Distribution of Emphasis for Mathematics

Mathematics Content Strands

2(3)* 3(4) 4(5) 5(6) 6(7) 7(8) 10(11)

Number and Operations 55% 50 50 45 30 20 15

Geometry andMeasurement 15% 20 20 25 25 25 30

Functions and Algebra15% 15 15 15 30 40 40

Data, Statistics, & Probability 15% 15 15 15 15 15 15

Total 100% 100 100 100 100 100 100

* 2(3) indicates end-of-grade 2 GLEs are tested at the beginning of grade 3

Session INo Calculator or tools

Mathematics Grades 3 & 4 Test Design

Three 45-minute sessions (Schedule 90 minutes each)*

12 multiple choice(one point each)

5 Short Answer(one point each)

6 Short Answer(two points each)




Session IICalculator and tool active

Session IIICalculator and tool active




*All students are allowed an additional 45 minutes without an accommodation. Additionally, extended time accommodations are available for all students and must be made prior to testing.

Session INo Calculator or tools








Session IICalculator and tool active

Session IIICalculator and tool active



2 Constructed Response

(four points each)


(four points each)


(four points each)

*All students are allowed an additional 45 minutes without an accommodation. Additionally, extended time accommodations are available for all students and must be made prior to testing.

Mathematics Grades 5-8 Test Design

Three 45-minute sessions (Schedule 90 minutes each)*

So what do the NECAP Assessment Standards (GLEs) look like….

….and are there any practice materials out there?


So what do the NECAP Assessment Standards (GLEs) look like….

http://www.maine.gov/education/lsalt/necap/standards.html

….and are there any practice materials out there?

http://www.maine.gov/education/lsalt/necap/released.html


Facilitated Questions:

What are the major differences between the NECAP GLEs at your grade level and your current program mathematics program?



Are any of the noted differences related to material already covered in your current program? Can understanding be attached to/from already secure knowledge?

For example: probability and fractions



What actions can be taken to address these differences? Who will be responsible for taking these actions?

• Short term• Long term



What resources and/or materials might be used to support any identified deficiencies?



Have you considered the many free, on-line resources that are available to all math teachers?

One example: http://illuminations.nctm.org/


……back to NECAP mathematics protocols and procedures…..


Scratch paper is allowed on all math sessions. (ALL answers must appear in the student answer booklet.)


For every grade, calculators (and other supplied tools) are not permitted for the 1st session, but are permitted for the last 2 sessions.


For calculator sessions, the following resources are allowed:

calculators that meet NECAP specifications,

NECAP Multiplication Tables,

NECAP Hundreds Charts, NECAP mathematics tool

kits, protractors and rulers.

25-50% of mathematics items are released.

NECAP Mathematics

Clear plastic rulers are provided to all students for use in sessions 2 & 3. The rulers are marked with both inches and centimeters.

NECAP Mathematics

NECAP accommodations and related policies are currently being reviewed by educators in all four states. (Refer to: http://www.maine.gov/education/edletrs/ilethome.shtml for more information.)

There are several differences in current NECAP accommodations. For instance, in the reading sessions, the questions and answers accompanying the reading passage can not be read to students who have the approved “read aloud” accommodations.

NECAP Overview - Accommodations

The MEA has not allowed modifications. NECAP currently allows the following modifications to the test:

F-1 Using calculator and/or manipulatives on Session 1 of

the Mathematics test F-2 Reading the reading test

HOWEVER, all affected sessions using these modifications are counted as incorrect.


Also, the responses for students who have been approved to use a word processor, typewriter, or computer must be transcribed by school personnel exactly as written into the student’s answer booklet prior to returning materials.

NECAP can not allow computer-generated responses to be returned in the special handling envelope for hand scoring due to the volume of returned materials.


Following the current review, updated accommodation policies and a crosswalk to 08-09 MEA accommodations will be available prior to the start of testing.

IEP teams should use the current 08-09 MEA accommodation policies and codes for Spring 2009 meetings. These policies are located at: http://www.maine.gov/education/mea/0809materials/policiesprocedures.pdfApproved accommodations from the MEA 08-09 list will be allowed for the 2009 NECAP.


The NECAP testing window will always begin on October 1st, or the first school day following October 1st, each year and is 3 weeks long.

Maine’s first administration of the NECAP tests will begin on October 1, 2009 and continue through October 22nd.

NECAP Overview – Administration Dates

Testing materials will be shipped from MP the week of September 14 -18, 2009. They should arrive in schools during the latter part of that week or more likely the beginning of the following week.

Following test administration, the materials will be picked up on October 23rd.

NECAP Overview – Administration Dates

NECAP test administration presentations (possibly in this electronic format) containing more specific information will occur in September 2009.

NECAP Overview – Administration Info.

NECAP is very similar to Maine’s current reporting system in that it:

Utilizes an 80-point scale, with an initial digit representing the grade being tested.

Classifies student achievement across 4 achievement levels with definitions similar with Maine’s present language.

The NECAP Reporting System

NECAP Achievement Levels

Proficient with Distinction (Level 4)

Students performing at this level demonstrate the prerequisite knowledge and skills needed to participate and excel in instructional activities aligned with the GLE at the current grade level. Errors made by these students are few and minor and do not reflect gaps in prerequisite knowledge and skills.

(MEA - Exceeds the Standards)

Proficient with Distinction (Level 4)

Students performing at this level demonstrate the prerequisite knowledge and skills needed to participate and excel in instructional activities aligned with the GLE at the current grade level. Errors made by these students are few and minor and do not reflect gaps in prerequisite knowledge and skills.

(MEA - Exceeds the Standards)


Proficient (Level 3) Students performing at this level demonstrate minor gaps in the prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. It is likely that any gaps in prerequisite knowledge and skills demonstrated by these students can be addressed during the course of typical classroom instruction.(MEA – Meets the Standards)

Proficient (Level 3) Students performing at this level demonstrate minor gaps in the prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. It is likely that any gaps in prerequisite knowledge and skills demonstrated by these students can be addressed during the course of typical classroom instruction.(MEA – Meets the Standards)


Partially Proficient (Level 2)

Students performing at this level demonstrate gaps in prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. Additional instructional support may be necessary for these students to meet grade level expectations.

(MEA – Partially Meets the Standards)

Partially Proficient (Level 2)

Students performing at this level demonstrate gaps in prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. Additional instructional support may be necessary for these students to meet grade level expectations.

(MEA – Partially Meets the Standards)


Substantially Below Proficient (Level 1)

Students performing at this level demonstrate extensive and significant gaps in prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. Additional instructional support is necessary for these students to meet grade level expectations.

(MEA – Does Not Meet the Standards)

Substantially Below Proficient (Level 1)

Students performing at this level demonstrate extensive and significant gaps in prerequisite knowledge and skills needed to participate and perform successfully in instructional activities aligned with the GLE at the current grade level. Additional instructional support is necessary for these students to meet grade level expectations.

(MEA – Does Not Meet the Standards)

NECAP assessment reports are very similar to MEA reports, will be available in late January 2010 and will consist of:

Individual student reports School and SAU reports Class analysis reportsThey will be delivered on the same secure online reporting tool as the MEA reports have been.


NECAP Student Report

60

The Item Analysis Report

61

The Item Analysis Report

NECAP reporting presentations containing more specific information will occur in late January and/or February 2010.


2009-10 Adequate Yearly Progress (AYP) determinations will be based on the following assessments of learning during the 2008-09 teaching year:

October 2009 NECAP reports 2008-09 PAAP data (except at

grade 3) first year LEP participation

based on 2008-09 ACCESS administration.

Maine Accountability Reporting

NECAP information, such as GLEs, released items, other content-related information, and administration information is or will be posted and regularly updated on links from the NECAP page on the DOE website at:

http://www.maine.gov/education/lsalt/necap/index.html.

Maine Accountability Reporting

Wanda Monthey

Team Leader

207-624-6626

[email protected]

Daniel Hupp

Director of Student Assessment

207-624-6827

[email protected]

Susan Smith

MEA/NECAP Coordinator

207-624-6775

[email protected]

Kim Schroeter

PAAP and Accommodations Coordinator

207-624-6774

[email protected]

Diana Dorion

ELA Content Specialist

207-624-6823

[email protected]

Patsy Dunton

ELA Content Specialist

207-624-6625

[email protected]

Sandra McKechnie

Program Assistant

207-624-6770

[email protected]

Rachelle Tome

Title 1A School Improvement - AYP

207-624-6705

[email protected]

Maine DOE Contact Information

Documents

The New England Common Assessment Program (NECAP)