11 Using the 2010 NECAP Reports February/March, 2011

11

Using the 2010 NECAP Reports

February/March, 2011

22

Welcome and Introductions

Susan SmithMEA/NECAP Coordinator

Maine Department of Education

Harold StephensNECAP Program Director

Measured Progress

33

Welcome and Introductions

Wanda MontheyTeam Leader624-6831wanda.monthey@maine.gov

Susan SmithMEA/NECAP Coordinator624-6775susan.smith@maine.gov

Dan HuppAssessment Director624-6827dan.hupp@maine.gov

Michele MailhotMath Specialist624-6829michele.mailhot@maine.gov

Patsy DuntonELA Specialist624-6625patsy.dunton@maine.gov

Lee Anne LarsenELA Specialist624-6628leeann.larsen@maine.gov

Rachelle TomeNCLB Title I Director, Accountability & AYP624-6705rachelle.tome@maine.gov

Sue FossettPAAP Coordinator624-6774susan.fossett@maine.gov

Department of Education Staff

44

Welcome and Introductions

NECAP Service Center

1-877-632-7774

55

Purpose of the Workshop

• Review the different types of NECAP reports

• Conduct a demonstration of the Analysis & Reporting System

• Review Maine NECAP data

66

Involvement of Local Educators

• Test Item Review Committees

• Bias and Sensitivity Review Committees

77

NECAP Grade Level Expectations

If students do not have the opportunity to learn the content of the NECAP Grade Level Expectations (GLEs), they will be at a disadvantage when taking the NECAP test.

This means all students, including special education students.

88

NECAP Grade Level Expectations

Remember that• Every item on the NECAP test is

specifically written to measure a NECAP GLE

• GLEs were adopted by the legislature as Maine’s Federal, State, and Local Accountability Standards and became part of the Maine Learning Results: Parameters for Essential Instruction

99

NECAP Grade Level Expectations

• GLEs are located at http://www.maine.gov/education/necap/standards.html

• GLEs should be fully incorporated into each school’s curriculum to ensure that students have an opportunity to learn the content assessed on the NECAP

1010

NECAP Grade Level Expectations

“We are focusing on Common Core State Standards now. Why should we pay attention to NECAP GLEs?”

• The NECAP test will be administered 3 more times in Maine schools: Oct. ‘11, ‘12 and ‘13.

• The assessment to measure the CCSSs does not yet exist and will not be administered until 2014-15.

1111

Basics of NECAP Test Design

• Fall test – previous year’s GLEs• Reading and Mathematics – grades 3-8• Writing – grades 5 and 8• Variety of Item Types

• Reading: multiple choice and constructed response• Mathematics: multiple choice, short answer (one point),

short answer (two points), and constructed response• Writing: multiple choice, constructed response, and

extended response

• Common and Matrix Items

1212

Basics of NECAP Test Design

Reading – Grades 3-8• 52 Common points

Mathematics – Grades 3 & 4• 65 Common points

Mathematics – Grades 5-8• 66 Common points

Writing – Grades 5 & 8 • 34 Common points

1313

Workshop Materials

1414

Types of NECAP Reports

Public Reports

Results Report School and District level

Summary Report School/District/State level

Password Restricted Reports

Released Items Summary DataSchool and District level

Achievement Level SummarySchool and District level

1515

Types of NECAP Reports

Confidential Reports

Student Report Information for Parents/Guardians

Item Analysis Report School level by student

Longitudinal Data Student level, across test administrations

Student Level Data Files Excel/csv files by grade on district and school confidential site

1616

The Family Educational Rights and Privacy Act (FERPA)

• Access to individual student results is restricted to: the student the student’s parents/guardians authorized school personnel

• Superintendents and principals are responsible for maintaining the privacy and security of all student records.

• Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties.

• FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html

1717

Student Report

1818

Looking at the Student Report

1919

Looking at the Student Report

This part of the report provides the individual

student’s achievement

level and scaled score

2020

Looking at the Student Report

This part of the report provides a comparison between the

performance of this student and his/her school, district, and state

2121

Looking at the Student Report

This part of the report gives specific information about the

student’s performance in content area subcategories

2222

Students at Proficient Level

2323

Teaching Year vs. Testing Year

• Rationale

• Inclusion of students in “Teaching Year” reports

2424

Looking at a School-Level Results Report

Schools can view reports for Testing

Year (2010-11)

2525

Looking at a School-Level Result Report

Or for Teaching Year (2009-10)

2626

District and School Results Report

27

School-Level Results Report – Grade Level Summary

27

28

School-Level Results Report – Grade Level Summary

28

2929

School-Level Results Report – Grade Level Summary

3030

School-Level Results Report – Content Area Results

3131

Looking at the Results Report – Content Area Results

Three years of data are typically shown on this report. This year, Maine school reports will include data for 2 years. After three years, a

Cumulative Total will be shown.

3232

School-Level Results Report – Content Area Results

3333

Looking at the Results Report – Content Area Results

Total Possible Points includes both common and matrix items (not field-test).

Total Possible Points also

represents the test’s balance of representation.

Mathematics

3434

Looking at the Results Report – Content Area Results

Please note: The Total Possible Points column is organized differently on the Reading Results Report

106 possible points are represented here – they are sorted by “Type of

Text”

The same 106 possible points are represented here – they are sorted

by “Level of Comprehension”

35

Looking at the Results Report – Content Area Results

Total Possible Points includes both common

and matrix items (not field-test).

Reading

36

Looking at the Results Report – Content Area Results

Total Possible Points are all the points included in the test

by item type.

Writing

3737

Looking at the Results Report – Disaggregated Results

3838

Looking at the Results Report – Disaggregated Results

Important Note: Disaggregated results are

not reported for sub-groups of less than 10

3939

Looking at the Results Report – Disaggregated Results

Because this is a small school, and so many of the sub-groups are smaller than 10, this part of

the report is not as useful.

But we can still look at district and state

disaggregated results.

Does this data match what we

know about the district’s

program?

4040

Summary Report

4141

NECAP District and School Student-Level Data Files

Contain:• All demographic information for each student that

was registered through MEDMS/Infinite Campus State Edition

• The scaled score, achievement level, and subcategory scores earned by each student in all content areas tested

NECAP files also contain:• Performance on released items• Student questionnaire responses

4242

Item Analysis Report

4343

Item Analysis Report

4444

Looking at the Item Analysis Report

4545

Looking at the Item Analysis Report

This part of the report gives specific information about the released items

and student performance on individual items

4646

Looking at the Item Analysis Report

4747

Looking at the Item Analysis Report

This part of the report represents all of the items used to

compute student scores - points are

displayed by subcategory.

Each student’s Scaled Score and

Achievement Level are shown.

4848

• Includes ALL Maine assessment reports using one password: NECAP, MEA, MHSA, PAAP and AYP

• Student Level CSV Files

• Interactive Capability• Construct item-level reports by sub-group• Construct achievement level graphs• View longitudinal data by student (including PAAP)• View item-level response percents/averages

• Account management:• Set up accounts to view data on groups of students• Set up teacher accounts so they can view pertinent data

http://iservices.measuredprogress.org/

MeCAS Analysis and Reporting System

4949

• Guide to Using the 2010 NECAP Reports • NECAP Analysis and Reporting System User and Training

Manual• Companion PowerPoint presentation• Grade Level Expectations • NECAP Accommodations Guide• Released Items documents

www.maine.gov/education/necap

Supporting Materials and Resources

5050

2 Years of NECAP Data

5151

2 Years of NECAP Data

Mathematics

Grade

2009NECAP % Proficientor Above

2010NECAP % Proficientor Above

% Change

2009NECAP Scaled Scores

2010NECAP Scaled Scores

Point Change

3 62 61 -1 342 343 +1

4 62 60 -2 443 443 0

5 64 60 -4 543 543 0

6 63 63 0 643 643 0

7 60 58 -2 742 742 0

8 59 59 0 842 842 0

5252

2 Years of NECAP Data

Reading

Grade2009

NECAP % Proficientor Above

2010NECAP % Proficientor Above

% Change

2009NECAP Scaled Scores

2010NECAP Scaled Scores

Point

Change

3 73 69 -4 346 345 -1

4 67 68 +1 444 445 +1

5 72 70 -2 546 545 -1

6 69 72 +3 645 646 +1

7 68 66 -2 745 745 0

8 69 73 +4 846 847 +1

5353

Maine tests have consistently shown that female students outperform male students

statewide in reading at every grade level.

Sub Group Data - Gender

5454

NECAP 10-11 Performance Analysis

Gender Differences in Reading Performance

Chart Statistics Based on 2010-11 School Year Student Performance

5555

Gender Gap - Reading

If your school/district data is similar to this state data, what strategies have you employed or plan to employ to address this gender gap?

5656

For instance, the NECAP GLEs address both narrative and

informational texts, so NECAP reading tests are designed to include both types equally.

Does your reading curriculum reflect that balance?

Gender Gap - Reading

5757

If you organized the released items in the interactive reporting tool by GLE and chose gender as a filter, what

might you predict about gender differences based on type of text?

Note: Reading GLE codes R4, 5, and 6 refer to literary text and R7 and 8 refer to

informational text.

Gender Gap - Reading

5858

Years ago, the MEA showed a considerable gender gap in

mathematics, with male students outperforming female students

statewide at every grade level, but closing that gap became a priority

and it was closed over time.

Gender Gap - Mathematics

5959

NECAP 10-11 Performance Analysis

Gender Differences in Mathematics Performance

Chart Statistics Based on 2010-11 School Year Student Performance

6060

What percent of students with disabilities score in the

proficient or proficient with distinction levels statewide?

Sub Group Data - SPED

6161

NECAP 10-11 Performance Analysis

Students with IEPs

Chart Statistics Based on 2010-11 School Year Student Performance

6262

IEP Data

What percentage of students scored at proficient or proficient with distinction in reading and mathematics in your school/district data?

Are all of your SPED staff familiar with the NECAP GLEs? Are they included in the same NECAP professional development as classroom teachers?

6363

Sub Group Data - SES

What percent of students identified as low socio-economic status score in the proficient or proficient with distinction levels statewide?

6464

NECAP 10-11 Performance Analysis

Students Identified as SES

Chart Statistics Based on 2010-11 School Year Student Performance

6565

SES Data

What percentage of students who scored at proficient and proficient with distinction in reading and in mathematics might you expect to find in your school/district data?

If the percent is below (or above) state averages, what strategies have you employed or plan to employ to increase performance?

6666

MDOE content specialists have held webinar presentations on instructional strategies to assist teachers in implementing the NECAP GLEs in their classrooms, available at: www.maine.gov/education/ necap/presentations.html

They will continue to present and post NECAP-related webinars on subjects such as DOK, incorporating CR-type questions in daily instruction and assessment, etc.

NECAP Implementation

6767

For more information regarding content-related webinars and other standards-related work, teachers may join the ELA or Mathematics Listserves through links at: http://www.maine.gov/education/standards.htm.

The MEA/NECAP Listserve link is at: http://www.maine.gov/education/necap/admininfo.html

Listserves

6868

Coming Attractions - 2011

• AYP reports released on March 9th

• AYP Performance - teaching year data

• AYP Participation – testing year data

• AYP 2011 targets:

• Reading – 75%

• Mathematics – 70%

• ADA – 93%

6969

Coming Attractions - 2011

• Administration window of the MEA science test for grades 5 and 8: May 9 – 20.

• MEA test administration materials: http://www.maine.gov/education/mea/admininfo.htm

• There will be a test administration webinar on April 27th from 9:30 -10:30 AM that will also be posted on the web.

7070

How Do I Find Public Assessment Reports?

Go to…

www.maine.gov/education/ necap/results