1
Guide to Using the 2006
NECAP & NH-Alternate
Reports:
Companion PowerPoint Presentation
February 2007
New Hampshire Alternate Assessment
New Hampshire Statewide Assessment
2
• Introductions•NHDoE Updates•Background•NH-Alternate Assessment
Reporting•Short break•NECAP Reporting
Agenda
3
New Hampshire Alternate Assessment
Welcome & Introductions
Tim Kurtz
Director of Assessment
NH Department of Education
Gaye Fedorchak
Alternate Assessment Coordinator
NH Department of Education
4
Measured Progress Service Center: 1-877-632-7774
Measured Progress Offices: 1-800-431-8901
New Hampshire Alternate Assessment
Welcome & Introductions
Harold StephensNECAP Program [email protected]
Amanda BreitmaierNH Alt. Assessment Program [email protected]
Amanda SmithNECAP Program Manager – Grade [email protected]
Mellicent FriddellNECAP Program [email protected]
Josh EvansNECAP Program Manager – Grades [email protected]
Tina HaleyNECAP Program [email protected]
Elliot ScharffNECAP Program Manager – [email protected]
Jennifer VarneyNECAP Program [email protected]
5
New Hampshire Alternate Assessment
NHDoE Contacts
Tim KurtzDirector of Curriculum and AssessmentPhone: (603) 271-3846E-Mail: [email protected]
Gaye FedorchakAlternate Assessment CoordinatorPhone: (603) 271-7383E-Mail: [email protected]
Susan Morgan - ELL Access SpecialistPhone: (603) 271-3719E-Mail: [email protected]
Carol AngowskiAssessment Program SpecialistPhone: (603) 271-8392E-mail: [email protected]
Linda Stimson - ELA CoordinatorPhone: (603) 271-2035E-mail: [email protected]
Rich Andrusiak - Mathematics CoordinatorPhone: (603) 271-3844E-Mail: [email protected]
Jan McLaughlin - Science CoordinatorPhone: (603) 271-3856E-Mail: [email protected]
Ken Relihan - Social Studies CoordinatorPhone: (603) 271-6151E-mail: [email protected]
Deb Fleurant - Bias and Sensitivity and Title I
Phone: (603) 271-3838E-mail: [email protected]
http://www.ed.state.nh.us/assessment
6
AssessmentNECAP
Piloted Reading, Math, and Writing in NovemberNo student, school, or district scores will be producedFirst operational test – October 2007
NH-Alt2006-2007 Grade 10 portfolios in Reading, Mathematics, and
Writing2007-2008 In addition to above, Grade 11 portfolios in science
AccountabilitySchools – AYP based on Graduation rate onlyDistricts – AYP based on May 2006 NHEIAP and 2005-2006
Alt
Update: High School Assessment
7
AssessmentNECAP
Pilot Science May 14-24, 2007Two Sessions – about 90 minutes eachNo student, school, or district scores will be producedFirst operational test (three sessions) – May 2008Workshops during the week of April 30 – May 4
NH-AltGrade 11 Science Portfolios will be added in school year
2007-2008
AccountabilityNone at this time, but NCLB reauthorization proposals
include it starting in school year 2007-2008.
Update: Science Assessment
8
i.4.see Workbench• Hub for district submitted student level data• Many new build-it checks on submitted data
i.4.see Dashboard• Check summative student demographic files that affect
AYP• Ability to change incorrect data• Notice will go to Superintendents, Special Education
Directors, i.4.see coordinators, and Principals• Hope to go live on Monday, February 12, 2007• Deadline on changes has not been set yet
Follow the Child Assistance Center• Phase I – April/May• Phase 2 – early summer
Update: Student Demographic & Assessment Data
New
9
Any questions about Updates?
Updates
10
New Hampshire Alternate Assessment
Purpose of the Workshop
• To assist schools and districts in using the variety of NH Statewide Assessment reports and support material in conjunction with the release of results
• To provide school and district personnel an opportunity to ask questions regarding the reports and support materials
11
• Development of Grade Level Expectations (NECAP)
• Development of Alternate Achievement Standards Linked to Grade Level Expectations (NH-Alt)
• Test Item Review Committees (NECAP)
• Bias and Sensitivity Review Committees (NECAP)
• Classroom Teacher Judgment Data (NECAP)
• Standard Setting Panelists (NECAP and NH-Alt)
• Technical Advisory Committee (NECAP and NH-Alt)
• NH-Alt Advisory Task Force (NH-Alt)
Local Educator InvolvementNew Hampshire Alternate Assessment
12
When:• NECAP: Oct 2006• NH-Alternate: School year 2005-2006
What:• Reading and Mathematics –
• NECAP: Grades 3-8 (tests previous year’s GLEs)
• NH-Alt : Grades 2-7, 10 (tested during instruction last year)
• Writing – • NECAP: Grades 5 & 8 (tests previous year’s
GLEs) • NH-Alt: Grades 4 & 7 (tested during
instruction last year)
Test BasicsNew Hampshire Alternate Assessment
13
NECAPVariety of Item Types
– Reading: multiple choice and constructed response– Mathematics: multiple choice, short answer 1 point, short answer
2 points, and constructed response– Writing: multiple choice, constructed response, and extended
response
Common and Matrix Items– Common items used to generate student scores– Matrix items used for equating and field testing
Basics of Test DesignNew Hampshire Alternate Assessment
14
NH Alternate Portfolio Assessment:
What is an “Item” in NH-Alt? – In each content area, two standards-based targeted skills are
identified that address student learning needs– Baseline data are collected for each targeted skill– Work samples are gathered over the course of the year to show
how the student is progressing on these two targeted skills– The collection of work samples and data presented in the
portfolio are scored as a single item– Progress data and student work samples are collected across the
school year (Sept – April) on each targeted skill as the student is learning the skill.
Reading: 2 Targeted Skills Mathematics: 2 Targeted Skills, Writing: 2 Targeted Skills
Basics of Test DesignNew Hampshire Alternate Assessment
15
NH Alternate Portfolio Assessment
Targeted skills are represented by a variety of work samples collected over the course of the year.– Collections include multiple samples of actual student work,
which typically show the student solving problems in many ways, i.e. multiple choice responses, short answer responses, and longer responses that look similar to constructed response problems. Responses often involve use of manipulatives, communication systems, or other instructional technology.
– Authentic classroom constructed response and short answer tasks provide the best methods for allowing students with severe disabilities to show their work in the way they are able to produce it.
– Provides richer evidence of access to standards-based content and student progress.
Basics of Test DesignNew Hampshire Alternate Assessment
16
Norm-Referenced: What is the student’s status or rank in a designated group?
Criterion-Referenced (Standards-Based): Has the student learned the skills or achieved the standard?
Both NECAP and NH Alternate Assessments are Criterion-Referenced Tests.
Types of Educational TestsNew Hampshire Alternate Assessment
17
Questions about the purpose of the workshop
or NECAP or NH-Alternate
background?
Questions and AnswersNew Hampshire Alternate Assessment
18
New Hampshire Alternate Assessment
Student Report (Confidential) with Information for Parents and Report Interpretation Guide
Student Roster Report (Confidential) School and District levels by student
District Student Level Data Files (Confidential)
Disaggregated Results by Content Area: (Public) School, District, & State reported separately for each grade
State Summary Reports (Public)
NH-Alt Students included in NECAP reports as “NT-Approved”
Basics of Test Design
19
New Hampshire Alternate Assessment
20
21
22
23
New Hampshire Alternate Assessment
• Names of Achievement Levels are the same as in NECAP -
Level 4: Proficient with Distinction
Level 3: Proficient
Level 2: Partially Proficient
Level 1: Substantially Below Proficient
• But - describe performance of grade-linked standards-based skills reduced in depth, breadth, and complexity for the student.
NH-Alt Achievement Levels
24
NH-Alt: Two Scoring Dimensions
New Hampshire Alternate Assessment
1. The Performance Dimension –
Most heavily weighted in score and includes: • Student Progress (from start of year baseline)
• Connections and Access to the General Curriculum (Student work samples must show linkage to grade-level content at reduced level of depth, breadth, complexity)
25
NH-Alt: Two Scoring DimensionsNew Hampshire Alternate Assessment
2. The Program Dimension -
Less heavily weighted in scoring, and includes: • Generalized Performance (Student uses skill
across different settings and situations)
• Self-Determination (Student attempts to self direct and monitor work)
• Supports (Does level of assistance provided support growth of independence and match student need?)
26
Sample Portfolio Score CalculationNew Hampshire Alternate Assessment
Scoring Sub-Dimension:
Base Points
(Range 1-4 pts.)
WeightGiven
WeightedSub-Score
Student Progress 4 pts X 4 16
Access to Curriculum 2.5 pts X 4 10
Skill Generalization 2 pts X 3 6
Self-Determination 4 pts X 1 4
Supports 4 pts X 1 4
Calculate Content Area Raw Score:
16 + 10 + 6 + 4 + 4 = 40
27
Raw Score To Achievement Level Conversion Chart
New Hampshire Alternate Assessment
If - Total Raw Score for the content area is:
Then -Achievement Level for the content area is:
47-52 Level 4: Proficient with Distinction
38-46 Level 3: Proficient
29-37 Level 2: Partially Proficient
13-28 Level 1: Substantially Below Proficient
28
Questions about
NH-Alternate Reporting?
Questions and AnswersNew Hampshire Alternate Assessment
29
• Student Report• Item Analysis Report• School/District Results Report• School/District Summary Report• District Student Level Data Files
NECAP: Types of Reports
30
The Family Educational Rights and Privacy Act (FERPA)
• Access to individual student results is restricted to:• the student• the student’s parents/guardians• authorized school personnel
• Superintendents and principals are responsible for maintaining the privacy and security of all student records.
• Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties.
• FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html
Family Educational Rights and Privacy Act
31
National Council on Measurement in Education
Educating Others
Evaluation and Research
Interpretation and Use
Scoring
Development
Marketing and Selling
Selection
Administration
www.ncme.org
32
Student Report
• Two copies mailed to the school (February 7)• color copy for parents/guardians• black and white copy for local use
• Dual sided: Parent Letter & Student Report
• Spanish version of Parent Letter and Student Report shell available for NECAP(available at: http://www.ed.state.nh.us/NECAP)
33
34
35
36
37
Questions about the Student Report?
Questions and Answers
38
• Rationale• Inclusion of students in “Teaching
Year” reports
Teaching School vs. Testing School
39
Teaching School vs. Testing School
Schools Can View Reports for Testing Year (2006-07)
40
Teaching School vs. Testing School
Or Teaching Year(2005-06)
41
Testing Year
Teaching Year
42
http://iservices.measuredprogress.org
43
New thisyear
Please note:
You must always log in through this page to view confidential
reports and data
44
45
46
47
Item Analysis Report
• Available at the school level via password protected website (User ID and passwords sent to Principals and Superintendents in January)
• One report for each content area/grade level combination
• Issued in both Testing Year and Teaching Year formats
• Use in conjunction with Released Items documents
48
49
50
51
Released Item Documents
52
Released Item Documents
• New information provided in the Released Item Support Materials - documents include:
• For all content areas, providing the multiple-choice items and the GLE code and text that each item is linked to
• For math, indicating which items were included in session 1 of the math test by adding a “no calculator” symbol above the items
• For reading, listing the passage name and type of text• For writing, listing passage name and mode of writing
53
Questions about the Item Analysis Report or the Released Item documents?
Questions and Answers
54
• Available on-line (January 30) athttp://reporting.measuredprogress.org/NECAPpublicNH/
• Grade Level Report• One for each grade in each school and district• Issued in both testing year and teaching year
formats
• Summary Reports• One for each school or district• Issued in both testing year and teaching year
formats
School/District Results
55
56
Page 1: Title PagePage 2: Grade Level Summary ReportPage 3: Reading ResultsPage 4: Disaggregated Reading ResultsPage 5: Mathematics ResultsPage 6: Disaggregated Mathematics ResultsPage 7: Writing Results (Grades 5 and 8 only)Page 8: Disaggregated Writing Results (ditto)
Grade Level Results Report
57
58
Questions about the Grade LevelSummary Report page?
Questions and Answers
59
• Pages 3, 5, and 7 of the NECAP School/District Results Report
• Provides current and historical summative information about achievement in tested content areas
• Provides information about achievement in subtopics of tested content areas
Content Area Results
60
Content Area Results - Historical Data
• Data is included for two years and is combined in the Cumulative Total row
• In the Cumulative Total row for the achievement level columns:• The N columns are summed• The % columns are found by dividing the total
number of students in the level by the total number of students tested
• The Mean Scaled Score column is a weighted average
61
62
Content Area Results – Subscore Results
• Results based on both common matrix equating items
• Many Reading items are included twice – once under Type of Text, and once under Level of Comprehension
• Standard Error Bars are included to help determine if differences between scores are statistically significant
63
64
Comparisons of NECAP Scores Across Years
• School and District-Level Scaled Scores and Achievement Levels
• Student-Level Scaled Scores and Achievement Levels
• Content Area Subscores
65
Number of Scaled Score Points Denoting Minimally Statistically Significant Difference for Average Group Results*
Grade SubjectNumber of Students Tested in Group (Class, School etc.)
10 25 50 100 200
3Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
4Reading 6 4 2 2 1
Mathematics 5 3 2 2 1
5
Reading 5 3 2 2 1
Mathematics 5 3 2 2 1
Writing 7 4 3 2 2
6Reading 6 4 3 2 1
Mathematics 6 4 3 2 1
7Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
8
Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
Writing 6 4 3 2 1
*Standard error of the mean difference with assumption n1=n2 and s1=s2
66
Percentage Difference in Student Achievement Level Classification Denoting Minimally Statistically Significant
Differences for Group Results*
Percentages of Students in Achievement Level(s)
Number of Students Tested in Group (Class, School etc.)
10 25 50 100 200
10 9 6 4 3 2
20 13 8 6 4 3
30 14 9 6 5 3
40 15 10 7 5 3
50 16 10 7 5 4
60 15 10 7 5 3
70 14 9 6 5 3
80 13 8 6 4 3
90 9 6 4 3 2
*0.05 level of statistical significance
67
Questions About the Content AreaResults Page?
Questions and Answers
68
• Pages 4, 6, and 8 of the NECAP School/District Results Report
• Provides information about achievement for different groups in a school/district
• Testing year reports have detailed information about all students enrolled in October 2006
• Teaching year reports include only those students with NECAP scores from Oct 2006.
• Detailed information for districts and the state may be found in the district and state reports.
• CSV files containing all this data will be posted to the NHDoE website in mid February.
Disaggregated Content Area Results
69
70
Questions about the Disaggregated Content Area Results page?
Questions and Answers
71
• Separate report for each school and district• Provides summative information for each
grade level within the school or district• Provides summative achievement level data
for whole school or district• Provided in both teaching year and testing
year formats• CSV files containing all this data will be
posted to the NHDoE website in mid February.
School/District Summary Report
72
Identifying the Report
• The first line designates the year the test was administered
• The second line is the name of the report• The third line differentiates between
“teaching year” and “testing year”
73
74
Questions about School/District Summary Reports?
Questions and Answers
75
Student Level Data Files
• CSV files available at the district level via password protected website (User ID and passwords sent to Superintendents in January)
• All demographic information for each student that was provided by the state
• Provided in both teaching year and testing year formats• The scaled score, achievement level, and subscores earned
by each student in all content areas tested• Also contain:
• Performance on released items• Student questionnaire responses• Optional reports fields data
76
77
Questions about Student Level Data Files?
Questions and Answers
78
Supporting Materials
Materials available now• Guide to Using the
2006 NECAP Reports
• 2006 Reporting Workshop Ppt
• Grade Level Expectations
• Released Item documents
• GLE Support Materials
Materials available soon• Roster of school and district
results (teaching year and testing year formats) (csv)
• Disaggregated Data for school & districts by grade level (teaching year and testing year formats) (csv)
• Student work samples without scores or annotations (pdf)
• Release Item distracter stats (pdf)
• 2006 Technical Manual