Preparing for 2005 Mid-Cycle IV Adequate Yearly Progress (AYP)

Determinations

Massachusetts Department of Education August, 2005

Massachusetts School & District Accountability System (MSDAS) Purpose

• Designed to gauge the progress of schools and districts toward getting all students to proficiency in English Language Arts and mathematics by 2014

• Enables policymakers, parents, and the public to:– assess the effectiveness and monitor the improvement

of all public schools and districts– hold school & district leaders accountable for that

performance and improvement– identify where State intervention is needed

Why Proficiency?

• Proficiency in core academic subjects is the gateway to:

– Opportunities for higher education– Meaningful choices for employment in our

21st-century high tech economy– Full participation in community and civic life

MSDAS Basics• School & District Performance Ratings are:

– Biannual (2002, 2004, 2006…), as required by state law– Descriptive terms – Ratings for Performance and Improvement– Based on aggregate MCAS student results

• Adequate Yearly Progress determinations are: – Issued every year, as required by federal law– For students in aggregate and for eight subgroups– Based on four factors:

• participation, performance, improvement, and attendance (K-8 & districts) or graduation rate (high schools)

What is AYP?

• Stands for Adequate Yearly Progress

• Means progress toward 100% of students achieving proficiency in key subject areas by 2014

• Measures progress against specific expectations each year

Composite Performance Index

• MA measures Proficiency with a 100-point index scale: the CPI

• Points awarded based on number of students performing at each level on MCAS

• Different performance measures for students with significant cognitive disabilities who participate in MCAS- Alt

• Just arithmetic! Multiply, add, then divide.

Table 1: MCAS Proficiency Index Table 2: MCAS-Alt Index

For students taking standard MCAS tests(and MCAS-Alt for students who do not have

significant cognitive disabilities)

For students with significant cognitivedisabilities taking MCAS-Alt

(up to 1% of all assessed students in a district)

MCAS SCALED SCORE(or MCAS-Alt equivalent)

POINTSAWARDED MCAS-ALT SCORE

POINTSAWARDED

200 – 208Failing/Warning – Low(Awareness)

0 Portfolio not submitted 0

210 – 218Failing/Warning – High(Emerging/Progressing)

25 Incomplete 25

220 – 228(Needs Improvement – Low)

50 Awareness 50

230 – 238(Needs Improvement – High)

75 Emerging 75

240 – 280(Proficient/Advanced)

100 Progressing 100

Composite Performance Index

Sample CPI CalculationStudents taking Standard MCAS tests

# of Students

Performance Level Index Points Total Index Points

5 Failing - Low 0 0

5 Failing - High 25 125

20 Needs Improvement - Low 50 1000

40 Needs Improvement - High 75 3000

25 Proficient 100 2500

5 Advanced 100 500

Students with Significant Cognitive Disabilities taking MCAS-Alt

2 Emerging 75 150

1 Progressing 100 100

Total Points Awarded 7375

Total # of Students Tested 103

CPI (Total Points divided by Total Students) 71.6

State Performance Targets for ELA and Mathematics 2002-2014

60.8

68.7

53.0

76.5

84.3

92.2

75.6

70.7

80.5

85.4

90.2

95.1

100.0

40

50

60

70

80

90

100

2001 & 02 2003 & 04 2005 & 06 2007 & 08 2009 & 10 2011 & 12 2013 & 14

Co

mp

osi

te P

erf

orm

an

ce I

nd

ex

Math ELA

(39.7% P&A)

(19.5% P&A)

How is AYP Calculated?

A + (B or C) + D = AYP

A Participation Rate: 95% or greater in MCAS & MCAS-Alt

+

B Performance: 80.5 or greater Composite Performance Index in ELA

68.7 or greater Composite Performance Index in math (B or C)

C Improvement: Meet or exceed Mid-Cycle IV Improvement Target

(Specific to districts, schools, and student groups)

+

D

Performance or Improvement on Additional Indicator:

K-8 & District: 92% or higher attendance rate, or 1% improvement over 2004

High School: 70% or higher Competency Determination (CD)

(100 – Cycle III CPI) /

5 x 2/3

18 AYP Determinations per School/District

Each school and district receives AYP determinations:

• in ELA/reading and in mathematics

• for students in the aggregate and each NCLB subgroup meeting the minimum size requirements

Subgroup Size

1. 40 or more students; AND

2. Constitutes 5% or greater of total; OR

3. 200 students

Same requirements for each year of a 2-year cycle

AYP Report Sample

When Schools Do Not Make AYP for Two Consecutive Years

• Schools that do not make AYP for two consecutive years in either subject for any group are identified for improvement

– All schools identified for improvement must develop a plan for improving student performance

– Title I schools identified for improvement must also:

• offer school choice (year 1 in improvement status)• offer supplemental services (year 2 if fail to make AYP

in year 1)

Corrective Action and Restructuring

• Schools identified for improvement that fail to make AYP for 2 additional years are identified for corrective action

• Schools in corrective action that fail to make AYP in 2005 are identified for restructuring

School Choice and Supplemental Educational Services: Notice to Parents

• Title I schools identified for Improvement, Corrective Action or Restructuring must offer school choice if alternative assignment options exist

• If school choice is required, parents

must be notified, in writing, as soon as possible after districts receive notice of the designation

School Choice and Supplemental Educational Services: Notice to Parents

• Title I schools identified for Improvement (Year 2+), Corrective Action or Restructuring must offer supplemental educational services (SES) to low-income students

• If SES is required, parents must be notified

of availability as soon as possible

AYP Appeals• Objective is to render fair, reliable decisions• Appeals may be filed to address perceived erroneous

or unfair determinations where:– there are student enrollment data discrepancies– data are missing or misplaced– circumstances render the reported data invalid or unreliable– application of the standard rules would render a result

inconsistent with statutory intent, public policy, or sound professional judgment

– school or district provides evidence that it has taken specific action to improve the performance of an identified subgroup and MCAS results demonstrate significantly improved results for that subgroup in the following year

MA School Accountability Process

1. Performance Ratings & AYP Determinations2. Panel Review3. Determination of Under-performance4. Fact-Finding 5. Performance Improvement Mapping 6. Improvement Plan Reviewed by Board7. DOE Monitors Improvement Initiatives8. Year 2 Follow-up Review9. Determination of Chronic Under-performance

Link Between NCLB and Our State’s On-Site Performance Review Process

• Schools identified for restructuring or corrective action undergo state panel review

• Panel Review Questions:1. Is there a sound plan for improving student performance?2. Are the conditions in place for successful implementation of such a plan?

2005 MCAS & AYP Reporting ScheduleReport/Event Mode of Delivery Date

1. Districts receive electronic files of preliminary 2005 MCAS student item analysis data

MADOE Security Portal

8/19

2. 2005 MCAS Raw Score-Scaled Score Conversion Tables and Updated Version of TestWiz available

MADOE / MCAS Web site TestWiz Web site

8/19

3. Districts receive 2005 preliminary MCAS student item analysis data (paper reports)

Paper reports delivered 8/19

4. Districts and schools review 2005 MCAS item analysis data and report data discrepancies

Report discrepancies online via: http://www.mcasservicecenter.com

8/19-8/26

5. Districts receive and preview preliminary 2005 district and school AYP determination data and report discrepancies to DOE

MADOE Security Portal

E-mail to ata@doe.mass.edu 8/29 –9/7

6. Notice to districts and schools preliminarily identified for improvement, corrective action, and restructuring (not released publicly)

E-mail & regular mail to superintendents and principals

8/30

7. Public release of 2005 MCAS Statewide Report of Results

Press Release and MADOE / MCAS Web site

8/31

2005 MCAS & AYP Reporting ScheduleReport/Event Mode of Delivery Date

8. Public release of preliminary list of districts and schools identified for improvement, corrective action, and restructuring

MADOE / School and District Accountability (SDA) Web site

www.doe.mass.edu/sda/ayp/cycleIVmid/ 9/8

9. Districts receive and review 2005 MCAS and AYP School and District reports (updated after discrepancy reporting)

MADOE Security Portal

9/20

10. Districts receive 2005 MCAS School, District, and Parent/Guardian reports (paper reports/ online reporting/data files)

Grade 10: all reports delivered (paper) Grades 3-8: Parent/Guardian Reports and Student Labels delivered (paper)

School and district results online* Data files posted on Security Portal

9/20

11. Public release of 2005 MCAS and AYP School and District reports (excluding K-2 schools and other special cases)

Press release and MADOE Web site - School and District

Profiles http://profiles.doe.mass.edu/ 9/22

12. Districts receive and review pre-publication version of 2005 AYP determination reports for all schools and districts

MADOE Security Portal

9/29

13.

Public release of 2005 AYP reports for all schools and districts and updated list of schools and districts identified for improvement, corrective action, and restructuring

MADOE Web site – School and District Profiles http://profiles.doe.mass.edu/

10/12

Online MSDAS Information

• MA School & District Accountability System Explanatory Materialshttp://www.doe.mass.edu/sda/

• Mid-Cycle IV Materialshttp://www.doe.mass.edu/sda/ayp/cycleIVmid/

• School & District Profiles/Directoryhttp://profiles.doe.mass.edu/

• Send questions to: ATA@doe.mass.edu