Student Teacher Accountability Reporting System (STARS)

FY 2008 Appropriation Certification Presentation

Certification Committee

July 25, 2007

 

 

2

Prior to STARS, ADS (Accountability Data System) was the State’s educational data and reporting system

ADS was originally designed in 1995 and piloted in 1997

ADS was created to address educational data reporting requirements circa 1995

– Federal and State requirements have significantly evolved, most notably with the 2001 No Child Left Behind (NCLB) Act

– Continued maintenance and enhancements to ADS became unacceptably expensive and compromised system integrity and usability

A series of independent audits were conducted on ADS:

– 2001: LFC, Audit of Data Used in Accountability Program

– 2003: LFC, Audit of Special Education

– 2004: LFC, Audit of Bilingual Education Program

– 2003-04: ECS, Study on Federal Requirements Compliance

– 2004: CCSSO, Accountability & Indicators Report

These audit findings prompted several detailed evaluations:

– 2004: CCSSO/DSAD, Case Study of New Mexico

– 2004: LFC/LESC/OEA/PED, Joint Needs Assessment

Pre-STARS Background

 

 

3

Audit and Study efforts resulted in two clear conclusions:

– PED needed to develop a system that meets Federal and State laws while addressing the concerns and directives of the Legislature and Audits

– PED needed to address district concerns and dissatisfaction in current methods and systems

A multi-year ‘fund as we build’ plan was developed specifically to:

– Minimize development and deployment risk through a milestone-based method

– Ensure effective and efficient progress

– Maintain accountability and transparency to the Legislature and tax payer

Initial funding of $6.6M was appropriated during the 2005 Session

A national search and solicitation for a development partner was conducted

– 11 nationally recognized vendors submitted proposals

– Selection criteria based on ‘Commercial Off-the-Shelf’ (COTS) solution as well as demonstrated track record with prior projects relevant to NM’s environment

The partner was selected and the initiative began January, 2006

STARS Was Born

 

 

4

We have exceeded our originally set objectives on time and on budget:

– Built and piloted an operational prototype

– Implemented uniform student ID methods statewide

– Rolled-out the fully operational data warehouse statewide (in less than 8 months from start)

– Collected SY 2007 40th, 80th, 12/1 SPED, 120th, and EOY data via STARS

– Calculated and Reported SEG, AYP, HQT

– Submitted EDEN Special Education data through STARS

We are now actively collecting and reporting student, teacher and salary data in support of:

– Federal reporting requirements

– NEA reporting requirements

– Governor’s Office and Legislative requests

We have accomplished all of these achievements in a manner that respects district autonomy by interfacing with more than 12 different district systems

These achievements have set the foundation for continued success in our next phases of development, implementation and associated funding needs

Achievement to Date – First 24 Months

On time, on budgetachievements

On time, on budgetachievements

 

 

5

We anticipated a higher level of data-reporting proficiency in the districts

We under estimated the amount of communication, outreach and training necessary to support the changes associated with the new STARS system

We under estimated the ‘silo’ affect – both within the districts and PED

We have learned from these challenges and are adapting our next phase activities accordingly

We Have Had Our Bumps Along the Way

 

 

6

Thirteen states started to build an educational data and reporting system when we did:

– None of them have made it as far as we have

– Only four others have completed what they set out to do

We have been highly effective given some comparative benchmarks:

– New York: $500M spent and still not performing to desired functionality

– North Carolina:$700M spent and also not performing to desired functionality

– Kentucky: $125-$250M spent and also not performing to desired functionality

– Idaho: Initial funding of $54M swelled to $180M and terminated the program

New Mexico’s performance is currently the envy of the country:

– We are frequently invited to national meetings to present our story

– Of particular interest is our:

– Approach (high flexibility and decentralization)

– Pace of achievement

– Cost effectiveness of achievement

– STARS Project Manager now sits on a national task force for a term of 3 years

Relative Comparisons

 

 

7

FY 2008 AppropriationIn the 2007 legislative session House Bill 2 (2007), Section 7, page 258, line 1, item 32 appropriated $2,500,000 based on the following appropriation.

(32) PUBLIC EDUCATION DEPARTMENT:To continue implementation of the student and teacher accountability reporting system. The appropriation is from the appropriation contingency fund dedicated for the purpose of implementing and maintaining educational reforms created in Section 12 of Chapter 114 of Laws 2004. The appropriation is contingent on the public education department reengineering its business processes before proceeding and expending additional funds and on the public education department developing and enforcing reporting compliance. The public education department shall provide monthly status reports to the legislative finance committee and the office of the chief information officer.

Based on this appropriation, the funding is contingent upon:• Reengineering its business processes before proceeding and expending funds. PED has redesigned the following processes:

• Data Review - districts are now able to submit and review immediately after submission and prior to the data being available for reporting. Programs are now able to review and audit data prior to developing reports.• Reporting – districts and users are now able to run their own reports instead of having PED push the report to them. Powerful decision support tools are available to users to assist them in their reporting needs.• Program Improvements: HQT is now fully automated; SPED data is electronically created and submitted through EDEN; Error correction for AYP is down from 50,000 errors to 93; AYP results are now available to the benchmark level.

• Developing and enforcing reporting compliance For the first time districts are able to review their data for accuracy prior to reporting. Due to this district submissions took longer this year; 40th approximately 66 days; 80th day approximately 60 days; 120th approximately 60 days; EOY approximately 21 days. For SY2008 PED will enforce the submission schedule, while providing a defined submission process, monitoring program, and performance metrics to track district submission

• Providing monthly status reports to the LFC and OCIO PED has been, and will continue to, supply status reports and executive briefings to all STARS stakeholders

 

 

8

Next Phase SummaryThe appropriation will be divided into two parts:

1. License and Hosting Fees

This section of the appropriation will be used to pay those fees.

Estimated Costs: $900,000.

2. Operational Support

Also in the FY2007 Appropriation Cycle NMPED requested operational budget to transfer the four term employees that were designated in HB2 for STARS development into a STARS operational support operational position. This request was also denied. As we continue to work with the districts and PED staff to better utilize STARS, a portion of the remaining appropriation will be used to fund these term position as internal operational support.

Additional tools, reports and training is necessary to assist stakeholders in fully optimizing their use of STARS. Support will include:

•District/Charter Level Submission Support

•District/Charter Level Reports and Cube Development

•Design and development of a Proof of Concept for the Statewide Education User Interface to provide PED and districts with the tools to communicate, collaborate, and mange content and data across the state.

•Design, develop, and implement STARS security and user provisioning. As STARS is roll-out to more and more users, Access control will be a major component of the STARS security strategy. STARS security and user provisioning will develop and implement a plan to mange the access to confidential student, staff and financial data.

•Training will continue to include specific district level training related to how to better access and evaluate the existing aggregated and disaggregated data now available to the districts. Users now have direct access to the student, teacher, financial, and facility data necessary to better evaluate the student progress. Training is required to assist district and school personnel to access and interpret the available data.

Estimated Costs: $1,6,00,000

 

 

9

STARS BudgetProject Element FY2005 FY2006 FY2007 FY2008Salaries and Benefits $0 $462 $0 $395Contractual Services

Phase 0 - ProcurePM $485 $200Content $160 $150Procure $72

IV&V $96 $75Prototype Deliverables Demonstrate Operational Prototype $348

Establish Hosting Facility $100

Full Prototype Deployment $418

Prototype Acceptance $811

Phase 1 - Data warehousePM $200

Content $120IV&V $65

Phase 1 - Data Warehouse Build Deliverables Training $50

Pilot Train and Change Mngt. $100

Pilot Implementation $1,233

Unique ID $425

State-wide Rollout $1,558Phase 1 Operational Support

Load Licensure Data $450 District Submission Support $300 Report Development $475 Education and Training $350

Phase 2 State Education Interface Maintenance, License, Hosting $875 Load Financial Data $20 District Support Submission $75 Report Development $100 Education and Training $75 State EducationUser Interface $535 STARS Security and Provisioning $425

TOTALS $6,519 $2,000 $2,500

 

 

10

STARS Operational Support TimelineID Task Name Duration Start Finish

2 Initiate Operational Support Activities 0 days 7/25/07 7/25/07

3 Obtain FY2008 Certification 1 day 7/25/07 7/25/07

4 Finalize Contracts 1 day 8/1/07 8/1/07

5 NMSBA Results from Vendor 1 day 7/16/07 7/16/07

6 Calculate SY2007 AYP Designations 13 days 7/17/07 8/2/07

7 Publish 2007 AYP Designations 1 day 8/3/07 8/3/07

8 Publish 2007 State Report Card 1 day 8/6/07 8/6/07

9 Communicate Data Template Changes to Districts1 day 8/15/07 8/15/07

10 Develop Additional Reports 40 days 8/2/07 9/26/07

14 Develop STARS Eduction User Interface75 days 8/2/07 11/14/07

18 Conduct SY2008 Data Conference 3 days 10/1/07 10/3/07

19 STARS SY2008 Set-up 24 days 9/3/07 10/4/07

20 Load PED Financial/Licensure Data 5 days 10/5/07 10/11/07

21 Load 40th Day District Data 15 days 10/5/07 10/25/07

22 Determine SY2008 SEG 10 days 10/26/07 11/8/07

23 Develop STARS Security and User Provisiong Model60 days 11/15/07 2/6/08

27 Load 80th Day District Data 15 days 11/28/07 12/18/07

28 Calculate and Report SY2008 HQT 10 days 12/19/07 1/1/08

29 Load 12/1 SPED District Data 15 days 12/3/07 12/21/07

30 Complete SY2008 SPED EDEN Data 1 day 2/4/08 2/4/08

31 Load 120th Day District Data 15 days 2/27/08 3/18/08

32 End of Year District Data 15 days 5/21/08 6/10/08

33 Close of STARS Year 0 days 6/10/08 6/10/08

7/25

6/10

Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun JulQuarter 1 Quarter 2 Quarter 3 Quarter 4 Quarter 5

 

 

11

Independent Validation & Verification

IV&V Tasks to include:

Periodic review of ‘Project’ • Project Plan and Execution

• Issues Identification and Resolution

• Risk Identification and Risk Mitigation Strategies

Review of ‘Project’ Deliverables

Assessing Organizational Readiness

Assessing Executive Awareness

 

 

12

Potential / Future Enhancements

Addition of Pre-K and Higher Ed data requirements.

Unique ID for Higher Ed

Develop Unique ID for certified staff.

Continued reduction in district work load

Add data elements such as NMEPLA and Dibbles testing, Nutrition, Transportation, etc.

Provide additional analysis tools such as GIS.

Provide analysis and reporting capabilities to larger audience including School Administrators, teachers and potentially parents.

 

 

13

CCSSO/DSAC Funded Component.

SEA Application

Architecture

Provide Assistance

Enterprise Directory &

Administrator Security

Student ID

Management & Record Collec tion

Educator Certification

Management

Staff Record Collection & HQE

State Curriculum Information

Management

Grant & Program Data Collection

EOY Finance Data Collection

Safety & Discipline Data Collection

Facilities Information Data

Collection

Data Warehouse

Decision Support Tools

Assess Funding

Establish Policy

Provide Org/Staff

Provide Decision Support

Manage Core

Processes

Provide Communi -

cation

Enabling Processes

Enabling Processes

Administer

Assessments

Certify Educators

Conduct Data Driven Analysis &

Interventions

Distribute Grants/Aid &

Ensure Compliance

Set Academic

Standards & Curriculum

Co llect & Report Data

State Assessment Results

Management

Core Processes

 

 

14

Procurement Process / Plan

Project Management Plan Certified

RFP Released July 2005• COTS

• Vendor ability to implement at district / state of equal or larger than NM

• 11 Nationally Recognized Vendors submitted proposals

Selection Process• Evaluation Team Reviewed / Scored Three Finalists in late

September 2005 (Deloitte,IBM, and Pearson)

• Secretary announced Deloitte, eScholar, COGNOS, and Bency at team awarded the bid

 

 

15

STARS Proposed Governance / Advisory Team

The STARS Project team

 

 

16

Our Vision for STARS

Personalized K-12 Database that connects people, data and resources

Principals/Administrators• Create improvement plans and communicate to teachers• Assess progress against goals• Adjust strategies and communicate to teachers• Compare performance to peer schools

Teachers• Assess historical performance of incoming students• Identify peer teachers and resources to meet student needs• Collaborate with peer teachers through professional learning

communities

US Department of Education• Comply with federal reporting requirements

State of New Mexico• Comply with state reporting requirements• Meet special data needs various state entities and

policy-makers

Parents• View teacher profiles• Review curriculum• Communicate with teacher• Identify tutoring and enrichment resources

Students• View profile to track progress toward goals• Access research resources