EDFacts Shared State Solution (ES3)

An Open Source Approach to doing EDFacts

An Update

26th Annual MIS ConferenceFebruary 13, 2013

1

Original EDFacts Data Flow

2

ES3 Architecture/Data Flow

3

EDFacts Submission Data

Why a Third Party Had To Do This

• States have been more than willing to share their efforts– State Specific Codes or Assumptions Built into the Routines– Documentation is “Spotty”– Most Solutions not Designed for Easy Customization

• ESP Had To Do This– Multiple Clients– Multiple Approaches– Each a “Work for Hire” with Public Funds– Deep Understanding of the Pain

4

State Customization

• State Config (each Reporting Year)

– State Name, Postal Code, State Agency Number– Root Path to File Storage Directory– Email Notification “From:” Address

• State Characteristics– Charter Schools– Grade 13, Ungraded Allowed

• State Code Translation Table– Reporting Period– Code Set Name– State Code– EDFacts Code

5

State Customization (con’t)

• Submission File Characteristics– EDFacts Settings: Header Record File Name, Header Record

File Type, Data Record Table Name– Submission File Subdirectory– Zip File Password (optional)– Invalid Records File Name

• SSIS Configuration– Database Locations– Reporting Period– Success and Error Email Recipients (To and CC)

6

Consistent and Robust Template

7

Web Front-end Being Developed

8

Features

• Standard Microsoft Stack• Very Modular• Customization for a Specific State Compartmentalized• Lots of “Eyes”• Email Notification of Processing Results• Attachments are Zipped and Can be Password

Protected• System Logging• Validation Reports• Management and Operational Reports

9

Progress

• 70 submission packages developed off of 15 unit record staging tables

• Additional submissions from state aggregate data sources

• Success with multiple data source types:– SQL Server– Oracle– Excel– Access– Text Files

10

Lessons Learned

• Directory• Code Sets

– Inconsistent across state systems– Translation is one-way

• Core EDFacts from far fewer staging tables• SLDS may not be ready on our schedule• Getting coordinator support in heat of submission cycle

– Next few months will tell us if better off cycle

• Need to be better at collaborating with state partners– The source queries– Staging designs

• Not everyone does what they are “supposed” to– Added staging flags for “include” at each level, submission

11

Plans and Schedule

• Finish Final 17 Files Due Now through June• Back-fill Files Skipped Over in the “Heat of the Moment”• Expand and Install UI at States• Build Review and Validation Reports• Build Management Reports

12

Benefits

• Based on CEDS and Other National Standards• Shared Risk with Other States

– MO, SD, ID, TN, USVI

• Minimized and Shared Maintenance• You Own and Can Manage• Multiple Users and Development• External Coordinator Back-up

• Distributable File Creation

13

ESP EDFacts Experience

• Ground Floor EDEN/EDFacts Design Support to USED• Designed and Built the EDFacts XML Validation

Schemas and XSLT Transform system• EDEN State Site Visits (2003 and 2004)• Multi-state EDFacts Reporting

– (DE, LA, NC, NH)

• SEISS Team

Nobody Has More Experience

14

Contact Information

Tom Ogle

Missouri Dept of Elementary and Secondary Education

tom.ogle@dese.mo.gov

Judy Merriman

South Dakota Dept of Education

judy.merriman@state.sd.us

Joyce Popp,

Idaho State Dept of Education

Jpopp@sde.Idaho.gov

Steven King

ESP Solutions Group

sking@espsg.com

15