Massachusetts Department of Education FFY2002 Annual Report: Response to OSEP Letter Attachment 2B – Overview of Educator Database Planning Project Note:

Archived

Massachusetts Department of EducationFFY2002 Annual Report: Response to OSEP Letter

Attachment 2B – Overview of Educator Database

Planning Project

Note: This presentation was prepared for the MASSDE Educator Database Planning Project subcommittee in January, 2005. The presentation provides a comprehensive overview of the MASSDE’s work-to-date on developing a comprehensive system for collecting educator data, including data on special education licensure and vacancy needs.

2

Archived

Table of Contents

Introduction• Project Drivers, Goals & Approach• Project Team & Subcommittee

Current State• Educator Data Flow• Methods, Systems & Data

Future State• Guiding Principles• Vision• Users• Requirements

Implementation Roadmap• Next Steps

3

Archived

Project Drivers and Goals

• No Child Left Behind (NCLB) requires “highly qualified” teachers in all classrooms by the end of 2005-2006.

• In order to meet this requirement, states must collect and report on educator data. Today, limited data and capabilities exist to help MASSDE and districts meet this requirement.

• Better data and additional analytical capabilities will help to improve recruitment, deployment, retention, certification, and professional development of MA educators. Ultimately, this work could set the foundation for linking educator quality with exceptional student performance.

• The goals of the Educator Database Planning project included the following:

Goal Timing Work Product

Understand MASSDE’s current educator data and reporting capabilities.

Weeks 1-2 Data Collection and Dissemination Inventory

Collect and document user needs for an educator database.

Weeks 3-4 User and Stakeholder Analysis

Future Vision

Requirements Analysis

Develop options (including cost, time estimates, resource needs, and implementation phases) for MASSDE to meet these needs.

Weeks 5-8 Implementation Activities and Roadmap

4

Archived

Approach - System Development Life Cycle (SDLC)

The System Development Life Cycle (SDLC) is a standard set of iterative steps used to develop, maintain and replace information systems.

Sample ActivitiesPlanning• Initiate project• Analyze current environment• Identify users• Develop vision • Collect and Analyze

Requirements • Develop strategy &

implementation plan (roadmap) • Identify project costs/benefits

Design• Define detailed system

specifications and technical requirements

• Create design document• Conduct JAD sessions• Develop and deploy

communication and change management plans

• Conduct usability testing• Data migration planning

Develop• Develop/select product• Test• Document traceability• Install• Data migration• Document• Train• User support plans

Implement & Enhance• Launch• User support• Monitor system performance• Retire legacy systems• Gather user feedback and

metrics on performance• Collect and evaluate change

requests• Select, design and

implement enhancements

Maintain, Evaluate & Enhance

ImplementationDevelopDesignPlanning & Analysis

We focused here

5

Archived

Project Team and Subcommittee

• Board of Higher Education

• Data Collection

• Commissioner’s Office

• Educator Licensure

• Educator Preparation and Quality

Steering Committee met during our 8-week project:

Subcommittee Kick-off11/17

WP #3: DOE Roadmap1/6

Project TimelineJan 6, 2004

Current data, requirements, & capabilities

User Needs and Analysis Options, recommendations, & roadmap

Nov. 15, 2004

WP #1: Draft DOE Capability Baseline11/29

WP #2: Draft Needs Analysis12/13

Subcommittee session11/22

11:30am

Subcommittee session

12/810:00am


12/110:00am

Subcommittee session12/15

11:00am


1/610:00am

• Finance

• IT Project Management

• Mass. Assoc. of School Superintendents

• Mass. Teachers Retirement Board

• Special Education

22 interviews and numerous working sessions with internal and external staff to discuss various educator data components.

Planning & Analysis

Initiate project

6

Archived

Current State

What is happening today with educator data at MASSDE?

7

Archived

Current State – Educator Data Flow

• Diversity of data “owners” supplying data

• Multiple submission timeframes (ongoing vs. periodic) and units (individual vs. groups)

• Various submission mechanisms

• Numerous systems that store educator data

• Numerous mechanisms for reporting

• Increased compliance requirements

• Increased interest in educator data

The complex flow of educator data is a function of several factors:

Planning & Analysis

Analyze Current Environment

8

Archived

Current State – Methods, Systems and Data

Identified systems with educator data and the categories of data that reside in each system.

• Multiple systems house “pieces” of data across categories

• There are opportunities to streamline/ optimize data.

Data Categories and ElementsIdentified what and how data is collected and

reported.

• Numerous collection and reporting tools. Each serve unique purpose.

• There are opportunities to streamline collection processes and improve reporting.

Data Collection and Reporting Inventory

To form our understanding of the current flow of educator data, we reviewed all MASSDE methods for data collection and dissemination and analyzed where educator data resides.

Planning & Analysis

Analyze Current Environment

9

Archived

Future State

What should the future of educator data at MASSDE be?

10

Archived

Future State – Guiding Principles

Opportunities exist to improve the quality of educator data as well as the mechanisms for data collection, storage and dissemination.

Planning & Analysis

Develop Vision

11

Archived

Future State – Educator Data Flow Vision

The future vision reflects additional suppliers of data submitting data through a streamlined process into enhanced MASSDE educator systems. The reporting side also includes increased functionality and additional consumers.

Planning & Analysis

Develop Vision

Recommendations focused here

For Future Consideration

12

Archived

Future State – Users of Educator Data

Partners Internal External Stakeholders

External individual or organization with “special” data sharing relationship with MASSDE thereby requiring appropriate access to educator data

MA DOE employee or Department

External individual or organization

Individuals or organizations with an interest in the Educator Data collected by MASSDE.

• Approved Educator Preparation Programs

• Board of Higher Education

• Schools/Districts

• Educators

• MA Teachers Retirement Board (MTRB)

• Private Schools & Collaboratives

• Professional Development Providers

• Testing companies

• Board of Education

• Career and Technical Education (CTE)

• Commissioner’s Office

• Educator Licensure (Teacher Certification)

• Educator Preparation & Quality

• Finance

• Office of Accountability and Targeted Assistance (ATA)

• Special Education Policy & Planning

• PQA

• Student Assessment

• Federal Government

• General Public

• Local City/Town Management

• MA State Legislature / Governor’s office

• Media

• Office of Educational Quality & Accountability (OEQA)

• Parents

• Policy Groups

• Researchers

MA Teachers Associations

• Mass. Teachers Association (MTA)

• Mass. Teachers Federation (MTF)

MA Educator Associations

• MA. Assoc. of School Superintendents (MASS)

• MA. Assoc. of School Committees (MASC)

• MA Secondary School Administrators Assoc. (MSSAA)

• MA Elementary School Principals Assoc. (MESPA)

• MA Assoc. of School Business Officials (MASBO)

We identified and grouped users of educator data into four categories based on their relationship to DOE.

Planning & Analysis

Identify Users

13

Archived

Future State – Requirements

This list of questions are representative of what would we like to be able to answer using educator data. Since we have limited resources, we cannot address all 30 at once.

Requirements/Questions

1 What is an educator's FTE status?

2 Is an educator "Highly Qualified"? (NCLB/IDEA)

3 Is this paraprofessional "Qualified"? (NCLB)

4 Is an educator licensed to teach in a particular subject matter?

5 What licenses do educators have at school X?

6 What licensure path/requirements are educator's using (aggregated by license type)?

7 How many educators are working on a waiver? (by school, by district, by discipline)

8 How many individuals who are granted waivers eventually get licenses?

9 How many educator FTE salaries are funded from what sources (school committee appropriations and collectively federal grants, state grants and special funds)

10 How do educators impact student performance?

11 Does student performance differ when educators are teaching outside their licensed field?

12 What is the retention rate for new and current educators?

13 Are districts successful in filling educator vacancies in high need subject areas?

14 Where are current shortage or excess areas by role, subject, location?

15 Where are the projected shortage or excess areas by role, subject, location?

16 What are the projected educator retirements in the next five years? (by field, by district, by license)

17 Where are new educators coming from? (another district? higher ed? another state, field outside of education)

18 What factors support teacher retention?

19 Why are educators staying or leaving the field?

20 How long do educator's stay in the education field?

21 How does educator retention relate to educator preparation?

22 Who is in the pipeline as a potential educator at institutes of higher education in MA? (totals, by degree, graduation date, location, program)

23 How many people complete educator preparation programs in Massachusetts each year?

24 How many Massachusetts educator prep program completers end up working in Massachusetts as educators after completing their degree?

25 How many Massachusetts educator prep program completers do not work in Massachusetts as educators after completing their degree?

26 What are our most successful educator preparation programs? Do completers of one program consistently outperform others (as measured by parent & staff satisfaction and student achievement)?

27 How are newly licensed educators meeting the educational requirements for their licenses?

28 What are most effective training and professional development programs?

29 What is the demographic makeup of our Educator workforce? (age, race/ethnicity)

30 What are the factors that contribute to student achievement?

Planning & Analysis

Collect and Analyze Requirements

14

Archived

Future State –Requirements

We analyzed requirements by gauging the perceived benefit of answering the question and the level of effort required.

Level of Effort - Criteria

Definition Rating Detail

Business Complexity for Provider

What level of effort will be required for the provider of the data to give the data to DOE to meet the requirement?

Estimated (H, M, L) based on:• How often is data required from

provider?• How easy is it to provide data to

DOE?

Business Complexity for DOE

What level of effort will be required for the DOE to define and acquire the data from a provider to meet the requirement?

Estimated (H, M, L) based on:• How many parties are providing

data?• Does DOE need to define/frame

data provided?• What level of support will DOE

need to provide?

Technical Complexity for DOE

What level of effort will be required technically to develop system to store and manipulate data to meet requirement?

Estimated (H, M, L) based on:• Technical complexity of new data

elements required• Number of data domains required• Number of data elements

required

Benefit - Criteria Definition Rating Detail

Compliance Captures whether the requirement helps DOE meet a Federal or State mandate

H=Meets a Federal/State requirementL=Does not address a Federal/State requirement

Decision Support Reflects how much this requirement supports better decision-making for DOE and users.

H=Improves decision makingM=May improve decision makingL=Does not improve decision making

User Demand Reflects the demand and urgency of requirement.

H=High level of demand by one or multiple users (urgent need)M=Requested by multiple usersL=Requested by one user

Improves Educator Quality

Captures whether the requirements helps to improve training, recruiting, retention, and/or training in an effort to improve overall educator quality.

H=Directly improves educator training, retention, recruiting, supportM=May improve educator training, retention, recruiting, supportL=Does not impact educator quality

Improves Student Achievement

Captures whether the requirement helps to improve student achievement.

H= Directly supports improved achievementM=May support improved achievementL=Does not impale student achievement

Planning & Analysis


15

ArchivedLevel of Effort -

CriteriaDefinition Rating Detail

Business Complexity for Provider

What level of effort will be required for the provider of the data to give the data to DOE to meet the requirement?

Estimated (H, M, L) based on:• How often is data required from

provider?• How easy is it to provide data to

DOE?

Business Complexity for DOE

What level of effort will be required for the DOE to define and acquire the data from a provider to meet the requirement?

Estimated (H, M, L) based on:• How many parties are providing

data?• Does DOE need to define/frame

data provided?• What level of support will DOE

need to provide?

Technical Complexity for DOE

What level of effort will be required technically to develop system to store and manipulate data to meet requirement?

Estimated (H, M, L) based on:• Technical complexity of new data

elements required• Number of data domains required• Number of data elements

required

Future State –Requirements

To analyze the level of effort, each of our questions was divided into required data elements. The level of effort criteria are all based at the data element level.

What is an educator's FTE status?

Required Data Elements Educator IDEducator RoleEducator District/SchoolEducator FTEEducator Date of HireEducator Termination Date

Planning & Analysis


For each data element:

• Do we have it?

• Which “domain” does it belong to?

16

Archived

Future State - View of Educator Data

• Assignment: Information about the position an educator holds and/or held in the past

• Development Activities: Information about the training and professional. development activities required and participated in

• Educational History: Information about the education an person obtained to become an educator

• Identity: Information about who an individual educator is

• Potential Jobs: Info about jobs available for educators. May also include info regarding future openings.

• Licensure: Information about the licenses required for educators and the licenses obtained

• Salary: Information about the funding sources for educator salaries and average salary amounts

• Students: Information about the students in Massachusetts’ educational system

As part of analyzing the data elements, we categorized each of our data elements into our eight “data domains”.

Planning & Analysis


17

Archived


Using the matrix, we analyzed the questions grouped by quadrant.

Planning & Analysis


18

Archived


After analyzing the results, we realized how the requirements were grouped would present challenges if used to prioritize system development.

3 Domains Impacted• Assignment• Educational History• Identity

7 Domains Impacted• Assignment• Development Activities • Educational History• Identity

7 Domains Impacted• Assignment• Development Activities • Educational History• Identity• Licensure• Potential Jobs• Salary

3 Domains Impacted• Assignment• Educational History• Identity

• Licensure• Potential Jobs• Students

Planning & Analysis


19

Archived


An alternative method of prioritizing the requirements is based on analyzing the required data elements, focusing on data elements that are not currently being captured or “insufficient” as currently captured.

Planning & Analysis


Captured, but insufficient to answer requirement

Educator Completed Professional Dev. ActivitiesEducator DegreeEducator District/SchoolEducator FTEEducator Prep Program Completer IDEducator Race/EthnicityEducator RoleOpen Position IDOpen Position/Role DistrictOpen Position/Role in DistrictRequired Licenses for Open Positions/Role

CapturedApplicable License PathsDistrict Requesting WaiverEducator Date of BirthEducator Ed Prep Program TypeEducator LicensesEducator Licenses Applied ForEducator Licenses Applied For DateEducator MTEL Test ResultsEducator Prep Program Educator Prep Program CompletedEducator Prep Program Completer Graduation DateEducator Prep Program CompletersEducator Prep Program ParticipantsEducator Prep SchoolEducator Waivers Federal-State Grants and Special FundLicensure Activities Completed to DateNumber of Educators Teaching Out of Field in DistrictNumber of FTE Paid From Each Funding SourceNumber of Long Term Substitutes in DistrictProgram Area (for Funding Purposes)School Appropriation TypeStudent Test Results

Not CapturedEducator Classes TaughtEducator Date of HireEducator Evaluation DataEducation Staff IDEducator Leave StatusEducator Prep Program Participant Expected DegreeEducator Prep Program Participant Expected Graduation DateEducator Previous PositionEducator Projected Retirement DateEducator Reason for Leaving or StayingEducator Termination DateEducator Years of Experience/ServiceLicenses Required for RolePotential Retention FactorsProjected Candidates from Outside of Education Prep ProgramsProjected Open Position (Role)Projected Open Position School/DistrictRequired Licenses for Projected Open Positions/RoleStudent Classes Enrolled In

Use Data Domains to Analyze These

20

Archived

Future State - Requirements

New assignment data is required to answer most of the requirements. Additionally, an identifier is required to answer the questions on an individual level.

Planning & Analysis


Identity – 26 Requirements Assignment Data – 23 RequirementsEducation History – 12 RequirementsProf. Development – 5 RequirementsPotential Job – 4 RequirementsStudent – 4 RequirementsLicense – 3 Requirements

21

Archived

Roadmap

How do we begin to move toward the future vision?

22

Archived

Roadmap - Purpose

MASSDE is beginning to map the preliminary steps to go from current state towards the future vision

Current State

Roadmap

Future Vision

Planning & Analysis

Develop Strategy and Implementation Plan

23

Archived

Roadmap – Next Steps

Throughout this process, MASSDE has used the System Development Life Cycle (SDLC) as a framework for understanding the iterative steps used to develop, maintain and replace information systems. The below graphic presents a possible framework for next steps.

Sample ActivitiesPlanning (CONTINUE)• Initiate project• Analyze current environment• Identify users• Develop vision • Collect and Analyze

Requirements • Develop strategy &

implementation plan (roadmap) • Identify project costs/benefits

Design• Define detailed system

specifications and technical requirements

• Create design document• Conduct JAD sessions• Develop and deploy

communication and change management plans

• Conduct usability testing• Data migration planning

Develop• Develop/select product• Test• Document traceability• Install• Data migration• Document• Train• User support plans

Implement & Enhance• Launch• User support• Monitor system performance• Retire legacy systems• Gather user feedback and

metrics on performance• Collect and evaluate change

requests• Select, design and

implement enhancements


ImplementationDevelopDesignPlanning & Analysis

Next Steps


ImplementationDevelopDesign

24

Archived

Next Steps

Business Process

Communication/Change Mgmt

Governance Technology Policy Formulation

• Survey schools districts to better understand their capabilities and issues.

• Develop detailed processes on how education staff IDs could be assigned.

• Develop Communication strategy and begin engaging Schools and Districts

• Estimate and obtain funding for project

• Assemble Steering Committee and Project Team

• Evaluate options for Data Collection Mechanism

• Evaluate option for Assignment Database

• Initiate policy discussion about the need for MASSDE to collect individual Assignment data

MASSDE has identified the following as possible next steps.

Documents

Massachusetts Department of Education FFY2002 Annual Report: Response to OSEP Letter Attachment 2B – Overview of Educator Database Planning Project Note: