The Future of Special Education Teacher and

Administrator Evaluations

2013 Summer InstitutePresentation byVickie L. CoeJordan M. BullingerLaPointe & Butler, P.C.

Part One

Educator Evaluations

MI Race to the Top Legislation Enacts legislation in 2009, effective January 4,

2010 amending the Revised School Code by adding 380.1249. – Performance evaluation system for teachers and

school administrators– Governor’s Council on Educator Effectiveness– Recommendations on evaluation processes– Effectiveness label

380.1249 Teacher ProvisionsDATE REQUIREMENT COMPONENTS

Within 60 days of 1-4-10

If not using effective-based terms, convert existing rating language

Include highly effective, effective, minimally effective and ineffective

9-1-11 LEA, ISD, and PSA boards to adopt and implement a teacher and administrator evaluation system

1. Evaluate job performance annually based on multiple rating categories, with student growth as a significant factor.

2. Student growth measured by national, state, or local assessments and other objective criteria.

3. Establish clear approaches to measuring student growth, and provide data to evaluatee.

4. Use evaluations to inform decisions re effectiveness; promotion, retention, development; tenure; removal.

380.1249 Teacher Provisions, cont.DATE REQUIREMENT COMPONENTS

Beginning 2013-14 SY

Annual year end evaluation (AYEE), unless school exempts because “HE” on three consecutive annual year end evaluations.

1. At least 25% of the evaluation shall be based on student growth and assessment data, using the student growth assessment tool required under the legislation enacted by the legislature after review of the recommendations from the Governor’s Council on Educator Effectiveness.

2. If student growth and assessment data available for at least 3 SY, the eval shall be based on the most recent 3 consecutive SY period. If not, then eval based on all student growth and assessment data available. School administrator may exempt student growth data for a particular pupil with approval of Supt.

3. Specific performance goals for improving effectiveness for the next school year, any recommended training, and IDP if/as required.

4. Classroom observation by prescribed teacher evaluation tool (unless local tool consistent with state tool), including review of lesson plan, state curriculum standard used, pupil engagement. Multiple, unless E/HE on 2 most recent AYEE.

380.1249 Teacher Provisions, contDATE REQUIREMENT COMPONENTS

Beginning 2014-15 SY

Ditto 1. Increases to 40%Ditto the remainder

Beginning 2015-16 SY

Ditto 1. Increases to 50%Ditto the remainder

380.1249 Administrative Eval Req. DATE REQUIREMENT COMPONENTS

Beginning 2013-14

Annual year end performance evaluation system for all building level and central office admins regularly involved with instructional matters, using 4 rating categories of HE, E, ME, I

1. At least 25% of the AYEE shall be based on student growth and assessment data. Aggregate student growth and assessment data for all buildings the admin works in, and for central office aggregate student growth and assessment data for the entire district.

2. The non-student growth/assessment data of evaluation to include self or designee’s proficiency in teacher evaluations; progress in meeting goals in school improvement plan; pupil attendance; student, parent and teacher feedback

3. Adopt/implement state eval tool for admins, unless local has consistent evaluation tool.

4. If ME or I, improvement plan including PD. If 3 consecutive I’s must dismiss; if 3 consecutive HE’s may do biennial year end evaluation.

Part Two

Introduction to MCEE Report

Caveats The following slides contain the MCEE’s

recommendations. The MCEE’s recommendations are NOT

the law. This presentation is not intended to be a

labor or tenure law tutorial. This presentation is not intended to provide

legal advice. The legislation and the MCEE Report

reference a waiver process.

Important Definitions Adopted by the Report

Teacher– Individual directly responsible for instruction

aimed at helping a group of students reach goals defined in a well-specified curriculum over an extended period of time such as a quarter, semester, or academic year. (Report – 6 (hereinafter R-__))

Teacher leader/master teacher– A teacher whose performance has been rated

“professional” for at least 3 years in a row and who has additional skills in supporting the development and improvement of practice. (R-6)

Important Definitions, continued Non-teacher

– Report indicates that it does not address non-teacher evaluations.

– Includes ancillary staff as an example of non-teacher.

– Leaves responsibility to define and evaluate non-teachers to the LEA. (R-11)

Important Definitions, continued Professional

– Exhibits knowledge and capabilities expected of a skillful educator.

– A teacher rated professional for three consecutive years:• May pursue opportunities for advanced

roles or leadership.• Receive bi-annual evaluation and receive

two-year improvement plan (goals). (R-8) – MCEE believes that majority of teachers will

receive a professional rating.

Important Definitions, continued Provisional

– Exhibits some professional knowledge and skill, but has specific substantial identified weaknesses that should be addressed through feedback and targeted PD.

– A teacher rated provisional or below for three consecutive years should be counseled out of his or her current role. (R-8)

Important Definitions cont. Ineffective

– Exhibits performance that has specific critical identified weaknesses.

– A teacher who receives an ineffective rating for two consecutive years should be terminated from further employment as a teacher in current LEA. (R-8)

Teacher Evaluation Overview

Part Three

Practice

“Practice” Practice

– The work that teachers do to prepare and conduct instruction and to assess, communicate about and improve students’ learning. (R-7)

Evaluation of PracticeMeasure Weight Example Recommendation

Observation At least 80%

Charlotte Danielson’s Framework for Teaching

1. Preliminary MCEE pilot study data suggests little difference among tools. MDE to pick one of tools through RFP process.

2. If final MCEE pilot study data suggests that any of these tools is less reliable or practical than the other, MDE should consider this information in tool selection.

Marzano Teacher Evaluation Model

The Thoughtful Classroom

5 Dimensions of Teaching and Learning

Evaluation of Practice, continuedMeasure Weight Examples Recommendation

“Other” No more than 20%

Teacher self-assessments “MCEE supports the use of multiple sources of evidence, but is also concerned about the burden this educator evaluation system will place on schools and LEAs, teachers and administrators.”

Professional development activities

Educator growth plans (developed by teachers or administrators)

Structured review of student work

Teacher artifacts (portfolio or evidence binder)

Feedback from students, parents, and/or other teachers using structured survey tools

Teacher self-reflection and progress on professional growth goals

MDE/LEA “Practice” Responsibilities (R-10)MDE LEA

Chooses observation tool. Chooses the MDE-selected observation tool, one of the remaining 3 MCEE piloted tools, or obtains waiver approval.

Provides sufficient base funding per teacher to support LEAs that use the MDE-selected observation tool.

Pays for extra costs if chooses non-MDE tool including costs of technical support, training, data management, data analysis and reporting.

Provides technical support for the MDE-selected observation tool.

Decides what, if any, “other” data will be used to evaluate teacher practice.

Provides training for the MDE-selected observation tool.

Decides how to aggregate data to determine a teacher’s observation rating.

Performs data analysis for MDE-selected observation tool.

Reports data to MDE.

Observer Responsibilities Who Role Training (R-10-13)Observer Must be a principal, assistant

principal, master teacher, curriculum director, superintendent or assistant superintendent

All require training. See specifics below and on next slide. Training should include:1. Training in the conceptual framework

underlying the observation protocol.2. Training in the scoring of rubrics.3. Training in evaluation steps.4. Practice in scoring the protocol under

the supervision of an expert.5. Training in how to communicate

evaluation results to teachers.

Note: MCEE recommends that all observers receive retraining every three years.

Observer Responsibilities, cont.Who Role TrainingAdministrator responsible for evaluations

Identifies who will be responsible for evaluating teachers.

Identifies the primary evaluator (see below) for a teacher by Sept 30th

Report is silent.

Administrator responsible for a teacher’s evaluation (aka primary evaluator)

Performs evaluation duties or assigns to qualified peer observer (see upcoming slide). Must, however, conduct at least one observation of a teacher.

Provides teacher with summative feedback from all observations and, in consultation with the teacher being evaluated, develops goals for improvement.

Initial: Vendor-provided training on LEA chosen observation tool. Additional training in coaching and providing feedback, including having opportunities for supervised practice.

Retraining: Triennial unless new to LEA , new to observation tool, or required by his or her evaluation.

Observer Responsibilities, cont.Who Role TrainingQualified peer observer (“QPO”)

1. Must be a teacher who has received a “professional” performance rating for at least three consecutive years AND has additional skills in supporting the development and improvement of practice.

2. During the first 3 years of implementation of the system, LEA has discretion with regard to “professional” rated teacher.

May observe teachers and provide observational data.

Provides teachers with feedback that is specific and framed in ways to provide a practical basis for development.

Perform assigned duties.

Cannot be sole source for observation data.

Initial: Vendor-provided training on chosen observation tool.

Retraining: Annually

Observation Mechanics (R-11-12)How much? OutcomeAt least 3 observations across the school year, one of which is unscheduled.

Formative feedback for each observation is specific and framed in ways that provide a practical focus for development. Provided by observer.

Summative feedback aggregates ratings from all observations and is provided by the primary evaluator. Results in the overall appraisal.

Observations should occur at least 30 days apart.

Observation does not need to be for full class period.When possible, at least one of teacher’s observations should be completed by someone who has expertise in the subject matter/grade level or the specialized responsibility of the teacher.

Part Four

Student Growth and Assessment Data

“Student Growth”

Student growth– Refers to the change in students’ knowledge

and skills across time. (R-13)

Linking Growth to Instruction Methods Description (R-13-14)Student growth and assessment tools (“SGAT”)

Tests that measure achievement and growth of individual students, e.g., same test pre and post. Less complex pre-test with more complex post-test; examining post-test controlling for baseline on pre-test.

Value-added models (“VAM”)

Statistical techniques that use data produced by growth and assessment tools to estimate the effects of teachers (and schools) on student achievement.

Student learning objectives (“SLO”)

Specific measurable academic goals set for groups of students. To yield useful information:1. Must be rigorous;2. Cover a reasonably representative portion of the

curriculum; and3. Accompanied by tools to measure progress.

Growth and Assessment Tools Assessments useful for measuring student

growth can be provided in multiple ways:– Full service assessments, which are developed,

administered, scored, and reported on centrally.– Model assessments, which are developed

centrally, but administered, scored and reported locally.

– Locally developed assessments, which are developed, administered, scored and reported on locally. (R-14)

State Obligations re: Student Growth & Assessment Tools (R-15)1. Assessments aligned to state-adopted standards in all core content areas

2. Assessments aligned to state-adopted content standards in high-volume non-core content areas where state-adopted content standards exist (e.g. arts, health and PE).

3. Provide guidelines for minimally required evidence of appropriate quality for assessments provided by third parties.

4. Provide guidelines for minimally required evidence of appropriate quality for locally developed assessments.

5. Provide guidelines for training on adequate rigor in the development and measurement of the achievement of SLOs.

6. Provide growth data back to LEAs for individual students on full-service yearly assessments.

7. Provide growth data back to LEAs aggregated for individual educators.

MCEE Recommendations re: SGATsSubject Recommendation (R-15)State No state mandate for additional assessments beyond those required by

federal and current state law.

If state revises or adopts new content standards in any content area, new assessments should not be used in evaluations for at least 2 years after adoption. New assessment items should, however, be piloted while also using the old items.

LEA If assessments meet basic quality requirements, allow LEAs to adopt:1. Third-party created assessment tools;2. Additional assessments provided by state or third parties; or 3. Locally developed assessments.

SGAT Should provide useful information about student knowledge and performance that is more specific than a scaled score. This can include:

1. Stand-level data (e.g. “number sense operations” as a subset of mathematics);

2. Data segmented for particular learning groups (e.g. student growth data on students below grade level, special populations, etc); and

3. Test score histories.

Tool Recommendations (R-16-21)Recommendation Issue(s) ResponseUse required state assessment tools

1. Not all grades & subjects tested every year

2. When tested, only once per year

1. Assessments can be given and used as a post-test for a different assessment covering the same content.

2. State should be required to provide value-added measures to LEAs.

Do not develop optional state assessment tools in specific content areas and all grade levels.

Several reasons why important but financially burdensome.

Illustration only, based on MDE’s current plans for development and/or adoption of assessments. Will require new ongoing appropriations to support adequately. (R-18-19, column 2).

Tool Recommendations continuedRecommendation Issue ResponseVendor-created assessments 1. Assessments may not

be aligned to state standards

2. Assessments not developed for all subject areas

1. LEAs should adopt policies that clearly lay out content expectations.

2. LEAs required to demonstrate adequate reliability and validity of assessments

ISD or LEA created assessments

None identified in Report 1. Should align with state standards, where applicable

2. LEA must demonstrate adequate reliability and validity of assessments

SLOs None identified in Report 1. Adequate training in the development and implementation of SLOs should be provided

General VAM Concepts (R-20) Relationship to student growth and assessment tools

– Statistical models that use data from SGATs to produce estimates of “value added” by individual educators to student learning.

How?– By controlling for factors over which educators have little to no

influence e.g., incoming achievement, race/ethnicity, socioeconomic status, gender, special education status, and English language learner status.

Measures of value added for an individual educator are based on the typical deviation of his or her students’ achievement or growth from the achievement or growth those students were expected to demonstrate given previous achievement and/or other factors over which the educator has little to no influence.

Highly controversial; considerable scientific disagreement.

General Propositions (R-20-21) MCEE’s conclusion: when comparing the use of VAM data to

the alternative of district-developed data models of teaching effects, the MCEE believes that VAMs are more reliable evidence.

VAMs should only be based on assessment data that provide information that support valid and reliable inferences about students’ growth.

In subject areas for which there are no VAM data available, educators should be evaluated based on alternate measures of student growth and achievement.

LEAs decide method for combining multiple VAM measures when teachers teach multiple content areas or sections. The state should provide VAM scores for individual educators on all state-mandated assessments and all optional assessments offered by the state.

Key Value-Added RecommendationsState provided VAM or growth data

1. At least one half of teacher’s student growth component should be based on state provided value-added estimates for teachers in core content areas (currently reading and math grades 4-8)

2. For teachers of other subject areas, LEA discretion to adopt state-provided VAM or growth data.

3. State-provided VAM or growth data to be based on 3-year averages, when possible.

4. State-provided VAM in core content may be used in a teacher evaluation even if teacher does not teach in core content area.

5. School-level VAMs may be used for individual teacher evaluations if reasonable connection or core content to actual teaching assignment, but may not comprise more than 10% of the individual teacher’s growth component. (R-21)

Complications in Calculating VAM Complications (R-21) Recommended adjustments (R-21-22)

1. Where a student has multiple educators for a specific content area (i.e. team teaching), determining method to apportion specific responsibility for student’s data.

2. Determining whether and how to control for multiple student and class, school, or LEA characteristics

3. The legislative requirement that an educator be able to appeal to have any specific student removed from his/her VAM score

1. Adjustments to state data system to capture fine-grained attendance data for individual students, proportional responsibility by student for educators, and a roster review to allow individual teachers to review data links for accuracy and appeal mechanism with approval by supervising administrators.

2. Money

3. Separate vendor to carry out VAM analysis

4. Allow for exclusion of students whose total attendance is less than 90% of the school year.

Combining Scores During the first two years of

implementation (2015-2016 and 2016-2017), LEAs will:– Produce teacher ratings based on qualitative

combinations of the categories.– Conduct a standard-setting process based on

data so that longer term will have profiles of teachers’ performance on specific aspects of instructional quality and student progress.

Example of a Qualitative Combination of Evaluation Categories

Example of Proportional Relationship of Evaluation Data

Applies to teachers in core content areas in grades for which growth data from state-mandated assessments exists.

Assumes, at a minimum, teacher-level VAMs comprise 50% of student growth section and 25% of the overall evaluation. Also assumes building-level VAMs comprise 10% of student growth section and 5% of overall evaluation. “Other” measures comprise 20% of practice section and 10% of overall evaluation.

Part Five

Administrator Evaluations

Who are Administrators? According to the law, administrators are:

– Principals– Assistant Principals– Curriculum coordinators– Superintendents– Assistant Superintendents– Career and technical education managers– Special education directors

Evaluation Process Overview Who Scope Evidence ToolsSuperintendents (evaluated by school board)

All other administrators (evaluated by supervisor)

Practice

Student growth

Required:1. Proficiency of skill in

evaluating teachers2. Progress made in the

school improvement plan

3. Attendance rates4. Student, parent and

teacher feedbackOptional: LEAs may

incorporate other pieces of evidence (e.g., professional contributions, peer input, and training/professional development

Either:1. MASA’s

School Advance Administrator Evaluation Instrument

or

2. Reeves Leadership Performance Rubric

Evaluation Chronology By October 1st, administrators and supervisors meet

– School principals and their supervisors are to establish an agreement with regard to how required “evidence” factors (see prior slide) will inform the summative decision at the end of the year.

– Other administrators will discuss how comparable information will inform the evaluation process.

By February 1st, mid-year evaluation conference– Supervisors provide verbal and written feedback, including

relevant information on teacher evaluation, student and parent feedback, attendance rates, and school improvement.

– Supervisors also provide clear information on any areas of concern that should be addressed by June.

Evaluation Chronology continued Summative evaluation meeting at the end of the

school year. Administrators provided with:– Verbal and written feedback – “Final” evaluation rating using both the selected rubric and

the information provided on teacher evaluations, school improvement progress, attendance, and student, teacher, and parent feedback.

Note: If state-run VAM estimates are component of evaluation, final evaluation rating unlikely until August since not released prior. MCEE recommends that, if decisions about personnel need to be made before state-run VAM data released, supervisors should use available evaluation data.

Administrator Ratings Professional (see slide 13)

– Same definition as for teachers with a slight variation. “Professional” administrator “exhibits the knowledge and capabilities expected of a skillful leader.

Provisional (see slide 14)– Same definition as for teachers

Ineffective (see slide 15)– Same definition as for teachers

Calculating Administrator Ratings Component Measure Weight

Practice LEA’s chosen rubric. Assessments within the rubric will be informed by the supervisor’s use of the range of evidence.

At least 50%

Student growth

Drawn from several sources.

At least half must come from building-level VAMs, where available. Currently available in reading and math in grades 4-8 (VAM scores weighted equally).

LEA determines what sources will comprise the remaining portion. Other sources of student growth data include:1. SLOs 2. Graduation rates3. Local common assessment performance4. State-provided assessments in other content areas5. Vendor-provided assessments in any content area6. Pass/fail rates7. Percentage of students on track to graduate

At least 50%

MCEE Recommendations re: Combining Scores (R-27) Weight of measures:

– During the 2013-2014 and 2014-2015, the MCEE recommends that LEAs may use student growth as a significant component, but not more than 50% of an individual administrator’s evaluation.

– To prepare for full implementation, LEAs should pilot the use of SGAT as 50% of an administrator’s evaluation.

As with teachers’ evaluations, final judgments will be determined by combining student growth scores and practice scores.

MCEE Recommendations re: MDE/LEA Responsibilities (R-25-26)MDE LEAChooses rating tool Chooses the MDE-selected rating

tool or the tool not selected by MDE.

Provides sufficient base funding per administrator to support LEAs that use the MDE-selected tool

Pays for extra expenses above the base funding supplied by the state

Provides technical support for the MDE-selected tool

Decides what, if any, “other” data will be used to evaluate administrator’s professional effectiveness and growth

Provides training for the MDE-selected tool

Develops a system to manage challenges to an administrator’s evaluation

Provides hosting and data analysis services for the MDE-selected tool

Reports data to MDE

Part Six

Waiver Process

MCEE Key Waiver Recommendations “Although the MCEE is committed to creating a

quality system of educator evaluation, the council realizes that some LEAs have used time and resources to develop their own evaluation systems.”

May request waiver for either the teacher evaluation system, the administrator evaluation system, or both.

If request a waiver and denied, may reapply the following year.

LEAs requesting a waiver must demonstrate that their processes and systems will have the same level of quality and rigor as those LEAs following the state requirements. (see, R-29 for required waiver components).

MCEE Key Waiver Recommendations,continued If an LEA submits an adapted form of a commercial

evaluation system, the LEA must demonstrate how the adaptations do not threaten the validity of the inferences based on use of the instrument.

If an LEA is using an evaluation system that does not have available documentation about its validity and reliability, the LEA must submit a plan for how it will gather relevant data on the system’s technical soundness. LEAs will receive probationary approval, contingent upon providing evidence of system’s reliability and validity within 3 years.

Part Seven

So now what?

The Devil is in the Details Legislative action

– Possible amendments– $$$$$$$$$

Based on MCEE recommendations, very clear that MDE has a lot of work to do.– Considerable time and money will need to be allocated to

develop a new infrastructure and/or revise the existing infrastructure to conform to the MCEE’s recommendations.

– For example:• Possible development of new assessment tools.• Initial and ongoing state-wide training. • Developing and implementing monitoring and quality control

measures.• Establishing new complaint/review process.

The Devil is in the Details, continued MCEE is recommending that a successor standing

committee be established. This successor standing committee would provide oversight of MDE and would act as a liaison between executive branch. Committee will also act as mediator to resolve issues as they arise.

Although recommendations answer some questions others are left unanswered.

Stay Tuned!

THANK YOU!!