Annual Professional Performance Reviews

EngageNY.org

Annual Professional Performance Reviews

Improved Evaluation and Support for Teachers to Ensure All Students Graduate

College and Career Ready

Supporting our teachers to ensure all students achieve and succeed

• Nothing impacts student learning in schools more than teachers

• All New York students deserve great teachers who can help them master the skills they need to be successful adults

• All New York teachers deserve: Clearer, higher expectations Feedback and support to meet these expectations

• The Annual Professional Performance Review (APPR): Focuses on student learning Helps teachers understand their impact on students Provides teachers meaningful opportunities for professional growth

• Grounded in research on effective evaluation systems, APPR ensures rigorous, transparent, and fair educator evaluations

When our teachers succeed, our students succeed, too.

EngageNY.org 2

EngageNY.org 3


• College and Career Readiness

• Teacher Effectiveness Matters

• New York State Annual Professional Performance Reviews (APPR) Overview

• APPR / Evaluation and Support in Our District

A college degree provides access to careers that generate significantly more income and

job stability

EngageNY.org 4

Note: Data is for people age 25 and older. Earnings are for full-time wage and salary workers. Annual earnings are based on weekly earnings times 52.Source: Bureau of Labor Statistics, Current Population Survey, “Earnings and unemployment rates by educational attainment,” http://www.bls.gov/emp/ep_chart_001.htm, accessed 6/24/13.

Unemployment Rate in 2012 Median Annual Earnings in 2012

2.1%

3.5%

4.5%

8.3%

12.4%

$90K

$68K

$41K

$34K

$24K

Master’s degree(MA, MS, etc.)

Bachelor’s degree

Associate’s degree

High school diploma

Less than high school diploma*

6.2%

Professional degree(MD, JD, etc.)

$55K

EngageNY.org

Our goal: Graduate all students college and career ready

June 2012 Graduation Rates

New York's 4-year high school graduation rate is 74% across all students. BUT… Only 35% of students are ready for college and careers,

and the gaps are disturbing.

Graduation under Current Requirements

% Graduating % Graduating

All Students 74.0 All Students 35.3

American Indian 58.5 American Indian 18.8

Asian/Pacific Islander 81.6 Asian/Pacific Islander 56.5

Black 58.1 Black 12.5

Hispanic 57.8 Hispanic 15.7

White 85.7 White 48.5

English Language Learners 34.3 English Language Learners 7.3

Students with Disabilities 44.7 Students with Disabilities 4.9

Calculated College and Career Ready*

*Students graduating with at least a score of 75 on Regents English and 80 on a Math Regents, which correlates with success in first-year college courses.

Source: SED Office of Information and Reporting Services

5

College and Career Readiness in our District

• We must increase graduation rates and college ready rates Include stats on graduation rates Include stats on college ready rates

• We must close achievement gaps to accomplish our goal to ensure all students graduate college and career ready

Include achievement gaps in grad rate

• We also are working to achieve additional academic goals Include other facts to summarize academic district goals – e.g.,

increase Regents with Advanced Designation, etc.

EngageNY.org 6

District to update

As we implement the Common Core to better prepare our students, we must adapt our

content AND practice• The Common Core State Standards

Clearly communicate what students are expected to learn at each grade level

Provide clear goals for student learning in order to help teachers ensure their students have the skills and knowledge they need

Are robust and relevant to the real world, reflecting the knowledge and skills that our young people need for success in college and careers

• The Common Core demands key instructional shifts

EngageNY.org 7

As we shift the content (what we teach), we also must shift our pedagogical and instructional practice (how we teach it).

These are intricately linked.

See Appendix for an illustration of how teacher practice rubrics align to the Common Core State Standards

EngageNY.org 8






• Develop and support teachers Help educators understand your impact on students Support teachers to achieve dramatic gains in student achievement Foster environment of continual growth for all students and educators

• Identify and retain our most effective teachers Celebrate and learn from your successes Reward teachers Retain our most effective teachers in classrooms

The Annual Professional Performance Review (APPR) is an improved evaluation and support tool that will help

all teachers – and students – perform at high levels

EngageNY.org 9

We must develop effective professionals in every classroom to dramatically increase student

achievement

EngageNY.org 10

Research shows effective teaching can be measured –Teachers make a significant difference

in student learning…The difference between an effective and ineffective teacher can

approach 11 months of learning for a student in one year

State Math Test Balanced Assessment

of Mathematics

State ELA Test

SAT9/Open-Ended Reading

Months of Learning Gained (Lost) Compared to an Average Teacher

Average Teacher

Source: Bill & Melinda Gates Foundation, Measures of Effective Teaching Project

+4.8 months

+2.9 months+1.4

months

+5 months

-5.8 months

-1.4 months -3.2

months

-2.7 months

For example, on the SAT9 assessment, •Students with the top 25% of teachers learned 5 months more of content than in an average classroom while…•Students with the bottom 25% of teachers learned nearly 6 months less of content than in an average classroom•For a total of nearly 11 months difference in learning in a single year

EngageNY.org 11

… and life outcomes over time

• Higher value-added teachers result in positive life outcomes

• Having a higher value-added teacher for even one year in grades 4-8 has substantial positive long-term impacts on a student’s life outcomes including:

Higher likelihood of attending college Lower likelihood of teen pregnancy Higher lifelong income Higher retirement savings

Source: Raj Chetty, John N. Friedman, Jonah E. Rockoff, “The Long-Term Impacts of Teachers: Teacher Value-added and Student Outcomes in Adulthood,” National Bureau of Economic Research Working Paper 17699, Dec 2011, http://obs.rc.fas.harvard.edu/chetty/value_added.html.

EngageNY.org 12

Changing educator evaluations will result in better teaching and learning

• Our previous evaluation system did not work1

Prior to changes in evaluation laws, most teacher evaluations rated 97-99% of teachers satisfactory or better

However, 81% of administrators and 57% of teachers said there was at least one poor performing tenured teacher in their school

73% of teachers said their evaluations did not identify any development areas and, of those whose did, 55% did not receive useful feedback on how to improve

59% of teachers and 63% of administrators said their districts are not doing enough to identify and retain the most effective teachers

• Research shows evaluations can improve a teacher’s impact on student learning2

Mid-career teachers who received classroom observations and full evaluations improved their “value-add” over previous years in math

Gains in value-added persist after the evaluation year1 The New Teacher Project, “The Widget Effect,” 2009.2 Eric S. Taylor and John H. Tyler, "The Effects of Evaluation on Performance: Evidence from Longitudinal Student Achievement Data of Mid-career Teachers,” The National Bureau of Economic Research Working Paper No. 16877, March 2011, http://www.nber.org/papers/w16877.pdf, accessed June 2013.

EngageNY.org 13






New APPR builds a comprehensive evaluation and support system to increase student

achievement

EngageNY.org 146 Design Standards are drawn from: The New Teacher Project, “Teacher Evaluation 2.0,” 2010.

EngageNY.org 15

Each teacher will receive an overall summative rating based on multiple measures

of teacher effectiveness*• Evaluations include teacher practice and student learning measures• Measures result in single composite teacher effectiveness score• 4 rating categories differentiate teacher effectiveness• Ratings: Highly Effective, Effective, Developing, Ineffective (HEDI)**

Student Growth on

State Tests or Comparable Measures***

State-provided growth on state assessments or

Student Learning Objectives

Student Learning

Measures of Teacher Practice

Evidence-based classroom

observations, surveys, review of

student work

Teacher Practice

Composite APPR Rating

and ScoreOverall annual evaluation with HEDI rating and score from 0-100

Locally-Selected

Measures of Student Learning

Student growth or achievement – rigorous and

comparable across classrooms

20% 20%60% 100%

40%* See Appendix for which classroom teachers will be evaluated under the new APPR** See Appendix for state guidance on HEDI ratings*** In 2014-15, the State will use a value-added measure. At that point, State-provided growth measures will count for 25% for applicable educators and local measures will count for 15%.

EngageNY.org 16






We designed our teacher evaluation and support system locally• State provided framework with specific requirements and guidance

based on research about what works*

• Together, our district and union designed/selected our: Overall evaluation and support strategy and implementation plan Teacher practice rubric (which one) Protocol to set and review Student Learning Objectives Locally-selected measures of student learning Procedures to use throughout the evaluation process including appeals

and development plans How educators earn points for the local and other subcomponents

• Educators played a critical role in designing and providing feedback on the system

Any details to support this (e.g., description of pilots, number of educators involved, quotes from educators)

EngageNY.org 17

District to update

* See Appendix for findings from national research on what a good evaluation system looks like (MET Project) as well as the state framework and local design choices

Majority (60%) of a teacher’s overall rating will be based on Measures of Teacher Practice

• Evidence-based observations of teacher practice using the X framework/rubric

If District has multiple teacher practice measures, indicate # of points associated with observations

• Teachers will be observed by principals X times per year Any differences by teacher type in number of observations? Details about announced/unannounced? Will announced have pre-conferences

sometimes, always, or never? Details about formal/informal observations? Video option if available?

• Any other observers? If so, who and how often?• Feedback will be provided after each observation

How? (written, verbal, conference, etc.) Within what time period?

• Any other measures besides observations (e.g., portfolio, survey, artifacts)? Describe each measure and points associated with it

• All lead evaluators have been trained and certified by our District

EngageNY.org 18

District to update

40% of a teacher’s overall rating will be based

on Measures of Student LearningEvery teacher will have 2 different measures of student learning:

1.State-calculated student growth or comparable measure State-calculated growth on state assessments or Student Learning Objectives (SLOs)

2.Locally-selected measures of student learning Student growth or achievement Rigorous and comparable across classrooms

EngageNY.org 19

All teachers will have at least one measure of student

growth• All students can show growth – It is our responsibility as educators

to meet students where they are & help them advance academically

• State-provided measures and SLOs focus on student growth, rather than proficiency – This allows us to:

Recognize students’ different starting points Acknowledge educators’ contributions towards student learning Identify strengths and gaps in student progress and help teachers to

better support students with different academic needs Allow teachers to achieve high ratings regardless of students’ incoming

levels of achievement

• Some teachers will receive State-provided growth measures while others will develop Student Learning Objectives*

Teachers of ELA and Math in grades 4-8 will receive state-provided growth measures in 2012-13 & 2013-14

EngageNY.org 20

Districts – an alternate headline, if true, is:

In our District, all student learning measures assess growth, not achievement

* See Appendix for an explanation of which teachers will receive State-provided growth measures and which will develop SLOs as well as an overview of SLOs

• State-provided growth measures compare a student’s performance on State tests between two years to that of similar students

• Proficiency does not tell the whole story Students enter teachers’ classrooms at different levels of proficiency or

academic achievement We must look at student growth from one year to the next

• Comparing a change in test scores is not enough either – In order to determine if a student grew a lot or a little, we have to:

Identify similar students – students who performed similarly in the past (had the same test score in the prior year)

Measure an individual student’s growth relative to these similar students This comparison helps us put an individual student’s growth into context

• Based on this information, we assign each student a Student Growth Percentile (SGP)

SGPs are on a scale from 1 to 99 SGPs tell us that an individual student scored better than X% of his/her peers

EngageNY.org 21

State-provided growth measures compare a

student’s growth to similar students

• Let’s say proficiency is 650• Within one class, students enter at a variety of achievement levels

Achievement levels are indicated by prior year test scores See dots in graphic below

• Consider one student – Tanisha She scores 600 in 2011-12 She scores 640 in 2012-13

EngageNY.org 22

2011-12 2012-13

• Is Tanisha proficient?• Do we know whether her growth

is high, low, or typical compared to other students?Tanisha’s Scale Scores

Dots represent other students

Understanding State-provided growth measures –

An example

EngageNY.org 23

• To understand Tanisha’s relative growth:1. Compare Tanisha to other students who had the same score (600) the year

before 2. Compare Tanisha’s current score (640) to similar students’ current scores

• Tanisha’s SGP is 60, meaning she performed better in the current year than 60% of similar students (students who started in the same place)

600

60% of similar students did not perform as well as Tanisha

640

Understanding State-provided growth measures –

An example

2011-12 2012-13

Multiple characteristics are included when identifying

similar studentsThis ensures that all educators have a chance to demonstrate effectiveness on this measure regardless of the composition of his or her classroom

EngageNY.org 24

Factors Included in 2012-13 State ModelAcademic

History• Up to 3 years student state exam scores, same subject• Prior year test score, different subject• Retained in grade• New to school in year other than entry year• Average prior achievement and range around average prior score in student’s

class/course (same subject)

Student With Disability (SWD)

• Whether a student is or is not a SWD• SWD spends less than 40% of time in general education setting• Percent SWD in student’s class/course

English Language

Learner (ELL)

• Whether a student is or is not an ELL • NYSESLAT scores • Percent ELL in student’s class/course

Poverty • Poverty status• Percent poverty in student’s class/course

State-provided growth measures also consider student enrollment and

attendance• Only students who meet the following criteria will count towards a

teacher’s score: Have two consecutive years of State test results Enrolled with the teacher for at least 60% of the course

• Differences in enrollment and attendance are considered Each student’s result is weighted by the proportion of time the student

was enrolled and in attendance with the teacher For example, if a student is enrolled for 90% of the course and

attended 90% of the time, that student’s result will count as 0.81 (0.9 x 0.9 = 0.81)

• Educators receive a Mean Student Growth Percentile (MGP) score Students receive an individual Student Growth Percentile (SGP) score Educators receive an MGP result (weighted average of student SGPs)

for each grade and subject with at least 16 student test scores If an educator has more than one (1) result, the results will be

averaged together based on the number of student scores in each

EngageNY.org 25

We expect 2012-13 state-provided growth score rating

distribution to be similar to 2011-12

In 2011-12, 84% of teachers received a state-provided growth score rating of effective or highly effective

EngageNY.org 26

Growth Score Rating 2011-12

Percent (%) of Teacher(Grades 4-8 ELA and Math)

Highly Effective 7%Effective 77%

Developing 10%Ineffective 6%

Growth and local measures of student learning work

together to support our academic goals

• Growth and local measures must be different from one another

• Teachers of ELA and Math in grades 4-8 will receive: A State-provided growth measure Local measures based on (district to summarize – growth on a different

assessment? % proficient on State assessments? Something else?)

• Teachers in other grades and subjects with State Assessments will: Create Student Learning Objectives (SLOs) based on State

assessments• Grade 3 ELA and math• Grade 8 Science• High School Regents courses• Students taking NYSELAT and/or NYSAA

Receive locally-selected measures based on (district to summarize – can create a table or use bullets to show specific local measures next to each type of teacher above)

EngageNY.org 27

District to update

For other grades & subjects, we made the following choices for

SLOs and locally-selected measures

EngageNY.org 28

Teacher Category

Assessment for SLO

Locally-Selected MeasuresAssessment Growth Proficiency

K-2 ELA and Math

Check appropriate box for each

6-8 Social Studies

6-7 Science

Arts

Physical Education

High School Electives

District to update •List assessment for each sub-group•Note whether locally-selected measures are focused on growth or proficiency•Use as many slides as you need – group grades & subjects together as necessary

The APPR will lead to meaningful feedback and support

EngageNY.org 29

• Evaluation and development is an ongoing cycle• APPR will support teacher development, leading to improved instructional

practice and, ultimately, improved student learning outcomes

Improved Student Learning

EngageNY.org 30

Some teachers will receive a Teacher Improvement Plan (TIP) based on their APPR

results• Teachers rated Developing or Ineffective will receive support

through a Teacher Improvement Plan (TIP)

• TIPs should assist teachers to work to their fullest potential and improve their practice

• TIPs must: Include specific standard-based goals for teacher progress Identify areas needing improvement, timeline for achieving

improvement, manner in which improvement will be assessed, differentiated activities to support improvement

Describe professional learning activities that educator must complete – should connect directly to areas needing improvement

List additional support/assistance educator will receive

Other important details about our District APPR

• Appeals will be handled in a timely and expeditious manner District to add appeals process details

• Tenured educators with two consecutive annual “Ineffective” ratings may be considered for termination through an expedited hearing process

• Other

EngageNY.org 31

District to update

We are building on lessons learned from our 2012-13 APPR process

• Successes from first year of implementation Collective bargaining completed for new system Observation and feedback process led to rich instructional

conversations Defining growth expectations for students across all courses was

challenging but worthwhile

• We are taking steps to strengthen the usefulness of the 2013-14 APPR as a tool for instructional improvement

Individual conversations about 2012-13 results will focus on development opportunities for 2013-14

[any changes to reduce time and administrative burden?] [any changes to SLO process?] [additional training opportunities for evaluators and/or teachers?] other

EngageNY.org 32

District to update – suggestions based on state-wide feedback and may not apply. Edit as appropriate.

Teachers can access many supports to learn more about our new teacher evaluation &

development system• In-Person Supports• Websites• Documents• Questions? Contact XYZ or visit ABC

EngageNY.org 33

District to update

EngageNY.org 34

Appendix

35

Our Danielson-based teacher practice rubric reinforces the Common Core

35

Common Core Content

Common Core Pedagogy

EngageNY.org 35

Districts – keep only if you use Danielson

Our NYSUT-based teacher practice rubric reinforces the Common Core

EngageNY.org 36

Districts – keep only if you use NYSUT rubric

Common Core Content

Common Core Pedagogy

EngageNY.org 37

Classroom teachers will be evaluated under new APPR

Evaluated by New APPR Not Evaluated by New APPR• Classroom teachers• Career & technical teachers• Special education teachers in team-teaching

classrooms• Speech teachers who are certified, are

teachers of record, and provide instructional services

• “Push-in” and “pull-out” teachers• Academic Intervention Services (AIS)

specialists• School librarians IF is a teacher of record

• Pre-kindergarten teachers• Pupil personnel services (e.g., school

psychologists, social workers)• Supplemental school personnel (teacher

aides, teaching assistants)• Substitute teachers• Teachers of adult, community, continuing

education• Licensed speech language pathologist who is

not certified and does not provide instructional services

• Certified speech and language therapists who provide ONLY related services

• Teachers performing instructional support services at least 40% of his/her time UNLESS s/he also serves as a teacher for at least 40% of his/her time

• Certified librarians who are NOT teacher of record

State provides standards for how districts assign points to rating categories

Standards for Rating

Categories

Student Growth on State Tests or

Comparable Measures

Results as compared to state average for similar students (or District if no

state test exists)

Locally-Selected Measures of

Student Learning

Results as compared to District- or BOCES-adopted expectations for growth or

achievement of student learning standards for

grade/subject

Other Measures of EffectivenessOverall performance and

results…

Highly Effective Well-above Well-above Exceed standards

Effective Meet Meet Meet standards

Developing Below BelowNeed improvement

in order to meet standards

Ineffective Well-below Well-below Do not meet standards

EngageNY.org 38

39 39

Use research-based observation rubrics Use multiple observations per teacher (ideally with

multiple observers) Train and calibrate all observers Include value-added measures which are more

predictive of future student learning than other researched measures

Combine observation measures, student feedback surveys, and value-added growth results on state tests to create a more reliable and better predictor of student learning than:

Any measure alone Graduate degrees Years of teaching experience

Combining “measures” is also a strong predictor of student performance on other kinds of student tests

Bill and Melinda Gates Foundation, Measures of Effective Teaching Project.

EngageNY.org 39

APPR is based on national research that tells us what

good evaluation systems look like

EngageNY.org 40

We designed our teacher and evaluation system locally within the state framework

State-Determined Locally-Determined

Overall Approach

State must approve each district’s APPR plan to ensure it is rigorous, transparent, and fair

Overall APPR strategy, measures, and implementation plan – developed through negotiations between district and union

Student Growth or Comparable

Measure

• Calculate student growth/value-added scores• Develop guidance for Student Learning

Objectives (SLOs)

• Determine SLO approach and requirements

• Support educators in developing SLOs

Locally-Selected Measure

• Identify menu of state-approved 3rd party assessments

• Select measures for this subcomponent• Ensure any locally-developed

assessments are rigorous & comparable

Other Measures of Educator

Effectiveness

• Identify menu of state-approved rubrics to assess performance based on standards

• Identify menu of state-approved surveys

• Select rubric from state-approved menu• Determine additional measures for this

subcomponent• Determine # of points for each measure

Scoring and Rating

• Scoring bands for: • Growth measure• Local measure• Composite score to determine an educator’s

rating category of Highly Effective, Effective, Developing and Ineffective (HEDI)

• Process to assign points for State Growth or Other Comparable Measures, including guidance for districts to assign points for Student Learning Objectives

• Scoring bands for Other Measures• Process to assign points in Locally-

Selected Measures and Other Measures• Must effectively differentiate educators’

performance

EngageNY.org 41

Some teachers will receive state-provided growth measures while others will develop Student Learning

Objectives

Example: A teacher with 1 section of 7th grade math and 4 sections of 7th grade science must have one SLO associated with the state-provided growth measure for math.

• Growth based on state assessments• Score counts for full growth

subcomponent

• At least one SLO must use state-provided growth measure

• Create other SLOs for other courses/sections

• Create SLOs for courses with the largest number of students

Student Learning Objectives (SLOs) must:•Cover courses with the largest number of students•Combine sections that use common assessments•Continue until at least 50% of students are covered by SLOs•Be weighted based on proportion of students covered in each SLO•Include all students in course sections subject to an SLO

Student Learning Objectives (SLOs) allow us to measure growth in non-tested subjects

EngageNY.org 42

• Student Learning Objectives (SLOs) will be used to assess the learning growth of students in classrooms (“non-tested subjects”) where there is no state assessment that can be used for state-provided growth or value-added measures

• Educators’ scores are based on the degree to which their SLOs/goals were attained

EngageNY.org 43

Helpful Resources• EngageNY Teaching and Learning Resource Website: http://engageny.org

• Approved Teacher Practice Rubrics: http://usny.nysed.gov/rttt/teachers-leaders/practicerubrics/

• Approved Survey Tools: http://usny.nysed.gov/rttt/teachers-leaders/approved-surveys/home.html

• State Growth Measures for webinars, guides to interpreting growth scores, technical report, FAQs, and sample reports: http://www.engageny.org/resource/resources-about-state-growth-measures

• Student Learning Objectives (SLOs) Guidance, Road Map, NYSED SLO Template, and Introductory Webinars: http://engageny.org/resource/student-learning-objectives/

• Approved Student Assessments for use by School Districts and BOCES in Teacher Evaluations: http://usny.nysed.gov/rttt/teachers-leaders/assessments/

• Common Core Toolkit for information about the Common Core State Standards, FAQs, myths vs. facts, and more: http://www.engageny.org/resource/common-core-toolkit

• Common Core Shifts for resources describing the six instructional shifts needed to effectively implement the Common Core: http://www.engageny.org/resource/common-core-shifts/

• Common Core Curriculum for curriculum modules and units in P-12 ELA and math that can be adopted or adapted locally: http://www.engageny.org/common-core-curriculum

• Common Core Video Series for 15 videos explaining the Common Core in depth and sharing how to implement them in your classrooms: http://www.engageny.org/resource/common-core-video-series/

• Further Questions? Email [email protected]