The Tulsa Model

Copyright © Tulsa Public Schools 2011© 2012, Tulsa Public Schools

The Tulsa ModelTLE Observation and Evaluation System / Process

OVERVIEW

2

Why did Oklahoma adopt a new teacher evaluation system?

(a) We needed a better method of identifying

and describing effective teaching.

(b) We needed a better method of identifying

and describing ineffective teaching.

3

Why Teacher Effectiveness Matters

4

Facts…

1. Effective Teachers impact student performance.

2. TPS TLE… • was “home grown”, tested and revised by

Oklahoma’s best… Oklahoma Teachers.• is research-based and externally

validated.• measures what matters.

The Tulsa Model Evaluation System

Observations

Evaluations

Conferences

Feedback /

Support

Rubric-Based

5

A CAUTION…

The Tulsa Model was approved by the OSDE

and must be followed as written. Specifically,

material or substantive alteration to the

Model is not permitted.

6

7

Tulsa Model’s Focus forAccomplishment

• Create a common framework and language for evaluation.

• Provide educators with clear expectations about what is being assessed, as well as standards that should be met.

• Send messages about what is most meaningful.• Increase the consistency and objectivity in

evaluating professional performances.

SwitchingGears

8

Tulsa Model’s Focus for Accomplishment

• Provide educators with information about where they are in relation to where they need to be for success.

• Identify instructional practices that drive student achievement.

• Give educators guidance for evaluating and improving their work.

9

Statutory Requirements

The Tulsa Model has a five (5) tier rating system:• Ineffective• Needs Improvement• Effective• Highly Effective• Superior

70 S.O. § 6-101.16 (SB 2033)

10

Statutory Requirements – Qualitative Characteristics – 50% of total rating*

• Organizational and classroom management skills

• Ability to provide effective instruction

• Focus on continuous improvement and professional growth

• Interpersonal skills

• Leadership skills

*No final decision on 50% QUANTATIVE MEASURES as of yet. 70 S.O. § 6-101.16 (SB 2033)

11


• Qualitative assessment must be evidence-

based and include observable and

measureable characteristics that are

correlated to student performance.70 S.O. § 6-101.16 (SB 2033)

12


Local board policies regarding evaluations must be based on TLE by 2013-2014 school year.

Data generated from the TLE will be used to drive educational decisions that include:

• Career teacher status• Mandatory exiting of ineffective teachers

13


Comprehensive remediation plan / instructional coaching

are required for teachers rated 1 - Ineffective or 2 - Needs

Improvement.

Reduction in Force policies will use teacher effectiveness

as the primary basis for releasing teachers.

14

What We Have Learned… and a Fact

1. The Tulsa Model is a performance-based evaluation SYSTEM and has an adherence PROCESS.

2. The Tulsa Model is controlled by Statutory provisions.

3. The evaluation data from the Tulsa Model will NOT be used for employment decisions… unless the local Board adopts the relevant model as its official teacher evaluation policy for the district.

CURRENTTULSA

MODEL

TULSA MODEL

15

Tulsa Model Evaluation System Description

RubricProvides the narrative definitions of professional proficiency (effectiveness) for all 5 rankings. A matrix structure provides clarity and specificity for the 20 Indicators that impact student performance.

1 2 3 4 5Ineffective Needs

ImprovementEffective Highly Effective Superior

NOT in Evidence>>> >>> >>>

Efficacy in Effectiveness

Achieved

SwitchingGears

16

TLE Evaluation System Description

Observation+ Process The Observation scores reflect the Evaluator's intentional study of the teacher’s classroom performance plus an assessment of other factors specified in the Rubric up to a given point in time.

Observations and the Observation Form serve as the precursors (foundations) to the Evaluation.

Is guided by the rubric.• A minimum of two (2) observations are required

before every evaluation.

17

TLE Evaluation System Description (cont.)

Evaluation FormWeb-based or paper-based. Your choice.

ConferencesFollow every observation and evaluation.

Customized Feedback and SupportFocusing the most intensive supports for 1 (“Ineffective”) and 2 (“Needs Improvement”).

18

The Tulsa Model System - Structure

Teachers• Classroom Management• Instructional Effectiveness• Professional Growth &

Continuous Improvement• Interpersonal Skills• Leadership

TAKEN FROM SB 2033 EFFECTIVENESS MEASURES

19

TLE Evaluation System – Indicators, etc.

Teachers(20 Indicators within 5 Domains, and Domains have assigned weights)

• Classroom Management (6) @ 30%• Instructional Effectiveness (10) @ 50%• Professional Growth & Continuous

Improvement (2) @ 10% • Interpersonal Skills (1) @ 5%• Leadership (1) @ 5%

DEFINING AND ADDING SPECIFICITY TO SB 2033 EFFECTIVENESS MEASURES

20

The RUBRIC… Definitions

Establishing a Common Language…

“Domains” are the Performance Measures taken from SB 2033.

“Dimensions” are the TITLES of the specific components that establish the parameters of the Domain, e.g., the branches off the main truck of a tree.

“Indicators” are those observable and measurable characteristics and practices that are correlated to student performance success.

21

Teacher Rubric

THE FOUNDATION FOR DEFINING EFFECTIVENESS

SwitchingGears

22

Full Rubric for Indicator 2…

23

Rubric, Observation and Evaluation

• Teachers• Counselors• Deans• Librarians• School Development Teachers• School Nurses• School Psychologists• Speech-Language Pathologists

ALSO AVAILABLE ON THE PORTAL

24

Rubrics and the Horse’s Tale

Which direction is thehorse’s head facing?

SwitchingGears

25

Perception

Perception is a unique interpretation of the

situation, not an exact recording of it.

Perception is a very complex cognitive process

that yields a unique picture of a person, event or

even the world.

26

Perception

The process of attaining

awareness or

understanding of

information.

27

Perception

28

Perception

29

An Administrative Goal for the Evaluation Process

To move from subjectivity to higher and

higher levels of objectivity and insight.

How?

Information, information,

information, and more information.

30

Determining an Indicator’s Rating

Each Indicator often has several definitional narratives for each level of proficiency. However, Evaluators must enter only one (1) score as to each Indicator (1, 2, 3, 4, 5, NO, NA). Use the “BEST FIT.”

3

31

Observation+ Process


The Observation scores reflect the principal's intentional study of the teacher’s classroom performance plus an assessment of all inclusive factors specified in the Rubric up to a given point in time.

It is the sum total of the instructional value that a Teacher brings to his/her students.

32


13 of the 20 Indicators are commonly apparent in a classroom observation:

From the Domain of Classroom Management:

1 Teacher plans for delivery of the lesson relative to short-term and long-term objectives.

2 Teacher clearly defines expected behavior.5 Teacher acknowledges student progress and uses assessment

practices that are fair and based on identified criteria.6 Teacher optimizes the learning environment through

respectful and appropriate interactions with students, conveying high expectations for students and an enthusiasm for the curriculum.

33


13 of the 20 Indicators are commonly apparent in a classroom observation:

From the Domain of Instructional Effectiveness:7 Teacher embeds the components of literacy into all instructional content.9 Teacher uses active learning, questioning techniques and/or guided

practices to involve all students.10 Teacher teaches the objectives through a variety of methods.11 Teacher gives directions that are clearly stated and relate to the learning

objectives.12 Teacher demonstrates / models the desired skill or process.13 Teacher checks to determine if students are progressing toward stated

objectives.14 Teacher changes instruction based on the results of monitoring.15 Teacher summarizes and fits into context what has been taught.16 Effective development and use of modified assessments and curriculum

for special education students and other students experiencing difficulties in learning.

34

Teacher Observation Form

THE TOOL TO OPERATIONALIZE THE RUBRIC

2012 -2013

TLE Observation Form

35

Rubric Converted to Observation Form

Observation Form retains the Effectiveness Level of 3, but uses (-) if less than effective or (+) if exceeds effectiveness.

36

Teacher Observation Form

20 CELLSFlexible

Structure

37

Completing the Form… a CONVERSATION

On the Observation Form, Tulsa uses a coding system of “-” for Ineffective or Needs Improvement; “3” for Effective; “+” for Highly Effective or Superior; “NO” for Not Observed; “NA” for Not Applicable. Plus comments.

Districts can opt to use ALL rankings, e.g., 1, 2, 3, 4, 5, NO, or NA. Plus comments.

What advantages and/or disadvantages can you see in using one option or the other on the Observation Form?

38

Teacher’s Observation Form

Bottom of page 1

Top of page 2

1

2

Observation+ to the Evaluation

Observations and the Observation Form serve as the precursors (foundations) to the Evaluation.

Is guided by the Rubric.• A minimum of two (2)

observations are required before every evaluation.

39

REMINDER

40

Evaluation Form

THE OFFICIAL PERFORMANCE DOCUMENTATION

SwitchingGears

41

Evaluation Form

Ranges were needed to define the Rankings.

42

Evaluation Form Calculations

Domain: C’room Management Rating Domain Average

Indicator 1 narrative sentence 3 2.33

Indicator 2 narrative sentence 2





43


Domain Average Domain Weight by %

Classroom Management 2.33 30%

Instructional Effectiveness 3.20 50%

Prof Growth / Cont. Impro. 2.50 10%

Interpersonal Skills 4.00 5%

Leadership 2.00 5%

The single number that “potentially is high stakes” is the Composite Weighted Average and it is arrived at via an Excel formula. In the above hypothetical case it would be:

=((2.33*30)+(3.20*50)+(2.50*10)+(4.00*5)+(2.00*5))/100

44


The Composite Weighted Average (CWA) would be 2.89.

Next the Ranges determine the Rating Classification:

Less than 1.8 = Ineffective

Equal to or greater than 1.8, but less than 2.8 = Needs Improvement

Equal to or greater than 2.8, but less than 3.8 = Effective

Equal to or greater than 3.8, but less than 4.8 = Highly Effective

Equal to or greater than 4.8 = Superior

In the above hypothetical, the CWA is 2.89; therefore, the Teacher is EFFECTIVE. However, if any Indicator reflects a 1 or a 2, then a PDP must be written.

45

Requirements of the Evaluation Form

1. Any ranking of 1 or 2 on any component (an individual Indicator) of this Evaluation requires a Personal Development Plan (PDP), following the prescribed format and conferencing on same.

2. Any ranking of 4 or 5 on any component (an individual Indicator) requires “summary” narrative comments under Evaluator Comments on the Evaluation Form.

46

Observation / Evaluation Process Fidelity

If an Evaluator BEGINS the process, then that SAME Evaluator conducts all steps along the way including the summative Evaluation.

There is no hand-off between Observations or between the Observations and the Evaluation.

If that occurs the entire process must begin again.

47

What We Have Learned…

1. The Rubric drives all subsequent actions… Observations, Evaluations, Feedback and Support Opportunities and Conferences.

2. The Tulsa Model Process cannot be interrupted by a change in Evaluator.

48

Does not lend itself to “catch-up”

The Tulsa Model scheduling requires INTENSE and FOCUSED Time Management.

Use Outlook, an electronic scheduler, a whiteboard or a calendar… BUT, begin planning the year by the 1st week in August.

Aug Sept Oct

Nov Dec Jan

Feb Mar Apr

May

SwitchingGears

49

50

The Conference Process

Observations shall NOT be conducted on the

day immediately following any extended break

in the instructional calendar (whether scheduled or

unexpected).

Artifact File or Portfolio

On a voluntary basis, an Educator may wish to

provide additional evidence of effectiveness in the

form of a portfolio or artifact file / binder for

purposes of INFORMING the Evaluator of additional

involvements, contributions and activities in support

of improving student performance and achievement.

51

Feedback and Support-Focused

• Rubrics are detailed roadmaps for improvement for all teachers.

• Observation conferences are a status check prior to formal evaluation.

• Requires customized teacher training and responses w/re to teachers ranked in bottom two tiers.

52

SwitchingGears

53

Feedback and Support-Focused

Language from SB 2033…

“Annual evaluations that provide feedback to improve student learning and outcomes;

Comprehensive remediation plans and instructional coaching for all teachers rated as needs improvement or ineffective;”

54

Customized Feedback and Supports for Teacher Improvement

1. “Push Pins” (less formal, yet documented) approaches to remedy area(s) of ineffectiveness or needing improvement

2. Personal Development Plans (PDPs)

55

Feedback / Support – Push Pin Approach

“Push Pins” (less formal, yet documented) approaches to remedy area(s) of ineffectiveness or needing improvement.

Approaches could include:• In passing, a brief conversation in hallway,

classroom, etc.• Requested brief conversation in office.• Written note in mailbox.• Email to staff member.

56

Feedback / Support – Push Pin Approach

It is a professional “judgment” call deciding upon the use of a “Push Pin” or a PDP.

Example 1: A 60 second verbal tirade to a fellow staff member (in a closed door faculty room) regarding the loss of a planning period because of actions of that staff member. Push Pin or PDP?

Example 2: Outdated bulletin boards. Push Pin or PDP?

Example 3: Leaving a class of AP students unsupervised for 15 minutes while talking on a cell phone in the hallway. Or, special needs students. Push Pin or PDP?

57

Feedback and Support - PDPs

A Personal Development Plan…1. Cites / Summarizes the situation and/or

Observation / Evaluation details

(referencing Indicator(s))

2. Provides a List of Expectations

3. Provides an Action Plan using a SMART format (specific,

measurable, attainable, resources attached, timeline included)

4. Provides a Follow-Up Progress Reporting (within a

maximum of a two month window)

NOTE: ALL PDPs require a CONFERENCE.

58

Feedback and Support - PDPs

A Personal Development Plan…• MAY be created based upon any Observation that results in a 1 –

Ineffective or 2 – Needs Improvement determination

• MUST be created based upon an Evaluation that results in a 1 –

Ineffective or 2 – Needs Improvement ranking

• OR, could be generated as a Stand-Alone document that illustrates

behavior(s) at Levels 1 or 2

59

What We Have Learned…

1. Time management is CRITICAL to the success of the TLE Process.

2. The statutory requirement for “feedback and support” can be achieved via Push Pin and Personal Development Plan (PDP) approaches.

3. A PDP has a prescribed format, with clear, focused, descriptive narrative writing being a requirement.

60

Transparency and Understanding

Either as paper or electronic copies…

ALL staff members MUST have access to the following:• Evaluator’s Handbook• Rubric• Observation Form• Evaluation Form• Structure for the Personal Development Plan

61

For more information: www.tulsaschools.orgOffice of Teacher and Leader Effectiveness918-746-6800

Tulsa Framework: http://sde.state.ok.us; left column: Teacher and Leader Effectiveness; Scroll to Frameworks; Tulsa Framework

Talia Shaull – [email protected] Jana Burk – [email protected]

Katy Ackley – [email protected] S. Kleindienst - [email protected]

Tulsa Model via Portal: http://tulsaschools.org; left column: Teacher and Leader Effectiveness Portal; Scroll to tabs of interest

Documents

The Tulsa Model