Embed Size (px)
Citation preview
S.B. 191 Implementation and Pilot Update
Educator Effectiveness
State Board of Education December 2014
2011-12 Development and testing,
principal pilot
2012-13 Full pilot
principal and teacher evaluation
2013-14 Statewide rollout
principal and teacher evaluation
2014-15 Full statewide
implementation
S.B. 10-191 Timeline of Implementation
“Practice year;” ineffective ratings did not count toward the loss of non-
probationary status
Flexibility in weighting student
growth
2
Created and refined a State Model Evaluation System with 11 educator roles (i.e., 11 different systems) – principals/assistant principals, teachers, counselors, social workers, psychologists, speech language pathologists, occupational therapists, orientation and mobility specialists, nurses, physical therapists, and audiologists.
Completed the first year of implementation Provided over 84 trainings across the state Topics: evaluation process, rubrics, measures of student learning
(MSL), inter-rater agreement (IRA), performance management system (COPMS), specialized service professionals (SSPs)
Executing our research plan and making changes to the rubric as a result of initial findings
Year In Review
3
Launched Colorado Online Performance Management System 92 local education agencies are in the
system. Others have expressed interest. Enhanced and expanded use of
Elevate Colorado to support inter-rater agreement Reviewed, approved and trained 58
training providers
Year In Review
4
5
What did we learn from our pilot districts in 2013-14?
Analysis of teacher evaluation data from the
pilot districts
Structure of Evaluation Teacher Evaluations
50% Student
Academic Growth
50% Professional
Practice
Quality Standards I-V: I. Mastery of content II. Establish learning environment III. Facilitate learning IV. Reflect on practice V. Demonstrate leadership
Evaluated using: (1) observations; and (2) at least one of the following: student perception measures, peer feedback, parent/guardian feedback, or review of lesson plans/student work samples. May include additional measures.
Quality Standard VI: VI. Responsibility for student academic growth
Evaluated using: (1) a measure of individually-attributed growth, (2) a measure of collectively-attributed growth; (3) when available, statewide summative assessments; and (4) where applicable, Colorado Growth Model data.
Principals have an evaluation system aligned with the teacher system 6
Pilot year 2 – teacher system piloted in 25 districts across the state 3,436 teachers from 166 schools in 23 districts (in pilot year 1, CDE
received ratings for 1,900 teachers in 25 districts) Findings continue to indicate that: The professional practice rubric captures multiple aspects of teaching Evaluators differentiate between different elements and between
teachers There is evidence for reliability There are differences in teachers’ ratings based on numerous school,
teacher, and student characteristics Complete report is available on the CDE website
State Model Evaluation System for Teachers
7
97% of teachers received a proficient or higher overall professional practice rating, compared with 92% in 2012-13. The largest number of teachers received a rating of accomplished. Teachers received the highest ratings on Standard 4 (Reflect on Practice). Teachers received the lowest ratings on Standard 3 (Facilitate Learning), same as in 2012-13.
8
Distributions: Standards and Overall
Notes. Percentages may not add to 100 percent because of rounding.
95% of teachers received a rating of proficient or higher on Standard 1 (Know Content). Element 1b (Literacy Development) is one of the lowest rated elements, as it was in 2012-13. Higher percentages of teachers receive below proficient ratings on Elements 1b (Literacy Development), 1c (Math Development), and 1f (Relevant Content).
9
Standard 1: Know Content Elements and Summative Rating
Notes. Percentages may not add to 100 percent because of rounding.
On Standard 2 (Establish Environment), 97% of teachers received a rating of proficient or higher. Element 2a (Predictable and Caring) is one of the highest rated elements, as it was in 2012-13. Thirteen percent of teachers received a below proficient rating on Element 2c (Unique Students).
10
Standard 2: Establish Environment Elements and Summative Rating
Notes. Percentages may not add to 100 percent because of rounding.
Similar to 2012-13, Standard 3 (Facilitate Learning) is the lowest rated standard on the rubric. Ninety-five percent of teachers received a rating of proficient or higher. Three of the lowest rated elements are in this standard: Elements 3d (Utilize Technology), 3e (High Expectations), and 3h (Use of Assessment), as was found in 2012-13. Thirteen to seventeen percent of teachers received below proficient ratings on these elements.
11
Standard 3: Facilitate Learning Elements and Summative Rating
Notes. Percentages may not add to 100 percent because of rounding.
Standard 4 (Reflect on Practice) is the highest rated standard on the rubric with 93% of teachers receiving a rating of proficient or higher and a notable 24% receiving a rating of exemplary. Elements 4a (Analyze to Improve) and 4c (Responsive to Environment) are among the highest rated elements. However, 11% of teachers received a below proficient rating on Element 4c (Responsive to Environment) and 13% received a below proficient rating on Element 4b (Professional Goals).
12
Standard 4: Reflect on Practice Elements and Summative Rating
Notes. Percentages may not add to 100 percent because of rounding.
On Standard 5 (Demonstrate Leadership), 97% of teachers receiving a rating of proficient or higher. As in 2012-13, two of the highest rated elements are in this standard: Elements 5a (School Leadership) and 5d (Ethical Standards). Element 5d is the highest rated element on the rubric. Element 5b (Professional Contributions) is the lowest rated element with 37% of teachers receiving a below proficient rating. Higher percentages of teachers receive below proficient ratings on Element 5c (Student Advocate) as well.
13
Standard 5: Demonstrate Leadership Elements and Summative Rating
Notes. Percentages may not add to 100 percent because of rounding.
Highest Rated Elements Element 5d* - Ethical Standards: High ethical standards. Element 2a* - Predictable and Caring: Predictable classroom learning environment in which
each student has a positive, nurturing relationship with caring adults and peers. Element 5a* - School Leadership: Demonstrate leadership in their schools. Element 4a - Analyze to Improve: Analyze student learning, development, and growth and apply
what they learn to improve their practice. Element 4c - Responsive to Environment: Respond to a complex dynamic environment. Highest Rated Standard Standard 4 - Reflect on Practice: Teachers reflect on their practice.
Summary of Highest Ratings
14 * Indicates similar standing to year one of the pilot, 2012-13.
Lowest Rated Elements Element 5b - Professional Contributions: Contribute knowledge and skills to educational
practices and the teaching profession. Element 3e* - High Expectations: Communicate high expectations for all students and plan
instruction that helps students develop critical-thinking and problem solving skills. Element 3h* - Use of Assessment: Use appropriate methods to assess what each student has
learned, including formal and informal assessments, and use results to plan further instruction. Element 3d* - Utilize Technology: Integrate and utilize appropriate available technology to
maximize student learning. Element 1b* - Literacy Development: Knowledge of student literacy development in reading,
writing, speaking and listening. Lowest Rated Standard Standard 3* - Facilitate Learning: Teachers plan and deliver effective instruction and create an
environment that facilitates learning for students.
Summary of Lowest Ratings
15 * Indicates similar standing to year one of the pilot, 2012-13.
There is a shift in the distribution from 2012-13 to 2013-14 with higher percentages of teachers receiving a final rating of accomplished rather than proficient.
Considering the 1,437 teachers for whom CDE received ratings in both years, 54% of teachers received the same overall professional practice rating in year two and 35% improved their performance. Eleven percent of teachers received a lower rating in year two.
16
Ratings Distribution in Subsequent School Year
There are statistically significant differences between districts in the percent of teachers in each performance category, meaning that the ratings distributions vary as a function of the district. Differences between districts are not as extreme as in 2012-13.
17
Ratings Distributions Across 22 Pilot Districts
This stacked bar chart is ordered from highest scores to lowest scores. District names and sample sizes have been removed to protect district confidentiality. Districts with fewer than 5 teachers participating in the pilot also have been removed. * = Statistically significant differences, indicating a relationship between district and overall professional practice ratings.
District *
There are statistically significant differences in teachers’ overall professional practice ratings based on school level and probationary status. ECE teachers and teachers with non-probationary status receive the highest ratings.
18
Ratings Distributions by Teacher Employment Characteristics
School Level *
Probationary Status *
Stacked bar charts are ordered from highest scores to lowest scores. * = Statistically significant differences, indicating a relationship between the teacher employment characteristic and overall professional practice ratings.
There are statistically significant differences in teachers’ overall professional practice ratings based on their years of experience and education. Teachers with more years of experience and with graduate degrees receive higher ratings.
19
Ratings Distributions by Teacher Experience and Education
Years of Experience *
Degree *
Stacked bar charts are ordered from highest scores to lowest scores. * = Statistically significant differences, indicating a relationship between the teacher characteristic and overall professional practice ratings.
There are statistically significant differences in teachers’ ratings based on Colorado’s school accountability measure, the School Performance Framework (SPF). Teachers in Turnaround schools receive the lowest overall professional practice ratings.
20
Ratings Distributions by SPF Category
SPF *
The stacked bar chart is ordered from highest scores to lowest scores. * = Statistically significant differences, indicating a relationship between SPF ratings and overall professional practice ratings.
There are statistically significant differences in teachers’ ratings based on the demographics of students in the school. Teachers in schools with higher proportions of minority students (i.e., high minority quartile) and higher proportions of students qualifying for free- and reduced-price lunch (i.e., high FRL quartile) receive lower overall professional practice ratings than teachers in schools with lower proportions of minority and FRL students.
21
Ratings Distributions by School-Level Student Demographics
Minority Quartile *
FRL Quartile *
Stacked bar charts are ordered from highest scores to lowest scores. * = Statistically significant differences, indicating a relationship between student demographics and overall professional practice ratings.
The relationship between the average level of student achievement (TCAP Reading % proficient or advanced) and teacher ratings is statistically significant. In general, as teachers’ overall professional practice ratings increase the achievement level of the students in the school does as well. The relationship between student growth (TCAP Reading MGP) and teacher ratings is not statistically significant.
22
Average Ratings by School-Level Student Achievement and Growth
Student Achievement * Student Growth
* = Statistically significant correlation between student achievement and overall professional practice ratings.
Using Data to Improve the System
23
Quantitative and qualitative data informed changes to the State Model Evaluation System at the beginning of the 2013-14 and 2014-15 school years. Changed the “not evident” category to “basic” Rubric shortened by 6 pages Professional practices language made more specific to help set
performance expectations Eliminated redundancies Most non-observable professional practices have been eliminated from
Standards 1, 2 and 3 CDE is hiring an external evaluator to review and ensure quality of
CDE’s full research plan, validation plan and analyses, and continuous improvement plan.
Continue to pilot and validate teacher, principal, and SSP rubrics
Principal and SSP pilot reports are expected in December, 2014
Continue to provide technical assistance to 160 districts using the State Model Evaluation System
Continue roll out and support of the online platform that enables districts to manage their evaluation process electronically
Next Steps in the Pilot of the State Model Evaluation System
24
Contact: Katy Anthes, Executive Director, Educator
Effectiveness Unit [email protected]
Britt Wilkenfeld, Assistant Director of Research, Educator Effectiveness Unit [email protected]
Questions
25
