Reorganizing the Professional Culture in Wisconsin Schools to Eliminate Racial Achievement Gaps A Wisconsin Educator Effectiveness Research Partnership (WEERP) Research Report October 2020Curtis J. Jones & Marlo A. Reeves - University of Wisconsin Milwaukee Katharine M. Rainey - Wisconsin Department of Public Instruction

In this longitudinal study of 211 Wisconsin schools, we examine how the strength of school professional culture, which includes teacher retention, trust between educators, collaboration, school commitment, principal leadership, and Educator Effectiveness (EE) processes, impacts the achievement gaps1 between Black, Latinx, and White students.2 We followed schools from 2015-16 to 2018-19 to allow school professional culture three full years to impact achievement.The study is organized into four sections:1. First we examine the size of the achievement

gaps between Black, Latinx, and White students. 2. Next we examine characteristics of schools

associated with student achievement. These include both student demographics and aspects of professional culture.

3. Then we compare the strength of professional cultures in schools serving Black, Latinx, and White students. As part of this, we examine the number of schools serving each group that were organized to improve, with a strong professional culture.

4. Finally, we use statistical modeling to identify aspects of school professional culture uniquely associated with the improvement of achievement for Black, Latinx, and White students.

This overarching goal of this study is to answer the following question: To what extent do schools organized to improve, with a strong professional culture, improve Black and Latinx student achievement and reduce achievement gaps?

Summary Findings

uwm.edu/sreed | sreed-info@uwm.edu

• Although we found large achievement gaps between White and Latinx students, the gaps for Black students were twice as large.

• Student demographics were the strongest school characteristics associated with student achievement. This suggests the Wisconsin school system is not adequately promoting racial equity.

• In reality, the Wisconsin school system works against equity, and thus represents a racist institution. Schools serving Black and Latinx students have much weaker professional cultures than schools serving White students. Educators in schools serving White students work more effectively together and are more likely to remain in their school.

• While few schools serving Black or Latinx students were organized to improve, with a strong professional culture, when they were, racial equity improved and achievement gaps were reduced.

BackgroundBackground

This study extends the work done as part of the statewide evaluation of the Wisconsin Educator Effectiveness (EE) system (see Appendix A) and the Wisconsin Educator Effectiveness Research Partnership (WEERP) between the University of Wisconsin Milwaukee (UWM), the University of Wisconsin Madison (UW), and the Wisconsin Department of Public Instruction (DPI). Since the 2016-17 school year, the statewide evaluation of EE has leveraged a statewide survey of educators called the Wisconsin Educator Development Support and Retention (WEDSR) Survey to answer several key questions about the professional culture of Wisconsin schools and their EE processes that have been used to inform improvement efforts for the EE system. Through this work, we learned teachers in schools that focus their EE processes on teacher development instead of accountability view their principal as a stronger leader (Jones and Gilman, 2018), are more satisfied with their job (Jones and Gilman, 2018), and are more likely to stay (Jones, Cain, & Gilman, 2019). We also learned that intercultural communication challenges and bias between White, Black, and Latinx educators represent significant barriers for schools organizing their professional culture to improve student achievement. This results in low retention of Black educators generally and low retention of White educators in many urban schools. Due to the overepresentation of White educators across all Wisconsin schools, their low retention in urban schools that serve many students of color interrupts the effectiveness of the EE process for developing educators (Jones, 2019). So while we have learned schools that increase the opportunities of educators to use performance feedback realize improved student achievement overall (Jones, Gilman, & Pyatigorski, 2019), it remains unclear if Wisconsin schools organizing around a learning-focused EE process result in improvements to the achievement of Black and Latinx students.

In this study, we explore how the professional culture in Wisconsin schools, which includes the EE process, teacher retention, trust between educators, and educator collaboration, relates to changes in the achievement of White, Black, and Latinx students. We examine the extent that schools with a strong professional culture, one that is organized to improve, improve Black and Latinx student achievement and reduce achievement gaps, which are among the largest in the US (National Assessment of Educational Progress (NAEP), 2019).

This study borrows from the ideas, methods, and terminology of the seminal study Organizing Schools for Improvement: Lessons from Chicago (Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010). In their study of Chicago schools, the authors identified the "ingredients" necessary for schools to improve. Many of the ingredients identified in that study are also included in the current study of Wisconsin schools. Aspects of a school's professional culture, such as having strong principal leadership and collaborative teachers, were found in the Chicago study to be critical for school improvement. For instance, schools strong in principal leadership were four times more likely to demonstrate substantial improvement in reading achievement than were schools weak in school leadership. Schools with a strong professional culture were identified as schools organized to improve. Schools with a weak professional culture were identified as schools organized to stagnate. The current study applies these ideas to Wisconsin schools to assess the impact of schools having a strong professional culture on the achievement gaps between Black, Latinx, and White Wisconsin students.

Page 2

Statewide, there were 211 schools with both baseline (2015-16) and post (2018-19) achievement results that also participated in the WEDSR. Of these, 59 had baseline (2015-16) and post (2018-19) achievement results for Latinx students, 67 for Black students, and 102 for White students (See Appendix C for more detailed school selection criteria and the characteristics of study schools). Among these schools, 19 had enough White, Black, and Latinx students for achievement results to be available for all three groups, only three had results for just White and Black students, 22 had results for just White and Latinx students, and eight had results for just Black and Latinx students. School achievement results are not publicly reported for student groups with fewer than five students.

While the achievement in study schools as a group was much lower than the state average (0.66 standard deviations in ELA and 0.71 in math), the reason for the lower overall school performance was the very large achievement gaps between White, Black, and Latinx students (see figure below). While White student achievement was slightly better than the state average (+0.18 in ELA and +0.12 in math), both Black (-1.76 in ELA and -1.91 in math) and Latinx (-0.78 in ELA and -0.91 in math) student achievement was much lower than the state average. Appendix B includes an explanation for how we calculate achievement gaps.

How large are the Wisconsin racial achievement gaps?How large are the Wisconsin racial achievement gaps?

Overall achievement results hide large achievement gaps between Black, Overall achievement results hide large achievement gaps between Black, Latinx, and White students.Latinx, and White students.

Standardized racial achievement gaps in Wisconsin schools serving... Standardized racial achievement gaps in Wisconsin schools serving...

98% of school averages for White student achievement were higher than the typical school serving Black students.

85%-86% of school averages for White student achievement were higher than the typical school serving Latinx students.

Page 3

Consistent with the large achievement gaps between White, Black, and Latinx students previously presented, schools with the highest percentage of White students, the fewest economically disadvantaged students, and the fewest students with an IEP had the highest achievement results. That student characteristics were the strongest factors related to achievement is the foundation of educational inequity. Student demographics should not define a school’s success.

Several aspects of school professional culture were related to student achievement. These factors were derived from both the WEDSR and other public data as described in Appendix B. Teachers in higher achieving schools were more committed to their school and more likely to stay. Teachers in schools with higher achievement also viewed their principal as a stronger leader and held a more positive perspective of their EE process, viewing their evaluator as more qualified to provide feedback and the feedback provided as more accurate. Unfortunately, as is clear in the next section, far fewer Black and Latinx students attended schools where the professional culture was organized to improve than was the case in schools with White students.

What do high achieving Wisconsin schools look like?What do high achieving Wisconsin schools look like?

High achieving Wisconsin schools were largely defined by student High achieving Wisconsin schools were largely defined by student demographics.demographics.

Correlations between school characteristics and achievementCorrelations between school characteristics and achievement

Student demographic characteristics were the most associated with school achievement.

Page 4

While differences in the strength of the professional culture between schools serving White and Latinx students were substantial, differences between schools serving White and Black students were even larger. While teachers in schools serving White students were 0.21 standard deviations more committed to their school than teachers in schools serving Latinx students, the difference between schools serving White and Black students was five times greater (1.04 standard deviations). This means that 86% of schools serving White students had teachers who were more committed to their school than the typical school serving Black students.

Considering their lower school commitment, it is no surprise that fewer teachers in schools serving Black students remained in their school from 2016 to 2019 (55.5% compared to 68.3%). This difference represents nearly a full standard deviation (0.98) lower teacher retention in schools serving Black students compared to schools serving White students. Further, Black and Latinx students attended schools where fewer teachers viewed their principal as a strong leader, viewed their performance feedback as accurate, and felt their evaluator was qualified to provide them feedback.

Considering the racialized perceptions of teachers regarding their school and other educators we documented in our previous study of trust and teacher retention (Jones, 2019), it is important to emphasize that the measures of professional culture presented in this report are derived from teacher perceptions and are not directly measured. In our previous study of teacher retention and trust we found that educator perceptions of the effectiveness of principals in diverse schools are likely influenced by both racial bias and the actual effectiveness of principals. Thus, it is unclear to what extent the professional culture in these schools as measured in this study accurately reflect the quality of leadership and of the feedback process. However, it ultimately does not matter for the student or their success why teachers have less positive perceptions of their school. Regardless of the reasons, White teachers, as a group, in schools comprised mostly of Black students are less committed to their school, less satisfied with their job, hold less trust with their principal, and view the performance feedback provided to them less favorably. Considering the vast majority of teachers in Wisconsin schools serving Black or Latinx students are White (Jones, 2019), it is clear how this results in few schools organized for the success of Black students.

To what exteTo what extent do differences in the strength of professional cultures nt do differences in the strength of professional cultures between schools rebetween schools reflect opportunity gaps between racial groups?flect opportunity gaps between racial groups?

Schools serving White students have much stronger professional cultures Schools serving White students have much stronger professional cultures than those serving Black and Latinx students. than those serving Black and Latinx students.

Page 5

The 0.85 and 0.58 standardized differences suggest 80% and 72% of schools serving White students have teachers who view performance feedback as more accurate and their evaluator as more qualified than the typical school serving Black students.

The 0.51 and 0.60 standardized differences suggest 70% and 73% of schools serving White students have teachers who view their principal as a stronger leader and have greater trust with their principal than the typical school serving Black students.

The 1.10 and 1.04 standardized differences suggest 86% and 85% of schools serving White students have teachers who are more satisfied with their job and committed to their school than the typical school serving Black students.

Differences in the strength of professional cultures between schools serving Black, White, Differences in the strength of professional cultures between schools serving Black, White, and Latinx studentsand Latinx students

Page 6

The figure below presents the number of study schools that were organized to improve (greater than one standard deviation better than the state average), i.e., they were strong in the aspects of professional culture most related to student achievement (see page 4). While roughly one out of four schools serving White students were organized to improve, fewer than one out of ten schools serving Black students and one out of five serving Latinx students were organized to improve.

How many schools serving Black and Latinx students were How many schools serving Black and Latinx students were organized to organized to improveimprove, with a strong professional culture?, with a strong professional culture?

Note: Aspects of school professional culture, organized to improve, were at least one standard devi-ation higher than the state average.

Few Wisconsin schools serving Black or Latinx students were Few Wisconsin schools serving Black or Latinx students were organized to organized to improve, improve, with a strong professional culture.with a strong professional culture.

Page 7

The professional culture of many schools serving Black students was characterized by weak teacher commitment to their school, teachers viewing their principal as less effective, and teachers viewing their EE process less favorably. We call the professional culture in these schools organized to stagnate. Very few schools organized to stagnate are likely to improve. The figure below presents the number of study schools organized to stagnate regarding aspects of the professional community (greater than one standard deviation lower than the state average). Schools serving Black students were organized to stagnate in between one-fourth and one-half of schools, depending on the aspect of professional culture. Also, roughly one out of every six schools serving Latinx students were organized to stagnate. Conversely, fewer than one out of ten schools serving White students were organized to stagnate.

How many schools serving Black and Latinx students were How many schools serving Black and Latinx students were organized to organized to stagnatestagnate, with a weak professional culture?, with a weak professional culture?

Note: Aspects of school professional culture, organized to stagnate, were at least one standard devi-ation lower than the state average.

Many Wisconsin schools serving Black or Latinx students were Many Wisconsin schools serving Black or Latinx students were organized organized to stagnate, to stagnate, with a weak professional culture.with a weak professional culture.

Page 8

We used statistical modeling to estimate the relationships between aspects of school professional culture and changes in school achievement from 2016 and 2019. These analyses adjust the relationships between aspects of school professional culture and school achievement growth by a number of factors as outlined in Appendix E. All aspects of school professional culture included in these analyses were standardized to improve interpretability.

The results suggest several aspects of a school's professional culture related to their overall achievement growth. Not looking within specific racial groups, the figure on page 10 presents the adjusted relationships between aspects of a school's professional culture and changes in achievement. Each bar represents the standardized relationship of one aspect of professional culture with overall school achievement improvement, i.e., how much does a school's achievement improve with every one standard deviation improvement in that aspect of school professional culture.

The most important aspects of school professional culture centered on the teacher role; student achievement improved in schools with highly committed teachers, who collaborated frequently, and who were highly satisfied with their job. Principal effectiveness was also important for improving achievement. Achievement improved in schools with teachers who viewed their principal as an effective leader and who trusted them. Finally, aspects of the EE process also related to changes in student achievement; achievement improved in schools where teachers viewed performance feedback as useful, viewed their evaluator as qualified to provide them feedback, and were provided more opportunities to use feedback. Appendix F provides more detailed model results.

It is notable that teacher collaboration and the opportunities provided to teachers to use feedback were associated with the improvement of school achievement but not school achievement attainment (page 4). These results are consistent with our previous studies of the EE process where we found schools that increased the number of opportunities for teachers to use feedback improved their student achievement results (Jones & Gilman, 2019).

Which aspects of school professional culture related to the improvement of Which aspects of school professional culture related to the improvement of achievement across student groups?achievement across student groups?

Overall student achievement improved in schools with teachers committed Overall student achievement improved in schools with teachers committed to their school, who viewed their principal as a strong leader, and viewed to their school, who viewed their principal as a strong leader, and viewed their EE process favorably.their EE process favorably.

Page 9

* Aspect of school professional culture is a statistically significant predictor of changes in ELA and/or math student achievement.

Summary model results relating aspects of school professional culture with the Summary model results relating aspects of school professional culture with the improvement of student achievementimprovement of student achievement

The strongest factors related to a school's improvement measures aspects of teacher experiences.

Schools where teachers felt their performance feedback was useful, accurate, and provided by a qualified evaluator improved.

Schools where teachers felt their principal was a strong leader and trusted their principal improved.

Page 10

Statistical modeling was used to measure the relationships between aspects of school professional culture and changes in Black student achievement from 2016 and 2019. The aspects of professional culture that predicted changes in Black student achievement were similar to those that predicted overall school improvement. Black student achievement improved in schools with highly committed teachers who collaborated frequently. Black student achievement also improved in schools with strong leadership. The difference between what mattered for Black student achievement and overall achievement was one of degree; a strong professional culture was much more important for the improvement of Black student achievement than overall school achievement. For example, the adjusted relationship of teacher collaboration with Black student math achievement (0.25) was over twice as large as it was for overall student math achievement (0.11). Appendix F provides more detailed model results.

Which aspects of school professional culture related to the improvement of Which aspects of school professional culture related to the improvement of Black student achievement?Black student achievement?

* Aspect of school professional culture of is a statistically significant predictor of changes in ELA and/or math student achievement.

The strength of a school's professional culture was much more important The strength of a school's professional culture was much more important for the improvement of Black student achievement than for the overall for the improvement of Black student achievement than for the overall improvement of achievement in schools.improvement of achievement in schools.

Summary model results relating aspects of school professional culture with the Summary model results relating aspects of school professional culture with the improvement of Black student achievementimprovement of Black student achievement

Page 11

The only aspects of school professional culture statistically related to Black ELA achievement improvement were teacher school commitment and teacher collaboration. However, as was mentioned before, very few schools serving Black students were organized to improve regarding any of the aspects of school professional culture measured in this study. Only four schools serving Black students were organized to improve regarding teacher commitment, with teachers highly committed to their school, and only three were organized to improve regarding teacher collaboration.

Because the professional culture in too few schools was organized to improve, we expanded our analysis to additional schools. In the 40 schools at least partially organized to improve (better than one standard deviation below the state average, or not organized to stagnate) regarding teacher school commitment, the Black ELA achievement gap decreased by 22%, moving from -1.62 to -1.27 standard deviations below the state average. In the 65 schools at least partially organized to improve (better than one standard deviation below the state average, or not organized to stagnate) regarding teacher collaboration, the Black ELA achievement gap decreased by 11%, moving from -1.74 to -1.55 standard deviations below the state average.

The change in the Black ELA achievement gap in schools not The change in the Black ELA achievement gap in schools not organized to stagnate organized to stagnate because of...because of...

The Black ELA achievement gap decreased by 11% to 22% in schools The Black ELA achievement gap decreased by 11% to 22% in schools with teachers who were at least somewhat committed to their school and with teachers who were at least somewhat committed to their school and engaging in a modest amount of collaboration.engaging in a modest amount of collaboration.

State averageState average

The Black ELA achievement gap in schools where the professional culture was at least partially organized to improve, with moderate teacher school commitment, was 1.62 standard deviations below the state average at the start of the study.

The Black ELA achievement gap in schools where the professional culture was at least partially organized to improve, with moderate teacher school commitment, improved to 1.27 standard deviations at the end of the study.

Page 12

The figure below presents changes in the Black ELA achievement gap for schools organized to stagnate, with a weak professional culture. Schools organized to stagnate with teachers less committed to their school and teachers who collaborate infrequently did not reduce the Black ELA achievement gap.

The change in the Black ELA achievement gap in schools The change in the Black ELA achievement gap in schools organized to stagnate organized to stagnate because of weak...because of weak...

Schools Schools organized to stagnateorganized to stagnate, with a weak professional culture, failed to , with a weak professional culture, failed to reduce the Black ELA reduce the Black ELA achievement gapachievement gap..

State averageState average

Page 13

More aspects of school professional culture were associated with changes in Black math achievement gaps. Again though, because the professional culture in too few schools was organized to improve, we expanded our analysis to include schools at least partially organized to improve, i.e., not organized to stagnate. Schools at least partially organized to improve regarding principal leadership, principal trust, teacher collaboration, and school commitment realized modest reductions in the Black math achievement gap of 8-14%. The achievement gap, although reduced, remained large.

The change in the Black math achievement gap in schools not The change in the Black math achievement gap in schools not organized to organized to stagnate stagnate because of...because of...

The Black math achievement gap decreased by 8% to 14% in schools with The Black math achievement gap decreased by 8% to 14% in schools with teachers at least somewhat committed to their school and who viewed their teachers at least somewhat committed to their school and who viewed their principal as at least somewhat effective.principal as at least somewhat effective.

State averageState average The Black math achievement gap for schools at least partially organized to improve, with at least modest teacher school commitment, improved from 1.63 to 1.40 standard deviations below the state average.

Page 14

The figure below presents changes in the Black math achievement gap for schools organized to stagnate, with a weak professional culture. Schools organized to stagnate with weak school leadership, teachers less committed to their school, and teachers who collaborate infrequently did not reduce the Black math achievement gap.

The change in the Black math achievement gap in schools The change in the Black math achievement gap in schools organized to stagnate organized to stagnate because of weak...because of weak...

Schools Schools organized to stagnateorganized to stagnate, with a weak professional culture, failed to , with a weak professional culture, failed to reduce the Black math reduce the Black math achievement gapachievement gap..

State averageState average

Page 15

Statistical modeling was used to measure the relationships between aspects of school professional culture and changes in Latinx student achievement from 2016 and 2019. Nearly all aspects of school professional culture included in this study were associated with the improvement of Latinx student achievement. Latinx math and ELA student achievement improved considerably in schools with teachers who were highly committed to their school and who collaborated frequently. Further, Latinx achievement improved in schools with teachers who viewed their principal as a strong leader and viewed their performance feedback as provided by a qualified evaluator, as accurate, and useful. Latinx achievement also improved when schools provided teachers more opportunities to use feedback to improve. Appendix F provides more detailed model results.

Which aspects of school professional culture related to the improvement of Which aspects of school professional culture related to the improvement of Latinx student achievement?Latinx student achievement?

The strength of a school's professional culture was much more important The strength of a school's professional culture was much more important for the improvement of Latinx student achievement than the improvement for the improvement of Latinx student achievement than the improvement for Black students and for schools overall.for Black students and for schools overall.

Summary model results relating aspects of school professional culture with the Summary model results relating aspects of school professional culture with the improvement of Latinx student achievementimprovement of Latinx student achievement

* Aspect of school professional culture is a statistically significant predictor of changes in ELA and/or math student achievement.

Page 16

The figure below presents the reduction in the Latinx ELA achievement gap realized in schools organized to improve, with a strong professional culture. Schools organized to improve with an effective EE process, strong school leadership, and teachers highly committed to their school realized sizable reductions in the Latinx ELA achievement gap. Schools with teachers highly committed to their school nearly eliminated the Latinx ELA achievement gap; at the end of the study the ELA achievement of Latinx students in these schools was nearly equal to the state average. Schools where teachers perceived performance feedback as highly accurate reduced the Latinx ELA achievement gap by 57%, moving from 0.79 to 0.34 standard deviations below the state average. Schools that provided teachers more opportunities to use feedback reduced the Latinx ELA achievement gap by 43%, moving from 0.76 to 0.43 standard deviations below the state average. Schools where teachers viewed their principal as a strong leader reduced the Latinx ELA achievement gap by 55%, moving from 0.64 to 0.29 standard deviations below the state average.

Note: For the sake of interpretability, this figure only presents a sample of aspects of school professional culture related to Latinx achievement gap reduction.

The change in the Latinx ELA achievement gap in schools The change in the Latinx ELA achievement gap in schools organized to improve organized to improve because of strong...because of strong...

Schools Schools organized to improve, organized to improve, with a strong professional culture, reduced with a strong professional culture, reduced the Latinx ELA achievement gap by as much as 89%.the Latinx ELA achievement gap by as much as 89%.

State averageState average

The Latinx ELA achievement gap in schools where the professional culture was organized to improve, with strong teacher school commitment, improved from 0.36 to 0.04 standard deviations below the state average.

Page 17

The figure below presents changes in the Latinx ELA achievement gap for schools organized to stagnate, with a weak professional culture. Schools organized to stagnate with an ineffective EE process, weak school leadership, and teachers less committed to their school did not reduce the Latinx ELA achievement gap.

Note: For the sake of interpretability, this figure only presents a sample of aspects of school professional culture related to Latinx achievement gap reduction.

The change in the Latinx ELA achievement gap in schools The change in the Latinx ELA achievement gap in schools organized to stagnate organized to stagnate because of weak...because of weak...

Schools Schools organized to stagnate, organized to stagnate, with a weak professional culture, failed to with a weak professional culture, failed to reduce the Latinx ELA reduce the Latinx ELA achievement gapachievement gap..

State averageState average

The Latinx ELA achievement gap did not improve in schools with a professional culture organized to stagnate, with weak teacher school commitment, feedback accuracy, and principal leadership.

Page 18

The figure below presents the reduction in the Latinx math achievement gap realized in schools organized to improve with a strong professional culture. Schools organized with an effective EE process, strong school leadership, and teachers committed to their school realized sizable reductions in the Latinx math achievement gap as well. Schools with teachers highly commitment to their school reduced the Latinx math achievement gap by 55%, moving from 0.62 to 0.28 standard deviations below the state average. Schools with an EE process that teachers perceived as providing highly accurate feedback reduced the Latinx math achievement gap by 52%, moving from 1.06 to 0.51 standard deviations below the state average. Schools providing teachers more opportunities to use feedback to improve reduced the Latinx math achievement gap by 48%, moving from 0.93 to 0.48 standard deviations below the state average. Schools where teachers viewed their principal as a strong leader reduced the Latinx math achievement gap by 54%, moving from 1.00 to 0.46 standard deviation below the state average.

Note: For the sake of interpretability, this figure only presents a sample of aspects of school professional culture related to Latinx achievement gap reduction.

The change in the Latinx math achievement gap in schools The change in the Latinx math achievement gap in schools organized to improve organized to improve because of strong...because of strong...

Schools Schools organized to improve, organized to improve, with a strong professional culture, reduced with a strong professional culture, reduced the Latinx math achievement gap by as much as 55%.the Latinx math achievement gap by as much as 55%.

State averageState average

The Latinx math achievement gap in schools where the professional culture was organized to improve, with strong teacher school commitment, improved from 0.62 to 0.28 standard deviations below the state average.

Page 19

The figure below presents changes in the Latinx math achievement gap for schools organized to stagnate, with a weak professional culture. Schools organized to stagnate with an ineffective EE process, weak school leadership, and teachers less committed to their school did not reduce the Latinx math achievement gap.

Note: For the sake of interpretability, this figure only presents a sample of aspects of school professional culture related to Latinx achievement gap reduction.

The change in the Latinx math achievement gap in schools The change in the Latinx math achievement gap in schools organized to stagnate organized to stagnate because of weak...because of weak...

Schools Schools organized to stagnate, organized to stagnate, with a weak professional culture, failed to with a weak professional culture, failed to reduce the Latinx math reduce the Latinx math achievement gapachievement gap..

State averageState average

The Latinx math achievement gap did not improve in schools with a professional culture organized to stagnate, with weak teacher school commitment, feedback accuracy, and principal leadership.

Page 20

That a school's professional culture did not relate to the improvement of White student achievement does not suggest it does not matter. Considering many of the correlations between aspects of school professional culture and achievement were statistically significant, their relationships were likely already “baked in” our models by including baseline achievement. Further, White students were already doing well as a group at the beginning of the study so there was limited impact the school professional culture could have on achievement. Appendix F provides more detailed model results.

More importantly, these findings suggest that strengthening a school's professional culture, with less teacher turnover, greater teacher commitment to their school, teachers working together more effectively, teachers viewing their principal as a more effective leader, and teachers viewing the EE process more favorably, represents a mechanism for achievement gap reduction rather than just overall school improvement.

Which aspects of school professional culture related to the improvement of Which aspects of school professional culture related to the improvement of White student achievement?White student achievement?

No aspects of school professional culture were associated with changes in No aspects of school professional culture were associated with changes in the ELA or math achievement of White studentsthe ELA or math achievement of White students

Summary model results relating aspects of school professional culture with the Summary model results relating aspects of school professional culture with the improvement of White student achievementimprovement of White student achievement

Page 21

Summary findingsSummary findings

In this longitudinal study of Wisconsin schools, we explored the impact school professional culture has on Black, Latinx, and White student achievement. We measured school-level achievement changes for each racial group from the 2015-2016 to the 2018-19 school years, thus allowing school professional culture three full years to impact achievement.

We first defined the problem and found large achievement gaps between Black, Latinx, and White students attending Wisconsin schools. Consistent with this, we found student demographic characteristics were the strongest factors associated with overall school achievement. We also found schools serving Black and Latinx students were much less likely to be organized to improve, with a strong professional culture, than schools serving White students. Teachers in schools serving Black and Latinx students were, as a group, less committed to their school, viewed their EE process less favorably, viewed their principal as a less effective leader, and were less likely to stay. This was especially true regarding schools serving Black students, with fewer than 10% organized to improve, with a strong professional culture, and as many as 50% organized to stagnate, with a weak professional culture. In these ways, the Wisconsin education system works against equity and thus represents a racist institution.

A key finding of this study is that when the professional cultures in schools serving Black or Latinx students were organized to improve achievement gaps were reduced. Put another way, as schools became more equitable achievement gaps dissipated. This was especially true for Latinx students, for which the achievement gap was completely eliminated under certain conditions. While achievement gaps were reduced for Black students in schools with a stronger professional culture, too few schools serving Black students were organized to improve, with a strong professional culture, to allow us to reliably measure its impact. Finally, we found that while professional culture was related to the achievement of White students, it was not associated with the improvement of achievement for White students. That schools with a stronger professional culture improved the achievement of Black and Latinx students but not White students suggests that organizing the professional culture within schools represents a strategy for improving educational equity between racial groups and reducing achievement gaps.

The results of this study have significant implications for addressing the large racial achievement gaps in Wisconsin. In this study, we explored whether our previous finding that an effective EE process positively impacts student achievement (Jones & Gilman, 2019) was true for student subgroups as well, i.e., can the EE process reduce racial achievement gaps. The results suggest that the overall impact of EE on school achievement, as measured in this study, was due to its impact on Black and Latinx students. Schools realized sizable reductions in achievement gaps when teachers were committed to their school, viewed their principal as a strong leader, and viewed their EE process favorably. We also found that racial achievement gaps were reduced in schools where teachers collaborated frequently and were provided more opportunities to use feedback.

Page 22

Many Black and Latinx students have to overcome their school to Many Black and Latinx students have to overcome their school to succeed.succeed.

It is important to emphasize that this study does not suggest it is impossible for students to succeed in schools with a weak professional culture. There are students who progress through schools with teachers less committed to their school, less supported by their EE process, and less supported by an effective principal. However, students who succeed in these schools do so in spite of their school. Students who overcome their school to become doctors or teachers should not be used to justify the continued inequities in education this study spotlights. Schools shouldn’t be a barrier to success. Unfortunately, this study suggests that many are for Black and Latinx students.

Addressing the Black student achievement gap may be more complicated Addressing the Black student achievement gap may be more complicated than just organizing the professional cultures within schools to improve.than just organizing the professional cultures within schools to improve.

Many of the schools in the current study serving Black students are likely in significantly disadvantaged communities. In the Chicago study (Bryk et al., 2010), schools in communities with high levels of crime, poverty, domestic violence, and students in "unconventional living arrangements", such as being homeless or in foster care, were called Truly Disadvantaged. The schools in these communities needed an extremely strong professional culture to improve. Truly Disadvantaged schools could still improve but doing so was much more difficult. Although the current study does not explore the community conditions in schools serving Black students, it is likely that some number of these schools are in Truly Disadvantaged communities. Further research will explore the intersection of school and community factors more in depth.

EE, principal leadership, and collaborative teachers are all necessary EE, principal leadership, and collaborative teachers are all necessary ingredients for school improvement.ingredients for school improvement.

This study does not allow us to separate out specific aspects of school professional culture to understand which are the most important for reducing achievement gaps. In the Chicago study (Bryk, et al., 2010), school organization was compared to the ingredients of a cake. Just like you need certain ingredients to bake a cake, schools need to have certain ingredients to be effective. All are necessary and important. A cake needs more flour than baking soda but still needs both. Similarly, the aspects of school professional culture examined in this study are some of the ingredients needed to build an effective school. Thus, it is irrelevant which is the most important. Schools missing some of the ingredients will struggle to improve. Unfortunately, the results of this study suggest many schools with Black or Latinx students are missing many of the necessary ingredients of effective schools. When schools were organized with these necessary ingredients, however, racial achievement gaps decreased substantially.

DiscussionDiscussion

Page 23

Students who rely the most on their school for their educational Students who rely the most on their school for their educational opportunities attend schools that are the least able to provide them.opportunities attend schools that are the least able to provide them.

A student’s educational opportunities come primarily from three places: their family, their community, and their school. Through grades and test scores, schools provide benchmarks of student success. For families with privilege, these benchmarks are often interpreted through a lens of what puts their children on a path to enter a competitive college or university. If a child is not meeting those benchmarks, a family can organize additional supports and experiences to promote their child's success in school. Further, if a school is viewed as ineffective at positioning their child for postsecondary success, a family with privilege can influence their school's practices and even switch schools if necessary. Also, families with privilege typically live in advantaged communities that are safe and provide access to additional learning opportunities. For students living in a family with privilege, their success is much less affected by the effectiveness of their school. These families adapt to the effectiveness of their child’s school and work to ensure their success. This is consistent with the results for White students in the current study.

For students of color living in poverty and in marginalized communities, their educational opportunities are much more determined by their school. Families in poverty have fewer opportunities to promote their students' academic growth. Families must rely on their children’s schools more for addressing their educational needs. If a child’s school is failing, there is little recourse for the family. Thus, schools act more as a hinge for determining the success or failure of students living in marginalized communities. Schools in marginalized communities with a strong professional culture have a disproportionately positive impact on the academic success and development of disadvantaged students. Consistent with this, in the current study schools with a professional culture organized to improve successfully narrowed the achievement gap for Black and Latinx students. Conversely, schools with a weak professional culture have a disproportionately negative impact on disadvantaged students. Due to Wisconsin's extreme segregation, low-income, Black students are packed into schools with a professional culture organized to stagnate, and their achievement reflects that. Together, these results show the potential of schools for providing educational opportunities for Black and Latinx students and for schools truly acting as a "great equalizer" of society. The results of this study, however, demonstrate this potential remains largely unrealized.

Page 24

In Wisconsin, we need to better understand to what extent schools have In Wisconsin, we need to better understand to what extent schools have the necessary ingredients of effective schools.the necessary ingredients of effective schools.

One limitation of this study is that the data collection system for the statewide evaluation of Wisconsin EE does not require all schools to participate at a consistent level. As such, we only have reliable data for a sample of schools. Further, since we only survey educators, we are not able to determine if schools have all the necessary “ingredients” for making an effective school. Given the importance of this study’s findings based exclusively on teacher voice, Wisconsin should consider rigorous statewide surveys of students and perhaps parents as well. This study's results, and the Chicago study (Bryk, et al., 2010), provide a recipe for school improvement and for turning around the large achievement gaps between racial groups in Wisconsin schools. Unfortunately, we at this time do not know which and how much of the necessary ingredients are present across Wisconsin schools.

The achievement gaps between Black, Latinx, and White students are The achievement gaps between Black, Latinx, and White students are due to institutional racism.due to institutional racism.

Even though we all have a responsibility to understand our biases and to work to prevent them from negatively impacting others, it is important to emphasize that this study should not be used to blame teachers or principals for the large achievement gaps found between Black, Latinx, and White students. Doing so would be counterproductive for improving the education system to be equitable. This study's findings place the blame for the inequities on institutional racism (see figure on page 26). Students who are marginalized and disadvantaged attend schools with weak professional cultures that serve to further marginalize and disadvantage them. Unfortunately, the communities where it is the most important schools have a strong professional promoting student learning and development, are much less likely to have them. These schools lack the tools and resources necessary for teachers to be effective and to prevent teacher bias from negatively impacting students. It is clear that these schools have many committed and caring educators who are motivated for students to succeed. However, educator bias (Jones, 2019; Starck, Riddle, Sinclair, & Warikoo, 2020) and other aspects of institutional racism still affect the overall professional culture of schools. For example, schools struggle to hire and retain teachers of color (Jones, 2019; Achinstein, Ogawa, Sexton, & Frietas, 2010) and retain teachers more generally when they have large numbers of students of color (Jones, 2019). Schools also struggle to effectively engage families of color and blame parents for their lack of engagement (DeCastro-Ambrosetti & Cho, 2005). These are just some of the ways Black and Latinx students are harmed by the Wisconsin education system as it is currently organized.

Page 25

ID

,_,

>, "C :::J .... Ill a, .s::. .... 0 en C C C en a, .c a, .s::. .... .... ro a. ro en .... C a,

a,

-�.s::. u

VISUALIZING INSTITUTIONAL

RACISM IN WISCONSIN SCHOOLS

Achievements gaps at the start and end of the study (2016-2019)

102 SCHOOLS

White student

achievement

better than the

state average

State Average

99 SCHOOLS

Black and Latinx

student achievement

far below the

state average

Opportunity gaps

during the study

Many schools (~25%) serving White

students organized to improve, with a

strong professional culture.

■

Very few schools ( ~ 10%) serving Black

or Latinx students organized to improve.

WEERP

No change in

White student

achievement (Stayed above the state average)

State Average

Improved

Black and Latinx

achievement

The achievement

gap grew for the

other -90% of

schools serving

Black or Latinx

students.

Wisconsin Educator Effectiveness Research Partnership Socially Responsible Evaluation in Educ::ation

uwm.edu/sreed I sreed-info@uwm.edu

;' Gl:E ... Ill Cl) n Ill :::r ::::!. 0 '< iii � 0 a. � C a. n

::i Cl) i::j·

a.

ID Ill Q. g.0 io' §" Cl) 'tl 3 .., Cl) 0 :::J " .... ID Ul Ill Ill Cl) "C

5. �:::J .... Ul :::r Ill Cl) .... Cl) C :::J a. a. � 0 .... -111 ....

Q, i6n Ill 0 .... _c 0 a. :" '<

i3 ;· g :T ID Cl> ,_,

Page 26

During to the disruption to schools caused by COVID-19 we must fix the During to the disruption to schools caused by COVID-19 we must fix the schools we are sending Black and Latinx students back to.schools we are sending Black and Latinx students back to.

Finally, we must acknowledge the impact that COVID-19 is having on Wisconsin schools. With many schools across Wisconsin closed, there is a growing concern that marginalized groups have been disproportionately affected and that achievement gaps have likely widened. Because of this, there is an urgency for students to return to their schools. However, we must consider the weak professional cultures typical of schools to which Black and Latinx students are returning. The school system has not worked for many of these students. Should we simply send them back with no consideration for this? There needs to be a statewide focus on racial equity and statewide educator training regarding bias, not just about how to effectively work with students and families of different racial groups, but also about how to work with educators from different backgrounds. The seeds of this are being planted within the EE system, as DPI revises EE to be more explicitly focused on developing educators to improve educational equity and reduce bias. Specifically, Educator Effectiveness will include self-reflection on identity and bias, as well as support for implementing and observing/coaching for equitable practices. The work being done in the Wisconsin Urban Leadership Institute (Cain & Jones, 2019), developing principals into equity leaders, is another example. However, it is not clear these initiatives are enough to overcome the organizational conditions in the schools Black and Latinx students attend. There must not only be an urgency for sending students back to school but also for fixing the schools to which we send them.

End note:

1. We use the term Achievement Gap to descriptively report on the difference in state achievement results for students from different racial or ethnic groups. We understand that the term Achievement Gap is deficit-focused and individual-blaming. We hope that this paper provides specific examples of how the term Achievement Gap fails to capture the real problem in the Wisconsin education system, that the amount of school opportunity a student experiences is directly related to their privilege.

2. We capitalize White and Black throughout the report based on recommendations published in June of 2020 by the National Association of Black Journalists. https://www.nabj.org/page/styleguide

Page 27

Achinstein, B., Ogawa, R. T., Sexton, D., & Freitas, C. (2010). Retaining teachers of color: A pressing problem and a potential strategy for “hard-to-staff” schools. Review of Educational Research, 80(1), 71–107. https://doi.org/10.3102/0034654309355994

Bryk, A. S., Sebring, P. B., Allensworth, E. L., & Luppescu, S. S. & Easton, J.(2010): Organizing Schools for Improvement. Lessons from Chicago. The University of Chicago Consortium on Chicago School Research: Chicago.

Cain, E. & Jones, C. J. (September, 2019). Results from the First-Year Evaluation of the Wisconsin Urban Leadership Institute. A Wisconsin Educator Effectiveness Research Partnership (WEERP) Evaluation Brief. Retrieved online at: https://uwm.edu/sreed/wp-content/uploads/sites/502/2019/10/WEERP-Wisconsin-Urban-Leadership-Institute-Brief-September-2019.pdf

Cherasaro, T. L., Brodersen, R. M., Yanoski, D. C., Welp, L. C., & Reale, M. L. (2015). The Examining Evaluator Feedback Survey (REL 2016–100). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central. Retrieved from http://ies.ed.gov/ncee/edlabs).

Danielson, C. (2013). The Framework for Teaching Evaluation Instrument: 2013 Instructionally Focused Edition. Princeton, NJ: The Danielson Group.

DeCastro-Ambrosetti, D. & Cho, G. (2005). Do parents value education? Teachers' perceptions of minority parents. Multicultural Education, 13(2), 44-46.

Grissom, J.A., & Youngs, P. (2016). Improving Teacher Evaluation Systems: Making the Most of Multiple Measures. New York, NY: Teachers College Press.

Jones, C. J. (November, 2019). Race, Relational Trust, and Teacher Retention in Wisconsin Schools. A Wisconsin Educator Effectiveness Research Partnership (WEERP) Evaluation Brief. Retrieved online at: https://uwm.edu/sreed/wp-content/uploads/sites/502/2019/11/WEERP-Brief-Nov-2019-Race-Relational-Trust-and-Teacher-Retention.pdf

Jones, C. J., Cain, E., & Gilman, L. (September 2019). The Impact of the Wisconsin Educator Effectiveness Process on New Teacher Turnover. A Wisconsin Educator Effectiveness Research Partnership (WEERP) Evaluation Brief. Retrieved online at: https://uwm.edu/sreed/wp-content/uploads/sites/502/2019/10/WEERP-New-Teacher-Retention-Brief-September-2019.pdf

ReferencesReferences

Page 28

Jones, C. J. (October, 2018). How are District Differences in the Implementation of Teacher Evaluations associated with Teacher Perceptions of School Leadership and Teacher Job Satisfaction? A Wisconsin Educator Effectiveness Research Partnership (WEERP) Evaluation Study. Retrieved online at: https://uwm.edu/sreed/wp-content/uploads/sites/502/2019/10/2018-WEDSR-Teacher-Survey-Results-and-District-Conditions-that-Promote-Teacher-Job-Satisfaction.pdf

Jones, C. J., Gilman, L., & Pyatigorsky, M. (August, 2019). The Impact of the Wisconsin Educator Effectiveness Process on Student Achievement. A Wisconsin Educator Effectiveness Research Partnership (WEERP) Evaluation Brief. Retrieved online at: https://uwm.edu/sreed/wp-content/uploads/sites/502/2019/10/WEERP-EE-Student-Achievement-Brief-August-2019-1.pdf

Klugman, J., Gordon, M. F., Sebring, P. B., & Sporte, S. E. (2015). A First Look at the 5Essentials in Illinois Schools. Research Report. University of Chicago Consortium on Chicago School Research.

Kraft, M. A., Brunner, E. J., Dougherty, S. M., & Schwegman, D. J. (2019). Teacher Evaluation Reforms and the Supply and Quality of New Teachers (Brown University Working Paper). Providence, RI. Retrieved from https://scholar.harvard.edu/files/mkraft/files/kraft_et_al._teacher_evaluation_-_updated_feb_2019.pdf

Liebowitz, D. D. (2019). Teacher Evaluation for Accountability and Growth: Should Policy Treat them as Complements or Substitutes. University of Oregon Working Paper. Retrieved from https://scholar.harvard.edu/files/dliebowitz/files/evaluation_compl_subs_july_2019. pdf.

McCaffrey, J. R., Lockwood, D. F., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating Value Added Models for Teacher Accountability [Monograph]. Santa Monica, CA: RAND Corporation. Retrieved from http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf

MET Project (2013). Ensuring Fair and Reliable Measures of Effective Teaching: Culminating Findings from the MET Projects’s Three-year Study. Seattle, WA: Bill & Melinda Gates Foundation.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrics, 73, 417-458.

ReferencesReferences

Page 29

Stronge, J. H. (2002). Qualities of Effective Teachers. Alexandria, VA: Association for Supervision and Curriculum Development.

Thompson, E.R. & Phua, F. T.T. (2012). A Brief Index of Affective Job Satisfaction. Group & Organization Management, 37(3), 275-307.

Wisconsin (2012). 2011 Wisconsin Act 166. Retrieved from

https://docs.legis.wisconsin.gov/2011/related/acts/166.pdf

ReferencesReferences

Page 30

This study was supported by the Wisconsin Department of Public Instruction as part of the Wisconsin Educator Effectiveness Research Partnership (WEERP). WEERP is a research-practitioner partnership between the Office of Socially Responsible Evaluation in Education at the University of Wisconsin Milwaukee, The Wisconsin Center for Education Research at the University of Wisconsin Madison, and the Wisconsin Department of Public Instruction.

WEERP conducts rigorous and relevant research to inform the efforts of Wisconsin Educators to improve educator effectiveness and achievement for all students.

Thank you to the following persons for their helpful feedback on drafts of the paper.

Elaine Allensworth, University of Chicago Sheila Briggs, Wisconsin Department of Public Instruction John Bowser, University of Wisconsin Milwaukee Anne Chapman, Wisconsin Policy Forum Cathy Clarkson, CESA 6 Kyle Fagan, American Institutes for Research Leon Gilman, University of Wisconsin Milwaukee Steve Kimball, University of Wisconsin Madison Emily Kite, University of Wisconsin Madison James Sokolowski, Milwaukee Public Schools

Curtis J. Jones is the Director of the Office of Socially Responsible Evaluation in Education at the University of Wisconsin Milwaukee and Co-Director of WEERP.

Marlo A. Reeves is a Senior Research/Evaluation Associate in the Office of Socially Responsible Evaluation in Education at the University of Wisconsin Milwaukee.

Katharine M. Rainey is the Director of Educator Development and Support at the Wisconsin Department of Public Instruction and Co-Director of WEERP.

For more information about this report or about the statewide evaluation of Wisconsin Educator Effectiveness, please contact Curtis Jones at jones554@uwm.edu or visit www.uwm.edu/sreed

AcknowledgmentsAcknowledgments

Page 31

Appendix A:Appendix A:Background on the Wisconsin Educator Effectiveness ProcessBackground on the Wisconsin Educator Effectiveness Process

The Wisconsin Department of Public Instruction (DPI) had already been exploring the possibility of EE systems when in 2012 the Wisconsin legislature passed Act 166 (Wisconsin, 2012), requiring districts to implement standards-based educator evaluation systems. The resulting Wisconsin educator evaluation process was motivated by research on the impact of teaching quality on student achievement (McCaffrey, Lockwood, Koretz, & Hamilton, 2003; Rivkin, Hanushek, & Kain, 2005) as well as research demonstrating improved teacher evaluation measures (Grissom & Youngs, 2016; MET Project, 2013).

Wisconsin’s teacher evaluation system is intended to promote the use of performance feedback to enhance the quality of teaching and student learning across the state. The system requires schools provide ongoing feedback to teachers about their professional practice based on a standards-based model of teaching. Districts may use the Danielson Framework for Teaching (Danielson, 2013), the CESA 6 Effectiveness Project based on the Stronge Teacher Performance Evaluation System (Stronge, 2002), or apply to use an equivalent teaching practice model. Each model includes a set of domains or standards and multiple levels of professional practice with indicators of practice at each level.

Veteran teachers are evaluated on a three-year cycle. Every third or “summary year,” teachers complete a structured process that involves a series of goal setting activities and evidence collection that includes observations and teaching artifacts (e.g., lesson plans, formative assessments, student work). Feedback is required based on the observations and other evidence using the teaching practice model and rubric. Evaluators, primarily school principals, engage with teachers in the process and provide feedback. First-year teachers also complete the summary year teacher evaluation process. In off years, or “supporting years,” teachers may receive feedback from a peer or coach instead of an evaluator.

Another aspect of the Wisconsin evaluation process involves teachers completing at least one Student Learning Objective (SLO) every year. This involves identifying, measuring, and tracking specific student learning outcomes across the year. A third aspect involves the creation of a Professional Practice Goal (PPG), whereby teachers identify an area of practice they plan to work to improve during the school year. At the end of their summary year, teachers receive culminating feedback that may include practice-based ratings based on observations and artifact evidence and separate ratings based on their SLO.

In Wisconsin, the Department of Public Instruction (DPI) encourages schools to use the evaluation process in a learning-centered, growth-focused manner that provides teachers with constructive and actionable feedback during a continuous improvement cycle. The DPI strongly discourages the use of evaluation ratings to compare teachers when making decisions (e.g., merit pay), but does suggest it is acceptable for data to inform decisions regarding individual teachers. This approach is justified by recent research on teacher evaluation reforms, which has shown the implementation of high-stakes evaluation systems lowers the attractiveness of becoming a teacher and reduces the supply of new teacher candidates (Kraft, Brunner, Dougherty, & Schwegman, 2019). Further, in a recent review and meta-

Page 32

analysis of teacher evaluation reform research, Liebowitz (2019) concludes that the approach with the greatest positive impact on teacher effectiveness emphasizes growth more than accountability. He argues that accountability should only be the focus on teachers with the lower performance ratings so that most teachers are able to focus on their growth.

Page 33

How do we measure achievement gaps?

The Forward Exam is the state of Wisconsin’s achievement test administered each spring to students in grades three through eight. School results are posted by DPI each year for a number of student groups. Results are posted within grade levels for students groups when at least five students in a group take the exam. To calculate Forward Exam standardized scores we calculated school English Language Arts (ELA) and Math Z-scores for White, African American, and Latinx student groups by grade level for all Wisconsin schools. Z-scores present the standardized difference in Forward Exam scores for each racial group within each grade level. We then calculated an overall school level Z-score for each racial group by averaging the grade level Z-scores (weighted by the number of students in that group who took the assessment in that grade). Thus, a school’s Z-score for a racial group represents the number of standard deviations better or worse that group scored on the Forward Exam than was typical across all Wisconsin Schools. This process was done for 2015-16 and 2018-19 achievement results. Thus, the study is able to measure changes in the achievement gaps for each racial group from 2015-16 to 2018-19. We chose to calculate racial achievement gaps by comparing groups to state averages rather than other racial groups within the same schools because the racial enrollment in many schools was homogeneous. This prevented us from calculating racial achievement gaps within schools.

How do we measure educator retention?

State public data files were analyzed to calculate both teacher and principal attrition. To do this, educators were matched to public data files over time. The number of educators who remained associated with a school represented the educator retention rate for that school. Educator retention rates were calculated at baseline by comparing the consistency of teachers from the 2014-15 to the 2015-16 school years and as a predictor of achievement gap change by comparing the consistency of teachers from the 2016-17 to 2018-19 school year. Principal retention was measured as a dichotomous variable of whether the school had the same principal from 2014-15 to 2015-16 and from 2016-17 to 2018-19.

How do we measure school professional culture?

The WEDSR Survey is a foundational data collection instrument for the Wisconsin Educator Effectiveness Research Partnership (WEERP). The WEDSR, which is administered to all Wisconsin teachers each spring, measures teacher perceptions of their evaluation processes and several other aspects of school professional culture as outlined below. For this study, responses within each school were averaged across each of the measures described below.

The Feedback Process – We used scales from the Examining Evaluator Feedback Survey (Cherasaro, Brodersen, Yanoski, Welp, & Reale, 2015) to measure teacher perceptions of the feedback process. Teachers were asked the degree they agree or disagree with:

Appendix B - Methods:Appendix B - Methods:Study MeasuresStudy Measures

Page 34

• three statements about Feedback Accuracy, such as “The feedback I received was an accurate portrayal of my teaching,”

• seven statements about Feedback Usefulness, such as “My evaluator’s feedback was provided as frequently as I needed it,”

• four statements about the Opportunity to Use Feedback, such as “I was able to observe expert teachers modeling skills that related to my feedback,” and

• five statements about the Use of Feedback, such as “I tried new instructional strategies in my classroom.

The internal consistency of these scales was .840, .938, .812, and .903, respectively. Confirmatory Factor Analysis results verify the four factors of the measure (Chi2 p < .001; RMSEA = .074; CFI = .995; SRMR = .047). Only teachers who indicated they received performance feedback in the current year were asked to answer these questions.

Principal Effectiveness – We used three survey scales to measure teacher perceptions of principal effectiveness. The first two scales are from the University of Chicago’s 5Essentials Survey (Klugman, Gordon, Sebring, & Sporte, 2015). With these, teachers were asked the extent they agree or disagree with eight statements about Principal Leadership, such as, “The principal participates in instructional planning with teams of teachers,” and eight statements about Principal Trust, such as, “The principal looks out for the personal welfare of the staff.” The internal consistency of these two scales in the current sample is .934 and .957 respectively. The third scale was derived from the Examining Evaluator Feedback Survey (Cherasaro et al. 2015). With this, teachers were asked the extent they agree or disagree with five statements about Evaluator Qualifications to Provide Feedback, such as, “My evaluator had sufficient knowledge of effective teaching practices to effectively evaluate me.” The internal consistency of this five-item scale was .929.

Teacher collaboration – We used one scale from the University of Chicago’s 5Essentials Survey (Klugman et al., 2015) to measure the amount of collaboration occurring between teachers. Teachers were asked how often they engaged in five different collaborative activities such as “Observed another teacher’s classroom to get ideas for your own instruction.” The internal consistency of this five-item scale was .763.

Teacher job satisfaction – We used the Brief Index of Affective Job Satisfaction (Thompson & Phua, 2012) to measure affective teacher Job Satisfaction. With this, teachers were asked the extent they agree or disagree with four statements about how they feel about their job, such as, “Most days I am enthusiastic about my job.” The internal consistency of this scale is .937. We also used one scale from the University of Chicago’s 5Essentials Survey (Klugman et al., 2015) to measure teacher School Commitment. With this, teachers were asked the degree they agree or disagree with four statements about their school, such as, “I wouldn’t want to work in any other school.” The internal consistency of this scale was .885.

Page 35

Appendix C: Appendix C: Which schools were included in this study?Which schools were included in this study?

Only schools with at least a 60% WEDSR total response rate aggregated across all three study years (2016-17, 2017-18, and 2018-19) AND that had assessment results for both the 2015-16 and 2018-19 school years for students in all taught grades were included in the analyses presented in this report. 211 schools met these criteria. Within the 211 schools, 12,396 out of 16,870 teachers completed a survey, representing a 73.5% response rate.

The 211 study schools included 167 elementary, 26 middle, seven combined elementary/secondary, and 11 high schools. Among the 211 schools, Latinx student achievement results were available for 59 schools, Black student achievement results for 67 schools, and White student achievement results for 102 schools. For a school to be included in an analysis of a specific student group, achievement results had to be available for students in that group for every tested grade in both 2015-16 and 2018-19.

Of the 211 schools, 49 were using the Stronge Effectiveness Performance Evaluation System (Stronge, 2002), and 162 were using the Danielson Framework for Teaching (Danielson, 2013) as part of their EE process. 102 of the 211 schools included in the study were part of the Milwaukee Public Schools (MPS). The schools included in the study were more diverse than Wisconsin schools overall, comprised of more than half students of color. The schools were also much lower achieving than schools across the state, with ELA and Math Forward Exam scores 0.66 and 0.71 standard deviations lower than the state average.

Sample SchoolsMean SD

Percent Black 27.4% 36.5%Percent Latinx 16.0% 22.9%Percent White 47.5% 39.3%Percent Asian 3.8% 6.8%Percent two or more races 3.5% 2.4%Percent Pacific Islander 0.1% 0.2%Percent Native American 1.7% 8.8%Percent students with a disability 17.5% 6.0%Percent students economically disadvantaged 63.3% 28.0%Percent students English Language Learners 7.0% 12.8%Percent female 48.0% 3.2%School size 440.5 199.6

School demographic characteristics of all 211 study schoolsSchool demographic characteristics of all 211 study schools

Page 36

Appendix C: Appendix C: Which schools were included in this study?Which schools were included in this study?

Sample SchoolsMean SD

Percent Black 58.7% 31.9%Percent Latinx 18.5% 22.5%Percent White 12.1% 17.8%Percent Asian 6.4% 8.5%Percent two or more races 3.8% 2.3%Percent Pacific Islander 0.1% 0.1%Percent Native American 0.5% 0.8%Percent students with a disability 20.3% 7.2%Percent students economically disadvantaged 83.0% 18.9%Percent students English Language Learners 8.1% 12.8%Percent female 47.7% 3.7%School size 499.0 221.0

School demographic characteristics of 67 schools serving Black studentsSchool demographic characteristics of 67 schools serving Black students

School demographic characteristics of 59 schools serving Latinx studentsSchool demographic characteristics of 59 schools serving Latinx studentsSample Schools

Mean SDPercent Black 16.9% 19.9%Percent Latinx 39.4% 28.6%Percent White 33.3% 29.3%Percent Asian 5.6% 7.8%Percent two or more races 3.9% 2.5%Percent Pacific Islander 0.1% 0.1%Percent Native American 0.7% 0.9%Percent students with a disability 18.0% 6.5%Percent students economically disadvantaged 70.3% 24.6%Percent students English Language Learners 16.3% 17.9%Percent female 48.2% 3.1%School size 593.1 228.8

Page 37

Appendix D: Appendix D: Which students were included in this study?Which students were included in this study?

In 2018-19, these 211 schools tested 6,310 Latinx students, 6,279 African American students, and 17,578 White students. Statewide, school results were available for 30,569 Latinx, 16,816 African American, and 198,354 White third to eighth grade students. Thus, even though the 211 schools represent only about 10% of all Wisconsin schools, these schools included 37% of Black and 21% of Latinx tested Wisconsin students.

School demographic characteristics of 102 schools serving White studentsSchool demographic characteristics of 102 schools serving White studentsSample Schools

Mean SDPercent Black 7.0% 12.2%Percent Latinx 16.1% 16.7%Percent White 66.7% 26.7%Percent Asian 4.6% 7.3%Percent two or more races 4.3% 2.5%Percent Pacific Islander 0.1% 0.1%Percent Native American 1.3% 3.4%Percent students with a disability 15.3% 4.3%Percent students economically disadvantaged 48.1% 22.5%Percent students English Language Learners 6.0% 7.8%Percent female 48.6% 3.0%School size 493.2 225.4

Page 38

How did we estimate the impact of aspects of school professional culture on racial achievement gaps?

We examined the relationship between measures of school organizational (SO) conditions and changes in achievement (ELA and Math weighted Forward Exam (FE) Standard Sores) by estimating a series of regressions, using maximum likelihood estimation (Generalized Linear Modeling with Robust Standard Errors and an identify link function) with the following equation:

FEs(2019) = β0 + β1FE-Ms(2016) + β2FE-ELA s(2016) + β3SOs(2017-2019) + β4MPSs + Xs(2019) + εs(2019)

Where:

FEs(2019) is the spring 2019 Forward Exam Math or ELA score for a specific student group in school s;

FE-Ms(2016) is the spring 2016 Forward Exam Math score for a specific student group in schools s;

FE-ELA s(2016) is the spring 2016 Forward Exam ELA score for a specific student group in school s;

MPSs is 1 if a school is in MPS and 0 if not;

Xs(2019) is a vector of school characteristics, such as school size, % F/R lunch, % teacher retention, and % white;

SOs(2017-2019) is a school organizational condition aggregated across the 2016-17, 2017-18, and 2018-19 school years.

εs(2019) is the error term for school s.

Appendix E:Appendix E:Modeling approachModeling approach

Page 39

Aspect of school professional culture βStd.

Error Lower UpperWald Chi-

Square df Sig.ELA results

Feedback accuracy 0.03 0.02 -0.02 0.08 1.57 1 0.211Feedback usefulness 0.05 0.02 0.01 0.10 5.31 1 0.021Opportunity to use feedback 0.03 0.02 -0.01 0.07 1.78 1 0.182Use of feedback 0.00 0.02 -0.05 0.05 0.00 1 0.986Qualifications of evaluator 0.04 0.02 0.00 0.08 3.92 1 0.048Principal leadership 0.07 0.02 0.03 0.12 10.96 1 0.001Principal trust 0.09 0.02 0.04 0.14 12.55 1 <.001Teacher collaboration 0.08 0.02 0.04 0.12 15.39 1 <.001Teacher affective job satisfaction 0.09 0.02 0.04 0.14 12.55 1 <.001Teacher school commitment 0.13 0.02 0.08 0.18 27.10 1 <.001Teacher retention (16 to 19) 0.07 0.02 0.03 0.11 9.80 1 0.002

Math resultsFeedback accuracy 0.05 0.02 0.00 0.10 4.60 1 0.032Feedback usefulness 0.06 0.02 0.01 0.10 6.22 1 0.013Opportunity to use feedback 0.07 0.02 0.02 0.11 9.48 1 0.002Use of feedback 0.02 0.02 -0.02 0.07 0.96 1 0.329Qualifications of evaluator 0.06 0.02 0.02 0.10 7.26 1 0.007Principal leadership 0.09 0.02 0.05 0.14 18.42 1 <.001Principal trust 0.07 0.02 0.03 0.11 10.19 1 0.001Teacher collaboration 0.11 0.02 0.07 0.15 30.17 1 <.001Teacher affective job satisfaction 0.08 0.03 0.03 0.13 9.19 1 0.002Teacher school commitment 0.13 0.02 0.08 0.18 28.83 1 <.001Teacher retention (16 to 19) 0.05 0.02 0.01 0.10 6.51 1 0.011

Separate model results for aspects of school professional culture predicting changes in overall school achievement

Appendix F: Appendix F: Results TablesResults Tables

Page 40

12

34

56

78

910

1112

1314

1516

1718

19

120

15-1

6 EL

A a

chie

vem

ent

1

220

15-1

6 M

ath

achi

evem

ent

.954

1

3Te

ache

r ret

entio

n in

201

5-16

.534

.598

1

4Sc

hool

size

.129

.095

.03

1

5St

uden

ts w

/ dis

abili

ty %

-.637

-.624

-.316

-.074

1

6St

uden

t eco

n di

sadv

%-.8

63-.8

72-.5

25-.0

88.6

131

7W

hite

stud

ent %

.812

.842

.636

-.106

-.543

-.881

1

820

18-1

9 EL

A A

chie

vem

ent

.945

.935

.583

.13

-.647

-.877

.839

1

920

18-1

9 M

ath

Ach

ieve

men

t.9

21.9

47.6

16.0

76-.6

21-.8

69.8

67.9

711

10Fe

edba

ck a

ccur

acy

.493

.467

.270

-.067

-.299

-.406

.453

.484

.497

1

11Fe

edba

ck u

sefu

lnes

s.0

93.0

38-0

.04

-.071

-.022

.019

-.001

.094

.082

.686

1

12O

ppor

tuni

ty to

use

feed

back

.083

.039

-.037

.006

-.012

.018

-.032

.07

.086

.600

.773

1

13Fe

edba

ck u

se

-.258

-.293

-.275

-.074

.234

.328

-.300

-.286

-.275

.222

.569

.618

1

14Q

ualifi

catio

ns o

f eva

luat

or.3

43.3

13.2

16-.0

63-.1

76-.2

68.3

07.3

52.3

61.8

70.7

42.6

75.3

541

15Pr

inci

pal l

eade

rshi

p.3

19.2

91.1

47-.0

91-.1

83-.2

01.2

40.3

39.3

52.7

22.7

26.6

69.3

56.8

051

16Pr

inci

pal t

rust

.331

.306

.177

-.072

-.178

-.222

.279

.344

.352

.746

.651

.618

.252

.807

.907

1

17Te

ache

r col

labo

ratio

n.1

09.0

81-.0

15.1

77-.0

39-.0

61-.0

57.1

52.1

52.3

24.4

94.6

12.3

62.3

91.4

90.3

781

18Te

ache

r affe

ctiv

e jo

b sa

tisfa

ctio

n.6

39.6

28.4

13-.0

24-.3

72-.5

44.5

82.6

59.6

55.6

12.3

55.3

57.0

07.5

30.5

71.6

08.2

541

19Te

ache

r sch

ool c

omm

itmen

t .6

43.6

06.3

45-.0

18-.3

75-.4

84.4

76.6

60.6

50.6

76.4

74.4

58.0

1.6

22.6

66.7

02.3

67.8

391

20Te

ache

r ret

entio

n 20

17 to

201

9.4

94.4

81.4

11.2

15-.2

59-.4

17.3

85.5

28.5

09.3

64.1

13.1

3-.2

55.3

07.2

23.3

07.1

66.4

44.6

16

Page 41

Aspect of school professional culture βStd.

Error Lower UpperWald Chi-

Square df Sig.ELA results

Feedback accuracy -0.03 0.06 -0.16 0.10 0.20 1 0.658Feedback usefulness 0.03 0.07 -0.10 0.16 0.20 1 0.658Opportunity to use feedback -0.02 0.06 -0.15 0.11 0.11 1 0.736

Use of feedback -0.01 0.06 -0.14 0.11 0.04 1 0.842Qualifications of evaluator -0.01 0.06 -0.13 0.12 0.01 1 0.931Principal leadership 0.07 0.06 -0.05 0.19 1.35 1 0.245Principal trust 0.09 0.06 -0.02 0.20 2.69 1 0.101Teacher collaboration 0.17 0.07 0.03 0.31 5.76 1 0.016Teacher affective job satisfaction 0.10 0.07 -0.03 0.23 2.14 1 0.144Teacher school commitment 0.21 0.06 0.10 0.32 12.77 1 <.001Teacher retention (16 to 19) 0.05 0.05 -0.04 0.14 1.13 1 0.289

Math resultsFeedback accuracy 0.06 0.08 -0.09 0.21 0.56 1 0.455Feedback usefulness 0.07 0.08 -0.08 0.23 0.82 1 0.365Opportunity to use feedback 0.03 0.08 -0.12 0.18 0.18 1 0.668Use of feedback 0.09 0.08 -0.06 0.24 1.51 1 0.218Qualifications of evaluator 0.11 0.07 -0.03 0.25 2.44 1 0.118Principal leadership 0.17 0.07 0.03 0.30 5.71 1 0.017Principal trust 0.15 0.07 0.02 0.28 5.26 1 0.022Teacher collaboration 0.25 0.08 0.09 0.41 8.97 1 0.003Teacher affective job satisfaction 0.13 0.08 -0.03 0.29 2.69 1 0.101Teacher school commitment 0.26 0.07 0.13 0.40 14.42 1 <.001Teacher retention (16 to 19) 0.06 0.06 -0.05 0.17 1.12 1 0.290

Separate model results for aspects of school professional culture predicting changes in overall school achievement for Black students

Page 42

12

34

56

78

910

1112

1314

1516

1718

19

120

15-1

6 EL

A a

chie

vem

ent

1

220

15-1

6 M

ath

achi

evem

ent

.940

1

3Te

ache

r ret

entio

n in

201

5-16

.258

.376

1

4Sc

hool

size

.561

.553

.341

1

5St

uden

ts w

/ dis

abili

ty %

-.550

-.548

-0.0

3-.2

961

6St

uden

t eco

n di

sadv

%-.7

43-.7

28-.0

85-.4

81.5

371

7W

hite

stud

ent %

.734

.756

.248

.454

-.456

-.848

1

820

18-1

9 EL

A A

chie

vem

ent

.917

.940

.310

.582

-.594

-.796

.788

1

920

18-1

9 M

ath

Ach

ieve

men

t.8

79.9

47.3

77.5

47-.5

40-.7

45.7

75.9

591

10Fe

edba

ck a

ccur

acy

.333

.320

-.013

.073

-.171

-.265

.362

.330

.347

1

11Fe

edba

ck u

sefu

lnes

s.0

83.0

63-.1

140

-.042

-.004

.11

.143

.138

.721

1

12O

ppor

tuni

ty to

use

feed

back

.117

.128

-.031

.035

-.028

-.12

.291

.181

.207

.752

.796

1

13Fe

edba

ck u

se

-.287

-.295

-.215

-.153

.238

.298

-.197

-.269

-.245

.262

.587

.518

1

14Q

ualifi

catio

ns o

f eva

luat

or.2

06.2

31.0

06.0

17-.0

54-.1

7.3

11.2

58.2

85.8

39.7

40.7

87.4

171

15Pr

inci

pal l

eade

rshi

p.1

75.1

89-.0

46-.0

1-.1

12-.1

02.2

19.2

56.2

69.6

48.6

48.7

18.3

69.7

761

16Pr

inci

pal t

rust

.149

.169

-.028

-.027

-.079

-.067

.22

.226

.228

.671

.558

.664

.255

.769

.892

1

17Te

ache

r col

labo

ratio

n.2

05.2

54.0

07.1

11-.1

48-.1

59.2

66.3

12.3

74.4

99.4

02.5

36.1

65.4

96.5

70.5

371

18Te

ache

r affe

ctiv

e jo

b sa

tisfa

ctio

n.4

92.5

02.2

17.2

-.186

-.303

.465

.525

.508

.470

.323

.394

0.4

74.5

31.5

34.3

761

19Te

ache

r sch

ool c

omm

itmen

t .6

00.5

97.1

85.2

4-.2

65-.4

02.5

22.6

57.6

50.5

68.3

68.4

29-0

.1.5

76.5

91.6

57.4

74.7

741

20Te

ache

r ret

entio

n 20

17 to

201

9.5

01.4

80.2

73.2

94-.2

33-.3

92.5

22.5

52.5

52.3

70.2

03.2

70-.1

83.3

58.2

70.3

30.3

10.3

84.6

64

Page 43

Aspect of school professional culture βStd.

Error Lower UpperWald Chi-

Square df Sig.ELA results

Feedback accuracy 0.19 0.06 0.07 0.30 9.89 1 0.002Feedback usefulness 0.12 0.07 -0.02 0.26 2.72 1 0.099Opportunity to use feedback 0.20 0.06 0.07 0.32 9.34 1 0.002

Use of feedback -0.01 0.08 -0.18 0.16 0.02 1 0.903Qualifications of evaluator 0.17 0.06 0.05 0.30 7.74 1 0.005Principal leadership 0.15 0.05 0.05 0.26 8.15 1 0.004Principal trust 0.15 0.05 0.05 0.26 7.90 1 0.005Teacher collaboration 0.15 0.06 0.04 0.26 6.73 1 0.010Teacher affective job satisfaction 0.17 0.06 0.04 0.29 6.75 1 0.009Teacher school commitment 0.20 0.07 0.07 0.34 8.85 1 0.003Teacher retention (16 to 19) 0.16 0.07 0.03 0.29 5.49 1 0.019

Math resultsFeedback accuracy 0.23 0.06 0.12 0.34 15.88 1 0.000Feedback usefulness 0.18 0.07 0.04 0.32 6.53 1 0.011Opportunity to use feedback 0.23 0.06 0.11 0.36 13.52 1 <.001Use of feedback -0.05 0.09 -0.22 0.12 0.31 1 0.577Qualifications of evaluator 0.22 0.06 0.11 0.34 13.64 1 <.001Principal leadership 0.23 0.05 0.13 0.33 21.09 1 <.001Principal trust 0.22 0.05 0.12 0.32 18.00 1 <.001Teacher collaboration 0.18 0.06 0.07 0.30 10.01 1 0.002Teacher affective job satisfaction 0.17 0.07 0.05 0.30 7.08 1 0.008Teacher school commitment 0.26 0.07 0.14 0.39 16.17 1 <.001Teacher retention (16 to 19) 0.12 0.07 -0.02 0.25 2.81 1 0.093

Separate model results for aspects of school professional culture predicting changes in overall school achievement for Latinx students

Page 44

12

34

56

78

910

1112

1314

1516

1718

19

120

15-1

6 EL

A a

chie

vem

ent

1

220

15-1

6 M

ath

achi

evem

ent

.857

1

3Te

ache

r ret

entio

n in

201

5-16

.237

.404

1

4Sc

hool

size

.106

.074

-.035

1

5St

uden

ts w

/ dis

abili

ty %

-.497

-.522

-.132

-.208

1

6St

uden

t eco

n di

sadv

%-.7

04-.7

78-.2

80-.0

8.4

891

7W

hite

stud

ent %

.461

.599

.473

-.311

-.248

-.707

1

820

18-1

9 EL

A A

chie

vem

ent

.871

.809

.289

.106

-.553

-.761

.500

1

920

18-1

9 M

ath

Ach

ieve

men

t.8

10.8

75.3

68-.0

02-.4

94-.7

72.6

40.9

041

10Fe

edba

ck a

ccur

acy

.239

.187

-.002

-.235

-.166

-.116

.216

.234

.224

1

11Fe

edba

ck u

sefu

lnes

s.0

57-.0

7-.1

54-.1

78.0

19.1

4-.0

78.0

53-.0

45.7

091

12O

ppor

tuni

ty to

use

feed

back

-.016

-.082

-.191

-.135

.024

.128

-.096

-.003

-.017

.618

.740

1

13Fe

edba

ck u

se

-.204

-.224

-.257

-.176

.097

.279

-.147

-.205

-.201

.311

.585

.629

1

14Q

ualifi

catio

ns o

f eva

luat

or.2

00.1

12-.0

24-.2

47-.1

21-.0

64.1

56.1

93.1

68.8

85.7

35.6

62.3

771

15Pr

inci

pal l

eade

rshi

p.2

17.1

49.0

15-.1

91-.1

08-.0

36.0

92.2

18.2

06.7

83.7

36.6

95.3

85.8

341

16Pr

inci

pal t

rust

.223

.154

.059

-.19

-.105

-.033

.133

.221

.207

.793

.672

.669

.349

.822

.924

1

17Te

ache

r col

labo

ratio

n.2

68.2

22-.0

93.0

97-.0

93-.1

79-.0

43.3

47.2

96.3

91.4

70.5

83.2

54.3

77.5

16.4

331

18Te

ache

r affe

ctiv

e jo

b sa

tisfa

ctio

n.4

16.3

190.

08-.1

45-.1

95-.2

84.3

00.4

23.3

80.5

41.4

39.4

31.2

30.4

81.5

90.6

07.4

231

19Te

ache

r sch

ool c

omm

itmen

t .3

92.2

11-.0

52-.1

53-.1

62-.1

05.0

33.3

83.2

83.6

16.6

12.5

68.2

79.5

71.6

78.6

92.5

16.7

991

20Te

ache

r ret

entio

n 20

17 to

201

9.2

14.1

85.1

4.1

37-.0

17-.1

38-.0

35.2

35.1

22.1

18.0

74.0

12-.1

8.0

47.0

33.0

98.0

99.1

05.3

57

Page 45

School Organizational Condition βStd.

Error Lower UpperWald Chi-

Square df Sig.ELA results

Feedback accuracy -0.01 0.04 -0.09 0.06 0.11 1 0.742Feedback usefulness 0.02 0.04 -0.05 0.09 0.24 1 0.624Opportunity to use feedback 0.02 0.04 -0.05 0.09 0.38 1 0.539

Use of feedback -0.01 0.04 -0.10 0.08 0.04 1 0.835Qualifications of evaluator 0.00 0.04 -0.07 0.07 0.00 1 0.992Principal leadership 0.00 0.04 -0.07 0.07 0.00 1 0.976Principal trust 0.00 0.04 -0.07 0.07 0.01 1 0.906Teacher collaboration 0.02 0.04 -0.05 0.09 0.40 1 0.528Teacher affective job satisfaction 0.03 0.04 -0.05 0.11 0.48 1 0.489Teacher school commitment 0.08 0.05 -0.01 0.17 3.33 1 0.068Teacher retention (16 to 19) 0.04 0.04 -0.04 0.12 1.14 1 0.286

Math resultsFeedback accuracy -0.07 0.04 -0.14 0.01 3.26 1 0.071Feedback usefulness -0.05 0.04 -0.12 0.03 1.57 1 0.211Opportunity to use feedback -0.02 0.04 -0.09 0.05 0.31 1 0.575Use of feedback -0.05 0.04 -0.13 0.04 1.26 1 0.262Qualifications of evaluator -0.04 0.04 -0.11 0.03 1.10 1 0.294Principal leadership -0.04 0.04 -0.11 0.03 1.10 1 0.294Principal trust -0.04 0.04 -0.10 0.03 0.99 1 0.319Teacher collaboration -0.04 0.04 -0.11 0.03 1.11 1 0.293Teacher affective job satisfaction -0.03 0.04 -0.11 0.05 0.48 1 0.491Teacher school commitment 0.00 0.05 -0.09 0.09 0.00 1 0.969Teacher retention (16 to 19) 0.00 0.04 -0.08 0.08 0.00 1 0.976

Separate model results for school organizational conditions predicting changes in overall school achievement for White students

Page 46

12

34

56

78

910

1112

1314

1516

1718

19

120

15-1

6 EL

A a

chie

vem

ent

1

220

15-1

6 M

ath

achi

evem

ent

.857

1

3Te

ache

r ret

entio

n in

201

5-16

.237

.404

1

4Sc

hool

size

.106

.074

-.035

1

5St

uden

ts w

/ dis

abili

ty %

-.497

-.522

-.132

-.208

1

6St

uden

t eco

n di

sadv

%-.7

04-.7

78-.2

80-.0

8.4

891

7W

hite

stud

ent %

.461

.599

.473

-.311

-.248

-.707

1

820

18-1

9 EL

A A

chie

vem

ent

.871

.809

.289

.106

-.553

-.761

.500

1

920

18-1

9 M

ath

Ach

ieve

men

t.8

10.8

75.3

68-.0

02-.4

94-.7

72.6

40.9

041

10Fe

edba

ck a

ccur

acy

.239

.187

-.002

-.235

-.166

-.116

.216

.234

.224

1

11Fe

edba

ck u

sefu

lnes

s.0

57-.0

7-.1

54-.1

78.0

19.1

4-.0

78.0

53-.0

45.7

091

12O

ppor

tuni

ty to

use

feed

back

-.016

-.082

-.191

-.135

.024

.128

-.096

-.003

-.017

.618

.740

1

13Fe

edba

ck u

se

-.204

-.224

-.257

-.176

.097

.279

-.147

-.205

-.201

.311

.585

.629

1

14Q

ualifi

catio

ns o

f eva

luat

or.2

00.1

12-.0

24-.2

47-.1

21-.0

64.1

56.1

93.1

68.8

85.7

35.6

62.3

771

15Pr

inci

pal l

eade

rshi

p.2

17.1

49.0

15-.1

91-.1

08-.0

36.0

92.2

18.2

06.7

83.7

36.6

95.3

85.8

341

16Pr

inci

pal t

rust

.223

.154

.059

-.19

-.105

-.033

.133

.221

.207

.793

.672

.669

.349

.822

.924

1

17Te

ache

r col

labo

ratio

n.2

68.2

22-.0

93.0

97-.0

93-.1

79-.0

43.3

47.2

96.3

91.4

70.5

83.2

54.3

77.5

16.4

331

18Te

ache

r affe

ctiv

e jo

b sa

tisfa

ctio

n.4

16.3

190.

08-.1

45-.1

95-.2

84.3

00.4

23.3

80.5

41.4

39.4

31.2

30.4

81.5

90.6

07.4

231

19Te

ache

r sch

ool c

omm

itmen

t .3

92.2

11-.0

52-.1

53-.1

62-.1

05.0

33.3

83.2

83.6

16.6

12.5

68.2

79.5

71.6

78.6

92.5

16.7

991

20Te

ache

r ret

entio

n 20

17 to

201

9.2

14.1

85.1

4.1

37-.0

17-.1

38-.0

35.2

35.1

22.1

18.0

74.0

12-.1

8.0

47.0

33.0

98.0

99.1

05.3

57

Page 47

The table below presents Cohen's U3 as a function of the standardized effect size difference for two groups. Cohen's U3 is the percent of one group that is above the mean of another group. For example, an effect size difference of .5 suggests 69.2% of one group is above the mean of the other group.

Cohen's d (effect size of difference)

Cohen's U3 (percent of group that is above the mean of another group)

.1 54.0%

.2 57.9%

.3 61.8%

.4 65.5%

.5 69.2%

.6 72.6%

.7 75.8%

.8 78.8%

.9 81.6%

1.0 84.1%

1.1 86.4%

1.2 88.5%

1.3 90.3%

1.4 91.9%

1.5 93.3%

1.6 94.5%

1.7 95.5%

1.8 96.4%

1.9 97.1%

2.0 97.7%

Effect Size Percentile Conversion TableEffect Size Percentile Conversion Table

Page 48