Instructional Culture Insight - TNTP · / 2 •Introduce the Insight survey school report, including definitions of the Index and domain scores; •Introduce the theory of local exemplars

© TNTP 2013

Reading Your Instructional Culture Insight Report Planning in response to Instructional Culture Insight results Spring 2013

/ 2

• Introduce the Insight survey school report, including definitions of the Index and domain scores;

• Introduce the theory of local exemplars as a strategy for school improvement;

• Demonstrate a proven strategy for using an Insight report to develop a responsive action plan.

Presentation Goals

Presenter

Presentation Notes

For Insight to realize its potential as a principal development and school support tool, it will require support from everyone in this room. The goal of our session today is to make sure you are well equipped and prepared to use Insight reports to support principals and to answer any questions they may have. To that end, by the end of our 90 minutes …

/ 3

How Insight Works o The importance of measuring instructional culture o Insight Index and domain scores o Theory of Positive Deviance

Anatomy of an Insight Report

o Choosing focus areas based on results

o Domain sequencing and prioritization

o Creating an action plan

Reading your Report to Plan for Change

o Introduction to the school sorter

o Identifying zone-level development needs and action planning

Presentation Agenda

1

2

3

/ 4

How Insight Works



Agenda

/ 5

Schools with strong instructional cultures retain more of their effective teachers and achieve greater academic success with students.

Instructional Culture Matters for Teachers and Students

Presenter

Presentation Notes

Culture matters – to teachers and to students – and it can be measured. In our research using Insight since 2010 we’ve seen two outcomes consistently correlation with culture: students performance outcomes and the retention of effective teachers.

/ 6

Insight is a tool to collect and aggregate teacher feedback to help principals set priorities around instructional culture.

Validated survey gathers teacher feedback on critical

aspects of school culture

1. SURVEY TEACHERS TNTP identifies strong local performers, assesses school

culture and shares this upward feedback with leaders

2. SHARE FEEDBACK

School Reports and Data Debriefs

Survey results provide information to prioritize and

plan necessary actions to improve culture

3. IMPROVE SCHOOL CULTURE

Insight Survey

Insight surveys happen twice per year, giving principals opportunities to monitor progress before the end of the year.

Presenter

Presentation Notes

Insight’s value is in the cycle of receiving feedback from teachers, reviewing results, planning and taking action to improve based on those results, and then reassessing.

/ 7

TNTP’s Instructional Culture Index is a validated, summative measure of a school’s instructional culture which can be used to compare schools. Topics, and items within those topics, identify levers for future improvement.

Presenter

Presentation Notes

Most teachers spend about 15 minutes on the survey. The survey is about 45 items in length, and we report them across 11 areas or domains of talent management. We also use three, pulled out on the right, to create the Instructional Culture Index, a validated summative measure of school culture, with positive correlation to student performance outcomes and the retention of effective teachers. In our analysis and reporting, we use the Index to compare schools and to identify what we call “top-quartile schools” – those with Index scores in the top 25% in a given region or district. Once those schools are identified, the topics and items within the survey allow schools to compare themselves to those top-performers, and to learn from their strength.

/ 8

• Within any community some individuals find solutions to shared challenges, even while facing the same limitations and working with the same resources.

• Positive Deviance focuses on identifying these individuals and isolating the practices that set them apart.

• Schools identified as “top-quartile” possess the strongest instructional cultures in the district. Insight reports make comparisons to these schools, so we can learn from and be motivated by their successes.

Positive Deviance: Learning from local exemplars to achieve transformative progress

What practices are different at this school?

Presenter

Presentation Notes

Positive Deviance as a theory of social chance posits that in every community there are individuals or groups whose uncommon behaviors and strategies enable them to find better solutions to problems than their peers, while having access to the same resources and facing similar. The Positive Deviance approach is an asset-based, problem-solving, and community-driven approach that enables the community to discover these successful behaviors and strategies and develop a plan of action to promote their adoption by all concerned. (http://www.positivedeviance.org) Book suggestion: Better by Atul Gawande

/ 9

How the Insight cycle works in schools: A cycle of continuous improvement

MEASURE

ASSESS

PLAN

ACT

1

2

3

4

Insight administration, Spring 2013

School leaders review results to identify areas of strength and opportunity

School leaders develop action plans to address areas of greatest need

Schools execute steps from action plans, drawing support from local exemplars, district resources and TNTP’s Teacher Talent Toolbox

Presenter

Presentation Notes

Making change in a school’s instructional culture takes time. By using Insight results to focus on priorities, taking action on those priorities, and then assessing results and repeating, school leaders can create meaningful change in their schools.

/ 10

How Insight Works



Agenda

/ 11

The first page of the report starts with the school’s Instructional Culture Index.* Insight reports use the Index to compare schools and identify local exemplars.

The Index is on a 1-10 scale, calculated according to the % of teachers agreeing with three items: • Teachers at my school

share a common vision. • My school provides a

clear expectation. • My school is committed

to my improvement.

* The Index is an independently validated predictor of student performance and effective teacher retention.

Presenter

Presentation Notes

The Index is reported on a scale of 1-10; we use this as a comparative measure among schools because it is predictive of student performance and teacher retention. In addition to these correlations to important outcomes, the Index is also universal in its comparisons: it includes teachers in all subjects and grades, both tested and untested by state assessments. The Index is also the backbone of our reporting structure: throughout the report you will see mention of “top-quartile” schools, which always refers to the schools with the top 25% of index scores in the district. Let’s remember that the Index is summative – it tells us how a school’s doing overall, and the rest of the survey points to specific areas of strength and opportunity…so we can maintain and build aspects of a strong culture. Let’s dig into the next level, of how the entire survey provides direction, first by helping to identify specific areas for development.

/ 12

The first page also presents an overview to the school’s performance on ten domains of instructional culture. Domain scores summarize the individual survey items included within each domain.

Top-quartile comparisons are the difference between your school’s scores and schools with the top 25% of Index scores in the district: these are the local exemplars we want to learn from.

Historical scores present the domain scores from the last administration of Insight.

Presenter

Presentation Notes

The domain scores in this table are reported on a 1-10 scale, and summarize all of the questions within that topic. Most importantly, let’s look at the right-most column. This shows the difference between my school and the average of the strongest 25% of schools in my district; my report shows that my Index is 3.2 below the average of those schools, and I’m 1.5 below that average in O&F. We provide this comparison because of our belief in the power of positive deviance: by looking at the performance of local exemplars, having the same resources and facing the same challenges, we can see what’s possible and desirable. When we survey in CMS again this fall, we’ll be able to provide historical comparison data, which will show change, ideally growth, since this current administration.

/ 13

Teacher responses to individual items follow on pages 3-7, organized by domain in the order they appear in the summary table.

Percentages represent the percent of teachers who “Agree” or “Strongly Agree” with each item.

Presenter

Presentation Notes

Digging into the rest of the report, we’ll see teacher agreement rates with individual items, such as this shot featuring learning environment. The column on the left shows school-level data; the middle column is the average of all schools in the district, and the right column is the average of the strongest 25% of schools, as measured by the Index.

/ 14

How Insight Works



Agenda

/ 15

Reading your report: a focused process

Where are you?

• Orient: How is your school doing overall? • Assess: What are your areas of strength and opportunity?

Where do you want to

be?

• Prioritize: What are your school’s most important needs? • Choose a destination: What will success look like?

How do you get there?

• Chart a course: What do you need to do to achieve success? • Gather your supplies: What resources will you need to

succeed?

Presenter

Presentation Notes

Reading survey results can be challenging for people, esp when it represents feedback on our work. I spent several years as a new teacher trainer for TNTP, and three times each season, we’d survey our new teachers on the value of the support they received, and I remember distinctly the challenge of focusing in on specific areas for improvement, as well as being objective about the results – not taking it personally. The process we’ve developed for reviewing Insight has proven helpful for addressing that need for an objective focus.

/ 16

Orient: How is this school doing overall?

“My school’s Index score of 6.3 and 67th percentile rank means that I scored higher than two-thirds of schools in my distict. “My highest performing teachers experience a weaker culture than their lower-performing peers.”

Presenter

Presentation Notes

First, it’s helpful to note how my school is performing in relationship to my peers, and also how my higher vs lower performing teachers are experiencing culture at my school.

/ 17

After assessing a school’s overall performance, the next step is to determine a particular domain of focus.

There are three things to consider when choosing a domain of focus:

Domains with scores lagging behind-top quartile values

Areas of focus within your school

Domains which will have the greatest impact

1

2

3

Presenter

Presentation Notes

Top-quartile schools are those with the strongest cultures; in the theory of positive deviance for social change, we want to consider how we can learn from their behaviors. It’s also vital to consider the things you’re already working on in your school, and taking the opportunity to reflect on your past progress. Finally, not all domains are of equal value: some domains are foundational to others…

/ 18

Insight Index

Compensation Hiring

Talent management domains build on each other: foundational domains provide the groundwork for successful implementation of others.

3

2

1

Learning Environment

Observation & Feedback

Student Growth

Measures

Professional Development

Retain Top Talent

Career Progression

Workload

Peer Culture

Evaluation

Planning

Presenter

Presentation Notes

This graphic is the result of research by the Insight team into our data set, with the research question “when a school needs to improve their index score, which domain should be the focal point?” The answer, of course, is “it depends” – and it’s pretty logical. For example, let’s say a school needs to improve in both evaluation and observation & feedback, two domains which are clearly linked. I believe that obs&feedback is the place to start – as a classroom teacher, I remember the evaluation process…which included a single observation as a data point. I never put much faith into that evaluation – and every rating was satifactory at least – which is good, because without active coaching in the form of feedback, I’d have been really unsatisfied with a poor eval if there’d never been effort taken to help me improve in my areas of need.

/ 19

Assess: What are this school’s areas of strength and opportunity?

Analyzing Results

“Our biggest gaps are in Learning Environment, Observation, and PD.

“Learning Environment is the most foundational competency and it also has the biggest difference, so let’s focus our efforts there first.

“That makes sense to me. I’d also like to make progress on Professional Development, but let’s see what resources we’ll need for each action plan and if we can try to make progress on both.”

When choosing a domain of focus, prioritize domains which are foundational to others.

“We’re roughly in line with the strongest schools in using Student Growth Measures.

Presenter

Presentation Notes

And then we dig into specific areas of strength and opportunity to pick a single domain to focus on; remember, by narrowing our vision, we can get more specific when we make plans for positive change.

/ 20

Prioritize: What are this school’s specific needs?

Reflecting on Survey Items

“We’re low in Learning Environment overall, but items #3, 4 and 5 are the lowest, and the furthest from top-quartile schools.”

“New teachers this year didn’t receive much guidance on developing management systems, and some are over-using office referrals to manage their classrooms. “We’ve also had concerns from teachers regarding behavior during lunch, which makes afternoons more challenging for students."

Considering the School’s Context

Survey items are most meaningful when triangulated with insider knowledge of a school’s context.

Presenter

Presentation Notes

By reviewing individual questions, we can determine where a school needs to put their focus; it’s also valuable/essential to include details of the school’s context. This way, we get a complete picture of what’s going on, and also encourage school leaders to engage as actively as possible with the review of results. When it comes to taking action, partnering with and supporting a school leader is essential.

/ 21

Reading multiple data points simultaneously

1

2

3

2

Using Multiple Data Point

This item’s agreement rate is low, but… …when we look at the overall indicators of successful observation and feedback cycles, teachers are responding positively. In fact, this school uses a mix of meetings and emails to deliver feedback following observations. A better place for this school to put their focus is on providing supports to help teachers be successful with action steps, and following up after the feedback is delivered to ensure successful implementation of action steps.

1

2

3

Presenter

Presentation Notes

It can be easy to get overly focused on low values in school results, for example, the item labeled 1 here. Less than half of teachers report having scheduled appointments to receive feedback; it’s worth asking the question – “is this really a problem?” When we look at other items, labeled with 2s, we see that teachers generally feel they get enough feedback, frequent feedback, and helpful feedback. Given these data points, focusing on more feedback meetings, as opposed to emails or quick conversations in the hallway, may not be the best area of focus. Rather, looking at the items labeled with a 3, providing supports to teachers to improve in their action areas and following up to check on successful implementation of those action areas.

/ 22

Choose a destination: What will success look like?

Defining Success

“We’ll know we’re successful when:

“Teachers write fewer office referrals, especially those who write the most.

“Teachers use more effective behavior management strategies in their classrooms, including positive-reinforcement and structured transitions, to prevent misbehavior before it starts.

“All staff are present and reinforce behavioral expectations in common areas (breakfast, lunch, dismissals, hallways, etc.).

“At the next administration, at least 60% of teachers agree with these three survey questions.”

Defining specific successes in improving instructional culture is essential for establishing a vision for reaching those goals and communicating direction to other stakeholders.

Presenter

Presentation Notes

Building from the results and from the school’s context, we set a vision of success. Some leaders will be motivated by setting a numerical goal for the next Insight administration, but I also suggest using other, internally measurable goals. The first three goals listed above can be monitored on a monthly or quarterly basis, and efforts can be made if the school falls off track.

/ 23

Chart a course: What do you need to do to achieve success?

Success looks like: We need to: Teachers writing fewer office referrals, especially those who write the most.

• Provide targeted development for these teachers

• Set goals with these teachers for reduction of office referrals

Teachers using more effective management strategies in their classrooms.

• Review resources from other elem. schools

• Set up focused peer observations • Implement short observation rubric

focused on positive reinforcement and structured transitions

All staff are present and reinforce behavioral expectations in common areas (breakfast, lunch, dismissals, hallways, etc.).

• Cultural reset: teach all students and teachers our expectations for all areas in the school

• Communicate expectations and rationale regarding common areas to all teachers

Presenter

Presentation Notes

It’s essential to develop specific actions for school leaders to take to impact the school’s culture; the next step from here is even to assign individuals to own components of the work.

/ 24

Gather your Supplies: Take advantage of external resources

MEETING STUDENT ACHIEVEMENT GOALS • Instructional Planning • Using Student Growth Measures

DEVELOPMENT & EVALUATION • Observation and Feedback • Professional Development • Teacher Evaluation

RECRUITMENT & RETENTION • Hiring • Teacher Career Progression • Retaining Top Talent

WORKING CONDITIONS • Learning Environment • Peer Culture and Collaboration • Managing Teacher Workload

The Toolbox offers several unique services, free and available to all: • Open access to the Teacher Talent

Toolbox website, including toolkits that outline promising practices in many areas of Talent Management.

• More than 250 downloadable and customizable resources from innovative schools, networks and districts.

• TNTP's latest research on what teachers value in the workplace.

http://www.tntp.org/toolbox

/ 25

tntp.org and tntp.org/toolbox

facebook.com/thenewteacherproject

twitter.com/tntp

Connect with us.

[email protected]

Documents

Instructional Culture Insight - TNTP · / 2 •Introduce the Insight survey school report, including definitions of the Index and domain scores; •Introduce the theory of local exemplars