Data Culture Survey

Analysis

August 2014

1

• Understand why we are conducting data culture surveys

• Analyze district and school results of the data culture survey and plan for implications at your school

• Provide feedback to the data culture team on the goals and direction moving forward

2

Objectives

• The 2020 Denver Plan calls for dramatic acceleration of student learning and growth with specific goals around: – College and Career Readiness– School Readiness – Eliminating the opportunity gaps – Support for the whole child

• The district believes that a focus on data driven instruction is a key lever for meeting the goals of the Denver Plan

3

Purpose

• In Leverage Leadership and Driven by Data Paul Bambrick-Santoyo states that a key principle of data driven instruction is that “Effective instruction is not about whether we taught it. It’s about whether the students learned it.” DDI includes creating a culture of using assessment and data to drive action and instruction.

• He presents numerous case studies of the dramatic student achievement gains (often higher than 15%) by focusing on implementing data-driven instructional systems and culture.

4

Why Data-Driven Instruction?

Informing our Improvement Strategy:Focus on Data Driven Instruction

• Hypothesis: Deep implementation of data driven instruction will accelerate student achievement in all categories

• Implementation:– RELAY– DDI district initiatives (SCAN, etc)– Data Culture Framework– Student Learning Objectives (SLOs)– Professional Learning Days (green/blue days)

• Initial findings:– Promising results in schools that have shown exemplary data driven

practice.

5

Initial Results: Growth Among Schools Engaged in Data Driven Instruction

DSST: Green Val-ley Ranch HS

DSST: Stapleton MS

Force Elementary School

Grant Beacon Middle School

High-Tech Early College

Skinner Middle School

STRIVE Prep - Federal

0

50

100

150

200

250

65 6957 60 57 50 49

9378.5

52

71

4964

49

68 7465

5749

57 60

226 221.5

174188

155171

158

TCAP MGP for Data-Driven SchoolsReading Math Writing Sum

6

DPS: 163

7

Monitoring our Progress from 2012

• Create a vision for data inquiry at the district, school, and classroom level•Align tools, resources, and

systems to support effective data use practices• Build capacity of school

leaders in utilizing data inquiry

What were we trying to learn by conducting the Data Culture Survey?

Questions we were trying to answer:

• In what areas—if any—has the district (teachers, schools, and the central office) made progress in the implementation of effective data use practices?• How—if at all—have understandings of effective data use

practices shifted since the prior survey administration?• In what areas is there room for improving data use practices

at the classroom, school, and central office levels?• How can the district invest its resources to address areas for

improvement and to build upon areas of success?

Objective: To develop a reliable picture of growth and gaps in the district’s data culture in order to prioritize professional development and other supports to promote adoption of effective data use practices

8

•Teachers 59%•School Leaders: 90%•Central Office Personnel: 72%

9

Participation Rates

10

Analyze: Celebrations

Identify Student Strengths ...

Provide Timely Descriptive F...

Make Instructional Decisions

Collaberation with Other Tea...

Form

ative

Ass

essm

ents

Inte

rim A

sses

smen

ts

87%

68%

85%

29%

91%

74%

90%

37%

Assessment Use

2014 2012

11

Analyze: Celebrations

Very Familiar Not Familiar at All

92%

79%73%

58%

Impact of Data Inquiry Cycle Knowledge on Implementa-tion

Formative AssessmentInterim Assessment

12

Analyze: Celebrations

31%

21%

16%

15%

14%

4%

Access to Data Inquiry Cycle

School LeadersDistrict TrainingDistrict Resources (Standards Toolkit)TeachersCoachesCentral Office Per-sonnel and Iss

13

Analyze: Celebrations

1 2

69%

82%

School Leader Awareness of DPS Data Inquiry Cycle

2012 2014

14

Analyze: Celebrations

School-Level PD District-Level PD

91%

85%

99%

94%

Professional Development for Effective Data UseTeachersSchool Leaders

15

Analyze: Identified Gaps

“How often do you examine formative data?”

“How often do you examine interim data?”

2012 2014

Rarely 5% 3%

Monthly 19% 16%

Weekly 44% 46%

Daily 33% 35%

2012 2014

Rarely 8% 9%

After some units

22% 21%

After most units

29% 28%

After all units 42% 42%

While teachers report increases in engagement in most data use practices, formative, interim and summative data frequency remains unchanged.

16

Analyze: Identified Gaps

1 2

56% 56%

Familiarity with DPS Data Inquiry Cycle

2012 2014

School leader and teacher familiarity with the inquiry cycle and access of the data inquiry toolkit varies across networks.

Familiarity with the inquiry cycle varies greatly from one network to the next:

School LeaderVery or Moderately Familiar: 74% to 100%

TeacherVery or Moderately Familiar: 28% to 49%

Use of the Data Inquiry Toolkit to access resources to support data use varies across networks.

Among school leaders, this percentage

varies from 14% to 79% depending on the network.

Among teachers, this percentage varies

from 7% to 24% depending on the network.

17

Analyze: Identified Gaps

18

Analyze: Identified Gaps

Teachers School Leaders Central Office Personnel

20%

43%

25%

Accessing Data Inquiry Toolkit

19

Analyze: Identified Gaps

Central Office Personnel School Leaders Teachers

50%

44%

31%

Common, Clear, and Consistent Vision for Data Decision-Driven

Series1

Analyze: Summary of findings from the survey

Teachers are slightly more engaged in formative, interim, and summative data use practices. Frequency of data use has not increased since 2012.

Awareness and knowledge of the DPS inquiry cycle and tools varies across the district.

Despite district articulation of an inquiry cycle, there remains a lack of clarity about how data will be used at the school level.

A significant majority of teachers report that existing school- and district-level PD has contributed to their ability to effectively use data.

1

2

3

4

20

2012 Area of Focus Rating Evidence

Create a vision for data inquiry at the district, school, and classroom level

While we have created the DPS Data Inquiry Cycle, it alone is not a vision for Data Culture. We will need to further clarify Data Culture Vision and expectations for data use practices.

Align tools, resources, and systems to support effective data use practices

The knowledge of the cycle and available resources varies across DPS. We will need to expand the availability of resources and continue to focus on aligning systems and tools to support effective practices.

Build capacity of school leaders in utilizing data inquiry

While school leaders have reported an increase in data use practices, teachers have not reported an increase. We will need to expand the audience and the availability of professional learning to continue to build capacity.

21

Analyze: Are we on track?

22

Plan

2012 Area of Focus Rating 2014-2015 Action Steps

Create a vision for data inquiry at the district, school, and classroom level

The development and use of the Data Culture Framework will communicate clear expectations for teacher and school leaders to engage in effective data use practices as well as provide a roadmap for implementation and building/enhancing data culture and assessment practices.

23

Plan

2012 Area of Focus Rating 2014-2015 Action Steps

Further promote awareness of data inquiry resources, tools, and expectations.

The Standards Toolkit can be better leveraged by leadership in order to support its use by teachers.

Expanding the School Data Culture Team creates an opportunity to provide differentiated support to network teams and TECs, build capacity for implementing data practices, and the use of available resources.

24

Plan

2012 Area of Focus Rating 2014-2015 Action Steps

Enhance professional learning that targets a variety of school skill sets across the district and includes opportunities for collaborative data analysis, instructionally-relevant content, and time for participants to apply skills.

The School Data Culture Team provides differentiated support and opportunities for professional learning, including the tier 1 and tier 2 schools in the Foundations of School Data Culture Standards Pathway.

Professional learning opportunities will move from optional sessions for school leaders only, to a more robust menu of options including network teams, coaches, teacher leaders, teachers through online and blended learning.

• Implement– Data Culture Framework – Blue/Green Day Guidance – Professional Learning: affinity groups, teacher leader cohorts, TEC’s,

school leaders’ course catalogue, online modules, networks and partners– Foundations of School Data Culture Standards Pathway

• Evaluate– Professional Learning Surveys – Blue/Green Day Surveys – Data Culture Framework Results – Data Culture Survey (Spring of 2015) – Observations, interviews, and focus groups

• Adjust – After fall and winter interims – After spring break – Summer 2015 25

Implement, Evaluate, Adjust: Plan for Monitoring Progress

• Analyze– Identify celebrations and gaps – Determine/brainstorm root cause

• Plan – To address gaps – Utilize tools and resources found on the standards

toolkit • Implement, Evaluate, and Adjust – How will you monitor progress? (consider systems in

place like your UIP and the data culture rubric)

26

Practice: Analyze your network or school’s results from the data culture survey

• Script what you’ll say using the template • Role play with a partner • Use cheat sheet to provide feedback • Redo implementing the feedback you’ve

received

27

Practice: Role play presenting findings (either district or school) to your staff

Appendix

28

Formative Data Use• To identify strengths and weaknesses in student learning (in both specific content areas as well as English language

development for English language leaners)• To provide timely descriptive feedback to students• To make instructional decisions

Interim Data Use• To evaluate student performance• To modify instruction for my whole class, small groups, and individual students• To monitor student progress towards content standards mastery• To identify student-specific strengths and weaknesses (in both specific content areas as well as English language

development for English language leaners) against a defined set of content standards• To set goals for student learning• In comparison with other teachers’ students to identify successful strategies and make changes to instruction to

improve student learning

Summative Data Use• To evaluate student performance (in both specific content areas as well as English language development for

English language learners) against a defined set of content standards• Of outgoing students to monitor and assess student progress towards goals• Of outgoing students to inform how I can improve my teaching for the upcoming year• Of incoming students to determine how to pace classroom instruction• Of incoming students to determine how to group students to maximize their learning

APPENDIX: Teacher Data Use Practices by Category

29

Includes using data to:► Establish a baseline of student

performance► Identify strengths and weaknesses in

student learning► Determine instructional needs► Inform student placements and

interventions► Allocate resources for reteaching content► Monitor alignment of the written

curriculum, classroom instruction, and student assessment

► Develop goals and relevant action items to be documented in the UIP

► Monitor the success of school improvement efforts

APPENDIX: School Leader Data Use Practices

30