S.T.A.T. Year Three Evaluation - BoardDocs, a Diligent Brand...Professional Development •S.T.A.T....

S.T.A.T. Year Three Evaluation

Dr. Jennifer R. Morrison

Dr. Steven M. RossAugust 8, 2017

S.T.A.T. Evaluation Model

Professional

Development Measurable Outcomes Goals

Classroom

Environment

Teacher

Practice

Digital

Content

Administrators

Classroom

Teachers

S.T.A.T.

Teachers

Student

Engagement

P21 Skills

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Year 1+

Year 2+

Years 3/4+

Year 1+

8/8/2017 2

Data Sources

• Interviews and Focus Groups (principals, S.T.A.T. teachers, classroom teachers)

• Classroom Teacher Survey (CRRE survey)

• Classroom observations in schools (OASIS-21 Instrument)

• Student behavioral data

• MAP data (Grades 1-3)

• S.T.A.T.-specific climate survey items (BCPS survey)

8/8/2017 3

S.T.A.T. Experience

2015-2016 2016-20172014-2015

Cohort 1:

LH Grades 1-3

Cohort 2:

LH Grades K, 4, 5

Ph 2 Grades 1-3

LH Grade 6

Cohort 3:

Ph 2 Grades K, 4, 5

Ph 2 Grade 6

LH Grade 7

LH Grades 9-1248/8/2017

Preview of Year Three Results

• Research on school-district technology integration initiatives shows1:– Higher student engagement

– Increases in student-centered instruction

– Improved student achievement

• Third-year results in BCPS show:– Continued changes from teacher- to student-centered learning

– Shifts to teacher coaching rather than presenting

– Focus on using data to customize instruction

– Strong impact on student engagement

1 Zheng, B., Warschauer, M., Lin, C., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-

analysis and research synthesis. Review of Educational Research. 8/8/2017 5

S.T.A.T. Evaluation Model

Professional

Development Measurable Outcomes Goals

Classroom

Environment

Teacher

Practice

Digital

Content

Administrators

Classroom

Teachers

S.T.A.T.

Teachers

Student

Engagement

P21 Skills

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Year 1+Year 1+

Year 2+

Year 3+

8/8/2017 6

Professional Development

• S.T.A.T. teachers continued to be viewed as critical to

implementation

• Principals viewed as supportive of teachers’

implementation

• Teacher collaboration and sharing

• Need clearly defined roles of the S.T.A.T. teacher

• Additional PD on using tools and programs, P21 skills

8/8/2017 7

S.T.A.T. Evaluation Model

Professional

Development Measurable Outcomes Goals

Classroom

Environment

Teacher

Practice

Digital

Content

Administrators

Classroom

Teachers

S.T.A.T.

Teachers

Student

Engagement

P21 Skills

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Year 1+Year 1+

Year 2+

Year 3+

8/8/2017 8

Observation Rating Scales

• Not observed: Not observed in class

• Rarely: Received little emphasis/time in class

• Somewhat/Occasionally: Receives modest emphasis/time

in class

• Frequently: Receives substantial emphasis/time in class

• Extensively: Highly prevalent in class

8/8/2017 9

Impact on Classroom Environment

8/8/2017 10

9.4%

+

6.5%

40.6%

63.2%

53.2%

50.0%

32.4%

40.3%

Cohort 1

Cohort 2

Cohort 3

Not observed General Subject Lesson Specific

Information and resources that reflect content being taught are visibly displayed

+ < 5.0%

37.5%

47.1%

53.2%

43.8%

38.2%

33.8%

18.8%

14.7%

13.0%

Cohort 1

Cohort 2

Cohort 3

Not Observed Somewhat Extensive

Information and communications that support independent thinking are visible

Summary: Classroom Environment

• Classrooms appear to be consistent with S.T.A.T. goals

• Little change in the physical environment

8/8/2017 11

Impact on Teacher Practices

12

9.4%

22.1%

15.6%

15.6%

13.2%

14.3%

28.1%

23.5%

24.7%

18.8%

19.1%

19.5%

28.1%

22.1%

26.0%

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Teachers acting as coach/facilitator

28.1%

30.9%

44.2%

21.9%

25.0%

19.5%

37.5%

22.1%

22.1%

6.3%

13.2%

11.7%

6.3%

8.8%

+

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Teacher presentation

+ < 5.0%

8/8/2017

Summary: Teacher Practice

• Teachers favoring coaching and facilitating instruction

over presentations

• Increased teacher collaboration, personalizing instruction,

interactive learning

8/8/2017 13

Impact on Digital Content

8/8/2017 14

+

+

8.0%

15.8%

21.1%

30.0%

36.7%

41.6%

62.0%

46.0%

32.5%

Cohort 1

Cohort 2

Cohort 3

Never At least once per month At least once a week At least once a day

Deliver instruction customized to students' needs

+ < 5.0%

72.0%

59.5%

37.3%

8.0%

10.2%

22.2%

8.0%

16.3%

24.8%

12.0%

14.0%

15.7%

Cohort 1

Cohort 2

Cohort 3

Never At least once per month At least once a week At least once a day

Post homework assignments

+ < 5.0%

Summary: Digital Content

• Successes with using technology:

– Research, students taking ownership of learning, increased

engagement

• Challenges with technology integration:

– Inappropriate/off-task use

8/8/2017 15

S.T.A.T. Evaluation Model

Professional

Development Measurable Outcomes Goals

Classroom

Environment

Teacher

Practice

Digital

Content

Administrators

Classroom

Teachers

S.T.A.T.

Teachers

Student

Engagement

P21 Skills

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Year 1+Year 1+

Year 2+

Year 3+

8/8/2017 16

Impact on Student Engagement

8/8/2017 17

28.1%

42.6%

39.0%

37.5%

29.4%

23.4%

18.8%

5.9%

15.6%

15.6%

22.1%

22.1%

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Student use of digital tools for learning

12.5%

16.2%

14.3%

9.4%

14.7%

5.2%

15.6%

23.5%

19.5%

40.6%

17.6%

26.0%

21.9%

27.9%

35.1%

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Independent work

Summary: Student Engagement

• Evidence of student engagement in learning task

• Mostly positive regarding student engagement

• Student behavior

– Mixed views

– Attendance and office referral rates stable, statistically

significant increases in suspension rates

8/8/2017 18

Impact on P21 Skills

8/8/2017 19

90.6%

89.7%

83.1%

+

+

+

+

+

6.3%

5.9

11.7%

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Project-based approaches to instruction

+ < 5.0%

56.3%

67.6%

66.2%

+

8.8%

5.2%

9.4%

10.3%

14.3%

12.5%

7.4%

9.1%

18.8%

5.9%

5.2%

Cohort 1

Cohort 2

Cohort 3

Not Observed Rarely Somewhat/Occasionally Frequently Extensively

Learning incorporates authentic/real world contexts

+ < 5.0%

Summary: Impact on P21 Skills

• Comparable observation results with baseline and across

Cohorts

• Some perceptions of improved P21 skills

• Needed PD

8/8/2017 20

S.T.A.T. Evaluation Model

Professional

Development Measurable Outcomes Goals

Classroom

Environment

Teacher

Practice

Digital

Content

Administrators

Classroom

Teachers

S.T.A.T.

Teachers

Student

Engagement

P21 Skills

Student

Achievement

Graduate Globally

Competitive

Students

MAP

PARCC

Year 1+

Year 2+

Years 3/4+

Year 1+

8/8/2017 21

Student Achievement: Lighthouse

• Mathematics

– Increases in students meeting growth expectations

– Upward trend for Grades 1-2 students

– Grades 1-3 exceeded national average

• Reading/ELA

– Increases in students meeting growth expectations

– Upward trend for Grades 1-2 students

– Grades 1-3 exceeded national average

228/8/2017

Student Achievement: Non-Lighthouse

• Mathematics

– Increases in students meeting growth expectations

– Upward trend for Grades 1-2 students

– Grade 2 exceeded national average

• Reading/ELA

– Increases in students meeting growth expectations

– Upward trend for Grades 1-3 students

– Grades 1-3 exceeded national average

238/8/2017

Perceptions of S.T.A.T.

• Principals, S.T.A.T. teachers, classroom teachers:

– Increases in student engagement, focus on instruction, variety

• Parents and students:

– Vast majority positive towards teachers’ use of technology

8/8/2017 24

Recommendations

• S.T.A.T. teacher roles and responsibilities

• Teacher instructional support

• Technology policies

• Improved communication

8/8/2017 25

Conclusion

• Highly positive perceptions of S.T.A.T. teachers

• S.T.A.T. valued for increases in student engagement, focus on instruction, support for learning

• S.T.A.T. viewed positively by all stakeholders

• Positive achievement trends on MAP

8/8/2017 26