45
Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing Gaps Between Policy and Practice

Implementation Science: Closing Gaps Between Policy and Practice

  • Upload
    santo

  • View
    38

  • Download
    0

Embed Size (px)

DESCRIPTION

Implementation Science: Closing Gaps Between Policy and Practice. Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013. Acknowledgements. - PowerPoint PPT Presentation

Citation preview

Page 1: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Circe Stumbo, West Wind Education Policy Inc. and CCSSO

CCSSO/SCEE National Summit on Educator Effectiveness

April 10, 2013

Implementation Science:

Closing Gaps Between Policy and Practice

Page 2: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Acknowledgements

Special thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission.

2

Page 3: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Sixth Year of Working with SEAs

State Implementation & Scaling-up of Evidence-based Practices Center

scalingup.org

3

Page 4: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Implementation Science`

A policy is one thingImplementation of a policy is a

very different thingStudents cannot benefit from a

policy they do not experience

Page 5: Implementation Science:  Closing Gaps  Between  Policy  and Practice

An Implementation Failure labeled as an Intervention FailureExample: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective

Chapin Hall Center for Children, 2002

Page 6: Implementation Science:  Closing Gaps  Between  Policy  and Practice

An Implementation Failure labeled as an Intervention FailureExample: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective But, over 25% was spent on in-office

interventions with parents or children(< 0 fidelity)

Chapin Hall Center for Children, 2002

Page 7: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Longitudinal Studies of CSR Programs

Source: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

Evidence-Base For Effectiveness

Every Teacher Trained

Every Teacher Continually Supported

Actual SupportsYears 1-3

Fewer than 50% of the teachers received some training

Fewer than 25% of those teachers received support

Fewer than 10% of the schools used the CSR as intended

Vast majority of students did not benefit

OutcomesYears 4-5

Page 8: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Formula for Success

8

Socially Significant Outcomes

Effective Policies/

Interventions

Effective Implementation

Methods

Enabling Contexts

Page 9: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Formula for Success

9

WHY: Socially

Significant Outcomes

Effective Policies/

Interventions

Effective Implementation

Methods

Enabling Contexts

(Vision!)

Page 10: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Formula for Success

10

WHY: Socially

Significant Outcomes

WHAT: Effective Policies/

Interventions

Effective Implementation

Methods

Enabling Contexts

Page 11: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Formula for Success

11

WHY: Socially

Significant Outcomes

WHAT: Effective Policies/

Interventions

Effective Implementation

Methods

WHERE: Enabling Contexts

Page 12: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Formula for Success

12

WHY: Socially

Significant Outcomes

WHAT: Effective Policies/

Interventions

HOW & WHO: Effective

Implementation Methods

WHERE: Enabling Contexts

Page 13: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Table Discussions

Share a situation you have experienced/ observed where one of the variables in the formula was zero

Page 14: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Implementation Science

Page 15: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Excellent evidence for what does not work Implementation without changing

supporting roles and functions Implementation by edict Diffusion/dissemination of information by

itself Implementation by “following the money” Training alone, no matter how well done

Paul Nutt (2002). Why Decisions Fail

Page 16: Implementation Science:  Closing Gaps  Between  Policy  and Practice

What does work? An Implementation Framework Implementation Teams Implementation Drivers Improvement Cycles Implementation Stages

http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

Page 17: Implementation Science:  Closing Gaps  Between  Policy  and Practice

An Implementation Framework

Implementation Teams Implementation Drivers Improvement Cycles Implementation Stages

http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

Page 18: Implementation Science:  Closing Gaps  Between  Policy  and Practice

IMPLEMENTATION

Implementation Science

Effective NOT Effective

Effective

NOTEffectiveIN

TERV

ENTI

ON Student

Benefits

(Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Poor outcomes

Inconsistent, not sustainable, poor outcomes

Poor outcomes; sometimes

harmful

Page 19: Implementation Science:  Closing Gaps  Between  Policy  and Practice

IMPLEMENTATION

Implementation Teams

Effective

INTE

RVEN

TIO

N

80%, 3 Yrs 14%, 17 Yrs

Impl. Team NO Impl. Team

NOTEffective

Balas & Boren, 2000 Green, 2008

Fixsen, Blase, Timbers, & Wolf, 2001

Page 20: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Implementation Team

A group that knows: the intervention/policy

implementation

improvement cycles

Page 21: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Simultaneous, Multi-Level Interventions

SchoolManagement (leadership, policy)Administration (HR, structure)Supervision (nature, content)

Teacher

RegionDistrict

Impl

emen

tatio

n Te

am

State

Partners

Page 22: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Iowa’s Collaborating for Kids (C4K)

22

Page 23: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Cascading Logic Model

Page 24: Implementation Science:  Closing Gaps  Between  Policy  and Practice

An Implementation Framework

Implementation Teams Implementation Drivers Improvement Cycles Implementation Stages

http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

Page 25: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Improvement Cycles

Practice-Policy Communication Loops

Plan-Do-Study-Act cycles

Usability Testing

25

Page 26: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Policy

Ena

bled

Practi

ce (P

EP)

Practi

ce In

form

ed

Policy

(PIP

)

Practice/Policy Communication

Loops

Page 27: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Policy

Ena

bled

Practi

ce (P

EP)

Practi

ce In

form

ed

Policy

(PIP

)

Practice/Policy Communication

Loops

Page 28: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Policy

Ena

bled

Practi

ce (P

EP)

Practi

ce In

form

ed

Policy

(PIP

)

Practice/Policy Communication

Loops

Page 29: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Feed

back

Lo

ops

Feed

back

Lo

ops

Page 30: Implementation Science:  Closing Gaps  Between  Policy  and Practice

North Carolina Example

30

Page 31: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Authentic Work Informs the

PEP-PIP Cycle

Plan-D

o-Stu

dy-A

ct

Plan-D

o-Stu

dy-A

ct

Page 32: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

Team

District Leadership and Implementation

Teams

Regional Implementation

Teams

State Transformation

Team

StateDepartment of

Education Leadership

Authentic Work Informs the

PEP-PIP Cycle

Usabil

ity

Studie

s

Usabil

ity

Studie

s

Page 33: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Table Discussions

How do teams at each level of your system interact? What work are the teams doing with each

other?

What feedback loops exist between teams?

How might you increase interaction among teams?

Have/might you incorporate improvement cycles into pilots and policy innovations?

33

Page 34: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Growing Implementation Capacity Using

Transformation Zones

Page 35: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Implementation Team

Minimum of 3 people (4-5 preferred)

Tolerate turnover; sustainable

Page 36: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team

N = 10 Schools

Growing Implementation Capacity

(Phase 1)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 9 Staff

Invest up front to build

capacity

Page 37: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team

N = 10 Schools

Growing Implementation Capacity

(Phase 2)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

Subsequent Regional Implementation Team (3 Staff)

Page 38: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team, Expanded

N = 20 Schools

Growing Implementation Capacity

(Phase 2)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent Regional Implementation Team (3 Staff)

Page 39: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team, Expanded

N = 20 Schools

Growing Implementation Capacity

(Phase 2)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent Regional Implementation Team (3 Staff)

Page 40: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team, Expanded

N = 20 Schools

Growing Implementation Capacity

(Phase 2)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent Regional Implementation Team (3 Staff)

Page 41: Implementation Science:  Closing Gaps  Between  Policy  and Practice

State Transformation Team

First School Implementation Team, Expanded

N = 20 Schools

Growing Implementation Capacity

(Phase 2)

Exte

rnal

Sup

port

& 2

FTE

s

First Regional Implementation TeamN = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent School Implementation Teams

N=20 Schools

Subsequent Regional Implementation Team (3 Staff)

STAFF

Page 42: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Table Discussions

What responsibility will your state take to ensure implementation of your policies?

How can you grow implementation capacity within your state?

How can you rethink staffing in order to invest in the early stages of implementation?

42

Page 43: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Poll

One key take-away

OR

What questions do you have?

43

Page 44: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Possibilities to consider….

44

SCEE webinars SCEE Topical

Meetings SCEE

Discussion Groups

GIC Individual state

follow-up

Page 45: Implementation Science:  Closing Gaps  Between  Policy  and Practice

Thank you

45