KIPP: Effectiveness and Innovation in Publicly-Funded, Privately-Operated Schools

KIPP: Effectiveness and Innovation in

Publicly-Funded, Privately-Operated SchoolsOctober 4, 2012

Presentation to the APPAM/INVALSI Improving Education ConferenceChristina Clark Tuttle Philip Gleason Brian Gill Ira Nichols-Barrer

Network of 125 charter schools serving over 39,000 disadvantaged students in 20 states and D.C.

Model involves high expectations and more time in school to prepare students for college

Early KIPP model served grades 5-8 (age 10-14)– Represent a majority of schools in operation (n=70)– First elementary and high schools opened in 2004

Mixed effects for charter schools generally

Positive pattern of findings for KIPP specifically

Background

2

What are the impacts of KIPP middle schools on student achievement and other student outcomes?

Do KIPP schools engage in selective entry or exit?

Does the performance of KIPP students suggest they are on a path toward college?

Research Questions

3

Quasi-experimental analysis of “all” 70 KIPP middle schools– De-identified data from states on student selection, test

scores, and attainment

Experimental (or lottery-based) analysis in 13 schools (1,000+ students)– School records– Parent and student surveys– Study-administered test

Validation of observational methods using experimental results

Evaluation Design

4

Mathematica® is a registered trademark of Mathematica Policy Research.

22 KIPP middle schools– 20 still operating– 2 “closed” by KIPP

Opened by SY2005-06– Allows for more than one cohort to be analyzed across

multiple years after KIPP entry

Located in jurisdictions with available data– Three consecutive years of longitudinally-linked student-

level data, typically through 2007-08– For both traditional public and charter schools– Between 3 and 8 cohorts per school

Pilot QED Sample Selection

5


Treatment group comprised students entering KIPP in 5th or 6th grade (n=5,993)

Defined three comparison groups:– District: all students within the district– Feeder: students in ES also attended by KIPP students at

baseline (and their MS)– Matched comparison: propensity-score matched comparison

group using baseline characteristics

Analyses– Student characteristics– Attrition and replacement– Impacts on achievement

Analytic Approach

6


Demographic Characteristics

7

Difference from KIPP is statistically significant at the 5% level


Baseline Achievement

8



Attrition Rates, by Grade

9



Average Baseline Achievement in Math, Stayers vs. Transfers

10

Difference from stayers is statistically significant at the 1% level

At both KIPP and district schools, early leavers are lower-achieving than students who stay


Incidence of Late Arrivals

11

KIPP schools replace more students than they lose in grade 6, but fewer in grades 7 and 8

District comparison schools replace more students than they lose in both grades 7 and 8

Replacement Ratio: Ratio of New Arrivals to Prior Attrition

Proportion of total

enrollmentGrade 5-6 Transition

Grade 6-7 Transition

Grade 7-8 Transition

KIPP 1.18 0.78 0.60 0.15

Feeder NA 1.32 1.35 0.14


Average Baseline Achievement in Math, On-Time vs. Late Arrivals

12

Difference from on-time arrivals is statistically significant at the 1% level

At KIPP schools, late arrivals are higher-achieving than on-time arrivals; at district schools, they are lower-achieving


Baseline reading and math scores, by grade

13

By 8th grade, KIPP classrooms comprise students higher-achieving at baseline

Baseline reading Baseline mathKIPP Feeder District KIPP Feeder District

5 -0.10 -0.09 0.03** -0.10 -0.09 0.04**

6 -0.03 -0.08** 0.05** -0.01 -0.07** 0.05**

7 0.06 -0.07** 0.05 0.07 -0.07** 0.05

8 0.07 -0.07** 0.05 0.13 -0.05** 0.06*

All -0.09 -0.09 0.03** -0.08 -0.09 0.03**

*Difference from KIPP is statistically significant at the 0.05 level**Difference from KIPP is statistically significant at the 0.01 level

Model specification:

Retain students who leave KIPP in the treatment group

“Freeze” scores for grade repeaters (more common in KIPP: 11% vs. 2% of 5th graders)

Estimating Impacts

14

4

1nittitnitit TnXy

Year 1 Year 2 Year 3 Year 4

Reading 0.09**(0.011)

0.16**(0.013)

0.24**(0.018)

0.16**(0.027)

Math 0.26**(0.011)

0.35**(0.014)

0.42**(0.020)

0.25**(0.027)

Number of KIPP schools 22 22 22 17

N (math) 11,242 8,019 5,439 2,576

N (reading) 11,218 8,041 5,447 2,570

Estimated Impact of Potential Exposure to KIPP

15

*Difference is statistically significant at the 0.05 level**Difference is statistically significant at the 0.01 level

Percentage of KIPP Schools with Positive and Negative Impacts in Reading, by Years after KIPP Entry

16

Percentage of KIPP Schools with Positive and Negative Impacts in Math, by Years after KIPP Entry

17


KIPP students are:– More likely to be a racial minority, eligible for FRPL– Less likely to be limited English proficiency or

special education – Lower-achieving at baseline than the district overall

but equivalent to other students at the same ES

Rates of attrition are similar in KIPP and district schools

Conclusions

18


Late arrivals present a mixed picture– Proportion of late arrivals relative to enrollment is similar

at KIPP and comparison schools– KIPP schools are less likely to replace in later grades– KIPP late arrivals are higher-achieving

Patterns of attrition and late arrivals mean later grades at KIPP comprise higher-performing students, but “peer effects” can explain no more than about a quarter of cumulative impacts

Estimated impacts on reading and math scores are positive, statistically significant, and of substantial magnitude

Conclusions

19


Please contact:– Christina Clark Tuttle

• [email protected]

View reports online at:– Impacts:

http://www.mathematica-mpr.com/publications/PDFs/education/KIPP_fnlrpt.pdf

– Selection and Attrition: http://www.mathematica-mpr.com/publications/PDFs/education/KIPP_middle_schools_wp.pdf

For More Information

20

Supplemental Slides

21


Location of KIPP Schools in Sample

22

KIPP state in study

Other KIPP state (as of 2005)

Recent KIPP state (as of 2012)


KIPP Feeder DistrictBlack 0.37 0.44** 0.45**

male 0.38 0.42 0.45**

Hispanic 0.24 0.29** 0.30**

male 0.27 0.30 0.30

FRPL 0.34 0.34 0.37**

Overall 0.34 0.34 0.36**

Average Cumulative Attrition by Subgroup

23

*Difference is statistically significant at the 0.05 level**Difference is statistically significant at the 0.01 level

Size of Impacts in Reading after Three Years

24

KIPP Schools

Size of Impacts in Math after Three Years

25

KIPP Schools

Documents

KIPP: Effectiveness and Innovation in Publicly-Funded, Privately-Operated Schools