The Impact of Targeted vs. General Education Investments: Evidence from Special Education and English Language Learners in Boston Charter Schools

Using novel variation in special education and English Language Learner classification from admissions lotteries, I find that students can achieve large academic gains without specialized services. Enrolling in a Boston charter school doubles the likelihood that students lose their special education or English Language Learner status, but exposes students to a high-performing general education program that includes high intensity tutoring, data driven instruction, and increased instructional time. The positive effects extend to college: charters nearly double the likelihood that English Language Learners enroll in four-year colleges and quadruple the likelihood that special education students graduate from two-year college. A multiple instrument strategy suggests that high quality general education practices drive the gains and finds no detrimental effect from lower classification rates.

Suggested citation: Setren, Elizabeth. (2019). The Impact of Targeted vs. General Education Investments: Evidence from Special Education and English Language Learners in Boston Charter Schools. (EdWorkingPaper: 19-100). Retrieved from Annenberg Institute at Brown University: http://www.edworkingpapers.com/ai19-100

Elizabeth SetrenTufts University

VERSION: July 2019

EdWorkingPaper No. 19-100

The Impact of Targeted vs. General Education

Investments: Evidence from Special Education and English

Language Learners in Boston Charter Schools

Elizabeth Setren*

July 25, 2019

Abstract

Using novel variation in special education and English Language Learner classification

from admissions lotteries, I find that students can achieve large academic gains without spe-

cialized services. Enrolling in a Boston charter school doubles the likelihood that students

lose their special education or English Language Learner status, but exposes students to a

high-performing general education program that includes high intensity tutoring, data driven

instruction, and increased instructional time. The positive effects extend to college: charters

nearly double the likelihood that English Language Learners enroll in four-year colleges and

quadruple the likelihood that special education students graduate from two-year college. A

multiple instrument strategy suggests that high quality general education practices drive the

gains and finds no detrimental effect from lower classification rates.

*Tufts University, Department of Economics, Medford, MA 02155, elizabeth.setren@tufts.edu. I am grateful toJoshua Angrist, Parag Pathak, and Amy Finkelstein for their guidance and support. I also thank David Autor, BarbaraBiasi, Sandra Black, Sydnee Caldwell, Sarah Cohodes, Sarah Cordes, Tom Downes, Esther Duflo, Susan Dynarski,Mayara Felix, Nora Gordon, Peter Hull, Scott Imberman, Magne Mogstad, Natalie Obergruber, Ben Olken, andChristopher Walters for helpful comments and seminar participants at Harvard Graduate School of Education, NBEREducation Fall 2017, MIT, Teachers College Columbia University, the Boston Fed, AEFP 2016, APPAM 2016, ASSA2019 and the Evidence Based Economics Summer Meeting 2016. Special thanks go to Carrie Conaway, Cliff Chuang,Eleanor Rounds-Bloom, and the staff of the Massachusetts Department of Elementary and Secondary Education fordata, suggestions, and assistance. Thanks also to Annice Correia and Eryn Heying for administrative support andAnthony Yu, Alex Jucta, Xialing Zhang, Veronica Salazar, and Hazal Can for assistance preparing the lottery records.This work was supported by a National Science Foundation Graduate Research Fellowship.

1

How to allocate educational resources to serve students with large achievement gaps remains

a pressing concern. The largest sources of federal funding for K-12 education, Title I for low-

income students and IDEA for special education students (15.4 and 12.8 billion respectively), pay

for targeted services that only affect those students (such as specially trained teachers, counselors,

therapists, curriculum consultants, and paraprofessionals). State spending policies often also fol-

low this categorical spending model which targets additional investment in low-income, special

education, and English Language Learner (ELL) students towards specialized services as opposed

to broader investments in school quality that would affect all students (such as increased instruc-

tional time, higher performing teachers, or tutoring programs). This funding allocation comes from

a strong assumption that marginal education spending aimed to help low-performing groups of stu-

dents should be focused on specialized supports instead of a balance between specialized supports

and general school quality.

Policies to improve the educational and career outcomes for special education and ELL students

are increasingly important as their prevalence in urban districts grows: fifty percent of Boston Pub-

lic School (BPS) students have either a special education or ELL status.1 Schools invest two to

four times more per pupil for special education students and ELL students (Hayes et al., 2013;

Chambers, Parrish and Harr, 2004). Even with this additional investment, the special education

and ELL achievement gaps are at least double the size of the low-income and black-white achieve-

ment gaps.2 Despite special education and ELL students’ increasing prevalence, higher costs,

and low academic achievement, little causal evidence exists for how to improve their educational

trajectories.

To understand the tradeoffs between investments in targeted special education and ELL sup-

ports versus general school quality, we either need randomness in access to specialized supports

or randomness in the allocation of funds to specialized supports and general school investments.

However, schools endogenously designate students as special education and ELL based on stu-

1Author’s calculations using the Massachusetts Department of Elementary and Secondary Education Student In-formation Management System data for Boston Public Schools for the 2013-2014 school year.

2Author’s calculations using the Massachusetts Department of Elementary and Secondary Education Student In-formation Management System data based on high school graduation.

2

dent needs and ability. Additionally, families endogenously sort into schools, making it difficult to

conduct this analysis.

Boston charter schools offer a unique opportunity to look at both types of random variation.

First, using randomized charter admissions lotteries, I find that charter enrollment at least dou-

bles the likelihood that a student in special education or ELL at the time of the lottery loses this

classification and subsequently access to specialized services. The classification changes occur

for students with a wide range of special education and ELL needs, enabling the first analysis of

the impact of specialized services for students far from the qualification margin for services. Past

work on ELL and special education focuses on the impact of services for marginally qualified stu-

dents. Research on ELL classification for marginally qualified students finds mixed effects (Pope,

2016; Matsudaira, 2005; Robinson-Cimpian and Thompson, 2015) and Hanushek, Kain and Rivkin

(2002) find that special education classification boosts math outcomes by analyzing students who

move in and out of special education programs, but these movements are not random. This is also

the first study to use random variation in special education classification.

Second, the Boston charter school lotteries offer variation in access to high performing schools

that spend less on specialized supports (Boston charter schools) versus lower performing schools

that spend more on specialized supports (BPS).3 Boston charter schools spend 44 percent less on

special education instructional spending compared to BPS (see Table A6).4 At the same time,

Boston charters implement a set of education practices that affect all students, including increased

instructional time, high academic and behavioral expectations, high intensity tutoring, data driven

instruction, and frequent teacher feedback. These practices have a strong positive relationship with

school effectiveness in charters and yield positive effects when implemented in traditional public

schools or schools converted to a charter model (Angrist, Pathak and Walters, 2013; Dobbie and

Fryer, 2013; Fryer, 2014; Abdulkadiroglu et al., 2016). However, little is known about the effect

of these practices or charter schools on special education and ELL students specifically.

3Lottery-based evidence shows that Boston charters generate large academic gains relative to applicants’ tradi-tional public schools options (Angrist, Pathak and Walters, 2013; Angrist et al., 2016; Abdulkadiroglu et al., 2011).

4Districts do not report ELL specific school expenditures. See Table A6 for detailed BPS and charter schoolexpenditure and grant information.

3

Critics argue that charter schools underserve special education and ELL students because char-

ters have lower enrollment rates of special needs students compared to district schools and charters

often lack the economies of scale of traditional public school districts to provide separate class-

rooms and other intensive resources (Government Accountability Office, 2012; Boston Globe Edi-

torial Board, 2015; Massachusetts Teachers Association, 2015).5 These criticisms hold in Boston,

however the lower special needs enrollment and reduced spending on intensive services stem from

charters moving students to more inclusive, general education settings. As such, this paper speaks

to policy debates about the tradeoffs of targeted services versus inclusion in general education

classrooms.

My lottery-based estimates show that the net effect of Boston charter enrollment, including

reduced access to specialized services and exposure to high quality general education practices,

generates large achievement gains for students with special education or ELL classifications at

the time of the lottery. The gains for these special needs students are similar to those made by

non-special needs students in charter schools. Charters also significantly increase the likelihood

that special needs students meet a key high school graduation requirement, become eligible for a

state merit scholarship, and take an AP exam. Special education students in charters score are over

twice as likely to score 1200 or higher on the SAT than their traditional public school counterparts.

Charters nearly double the likelihood that ELL students enroll in four-year colleges and quadruple

the likelihood of two-year college graduation for special education students.

Charters generate academic gains even for the most disadvantaged charter applicants. Special

needs students who scored in the bottom third on their state exams in the year of the lottery expe-

rience gains of over 0.24 standard deviations in math. English Language Learners with the lowest

baseline scores have the largest English exam gains. Students with the most severe needs at the

time of the lottery – special education students who spent the majority of their time in substantially

separate classrooms and ELLs with beginning English proficiency – perform significantly better in

charters than in traditional public schools.

5I will refer to students with special education and ELL classifications at the time of the charter lottery as specialneeds students.

4

Lastly, I use a multiple instrument strategy that harnesses school-specific variation in reclassifi-

cation rates and pre-lottery characteristics of charter applicants to separately estimate the academic

effects of general education practices and specialized services. I find suggestive evidence that spe-

cial education and ELL classification removal have weak positive effects on test scores. The weak

positive correlation between individual charter schools’ classification removal effects and special

education and ELL achievement gains supports this finding. At the same time, school practices that

predict gains for general education students also predict gains for special needs students. Together,

this suggests that the achievement gains stem mostly from the general education practices and that

there is a weak positive effect from reduced specialized services.

Combined, the findings show that it is feasible for special needs students to make large aca-

demic gains without special needs services in a high quality general education program and that

the effects extend throughout the ability distribution and level of need. This suggests that increased

focus on general school quality investments can improve special education and ELL student out-

comes.

The next section provides background on Boston charter schools, discusses the special needs

classification process, and describes the data analyzed here. Section 2 details my empirical strategy

and reports the effect of charter enrollment on special needs classification. Section 3 reports the

academic effects of charter enrollment and Section 4 investigates mechanisms. The final section

concludes.

1 Background and Data

1.1 Special Needs Classification Processes

The special education classification process begins when a parent, teacher, or school staff requests

an evaluation for a student. This can happen at any grade or age. After a request, the district or

a private psychologist conducts an evaluation. The school holds a meeting with the parent(s) to

decide the student’s classification. If the student is classified, the school develops an Individualized

5

Education Program (IEP) that details the supports the student will receive. Students are designated

to full inclusion, partial inclusion, or substantial separate classrooms. Students in full inclusion

spend less than 21% of their time outside of the general education classroom. Partial inclusion

students spend between 21% to 60% of their time in a separate setting, and substantially separate

students spend over 60% of their time receiving special education services. Schools are required

to re-evaluate students’ classification and level of services every three years.6

To classify students as English Language Learners, Massachusetts public schools survey the

parent(s) of all new students, including those coming from within the same district, to identify

students whose primary language at home is not English.7 Once identified, these students take an

English Proficiency exam. A licensed ELL teacher or administrator interprets the test to decide

whether the student will be classified as ELL and to determine the set of services they will receive.

Every Spring, ELL students take a state standardized English proficiency exam, and their teachers

and ELL specialists evaluate their results to reconsider their ELL status and services. The state has

suggested guidelines for mapping the English proficiency exam scores to whether students become

or remain ELL and the level of services they receive; however, schools have discretion.

Schools aim to improve English language ability of ELL students so that they no longer need

the ELL classification and services. This goal of removing classification does not exist for special

education students; rather, schools aim to provide the proper set of supports to enable the child

succeed academically.

1.2 Classification Incentives

The financial and accountability incentives for special needs classification go in opposite directions

and impact charters more than traditional public school districts. The state and local school fund-

ing formula in Massachusetts does not include special education enrollment to discourage over-

classification. For the same reason, the federal Individuals with Disabilities Act (IDEA) Grant,

6Cullen (2003); Kubik (1999) analyze how financial incentives affect special education classification. Cullen andRivkin (2003) overviews the classification incentives and stratification in school choice programs.

7The survey is offered in 28 languages and administered by specially trained professionals (including teachers,principals, and guidance counselors). The training aims to detect if families falsely report English proficiency.

6

which provides funding to the states for special education costs, does not consider the number

of classified students in its funding formula. As a result, school funding formulas disincentivize

special education classification due to higher costs for special education services.8

The funding formula includes lagged ELL enrollment, but districts face financial disincentives

to classify students if the costs of services exceed additional funding. Massachusetts also dis-

tributes federal funding for ELL programs through a lagged ELL enrollment formula. A 2015

Massachusetts state commission found that the state formula did not provide enough funding to

meet the costs of educating ELL students and recommended an additional $2,361 for each ELL

student (Chang-Diaz and Peisch, 2015). This implies that schools face financial disincentives to

classify both special education and ELL students. Smaller school districts, including charter school

districts, face relatively larger disincentives because of economies of scale for providing special

needs services.

Accountability incentives encourage schools to properly classify special needs students. The

state inspects schools for proper identification of special needs and provision of services. The

state accountability system considers the outcomes of special needs students in addition to overall

student performance, which incentivizes providing the proper set of services for this group of

students to succeed academically.9 Charter schools face higher accountability standards and the

threat of de-authorization, so these incentives affect charters more acutely than traditional public

schools.

1.3 Data and Sample

To study the effect of charter attendance for special needs students, this paper uses the admissions

lotteries of 30 Boston elementary, middle, and high charter schools from the 2003-04 to 2014-15

school years. These schools account for 89 percent of Boston charter entry grade enrollment in

8Districts in Massachusetts can also receive reimbursement for special education costs of individual students abovea high threshold and Medicaid (reimburses school districts for qualifying medical services).

9This might also incentivize over-classification to increase the performance of special education students as awhole. The state inspections and financial disincentives counter this incentive.

7

2012-13.10 Schools are excluded from the study if they closed,11 declined to participate,12 had

insufficient records,13 did not have any oversubscribed lotteries,14 or serve alternative students.15

Appendix Table A1 describes the schools and application cohorts in the sample. In having near

full coverage of an entire city’s charter sector, including all grade levels, this paper overcomes the

common criticism of lottery-based charter school studies that the set of schools that elect to share

data might differ from the rest of the city’s charters.

I match lottery records to state administrative education data for detailed student demographics,

enrollment, and outcomes. This data provides both baseline characteristics of students from the

time of the lottery and post-lottery outcomes. It includes special education status, disability type,

and level of classroom inclusion for special education students and ELL status, native language,

and test scores on the annual English proficiency exam for ELLs. I categorize ELL students as be-

ginning, intermediate, or advanced English proficient using their English proficiency exam scores

and state guidelines for the amount of services to provide ELLs. I study students with special

needs classifications at the time of the lottery because special needs status can change over time.

Throughout the paper, mentions of special education and ELL students refer to those with baseline

classifications. Similarly, analysis by level of inclusion or English proficiency refers to baseline

characteristics. More details about the data and matching procedure appears in the Data Appendix.

This paper’s main analysis estimates the impact of charter school attendance on academic out-

comes for students by their pre-lottery special needs status. As a result, applicants who are not

enrolled in Massachusetts public schools the year of the lottery are excluded because they do not

have a pre-lottery special needs status. This excludes 95.4% of pre-k applicants and 70.7% of

10The sample expands upon the 11 Boston charter schools included in Angrist, Pathak and Walters (2013) byincorporating charter elementary schools, adding nine additional charter middle and high schools, and extending thesample to include the 2011-12 through 2014-15 school years.

11Uphams Corner Charter School closed in 2009. Fredrick Douglas Charter School and Roxbury Charter HighSchool both closed in 2005.

12Kennedy Academy for Health Careers (formerly Health Careers Academy) and Helen Davis LeadershipAcademy (formerly Smith Leadership Academy) declined to participate.

13Boston Renaissance and Dudley Street Neighborhood Charter School had insufficient records.14UP Academy Dorchester opened in 2013-14 and did not have an oversubscribed lottery.15Boston Day and Evening Academy Charter serves alternative students, including those who are overage for high

school, dropouts, and students with behavioral and attendance issues. In addition to serving a different population thanthe other Boston charters, Boston Day and Evening Academy uses rolling admissions instead of a lottery, making theschool not appropriate for this paper’s empirical strategy.

8

kindergarten applicants. These excluded applicants are used to investigate the effect of attending a

charter school on special needs initial classification.

1.4 Representation of Special Needs Students

Until recently, special needs students have been underrepresented among students applying to and

attending charters. In 2010, the Massachusetts state legislature passed a law that required charter

schools to increase efforts to recruit and retain special education and ELL students. Figure A1

shows that the special education application gap has narrowed for both middle and high school. In

Spring 2004, 22.1 percent of BPS students in 4th and 5th grades had a special education status.

Comparatively, only 17.0 percent of charter applicants in those grades had a special education

status. By the Spring 2014 lottery, the prevalence of special education students in middle school

charter lotteries was similar to BPS: 22.6 and 23.1 percent respectively. The gap also closed for

high school, with 20.3 percent of applicants with a special education status in charters, compared

to 19.5 percent of BPS 8th graders. Gaps in enrollment have also narrowed. Figure A1 shows

that gaps between BPS and charters remain in middle school special education enrollment in entry

grades, but special education students are overrepresented in 9th grade in charters.16

Gaps in ELL application and enrollment rates in BPS compared to charters were historically

larger, but they have also narrowed. Figure A2 shows that in Spring 2004, ELL students were

almost three times more prevalent in BPS than in charter middle and high school lotteries. In the

past decade, ELLs have become more prevalent in BPS, and the gap has closed. By Spring 2014,

ELLs have similar prevalence in BPS and charters: 24 percent in each for high school and 30 and

27 percent respectively for middle school.

Differences between the application and enrollment trends result from parental choices in re-

sponse to other school options and the sibling lottery preference. Figures A1 and A2 show that

the enrollment gaps have reversed for special education students in high school. The trends are

noisier for ELL students, but the middle school ELL enrollment gap has almost halved from 18

16I do not display the application and enrollment trends for elementary school charters because a low proportion ofpre-k and kindergarten charter applicants have a pre-lottery special needs status.

9

percent at its peak in 2007 to 9 percent in 2014. Similarly, the high school ELL enrollment gap has

halved from 9.5 percent in 2009 to 4.3 percent in 2014. Because ELL students were historically

underrepresented in charters, the sibling lottery preference means that ELL students have a lower

likelihood of getting a charter offer compared to non-ELL students. This likely contributes to the

current ELL enrollment gap.

By Spring 2014, students across the pre-lottery levels of special education classroom inclusion

and English language proficiency are, for the most part, similarly represented in charter lotteries

and BPS as shown in Figures A3 and A4. Small gaps remain for substantially separate inclusion

students in middle school and high school and for beginning English speakers in high school.

2 Classification

2.1 Empirical Framework & Descriptive Statistics

I use charter lottery offers as instruments to estimate the causal effect of attending charter schools

in a two-stage least squares setup. The second-stage equation links charter school attendance with

outcomes as follows:

yigt = αt +βg +∑j

δ jdi j +X ′i θ + τCigt + εigt (1)

where yigt is the outcome of interest for student i in grade g in year t. The terms αt and βg

represent outcome year and grade effects. The di j are dummy variables for all combinations of

charter school lotteries (indexed by j) present in the sample (henceforth referred to as experi-

mental strata). These experimental strata control for the fact that the set of school applications

determines the probability of receiving an offer. Baseline demographic characteristics from the

year of the lottery, represented by vector Xi, include gender, race, subsidized lunch status, ELL,

special education, and a female-minority interaction.

The treatment variable, Cigt , equals one if the student enrolled in a charter any time following

10

the lottery and until the time schools reported special needs classification.17 For models testing

charter effects on college preparation measures and high school graduation, Cigt indicates charter

enrollment between the lottery and the test or graduation date. Standard errors are clustered on

the school, grade, and year of the outcome. The parameter τ captures the causal effect of charter

school enrollment. I estimate the model separately for each baseline special needs status: special

education, ELL, and non-special needs.

When estimating the math or English exam effects, Cigt represents years spent in a charter from

the time of the lottery to the the test date. Students take exams in grades 3 through 8 and grade

10, so elementary and middle school applicants who appear in multiple testing grades contribute

multiple observations to the estimation. To account for this, the standard errors, εigt , are clustered

on the unique student identifier in addition to the school, grade, and year of the test. For math

and English test results, the parameter τ estimates the causal effect of a year of charter school

attendance.

I use two instruments for charter attendance: whether a student receives a random offer on

the day of the lottery (immediate offer) or whether a student receives an offer from the randomly-

ordered waitlist (waitlist offer). Z1i is equal to one if the applicant received an immediate offer to

attend a charter and zero otherwise. Z2i designates whether the applicant received a waitlist offer.

Appendix Table A1 details the schools and application cohorts with immediate and waitlist offers.

The first stage equation for the instrumental variables estimation is:

Cigt = λt +κg +∑j

µ jdi j +X ′i Γ+π1Z1i +π2Z2i +ηigt , (2)

where π1 and π2 capture the effects of receiving immediate or waitlist offers on charter atten-

dance. Like the second-stage equation, the first stage includes year and grade effects, experimental

strata dummies, and baseline demographic controls.

Because they are randomly assigned, charter offers are likely to be independent of student

17Students for whom Cigt equals zero enroll in non-charter public schools, including traditional public schools,pilot schools, exam schools, and innovation schools. For simplicity, I refer to this group by the most common type:traditional public schools.

11

background and ability within experimental strata. The pre-lottery demographics and test scores

are similar for offered and non-offered students, as shown in Columns (3) and (4) of Table 1.

Differences in baseline characteristics by offer status are small, mostly statistically insignificant,

and the p-values from joint tests are high. The subset of students with baseline special needs also

have comparable characteristics across offer status, as seen in Columns (6) and (7) for special

education and Columns (9) and (10) for ELL.

Differences between charter applicants and Boston Public School (BPS) students are docu-

mented in the first two columns of Table 1. Lottery applicants are less likely to have a special ed-

ucation status than BPS students. The two populations have similar rates of ELL students (though

as discussed above, this is not historically true). All levels of English proficiency are more repre-

sented in charter applicants than in BPS students. Lottery applicants have slightly higher baseline

test scores compared to BPS students (0.042 and 0.093 standard deviations in math and English

respectively).

Columns (5) and (6) of Table 1 shows that BPS students and lottery applicants who have special

education statuses at the time of the lottery have similar demographic characteristics. Columns (9)

and (10) show that the BPS and charter applicant ELLs have similar rates of female, Latino, and

reduced-price lunch students, but different rates of black students. Special education and ELL

charter applicants have large achievement gaps relative to the general charter lottery applicant pool

and have baseline test scores one standard deviation below the state mean. However, their average

test scores are 0.16 to 0.018 standard deviations higher than special education and ELL students in

BPS.

Despite the positive selection on test scores, the special education and ELL students who apply

to charters represent a range of needs. The charter applicant pool includes students who receive

high, moderate, and low levels of services, though students from substantially separate classrooms

are slightly under represented and students from partial inclusion classrooms are slightly over

represented in charter lotteries relative to BPS. Also, students of all levels of English proficiency

are more prevalent in charter lotteries than in BPS.

Ten percent of both BPS and charter applicants have a specific learning disability, the most

12

common disability type (see Table A2).18 Four percent of BPS and charter applicants also have

the second most common disability, communication impairment, which includes articulation, lan-

guage, or voice impairments and stuttering. Health, intellectual, physical, and sensory impairments

and multiple disabilities each comprise under one percent of both BPS and charter applicant stu-

dents and have similar rates in both populations. Autism, developmental delay, emotional impair-

ment, and intellectual impairment each comprise one to three percent of the BPS population and

are underrepresented in charter lotteries.19

Immigrant students comprise about a third of the ELL population in both BPS and charter

lotteries (see Table A2). Elementary school-aged immigrant ELL students are more prevalent in

charter lotteries than BPS while high school-aged immigrant ELL students are underrepresented.

Spanish speakers comprise almost 60 percent of BPS students and charter applicants and 13 percent

of the charter applicant sample. A variety of other languages comprise the rest of the ELL students,

with Haitian Creole and Chinese as the next most common.

These summary statistics show that the study sample covers students with a wide range of

special education and ELL characteristics including students with low baseline test scores, high

levels of special education need, and low levels of English proficiency. Given that I cannot observe

student motivation and parental knowledge of students’ ability, it is possible that these applicants

could be positively selected in unobservable ways compared to the general BPS student body.

2.2 Special Needs Classification

Receiving a lottery offer increases the likelihood of enrolling in a charter and the amount of time

spent in a charter school (see Table A3 for these first stage estimates). Special needs applicants

with immediate and waitlist offers spend approximately a year and 0.64 years longer respectively in

charters compared to those without offers. Immediate and waitlist offers also boost the likelihood

18Federal law 34 C.F.R. §§300.7 and 300.541 defines specific learning disability as “a disorder in one or more ofthe basic psychological processes involved in understanding or in using language, spoken or written, that may manifestitself in an imperfect ability to listen, think, speak, read, write, spell, or to do mathematical calculations, includingconditions such as perceptual disabilities, brain injury, minimal brain dysfunction, dyslexia, and developmental apha-sia.”

19For detailed information about disability type rates by grade-level group, see Table A2.

13

that special needs students will enroll in charters one year after the lottery by over 60 and 39

percentage points respectively.20 Of those students that enroll in charters, they attend charters for

an average of 3.0, 2.4, and 2.7 years respectively for elementary, middle, and high school in the

data.

Charters remove special needs classifications and move special education students to more in-

clusive settings at the time of enrollment21 at a higher rate than traditional public schools. Column

(2) of Table 2 shows that applicants with a special education status at the time of the lottery are

11.8 percentage points more likely to have their classification removed in charters than in tradi-

tional public schools.22 Charters even remove special education status from students with more se-

vere disabilities: applicants from substantially separate classrooms are 17.3 percentage points less

likely to keep their special education status in a charter compared to a traditional public school. The

classification removal effects appear consistent across disability type, though the point estimates

are noisy due to relatively smaller sample sizes (see Table A4).

Charters move special education applicants to more inclusive classrooms23 27.1 percentage

points more often than traditional public school (see Column (10) of Table 2). This means that stu-

dents spend more time in a general education classroom and less time receiving services outside

of the mainstream classroom. Charters move students across all ranges of need to more inclusive

settings. Charters increase inclusion for students from substantially separate classrooms 66.0 per-

centage points more (see Column (4) of Table 2) than traditional public schools. Instead, charters

place these high-need students in full inclusion classrooms 27.4 percentage points more often than

traditional public schools or remove their classification entirely (see Column (8) of Table 2). Char-

ter enrollment leads to a significant increase in classroom inclusion across all disability types (see

20The first stage for charter enrollment does not equal one because some students with offers elect to go to tradi-tional public schools and some students without offers ultimately enroll by moving off of a waitlist after our data wascollected.

21Data is collected on October 1st. Given this short time span, schools likely do not have sufficient time to alter theinitial classification given at the time of enrollment before the reporting date.

22I consider students to have their classifications removed if they had a classification the year of the lottery, haveno classification on the October 1st following the lottery, and continue to have no classification for the next two years.I follow the same practice for changes in classroom inclusion.

23Increased inclusion includes removing classification, moving from substantially separate inclusion to partial orfull inclusion, and moving from partial inclusion to full inclusion.

14

Table A4).

Charters remove ELL status at the time of enrollment 31.8 percentage points more often than

traditional public schools (see Table 3). Students with intermediate and advanced English profi-

ciency drive the differences in classification. Those with beginning English proficiency rarely have

their ELL classification removed at the time of enrollment. The effects are not driven by a spe-

cific language: applicants that speak Spanish, Haitian Creole, and other languages each experience

significantly lower rates of maintaining their ELL classification in charters (see Table A5).

Charters classify new enrollees to Massachusetts public schools (who have not yet been eval-

uated for special needs services in the state) as special needs less often than traditional public

schools. Only 1.4 percent of applicants who attend a traditional public school become classified

as special education at the time of enrollment.24 Attending a charter leads to an even lower special

education classification rate close to zero (see Column (2) of Table 2).25

Traditional public schools designate 63.7 percent of non-native English speakers, the potential

candidates for ELL services, as ELL. The rate is 26.1 percentage points lower in charters (see Table

3).

The classification and inclusion effects are consistent across grade-level and appear to persist

for two years, though with less precision (see Tables A6 - A8).

BPS moves 3.5 percent of special education applicants to less inclusive classrooms the Fall

after the charter lottery and charters do so at a similar rate (see Columns (11) and (12) of Table 2).

2.3 Explanations for Classification Removal and Increased Inclusion Effects

Learning gains cannot justify the classification differences because the special needs status changes

occur at the beginning of the school year following the lottery. At this point, schools have not had

time to generate substantial learning gains.24The state actively recruitments students with special needs for early intervention pre-k that starts at age 3. There-

fore, a large portion of students who qualify for special education services at a young age already have a classificationat the time of the lottery.

25I find that less than 0.8 percent of charter applicants who apply from a Massachusetts public school are designatedspecial education for the first time in the fall after the lottery. Two years after the lottery, the rate of new classificationsis around 3.5 percent. There is no significant difference in the classification of latent disabilities between the charterand traditional public schools.

15

Massachusetts law requires schools to assess the English proficiency of all incoming non-native

English speaking students. Therefore, schools assess all incoming ELLs, but charters use their dis-

cretion when interpreting the English proficiency exam and remove ELL classification 2.8 times

more often than traditional public schools. This supports the idea that charters have lower prefer-

ence for ELL classification. The lower special education classification in charters for new pre-k

and kindergarten students also implies that charters have a lower preference for classification com-

pared to traditional public schools.

Unlike English language proficiency, Massachusetts does not require schools to assess all new

enrolled students for special education needs. Therefore, factors other than schools’ classification

preferences could contribute to different special education classification practices. Better trans-

fer of student records, which include special education information, between BPS district schools

compared to between BPS district schools and charter schools plays a major role in special ed-

ucation classification changes. As a result, charters learn of special needs classifications from

voluntary parental reporting before they receive school records. The initial reliance on parental

reporting could contribute to fewer students maintaining their special education classifications in

charters. A survey conducted by the Massachusetts Department of Elementary and Secondary

Education that resulted from this study found that the most common reason for special education

classification removal was parent(s) not disclosing.26 The reasons why parents decline reporting

special education status could include stigma, individual preferences, not knowing their child re-

ceived special education services, assuming the school received the records, and not understanding

what special education means. Additionally, parents can refuse their child’s special education clas-

sification. Parental refusal of special needs status could differ in charters compared to traditional

public schools.

Charters’ preference for high levels of special education inclusion, often cited in charter schools’

annual reports, likely also play a role in higher levels of inclusion. Additionally, the relatively

26The survey investigated all cases of special education classification removal in the 2012-13 through 2014-15school years. All sample charters participated. Forty-nine percent of the cases cited parent(s) not disclosing. Theother reasons include unknown (12 percent), record error (12), student found ineligible for services after lottery byBPS (8), student transferred out of charter soon after enrolling (7), parent declined services (7), student determinedineligible by charter (3), and charter gave services later in the year (2).

16

smaller size of charter schools make it less likely for them to have the economies of scale to pro-

vide substantially separate and partial inclusion services to students compared to traditional public

schools.

2.4 Special Needs Inputs and Implications of Special Needs Reclassification

Students who have their special needs status removed have substantially different educational ex-

periences than those that remain classified. Schools are only legally obligated to provide special

education or ELL services to students with special needs classifications. Therefore, the higher

rate of classification removal in charter schools likely results in baseline special needs students re-

ceiving fewer special education and ELL services. Additionally, students who are moved to more

inclusive classrooms spend less time receiving services. Classification differences likely contribute

to the large differences in special needs educational inputs between charter and BPS.

Students who enroll in charters experience lower special education and ELL staff-to-student

ratios (Columns (4) and (6) of Table A10). Lottery applicants who enrolled in BPS have roughly

1.9 special education and 1.5 ELL staff per 100 students. Enrolling in a charter school exposes

lottery applicants to 1.1 fewer special education staff and 1.3 fewer ELL staff per 100 students.

Lower counts of special needs teachers drives the lower special needs staff-to-student ratio in

charters.

Despite charters having fewer classified special needs students, they employ mostly similar

proportions of special needs specialists27 and content support teachers.28 The similar rates of spe-

cialists in charters and traditional public schools suggest that either specialists work with students

who remain classified more intensively or that they also serve students without special needs classi-

fications. Charters also spend 44 percent less on special education instructional spending compared

27Special needs specialists include special education and ELL directors who oversee service provision, specialeducation diagnosticians, therapists, and counselors.

28Content support teachers coach teachers in how to better serve those with special education needs or limitedEnglish proficiency in the classroom or teach alongside another teacher, providing additional attention and differen-tiation. They could more broadly help students without special education or ELL statuses who might also benefitfrom the additional attention or a more accessible learning environment. In particular, these interventions could helpstudents with baseline special education and ELL statuses who had their classification removed.

17

to BPS (shown in Table A11).29

3 Academic Effects

Charter enrollment leads to two effects for special needs students: higher likelihood of classifi-

cation removal and exposure to the charter school environment. The charter school environment

and classification removal could have complementary or opposing effects. The high academic and

strict behavior standards common in Boston charter schools could leave special needs students be-

hind or motivate them to meet higher expectations. Special needs students could thrive in a more

inclusive classroom environment or fall behind without the specialized services they previously

received.

Prior research suggests no effect or limited gains from ELL classification removal (Chin,

Daysal and Imberman, 2013; Pope, 2016; Matsudaira, 2005) except Robinson-Cimpian and Thomp-

son (2015) who estimate a negative effect on when lower ability ELLs marginally qualify for clas-

sification removal. To the best of my knowledge, no causal evidence exists for special education

classification removal.30

In this section, I present causal estimates of the effect of charter enrollment on special needs’

students outcomes which bundles the two treatments of classification removal and charter envi-

ronment. In Section 4, I estimate the academic effects of classification removal and the charter

environment.

3.1 Charter School Effects

Test Score Effects

Charter school attendance has large positive effects for math and English state exam scores for

students with special education or ELL status at the time of the lottery. Table 4 documents the

29Districts do not report ELL specific school expenditures. See Table A11 for detailed BPS and charter schoolexpenditure and grant information.

30Using non-random movements of students in and out of special education programs, Hanushek, Kain and Rivkin(2002) find positive impacts on math test scores.

18

large and statistically significant gains for special needs applicants. A year of charter attendance

increases math test scores by 0.261 standard deviations for special education applicants and by

0.326 standard deviations for ELL applicants. Charters generate English score gains of 0.205 and

0.241 standard deviations for special education and ELL applicants (shown in Table 4). Positive

charter effects are statistically similar for special education and non-special needs students. Point

estimates for ELLs are statistically significantly larger than non-special needs effects.

One year of charter attendance narrows the ELL achievement gap by 84 percent in math and

39 percent in English.31 The larger gap between special education and non-special needs students

narrows substantially as well. With one year of charter enrollment, the special education gap

decreases by 30 percent in math and 20 percent in English.

The ordinary least squares (OLS) estimates (shown in Table A12) have comparable estimates

to the two-stage least squares. This suggests that the OLS is unbiased in the sample of lottery

applicants. Therefore, there is not significant selection into complying with the results of the

lottery: accepting a charter offer if it is received and not attending a charter if the student does not

receive an offer.

The reduced form or intent to treat estimates (shown in Table A12) also have comparable es-

timates to the two-stage least squares. Therefore, even without accounting for lottery compliance,

randomly assigned charter offers have a strong positive relation to test scores.

The effects of charter attendance appear to accumulate in the first two years and then level off.

The first year of charter attendance generates gains of 0.397 and 0.457 standard deviations in math

for special education and ELL applicants respectively (see Figure A5). The charter enrollment

effect nearly doubles for special education students and grows by 1.6 times for ELLs in the second

year. After the third year, the charter effects stabilize and are comparable to the prior year. A

similar pattern occurs for the English exam.32

The annual English proficiency exam – which schools use to reevaluate ELL students’ classifi-

31Achievement gaps are calculated by comparing the ELL scores in charter and traditional public schools with thenon-special needs student scores in traditional public school (using Table 4 estimates).

32This analysis focuses on middle school applicants because they take the state standardized exam in the three yearsfollowing the lottery. The test schedule for elementary and high school applicants does not lend itself to this analysis.

19

cation and services – also suggests that charter schools improve English skills for ELLs. Attending

a charter makes students 27.5 percentage points less likely to take the English proficiency exam

because charters remove ELL status at higher rates than traditional public schools (see Column (2)

of Table A13). Charters likely remove classification from the ELLs with relatively higher English

proficiency: leading to negative selection. Therefore if traditional public schools and charters have

the same effect on English language proficiency, charters would have a negative effect on English

proficiency scores. Instead, charter students perform similarly or significantly better compared to

traditional public school students. This suggests positive charter effects on English proficiency

(see Column (4) of Table A13).

College Preparation and High School Graduation

Charters also have positive effects on longer-term outcomes that likely have a strong, lasting link to

human capital and future earnings through educational attainment. Panel A of Table 5 shows that

charter special education and ELL students are 24.4 and 36.7 percentage points respectively more

likely to reach a key high school graduation requirement: reaching proficiency on the 10th grade

math and English exams.33 Students who do not meet this requirement need to fulfill remedial

coursework to graduate. Therefore, fulfilling this requirement keeps students on the path towards

high school graduation and enables them to take more college preparation courses.

Charters also boost the likelihood that special education students and ELL students will become

eligible for the Adams state merit college scholarship by 11.3 percentage points and 28.7 percent-

age points each. The Adams Scholarship awards free tuition to Massachusetts public universities

based on 10th grade math and English exams and has stricter conditions than the proficiency grad-

uation requirement.

Evidence in Panel B of Table 5 suggests that charter enrollment has weak positive effects on

college preparation exams for special needs students. Special needs charter and traditional public

school students take the SAT at similar rates, but charter enrollment has a positive effect on the

33This requirement is called Competency Determination.

20

likelihood that ELL students score above a 900 out of 1600 on the SAT.34 Only seven percent of

ELL charter applicants score above 1000 on the SAT and the likelihood of reaching 1000 is not

statistically significantly different among charter and traditional public school students. Estimates

of charter enrollment’s impact on special education students’ scoring above 800, 900, and 1000 on

the SAT are positive, but not statistically significant.

Special education and ELL students are 30.8 and 28.4 percentage points more likely to take

at least one AP exam in charters compared to in traditional public schools (see Panel B of Table

5). However, there is no significant effect of charter enrollment on scoring a 3 or higher, which is

required to earn college credit.

Charter enrollment lowers the likelihood that special education and ELL students will graduate

high school in four years by 29.9 and 18.3 percentage points respectively, though the estimate for

ELLs is not significant (see Panel C of Table 5). This is surprising given the gains in reaching the

proficiency graduation requirement. Angrist et al. (2016) suggest that students could take longer

to graduate from charters because they need additional time to meet charters’ rigorous graduation

requirements or because they choose to save money by remaining in high school for an additional

year rather than taking remedial coursework in colleges.

Estimates of charters’ effect on five-year graduation rates supports the theory that special needs

students take longer to graduate from charter schools relative to traditional public schools: Panel D

of Table 5 shows no significant difference between charter and traditional public school five-year

graduation rates. However, the noisy negative estimates (0.123 and 0.093 percentage points lower

likelihood of graduating in five years for special education and ELL students respectively) warrant

additional investigation. The difference in graduation rates is not driven by differences in dropout

rates. Instead, those special needs students who do not graduate in five years appear to transfer to

other schools (Panel D of Table 5).

Special education students with high levels of need can qualify for transition services from age

18 - 22 if they remain enrolled in school. Therefore, remaining in school longer could be positive

34The SAT score results include math and verbal sections. They exclude the writing section because it was notrequired in all years of the sample.

21

since it provides them with supports to help ease the transition to adulthood through teaching life

and job skills. The students who remain enrolled after four years of high school likely qualify for

these supports: they have qualifying disabilities (intellectual, communication, emotional, or learn-

ing disabilities) and high levels of need (they were enrolled in substantially separate classrooms

in eighth grade). Also, over 80 percent of special education charter applicants who do not grad-

uate in five years transfer to alternative high schools or to traditional public schools that provide

transitional services.

College Enrollment and Graduation

The Boston charter schools emphasize a college preparation curriculum. Table 6 investigates

whether the focus on college, higher AP taking rates, and increased scholarship qualification trans-

late into higher college enrollment and completion rates. Since charters have a negative effect on

graduating high school on time, Panel A of Table 6 shows college enrollment within 18 months of

students’ projected high school graduation dates. Forty-seven percent of special education appli-

cants and fifty-six percent of ELL applicants who attend traditional public schools enroll in college,

compared to sixty-four percent of their general education peers (see Column (1), (3), and (5) of

Panel A of Table 6). Charters generate positive effects on college enrollment, though estimates are

noisy for special education and marginal for ELL students.

Of special education college enrollees from traditional public schools, slightly more than half

choose four-year institutions over two-year colleges. About two-thirds of college-enrolled ELL ap-

plicants who attended traditional public schools choose four-year colleges. Charters nearly double

the likelihood that ELL applicants enroll in a four-year college. The estimates for special educa-

tion students are positive, but inconclusive. The increased four-year enrollment is driven by public,

in-state college enrollment. Estimates of charter school effects on two-year enrollment are close to

zero, but noisy for special needs students.

Charters have a noisy, positive effect on graduation from four-year colleges and a significant

positive effect on two-year college graduation for special education applicants. The ELL applicant

cohort that reaches college graduation age is too small to display college graduation findings.

22

Despite positive effects on initial college enrollment, Column (6) of Panel B of Table 6 shows

noisy zero effects of charters on college graduation for general education students.

3.2 Heterogeneity

Charters generate test score gains for even the most disadvantaged special needs students. Panel

A of Table 7 shows gains of 0.256 standard deviations in math for special education students with

the highest need. Students with less severe needs, those who apply from partial and full inclusion

classrooms, also experience gains of 0.328 and 0.269 standard deviations respectively. English

exam gains for special education students are positive and of similar magnitude across level of

inclusion, but they are imprecise for substantially separate and partial inclusion students.

Those with the lowest level of English proficiency experience math and English test score gains

of over 0.400 standard deviations in charters as seen in Panel B of Table 7. Charters also generate

math and English test score gains for ELLs with intermediate and advanced English proficiency.

Baseline test scores provide an alternative approach to analyze whether charters benefit the

neediest students. Column (2) of Table 8 shows that the bottom third of special education students,

as measured by their combined pre-lottery math and English exams, score 0.255 standard devia-

tions higher in math and 0.189 in English in charter schools. Column (4) shows that charters also

have positive effects for the bottom third of ELLs. While the higher-baseline performing students

also experience charter gains, the bottom third of ELLs experience the largest gains for English.

Elementary, middle, and high school charter applicants all experience positive effects even

though the characteristics of students with special education and ELL classifications vary across

age groups (see Table A14).

Charter gains are strongest for those with specific learning disabilities, which comprise 40

percent among charter special education applicants (see Table A15). The severity of learning dis-

abilities varies across lottery applicants: at the time of the lottery, thirty-seven percent of students

with learning disabilities come from a full inclusion classroom, 44% from partial inclusion, and

19% from substantially separate classrooms. Charters generate significant math and English gains

23

for ELLs who speak Spanish and Haitian Creole, the most common native languages of applicants

after English (shown in Table A16). While the other disability types and other languages are not

prevalent enough to estimate alone, students with non-learning disabilities and ELLs who speak

a language other than Spanish or Haitian Creole both experience significant gains in math (see

Tables A15 and A16).

4 Mechanisms

4.1 Classification Removal and School Environment

Do the academic gains documented above stem from general charter school characteristics that

affect all attendees or from classification removal and increased inclusion? Legal requirements

and best practices operate under the assumption that special needs students require services and

accommodations to succeed. Does charter classification removal and increased inclusion help or

hinder special needs students?

The similar charter achievement effects for special needs and non-special needs students sug-

gest that general charter school practices have a consistent effect for both groups. However, the

similar effect sizes could mask differences in the mechanisms that led to the gains. For exam-

ple, positive effects of general charter school practices for special needs students could outweigh

negative effects of the classification changes.

Classification Removal

I find suggestive evidence that the general charter school environment drives the academic gains

and find no negative effects of lower classification rates. Figure 1 plots the relationship between

test score effects of each individual charter school cohort against their reclassification effects (see

Online Appendix C.1 for the detailed estimation strategy). Charter school cohorts that experienced

higher reclassification rates also had higher special needs student test outcomes: test score effects

have a weak positive correlation with special education increased inclusion effects and ELL clas-

24

sification removal effects. Test score and special education classification removal effects have a

positive relationship for English and an imprecise relationship for math.35 The weak positive cor-

relations suggest that classification removal and increased inclusion contribute positively to student

growth, but cannot fully explain the charter test score gains. Therefore, school practices other than

special needs classification and services likely play an important role.

To provide additional evidence, I estimate a model with three endogenous variables: years in

charter, classification removal, and the interaction of years in charter and reclassification. The co-

efficients on these variables show the effect of a year of charter attendance holding classification

constant, the effect of classification removal, and the differential effect of classification removal

between charters and BPS. This estimation requires quasi-random variation in charter enrollment

and student reclassification in charters and in BPS. Unlike the lottery which randomly offers stu-

dents seats at charters, schools non-randomly make reclassification decisions based upon students’

needs. To address this selection issue, I harness school-specific variation in reclassification rates

and pre-lottery characteristics of charter applicants. I use individual charter lottery offers and the

interaction of these offers with students’ pre-lottery classification removal likelihood as instru-

ments (See Online Appendix C.2 for the full estimation strategy).36

The two-stage least squares estimation suggests weak positive effects of classification removal,

similar to Figure 1. However, the estimates are noisy estimates for the special education sample and

the ELL sample suffers from a weak instrument problem. For increased precision, I estimate the

Ordinary Least Squares (OLS) version with the same lottery applicant sample.37 OLS yields simi-

lar, but more precise estimates compared to the two-stage least squares estimation (see Columns (2)

35If schools that remove classification and increase inclusion more are effective due to other practices then thisexercise overstates the importance of reclassification. The relationship between non-special needs test score effects andcharter school reclassification effects is small and insignificant for special education and ELL classification removal,but small, positive, and marginally significant for special education increased inclusion. Therefore, there is littleevidence of other school practices correlated with classification removal and increased inclusion driving the correlationbetween reclassification and special needs academic effects (see Figure A6).

36Abdulkadiroglu, Angrist and Pathak (2014); Kling, Liebman and Katz (2007); Kline and Walters (2016); Co-hodes (2015) also interact site-specific indicators and baseline characteristics with random or quasi-randomly assignedoffers to generate new instruments to identify models with multiple endogenous variables.

37The similarity of the OLS and two-stage least squares estimates for the effect of charter attendance on test scoresand on classification removal (compare Table 4 to Table A12 and Tables 2 and 3 to Tables A17 and A18) suggests thatthe OLS estimates are unbiased.

25

and (6) of Table 9). Holding classification constant, one year in a charter boosts math and English

test scores of special needs lottery applicants by 0.2 to 0.3 standard deviations on average. Clas-

sification removal increases math test scores by 0.239 and 0.166 standard deviations for special

education and ELL students respectively. English test scores increase by 0.321 and 0.196 standard

deviations for special education and ELL students after classification removal. Special education

classification removal has a similar effect in charters and traditional public schools. For ELLs,

classification removal in charters has a smaller positive effect relative to classification removal in

traditional public schools.

Using these results, I calculate the upper and lower bound of the effect of charter classification

removal on test scores. Then, I scale the bounds by the charter classification removal effect: the

percent of applicants who lost their classification in charters, but would have kept their classifica-

tion in a traditional public school. This exercise shows that classification removal is linked to 0.003

to 0.063 standard deviation test score gains, which accounts for 0.9 to 25.4 percent of the charter

test score effects. Therefore, both the relationships between test scores gains and classification

removal and a multiple endogenous variable estimation approach suggest that the general char-

ter school environment, and not classification removal, drive the special needs students’ academic

gains in charter schools.

School Quality

Charter schools that serve special needs students well also serve general education students well.

Figure 2 displays the strong positive relationship between schools’ special needs and non-special

needs test score effects.

To contrast the relative importance of classification practices with overall school quality, I es-

timate a multiple endogenous two-stage least squares using years in charter, an index of school

quality, and classification removal effects. I add the math and English two-stage least squares ef-

fects for non-special needs students for each individual charter school to create a school quality

index (see Online Appendix C.1 for estimation strategy). The multiple endogenous variables esti-

mates yield noisy estimates for classification removal and precisely positive estimates for school

26

quality. Enrolling in a school with a one standard deviation higher non-special needs student test

score effect significantly increases special education and ELL students’ math scores by 0.201 and

0.325 standard deviations), while classification removal has a noisy positive estimate (see Columns

(3) and (7) of Table 9). The analogous OLS estimates show that classification removal has a similar

effect to one standard deviation increase in school quality for special education math and a much

smaller effect for ELL math and English. School quality has a smaller effect relative to classifica-

tion removal on special education students’ English outcomes. Since classification removal only

affects a subset of students but school quality affects all applicants, even though the classification

removal effects and the school quality effects have similar estimates for special education, school

quality explains a larger portion of the charter test score gains. This analysis shows the importance

of general education practices in explaining special needs’ charter gains.

4.2 School Practices

Special needs students who apply and do not receive charter lottery offers attend schools with

markedly different characteristics. Their BPS schools have more experienced, more licensed, and

higher paid teachers and spend about $1,700 more per pupil relative to the Boston charter schools

(see Table 10). Over half of Boston charters have a longer school year and over 95 percent of

Boston charters have a longer school day compared to BPS.38 Tutoring programs exist in all Boston

charters and about a third of charters require tutoring for all students. Boston charters commonly

use “No Excuses” practices, including high academic and behavior expectations, selective teacher

hiring, frequent testing and teacher feedback, and data-driven instruction.

The set of school practices that positively correlate with charter school effectiveness for general

education students also correlate with test score gains for special needs students. Column (3) of

Table 10 displays the correlation between charter school special education math effects and school

practices. Columns (4) and (5) display the analogous correlations for ELL and other students. An

index of “No Excuses” school practices,39 strict behavior code, longer school day, and emphasis

38BPS has 180 school days and 6.5 hours in the day.39The “No Excuses” index includes equal weight for discussion of the following items in the annual school report:

27

for high expectations in academics, characteristics that Angrist, Pathak and Walters (2013); Dobbie

and Fryer (2013) find linked to overall charter gains, are also positively correlated with special

education and ELL student gains.

School characteristics that do not correlate with general education student gains, expenditure

per pupil, student teacher ratio, teacher licensure, teacher experience, and teacher salary, also have

a null or a negative effect on special needs student outcomes. Special needs school characteristics

are weakly correlated with special needs charter effects (see Panel B of Table 10).

4.3 Peer Composition

Charter lotteries in the bottom quartile for special needs student representation have similar aca-

demic effects as those in the top quartile (see Table A19). The similar point estimates counter the

idea that charter special needs gains stem from fewer special needs students in the classroom. Lot-

teries with an average of 41 percent of applicants with ELL status have over 0.2 standard deviation

effects. Additionally, charter cohorts with the lowest special needs representation have gains of

around 0.2 standard deviations, suggesting that economies of scale cannot fully explain the charter

gains. The limited evidence that special needs economies of scale correlating with academic ef-

fects further supports the importance of general school practices in explaining special needs charter

gains.

5 Conclusion

Enrolling in a Boston charter school exposes students to a dual treatment. Lottery estimates find

that Boston charter schools remove special education and English Language Learner classifica-

tions at a higher rate than traditional public schools and move special education students to more

inclusive classrooms. Additionally, students experience the charter environment, including a set

high expectations for academics, high expectations for behavior, strict behavior code, college preparatory curriculum,core values in school culture, selective teacher hiring or incentive pay, emphasis on math and reading, uniforms, hiresTeach for America teachers, Teaching Fellows, or AmeriCorps members, affiliated with Teach for America alumni,data driven instruction, and regular teacher feedback.

28

of general education practices including high intensity tutoring, increased instructional time, high

expectations, and data driven instruction.

Using randomized admission lotteries, this paper finds strong positive net effects of Boston

charter schools’ dual treatment for special education and English Language Learner students.

Charters generate substantial gains for special needs students in math and English standardized

exams, English proficiency, and college preparation, enrollment, and completion outcomes. Even

the most disadvantaged special needs students perform better in charter schools compared to tra-

ditional public schools.

Charter attendance substantially decreases the special needs achievement gap. Among stu-

dents attending BPS schools, special education students and ELL students score about 0.87 and

0.39 standard deviations respectively below non-special needs students in math. Since charters

generate math gains of 0.268 standard deviations for special education students, one year in a

charter reduces the special education achievement gap by 30.8 percent. ELL students score 0.345

standard deviations higher in charters, narrowing the ELL achievement gap by 88.4 percent.

The findings show that schools can boost special needs students’ academic outcomes without

the traditional set of special needs services. Frequent use of tutoring and data-driven instruction

enables charters to identify and provide support to struggling students, regardless of special needs

status. “No Excuses” school practices, strict behavior code, longer school day, and emphasis on

high academic expectations positively correlate with charter school effectiveness for special needs

and general education students.

I find no evidence that classification removal or increased inclusion lowers outcomes for stu-

dents. Classification removal and increased inclusion can explain between 1 and 25 percent of the

special needs achievement effects. Charter schools that generate large non-special needs student

gains also generate gains for special needs students. Together, these findings imply that elements

of the charter school experience that affect all students, not just those classified as having special

needs, drive the positive gains for special needs students.

It is worth noting that the results apply to Boston charter lottery applicants. While special needs

students are currently well represented in the charter lotteries, Boston charters could have different

29

effects for the students who do not apply. By extension, my estimates may not reflect the effects

of expanding the number of seats in Boston’s charter sector or requiring charters to recruit more

special needs students. Similarly, it is unknown whether these results extend to other locations:

the set of general education practices employed by Boston charter schools are common among

urban charter schools, but the special education and English Language Learner practices of charter

schools are not well documented.

This study highlights the importance of high quality general education practices as an effective

policy lever to improve special education students’ and English Language Learners’ outcomes.

I cannot comment on the impact of an ideal implementation of specialized services or general

education practices, only on one city’s practices. The finding that special education students and

English Language Learners can make large academic gains without specialized services in a high

quality general education program calls for greater attention to general practices, in addition to the

current focus on specialized supports, to improve special needs students’ outcomes.

30

Notes: This figure plots the school-specific math and English Ordinary Least Squares (OLS) effects of years in charter schools for special needs students against the school-specific post-application special needs reclassification OLS effects of charter enrollment by the fall following the lottery. The figure plots elementary, middle, and high school estimates. Each dot represents a charter school application cohort. Experimental strata with samples too small to estimate are not displayed. The fitted line is the regression of the test score effect on the reclassification effect, weighted by the inverse of the average variance of the effects.

Remove Classification Post-Application Effect Increase Inclusion Post-Application Effect Remove Classification Post-Application Effect

Panel B: English

Engl

ish

Effe

ct (i

n St

anda

rd D

ev.)

Remove Classification Post-Application Effect Increase Inclusion Post-Application Effect Remove Classification Post-Application Effect

Mat

h Ef

fect

(in

Stan

dard

Dev

.)Figure 1: Correlations of Reclassification and Academic Effect Sizes by School x Cohort

Special Education - Remove Classification Special Education - Increase Inclusion English Language Learner - Remove ClassificationPanel A: Math

Panel B: English

Spec

ial N

eeds

Tes

t Sco

re E

ffect

(in

Stan

dard

Dev

.)

Non-Special Needs Test Score Effect (in Standard Dev.) Non-Special Needs Test Score Effect (in Standard Dev.)Notes: This figure plots the school-specific math and English Ordinary Least Squares effects of years in charter schools for special needs students and non-special needs students. The figure plots elementary, middle, and high school estimates. Each dot represents a charter school application cohort. Experimental strata with samples too small to estimate are not displayed. The fitted line is the regression of the special needs test score effect on the non-special needs test score effect, weighted by the inverse of the average variance of the effects.

Non-Special Needs Test Score Effect (in Standard Dev.) Non-Special Needs Test Score Effect (in Standard Dev.)

Figure 2: Correlations of Effect Sizes by School x CohortSpecial Education English Language Learner

Panel A: Math

Spec

ial N

eeds

Tes

t Sco

re E

ffect

(in

Stan

dard

Dev

.)

Boston Public Schools (BPS)

Students

MeanNon-Offered

MeanImmediate

Offer Any OfferBPS

MeanNon-Offered

MeanImmediate

Offer Any OfferBPS

MeanNon-Offered

MeanImmediate

Offer Any OfferBaseline Charateristics (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)Female 0.480 0.503 0.010 0.001 0.337 0.342 0.012 0.013 0.471 0.482 0.006 -0.012

(0.013) (0.013) (0.030) (0.030) (0.028) (0.028)Black 0.392 0.461 -0.023 -0.017 0.450 0.477 0.005 -0.006 0.191 0.269 0.009 -0.002

(0.013) (0.013) (0.031) (0.031) (0.025) (0.024)Latino/a 0.363 0.369 0.013 0.005 0.360 0.360 -0.015 -0.016 0.627 0.624 -0.035 -0.004

(0.012) (0.012) (0.029) (0.029) (0.027) (0.027)Subsidized Lunch 0.753 0.749 0.002 -0.007 0.797 0.757 0.031 0.012 0.847 0.844 -0.003 0.001

(0.011) (0.011) (0.024) (0.025) (0.020) (0.018)Baseline Math Test Score -0.449 -0.407 0.016 -0.012 -1.181 -1.002 -0.012 0.018 -0.820 -0.736 0.017 -0.047

(0.027) (0.027) (0.066) (0.066) (0.057) (0.055)Baseline English Test Score -0.548 -0.455 -0.028 0.004 -1.424 -1.214 -0.036 0.062 -1.142 -0.980 -0.028 -0.003

(0.028) (0.028) (0.069) (0.068) (0.062) (0.060)Special Education 0.226 0.192 0.007 -0.002 1.000 1.000 - - 0.212 0.190 -0.008 0.001

(0.011) (0.011) - - (0.022) (0.022)0.080 0.050 0.005 -0.004 0.355 0.260 0.016 -0.017 0.088 0.067 -0.021 -0.008

(0.005) (0.006) (0.025) (0.026) (0.011) (0.012)Partial Inclusion 0.056 0.057 0.008 0.002 0.249 0.296 0.021 0.014 0.058 0.059 0.015 0.011

(0.007) (0.007) (0.030) (0.030) (0.016) (0.015)Full Inclusion 0.093 0.082 -0.005 -0.001 0.411 0.425 -0.035 -0.004 0.076 0.061 0.000 -0.001

(0.007) (0.007) (0.030) (0.030) (0.014) (0.013)English Language Learner 0.231 0.258 -0.008 -0.003 0.217 0.254 -0.023 -0.010 1.000 1.000 - -

(0.011) (0.011) (0.026) (0.026) - -Beginning Proficiency 0.017 0.025 -0.006 -0.007 0.015 0.024 -0.007 -0.006 0.073 0.098 -0.019 -0.028

(0.003) (0.003) (0.006) (0.005) (0.012) (0.013)Intermediate Proficiency 0.071 0.121 0.002 0.005 0.094 0.144 0.008 -0.002 0.306 0.465 0.033 0.035

(0.009) (0.008) (0.022) (0.022) (0.028) (0.027)Advanced Proficiency 0.049 0.058 0.001 0.004 0.026 0.029 -0.009 0.001 0.205 0.216 0.010 0.018

(0.008) (0.007) (0.015) (0.013) (0.027) (0.027)Observations with School/Offer Type 194712 7591 5085 10408 43918 1458 1007 2076 44998 1956 1119 2188

P-value 0.661 0.661 0.592 0.924 0.499 0.995

Table 1: Descriptive Statistics and Covariate Balance

All Lottery Applicants

Substantially Separate Classroom

Notes: This table shows descriptive statistics for Boston Public School (BPS) students and charter lottery applicants. Column (1) shows means for BPS attendees in charter application grades (Pre-K, K, 1, 3, 4, 5, and 8) for 2003-04 through 2013-14. Column (2) shows means for charter lottery applicants who did not receive offers. Columns (3) and (4) report coefficients from regressions of observed characteristics on immediate offers and any offers, controlling for experimental strata dummies. P-values come from tests of whether all non-test score coefficients equal zero. Baseline test scores are only available applicants to the 4th grade or higher. Columns (5) through (12) report analogous results for the subsample with special education classification and ELL classification in the lottery application year.

Special Education at Baseline English Language Learner at Baseline

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Baseline Status (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)All Special Education 0.890 -0.118 0.151 0.271 0.035 0.001

(0.032) (0.035) (0.014)N 3153 3153 3153

0.967 -0.173 0.834 -0.660 0.049 0.028 0.051 0.274 0.114 0.380(0.036) (0.065) (0.039) (0.052) (0.063)

N 808 808Partial Inclusion 0.914 -0.079 0.627 -0.526 0.173 0.374 0.225 0.409 0.024 -0.004

(0.048) (0.069) (0.063) (0.066) (0.021)N 1004 1004 1004

Full Inclusion 0.823 -0.126 0.617 -0.029 0.117 0.080 0.070 0.016(0.052) (0.062) (0.043) (0.028)

N 1278 1278 12780.014 -0.011 0.001 -0.002 0.003 0.003 0.008 -0.008

(0.006) (0.002) (0.004) (0.005)

N 2665

Moved to Less Inclusive Classroom

Table 2: Effect of Charter Enrollment on Special Education Classification

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on special education classification and level of classroom inclusion in the fall following the charter lottery. Traditional public means show the proportion of charter applicants that do not enroll in charter schools with a given special education status. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately for each baseline classroom inclusion type. Students in full inclusion spend less than 21% of their time outside of the general education classroom. Partial inclusion students spend between 21% to 60% of their time in a separate setting, and substantially separate students spend over 60% of their time receiving special education services. Moved to a more inclusive classroom reflects moving from substantially separate to partial or full inclusion or moving from partial to full inclusion. Effects persist for up to two years following the charter application. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by school-grade-year.

Substantially Separate Classroom

New Students to MA Public Schools (No Prior Special Ed. Evaluation)

Any Special Education Substantially Separate

Classroom Partial Inclusion Full Inclusion

Classification Removed or Moved to More Inclusive

Classroom

34

Trad. Public mean Charter effectBaseline Status (1) (2)

0.825 -0.318(0.044)

N 3763Beginning Proficiency 0.996 -0.037

(0.030)N 287

Intermediate Proficiency 0.954 -0.343(0.055)

N 1810Advanced Proficiency 0.589 -0.270

(0.076)N 1008

0.637 -0.261(0.061)

N 856Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on English Language Learner classification in the fall following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by baseline English proficiency level. Effects persist for up to two years following the charter application. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

Table 3: Effect of Charter Enrollment on English Language Learner ClassificationRemain English Language Learner

All English Language Learners

New Non-native English Speaking Students (No Prior English Lang. Learner Evaluation)

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

(1) (2) (3) (4) (5) (6)Math -0.982 0.261 -0.503 0.326 -0.113 0.268

(0.053) (0.045) (0.023)N 4824 5404 16643

English -1.146 0.205 -0.728 0.241 -0.113 0.163(0.054) (0.043) (0.022)

N 4829 5416 16622Notes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores. Traditional public means show the average score of charter applicants that do not enroll in charter schools. Immediate and waitlist offer dummies instrument for years spent in charter schools. Columns (1) and (2) show estimates for applicants with baseline special education status, columns (3) and (4) for applicants with baseline English Language Learner classification, and Columns (5) and (6) for other students. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Estimates pool post-lottery outcomes for test-taking grades. Standard errors are clustered by student identifier and school-grade-year.

Table 4: Test Score Effects of Years in Charter by Baseline Special Needs StatusSpecial Education English Language Learner Non-Special Needs

36

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

(1) (2) (3) (4) (5) (6)

0.376 0.244 0.561 0.367 0.766 0.154(0.110) (0.162) (0.054)

Eligible for State Merit Scholarship 0.042 0.113 0.128 0.287 0.257 0.340(0.051) (0.129) (0.058)

N 1007 484 3892

Take SAT 0.433 0.090 0.561 -0.005 0.632 0.124(0.100) (0.160) (0.054)

SAT Score 800 or Higher 0.133 0.054 0.264 0.089 0.463 0.205(0.077) (0.148) (0.057)

SAT Score 900 or Higher 0.070 0.067 0.137 0.216 0.302 0.194(0.059) (0.109) (0.055)

SAT Score 1000 or Higher 0.032 0.039 0.071 -0.066 0.168 0.131(0.041) (0.100) (0.045)

Take AP 0.100 0.308 0.325 0.284 0.349 0.279(0.081) (0.150) (0.059)

Number of AP Exams 0.188 0.526 0.772 -0.030 0.823 1.016(0.186) (0.586) (0.218)

AP Score 3 or Higher 0.039 0.058 0.148 0.020 0.155 0.112(0.048) (0.143) (0.048)

N 1369 691 4685

Graduate in Four Years 0.567 -0.299 0.656 -0.183 0.701 -0.013(0.105) (0.137) (0.048)

N 1369 691 4685

Graduate in Five Years 0.666 -0.123 0.726 -0.093 0.778 0.056(0.101) (0.122) (0.044)

Remain Enrolled in Same School 0.083 0.046 0.045 -0.009 0.036 -0.030(0.065) (0.062) (0.027)

Transfer to Another School 0.090 0.090 0.109 0.108 0.078 -0.011(0.045) (0.074) (0.026)

Drop out of High School 0.151 -0.030 0.121 -0.005 0.106 -0.025(0.070) (0.083) (0.030)

N 1185 546 4188

Graduate in Six Years 0.685 -0.037 0.727 -0.065 0.798 0.029(0.098) (0.134) (0.044)

N 961 363 3579

Table 5: Effects of Charter Enrollment on College-Preparedness and High School Completion by Special Needs Status

Special Education English Language Learner Non-Special Needs

Panel A: High School PerformanceMeet High School Proficiency Graduation Requirement

Panel B: SAT and AP Exams

Panel C: High School Graduation

Notes: This table reports the two-stage least squares estimates of the effects of charter enrollment on longer-term outcomes. Immediate and waitlist lottery offer dummies instrument for any charter enrollment by the end of 10th grade. The 10th-grade state standardized exam score determines whether students meet the high school proficiency graduation requirement (called Massachusetts Competency Determination) and the State Merit College Scholarship (John and Abigail Adams Scholarship). The latter has higher standards for eligibility. SAT scores are out of 1600 and include the math and verbal sections. Students who do not take the SAT are coded as “0” for the SAT score indicator variables. Panel A’s sample includes students projected to graduate in Spring 2008 – 2016. Panel B and four-year graduation includes students projected to graduate in Spring 2008 - 2017. Panel D is restricted to students projected to graduate in 2008 – 2016. Panel E is restricted to students projected to graduate in 2008 – 2015. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by 10th grade school and year.

Panel D: Status in Year After Projected Four-Year High School Graduation

Panel E: Status in Two Years After Projected Four-Year High School Graduation

Trad. Public mean Charter effect

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4) (5) (6)

0.470 0.060 0.558 0.254 0.639 0.145(0.106) (0.160) (0.054)

Two-year 0.248 0.014 0.230 -0.012 0.202 -0.072(0.100) (0.128) (0.046)

Four-year 0.254 0.099 0.377 0.300 0.486 0.187(0.098) (0.155) (0.055)

Four-year Public 0.098 0.128 0.185 0.337 0.219 0.128(0.081) (0.159) (0.053)

Four-year Private 0.171 -0.046 0.220 -0.036 0.292 0.055(0.092) (0.150) (0.062)

Four-year Public in MA 0.087 0.111 0.179 0.318 0.190 0.102(0.076) (0.159) (0.053)

N 1207 550 4280

0.094 0.123 0.182 -0.021(0.095) (0.061)

Two-year 0.039 0.117 0.057 -0.003(0.055) (0.034)

Four-year 0.077 0.105 0.174 -0.024(0.083) (0.061)

N 625 2544

Table 6: Effects of Charter Enrollment on College Enrollment and Graduation by Special Needs Status

Notes: This table reports the two-stage least squares estimates of the effects of charter enrollment on college enrollment and graduation. Immediate and waitlist lottery offer dummies instrument for any charter enrollment by the end of 10th grade. College enrollment and graduation data come from the National Student Clearinghouse. Panel A investigates college enrollment for students projected to graduate high school in Spring 2008 - 2016. Panel B displays estimates for college graduation for students projected to graduate high school in Spring 2008 - 2014. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by 10th grade school and year.

Any

Any

Special Education English Language Learner Non-Special Needs

Panel A: College Enrollment within 18 Months of Projected 4-Year High School Graduation

Panel B: College Graduation in Four Years

Trad. Public mean Charter effect

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4) (5) (6)Math -1.392 0.256 -1.148 0.328 -0.606 0.269

(0.114) (0.093) (0.072)N 1004 1656 2090

English -1.614 0.204 -1.243 0.171 -0.791 0.216(0.135) (0.104) (0.065)

N 1004 1658 2092

Trad. Public mean Charter effect

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4) (5) (6)Math -1.392 0.404 -0.652 0.370 -0.129 0.296

(0.138) (0.062) (0.072)N 289 2710 1799

English -1.961 0.498 -0.904 0.315 -0.251 0.162(0.145) (0.057) (0.063)

N 292 2719 1801

Beginning Proficiency Intermediate Proficiency Advanced Proficiency

Notes: This table reports two-stage least squares estimates of the effects of years spent in charter schools for baseline special needs subgroups: by special education level of classroom inclusion and by English proficiency level. The sample includes elementary, middle, and high school lottery applicants. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

Table 7: Test Score Effects of Years in Charter for Special Needs SubgroupsPanel A: Baseline Special Education Level of Classroom Inclusion

Substantially Separate Classroom Partial Inclusion Full Inclusion

Panel B: Baseline English Language Learner English Proficiency Level

39

Trad. Public mean Charter effect

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4) (5) (6)

Bottom third -1.699 0.255 -1.337 0.248 -0.905 0.357(0.088) (0.090) (0.040)

N 1360 1491 5077

Middle third -1.067 0.219 -0.539 0.334 -0.100 0.284(0.078) (0.065) (0.032)

N 1540 1613 5285

Top third -0.302 0.314 0.254 0.328 0.592 0.185(0.069) (0.061) (0.026)

N 1597 1706 5123

Bottom third -1.812 0.189 -1.474 0.400 -0.789 0.175(0.110) (0.073) (0.040)

N 1418 1486 5021

Middle third -1.187 0.114 -0.722 0.305 -0.080 0.173(0.077) (0.076) (0.028)

N 1487 1580 5224

Top third -0.443 0.131 0.009 0.140 0.451 0.106(0.064) (0.056) (0.026)

N 1592 1617 5213

Panel B: English

Notes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores by baseline test performance and special needs status. Columns (1) and (2) report estimates for the baseline special education students by terciles of their baseline math and English test scores. Columns (3) and (4) report these estimates for baseline English Language Learners and Columns (5) and (6) for baseline non-special needs students. The sample includes elementary, middle, and high school lottery applicants. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

Table 8: Test Score Effects of Years in Charter by Pre-lottery Test Performance and Special Needs Status

Special Education English Language Learner Non-Special NeedsPre-lottery Test Performance within Special Needs Status

Panel A: Math

40

2SLS OLS 2SLS OLS 2SLS OLS 2SLS OLSEndogenous Variables (1) (2) (3) (4) (5) (6) (7) (8)

Charter Enrollment 0.204 0.229 0.164 0.187 0.344 0.292 0.212 0.206(0.040) (0.018) (0.039) (0.016) (0.049) (0.027) (0.040) (0.017)

First-stage F 9.213 8.786 4.330 5.477

Remove Classification 0.275 0.231 0.334 0.214 0.537 0.124 0.130 0.028(0.339) (0.071) (0.360) (0.056) (0.233) (0.043) (0.115) (0.032)

First-stage F 8.567 10.178 1.040 26.639

0.014 -0.031 -0.222 -0.071(0.100) (0.049) (0.119) (0.033)

First-stage F 13.623 3.026

School Quality Index 0.201 0.203 0.325 0.337(0.070) (0.029) (0.079) (0.044)

First-stage F 9.662 13.826

Overid. p-value 0.210 0.346 0.046 0.157N

Charter Enrollment 0.167 0.193 0.154 0.176 0.279 0.224 0.195 0.176(0.043) (0.018) (0.042) (0.018) (0.051) (0.022) (0.046) (0.017)

First-stage F 9.472 8.920 4.296 5.371

Remove Classification 0.315 0.319 0.324 0.270 0.467 0.159 0.228 0.085(0.341) (0.065) (0.356) (0.057) (0.255) (0.043) (0.122) (0.034)

First-stage F 8.434 10.152 1.104 27.272

-0.004 -0.057 -0.128 -0.051(0.115) (0.048) (0.129) (0.030)

First-stage F 13.358 3.037

School Quality Index 0.062 0.071 0.222 0.187(0.075) (0.031) (0.080) (0.037)

First-stage F 9.689 14.057

Overid. p-value 0.398 0.409 0.025 0.036N

Notes: This table displays multiple endogenous variable two-stage least squares (2SLS) and Ordinary Least Squares (OLS) estimates of two separate models which investigate mechanisms behind the charter test score effects. Columns (1) and (5) display the 2SLS estimates of a model with three endogenous variables: years in charter, classification removal by the fall following the lottery, and the interaction of the two. Columns (3) and (7) display the 2SLS estimates of a model with the following endogenous variables: years in charter, classification removal by the fall following the lottery, and a school quality index. The school quality index is the sum of the non-special needs math and English individual school 2SLS effects relative to Boston Public Schools. Instruments for both models include individual charter offers and individual charter offers interacted with a predicted reclassification index. See the online appendix for details about the predicted reclassification index. The even numbered columns display the OLS version of the previous column.The sample includes middle, and high school lottery applicants with baseline test scores and special education or ELL classifications at the time of the charter application. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Estimates pool post-lottery outcomes for test-taking grades. Standard errors are clustered by student identifier and school-grade-year.

Classification Removal School Quality School Quality

Classification Removal

Charter X Remove Classification

Panel A: Math

Panel B: English

Table 9: Multiple Endogenous Variable Test Score EstimatesSpecial Education Test Scores English Language Learner Test Scores

Charter X Remove Classification

Endogenous Variables Include:

Endogenous Variables Include:

Endogenous Variables Include:

Endogenous Variables Include:

3705 3844

3693 3830

41

Boston Public Schools Mean

Charter Sample Mean

Special Education

English Language Learner

Non-Special Needs

School Practices (1) (2) (3) (4) (5)

"No excuses" index 0.826 0.285 0.505 0.884(0.120) (0.446) (0.104) (0.304)

Strict behavior code 0.818 0.187 0.194 0.247(0.395) (0.094) (0.042) (0.072)

Longer school year 0.591 -0.010 0.130 -0.036(0.503) (0.065) (0.078) (0.061)

Longer school day 0.955 0.335 0.409 0.433(0.213) (0.032) (0.051) (0.025)

Emphasize high academic expectations 0.955 0.335 0.409 0.433(0.213) (0.032) (0.051) (0.025)

Total per pupil expenditure $18,766 $17,079 0.000 0.000 0.000($2,438) (0.000) (0.000) (0.000)

Student to teacher ratio 12.678 12.126 -0.006 -0.015 -0.008(1.790) (3.092) (0.015) (0.004) (0.012)94.974 52.265 -0.003 -0.005 -0.003(4.554) (17.173) (0.002) (0.003) (0.001)12.353 2.625 -0.023 -0.085 -0.061(3.355) (1.489) (0.026) (0.029) (0.016)

Average teacher salary $78,237 $65,380 0.000 0.000 0.000(10774.157) (0.000) (0.000) (0.000)

Special education compliance index 0.685 0.723 -0.323(0.041) (0.517)

English Language Learner compliance index 0.511 0.696 0.335(0.066) (0.629)

Special education remove classification effect 0.068 -0.069(0.111) (0.419)

Special education increased inclusion effect 0.225 0.379(0.229) (0.194)

English Language Learner remove classification effect 0.300 0.347(0.193) (0.260)

Special education instructional spending per pupil $2,299 $988 0.000(2,008) (519) (0.000)

Special needs staff to student ratio 0.030 0.015 -1.445 6.048(0.012) (0.011) (4.034) (4.894)

N 114 22 22Notes: This table reports coefficients from regressions of school-specific treatment effects for each special needs subgroup on 2012-13 school practices in Columns (3) - (5) (one regression for each school practice and student type combination). School-level BPS data is weighted by the proportion of lottery applicants who enrolled in the school. Only district-level data was available for total per pupil expenditure and the compliance indices. All costs are in 2015 CPI-U adjusted dollars. Column (2) displays the mean characteristics for sample charter schools with lottery cohorts with test results (those that reach grade 3 or higher by 2013-2014). Data come from charter school annual reports, Massachusetts Department of Elementary and Secondary Education School District Profiles, Education Personnel Information Management System, School District Expenditures, and Charter School End of Year Financial Reports. Data also come from MA DESE charter inspections including Renew Inspection Reports, site visits, Summary of Reviews, and Coordinated Program Reviews. See the Data Appendix for information on the “no excuses” index.

Table 10: School Characteristics and their Correlation with Individual Charter School Test Score Effects

Correlates of School Practices and Charter Effectiveness by Special Needs Group

Panel A: General School Characteristics

% of Teachers licensed in teaching assignment

Years of teaching experience in Massachusetts

Panel B: Special Needs School Characteristics

42

References

Abdulkadiroglu, Atila, Joshua Angrist, and Parag Pathak. 2014. “The Elite Illusion: Achieve-

ment Effects at Boston and New York Exam Schools.” Econometrica, 82(1): 137–196.

Abdulkadiroglu, Atila, Joshua Angrist, Peter Hull, and Parag Pathak. 2016. “Charters With-

out Lotteries: Testing Takeovers in New Orleans and Boston.” American Economic Review,

106(7): 1878–1920.

Abdulkadiroglu, Atila, Joshua Angrist, Susan Dynarski, Thomas J. Kane, and Parag Pathak.

2011. “Accountability and flexibility in public schools: Evidence from Boston’s charters and

pilots.” The Quarterly Journal of Economics, 126(2): 699–748.

Angrist, Joshua D., Parag a. Pathak, and Christopher R. Walters. 2013. “Explaining Charter

School Effectiveness.” American Economic Journal: Applied Economics, 5(4): 1–27.

Angrist, Joshua D, Sarah R Cohodes, Susan M Dynarski, Parag a Pathak, and Christopher R

Walters. 2016. “Stand and deliver: Effects of Boston’s charter high schools on college prepara-

tion, entry, and choice.” Journal of Labor Economics, 34(2): 275–318.

Boston Globe Editorial Board. 2015. “Charter Schools Take a Commendable Step.” The Boston

Globe, April 20.

Chambers, J. G., T. B. Parrish, and J. J. Harr. 2004. “What Are We Spending on Special

Education Services in the United States, 1999-2000?” Special Education Expenditure Project,

American Institutes for Research, Report 1, June.

Chang-Diaz, Sonia, and Alice Peisch. 2015. “Foundation Budget Review Commission.”

Chin, Aimee, N. Meltem Daysal, and Scott Imberman. 2013. “Impact of Bilingual Education

Programs on Limited English Proficient Students and Their Peers: Regression Discontinuity

Evidence from Texas.” Journal of Public Economics, 107: 63–78.

43

Cohodes, Sarah. 2015. “The Long-Run Impacts of Tracking High-Achieving Students: Evidence

from Boston’s Advanced Work Class.” Harvard Kennedy School, thesis chapter, May.

Cullen, Julie Berry. 2003. “The impact of fiscal incentives on student disability rates.” Journal of

Public Economics, 87: 1557–1589.

Cullen, Julie Berry, and Steven G. Rivkin. 2003. “The Role of Special Education in School

Choice.” In The Economics of School Choice. , ed. Caroline M. Hoxby, Chapter 3, 67–106.

National Bureau of Economic Research: University of Chicago Press.

Dobbie, Will, and Roland Fryer. 2013. “Getting Beneath the Veil of Effective Schools: Evidence

from New York City.” American Economic Journal: Applied Economics, 5(4): 28–60.

Fryer, Roland. 2014. “Injecting Charter School Best Practices into Traditional Public Schools:

Evidence from Field Experiments.” Quarterly Journal of Economics, 129(3): 1355–1407.

Government Accountability Office. 2012. “Charter Schools Additional Federal Attention Needed

to Help Protect Access for Students with Disabilities.” Report to Congressional Requesters,

GAO-12-543, June.

Hanushek, Eric, John Kain, and Steven Rivkin. 2002. “Inferring Program Effects for Special

Populations: Does Special Education Raise Achievement for Students with Disabilities?” The

Review of Economics and Statistics, 4(84): 584–599.

Hayes, Cheryl, Shawn Stelow Griffing, Nalina Ravindranath, Irina Katz, Justin Silverstein,

Amanda Brown, and John Myers. 2013. “Cost of Student Achievement: Report of the DC

Education Adequacy Study.” The Finance Project, December.

Hull, Peter. 2015. “IsoLATEing: Identifying Counterfactual-Specific Treatment Effects by Strati-

fied Comparison.” MIT Department of Economics, unpublished paper, December.

Kirkeboen, Lars, Edwin Leuven, and Magne Mogstad. 2016. “Field of Study, Earnings, and

Self-Selection.” Quarterly Journal of Economics, 131(3): 1057–1111.

44

Kline, Patrick, and Christopher Walters. 2016. “Evaluating Public Programs with Close Sub-

stitutes: The Case of Head Start.” Quarterly Journal of Economics, 131(4): 1795–1848.

Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz. 2007. “Experimental Analysis of

Neighborhood Effects.” Econometrica, 75(1): 83–119.

Kubik, Jeff. 1999. “Incentives for the Identification and Treatment of Children with Disabilities:

The Supplemental Security Income Program.” Journal of Public Economics, 73: 187–215.

Massachusetts Teachers Association. 2015. “Week of Action on Charter Schools.”

Matsudaira, J. 2005. “Sinking or swimming? Evaluating the impact of English immersion versus

bilingual education.” University of California, Berkeley, unpublished paper, December.

Pope, Nolan. 2016. “The marginal effect of K-12 English language development programs: Evi-

dence from Los Angeles Schools.” Economics of Education Review, 53: 311–328.

Robinson-Cimpian, Joseph P., and Karen D. Thompson. 2015. “The Effects of Changing Test-

Based Policies for Reclassifying English Learners.” Journal of Policy Analysis and Manage-

ment, 35(2): 279–305.

45

For Online Publication

A Data Appendix

This paper utilizes data from several sources. The charter applicant information was collected from

the individual charter schools. This data includes immediate and waitlist offers as well as factors

that impact an applicant’s ranking in the lottery, including sibling status, disqualifications, late ap-

plications, and applying from outside of Boston. Student demographic and school enrollment data

comes from the Student Information Management System (SIMS), which includes all of the public

school students in Massachusetts. Student standardized test scores come from the state database

for the Massachusetts Comprehensive Assessment System (MCAS). The paper also uses English

proficiency exam data, SAT and AP records, the Massachusetts Education Personnel Information

Management Systems (EPIMS) data, and National Student Clearinghouse (NSC) data on college

enrollment and graduation. This Appendix describes each data source and explains the process

used to clean and match them.

A.1 Lottery Data

Massachusetts legally requires charters to admit students via lottery when there are more applicants

than seats for a given grade. This paper uses charter lottery records from Spring 2004 to Spring

2014. The sample includes 10 elementary schools, 10 middle schools, five schools serving middle

and high schools, and five high schools. For the full list of schools and years, see Appendix Table

A1. Because of limited public pre-k enrollment, I exclude Spring 2014 pre-k lotteries from analysis

due to relatively low match rates to the administrative data.

The lottery data typically includes applicants’ names, dates of birth, and lottery and waitlist

offer information. Offers to attend the charter school can occur on the day of the lottery (referred

to here as immediate o f f er) or after the day of the lottery when students from the randomly

sequenced waitlist are contacted as seats become available (referred to as waitlist o f f er).

In some years, certain schools gave all applicants offers, so only the immediate offer instru-

46

ment, not the waitlist offer instrument, can be used for that cohort. For a few lotteries, records did

not distinguish the timing of offers, so only one instrument can be used for these cohorts. In other

cases, no waitlist offers were given to non-siblings. The lotteries affected by these circumstances

are noted in Appendix Table A1.

A.2 SIMS Data

This research uses SIMS data from the 2003-2004 school year through the 2014-2015 school year.

Each year has a file from October and the end of the school year. The observations are at the

individual student level. Each student has only one observation in each data file, except when

students switch grades or schools within year. The data includes a unique student identifier known

as the SASID. This identifier is used to match the SIMS data to the MCAS, English Proficiency

Exam, and SAT and AP data described below.

The SIMS dataset contains grade level, year, name, date of birth, gender, race, special education

and limited English proficiency status, level of classroom inclusion and type of disability for special

education students, free or reduced price lunch status, school attended, suspensions, attendance

rates, native language, and immigrant status. Students appear in the state administrative data if they

attend a Massachusetts public school. Those who enroll in private or parochial schools or move

out of state have missing outcomes data in years they are not in Massachusetts public schools.

A student is coded as attending a charter in a school year if there is any record in the SIMS of

attending a charter that year. Students who attend more than one charter school within a year are

assigned to the charter they attended the longest. If a student attended more than one traditional

public school in a year, the analysis uses the school where the student attended for the majority of

the year. In the case of attendance ties, the school for the analysis sample was randomly chosen.

For baseline characteristics, I designate a student as special education, ELL, or free/reduced lunch

if they have this status for either the October or end-of-year file for the application year.

47

A.3 State Standardized Exam (MCAS) Data

Massachusetts Comprehensive Assessment System (MCAS) data is used for the 2003-04 through

2013-2014 school years. An observation in the MCAS data refers to an individual student’s test

score results for a given grade level and year. The MCAS math and English Language Arts (ELA)

is administered in grades 3 through 8 and grade 10. Baseline math and ELA scores in the year of

charter application are used to check the balance for middle and high school lotteries. The raw test

scores are standardized to have a mean of zero within a subject-grade-year in Massachusetts.

The state requires special education and English Language Learners take the MCAS exam

regardless of the program and services they receive. Under 0.5 percent of the special education

students in my sample take an alternative MCAS exam. All ELLs in my sample are required to

take the regular MCAS exam.

Qualifying special education and ELL students can receive accommodations on the MCAS

exam to meet their specific accessibility needs. 40Enrolling in a charter school does not affect the

likelihood of students with special education or ELL statuses at the time of the lottery receiving

testing accommodations. This finding is robust across baseline level of classroom inclusion and

level of English proficiency. Therefore, accommodations do not threaten the validity of findings.

A.4 English Proficiency Exam (MEPA/ACCESS)

English Language Learners in kindergarten through 12th grade in Massachusetts take an annual

English proficiency exam. From 2005-2012, the state used the Massachusetts English Proficiency

Assessment (MEPA), and starting in 2013, the state switched to the Assessing Comprehension and

Communication in English State-to-State for English Language Learners (ACCESS). I standardize

the exam scores to center around the state mean for each year. I use state recommendations for

interpreting the scores of the exam to categorize students as beginning, intermediate, or advanced

English proficiency.

40Accommodations include changing the format of the test (e.g., paper-based vs. computer-based, large print,braille, read aloud), changing the test procedures, supports and devices to facilitate students’ test responses (e.g.,reference sheets, checklists, transcription).

48

A.5 SAT and AP Data

I use SAT and AP data files provided to the Massachusetts Department of Elementary and Sec-

ondary Education by the College Board. The data include scores on all AP and SAT tests for

students projected to graduate in 2008 through 2015. For students who took the SAT more than

once, their data includes only the most recent exam score.

A.6 Staff Data

I develop school level totals of full-time equivalent teachers and staff by various categories using

the Massachusetts Education Personnel Information Management Systems (EPIMS) data. I use the

state designations for staff type (i.e.. special education therapist, ELL co-teacher/support content)

and generate a total number of full-time equivalent teachers in each staff position for that school.

This means that if one school has two half-time ELL teachers, they are counted as having one

full-time equivalent ELL teacher. The EPIMS data ranges from the 2007-08 through the 2013-14

school years. I use a snapshot of the school staffing from October of these years.

A.7 National Student Clearinghouse Data (NSC)

College enrollment and graduation data comes from the National Student Clearinghouse (NSC)

database, which contains enrollment information for 94 percent of college students in Massachusetts.

The data include all students who graduated from a Massachusetts public high school from 2003-

2017 and students who ever enrolled in grades 8-12 in a Massachusetts public school from 2003-

2016. NSC searches used name and date of birth as criteria. The data include student unique

identifiers which merge to the state administrative SIMS data. College characteristics are coded

using the first college a student attends after their final observation in the SIMS.

A.8 Matching Data Sets

Lottery records were matched to the state administrative student-level data using applicants’ names,

date of birth, grade, and year. The applicants who uniquely and exactly match the grade, year,

49

name, and date of birth (if available) in the state records are assigned the matched SASID. Then

the names in the lottery and SIMS data are stripped of spaces, surnames (i.e.. Jr. IV), hyphens, and

apostrophes. Students who exactly match after that cleaning process are also assigned the matched

SASID. Then reclink, a fuzzy matching STATA program, is used to suggest potential matches for

the unmatched students. This matches students with slight spelling differences and those who ap-

pear in one grade older or younger than the lottery application grade. These suggested matches

are hand checked for accuracy. The remaining unmatched students are searched for by hand in the

data. Students in this category were not matched in the earlier methods because their names were

misspelled or their first and last names were recorded in the wrong field.

This matching process successfully assigns most applicants a unique student identifier. Ap-

pendix Table A20 shows the match rates to the administrative data for each year. Overall, 91.2

percent of applicants to elementary lotteries, 94.9 percent of applicants for middle school, and 96

percent of applicants for high school matched. Any student who enrolls in private, parochial, or

out-of-state school does not appear in the state records.

Students with offers are significantly more likely to match to the data by 4.3 percent for el-

ementary school and 3.8 percent for middle school. There is no significant difference for high

school. This means that elementary and middle school applicants without offers are slightly more

likely to go to private, parochial, or out-of-state schools. As a result, my findings show causal

estimates for the set of students who ultimately enroll in Massachusetts Public Schools.

A.9 Sample Restrictions

Appendix Table A21 shows the sample restrictions imposed upon the raw lottery records. The

sample excludes duplicate applicants within an individual school’s lottery and applicants who re-

ceive higher or lower preference in the lottery. Those with higher or lower preference include

late applicants, those who apply to the wrong grade, out-of-area applicants, and siblings. These

groups generally have no variation in offer status. If a student applied to multiple charters in dif-

ferent years, I keep only the first application year for that student. Except for estimating the effect

50

of charter attendance on initial special needs designation for new Massachusetts public school

students, the sample is further restricted to those with baseline demographics data. With the re-

strictions imposed, the original raw elementary school sample of 13,281 is narrowed to 6,569. For

middle and high school, the raw samples of 24,170 and 18,688 are restricted to 9,501 and 6,555

respectively.

B Threats to Validity

B.1 Selective Attrition

At the time of the lottery, students with and without random charter offers should be similar.

Differential attrition by offer status may lead to selection bias. For example, if not receiving a

charter offer makes students less likely to attend Massachusetts public schools, not receiving an

offer may alter the likelihood that a student appears in the data.41 Differential attrition generates

selection bias. To test for selection bias, I test the impact of charter offers on the probability that

lottery applicants contribute to state math and English exam scores and whether they have a non-

missing special needs status post-lottery.42 Small differences in the follow-up rates by offer status

imply that limited selection bias from differential attrition.

Differential attrition for middle and high school lottery applicants with baseline special needs

is not statistically significant, as documented in Table A22. Elementary school lotteries have some

differential attrition. Special needs students with charter offers are marginally more likely to take

a state math or English exam. These differences are fairly small. Elementary ELL students with

charter offers are 2.8 percentage points more likely to contribute to exam data than students without

charter offers, 83 percent of whom take the exams. These relatively small differences seem unlikely

to explain the elementary school exam results. For classification, 21.2 and 8.1 percent respectively

of the non-offered special education and ELL elementary applicants attrit from the data, compared

to essentially none of those with offers. These differences are significant and substantial, but they

41Students who leave the state or enroll in private or parochial schools do not appear in the data.42Post-lottery is defined as the October 1 after the lottery occurs.

51

are not large enough to explain the ELL classification effect or to fully explain the special education

classification effects.

B.2 School Switching

Charter critics often argue that large achievement gaps between charter and district schools stem

in part from charters encouraging lower performing students to leave. This paper’s results are not

directly affected by whether students enroll or remain in charter schools because the lottery offer

status comparisons (the two-stage least squares reduced forms) drive the estimates. The group

with lottery offers includes those who enroll and remain in charters as well as those who switch to

other schools. Similarly, the group without lottery offers includes some students who manage to

eventually enroll in a charter school.

However, excess school switching in charters could potentially inflate my estimates if students

who leave would generate negative peer effects (i.e. through disruption). Therefore, Table A23

investigates whether students in charters and traditional publics move schools one year following

the lottery at different rates. The lottery applicant population appears very mobile: roughly 50

percent of special needs elementary and middle and 30 percent of high school traditional public

school students switch schools.

For elementary and middle school, a large portion of these school moves are mechanical. When

I exclude applicants who need to switch schools because they reach the highest grade offered in

their school, 30.8 percent of special education and 21.2 percent of ELL elementary applicants in

traditional publics switch schools. Similarly, switch rates drop to around 15 percent for middle

school special needs applicants in traditional public schools.

The switching rate for elementary and middle school special education students is not statis-

tically significantly different in charter compared to traditional public schools. Elementary ELL

students are 13.8 percentage points less likely to switch schools in charter schools. In middle

school, ELL switching rates in charter schools are marginally significantly lower by 6.3 percent-

age points.

52

Special education high school applicants are 29.9 percentage points more likely to switch in

charters, more than double the school movement rate in traditional public schools. The differ-

ential switching comes from two early years. Without these years in the sample, the switching

rates of special education students in charters and traditional public schools are not statistically

significantly different, and the test score findings are essentially unchanged.

The estimates for ELL high school students are noisy, but not significantly different across

school type. Since special needs students are overall similarly or less mobile in charters, it is

unlikely that high mobility out of charters drives the main results.

B.3 Fallback Schools of Charter Applicants

Differences in the quality of students’ fallback schools if they do not get into charters could po-

tential explain some of the findings. For example, perhaps charters are not similarly effective at

serving special education students, ELLs, and general education students, but the counterfactual

school for a special education or ELL charter applicant performs considerably worse than the fall-

back option for a general education charter applicant.

To investigate this, I estimate OLS value-added for schools attended by untreated charter lottery

compliers. I ran 2SLS regressions of school-value added interacted with a traditional public school

indicator on a set of variables equal to one minus a charter enrollment indicator. I used lottery

offers as instruments and controlled for demographics and experimental strata. School value-

added estimates come from OLS regressions of test scores on a set of school indicator variables,

controlling for lagged test scores and student demographics.

I find no statistically significant differences between the untreated complier means fallback

schools of special education, ELL, and non-special needs students (see Table A24). Therefore,

there is no evidence that differences in students’ fallback school quality contribute to the findings

and charters appear to serve special education, ELL and general education applicants similarly

well.

53

C Estimation of Mechanisms

C.1 School and Cohort-Level Reclassification and Academic Effects Esti-

mation

I use the following model to estimate the individual charter school cohort academic effects dis-

played in Figure 1:

yigt = ∑t

∑s

ρstCigst +X′i θ +αt +βg +∑

jδ jdi j + εigt (3)

where yigt represents student i’s test score in grade g and year t and Cigst denotes the years

student i spent in charter school s by year t and grade g. Similarly, I estimate individual charter

cohort reclassification effects using

rigt = ∑t

∑s

ϑstCigst +X′i θ +αt +βg +∑

jδ jdi j + εigt (4)

where rigt indicates reclassification at enrollment for student i and Cigst indicates charter en-

rollment in the year after the lottery. I estimate equations (3) and (4) separately by baseline special

needs status. Two-stage least squares estimates using individual school immediate and waitlist

offers and OLS estimates yield similar results. I focus on the OLS estimates for precision. Figure

1 plots the cohort test score effects ρst against the reclassification effects ϑst .

C.2 Multiple Endogenous Variable Empirical Strategy

The individual charter lottery offers randomize not only whether students can enroll in charters, but

also student exposure to different reclassification rates. The interaction of individual charter offers

with students’ reclassification likelihood captures variation in classification removal for similar

students. I use individual charter lottery offers and the interaction of these offers with students’

pre-lottery classification removal likelihood as instruments for charter enrollment, classification

removal, and the interaction of charter enrollment and classification removal. In a constant effects

54

framework, these instruments identify causal effects for charter compliers. Heterogeneous effects

across the interacted characteristics make the estimates difficult to interpret (Kline and Walters,

2016; Hull, 2015; Kirkeboen, Leuven and Mogstad, 2016), but test score effects are not statistically

significantly different across baseline level of classroom inclusion, English proficiency, or test

score terciles (see Tables 6 and 7).

To create the pre-lottery reclassification likelihood variable, I estimate the relationship between

students’ baseline characteristics (represented by Ti) and an indicator for whether school change

their classification in the Fall of the following year, Li using the following model

Li = λTi +αt +βg + εitg. (5)

First, I estimate the model for Boston 5th, 6th, and 9th grade students who do not apply for

charter schools.43 I use the full range of available baseline student characteristics from the prior

grade, including gender, race, free or reduced price lunch, suspensions, days truant, and test scores.

I estimate the model separately for the different types of reclassification (special education clas-

sification removal, special education increased inclusion, and ELL classification removal). The

estimation for special education students includes baseline level of classroom inclusion and the

estimation for ELLs includes an indicator for native Spanish speakers and the baseline English

proficiency exam. The model also controls for year and grade effects using αt and βg.

I use the estimates from equation (5) which show how each student characteristic relates to

likelihood of classification removal in charter application grades in Boston to predict the likelihood

that charter applicants will have their special needs classification changed. Then, I center this pre-

lottery reclassification likelihood variable around the BPS mean for Li within a grade-year.44

The second stage equation links charter attendance and classification removal to test score

outcomes as follows:

43I focus on middle and high school applicants for this analysis so that I can include baseline test scores in equation(5) .

44Figure A7 plots the proportion of students with classification changes by pre-lottery reclassification likelihoodfor treated compliers, untreated compliers, and the full lottery applicant sample. It shows the positive relationshipbetween the predicted reclassification likelihood index and the proportion of students reclassified.

55

yigt = τ1Cigt + τ2Rigt + τ3CigtRigt + γLi +αt +βg +∑j

δ jdi j +X ′i θ + εigt (6)

where yigt is the test score of student i in grade g and year t. I estimate the three endogenous

variables Cigt (years in charter), Rigt (an indicator for classification removal or increased inclusion

by October 1st following the lottery), and CigtRigt (their interaction). I also control for pre-lottery

reclassification likelihood (Li), year and grade effects, experimental strata, and a vector of pre-

lottery demographic characteristics. Middle school applicants have multiple observations – one

for each grade in which they take the exam – so I cluster standard errors by student and the school,

grade, and year of the test. I estimate each model separately for the different types of classification

removal(special education and ELL) and restrict the sample to students with the corresponding

baseline special needs status.

The first stage for years spent in charter can be written as follows:

Cigt = ∑k

ρkZki +∑k

ψkZkiLi +ϕLi +λt +κg +∑j

µ jdi j +X ′i Γ+ηigt , (7)

where ρk represents the effect of receiving an offer, Zki, from charter school k on charter atten-

dance and ψk captures the effect of a one standard deviation increase in pre-lottery reclassification

likelihood, Li, on charter attendance for students with offers at charter school k.45 The first stages

for Rigt and CigtRigt have analogous specifications. The new set of instruments yield charter effect

estimates similar to the main estimates.

45Student sorting into charter schools based on classification removal rates poses a potential threat to the use ofschool interactions as instruments. There is no clear evidence of this: the average predicted reclassification index ofapplicants is not correlated with charter special education increased inclusion effects or the charter ELL classificationremoval effects.

56

Notes: The graphs on the left plot the percent of students with a special education status at the time of the lottery for charter applicants and Boston Public School (BPS) students in charter application grades (4, 5, and 8). The graphs on the right plot the percent of students with special education status at the time of the lottery for charter enrollees and BPS students in charter entry grades (5, 6, and 9). Using the special education status at the time of the lottery ignores any post-lottery changes to classification.

Figure A1: Special Education Prevalence in Charters and Boston Public Schools (BPS)Lottery Enrollment

Panel A: Middle School

Panel B: High School

Perc

ent

Perc

ent

57

Notes: The graphs on the left plot the percent of students with English Language Learner (ELL) status at the time of the lottery for charter applicants and Boston Public School (BPS) students in charter application grades (4, 5, and 8). The graphs on the right plot the percent of students with ELL status at the time of the lottery for charter enrollees and BPS students in charter entry grades (5, 6, and 9). Using the ELL status at the time of the lottery ignores any post-lottery changes to classification.

Perc

ent

Perc

ent

Figure A2: English Language Learner Prevalence in Charters and Boston Public Schools (BPS)Lottery Enrollment

Panel A: Middle School

Panel C: High School

58

Notes: This figure plots the percent of students with special education substantially separate, partial, and full classroom inclusion at the time of the lottery for charter applicants and Boston Public School students in charter application grades (4, 5, and 8).

Figure A3: Baseline Level of Inclusion of Charter Applicants and Boston Public School (BPS) Students Substantially Separate Partial Inclusion Full Inclusion

Panel A: Middle School

Panel B: High School

Perc

ent

Perc

ent

59

Notes: This figure plots the percent of students with beginning, intermediate, and advanced English proficiency at the time of the lottery for charter applicants and Boston Public School students in charter application grades (4, 5, and 8). English proficiency is measured by the required annual state exam for English Language Learners.

Figure A4: Baseline English Proficiency of Charter Applicants and Boston Public School (BPS) StudentsBeginning Proficiency Intermediate Proficiency Advanced Proficiency

Panel A: Middle School

Panel B: High School

Perc

ent

Perc

ent

60

Figure A5: Test Score Effects of Charter Enrollment by Years Following Lottery

Notes: This figure shows the two-stage least squares estimates of charter enrollment on math test scores for pre-lottery special education and ELL middle school applicants. Each data point reflects a separate estimation for 0, 1, 2, and 3 years after the lottery. The baseline year is represented by year 0. Error bars report the 95 percent confidence intervals. Immediate and waitlist offer dummies instrument for years spent in charter schools. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by school-grade-year.

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3

Mat

h Te

st S

core

Effe

cts

(in S

tand

ard

Dev

)

Year After Charter Lottery

Special Education

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3

Mat

h Te

st S

core

Effe

cts

(in S

tand

ard

Dev

)

Year After Charter Lottery

ELL

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3

Engl

ish

Test

Sco

re E

ffect

s (in

Sta

ndar

d D

ev)

Year After Charter Lottery

Special Education

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3

Engl

ish

Test

Sco

re E

ffect

s (in

Sta

ndar

d D

ev)

Year After Charter Lottery

ELL

Mat

h Ef

fect

(in

Stan

dard

Dev

.)Figure A6: Correlations of Reclassification and General Education Academic Effect Sizes by School x Cohort

Special Education - Remove Classification Special Education - Increase Inclusion English Language Learner - Remove ClassificationPanel A: Math

Notes: This figure plots the school-specific math and English Ordinary Least Squares (OLS) effects of years in charter schools for non-special needs students against the school-specific post-application special needs reclassification OLS effects of charter enrollment by the fall following the lottery. The figure plots elementary, middle, and high school estimates. Each dot represents a charter school application cohort. Experimental strata with samples too small to estimate are not displayed. The fitted line is the regression of the test score effect on the reclassification effect, weighted by the inverse of the average variance of the effects.

Remove Classification Post-Application Effect Increase Inclusion Post-Application Effect Remove Classification Post-Application Effect

Panel B: English

Engl

ish

Effe

ct (i

n St

anda

rd D

ev.)

Remove Classification Post-Application Effect Increase Inclusion Post-Application Effect Remove Classification Post-Application Effect

Notes: This figure displays the proportion of students reclassified by predicted reclassification index value (grouped into five bins).

Figure A7: Relationship between Proportion Reclassified and Predicted Reclassification IndexSpecial Education Remove Classification Special Education Increase Inclusion English Language Learner Remove Classification

Prop

ortio

n Re

clas

sifie

d

Predicted Reclassification Index Predicted Reclassification Index Predicted Reclassification Index

Proportion with Same Language Proportion with Same LanguageMat

h Ef

fect

(in

Stan

dard

Dev

.)

Engl

ish

Effe

ct (i

n St

anda

rd D

ev.)

Figure A8: Correlations of Charter Academic Effect Sizes by the Average Proportion of Students in Grade with the Same Language as ELL Students

Notes: This figure plots the school and cohort-specific math and English Ordinary Least Squares (OLS) effects of years in charter schools for English Language Learner students against the average proportion of students in the individual charter school grade that speak the same non-English. Languages include Spanish, Haitian Creole, and Chinese. All other non-English languages individually comprise a small portion of the sample. Students who speak other non-English languages are considered to speak the same language in the proportion (signaling the school has a critical mass of non-common languages). The figure plots elementary, middle, and high school estimates. Each dot represents a charter school application cohort. Experimental strata with samples too small to estimate are not displayed. The fitted line is the regression of the test score effect on the average proportion of same language speakers, weighted by the inverse of the variance of the academic effects.

Application Year/School

Bridge Boston Brooke East

BostonBrooke

MattapanBrooke

RoslindaleCodman

Conservatory Lab

Dorchester Collegiate Academy

KIPPMatch

Community Day

Neighborhood House

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)Entry Grade Pre-K K K K Pre-K Pre-K 4 K Pre-K & 2 Pre-K

20032004 Y2005 Y*2006 Y*2007 Y Y2008 Y Y2009 Y** Y+ No records Y2010 Y Y* No records Y2011 Y+ Y+ Y Y No records Y Y2012 Y Y+ Y Y* Y Y Y Y2013 Y Y Y Y Y+** Y Y Y2014 Y Y Y Y Y+ Y+ Y Y Y

N 561 2300 1296 785 114 739 52 159 1082 1932

Application Year/School

Dorchester Prep (UCS)

Brooke Roslindale

Brooke Mattapan

Brooke East

Boston

Excel East Boston

Excel Orient Heights

Lucy Stone (UCS)

Mission Hill (UCS)

KIPP BostonUP Academy

Boston

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)Entry Grade 5 5 5 5 5 5 5 5/6 5 6

2003 Y*2004 Y*2005 Y*2006 Y** Y2007 Y Y2008 Y Y Y2009 Y Y Y2010 Y Y2011 Y Y Y Y Y2012 Y Y Y** Y Y Y Y Y* Y2013 Y Y Y Y Y Y Y Y* Y2014 Y** Y** Y Y** Y** Y Y**

N 1035 254 738 367 519 333 1430 2291 429 1021

Application Year/School

Academy of the Pacific

Rim

Boston Collegiate

Boston Prep Codman Academy

Match MSBoston Green

AcademyCity on a Hill

City on a Hill IICodman Academy Match HS

(1) (2) (3) (4) (5) (1) (2) (3) (4) (5)Entry Grade 5/6 5 6 5/6 6 9 9 9 9 9

2003 Y No records Y

2004 YIncomplete

records Y* Y** Y2005 Y Y Y** Y Y2006 Y Y Y Y Y2007 Y Y Y Y No record Y2008 Y Y Y Y Y* Y Y2009 Y Y Y Y Y Y Y2010 Y Y Y Y Y Y Y2011 Y Y Y Y Y Y Y2012 Y Y Y Y Y** Y Y2013 Y Y Y Y Y Y Y** Y2014 No records Y Y+ Y Y Y** Y Y Y

N 1852 3025 1636 69 2137 901 4624 1102 1737 2766

Panel C: Combined Middle and High Schools (5th-6th - 12th Grades) Panel D: High School

Incomplete records

Incomplete records

Not entry grade

Notes: This table shows study charters and their application cohorts. The counts contain the number of students applying to each school in the study sample, not including siblings, out of area applicants, duplicates, disqualified applicants, and students not matched to the state data. In 2012, Uncommon Schools (Roxbury Prep, Dorchester Prep, and Grove Hall) held a joint lottery. APR had 6th grade lotteries from 2005-2007 and 5th grade lotteries from 2007-2014. Roxbury Prep began using 5th grade lotteries in Spring 2012. This table excludes closed schools and schools that did not provide usable lottery records. * Only ever offer information is available. ** There is no variation in waitlist offers. + Lotteries for additional entry grades are included in the analysis sample.

No recordsNot open

Not entry grade

Not open

Not openNot open

Panel B: Middle School

Not open

No records

Not openNot open Not open

Not openNot open

Not open

No records

Not entry grade

Incomplete records

A1: Lottery Participation by Schools and CohortsPanel A: Elementary School

Not openNot open

Not open

No records

Not open

No records Not open

Not open

Not open

Not open

Declined

Boston Public Schools (BPS)

StudentsAll Lottery Applicants

Boston Public Schools (BPS)

StudentsAll Lottery Applicants

Boston Public Schools (BPS)

StudentsAll Lottery Applicants

Boston Public Schools (BPS)

StudentsAll Lottery Applicants

Baseline Characteristics (1) (2) (3) (4) (5) (6) (7) (8)

Special Education 0.226 0.196 0.191 0.158 0.235 0.198 0.221 0.205Disability Type

Autism 0.012 0.006 0.030 0.017 0.011 0.005 0.007 0.002

Communication Impairment 0.043 0.040 0.043 0.048 0.052 0.045 0.026 0.031

Developmental Delay 0.015 0.012 0.102 0.081 0.005 0.007 0.000 0.000

Emotional Impairment 0.027 0.017 0.002 0.003 0.027 0.017 0.036 0.020

Health Impairment 0.007 0.007 0.001 0.003 0.008 0.009 0.008 0.007

Intellectual Impairment 0.028 0.016 0.000 0.001 0.030 0.014 0.036 0.022

Neurological Impairment 0.003 0.002 0.001 0.000 0.003 0.002 0.003 0.002

0.004 0.004 0.004 0.003 0.005 0.006 0.002 0.001

Sensory Impairment 0.003 0.001 0.002 0.001 0.003 0.001 0.002 0.001

Specific Learning Disability 0.098 0.100 0.002 0.008 0.110 0.101 0.114 0.126

Multiple Disabilities 0.005 0.005 0.000 0.001 0.006 0.008 0.003 0.002

ELL 0.231 0.230 0.361 0.454 0.249 0.257 0.148 0.125

Immigrant and ELL 0.076 0.074 0.036 0.087 0.082 0.078 0.081 0.065

Spanish Speaker and ELL 0.135 0.133 0.173 0.258 0.153 0.151 0.089 0.070

Haitian Creole Speaker and ELL 0.014 0.005 0.020 0.013 0.016 0.004 0.007 0.002

Chinese Speaker and ELL 0.021 0.037 0.019 0.057 0.023 0.046 0.017 0.017

Other Language and ELL 0.152 0.154 0.278 0.325 0.166 0.172 0.079 0.078

N 194712 17999 23858 1943 110289 9501 60565 6555

Physical Impairment

Notes: This table displays the proportion of students with each disability and ELL characteristic in BPS and the charter school lotteries for charter application grades (Pre-K, K, 1, 3, 4, 5, and 8). Students can have more than one disability type and can speak more than one language.

Panel A: Special Education Characteristics

Panel B: English Language Learners (ELL) Characteristics

Table A2: Special Education and English Language Learner Descriptive StatisticsAll Grades Elementary School Middle School High School

66

Immediate Offer Waitlist Offer

Immediate Offer Waitlist Offer

Immediate Offer Waitlist Offer

(1) (2) (3) (4) (5) (6)Years in Charter 0.966 0.638 1.105 0.701 1.147 0.737

(0.062) (0.063) (0.080) (0.065) (0.046) (0.043)N

Enroll in Charter 0.512 0.342 0.593 0.431 0.546 0.358(0.036) (0.035) (0.042) (0.038) (0.027) (0.025)

NNotes: This table reports the first stage estimates for the effect of lottery offers on years spent in charter schools and an indicator for charter enrollment by the fall following the lottery. Standard errors are clustered by school-grade-year for enroll in charter and bystudent identifier and school-grade-year for years in charter.

3131 3711 9546

A3: Effect of Lottery Offer on Charter Enrollment and Years in CharterSpecial Education English Language Learner Non-Special Needs

4877 5433 16675

67

Trad. Public mean

Charter effect

Trad. Public mean Charter effect

Disability type (1) (2) (3) (4)Learning 0.907 -0.068 0.153 0.208

(0.044) (0.052)N 1643 1643

Communication 0.843 -0.107 0.188 0.374(0.068) (0.075)

N 654 654

Other 0.910 -0.175 0.138 0.285(0.054) (0.067)

N 1031 1031

Classification Removed or Moved to More Inclusive

Classroom

Table A4: Post-Application Special Education Classification by Disability Type

Notes: This table reports the two-stage least squares estimates of the effects of Boston charter enrollment on special education classification and level of classroom inclusion in the fall following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by disability type. Disability types in the “Other” category had smaller samples. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

Any Special Education

68

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

(1) (2) (3) (4) (5) (6)Keep ELL Classification 0.841 -0.274 0.886 -0.383 0.812 -0.264

(0.040) (0.064) (0.069)N 2173 607 929

Table A5: Post-Application English Language Learner Classification by Native LanguageSpanish Hatian Creole Other

Notes: This table reports the two-stage least squares estimates of the effects of Boston charter enrollment on English Language Learner classification in the fall following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by native language. Languages in the “Other” category had too few students to individually estimate. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Baseline Status (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)All Special Education 0.866 -0.078 0.314 0.203

(0.039) (0.054)N 2176 2176

0.979 -0.026 0.704 -0.337 0.079 0.010 0.145 0.260 0.245 0.295(0.039) (0.105) (0.058) (0.093) (0.101)

N 549 549Partial Inclusion 0.910 -0.144 0.408 -0.200 0.371 0.129 0.461 0.273

(0.061) (0.083) (0.094) (0.088)N 701 701

Full Inclusion 0.756 -0.060 0.474 0.131 0.244 0.060(0.072) (0.080) (0.072)

N 884 8840.154 -0.027 0.011 -0.014 0.011 -0.018 0.067 0.002

(0.023) (0.005) (0.007) (0.021)N 1138

Substantially Separate Classroom

New Students (No Prior Special Ed. Evaluation)

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on special education classification and level of classroom inclusion two years following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by baseline classroom inclusion type. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by school-grade-year.

A6: Effect of Charter Enrollment on Special Education Classification Two Years After Application

Any Special Education Substantially Separate

Classroom Partial Inclusion Full InclusionMove to More Inclusive

Classroom

70

Trad. Public mean Charter effectBaseline Status (1) (2)

0.615 -0.403(0.068)

N 2314Beginning Proficiency 0.982 -0.203

(0.128)N 146

Intermediate Proficiency 0.760 -0.439(0.106)

N 1128Advanced Proficiency 0.270 -0.224

(0.068)N 642

0.565 -0.336(0.093)

N 308

A7: Effect of Charter Enrollment on English Language Learner Classification Two Years After Application

Remain English Language Learner

All English Language Learners

New Non-native English Speaking Students (No Prior English Lang. Learner Evaluation)

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on English Language Learner classification two years following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by baseline English proficiency level. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

71

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Baseline Status (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

All Special Education 0.907 -0.190 0.161 0.294(0.069) (0.125)

N 254 2540.903 -0.016 0.629 -0.401 0.016 0.066 0.177 0.556 0.226 0.539

(0.107) (0.169) (0.093) (0.124) (0.175)N 72 72

Partial Inclusion 0.895 -0.445 0.500 -0.551 0.289 0.226 0.342 0.464(0.226) (0.222) (0.254) (0.287)

N 49 49Full Inclusion 0.910 -0.144 0.690 0.034 0.060 0.141

(0.136) (0.171) (0.131)N 126 126

0.014 -0.011 0.001 -0.002 0.003 0.003 0.008 -0.008(0.006) (0.002) (0.004) (0.005)

N 2665

All Special Education 0.927 -0.161 0.125 0.301(0.044) (0.049)

N 1726 17260.976 -0.140 0.897 -0.683 0.036 0.016 0.028 0.259 0.071 0.286

(0.064) (0.098) (0.066) (0.076) (0.092)N 403 403

Partial Inclusion 0.935 -0.143 0.665 -0.645 0.156 0.413 0.193 0.462(0.066) (0.087) (0.079) (0.084)

N 611 611Full Inclusion 0.886 -0.226 0.692 -0.100 0.097 0.117

(0.077) (0.090) (0.059)N 683 683

All Special Education 0.841 0.030 0.180 0.112(0.103) (0.092)

N 1173 11730.975 -0.442 0.819 -0.468 0.071 -0.171 0.042 0.065 0.130 0.101

(0.077) (0.123) (0.095) (0.077) (0.126)N 333 333

Partial Inclusion 0.884 0.270 0.589 -0.472 0.179 0.633 0.254 0.470(0.185) (0.191) (0.172) (0.177)

N 344 344Full Inclusion 0.726 0.335 0.511 0.341 0.156 -0.147

(0.187) (0.198) (0.132)N 469 469

Substantially Separate Classroom

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on special education classification and level of classroom inclusion in the fall following the charter lottery. Traditional public means show the proportion of charter applicants that do not enroll in charter schools with a given special education status. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately for each baseline classroom inclusion type. Students in full inclusion spend less than 21% of their time outside of the general education classroom. Partial inclusion students spend between 21% to 60% of their time in a separate setting, and substantially separate students spend over 60% of their time receiving special education services. Moved to a more inclusive classroom reflects moving from substantially separate to partial or full inclusion or moving from partial to full inclusion. Effects persist for up to two years following the charter application. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Standard errors are clustered by school-grade-year.

Panel A: Elementary School

Substantially Separate Classroom

New Students (No Prior Special Ed. Evaluation)

Panel B: Middle School

Substantially Separate Classroom

Panel C: High School

A8: Post-Application Special Education Classification

Any Special Education Substantially Separate

Classroom Partial Inclusion Full InclusionMove to More Inclusive

Classroom

72

Trad. Public mean Charter effectBaseline Status (1) (2)

0.900 -0.198(0.075)

N 818Beginning Proficiency 0.989 -0.033

(0.029)N 110

Intermediate Proficiency 0.986 -0.126(0.074)

N 349Advanced Proficiency 0.739 -0.604

(0.297)N 25

0.637 -0.261(0.061)

N 856

0.794 -0.328(0.059)

N 2231Beginning Proficiency 1.000 0.000

(0.000)N 130

Intermediate Proficiency 0.953 -0.420(0.075)

N 1105Advanced Proficiency 0.570 -0.199

(0.085)N 774

0.802 -0.375(0.140)

N 714Beginning Proficiency 1.000 -0.042

(0.047)N 47

Intermediate Proficiency 0.921 -0.384(0.143)

N 356Advanced Proficiency 0.618 -0.152

(0.375)N 209

All English Language Learners

Panel C: High SchoolAll English Language Learners

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on English Language Learner classification in the fall following the charter lottery. Immediate and waitlist offer dummies instrument for enrollment in charter schools. Estimation is run separately by baseline English proficiency level. Effects persist for up to two years following the charter application. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

A9: Post-Application English Language Learner ClassificationRemain English Language Learner

Panel A: Elementary SchoolAll English Language Learners

New Non-native English Speaking Students (No Prior English Lang. Learner Evaluation)

Panel B: Middle School

73

Trad. Public mean Charter Effect

Trad. Public mean Charter Effect

Trad. Public mean Charter Effect

(1) (2) (3) (4) (5) (6)Total Staff 0.120 0.045 0.019 -0.011 0.015 -0.013

(0.011) (0.001) (0.001)Teachers 0.079 0.013 0.010 -0.010 0.003 -0.002

(0.004) (0.001) (0.001)Specialists - - 0.003 -0.001 0.000 0.000

(0.000) (0.000)Content Support - - 0.004 0.002 0.001 0.001

(0.001) (0.001)N (students)Notes: This table shows two-stage least squares estimates of the effect of charter enrollment on the staff-to-student ratios. Immediate and waitlist offer dummies instrument for any charter enrollment in the year following the lottery. The sample includes all lottery applicants applying in the 2007-08 through 2013-14 school years. Staffing and student counts data are collected in October of each year. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

A10: Effect of Charter Enrollment on the Staff-to-Student Ratios Students ExperienceAll Staff Special Education Staff English Language Learner Staff

14346

74

Boston Public Schools

Boston Charter Schools

Boston Public Schools*

Boston Charter Schools

(1) (2) (3) (4)

Total $19,214 $16,759 $1,361

(2,502) (713)

Total Instructional Spending $8,913 $9,769 $2,365 $1,325

(2,395) (1,470) (2,365) (692)

Retirement & Insurance $3,282 $1,345 -

(410)

Other Teaching Services $1,307 $872 $504 $168

(842) (652) (725) (209)

Professionals $309 $360 $5 $72

(183) (489) (62) (146)

Paraprofessionals $974 $249 $498 $17

(772) (398) (697) (49)

Contractors $120 $204 $6 $76

(373) (331) (015) (144)

Classroom & Specialist Teachers $6,051 $5,521 $1,567 $808

(1069) (844) (1,231) (605)

Professional Development $310 $190 $86 $16

(134) (205) (75) (52)

Pupil Services $2,601 $1,994 $36

(726) (110)

Operations & Maintenance $1,249 $1,020 -

(517)

Administration $557 $2,632 -

(1,471)

Guidance, Counseling, & Testing $117 $715 $23 $210

(346) (419) (291) (196)

Instructional Leadership $821 $1,627 $159 $100

(400) (0,641) (231) (117)

Materials, Equipment, & Tech $308 $843 $27 $22

(406) (588) (035) (45)

Federal Grants $1,396 $1,257 $389 $246

(683) (115)

State Grants $89 $6

(15)

Medicaid Reimbursement $119 $24

- (35)

Notes: This table shows the per pupil expenditures and grants per pupil for total spending and special education spending for the 2013-14 school year in 2015 CPI-U adjusted dollars. Districts do not report English Language Learner specific school expenditures. Total enrollment is used to calculate special education spending per pupil (instead of special education enrollment). Items without school-level BPS data do not have standard deviations. If school-level Boston Public Schools (BPS) data is available, BPS statistics are weighted by the proportion of lottery applicants that enroll in individual BPS schools.

A11: School FinancesTotal Special Education

Panel A: Per Pupil Expenditures

Panel B: Federal and State Grants Per Pupil

75

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

(1) (2) (3) (4) (5) (6)

Math -0.983 0.231 -0.504 0.257 -0.113 0.184(0.017) (0.019) (0.011)

N 4826 5407 16648

English -1.147 0.192 -0.729 0.212 -0.114 0.134(0.017) (0.017) (0.009)

N 4831 5419 16627

Math -0.868 0.203 -0.432 0.270 -0.045 0.246(0.047) (0.049) (0.027)

N 4826 5407 16648

English -1.065 0.171 -0.669 0.197 -0.059 0.145(0.048) (0.046) (0.024)

N 4831 5419 16627

Panel B: Reduced Form Estimates

Notes: Panel A reports the Ordinary Least Squares estimates of years spent in charter school on state standardized test scores. Panel B shows the Reduced Form estimates of the effect of getting any charter offer on state standardized test scores. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

A12: Test Score Effects of Years in Charter by Baseline Special Needs StatusSpecial Education English Language Non-Special Needs

Panel A: Ordinary Least Squares Estimates

76

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4)0.614 -0.275 0.389 -0.001

(0.056) (0.085)N 3198 1824

A13: Effect of Charter Enrollment on English Proficiency Exam Scores for Baseline English Language Learners

Take English Proficiency Exam

English Proficiency Exam Score

Notes: This table reports the two-stage least squares estimates of charter enrollment on whether English Language Learners take the annual Spring English Proficiency exam and their scores. Immediate and waitlist offer dummies instrument for charter enrollment in the year following the lottery. Students who remain classified as English Language Learners take the English Proficiency exam. Models control for gender, ethnicity, female x minority interaction, baseline special education, baseline subsidized lunch, experimental strata, year-applied dummies, grade-applied dummies, and baseline English proficiency exam score. Estimates are clustered by school-grade-year.

77

Trad. Public mean Charter effect

Trad. Public mean Charter effect

Trad. Public mean Charter effect

(1) (2) (3) (4) (5) (6)

Math -0.737 0.309 -0.326 0.386 -0.087 0.184(0.123) (0.101) (0.046)

N 171 541 591

English -1.186 0.478 -0.519 0.360 -0.128 0.199(0.148) (0.100) (0.046)

N 169 539 590

Math -1.025 0.245 -0.550 0.306 -0.129 0.257(0.059) (0.052) (0.026)

N 3608 4369 12053

English -1.176 0.177 -0.763 0.200 -0.102 0.142(0.062) (0.050) (0.024)

N 3595 4373 11986

Math -0.920 0.240 -0.419 0.412 -0.086 0.333(0.092) (0.139) (0.053)

N 1030 493 3926

English -1.069 0.160 -0.758 0.412 -0.135 0.214(0.099) (0.170) (0.042)

N 1050 503 3974

Panel C: High School

Notes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores. Immediate and waitlist offer dummies instrument for years spent in charter schools. Columns (1) and (2) show estimates for applicants with baseline special education status, columns (3) and (4) for applicants with baseline English Language Learner classification, and Columns (5) and (6) for other students. All models control for gender, ethnicity, female x minority interaction, baseline special education, baseline ELL, baseline subsidized lunch, experimental strata, year-applied dummies, and grade-applied dummies. Estimates for elementary and middle school sample pool post-lottery outcomes for grades 3-5 and 5-8 respectively. Standard errors are clustered by school-grade-year and the elementary and middle school estimates are also clustered by student identifier.

A14: Test Score Effects by Baseline Special Needs StatusSpecial Education English Language Learner Non-Special Needs

Panel A: Elementary School

Panel B: Middle School

78

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Exam (1) (2) (3) (4)Math -1.055 0.337 -0.917 0.212

(0.068) (0.083)N 2783 2316

English -1.199 0.236 -1.110 0.180(0.070) (0.090)

N 2785 2320

Notes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores for students by their baseline disability type for elementary, middle, and high school applicants. Other disability types include emotional disabilities, intellectual disabilities, autism, communication, physical disabilities, multiple disabilities, developmental disabilities, and health disabilities. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

A15: Test Score Effects of Years in Charter by Baseline Special Learning Other Disability Types

79

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Exam (1) (2) (3) (4) (5) (6)Math -0.567 0.273 -0.731 0.587 -0.236 0.256

(0.058) (0.127) (0.095)N 3120 931 1331

English -0.786 0.210 -0.816 0.451 -0.564 0.083(0.056) (0.124) (0.107)

N 3134 931 1329

A16: Test Score Effects of Years in Charter by First Language of Baseline English Language Spanish Hatian Creole Other

Notes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores for students by their first language for elementary, middle, and high school applicants. Languages in the “Other” category had too few students to individually estimate. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

80

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Trad. Public mean

Charter effect

Baseline Status (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)All Special Education 0.890 -0.117 0.151 0.224

(0.020) (0.022)N 3153 3153

0.967 -0.126 0.834 -0.536 0.049 0.046 0.051 0.250 0.114 0.344(0.028) (0.047) (0.024) (0.035) (0.044)

N 808 808Partial Inclusion 0.914 -0.120 0.627 -0.410 0.173 0.283 0.225 0.333

(0.028) (0.044) (0.037) (0.039)N 1004 1004

Full Inclusion 0.823 -0.097 0.617 0.004 0.117 0.081(0.027) (0.031) (0.022)

N 1278 1278-0.503 0.326 0.001 -0.002 0.003 0.001 0.008 -0.002

(0.045) (0.001) (0.003) (0.004)N 5404

Notes: This table reports ordinary least squares estimates of the effects of Boston charter enrollment on special education classification and level of classroom inclusion in the fall following the charter lottery. Estimation is run separately by baseline classroom inclusion type. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

Substantially Separate Classroom

New Students (No Prior Special Ed. Evaluation)

A17: Ordinary Least Squares Estimates of Charter Enrollment on Special Education Classification

Any Special Education Substantially Separate

Classroom Partial Inclusion Full InclusionMove to More Inclusive

Classroom

81

Trad. Public mean Charter effectBaseline Status (1) (2)

0.825 -0.312(0.031)

N 3763Beginning Proficiency 0.996 -0.050

(0.030)N 287

Intermediate Proficiency 0.954 -0.335(0.039)

N 1810Advanced Proficiency 0.589 -0.255

(0.045)N 1008

0.637 -0.225(0.047)

N 856Notes: This table reports ordinary least squares estimates of the effects of Boston charter enrollment on English Language Learner classification in the fall following the charter lottery. Estimation is run separately by baseline English proficiency level. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

A18: Ordinary Least Squares Estimates of Charter Enrollment on English Language Learner Classification

Remain English Language Learner

All English Language Learners

New Non-native English Speaking Students (No Prior English Lang. Learner

82

Bottom Quartile Top Quartile Bottom Quartile Top Quartile(1) (2) (3) (4)

Math 0.264 0.321 0.241 0.315(0.036) (0.036) (0.049) (0.029)

N 5711 7148 3656 9703

English 0.196 0.207 0.152 0.199(0.035) (0.035) (0.043) (0.028)

N 5640 7156 3608 9706

13.55% 23.32% 14.08% 41.22%(3.99) (5.36) (10.49) (25.11)

A19: Test Score Effects of Years in Charters for Lotteries with High and Low Proportions of Special Needs Applicants

Special Education English Language Learner

Mean % of Lottery Applicants with Special Needs StatusNotes: This table reports the two-stage least squares estimates of the effects of years spent in charter schools on test scores for lotteries with the highest and lowest quartile of special needs representation. Immediate and waitlist offer dummies instrument for years spent in charter schools for elementary, middle, and high school lottery applicants. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

83

Number of Applications

Proportion Matched

Immediate Offer Any Offer

Number of Applications

Proportion Matched

Immediate Offer Any Offer

Number of Applications

Proportion Matched

Immediate Offer Any Offer

Lottery Year (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)2004 150 0.867 0.139 0.074 268 0.989 -0.006 -0.007 638 0.991 -0.015 -0.010

(0.029) (0.071) (0.026) (0.013) (0.013) (0.015)2005 141 0.865 - 0.090 616 0.987 0.005 0.002 601 0.990 0.000 -0.003

- (0.056) (0.011) (0.013) (0.010) (0.010)2006 166 0.910 - 0.098 742 0.991 0.001 0.004 669 0.991 0.002 -0.005

- (0.024) (0.008) (0.016) (0.010) (0.013)2007 303 0.901 0.077 0.043 924 0.984 0.019 0.034 997 0.978 0.008 0.013

(0.026) (0.031) (0.008) (0.013) (0.009) (0.009)2008 322 0.913 0.089 0.082 1018 0.957 0.042 0.061 837 0.957 0.038 -0.002

(0.018) (0.025) (0.013) (0.019) (0.011) (0.030)2009 472 0.960 0.031 0.051 1106 0.977 0.004 0.011 898 0.971 -0.017 0.023

(0.013) (0.015) (0.011) (0.010) (0.020) (0.015)2010 558 0.937 0.013 0.020 1041 0.924 0.065 0.071 917 0.954 0.013 0.027

(0.028) (0.024) (0.016) (0.017) (0.012) (0.013)2011 1610 0.940 0.032 0.033 2614 0.954 0.018 0.025 1234 0.930 0.012 0.020

(0.012) (0.011) (0.007) (0.007) (0.010) (0.013)2012 1864 0.911 0.048 0.048 2503 0.939 0.001 0.033 1499 0.951 0.000 -0.030

(0.014) (0.013) (0.011) (0.011) (0.008) (0.021)2013 1422 0.884 0.032 0.052 2712 0.902 0.045 0.078 1537 0.951 -0.003 -0.120

(0.018) (0.018) (0.012) (0.015) (0.009) (0.078)2014 1085 0.890 0.009 0.020 1938 0.961 0.027 0.036 1403 0.952 0.023 0.111

(0.022) (0.021) (0.007) (0.014) (0.010) (0.106)All Cohorts 8093 0.912 0.036 0.043 15482 0.949 0.023 0.038 11230 0.960 0.007 0.006

(0.007) (0.006) (0.003) (0.004) (0.003) (0.005)Notes: This table summarizes the match from the state administrative data to the lottery records. The sample excludes late applicants, siblings, disqualified applicants, duplicate names, and out-of-area applicants. Columns (3) and (4) report coefficients from regressions on a dummy for a successful state data match on immediate and any charter offer dummies for the elementary school sample. Year-specific regressions control for charter school dummies. All cohort regressions control for school-by-year dummies.

A20: Match from Lottery Data to Administrative DataElementary School Middle School High School

Reg of Match on Offer Reg of Match on Offer Reg of Match on Offer

84

Year of application 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 All

Total number of records 160 166 194 364 396 602 702 2899 2963 2537 2298 13281Excluding disqualifed applications 160 166 194 360 396 602 702 2889 2956 2479 2280 13184Excluding late applications 160 166 194 360 396 602 700 2882 2956 2470 2279 13165Excluding out of area applications 160 160 194 357 395 590 687 2832 2874 2408 2233 12890Excluding siblings 151 140 166 325 338 525 621 2330 2508 2101 2038 11243Excluding records not matched to SIMS 131 123 151 296 310 507 585 2225 2336 1942 1858 10464Keep only first year of charter application 131 123 151 273 294 491 555 1965 2069 1633 1398 9083Excluding repeat applications 131 121 151 273 294 491 551 1954 2041 1618 1396 9021Reshaping to one record per student 130 119 138 261 284 409 393 1336 1427 1041 918 6937Has any demographics 130 119 150 262 285 426 484 1391 1430 1060 832 6569Has demographics for baseline and/or year 1 29 37 54 205 228 345 392 1156 1131 874 805 5256Has baseline demographics 1 5 3 26 56 68 62 613 472 249 388 1943

Total number of records 341 739 913 1143 1422 1595 1467 4283 4312 4766 3189 24170Excluding disqualifed applications 341 738 911 1135 1404 1594 1444 4273 4305 4760 3189 24094Excluding late applications 340 738 909 1135 1363 1566 1397 4163 4196 4583 3187 23577Excluding out of area applications 340 733 900 1123 1353 1548 1379 4094 4071 4513 3136 23190Excluding siblings 300 677 836 1021 1223 1408 1249 3758 3760 4320 2865 21417Excluding records not matched to SIMS 266 634 801 1000 1181 1378 1179 3627 3573 4016 2792 20447Keep only first year of charter application 266 617 770 962 1093 1282 1038 3308 2962 3469 1975 17742Excluding repeat applications 266 617 770 962 1093 1282 1038 3308 2962 3458 1960 17716Reshaping to one record per student 265 523 586 760 868 963 812 2055 1715 1900 1176 11623Has baseline demographics and in Boston at baseline 176 382 437 571 679 722 623 1790 1499 1594 1028 9501

Total number of records 940 884 942 1330 1211 1300 1500 1835 2049 3280 3417 18688Excluding disqualifed applications 940 883 942 1327 1210 1289 1500 1818 2040 3278 3417 18644Excluding late applications 930 880 942 1327 1191 1289 1500 1818 1986 3235 3417 18515Excluding out of area applications 930 880 939 1327 1191 1276 1465 1787 1979 3136 2762 17672Excluding siblings 905 864 939 1298 1153 1214 1376 1727 1952 3082 2658 17168Excluding records not matched to SIMS 858 817 919 1271 1108 1184 1335 1642 1882 2980 2571 16567Keep only first year of charter application 858 810 910 1161 919 925 984 1208 1369 2192 1416 12752Excluding repeat applications 858 810 910 1161 919 925 984 1208 1366 2187 1414 12742Reshaping to one record per student 632 590 656 827 604 629 591 736 786 928 652 7631Has baseline demographics and in Boston at baseline 508 478 536 751 487 529 503 628 735 848 552 6555

A21: Sample Selection

Panel A: Elementary School

Panel B: Middle School

Panel C: High School

Notes: This table shows the sample restrictions imposed for lottery analysis.

85

Trad. Public Attrition Rate

Attrition Differential by

Offer StatusTrad. Public

Attrition Rate

Attrition Differential by

Offer StatusTrad. Public

Attrition Rate

Attrition Differential by

Offer StatusOutcome (1) (2) (3) (4) (5) (6)Math Exam 0.229 0.008 0.182 -0.003 0.224 -0.028

(0.013) (0.010) (0.006)5861 6234 19442

English Exam 0.225 0.003 0.180 -0.009 0.222 -0.029(0.013) (0.010) (0.006)5861 6234 19442

Classification Status 0.109 -0.049 0.090 -0.042 0.101 -0.099(0.032) (0.024) (0.019)3245 3709 10348

Notes: This table reports the two-stage least squares estimates of the effect of years spent in charter schools on attriting from the sample for test score and reclassification outcomes. Standard errors are clustered by student identifier and school-grade-year. See Table 4 notes for detailed regression specifications.

A22: Attrition

Special Education at Baseline English Language Learner at Baseline Non-Special Needs at Baseline

86

Trad. Public mean Effect

Trad. Public mean Effect

Trad. Public mean Effect

(1) (2) (3) (4) (5) (6)

Any Switch 0.498 0.253 0.373 -0.002 0.440 -0.120(0.151) (0.057) (0.045)

N 296 864 858

0.308 0.095 0.212 -0.138 0.230 -0.173(0.139) (0.046) (0.041)

N 296 864 858

Any Switch 0.549 -0.160 0.556 -0.176 0.598 -0.393(0.051) (0.043) (0.031)

N 1820 2314 5263

0.160 0.018 0.144 -0.063 0.205 -0.119(0.039) (0.032) (0.023)

N 1820 2314 5263

Any Switch 0.296 0.257 0.337 0.068 0.262 0.068(0.102) (0.117) (0.057)

N 1259 741 4040

0.206 0.299 0.178 0.178 0.168 0.073(0.099) (0.114) (0.055)

N 1259 741 4040

Panel B: Middle School

Switch excluding transitional grades

Panel C: High School

Switch excluding transitional grades

Notes: This table reports two-stage least squares estimates of the effects of Boston charter enrollment on switching schools one year following the lottery. Students who do not appear in Massachusetts public schools in October following the charter application are not counted as school switchers. The switch excluding transitional grades equals one for students who switch schools in grades other than the exit grade of their first school. It does not equal to one if the school closed the year the student switched. Standard errors are clustered by school-grade-year. See Table 2 notes for detailed regression specifications.

Switch excluding transitional grades

A23: Effect of Charter Enrollment on School Switching by Baseline Special Needs Status

Special EducationEnglish Language

Learner Non-Special Needs

Panel A: Elementary School

87

Special EducationEnglish Language

Learner Non-Special Needs

(1) (2) (3)

Untreated Complier Mean: Math 0.003 0.013 0.003

(0.011) (0.010) (0.006)

N 1567 2002 4556

Untreated Complier Mean: English -0.017 -0.029 -0.018

(0.011) (0.009) (0.006)

N 1567 2002 4556

A24: Value-Added of Fallback Schools for Charter Applicants

Notes: This table summarizes OLS value-added estimates for schools attended by untreated charter lottery compliers. Untreated complier means are estimates from 2SLS regressions of school-value added interacted with a traditional public school indicator on a set of variables equal to one minus a charter enrollment indicator. The model uses lottery offers as instruments and controls for demographics and experimental strata. School value-added estimates come from OLS regressions of test scores on a set of school indicator variables, controlling for lagged test scores and student demographics.

88