34
School choice and student achievement – new evidence on open-enrolment Martin Söderström WORKING PAPER 2006:16

School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

School choice and studentachievement

– new evidence on open-enrolment

Martin Söderström

WORKING PAPER 2006:16

Page 2: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

The Institute for Labour Market Policy Evaluation (IFAU) is a research insti-tute under the Swedish Ministry of Industry, Employment and Communica-tions, situated in Uppsala. IFAU’s objective is to promote, support and carry out: evaluations of the effects of labour market policies, studies of the function-ing of the labour market and evaluations of the labour market effects of meas-ures within the educational system. Besides research, IFAU also works on: spreading knowledge about the activities of the institute through publications, seminars, courses, workshops and conferences; influencing the collection of data and making data easily available to researchers all over the country. IFAU also provides funding for research projects within its areas of interest. The deadline for applications is October 1 each year. Since the researchers at IFAU are mainly economists, researchers from other disciplines are encouraged to apply for funding. IFAU is run by a Director-General. The authority has a traditional board, consisting of a chairman, the Director-General and eight other members. The tasks of the board are, among other things, to make decisions about external grants and give its views on the activities at IFAU. A reference group including representatives for employers and employees as well as the ministries and authorities concerned is also connected to the institute. Postal address: P.O. Box 513, 751 20 Uppsala Visiting address: Kyrkogårdsgatan 6, Uppsala Phone: +46 18 471 70 70 Fax: +46 18 471 70 71 [email protected] www.ifau.se Papers published in the Working Paper Series should, according to the IFAU policy, have been discussed at seminars held at IFAU and at least one other academic forum, and have been read by one external and one internal referee. They need not, however, have undergone the standard scrutiny for publication in a scientific journal. The pur-pose of the Working Paper Series is to provide a factual basis for public policy and the public policy discussion.

ISSN 1651-1166

Page 3: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

School choice and student achievement – new evidence on open-enrolment*

by

Martin Söderströma

December 4, 2006

Abstract This paper studies the effects of open-enrolment on student performance in the context of an admission reform in Stockholm. Before 2000, students had priority to the public upper secondary school situated closest to where they lived, but from the fall of 2000 and onwards, admission is based on grades only. The reform imposed strong incentives for school competition: all students can apply to all schools, there is no targeting of students to schools, and funding follows the students. It is shown that the students in Stockholm per-form no better with increased choice availability. In fact, high ability students seem to perform worse after the reform.

* I would like to thank Peter Frediksson, Per-Anders Edin, Torbjørn Hægeland, Kjell Salvanes, Roope Uusitalo, Hessel Oosterbeek, Eric Plug, and seminar participants at Uppsala University, Labor Institute of Economic Research, and Tinbergen Institute for helpful comments. Thanks also to Louise Kennerberg and Helge Bennmarker at IFAU for help with data. Financial support from the Swedish Council for Working Life and Social Research is gratefully acknowledged. a Department of Economics, Uppsala University. E-mail: [email protected].

IFAU – School choice and student achievement – new evidence on open-enrolment 1

Page 4: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table of contents

1 Introduction ............................................................................................... 3

2 The Swedish school system....................................................................... 5

3 The Stockholm admission reform.............................................................. 7

4 Data............................................................................................................ 8

5 Results ..................................................................................................... 12 5.1 Between school variance in student outcomes ........................................ 12 5.2 Reform effects on student outcomes........................................................ 13 5.3 Robustness checks ................................................................................... 17

6 Conclusions ............................................................................................. 20

References.......................................................................................................... 22

Appendix A – Model specification .................................................................... 24

Appendix B – Tests for grade inflation.............................................................. 27

IFAU – School choice and student achievement – new evidence on open-enrolment 2

Page 5: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

1 Introduction In most countries, students attend their nearest school. School choice can simply be described as the possibility for students to choose other alternatives than their neighborhood school. The argument for increasing choice availability relies on the idea that students (or parents) prefer better schools. If there are financial incentives for schools to attract students, competition will then increase productivity, since it will be profitable to be a better school.1

The effects of increasing the possibilities to choose among public schools have been debated over the last decades.2 The available evidence mainly concerns charter school experiments in the US. In this literature, one can find some reliable evidence when the authors control for the selection issues associated with these reforms.3 Cullen, Jacob & Levitt (2005) is one of the few studies on the effects of open-enrolment on student achievement. They study 9th graders in Chicago who are guaranteed a slot in a pre-assigned neigh-borhood high school, but have the possibility to opt for other public schools in the district. Their results show that students are opting out of neighborhood schools, but that this behavior is not associated with any academic gains except for students in career academics.

It is easy to explain why there are so few studies on open-enrolment: reforms of this type do usually not qualify as choice reforms since they fail to create competitive environments. That is, the choice process is often restricted by targeting specific students to specific schools, or there are no financial incentives for schools to attract students. This paper aims at shedding new light on the question of open-enrolment and student achievement by analyzing a reform that generated truly strong incentives for competition.

In 2000, the Stockholm municipality changed the admission system of public upper secondary schools. A residence-based principle was abolished for a strict grade procedure. This means that all students can apply to all schools. Furthermore, funding follows the students, and since there are financial fixed costs this creates incentives for schools to attract students. There are no

1 For a discussion of the pros and cons of school choice, see Hoxby (2003b), and Fiske & Ladd (2000). 2 See Björklund et al. (2005) for an overview of the school choice debate. 3 Hoxby (2003a), Hanushek et al. (2005), and Bettinger (2005) are examples of studies on charter school experiments. However, their results differ: Hoxby (2003a) finds that charter schools improve student performance, while Hanushek et al. (2005) and Bettinger (2005) do not find such an effect.

IFAU – School choice and student achievement – new evidence on open-enrolment 3

Page 6: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

exceptions in the admission procedure, and there is no targeting of students to schools. Hence, competition is so strong, that unlike most choice reforms (which usually are designed to help low-ability students), this reform could actually force low-ability students away from their neighborhood school.

Söderström & Uusitalo (2005) have shown how the Stockholm reform affected the allocation of students across schools within the city. They show that student mobility and segregation – measured in terms of ability, immigrant status, and family background – increased.4 This is clear evidence that the Stockholm students are exercising choice.

This paper studies what impact the Stockholm reform had on student performance by utilizing a difference-in-differences strategy. The change in performance for Stockholm students before and after the reform is compared to the corresponding change for students in the surrounding municipalities, where the residence-based admission procedure is still in use.

The results show that the students in Stockholm perform no better compared to the students in the surrounding municipalities after the reform. In fact, the point estimate suggests a decrease of about half a percentile rank on average; this estimate is insignificant however. The interaction estimates suggest some surprising and some interesting results. Surprisingly, estimates by ability show that losses due to the reform seem to be concentrated to the higher end of the ability distribution. Further, as a consequence of the reform, the school performance of immigrants was reduced by one and a half percentile ranks. Given the nature of the reform and the results in Söderström & Uusitalo (2005), this is not surprising, but still of interest. Also children from high-income families were hurt by the reform. The performance of these children decreased with a magnitude similar to that of immigrant children.

The robustness checks suggest that the difference between groups to some extent can be explained by differences in completion probabilities. For example, the negative reform effects for immigrants and students from high-income families are explained by changes in completion rates. Further, high-ability students in Stockholm tend to complete to a lesser extent after the reform. However, the negative reform effect for this group is still significant conditional on completion. The robustness checks also show that the negative

4 Maybe the most interesting result in Söderström & Uusitalo (2005) is that segregation by immigrant status increases also conditional on ability. This suggests that the opportunities of the reform were not conceived by all immigrants, or that immigrants have different attitudes towards schooling.

IFAU – School choice and student achievement – new evidence on open-enrolment 4

Page 7: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

reform effect is larger among those subjects not being common between students.

Throughout the paper, school performance is measured by school grades. A priori this is not ideal. Therefore, considerable effort is put into examining whether the results are confounded by issues such as grade inflation and relative grade setting. These sensitivity checks suggest that the results are not distorted by having grades rather than, e.g., test scores, as a measure of student performance.

The contribution of this paper is twofold. First, it is shown that this open-enrolment reform has not enhanced student performance, at least not with respect to the first cohort of students affected by the reform. Second, it is shown that those who should gain in the choice process (i.e. those with high grades from compulsory school) do in fact perform worse with increased choice availability. The natural explanation seems to be that students misin-terpret school productivities when they choose schools. That is, they only observe raw measures of student performance, such as grades and test scores, but they do not know whether they reflect educational production or student characteristics.5 Hence, the findings and conclusions are very much in line with Cullen, Jacob & Levitt (2005).

The paper proceeds as follows. Section 2 describes the Swedish school system, Section 3 the admission reform, and Section 4 the data. Section 5 presents the results, and Section 6 concludes.

2 The Swedish school system The Swedish schools are governed by the municipalities within the guidelines and criteria set by the National Agency for Education. Each municipality is given funding from the federal budget, and is then free to determine how much to spend in the educational sector. At the municipal level, a specific amount of money is attached to each student, following the student to whatever school he or she attends.

Sweden has nine years of compulsory schooling, between the ages of 7 and 16. Almost all students that complete 9th grade continue to upper secondary schooling, which consists of three-year programs. The programs can be of three

5 A more far-fetched explanation is that students choose schools based on non-academic criteria.

IFAU – School choice and student achievement – new evidence on open-enrolment 5

Page 8: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

different types: national programs (all giving eligibility to post-secondary education), individual programs (intended to prepare the students for national programs later on), or special programs (for example handicraft or art).

These programs were introduced in 1994 along with goal-oriented grades. The programs consist of courses, ranging from 50 to 200 points depending on the extent of the course. In total a program consists of 2 500 points.6 Each course is given one of four possible grades; Fail, Pass, Pass with distinction, and Pass with special distinction, which earn the student 0, 10, 15, or 20 credits respectively. The final grade point average (GPA) is calculated as a weighted average over courses:

⎟⎠

⎞⎜⎝

⎛⎟⎠

⎞⎜⎝

⎛= ∑∑

==

n

ii

n

iii PCPGPA

11/*

where Pi denotes points, Ci credits, and subscript i indicates course. Thus, more weight is given to longer courses, and the GPA is ranging from 0 (worst) to 20 (best). The subjects differ between programs; consequently the GPA is based on different subjects in different programs. However, there are eight subjects that all students have to take irrespective of program attended, the so called “core subjects”.7 The average of the core subjects (GPAcore) is the main outcome measure used in this paper.

All grades are set by the teacher, and should be given according to subject-specific assessment criteria determined by the National Agency for Education. As further guidelines there are national tests in Swedish, Math, and English. There is no supervision of the teacher grade setting, and hence there is no guarantee that grading standards are equal across teachers.

Students who began upper secondary schooling prior to 1994 were graded in a norm-based reference system. It is documented that the average grade has been increasing in Swedish upper secondary schools since the introduction of the goal-oriented grade system (Wikström, 2005). Wikström & Wikström (2005) study whether this increase is due to grade inflation induced by

6 This holds for 2000. Before 2000, courses ranged 20-200 points with a total of 2 150 or 2 370. 7 The core subjects are Mathematics, Swedish, English, Sport, Religion, Art, Natural Science, and Social Science.

IFAU – School choice and student achievement – new evidence on open-enrolment 6

Page 9: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

increased school competition.8 They do not find that intra-municipal school competition induces grade inflation among public schools; however, their results suggest that private schools inflate grades.

Historically, private schools have been very rare in Sweden. However, after a school reform in the beginning of the 1990s which made private schools entitled to municipal funding the number of private schools has been in-creasing, particularly in the metropolitan areas. The few private schools that are financed by tuition fees and are using selection rules are not entitled to municipal funding.

3 The Stockholm admission reform The responsibility for the educational system rests in the hands of the municipalities. Hence, the municipalities can choose admission procedure to upper secondary schooling. Stockholm changed this procedure in 2000.9 The students who began upper secondary schooling in the fall of 2000 were the first cohort in a system where admission is based on grades only. Students apply for a specific program in a specific school. If they are not accepted to their first choice, a second is considered, and so forth. Note that there can only be one first choice, not one first choice per school. Prior to 2000, students applied for a program only. Given acceptance (based on grades), students were allocated to their nearest school.10 It should be noted that the reform did not change the way in which students from outside Stockholm could apply for schools within Stockholm.11

8 The authors compare grades from upper secondary schooling with SweSAT test scores, the two main routes for applying to university. The SweSAT is given twice a year throughout the country and is open to anyone. 9 The decision was taken by the centre-right wing coalition of Stockholm on October 18, 1999. 10 The residence-based principle is best described with an example. The Local Admission Unit counted the number of slots in total for each program in the public schools, for example 300 for the program in Natural Science. The students only applied for a program, and they were ranked according to grades. To the program in Natural Science, the 300 with best grades were accepted. Given acceptance, the Local Admission Unit studied every single student, and allocated them to schools by address, minimizing travelling distance to school. 11 A student from outside Stockholm can apply for a national program within Stockholm if that program is not given in the municipality where the student resides, and if the home-municipality does not have an agreement with another municipality.

IFAU – School choice and student achievement – new evidence on open-enrolment 7

Page 10: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

The Stockholm admission reform fits the criteria of being considered a school choice reform. The funding follows the student, making the schools financially dependent on how many students they attract. Furthermore, schools have the possibility to take on more students, or lose students. However, in contrast to most other choice reforms - for example voucher initiatives in the US, which often are directed to increase choice availability among low ability students - this reform could actually be detrimental to low-ability students if they were no longer admitted to their neighborhood school.

The argument in favor of school choice is that extending choice increases competition. Hence, the next question is if there was an increase in competition. Söderström & Uusitalo (2005) conclude that the mobility of students, as well as sorting of students over schools, increased in Stockholm in response to the reform. For example, the average commuting distance for a student increased from 4.2 km in 1999 to 4.8 km in 2000, and the segregation by previous grades increased from 40% to 54% over the same period. This is clear evidence that students are exercising choice, leading to increased school competition. There is also considerable evidence in how schools nowadays are promoting themselves.12 But perhaps the best argument for increased competition is the fact that some public schools are struggling for survival. In fact, it was recently decided that one of the schools had to be shut down. Thus, arguably school competition has increased in Stockholm after the reform.

4 Data The data come from Statistics Sweden. It is constructed by all students registered the first semester at an upper secondary school situated in the Stockholm County.13 Stockholm County consists of 26 municipalities. The Stockholm municipality (hereafter simply denoted Stockholm) is the treatment group and the other 25 municipalities constitute the potential comparison group. Data are available for six consecutive years, from 1995 to 2000, and give information on at which school and program the students are registered. Note that data only include one post-reform cohort. To these data, personal,

12 For example, there are exhibitions for ninth-graders, and advertisements in newspapers. 13 The population of students is defined using the register of applicants and admissions to upper secondary schooling.

IFAU – School choice and student achievement – new evidence on open-enrolment 8

Page 11: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

family, compulsory school, and upper secondary school information are matched to the students.14

The data are restricted in three ways. First, only students attending national programs are used. The reason is that individual programs are preparing the student for taking national programs later on, and the special programs are diverse and not easily comparable. In both treatment and comparison groups, about three quarters of the students register at national programs. Second, since grades from upper secondary schooling are only available until 2003, grades are only used for those students completing in the stipulated three years time. Third, a small number of individuals (less than 1% of the sample) are excluded due to missing information on compulsory school grades.

Table 1 reports background characteristics for Stockholm (S) and the Comparison group (C) for each year. The students are sorted into the two groups depending on where the upper secondary school they are registered at is located. Immigrant status is defined as those who are born outside Sweden, or has at least one parent who is born outside Sweden. Parental education is an indicator equaling unity if the student has at least one parent with a university degree. Parental earnings (in thousands of SEK) are the sum of the parents’ earnings, thereby also capturing the effect of having one or two (working) parents. The compulsory school GPA is percentile ranked over both groups.15 Compulsory school GPASME corresponds to the average of the percentile ranked grades in Swedish, Math, and English. Table 1 also includes information on the share of students in private schools, and the number of schools and students. Data include further information (not shown in tables) on compulsory school attended, residential information (municipality where the student resides), and program attended.

14 Different registers from Statistics Sweden are used. The education registers of compulsory and upper secondary schooling give educational information and grades. Information on personal and family characteristics comes from a longitudinal register of education and income, which is a combination of income tax registers, population censuses, and other sources. 15 Note that students leaving compulsory school prior to 1998 were graded on a norm-based scale from 1-5, and from 1998 and onwards in the goal-oriented system.

IFAU – School choice and student achievement – new evidence on open-enrolment 9

Page 12: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table 1 Characteristics by year of entry into upper secondary school.

1995 1996 1997 1998 1999 2000

Female S 0.488 (0.500)

0.490 (0.500)

0.477 (0.500)

0.502 (0.500)

0.501 (0.500)

0.514 (0.500)

C 0.469 (0.494)

0.474 (0.499)

0.468 (0.499)

0.479 (0.500)

0.475 (0.499)

0.462 (0.499)

Immigrants S 0.309 (0.462)

0.309 (0.462)

0.315 (0.464)

0.315 (0.465)

0.369 (0.482)

0.370 (0.483)

C 0.302 (0.459)

0.325 (0.468)

0.325 (0.468)

0.327 (0.469)

0.364 (0.481)

0.363 (0.481)

Age S 16.27 (0.69)

16.27 (0.74)

16.28 (0.75)

16.30 (0.90)

16.26 (0.80)

16.32 (1.03)

C 16.20 (0.52)

16.19 (0.49)

16.20 (0.51)

16.21 (0.53)

16.22 (0.52)

16.21 (0.51)

Parental education

S 0.469 (0.499)

0.479 (0.500)

0.489 (0.500)

0.509 (0.500)

0.519 (0.500)

0.525 (0.499)

C 0.415 (0.493)

0.424 (0.494)

0.416 (0.493)

0.442 (0.497)

0.422 (0.494)

0.428 (0.495)

Parental earnings

S 312.5 (292.0)

327.2 (284.1)

342.7 (308.5)

365.5 (342.7)

388.6 (338.3)

408.9 (411.0)

C 320.0 (240.7)

337.0 (252.7)

343.5 (302.2)

356.1 (302.0)

366.5 (308.0)

381.1 (362.9)

Compulsory school GPA

S 51.10 (29.36)

50.89 (29.63)

51.43 (29.65)

52.37 (28.82)

55.75 (28.68)

55.39 (28.96)

C 50.13 (28.47)

50.31 (28.33)

50.01 (28.30)

48.65 (28.99)

46.29 (28.62)

46.69 (28.16)

Compulsory school GPASME

S 51.31 (22.99)

51.36 (23.43)

51.76 (23.60)

52.02 (22.32)

54.32 (22.41)

54.26 (22.56)

C 49.90 (22.14)

49.98 (22.35)

49.62 (22.58)

49.27 (21.90)

47.99 (21.37)

47.98 (21.37)

Share of private school students

S 0.055 (0.228)

0.062 (0.242)

0.051 (0.219)

0.126 (0.332)

0.188 (0.391)

0.207 (0.405)

C 0.068 (0.252)

0.065 (0.247)

0.073 (0.260)

0.077 (0.267)

0.095 (0.294)

0.121 (0.326)

# schools S 30 32 30 33 34 41 C 44 46 48 47 50 56 # students S 6 754 6 823 6 683 5 989 6 245 6 556 C 10 883 11 439 10 928 9 975 9 477 9 686

Note: Standard deviations are reported in parentheses. Immigrant status is defined as those who are born outside Sweden, or has at least one parent who is born outside Sweden. Parental education is an indicator equaling unity if the student has at least one parent with a university degree. Parental earnings are the sum of the parents’ earnings, measured in thousands of SEK. Compulsory school GPA is percentile ranked over both groups per year. Compulsory GPASME is the average of the percentile ranked grades in Swedish, English, and Math.

IFAU – School choice and student achievement – new evidence on open-enrolment 10

Page 13: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

According to Table 1, the composition of students in 1995 is fairly similar in the two groups. However, the composition in Stockholm appears to change over time. For example, the share of females in Stockholm increases from 47.7% in 1997, to 51.4% in 2000, while it remains fairly constant in the comparison group. The share of students from families with an academic background increases from 48.9% to 52.5% in Stockholm over the same period, and parental earnings increase by more than in the comparison group. The share of immigrants, on the other hand, has a similar time pattern in the two groups.

The explanation for the observed changes in student composition seems to be found in the share of students attending private schools. Between 1997 and 2000, this share increased from 5.1% to 20.7% in Stockholm. The number of schools increased from 30 to 41 over the same period, exclusively due to new private schools. Further, compulsory school GPA increases from 51.4 in 1997, to 55.4 in 2000. As the average GPA for public school students only increased from 50.4 to 52.8 during the same period it appears that there was an inflow to Stockholm of high ability students attending private schools. It should be noted that the bulk of the increase in the share of private school students in Stockholm occurred between 1997 and 1999. For the reform year (between 1999 and 2000), the share of students in private schools increased by about the same amount in both groups.

Table 2 displays information on student performance at the upper secondary level. Grades from upper secondary school are shown as a weighted average over all grades (GPA), and over core subjects (GPAcore). The grade measures are percentile ranked over the full sample of completing students. Upper secondary schooling is intended to last for three years, and a considerable fraction of the students fails to get a final grade within these three years. The share of students not completing in the stipulated time corresponds to these students.

Grades are higher in Stockholm than in the comparison group, and the difference gets larger over time, as can be seen in Table 2. The GPA in Stockholm increases from 52.86 in 1995 to 54.32 in 2000, and the same pattern holds for GPAcore. These changes come as no surprise since Table 1 shows that Stockholm students have increasingly better compulsory school grades. Table 2 also shows that the share of students completing in three years varies around two thirds, with no clear pattern and no marked differences across the two groups. In all, Table 2 does not indicate any reform effects.

IFAU – School choice and student achievement – new evidence on open-enrolment 11

Page 14: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table 2 School outcomes by year of entry into upper secondary school.

1995 1996 1997 1998 1999 2000

GPA S 52.86 (29.51)

53.03 (29.81)

54.01 (29.45)

53.48 (29.11)

54.42 (28.99)

54.32 (29.14)

C 48.34 (28.35)

48.30 (28.10)

47.84 (28.27)

48.06 (28.47)

46.94 (28.33)

47.20 (28.27)

GPAcore S 53.88 (29.30)

53.93 (29.69)

54.81 (29.14)

54.37 (28.61)

55.30 (28.68)

55.67 (28.74)

C 48.81 (28.43)

48.84 (28.15)

48.37 (28.40)

48.57 (28.72)

47.41 (28.51)

47.03 (28.28)

Share of students completing in the

S 0.621 (0.485)

0.607 (0.488)

0.626 (0.484)

0.605 (0.489)

0.666 (0.472)

0.663 (0.473)

stipulated time C 0.661 (0.473)

0.639 (0.480)

0.667 (0.471)

0.625 (0.484)

0.633 (0.482)

0.650 (0.477)

Note: Standard deviations within parentheses. GPA and GPAcore are for each year percentile ranked over the full sample of completing students. The core subjects are Mathematics, Swedish, English, Sport, Religion, Natural Science, Social Science, and Art. The share of students completing in the stipulated time corresponds to the students who receive a final grade after three years of study at the upper secondary level.

5 Results This section begins with a brief descriptive analysis. Then the difference-in-differences model is specified and estimated on the full sample to identify average treatment effects, and on sub-samples to identify heterogeneous treatment effects. Robustness checks include the impact of sorting, relative grade setting, completion rates, and grade inflation. 5.1 Between school variance in student outcomes According to the descriptive statistics in Table 2, Stockholm students have higher upper secondary grades than the students in the comparison group. This difference becomes larger over time but it is hard to spot any indication of a reform effect. In order to examine whether there are any easily depicted effects of the reform on grades, the school level is studied. Since the sorting of students with respect to compulsory school grades changed after the reform, we should also expect the differences between upper secondary schools to increase in terms of student performance.

IFAU – School choice and student achievement – new evidence on open-enrolment 12

Page 15: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table 3 presents ANOVA-results on the between school variance in student outcomes. Percentile ranked core grades are regressed on school fixed effects, and the results presented are the R2-adj from these regressions.

The upper part of Table 3 shows the result for all schools. It can be noted that the between school variance is larger in Stockholm, and that it is fairly stable for the years prior to the reform. More importantly, the results indicate a reform effect in Stockholm. In 1999, 20.7% of the variation in core grades could be explained by school attended; this figure increases to 29.7% in 2000. The magnitude of the increase is much larger than in the comparison group.

Table 3 Between school variance in student outcomes.

R2-adj from regressions of core grades on school fixed effects

1995 1996 1997 1998 1999 2000 All schools

Stockholm 0.116 0.178 0.184 0.174 0.207 0.297 Comparison 0.062 0.061 0.072 0.104 0.107 0.134

Public schools

Stockholm 0.089 0.145 0.160 0.125 0.155 0.261 Comparison 0.058 0.039 0.055 0.069 0.059 0.077

The observed pattern could be due to private schools. The lower part of

Table 3 presents the same analysis, now using students in public schools only. The pattern is even stronger among public schools: the between school variance among public schools in Stockholm increases from 15.5% to 26.1% between 1999 and 2000.

Due to the selection issue discussed above, this result comes as no surprise. An increased sorting of students by ability over schools contributes to the observed pattern. Nonetheless, this is a clear indication that the reform affected students at upper secondary level. The next section studies the impact of the reform on student performance. 5.2 Reform effects on student outcomes In order to identify the causal effect of the reform on student performance, a difference-in-differences analysis is used. That is, a comparison group is used

IFAU – School choice and student achievement – new evidence on open-enrolment 13

Page 16: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

to resemble what the situation would have been like in Stockholm without the reform. A fundamental criterion for this identification strategy to work properly is that there are no differences in trends in the outcome measure between the two groups. If such a difference exists, it will be captured by the reform indicator and bias the estimated reform effect.

For this study, five pre-reform years (1995-1999) and 25 municipalities constituting a potential comparison group are available. Hence, pre-reform trends at the municipal level can be estimated, and by applying a common support restriction municipalities with trends significantly different from Stockholm will be excluded from the Comparison group. Appendix A de-scribes how this procedure is conducted, and how four municipalities were excluded according to the common support restriction. The model specification then boils down to a difference-in-differences analysis for 1999-2000. It is worth noting that potential problems with the inflow of private school students to Stockholm (see Table 1) are minimized using this procedure, since this compositional change mainly occurred prior to 1999.

Equation (1) below is estimated over the years 1999-2000. The dependent variable, , is the weighted average of core grades (percentile ranked).core

istGPA 16 Subscript i denotes student, s school, and t time. Explanatory variables are personal characteristics (Xist), school dummies (γs), time dummies (γt), and a product of a Stockholm school indicator with the year 2000 (SS2000). The reform effect is identified through SS2000. The personal characteristics include age, gender, immigrant status, parental education, parental earnings (percentile ranked since the mean and variance vary over time), program, municipality where the student resides17, compulsory school GPA18, and compulsory school attended.

(1) 2000

istittsistcoreist SSXGPA εϕγγβα +++++=

Students may be affected differently by the reform. Some students did react

to the new opportunities and attended a school which they otherwise would not

16 If a student has not completed upper secondary schooling in three years, (s)he is given the grade 0. 17 Students living outside the Stockholm County are grouped into one category. 18 The specification assumes a linear effect of compulsory school grades. More flexible functional forms with dummies per quintile or decile have been tried, and the results are virtually unchanged. The results are also very similar using ordinary grades instead of percentile ranks.

IFAU – School choice and student achievement – new evidence on open-enrolment 14

Page 17: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

have had access to. Other students were forced away from their neighborhood school due to competition. By estimating separate regressions by gender, grades from compulsory school, family background and immigrant status, potential heterogeneous treatment effects are examined. Table 4 displays the results (reform effect estimates) for the full sample, and when the sample is restricted to specific groups.

Column (1) in the upper part of Table 4 presents the average treatment effect. According to the table, the students in Stockholm perform half a percentile worse after the reform, however, the result is not significant. Column (2) includes only students attending public schools. Compared to the average effect in column (1) there is a small decrease in the magnitude of the reform effect. The estimate is still insignificant, though. Thus, it does not matter whether we consider public or private schools. In column (3) only those students attending theoretical programs are included, and the effect is larger in absolute value, 1.13, but still insignificant.19

Column (4) and Column (5) present separate regressions by gender. The results are striking. Females perform significantly worse after the reform with a point estimate of -1.81, while the reform effect for males is insignificantly positive.

The mid-section of Table 4 shows the impact on students with different ability, measured by compulsory school grades. The results are displayed for the four quartiles of ability; Q1 being the lowest, and Q4 the highest achieving quartile. The results indicate substantial differences between groups. The lowest achieving quartile has a positive insignificant estimate, 1.15. Quartile 2 and 3 perform significantly worse after the reform with point estimates of -2.00 and -2.47 respectively. The top achieving quartile has an insignificant estimate of -2.29 (p-value 0.125).

The lower part of Table 4 presents result where student performance is studied according to immigrant status and socio-economic background. Low parental education indicates students for which neither parent has a university degree. Low parental earnings indicate if the students’ parents belong to the lowest quartile in the parental earnings distribution. Correspondingly, high parental earnings are the top quartile of the earnings distribution. Students with high-income parents are performing significantly worse after the reform, with a point estimate of -1.75 percentile ranks. Likewise, immigrants do perform 1.5 percentile ranks worse (significant at the 10%-level). 19 The programs in natural and social sciences are academic tracks, denoted theoretical programs.

IFAU – School choice and student achievement – new evidence on open-enrolment 15

Page 18: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table 4 Reform effects.

Average reform effect and effects in public schools, theoretical programs, and by gender

All students Public schools Theoretical programs

Females Males

SS2000 -0.567 (0.587)

-0.369 (0.624)

-1.126 (0.848)

-1.805** (0.878)

0.858 (0.805)

R2-adj 0.545 0.542 0.495 0.530 0.551

N 28 811 24 295 17 039 13 951 14 860

Reform effects by quartiles of the ability distribution

Q1 (low) Q2 Q3 Q4 (high)

SS2000 1.147 (0.733)

-2.001* (1.067)

-2.473* (1.364)

-2.294 (1.496)

R2-adj 0.178 0.162 0.198 0.209

N 7 441 7 280 7 139 6 951

Reform effects by immigrant status and socio-economic background

Immigrants Low parental education

Low parental earnings

High parental earnings

SS2000 -1.546* (0.921)

-1.070 (0.748)

0.710 (1.109)

-1.746* (1.350)

R2-adj 0.533 0.519 0.496 0.491

N 10 421 15 149 7 041 7 038 Note: The outcome variable is percentile ranked core grades. Regressions include a constant, and controls for gender, age, immigrant status, compulsory school grades, parental education and earnings, time effects, upper secondary school attended, compulsory school attended, program, and municipality where the student resides. Low parental education identifies students without parents with a university degree. Low (high) parental earnings define students with parents in the lowest (highest) quartile of the parental earnings distribution. Robust standard errors are in parentheses. Significance level: * = 10%, ** = 5%, and *** = 1%.

The results seem to be driven by high ability students. This result is

surprising. One would have expected that students with high grades from compulsory school should gain from the reform since their choice opportunities increased. But those who can exercise choice due to good grades do not seem to have benefited from this opportunity; their grades have even turned for the

IFAU – School choice and student achievement – new evidence on open-enrolment 16

Page 19: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

worse in some cases.20 Possible explanations to this unexpected pattern are discussed below.

5.3 Robustness checks The results could be driven by the inclusion of school fixed effects in the model. That is, if the school effects capture the ranking of individuals within schools, this could potentially drive the estimated reform effects. To investigate this issue the estimations are done without school fixed effects. If the hypothesis just stated has an impact, the differences between the quartiles of the ability distribution in Table 4 should diminish. The results (not shown), however, barely change but become more precisely estimated. For example, the lowest achieving quartile performs 1.1 percentile ranks better after the reform, and Quartile 3 perform 2.2 percentile ranks worse after the reform (both estimates are significant at the 5%-level).

Another worry is that teachers are reluctant to give high grades to all students in a high achieving class, i.e. grade-setting is relative. In principle, this should not be the case since the grading standard is goal-oriented. By including average peer quality in the regressions, it is possible to get a test for the impact of the student composition, which may affect individual outcomes through, for example, peer effects or relative grade-setting.21 The average compulsory grade per school and program is included additively in the regressions underlying Table 4. In all specifications the reform estimates are totally unchanged, and the direct effect of the peer quality is small and negative (and in most cases insignificant). The conclusion is that the student composition does not drive the results; even though there may be relative grade setting it does not affect the estimated reform effects.

The analysis has so far included students not completing in the stipulated three years time, giving them the grade 0. Table 5, which is organized in the same way as Table 4, presents results conditional on completion. The first row 20 Due to the design of the reform and the residential segregation by income in Stockholm, one did suspect that high ability students from low income families should gain by the reform, and that low ability students from high income families should be potential losers. Interaction estimates of ability and parental earnings do not support this conjecture. 21 The peer effect argument is that individual achievement is affected by the other students in the class-room. The literature on peer effects has shown that the peer composition do affect individual performance; see for example Sacerdote (2001), and Falk & Ichino (2006). The presence of relative grade-setting would imply that it is more difficult for high-achieving students in Stockholm to get high grades after the reform, since they on average attend classes with better peers.

IFAU – School choice and student achievement – new evidence on open-enrolment 17

Page 20: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

in each section shows the restricted sample size, and on average, about one third of the students are excluded. Row (2) shows the reform effects using GPAcore, and row (3) extends the outcome variable to the ordinary GPA. The results obtained with GPAcore will be considered first.

The results in the second row of Table 5 indicate that completion does matter, at least to some extent. The average treatment effect is more or less unchanged when conditioning on completion, but the effects in the sub-groups change. Overall, the differences between treated and comparison groups become smaller. In particular, unconditional on completion we observed a large difference in the point estimates between genders. This difference does not exist conditional on completion, where both genders have an insignificant effect of about -0.6.22 However, for theoretical programs the reform effect become negative and significant (at the 10%-level), with a point estimate of 1.02.

The effects over the ability distribution change as well. All quartiles now have negative point estimates, and the differences between the quartiles are smaller. The largest effect is found in quartile 3 (the 50-75th percentiles of the compulsory grade distribution) with a significant point estimate of -2.19. The expected pattern of completion rates is seen in number of completed students per quartiles. Only 37% of the lowest achieving quartile actually completes, while the corresponding figure for the top achieving quartile is 87%.

Comparing the bottom sections of Table 4 and 5, we see that the significantly negative effect for immigrants is explained by changes in the probability of completion. Also concerning socio-economic background the negative reform effect gets closer to zero conditioning on completion.

According to Table 5, variations in completion rates seem to be driving many of the notable differences between treated and comparison groups observed in Table 4. In general, completion rates have gone done in Stockholm among groups of high-achievers after the reform, while the opposite is true among low-achievers. The sole exception from this general pattern is immigrants.

The third row of each section in Table 5 displays the results on the ordinary GPA. The average effect, -1.15, is significant at the 5%-level. In the theoretical programs the reform effect is -1.97, significant at the 1%-level.

22 By means of the information provided on sample sizes, it can be seen that 69% of the females do complete, and only 62% of the males.

IFAU – School choice and student achievement – new evidence on open-enrolment 18

Page 21: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table 5 Reform effects conditional on completion.

Average reform effect and effects in public schools, theoretical programs, and by gender

All students Public Theoretical Females Males schools programs

N 18 853 15 780 12 382 9 652 9 201

GPAcore

SS2000 -0.576 (0.505)

-0.710 (0.559)

-1.024* (0.615)

-0.573 (0.703)

-0.603 (0.752)

GPA

SS2000 -1.145** (0.575)

-1.376** (0.636)

-1.967*** (0.714)

-1.541* (0.796)

-0.835 (0.853)

Reform effects by quartiles of the ability distribution

Q1 (low) Q2 Q3 Q4 (high)

N 2 751 4 465 5 596 6 041

GPAcore

SS2000 -1.015 (1.532)

-0.313 (1.182)

-2.191** (1.059)

-0.773 (0.782)

GPA

SS2000 -0.824 (1.698)

-0.441 (1.310)

-3.305*** (1.175)

-2.566*** (0.922)

Reform effects by immigrant status and socio-economic background

Immigrants Low parental education

Low parental earnings

High parental earnings

N 6 299 8 882 3 872 5 407

GPAcore

SS2000 -0.088 (0.920)

-0.460 (0.797)

-0.052 (1.216)

-0.803 (0.913)

GPA

SS2000 -1.047 (1.041)

-0.859 (0.897)

-1.070 (1.386)

-1.692 (1.041)

Note: Row (1) shows the size of the restricted sample, and row (2) and (3) shows the reform effects conditional on completion by using GPAcore and GPA as the outcome measure respectively. Regressions include a constant, and controls for gender, age, immigrant status, compulsory school grades, parental education and earnings, time effects, upper secondary school attended, compulsory school attended, program, and municipality where the student resides. Robust standard errors are in parentheses. Significance level: * = 10%, ** = 5%, and *** = 1%.

IFAU – School choice and student achievement – new evidence on open-enrolment 19

Page 22: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

With respect to the differences across the ability distribution, the pattern remains: the observed effects are largest among those with high compulsory grades. In the highest and second highest achieving quartiles the effects are -2.57 and -3.31 percentile ranks respectively, significant at the 1%-level. Hence, when students who do complete in the stipulated time are compared over the ordinary grades instead of the core subjects, the effects are larger and tend to be statistically significant to a greater extent. It thus seems that the negative reform effect is stronger for those subjects not being core subjects.

The final sensitivity analysis concerns grade inflation. Appendix B provides tests for the potential impact of grade inflation at compulsory and upper secondary level. The most relevant concern is perhaps grade inflation in Stockholm at compulsory level, since there are incentives for parents and students to put pressure on teachers to inflate grades after the reform. However, by restricting the compulsory grade variable to only include those subjects where there are national tests (and where there is arguably less or no grade inflation), the estimated reform effects are unchanged. Furthermore, the analysis gives no indications that grade inflation at upper secondary level drives the results.

6 Conclusions This paper evaluates a reform of the admission system to public upper secondary schools in Stockholm. A residence-based procedure was changed into a grade procedure. This open-enrolment reform has earlier been shown to strongly affect the distribution of students over schools. In this paper it is shown that the Stockholm students perform no better with increased choice availability. In fact, it seems that the effect is negative for students with high compulsory school grades.

Given the nature of the reform, potential distributional consequences along the ability dimension would have been expected to be the reverse. That is, low ability students were conceived as potential losers since they became restricted in the choice process. It should be noted that some of the differences between groups can be explained by differences in completion probabilities. High-ability students in Stockholm tend to complete to a lesser extent, while low-ability students completes to a greater extent. However, some of the surprising distributional effects seem to persist over alternative specifications.

IFAU – School choice and student achievement – new evidence on open-enrolment 20

Page 23: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

The most plausible explanation seems to be that students misinterpret school productivities when exercising choice, a point also made by Cullen, Jacob & Levitt (2005). Students only observe crude measures of student achievement in terms of grades and test scores, and they have no opportunity to judge whether these outcomes reflect educational production or student characteristics, i.e. educational input.

Having said this, one should keep in mind that this paper only uses one post-reform cohort, and school competition may need more time. The longer run effects are perhaps more favorable to the proponents of school choice.

IFAU – School choice and student achievement – new evidence on open-enrolment 21

Page 24: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

References Bettinger, E P (2005). The effect of charter schools on charter students and

public schools. Economics of Education Review 24(2).

Björklund, A, M A Clark, P-A Edin, P Fredriksson and A B Krueger (2005). The Market Comes to Education in Sweden. An Evaluation of Sweden’s Surprising School Reforms. Russell Sage Foundation, New York.

Cullen, J B, B A Jacob and S D Levitt (2005). The Impact of School Choice on

Student Outcomes: An Analysis of the Chicago Public Schools. Journal of Public Economics 89(5-6).

Falk, A and A Ichino (2006), “Clean Evidence on Peer Effects”, Journal of

Labor Economics, Vol. 24. Fiske, E B and H F Ladd (2000). When Schools Compete: A Cautionary Tale.

Brookings Institution Press, Washington, D.C. Hanushek, E A, J F Kain, S G Rivkin and G F Branch (2005). Charter School

Quality and Parental Decision Making With School Choice. NBER Working Paper No. 11252.

Hoxby, C M (2003a). School choice and school competition: Evidence from the United States. Swedish Economic Policy Review 10(2).

Hoxby, C M (2003b). School Choice and School Productivity (Or, Could

School Choice be a Rising Tide that Lifts All Boats?). In: C Hoxby (Ed), The Economics of School Choice, The University of Chicago Press, Chicago.

Sacerdote, B (2001), “Peer Effects with Random Assignment: Results for

Dartmouth Roommates”, Quarterly Journal of Economics, Vol. 116(2). Söderström, M and R Uusitalo (2005). School choice and segregation: evidence

from an admission reform. Working Paper 2005:7, Institute for Labour Market Policy Evaluation, Uppsala.

IFAU – School choice and student achievement – new evidence on open-enrolment 22

Page 25: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Wikström, C (2005). Grade Stability in a Criterion-Referenced Grading System: the Swedish example. Assessment in Education: principles, policy and practice 12(2).

Wikström, C and M Wikström (2005). Grade Inflation and School

Competition: an empirical analysis based on the Swedish upper secondary schools. Economics of Education Review 24(3).

IFAU – School choice and student achievement – new evidence on open-enrolment 23

Page 26: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Appendix A – Model specification This study utilizes a difference-in-differences approach. A potential worry is that differences in trends in the outcome measure between groups will bias the reform effect. If such a difference exists, one can use a pre-reform common support restriction when defining the comparison group, in order to make sure that the counterfactual is well defined.

In order to investigate whether pre-reform differences in trends exist, Equation (A1) is estimated over the years 1995-1999. The core grades ( ) are regressed on a set of individual characteristics (Xcore

ismtGPA ismt), school effects (χs), time effects (χt), municipality effects (χm), and interactions for schools being located in Stockholm with time (SSit). Subscript i denotes student, s school, t time, and m municipality.

(A1) ismtitmtsismtcoreismt SSXGPA μθχχχδκ ++++++=

By studying the interaction-terms of schools situated in Stockholm with

time, the presence of a descending trend cannot be ruled out, see column (1) of Table A1. In 1996, the interaction term was (insignificantly) positive with a point estimate of 0.67. This interaction effect becomes smaller and eventually negative over the years.

Therefore, the five pre-reform years (1995-1999) is used to define the Comparison group by applying a common support restriction. This is done in the following way. Linear municipality-specific trends are included additively to Equation (A1), and are estimated conditional on the covariates. If a munic-ipality has a trend significantly different from Stockholm at the 1%-level, the municipality is excluded from the Comparison group. This procedure implied that 4 of the 25 municipalities in the Comparison group were excluded.23 A difference-in-differences analysis is then performed over 1999 and 2000, according to Equation (A1). The result, displayed in column (2) of Table A1, indicates an insignificant negative reform effect of 0.57 percentile points.

23 In 2000, about 1 500 students attended a school in these municipalities.

IFAU – School choice and student achievement – new evidence on open-enrolment 24

Page 27: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table A1 Specification of baseline model.

Full sample Matching; exclusion

restriction 1%

Matching; exclusion

restriction 25%

Constant 63.495*** (3.225)

63.843*** (16.201)

-5.254 (6.519)

Female 1.048*** (0.192)

0.309 (0.324)

0.056 (0.350)

Age -4.138*** (0.142)

-3.045*** (0.261)

-3.305*** (0.284)

Immigrant -1.239*** (0.193)

-0.948*** (0.319)

-1.122*** (0.345)

Parental education 2.604*** (0.195)

2.458*** (0.322)

2.310*** (0.346)

Parental earnings 0.050*** (0.003)

0.054*** (0.006)

0.055*** (0.006)

Compulsory school grades

0.746*** (0.004)

0.790*** (0.007)

0.796*** (0.008)

SS1996 0.670 (0.537)

SS1997 0.577 (0.540)

SS1998

-0.087 (0.570)

SS1999

-0.635 (0.573)

SS2000

-0.568 (0.587)

-0.493 (0.623)

R2-adj 0.500 0.545 0.545

Time period 1995-1999 1999-2000 1999-2000

N 85 172 28 811 24 797

Note: The outcome variable is percentile ranked core grades. Regressions include controls for time, upper secondary school attended, compulsory school attended, program, and municipality where the student resides. Robust standard errors are in parentheses. Significance level: * = 10%, ** = 5%, and *** = 1%.

IFAU – School choice and student achievement – new evidence on open-enrolment 25

Page 28: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

The result could be sensitive to the level of exclusion, meaning that the procedure does not fully cope with differences in trends. Therefore, another specification is used, where a municipality is excluded in the first step if it has a pre-existing trend significantly different from Stockholm at the 25%-level. This implied that another 6 municipalities were dropped. The result, presented in Column (3), is very similar to the one in column (2). Hence, the 1% exclusion restriction is considered sufficient, and is chosen to constitute the baseline model.

IFAU – School choice and student achievement – new evidence on open-enrolment 26

Page 29: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Appendix B – Tests for grade inflation The empirical analysis uses grades as the outcome measure, and it also uses grades as a measure of initial ability. This appendix studies whether grade inflation at the compulsory level and/or the upper secondary level is driving the results. Grade inflation at compulsory level An indirect test for grade inflation at the compulsory level is conducted by restricting the compulsory grade variable to the average of the grades in English, Math, and Swedish (denoted GPASME in the descriptive statistics). Grade inflation should not be a concern for these subjects (to the same extent) since the teachers have National test scores as guidelines in the grade-setting.

In this analysis, the same first-step common support restriction is used as in Appendix A, but now with the compulsory GPASME instead of the ordinary compulsory GPA as a measure of initial ability. This implies that six municipalities are being dropped according to the 1% exclusion restriction. The model is then estimated in three specifications, corresponding to the first column in the upper parts of Table 4 and Table 5, which is reproduced in the upper part of Table B1. That is, the average treatment effect in three speci-fications. Column (1) includes all students, and Column (2) and (3) are conditional on completion. Column (1) and (2) use core subjects as the outcome measure, and Column (3) uses ordinary grades.

IFAU – School choice and student achievement – new evidence on open-enrolment 27

Page 30: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Table B1 Reform effects using compulsory school GPASME.

Full sample;

Outcome variable: GPAcore

Conditional on completion;

Outcome variable: GPAcore

Conditional on completion;

Outcome variable: GPA

Compulsory GPA

SS2000 -0.567 (0.587)

-0.576 (0.505)

-1.145** (0.575)

R2-adj 0.545 0.692 0.602

N 28 811 18 853 18 853

Compulsory GPASME

SS2000 -0.563 (0.654)

-0.397 (0.547)

-0.939 (0.638)

R2-adj 0.483 0.654 0.539

N 26 327 17 627 17 627

Note: Regressions include a constant, and controls for gender, age, immigrant status, compulsory school grades, parental education and earnings, time effects, upper secondary school attended, compulsory school attended, program, and municipality where the student resides. Robust standard errors are in parentheses. Significance level: * = 10%, ** = 5%, and *** = 1%.

The estimates are very similar. With GPASME the point estimates are identical for the full sample, and somewhat smaller when conditioning on completion. Hence, this test does not suggest that grade inflation at the compulsory level in Stockholm is driving the results. Grade inflation at upper secondary level Grade inflation at the upper secondary level is studied with use of National test scores in English, Swedish, and Math. For a national sample of around 10% of the upper secondary schools, tests are collected on a yearly basis, and are available for this study. Two things should be noted about the test score data. First, test scores cannot be matched at the individual level, only per program and school, making the analysis relying on class averages (i.e. program per school). Second, there is no information on when the students complete upper secondary schooling, making the analysis relying on the assumption that the students taking national tests complete in the stipulated time. We know that this

IFAU – School choice and student achievement – new evidence on open-enrolment 28

Page 31: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

is an approximation, but it is only a problem if the completion probability varies systematically between groups over time.

With grades (Gpst) as the outcome measure, Equation (B1) is estimated at the class level for each subject. Subscript p denotes program, s school, and t time. Explanatory variables are class averages of background characteristics (Xpst), program fixed effects (λp), school fixed effects (λs), time fixed effects (λt), national test scores (NTpst), and a vector of products of a Stockholm school indicator and time (SSpt).

(B1) pstptpst

NTtsppst

Xpst SSNTXG ητρλλλρφ +++++++=

The coefficients of interest are the τ:s, which show how grades evolves in

Stockholm compared to the Comparison group, conditional on the set of covariates and the National test scores. Hence, this procedure tests for systematic differences between grades and test scores in the two groups. The results are shown in Table B2. The coefficients are insignificant, and it is impossible to spot any trends. Hence, the results do not indicate systematic differences in grade inflation between the groups.

Table B2 Tests for grade inflation at upper secondary level.

Math English Swedish

SS1997 -0.725 (1.125)

0.876 (0.724)

0.609 (1.232)

SS1998 -0.964 (1.316)

1.166 (0.859)

-1.179 (0.814)

SS1999 -1.750 (1.295)

-0.212 (0.592)

-0.413 (2.228)

SS2000 -1.361 (1.202)

0.282 (0.642)

-2.158 (1.581)

R2 0.898 0.913 0.884

N 139 174 136

Note: Regressions are at the class level, and include a constant, and controls for gender, age, immigrant status, compulsory school grades, parental education and earnings, time effects, school effects, and program. Robust standard errors are in parentheses, and regressions are weighted by class size. Significance level: * = 10%, ** = 5%, and *** = 1%.

IFAU – School choice and student achievement – new evidence on open-enrolment 29

Page 32: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

Publication series published by the Institute for Labour Market Policy Evaluation (IFAU) – latest issues Rapporter/Reports

2006:1 Zenou Yves, Olof Åslund & John Östh ”Hur viktig är närheten till jobb för chanserna på arbetsmarknaden?”

2006:2 Mörk Eva, Linus Lindqvist & Daniela Lundin ”Påverkar maxtaxan inom barnomsorgen hur mycket föräldrar arbetar?”

2006:3 Hägglund Pathric ”Anvisningseffekter” – finns dom? Resultat från tre arbetsmarknadspolitiska experiment”

2006:4 Hägglund Pathric ”A description of three randomised experiments in Swedish labour market policy”

2006:5 Forslund Anders & Oskar Nordström Skans “(Hur) hjälps ungdomar av arbetsmarknadspolitiska program för unga?”

2006:6 Johansson Per & Olof Åslund ”’Arbetsplatsintroduktion för vissa invandrare’ – teori, praktik och effekter”

2006:7 Calleman Catharina ”Regleringen av arbetsmarknad och anställningsförhål-landen för hushållstjänster”

2006:8 Nordström Skans Oskar, Per-Anders Edin & Bertil Holmlund ”Löneskillna-der i svenskt näringsliv 1985–2000”

2006:9 Engström Per, Hesselius Patrik & Malin Persson ”Överutnyttjande i tillfällig föräldrapenning för vård av barn”

2006:10 Holmlund Bertil, Qian Liu & Oskar Nordström Skans ”Utbildning nu eller senare? Inkomsteffekter av uppskjuten högskoleutbildning”

2006:11 Sibbmark Kristina & Olof Åslund ”Vad för vem och hur gick den sen? En kartläggning av arbetsförmedlingarnas insatser för utrikes födda under 2005”

2006:12 Fredriksson Peter & Björn Öckert ”Är det bättre att börja skolan tidigare?”

2006:13 Sibbmark Kristina ”Arbetsmarknadspolitisk översikt 2005”

2006:14 Wang Iris J Y, Kenneth Carling & Ola Nääs ”Är kommunala sommarjobb en gräddfil till arbetsmarknaden?”

2006:15 Söderström Martin ”Betygsintagning och elevers studieframgång i Stockholms gymnasieskolor”

Working Papers

2006:1 Åslund Olof, John Östh & Yves Zenou “How important is access to jobs? Old question – improved answer”

Page 33: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

2006:2 Hägglund Pathric “Are there pre-programme effects of Swedish active labour market policies? Evidence from three randomised experiments”

2006:3 Johansson Per “Using internal replication to establish a treatment effect”

2006:4 Edin Per-Anders & Jonas Lagerström “Blind dates: quasi-experimental evidence on discrimination”

2006:5 Öster Anna “Parental unemployment and children’s school performance”

2006:6 Forslund Anders & Oskar Nordström Skans “Swedish youth labour market policies revisited”

2006:7 Åslund Olof & Per Johansson “Virtues of SIN – effects of an immigrant workplace introduction program”

2006:8 Lalive Rafael “How do extended benefits affect unemployment duration? A regression discontinuity approach”

2006:9 Nordström Skans Oskar, Per-Anders Edin & Bertil Holmlund “Wage disper-sion between and within plants: Sweden 1985–2000”

2006:10 Korkeamäki Ossi & Roope Uusitalo “Employment effects of a payroll-tax cut – evidence from a regional tax exemption experiment”

2006:11 Holmlund Bertil, Qian Liu & Oskar Nordström Skans “Mind the gap? Esti-mating the effects of postponing higher education”

2006:12 Fredriksson Peter & Björn Öckert “Is early learning really more productive? The effect of school starting age on school and labor market performance”

2006:13 Pekkarinen Tuomas, Sari Pekkala & Roope Uusitalo ”Educational policy and intergenerational income mobility: evidence from the Finnish compre-hensive school reform”

2006:14 Wang Iris J Y, Kenneth Carling & Ola Nääs “High school students’ summer jobs and their ensuing labour market achievement”

2006:15 de Jong Philip, Maarten Lindeboom & Bas van der Klaauw “Screening dis-ability insurance applications”

2006:16 Söderström Martin “School choice and student achievement – new evidence on open-enrolment”

Dissertation Series

2006:1 Hägglund Pathric “Natural and classical experiments in Swedish labour market policy”

2006:2 Savvidou Eleni “Technology, human capital and labor demand”

Page 34: School choice and student achievement · School choice and student achievement – new evidence on open-enrolment* by Martin Söderströma December 4, 2006 Abstract This paper studies

2006:3 Söderström Martin “Evaluating institutional changes in education and wage policy”

2006:4 Lagerström Jonas “Discrimination, sickness absence, and labor market policy”