Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
The Gross And Net Effects Of Primary School Denomination On Pupil Performance
Geert Driessen, Orhan Agirdag & Michael S. Merry
Corresponding author:
Geert Driessen
ITS
Radboud University
P.O. Box 9048
6500 KJ Nijmegen
The Netherlands
www.geertdriessen.nl
Orhan Agirdag
Laboratory for Education and Society
University of Leuven
Belgium
Department of Educational Sciences
University of Amsterdam
The Netherlands
Michael S. Merry
Faculty of Social and Behavioural Sciences
University of Amsterdam
The Netherlands
2
Abstract: Notwithstanding dramatically low levels of professed religiosity in Western
Europe, the religious school sector continues to thrive. One explanation for this paradox is
that nowadays parents choose religious schools primarily for their higher academic
reputation. Empirical evidence for this presumed denominational advantage is mixed. We
examine and compare several studies purporting to show a denominational school effect, and
then turn our attention to the Dutch case. Owing to its longstanding and highly varied
denominational school sector, the Netherlands arguably provides a unique context in which to
examine whether there are school sector effects. In this study multilevel analyses were
performed. Data include 19 cognitive and non-cognitive outcome measures in 2011
administered to 27,457 pupils in grades 2, 5 and 8 of 386 primary schools. Results show that
after controlling for input differences at pupil and school level no substantial output
differences between religious schools and public schools remain. However, Islamic schools
appear to be one important exception which turn out to have a great value-added potential.
Implications of these findings are discussed.
Key words: denomination; religious schools; school choice; effects; primary school; the
Netherlands; multilevel analysis
3
The Gross And Net Effects Of Primary School Denomination On Pupil Performance
Background, Perspective and Objective
A Paradox: Denominational Schools in ‘Secular’ Western Europe
Across much of Western Europe there is ample evidence to suggest a sharp decline in
religiosity. This, of course, is not a completely new development, as a process of
secularization has been underway since the 1960s. It also should be said that this process does
not apply equally to all religions; nor does it apply to all European countries to the same
degree (Norris and Inglehart 2011). Concurrent with the steady decline in traditional religious
belief and church membership, in many countries the market share of religious schools
remains virtually unchanged (Dijkstra, Driessen, and Veenstra 2001; Merry 2015). That is,
the proportion of parents selecting denominational schools for their children is almost the
same today as it was half a century ago; in some countries there even is an increase in
demand for religious schools. Various explanations for this paradoxical situation have been
put forward (Dronkers 2004; Goldring and Philips 2008).
First, both Catholic and Protestant religious schools are typically supported by
Christian political parties (often the ruling parties), but they also enjoy support from laws
protecting freedom of religious education, as well as the vested interests of administrative and
educational professionals attached to the different denominational organizations. Second,
while perhaps only a minority, a committed core of religiously devout parents continue to
choose religious schools because they reinforce their beliefs and values; these may be
Christian, Muslim or Jewish parents for whom a religious upbringing is paramount. Third, it
is also the case that many parents – particularly in smaller towns and rural areas – choose the
existing local denominational school for its proximity and convenience. Fourth, religious
schools may be more attractive, even for non-religious parents, because they pay more
attention to moral values than the ‘neutral’ public schools. Finally, though seldom explicitly
stated, parents may choose a religious school on the basis of the pupil composition;
particularly middle-class parents are keen to avoid (public) schools composed of high
concentrations of ethnic minorities, whose presence is believed to compromise the general
quality of education.
Each of these explanations is plausible and supported in the literature. However, mono-
causal explanations for the attractiveness of religious schools is simply not possible; instead,
a complex interplay of various factors is at work. On the other hand, Dijkstra, Dronkers, and
Karsten (2004; also see Allen, Burgess, and McKenna 2014; Denessen, Driessen, and
4
Sleegers 2005) conclude that nowadays the most important reason for the survival of the
religious school sector is their reputation for producing higher academic attainment. At the
same time, it is not clear to what degree the academic reputation of religious schools
coincides with their actual impact on pupils’ cognitive and non-cognitive outcomes. The
evidence concerning the impact on academic achievement is mixed, and very few studies
have focused on non-cognitive effects. Hence, the aim of the present study is to examine how
school sector differences are related to pupil outcomes, both in terms of academic
performance as well as non-cognitive outcomes. Our ultimate focus will be on the primary
education sector in the Netherlands and we will perform a secondary analysis of recently
collected large-scale data.
Funding, Quality and Output
Since the Coleman, Kilgore, and Hoffer (1981) report, numerous studies have examined the
outcome effects of denominational schools, both in the United States and – to a lesser extent
– in Europe. Underlying much of this research is the question of whether the state, in addition
to financing public schools, also is responsible for funding private religious schools. Indeed,
the mere fact that so many children attend religious schools appears to support the idea of
equally funding all schools given the state’s parens patriae role entailing responsibility for
the education of all children irrespective of the type of school they attend (Merry 2007).
Meanwhile, school choice is increasingly promoted by governments as a way to
stimulate competition between schools with the dual aim of improving school quality as well
as providing parents with more options. In a system of education designed as a quasi-market
it is assumed that in order to develop a good reputation and thereby attract more pupils,
competition will induce both public and private schools to improve their quality while at the
same time reducing their expenses. Hence it is expected that school choice will yield both
better learning outcomes and more efficiency (Chubb and Moe 1990; Lauri and Pöder 2013).
Further, if religious schools outperform non-religious schools, then there are reasons to think
that religious schools may serve as a useful role model (Burgess et al. 2015; Jeynes and
Beuttler 2012). These matters have implications not only for how schools are financed but
also as it concerns the increasing amount of attention on the quality of education being
offered (Ritzen, Van Dommelen, and De Vijlder 1997). Here the notion of educational
quality is deduced primarily in terms of pupil achievement. Accordingly, researchers have
been keen to determine the possible relationship between school sectors (denominational vs
non-denominational) and their educational results. While much research has been carried out,
5
the results remain surprisingly inconclusive, in part because the performance of schools can
only be understood in relation to their specific context. These contextual differences pertain
not only to nationality; indeed, schools within the same country also may be difficult to
compare given the range of relevant variables – financing, pupil composition, teacher
retention, testing instruments, etc. – bearing upon any given school’s academic performance
(Avram and Dronkers 2011).
Yet if religious schools enjoy a superior reputation with respect to their educational
performance, it is worthwhile finding out whether there is any corroborating empirical
evidence for this. Lately, several comprehensive studies examining denominational outcome
effects have been published. Together these large-scale studies help us to analyze and
compare both cognitive and non-cognitive output effects of religious versus public schools in
a large number of countries.
At the turn of the century, Dijkstra, Driessen, and Veenstra (2001) reviewed all of the
research evidence for public versus private denominational school effects in the Netherlands.
They cautioned against drawing unequivocal conclusions about systematic sector effects. In
the studies they reviewed, denomination-specific differences are sometimes reported, and
other times not. When such effects occur, they tend not to reveal much of a pattern.
Nevertheless, there are indications of a relationship between school sector and corresponding
outcomes. The authors found that the school sector effects are for the most part negative for
public schools, and positive for private religious schools. This applies slightly more to
primary education than secondary education, and more to younger data than older data.
A few years later, Dronkers (2004) reviewed the evidence concerning the differences in
the effectiveness of public and religious state-funded schools in seven European countries.
He concluded that differences in cognitive outcomes clearly exist in Belgium, France,
Hungary, the Netherlands, and Scotland, but less obviously in Germany. These differences
could not be explained by the social characteristics of pupils, parents, schools and
neighborhoods. Non-cognitive outcome differences were not found in Belgium, only partly in
Germany, and only negligibly in the Netherlands. According to Dronkers, this comes as a
surprise as it is especially the non-cognitive contribution that is the chief raison d’être for the
establishment of state-funded religious schools.
Avram and Dronkers (2011) analyzed the PISA waves of 2000, 2003 and 2006 of 16
European countries. They argued that the establishment of denominational schools is
primarily motivated by the desire to socialize children into the prevailing (religiously and
morally inspired) norms and values of a particular community. Therefore they focused on two
6
non-cognitive outcome measures, namely emotional integration within the school
community, and the concern and feelings of responsibility towards the environment. Yet with
the exception of Austria, Belgium and Spain as it concerned emotional integration, the
analyses revealed no salient differences between denominational and public schools. One
explanation for this may be that public schools are just as effective as religious schools in
conveying important norms and values.
Meanwhile, Jeynes (2012) conducted a statistical meta-analysis on religious school
effects predominantly in the United States, which comprised 90 studies and used
sophisticated controls for socioeconomic status and race/ethnicity. He found that pupils who
attend religious schools perform better than their counterparts in public schools, both in terms
of academic and behavioral outcomes, and controlling for socioeconomic and racial/ethnic
backgrounds. Interestingly, he also found evidence suggesting that religious and public
schools have much to learn from each other. Religious schools, he maintained, tend to score
more favorably on high expectations and more challenging course offerings, whereas public
schools score higher on classroom discussion and the range of elective courses.
In sum, most comparative studies examine secondary rather than primary education; the
focus typically is on cognitive effects rather than non-cognitive effects; and finally, many
studies concentrate on just one or only a few output measures. From these studies it can be
concluded that there is a tendency for religious schools to achieve better than non-religious
public schools, and this is clearer with regard to cognitive outcomes than non-cognitive
outcomes. However, the findings are not as unambiguous as many assume.
Explanations for Output Differences
In so far as output differences that appear to be attributable to the schools’ denomination are
concerned, several explanations have been offered, not all of which have been empirically
validated. For example, the selectivity thesis suggests that the differences between religious
(and hence ‘private’) and non-religious (more typically ‘public’) schools can be explained by
the fact that private schools attract better and more motivated pupils than public schools
(Willms 1985). The explanation for sector differences mentioned most, however, builds on
the social capital model developed by Coleman (1988). Social capital describes the norms of
trust and reciprocity that arise out of our social networks. Coleman argued that Catholic
schools are supported by a functional community sharing two indispensable traits of social
capital.
7
First, there is a shared trustworthiness among members of the community; as a general
rule, those participating in the shared norms of community exhibit a higher degree of trust
towards others doing the same. This reciprocal trust facilitates the exchange of useful
information, but also a variety of other resources germane to a healthy community. Second,
there exists what Coleman calls a ‘density of outstanding obligations,’ meaning that available
resources within a particular structure can be augmented by the mere fact that others within
the community can be called upon to contribute to achieving the goals shared by all. Hence
the absence of one member (e.g., a parent) from a particular activity or event can be
compensated for both by the presence and attentive involvement of another. Taken together,
these elements support and sustain stronger intergenerational networks between pupils,
parents and teachers and combine to ‘provide effective rewards for high achievement’ in a
particular school environment. But social capital is also strengthened and maintained by the
families comprising the broader social network, and whose interests and expectations are
shared with others similarly situated and invested in the school’s ‘performance’. Even when
the internal composition of a school may be highly diverse, these shared features are believed
to facilitate the achievement of common interests and goals.
Morgen and Sorensen (1999) empirically tested Coleman’s social capital hypothesis
and they found that parental networks were in fact negatively related to achievement gains.
Moreover, while peer-networks were positively related to achievement gains, they could not
account for the religious (Catholic) school advantage. Hofman et al. (1996) tested the
hypothesis that it is not only the functional community (i.e. the parental networks) that affects
academic achievement, but also the interactions between the parental network around a
school and the governance structure of the school. Indeed, they found that differences in
achievement for religious and non-religious schools are mediated by characteristics at both
the family and the institutional level. In other words, characteristics at the institutional level
are necessary if parental networks are to have the intended effects. Other studies found that
the positive effect of religious schools was mostly related to the types and rigor of classes on
offer, rather than, say, more equitable treatment of pupils (Carbonaro and Covay 2010).
Research Objective
Having canvassed several comparative multi-national studies, we now turn our attention to
the Netherlands. The Dutch case offers many uniquely illuminating insights, particularly
when we take the following into consideration. First, contrary to the situation in some other
countries, the division of public and private education in the Netherlands is based on the form
8
of administration and objectives of the school and not on financing. Public schools (i.e.,
nondenominational schools) are administered under the auspices of the community
government, whereas private schools (consisting almost entirely of denominational schools)
are administered by private legal institutions (an association or foundation) based on a
particular religion or philosophy of life. Second, because of the constitutional freedom of
education, which includes the freedom to establish schools, to organize the teaching and to
determine the religious, ideological or educational principles on which this is based, the
Dutch education system accommodates a remarkable variety of religious schools. In 2013, 32
percent of all primary schools were public schools, 30 percent were Catholic and another 30
percent Protestant. In addition there are several smaller denominations, such as Islamic (0.6
percent) and Hindu (0.1 percent) . Third, close to 70 percent of Dutch primary pupils
continues to attend denominational schools, and this percentage has hardly changed since the
1950s (Ministry ECS 2014). Fourth , the Netherlands has for nearly a century been
committed to the equal funding of all schools, irrespective of their religious or ideological
character (Ritzen, Van Dommelen, and De Vijlder 1997). Both public schools and private
denominational schools receive exactly the same financial support from the state. The most
decisive factor for funding is the school’s pupil population in terms of social class
background. Under the Educational Disadvantage Policy schools that meet a certain threshold
of disadvantaged pupils receive extra funding (Driessen and Merry 2014). As a consequence,
a school with predominantly disadvantaged pupils is entitled to nearly twice the budget that a
school with a higher concentration of middle-class pupils receives.
As a general rule, public and private denominational schools cater to different
populations both in terms of the secular-religious dimension as well as the socioeconomic
and ethnic dimension. Nationally representative large-scale data recently became available
that make it possible to analyze a comprehensive range of output indicators regarding both
cognitive and non-cognitive measures. The main question of this study is: are there any
output differences between public and the various denominational schools – taking into
account their differing pupil populations? We hypothesize, first, that religious schools will
perform better than public schools in the area of non-cognitive measures for the simple
reason that religious schools were explicitly established with this aim in mind. Our second
hypothesis is that we expect that religious schools, owing to their distinctive mission, will
distinguish themselves from their public counterparts and consequently also perform better
academically than other schools serving a comparable pupil population.
9
Method
Sample
The data analyzed here are from the Dutch large-scale COOL5-18 study (Driessen, Mulder,
and Roeleveld 2012). In the school year 2010/11, a total of 553 primary schools participated
in this study (or 8% of all Dutch primary schools) with nearly 38,000 pupils in grades 2, 5
and 8 (6-, 9- and 12-year-olds). The COOL5-18 sample consists of a core sample of 406
schools, which is representative for all Dutch primary schools, and an additional sample of
147 schools with large numbers of disadvantaged pupils. A new sample was drawn from all
of the public (i.e. nonreligious), Protestant, Catholic and Islamic schools in the representative
sample, thereby excluding 28 schools from several smaller denominations. This sample was
supplemented with the 8 Islamic schools from the additional sample. Analysis showed that in
terms of socioeconomic disadvantage the 17 Islamic schools in the new sample were
representative for all of the (at that time) 41 Islamic schools. The resulting sample of 386
schools included a total of 27,457 pupils in grades 2, 5 and 8 of 143 Public, 101 Protestant,
125 Catholic and 17 Islamic schools.
Instruments and Variables
In the COOL5-18 study the following measures were used:
• In all three grades 2, 5 and 8 standardized Language and Math tests were administered
developed by CITO, the National Institute for Educational Measurement (www.cito.nl).
• In grades 5 and 8 two standardized scales were used to establish the pupils’ motivation,
namely Self-Efficacy and Task Motivation (Midgley et al. 2000; Seegers, Van Putten,
and De Brabander 2002).
• In grade 8 the Citizenship Competences Questionnaire was used, which consists of four
subscales, namely Knowledge, Reflection, Skills, and Attitudes (Ten Dam, Geijsel, and
Reumerman 2011).
• In grade 8, which is the final grade of Dutch primary education, the standardized
Primary School Leavers’ Test also developed by CITO was administered. In addition to
total test scores, scores on the Language, Math, and Study Skills subtests are available.
• In grade 8 the pupils received the secondary school recommendation which was given
by their class teacher. In the Dutch hierarchically ordered school system five tracks can
be discerned.
10
The central variable of interest here is the denomination of the school. To correctly appreciate
the schools’ cognitive and non-cognitive output according to denomination and reduce
possible selection bias effects (Hofman et al. 1996), two new measures were calculated, the
gross and the net output. To arrive at the net output the gross output (i.e., the ‘raw’ scores)
was corrected for by several factors at the level of the pupil and in addition to this also at the
level of the school. This output often is seen as an indicator of a school’s learning outcomes,
for instance by the Dutch Inspectorate of Education. Interpretations then are: what is added
by a school to the pupils’ initial intellectual and social capital; or the relative achievement of
pupils in one school compared to pupils in other schools after controlling for differences
between pupils outside of the control of the school that influence achievement). In
educational research such factors most found to contribute to achievement differences are
socio-economic background and ethnic origin (Jeynes 2012; Strand 2014). In the COOL5-18
study two indicators for these correction variables are available, namely parental educational
level and country of origin. The first factor discerns the three categories applied in the Dutch
Educational Disadvantage Policy: medium and higher education; low education; and very
low education. The second factor, which until recently also was used in the Educational
Disadvantage Policy as a second indicator of disadvantage, discerns four categories: Dutch
and Western; Turkish; Moroccan; and other non-Western. The information on both factors
came from the schools’ administrations. At the school level comparable factors were used:
the percentages of pupils with low educated parents and with very low educated parents, and
the percentages of Turkish, Moroccan, and other non-Western pupils. The information on
these factors came from the Ministry of Education’s administration.1
In the COOL5-18 study several sources for missing values can be discerned (see
Driessen, Mulder, and Roeleveld 2012). The actual number of missing values differs strongly
for each of the grades, instruments and variables. First, not all of the schools provided
information on parental educational and ethnic origin. Depending on grade and instrument
the share of missing values varies from 2.5 to 5.0%; these cases were removed from the
sample. Second, not all schools administered all of the instruments and in addition not all of
the pupils were present when the data were collected. In the COOL5-18 study the response on
all of the instruments was checked for any correlation with the pupils’ social and ethnic
backgrounds. The analyses revealed that there was no relation whatsoever. Third, several of
1 By controlling for parental educational level any effects associated with extra budgets received by
schools as part of the Educational Disadvantage Policy are also ruled out.
11
the cognitive tests are regular tests used by the schools and not specifically administered as
part of the COOL5-18 study. CITO, the institute responsible for the construction of the tests,
recently developed new versions of the existing tests, which resulted in some schools still
using the old versions, and some schools using the new versions. As the scores on both
versions are not comparable we chose the version with the most valid cases for our analyses.
Table 1 provides an overview of the resulting statistics for the outcome measures. To
facilitate comparability and interpretation of the results of the ultimate multilevel analyses, all
of the scores on these instruments were transformed into standardized z-scores with a mean
of 0 and a standard deviation of 1.
To give a description of how pupils with different backgrounds are distributed across
the various school denominations, in Table 2 we present the relationships between
denomination and parental education and ethnicity.
[Insert Table 2 about here]
Table 2 shows that in the public school sector 81.4% of the pupils’ parents have a medium or
higher education and 82.2% is of native Dutch or Western origin. The Protestant and Catholic
sector cater to significantly fewer pupils with very low educated parents (2.7 and 3.5% vs.
8.5%) and non-native Dutch parents (7.2 and 9.6% vs. 17.8%) than the public schools. The
Islamic school sector deviates the most from the other denominations. More than half
(51.8%) of its pupils have low or very low educated parents, while for the other
denominations the percentages vary between 12.0 to 18.6. In addition, 84.1% of the pupils at
Islamic schools have an immigrant background, while the mean percentage of the other
denominations is 11.5. We also computed the mean percentages of the parental education and
ethnicity categories per school denomination but now at the school level; this analysis yielded
similar results as those at the pupil level. What these findings show is that the Dutch
denominational primary school sector is to a certain extent (self-)segregated according to the
pupils’ social milieu and ethnic background. Again, Islamic schools differ very strongly from
the other denominations, but public schools also cater to significantly more pupils from lower
social milieus and immigrant backgrounds, i.e. disadvantaged pupils, than the Protestant and
Catholic schools. The question then is what the impact of these individual and compositional
differences is on the schools’ output, and whether there are any remaining output differences
which can be ascribed to sector differences.
12
Analytical Strategy
Because of the nested structure of the data, pupils within schools, multilevel analysis was
performed using the SPPS mixed models module (Field 2009; Heck, Thomas, and Tabata
2010). Several random intercept fixed slopes models were tested. In model 1, the central
variable of school denomination was introduced with three dummy categories: Protestant,
Catholic, and Islamic. The reference category consists of the public schools. The results of
this model can be interpreted as the gross effect of school denomination. Because the output
measures all have been standardized, these coefficients can be evaluated according to the rule
of thumb supplied by Cohen (1988), who denoted 0.20 as a small effect, 0.50 as a medium
effect and 0.80 as a strong effect. In model 2, a first series of correction variables at the level
of the pupil were added, namely parental educational level and immigrant origin. In the final
model 3, a second series of correction variables at the level of the school were introduced: the
percentage equivalents of the correction variables at the pupil level.2 The results of this
model can be interpreted as the net effect of school denomination. Thus, the focus in our
analyses is not on model testing per se, but on what remains of the individual denominational
effects after having corrected for differences between denominations regarding socio-ethnic
pupil and school backgrounds.
Results
To give a concrete example of the analytic strategy applied, in Table 3 the results of the
effects of one of the outcome measures are presented, viz. for the reading skills of the grade 5
pupils are presented (for the relevant descriptive statistics, see Table 1, row 5) .
[Insert Table 3 about here]
In model 1, the central variable of school denomination is introduced with three dummy
categories: Protestant, Catholic, and Islamic. The reference category consists of the public
schools, which have a mean score of -0.0878 (i.e. the intercept). The results of this model
denote the gross effect of denomination. Protestant and Catholic schools perform
significantly higher (0.1389 and 0.1676, respectively; p = 0.021 and p = 0.002) on reading
2 We checked for possible multicollinearity of the indicators of education and ethnicity, but with a
maximum correlation of 0.35 this proved to be no problem.
13
skills than public schools. Islamic schools achieve significantly lower (0.3717; p = 0.001)
lower than public schools.
In model 2, a first series of correction variables at the level of the pupil are added.
These are parental educational level with dummy categories low education and very low
education, with the reference category medium or higher education, and ethnic origin with
dummy categories Turkish, Moroccan and other non-Western, with the reference category
Dutch and Western. This model shows that the initial lead of the Protestant schools is reduced
with around 0.06 standard deviation and becomes statistically insignificant (p = 0.169). This
indicates that the higher reading scores of the Protestant schools are in part a result of the
more favorable backgrounds of their pupils. The same applies to the Catholic schools: their
lead is reduced with around 0.05 standard deviation, but here the resulting estimate of 0.1143
remains significant (p = 0.020) different from the estimate of the public schools. At the same
time this model also shows that the considerable initial reading achievement gap of Islamic
schools as compared to public schools is almost completely reduced to zero (-0.0348) by
correcting for parental education and ethnicity and becomes insignificant (p = 0.750). Thus,
the gap reported in model 1 was obviously a result of the lower parental education level of
the pupils at Islamic schools and the fact that many of them have an immigrant background.
In the final model 3, a second series of correction variables at the level of the school are
introduced. These are the percentage equivalents of the correction variables at the pupil level
(minus the reference categories). The results of this model denote the net effect of
denomination on reading achievement. This model shows that none of the denomination
categories scores significantly different from the reference category of public schools any
longer. This means that after the corrections at the pupil and school level, the Catholic
schools no longer perform better than the public schools, but also that the Islamic schools no
longer perform lower than the public schools; they even achieve somewhat higher – albeit not
significantly (p = 0.460). This model also shows that with the exception of the percentage of
low-educated parents at a school (p = 0.028) none of the correction variables at the school
level has a significant effect, above the effect of the correction variables at the pupil level.
To summarize these findings, it can be concluded that although Islamic schools initially
score far below the other denominational schools with regard to reading performance in grade
5, after correcting for parental education and ethnicity nothing of this negative effect remains.
This implies that Islamic schools have great value added power.
In the COOL5-18 study a total of 19 cognitive and non-cognitive effect measures are
available. In Table 4 the final results of the complete set of analyses are summarized. (The
14
descriptive statistics for these analyses are presented in Table 1.) As the focus is on
denominational effects, this table only contains the gross and net effects of denomination.
Critically, the question is whether pupils are better off in denominational schools than in
public schools. The coefficients presented therefore are always relative to the output of the
public schools.
[Insert Table 4 about here]
When we first look at the gross effects, Table 4 shows that Protestant schools perform
somewhat better than public schools, certainly with regard to the cognitive measures (e.g.
Language and Math). Regarding the non-cognitive measures (Self-Efficacy and Task
Motivation in grade 5 and Citizenship Reflection and Skills) they perform somewhat worse.
In terms of academic performance, Catholic schools do not differ much from public schools.
Islamic schools perform the worst on nearly all cognitive measures. It is striking, however,
that on almost all non-cognitive measures (with the exception of Citizenship Knowledge)
they perform better or even much better than the other denominational schools.3
When we next look at the net effects, the analysis results show that of the 30 significant
gross effects only 2 net effects remain significant. Surprising is the fact that after having
corrected for social and ethnic differences in pupil backgrounds, Islamic schools even
achieve better (though only in one case significantly) than the other denominational schools
on nearly all output measures, especially with regard to math achievement (p = 0.03).
Summing up, although Islamic schools in an absolute sense achieve lowest on all cognitive
measurements, they succeed in raising their pupils’ achievement more than the other
denominational schools. With regard to the non-cognitive measurements (except Knowledge)
they already score highest in an absolute sense.
3 One could argue that the high scores of the pupils at Islamic schools on the Citizenship competences
questionnaire are, to a certain extent, the result of social desirability bias. However, the fact that their
scores on the Knowledge subscale are significantly lower than those of the other denominations,
which is understandable in the light of different pupil populations, the fact that for the other three
subscales the results are in the opposite direction does not support this suspicion and offers further
proof of the instrument’s validity.
15
Conclusions
The purpose of this study has been to examine how school sector differences are related to
pupil outcomes. The question arises given the prominent role that religious schools continue
to play in Western Europe. Indeed, notwithstanding a ‘secularizing’ trend in Western Europe,
the market share of religious schools remains impressively high. While half a century ago the
most important motive for choosing a religious school was its denomination, nowadays the
primary motive – at least as reported by parents – is the quality of education on offer. Parents
continue to select religious schools because of their academic reputation. (Dijkstra, Dronkers,
and Karsten 2004) and parents similarly placed within their social networks reinforce these
perceptions (cf. Allen, Burgess, and McKenna 2014).
This study aimed to investigate whether these perceptions are consistent with the
reality. To that end we examined the Dutch case by using recent large-scale data and tested
whether any output differences remained after controlling for input differences in terms of
social and ethnic background at the pupil and the school level. We put forward two
hypotheses: first, we predicted that religious schools would perform better than non-religious
public schools in the area of non-cognitive measures; second, we predicted that religious
schools would distinguish themselves in ways that would enable them to also outperform
their non-religious counterparts academically, even when serving a comparable pupil
population.
To start with the latter, before controlling for individual and compositional differences,
the results suggest that Protestant and Catholic schools perform better (though not always
significantly) on all academic outcomes than non-religious public schools. However, after
taking the input differences into account, no significant output differences between these two
types of religious schools and non-religious schools remained. Hence our hypothesis that
religious schools perform better academically needs to be rejected for the Protestant and
Catholic sector. The difference between the gross and net effects can partly explain the
continued popularity of Christian denominational schools. That is, parental perceptions about
religious school quality cannot ‘control’ for input measures. At best, these perceptions reflect
gross differences between schools. The simple fact that the raw academic scores are higher in
religious schools often is interpreted by many parents as an indication of their superior
performance. In other words, there is a discrepancy between perceptions (which obviously do
not control for input at the pupil or school-level) and the reality of value-added scores that
come closer to the actual performance of a school (cf. Burgess et al. 2015).
16
Importantly, this finding does not hold for Islamic schools. Most tend to have low raw
scores, which may be regarded as an indication of low quality by many parents. Added to
these one-sided perceptions, many policy-makers continue to express concerns about the
effectiveness of Islamic schools. However, when net differences in achievement scores are
more closely examined, it becomes clear that Islamic schools are by far adding the greatest
educational value. In fact, with regard to a number of achievement measures, Islamic schools
succeed in improving their position from the most disadvantaged (gross) sector to a sector
which does not differ from the other sectors (net), although the difference is only statistically
significant in one specific case, i.e. with respect to fifth grade math scores. One explanation
provided for this result may be a more positive school climate, better teacher-pupil
relationships (see Agirdag, Van Houtte, and Van Avermaet 2012), and extra attention given
both to language and math skills but also the cultural background of the pupils, each of which
are of crucial importance to pupils with disadvantaged backgrounds (cf. Merry & Driessen
2014). Taken together, these findings suggest that the academic performance of Islamic
schools may owe to being part of a functional community, and thus support Coleman’s social
capital theory; other outcomes suggest a better match between the specific circumstances and
needs of these pupils and the chosen educational approach that Islamic schools aim to
provide.
With respect to our second hypothesis concerning non-cognitive outcomes the results
initially appear to be the opposite to those concerning the cognitive measures: Christian
denominational schools score somewhat lower than public non-religious schools while
Islamic schools perform much better, especially in matters concerning citizenship. The latter
finding suggests that the distrust of many opponents to a separate Islamic school sector are
unwarranted. But again, as was the case with the cognitive outcomes, after controlling for
pupil and school demographics no relevant denominational differences remained. This
implies that our first hypothesis must be rejected as well.
In fact, our findings show there to be hardly any net denominational differences,
whether in cognitive or in non-cognitive output measures. Insofar as gross differences exist,
these might simply be explained by the social and ethnic backgrounds of the pupils. But
Dronkers (2004) also suggests that the higher academic effectiveness of religious schools is
mainly restricted to a specific historical period, viz. the 1960s to the 1990s. During the 1990s
the small-scale advantages of religious schools began to disappear for the simple reason that
many individual Protestant and Catholic schools had become too large; additionally, many
mainstream denominational schools were increasingly governed by large-scale professional
17
organizations supervising more and more schools, thus drastically reducing the positive
effects of communal bonds. And indeed, in this study we also found no significant non-
cognitive differences between the Christian school sectors and the public school sector. This
lack of substantive difference between the public-private sector in the Netherlands is not a
little surprising given that such purported differences are the raison d’être of religious
schools.
However, there remain a number of unanswered questions. For example, in this study
we did not capture specific aspects traditionally associated with religion, such as socialization
according to specific religious value systems. On the other hand, it remains unclear whether
religion still plays a prominent role in the average mainstream Protestant or Catholic school.
There is also the fact that primary schools in the Netherlands are obliged to offer both
citizenship education as well as lessons regarding religious and ideological movements.
Public (‘neutral’) schools may therefore be just as successful in transmitting values as
religious schools are (Dronkers and Avram 2014). And finally, it may be the case that
religious schools are less about dogma or belief than about being with others sharing a similar
background and/or set of concerns (Merry 2015). In other words, religious schools may
simply be proxies for social communities where parents select a religious school not in order
to socialize their children into a specific religion, but rather to congregate with others with
whom they share a social class background, or other educational priorities.
In other words, what we still lack (and what likely only in-depth qualitative studies can
provide) is greater insight into the motivations of parents but also the actual differences in
everyday school-life between the various categories of schools. Gathering more information
about parental motivations, but also the extent to which differences between denominational
and non-denominational schools are (in)significant may even come to influence policy
decisions concerning the operation and funding of religious schools. Whatever the case, we
believe that our conclusions about the absence of clear denominational effects will be
instructive for, and help to stimulate, research of a similar type in other countries with a
mixed educational economy in terms of religious and non-religious school provision.
References
Agirdag, O., Van Houtte, M., and Van Avermaet, P. 2012. “Ethnic school segregation and
self-esteem: The role of teacher–pupil relationships”, Urban Education 47: 1135-59.
18
Allen, R., Burgess, S., and McKenna, L. 2014. School Performance and Parental Choice of
School. London: Department for Education.
Avram, S., and Dronkers, J. 2011 “School sector variation on non-cognitive dimensions: are
non-pubic but publicly supported schools more effective?”, Educational Research and
Evaluation 17: 115-39.
Burgess, S., Greaves, E., Vignoles, A., and Wilson, D. 2015. “What parents want: school
preferences and school choice”, The Economic Journal 125: 1261-1289.
Carbonaro, W., and Covay, E. 2010. “School sector and student achievement in the era of
standards based reforms”, Sociology of Education 83: 160-82.
Chubb, J., and Moe, T. 1990. Politics, Markets and America’s Schools. Washington, DC:
Brookings Institute.
Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ:
Erlbaum.
Coleman, J. 1988. “Social capital in the creation of human capital”, American Journal of
Sociology 94: S95-S120.
Coleman, J., Kilgore, S., and Hoffer, T. 1981. Public and Private High Schools., Washington:
National Center for Educational Statistics.
Denessen, E., Driessen, G., and Sleegers, P. 2005. “Segregation by choice? A study of group-
specific reasons for school choice”, Journal of Education Policy 20: 347-68.
Dijkstra, A., Driessen, G., and Veenstra, R. 2001. “Academic achievement in public, religi-
ous, and private schools. Sector and outcomes differences in the Netherlands”, paper
Annual Meeting AERA, Seattle, April 9-14.
Dijkstra, A., Dronkers, J., and Karsten, S. 2004. “Private schools as public provision for
education: school choice and market forces in the Netherlands”, In Educating Citizens.
International Perspectives on Civic Values and School Choice, edited by P. Wolf and
S. Macedo, 67-90. Washington: Brookings Institute Press.
Driessen, G., and Merry, M. 2014. “Trends in educational disadvantage in Dutch primary
school”, Educational Review 66: 276-92.
Driessen, G., Mulder, L., and Roeleveld, J. 2012. Cohortonderzoek COOL5-18. Technisch
Rapport Basisonderwijs, Tweede Meting 2010/11 , Nijmegen: ITS.
Dronkers, J. 2004. “Do public and religious schools really differ? Assessing the European
evidence”. In Educating Citizens. International Perspectives on Civic Values and
School Choice, edited by P. Wolf and S. Macedo, 287-314. Washington: Brookings
Institute Press.
19
Dronkers, J., and Avram, S. 2010. “A cross-national analysis of the relations of school choice
and effectiveness differences between private-dependent and public schools”,
Educational Research and Evaluation 16: 151-75.
Field, A. 2009. Discovering statistics using SPSS. London: Sage.
Goldring, E., and Phillips, K. 2008. “Parent preferences and parent choices: the public–
private decision about school choice”, Journal of Education Policy 23: 209-30.
Heck, R., Thomas, S., and Tabata, L. 2010. Multilevel and longitudinal modeling with IBM
SPSS. New York: Routledge.
Hofman, W., Hofman, A., Guldemond, H., and Dijkstra, A.B. 1996. “Variation in
effectiveness between private and public Schools: the impact of school and family
networks”, Educational Research and Evaluation 2: 366-94.
Jeynes, W. 2012. “A meta-analysis on the effects and contributions of public, public charter,
and religious schools on student outcomes”, Peabody Journal of Education 87: 305-35.
Jeynes, W., and Beuttler, F. 2012. “What private and public schools can learn from each
other”, Peabody Journal of Education 87: 285-294.
Lauri, T., and Pöder, K. 2013. “School choice policy: seeking to balance educational
efficiency and equity. A comparative analysis of 20 European countries”, European
Educational Research Journal 12: 534-52.
Merry, M. 2007. “Should the state fund religious schools?” Journal of Applied Philosophy
24: 255-70.
Merry, M. 2015. “The conundrum of religious schools in twenty-first-century Europe”,
Comparative Education 51: 133-156.
Merry, M., and Driessen, G. 2014. “On the right track? Islamic schools in the Netherlands
after an era of turmoil”, Race, Ethnicity and Education doi
10.1080/13613324.2014.985586
Midgley, C. et al. 2000. Manual for the Patterns of Adaptive Learning Scale , Ann Arbor:
University of Michigan.
Ministry ECS 2014. Key Figures 2009-2013. Education, Culture and Science , Den Haag:
Ministry of Education, Culture and Science.
Morgan, S., and Sørenson, B. 1999. “A test of Coleman’s social capital explanation of school
effects”, American Sociological Review 64: 661-81.
Norris, P., and Inglehart, R. 2011. Sacred and Secular: Religion and Politics Worldwide,
New York: Cambridge University Press.
20
Ritzen, J., Van Dommelen, J., and De Vijlder, F. 1997. “School finance and school choice in
the Netherlands”, Economics of Education Review 16: 329-35.
Seegers, G., Van Putten, C., and De Brabander, C. 2002. “Goal orientation, perceived task
outcome and task demands in mathematics tasks”, British Journal of Educational
Psychology 72: 365-84.
Strand, S. 2014. “Ethnicity, gender, social class and achievement gaps at age 16:
intersectionality and ‘getting it’ for the white working class”, Research Papers in
Education 29: 131-71.
Ten Dam, G., Geijsel, F., and Reumerman, R. 2011. “Measuring young people’s citizenship
competences”, European Journal of Education Research, Development and Policy 46:
354-72.
Willms, J. 1985. “Catholic-school effects on academic achievement: new evidence from the
high school and beyond follow-up study”, Sociology of Education 58: 98-114.
21
Table 1. Descriptive statistics for the outcome measures Grade Domain Mean SD n
2 Language 64.04 10.85 6891
Math 59.32 13.74 6505
5 Reading 25.36 14.15 8084
Math 70.89 15.45 8424
8 Reading 53.58 19.41 4451
Math 109.92 13.16 6048
8 Language 74.11 12.61 4862
Math 42.35 11.03 4826
Study Skills 29.26 6.25 4809
Total 534.96 9.36 5004
8 Recommendation 3.46 1.23 6138
5 Self-Efficacy 3.82 0.66 8067
Task Motivation 4.17 0.63 8091
8 Self-Efficacy 3.70 0.60 7839
Task Motivation 3.94 0.61 7844
8 Knowledge 0.78 0.16 7887
Reflection 2.25 0.56 7884
Skills 3.01 0.40 7884
Attitudes 2.95 0.43 7896
22
Table 2. Parental education and ethnicity by school denomination (percentages; n = 26,039) Parental education Ethnicity
Medium
and higher
Low Very low Dutch Turkish Moroccan Other non-
Western
Public 81.4 10.1 8.5 82.2 5.7 4.9 7.2
Protestant 88.0 9.3 2.7 92.7 1.3 1.5 4.4
Catholic 87.1 9.4 3.5 90.4 2.0 2.3 5.3
Islamic 48.1 8.2 43.6 15.9 24.7 42.0 17.4
Total 83.5 9.6 6.9 84.6 4.2 4.9 6.3
Eta 0.31 0.36
23
Table 3. Effects of denomination on reading skills in grade 5 Model 1 Model 2 Model 3
(Gross) (Net)
Level Intercept -0.0878* 0.0476 0.1869***
School Denomination Public Reference reference reference
Protestant 0.1389* 0.0745 0.0168
Catholic 0.1676** 0.1143* 0.0613
Islamic -0.3717*** -0.0348 0.1012
Pupil Parental education Medium and higher reference reference
Low -0.3955*** -0.3806***
Very low -0.3759*** -0.3483***
Parental ethnicity Dutch and Western reference reference
Turkish -0.4426*** -0.3754***
Moroccan -0.3316*** -0.2836 ***
Other non-Western -0.1590*** -0.1272**
School Parental education % Low -0.0063*
% Very low -0.0039
Parental ethnicity % Turkish -0.0044
% Moroccan 0.0015
% Other non-Western -0.0045
*p < 0.05 ** p < 0.01 *** p < 0.001
24
Table 4. Gross and net effects of denomination (reference category: public schools) Gross effect Net effect
Grade Domain Protestant Catholic Islamic Protestant Catholic Islamic
2 Language 0.14* 0.07 -0.70*** -0.00 -0.06 -0.10
Math 0.07 0.09 -0.23 -0.02 -0.01 0.06
5 Reading 0.14* 0.17** -0.37*** 0.02 0.06 0.10
Math 0.18*** 0.12** -0.24* 0.07 0.03 0.27*
8 Reading 0.09 0.16* -0.25 -0.08 -0.00 0.29
Math 0.19* 0.15* 0.13 0.06 0.03 0.25
8 Language 0.22* 0.09 -0.34* 0.08 -0.03 0.05
Math 0.16* 0.09 -0.00 0.05 0.00 0.20
Study Skills 0.20* 0.12 -0.23 0.06 0.02 0.06
Total 0.20** 0.14* -0.20 0.06 0.03 0.13
8 Recommendation 0.13* 0.07 -0.15 0.02 -0.01 0.03
5 Self-Efficacy -0.08* -0.06 0.29*** -0.04 -0.02 -0.01
Task Motivation -0.10* -0.02 0.27*** -0.05 0.02 0.04
8 Self-Efficacy -0.07 -0.05 0.37*** -0.04 -0.03 0.10
Task Motivation -0.08 0.04 0.60*** 0.00 0.11** -0.08
8 Knowledge 0.03 0.09 -0.44*** -0.08 -0.01 -0.12
Reflection -0.11* -0.06 0.55*** -0.06 -0.02 0.10
Skills -0.14** -0.03 0.57*** -0.10 0.00 0.00
Attitudes -0.10 -0.02 0.59*** -0.06 0.02 0.05
*p < 0.05 ** p < 0.01 *** p < 0.001