Economics Working Paper Series

Centre for Training and Research in

Public Finance And Policy

Reserve Bank of India, Industrial Economics

Cell

PAPER TITLE Small Bang for Big Bucks? An Evaluation of a Primary School Intervention in India

PAPER AUTHOR

Jyotsna Jalan,

& Elena Glinskaya,

WP NO 9

CENTRE FOR TRAINING AND RESEARCH IN PUBLIC FINANCE AND POLICY WEBSITE: WWW.CTRPFP.AC.IN

Centre for Studies in Social Sciences,

Calcutta

2

Small Bang for Big Bucks? An Evaluation of a Primary School Intervention in India 1

Jyotsna Jalan, Centre for Studies in Social Sciences, Calcutta, India

Elena Glinskaya, The World Bank, Washington DC, USA

Abstract

Primary school interventions have regularly been undertaken by governments in many countries to increase enrollments and improve quality of learning. Such initiatives included building schools, training teachers, providing school-meals at noon, revising textbooks etc.. In India too, several policy initiatives have been undertaken over the years. A recent intervention is the District Primary Education Program (DPEP). Under the DPEP, efforts were made to design project components that would reduce gaps in enrollments, dropouts and learning achievements across gender and disadvantaged social groups. By December 2001, over 1.5 billion US dollars had been committed to the project and the school system under DPEP was expected to cover more than 50 million children. Question is whether the DPEP was successful in increasing enrollment rates particularly among the targeted groups and/or in improving quality of learning of the already enrolled children. Our estimates show some impacts on enrollments of the socially disadvantaged minority groups (the scheduled tribe and scheduled caste) especially in one specific state where concurrent to the DPEP, another state government-supported primary school initiative was also started. Contrary to the stated objectives of the program, however, there was no closing of the gender gap either in terms of enrollments or with respect to educational achievements. JEL Classification codes: C310, I280, O150

Keywords: Education, Primary school intervention, Evaluation 1 Suchismita Banerjee provided excellent research assistance in this project. We thank Chang-Tai Hsieh for extensive comments that have contributed significantly in making this a better paper. Sajitha Bashir, Prema Clarke, Shanta Devarajan, E. Jimmenez, Lant Pritchett, Martin Ravallion, Stephen Howes, Michelle Riboud, Arijit Sen and Vandana Sipahimalanirao made helpful suggestions at various stages of the project. We also thank seminar participants in a talk given at the Ministry of Human Resource Department, Government of India, New Delhi and The World Bank, Washington DC for very insightful comments. The views expressed in this paper are those of the authors and do not reflect the views of their employers. Financial support from Bank-Netherlands Partnership Program “Mainstreaming Poverty Impact Evaluation in Operations” (TF 029906) is gratefully acknowledged.

1

Introduction

Governments around the world have invested and continue to invest substantial public

resources to provide elementary education to their children. However, the effectiveness

(measured in terms of observable indicators) of such expenditures is still ambiguous. The Coleman report (1966) released more than 30 years ago concluded that after controlling for

family background, there was little evidence that the level of school resources had a

statistically significant effect on student test scores. Since then, numerous studies by

Hanushek (1985) and others persuasively argue that providing additional resources for school are not closely related to variations in school outcomes unless coupled with other

fundamental reforms. But others like Card and Krueger (1992) provide some support that resources do matter. They show stronger positive results between school resources and

students’ educational attainments and subsequent labor market outcomes as compared to that between resources and test scores.

All the above studies are based on empirical evidence using data from the US. There is limited similar evidence available for developing countries. Some studies on developing

countries include the impacts on completed schooling and enrollment of schooling infrastructure in Indonesia (Duflo, 2001), in Brazil (Birdsall, 1985), in South Africa (Deaton

and Case, 1996) etc.. At least to our knowledge, there is no evidence on the effectiveness of

an overall increase in overall resources for primary school education (similar to the US) for

developing countries. Our paper is an attempt to fill this vacuum using evidence from an elementary school project in India.

Since independence, for successive governments in India, “education for all” for children 14 years and younger has been the political mantra. Several policy initiatives have

been undertaken over the years by both the central and the individual state governments2,3.

2 These schemes include Non-formal Education program (1979-80), Operation Blackboard for small rural schools (1986), Total Literacy Campaigns (1988), District Institutes of Education and Training (1988), Minimum Levels of Learning (1989). See World Bank (1997) for further details. 3 Some of these schemes include Shiksha Karmi and Lok Jumbish schemes in Rajasthan, the Bihar education Project, Education Guarantee Scheme in Madhya Pradesh, Uttar Pradesh Basic Education Project (UPBEP), Andhra Pradesh Primary Education Project. In fact the UPBEP project was a pilot for the subsequent DPEP initiative. Under UPBEP, US$165 million was provided under the World Bank IDA terms to the Government of Uttar Pradesh. See World Bank (1997) for further details.

4

During 1951-52 and 2002-03, total education expenditures by both the center and the state as a percentage of the GDP increased from .64 percent to 3.61 percent of which more than a third was on primary school education4. Per student public expenditure at the primary school levels also increased from Rs.99.50 in 1951-52 to Rs.184.50 (at 1980-81 prices) in 1990-91

indicating an increase of 2.14 percent per annum. So have these increased expenditures in education by the government reflected in improved educational outcomes?

Simple education outcome statistics suggest that school enrollments have increased significantly over the decades, but education indicators for disadvantaged groups like females, scheduled castes and scheduled tribes and for the general population in some

geographical areas continue to be dismal. According to the 1991 census, literacy rate among

rural women was 30.6 while the national female literacy rate for India was 39.3. In some states, the female literacy rate in rural areas was as low as 11.6. Scheduled castes and tribes

had even lower literacy rates (23.7 and 18.2 respectively in 1991), enrollment and achievement rates, and higher dropout rates compared to the general population. So how

well has the government done in providing primary school education in India? We try and answer this question in the context of a large-scale primary school intervention undertaken by

the government of India in 1993-94.

The Government of India launched a major primary school education initiative through the District Primary School Education Program (DPEP) in 1994. Under the scheme, districts with poor educational indicators were selected to receive financial assistance

towards improvement of school infrastructure, teacher training, textbook improvement, etc..

The principal objectives of the DPEP were to provide access for all children to primary

school or its equivalent non-formal education, to reduce overall dropout rates, and to reduce gaps in enrollments, dropouts, and learning achievements among gender and social groups.

DPEP provided an additional $9.1 per student per year in program districts.

The DPEP was a centrally sponsored scheme where 85 percent of the approved project cost within each district was borne by the central government and the remainder by

the state government. The central government's contribution in the DPEP was funded by

4 Source: Selected Education Statistics, Ministry of Human Resource Department, Government of India

5

multi- and bi-lateral aid agencies like the World Bank, European Union (for Madhya Pradesh), Government of Netherlands (for Gujarat), Department of International Development (DFID) (for Orissa, West Bengal and Andhra Pradesh), and United Nations International Children’s Fund (UNICEF) (for Bihar). To date, these international institutions

have committed resources to the tune of over $1.5 billion for the project of which the World Bank’s commitment has been slightly over $1 billion. This makes DPEP one of the largest

donor assisted programs in the world

DPEP is a very intensive primary school intervention by the central government that was implemented in more than 271 districts spread over 18 states (15th joint Review Mission

Report, DPEP Bureau). The system covered 51.3 million children and 1.1 million teachers in

3,75,000 schools. More than 40,000 new school buildings were constructed, about 66,000 works undertaken to provide ancillary facilities like drinking water, toilets, etc. Grass root

level participation was strengthened considerably with several thousands of village education and school management committees established under the auspices of the program. To date,

more than a million teachers and 3 million community members have been trained under the program.

Question is to whether additional resources that were available to the project districts under the DPEP led to improvement in education outcome indicators especially among

disadvantaged groups like girls and children belonging to scheduled caste and tribes?

As of 1999-00 (post-project year in our data), the first phase of the project (DPEP- I) had been in place for a little more than 5 years and thus covered at least one cycle of primary

education among school-going children. In this paper, we evaluate DPEP-I in terms of its stated objectives: (a) whether primary school enrollment rates have increased since the inception of the project among different age groups and for specific sub-populations (female

children, and children from scheduled caste and tribes);5(b) whether the program has encouraged students to remain in the schooling system beyond primary school and hence 5 Due to small sample problems, we group scheduled castes and tribes together into the minority group. In an earlier version of the paper, we did find differences in the impacts across the two groups but our estimates were often not robust due to small sample problems.

6

4

7

reduced dropout rates; and (c) whether fewer children report not getting any education and whether more children complete the primary school cycle in the post-DPEP period.6

The first phase of DPEP was started in 1994, where 42 districts in rural India, across

the seven states of Assam, Haryana, Karnataka, Kerala, Madhya Pradesh, Maharashtra, and Tamil Nadu were chosen as project districts.7Program placement was not random but was

primarily on the basis of low female literacy rates in 1991.8 Total project costs committed

for DPEP-I were $310.5 million over a seven year period that was subsequently extended by

a year. Under the first phase of the project, DPEP covered 63,114 regular schools, 32,467 Alternative School (AS) centers covering 12,03,399 students, 45,004 early education centers

(including anganwadi (child-care) centers and mother associations), constructed 4,222 new

schools and added 5,745 new classrooms. Community groups like Village Education Committees (VECs) and Mother-Teacher Associations (MTAs) were also

established- 1,49,810 community groups were set up between 1994-2000.9

The original DPEP guidelines put significant emphasis on research, monitoring and evaluation of the program. The Research, Evaluation and Studies Unit (RESU), which

became functional from March 1995, compiled 255 research abstracts in primary education between 1994-2000 conducted at the nationa l, state and district levels.10 A defining feature

of all these studies is that they have focused primarily on changes in educational outcomes 6

The other three phases began only a few years prior to 1999-00, so the available data is inadequate to

evaluate the program impacts of the later phases. Our paper thus focuses only on Phase I of the program.

7 Subsequently, the program has expanded in a phased manner to 242 (273 bifurcated districts) in 18 states under Phases II (1996-97), III (1997-98) and IV (1999-00) of the project.

8 The DPEP guidelines indicate successful implementation of the Total Literacy Campaign (TLC) in the

districts along with low female literacy rate (below the national level) as the two criteria used to choose districts. TLCs were community based efforts to improve community-level adult literacy. For further details see Govinda (2002). However, by 1994, TLC had been implemented in almost all districts in India. 9 The figures for regular and newly constructed schools, and addition of new classrooms are from Aggarwal (2000). The other figures are from the various Joint Review Commission reports and in many cases, they are numbers for DPEP-I states (including the expansion districts of 1997) rather than for the original 42 districts only.

10

(2001).

Some of the nation-wide studies include those by Pandey (2000), Aggarwal (2000a, 2000b), Menon

8

5

9

that have happenedwithin

DPEP districts. On this basis, these studies have by and large declared the DPEP to be a success in terms of its stated objectives.

However, the important point to note is that since independence, there has been a

secular “improvement” in the various educational indicators in India.11So, within any district selected under the DPEP, we would expect the primary school indicators to improve

over time even if the district did not get DPEP assistance. Thus, in evaluating the DPEP, we

have to determine the extent of improvements in primary school education over and above

the improvements that would have happened in the “natural course of events”. In other words, we have to determine the net impact of the program. If instead, we (like some of the

studies mentioned above) measure the gross improvements in the program districts, we will

probably be seriously overestimating its impact.

We use propensity score matching (PSM) methods to estimate the net impacts of the DPEP. PSM balances the distributions of the observed covariates between a treatment group

and a control group based on similarity of their predicted probabilities of being declared a project district (their “propensity scores”). The method does not require a parametric model

linking program placement to outcomes, and thus allows estimation of mean impacts without arbitrary assumptions about functional forms and error distributions.12

The DPEP under its project guidelines did not collect data on different educational indicators for non-project districts. Given this constraint, in order to estimate the net impacts

of the program, we have to use available secondary data. This also means that we are unable

to evaluate the different specific components of the project. At best, we can evaluate the

program in terms of general education indicators like enrollments, educational atta inment, cohort progression and dropout rates. However, conditional on these limitations, this paper is

the first rigorous quantitative evaluation of the program since its inception. 11 For example, literacy rates in India have steadily increased over time from 18.3 percent in 1952 to 52.2 percent in 1991 (World Bank, 1997). 12 However, we only estimate the direct effects of program participation (i.e. the “partial equilibrium” effects of the treated). We do not attempt to estimate or control for the possible indirect effects of the program on the non-DPEP districts within a “general equilibrium” framework.

10

6

11

We combine data from the 1991 census with two rounds of the all-India household data from the National Sample Survey (Round 50, 1993-94 and Round 55, 1999-2000)

collected routinely by the National Sampling Survey Organization (NSSO), Department of Statistics and Program Implementation, Government of India. We report the impacts for all

the seven states where DPEP was implemented under Phase I of the project. We also report

the impacts for the state of Madhya Pradesh separately. We do this because in Madhya

Pradesh, two substantive state level primary school interventions Alternative Schooling

(AS) and Education Guarantee Scheme (EGS) were started at around the same time as DPEP Phase I.

Our impact estimates show some positive net impact of the DPEP on primary school enrollment rates, stock of completed primary school education, and progression into higher

levels of education beyond primary. However, the impacts are not uniform. The more striking impacts are noticed for children belonging to the minority group (i.e. scheduled caste

and scheduled tribe) and for children living in the state of Madhya Pradesh.

Our analysis clearly shows that the net impact of the program is substantially smaller than the gross improvements. Further, it matters how the net impact is calculated. A simple

method to measure the net impact would be to compare the average outcomes in treatment districts with those in non-treatment districts. Aggarwal (2000) does precisely this and finds that between 1993-94 and 1996-97, the net impact of DPEP Phase I on enrollments was 5

percent, with it being as much as 16.8 percent in Assam. In contrast, our propensity score

matching results indicate that there was no impact on the enrollment rates of primary school

age children.

Apart from increasing access, and reducing dropouts in the overall population, an avowed aim of DPEP was to reduce gender and social disparity in access, retention and

dropouts. Our results show that the program’s performance in the Phase I districts in

reducing gender-disparity has not been very successful. However, DPEP Phase I has had an positive net impact on the three outcome indicators for the minority group in all DPEP-I

states and in the state of Madhya Pradesh.

12

7

13

Our paper is organized as follows. In the next section we discuss the various aspects of the DPEP proje ct in general. In Section 3, we provide an intuitive description of the statistical methodology that we use in our analysis (we provide technical details about the methodology in the Appendix), and Section 4 describes our data sources. Our results are

presented in Section 5. Interpretation of the size of the program impacts in the context of a simple model is given in Section 6. Section 7 contains some concluding comments.

2

The DPEP Guidelines

The DPEP project was approved in early 1994 as a centrally sponsored intervention in primary school education in India. “…DPEP was designed to help achieve the objective of

Universal Elementary Education (UEE) outlined within the policy framework of the revised National Policy on Education 1986 (as updated in 1992) and the Programme of Action,

1992” (16th Joint Review Mission). Under the first phase of the project, 42 districts spread over the seven states of Assam, Haryana, Kerala, Karnataka, Madhya Pradesh, Maharashtra

and Tamil Nadu were chosen as project sites. At that time, it was envisaged that the project

would subsequently cover all districts with female literacy rates below the national

average.13 Today the project covers more than 240 districts spread over 18 states of India.

As Table 1 indicates, the choice of the initial 42 districts to a great extent satisfied the above criterion. With the exception of Kerala where no district has a female literacy rate below the national average, in all other states, all chosen districts had low female literacy rates. However, Table 1 also shows that the districts chosen were not necessarily the ones

with the worst female literacy rates.14

Keeping the principles of the National Policy of Education (1986, updated in 1992) in

mind, the charter of the DPEP sought to operationalize the following: 13

Explicitly, two criteria had to be satisfied for a district to be chosen as a project district: low female literacy rate and districts where Total Literacy campaigns had been successfully implemented. The latter criterion was satis fied by nearly all districts in India at the start of the program.

14 The project document does not state that districts with the lowest female literacy rates in a state were to be the first to be chosen as project districts. However, under DPEP -I, in addition to the stated criteria of low female literacy and implementation of TLC, the policymakers implicitly chose to pick districts where they felt that there was considerable scope for the project to be successful (Pandey, 2000).

14

8

15

“… efforts would be made to develop district specific projects, with specific activities, clearly defined responsibilities, definite time-scheduled and specific

targets. Each district project will be prepared within the ma jority strategy framework and will tailored to the specific needs and possibilities in the district. Apart from

effective universalization of elementary education, the goals of each project would include the reduction of existing disparities in educationa l access, the provision of

alternative systems of comparable standards to disadvantaged groups, a substantial

improvement in the quality of schooling facilities, obtaining a genuine community

involvement in the running of schools, and building up local level capacity to ensure effective decentralization of educational planning. That is to say, the overall goal of

the project would be reconstruction of primary education as a whole in selected

districts instead of piecemeal implementation of schemes. An integrated approach is more likely to achieve synergies among different program components”. (DPEP

Principles, 1994)

To achieve its stated goals, the program adopted measures to strengthen the capacities of national, state and district institutions for planning, management and evaluation of primary

education. At the national level, the Ministry of Human Resource Development, Department of Education was responsible for overall control and management of the Program; it

established a DPEP bureau headed by a Joint Secretary and six Directors/Deputy Secretaries.

At the State level, the program was implemented through registered state level autonomous societies with the Chief Minister of the state as ex-officio chairperson. At the district and

sub-district levels , program implementation plans were undertaken in consultation with

General Committee, and the Executive Committees. Action plans and budgets were

developed at the district level with active participation from the community, NGOs, and teachers.

To get financial assistance under the program, states had to maintain their 1991-92 expenditure levels (in real terms) on elementary education. Individual state’s 15 percent contribution to the DPEP could not be included as part of these expenditures. Bashir and

Ayyar(2001) conclude that in general state governments fulfilled this condition. Real

expenditures increased relative to the first year of the project. Expenditures on civil works were restricted to 24 percent of the total (later increased to 33 percent) and management cost

16

9

17

to 6 percent. Emphasis of project expenditures on non-salary inputs for primary education was a major departure from earlier primary school interventions. DPEP did not finance non-

educational incentives like free uniforms, incentives for attendance, nutrition etc. except under special circumstances.

The DPEP recognized the role of local community initiatives in promoting enrollment, retention, achievement and school effectiveness. Communities were mobilized to make

communities to understand the importance of primary school education and to persuaded

them to send their wards to school. It encouraged the formation of community level structures like that Village Education Committees (VEC) and local bodies like Mother-

Teacher Assoc iations (MTA), so that they could assist in creating awareness campaigns for

providing the necessary institutional infrastructure for increasing enrollment, retention, and for facilitating schools and Non-Formal Education (NFE) centers.

DPEP stressed on development and introduction of textbooks based on the minimum levels of learning (MLL) guidelines and on the availability of textbooks and learning materials in major tribal languages. MLLs while lacking a theoretical framework defined a

set of basic skills to be acquired by students in mathematics, language and environmental studies. At the project inception stage, it was anticipated that 10 percent of the total base

costs would be spent on developing and revising teaching materials (DPEP Staff Appraisal

Report, The World Bank, 1994).

Except for para-teachers, most teachers in DPEP schools possessed a one year pre-service teacher certification degree for instruction in elementary education. DPEP introduced regular

in-service training ranging from 3 – 20 days for a large number of teachers that included a wide variety of topics. This included encouraging the teachers to make the classroom

atmosphere less intimidating and friendlier and to sensitize them to the needs of girl students and to that of children belonging to minority groups.

Another innovation of the DPEP was to provide for early childhood education in the expectation that enrollment and retention of girls would increase by providing an alternate

source of sibling care during school hours. It also encouraged integrated education of disabled children. It proposed to involve the community to identify type, degree and extent

18

10

19

of disabilities amongst the primary level age group. Necessary skills were to be imparted to the parents and the disabled children. Teachers were to be trained to detect disabilities, use

aids and appliances, and implement individualized education plans.

Lastly, the program emphasized on initiatives to strengthen educational planning and

management capacities in the project states. Project undertook improvements in the role and

functioning of the District Institute of Education and Training (DIETs). This would involve

setting up of separate State Institute of Educational Management and Training (SIEMAT) to

augment state level structures like State Council of Educational Research and Training (SCERT). Training activities of teachers in private aided schools, pre-primary teachers other

than those under Integrated Child development Scheme (ICDS), VEC/MTA members and

NGOs were also to be financed under the project.

All the above initiatives were taken to (a) provide access for all children to primary school or its equivalent alternative education, (b) to reduce overall dropout rates, and (c) to

reduce gaps in enrollments, drop-outs and learning achievements among gender and social groups to less than 5 percent.

In this paper we intend to determine the net impact (between 1993/94 and 1999/2000) of the first phase of the DPEP on the three stated objectives described above. Ideally, we

would like to know the net impact of the DPEP on the scholastic achievements of primary school students in the participating districts, and also the precise mechanics by which the

DPEP affected these outcomes. However, answering these difficult questions require

detailed data availability.

3

Methodology

We want to assess whether the DPEP has had a “net impact” on primary schooling outcomes. In principle, the net impact would be the difference between the observed outcomes in the treatment districts and the (unobserved) outcomes in the same districts if there were no

DPEP. In practice, of course, the exact counterfactual of a declared DPEP district is non-

existent. Constructing counterfactuals is the central problem in the literature on evaluating social programs.

20

11

21

A simple method to get around the above problem would be to compare the average outcomes in treatment districts with those in non-treatment districts. However, program placement is not random and observed outcomes of the program may themselves be

influenced by the selection criteria. In our case, there are non-program districts with high female literacy rates, and good educational facilities. To the extent that both these factors

influence educational outcome indicators, the observed educational outcomes of the non-

treatment districts, on average, do not measure the outcome indicators of the treatment

districts in absence of the program. Therefore, differences in average outcomes across treatment and non-treatment districts do not isolate the impacts of the program on the treated.

Such “selection bias” will generally arise in situations where there is a direct relationship

between outcomes and choice of treatment districts.

There are several approaches used in the literature to deal with the selection bias problem. One could randomly pick the treatment districts so that the non-treatment districts

are the true counterfactuals and there is no selection bias problem. In practice, however, such random experiment designs are not feasible especially in a large-scale program like the

DPEP.

Alternatively, one can use econometric techniques to “adjust out” the systematic differences between the non-experimental comparison group and the treatment group. Propensity score matching technique (the method we adopt in this paper) is one such

econometric method where statistical models are used to generate a comparison group that is

similar in all respects to the members in the participant group, except that they do not get the

treatment. Each program participant is paired with an observably equivalent non-participant.

The “no-treatment” matched district is declared a counterfactual for “treatment” district when

the difference between the predicted probabilities is the minimum among the set of all potential matches.15 Once the “matches” are made, the difference in their outcomes is

15 We match program districts with non-program districts on the principle of sampling without replacement. That is, our method chooses the “best” possible match on the basis of the Mahalanobis distance metric. A potential control district can thus be matched with more than one treatment district. This procedure ensures that each treatment district is matched with a non-treatment district that is closest to it in terms of the observed covariates but did not have the program.

22

12

23

interpreted as the mean effect of the program on the treated. 16 Matching is justified on the fact that conditioning on observed covariates, the potential selection bias in program placement is eliminated.

In estimating the propensity score, female literacy rates would certainly be a determinant in our model. However, since in addition to low female literacy rates, districts

were also chosen on the probable success of the program if implemented. To control for this

we included a host of indicators like population density, scheduled caste/scheduled tribe

(SC/ST) concentration, female literacy rates for overall population and specifically for SC/ST population, proportion of people who are born in the enumeration area, housing indicators of

the households (electricity, water, toilet, fuel usage, roofing material, floor material), village

infrastructure characteristics (whether village has a bank, communication systems, education facilities, etc.) that would proxy for the backwardness of a particular district.

In our particular case, some additional issues need to be noted. First, since there are institutional differences in educational systems across states, we choose treatment and control districts from within the same state.17 Second, the DPEP project was implemented in a

phased manner all-over India. DPEP-I was implemented in late 1994 and three years later, 40 additional districts in the same states were chosen as project sites under DPEP-II.18 Given

that our two comparison periods are 1993-94 and 1999-00, we have to exclude the additional

districts under DPEP-II from the set of potential candidates to construct the counterfactual otherwise we will underestimate the impact of the program. Therefore, out of a total of 163

districts in the seven DPEP-I states, we select out the 42 districts that were project districts in

1994 and the 40 additional districts that were brought under the DPEP framewor k in phase 16

To ensure that we have the best possible model to predict the propensity scores, several checks and

balances are introduced in our methodology (See, Technical Appendix for details). 17 This meant that we estimated separate propensity score functions for each state to do the matching. 18 Six additional states Andhra Pradesh, Uttar Pradesh, West Bengal, Orissa, Himachal Pradesh and Gujarat were also declared project states under DPEP -II. However, we only focus on the DPEP -I states in our study.

24

13

25

II.19 We have a sample of 93 districts remaining to find the appropriate counterfactuals for the DPEP-I districts.20

Table 1 clearly shows that low female literacy rate was not the only criterion used in

the choice of program district. In almost all states, capability of districts to absorb the program was also kept in mind while selecting them as program districts. (See Pandey, 2000)

The latter criterion is, however unobserved by us but as mentioned earlier is taken care of by

including economic progress indicators like agricultural yield, availability of infrastructure

and public services in the district. Any remaining unobserved effect is assumed to be differenced out using the difference-in-difference (DID) matching estimator that takes the

difference between outcomes before and after program in project areas, minus the

corresponding difference in the matched comparison areas. The DID estimator allows for temporally invariant differences (fixed effects) between participants and non-participants.21

We adopt a two stage matching method to get around this problem. First, we use the 1991 district level census data to match program districts with non-program districts. Once we have matched the districts, we use household level data to match households within the

matched districts. For example, if district A is matched to district B using propensity score matching methods at the district level, in the second stage, we match households in district A

with only households in district B. This way, we control for the effects of district

characteristics and of family background to estimate the impact of the program.

Finally, the data that we use is not a panel data set, but are independent cross-sections over time. So we use methods that are appropriate for a time-series of cross-sections as

discussed in Heckman et. al. (1998). See Technical Appendix for further details.

19 This is based on Census of India data for 1991.

20 DPEP als o undertook certain state level activities like providing teachers training materials etc. Benefits from these were not necessarily restricted to program districts only. (The financial resources invested in this component were however quite small relative to the overall program). However, since this component of the project is common across treatment and control districts, the common impact will wash out when we take the differences in outcome indicators between treatment and control districts.

21 In existing empirical work (for example, Smith and Todd (2000)), the DID estimator is found to be more robust than the cross-sectional matching estimators. A DID estimator however, requires data on periods before and after the start of the program.

26

14

27

4

Data and Variables

We combine the 1991 census data with household level data to estimate the impact of the

program. For the household level data, we use the 1993-94 (pre-project) and the 1999-2000 (post-project) National Sample Survey (NSS) household surveys that collect data on age-

specific enrollments. The NSS adopts a two-stage stratified sampling design, but in neither

stage is the district a stratification unit. Two issues may arise as a result of this. First, not all

districts may be in the survey. This may mean deleting some of the program districts and/or

choosing the second (or even third or higher) best matched counterfactual. Second,

aggregation of households within districts to construct district level statistics could lead to

significant sampling error. In this paper, while the first is a non-issue, our econometrics methodology takes care of the second.

The first period of the household data is treated as pre-program data and the latter period as post-program information. We use data from the seven DPEP-I states. Excluding the expansion districts under DPEP-II in these states, our data covers 41 of the 42 project

districts (Kaithal district in Haryana is covered by the NSS but for the 50th round there is no way to separate the households in Kaithal from the households in Kurukshetra, a non-project

district.). We also exclude the district of Karbi Anglong in Assam from our set of treatment

districts because available financial data indicates that flow of DPEP funds into this district started from 1997, nearly 3 years after the start of the project in other DPEP-I districts. Thus

we have households residing in 40 treatment and 87 non-project districts.

Table 2 reports literacy indicators from the 1991 census. We divide all districts in the seven project states into project districts (either under DPEP-I or DPEP-II) and non-project

districts. The numbers suggest that at least on the face of it, the program implementers have chosen the program districts prudently. In the aggregate as well as in all sub-categories,

mean literacy rates in the DPEP districts are worse than those in the non-program districts.

Descriptive statistics on the sample used in the paper are reported in Table 3. Even though the 1993-94 and the 1999-00 are independent cross-section samples, on average, the descriptive statistics match up quite closely. There are approximately 5,900 households

28

15

29

residing in the project districts and 12,500 households in the non-project districts. The per capita expenditures of households in these districts are lower when compared with the non-

project districts. One striking feature is the higher percent of Scheduled Tribes residing in

the project districts as compared to the non-project districts. This is congruent with the 1991

census data that indicate that in at least 4 of the 7 DPEP-I states, scheduled castes/tribes constituted more than 50% of total population.

We construct four main outcome variables: (i) primary school enrollments, (ii) dropout rates (iii) progression from primary to higher levels of education and (iv) highest educational level completed. Each measure was constructed at the household level.

Current enrollment measures the proportion of children enrolled in school in a specific age group at the time of the survey. We allow for under-age and over-age

enrollments by reporting the impact estimates for the age-group 5-11 years instead of the standard primary school age of 6-10 years.22 We also report impacts for the age-group 12-15

years. Using information collected on children currently not attending school but previously

enrolled in school, we report the impacts on dropout rates for the two sub-categories: 5-11

and 12-15 years.

We also look at the impact of the program on cohort progression and on dropout rates using data from the two surveys. We calculate proportion of 5-9 year olds in a household

who attended primary or pre-primary school in 1993; similarly, we calculate the proportion

of children aged 11-15 in 1999-00 that attended middle or secondary school. The

“progression” outcome variable is then calculated as difference between these two measures.

Our final indicator, educational attainment is measured as the proportion of children in the age-group 12-15 years with no education (illiterate), and those with completed primary

school education. This is based on individual’s responses about their highest education level completed education in the survey. 22

A 1992 all-India schools survey estimates that 21 percent of boys and 19 percent of girls enrolled in grade 5 were overage (World Bank, 1997).

30

16

31

We report the program impacts for all children, for girls and boys separately and for minority children where we have grouped children belonging to scheduled caste and tribes together.23

5

Program Impacts

5.1 Comparison of change in trends between DPEP and non-DPEP districts Figures 1-7 show averages of the outcome indicators (defined in the earlier section) for DPEP and all non-DPEP districts over the two time periods: 1993-94 and 1999-2000. We also

graph the averages of districts with female literacy rates less than 40 percent according to

census 1991 (a sub-set of all non-DPEP districts); we call this set “potential controls” for the treatment districts. 24 These graphs demonstrate the following points:

(i) Initial mean levels (i.e. levels in 1993-94) of the outcome indicators in DPEP districts were considerably lower than the initial averages in all non-DPEP districts.

However where we also plot the averages for potential control districts we observe these means to be significantly closer to those of the DPEP districts. For example, the difference in

school attendance among 5-11 year old minority children between DPEP and non-DPEP districts was eighteen percentage points but that between DPEP and potential control districts

was only three percentage points.

(ii) Between the two periods, mean levels of the outcome indicators improved in nearly all groups across the different sub-populations (overall, males, females, minorities).

For example, proportion of children 12-15 years old with no education declined from 8 to 6

percent in non-DPEP districts and from 24 to 16 percent in DPEP districts; proportion of

children 12-15 years old with completed primary education increased from 80 to 83 percent in DPEP districts and from 61 to 68 percent in non-DPEP districts.

(iii) The rate of change in the mean outcome levels in DPEP districts was higher than that in non-DPEP districts overall. For example, change in dropout rate among 5-11 year old 23

In an earlier draft, we had separated out the two minority groups and we did find differences between them. However, the results were not robust due to small sample problems for a number of impacts reported in the paper. So in this version, we have grouped all minorities into a single category.

24 In Kerala even the treatment districts have female literacy rates that are far greater than 40 percent. So for Kerala, we include all non-DPEP districts in the set of potential controls.

32

17

33

in DPEP districts was 2 percentage point as compared to 1 percentage points in non-DPEP districts; that among 12-14 year old in DPEP districts was 7 percentage point as compared to 1 percentage points in non-DPEP districts.

(iv) The rate of change in the mean outcome levels in potential control non-DPEP

districts was higher than that in all non-DPEP districts and was similar to the rate of change

in DPEP districts. For example, enrollment rates of all 5-11 year old children increased from

85 to 86 percent in non-DPEP districts, and from about 70 to about 75 percent in both DPEP

and potential control non-DPEP districts.

(v) Notwithstanding improvements in mean outcomes over time in DPEP districts and at a rate faster than that observed in non-DPEP districts, the levels of nearly all outcome

indicators in 1999-2000, failed to exceed the 1993-94 levels for the non-DPEP districts.

Even though the rate of change in the proportion of 12-15 year olds reporting completed primary school levels was nearly three times higher in DPEP districts than in non-DPEP

districts, the mean percent of kids in DPEP districts in 1999/2000 reporting completed levels

of primary school education was 68 percent; this is considerably lower than the average of 80

percent recorded for non-DPEP districts in 1993-94. The above observations demonstrate substantial improvements in education

indicators in DPEP districts. At the same time, they also show improvements in the non- DPEP districts. The issue then is to determine the marginal contribution of DPEP-I to

primary school education, i.e., its net impact. To do this, we need to construct the “correct”

counterfactual comparison group for the DPEP districts. This is what we do in the next sub- section.

5.2 Impact estimators of the DPEP We used a two stage matching procedure to estimate the impacts of the DPEP. First, we

matched districts within states and then matched households within the matched districts.

Among the set of observed characteristics included in the district propensity score model, the only statistically significant variables were the state dummy variables and female literacy

rate.25,26 We therefore, matched project with non-project districts only on the basis of

25 The “balancing property” test of the propensity score model is satisfied. 26 The other variables included in our specification are: district population density, proportion of houses which are permanent and semi-permanent in district, proportion of households in district with power, and

34

18

35

observed female literacy rates within a particular state. However for robustness reasons, we also matched districts using the predicted propensity scores from the estimated model. We

found that the matched control districts picked by the two alternative methods for the treatment districts were almost always identic al. 27,28 We matched households in treatment

districts with households in matched control districts in the second stage.29

We report the net impacts on the different aspects of primary school education (access, completion levels, dropouts and progression rates) for various population sub-groups

(females, males, minority group etc.). The underlying stochastic process for the different outcome indicators defined over each sub-population is different. Therefore, for the impact

reported for each outcome indicator, we estimated a separate propensity score model to find

similar households in treatment and non-treatment districts. Details on the individual logit models are available from the authors.

The impacts are reported for two sub-samples: full sample (includes households from 40 districts spread over the seven states), and separately for those households residing in Madhya Pradesh (Tables 4-7). We do this because in Madhya Pradesh, two substantive state

level interventions called the Alternative Schooling (AS) and the Education Guarantee System (EGS) were started in 1995 and 1997 respectively. The AS started with 418 centers

across the 19 DPEP-I districts, which by the year 2000, had expanded to 2,970 centers in the

same districts (Jha, 2000). The EGS on the other hand has more than 16,000 centers in entire state (Gopalakrishnan and Sharma (1998)).30

access to primary school, proportion of immigrants in district, scheduled caste/tribe concentration in district, and female literacy rate among scheduled caste/tribe.

27 We restrict the matching of districts to within states only to control for state specific rules and regulations in individual states. For example, in the state of Assam there is automatic promotion until primary school. This is not true in the other states. 28 In the paper we do not report the logit model at the district level. However this is available from the authors on request. 29 We also matched treatment households with households from non-treatment districts from the same state having a female literacy rate lower than 40 percent in 1991. The impact estimates are similar to those estimated from the two stage matching.

30 AS and EGS are two large-scale decentralized, non-formal primary school education systems sponsored by the government of Madhya Pradesh. Both these schemes cost less to the state government than setting up a formal primary school. The objectives of both these schemes were to provide primary school facilities to the most needy sections of society and those that cannot join the formal school system (for example,

36

19

37

The data at hand (or even independent sources) do not help us distinguish enrollments in a DPEP school from enrollments in either of these schemes. In many cases, these alternative educational interventions have worked in a complementary manner and often

under the umbrella of the same administrative body viz. Rajiv Gandhi Shiksha Mission.31 Given this, the best we can do is present the impacts for Madhya Pradesh separately and see

whether the pattern in MP is different from the other states.

Impacts on enrollment rates.

Table 4 reports the impacts on school and primary school enrollments respectively among 5-11, 12-15 year olds for the overall population, for

males, females and minorities (scheduled caste and tribes grouped together). The impact on

enrollment rates is defined as: (PT – Pc)1999-00 - (PT – Pc)1993-94 where Pi (i=Treatment, Control) is the proportion of specific age group enrolled in school at the time of the survey.

In the aggregate, DPEP-I has had no effect on enrollment rates on all children, on girls or on boys, belonging to either of the age-categories. A goal of the DPEP has been to improve primary school access for disadvantaged groups like scheduled castes and tribes. In

this regard, the program has been successful in substantially increasing primary school enrollment rates among 5-11 year olds belonging to the minority group. Compared to a non-

DPEP district, in a DPEP district, 4.5 percent of minority kids in the age-group 5-11 years

were more likely to enroll in primary school. In the state of Madhya Pradesh the impact of the program in raising school enrollments among minorities is even stronger. However our

estimates do not show any impact on the school enrollments of female children. So even

though reducing gender disparity in enrollment rates was an avowed aim of the program, in

reality there was no impact of the program. This is true even for the state of Madhya Pradesh.

migrant children, children living in remote areas, those involved in labor etc.). Both these schemes emphasized on the importance of local community participation. For example, under both schemes, the local community body (Panchayat) is responsible for recommending the names of teachers, and the village education committee (or administrative bodies with similar functional responsibilities) is the managing authority with various responsibilities. AS has now been merged into the EGS schools. 31 For example, the pedagogical guidelines advocated by the DPEP are best represented in the Alternative Schools.

38

20

39

Impacts on dropout rates and cohort progression from primary to middle (or higher) level. In Tables 5 and 6 we report impact estimates on dropout rates and on the proportion of 5-9 year olds enrolled in primary school in 1993-94 and progressed to middle school (or higher) by 1999-00. The impact on cohort progression is defined as:

T C T C

(M - M ) (P - P where M is the proportion of i=(11-15) year olds enrolled i i 1999 00 j j )1993 94

in middle school or higher in 1999-00 and P is the proportion of j=(5-9) year olds enrolled in

primary school in 1993-94. The impact on dropout rates is defined in the same way as the

impact on enrollment rates.

While DPEP may not have been very successful in encouraging non-minority new

students to the schooling system, enrollments did increase as a result of the DPEP for minority students especially in the state of MP. Our estimates show that the DPEP did have some impact in retaining already enrolled students particularly in the older age (12-15 years)

category. For the minority group, impact was stronger for the younger age-group (5-11 years). For the older age group, minority children in DPEP districts seemed to have fared

worse than in non-DPEP districts. This pattern is similar for the state of Madhya Pradesh

too. However, here we see reduced dropout rates among girls in the older age-category and

reduced dropout rates among boys in the younger age-category (5-11 years).

Impact estimates for cohort progression supports our results for dropout rates showing that there is a net increase in the proportion of 5-9 year old cohort of boys and minorities who progress to middle school by 1999-2000 in the program districts. This is

observed among all sub-populations with the exception of female children. In line with the

patterns observed for the other two indicators, the net impacts are stronger for treatment

households in Madhya Pradesh.

Impacts on completed levels of education. In Table 7, we report impacts on highest level of completed education among 12-15 year olds to assess the impact of the program on

the stock of completed levels of education. We chose 12-15 years as the relevant age-

category, to capture students who have completed their primary school education in stipulated years and to allow for some repetition and late starters. The impact on completed

T C T C

education levels is defined as: (Pi- P i

(Pi- Pi) )199900

21

40

1993 94 where Pij

(j=Treatment,

41

Control) is the proportion of 12-15 year olds with (i=no education, completed primary) levels of education.

For this outcome indicator, there are some impacts of the DPEP. Fewer children and

especially male children in the age-category 12-15 years report as having gotten no education in the DPEP districts. However our estimates suggest that proportion of children with

“completed primary school” education in the age-category 12-15 years residing in DPEP

districts did not differ significantly from those in non-DPEP districts. Once again, there were

no impacts on the female sub-population. In the state of Madhya Pradesh, a similar pattern is

noticed with one difference. There are some impacts on completion rates among all children

and among girls in particular. Girls in the age-group 12-15 years residing in DPEP districts

in MP are 12.48 percent more likely to have completed their primary school education than girls residing in non-DPEP districts.

Unlike enrollments and dropout rates, our estimates indicate that the DPEP has had no impact on the stock of education (as measured by our two indicators) among 12-15 year olds belonging to the minority community. Moreover, minority children living in DPEP

districts continue to have lower primary school completion rates when compared to those living in non-DPEP districts.

In summary, our net impact estimates of DPEP-I indicates the following results. First, the average impact on enrollments is the largest for children belonging to the minority

group and a substantial part of the positive impact of DPEP-I emanates from the

improvements recorded within the state of Madhya Pradesh. As mentioned before, Madhya

Pradesh had other education programs like the EGS and AS running concurrently with DPEP-I, and it is not possible to disentangle the effects of the different programs. Second,

our estimates show negligible impacts on enrollments of non-minorities, but some impacts on dropouts and educational attainment suggesting that the program may have been instrumental

in improving the quality of learning of the already enrolled rather than attract new enrollees. For minorities, however there is not much impact on the already enrolled students. Finally,

our most surprising result is a near absence of program impact on female children, whether

the outcome indicator is enrollments, or completion rates, or cohort progression rates even though the program’s purported objective was to improve education outcomes of females.

42

22

43

6

Interpretation of the impact estimates32

How should we interpret the magnitude of the impacts reported in the previous section? In a

simple model where families do a cost-benefit analysis over one’s lifetime to determine the optimal amount of schooling, one can show that the optimal amount of schooling is given

by:33

= s T

1

ln

(r g ) � ∝(r g)

Here s refers to years of schooling, T refers to age of retirement, r is the interest rate, is the returns to education (which is a function of demand for skilled workers and school quality), g

is the growth rate of total factor productivity, and ∝ is the ratio of private direct costs of schooling to the opportunity cost of student time.

This equation summarizes the supply and dema nd factors that potentially affect schooling outcomes. For example, poor countries have low enrollment rates because

effective discount rates are high in poor countries (e.g., due to subsistence consumption constraints or underdeveloped financial markets). This is represented by an increase in r.

Another explanation could be that “school quality” is low in poor countries.34 This can be

32 This section is based on detailed review comments provided by Chang-Tai Hsieh. We thank him for his inputs in making this a better paper but remaining errors are ours.

33Maximizing

T ⟩t

+ e u c t dtsubject to[ ( )] o

+

T

s

s

e

( g r t)

dt

ε+

T

0

rT

e c t dt

( )

+

+

s

0

e(

g )

∝

dt

. The

first expression on the left hand side of the budget constraint is lifetime income, which increases with schooling attainment. Turning to the right hand side of the budget constraint, the first term is lifetime consumption. In turn, since ∝ refers to the ratio of private direct cost of schooling to the opportunity cost of schooling time, the second expression is the sum of the opportunity cost and the direct cost of schooling. See Card (2000) and Bils and Klenow (2001) for papers that use this basic approach to discuss the determinants of educational outcomes.

34 There is also abundant anecdotal evidence of low school quality in poor countries, but the empirical

estimates (e.g., those compiled by Psacharopoulos, 1994), suggest that if anything, returns to schooling

44

seem to be higher in poor countries. 23

45

represented by a fall in .35

An alternative, supply -side story, is the absence of schools within reasonable commuting distance in regions where poor people live. This can be

represented as a high ∝.

So how would a program like the DPEP enter this equation? It could increase the

school quality, which can be viewed as an increase in or it lowers the cost of schooling,

which we can represent as a fall in ∝.36

DPEP I provided US$310.5 million over seven years, the total number of students in the DPEP I districts in 1997-1998 was 17.720 million, which is US$9.1 per student-year. The question is what effect did this additional $9 per student have on school outcome

indicators?

Suppose that we take $20-$50 as reasonable estimates of the average annual cost of educating a primary school student for one year.37Therefore, a subsidy of $9.1 per year

would lower the private cost of schooling (assuming that the additional $9 available to the government is translated into a $9 reduction in private costs) by 20 to 40 percent. Under the

assumption that initially, ∝ is roughly 0.75, a 40 percent reduction in schooling costs lowers

∝ to 0.45. We also fix the other parameter values as T=60, r=0.05, and g=0.025.

If we assume that the DPEP funds represented a 20 percent reduction in schooling costs, this would increase years of schooling from 0.8 (?=.2) to 1.7 (?=.1) years. Under the assumption that

the DPEP represented a 40 percent fall, this would imply an increase in educational

attainment of 1.6 (?=.2) to 3.4 (?=.1) years. Translating this into enrollment rates, an

increase in average years of schooling of one year is equivalent to a 16-17 percentage point

35 This equation suggests that an additional reason for low schooling attainment in poor countries is low life expectancy (T is lower). 36 One can think about school construction as a fall in ∝. If there is no school in a village, a household living in the village has the choice of taking a child to another village (or relocating to another village) that has a school. However, this is presumably costly. By constructing a school in a village, the cost to households wishing to send their children to school is lowered.

37 This estimate is based on the following calculation. In 1995/96, total expenditures on primary schooling in India (central, state, and household spending) amounted to Rs 245 billion and total primary enrollment was 151.5 million, which amounts to Rs 1,612 per student-year, or roughly $40 per student-year.

46

24

47

increase in the primary school enrollment rate (assuming six years for primary school).38 Assuming that the program was well implemented, the investment made by DPEP should have translated into quite large improvements in enrollments.

So how do our estimates measure up to the predictions of the simple model? The model in this section predicts that enrollments should have gone up by 16-17 percent to

declare the intervention a success. Our impact estimates from the previous section suggest

that even though enrollment rates for prime-age group (5-11 years) among the minorities

increased due to the program, it is only in the state of Madhya Pradesh that we observe an increase in enrollments that is commensurate with the predictions of the model.39For the

other sub-populations, the program did not lead to an increase in enrollment rates.

So should one conclude that the DPEP is a failure? The calibration exercises assume

that the DPEP represented a net increase in school spending in each district. Although an explicit rule was that states benefiting from DPEP funds could not reduce their own

expenditures on elementary education, there was no such rule for within state (across districts) allocation.40 It could well be the case that the effective increase in school spending

in the DPEP districts was much lower than what the total amount disbursed by the DPEP would lead one to conclude.41

However, one doubts whether this actually happened. State governments’ primary school expenditures fund primarily teachers’ salaries rather than fund investments in

improving the quality of primary education. DPEP is quite explicit in its guiding principles

that the funds are to be used for non-salary primary school investments with management

cost (which includes project staff) limited to 6 percent of total project cost. So at least the

38

The increase in average years of schooling is the product of the increase in enrollment rates and the

number of years of schooling represented by that level of schooling.

39 This is if we do not add the additional resources available due to the other state-sponsored non-formal primary school intervention.

40 Bashir and Ayyar(2001) provides some evidence that program states did maintain their pre-DPEP expenditures on primary school education.

41 Of course it is also feasible that individual state governments withdrew existing resources from the non- DPEP districts and invested it in the DPEP districts which would mean that even a larger impact of the program should have been observed.

48

25

49

state government expenditures could not have been re-allocated away from the DPEP districts. Insofar as the central government funds were concerned, at least during the period under study there was no major primary school initiatives undertaken other than the DPEP with the exception of the mid-day meals. Again the DPEP guidelines (at least Phase 1 of the

project) did not allow supporting school-feeding programs.

Second, one cannot assume that the money was used to finance educational investments to attract new students. Rarely do we find convincing evidence that school

inputs leads to improvements in educational outcomes partly because many inputs may improve the student’s welfare, but may not necessarily translate into improvements in

educational outcomes. That is, it could well be the case that the funds were used for

expenditures that benefited students who were already enrolled rather than attract new students. Our results do suggest statistically significant impacts on indicators like drop out

rates, cohort progression and educational attainments – all variables pertaining to already enrolled students.

Finally, since DPEP brought about a revolutionary change in the way primary school

interventions were previously thought of, there could have been a “district collector effect”. That is, a district collector previously posted in a DPEP district and then subsequently

moving to a non-DPEP district could carry and implement his or her acquired DPEP human

capital at the time of transfer. That is implement DPEP type of reforms in non-DPEP districts using a lot of the investments (like textbooks, teacher training program etc.) that

DPEP had already made. This would lead to a diffusion of impacts of the DPEP program.

While this is a possibility, we are less concerned about this being an issue because the time

frame that we consider is one primary school cycle (6 years) and we think this is too short for the “district collector effect” to be important.

7

Conclusions DPEP represented a major primary school intervention by the central government (with some matching funds by the state government). DPEP funds represented approximately 2 percent

of total (center and state) primary school funding in India. Among the project states, it

increased the average allocation on primary school education by 17-20 percent. Besides the

additional resources available to the project districts and states, DPEP revolutionized the way

50

26

51

primary school interventions were previously designed. More importance was given to improving the quality of schools rather than on simply expanding the primary school system.

In this paper we have evaluated the DPEP-I education project using propensity score

matching methods. The unexpected element in our program impact estimates is that the impacts are not as substantial as warranted by the massive coverage of the program or as

claimed in the existing DPEP evaluation literature. Net program impacts on minority

children are the most impressive but here too, the net improvements in enrollment rates are

only 4.6 percentage points over the six years. It is in the state of Madhya Pradesh and especially for the minority group that we observe significant impacts and often large impacts

of the program. For example, the impacts on enrollments for the age-group 5-11 years of the

minorities is very close to what our simple model predicts for the program to be declared successful. It is only in the educational attainment indicator that the minorities as a group

does not do too well.

Inline with the US evidence provided by Hanushek and others, our analysis convinc ingly shows that the direct impacts of the program especially on school enrollments

were negligible. So where does it leave us?

Even though the DPEP in its program design ushered in a revolution in the way primary school programs were perceived in India, our analysis finds small impacts of the program. These impacts are certainly not congruent with the scale of the program. Other

than the above mentioned biases, a possible explanation could be that the limitations of the

primary school system are so severe that marginal improvements in the existing system will

not get the desired results. Perhaps what is required is a complete overhaul of the existing system like restructuring the incentive system for primary school teachers. Pritchett (2005)

discusses one such way of overhauling the system.

Finally an important conclusion is that interventions must be designed in a way and accompanied by the collection of data that would make evaluation possible. Otherwise, it is

simply very difficult to know whether the intervention had the desired results, and what

aspects of the policy worked well and might be potentially replicable, and what parts of the program worked less well.

52

27

53

References Aggarwal, Y. (2000a) “Monitoring and Evaluation Under DPEP: Measurement of Social

Impact”, NIEPA, New Delhi Aggarwal, Y. (2000b) “An Assessment of Trends in Access and Retention” National

Institute of Educational Planning and Administration, New Delhi Bashir, S. and U. Ayyar (2001) “District Primary Education Programme, Encyclopedia of I

Indian Education, New Delhi: National Council of Education Research and Training

Birdsall, N. (1985) “Public Inputs and Child schooling in Brazil” Journal of Development

Economics, 18(1), 67-83 Bils, Mark and Peter Klenow (1998) “Does Schooling Cause Growth?” Working Paper

6393, National Bureau of Economic Research, Cambridge: Massachusetts

Card, David (2001) “Education” in Handbook of Labor Economics, Volume III, edited by David Card and Orley Ashenfelter, North Holland

Card, D. and Alan B. Krueger (1992) “Does School Quality Matter? Returns to Education and the Characteristics of Public Schools in the United States” Journal of Political

Economy, 100:1, 1-40 Case, A. and A. Deaton (1996) “School Quality and Educational Outcomes in South Africa”

mimeo, RPDS, Princeton University Chin, A. (2002) “The Returns to School Quality When School Quality is Very Low:

Evidence from Operation Blackboard in India”, mimeo, University of Houston

Coleman, James S., E.Q. Campbell, C.J. Hobson, J. McPartland, A.M. Mood, F. Weinfeld, and Robert L. York, Equality of Educational Opportunity, Washington, DC: US

Government Printing Office

Duflo, Esther (2001) “Schooling and Labor Market Consequences of School Construction in

Indonesia: Evidence from an Unusual Policy Experiment” American Economic

Review, 91

Dehejia, R. H., and S. Wahba (1998), “Propensity Score Matching Methods for Non- Experimental Causal Studies”, NBER Working Paper 6829, Cambridge, Mass

Dehejia, R. H., and S. Wahba (1999), “Causal Effects in Non-Experimental Studies: Re- Evaluating the Evaluation of Training Programs”, Journal of the American Statistical

Association, 94, 1053-1062

Gopalakrishnan, R. and A. Sharma (1998), “Education Guarantee Scheme in Madhya Pradesh”, Economic and Political Weekly, 277-282

54

28

55

Govinda, R. (2002) India Education Report, National Institute of Educational Planning and Administration, New Delhi, India

Government of India (1991) Census of India, 1991 (1995) DPEP Guidelines, NCERT, New Delhi

Government of Madhya Pradesh (2000) From Your School to Our School, Rajiv Gandhi Shiksha Mission, Government of Madhya Pradesh, Madhya Pradesh

Hanushek, E. (1986) “The Economics of Schooling: Production and Efficiency in Public

Schools” Journal of Economic Literature, 24, 1141-1177

Heckman, J., H. Ichimura, and P. Todd (1997), “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme”, Review of

Economic Studies, 64, 605-654

Heckman, J., H. Ichimura, J. Smith, and P. Todd (1998), “Characterizing Selection Bias using Experimental Data”, Econometrica, 66, 1017-1099

Heckman, J., H. Ichimura, J. Smith, and P. Todd (1996), “Nonparametric characterization of selection bias using experimental data: A study of adult males in JTPA. Part II,

Theory and Methods and Monte-Carlo Evidence,” Mimeo, University of Chicago Imbens, G. and J. Angrist (1994), “Identification and estimation of local average treatment

effects”, Econometrica, 62, 467-476 Jalan, J. and M. Ravallion (2003), “Estimating the Benefit Incidence of an Antipoverty

Program by Propensity Score Matching”, Journal of Business and Economic

Statistics, 1, 19-30 Jalan, J. and M. Ravallion (2003), “Does piped water improve child health for poor

families in rural India?” (with M. Ravallion) Journal of Econometrics,12(1), 153-

173

Jha, J. (2000), “Education Guarantee Scheme and Alternative Schooling: Community- based initiatives in Primary Education in Madhya Pradesh” in From Your School

to Our School, Government of Madhya Pradesh, Madhya Pradesh Lazear, Edward (2001) “Educational Production” Quarterly Journal of Economics, 3, 777-

804 Menon, P. (2001), “Content Analysis of training Modules for Village Education

Committees: A Study of seven DPEP states (Phase I)”, NIEPA, New Delhi

NCERT (1993), Sixth All India Education Survey, NCERT, New Delhi

56

29

57

Pandey, R. S. (2000) Going to Scale with Education Reform: India's district Primary Education Program, 1995-99, Washington DC: The World Bank

Pritchett, Lant (2005), “Ed ucation and the PRI Structure in India”, mimeo, The World

Bank, New Delhi, India

Psacharopoulos, George (1994), “Returns to Investment in Education: A Global Update”

World Development, 22(9), 1325-1343 Rosenbaum, P. and D. Rubin (1983), “The Central Role of the Propensity Score in

Observational Studies for Causal Effects,” Biometrika, 70, 41-55 Rosenbaum, P. and D. Rubin (1985), “Constructing a Control Group using Multivariate

Matched Sampling Methods that Incorporate the Propensity Score,” American

Statistician, 39, 35-39 Rosenzweig, Mark (1995) “Why are there returns to schooling?” American Economic

Review, 85, 153-158 Rubin, D. (1974), “Estimating causal effects of treatments in randomized and non-

randomized studies”, Journal of Educational Psychology, 66, 688-701

Rubin, D., and N. Thomas (1996), “Matching Using Estimated Propensity Scores: Relating

Theory to Practice,” Biometrics, 52, 249-264 Sharma, A. and R. Gopalakrishnan (1997) “Bringing the people back in: From Lok Sampark

Abhiyan to Education Guarantee System in Madhya Pradesh”, Government of Madhay Pradesh, Bhopal, Madhya Pradesh

Smith, J. and P. Todd (2001), "Reconciling conflicting evidence on the performance of

propensity score matching methods" American Economic Review , 91(2), 112-118

Wooldridge, J. (2002) Econometrics of cross-section and panel data , Cambridge and MIT

University Press, Cambridge, Massachusetts World Bank (2002) DPEP: 15thJoint Review Mission Progress Report, New Delhi: The

World Bank

(2001) DPEP: 14thJoint Review Mission Progress Report, New Delhi: The World Bank

(2001) DPEP: 13thJoint Review Mission Progress Report, New Delhi: The World Bank

(2000) DPEP: 12thJoint Review Mission Progress Report, New Delhi: The

World Bank

58

30

59

(1997) Primary Education in India, Washington DC: The World Bank

(1994) India: District Primary Education Project, Staff Appraisal Report,

Washington, D C: The World Bank

(1993) India: Uttar Pradesh Basic Education Project, Staff Appraisal Report, Washington, D C: The World Bank

60

31

61

Appendix42

We use the “counterfactual framework” proposed by Rubin (1974) and subsequently used by both statisticians and econometricians (Rosenbaum and Rubin (1983), Imbens and Angrist

(1994), Heckman, Ichimura and Todd (1997) among others) to estimate the average treatment effects.

Let y1

denote the outcome with treatment and y0 outcome without treatment. Recognize that a unit cannot simultaneously be in both states. So, we cannot observe both y1

and y0 at the same time for the same unit. The econometrics problem that we have at hand is that of “missing data”.

Let t be a binary indicator, where t=1 indicates participation in the program and t=0

otherwise. (y1, y0, t) represents a random vector from the population of interest. For a random draw i from the population, the relevant vector is (y1i, y0i, ti). The implicit

assumption that we make is that treatment of unit i affects only the outcome of unit i and does

not affect any other unit’s outcome. Moreover, (y1, y0) could be correlate d with t.

To measure the impact of the program, we are interested in the difference of outcomes with and without treatment. Several possible estimators are possible. We use the

standard estimator of the average treatment on the treated (ATE) defined as: ATE = E(y1 – y0|

t=1). That is, the mean effect of the program on the participants. Furthermore, if x is a

vector of observed covariates, ATE can be redefined as: ATE = E(y1 – y0| t=1, x ).

Right at the beginning of the discussion we had posed the econometric problem underlying the estimation of program impacts as that of “missing data”. That is, for each

treated (non-treated), at any point in time, we observe only y1 (y0). The observed outcome is: y = y0 + t (y1 - y0). Question therefore remains as to what do we do about the “missing data”

problem?

42 This section draws heavily from Smith and Todd (2000) and Wooldridge(2002).

62

32

63

There are different approaches for imputing the counterfactuals. Some of the non- experimental estimators are as follows. The cross-section mean difference indicators use data on t=0 as a proxy for missing data. In this case, the impact estimator is:

cs = E(y1 | t=1) - E(y0 | t=0) + [E(y0 | t=1) - E(y0 | t=0)]

[Evaluation Bias] where the term in the parentheses is the evaluation bias. The before-after estimators treat pre-

program data on participants to proxy for post-program outcomes in the no-treatment state.

The impact estimator in this case is

b1 = E(y1 | t=1) - E(y0 | t=1) + [E(y0 | t=1) - E( y 0'|t=1)]

[Evaluation Bias]

where y0' is the missing data. The term in parentheses is the evaluation bias. Advantage of

the before-after estimator over the cross-sectional estimator is that it allows for unobserved fixed effects which difference out. However, the estimator is sensitive to time effects and to

Ashenfelter’s Dip problem. Finally, one can also compute the difference-in-difference estimator that takes differences over before-after outcomes for participants and non- participants. The impact estimator in this case is

b2 = E(y1 | t=1) - E(y0 | t=1) + [E(y0 - y0'|t=1) - E(y0 - y 0'|t=0)]

[Evaluation Bias] This estimator takes care of the sensitivity to time effects problem in the before-after

estimator.

Matching estimators pair each program participant with an observably similar non-

participant and interpret the difference in their outcomes as the effect of the program.

Matching estimators require that conditional on a vector of observed characteristics, x , (y1,

y0) are independent of t. This is the “ignorability” condition introduced by Rosenbaum and

Rubin (1983).

Matching may be difficult to implement when the set of conditioning variables x is

large. Rosenbaum and Rubin (1983) define what they term as strong “ignorability”

conditions that help in reducing the dimensionality problem of the conditioning variables while implementing the matching method.

64

33

65

Let p( x ) = Prob(t=1| x ) be the probability of treatment given the covariates x . p( x )

is the propensity score function in the evaluation literature. Given p( x ), and under the

assumptions (i) E(y1 – y0| t=1, x ) = E(y1 – y0| x ) (ii) 0< p( x ) <1 for all x , Rosenbaum and

Rubin (1983) show that E(y1 – y0| t=1, p( x )) = E(y1 – y0| p( x )). This implies that when the outcomes are independent of participation conditional on x , they are also independent of participation conditional on the propensity score function.

What does the condition 0< p( x ) <1 for all x imply? Basically it imposes

restrictions on the probability function such that ATE is identified. While it is still possible

that ATE is identified whenp( x )=0 for some x , from a practical point of view we want to

exclude units that have zero probability almost surely of being treated. We also exclude units

with p( x )=1. We cannot estimate the ATE by including population units that are treated

with certainty, conditional on x .

If the above two assumptions are satisfied, then the y0 distribution observed for the matched non-participant group can be substituted for the missing y0 distribution for

participants. The mean impact of the program can thus be written as:

= E(y1 - y0|t=1) = E(y1 | t=1) – E x |t=1{Ey(y| t=1, x )} = E(y1 | t=1) – E

x |t=1{Ey(y| t =0, x )}

where the second term is estimated from the mean outcomes of the matched comparison group.

For non-experimental data, we may not find a set of observed conditioning variables for which the conditions hold. This is the problem of no common support region. If there are

regions where the support of x does not overlap for the treated and non-treated, then there

may be a fraction of program participants for whom no match could be found in the data.

According to various studies by Heckman, Ichimura and Todd (1997, 1998), matching on the

no common support region is the primary cause of a bias in a matching estimator.

66

34

67

Let P = Pr(t=1| x ) be the estimated propensity score. Then the matching estimator is

defined as

1 m = [ | = 1, ]

y Eˆ ( y t P

Eˆ ( |

=

= n1

1 i ) I Sp

i1 0i i

y t P W j y where i 1, ) (i, ) . I1 denotes the set of program participants, I0 set of

0 i j I0

0 j

non-program participants, Sp the region of common support, n1 the number of persons in the intersection of the region of common support and the set of program participants. The match

for each participant is constructed as a weighted average over the outcomes of the non- participants, where the weights depend on the distance between Pi and Pj.

Let C(Pi) be a neighborhood for each i in the participant sample. Neighbors for i are

non-participants jI0 for PjC(Pi). The persons matched to person i are those in set Ai where

Ai = {jI0|PjC(Pi)}. Alternative matching estimators differ on the basis of the construction of the weights W(.,.) and how the neighborhood is defined.43

Some of the commonly defined matching estimators are the nearest neighbor

matching estimator where C(Pi) = minj || Pi - Pj||, jI0. That is, the non-participant with the

value of Pjthat is closest to Piis selected as the match. In this case the W(.,.) function is

unity.44 Similarly, there are the caliper matching estimators, where a match is declared only if

the C(Pi) = minj || Pi - Pj||, jI0 is restricted to be within a certain bound. Kernel and local linear matching estimators uses a kernel weighted average over multiple persons in the

comparison group.

The estimators described above assume that after conditioning on a set of observed

covariates x , the mean outcomes are conditionally mean independent of participation in the

program. However, this may not necessarily hold which could lead to the violation of ignorability conditions required for matching. In case of the DPEP, in the first phase of the

43 Jalan and Ravallion (2000b) discuss the choice further, and find that their results for estimating income

68

gains from an anti-poverty program are reasonably robust to the choice. 44 Rubin and Thomas (2000) use simulations to compare the bias in using the nearest five neighbors to just the nearest neighbor; no clear pattern emerges.

35

69

project, program administrators identified project districts on the basis of low female literacy rates but also took the potential capability of the district to absorb the project while declaring

it a project site. The latter criterion is very difficult to observe and if ignored, the correlation

between program participation and the outcomes would still persist even if conditioned on

female literacy rates and other observed district characteristics. In these cases, a difference- in-difference matching estimator (DID) allows for temporally invariant differences in

outcomes between participants and non-participants. The DID estimator requires that E(y0t - y0t’|P, t=1) = E(yt - yt’|P, t=0), where t and t’ are time periods before and after the program

enrollment date. This estimator also requires the support condition to hold in both periods. The impact of the project is thus defined as the difference between outcomes in the project

areas after the program and before it, minus the corresponding outcome difference in the

matched comparison areas.

If repeated cross-section data are available instead of information on the same households, the estimator can be implemented as:

DID

= 1

[ Y

1 W (i, j Y

0

) ]

1

1 W (i, j Y

0

) ]

n 1t

i ) I S

1t p

it ) j I S

0t p

jt n 1t'

i ) I 2 S

1t p

[Yit2 j ) I02 S

t p

jt2

I1t,I1t’

,I0t,I0t’ denote the treatment and comparison group datasets in each time period. In our

analysis, we use the matching estimator defined for the repeated cross-section estimator.

The success of any matching estimator depends on the availability of observable data to construct the conditioning set such that the “ignorability” conditions are satisfied.

Furthermore, choice of variables to estimate the propensity score is also an integral component of the matching procedure. While theory does not suggest ways in which to

choose the x vector, the balancing property proposed by Rosenbaum and Rubin (1983) help

determine interactions and higher-order terms to be included for a given set of x such that the

conditions are met. The intuition behind this proposition is that after conditioning on

Prob(t=1| x ), additional conditioning on x , should not provide any new information about t.

The test therefore is whether or not there are differences in x between t=1 and t=0 groups.

In this paper, we follow Dehejia and Wahba (1998, 1999) and divide the observations

70

into strata based on the estimated propensity scores. These strata are chosen so that there is 36

71

no statistical difference in the mean of the estimated propensity scores between the treated and the untreated observations within each stratum. From a practical point of view, we start with a very coarse strata and make it finer until the above condition is satisfied. Once the

strata are fixed, t-tests are used to detect mean differences in each x variable between the

treated and non-treated groups. When significant differences are found for particular

variables, higher order and interaction terms are added to the logistic model and the testing procedure repeated until no further differences emerge.

72

37

73

ASSAM (Districts in 1991: 23)

1. Dhubri (23.0) 4. Darrang (30.0) 5. Karbi Anglong (30.0) 11. Marigaon (37.0) KERALA (Districts in 1991: 14) 2. Kasaragod (75.31) 3. Wayanad (77.64) 5. Malappuram (83.91)

Table 1: DPEP-I Districts

HARYANA (Districts in 1991: 16)

2. Kaithal (23.68) 3. Jind (24.35) 4. Hisar (24.38) 6. Sirsa (27.43) MADHYA PRADESH (Districts in 1991: 47) 3. Rajgarh (9.46) 4. Guna (10.12) 5. Sidhi (11.40) 7. Surguja (12.50) 8. Shahdol (12.85) 11. Ratlam (13.94) 12. Chattarpur (14.12) 13. Panna (14.85) 15. Sehore (15.07) 17. Tikamgarh (15.39) 18. Dhar (15.64) 25. Mandsaur (19.88) 26. Raisen (20.45) 27. Bilaspur (20.92) 29. Satna (22.19) 30. Rajnandgaon (22.24) 32. Rewa (22.81) 33. Raigarh (23.48) 40. Betul (26.71)

KARNATAKA (Districts in 1991: 20)

2. Raichur (16.48) 6. Kolar (29.06) 7. Belgaum (31.07) 8. Mandya (32.12)

MAHARASHTRA (Districts in 1991: 30) 2. Parbhani(22.80) 3. Nanded (24.30) 6. Aurangabad (28.39) 8. Latur (35.35) 10. Osmanabad (35.80)

TAMIL NADU (Districts in 1991: 22)

1. Dharmapuri (31.86) 2. South Arcot (34.29) 5. TSambuvarayar(35.9)

Note: Number preceding the name of the district indicates the rank of the district in the state with respect to its female literacy rate. A smaller number indicates lower literacy rate. Numbers in parentheses are the observed female literacy rates in 1991 from census data

74

38

75

Table 2: Descriptive statistics on education indicators in DPEP and non-DPEP districts

Indicator Female literacy rates

Male literacy rates

Female literacy rates for scheduled castes

Male literacy rates for scheduled castes

Female literacy rates for scheduled tribes

Male literacy rates for scheduled tribes Source: Census 1991

DPEP – I 0.2648 (0.157) 0.5566 (0.124) 0.2100 (0.149) 0.4878 (0.135) 0.1589 (0.131) 0.3776 (0.199)

Non-DPEP - I* 0.3923 (0.181) 0.6669 (0.116) 0.3449 (0.195) 0.6097 (0.143) 0.2768 (0.198) 0.4841 (0.228)

*This category excludes the expansion districts under DPEP-II in the DPEP-I states

76

39

77

Sample Size

Table 3: Descriptive statistics of the NSS sample

50th Round (1993-94) 55th Round (1999-00) DPEP Non-DPEP* DPEP Non-DPEP*

States: Assam, Haryana, Karnataka, Kerala, Madhya Pradesh, Maharashtra, Tamil Nadu

Number of districts Number of households Demographic Structure Household size Proportion of children in household:

5-11 years 12-15 years

Proportion of girls: 5-11 years 12-15 years

Social Status Percent of scheduled tribes Percent of scheduled castes Employment status

41** 5,986

4.92

15.51 8.36 8.32 3.47

16.7 18.0

87 12,576

4.65

14.60 8.15 7.94 3.42

9.0 16.8

42 5,438

5.1

15.75 8.86 8.45 3.96

18.8 19.6

87 13,027

4.6

14.90 8.42 7.77 3.81

10.6 19.4

Percent of self-employed in non-agriculture 10.4 11.6 9.3 12.3 Percent of agricultural labor Percent of other (non-agricultural) labor

Source: Estimates based on NSS data

36.6 5.9

35.9 11.5

37.1 6.5

36.5 12.1

* :

**:

excludes districts which were declared as project districts under DPEP-II Kaithal district in Harayana was excluded from the samp le because the data does not allow us to distinguish it from Kurukshetra, a non-project district

78

4 0

79

80

100%

90%

80%

70%

60%

50%

1993

1999

All Children

Figure 1: Enrollment Rate among 5-11 year old by DPEP status (percent of all children in 5-11 group)

1993 1999 1993 1999

Male Female

1993

Minority

1999

non-DPEP

DPEP-I

42

81

non-DPEP (with bel ow national average female literacy level)

82

100% 90% 80% 70% 60%

50% 40%

1993

1999

All Children

Figure 2: Enrollment Rate among 12-15 year old by DPEP status (percent of all children in 12-15 group)

1993 1999 1993 1999

Male Female

1993

Minority

1999

non-DPEP

DPEP-I

43

83

non-DPEP (with belo w national average female literacy level)

84

50% 40% 30% 20% 10% 0%

1993

1999

All Children

Figure 3: Dropout rate among 5-11 year old by DPEP status (percent of all children in 5-11 group) 1993 1999 1993 1999

Male Female

199

3

Minority

1999

non-DPEP

DPEP-I

non-DPEP (with below national average female literacy level)

85

44

86

50%

40%

30%

20%

10%

0%

1993

1999

All Children

Figure 4: Dropout Rate among 12-14 year old by DPEP status (percent of all children in 12-14 group) 1993 1999 1993 1999

Male Female

1993

Minority

1999

non-DPEP

DPEP-I

45

87

non-DPEP (with below national average female literacy level)

88

100% 90% 80% 70% 60%

50% 40%

Figure 5: Cohort Progression by DPEP status

(percent of all 5-9 years old in primary or pre-primary in 1993-94; and percent of all 11-15 years old in middle or secondary in 1999-00)

1993 1999 1993 1999 1993 1999 1993

1999

All Children Male Female Minority

non-DPEP

DPEP-I

46

89

non-DPEP (with belo w national average female literacy level)

90

50%

40%

30%

20%

10%

0%

1993

1999

All Children

Figure 6: 12-15 year old with no Education by DPEP status (percent of all children in 12-15 group)

1993 1999 1993 1999

Male Female

1993

Minority

1999

non-DPEP

DPEP-I

47

91

non-DPEP (with below national average female literacy level)

92

100% 90% 80% 70% 60% 50% 40%

1993

Figure 7: 12-15 year old with at least Completed Primary Education by DPEP status (percent of all children in 12-15 group)

1999 1993 1999 1993 1999 1993

All Children Male Female

Minority

1999

non-DPEP DPEP-I

48

93

no

n-DPEP (with below national average female literacy level)

94

Table 4: Net impact of DPEP on primary school enrollment (percentage points) 5-11 years 12-15 years

All children Girls Boys Minorities

All children Girls Boys Minorities

Naïve estimator

4.177* (4.00) 5.335* (3.67) 3.151* (2.48) 6.268* (3.18) 9.999* (5.62)

10.954* (4.53) 7.594* (3.49)

13.262* (4.83)

DID estimator

1.439 (1.023) .035 (.02) -.624 (-.37)

4.561** (1.65)

Madhya Pradesh 1.799 (.75)

-1.359 (-.41) 2.856

(1.001) 17.143* (4.55)

Naïve estimator

7.388* (5.33)

10.665* (5.13) 5.112* (3.08)

10.027* (3.77)

10.937* (4.80)

13.540* (3.98) 8.923* (3.26)

13.764* (3.76)

DID estimator

-1.241 (.65) .481 (.17)

-1.186 (.52)

-3.527 (.99) 3.382 (1.07) 4.889 (1.04) 3.844 (1.04) 9.626* (2.02)

Notes: t-statistics in parentheses * indicates statistical significance at 5 percent or lower ** indicates statistical significance between 5-10 percent Naïve estimator: mean difference in levels between 1999-00 and 1993-94 in treatment districts only

DID estimator: mean difference between treatment and propensity score matched control districts between

1999-00 and 1993-94

95

49

96

Table 5: Net impact of DPEP on dropout rates (percentage points) 5-11 years 12-14 years

All children Girls Boys Minorities All children Girls Boys Minorities

Naïve estimator -2.724* (2.68)

-3.945* (2.77)

-1.772* (1.43) -2.338 (0.89)

-8.920* (5.06)

-10.119* (4.20)

-7.370* (3.44) -.588 (0.16)

DID estimator

-.173 (.13) 1.358 (.71) .724 (.44)

-7.471* (1.96)

Madhya Pradesh -1.147 (.48) 2.506 (.75)

-5.087** (1.788)

-9.168** (-1.73)

Naïve estimator -7.448* (5.27)

-11.105* (5.01)

-5.279* (3.19)

-7.401* (6.23)

-13.174* (5.43)

-16.984* (4.54)

-10.306* (3.62) -.517 (.10)

DID estimator -3.263** (-1.692) -3.831 (1.28) -.554 (.24)

18.14* (3.20)

-6.136* (-1.90)

-10.225* (1.98) -3.613 (.94)

23.323* (2.98)

Notes: t-statistics in parentheses * indicates statistical significance at 5 percent or lower ** indicates statistical significance between 5-10 percent

Naïve estimator: mean difference in levels between 1999-00 and 1993-94 in treatment districts only DID estimator: mean difference between treatment and propensity score matched control districts between

1999-00 and 1993-94

97

50

98

Table 6: Net impact of the DPEP on cohort progression rates (percentage) Cohort of 5-9 years in primary school in 1993-94 who progresses to middle school in 1999-00

All children

DPEP I states Naïve estimator DID estimator

9.059* 2.651 (6.91) (1.52)

Madhya Pradesh Naïve estimator DID estimator

11.214* 4.162 (5.42) (1.48)

Girls Boys

Minorities

Notes: t-statistics in parentheses

11.014* (6.05) 3.630* (3.83)

10.869* (4.56)

1.568 (.66)

5.490* (2.44) 7.324* (2.32)

14.862* (5.27) 7.441* (2.78)

10.503* (3.33)

4.287 (1.12) 6.080* (2.19)

19.149* (4.63)

* indicates statistical significance at 5 percent or lower

** indicates statistical significance between 5-10 percent Naïve estimator: mean difference in levels between 1999-00 and 1993-94 in treatment districts only

DID estimator: mean difference between treatment and propensity score matched non-treatment districts between 1999-00 and 1993-94

99

51

100

Table 7: Net impact of DPEP on educational attainment among 12-15 year olds (percentage points)

All children

Children with no education

Naïve estimator DID estimator -7.401* -3.837* (6.23) (2.40)

Children with completed primary education

Naïve estimator DID estimator 6.409* .600 (4.53) (.35)

Girls Boys Minorities

All children Girls Boys Minorities

10.200* (5.43)

-5.466* (4.11)

-11.892* (4.84)

-12.230* (5.84)

-13.992* (4.21)

-10.430* (4.48)

-15.669* (4.45)

-1.452 (.58)

-5.139* (2.85) 1.235 (.37) Madhya Pradesh

-6.450* (2.25) -4.072 (.909)

-10.243* (3.28) -2.411 (.52)

7.072* (3.38) 5.736* (3.30)

10.683* (3.97)

10.224* (4.42) 8.222* (2.48)

11.209* (3.84)

14.779* (4.15)

4.147 (1.50) .497 (.21)

-9.744* (2.72)

5.414** (1.72)

12.480* (2.65) 5.249

(1.314) -1.175 (.25)

Notes: t-statistics in parentheses * indicates statistical significance at 5 percent or lower

** indicates statistical significance between 5-10 percent

Naïve estimator: mean difference in levels between 1999-00 and 1993-94 in treatment districts only

DID estimator: mean difference between treatment and propensity score matched control districts between 1999-00 and 1993-94

101

52