Mexico: School-Based Mexico: School-Based ManagementManagement

Paul Gertler

Harry Anthony Patrinos

Marta Rubio-Codina

MotivationMotivation

• Increasing number of projects with School-Based Management (SBM) components

• Relatively little evidence being produced on outcomes

• Analysis of large-scale compensatory education program in Mexico with SBM component, thru a retrospective evaluation

OutlineOutline

• Background on compensatory program $ SBM component

• Data & identification strategy

• Results

• Conclusions

Mexico’s Compensatory Mexico’s Compensatory ProgramsPrograms

• Supply-side intervention started in 1991

• Managed by SEP (Secretariat of Public Education), implemented by CONAFE (Consejo Nacional de Fomento Educativo, or National Council of Education Promotion)

• Channel resources to the worst performing schools: – Reach the most disadvantaged– Reduce schooling inequalities

• Increase schooling availability and school quality

• Fine targeting of rural areas according to marginality

• Since 1991, program has constantly evolved in terms of targets & interventions

CONAFE InterventionsCONAFE Interventions

• Learning Materials Provision: – Provision of school and student supplies

• Teacher Training

• AGEs (Programa de Apoyo a la Gestión Escolar, or School Management Support Program):– Monetary incentives and other support to increase parental

involvement in school management.• Parental Associations exist by law but are rather dysfunctional

• Other interventions (not evaluated):– Improved technology and infrastructure– Teacher monetary incentives

Compensatory Education Coverage

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004

Initial Preschool Primary Secondary

HighlightsHighlights• Each year 4.5 million primary & lower secondary students receive school

materials (notebooks, pencils, rulers, geometry kits); 16,000 primary schools & telesecundarias received educational materials & computers

• Construction or rehabilitation of educational facilities, including classrooms, toilets, playgrounds within schools: almost 10,000 works each year

• About 13,400 teachers receive monetary incentives intended to reduce mobility & improve teaching; 122,000 teachers in 40,000 schools receive technical & pedagogical feedback

• Parental organizations from more than 12,000 kindergartens & 47,000 schools receive funds to improve school

• Initial Education component serves almost 0.5 million parents through orientation in early stimulation & child-care techniques

• Multicultural and indigenous education activities undertaken

Compensatory Education in MexicoCompensatory Education in Mexico

Budget $221 million

No. of children 5.6 million

Avg cost per child $50

Program Relatively Low-costProgram Relatively Low-cost

$50/student –

• Compared to: $527 telesecundaria, $477 general middle school

AGEsAGEs

• Support & finance training for Parent Associations (APFs)

• Parents trained in school management of funds transferred to APFs, participatory skills, information on achievements of students & ways parents can help improve learning

• Financial support to AGEs consists of annual grants transferred quarterly to APFs’ school accounts, from $500-$700 a year, according to size of school

AGEs: Example of SBM with AGEs: Example of SBM with Limited PowersLimited Powers

AGEs Mexico

PEC Mexico

Educo El Salvador

Autonomous Schools Nicaragua

Chicago Netherlands

Books/infrastructure Yes Yes Yes Yes Yes Yes Teacher training No No No No Yes No Hire teachers No No Yes Yes Yes Yes Teacher salaries No No No Yes No Curriculum No No No No Yes Yes

More on AGEsMore on AGEs

• Cannot spend money on wages & salaries for teachers• Most of the money goes to infrastructure• AGEs designed to promote school-parent cooperation• AGEs increase school autonomy through participation

Gets parents into the school, thru formal channel:

1. AGEs allow parents to see teacher show up

2. Parents can witness if their children pay attention

3. Participation gives parents power

46% of Primary Schools have 46% of Primary Schools have AGEsAGEs

Identification IIdentification I• Treatment Schools: schools intervened (continuously) between

1998 - 2001

• Control Schools: schools intervened starting in 2001 or not yet intervened

• CONAFE Treatment:– Dummy =1 if intervention year– Number of Periods school has received CONAFE

• OPORTUNIDADES Treatment:– Dummy =1 if intervention year– Ratio of OPORTUNIDADES beneficiaries in the school (intensity)

• Intermediate School Quality Measures:– Repetition Rates– Failure Rates– Intra-Year Drop Out Rates

Identification IIIdentification II• School level dif-in-dif estimation: School FE (s): changes in

outcomes as a function of changes in treatment status• Identification assumption: change in controls measures what change

in treatment would have been without AGEs (counterfactual); cannot test directly, but test pre-intervention trends are equal, likely that assumption is fine

• Test: pre-intervention trends equal between treatment & control schools

• State * Time Dummies: st : controls time varying changes in state education policy and other state changes in economic trends

• Treatment-Specific Time Trends (eg, different evolutions b/w treatment & control schools over time)

• Time varying school characteristics: intensity of Carrera Magisterial• Robust SE Clustered at the School (allow serial correlation within a

school over time)

Data Sources & Sample Data Sources & Sample • CONAFE Administrative Data (1991 - 2002)

• OPORTUNIDADES Administrative Coverage Data (1997-2003)

• Outcomes: School Census (Censo Escolar 911), 1995- 2003

• 2000 Mexican Census & 1995 Conteo

• 2 sub-samples:– All non-indigenous (general) rural schools– Q34: Non-indigenous rural schools in the top 2 quartiles of the CONAFE 2000

Targeting Index Distribution• Sample likely to better balanced• Pre-intervention trend

Sample SizesSample Sizes

All Schools

OP Treatment

OP Control

Total Number of Schools (%)

CONAFE Treatment 1,912 217

2,129

(39.55%)

CONAFE Control 2,571 683

3,254

(60.45%)

Total Number of Schools (%)

4,483

(83.28%)

900

(16.72%) 5,383

Q34 Schools

CONAFE Treatment 1,365 123

1,488

(36.01% )

CONAFE Control 2,101 543

2,644

(63.99%)

Total Number of Schools (%)

3,466

( 83.88%)

666

(16.12%) 4,132

Sources of VariationSources of Variation

• Number of schools attended increases over time

• Differences in CONAFE and OPORTUNIDADES phase-in over time & space

• Intensity of OP treatment (OP Ratio) varies across schools

• Variation in timing of type of treatment within CONAFE schools

0

5

10

15

20

25

30

35

40

45

Pe

rce

nta

ge

Sc

ho

ols

(%

)

1998 1999 2000 2001

School Year

Figure 2: Percentage of Schools By Treatment Status Over Time

None

CON Only

OP Only

Both

.07

.08

.09

.1.1

1F

ailu

re R

ate

1995 1996 1997 1998 1999 2000 2001Year

AGEs Treatment AGEs Control

AGEs vs. Non-AGEs SchoolsFigure 2: Failure Rate Trends

.075

.08

.085

.09

.095

Re

petit

ion

Rat

e

1995 1996 1997 1998 1999 2000 2001Year

AGEs Treatment AGEs Control

AGEs vs. Non-AGEs SchoolsFigure 3: Repetition Rate Trends

.03

.05

.07

Dro

p O

ut R

ate

1995 1996 1997 1998 1999 2000 2001Year

AGEs Treatment AGEs Control

AGEs vs. Non-AGEs SchoolsFigure 4: Drop Out Rate Trends

Mod 1A Mod 1B Mod 1C Mod 2A Mod 2B Mod 2C Mod 3A Mod 3B Mod 3C Mod 4A Mod 4B Mod 4C

AGEs =1 -0.004* -0.004* -0.004* -0.004* -0.005* -0.004* 0.000 -0.000 -0.000 0.383 0.360 0.356(0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.385) (0.385) (0.391)

Other Interventions

Ratio of Oportunidades Students in the School -0.010** -0.010** -0.008** -0.008** -0.013** -0.013** -2.596** -2.582**(0.003) (0.003) (0.003) (0.003) (0.002) (0.002) (0.740) (0.737)

Proportion of Teachers under Carrera Magisterial -0.003+ -0.003+ -0.004* -0.004* -0.002 -0.002 0.275 0.274(0.002) (0.002) (0.002) (0.002) (0.002) (0.002) (0.399) (0.399)

Other CONAFE InterventionsInfrastructure =1 0.002 0.000 0.000 -0.457

(0.003) (0.003) (0.003) (0.732)Equipment =1 -0.000 -0.001 -0.001 3.734*

(0.005) (0.005) (0.004) (1.486)Incentives =1 0.009 0.013 -0.002 0.918

(0.012) (0.011) (0.007) (2.234)

Student Supplies =1 -0.001 -0.002 -0.001 0.677(0.002) (0.002) (0.002) (0.856)

Training =1 0.001 0.002 0.002 -1.191*(0.002) (0.002) (0.002) (0.538)

School Fixed Effects Y Y Y Y Y Y Y Y Y Y Y YTime-Varying School Charact. Y Y Y Y Y Y Y Y Y Y Y YState Specific Time Trends Y Y Y Y Y Y Y Y Y Y Y YTreatment Specific Trends Y Y Y Y Y Y Y Y Y Y Y YNumber of Observations 30190 30190 30190 30190 30190 30190 30190 30190 30190 30190 30190 30190Number of Schools 6,038 6,038 6,038 6,038 6,038 6,038 6,038 6,038 6,038 6,038 6,038 6,038

Mean Failure Rate 0.09 0.09 0.09 0.09 0.09 0.09 0.04 0.04 0.04 131.23 131.23 131.23

DROP OUT RATE TOTAL ENROLLEMENT

Table 5: Effect of AGEs on School Aggregate Educational Outcomes: Failure, Repetition and Intra-Year Drop Out Rates; and Total Enrollement from 1998 to 2001 -Subsample of General Rural Primary Schools

Notes: +significant at the 10%, *significant at the 5%, **significant at the 1%. LS regressions with school FE. Robust SE clustered at the school level in parantheses. AGEs treatment schools are schools that continuously receive the Apoyo a la Gestión Escolar (AGEs), starting in 1998 (or later) until 2001. Extreme values for the dependent variables trimmed at the top 0.5% of the dependent variable distribution.Sample restricted to schools with no missing information on any of the dependent variables studied.

FAILURE RATE REPETITION RATE

Threats to Identification IThreats to Identification I• Placement Bias: interventions are non-randomly allocated

– Sign of bias depends on whether better/worse schools are intervened first– Time*State dummies capture shifts/changes in the allocation of resources

in each state

• Treatment status correlated with school characteristics:– If time invariant characteristics, school FE solve the problem– If time variant characteristics…

• … re-do analysis on the sub-sample of CONAFE Schools that receive all 3 CONAFE interventions (AGEs & Supplies & Teacher Training)

• Trend * Treatment capture evolution of treated schools over time

• Results robust although larger estimated AGEs effects Better schools might be receiving AGEs first. AGEs effect might disappear…

…CAPAGEs intervention starting in 2003: provide guidance in the administration of the AGEs monetary support

Threats to Identification IIThreats to Identification II• Treatment might affect the distribution of students’ skills in the school;

if so, then treatment correlated with unobserved ability– Check whether changes in total school enrollment over evaluation period – No significant changes in enrollment observed

• Spillover effects within the school: – School average rate includes OP beneficiary & non-beneficiary students– If positive spillover effects from OP beneficiaries to non-beneficiaries,

aggregate school effects can come from learning improvements from beneficiary and/or non-beneficiary students

– OPORTUNIDADES effect potentially over-estimated

• Spillover effects across schools/geographical areas:– Better/worse performing students select themselves into CONAFE

schools– Unlikely given:

(i) school choice is unlikely to be a dependent variable in rural Mexico(ii) small non-significant variation in enrollment in CONAFE school

AGEs are Cheap and EffectiveAGEs are Cheap and Effective

• While unit cost of compensatory program overall is $50• AGEs are much less costly component• Parents at participating school receive $500-$700 a year• 45,000 schools; 4.5 million students• Total cost of AGE school grants is $26 million a year

Annual unit cost: $5.86 per studento Oportunidades – also effective, with other important benefits –

pays students at primary level $100-$200 depending on grade

Pathway: Not infrastructure – but Parental Participationo Qualitative assessment confirms

ConclusionsConclusions• Demand-side intervention have effect on learning outcomes:

– nutritional component?

• OPORTUNIDADES is very effective in keeping students in school

• Results by intervention show large effects from AGEs that remain even after controlling for other education policies:– decentralization at the local level is effective & low cost

Large efficiency gains

• Difficult to econometrically identify the effect of school input provision

• No evidence of synergies