CRES/UNU-MERIT2013, International Conference

« Sustainable Growth and Development in Africa»

DIAGNE AbdoulayeLÔ Mouhamadou Moustapha

SOKHNA OusmaneDIALLO Fatoumata Lamarana

Dakar, Monday 25 November, 2013

Impact Evaluation of School Feeding Programs on Internal Efficiency of schools and Learning Abilities of

Students in Rural Primary Schools in Senegal

Outline

I. Context and problems

II. Research questions and objectives

III. Experimental Design

IV. Estimation method of impacts

V. Main Results

VI. Conclusion, Discussions and Prospects

2

I. Context and problems

• Education is now considered among the most important factors of development.

• Access and quality of education form the second priority of the International Community MDGs.

• Despite these considerations and this international commitment, progress in education of quality for all has been slow and unequal.

• More than 121 millions children of school age are still out of school. Most of them live in sub-Saharan Africa particularly in rural areas.

• Children in school face with the education quality issue.

3

• Hunger, nutritional deficiencies and chronic fatigue can be a major barrier to schooling and a factor of low education quality for poor households children.

• Feeding children in schools can both impact significantly on these deficiencies and the children poverty.

• Indeed, many studies attempted to establish relationships between school feeding and education performances.

4

I. Context and problems (cont’)

• Some studies use non experimental evaluation (Patrick J. and al, 2010 ; Sanchez, 2009; Farzana, 2007; Essama, 2006 ; Flores, 2005 ; Ahmed, 2004 ; Ahmed and Del Ninno , 2002 ; etc.)

• Other adopt randomized experimentation (Aguiro and Woohard, 2007 ; Vermeersche and Kremer, 2005 ; Imbens, 2004 ; Vermeersch and Kremer, 2004).

• This second group of studies is characterized by a great diversity both in terms of treatment, beneficiaries, treatment, sample size, time of exposure to treatment, type of effects, etc.

5

I. Context and problems (cont’)

Contribution of the study

This research :

• Is among the first experimental evaluations in the field of education in sub-Saharan Africa.

• Helps to fill the lack of evidence on the actual impact of school feeding programs on school performance and nutritional status of children in Senegal.

• Providing impact on four learning abilities : reasoning, memory, comprehension and knowledge.

• Makes a detailed analysis of the heterogeneity of the school feeding impacts.

• Highlighting the influence of school governance in achievement.

• Highlighting interaction between SFP and 5 inputs of education quality.

6

I. Context and problems (end)

II. Objectives and research questions• Objectives

Evaluate the potential contribution of school feeding programs to access to quality primary education for boys and girls in rural Senegal.

• Main Research questions

Nutrition• Does canteen program can improve the nutritional status of beneficiaries

children ?• Does canteen program have positive externalities ?

Quality

• Does canteens program contribute to improved cognitive learning and learning abilities of boys and girls ?

• The different cognitive skills are they affected the same way ? • On what kind of cognitive skills, school canteens have the greatest impact ?

7

Internal efficiency of schools

• Does school canteens contribute to lower repetition and dropout rates of beneficiaries schools ?

• Does school canteen program contribute effectively to attract more students to the school ?

• Does school canteen program promotes the education of girls more than boys ?

8

II. Objectives and research questions (end)

1. Description of the InterventionNature of the Intervention Implementation of school feeding and provision of meals to all students.

Intervention areas Four poor regions of Senegal located in rural areas : two provinces in

Centre of the country (Fatick and Diourbel) and two in the South (Kolda and Sedhiou).

Main characteristics of these regions: (1) low presence of schools canteens compared to the others parts of the country; (2) very effective delay in terms of schooling; and (3) high prevalence of poverty and vulnerability.

Main Actors Ministry of Education (through DCS and DPRE); WFP; INEADE; IDEN;

Communities.

III. Experimental Design

9

Food allocation Composition of the food basket : corn, leguminous plants, oil, iodized salt, proteins and condiments.

WFP provides the food in dry ration and Community brings seasoning.

Source: Division of the school canteens /DPRE, ME

10

III. Experimental Design (cont’)

2. Sampling Criteria of targeted regions

Poverty and food insecurityExistence of standard evaluation systemBe in the intervention area for the partners

Criteria of eligibility of schools No school canteen Have At least 50 students and maximum 600 Have a management committee operations (for schools of treatment)Have at least second and fourth grades (CP and CE2 in Senegal)

Selection of the schools in the sample

Sample size: 120 schoolsSchools repartition between two groups: random assignment of 60 schools in treated

group and 60 others in control group. Number of observations: First wave (pre-test) : 2917 students (1472 boys and 1445

girls) and Second wave (post-test) : 2882 students (1445 boys, 1427 girls).Duration of the experimentation: One academic year. 11

III. Experimental Design (cont’)

3. DataPrimary data collected Strategy - Survey with two wages (baseline [2009] + follow-up [2010])

4 questionnaires- Census around the school (radius < 4 km): number of children (7 and 7-12

years old), ...- Household: age, gender, education level of head, ...- School and its environment: number of students by grade, repetition,

dropout, ...- Teacher/grade: age, gender, diploma, experience, class size, …

Standard Evaluation - Periods: pre-test and post-test; grades: CP and CE2; disciplines: french and

mathematics. Fact sheets (day and month follow-up) - quality/quantity of food served at the canteen.

Secondary data - Data bases of MEN, IA, IDEN.

12

III. Experimental Design (end)

Protocol validation : - Tests (Student, Kolmogorov-Smirnov, Mann-Whitney)Tests results : - School level (Homogeneity); - Student level (No homogeneity).Method : The unbiased estimator of the Average treatment effect (ATE) is

calculated with the Difference-in-difference method.

• Partial adherence corrected by Instrumental Variable (ATT);• Controlling the equation by the preprogram characteristics ;• Model of interaction with canteen and school inputs of quality .

13

IV. Estimation method (cont’)

T : traitement (1,0)t : temps (1,0)Y : mean scoreX : explanatory variablese : error

1. Impact on dietary intakeSchool feeding improves dietary intake of students who received

treatment.

External effects on dietary intake School canteens improve significantly the dietary intake of children living

with treated students.

V. Main Results

14

Beneficiaries Energie(kcal) Lipide (g) Protéine (g) Glucide (g)

Direct (Student) 329.353 2.672 7.790 25.731

Standard Error (45.399)*** (1.294)*** (1.890)*** (8.979)***

Indirect (Brothers at home) 135.11 8.39 6.331 17.06

(46.321)*** (1.465)*** (2.014)*** (9.037)***

2. Impact on cognitive acquisitions

• School feeding increase global cognitive acquisitions by 5.5 points of percentage (pp)

• The impact is higher in mathematics than French. But both are significativement improved (Gove et Cvelich, 2010 for the importance of this result).

• Results are significant at 5 %.

Outcomes OLS Method Fixed effects ATE ATE ATT (IV) ATT (IV) ATE ATE

No control Control No control Control No control Control

Global Score 6.03 5.51 9.11 8.33 6.03 5.51

Standard Error ( 2.522)** (1.11)*** (1.592)*** (4.134)** (2.521)** (2.735)**

French 5.59 4.60 (omitted) (omitted) 5.59 4.605

(1.162)*** (3.05) - - (2.837)* (3.047)

Mathematics 6.35 6.32 (omitted) (omitted) 6.35 6.31

(2.69)** (2.88)** - - (2.69)** (2.88)**

V. Main Results (cont’)

A. Heterogeneities of impact

1. Grade

Only global cognitive acquisitions of student in the second year (CP) are significantly increased by the program (10.56 pp).

2. Age The program had large significant impact on cognitive acquisitions

scores of over 10 years olds who had delayed school entry. It increased their score by 10.65 points of percentage

This result confirms Adelman et al. (2008) study in Uganda.

3. GenderThe program has more success in boys (10.72 pp) than in girls (9.6 pp),

especially in second year. The results are the same in French and Math. Boy are more sensitive.

V. Main Results (cont’)

4. School governance

Schools with a Student Parents Association, a Cooperative or a Management Committee have better results.

4. Class size

Negative effect have been found on scores in classes exceeding 60 students. This result is frequent in the literature (Grantham-McGregor, 2005; Papamandjaris, 2000; Chang 1996).

B. Interactions : Complementarities and Substitutability

There are interaction between the school feeding and the traditional factors of education quality. The impact of school feeding on cognitive acquisitions is higher when the class size is small and when children come from poor households.

V. Main Results (cont’)

3. Impact on Learning Competences

• The cognitive competences of student in 2nd year increased more than those in fourth year.

• The highest impacts are obtained in Reasoning and Memory abilities.

2nd year (CP) 4th year (CE2)

No control Control No control Control

Comprehension 10.04 7.61 4.58 1.91 Standard Error (4.055)** (4.519)* (3.308) (3.569)

Reasoning 12.09 15.46 5.76 2.28(4.569)*** (5.128)*** (3.915) (4.119)

Knowledge 10.24 9.432 10.54 10.61(4.57)** (4.67)** (4.271)** (4.42)**

Memory 23.38 19.92 - -(3.80)*** (8.37)** - -

V. Main Results (cont’)

Hetorogenity on cognitive competences

• Comprehension : results are significant only on student in grade 2 (CP) who are not delayed in entry in school.

• Reasoning : significant results on 6-7 years old (23.92 pp) and 8-9 years old (15.59 pp ) in grade 2 (CP)

• Memory : significant results on 6-7 years old (33.23 pp) and 8-9 years old in grade 2 (17.20 pp)

• Knowledge : best results on student who are delayed in entry (13.35 pp)

V. Main Results (cont’)

4. Impact on internal efficiency

• The program reduces both grade repetion and drop out rates. But the results are not significant, even at 10 % level.

• Then program has not significantly impact on internal efficiency.

V. Main Results (cont’)

21

The average cost of School Feeding Program, is $ 0.19 per student per day (= $ 34 for a school year of 180 days) (WFP, 2001).

Due to strikes hourly quantum is 733/900 hours in 2009 (DRSP II, 2010).

On this basis, the annual cost per student in the school canteen is 27.69 U.S. dollars (or 13,854 FCFA on $ 1 = 500 FCFA).

Sources : Faye, S (2010) and authors’calculation.

Facteurs positifs Estimated coefficient (in%)

Annual cost per student

Cost-effectiveness report (in%)

Share in Sch. Feed. Cost

Proportion of teachers between 5 -10 years experience 25,341 2867 0,884% 13,4

Number of books per student 6,171 2305 0,268% 4,1

Teachers’ Evaluation system 0,783 110 0,712% 10,8

School feeding 9,11 13854 0,066% 1,0

5. Cost effectiveness analysis of School feeding program

V. Main Results (cont’)

VI. Conclusion, Discussions and Prospects

Our results suggest that school feeding can be an effective way to accelerate progress towards quality of education for all.

It can also reduce gender discrimination in education.

In order to fully enhance education performance, a strong school governance is essential.

Through the school canteen, it is possible to reduce child malnutrition.

School canteens are a way to fight against malnutrition, and hence children poverty.

However, one should check if the effect of feeding would be greater if they were coupled with a school deworming program.

Finally, it should be noted that school feeding programs are criticized because of the cost of their implementation.

Is it very efficient to support school feeding programs? Only cost-effectiveness analysis would address these concerns.

23

VI. Conclusion, Discussions and Prospects (Cont’)

I. How to make school feeding less expensive (sustainability of SFP)?

II. How to make more effective canteens by incorporating more local

agriculture (Increase the effectiveness of SFP)?

III. how sustainably integrate deworming in the School feeding program

(making the SFP most healty)?

IV. How to take into account the inequality between student from poor

households ?

V. how to involve more governments in the funding of these programs.

VI. Best results by correcting the attrition bias (more robust and relevant

results). 24

VI. Conclusion, discussions and Prospects (END)

Thank you for your attention

25