171101 ABE Final Report 22.09.17 - UNICEF › evaldatabase › files › Ethiopia... · An Impact Evaluation on the effectiveness of Alternative Basic Education in Ethiopia – (Draft)

CENTER FOR

EVALUATION AND

DEVELOPMENT

An Impact Evaluation of

Alternative Basic

Education in Ethiopia

–

The case of the regions

Afar, Oromia and Somali

Final Evaluation Report

October 2017

This evaluation was commissioned by UNICEF Ethiopia.

The expressed point of view in this document may not necessarily represent the views of

UNICEF Ethiopia nor the Ministry of Education of Ethiopia.

This report was prepared by C4ED.

An Impact Evaluation on the effectiveness of Alternative Basic Education in Ethiopia –

(Draft) Final report

ii Center for Evaluation and Development – C4ED (May 2017)

ACKNOWLEDGMENTS

The evaluation consultant team acknowledges useful guidance and support provided by the staff

of UNICEF Country Office and Regional Offices in Ethiopia throughout the evaluation process.

The evaluation consultant team is thankful to Mr. Ibrahim Worku, Mr. Abdihakim Mohammed

Aladi, Mr. Seyoum Jonoto Mehari, and Mr Seraj Mustapha Adem for their local expertise and

precious support provided in planning, coordinating and facilitating the field mission in the

regions of investigation.



iii Center for Evaluation and Development – C4ED (May 2017)

Title: An Impact Evaluation of Alternative Basic Education in Ethiopia – The case of

the regions Afar, Oromia and Somali

Timeframe of the evaluation: October 2016 – October 2017

Date of the report: October 2017

Country: Ethiopia

Evaluators: Dr. Adrien Bouguen, Prof. Dr. Markus Frölich, Mariam Nikravech, Linh

Nguyen, Dr. Asmus Zoch

Name of the organization commissioning the evaluation: UNICEF Ethiopia

Name of UNICEF staff contact point for the evaluation: Dorothy Aanyu Angura, Chief

Education Officer and Alembanchi Molla, Education Officer, Access and Equity Section,

UNICEF Ethiopia Country Office



iv Center for Evaluation and Development – C4ED (May 2017)

TABLE OF CONTENT

Acknowledgments ............................................................................................................................................................. ii

List of Acronyms ................................................................................................................................................................. v

1 Executive Summary .................................................................................................................................................. 1

2 Introduction................................................................................................................................................................. 7

3 Object, Scope And Purpose of the Evaluation ................................................................................................ 7

3.1. Purpose and scope ..................................................................................................................................... 7

3.2. Background and context ......................................................................................................................... 7

3.3. Theory of Change .................................................................................................................................... 15

4 Impact Evaluation methodology ...................................................................................................................... 17

4.1 Data collection .................................................................................................................................................. 18

4.2 Data Analysis .................................................................................................................................................... 23

Evaluation questions ............................................................................................................................................ 24

5 Findings ...................................................................................................................................................................... 25

Evaluation Question 1 - Impact / Effectiveness ......................................................................................... 25

Evaluation Question 2 - Impact / Effectiveness ......................................................................................... 37

Evaluation Question 3 - Effectiveness ........................................................................................................... 41

Evaluation Question 4 - Effectiveness ........................................................................................................... 45

Evaluation Question 5 - Relevance .................................................................................................................. 62

Evaluation Question 6 - Efficiency .................................................................................................................. 69

Evaluation Question 7 - Sustainability .......................................................................................................... 71

6 Conclusions ............................................................................................................................................................... 77

7 Recommendations ................................................................................................................................................. 77

7 Annexes ...................................................................................................................................................................... 83



v Center for Evaluation and Development – C4ED (May 2017)

LIST OF ACRONYMS

ABE Alternative Basic Education

ABEC Alternative Basic Education Center

BoFED Bureau of Finance and Economic Development

CMC Center Management Committee

E.C. Ethiopian calendar

ESDP Education Sector Development Programme

EQ Evaluation Question

FGD Focus Group Discussion

GER Gross Enrollment Rate

GEQIP General Education Quality Improvement Plan

GTP Growth And Transformation Plan

JC Judgement Criteria

LQE Local Qualitative Expert

MoE Ministry Of Education

MoFEC Ministry of Finance and Economic Cooperation

NER Net Enrollment Rate

PTA Parents Teacher Association

PTSA Parents Teacher Students Association

RCT Randomized Control Trial

REB Regional Education Bureau

SIP School Improvement Program

ToRs Terms of References

TTC Teacher Training College

UNICEF United Nations Children's Fund

WEO Woreda Education Office

ZEO Zonal Education Office



1 Center for Evaluation and Development – C4ED (May 2017)

1 EXECUTIVE SUMMARY

This is an Impact Evaluation report on Alternative Basic Education in Ethiopia (ABE) - The case of the regions Afar, Oromia and Somali”. The evaluation was conducted between October 2016 and August 2017. Background

The ABE program is an intervention that resulted from a study conducted in 2000 by the Ministry of Education called “Alternative Routes to Basic Education" that recommended a focus on enhancing the ABE program as articulated in the ESDP II (2002/03 – 2004/05). The ABE program was developed under the leadership of the Ministry of Education. UNICEF supported and implemented the program in partnership with the Regional Education Bureaus in the four regions of Afar, Somali, Oromia and Benishangul Gumuz. The ABE intervention aims at providing opportunities for out-of-school children, especially in the age range 7-14, to have access to good quality basic education. To achieve this goal the program supported the construction and equipping of the Alternative Basic Education Centers (ABECs) addressing the educational needs of hard-to-reach children in pastoralist areas in Ethiopia. Over the last ten years UNICEF supported the setting up of 1,678 ABECs, enrolling over 276,777 students (45% girls) in the marginalized communities. In this time-period UNICEF spend about around 10,000 Birr per ABEC and 17,306,000 Birr in total on the establishment of ABECs and 588,000 Birr on rehabilitation costs. In addition, approximately 5,000 facilitators/teachers of ABECs have received training to enhance their instructional skills. Purpose and Objectives of the Evaluation

In August 2016, the Center for Evaluation and Development (C4ED) at the University of Mannheim was contracted by UNICEF Ethiopia country office as an independent evaluator to evaluate ABE program in Ethiopia. In particular, the evaluation assessed whether ABECs contributed to the achievement of the ESDP V objectives of providing access for out-of-school children and in particular children from pastoralist communities. Furthermore, this study explored the specific role UNICEF has played in supporting ABECs. It sought to find out the extent of the impact, effectiveness, relevance, efficiency and sustainability of UNICEF’s support to ABECs. It had the main purpose of providing evidence for decision-making for the improvement of implementation of ABE program support to the Government of Ethiopia (GoE).

The evaluation addressed four specific objectives which included the following:

1. To assess the impact and effectiveness of ABE, and in particular of ABECs seeking answers to questions such as:

o To what extent has the provision of ABECs, in particular ABEC supported by UNICEF, increased access to education for underserved populations, especially children from pastoral communities?

o How have ABECs increased the cognitive development of children in terms of basic math skills, reading, writing, computing and understanding their environment and enabled successful transition of students to formal education?

o Are ABECs a flexible delivery method? o Do ABECs provide quality education?




2. Assess the relevance of ABECs by asking: o Are the modalities of implementation of ABE, in particular under the ABEC

modality, more relevant to the needs of children in pastoral communities, in

comparison to formal primary?

3. To assess efficiency the of ABECs by probing: o Was the ABE program implemented in the most efficient way to increase access

to basic education?

4. To assess sustainability the of ABECs by inquiring: o How did the ABE program encourage ABE-related activities within communities?

Do more active communities demand for more ABE services?

The scope of this evaluation encompasses the ABE program from 2010-2016 and focuses on three regions supported by UNICEF and the Ministry of Education (MoE) of Ethiopia: Afar, Oromia and Somali.

The intended main users of the evaluation findings are UNICEF Ethiopia and the Ministry of Education (MoE), Education specialists and stakeholders at national and local levels

Evaluation Methodology

The evaluation applied a mixed-method approach including, a large quantitative data

collection, in-depth qualitative interviews and structured analysis of secondary / administrative

data. Given that ABE center locations were not assigned randomly and that no baseline data was

collected prior to the beginning of the intervention, the impact evaluation design was based on

non-experimental methods. This entailed the selection of a group of control villages, where no

ABEC was established, and using households, parents and kebele data to account for the initial

differences between the ABEC villages and the control village types. The impact was then

assessed by comparing the performance of the children in ABEC villages to the ones in non-ABEC

villages. The following options were used:

1. Comparing villages with UNICEF supported ABECs against those with other

(government) ABECs,

2. Comparing villages with UNICEF supported ABECs against those with formal primary

schools,

3. Comparing villages with UNICEF supported ABECs against those with no other primary

school education facility.

In the main analysis, multivariate regression models were used to control for heterogeneous student backgrounds, such as different household, caregiver and village characteristics. Throughout the evaluation process great efforts were made to generate high quality data and credible evidence considering the time, budget and the prevailing circumstance in the field. Yet, since the data collection was only carried out in the three regions and in remote areas, by design, the findings of this study cannot be generalized for the whole country. Furthermore, due to the drought and emergency situation in the Somali region, findings from this region are limited to the Jijiga woreda.




Main findings and conclusion

Impact and effectiveness

In general, ABECs are an important tool to deliver primary education to remote pastoral communities. The evaluation showed that ABECs have helped increase enrollment rates in villages where no other primary school alternative was available. Overall in this remote setting enrollment rate was found to be low and far from reaching the goal of universal enrollment. The average net enrollment rate for children between the ages of 7 and 11 was 71% in villages with ABEC in comparison to an enrollment rate of 74% in villages with primary schools and 50% in villages without any primary schooling facility. Regional effects showed larger enrollment rates in Afar (75%) than in Oromia (66%) and Somali (55%). In terms of the gender, it was the largest in the Somali region (45% for girls vs. 62% for boys), followed by Oromia (60% vs. 66%). In Afar no gender gap could be observed.

To the question of why their children are not going to school, especially in villages that have no primary education facility, more than half of the parents claim the school is too far way (57%). This answer was cited much less in villages with an ABEC (31%) or with a formal primary school (15%). Overall, the Child must help with household chores was named as the main reason why children did not go to school. This was stated by 67% of parents in villages with primary schools and by 53% of parents in villages with an ABEC. This is in line with the findings from the in-depth interviews, where parents raised the need for child labor to contribute to economic and livelihood activities in those pastoral communities.

The unbiased net effect of having an ABEC in the village can only be estimated using multivariate regression models controlling for heterogeneous student background. The results of this model showed an average increase of 16 percentage points’ enrollment rates in villages with an ABEC in comparison to having no primary facility. Given that in villages without any school only half of all children are attending primary school, this result highlights the importance of ABECs in bringing basic education to remote communities. Despite this positive enrollment effect, transition from ABECs to the second cycle of primary education at formal primary schools remains low and gender-biased: Only 21% of boys and 14% of girls living in ABEC villages continue primary education after going to an ABEC, while in comparison, 38% of boys and 19% in villages with a formal primary school reach the second cycle of primary education. This stresses the margin of improvement the ABE program could achieve in terms of attendance in the coming years.

In terms of cognitive development, the multivariate model showed that going to an ABEC significantly increased cognitive performance. Based on a mathematics test that categorized children in seven different levels (from innumeracy to highly skilled), going to an ABEC increased children’s mathematic skills by about half a category in comparison to children that do not attend any primary school. The same can be said using the vocabulary test (number of words recognized) and the literacy test (ability to read a simple text): going to an ABEC improved the ability to recognize words (+ 3 words from an average of 11 words recognized) and decreased the probability of being illiterate by about 7 percentage points. Yet, the impact remained larger for primary schools (+0.8 categories in mathematics, + 4 words in vocabulary and reduction of 10 points in illiteracy rate), suggesting that, primary schools still performed better in terms of cognitive development. Overall, the learning outcomes are on a low base level with only 32-37% of 10-11 year old students from ABEC and primary schools being able to read at least one word from a standard level two school text book.

Observing UNICEF supported ABECs and other ABECs, no significant differences in terms of children’s cognitive performance could be found. The reasons for the mixed results regarding




the impact of UNICEF’s support could be explained by the findings about the ABEC environment and the training of ABEC facilitators. UNICEF supported ABECs are more likely to be new or in a very good condition (44% vs 14%), in a formal school building (73% vs. 61%) and not outside under a tree (22.5% vs 26%) than other ABECs. Further, UNICEF supported ABECs have more class rooms and a significantly lower student teacher ratio than other ABECs (50 for UNICEF supported ABEC vs 66 for other ABEC and 43 for formal primary). Additionally, there is a higher incidence of water being present for drinking and washing hands. The same is true for toilet facilities and classroom equipment like proper seating and desks for the students. On the other hand, government ABEC had a higher textbook-student ratio and teachers that have more teaching experience, receiving a higher salary and formal teacher training.

In conclusion, the evidences gathered showed that UNICEF supported ABECs have better infrastructure than non UNICEF supported ABECs, but at the same time had teachers with lower qualification and less experience. This pointed out the need to focus on improving the quality of the facilitators. Since all ABECs are run by the government and not by UNICEF, the lower teacher qualification can only be explained by UNICEF’s mission to reach the most vulnerable children and remote communities. These communities faced great challenges attracting qualified teachers. This had a negative impact on learning outcomes.

Relevance, Efficiency and Sustainability

An important question of the evaluation was the relevance of ABE curriculum to the learning needs of the communities. The Regional Education Bureau (REB) officials are responsible for contextualizing and translating the curriculum and developing the necessary teaching material. Encouragingly, efforts had been made to translate the curriculum into regional languages and shorten the four year curriculum into three years to adapt to the needs of local communities. Yet, the analysis of the interviews also revealed little coherent stand among Ethiopian stakeholders on the relevance of a condensed ABE curriculum. The federal Ministry requested the adoption of the formal primary school curriculum instead of the ABE condensed curriculum. At this stage, two different curricula exist at local level: the condensed ABE curriculum and the formal primary school curriculum which is favored by the federal government and adopted by many ABE facilitators. Efforts to contextualize the curricula are still needed together with consideration to develop a distinct ABE Curriculum extending to the second cycle of primary school.

According to key stakeholders at regional and woreda level and from UNICEF, the insufficient priority given to ABE at regional and federal levels constrains the sustainability and full quality delivery of ABE as a mainstream education modality for these marginalized communities. To achieve sustainability it must be guaranteed that the authorities take over ownership and have the capacity to continue after UNICEF funding ends.

To support the quality of ABE, UNICEF’s support was directed to the development of human capital at REB and woreda level, with capacity building interventions in various thematic areas: mainstreaming of SIP principles, EMIS/GIS system training and improvement of formative/continuous assessments. However, the sustainability of these activities was undermined as there was high staff turn-over which prevented sustainable capacitated human resources and internal continuous professional development. The limitations of the accountability and the M&E system, shared during interviews, are hurdles to the effectiveness and potentially the sustainability of ABE.

At the local level, beneficiaries and local officers’ interviews revealed that ABE’s main achievements in terms of sustainability could be found in the domain of community mobilization and increased social demand for quality education. The Center Management Committees (CMCs),




together with the ABE facilitators and the cluster supervisors participated in disseminating awareness-raising messages to the community. Yet, the capacity of CMCs to sustainably manage ABECs was limited because of the lack of local resources and qualifications, which were not complemented with the provision of sufficient CMC trainings. Decisions taken at community-level remain symbolic – namely, the choice of the location, back-to-school campaigns, the monitoring of out-of-school children – but were not enough to raise resources and sustainably support the enhancement of the quality of education.

Lastly, the budget modalities employed to finance ABE must be sustainable to ensure that the resources can be turned into maximum outputs, outcomes and impacts. The evaluation focused on UNICEF’s support to ABE, based on a review of UNICEF’s Annual Work Plans in Afar, Oromia and Somali. The review showed an evolution of UNICEF’s support away from construction and furnishing of ABECs towards soft components (training, capacity building, awareness raising). This evolution was justified by the need to rationalize public spending and to increase the ownership of national authorities on ABE. This also operationalizes the urge to concentrate resources on quality improvement needs, assessed in terms of quality of teachers, the need to upgrade and expand the upper cycle of primary school.

Recommendations

The evaluation provides a number of short-term, medium-term and long-term recommendations. The recommendations are addressed to the Ministry of Education, UNICEF Ethiopia and stakeholders at national and local levels, to engage all major stakeholders in an ongoing effort to improve ABE and to bring quality education to rural pastoral communities.

Short-term Recommendations Addressee

• To ensure long-term success and improve learning outcomes it is essential that MoE take over full ownership for ABE as a primary education modality. That means introducing the ABE teaching and modalities into the revised and endorsed Pastoralist Education Strategy.

MoE

• Efforts should be dedicate to strengthening the supervision and monitoring of ABECs. This includes conducting frequent supervisory support, increasing the financing of the existing resource centers, and strengthening capacity building for cluster supervisors and Woreda education officers.

MoE &

UNICEF

Ethiopia

• The performance accountability system should be reinforced with the development and regular use of professional performance assessment. Criteria and processes should be developed and approved by a committee at Federal level, including UNICEF and the REBs.

MoE

• To improve learning outcomes for ABE students the National Learning Assessment test of grade 4 learners from the formal primary school system should be extended to ABEC. This would also increase ownership by MoE for the ABE system

MoE

Medium-term Recommendations

• To improve primary school completion rates and the learning environment, the implementation of ESDP IV/V strategies for hard-to-reach children should be further implemented

MoE, REB

• Extended support to ABE facilitators should be prioritized via adequate MoE, REB,




teacher training on active teaching methods, professional assessment and supervision and experience sharing among teachers

UNICEF

• Provide context-specific incentives for the ABE facilitators. This can include, for example, the opportunity to upgrade to formal teachers

MoE, REB

• Improve the physical environment of the ABEC as per the standard, making schools child friendly in all aspects

MoE, REB

• Supply essential teaching and learning materials including teaching aids, related reference books, workbooks

MoE, REB

• Upgrade some selected ABEC from lower primary (level 1-4) to upper primary (level 1-6) as many ABECs end at level 4. This situation makes it difficult for children to find places for further learning. Upgrading will make the learning place close to the children’s home. This will enable pastoralist children to complete primary education easily and be prepared to start their secondary school learning.

MoE, REB

• Establish small multi-grade approach teaching so that more children have access to a primary school within close proximity to their home

MoE, REB

• Based on the assessment that some pastoralist communities still perform seasonal migration there is a clear need to promote and provide mobile schools as well as institutionalize and scale up the Networking Card System in all pastoralist districts.

This should include advocacy, planning, procurement and training at the

woreda and community levels. UNICEF is best placed to advocate for this at the

federal and regional levels with the support of local NGOs collecting the good

practices and lessons learned at the local level. Training on the Learner

Networking Card could be led as an additional module to the on-job facilitator

training and to the communities via cluster supervisors and woreda education

officers in partnership with local NGOs.

MoE, REB,

UNICEF

• Contextualization of the ABE curriculum should be continued based on the formal primary school curriculum together with a consideration to develop a distinct ABE curriculum extending to the second cycle of primary school. Contextualization should include local implementing partners, representatives of communities, and ABE teachers/facilitator. Local education research institutions should be included to add research evidence to demonstrate how a child- and context-sensitive curriculum can be developed, meeting the quality of the formal primary education.

MoE,

REBs

Long-term Recommendations

• In order to reach universal school attendance in remote rural areas and pastoral communities, incentives to attend ABEC / primary schools will have to be increased and poverty must be addressed in a more holistic way

MoE, REB,

UNICEF




2 INTRODUCTION

The impact evaluation on the effectiveness of Alternative Basic Education in Ethiopia was funded by UNICEF and conducted to inform the Ministry of Education of Ethiopia and UNICEF Ethiopia. This final evaluation report presents the findings, conclusions and recommendations of the evaluation. The first part describes the scope and the objectives of this evaluation, evaluation questions, judgment criteria and indicators. It also includes key elements on socio-economic, politic, demographic and programmatic context. The second part entails the evaluation methodology. The third, the main part of this report, presents evidence-based findings responding to the evaluation criteria. In a fourth section, conclusions and lessons learned are presented. Finally, the last section of the report presents recommendations supported by evidence and conclusions and review with the involvement of the relevant stakeholders.

3 OBJECT, SCOPE AND PURPOSE OF THE EVALUATION

3.1. PURPOSE AND SCOPE

The overall objective of this evaluation is to assess the impact, effectiveness, efficiency, relevance and sustainability of UNICEF’s support to the Alternative Basic Education (ABE) system. This will be done to shed light on decision-making for the future design and implementation of ABE support to Government of Ethiopia (GoE) and for wider learning.

The scope of this evaluation encompasses the ABE program from 2010-2016 and focuses on three regions supported by UNICEF and the Ministry of Education (MoE) of Ethiopia, namely Afar, Oromia and Somali. This evaluation takes place at an important point in time after the ESDP V was launched in 2015. It will be accessed if the ABE program can achieve the ESDP V objectives of broadening access for out-of-school children and in particularly children from pastoralist communities.

Quantitative and qualitative field data collection was implemented in the three regions to assess the modalities of ABE and its impact on children’s access to education and their learning outcomes. The choice and justification of the sample selection is described in detail in section 4.1.

The evaluation aims for strong external validity in order to enable transferability of the evaluation findings to other regions of the country.

3.2. BACKGROUND AND CONTEXT

Ethiopia is the second most populated country in the African continent, with a population of 99 million (CSA 2015) inhabitants that make up of over 90 ethnic and linguistic groups, according to the Education Sector Development Programme V 2015/2016 of the Federal Ministry of Education (MoE 2015a). With an annual growth rate of 2.6%, the population is largely composed of youth, with 44% under the age of 15 years old (MoE 2015a). According to the Welfare Monitoring Survey 2011, the literacy rate is 46.8% (CSA 2012). Furthermore, a large share of the population (80%) resides in rural areas (ECOSOC 2016), which makes Ethiopia one of the least urbanized countries in the world (MoE 2015a).

Ethiopia has seen encouraging economic improvements over the past twenty years. In 2014, the GDP increased by 10.3% (World Bank 2015). This is reflected in the GDP per capita, which increased from USD 300 in 2010 to USD 489 in 2013 (United Nations Statistics Division 2016). Agriculture remains a major source of income for the majority of the population despite a slight decrease of its share from 43% in 2012/2013 to 40% in 2013/2014 (MoE 2010, MoE 2015a). Ethiopia’s long-term vision is to become a lower middle-income country by 2025 (National




Planning Commission 2015). This vision is spelled out in its national development policy, currently named the Second Growth and Transformation Plan which follows the Growth and Transformation Plan 2010-2015 and builds on the overarching objectives to move towards a middle-income economy by 2025. The implication of the second GTP on the education sector translates into a need to further expand access to quality basic education to build skilled human resources.

AFAR

The Afar Regional State is inhabited by 1.7% of the national population according to the Central Statistics Agency (CSA 2008). The latest census reports show that Afar has a total population of 1,411,092, of which 86.6% lives in rural areas. The Afar region mainly consists of lowlands, with a predominately hot and dry semi-desert climate (Samara University 2009). The economy is predominantly based on a pastoralist economy. Main economic activities are subsistence-based animal husbandry, mixed-livestock management including but not limited to camels, cattle goats and sheep, which often rely on seasonal and periodic mobility (Samara University 2009). Consequently, many Afar people migrate seasonally in search of grazing places and water sources for their herds (Dyer and Engdasew 2016). Afar is one of the four regions classified by the Ethiopian government as an Emerging Regional State (DRS).

SOMALI

The Somali Regional State is located in the east and southern part of Ethiopia. According to the 2007 Census (CSA 2008), the Somali region has over 4,445,219 inhabitants, most of whom (96.23%) belong to the Somali ethnic majority. Somali is the main working language of the state and is spoken by 95.9% of the Somali population. The topography of Somali is dominated by lowlands plains at 80%, most of the region lying below 900 meters above sea level. The climate in Somali is 80% arid or semi-arid with variable, low rainfall distribution. Similar to the Afar region, Somali economy largely depends on livestock production and management (Dyer and Engdasew 2016). Somali is one of the four regions classified by the Ethiopian government as an Emerging Regional State (DRS).

OROMIA

The Oromia Regional State occupies the largest share of Ethiopia’s territory and population, with a total population of 26,993,933 inhabitants, according to the 2007 Census. About 87.71% of the population lives in rural areas and the majority (87.83%) belong to the Oromo ethnic group (CSA 2008). The region is characterized by great geophysical diversity (Geleta 2010), with highlands, plateaus, steep mountains, and lowlands plains in the east. Three types of climate characterize the region: the dry climate, tropical rainy climate and the temperate rainy climate (Government of Ethiopia 2016). The inhabitants rely largely on agricultural activities, such as small crop production (maize, teff, coffee), which serves as their main source of income. Around half (51.2%) of the crop production of Ethiopia is accounted for in the Oromia region.

EDUCATION IN ETHIOPIA

The education sector in Ethiopia has achieved a significant progress in recent decades in terms of access to primary education. The Government of Ethiopia has made a significant investment in improving access to and quality of educational services. This is particularly reflected in measures taken, such as the abolishment of school fees; which took place after the adoption of the new Education and Training Policy in 1994. The percentage of spending on education of total government spending almost doubled between 1990 and 1995 and then gradually increased from 14% in 1995 to 24% in 2009 (ODI 2010). More recently, the introduction of mother-tongue instruction and gender mainstreaming have been implemented to further reduce barriers to education (UNESCO 2016).




The number of children enrolled in primary school rose to more than 18 million in 2013/14, whereas only 3 million enrolled children in 1996. During the same time period, the number of primary schools, including ABE schools, also increased substantially, from 6,958 in 2000/2001 to 32,048 in 2013/2014.

The Gross Enrollment Rate (GER) for primary schools has risen substantially across all regions from 57.4% in 2000/2001 to 102.12% in 2015 (UNESCO 2016). This positive trend has been equally observed for both sexes. Net Enrolment Rate (NER) also shows a positive trend, increasing substantially from 48.8% in 2000/2001 to 85.59% in 2015 for both sexes (UNESCO 2016). In terms of gender parity enrolment, the gender parity index (GPI) is equal to 0.91 for Net Enrolment Rate at Primary levels (Grade 1 to 8) and even 0.96 for secondary education in 2015 (World Bank 2016). Since the GPI was only 0.73 (0.58) for primary (secondary) education in 2003, these numbers indicate a fast improvement of girls’ school participation. Nevertheless, the participation rate of girls is still lower than for boys in lower and upper primary education.

Even though there have been huge improvements with respect to access to primary education, universal attainment has not been reached. A primary school completion rate of 54.4% (UNESCO 2016) indicates that many students leave the education system prematurely. This emphasizes the importance of further reducing dropout and repetition within the school system, particularly in early years where drop-out rates stand the highest. Due to the low completion rates of primary education, gross enrollment rates for secondary education have changed only slightly and remain lower than the targeted rates (MoE 2015).

The Ministry of Education (MoE) has made strong efforts to provide access to primary education for out-of-school children through school construction, the provision of alternative education modalities (ABE centers, para-boarding schools, mobile schools) and through the special support program to the four emerging regions. The MoE designed the Alternative Basic Education program in 2006 to improve the access to education for children who are unable to attend the four year formal lower primary education. The national ABE Strategy provides guidance for the establishment of new ABE centers (ABECs) and in parallel the extension and transformation of ABECs into formal primary schools. ABE schools are primarily intended to address the diversity of needs and contexts when providing access to education, particularly to out-of-school children. ABE has contributed considerably to the improvement of enrollment rates for disadvantaged and previously under-served ethnic groups as well. Since the national ABE strategy, was founded in 2006, new ABECs have been established, and existing ABECs will be upgraded or turned into regular schools (MoE 2015). By 2014-2015, the number of ABECs in Ethiopia reached 3,932 (MoE 2016).

PRIMARY EDUCATION SYSTEM IN ETHIOPIA

Current structure of the primary and secondary education system in Ethiopia

The Ethiopian education system is structured through a multi-tiered structure. From pre -school, primary and secondary education, there is also technical, vocational training and education, and post-secondary higher education. The General Education in Ethiopia is comprised of :

� First cycle primary education, Grades 1-4 in formal schools or Level 1-4 / 1-3 in Alternative Basic Education1;

� Second cycle primary education, Grades 5 to 8;

� First cycle or ‘General’ secondary education, Grades 9 -10;

1 While in some regions the ABE system has moved to providing levels 1-4, other ABEC still provide levels 1-3 in particularly for out-of-school children.




� Second cycle secondary education, Grades 11 to 12

Figure 1 Primary and Secondary Education System of Ethiopia

Source: MoE 2015

The government decentralized structure is reflected in the educational sector and in the executive responsibilities for ABE (Anís 2007, Onwu and Agu 2010, MoE 2015). The devolution of discretionary power to administrative sublevels results in a certain amount of regional variation regarding strategic planning and implementation capacity (Onwu and Agu 2010). The Federal Ministry of Education is responsible for the overall policy (supervision, coordination, goal and standard setting, quality assurance, among others) though it primarily takes “a supportive role” as to operations (MoE 2011) for primary and secondary education.

The executive responsibilities for ABE are led by the Regional Education Bureaus (REBs), with involvement of the other levels of government (woreda education office [WEO] and kebele education office level).

SOCIO-GEOGRAPHIC CONSTRAINTS TO EDUCATION

Around 12 to 15 million people are estimated to live in pastoralist groups in the eight regional states of Ethiopia. In Somali and Afar, pastoralist communities respectively represent 37.97% and 29.43% of the regional population, according to the 2007 Census (CSA 2008). In Oromia, the situation is more nuanced. Most of the rural population consists of settled agriculturalists, however some districts are scarcely populated and inhabited by pastoralist and agro-pastoralist herder communities.




Pastoralists are people who live and derive most of their food sources and income from raising domestic livestock, such as herding or livestock rearing. Semi-pastoralism applies to the engaging in the same activities for only a part of the year. In comparison, agro-pastoralists combine raising livestock with crop farming. Regardless of the lifestyle there is always a need for mobility. These groups can be fully mobile, meaning they have no recognized place of residence. However, most of them have a recognized place of residence from which they move intermittently. Settlement patterns are characterized by these seasonal movements in search of lusher pastures and water for livestock and community (Carr-Hill 2005) but also periodic movements of livestock dealers and the by-products they bring to markets. These frequent migration movements are also enhanced with the occurrence of natural disasters such as floods, frequent droughts, water and food shortages, and violent conflicts (UNICEF quoted by IRIN 2013, MoE 2015).

In Ethiopia, geographical and social factors also related to pastoralism have posed specific challenges achieving universal access to primary education. The scattered population and remoteness of villages, coupled with seasonal and periodic movements of pastoralist and demi-pastoralist communities make it difficult to deliver quality education and attract qualified teachers and staff (SDI 2011, MoE 2015).

On the demand side, from a young age children are involved in labor activities like cattle grazing and girls are at risk of early child marriage which often ends their formal education. In general, parents’ poor literacy levels and low perceived benefits from their child’s participation in education can explain most of the high dropout rates in the regions (SDI 2011, MoE 2015, UNDP 2015). On the supply side, often poor accessibility / a large distance to school are important obstacles to education participation.

The highest percentage of out of school primary children is documented in the two predominantly pastoralist regions Afar (69.6%) and Somali (53.9%) (OOCS, Study Report 2012, EMIS Data 2010). By contrast, Oromia shows a smaller percentage of out of school children at the primary age (21.7%) but it is also the region with the highest absolute number of out of school children at the primary age (1,396,848 children).

POLICY FRAMEWORK - THE ETHIOPIAN EDUCATION SECTOR AND ALTERNATIVE BASIC EDUCATION

The Education Sector Development Programme (ESDP) was initially launched by the Government of Ethiopia in 1997/98. ESDP was initially inspired by the National Educational

and Training Policy of 1994 and all subsequent ESDP plans are consistent with this policy base. ESDP sustains the long-term vision of transforming Ethiopia into a lower middle-income country which requires increased access to quality education to improve literacy and numeracy level of inhabitants. Under this ETP policy umbrella, the third Education Sector Development

Programme (2005/2006-2010/2011) mainstreamed education for pastoralists as a priority in all sub-sectors of the educational system. This evolution stems from the assessment that a large share of the Ethiopian population living in rural and dispersed areas and the existence of many pastoralist and agro-pastoralist groups creates challenges for the education sector in providing equal access to quality education at all levels.

The National Pastoral Education Strategy, developed in 2008, has guided this sector focus since ESDP III and identified strategies to promote primary and secondary education. “The

Pastoralist Education Strategy (MoE 2008) also proposed a range of modalities to improve access,

via: institutionalizing different educational modalities (Alternative Basic Education [ABE], mobile

schools, Para-boarding schools, hostels, distance education, adult education, education radio

Programs and formal primary education). It also called for improving the quality and relevance of

primary education; and for addressing pastoralists’ cultural, environmental and economic

constraints” (MoE 2017).




At the time of the evaluation, the National Pastoralist Education Strategy is currently under revision and is expected to come into effect in 2017.

The Government of Ethiopia operated a shift of primary education priorities adding equity, quality and efficiency to access (SDI 2011). This shift is reflected in the ESDPs from 2010 onwards. ESDP IV and V both address the need to target hard-to-reach children, “most of whom

are in the emerging regions or belong to specific groups: the pastoralist, semi-pastoralist and

indigenous groups, children with special needs and vulnerabilities” (MoE 2010).

ESDP IV spelled out two sets of strategies for hard-to-reach children. The first part of the strategy proposed to expand the number of primary schools with a special emphasis on reducing the distance between school and home and to upgrade ABECs into full primary schools. To continue the efforts and to expand the access to education more ABECs were created. ABECs were considered an interim solution to provide access to education for children in hard to reach areas. This strategy intended a gradual phase out of ABECs, with their transformation into formal primary schools and the introduction of other solutions to address the needs of those children who still cannot access formal schooling. As part of a 2nd set of strategies, the use of multi-grade classes, provision of special support programs, scholarships and school feeding were put forward.

In 2015, ESDP V was launched as the central strategy document for educational development in Ethiopia, with an action plan for 2015/16-2019/2020. ESDP V defined one of the key objectives as “broadening access for out-of-school children, with a focus on adolescent

girls, children from pastoralist communities and children in emergency contexts” (MoE 2015a).

As a means of achieving this objective, the ESDP V seeks to improve retention rates in the first cycle of primary school and increase transition into the second cycle of primary school. A possible strategy proposed in line with this objective is to extend the number of grades offered in primary schools and ABECs and upgrade the ABECs into formal primary schools. Where they currently offer two or three levels2, they would be expanded to a full four levels. Selected ABECs will then be extended to level six (which will be equivalent to Grade 6 of formal education), while other ABECs will be promoted to formal primary schools. A second set of strategy include the development of an ABE curriculum and materials for the additional two levels. This was to be prioritized in the first year of ESDP V.

The evaluation mission took place during the revision of Strategy for Promoting Primary Education in Pastoralist Areas (March 2017) to reflect the intentions comprised in the ESDP V. Therefore, it does not include the new Strategy in the scope of the assessment. However, it does take into consideration the 2008 National Pastoralist Education Strategy as well as results from situation analysis of pastoralist education performed in 2016 to inform the Strategy revision (Dyer and Engdasew 2016).

One issue is that the expansion of access to education has stressed gaps between available funds and the need for improvement and to maintain the quality of education (MoE 20083). Therefore, MoE has operated a shift towards quality of learning environments since 2008 with the implementation of the General Education Quality Improvement Programme (GEQIP) as part of the ESDP III. GEQIP 1 and 2 have included support to Alternative Basic Education Centers. Indeed, the expansion of the education system, in particular through high proportion of recurrent costs, mostly teacher salary has constrained the budget for other types of activities such as training, textbook provision but also assessment, supervision.

2 Most ABECs offer three levels which corresponds to grade one to four of regular formal lower primary education. 3 GEQIP Plan.




The first phase of GEQIP (2009-2013) has been rolled out and the second phase of GEQIP (2013-2018) is ongoing.

ALTERNATIVE BASIC EDUCATION

It should be noted that universal education cannot be achieved by relying on formal education alone. In response, since 2006 the MoE in Ethiopia has designed and implemented a basic education program that is flexible and responsive, and delivered outside formal schooling. Under the National Pastoralist Education Strategy (2008), the MoE developed the Alternative Basic Education Implementation Guidelines and the Alternative Basic Education Standards in pastoralist and agro-pastoralist areas (MoE 2011), integrating ABE Centers into the educational system as an alternative path to formal basic education for the first cycle of primary school (Samara University 2009 E.C.).

Although a National Alternative Basic Education Strategy was drafted, this was not endorsed by MoE and ABE has remained as part of a broader mix of primary educational modalities to expend access to primary education, under the 2008 National Pastoralist Education Strategy. Within the revision of the National Pastoralist Education Strategy several measures have been proposed to increase the role of ABE in the delivery of quality basic education.

Originally following an ABE curriculum which was designed as a condensed form of the formal primary school curriculum, ABE schools are now directly following the formal education curriculum. Similarly, while Alternative Basic Education was first conceived as a three-year accelerated cycle, the ABE school year is now aligned with the formal four-year cycle. Minimum qualification requirements of ABE teachers/facilitators were raised from grade 8 to grade 10 under the revision of the National Pastoralist Education Strategy.

In addition, further efforts need to be made to deliver quality accelerated teacher program trainings over the summer to ABE facilitators. The major difference which exists between ABE and formal school modalities, is the flexible delivery in terms of adaptation of times of learning to accommodate local conditions and needs of learners and parents from the pastoralist/semi-pastoralist communities.

ABE Goals. “Alternative Basic Education Program is “an organized basic education program

mainly flexible and responsive in character and is delivered outside formal schooling. It is an

alternative to formal education and has a four-year cycle. It is equivalent to the first cycle formal

primary education (Grade 1-4). Under this Program, education is provided for out-of-school

children. “(MoE 2003 EC).

ABE beneficiaries. ABE is meant to improve access to quality basic education for children excluded from or beyond the reach of formal education, which according to the ESDPs, concerns especially “pastoral and agro-pastoral region children” (MoE 2010) and “disadvantaged and

previously under-served ethnic groups” (MoE 2015). It targets children at the age of being enrolled in the primary cycle of primary school (7 to 10 years old) but also over-age children with ranges from 11 to 14 years old (MoE 2011).

Flexible delivery

To suit the special needs of children from (semi-/agro-) pastoralist and other hard-to-reach groups, ABE is delivered in a flexible manner (MoE 2008 E.C.). Class timetables are agreed upon with the community and may include a shift system and relatively short class duration of 30-35 minutes (Onwu and Agu 2010).

Most ABE activities take place in Alternative Basic Education Centers (ABECs), built with low-cost construction materials and drawing on community support. ABE includes the option for




overage children (11-14) to attend a three-year program (MoE 2017). Flexibility is also to be achieved through the learner networking card, piloted in Somali, which allows children to join a different ABEC wherever s/he moves and through the recruitment of teachers/facilitators who have close bounds to the community (Getachew 2006, Anís 2007, Dyer and Engdasew 2016).

Learner Orientation

The learner orientation concept relies on curricula and teaching materials (books and facilitators/ teachers guides) which are adapted to children’s day-to-day reality and way of life (MoE 2003 EC, Dyer and Engdasew 2016). Based on the specific social context, teachers who have close ties to the community and its leaders are invited to share their indigenous knowledge and skills to learners in the classrooms (MoE 2008 EC). Curriculum Guidelines are provided at the Federal level, translated into regional languages, and adapted to the regional context by the Regional Education Bureaus.

Inclusive Education

ABE seeks to address the needs of disadvantaged children in order to promote social inclusion. In this regard, inclusive education is a non-discriminatory approach which provides opportunities for all learners, with or without disabilities, to learn together with available educational provision and in settings with an appropriate network of support services (MoE 2011), with specific focus on gender equity, access for children with disabilities, and marginalized social groups.

UNICEF’S SUPPORT TO ALTERNATIVE BASIC EDUCATION

Education is one of the core program areas of UNICEF Ethiopia: “Together with the Ethiopian

Ministry of Education, [UNICEF Ethiopia] works to address the causes of lack of access,

participation and completion of basic and lower secondary education including challenges of

enrollment, gender and geographical disparities, quality, completion and dropout rates, school

environment and capacity to implement sector plans “(UNICEF Ethiopia 2016).

UNICEF Ethiopia, together with the MoE, has been involved in improving access to basic education to the most vulnerable children and localities, in particularly those from (semi-) pastoralist communities. These goals are also set by UNDAF’s outcome on equitable access and quality primary education for boys and girls. UNICEF Ethiopia’s 2008-12 and 2012-15 Country Program aimed at contributing to the following results at national and regional levels:

i) The MoE, REBs, WEO and schools have the capacity to improve sector performance, develop and interpret better policies, and build systems to improve access and learning achievement;

ii) Children from the prenatal stage to age seven living in at least 120 lagging woredas (districts) are supported with integrated health, nutrition, protection, early stimulation and school readiness intervention and are ready to learn;

iii) All out-of-school girls and boys living in selected lagging woredas, including those affected by emergencies, have access to, participate in and complete general education of high quality;

iv) Girls and boys living in at least 120 lagging woredas demonstrate core learning competencies at grade 4 through the School Improvement Plan.

UNICEF supports the Government of Ethiopia in the delivery of its national plans and policies, which for the education sector is that of the ESDP. In that regard, UNICEF provides direct




support to the Federal Ministry of Education’s strategy for Alternative Basic Education and to Regional Education Bureaus in the 9 regions and 2 chartered cities of Ethiopia.

Documentary review of UNICEF Annual Work-plans sheds light on an orientation shift of UNICEF support: while ABECs construction and furnishing was a priority in Afar, Oromia and Somali (AWPs 2006, 2007 E.C.)4, a shift occurred towards a larger share of soft components such as capacity building trainings targeted to various actors (REB, Woreda Education Officers, ABE facilitators/teachers, cluster supervisors and Center Management Committee members) for planning, management, monitoring but also to upgrade of ABECs into second cycle primary school (AWP 2008 E.C., 2009 E.C.). UNICEF staff consulted by the evaluation team indicated that the reason for this shift is the non-sustainability and low cost-effectiveness of the construction and furnishing of new centers. Therefore, there has been a shift to investing in soft components that allow enhancing ownership of the program by national stakeholders.

3.3. THEORY OF CHANGE

The theory of change (ToC) is the logical chain of changes assumed behind the implementation

of an intervention. It runs through the whole process between ABE’s inputs, outputs, outcomes

and the reaching of the ultimate goals.

In the absence of a program ToC, the evaluation consultants reconstructed a ToC based on their

understanding of the Education Sector Development Programme V (2016-2020) and Alternative

Basic Education, from the 2008 National Pastoralist Education Strategy as well as from meetings

with UNICEF staff in Addis Abeba in October 2016 and January 2017. Further program

documentation, such as the Alternative Basic Education Strategy5 (MoE 2009 EC), were

consulted to inform the design of the ToC.

Figure 2 presents the Alternative Basic Education ToC, where the activities are in the first

column, the outputs in the second, the short term outcomes in the third, the intermediate and

long term outcomes in the fourth and the impacts in the fifth and last column.

The activities were the inputs/interventions carried out by MoE and UNICEF and included

activities like the creation of new ABECs or the transformation of existing ABEC into formal

schools, teacher training and awareness raising, opening multi-grade classes and many more.

These activities should than result into outputs like: increased school capacity and trained

teacher and ABE facilitators,

Following from these outputs are short term outcomes including the increased access to

education, having teachers with improved skills for teaching and the development of a

curriculum that is adapted to the needs of the pastoralist community. These short-term

outcomes than translate into medium and long-term outcomes like: increased primary school

attendance and enrollment rates, students’ learning outcomes are strengthened which in return

will lead to higher transition rates to secondary education. Finally, the expected impact is an

increased skilled labor force, higher youth participation in Ethiopia’s political, economic and

social life, as well as increased well-being of the youth.

4 Cost breakdown shows costs of ABEC construction, WASH facilities, furnishing, and educational material items in an ABEC. Additionally, a slight share of the budget is dedicated to facilitator and Center Management Committee members training. 5 This document was not endorsed by MoE. It enabled however the consultant team to identify key features of ABE to design the ToC.




Figure 2 Theory of Change


4 IMPACT EVALUATION METHODOLOGY

A rigorous impact evaluation allows to evaluate whether the program of interest generated the

expected changes and to measure its magnitude. Depending on how the impact evaluation is designed,

it can help to understand the mechanisms that led to the change, which is an important feature if the

program needs to be reproduced elsewhere.

A good impact evaluation essentially relies on the quality of the chosen counterfactual situation. The

counterfactual situation corresponds to what would have happened if the ABE interventions had not

been implemented. Specifically, we would like to know: What would have happened in the villages

where the ABECs were implemented if the ABE had not been implemented? Would children have

attended primary school anyhow, for instance by going to another formal school? Would they have

acquired the same educational skills (reading, calculus…)?

Of course, since at one point in time one specific village has either benefited or not from an ABE center,

by definition, the counterfactual situation is just a conceptual view and will never be observed. As a

result, the task is to construct or mimic the counterfactual by selecting control villages that would

behave as close as possible to what would have happened to the ABE villages in the absence of the

program.

Given that ABE centers’ locations were not assigned randomly and that no baseline data was collected

prior to the beginning of the intervention, a randomized controlled trial is unfortunately not feasible in

this case. The impact evaluation design was based on non-experimental methods. In a nutshell, such

methods rely on selecting a group of control villages, not affected by the ABEC and using households,

parents and kebele data to account for the initial differences between the ABEC villages. The impact

was then assessed by comparing the performance of the children in ABEC villages to the ones in non-

ABEC villages. The following options have been chosen:

4. Comparing villages with UNICEF-supported ABECs against those with other (government)

ABECs,

5. Comparing villages with UNICEF-supported ABECs against those with formal primary schools,

6. Comparing villages with UNICEF-supported ABECs against those with no other primary school

education facility.

To make sure that villages were comparable control villages were chosen from the same kebele than

their treatment counterpart. The finalized evaluation strategy is based on a mixed-methods approach

involving:

1. A quantitative impact evaluation based on primary data collected in the field;

2. A qualitative approach based on qualitative field data;

3. Cross-checking the results from (1) and (2), as well as using secondary data to validate the

findings.

The selected approach consisted mainly of quantitative surveys with randomized sampling which

generated a large amount of quantitative evidence and ensured strong internal and external scientific

validity. In addition, the review and the quantitative analysis of secondary data provided a sound

understanding of the effectiveness and efficiency of the program throughout the implementation




years. Qualitative methods were useful to illustrate and consolidate or nuance the findings from the

quantitative study (Patton, 1980).

The mixed methods approach enabled to triangulate across data collection and analysis methods in

order to enhance the evaluation credibility and take the contextual complexity into account. The

triangulation of the evidence gathered from diverse methods and sources to determine key results

would lead to strong conclusions and recommendations. It is noteworthy to say that non-

corroborating evidence is not a sign for weak analysis but on the contrary reveal nuances and

unanticipated effects that the evaluators strove to explain.

4.1 DATA COLLECTION

There are a variety of modalities by which alternative education is delivered to students and communities. The focus of this study are ABECs, excluding mobile schools, para-boarding schools and hostels. To get a full picture on ABEC and to evaluate their impact, effectiveness, efficiency and relevance for the education system an extensive data collection was undertaken by using both quantitative and qualitative interviews.

Quantitative data collection was conducted over a period of about 10 weeks from the beginning of February until the middle of March 2017. To ensure that high quality data could be collected and the instruments are relevant as well as accurate, some pretesting of the data collection tolls was conducted in December 2016. In addition, further testing of the instrument was done at the end of January 2017. The reginal scope can be seen in Figure 3 and 4. These three regions were selected to have a mix of flatlands and hill regions, with an overall focus of reaching children living in pastoralist regions.

Figure 3: Sampled Woreda in Afar and Somali region6

6 In Figure 4 and Figure 5 the woredas where the quantitative and qualitative data collection took place are displayed in green.




Figure 4 Sampled Woreda in Oromia region

Sampling strategy: As described before, analyzing the effect of the ABEC run by the government and

supported by UNICEF crucially depends on building an acceptable counterfactual scenario, which is

what would have happened if:

1. The ABEC did not exist in a village and pupils had no close access to primary education.

2. There is an ABEC run by the government but without the support of UNICEF.

3. There is a formal primary school in the village.

The main advantage of this strategy is that it allows analyzing the contribution of having a UNICEF

ABECs, a government ABECs, a formal primary school or no school at all within similar type of villages.

The sample selection has been performed respecting as much as possible the proportion of ABECs and

primary schools in each woreda. Hence, the analysis of this report is representative at the local level.

The four types of sampling criteria

UNICEF-supported ABEC These ABEC are run by the government but received some form of direct support from UNICEF e.g. construction and furnishing of ABECs.

Government ABEC These ABEC are run by the government and receive no direct support from UNICEF (but could have received indirect support e.g. through capacity building training of woreda Education Officers, ABE facilitators, etc.).

Primary schools Traditional primary schools run by the government.

No school villages These are villages (smaller parts of the kebele) which have no own ABEC or primary school.

For the sample selection, 115 kebele with UNICEF-supported ABEC, kebele with formal primary

schools and those having government ABECs have been randomly selected. The result of this sampling




can be seen in Table 1. Most kebele and schools were sampled in Oromia (43), followed by Afar (39)

and Somali (33).7

Table 1: Type and number of schools visited for the quantitative data collection

Region UNICEF-

supported

ABEC

Government

ABEC Formal

primary

school

Total

Afar 17 11 11 39

Oromia 21 12 10 43

Somali 3 23 7 33

Total 41 46 28 115

In total this sampling strategy provided a sample size of 2,328 households with 14,639 household

members.8 For the core group of 7-11 year olds that are eligible for the first cycle of primary schooling,

the total sample size was 3,601. This large sample has enabled the identification of different schooling

effects and contributed to obtain regional as well as sub-group analysis by gender and socio-economic

characteristics.

Table 2: Number of individuals visited in each region by school type (in village)

Region Age group ABEC Primary School No School Total

Afar All 1,759 1,037 1,198 3,994

7-11 462 258 304 1,024

Oromia All 2,816 1,738 2,293 6,847

7-11 646 421 523 1,590

Somali All 1,892 499 1,365 3,756

7-11 455 117 415 987

Total All 6,472 3,274 4,893 14,639

7-11 1,563 796 1,242 3,601

Instruments: In total four questionnaires had been developed: a kebele, a school, a household and a child questionnaire.

Each kebele questionnaire has been administered with the kebele leader. This questionnaire asked about the kebele size and infrastructure, as well as all villages and schools within the village. The village information from this questionnaire was then used to sample two villages: the target (with our primary education facility) and the control village.

7 The original sample strategy was to have equal number of kebele for each region to make cross-regional comparisons feasible. However, for the Somali region, the original plan of observing 40 schools had to be revised due to the state of emergency. Furthermore, the evaluation team only managed to identify 3 ABEC that were supported by UNICEF in the Jijiga Woreda of Somali. For this reason, government ABEC are over-sampled in this region. 8 A detailed description about the household sampling can be found in the Appendix 7.




The school questionnaire contained detailed question about the school infrastructure, number of students and teachers, as well as teacher qualification. Mostly, it has been done with the school headmaster or the school facilitators. This questionnaire provides valuable information to judge the quality of the school and education received by the children.

The household questionnaire was done with each of the 2,328 households. The questions asked about the household composition, basic information for each household member like age or education, the housing condition, and assets of the household. In a separate section, detailed questions about the caregiver and the primary school aged children provide important information about the backgrounds of the children in this study. They will be core to control for unobserved heterogeneity when analyzing school outcomes. They also offer further background information on hardships and constraints that rural children face when trying to access primary education.

Finally, up to two children from each household in the core primary age group have been tested using the child questionnaire instrument. This instrument was developed to test the cognitive development of the children in the sample. It contained a math, a vocabulary and a reading section.

Limitations: In general, the main concerns of the quantitative data collection were reaching the remote pastoral communities, following the sampling strategy/protocol, translating the questionnaire to the local languages and testing for the relevance of the instruments of the study. While the purpose of this study was to evaluate the effectiveness of the ABE system bringing education to the pastoralist communities, reaching those remote communities for the evaluation team turned out to be very challenging. First, simply navigating to the right school and kebele is a difficult task as the kebele and school names in our dataset were not always known to the local authorities. Second, the only way to reach the targeted kebele and villages was by long drives using 4 x 4 vehicles.9 The third major concern was the state of emergency in the Somali region due to the draught that the region was facing when the evaluation took place. Since many households had left their communities and most schools were closed we had to completely remove the woredas Birqod, Ararso and Yocale from the sample. Given that the remaining woreda for Somali Jijiiga is close to the main capital, the sample might not be representative for the full region.

To make sure that each interview partner and household member could give accurate answers to all questions; all four instruments had to be translated into the local languages. Furthermore, the special conditions and needs of the pastoral communities made the testing of the instruments for relevance and applicability a necessity. In particularly, the child test had to be specifically developed to provide a meaningful picture of cognitive development in all three regions and would be relevant in the rural context. For example, we had to drop many items from the vocabulary test which seemed to be not applicable in the context of rural Ethiopia.

Qualitative data collection was held during the inception phase in October 2016 and during the field phase between January and March 2017, mostly in the form of in-depth semi-structured interviews, including but not limited to:

Interviews: in-depth semi-structured interviews were led with various education stakeholders at the federal, regional and woreda level, in Addis Abeba and throughout the regions. Additional interviews with operators (implementing NGOs, cluster supervisors, Teacher Training College ABE officer) and beneficiaries were organized in three selected woredas (one per region) and in six visited villages where there is a UNICEF-supported ABE center. The list of informants met is available in Annex 3 and list of visited locations is available in Annex 4.

9 In some cases, the evaluation team even had to walk for some hours by foot to get to the sampled kebele.




Focus Group Discussions (FGDs): FGDs were organized with stakeholders at federal level as well as with ABE Center Management Committee (CMC) and other members of the community in visited villages.

Transect Walks: transect walks were led in the visited villages with community members to capture information on access to the school facility and to lead further observations. Five out of six villages were visited through the transect walk approach because of low accessibility and time constraint of local informants.10

Qualitative sample selection: Three woredas were selected from the sample of UNICEF-supported woredas in the selected zones of Somali (Jigiga), Afar (Zones 1, 3 and 4) and Oromia (Bale, Borena, East Harerge, Guji). Five UNICEF-supported the ABECs and one of the government-run ABECs were selected based on the school list in the three sampled woredas.

The villages that were visited had the characteristics of agro-pastoralist communities and rely mostly on cattle raising and crop cultivation activities. Two communities, one in Oromia (Homochesa) and one in Somali (Q/Axmed), reported practicing seasonal migration. The villages that were visited benefited from the presence of an ABEC supported by UNICEF, except for one (Jiciir, Somali), which is run by the government only.

Limitations: The main limitation of these primary data collection methods is the balance between costs and benefits of participating in the evaluation of study participants. Particularly, at the beneficiary level, the research team faced obstacles due to the limited time that the participants could grant to the research team during the study. To respect their time constraints, some questions were not addressed and the interview was adapted on an ad hoc basis to address the key points of the study. To prevent this issue from causing further obstacles, the strategy included scheduling the visits one day ahead and relying on a local informant who served as the liaison between the team and the respondents.

Secondary data: In order to complement the primary data collection, documentary and secondary

data review was performed. A complete list of documents that were used is available in Annex 5. It

consists of analyzing data which was not available earlier during the evaluation process or it was

collected from the stakeholders. The secondary data set provided information on the local context of

the area and about the education sector ion Ethiopia.

Limitations: The heterogeneity and the inaccessibility of some secondary data were a challenge. In

addition, the informants' were not always available to provide the secondary data. This challenge

could be partially alleviated, with the support of a Local Qualitative Expert. There were still significant

information gaps which required extensive time from the evaluation team to clarify and refill. In the

absence of complete implementation monitoring data of the interventions, the evaluation team could

only partially assess the extent to which activities were effectively implemented.

10 Some pictures from the “Transect Walks” in Somali and Afar region can be found in Annex 10.




Ethical aspects

The evaluation sought to give a voice to many stakeholders at all levels to inform the evaluation

with a multiplicity of perspectives and critical insights. Respondents from donor organizations,

including UNICEF, as well as federal, regional and woreda officials and officers, implementers

and beneficiaries were interviewed.

To ensure the involvement of all stakeholders and secure their buy-in, letters of introduction

issued by UNICEF and the MoE were presented to various stakeholders at the regional, woreda

and the kebele level, as part of the standard procedure. To access the next level in

administration’s “information” the evaluation team secured official authorization from the

relevant administration at Ministry, regional and woreda level.

Informed consent was provided verbally before the start of each interview and FGDs and

participants were invited to sign or finger-print a written informed consent form as a form of

consent to participate in the study.

The evaluation consultants strove to incorporate gender equality and equity components as

part of the data collection methods employed, both in the questioning but also in the protocols

used. To that respect, the team strove to interview both female and male respondents.

The evaluation team paid great attention to being culturally sensitive to local customs.

Enumerator teams were hired to perform quantitative data collection and they were trained to

administer the survey in an integer, respectful manner.

4.2 DATA ANALYSIS

As described above, since no baseline data was administered, the impact evaluation was based on a non-experimental method using control villages to estimate the program effects.11 In a first step, the mean value differences between different village types was computed and their statistical significance assessed using t-tests. However, since there are many reasons to believe that villages that benefit from an ABEC or primary school are different from villages without primary school facilities, in a second step, multivariate regression models have been used. In this model, the variable of interest (e.g. primary school attendance or learning outcomes) was regressed against the village type (ABEC village, ABEC supported by UNICEF village, primary school village or no school village) or school type, controlling for variables influencing children’s school performance.

The regression model is characterized by the following equations:

�� _�� _�� _��

� ��_��

In this model �� stands for the outcome variable of interest e.g. school attendance, math score or reading score.

11 To analyze the captured primary data, the evaluation team transferred the answers from the questionnaires into Stata format. Stata is a statistical software to analyze big data sets and to run complicated econometric models.




• The �� _�� : are the different type of observed villages introduced above that should capture the ABEC effects.

• The household_characteristics includes the variables: “Housing index” (an index on housing materials and sanitation), an “Asset index” (an index on household assets like TV or radio) and “Household size”

• The caregiver_characteristics variables are: “Female caregiver”, “Age caregiver”, “Education caregiver”, “Raven score caregiver” (A small intelligence test) and a Parent index(an index about parental behavior like reading to the child or doing homework together )

• And the child_characteristics which are: “Female child”, “Age of child” and “Z-score, BMI-for-age”

In addition, kebele fixed effects are included into the model to control for fixed heterogeneity (differences in wealth and infrastructure) between the different kebele.

When interpreting the model results, it should be highlighted that all findings are potentially subject to biases, as not all initial differences between village types could have been controlled for. Yet, these results provide the best estimates of the effect of having an ABEC in a village on pupil school enrollment and child test results. Furthermore, in the case of the Somali and part of the Oromia region the findings must be examined within the context of the drought, which does and will continue to affect the child’s performance and their ability to enroll in school.

Validity and reliability: These efforts were devoted to ensure validity and reliability through

triangulation. The use of mixed qualitative and quantitative methods and the collection of data from

independent sources makes the study more credible. It consisted of a multiplicity of perspectives, from

various stakeholders, implementers, and beneficiaries with the intent to minimize bias. However,

given the nature of qualitative research and the sample size, which was determined by practical and

research considerations, generalizations must be made carefully.

EVALUATION QUESTIONS

A set of evaluation questions (EQ) (see Table 3) were designed based on the areas of inquiry defined in the Terms of References (ToRs) (Annex 1). While preserving the different research focus of UNICEF and MoE, the questions were ordered into the DAC/OECD and UN evaluation criteria framework including impact, effectiveness, efficiency, relevance and sustainability.

Table 3: UN Evaluation Criteria

Impact / Effectiveness

EQ1 � To what extend has the provision of ABECs, in particular by ABECs supported by UNICEF, increased access to education for undeserved population in particular children from pastoral communities, their enrolment and retention rates, with an eye kept on gender equity?

EQ2

� How have ABE schools and centers increased the cognitive development of children in terms of basic skills, reading, writing, computing and understanding their environment and enabled successful transition of students to formal education?

Effectiveness

EQ3 � Are ABECs a flexible delivery mechanism? EQ4 Do ABECs provide quality education? Relevance

EQ5

�Are the modalities of implementation Alternative Basic Education (ex: the curriculum), more relevant to the needs of children in pastoral communities, in comparison to formal primary?

Efficiency

EQ6 �Was the ABE program implemented in the most efficient way to increase access to basic education? Sustainability

EQ7 �How did the ABE program encouraged ABE-related activities within communities? Do more active communities demand for more ABE services?




5 FINDINGS

This section presents the results of the quantitative and qualitative data collection. The evaluation

questions are grouped into 5 broader topics, namely the impact of ABECs on education outcomes, and

their effectiveness, relevance, efficiency and sustainability. The evaluation findings are based on the

quantitative and qualitative data collected. The results are complimented with secondary data to put

the findings of the study in broader perspective.

EVALUATION QUESTION 1 - IMPACT / EFFECTIVENESS

EQ1: To what extent has the provision of ABECs, and specifically Alternative Basic Education

centers established by UNICEF, increased access to education for under-served population; in

particular, children from pastoral communities, their enrolment and retention rates, with an

eye kept on gender equity?

The first evaluation question strives to assess the “core” effect of ABECs on school enrollment rates

and school performance.12 The former provides an idea of the impact of access to schooling while the

later evaluates the quality of the education received. Therefore, the first evaluation question asks: to

what extent has the provision of ABECs increased school enrollment and attainment in the “treatment”

villages. In this part, the main analysis will be performed at the village level and between villagers that

have an ABEC, formal primary schooling and no school. Sub-analysis will determine if there are

regional effects and significant differences between UNICEF-supported ABECs and solely government

supported ABECs.

JC 1.1 ACCESSIBILITY OF ABECS

Total number of ABECs in Afar, Oromia and Somali over time (i112). The review of Afar Education Statistics Annual Report shows the total number of functional ABECs has remained stable across the years from 2002 to 2008 E.C. with 394 ABECs in 2002 and 423 ABECs in 2008 E.C. (Afar REB, 2016/2008 E.C.). In comparison, the number of formal primary schools has been increasing, due to new constructions but also because of upgrades of ABECs, reaching, a total number of 637 primary schools, in 2016 (2008 E.C.) (See Figure 7).

In Afar, the absolute number of ABECs as well as the mix of modalities to deliver primary education varies widely across the various woredas. The number of ABECs spans from 2 centers in the woreda of Bidu to 35 in Adear, as reported in the Afar Education Bureau Educational Statistical Abstract (2016, 2008 E.C.).

12 Note in the inception report EQ1 & EQ2 were ordered in the other way around. However, to make the argument more

comprehensible we changed the order in this report and will first talk about school access and then cognitive development for

those that are going to school.




Figure 5 ABEC and Primary Schools in Afar

Source: REB Afar 2008 E.C. 13

In Oromia, the total number of functional ABECs has increased from 2,124 ABECs in 2005 E.C. to 3,115 in 2008 E.C. In parallel, the number of formal primary schools has also positively increased, reaching 1,299 formal primary schools in total in 2006 E.C. (See Figure 8)14. The distribution of primary education modalities also shows high disparities between woredas in terms of access to education and in terms of mix of education modalities15.

Figure 6 ABEC and Primary Schools in Oromia

Source: REB Oromia (2008 E.C.)

13 The number of primary schools in Afar region (785) for year 2004 E.C. seems to be an error and is therefore indicated as missing. 14 Data are missing for year 2007 EC to 2008 EC.

15 Formal primary education and Alternative Basic Education.

2002 2003 2004 2005 2006 2007 2008

ABE 394 426 440 425 360 417 423

Primary 365 420 537 571 604 637

0

100

200

300

400

500

600

700N

um

be

r o

f sc

ho

ols

AFAR - ABEC and Primary Schools (2002 - 2008)

2002 2003 2004 2005 2006 2007 2008

ABE 0 0 0 2124 2103 2491 3115

Primary 682 810 944 1110 1299 0 0

0

500

1000

1500

2000

2500

3000

3500

Nu

mb

er

of

Sch

oo

ls

OROMIA - ABEC and Primary Schools (2002 - 2008)




In Somali, the total number of functional ABECs has increased from 1,950 ABECs in 2002 E.C., to 2012 ABECs in 2006 E.C.. The number of formal primary schools has doubled over the same period of time from 682 to 1299 (See Figure 10).

Figure 7 ABEC and Primary Schools in Somali

Source: REB Somali (2006 E.C.)

In Somali, the mix of primary education modalities is available at zone level (REB 2014, 2006 E.C.). It shows a high disparity between zones in terms of the cumulated number of primary schools and ABECs: while the cumulated number of ABECs and primary schools recorded in Faafan zone reached 898 institutions, this number does not exceed 109 in Nogob; despite the diversity of area size and population (EMIS 2014). Education statistics’ annual abstracts tend to display an unbalanced mix of primary education modalities, with seven zones out of nine where ABECs outnumber the number of formal primary schools. In the zone of Faafan, the number of ABECs reaches 602 units alongside 296 primary schools which could be explained by the pastoralist nature of this region (REB 2014, 2006 E.C.).

The total number of UNICEF-supported woredas in Afar, Oromia and Somali over time

(indicator i111): Not all woredas in Afar, Oromia and Somali have UNICEF-supported ABECs. UNICEF’s technical and financial assistance focuses resources on a selected number of woredas is justified by a need to ensure a complementary coverage with other development partners in each region and demonstrating various modalities. The target is jointly discussed with REB, based on selection criteria such as high grade one drop-out rates and high drop-out rates in general, a low gender parity index and giving priority to emergency and disaster affected woredas.

In Somali, the number of woredas targeted by UNICEF’s is 18 out of 24 in total for the period 2012 to 2015. In Afar, this number has decreased from 32 in 2009 to 8 in 2012-2015 and to 4 (out of 49 woredas) in 2016. In Oromia, 11 pastoralist woredas (out of all 37 woredas) have received support for ABECs during the Country Cycle 2012-2015, according to AWPs and a list of UNICEF-supported pastoralist woredas shared by UNICEF.

2002 2003 2004 2005 2006

ABE 1950 2495 2361 2201 2012

Primary 682 810 944 1110 1299

0

500

1000

1500

2000

2500

3000

Nu

mb

er

of

Sch

oo

ls

SOMALI - ABEC and Primary Schools (2002 - 2006)


The reported rate of ABEC transformed into formal schools in Afar, Oromia and Somali (i125): ESDP IV focused on the opening of ABECs where and when necessary, alongside transformation of existing centers into formal schools (MoE 2012). This strategy has been revised for the next cycle as ESDP V suggests efforts to ensure increased transition to the second cycle of primary school as a priority over the construction of new ABECs. To improve retention rates and increase access to schools for students in transition to the second cycle, two strategies are defined in the Plan: upgrading the ABEC into a formal primary school, on the condition that the upgrade includes an extension of classes, and extension of two additional levels offered in ABECs (MoE 2015a).

Decision to upgrade an ABEC is taken at woreda level, on the demand of the community via its Center Management Committee (CMC). An upgrade can happen only if the ABEC fulfills the following upgrading criteria: a minimum of 200 students (MoE 2011), a standardized number of classes and verifying that the community is settled in the same place for a minimum of eight months. Stakeholders at the regional and woreda level and cluster supervisors have reported that the upgrade of the ABEC is desirable, especially if it follows social demand. Upgrade of the ABEC into a formal primary school implies an alignment on the formal curriculum and alignment of teaching time on the formal primary education. In reality, the decision to upgrade into a formal primary school can be loosely implemented and does not necessarily reflect the observations in the field.

In 2016/2017, 53 ABECs out of 423 were reported to be upgraded to primary schools in Afar region. In the woreda of Asayta, nine out of ten UNICEF-supported ABECs were upgraded to a formal primary school in the beginning of the 2016/2017 school year (see Table 4). In Oromia in total 21, and 16 out of the 37 UNICEF-supported ABECs have been transformed. In Somali in total 16, and 15 out of the 24 UNICEF-supported ABECs were upgraded to primary schools.

However, during the field visits, the evaluation team learned that the improvements of one of the visited upgraded UNICEF-supported ABECs were not fully implemented. The school building was not built with blocks and no additional teachers have been sent to the school. Furthermore, building upgrades are conditional on the presence of an existing block. Communities in remote areas see the provision of construction materials for blocks hindered because of low accessibility and high distance to the main roads. Despite attempts to mobilize local resources (woods) for construction and transportation (donkeys and camels), the issue of village remoteness still constitutes a major impediment to upgrade. (i125)

Table 4 UNICEF-supported ABECS in Afar, Oromia and Somali

Number of UNICEF-

supported Woredas

Number of UNICEF-

supported ABECs

Number of upgraded

UNICEF-supported ABEC

Afar 8 (4) 49 9

Oromia 11 37 16

Somali 18 24 15

Source: UNICEF (2016)

At community level, FGDs and in-depth interviews captured mixed opinions about upgrades. Although beneficiaries attribute overall importance to upgrades as a way to better address their education needs, community member respondents and ABE facilitators were invited to entertain a more nuanced view on the benefits of an upgrade of the ABECs. The perceived risk of losing flexibility provided by ABE constitutes a main argument to preserve ABE. They pointed out a need to extend the provision of ABE to Level 8, equivalent to Grade 8 of formal primary school.



29 Center for Evaluation and Development – C4ED (October 2017)

Mean distance from home to ABEC (i114): One of the first indicators for school accessibility is the average distance to school by village type. A simple analysis in Table 5 shows that children need to travel much further to school if they have no school located in their village. While this is expected, an interesting observation will be to see how this will influence school enrollment (i122) by village type and education levels by treatment group. In addition, the numbers indicate that schools are more scare (further away) in the Afar region.

Table 5 Mean distance to sampled school in km for children 7-11

Region ABEC village Primary school

village

NO school village ALL villages

Afar 5,3 2,8 6,8 5,1

Oromia 1,1 3,7 5,3 3,1

Somali 2,9 2,3 3,4 3,0

Total 2,9 3,2 4,9 3,6

Source: Authors‘ own calculations

Perceived accessibility to ABEC (i115): The location of the ABEC construction is decided in consultation with the communities. The communities designated land that is best suited for the (study/community/etc.), while addressing other formal criteria such as, being located within a radius of maximum 3 km from the most remote child in the community, it being flat land, and being free from dangers like wild animals (MoE 2011). The reported location selection process (from CMCs and parents and crossed checked with WEO officers) showed that this participatory process is at work in the ABE implementation and that it ensures the most suitable location of the ABEC within the community. In addition, Table 5 indicates that this goal was achieved at least for the Oromia and Somali region.

CMCs and parents as well as stakeholders from REB and WEO in the three regions appreciate the increased access to education allowed by the presence of a UNICEF-supported ABEC in their village. This joins the evidence provided by the team from University of Semara (2016) and Dyer and Engdsasew (2016) that the construction of an ABEC represents a great change, compared to the past where first education infrastructure was too far away and education de facto not accessible for children in their community.

Regional officials and woreda officers stated how important the role of UNICEF is in its support to ABE through construction of ABECs, furnishing and providing teaching aid. A woreda education officer stated that “What makes UNICEF special is that it’s being a pioneer in addressing remote and inaccessible

areas to get a chance of attending at least ABEC” (i115 and see i537).

Perceived hurdles to access ABEC (i116 and i712) and reasons for not attending school (i133). Economic and physical hurdles to access education provided by the ABECs remain numerous, despite the noteworthy increase in terms of enrollment to which ABE has impacted (see also i221).

In villages endowed with UNICEF-supported ABECs, FGDs with CMCs, interviews with parents and ABE facilitators confirmed this evidence on reasons for children to drop out of school. Although the provision of an ABEC within the kebele has decreased the distance to school, distance still remains an issue for children living on the fringes of the main village. Topographical and geographical characteristics of the land such as river, hills are aggravating factors which still prevent access to school. Transect walks and in-depth interviews, highlights identified by participants were dangers linked to wild animal attacks, absence of artificial light after dark, river crossings, and seasonal floods.




Case Study: Transect Walk in Oromia

The transect walks allowed to reconstruct the path from home to school of a child. They were led with a group of voluntary villagers who guided the researchers from a point A, the ABEC, to a point B, the shelter of an ABE student. During the transect walk in one of the village benefiting from an ABEC, dangers and hurdles were described. The villagers also commented on elements of the environment and shared stories of villagers met on the way.

The transect walk revealed that the field where the school was constructed is the property of one of the villagers. The field was considered the most appropriate for the construction of the school due to its flat surface, compared to the hilly topography of the rest of the village. Nevertheless, the location is not central and most children must travel a minimum of 40 minutes to reach the ABEC. The villagers perceived the ownership of the land as a sustainability issue as they see the need to pay the owner back for the rent of the land.

The transect walk also showed the dysfunctionality of gender-based bathrooms. A hard wind destroyed the bathroom roof which was never replaced. During the continuation of the transect walk, the villagers shared that the absence of public light is an issue affecting both boys and girls when travelling early in the morning to the ABEC. Attacks of wild animals (snakes) were reported as a common problem.

Comparing the results from the qualitative interviews to the quantitative statistics in Table 6, shows that distance to school (41%) and the fear for the child’s safety (16%) are important features that explain why children do not enroll in school. In particular, in villages that have no schools, more than half of the parents claim their child does not go to school because the school is too far way (57%). Importantly, in villages that have no schools, more than half of the parents (57%) claim their child does not go to school because the school is too far way.

In general, the single most important reason parents did not send their children to school was because the child had to do their household chores. It is noteworthy that in the villages with primary schools 67% of parents that did not send their child to school stated this was because the child was needed at home. Still, this was only important for 53% of parents in villages with an ABEC. This might be an indication that ABECs offer more flexible hours for children, which would enable them to fulfil their household duties and still attend school. This is in line with the qualitative interviews, showing the need for child labor to contribute to economic and livelihood activities, was still raised as a reason for parents not sending their children to school. In addition, poverty was mentioned as a main reason why parents were not able to send their children to school. For example, the inability to buy education supplies (pencil, notebooks) or clothes (student uniforms) is a burden to parents and might push them to withdraw their children from primary education.

The beneficiary interviews with parents allowed pinpointing parental strategies to combine their children’s education with the lack of resources to secure a stable livelihood. Due to this, several parents may decide to delay the age of school entry and keep the youngest child in the household to do chores and other household activities, such as fetching water or looking after the cattle. This delay allows, educational expenses to be spread over time while also having needed support from children in the home. Alternatively, some households reported that they may decide to only allow one children to receive an education and not send the others to school at all.




Table 6: Reasons for not attending any school

Reasons for not attending school (as a share

of individuals not attending any school, aged

7-11 in that village)

ABEC

village

Primary

school

village

NO school

village

ALL

villages

(1) Enrollment was turned down / not possible 6% 4% 2% 3%

Male 7% 5% 2% 4%

Female 5% 3% 2% 3%

(2) Afraid that child might get hurt on their way 13% 9% 21% 16%

Male 9% 11% 20% 15%

Female 16% 8% 22% 17%

(3) School is too far away 31% 15% 57% 41%

Male 28% 20% 56% 41%

Female 33% 10% 58% 41%

(4) School facility is not adequate for my child (no /

bad building, no equipment)

8% 1% 4% 5%

Male 8% 2% 3% 5%

Female 8% 0% 4% 5%

(5) Teacher is not well qualified 3% 1% 1% 2%

Male 1% 1% 2% 1%

Female 4% 2% 1% 2%

(6) Teacher is not present / class is canceled too

often

7% 2% 1% 3%

Male 6% 1% 1% 3%

Female 7% 4% 2% 4%

(7) Personal disputes with teacher 1% 0% 0% 1%

Male 2% 0% 0% 1%

Female 1% 0% 0% 0%

(8) School is too expensive 11% 12% 10% 11%

Male 12% 13% 10% 11%

Female 10% 10% 11% 10%

(9) Child must help with household chores 53% 67% 36% 47%

Male 53% 65% 38% 47%

Female 53% 69% 35% 47%

(10) Child has long term illness / disability 4% 4% 3% 3%

Male 5% 4% 2% 3%

Female 3% 4% 3% 3%

(11) I did not think about sending child to school 15% 18% 9% 12%

Male 13% 15% 8% 11%

Female 17% 21% 10% 14% Source: Authors‘ own calculations




In terms of gender specific issues, it seemed there were only a few questions where parents gave different answers for girls than boys. In particular, for girls from ABEC villages, parents were more concerned about their physical safety. In general, the largest differences were found in question number 11 “I did not think about sending child to school” which was a slightly more important reason for not sending girls (14%) more than boys (11%) to school.

Traditional attitudes that support child marriage create an additional obstacle, as it is still counted as a reason to stop schooling for some girls. In Afar region, one interview with an ABE facilitator illustrates this:

“- ABE Facilitator: [...] Besides, there are marriage arrangements which prevent girls from going to school. This year, there was a girl who would have been married, but after a lot of effort and cooperation with the police station of Aysaita I managed to stop the marriage. Now she is still in school.

- Interviewers: Is it common for girls to stop learning after being married?

- ABE Facilitator: Yes, it is logical and habitual action in our community.

- Interviewers: Have you experienced other female students who were forced to quit school because they got married?

- ABE Facilitator: Yes, there are many in number, I knew for instance six girls who stopped their learning suddenly due to marriage.”

Tensions between neighboring clans were also raised during qualitative interviews as an impediment to sending children to school. In the case of one visited village in Oromia, clan conflict triggered forced migration of certain communities, which interrupted education. This confirms similar findings from the situation analysis undertaken by Dyer and Engdasew in the Somali region (2016).

ABE and Qurʾānic schools

In communities where Islam plays an important role, Quranic school represent a competing model to secular primary school, including ABEC. Depending on the influence of the sheik, and the availability of education infrastructure, the role of Qurʾānic school can serve as a substitution to primary education delivered through ABE.

Parents may prefer religious education but that depends on the relative importance of Islam within the community, the influence of the sheik, as well as perceived quality of primary education. Proximity to the Qurʾānic school can also be a decisive factor to enroll a child into the Qurʾānic school rather than in primary school. Without implying a total rejection of primary education, it does cause significant delays for children who enter the secular primary education, meaning students enter school later in life, rather than the appropriate school age. The risk for female students is that they reach an age where they can be married and are forced into that instead of completing their secular primary education.

The time that religious education takes place may conflict with primary education time, which could prevent children to attend primary school. As religious education is provided in the evenings it may be more convenient for the children to be able to uphold their pastoral activities as well. This could explain why parents may prefer religious education over secular education.

Initiatives from the communities and WEOs intended to establish a link with religious leaders in order to coordinate time for religious education and secular education to take place:

“ABE facilitator: - […] Here in the community there many children who are going to Qurʾānic School. The

class is given in sessions which are clashing with ABE morning (8:00am-12:30am) school program. We




[The school CMC] sent a letter to the Qurʾānic school committee requesting them to free kids in the

morning session and deliver the Qurʾānic lesson at night time so that the children will come to the

ABEC.”

Furthermore, to conciliate religious and secular education and secure buy-in of religious leaders, the National Pastoralist Education Strategy recommends the use of Qurʾānic schools, “which are found in

most villages of pastoralist areas as venues for alternative basic education with the permission of the

community and religious leaders” (MoE 2008 E.C). This modality shows potential that UNICEF supports in 2008 the undertaking of a local study on how Qurʾānic teachers can be used as ABE facilitators in partnership with the University of Jijiga.

Perceived availability of ABEC/education provision during seasonal migration (i117). The mix of educational modalities of mobile schools, ABECs, para-boarding schools, and hostels permit continuous access to children’s education in pastoralist areas (Strategies for Promoting primary and Secondary Education in Pastoralists Areas, 2008 E.C; ESDP V MoE 2015a). The ABECs are expected to adopt a flexible approach that responds to the pastoralist community lifestyle, which is characterized by seasonal mobility in search of greener pastures and water for their cattle. This migration means the school closes when the majority of the community moves away for part of the season and the school reopens when the community comes back. That leads us to recommend that during times of seasonal migration, children should be able to attend school in their new location (see Learner Networking Card i333).

The Alternative Basic Education Implementation Guidelines and Standards (MoE 2011) described a specific feature of ABE when pastoralist communities remain established in one place for a period of time shorter than eight months a year. The ABE facilitator/teacher is expected to move together with the community. However, this feature is rarely implemented as many ABE facilitators/teachers are not originally from the community and are therefore reluctant to move with them, which creates obstacles to the education delivery.

The field interviews that were led in two agro-pastoralist communities that migrate seasonally, showed that migration does cause the interruption of education. In addition to the rigidity of mobile ABE delivery, very low level of school attendance is due to the absence of schools in the place of migration.

In Somali region, this was noted during a FGD with the CMC:

- Interviewer: Can children have access to education in the place where you move during migration?

- Female Participant: No, they will not get any kind of education at the places we go.

Similarly, when returning to their main community, repercussions from migration can translate into a high risk of drop-out rates. In Oromia, this was revealed during the interview with an ABE facilitator:

- Interviewer: Last year, have you seen an increase of children missing school?

- ABE facilitator: Yes there were. Especially last year due to migration there were children who were absent from the school and did not join school back at all.

Partial dropout rates in school has increased due to the drought, mostly in Somali and Oromia. The severity of the drought resulting in migration and displacement has been a primary factor to absenteeism and ultimately school dropout. FGDs and interviews have highlighted the fact that some community households must prepare themselves for, “survival mode”, which means parents have to prioritize livelihood activities, migrating, raising cattle and searching for water, over education. In




EQ3, the effectiveness of the Learner Networking Card is discussed to better understand how ABE can ensure a continuum of education during “normal” seasonal migration.

In the majority of pastoralist communities, the mobile school modality has been assessed by WEO, REB and UNICEF informants and tends to be the preferred option. However, key informants at UNICEF reported a low degree of implementation and suggested to complement ABE provision with mobile school in case of pastoralist communities. UNICEF’s support to mobile schools has been provided in Afar and Somali during the implementation year 2012/13 and 2013/14. Yet, after initiating the mobile school approach (including establishing the mobile school, the motivation of teachers, transportation means, and providing a mobile library), it was not replicated in the next years of the Country Cycle.

JC 2.2 ACCESS TO EDUCATION

The total number and age of children enrolled in each grade by region according to gender:

Gross Enrollment Rate (GER) and Net Enrollment Rate (NER) (i121). An indicator of access to primary education is the number of children enrolled in primary schools based on the population of that age group. This can be deduced by the level of education, regardless of age (gross enrollment rate, GER) or from the official school age per grade/group (net enrollment rate, NER). Whether looking at the EMIS datasets or the latest regional Annual Abstracts, based on EMIS data, the numbers vastly differ from each other. While the Annual Abstracts report a 96.8% GER rate for Afar, 120% GER rate for Somali and 126% for Oromia. In addition, smaller percentages 70%, 88% and 95% for Afar, Somali and Oromia were calculated using EMIS. Given the large discrepancies between the EMIS and the Annual Abstracts data, this creates questions for the reliability of the secondary data reliability; this weakens the overall analysis when and forces us to be cautious in drawing conclusions.16

The NER could possibly provide a more accurate image of enrollment and exclusively includes the number of students from the official age-group for a given grade. In Afar, the NER increased from 38.3% in 2002 E.C. to 65.8% in 2008, while in Somali it passes from 43% in 2002 E.C. to 57% in 2006 E.C. In Oromia, the NER stayed at a constant level of 88% for the 2002-2006 E.C. period (EMIS 2016)17.

While there appeared to be a positive trend in both the GER and NER (even though the levels seemed to be uncertain), most numbers still indicate a steady gap in enrollment between girls and boys. This gap is more extreme, with each higher grade, especially as they enter secondary school. This can be illustrated through evidence collected from the Woreda Education Office records in Oromia which was documented during field visits. These records show an extending enrollment gap between girls and boys: despite a similar enrollment rate for Level 1 and Level 2 between boys and girls (respectively 62% and 65% and 56% and 56%), significant disparities in enrollment appear reaching Level 3 with (65% boys and 41% for girls). Similar findings were collected from the Woreda Education Office in Jijiga, which recorded an overall lower enrollment rate for girls (4,052) compared to boys (5,241) in the ABECs in 2008 E.C., and in primary education. 18 19 (i221)

Enrollment rate sampled by UNICEF-supported ABEC village vs. primary school village by grade

and by gender (i122). Table 7 shows the child school enrollment rate for the three different types of villages: those with ABEC, formal primary schools, and villages without schools at all. The first significant finding is that the enrollment numbers in the evaluation sample are much closer to the woreda estimations than the official reports for the regions. That is school enrollment in the observed kebele are much lower than the official statistics for the whole country. However, it should be noted

16 To increase data reliability of education data in Ethiopia the evaluation team recommends investing into capacity building and training of the local authorities and the education statistic department. 17 Data are not available for Oromia and Somali region in 2007 and 2008 E.C. 18 Jijiga WEO Annual Records 2008 E.C. 19 Goro Dolla WEO Enrollment Plan for academic year 2009 E.C.




that the sample the evaluation team identified was derived from particularly rural and remote areas; therefore it is not representative on a regional level for all students, particularly those living in the larger cities.

Table 7: Net enrollment rate by village type, region and gender20

Net enrollment rate in % ABEC village Primary school

Village

NO School

Village

ALL Villages

All regions

Aged 7-11 71% 74% 50% 64%

Male 75% 77% 50% 67%

Female 65% 71% 49% 61%

Afar

Aged 7-11 78% 88% 58% 75%

Male 77% 89% 57% 75%

Female 79% 87% 59% 75%

Somali

Aged 7-11 68% 76% 34% 55%

Male 77% 79% 40% 62%

Female 56% 72% 25% 45%

Oromia

Aged 7-11 67% 65% 58% 63%

Male 73% 68% 56% 66%

Female 61% 61% 59% 60%


Yet, enrollment levels of 75% in Afar, 55% in Somali and, 63% in Oromia are far from universal enrollment. In the case of villages with no schools, enrollment rates were only 50% of children ages 7-11. On a positive note, enrollment rates did increase to 71% and 74% in areas that had ABEC and primary schools present in a village. While these are not causal effects and only correlations, there is still a strong indication that creating more schools in each village could help reduce the gap in school attainment in these hard to reach regions, even if these are only ABEC available.

Looking at the gender gap, it was the largest in the Somali region (45% for girls vs. 62% for boys),

followed by Oromia (60% vs. 66%). In Afar no gender gap could be observed. The gap seemed to be

smaller in primary schools (6%) than in ABEC (10%).

One important problem when looking at the raw numbers in Table 7 is that they do not control for unobserved heterogeneity of children background. Hence, simply comparing enrollment rates in ABEC villages and in no school villages may be misleading: It is possible for instance that schools are first built in wealthier places with more educated parents, who are also more likely to send their children to school. To control for some of the unobserved heterogeneity, results from the following Table 8 rely on a multivariate linear regression analysis as outlined in part 4.2. In the first column (1) of Table 8, the coefficients from the simple model without control variables are shown. Only the kebele and the regional fixed effects are included to control for regional differences in school attainment. Table 8’s first column indicates that attendance increases by 24 percentage points when a school is built while the level reaches 50% when no school is available in the village (see Table 7).

20 In this context enrollment rates are the % of students answering they are going to school.




Table 8: Linear regression of school attendance

(1) (2) (3)

VARIABLES School

attendance School

attendance School

attendance

ABEC village 0.237*** 0.167*** (0.0304) (0.0233)

Primary school village 0.243*** 0.112*** 0.113*** (0.0366) (0.0272) (0.0273)

UNICEF-supported ABEC village 0.179*** (0.0347)

Government run ABEC village 0.159*** (0.0281)

Housing index 0.0402*** 0.0403*** (0.00849) (0.00846)

Asset index -0.0137 -0.0136 (0.00904) (0.00904)

Household size -0.0139*** -0.0140*** (0.00328) (0.00329)

Female caregiver 0.0253 0.0248 (0.0235) (0.0234)

Age caregiver -0.00220*** -0.00219*** (0.000772) (0.000770)

Education caregiver 0.279*** 0.280*** (0.0258) (0.0259)

Raven score caregiver 0.00565 0.00583 (0.00411) (0.00417)

Female child -0.0689*** -0.0691*** (0.0152) (0.0152)

Age of child 0.0572*** 0.0573*** (0.00571) (0.00571)

Z-score, BMI-for-age -0.0129*** -0.0129*** (0.00275) (0.00275)

Parent index 0.0902*** 0.0906*** (0.0157) (0.0156)

Constant 0.913*** 0.583*** 0.583*** (0.0651) (0.0827) (0.0827)

Observations 3,599 3,578 3,578 R-squared 0.250 0.384 0.384

Robust standard errors in parentheses, *** p<0.01, ** p<0.05, * p<0.1. Note: not reported are regional and the kebele fixed effects. The missing category for the treatment variables “ABEC village” and “Primary school village” is “no school village”. Source: Authors‘ own calculations

After including household and individual control variables (in column 2), the effect of having an ABEC in comparison to no school at all drops to respectively 17 percentage points and 11 percentage points. The effect size reduction between column 1 and 2 highlights the fact that villages where a ABEC is built, a primary school is build and no school is built at all are globally very different: no school villages are poorer, have parents who are less educated, have larger households. Interestingly, the primary school estimate is more affected than ABEC estimates by the control variables: this indicates that primary school village are on average richer than ABEC villages. Yet, both results remain strongly significant and are now significantly different: ABEC seemed to outperform primary in term of school attendance.




In the third column (3) of Table 8, the specific effect of UNICEF ABE is compared to the government ABE. Both ABEC have a positive effect on attendance (16 percentage points and 18 percentage points) and their difference is not significantly difference: we hence conclude that UNICEF and government ABEC have a similar impact on attendance rate.

To sum up, as expected, building a school in a village has a string effect on attendance. This effect remain whatever control variable is included in the regression confirming this basic finding. Interestingly, when control variables are used, ABEC fare better than primary schools. Yet, we are not able to differentiate the effect of UNICEF-supported ABE and government supported ABE. In the following, we look how higher attendance affected student performances.

EVALUATION QUESTION 2 - IMPACT / EFFECTIVENESS

EQ 2 - How have ABE schools and centers increased the cognitive development of children in

terms of basic skills, reading, writing, computing, and understanding their environment and

enabled successful transition of students to formal education?

The second evaluation question strives to assess the relation between the activities of ABEC and their effects on cognitive development for girls and boys. Cognitive development is known in the long-run to increase labor income and well-being. In addition, it can improve the participation of Ethiopian youth in social, political and economic life. The impact of ABECs on the cognitive development of the children was measured by a range of basic skills: their ability to solve basic math problems, the understanding of simple vocabulary and their concepts, as well as reading a simple 100-word school text book in their local language.

JC 2.1 COGNITIVE DEVELOPMENT/CHILD EDUCATIONAL PERFORMANCE OF CHILDREN

Scores from cognitive performance tests in mathematics, vocabulary and reading (i.211). In Table 9, the average test results by village type from the child instrument are summarized. All variables are standardized test scores except for the mathematics category and illiterate variables. The mathematics category is based on a math tests that categorized children in seven levels of comprehension, from innumeracy to highly skilled. The illiterate variable signifies the inability of the child to read out a single word from a grade two textbook in the local language.

As shown in table 9, villages that have a primary school have, on average, higher test scores for all learning indicators. However, these observed differences cannot be interpreted as causal effects. Results are affected by both unobserved heterogeneity and differential compliance between village types. As show in Table 7 and 8, attendance in each school type is different in each village type. Therefore, in Table 10, the test results are given for actual school attendance. As one can see, the differences in child test scores between children that attend school and those that do not are much more pronounced. For example, children that go to an ABEC or primary school reach on average more than one category higher in the mathematics score. Similarly, the differences in the vocabulary score are twice as large for those children attending a primary education facility.




Table 9: Descriptive statistics: Child test scores by village type

Child test scores UNICEF-supported

ABEC village Government

ABEC village Primary

school village

NO school

village

Math category 1.85 2.01 2.25 1.61

Math score -0.03 0.02 0.12 -0.10

Vocabulary score 11.47 11.16 13.76 9.75

Reading score 2.00 1.82 3.11 1.37

Illiterate 77% 77% 73% 86%


Looking at the sub-group of younger children (ages 7-9) versus older pupils (ages 10-11), we can see that for all test scores there appears to be learning that takes place, while test score improvements for out of school children is less pronounced. Yet, results from table 10 probably suffer from selection bias: children with higher inherent ability are more likely to be sent to school than low achiever.

Table 10: Descriptive statistics Child test scores by school attendance (for eligible children age 7-11)

Child test results ABEC attendance Primary school

attendance

NO school attendance

Math category 2.159 2.327 0.968

Age 7-9 1.721 1.832 0.909

Age 10-11 2.644 2.794 1.059

Math score 0.065 0.161 -0.334

Age 7-9 -0.080 -0.022 -0.365

Age 10-11 0.226 0.332 -0.287

Vocabulary score 12.438 13.726 6.468

Age 7-9 10.365 12.083 5.729

Age 10-11 14.731 15.277 7.606

Reading score 2.106 3.109 0.091

Age 7-9 1.247 1.480 0.071

Age 10-11 3.003 4.540 0.118

Illiterate 75% 71% 97%

Age 7-9 81% 80% 97%

Age 10-11 68% 63% 96%


To control for the unobserved heterogeneity and to get closer to the true effect of going to an ABEC versus going to a primary school or no school at all we present the results from a linear multivariate model in table 11. In the first column (1), math test is regressed against school attendance, controlling




for regional and kebele fixed effects. The results indicate that both children from ABEC and primary schools reach a much higher level of education than those children who are out of school (+0.7 category) from a control average of 0.9 categories.

Table 11: Linear regression of schooling type on math scores

(1) (2) (3) (4) (5)

VARIABLES Math

category Math

category Math

category Vocabulary

score Being

illiterate Child attends ABEC 0.729*** 0.570***

(0.0764) (0.0770) Child attends UNICEF-supported

ABEC 0.552*** 3.089*** -0.0693** (0.0979) (0.719) (0.0295) Child attends government ABEC 0.582*** 2.392*** -0.0659*** (0.0974) (0.555) (0.0196) Child attends primary school 1.152*** 0.837*** 0.837*** 4.225*** -0.0997*** (0.0740) (0.0728) (0.0728) (0.479) (0.0193) Housing index 0.161*** 0.160*** 1.260*** -0.0356*** (0.0422) (0.0422) (0.299) (0.0111) Asset index 0.0177 0.0177 0.146 0.00652 (0.0393) (0.0393) (0.271) (0.00608) Household size 0.0214* 0.0215* -0.0173 0.00200 (0.0117) (0.0117) (0.0820) (0.00342) Female caregiver -0.0607 -0.0597 0.141 0.0447* (0.0962) (0.0955) (0.608) (0.0262) Age caregiver 0.00436 0.00435 -0.00389 -0.000185 (0.00290) (0.00290) (0.0208) (0.000735) Education caregiver 0.278*** 0.278*** 1.357*** -0.0922*** (0.0696) (0.0697) (0.419) (0.0200) Raven score caregiver -0.0118 -0.0119 0.0981 -0.00309 (0.0177) (0.0178) (0.112) (0.00525) Female child -0.306*** -0.307*** -1.055*** 0.0700*** (0.0489) (0.0490) (0.285) (0.0134) Age of child 0.309*** 0.309*** 1.503*** -0.0367*** (0.0236) (0.0236) (0.141) (0.00589) Z-score, BMI-for-age 0.0443** 0.0444** -0.0455 0.000856 (0.0179) (0.0179) (0.106) (0.00436) Parent index 0.0741 0.0738 -0.0652 0.00757 (0.0707) (0.0707) (0.421) (0.0166) Constant 0.780*** -1.758*** -1.769*** -9.610*** 1.103*** (0.160) (0.280) (0.281) (1.758) (0.103) Observations 2,656 2,654 2,654 2,758 2,425 R-squared 0.332 0.407 0.407 0.447 0.452 Robust standard errors in parentheses, *** p<0.01, ** p<0.05, * p<0.1. Note: not reported are regional and kebele fixed effects. The category that is left out for the treatment variables is “child attends no school”. Source: Authors‘ own calculations

After controlling for important individual household and parental characteristics found in column (2) both coefficients for attaining an ABEC and primary school drop of about 25%. This means that we would overestimate the true effect of going to school if we don’t control for these background characteristics. In terms of magnitude, the results indicate that going to an ABEC increase the math




category of about a half point, an increase of about 50% of the control group result. The effect is larger for primary schools (about 0.8 category improvement) and significantly different from ABEC. The same conclusion can be draw from other outcomes (see column 3, 4 and 5): attending ABE increase vocabulary test by 3 words (from a control value of 6 words) or reduce illiteracy by about 7 percentage points. As mentioned, even if care is warranted when using this result, the magnitude of the effects are so strong that we have little doubt about the direction of the causality.21

Finally, in column 3, 4 and 5 the hypothesis that UNICEF-supported schools perform better than other ABEC is tested and further outcome are provided. Results are generally similar in both ABEC except maybe for the vocabulary test where UNICEF ABEC seemed to perform slightly better. Given the size of the potential bias, one should not draw a general conclusion from that small different: overall, ABEC did improve performance but are less effective as a primary school. The reasons for the mixed findings for the UNICEF effect will be observed in more detail in the next sections when considering the ABEC environment and the quality of the ABEC facilitators.

JC 2.1. TRANSITION OF STUDENTS TO SECOND CYCLE OF PRIMARY EDUCATION

The Estimated rate of student transition from ABECs to second cycle of primary schools. The impact of ABE schooling should also be addressed in terms of continuity of schooling. ABE was initially designed as an alternative modality to the first cycle of primary school up to grade 4 from the formal curriculum. In the absence of upper (or secondary) cycle of ABE, ABE students graduating are expected to move on to formal primary schools. The question is whether ABEC can ensure that transition, a question which has not been answered in previous studies (Onwu 2010).

In Table 12, one can observe that children living in villages with ABEC had on average about the same years of education than those living in villages with primary schools. Furthermore, there seemed to be a prevalent gender bias, boys being more likely to receive secondary schooling than girls. The difference seemed to be the largest in primary school (37% of boys vs. 19% for girls).

Table 12: Years of education and secondary schooling by village type for 12-15 year olds

Education outcomes for 12-15 year olds ABEC village Primary

school village No school

village

Average years of education reached 2.4 2.8 2.5

Male 2.7 3.4 2.7

Female 2.1 2.2 2.3

Secondary schooling reached 17.2% 28.7% 23.6%

Male 20.6% 37.5% 26.8%

Female 13.6% 18.7% 20.2%


21 Besides, control variables have all the expected direction: there seem to be significant and positive effects for staying in a wealthier household (Housing index) and having better educated parents/caregivers (Education caregiver). As expected, older children perform significant better, indicating that for each year the child is in school it will improve by 0.3 categories in the mathematics test.




The perceived success and challenges of transition of ABE children to second cycle of primary

school. While various stakeholders (an ABE facilitator, WEOs and REB officers) reported that former ABEC students perform equally well as their fellow primary school students; absent of disaggregated longitudinal data obstruct calculating transition rates from ABEC to the second cycle of primary school. However, the findings presented by the quantitative data, rather support more critical stands of stakeholder informants that ABE students do not perform as well as students from the formal school (i223, EQ5).

Absence of ABE levels corresponding to grade 5 to 8 of formal school and distance to the nearest primary school remain a main issue hampering transition to upper primary school. Furthermore, the upgrade of ABECs into formal primary schools does not automatically lead to the creation of upper levels. As Dyer and Engdasew noted (2016), the options for transition are limited, as there tend to be no nearby primary school to join after completion of ABE.

In Oromia, the challenge of transition to primary education was taken in consideration by REB with support from UNICEF. Oromia’s ongoing design of an ABE curriculum and syllabi for grades 5 to 8 (AWP 2009) shows an effort to address this issue. However, this initiative is not currently mainstreamed at federal level nor in other regions

EVALUATION QUESTION 3 - EFFECTIVENESS

EQ 3: Is Alternative Basic Education (ABE) a flexible delivery mechanism of quality education?

This question covers the effectiveness of Alternative Basic Education to deliver quality education in a way that accommodates the local conditions and the needs of learners and their parents. Flexibility is one of the key strengths of Alternative Basic Education enshrined in the National Pastoralist Education Strategy (MoE 2008) and National Alternative Basic Education Implementation Guidelines (MoE 2003 E.C.).

The flexibility of ABE can first be measured in terms school timing: school hours during the day and during year should be adapted to the local needs of the communities. It can also be captured by ABE’s capacity to provide continuous education when the community needs to migrate for part of the season. This flexibility lies in the capacity of an ABE facilitator/teacher to teach and plan lessons accordingly to communities’ needs.

However, being flexible and sensitive to the needs of the community should not come as a detriment to the quality of education given, spelled out by MoE in the General Education Quality Improvement Package (MoE 2008). The effectiveness of ABE to deliver quality education outputs will be examined as part of EQ 4.

JC 3.1 TIMETABLE AND CALENDAR ARE ADAPTED TO THE NEEDS OF PASTORALIST COMMUNITIES

ABE timetable and calendar. The 2011 Alternative Basic Education Standards in pastoralist and agro-pastoralist areas set by the MoE indicates how the teaching learning program should be organized within a school year. It determines the yearly calendar, comprising of two semesters which are both separated with a weeklong break.

The 2011 Alternative Basic Education Standards provide relative flexibility with regards to the start of day for the school year, which is decided in consultation with the communities and the ABE facilitator.




Nevertheless, this flexibility described in the “Standards” remains to a certain extent limited. As shared by Woreda Education officers, the ABE calendar is at the same time based on the formal school starts in September of the Ethiopian calendar, but the registration timing can be flexible and open for longer periods of time. This means that although the school year has started, students can still enroll within the next one to two months. (i311)

Similarly, a daily schedule is to be decided in consultation with the local communities and the ABE facilitator/teacher, provided that the basic standards are respected: six schooling days per week and each day includes four periods that are at least 40 minutes each. The relatively shorter duration of school days, with fewer periods and greater amount of school days creates flexibility from the basic primary school standards 22 and aims at reducing absenteeism. Timetables were reported to be shaped in close consultation with the communities via the CMC and the ABE facilitator. (i314 and i315)

Communities expressed high satisfaction during FGDs and parent interviews regarding the flexibility

of the daily and yearly ABE calendar. This flexibility was also highlighted by other evaluations and

studies on ABE (PBEA 2016; Dyer and Engdasew 2016). Yet, the availability of the ABE

facilitator/teacher at the time of the day was not always an option, therefore creating an obstacle to

gathering interviews. (i316)

JC 3.3 FLEXIBLE ACCESS TO SCHOOL OUTSIDE OF THE VILLAGE

School attendance when children and families seasonally migrate (i331): ABE also accommodate the delivery of school instruction based on migration patterns in pastoralist and agro-pastoralist communities. This mobility involves all or part of the community and often includes primary school age children.

In our sample, 24% of the households are raising livestock as their only farming activity (see Table 13).23 40% of these households migrated during the last seasonal movement. Altogether, 29% of the pastoralist households stated bringing at least one of their children with them when they migrated. Unfortunately, only a few households gave information on whether a child went to school during the seasonal periods. Of the 36 children that we do have information on, 24 (66%) continued going to school. Of those 24, 19 claimed they went to a different ABEC than the school they usually attended. This indicates that very few children drop out from school because of migration (12 out of 36) and that many of them use ABEC while migrating. (i332)

Table 13 Household farming activities, seasonal movement of households and school attendance during seasonal

movement

Name of indicator Percentage/

Quantity

Households that engage in farming activities, as a percentage of all households (excluding missing information)

93%

By farming activity, as a percentage households that engage in farming activities:

Cattle/flock raising 24%

Crop cultivation 2%

22 Standard Primary Education timetable include five schooling days per week and each day includes six periods of 40 minutes each, amounting to four hours of schooling per day. 23 Unfortunately, we don’t have migration information for those households that answered “Cattle/flock raising and crop cultivation” due to a skipping error in the data. Therefore, in the following we must limit the analysis for this issue on the 24% that reported “Cattle/flock raising”




Cattle/flock raising and crop cultivation 74%

Households that move according to season, as a percentage of households that engage in cattle/flock raising

40%

Children who move according to season, as a percentage of children aged 7-11 29%

Number of children who attended school during seasonal movement 24/36

By type of school attended:

Usual school that child attends 5

ABEC in other village 19


Existence of a Learner Networking Card (i333). Table 14 shows the frequency of Learner Networking Card and school transcripts recordings of the competences of students at ABECs. It shows that they are used most frequently in Somali, where the Learner Networking Card was first introduced. In Somali, nearly 70% of the ABECs give out the Learner Networking Cards or similar school transcripts. While in Somali UNICEF-supported and government-run ABECs display similar results for the two other regions, the Learner Networking Cards and other school transcripts are more often distributed in the UNICEF-supported ABECs.

Table 14 Provision of learning networking cards by schools

Schools which provide learning networking card, as a percentage of all schools of a given type

UNICEF-supported

ABEC

Government ABEC

Afar 53% 45%

Oromia 43% 33%

Somali 67% 69%


Reported satisfaction for the Learner networking card (i334). Community mobility patterns embrace a vast spectrum but are motivated by similar reasons, mainly the search for water and pastures for cattle. Emergency situations caused by extreme weather events/natural disasters, such as droughts or flooding force communities to move out at a sudden notice, usually outside of their normal seasonal migration patterns. The drought induced by below average 2016 rains in south and southeast Ethiopia, mostly in low-land regions of Somali and Oromia has aggravated this need to migrate. Consequently, the risk of school interruption for agro-pastoral and pastoralist children related to migration has intensified.

In-depth interviews with key informants allowed capturing information on the Learner Networking Card, an innovation introduced by Save the Children (SCI) and UNICEF in Somali to limit the interruption of education during seasonal migration. This innovation relies on migration-related decision-making mechanisms at work in pastoralist communities. Community informants (Sahan) identify potential migration destinations with water and pasture. Once identified, the community moves to this location or nearby during the dry season.

Building on this mechanism, SCI and UNICEF have identified three main pillars of action:




The preparedness pillar advocates at community level to include education as a criterion for migration place along with other criteria like water and grazing pastures. Training will be delivered to the sahan and CMCs to support the identification of the migration place which education places, like schools.

As part of the response pillar, voluntary teachers will move along with the communities that are identified. Additionally, information will be provided to community to issue and later collect the networking cards. This will include the need to follow up in order to monitor the effective use of the networking cards.

The return/mitigation pillar includes follow-up actions at the community level to prevent drop out and absenteeism when the community comes back to their main lodging/residence.

One challenge identified is that communities move to areas where the Learner Networking Cards may not be used. WEO respondents described the functionality of the Learner Networking Cards as “difficult”. The limit of this tool stems from the unpredictability of pastoralist communities’ migration and lack of awareness from the families to pick up the LNC prior to migrating for the season. In addition, the level of involvement of ABE CMC and Cluster supervisors are not always designed to closely monitor migration movements of communities.

The federal and regional authorities slowly recognize the help of this tool via inclusion in the ESDP V into Education during emergencies. The need for advocacy at the federal level as well as to all woredas is still needed to institutionalize the Networking Card System in printing cards and planning its distribution. (i333 and i334)

JC 3.4 FLEXIBLE PRESENCE OF ABE FACILITATORS / TEACHERS

The flexibility of ABE delivery is linked to the presence of ABE facilitators/teachers in class. The main obstacle to achieve full flexibility is the limited availability of ABE facilitators within the communities.

Facilitators in the community. Overall, only 70% of teachers/facilitators live in the village where their school is located. 69% of facilitators in ABEC run by government live in the same village, whereas this figure drops to 65% for facilitators of UNICEF-supported ABEC. In both types of ABEC, this remains lower in comparison with the share of formal primary school teachers who live in the village where the school is located (76%) (see Table 15).

Table 15 Location of teachers' home

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC

Formal primary school

All schools

Location of teacher’s home, as a percentage of all teachers for a given type of school

In the village where school is located 65% 69% 76% 70%

In another village in the same kebele 23% 17% 12% 18%

In another kebele in the same woreda 12% 10% 12% 11%

In another woreda 0 4% 0 1%





It is noteworthy that the share of teachers living in another village in the same kebele is 23% in UNICEF-supported ABE and 17% in government-run ABEC while 12 % of formal primary school teachers lives in another village in the same kebele.

Commuting time of teacher/facilitator to school (i342). Qualitative interviews revealed long distances to travel from home to school, most of the time by foot, given low public transportation availability and costs of transportation, up to two hours for one way.

Difficulties recruiting qualified teachers and facilitators within the community force the WEOs to recruit outside of these villages. However, this causes problems for ABE facilitators/teachers in terms of attending class, which were revealed in qualitative interviews with ABE facilitators and parents. Long commuting distances of facilitators who do not live in the communities combined with lack of public transportation, coupled with low salary incentives, are the main obstacles for the facilitators to be available to teach every day and deliver education at the time that best suits the communities’ wish. To alleviate teachers’ absenteeism, local community initiatives have emerged to ensure quality delivery by providing facilitators with temporary accommodation and food. This is in line to some extent with the recommendations of the 2011 Alternative Basic Education Standards in Pastoralist and Agro-Pastoralist Areas (MoE) on community contribution to ABE. (i342)

EVALUATION QUESTION 4 - EFFECTIVENESS

EQ 4: Do ABECs provide quality education?

Being cognizant that low quality educational inputs translates into low educational outcomes, high drop-out, and repetition rates, the MoE, supported by UNICEF, has strived to include the improvement of education quality as a main objective to ensure acquisition of basic skills, numeracy, and literacy (MoE 2010 ; MoE 2015a).

The specific objectives for the quality improvements are integrated by the MoE in the General Education Quality Improvement Package (GEQIP). The first phase of GEQIP was launched in 2009 (2009-2013) and was followed by the second phase GEQIP II (2013 to 2018) which set up the specific objectives and modalities to improve quality education delivery (MoE 2009; MoE 201324). The special focus of GEQIP II on “improving learning conditions in primary and secondary schools” (MoE 201325), with the introduction of the School Grant Program and the School Improvement Plan (SIP), in each school, requests spending at least fifty percent of the School Grant in the teaching and learning environment domain (MoE 2015). The main education delivery modality in pastoralist areas is Alternative Basic Education (ABE) which is included in the GEQIP II and this is why assessing the quality of ABE deserves close attention.

Under this EQ, the evaluation team assesses the effectiveness to deliver quality outputs related to the school environment as well as teacher quality. This section also includes a quick assessment of UNICEF’s role to support. Questions related to other features of quality of education, curriculum, and the engagement of communities will be addressed in following EQs.

24 GEQIP Plan 25 GEQIP II




JC 4.1. QUALITY OF ABE ENVIRONMENT

The quality of learning environment is a major factor that influences the educational performance of children. In pastoralist areas, ABECs are set up as one of the main structures/institutions to deliver primary education. Based on the low-cost construction and community contribution in deciding construction and management, ABECs appear to be an affordable solution to develop access to education in remote pastoralist areas. This modality must nevertheless ensure quality learning conditions to support the sound development of skills.

The set of standards put forth in the Alternative Basic Education Standards in Pastoralist and

Agro-pastoralist Areas (MoE 2011) reflects the objective of the Ethiopian government to ensure quality for the construction and equipment of ABEC, despite its low cost. These physical infrastructure standards describe the type of services that each ABEC should include, in terms of the classrooms (the number and classroom’s dimension), offices for facilitators, storage for education material, presence of sanitation facilities, lighting system (see Table 16).

These standards also describe the type and amount of teaching learning material, combined desks, tables and chairs for facilitators, black board. Additional play and teaching aid material will be provided.

Table 16 ABEC construction standards

S.no Description of services Quantity area

1 Classrooms 2 6X7= 42 each m²

2 Facilitators Office 1 4x4=16 m²

3 Store 1 3x3=9 m²

4 Separate latrines for female and male students 2 1.5x0.80=1.2 m²

5 Residence for female and male facilitators 2 3.0x4=12 m²

6 Separate latrines for female and male facilitators 3.0x2=6 3.0x2=6 m²

7 Light (from Biogas, solar Masho, etc)

Source: MoE (2011)

While community contribution is stated in terms labor and material contribution during the ABEC construction in the standards and guidelines, the loose definition of the roles of the community and that of the government stakeholders in the ABEC’ construction constitutes a risk and places the main responsibility of construction and maintenance on the communities. (i411, see also EQ7)

UNICEF’s support to ABE construction also employs quality and quantity standards. These standards go beyond the standards set by MoE, in terms of number of classrooms to build and number of separate latrines for boys and girls. Similarly, for furnishing and equipping, UNICEF’s standards set a higher target of furnishing elements and equipment (Table 17).




Table 17 UNICEF ABEC construction standards

No Description of Activities Target / Quantity

Construction of the center

1. Construction of 3 levels ABE classrooms 3 classrooms

2. Construction of teachers’ office including store 1 office

Furnishing the ABEC

3. School furniture for 3 classrooms 50 each and a total of 150 children

75 combined desks

(2 children X 75 combined desk= 150 children)

4. Blackboards for 3 classrooms 3 blackboards

5. Shelves 2 shelves

6. Teachers’ table in each classroom 4 tables

Teachers’ chair in each classroom and teachers’ office

7 chairs

Providing educational materials

7. Exercise books (10), pen (6) and pencil (6) for 150 children

150 children

8. Duplication of text books (4 textbooks for each level and 12 for 3 levels) for 150 children

150 children

9. Teachers’ guide for each level 12 books

10. Printing and distribution of T/L Resources 150 Children

11. Playing materials

WASH facilities in the center

12. 4 Toilets 2 separate latrines for girls and boys

Hand washing facility 1 hand washing facility

Shallow well (potable water) 1 water well

Simple disposable pit

Source: UNICEF (2016)

Number of classrooms in ABECs and in formal schools (i113). The calculations come from the quantitative data given in Table 18, which shows that UNICEF-supported ABECs have on average more classrooms available (2.8) compared to ABECs run by the government (2.1). UNICEF ABECs are therefore close to reaching the standards of three classes per ABEC as set by MoE. Even more important, the number of students per classroom is significantly lower in UNICEF-supported ABECs (43) than in contrast to the government ABECs (70). The class size, measured as the average number of students per section, or class (student section ratio)26, is a strong factor that can have an impact on the quality of the teaching and learning environment27. A too high class size undermines the teaching learning process. On the other hand, the concern to maximize the cost-effectiveness of ABEs invites to

26 There can be several sections of pupils within one classroom that the teacher has to attend to separately. 27 There are consistent evidence in the literature indicating that class-size is one of the important predictor of performance in developed (Krueger & Whitmore, 1999) as well as in developing countries (Duflo et al. 2015)




avoid low student section ratio, which reflects the underutilization of resources. A class size of about 43 in the UNICEF-supported ABECs is close to the 1:40 goal of the Alternative Basic Education Standards in Pastoralist and Agro-pastoralist areas and the standard of ‘regular’ primary schools (MoE 2011). This indicates a positive impact in favor of the continued UNICEF support in constructing adequate ABEC with sufficient classrooms for the number of students. (i416)

Qualitative interviews revealed that multi-grade teaching is commonly used in ABEC, where the number of classrooms and of teacher/facilitators is less than the number of grades.

Table 18 Number of schools by region and number of classes by school type

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC

Formal primary

All schools

Number of schools surveyed by region:

Afar 17 11 11 37

Oromia 21 12 10 43

Somali 3 10 7 33

Average number of class rooms for all levels 2.8 2.1 4.6 3.0

Number of students per classroom 43 70 43 52


School infrastructure in ABEC and formal schools (i412). The significant observed differences are the type of infrastructure of ABECs and formal primary schools. 93% of formal schools consist of a school building (blocks). ABECs supported by UNICEF are constructed at 73% of blocks, while only 60% of ABECs run by the government are (see Table 19). The observed difference is likely due to UNICEF’s support in the construction of ABECs. ABEC buildings made of local construction material (woods) or tents were observed in 5% of UNICEF-supported ABECs and in 13% of other ABECs.

Finally, outdoor schooling (mostly under a tree) is observed in 26% of ABEC run by government, compared to 22.5% of UNICEF-ABECs and 7% of formal primary schools.28 UNICEF’s support in school construction seems to have primarily affected villages where schooling was initially given in another building.

28 What it means to have an outdoor school vs. a normal school built of bricks can be observed in Figure 11 & Figure 12. Figure 12 show that the children literately get taught under a tree to at least cover some shade.




Figure 8 UNICEF ABE Center Block - Somali

Own source

Figure 9 UNICEF ABE under tree

Own source




The quality of ABEC environment is assessed in terms of the Water, Sanitation and Hygiene (WASH) facilities available. To that regard, access to a source of drinkable water remains a major challenge both in ABECs and in formal primary schools. A large proportion of the government run ABECs (85%) does not have proper access to drinkable water (see Table 19). In contrast, access to drinkable water is unavailable in 56% of UNICEF-ABECs and in 57% of formal primary schools. This indicates that UNICEF’s support in the provision of WASH facilities was effective because of the great need.

Table 19 School infrastructure indicators

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC


ALL schools

Place where classes are conducted

A school building 73% 61% 93% 73%

Other building 5% 13% 0 7%

Outside (e.g. under tree) 22.5% 26% 7% 20%

Last 4 years’ changes in school building quality

Worsened 30% 20% 29% 26%

Stayed the same 25% 36% 29% 30%

Improved 40% 40% 39% 40%

Unknown 5% 4% 4% 4%

Source of drinking water in school

Drinkable water is not available 56% 85% 57% 68%

Pond/river/stream/unprotected dug well/rainwater

34% 10% 21% 22%

Water is bought/public tap/piped in dwelling or on premises

2.5% 0 14% 4%

Tubed/piped well or borehole 7.5% 4% 7% 6%

Source of water for hand washing in school

Water is not available 58.5% 83% 61% 69%

Pond/river/stream/unprotected dug well/rainwater

34% 13% 29% 24%

Water is bought/public tap/piped in dwelling or on premises

2.5% 0 0 1%

Tubed/piped well or borehole 5% 4% 11% 6%

Toilet facility in school

Open land 63.5% 87% 36% 66%

Pit latrine 36.5% 13% 57% 32%

Toilet connected to sewerage or septic 0 0 7% 2%

Gender separated 17% 11% 46% 22%

School building quality index (wall, roof, electricity)

0.30 -0.65 0.45 -0.02





For all the three school types, water was mainly available in the form of pond/river/stream /unprotected dug well or rainwater. Tubed and piped well or boreholes are rarely detected (4% in other ABECS, 7.5% in UNICEF-supported ABECs and in 7% of the formal primary schools). While they are purchased, stored, or public water remains almost inexistent in ABECs and rare in formal public schools (14%).

These results are in line with the evidence collected during the qualitative field work. Although water supplies were observed in two of the surveyed UNICEF-supported ABECs, in the form of water storage tanks and uncovered dug well, they were not functional because of technical defect and of irregular provision of water. The low availability of functional water supply such as water tank (Roto Tank), dug well or WASH facilities and the fact that separate latrines are seldom present, indicate that ABE’s construction remains imperfect and are sources of dissatisfaction for local communities.

The availability of gender-separated sanitary facilities (i414). Toilet facilities are rare in government-run ABECs where pit toilets make up for 13% of total toilet facilities and no toilets are connected to sewerage or septic systems. 36.5% of UNICEF-supported ABECs have toilet facilities under the form of pit toilets. This shows that UNICEF-supported ABECs are built to a higher standard than other ABECs but are still far from ABEC construction standards (see Table 16 ABEC construction standards), in particularly for gender separation (only 17%) and less than in formal primary schools 57%. This means that proper toilets for both girls and boys needs to be addressed.

There was additional qualitative field work that provided insights into the safety of the school areas. According to Alternative Basic Education Standards in Pastoralist and Agro-Pastoralist Areas (MoE 2011), the area should be free of potential risks (wild animals, valley, floods) and also be child-friendly. The parents and CMCs illustrated a nuanced reality. The site selection relied on a community needs assessment of the most appropriate place under geographical and economic constraints. Yet, in the visited ABECs, the school area was not clearly demarcated and was not entirely secured. At best, there were bush fences that delineate the school areas and provide an absolute protection (see figure 11 and 15 in Annex 10).

The perceived quality of ABEC infrastructure (i413). There are important differences in the perceived quality of the school buildings are observed between school types. Whereas three quarter of the respondents in UNICEF-supported ABECs (44%) would describe the school as in an acceptable condition or better (75%), only 43% would say so for the government-run ABECs. (see Table 20 Perceived quality of school infrastructure). When asked about the changes to the school building in comparison to four years back, a majority of respondents (40%) indicated improved school buildings (see Table 19) but there is no significant difference between UNICEF and non-UNICEF ABE. Taken all together, these figures indicate an overall larger local satisfaction from UNICEF-supported ABE than from government ones.

Table 20 Perceived quality of school infrastructure

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC


ALL schools

Perceived quality of school building:

Very old or poorly built 25% 57% 27% 38%

Acceptable condition 31% 29% 54% 37%

New or in a very good condition 44% 14% 19% 26%





The qualitative findings support this evidence, particularly as far as the government-run ABECs are concerned. Despite increasing access to education, the below-average quality of the government-run ABECs is criticized both by regional officials and WEO officers. There is praise as well with regards to the quality of UNICEF’s contribution to ABEC’s construction. Similarly, the MoE ABE focal experts estimate that 60% of the ABE infrastructure does not meet the ABEC standards, due to a lack of sufficient budget. The beneficiaries expressed a more nuance view on the quality of the UNICEF-supported ABECs. Although they provided an overall satisfactory assessment of UNICEF ABEC’s construction quality, there was a lack of maintenance or renovation which was perceived as a negligence of the authorities in trying to provide quality education. When we visited UNICEF-supported ABECs, the main issues that were raised in terms of infrastructure quality, concerned the run-down condition of the buildings and the lack of ventilation systems.

The field visits exposed high expectations from the beneficiaries from agro-pastoralist and pastoralist communities to receive a fabricated primary education school. A facility made of block is seen from the beneficiary’s perspective as a sign of consideration from the authorities and to improve education quality. In contrast, the government’s preference for cost-efficient construction depended largely on the availability of local materials from the community, or from mobile ABEC, such as tents (see also EQ7). (i413)

The quantity and quality of school equipment (i415). The difference between UNICEF and non-UNICEF ABEC are particularly significant when looking at school equipment. The number of combined desks installed in all the classrooms in the government-run ABEC reached 47 units on average. In UNICEF-supported ABECs, this number raises to 78 units, which is still not even close to the average number observed in formal primary schools (145 units). Furthermore, when it is available in sufficient quantity, the high state of deterioration makes the equipment unusable or dangerous, impeding the teaching and learning process. 52% of ABE run by the government do not have any combined-desk or table. In comparison, this number takes the value of 29% of UNICEF-supported ABECs, which reflects relative effectiveness of UNICEF’s support to provide furniture to ABECs. This figure remains nevertheless lower than formal primary schools, which are defined by the absence of combined-desks only in 10% of the cases.

More strikingly, the difference in the number of available textbooks shows the important disparities between ABECs and formal primary schools. While formal primary schools have on average 733 textbooks available, this figure states that there are only 180 textbooks available, unresponsively UNICEF-supported or others (Table 21 School facility and equipment). (see also i419)

The respondents from agro-pastoralist communities and the ABE facilitators during the qualitative interviews indicated a lack of functional equipment, namely combined desks and chairs or their deterioration. The perceived changes in school facilities and equipment show a net perceived improvement in formal primary schools (61%). More modestly, 37.5% of the respondents in UNICEF-supported ABECS observed improved school facilities and equipment while 30% stated a deteriorating effect and 30% reported no particular changes. In comparison, the perceived improvement of quality of school facilities and equipment is more than 10 points lower in ABE run by the government, and most respondents (44%) assessed that the quality remained the same across the years.




Table 21 School facility and equipment

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC


All schools

School age 4.75 6.02 7.75 6

Average money schools receives from the government (in Birr)

5234 7071 12547 7751

Average number of rooms 4 3 7 4

Average number of classrooms 3 2 5 3

Average number of combined-desks & tables for children

78 47 145 81

No combined-desk nor table 29% 52% 10% 34%

Average number of blackboards in school 3 2 5 3

School with flipcharts/white boards in school 3% 20% 19% 14%

Average number of textbooks in school 180 180 733 330


The presence of school served food and safety net implementation (i417). The unavailability of school food programs in ABECs has been pointed out by respondents from WEO and communities as a major lack of education provision, resulting in increased long-term drop-out rates, in particular for students who severely lack an opportunity to access education. UNICEF respondents have notified that UNICEF is advocating in favor of the inclusion for ABE into the school food program, yet so far this has not been fully implemented.

Table 22: School support outside of regular classes

Does school provide any further support

except regular classes

Type of school

UNICEF-supported

ABEC

Government ABEC


ALL schools

No 70% 58% 29% 55%

Yes 30% 42% 81% 45%


The quality and quantity of ABE teaching materials (i418). MoE (2011) have a set standardized list of elements which should be provided in particular classrooms (see Table 16 ABEC construction standards). The number of ABE students should determine the amount of teaching learning material available in a particular classroom. The quantitative results indicate that on average, the number of blackboards in schools is three in UNICEF-supported ABECs and is two in other ABECs. Other type of teaching material, like flipcharts and whiteboards were almost never recorded in field visits for ABECs and formal primary schools (see Table 21 School facility and equipment). This is in line with the findings of the qualitative interviews that mention that the absence of school material is very common and teachers have to often buy them themselves (i418).

The textbook-student ratio and perceived quality of ABE learning materials (i419 and i4110).

Each of the four subjects taught in ABE is covered in a textbook for each level. The ABE facilitators and




teacher respondents considered the ABE learning material very useful and relevant in encouraging quality education, based on the formal primary education standards (see i534).

Providing textbooks to each student was a strong factor determining the quality of the teaching and learning process. An inadequate supply of textbooks may impede children to pursue learning during and outside of school time. ESDP IV (MoE 2010) and ESDP V (MoE 2015a) explicitly spelled out the objective to distribute textbooks to ensure that all students have access to a complete set of textbooks (4:1 ratio). However, the shortage of textbooks is considered by all the stakeholders met as one of the main challenges not only in ABE but also in formal schools. Despite the massive textbook production and distribution under GEQIP, this objective seems to be far from being met when it comes to the actual availability and use of textbooks in ABECs.

The quantitative data revealed a low average of textbook to student ratio and there are significant disparities between ABECs and formal primary schools. Surprisingly, the average ratio is slightly above 1.3:1 which means 1 textbook per student in UNICEF-supported ABECs compared to above 2:1 in ABEs run by the government. In both cases, these figures remain lower than the textbook-student ratio of nearly 3:1 recorded in formal primary schools (see Table 23 Textbook student-ratio by type of schools).

Table 23 Textbook student-ratio by type of schools

Name of indicator

Type of school

UNICEF-supported

ABEC

Government ABEC


ALL schools

Average textbook-student ratio 1.33 2.04 2.96 2.07


While the textbook shortage is assessed both for formal primary schools and ABECS, WEOs, which oversee distributing textbooks received from REBs, may try to overcome shortage by reallocating books from one type of school to another. In one of the visited woreda, the shortage in ABE textbooks was averted by providing formal curriculum textbooks. Frequent changes in administrative boundaries were raised as an important obstacle to quality, on-time delivery of the teaching material as well as the furniture to support the infrastructure of the school. However, the reallocation mechanisms are not straightforward. Although the shortage of textbooks in this same woreda was reported both for formal and ABE textbooks, ABECs seem to be more likely affected by the issue. The solutions to cope with the textbooks’ shortage are sometimes aggravated at the local level. In one visited ABEC, the ABE facilitators encouraged a rotating system where children from the same household and in the same ABE level share the same book. (i4110)

JC 4.2 QUANTITY AND QUALITY OF ABE FACILITATORS/TEACHERS

The delivery of quality education by its very nature relies on high skilled teachers. The MoE, supported by UNICEF, have mainstreamed efforts to improve teachers and ABE facilitators’ skills. The guidelines on ABE implementation in pastoralist and agro-pastoralist communities (MoE 2003) includes training of ABE Facilitators, provision of courses in the socio-economic and cultural realities of pastoralist areas and building capacities of supervisors.

The student to teacher ratio in Afar, Oromia and Somali (i422, i341 and i229). The Alternative Basic Education Standards in Pastoralist and Agro-Pastoralist Areas (MoE 2011) were set in a way that each newly constructed ABEC should have at least two facilitators. In case the number of students registered is between 81 and 120, there should be at least three facilitators to reach a 40:1 student




teacher ratio. This ratio is similar to the national primary education minimum standards that should ensure quality education. (i341 and also EQ4).

The observed average student/teacher ratio largely exceeds the standards in government-run ABE where 66 students per teacher were measured. This figure is comparable to the regional average in the Afar region for ABECs (61:1) but larger than the one observed in Oromia (37:1)2930. Unfortunately, no data on student/teacher ratio is available for the Somali region. UNICEF-supported ABE and primary school fare better again in that aspect with a student / teacher ratio of 50:1 and 43:1 respectively.

The ABE facilitator gender gap remains hard to narrow in all regions. Secondary data from Afar Education Bureau Education Statistics Annual Abstract (2008 E.C.) show that despite efforts to prioritize female candidates during the recruitment process, the share of female ABE facilitators remains extremely low, variating between 5% of the ABE facilitators in 2008 E.C. and 13%, maximum reached in 2003 E.C. in Oromia. In comparison, the share of female ABE facilitators/facilitators in the sample amount to 15% of the total number of ABE facilitators. According to WEOs’ stakeholders, the difficulty to recruit female ABE facilitators is due to the challenging working conditions and the remote areas where many of the ABECs are located.

Table 24 Student-teacher ratio by type of school

Name of indicator

Type of school

ABE UNICEF

ABE Government

Formal primary

All schools

Average student-teacher ratio 50.29 66.41 43.14 54.63


Teacher Time-on-Task (i343). The teacher time-on-task reflects the teacher’s use of classroom time. ABE standards recommend that classes take place six days a week and classroom time consisting of four periods that are roughly 40minutes each. This amounts to a total instruction time of about 16 hours per week.

Instruction time is here estimated by compiling the number of hours spent teaching per week. ABE teachers/facilitators in UNICEF-supported ABECs show an average longer time teaching with 14 hours per week teaching, close to the time spent in formal primary schools (15.5 hours per week). In contrast, ABECs run by the government show lower average teaching time (11.23 hours per week) which does not meet the standards set. On the contrary, UNICEF’s support correlates with higher amount of time, indicating a potential impact of UNICEF’s support on the quality of teaching.

Table 25 Teacher Time on Task

Name of indicator

Type of school

ABE UNICEF

ABE Government

Formal primary

All schools

Average teaching time in hours per week 14.00 11.23 15.49 13.61


29It reached 94:1 in 2006 E.C. This extreme value deserves carefulness in its inclusion in the analysis and is solely provided as note. 30 This extreme value deserves carefulness in its inclusion in the analysis and is solely provided as note.




Academic qualification of ABE facilitators/teachers and perceived match to the minimum

requirements (i423 and i424). It appeared clear from interviews with key stakeholders at MoE, REB and WEO level and Teacher Training College that the insufficient supply of qualified certificated ABE facilitators or even those with minimum academic qualification of Grade 10 remains a major challenge to the provision of quality primary education. In practice, the shortage of candidates with minimum Grade 10 level leads WEO to recruit ABE facilitators under the minimum requirement, including below Grade 8.

In Afar, in 2008 E.C., the number of ABE facilitators with an education profile of Grade 8 or below at the time of recruitment represents more than 59% of the facilitators (Afar REB 2008 E.C.). Nevertheless, this figure is in decline compared to previous six years as the share of grade 8 slowly decreased from 75% in 2002 E.C. to 66% in 2007 E.C. More surprisingly, the share of ABE facilitators recruited with a maximum Grade 10 level education is 94% and stable across the years and represents above 90% of recruited ABE facilitators.

It is important to acknowledge the inadequacy in qualified teachers, and sharp emphasis is given under the current ESDP V to facilitator capacity building: ABE facilitator capacity buildings and the revision of the ABE facilitator training program were included as part of the first phase of GEQIP. (i423)

Based on the survey results, there were sizeable disparities in the academic qualifications of teachers/facilitators emerge across the three observed school types. On average, teachers are less qualified in ABECs than in formal primary schools. Unfortunately, UNICEF’s support to improve teacher quality did not seem to have made a significant impact in this area.

While in formal primary schools as well as in UNICEF-supported ABECs, no facilitators were reported to have the maximum academic qualification of having attended primary school, which is equivalent to up to grade 8, and 6% of ABE facilitators in ABECs run by the government reported to only have graduated from primary education, which is lower than the minimum requirements of Grade 10. In contrast, while nearly half of teachers hold a tertiary degree in formal primary schools, only one fourth of ABE teachers in ABECs run by the government hold this degree and, only 9% of the UNICEF-supported ABECs facilitators. (Table 26 Academic qualification of ABE facilitators/teachers). (i424)

Table 26 Academic qualification of ABE facilitators/teachers

Name of indicator

Type of school

UNICEF-supported

ABEC

Government

ABEC


ALL schools

Academic qualification of teachers

Primary 0 6% 0 2%

Grade 10-12 62% 25% 39% 44%

TTI 13% 22.5% 12% 15%

Other type of certificate 16% 22.5% 2% 13%

Tertiary 9% 24% 47% 26%


Facilitators/teachers trainings (i425 and i426). Teacher trainings activities are organized by various stakeholders, mainly WEO Education and Training Board, REBs with support of UNICEF, under oversight of REB. These trainings are carried out with the objective to train facilitators and teachers to




deliver quality Alternative Basic Education but also to provide ABE facilitators and teachers with the opportunity to obtain a certificate.

After the recruitment phase, three months of summer pre-service (on-job) training on basic pedagogics and subjects are provided to all ABE facilitators at Teacher Training Colleges (TTC). It includes classes on multi-grade teaching and learner-oriented teaching-learning methods. Facilitators have the possibility to follow the on-job training each summer, as well as the opportunity to pass their certification credentials. Efforts have been made to expend facilitators’ access to training in the ABE system. Yet, disparities still seem to exist in the access to trainings and some ABE facilitators reported that they did not have refresher training during their career.

UNICEF has supported facilitators training throughout the implementation years of the 2012-2015 Country Cycle. In all three regions of examination, the funds allocation shift from ABE Center constructions to soft component, pre-service (on-job) and in-service (refresher) training for ABE facilitators31. Supporting both, UNICEF’s training aim at building capacity following MoE’s primary education standards for teaching and learning process methodology, continuous assessment, classroom management and the use of multi-grade approach (Error! Reference source not found.). UNICEF’s contribution to training is considered by all respondents as very useful, as it allows making up for the lack of training funds allocated. Yet, the insufficient engagement of the authorities in training facilitators is a possible reason for UNICEF to decrease funding for these activities (see EQ 7).

Survey results show that UNICEF’s efforts to support training capacities are still insufficient. Overall, more than a third of facilitators/teachers have not received a teacher training (36% on average) and UNICEF-supported ABE facilitators did not appear benefit from more training (29%).

The training modalities vary by school type: while government ABE and primary school teacher received training from the Teacher Training College, teachers from UNICEF ABE globally received less training and were more likely to attend the summer camp or other training facilities.

The average number of teacher training days for teachers/facilitators who have received a training remains significantly higher for formal school teachers than for ABE facilitators. Formal primary teachers spent on average of nearly twice as many days in training (286 days) than the ABE facilitators/teachers (142.8 for UNICEF-supported ABE facilitators/teachers and 147.8 for government-run ABE facilitators), revealing a significant gap in training provisions, which was to the detriment of ABE facilitators/teachers. (i425)

As an illustrative example, in the case of Somali, the Dr. Abdulmajid Hussein College of Teacher Education of Jijiga (formal Jijiga Teacher Training Institute) has trained a total of 1304 ABE facilitators from all the zones of the region from 2004 until 2008 E.C. Yet, these figures remain under the required amount to equip all ABEC with trained ABE facilitators. (Table 28 Facilitators trained at the Teacher Training College of Jijiga). As Dyer and Engdasew revealed in their situation analysis in Somali region: in 2015, the REB intended to train 1350 teachers but only 340 could travel to Jijiga for the training (Dyer and Engdasew 2016). Somali is not an exception with similar gaps are also reported in Oromia.

31 This is based on AWPs review for Afar, Somali and Oromia




Table 27 Type and duration of facilitators/teachers training

Name of indicator

Type of school

ABE UNICEF

ABE Government

Formal primary

All schools

Type of facilitators/teacher training

Teacher Training College 29% 50% 49% 41%

Summer training/camp 16% 13% 12% 14%

Other 16% 8% 2% 9%

None 40% 29% 37% 36%

Average number of days of teacher training (if trained)

142.75 147.79 286.33 190.16


ABE facilitators/teachers who attended the refresher trainings stressed the usefulness of trainings to develop and consolidate teaching skills. Trainings financed by UNICEF and delivered through the REB and WEO were deemed very useful and satisfactory to equip facilitators with further skills in delivering learner-oriented education, but also to manage the center in cooperation with the CMC or PTA and creating awareness on education in the context of pastoralist communities.

Table 28 Facilitators trained at the Teacher Training College of Jijiga

Year No of facilitators trained

2004 E.C/2012 270

2005 E.C/2013 56 - Due to selection & recruitment problem.

2006 E.C/2014 248

2007 E.C/2015 340

2008 E.C/2016 390

Source: Teacher Training College Jijiga

Training recalled by ABE facilitators/teachers covered teaching-learning methodology concepts, lesson planning, subject-based training, classroom and multi-grade classroom management. On the other hand, some potential training topics suggested are rarely covered including but not limited to child protection, HIV/AIDS awareness, children with special needs, disability and gender sensitivity in classroom management.

All interviewed ABE facilitators appreciated the usefulness of the trainings. Many reported that the refresher training was adequate but did not set them up for success as teachers in many ways, due to the unique contexts of each region or woreda. The reported satisfaction surveyed for training should be also nuanced given its short and varied period (30 to 45 days) insufficient to cover the topics needed and ensure acquisition of minimum skills, when ABE facilitators have not finished schooling. Moreover, the absence of fix the training curriculum in TTCs prevents from ensuring quality of the




training and accreditation. Monitoring the quality of the trainings is weakly enforced due to the lack of supervision from the WEO on the persons mandated to deliver the trainings.

Recruitment of facilitators/teachers (i427). The recruitment of ABE facilitators is operated by the WEO, based on the guidance of the REBs, which reflects the ABE Standards (MoE 2011): candidates should have completed Grade 10 and have proven skills in the local language. Priority is given to facilitators from the community, or if not available, from neighboring kebele and woreda. In practice, these criteria are seldom met. To cope with the shortage of proper profiles, recruitment procedures also include the possibility to recruit at Grade 8 after approval of the REB. This recruitment of facilitators who did not complete at least Grade 10 is perceived as a high risk as they are not able to meet prerequisites and to fully take advantage of the training provided by the Teacher Training Colleges (TTC). According to respondents from TTC, the absence of the TTC in the recruitment process is problematic to ensure the aptitudes of recruited facilitators to be effectively trained, given their starting level, often below requirements.

Additionally, further findings from a study led by Dyer and Endgadsew (2016) tends to highlight the pitfalls of the selection process of facilitators, such as the absence of entrance examination or favoritism based on clan affiliation. Interviews with ABE facilitators but also informants at WEO suggested that privilege based on ethnic identity may to some extend influence the access to in-service (refresher) teacher training.

While nearly all formal primary school teachers (96%) hold a permanent position, this share is lower in ABECs that UNICEF supports, where about 20% of teaching staff are ABE facilitators, under limited contract with the WEO or with an NGO (7%), other (4%) or have no contract at all (7%).

Table 29 Types of teachers' contracts by type of schools

Name of indicator

Type of school

UNICEF-supported

ABEC

Other ABEC


ALL schools

Type of contract, as a share of all teachers of a given type of school:

No contract 7% 10% 2% 6%

Community teacher contract 5% 4% 2% 4%

Contract with NGO 2% 0 0 1%

Permanent public contract (Permanent teacher position)

82% 85% 96% 87%

Other 4% 2% 0 2%


The salary of facilitators/teachers (i428). The monthly salary depends on the type of position

of the teacher. While The level of ABE facilitator salary is based around 700 ETB, it sometimes reaches above 1000 ETB in case where the facilitator graduated from certificate program and holds a permanent ABE teacher position. The salary averages reported is around1254.61 Birr in UNICEF-supported ABE but can be as much as 1650 Birr in ABE-run by the government. This difference could partly be explained by different experience, training and status. This is further evidence that facilitators recruited in UNICEF ABE might be of slightly lower quality. In any cases, ABE salaries remain much lower than in the formal primary school, where teachers earn on average 2092.15 Birr per month.




Table 30: Monthly salary of facilitators/teachers

Name of indicator

Type of school

ABE UNICEF

ABE Government

Formal primary

All schools

Average monthly salary in BIRR 1254.61 1647.39 2092.15 1656.79


The perceived challenges related to facilitators/teachers (i4210). The high turn-over of ABE facilitators was mentioned by WEO officers and regional officials and officers as a major issue, undermining the overall progress in terms of staff trained. Harsh working conditions and low economic incentives are reported as causes of ABE facilitator defection and drop out. This goes along with high reported absenteeism, and in one reported case, a misuse of funds from ABE facilitators. Eventually, the lack of accountability and insufficiency of supervision and monitoring intensifies under-optimal facilitator performance and the risks of poor ABEC management. Cluster supervisors, in charge of the monitoring to ABECs and primary schools of each clusters have limited means at their disposal (vehicle and fuel) to access remote and hard-to-reach schools on a regular basis and to provide on-site support to ABE facilitators/teachers. The following quote was shared by an ABE facilitator met in Afar and illustrates the lack of encouragement received from the cluster supervisors:

Interviewer: - Do you appreciate the supervision you received from the cluster supervisor?

ABE Facilitator: - Had he been very good supervisor, I would have been very successful ABE facilitator.

The village would have benefited from the experience shared among the cluster center and the satellite

schools. For me, he would have been an additional asset in improving students’ performances.”

The results of the quantitative survey reveal that in all type of schools the low teachers’ salary is a source of dissatisfaction. This is particularly predominant in ABECs where 57.7% of respondents answered, “very true” on that specific challenge. In addition, 37.5% found that challenge “somewhat

true” and only 5% “not at all”.

This is fully supported by findings from the qualitative interviews. The ABE teacher/facilitator salary was consistently reported, as very low and insufficient salary identified as a main factor of dissatisfaction for ABE facilitators. More broadly, ABE facilitators expressed low satisfaction regarding their position, mostly due to the lack of recognition. The additional responsibility that some ABE facilitators overtake as ABEC director do not generate additional payment.

There were occasional delays in payment were reported but without being a main reason of dissatisfaction. Despite occasional incentive payments to ABE facilitators to prevent defection, noted in Afar according to WEO officer, no additional funds can be realistically provided to teachers due to very low funds available at woreda level.

In general, the qualitative interviews revealed that ABE facilitators feel neglected in comparison to formal school teachers due to a lower salary, contract-based employment, and fewer trainings offered were just among the mentioned complains. Nevertheless, the participation in in-service refresher trainings up to a certificate level and diploma, as well as the perspective to reach a teacher position constitute a source of satisfaction.

In addition, during the survey, the low available means to teach, such as teaching supplies, like chalk, were also reported as a challenge. In UNICEF-supported ABECs, 47.5% of parent respondents declared the statement as “very true” and 30% as “somewhat true”. By cumulating both type of answers and comparing to ABECs run by government (51% “very true” and 16%”somewhat true”), problems of




supply appear to be more of a challenge in UNICEF-supported ABECs. In comparison to formal primary school, this reveals important disparities in provision of teaching supply between ABECs and formal primary school, where an insufficient teaching supply were reported less problematic.

Interestingly, the high number of students per teacher/facilitator (see i429) is assessed as a challenge by many respondents in UNICEF-supported ABECs (45%), albeit less importantly than in government-run ABECs (53% of respondents found the statement on this challenge “very true”). By contrast, half of the respondents (50%) assessed that this is “not at all” a problem in formal primary schools, showing insightful difference in perception of the teaching environment quality.

Surprisingly, an absence of the teachers is not raised as a challenge by respondents (70%) on average, with similar rate in ABECs and in formal primary schools. If the absence of the teacher is not considered as a problem by respondents, it can nevertheless be a hindering factor in educational outcomes. The absence of objective monitoring data on teacher absenteeism and complexity to measure it in the primary data collection does not allow receiving the trend on actual absenteeism of teachers/facilitators in ABECs.

Table 31 Perceived challenges related to facilitators/teachers

Challenges related to facilitators/teachers

Type of school

ABE UNICEF

ABE Government

Formal primary

All schools

Salary of teachers/facilitators is too low

Not at all 5% 9% 21% 11%

Somewhat true 37.5% 35.5% 36% 36%

Very true 57.5% 55.5% 43% 53%

Not enough supplies like chalk etc.

Not at all 22.5% 33% 46% 33%

Somewhat true 30% 16% 18% 21%

Very true 47.5% 51% 36% 46%

Low qualification of teachers/facilitators

Not at all 25% 42% 24% 32%


Very true 42% 33% 28% 33%

Too many students for the number of teachers/facilitators

Not at all 37.5% 29% 50% 37%

Somewhat true 17.5% 18% 21% 19%

Very true 45% 53% 29% 44%

Teachers/facilitators are absent

Not at all 75% 71% 67% 70%


Very true 8% 12% 9% 11%





EVALUATION QUESTION 5 - RELEVANCE

EQ 5: Are the modalities of implementation Alternative Basic Education (ex: the curriculum),

more relevant to the needs of children in agro-pastoralist and pastoralist communities, in

comparison to primary?

This EQ’s goal is to assess the extent to which Alternative Basic Education is suited to the priorities of the target group, specifically children in agro-pastoralist and pastoralist communities. In line with the OECD DAC principles for the Evaluation of Development Assistance, this question can be presented as a subset of questions:

� To what extend are the overall objectives of the program valid? � Are the activities and outputs consistent with the overall goal? � Are they consistent with the intended impact and effect?

The notable expansion of ABE within this strategy leads to question whether this implementation modality is appropriate with the learning needs of the communities and whether the activities are consistent with the overall goal. This question focuses on the extent to which ABE modalities are valid with the objectives to deliver primary education equivalent to formal primary school, while being a suitable solution to adequately answer the needs of communities.

In that regard, an important feature that the evaluation team strives to assess is the relevance of ABE curriculum to the learning needs, in relation to the formal primary curriculum. This includes the children’s assessments and textbooks. As a primary education modality which should be adapted to the lifestyles of pastoralist and agro-pastoralist communities, it is also important to question whether the flexibility described in ABE is appropriate to the objectives, in comparison to primary school. The implementation of flexibility is already treated as part of EQ 3, therefore the EQ 5 will be restricted to address the validity of this program aspect to answer the overall objectives stated earlier.

Secondly, is the decentralized management and implementation of activities consistent with the overall goal and is UNICEF’s support taking this in consideration? This sub-question both takes stock of the effectiveness of processes but also questions whether this is well suited to assess needs and adapt quickly to the needs. As transversal question, is UNICEF’s support aligned with country policies and priorities?

To answer this EQ, desk review coupled with in-depth discussion with stakeholders from MoE, REBs and WEO, as well as cluster supervisors and CMC allow capturing elements of answer.

JC 5.1ALIGNMENT WITH FORMAL PRIMARY CURRICULUM

Monitoring and evaluating the academic achievement of students via assessment is a critical

component of quality education (i511). The horizontal alignment measures the match of the ABE assessment to the content standards, based itself on the formal curriculum content.

The evaluation reader should take note that demonstration of the alignment between assessment and standards must rely on sophisticated methods which could not apply during this mission due to limited resources and time. Instead, the evaluation mission intended to identify evidences informing on the assessment practices in ABE. Assessment of horizontal alignment with formal primary curriculum relies on documentary review, completed with key stakeholder interviews (including ABE facilitators).




A balanced mix of formative and summative assessment allows to monitor and to evaluate the students learning achievement based the curriculum standards. The absence of clear assessment guidelines across ABE, reported during stakeholder interviews with ABE focal persons at MoE leaves room for ABE facilitators to define assessment procedures and items. Interviewed ABE facilitators reported using a mix of the formative continuous assessment daily, and a summative assessment at the end of the semester, based on in-job training’s learnings. Generally, the absence of further guidance on the assessment can represent a potential risk for accountability.

JC 5.3. ABE CURRICULUM IS ADAPTED TO NEEDS OF PASTORALIST COMMUNITIES

The development and adaptation of the ABE curricula (i532, i533, i534 and i535). The REBs are responsible for contextualizing and translating the curriculum and developing the necessary teaching material correspondingly. The curriculum and textbooks were unanimously reported as available in local languages in the three regions and are of good quality in the content they offer. (i535) Yet, Dyer and Engdasew (2016) note that in the case of Somali, the curriculum should be more contextualized.

UNICEF’s support is aligned with this need for more contextualization and with the needs of regions to adapt the curriculum to their needs. In Somali, this takes place via the funding of curriculum revision to take into account gender sensitive considerations in four subjects based on regional context, focusing on equity and social cohesion. Similarly, this alignment to the regional needs also appears in Oromia where UNICEF where the REB launched the development of a second cycle of ABE curriculum up to Grade 8. (i533) (i534)

Table 32: Number of levels / grades by school type

UNICEF-

supported ABEC

Other ABEC Formal primary school

ALL schools

Grade / level 1 10% 9% 4% 8%

Grade / level 2 18% 44% 10% 26%

Grade / level 3 55% 33% 29% 40%

Grade / level 4 18% 14% 36% 21%

Grade 5 or above 0% 0% 21% 5% Source: Authors‘ own calculations.

The coexistence of the two types of curriculum (four-year curriculum and three-year curriculum) in each region was observed. Despite the change activated at regional and woreda level on the curriculum used, the change is not necessarily followed at school level, as some schools reported to still use the ABE condensed curriculum. Moreover, the condensed curriculum was justified by over-age students, this distinction between over-age and in-age students is not realistically doable within the same school. (i531)

Existence of feedback mechanisms to REB (i536). As ABE curriculum in its condensed version directly emanates from the formal curriculum, adaptation to the needs of pastoralist and semi-pastoralist communities is assessed at high level. Feedback mechanisms allow CMC formulate their needs to Cluster Supervisors and WEO, however no accountability systems secure proper responses to the needs expressed. In one visited woreda, the cluster supervisor mentioned the responsiveness of the WEO to find solution to a reported need, in contrast with the low involvement of zonal education office regarding ABECs. (i536)




The reported satisfaction on the quality of the education delivered (i537). At the local level, access to first cycle of primary education was reported as satisfactory during FGDs with the CMC members and parents from the community. However, in all areas, parents expressed their concern about the ABEC’s poor quality, and consider this as negligence from the authorities. The quality of ABE is also considered to be less compared to formal education by the WEO respondents interviewed. The concerns that were raised were the quality of infrastructure, the lack of educational equipment, but also the facilitators/teacher qualification and management skills. Despite the low educational attainment levels of community members, most communities, their CMC showed a strong commitment to the ABEC. They identified issues related to teacher quality and school administration which could be pinned down as a cause for poor education outcomes. This shows that education quality in terms of capable personnel does matter to communities. However, a lack of proper training may prevent the community members and parents from identifying and preventing weaknesses in teacher quality. (see also EQ7)

JC 5.4 COMMUNITY EDUCATION NEEDS ARE ADEQUATELY ASSESSED

The identification of pastoralist learning needs in the program documents (i541). The ESDP III to IV observe the particularities of special learning needs for out-of-school children in scattered communities living along a pastoralist or agro-pastoralist lifestyle in remote areas.

This translated into the formulation of the Pastoralist Education Strategy in 2008 and its revision of the National Pastoralist Education Strategy during the 2016/17 year. During the time of this evaluation, this also shows endeavor to adapt delivery of ABE to pastoralist children needs.

The revision took place after the conduction of a situational analysis (Dyer and Engdasew 2016), mandated by the MoE and funded by UNICEF and the British Council to assess the achievement of the previous National Pastoralist Education Strategies from 2008. The conclusions implied that the implementation tactics were not in line with the proposed strategy. As reported by an UNICEF informant, the 2008 Pastoralist Education Strategies did not receive a widespread adhesion from the regional states. Absence of validation process and involvement of regional stakeholders in the design and validation of the Pastoralist Education Strategies might be a reason for low adhesion from regional states.

The drafting of the Alternative Basic Education Strategy in 2006 and its revision in 2009 show efforts to more precisely shape the implementation needs and modalities that the Alternative Basic Education should undertake. However, none of these documents were endorsed by the MoE which may indicate several conflicting visions of the program.

The alignment of UNICEF with needs and priorities defined in country strategies is very relevant. Country Strategy 2012-2015 aligns its support to the GTP with expansion and quality improvement of education services, following major priority actions of the ESDP. The Country Program Document 2016-2020 aligned with the GTP II. Both country programs focus on children in pastoralist communities as the main target, associated the ESDPs.

The shift away from construction to the support to quality as spelled out in ESDP III and IV is taken into consideration by UNICEF with an increased support for capacity building and curriculum development. Similarly, the alignment with priorities from ESDP V is reflected with the focus of UNICEF’s support on general education quality improvement, with a strong emphasis on sector planning and policy dialogue. (see EQ4, JC5.3)

Regional stakeholders value the support of UNICEF, which is in line with the regional policy. Nonetheless, they perceived the shift of UNICEF priority to other policy agenda, such as support to policy level, as a medium-term challenge to deal with increasing regional educational needs. UNICEF




has provided training activities to disseminate the strategy, and to translate it to create ownership and stimulate sound implementation at regional level.

Gender equality considerations have been partially pursued with UNICEF ABE interventions, but not explicitly throughout all regions. These trainings include promoting female facilitators to train in Somali. Education for special needs children remain an underutilized element of UNICEF’s support to ABE.

UNICEF’s geographical targeting is also relevant to the needs assessed in the country strategy. ESDP focuses on emerging regional states with increased focus on Somali and Afar, two regions with GER below 50% during ESDP III and IV, and on the four regional states Afar, Somali, Gambella and Benishangul-Gumuz during ESDP V. During the Country Cycle 2012-2015, UNICEF geo-targeted 120 deficient woredas in support of ESDP III, reflecting UNICEF’s concern to answer the needs for the most vulnerable population. This needs assessment was based on vulnerability criteria. The new Country Cycle 2016-2020 does include a national implementation component with geo-targeting focused on the four developing regions: Afar, Benishangul-Gumuz, Gambella, and Somali supporting priorities from ESDP V.

The National Pastoralist Education Strategy from 2008 was based on an acknowledgement that out-of-school children in pastoralist areas have special needs which are exasperated when trying to access to school and special modalities, where formal primary school failed. After eight years of the National Education Strategy, the assessment led by the international team of Prof. Dr. Dyer and Dr. Engdasew allowed the current revision of the National Pastoralist Education Strategy. (i542)

The perceived adaptation of ABE to the community education needs assessed (i543). The initial identification of specific learning needs for pastoralist and agro-pastoralist communities, remoteness from primary school, children involvement in pastoralist and agriculture activities, seasonal migration patterns, are the raison behind ABE. Yet, ABE, through ABECs, is unable to provide quality education to the same level as formal primary school, justifying the need to upgrade ABECs into formal primary schools. (see EQ3 and EQ4)

While one might assume that the upgrade of an ABEC would cause an end to the flexible delivery, the upgrade is perceived to adapt to the way of life for communities who embraced a non-migratory way of life, based on agro-pastoralist activities. Respondents from beneficiary groups, WEO stakeholders and UNICEF regional office justified this with the decreasing needs of such communities in relation with ABE flexibility, but before the need to access quality education, this was perceived as not accomplished via ABE.

By contrast, for communities that rely exclusively on pastoralism, the modality ABEC is perceived as a rigid modality which could be better addressed with other alternative modes of primary education. Local WEO stakeholders in Oromia and Afar and UNICEF informants reported the need to develop alternative delivery modalities such as mobile schools. (i543, see also i217)

JC 5.5 DECENTRALIZED MANAGEMENT AND IMPLEMENTATION OF ABE IS ADAPTED TO THE NEEDS

The relevance of the Alternative Basic Education Program also relies on the capacity of its management framework to provide adequate and timely responses to the pastoralist and agro-pastoralist needs. The devolution of the management and implementation of ABE to regional states implies high need of coordination at all levels to ensure success and achieve the goals. The decentralized management procedures and coordination mechanisms were assessed mainly by desk review of the ABE Implementation Guide, which was completed with an analysis of procedures and perceptions reported by key stakeholders and external experts based on their level of satisfaction.




The opportunities and challenges on decentralized implementation of ABE (i553). The decentralized implementation of ABECs is considered to be a highly important asset which might allow for receptive and relevant responses to the educational needs of children, especially those from pastoralist and agro-pastoralist communities. The autonomy of regional states acquired through the devolution of decision-making, finance and management is an opportunity to best address the needs. As an Oromia REB official mentioned: “Sometimes what we are doing is out of the box”. This statement refers to the decentralization of the budget and responsibilities to regional states and illustrates the opportunities to make decisions without following guidelines from the MoE. Nevertheless, decentralization must rely on strict implementation and monitoring processes and guidelines as well as local capacity to ensure the quality of the outputs and strong outcomes.

The mechanisms and level of coordination and information exchange between central and

decentralized bodies assessed (552).

The coordination between Federal MoE and REBs were reported to be functioning globally, although they are characterized by communication gaps. Quarterly meetings, the assignment of regional experts from each directorate in the region ensure regular exchange on achievements and needs between the regional and federal level. Nevertheless, officials from REB and ABE focal experts from MoE noted loopholes in coordinating between the Federal MoE and the REBs, related to the communication gaps. Increased autonomy from the federal GoE sought by regional stakeholders tends to accentuate the coordination issues.

At the regional level, coordination takes place through monthly Education Task Force Meetings involving REBs and all development partners, including UNICEF. This coordination with development partners allows discussing identification of gaps to ensure complementarity of coverage, the impact of the drought and ensuring ongoing education were among the challenges. In the case of the Somali region, REB and WEO respondents reported that UNICEF was very engaged to ensure coordination of action.

The coordination of REB with lower implementation levels is also ensured through coordinating with the Regional Informal Education and Community Development directorate, which fully manages ABE, in close link with zonal, woreda and kebele level. The coordination mechanisms and their relevance to respond to the pastoralist and agro-pastoralist children’s learning needs are not consistently assessed by the various stakeholders. This plurality of perspectives may reflect both some contextual particularities and differentiated scope of action in overall ABE management.

The interviews with WEO and REB officers from the three regions illustrate a nuanced reality of the mechanisms to implement ABE. In Somali, there were close links to the REB and smooth coordination was reported in the woreda of Jijiga. This may come from the geographical close proximity between the WEO and the REB, and may not be extrapolated to other Somali WEO contexts.

In Oromia, the relationships between the REB and the WEO were reported working well, mostly going through the zonal level to report demands of school to the REB. This way, the demand for education equipment supply is addressed through the intermediate of zonal education office.

In Afar, coordination was reported weak, notably due to low level of commitment of both the WEO and the REB in support to ABE. The low perception of ABE as a useful investment from these stakeholders mostly holds from the fact that ABE is considered as a lower-end education modality. The stakeholders from the REB and from the WEO reported that ABE remains in a functioning state primarily due to UNICEF, perceived as only stakeholders which values ABE. On the other hand, the problem of commitment due to lack of decision power prevent the WEO from acting. It is only Woreda Council and Regional Council who can act.




The existence of management guidelines at Woreda level (i552). Although the School Improvement Program (SIP) management guidelines exist at WEO level, as part of the GEQIP implementation, there were reports that were not sufficiently disseminated at the cluster and ABEC level.

During several field visits, it was observed that although GEQIP SIP guidelines were available in visited WEOs, this was not fully the case at the ABEC level. ABE facilitators were not entirely fully capacitated to use the grants.

This is combined with the lack of sufficient capacity at ABEC level to manage the center. The ABE facilitators/teachers and directors are partly receiving training on school management, on the implementation of the School Improvement Plan. This was also applicable to ABE, as they reported very low level of capacity to do so, which was coupled with the absence of available guidelines at the school level. Additionally, none of the members from the CMC met during field phase received training on school management. This tends to support that local capacity remains limited to effectively manage funds to fulfill the SIP. Budget restriction was reported as a main reason for low capacity building. (i554, see also JC5.6)

To monitor the implementation and educational achievement of ABE (i551), a reliable monitoring system is fundamental. ESDPs enacted as guiding objectives for the improvement of a monitoring system. The education Management Information System (EMIS) booklets compile key indicators on access, coverage, internal efficiency, equity and quality nation-wide. Likewise, different regions produced their own abstracts.

When comparing Regional Education Statistics Annual Abstracts, it is striking to see the variation of basic implementation and educational performances information available. The paucity of data and discrepancies in data across regions and across woredas is a main challenge which impedes the proper monitoring of educational outputs and outcomes for improvement. This goes far beyond what is problematic for ABE, but it is particularly striking to see that EMIS data on ABE are under-recorded compared to formal primary school.

The existence of school registers in ABECs (i555) can inform on the basic monitoring capacity available at the school level. School paper-based registers are utilized, and implemented at every formal primary school (100%) to take stock of enrollment and the daily attendance. By contrast the full use of registers is not implemented in all ABECs visited (89% in government-run ABECs) and is interestingly the lowest in UNICEF-supported ABECs (83%) (Table 33 Existence of school registers at schools).

Educational achievements are also recorded at the school level by ABE facilitator, based on the collective and formative achievements, which are reported in certain cases, quarterly and in some yearly. The school registers observed during field visits suggest a locally developed solution rather than a systematized process, making the quality data uneven and barely possible to disaggregate, merge and compare.

Table 33 Existence of school registers at schools

Name of indicator

Type of school

UNICEF-supported

ABEC

Government

ABEC


ALL schools

Schools with school register 83% 89% 100% 89%





The cluster supervisors are in charge of collecting data from the formal schools and ABECs in part of each Cluster Resources Center. The supervisors pick up the summaries prepared by ABE facilitators which gathers implementation data (enrollment, attendance) and educational achievements.

The challenging work supervision conditions are an impediment to accomplish monitoring, as each supervisor is in charge of five to seven schools and ABECs distant from each other and are hard to access. This is coupled with low financial support for transportation. Although a budget from GEQIP is allocated for fuel and a motorcycle for supervision, this still seems to be problematic in ensuring thorough supervision. Consequently, there low degree of supervision tends to encourage absenteeism as a strong side-effect (Dyer and Engdasew 2016).

Supervision accountability is also a missing element which constrains the monitoring quality, due to limited government funds for monitoring supervisors. Moreover, the low capacity level of CMCs to support monitoring ABEC also impedes on accountability.

JC 5.6. ALL LEVEL OF STAFF IS WELL TRAINED TO MONITOR ABE AND SCHOOL ACHIEVEMENTS AND IMPLEMENTATION

The monitoring capacity building trainings (i561, i562 and i563). UNICEF has been a leading development partner to provide capacity building trainings to various ranges of implementers, like REB, Zonal and woreda education offices and cluster supervisors. Given the lack of monitoring capacity, training has been given to ABE facilitators as well as CMCs were also provided in Afar as part of the efforts to mainstream the School Improvement Program.

These efforts are widely acknowledged and the training and are deemed very useful, high quality, albeit not sufficient. The trainings provided by UNICEF encompass topics on quality improvements such as on EMIS and school grant management. The trainings were reported to be the appropriate place to share good practices. As an example, UNICEF’s training in Afar allowed improved supervision practices, reported in one woreda which was in random selection of ABEC. This prevents artificial display cases during supervision.

Nevertheless, these trainings remain insufficient to help WEO staff and cluster supervisors and lead to sound systematization of monitoring practices. Despite the trainings, the reported actual level of monitoring of ABE implementation and achievement falls short due to high staff turn-over and lack of institutional memory. During one field mission, educational documentation such as reports or abstracts were not available to the personnel in charge at the time had left their position with the document. This anecdote from the field mission illustrates the complexity of relying on stable and capacitated staff.

This lack of capacity to manage education and to monitor key performance indicators turns into low implementation achievements and coordination between the levels on one side and high discrepancy in monitoring data collected on the other. Ultimately this makes inadequate evidence for strategic planning, and to monitor the progress of implementation reforms, not only for ABE but more generally. (i562 and i563).




EVALUATION QUESTION 6 - EFFICIENCY

EQ 6: Was the Alternative Basic Education program implemented in the most efficient way to

increase access to basic education?

ABECs were designed as a low-cost modality to mitigate the issues of poor access to primary education for out-of-school children in pastoralist areas. A low-cost infrastructure and local educational personnel recruitment from the community, combined with flexible delivery methods were employed with the hope that it would increase access to basic education for hard-to-reach children in pastoralist areas.

With this evaluation question, the evaluation team seeks to assess the extent to which ABE has

converted its resources (funds, expertise, time) into results to achieve the maximum possible

outputs, outcomes and impacts, by means of the minimum possible inputs. However, estimating all the costs and benefits of ABE is a particularly challenging task. The limited availability of cost-related information and partial information on the benefits from ABE make the assignment difficult to evaluate.

The suggested strategy is to consider the estimated cost of ABEC activities related to selected outcomes of interest (i.e. cognitive score, enrollment).

In addition, estimating the ABC program efficiency questions the related concept of cost-effectiveness of ABE. In that regard, the evaluation assesses the extent to which the ABE program has achieved or is expected to achieve with its results or benefits at a lower cost compared with alternatives (World Bank, 2008). The main obstacle encountered while conducting this assessment is the very availability of reliable information, notwithstanding the complexity to put a monetary value to all costs and all benefits of ABECs and of other modalities of primary education (Formal schools, mobile schools, para-boarding schools, hostels). This does not allow for a complete comparative analysis and therefore, this assignment has limited perception of various stakeholders on the cost-effectiveness of ABE. An estimation of the costs of mobile schools, and para-boarding school, received based on the AWPs shared will help giving a reference compared to ABE.

Altogether, taking stock of the budget modalities will be a first necessary step: the budget modalities employed to finance ABE must be suitable to ensure that the resources can be turned into maximum outputs, outcomes and impacts. This will focus on UNICEF’s support to ABE, based on a review of the Annual Work Plans in the three regions of the assignment, Afar, Oromia and Somali. The cost effectiveness and efficiency will also be assessed for UNICEF’s support to ABE.

JC 6.1 THE BUDGET AND BUDGET MODALITIES ARE ADAPTED TO THE NEEDS FOR BASIC EDUCATION

The GoE contributes to ABE via different budget modalities (i612).

� Public Budget support / Promoting Basic Service Program (PBS): the government funds, through the release of block grants, are managed by the Ministry of Education and finance both recurrent costs corresponding to teacher salary and capital costs corresponding to school construction

� Pooled Fund as part of the General Education Quality Improvement Package (GEQIP): The first phase of GEQIP (2009-2013) was rolled out and the second phase of GEQIP (2013-2018) is still ongoing. The GEQIP Phase I was included to support to ABECs. Challenges have been identified at the end of the first phase, with respect to the capacity of emerging regions to implement reforms. Moreover, GEQIP I did not include an explicit focus on children with special needs. Based on a social assessment mandated by MoE, GEQIP II was designed to build on lessons learned from the first phase.




UNICEF has supported the MoE in its ABE modalities via two main modalities. Firstly, UNICEF has been wiring its contribution to REBs, via Regional BoFEDs, based on the Annual Work Plans on UNFAD Basic Social Services – Learning and Development Pillar. These budget modalities have been assessed as very useful by stakeholders from REBs.

Since 2016, UNICEF has participated in the GEQIP Pooled Fund, managed by the Ministry of Finance and Economic Development (MOFED) to support to School Improvement Program focused on Quality. This reflects a relevant shift in strategic orientation to focus on improving the quality of education, instead of a support to ABE expansion through construction, judged not sustainable. (i612)

The evaluation focused on UNICEF’s support to the Regional States, based on the Annual Work Plans for UNFAD Pillar 2: Basic Social Services – Learning and Development (Education) reviews for Afar, Oromia and Somali regions. This does not include UNICEF funds to the Federal Ministry of Education nor its participation to GEQIP pooled funds.

The AWP review is based on the following methodology: Each budget line that explicitly refers to ABE was taken into consideration. It excludes activities that were directed to woreda, zonal or regional level, such as development of EMIS/SMIS, unless it was explicitly related to ABE.

In the case where an activity is provided to both formal school and to ABECs, the budget allocated to ABECs is an estimate based on the share of ABECs in the region as part of the total number of primary schools.32

The final total of UNICEF’s support to ABE in comparison to the absolute amount of UNICEF’s support to primary education sector between 2006 and 2009 E.C. in Afar, Oromia and Somali is illustrated in ABE Error! Reference source not found., Error! Reference source not found., and Figure 3 in Annex 12. While the overall budget share allocated to Basic Social Services – Learning and Development shows that it is shrinking over years in the three regions, the share of the support allocated to ABE as part of the total support is also drastically reduced.

In the Afar region, the share of UNICEF’s support for activities related to ABECs dropped from 47% in 2006 E.C., making it the most important basic education modality supported, to 23%.

In the Oromia region, the share of UNICEF’s support follows a similar trend. In 2007 E.C., the activities in support of Alternative Basic Education amounted to 37% (1,036,900 USD) of UNICEF’s total support to basic education in the region (2,794,125 USD). In 2009 E.C. this share decreased by 20 points, dropping to 17% (253, 500 USD) of the total budget (1, 464, 017 USD).

In the Somali region, the share of support to interventions focused on ABECs represented nearly 59% of UNICEF’s support in 2006 E.C. (1, 184, 002) and plunged to 24% in 2009 E.C. (97,108 USD). (i611)

At the regional level, the decrease of budget is perceived as a constraint on the provision of

education, according to REB officials. The shift of budget modalities from AWP support to the participation into the federal pooled funds may further constrain the financial capacity of regional

32 This estimate does not take into account the proportion of ABECs as part of the education modality mix compared to formal primary school and other the alternative modalities. One possible limit to this approach is that it may exclude activities that were targeted both to ABECs and primary schools, or activities at woreda, zonal and REB levels that have an indirect impact on ABECs through the management improvement/monitoring capacity. One limit that is specific to Oromia AWP is the absence of disaggregated activity lines which do not allow for detailed analysis of sub-activities (construction, capacity building, furnishing and other) targeted to ABECs. The bulk budget line was therefore taken into account.




states to implement the decentralized management of ABE and jeopardize efforts to improve the quality of education.

This evolution of UNICEF’s support towards the GEQIP pooled funds and shrinkage of budget as part of AWPs was justified by the need to rationalize public spending and to increase the ownership of national authorities on ABE via policy dialogue. This also operationalizes the urge to concentrate resources on quality improvement needs, assessed in terms of quality of teachers, the need to upgrade and expand ABECs to the upper cycle of primary school. In parallel, the recent participation of UNICEF in the pooled funds also relates to this need to commit to increasing the quality of primary education and to increase the ownership of the national authorities of the program. (i613 and i614)

EVALUATION QUESTION 7 - SUSTAINABILITY

EQ 7: How did the ABE program encourage ABE-related activities within communities? In

general, do more active communities demand for more ABE services?

The seventh and last evaluation question of the assignment focuses on the sustainability of ABE.

Sustainability must be gaged in terms of ownership of the authorities and their capacity to continue after UNICEF funding ceased but also the ownership and capacity of communities to manage and increase the demand for ABE.

JC 7.1 COMMUNITY AND PARENTS GIVE IMPORTANCE TO EDUCATION

The establishment of the ABECs in pastoralist and agro-pastoralist communities has been accompanied with awareness-raising activities concerning the importance of education led jointly by WEOs and UNICEF in the communities. Participants in FGDs in villages that benefit from a UNICEF-supported ABE revealed that education was a minor matter when the school did not exist. Yet, the establishment of ABEC and awareness raising activities provided by WEOs, Cluster Supervisors and ABE facilitators have changed the overall perceived importance to education: “Our way of thinking in

regard to education has changed a lot.” (CMC member, Oromia).

The perceived importance of schooling was also reflected in parent respondents’ attitude of choosing work over education. Respondents unanimously responded that this is not desirable to ask their child to not go to school when there is an important chore to execute at home/work. Although this may not be related to an actual change in behavior, these attitudes towards education may indicate an increased enthusiasm to prefer primary education over involvement of school-age children in pastoralist activities.

An awareness on the importance of girls’ education is a notable achievement of ABE program and other educational initiatives of government and development partners, reported by all levels of respondents. Girls’ education is reported as crucial so that they know their human rights and empower themselves as equals to their male peers. Enhanced access to school through ABEC, but also awareness raising campaigns to communities led by woreda officers, cluster supervisors as well as ABE facilitators contributed to this change in attitudes with respect to girls’ access to education results. (i711)

The economic and social returns on education associated with going to school take on a variety of short-term features in villages where an ABEC is in place. An educated child can support his/her




parents with every-day tasks that require basic literacy and numeracy skills, such as dialing a phone number or writing a letter.

Long terms pay-offs for the children, the family and the community were also identified in villages benefiting from a UNICEF-supported ABE. The community members perceived that primary education is a necessary step to access secondary and higher education, and paves the way for qualified carriers, such as engineers, government officers, or doctors.

The benefits of education also extend to the community—not only will educated children enable their household to access a higher socio-economic status but also they will be able to contribute to the community socio-economic status elevation and empowerment, with increased skills to cope with the community’s resource scarcity and participate in the decision-making. “You cannot be a community

leader and even lower position in the kebele if you do not learn” (CMC member, Oromia). (i714)

However, the beneficiary interviews suggest that the perceived immediate costs of having a child at school may still be an obstacle to sending him/her to school. Namely, the opportunity costs resulting from school attendance of education remain key determinant to schooling, despite the relative proximity of the ABEC and absence of school fees. The schedule flexibility offered by ABE has been appreciated as it tends to avoid conflicting timing of school with pastoralist activities. Due to imperfect implementation this may still not prevent drop-out (see i316).

This supports existing literature’s findings that the education opportunity cost of children’s time remains high in rural pastoralist and agro-pastoralist context (Mason and Khandker 1996 ; Hedges et al. 2015 ; Heffernan et al. 2001). Although this is never explicitly referred to as a cost, the need for child labor conflicts with delivering education, is a reason why parents in the villages may not send their children to school. The urge to involve the child in economic activities of the household (like chores) is closely tied with the perceived severity of the drought on the community, mostly in Somali and in South Oromia. (i713)

JC 7.2 COMMUNITY AND PARENTS GIVE IMPORTANCE TO ABE

The establishment of the ABEC is perceived as an opportunity to increase access to education. Activities of construction, selection of the location were appreciated. The relative flexibility of the school schedule is a main source of satisfaction. (i721)

JC 7.3 PARENTS / COMMUNITIES TAKE PART IN THE ABE DECISION-MAKING PROCESS

The focus on engaging communities via the CMC in school planning and management has been put forth with ESDP III (MoE 2005). The following ESDP IV (MoE 2010) and ESDP V (MoE 2015a) have reassessed the requirement that communities are involved in using the available funds, namely school grants and block grants in order to maintain minimum school standards that support improvement in learning outcomes. The National Pastoralist Education Strategy (MoE 2008) also states this as a guiding principle: “Mobilize and develop a sense of ownership in the community so as to enable it to

actively participate in educational planning, building schools, monitoring and evaluation, etc.”

“Parents and community members, through PTAs and CMCs, have a vital role to play in deciding how the

School Grant will be used to improve the quality of education children receive, and in performance

monitoring.“ (MoE 2011).

To support the REB in implementing this principle, UNICEF has dedicated a share of resources to the REB to raise awareness on the importance of education at the beneficiary level.




The review of AWPs allows identifying the scope of UNICEF’s support to various regions in relation to community capacity building. The absence of complete monitoring data on the implementation of the interventions, diminishes the analysis’ power.

The planned activities have directly included the mobilization of the communities to form a CMC during the establishment of ABECs. Table 34 describes the planned interventions relative to facilitator and CMC members training at the establishment of the ABEC with the training of five to seven of CMC members.

Additionally, with respect to the establishment activities, there were plans for further awareness raising activities to sensitize communities on the education of hard-to-reach pastoralist children, via capacity building activities directed to CMCs and to ABE facilitators/woredas/Cluster supervisors to deliver further awareness-raising activities such as, back to school campaign. Such activities are relevant, although the issue of high turn-over tends to limit the effectiveness and the impact of the interventions.

The planned UNICEF funds for CMC capacity building activities remain timorous in terms of coverage, number of CMC to train and funding, relatively to other more costly activities, such as ABEC construction. In line with the programmatic activities, the CMC members participating in the FDGs did not recall any participation in training on ABE management at all33. There seem to be a gap in the implementation between the objective of providing continuous orientation and training and the means provided to fulfill this objective. Nevertheless, the initial awareness raising activity that came with the establishment of the ABE may be significantly responsible for an increase of awareness described above and translating into an increased demand for education.

Table 34 Planned UNICEF funded interventions for facilitators and CMC members training

Facilitators and CMC members training

13. Facilitators on-job training 2 times per year 3 facilitators

14. Facilitators upgrading trainings (School Cluster Resource Center and Distance learning)

3 facilitators

15 Center Management Committee members (5 to 7 members) training

7 members

Source: UNICEF

The outcomes of ABEC establishment as far as community participation is promising. The communities that benefit from an ABEC and had a CMC, or alternatively have a Parent Teacher Association (PTA), composed with members of the community, parents, and the Kebele head. There were five to eight male and female members of the CMC who reported to be elected by the community members to manage the ABEC together with the ABE facilitators and the Kebele leader. In contrast, the inclusion of student members was not observed. In the visited communities, the CMC selection took on various forms, from participatory community-based elections to election restricted from elders of the community, and the kebele management board. There were no specific guidelines on how the CMC

33 Note the qualitative sample size does not support capturing this activity during the field phase.




selection process should happen and it seems to rely mostly on the local political structures already in place (i731, i732).

The establishment of the CMC drove a formalization of education-related community

discussions. Communities visited during the field mission reported holding CMC meetings on a regular basis, bi-monthly or weekly, in addition to extraordinary meetings in case urgent issues arise (i734). These issues are related to education delivery quality are discussed during CMC meetings and two CMCs out of six indicated during the FGDs taking notes during the meetings, written by those who are literate. (i735)

Despite the notable positive contribution of ABE in formalizing the community’s responsibility to education, a real capacity of the CMC to manage the ABE resources remain limited. The actual realm of competencies of the CMC, although promising, does not make up for the fact that the CMC is often limited in its power to manage the utilization of funds. CMCs often confine their actions to awareness-raising activities, such as identifying and encouraging parents of out-of-school children to send them to school. In some cases, the CMCs reported participating in teacher recruitment by recommending a candidate to the WEO through the Cluster Supervisor. To some extent, CMCs play a role in reflecting the needs of the communities in terms of adapting the schedule, operationalizing the concept of flexibility: “I want to add that as you know this community is sometimes doing agriculture. The

agricultural outputs are being mostly collected in the morning which clashes with the children’s schooling

since the children also help in the agriculture work. In this critical time, the committee [CMC] is able to

rearrange the schedule and shift the school to the afternoon session after meeting with community” (CMC member, Oromia). Nevertheless, the spectrum of decisions taken by CMC remains somewhat limited and often it is not sufficient to ensure a proper management of the ABEC, when it is left to the sole supervision of ABE facilitators or directors.

JC 7.4 PARENTS/ COMMUNITIES ARE INVOLVED IN RUNNING THE ABECS

Support to the ABE construction is the first and main type of community contribution to the ABEC. This comes as a relative success with a reported high involvement of community members in the provision of local construction material (wood, stones) water and labor force, according to CMC members met during FGDs and parents.

By contrast, in-cash contributions to the ABECs are very rare, given the scarce resources of pastoralist monetary resources. It is still interesting to mention that in both UNICEF-supported villages visited in Oromia, community members were invited to contribute in cash to an amount of several birr for the general costs of the school and for the salary of the teacher.

Echoing previous findings, the role that the CMCs can play is extremely limited as regards to monitoring and evaluating the quality of teaching and learning, which is due to lack of capacities. Aside from the relative lack of activities to build CMC members’ capacity, the poor or inexistent literacy and numeracy skills of its members limits the possibility to effectively monitor and evaluate the quality of the school and to manage funds. Similarly, community contribution to the management of the ABECs, in particular to the implementation of the SIP, still remains limited.

The role of CMC in the day-to-day school management has taken the form of mobilization to promote school enrollment and attendance through the back-to-school campaign. CMCs control absenteeism in cooperation with the ABE facilitators by reaching out to parents whose children missed school and encouraging them to register their children to send them to school. Beyond a mere warning and awareness-raising, two CMCs in the Afar and Somali regions reported coercive actions (like a fine of up 30 to 100 ETB) to punish parents for not sending their children to school. (see i750)




To ensure consistent education delivery, the support of the community to the ABE facilitators/teachers is generally good. ABE facilitators who were interviewed described very supportive communities which provide meals or occasional accommodation, depending on their capacity. Most communities have reported good relationship to the ABE facilitators, even when they are not from the community, and have reported a mutual collaboration.

The relatively low capacity to manage the school funds nuances the reported general feeling of empowerment that the community members expressed, in particular that CMC can report their needs to higher level. (i762)

JC 7.5 PARENTS/COMMUNITIES ARE INVOLVED IN AWARENESS RAISING ACTIVITIES ON ABE AMONG THE COMMUNITY

ABE’s main achievements in community mobilization can be found in the domain of awareness-raising. CMCs, together with ABE facilitators and cluster supervisors participate in disseminating awareness raising messages to the community, in particular through the back-to-school campaigns and regular community meetings.

Empowering women in education is a powerful lever to promote universal education but deserves more solid intervention focus. If women’s involvement is made possible via CMC, equality still cannot be fully achieved. UNICEF’s supported CMC capacity building and awareness raising do not do enough to include gender equity as an objective, with the exception of the Somali region explicitly including a mandatory quota of 20% of female participants. One out of six CMCs visited did not count any woman in its members. This indicates an inclusion of women in education related matters, but an absence of quantitative data on CMC members does not allow us to quantitatively confirm this evidence. In contrast with this finding, female parents interviewed reported that they were not be addressed by the CMC, or were left out of community education-related discussion. (i750, see i742)

JC 7.6 PARENTS AND COMMUNITY DEMAND FOR ABE

An increase in the demand for education has accompanied the establishment of the ABECs in the villages. This observation can be supported with the expectations related by the various beneficiaries met. While the opening of the ABE is recounted to have met community expectations in terms of access, expectations for the future shifted to the quality aspects, the quality of the infrastructures and equipment, as well as expansion of classes to guarantee access to secondary cycle education. In particular, renovation is seen as a condition for success in order to provide quality education. (i761, i765)

Establishing ABECs in the villages has been reported as beneficial by all the communities. However, education access for children with special needs is still an issue. Covered as part of ABE facilitators training, the ABE facilitators/teachers do not have the means to actually provide any support to special needs children, in particular in situations of mental or physical handicaps. Adolescent girls, like ones who are going through puberty, are also lacking special support, making the fear of drop-outs an issue for concern. UNICEF supports adolescent girls through other types of interventions which are not explicitly included in the support to ABE. (i764)

JC 7.7 GOVERNMENT INCREASED OWNERSHIP OF ABE PROGRAM

Increased government ownership of the ABE program is a success factor, which can ensure the sustainability of the program when the donor funds stopped. Yet the very question of whether the sustainability of ABE is desirable or not is a major point that makes its development unclear. ABE is




characterized by relatively low interest of authorities, in comparison to formal primary school and is seen as a temporary measure doomed to be eventually phased out.

A significant decrease in UNICEF’s contribution to regional states has been reported by stakeholders at regional level as a risk for the continuation of ABE, as it affects the decentralized model of education delivery. On the other hand, this decrease of UNICEF’s funding has been justified by the low level of government investment in the program relative to UNICEF’s large contribution; in particular in ABEC construction, furnishing the classrooms, textbook provisions, and teacher training.

Similarly UNICEF’s contribution to summer trainings has reached an end because of low governmental endeavors. Yet, this is somehow mitigated by the inclusion of increased focus on the development of teacher capacity as part of the GEQIP, revealing an increased government’s ownership to train teachers for enhanced education quality. As ABE is integrally part of GEQIP, potential improvement of government ownership can be expected, under the condition that an accountability system is set in place to ensure the equity of funds delivered to ABEC and to formal primary schools.

UNICEF’s support has been directed to the development of human capital at the REB and woreda level, with capacity building interventions in various thematic areas: mainstreaming of SIP principles, EMIS/GIS system training and improvement of formative/continuous assessment. The staff salaries are covered by government funds, which represent a positive potential for sustainability. However, the impact of these activities is undermined as there is high staff turn-over and this prevents sustainable capacitated human resources and internal continuous professional development. (see also EQ5)

In addition, the activities comprising the development of manuals, curricula, guidelines, and standards were led in cooperation with the REBs in order to ensure continuation of capacity building and increased ownership of the program. To secure buy-in, validation workshops have also been organized. However, according to external expertise, the organization of validation workshops remains a formality which is not followed with attendance of stakeholders for validation and does not translate, at the authorities’ level into sustained practices.

In the Somali region, an advocacy platform called, the Pastoralist Education Forum, took place in 2016 for the first time with participation of UNICEF-supported pastoral woredas in the region. All the NGOs working on education participated in the workshop. As part of the activities, the new National Pastoralist Education Strategy that is now under revision was discussed with active participation of NGOs and WEOs. This was considered a positive practice to allow for mainstreaming of ABE, as part of pastoralist education modalities.

The performance of the ABE system goes beyond UNICEF scope and must involve the authorities commitment at all levels. The lack of a proper accountability system, including a functioning M&E system, and assessment of staff performance at all levels, undermines the effectiveness and potential sustainability of ABE as it can impact the quality of delivering education. The path recently taken by UNICEF to focus its support to policy level reflects attempts to create awareness and urge the government to contribute more to ABE (see also EQ2).




6 CONCLUSIONS

These conclusions are based on the different evaluation questions (EQ) shown in Table 3. Conclusions are phrased to answer specific program aspects.

CONCLUSION 1: ACCESS

Conclusion based on EQ1.

� This study has found a significant improvement of primary school enrollment rates for children aged 7-11 in villages with ABECs in comparison to similar villages without any school: School enrollment probabilities increase by 17 percentage points (pp). There is suggestive evidence that this improvement essentially came from a reduced distance to school. It appears that the mechanism of network cards was being used only marginally.

� ABECs and primary schools were equally able to increase attendance in villages where no school exists. At about a third of the cost of a formal primary school, ABECs can be considered a cost-effective way to increase attendance. Yet, the fact that only 50 % of primary school aged children attended school when no primary school was available in the village indicates that ABECs do not fully solve the problem. The impact of ABECs on attendance only accounted for about half of the attendance needed to reach the national average (87%).

� Apart from distance, the main reason for not sending children to school is household work. This applied to pastoralist and non-pastoralist households. This indicates that the economic and social situation of a household was instrumental in explaining low attendance.

CONCLUSION 2: SCHOOL PERFORMANCE


� The results of EQ2 indicate that going to an ABEC increased the performance of children in mathematics significantly on the highest level by about half a category in comparison to children that do not attend any form of primary schooling. The effect was larger for primary schools: (The effect size was 0.9 category levels); therefore children seemed to learn more when going to a primary school in comparison to an ABEC.

� Comparing UNICEF-supported ABECs and other ABECs, no differences in mathematic performance has been found. However, children from UNICEF-supported ABEC had larger vocabulary scores and were less likely to be illiterate.

CONCLUSION 3: FLEXIBILITY


� A key takeaway is the ability of ABECs to deliver education in a flexible way in terms of daily schedule. In a situation where many pastoral parents rely on their children to help in the household and with the cattle, ABECs can play an important role in reaching out-of-school children.

� By definition, if an ABEC is constructed as a building, children cannot continue schooling in that ABEC during seasonal migration. The Learner Networking Card was a promising innovation in that context, allowing children to go to another ABEC in a different community when moving. However, the Learner Networking Card lacks ownership from national and




regional governments so far and has not been used much. It could however be an important tool in emergency situations like the current drought in Somali, though.

CONCLUSION 4: FACILITATORS’ QUALITY


� The findings from the quantitative study indicate that government ABECs had teachers that

have more teaching experience, receive a higher salary and have formal teacher training. This

could be because UNICEF- supported ABECs were in the poorest areas that have attracted less

trained teachers.

� Increased efforts on the part of UNICEF to provide facilitator trainings might have translated into more qualified teachers in general, but there was not any particular targeting of UNICEF-supported ABEC facilitators.

CONCLUSION 5: QUALITY

Conclusion based on EQ1, EQ4 and EQ7.

� It was observed that UNICEF-supported ABECs were more likely to be either new or in very good condition (44% vs. 14%), in a proper school building (73% vs. 61%) and not outside under a tree (22.5% vs. 26%) than other ABECs. Additionally, UNICEF-supported ABECs had more classrooms and a significantly lower student teacher ratio. Furthermore, water for drinking and washing hands was more frequently available in UNICEF-supported ABECs (about 40% in primary schools and UNICEF-supported ABEC vs 17% for all other ABEC).

� The same was true for toilet facilities and classroom equipment like proper seating and desks for the students. In general, UNICEF contributed to an increase of ABEC infrastructure and classroom equipment; however it still did not reach the level of formal schools.

CONCLUSION 6: MANAGEMENT FRAMEWORK/SUSTAINABILITY

Conclusions based on EQ3 and EQ7.

� Desk reviews and in-depth interviews revealed that capacity building activities at regional and woreda / kebele levels could still be improved. Woreda Education Officer, Cluster Supervisors and CMCs did not have the sufficient abilities and guidelines to manage ABE implementation and ensure education quality (based on EQ3 and EQ7). High staff turn-over further constrains sustainable capacitated human resources and internal continuous professional development. The lack of a proper accountability system, including a functioning M&E system, and assessment of staff performance at all levels, undermines the effectiveness and potentially the sustainability of ABE

� The evaluation showed that ABE had not been given enough priority at regional and federal levels. This did constrain the delivery of ABE and will constrain sustainability in the long run.

CONCLUSION 7: SUSTAINABILITY OF ABE AT LOCAL LEVEL

Conclusions based on EQ7.

� The establishment of ABECs in the communities with the support of UNICEF raised the awareness of the parents on the benefits of educating their children and increased the demand to upgrade and transform ABECs to conventional/ formal primary schools




� This has translated into the involvement of the communities via CMCs in tracking out-of-school children, enhancing enrollment and preventing drop-outs.

� However, the capacity of CMCs to manage ABECs is limited due to the lack of local resources and qualifications. Decisions taken at community-level remain symbolical, e.g. the choice of the ABEC location.

CONCLUSION 8: RELEVANCE OF UNICEF’S SUPPORT TOWARDS ABE

Conclusion based on EQ5 and EQ7.

� UNICEF’s support to ABE was aligned with the country policy and strategy in place. UNICEF is a respected partner and Ethiopian stakeholders considered the intervention to be important and professionally conducted.

� The ABE construction activities that UNICEF funded were coherent with the initial need of ABE to extend the access to education. Yet, UNICEF’s shift of focus towards the quality of education was relevant, albeit insufficient to enhance the quality of all ABE facilitators and to extensively improve the management monitoring capacity of education officers at regional, woreda and kebele levels.

� The new focus of the UNICEF 2016-2020 Country Programme on planning and policy dialogue in support of ABE is adequate to develop Ethiopian stakeholders’ ownership of ABE and to improve capacity building at federal, regional and local levels.

CONCLUSION 9: RELEVANCE OF ABE


� The ABE curriculum originated directly from the formal primary school curriculum. The initial attempt to condense the curriculum into three years instead of four and to translate it into regional languages showed that some effort was made to adapt to the needs of local communities. Key-informants stated that the efforts remained insufficient and were not maintained to fully fit the needs and lifestyle of the pastoralist communities. The federal Ministry requested the adoption of the formal primary school curriculum instead of the ABE condensed curriculum. There is a lack of coherent stand among Ethiopian stakeholders on the relevance of a distinct and condensed ABE curriculum.

� The program documents on ABE specifically identified the learning needs to pastoral children. The development of a specific ABE modality was relevant. However, the lack of endorsement of a clear and binding pastoralist education strategy with ABE as the cornerstone has prevented efforts from fully matching learning needs of pastoralist children. ABE has remained a temporary solution to extend access to education and did not receive the necessary resources nor supervision to provide quality education.




7 OVERALL ASSESSMENT

The overall assessment of this project was done based on the conclusions and findings of each

evaluation criteria. The assessment was done on a scale reaching from very satisfactory, moderately

satisfactory and unsatisfactory

Evaluation

Criteria

Assessment

Effectiveness / Impact

The effectiveness and impact of UNICEF support to ABE was moderately satisfactory. On the one hand, the increased availability of ABECs did improve primary school attendance rates, particularly in the remote and pastoral communities. Furthermore, the building of new ABEC by UNICEF improved the school infrastructure significantly in comparison to the old/other ABECs run by the government. On the other hand, in terms of learning outcomes, UNICEF-supported ABECs did not perform any better than other ABECs and significantly worse than formal primary schools.

Relevance The relevance of the ABE curriculum and the implementation modalities to the needs of pastoralist communities was moderately satisfactory. The attempts to develop an ABE modality were not supported by a strong strategy or coherent guidelines. UNICEF’s support to ABE was relevant as it strived to adapt to the needs of ABE improvement of the quality of education with UNICEF’s recent participation in GEQIP pooled funds and to the increased ownership of the Ethiopian stakeholders. Yet, the overall budget shrinkage and focus on policy dialogue risks undermining efforts on quality of education in ABE.

Sustainability The achievements of ABE regarding sustainability were moderately satisfactory to unsatisfactory. Local awareness on the quality of education was improved and led to an increased demand for education. However, the sustainability of local capacities of staff was undermined by high staff turn-over, insufficient provision of training, guidelines and budget provision to support the management of ABE. The lack of interest at the regional and federal levels further constrained the sustainability of ABE.




8 RECOMMENDATIONS

The recommendations presented below are based on the findings and conclusions of the evaluation as well as on consultation with all key stakeholders.

The recommendations are addressed to the Ministry of Education, UNICEF Ethiopia and stakeholders at national and local levels, to engage all major stakeholders in an ongoing effort to improve ABE and to bring quality education to rural pastoral communities.

Recommendations are divided into three categories, short-term, medium-term and long-.term recommendations.

Recommendations Addressee Based on

conclusion

number:

Short-term Recommendations • To ensure long-term success and improve learning outcomes it is

essential that MoE take over full ownership for ABE as a primary education modality. That means introducing the ABE teaching and modalities into the revised and endorsed Pastoralist Education Strategy.

MoE

Nr. 1, 6, 7

• Efforts should be dedicate to strengthening the supervision and monitoring of ABECs. This includes conducting frequent supervisory support, increasing the financing of the existing resource centers, and strengthening capacity building for cluster supervisors and Woreda education officers.

MoE & UNICEF Ethiopia

Nr. 6, 7

• The performance accountability system should be reinforced with the development and regular use of professional performance assessment. Criteria and processes should be developed and approved by a committee at Federal level, including UNICEF and the REBs.

MoE

Nr. 6

• To improve learning outcomes for ABE students the National Learning Assessment test of grade 4 learners from the formal primary school system should be extended to ABEC. This would also increase ownership by MoE for the ABE system

MoE Nr. 2, 6

Medium-term Recommendations • To improve primary school completion rates and the learning

environment, the implementation of ESDP IV/V strategies for hard-to-reach children should be further implemented

MoE, local stakeholders

Nr. 1, 6

• Extended support to ABE facilitators should be prioritized via adequate teacher training on active teaching methods, professional assessment and supervision and experience sharing among teachers

MoE, REB, UNICEF

Nr. 4

• Provide context-specific incentives for the ABE facilitators. This can include, for example, the opportunity to upgrade to formal teachers MoE, REB Nr. 4

• Improve the physical environment of the ABEC as per the standard, making schools child friendly in all aspects MoE, REB Nr. 5

• Supply essential teaching and learning materials including teaching aids, related reference books, workbooks MoE, REB Nr. 5

• Upgrade some selected ABEC from lower primary (level 1-4) to upper primary (level 1-6) as many ABECs end at level 4. This situation makes it difficult for children to find places for further learning. Upgrading will make the learning place close to the children’s home. This will

MoE, REB Nr. 1




enable pastoralist children to complete primary education easily and be prepared to start their secondary school learning.

• Establish small multi-grade approach teaching so that more children have access to a primary school within close proximity to their home MoE, REB Nr.1

• Based on the assessment that some pastoralist communities still perform seasonal migration there is a clear need to promote and provide mobile schools as well as institutionalize and scale up the Networking Card System in all pastoralist districts.

This should include advocacy, planning, procurement and training at the woreda and community levels. UNICEF is best placed to advocate for this at the federal and regional levels with the support of local NGOs collecting the good practices and lessons learned at the local level. Training on the Learner Networking Card could be led as an additional module to the on-job facilitator training and to the communities via cluster supervisors and woreda education officers in partnership with local NGOs.

MoE, REB, UNICEF

Nr.1, 3

• Contextualization of the ABE curriculum should be continued based on the formal primary school curriculum together with a consideration to develop a distinct ABE curriculum extending to the second cycle of primary school. Contextualization should include local implementing partners, representatives of communities, and ABE teachers/facilitator. Local education research institutions should be included to add research evidence to demonstrate how a child- and context-sensitive curriculum can be developed, meeting the quality of the formal primary education.

MoE, REBs Nr.9

Long-term Recommendations • In order to reach universal school attendance in remote rural areas

and pastoral communities, incentives to attend ABEC / primary schools will have to be increased and poverty must be addressed in a more holistic way

UNICEF, MoE, local stakeholders

Nr. 1




9 ANNEXES

Annex 1 – Terms of references

Annex 2 – Theory of Change

Annex 3 – List of informants

Annex 4 – List of locations visited for quantitative and qualitative data collection

Annex 5 – List of documents consulted

Annex 6 – Evaluation Matrix

Annex 7 – Sampling Protocol

Annex 8 – Oromia schools and UNICEF-supported woredas

Annex 9 – Further regression outputs

Annex 10 – Pictures

Annex 11 – ABE facilitator/teacher training activities planned in AWPs

Annex 12 – Evolution of UNICEF’s support to ABE per Region

Annex 13 – Interview Checklist

Annex 14 – Formatted Transcripts

Documents

171101 ABE Final Report 22.09.17 - UNICEF › evaldatabase › files › Ethiopia... · An Impact Evaluation on the effectiveness of Alternative Basic Education in Ethiopia – (Draft)