Public Disclosure Authorizeddocuments.worldbank.org/curated/en/947801468312895835/pdf/ICR… · The Project Development Objective (PDO) is to increase access to improved basic education

Document of The World Bank

Report No: ICR2138

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-94417)

ON A

GRANT

IN THE AMOUNT OF US$15 MILLION

TO THE

REPUBLIC OF THE SUDAN

FOR A

BASIC EDUCATION PROJECT

November 26, 2013

Human Country Development Department AFCE4 Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS Exchange Rate Effective November 26, 2013

Currency Unit = SDG

SDG 1.00 = US$ 0.1751 US$ 1.00 = SDG 5.708

FISCAL YEAR 2014

ABBREVIATIONS AND ACRONYMS

BEP Basic Education Project BERP Basic Education Recovery Project CCU Central Coordination Unit CDF Community Development Fund CMU Country Management Unit CPA Comprehensive Peace Agreement DG Director General EFA Education for All EMIS Education Management Information System ESMF Environmental and Social Management Frame Work ESR Status of the Education Sector in Sudan report ESSP Education Sector Strategic Plan FMoGE Federal Ministry of General Education GER Gross Enrollment rate GoNU Government of National Unity GoSS Government of South Sudan ISR Implementation Support and Result Report M&E Monitoring and Evaluation MAP Management and Administration Program MDTF Multi-Donors Trust Fund MS Moderately Satisfactory MTR Mid-Term Review MU Moderately Unsatisfactory NCCER National Center for Curriculum and Educational Research NER Net Enrollment Rate PDO Project Development Objective PIRLS Progress in International Mathematics and Science Study PIU Project Implementation Unit SDS Service Delivery Survey TIMSS Trends in International Mathematics and Science Study TTI Teacher Training Institute

Vice President: Makhtar Diop

Country Director: Bella Bird

Sector Manager: Sajitha Bashir

Project Team Leader: Elizabeth Ninan

ICR Team Leader: Thanh Thi Mai

REPUBLIC OF THE SUDAN Basic Education Project (BEP) Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ........................................................................................................ 5

2. Key Factors Affecting Implementation and Outcomes ...................................................................................................... 11

3. Assessment of Outcomes ................................................................................................................................................... 17

4. Assessment of Risk to Development Outcome .................................................................................................................. 24

5. Assessment of Bank and Borrower Performance ............................................................................................................... 24

6. Lessons Learned ................................................................................................................................................................. 27

7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors ........................................................................ 28

Annex 2. Outputs by Components ......................................................................................................................................... 30

Annex 3. Economic and Financial Analysis ........................................................................................................................... 37

Annex 4. Grant Preparation and Implementation Support/Supervision Processes ................................................................. 41

Annex 5. Beneficiary Survey Results .................................................................................................................................... 42

Annex 6. Stakeholder Workshop Report and Results ............................................................................................................ 45

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ................................................................................ 48

Annex 9. List of Supporting Documents ................................................................................................................................ 49

MAP ....................................................................................................................................................................................... 50

A. Basic Information

Country: Sudan Project Name: Basic Education Project

Project ID: P112096 L/C/TF Number(s): TF-94417

ICR Date: 10/09/2013 ICR Type: Core ICR

Lending Instrument: SIL Grantee: SUDAN - GOVERNMENT OF NATIONAL UNITY

Original Total Commitment:

USD 15.00M Disbursed Amount: USD 14.89M

Revised Amount: USD 15.00M

Environmental Category: B

Implementing Agencies:

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 12/18/2008 Effectiveness:

Appraisal: 04/06/2009 Restructuring(s):

Approval: 05/12/2009 Mid-term Review:

Closing: 06/30/2011 05/31/2013 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Substantial

Bank Performance: Moderately Satisfactory

Grantee Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately Satisfactory Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments (if

any) Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA): None

1

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Primary education 82 82

Public administration- Education 18 12

Theme Code (as % of total Bank financing)

Education for all 90 90

Gender 10 10 E. Bank Staff

Positions At ICR At Approval

Vice President: Makhtar Diop

Country Director: Bella Bird Kenichi Ohashi

Sector Manager: Sajitha Bashir Lynne D. Sherburne-Benz

Project Team Leader: Elizabeth Ninan Dulvy Yasser Aabdel-Aleem Awny El-Gammal

ICR Team Leader: Thanh Thi Mai

ICR Primary Author: Thanh Thi Mai F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The Project Development Objective (PDO) is to increase access to improved basic education in four selected states in northern Sudan. Revised Project Development Objectives (as approved by original approving authority) The PDO remained unchanged during the project life. The PDO indicators were revised and approved during the first project restructuring in June 2011 (see Restructuring) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Students enrolled in primary education in selected schools (boys and girls)

2

Value quantitative or Qualitative)

Group 1: 7209 (3491 B & 3718 G) - baseline 2009 Group 2: 1949 (1042 B & 907 G) - baseline 2011

20% increase from baseline

Group 1: 7570 (3366 B & 3904 G) Group 2: 2047 (1095 B & 952 G) 5% increase from baseline

Group 1: 7964 (3705 B & 4259 G) - 10.4% increase from baseline Group 2: 2409 (1246 B & 1145 G) - 23% increase from baseline

Date achieved 09/30/2009 06/30/2011 06/30/2011 08/30/2013

Comments (incl. % achievement)

Group 1 includes 17 schools (from original 25 project-built schools of which 8 are currently closed due to conflict) Group 2 includes 5 new schools added after the second restructuring. Targets (revised in June 2011) were surpassed.

Indicator 2 : Average number of days operational for schools supported - Revised as "Average number of days targeted Category B schools are open annually"


123.7 204 160.8 149.6

Date achieved 09/30/2009 06/30/2011 06/30/2011 05/31/2013 Comments (incl. % achievement)

Average number of days open is 149.6 for 7 Category B schools. 1 is outlier, closed for most of the year due to teacher retention issues. If this is removed, the No of school days category B schools open would be 159.3, 29% increase from baseline

Indicator 3 :

Percentage of teachers evaluated positively following in-service training. Revised as " Improvement in teachers' subject knowledge as measured by pre and post in-service training tests"


Pre-test: Core: 25.52 Math: 26.13 Eng: 21.57 Arabic: 39.08

20% increase 20% increase

Core: 32.7 (28%) Math: 31.74 (19%) Eng: 23.42 (9%) Arabic: 51.9 (33%)


Subject knowledge targets exceeded for Arabic and Core program. For Math, target almost achieved and for English lower than target.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Number of additional classrooms built or rehabilitated. Value (quantitative or Qualitative)

0 220 170 202 (18 rehabilitated)

Date achieved 09/30/2009 06/30/2011 06/30/2011 05/31/2013 Comments Revised targets surpassed. Original targets achieved

3

(incl. % achievement) Indicator 2 : New basic education framework established

Value (quantitative or Qualitative)

Old curriculum New basic education curriculum

Refocused to B.Ed. program and English Grade 3&4 only

B.Ed. program and English Grade 3 textbooks & Grade 4 syllabus developed


Revised targets achieved. Original target only partially achieved

Indicator 3 : Number of teachers provided in-service training Value (quantitative or Qualitative)

11350 11380

Date achieved 09/30/2009 06/30/2011 05/31/2013 Comments (incl. % achievement)

Targets achieved.

Indicator 4 : Number of upgraded Teacher Training Institutions

Value (quantitative or Qualitative)

0 4

2 (in Blue Nile soil condition was not appropriate and in South Kordofan due to conflict)

2


Revised targets achieved, original targets only partially achieved.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements

(USD millions) 1 11/03/2009 Satisfactory Satisfactory 0.00 2 01/28/2010 Satisfactory Satisfactory 1.19 3 01/22/2011 Moderately Satisfactory Moderately Satisfactory 5.50 4 09/21/2011 Moderately Satisfactory Moderately Satisfactory 9.14 5 04/30/2012 Moderately Satisfactory Moderately Unsatisfactory 11.39 6 12/26/2012 Moderately Satisfactory Moderately Satisfactory 12.37 7 06/22/2013 Moderately Satisfactory Moderately Satisfactory 14.58

H. Restructuring (if any) Not Applicable

4

I. Disbursement Profile

5

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal Country context

1. The legacy of the civil war, limited infrastructure and the inequitable distribution of public goods and services put Sudan (unified Sudan and South Sudan before 2011) as one of the least developed countries in the world, with over 46 percent of the population living below the statutory poverty line of SDG114 per month (approximately US$30). Displaced populations, nomadic groups, rural populations, children, and female children in particular, were especially vulnerable to higher incidences of poverty, and poverty related health problems. The interaction of high levels of poverty, social exclusion, spatial inequality and fragile social cohesion underpinned rivalry over scarce resources, mass group mobilization and conflict. 2. The Comprehensive Peace Agreement (CPA) signed in 2005 put an end to the civil war in Sudan and set out a process for conflict resolution between North and South Sudan. Development partners supported Sudan’s peace building through two Multi-donor Trust Funds: (i) MDTF-National and (ii) MDTF-South. The Basic Education Project (BEP) was designed and approved in 2009, more than three years after the CPA. The project was part of the MDTF-N, focusing on the four states in Northern Sudan. 3. The post-CPA period was marked by significant economic growth and rapidly rising per capita GDP (average 7.05 percent per annum). There was a general optimism that the economy would continue to grow. However, the situation worsened after the Project began. The Project became effective in late 2009, and by July 2011, South Sudan seceded from Sudan. This led to renewed conflict in the border regions and continuing conflict in Darfur as well as a period of fiscal crisis with the loss of oil revenue and eventually the shutting down of oil production. Inflation was running as high as 40 percent. There was a change in leadership at the national, state and local levels following the secession which led to some disruption in the government programs. Education sector context Access 4. According to Government of National Unity (GoNU)’s Baseline Survey (2008), the average Gross Enrollment Rate (GER) in basic education for the country was 71.7 percent and the Net Enrollment Rate (NER) was 65.3 percent; for females the GER was 65.7 percent and NER 58.8 percent The positive impact of the CPA has been demonstrated through significant increases in basic school enrollment in Sudan as a whole, and especially in communities previously affected by conflict. Basic school enrollment in Sudan increased by almost a million between 2004/05 and 2008/09, corresponding to an average annual growth of 5.7 percent. The peace dividend was evident in the dramatic increases in access to education in states previously affected by conflict (the Darfur states, states bordering Southern Sudan and the Eastern states). However, the physical infrastructure in a large number of schools in the country remained extremely poor. Many of the classrooms were built out of tree branches and hay. During the rainy season, many students were unable to continue their education.

6

Quality

5. According to the result of the first Service Delivery Survey (SDS) conducted in 2009, student learning levels in the sampled schools in Sudan were low. The mathematics and reading tests were taken from the 1995 and 2003 Trends in International Mathematics and Science Study (TIMSS) grade 4 assessments, and the reading questions were taken from the 1991 and 2001 Progress in International Reading Literacy Study (PIRLS) grade 4 assessments. The vast majority of Grade 5 students (84 percent for mathematics and 72 percent for reading) answered less than half these questions correctly. 6. The findings highlighted the lack of resources in many basic schools and family background variables, such as wealth and parental education, were related to low level of student learning outcomes. Specifically: Classroom resources were generally scarce, with shortages of textbooks, desks, and functional

chalkboards in many of the sample schools; A large share of students did not have textbooks; The chalkboard was the only most important teaching tools, and in many cases chalkboards

were not in usable condition; Many students did not have desks, but instead sat on chairs without desks or on the ground,

particularly in rural schools; Syllabus coverage was low in many schools, limiting learning opportunities ; and Only 10 percent of basic education teacher were officially qualified and there was no

systematic in-service training provision Efficiency 7. The SDS 2009 also showed that in Sudan for Grades 1-5, average dropout was about 8% and repetition was about 2%. Location, gender and poverty remained very influential in determining a child’s likelihood of attending school. Whether a child lived in an urban or rural area was the single most powerful predictor of whether a child will access schooling. GER for urban children was 17 percentage points higher than for rural children. A poor rural girl was the most disadvantaged and was about 25 percentage-points less likely to ever access basic school than a rich urban boy. In rural schools, only 41 percent of enrolled students were girls. Management structure 8. Sudan had a decentralized system of education service delivery. Within the education sector, the federal government, through the Federal Ministry of General Education (FMoGE) in Khartoum, was in charge of planning, coordinating, and monitoring across the general education levels. FMoGE directed policy development in three key areas: secondary school certification, the qualification framework for teachers, and development of the basic and secondary education curricula. Policies related to human resource management and certification for basic education fell under the Preview of the state governments. The states and mahalyas (localities) were responsible for the delivery and financing of preschool, basic, and secondary education. Sector strategy

7

9. The GoNU developed the Five year (2007-2011) Education Sector Strategic Plan (ESSP) based on the 2001 Education Act. The Plan expressed the country’s commitment to the Education for All (EFA) goals and outlined activities to meet the Millennium Development Goals (MDG) targets by 2015. Goals included: • Closing the gender gap and reducing regional imbalances; • Ensuring universal primary education and improving quality; • Improving quality and relevance by revising the national curriculum; • Improving efficiency; and • Expanding the role of the private sector in education. 10. Though the ESSP set clear targets within a specific time frame, the detailed strategies or financial implications for achieving these targets were not fully analyzed. Realizing complexity of the situation and the magnitude of the tasks ahead, the MoGE decided to strengthen partnership arrangements with the international community represented by donors active in the education sector. To this end, the MoGE expressed its commitment to joining the Education for All - Fast Track Initiative (EFA/FTI) partnership. Bank and Development partners’ program 11. The MDTF’s commitment to assist Sudan in the education sector was confirmed in the Bank’s Interim Strategy Note for the Republic of Sudan (Report No. 43036-SD, March 20, 2008). The BEP Project was approved in 2009, among the 15 projects funded by the MDTF-N. BEP target areas 12. The BEP supported four of the States in Sudan - Red Sea, South Kordofan, North Kordofan, and Blue Nile (see Map page 46). Table 1: Characteristics of the four states in BEP Red Sea North

Kordofan South

Kordofan Blue Nile Sudan

average Poverty incidence 57.7% 57.9% 60% 56.5% 46.5% Basic education Gross Enrolment Rate

37% 78% 82% 65% 73%

Total enrolment 110,707 529,672 273,102 102,786 Retention rate1 31% 72% 92% 40% 68% Share of education spending at state level2

12% 23% 14% 12%

1 defined as number of students entering grade 1 completed grade 8 (cohort survival rate) 2 % of education budget in the state budget 13. The four states were among the poorest in Sudan. The poverty incidence in these states was higher than most of the states in Northern Sudan and was only lower than the Darfur states. In terms of access to basic education, Red Sea and Blue Nile were significant below the national average while North Kordofan and South Kordofan were performing better than the national average. About half of the classrooms in these four states were in poor condition without a chalkboard and furniture.

8

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 14. The Project Development Objective (PDO) is to increase access to improved basic education in four selected states in northern Sudan. The main indicators through which its principal outcomes will be measured are: • Enrolment in the selected schools (number of boys and girls). • Number of functioning days in the schools supported. • Percentage of teachers evaluated positively following in-service training in the four states

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO remained unchanged during the project. 15. The PDO key indicators, however, were revised as part of the first restructuring in June 2011 (after the Mid-term review in October 2010). Table 2: PDO level indicator revision (Restructuring June 2011) PDO Level Results Indicators

Original Target

Revised Target at MTR

Reasons for revision

Students enrolled in primary education in selected schools. (number of boys and girls).

10,320- 11,180 (20-30% increase)

9095 5% increase

No new schools were being built, but existing schools were being upgraded from local materials to brick structures. The expected increase in enrollment would be similar to the average annual growth rate for basic education in northern Sudan over the last 10 years, which was 5%. Also, the majority of the schools would have only be completed by mid- 2011 so a major increase in enrollments was only expected the following school year

Average number of days functional for schools supported.

Category A schools (175 days) Category B schools (124 days)

Average number of days targeted schools are open annually Category B (160 days)

Definition of ’functional’ was ambiguous. Focus would be on number of days a school is open in the year. Target was set for 30% increase in average days a school is open for Category B schools (i.e. those which are closed during the rainy season). A larger increase could not be expected since the roads and infrastructure in communities make it difficult for children to get to the schools.

Percentage of teachers evaluated positively following in-service training in the four states

15-20% increase in positive evaluations

Improvement in teachers’ subject knowledge (based on trainings received) as measured by pre

The indicator was ambiguous. The pre and post tests will only capture the improvement of the teacher’s subject knowledge based on the trainings received.

9

and post tests Average increase of 15-20% in the scores of the post test evaluation

1.4 Main Beneficiaries The primary project beneficiaries are students (boys and girls) in the project schools and teachers in the four states. It should be noted that the population of the project schools went through changes and adjustments throughout the project life. Table 3: BEP schools by states by project end Red Sea North

Kordofan South

Kordofan Blue Nile Total

A B A B A B A B A B Group 1 (original plan) Built 3 6 7 2 6 13 11

Cancelled 3 3 6 Built & closed 2 2 3 2 5 Built & open 3 6 5 3 11 6

Group 2 (added after June 2012)

Built & open 4 1 5

16. Schools were classified as Category A and Category B. Category A schools had an average school year of around 210 days, while for Category B it was 140 days. In the Category B schools, instruction took place only in the non-rainy season as the quality of infrastructure made it impossible for children to attend schools while it was raining. The original project schools identified in 2009 (Group 1) consisted of 30 schools (13 category A and 17 category B). Due to the conflict in Blue Nile and South Kordofan, six schools had to be cancelled. Furthermore, seven schools already built from this group remained closed to date as a result of the conflict hence the final achievement was recorded for 17 schools in Group 1 against the 2009 baseline. Group 2 (5 schools) was added in late 2012 as a result of the second restructuring and the final achievement was recorded for these five schools against the 2011 baseline. Table 4: Project beneficiaries

Direct beneficiaries

Group 1 schools

Group 2 schools Total

Baseline 2009

Target Actual August 2013

Baseline 2011



Students (B/G)

7209 3491 (B) 3718(G)

7570 3366 (B) 3904(G)

7964 3705 (B)4259(G)

1949 1042 (B)907(G)

2047 1095 (B) 952(G)

2409 1264 (B) 1145(G)

5% of baseline

10373 4669 (B) 5404 (G)

Teachers (M/F)

11350 6828 (F)

11380 7825 (F)

In addition, the project also benefited the National Center for Curriculum and Educational Research (NCCER) and two Teacher Training Institutions in Red Sea and North Kordofan states.

1.5 Original Components (as approved)

10

Component 1: Enhancing Access to Basic Education (US$5.006 million) included: Rehabilitation and addition of classrooms, offices and teachers’ housing in areas where

interventions will enhance girl’s access Component 2: Contributing to enhancing the quality of basic education (US$ 7.49 million). This component had three sub-components: Component 2.1 Curriculum review (US$0.325million). Evaluation of basic curriculum for each subject in the light of the CPA to identify the extent of

changes necessary to include the multi religious, multi ethnic and multilingual reality both within the north and in southern Sudan

Support for a forum to discuss the relevance of the pre-service teacher training curriculum Component 2.2 In-service teacher training (US$3.097million). Provision of in-service training in Arabic, English, mathematics, and core programs to about

11,300 primary school teachers with only secondary education in the four states. The length of the training would vary from 15 to 21 days across the states. A pre and post training questionnaire would be administered to understand and measure the effectiveness of the training.

Component 2.3 Physical upgrading to a number of Teacher Training Institutes (TTIs) in the four states (US$4.067million): Physical upgrading of four TTIs (Al-Dalang TTI in South Kordofan (rehabilitations and

additions), Ganoub Al-Roseirus TTI in Blue Nile (new), Al-Obied TTI in North Kordofan (additions and rehabilitation), and Port Sudan TTI in Red Sea State (new).

Component 3 Monitoring, Evaluation and Project Management (US$2.35million). Support for M&E mechanism to ensure that a baseline, relevant to each of the project

interventions, was collected and outcome indicators were systematically assessed Financing project coordination teams at the central and states’ level as well as operational costs

related to project implementation

1.6 Revised Components 17. The project component structure remained but the scope of each component and sub-components were adjusted throughout the project life. Specifically: Component 1: Enhancing Access to Basic Education (US$ 6.21m): Rehabilitation of existing classrooms (from local materials to permanent structure), and

construction of offices and teachers’ housing Provision of furniture, access to water and sanitation and learning materials to the project

schools

11

Component 2: Contributing to enhancing the quality of basic education (US$6.16m) Component 2.1 Curriculum review Organization of 10 workshops related to curriculum development, education system policy and

Bachelor of Education pre-service program Revision of the B.Ed. program Revision of Grade 3-4 English curriculum and development of textbooks and teachers’ guides Component 2.2 In-service teacher training Provision of in-service training in Arabic, English, mathematics, and core programs to about

11,300 primary school teachers with only secondary education in the four states. Component 2.3 Physical upgrading to a number of Teacher Training Institutes (TTIs) in the four states Physical upgrading of two TTIs (Al-Obied TTI in North Kordofan (additions and

rehabilitation), and Port Sudan TTI in Red Sea State (new). Component 3 Monitoring, Evaluation and Project Management (US$2.48m) Support for the M&E system Financing project coordination teams at the central and states’ level as well as operational costs

related to project implementation Details of the adjustments are presented in section 1.7, 2.2 and Annex 2.

1.7 Other significant changes The project underwent two restructurings (details presented in section 2.2) 18. First restructuring in June 2011 to (i) extend the project closing date by 12 months from June 30, 2011 to June 30, 2012; (ii) amend the institutional arrangements related to procurement and civil works supervision aspects of the project; (iii) revise some of the targets related to the Project Development Objective (PDO) indicators; and (iv) reallocate the Grant proceeds amongst the categories of expenditures; and

19. Second restructuring in June 2012 to: (i) introduce changes to the institutional arrangements that would streamline implementation now that some activities have been completed; (ii) allow resources currently held up in conflict affected states to be used in other states, per Government’s request; and (iii) extend the closing date of the BEP from June 30, 2012 to May 31, 2013.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 20. The BEP preparation can be considered as an emergency project as the preparation time from Concept review to Grant Funding Proposal approval was less than five months. The design team took

12

the lessons learnt from the Community Development Fund (CDF) project in the MDTF-N that involved interventions in the education sector (primarily school construction). The design adopted a strategy that in the project states where CDF was active, the delivery of school and TTIs construction would be outsourced to CDF. 21. BEP was also designed to go beyond providing education facilities. The reform of the school curriculum and the pre-service teacher training program were intended to lay grounds for system-wide reforms. These interventions were also expected to lay the foundations for the Bank and other partners’ longer-term engagement with Sudan’s education sector. 22. In spite of the short preparation timeframe, the preparation was participatory and consultative. Consultation with the four states was conducted. The consultation confirmed that direct support at the state level was likely to produce faster results and stimulate a more targeted development of education. State governments’ and State Ministries of General Education (SMoGEs)’s commitment had been strong and their full participation at different levels ensured the project activities were incorporated into the states’ annual plans for 2009-2011. 23. At appraisal, the design team identified the following risks associated with the project: Table 5: BEP risk assessment at appraisal Risks Risk Mitigation Measures Risks Rating With

Mitigations To PDO The Federal and State Ministries of Education have limited experience working on donor financed projects. Low implementation capacity could jeopardize a successful implementation of the program and endanger its quality and effectiveness. Lack of coordination between Federal and state MoE. Lack of dedicated allocations to education at the state level stops activities because MDTF resources cannot pay taxes.

Dedicated staff will be assigned to support project implementation. Staff will be competitively recruited and remunerated to ensure adequate capacity. The CDF and EO have been identified to support the infrastructure component. Components implementation has been divided between FMoE and SMoE based on overall mandate in the sector. Infrastructure is implemented by SMoE while curriculum update is implemented by FMoE. Federal Government committed to channeling extra resources to the four states to cover such expenditures. States’ governments committed to pay for these cost even from local resources.

S

M

M

To Component Results

No prior Bank procurement experience with the SMoE and limited experience within the MoGE. Basic education curriculum update may take a long time and be politicized. Support to TA for this process may be

Infrastructure components which absorb the bulk of funds are being implemented by two competent units, the CDF which has on-going procurement experience with the WB and the Engineering Office, which has a long experience in Gov. Procurement. GoNU totally owns the update process and federal technical committees have already been formed. Selection of TA will be decided by the

S

M

13

sensitive politically. Tensions between North and South regarding the curriculum.

curriculum development center on technical basis and from countries where Sudan is comfortable with. A MOU was signed in October, 2009 between the FMoE and MoE of South Sudan agreeing on a process for a joint review of the curriculum.

S

Overall Risk Rating S 24. Most of the risks identified here related to the lack of experience in the MoGE and SMoEs in managing donor supported projects. The design team expected that for the project investment activities (Component 1 and 2.3) CDF would bridge this capacity gap using its experience of managing the CDF program in Sudan. Actual project implementation showed that while CDF had experience with World Bank funded projects before, its commitment and capacity to implement the construction activities for BEP was limited. There were significant delays in the planning, procurement and execution of these contracts and the quality of on-site supervision was not satisfactory. This led to the need to restructure the implementation arrangement of the construction-related components. 25. For the curriculum interventions, although the design team foresaw the political risk associated with curriculum development and the potential tension between North and South Sudan, it was expected that the MoU signed in 2009 between FMoE and MoE of South Sudan would enable the joint review of the curriculum. The political process, however, went far beyond this premise and resulted in the separation of South Sudan in 2011, making the unified North/South curriculum not possible.

2.2 Implementation 26. The factors that affected the implementation of the project can be grouped into: (i) external factors; and (ii) internal factors. 27. The external factors consisted primarily of the political development leading to the split of Sudan and South Sudan and the ethnic conflicts erupted in Blue Nile and South Kordofan states in 2011. South Sudan/ Sudan split 28. The CPA established the basis for power sharing pending the outcomes of elections; constituted the Government of South Sudan (GoSS) with administrative authority over the 10 states of the South; entrenched an oil revenue sharing agreement whereby the Government of National Unity (GoNU) and GoSS divided evenly revenues derived from exploitation of oil wells in Southern Sudan. It further provided for a referendum on the status of South Sudan. The outcome of referendum in January 2011 led to creation of the new state of South Sudan in July 2011. 29. The South’s secession has had fundamental repercussions on Sudan as evidenced by significant stresses in the macro-fiscal situation, the structure of the economy, and the political economy. The most important and immediate was the loss of the oil revenue, aggravated by the several years of oil-export driven growth, extravagant public investments, and expanded public employment. In political economy terms, the secession has rendered the remaining territorial disputes with South Sudan (e.g., over Abyei, borders, etc.) and with other populations within Sudan (e.g. South Kordofan) particularly

14

difficult as the state views further compromise on territory as a threat to the integrity of the remaining Sudan. 30. The separation of South Sudan had impacted BEP in three ways. Fiscal resources available to the sector became more constrained. Many states’ and federal level education institutions experienced personnel changes. These had indirectly impacted the implementation of BEP at the state level. Finally and perhaps most notably, the development the unified curriculum was no longer possible. Conflict in Blue Nile & South Kordofan states 31. Even after the separation of South Sudan, not all conflict has been arrested and tensions continue. In most regions of Sudan conflict is endemic at the local level. This was especially the case as it related to competition over access to natural resources between pastoralists, agro-pastoralists and settled farmers (including commercial farms). The most disruptive conflict remained the conflict in the “Three Protocol Areas” (Blue Nile and South Kordofan states and Abyei area) where previous peace protocols had not yet been fully implemented. This conflict has been affecting Blue Nile and South Kordofan states since late June 2011. Several schools in these two states had not been reachable by the BEP team since the conflict began. In the end, due to this conflict the construction of five schools and the TTI in South Kordofan needed to be cancelled. Seven of the project-built schools in Blue Nile and South Kordofan remained closed to date. 32. Factors that are internal to the project included: (i) the decision to delegate major construction activities to CDF; and (ii) weak implementation capacity of implementing agencies (the education ministries, at both federal and state levels as well as contractors. The choice to delegate the planning, procurement and supervision of construction activities (schools and TTIs) to CDF was based on the assessment that CDF had more experience with implementing World Bank-funded projects than the Ministry of General Education in three of the four BEP states. However, CDF did not satisfactorily deliver what was planned. The procurement plans were submitted late by the project and only approved in June 2010, seven months after project effectiveness. Clearing of award recommendations was often delayed by incomplete and/or incorrect Bid Evaluation Reports. The design and procurement of the TTIs were also held back as CDF did not have the capacity to provide technical specifications for the TTI design. Quality of the construction of the several schools was also affected as CDF did not have sufficient staff dedicated to on-site supervision. This led to the need to restructure the project in mid-2011 to revise this implementation arrangement (limiting CDF role to existing schools and delegate states to be the implementing agencies for the remaining schools and TTIs). The states, while having high ownership of the project activities, did also have capacity constraint. By June 2012 another restructuring of implementation arrangement was done to consolidate the implementation of the TTIs’ and Group 2 school construction to the central PIU. With this restructuring and the stronger leadership and implementation capacity of the federal PIU, BEP completed its activities by the new closing date of May 31, 2013. As to the project duration, BEP was approved in May 2009 and was expected to close in June 2011, the same time as the MDTF-N. The Project closing dates were therefore had to be extended twice following the extension of the mother trust fund MDTF-N, which was initially extended to June 30, 2102 and then extended further to 2013.

15

Table 6: Summary of changes introduced in the two project restructurings First Restructuring June 2011

Second Restructuring June 2012

Status

Proposed changes Status Proposed changes

PDO indicators

As at appraisal Revision of lower targets for the indicator on enrolment increase and clarifications of the indicator for school functioning time and in-service teacher training

No change since June 2011

Institutional arrangements

Community Development Fund (CDF) was designated to provide procurement and supervision support for civil works activities to the BEP in three states but its capacity (design, procurement and supervision) was limited (evidenced by deviations from the technical specifications contained in the bidding documents; poor quality of bricks; and a general lack of site supervision)

Limit CDF’s supervision to the 15 schools that had begun construction while proposing alternate supervision arrangements (using State Ministry of Engineering Affairs and State Ministry of Education’s Engineering office) for the remaining schools. The size and complexity of the bidding documents for the TTIs exceed the capacity of the three SCUs hence these works will now be undertaken by the central coordination unit (CCU)

1 Central Coordination Unit and 4 State Coordination Units

Consolidation of 5 stand-alone implementing entities (CCU & 4 SCUs) into one single Project Implementation Unit (PIU) as major work of the project has been accomplished

Allocation of grant proceeds

Re-allocation of funds between States and Central Project units based to actual costs needed

$1.2M stalled in Blue Nile and South Kordofan due to conflict to be re-programmed for additional school construction to ensure that these resources are effectively utilized before the closing date

Project closing dates

June 30, 2011 Extend to June 30, 2012 June 30, 2012 Extend to May 31, 2013

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 33. The project M&E design included the following features: (i) Monitoring and evaluation will focus on the components highest in investment, the access component and the teachers’ training sub-component; (ii) Given that the number of schools to be supported under the project is small compared to the number of schools in each locality and state, monitoring of output and outcomes will be assessed at the schools’ level; and (iii) The project will use the existing capacities of M&E in the CDF in the three states to be supported where the CDF will be involved in implementation.

16

34. During implementation, the project followed the design principle of collecting data at each of the project school. The baseline data collection was done by CDF but due to some questionable data that needed subsequent verification, the project eventually relied on its own staff to carry out the M&E function. Designated M&E officers were engaged at the federal and all four state coordination units. With the support from the WB team, a detailed data collection protocol was developed, covering the indicators contained in the result framework. The data was updated quarterly and shared with the WB team. Data verification and spot check was conducted through field visits. After the project had closed, the final monitoring report was prepared as part of the project exit and hand-over process. 35. The M&E framework was utilized extensively for project monitoring purpose. Data related to the result indicators (both at the PDO and component levels) were used to inform about the project progress and performance. In particular, data on school enrolment informed the wider debate of access to education and the challenge of retain students (boys and girls) in schools. Data on the utilization of the project-provided facilities was also collected and reviewed.

2.4 Safeguard and Fiduciary Compliance

Safeguards

36. Construction of schools and TTIs was classified as environment Category B which triggers OP 4.01. The project developed an Environmental and Social Management Framework (ESMF) that describes measures to mitigate negative project impacts and the environmental monitoring arrangements. These included: (i) Environmental considerations included as criteria for site selecting; (ii) Environmental assessment would become an integral part of the infrastructure activities; and (iii) Environmental guidelines to be followed and used. At the state level, the M&E officer was designated as the focal point responsible for ensuring full implementation of the ESMF. The ESMF document was translated into Arabic language and disclosed to stakeholders. The project completed the required environmental forms for the ongoing schools.

37. The project had no significant irreversible environmental and social impacts. Construction and upgrading of school infrastructure had been at existing school sites, with no new sites acquired. The availability of water in many schools was a challenge as there are no nearby sources of water. In addition, several schools did not have fences to ensure the safety of students and protect the school from livestock encroachment. It was agreed at Mid-Term Review that BEP would endeavor to address the issue of water, fencing and trashcans using the available budget. Additionally, community awareness about the importance of maintaining the school environment, along with informal action plans led by the Project M&E officers had been developed.

Fiduciary 38. BEP in general complied with the fiduciary requirements. The project appointed and maintained procurement and financial management staff in the implementing units at the federal and state levels. For procurement, the main challenge was the delay in submitting the procurement plan/re-plan and bidding documents. For financial management, the project submitted the financial reports on time although occasionally revisions were required. Audit reports were submitted timely and the Bank provided comments and inputs to strengthen the project budgeting, accounting and control processes.

2.5 Post-completion Operation/Next Phase

17

39. At closing the project team had carried out a thorough project exit and hand-over, including the hand-over of key outputs such as assets, data and documents to responsible agencies at the Federal MoGE and State MoGEs. 40. Implementation of BEP has served as the platform for deeper engagement of the WB and other partners in the education sector in Sudan. In June 2011, the Bank together with the other primary donors to the Sudanese education system commenced a series of meetings to support the work of the MoGE to develop its strategic plan. The Status of the Education Sector in Sudan report (ESR, 2012) produced by the MoGE, with technical support from World Bank and other partners, provided a detailed diagnostic of the education sector and suggested underlying reasons for poor learning performance, including teacher deployment; teacher utilization; distribution of textbooks; the learning environment and the health status of learners. This ESR enabled Sudan to access financing to the Global Partnership for Education (former FTI/EFA Catalytic Fund). The Basic Education Recovery Project (BERP) was approved by GPE in March 2013, with the WB acting as the supervising entity. 41. The BERP (US$76.5 million) aimed at improving the learning environment in targeted areas; increasing the availability of textbooks; and strengthening education planning and management mechanisms in the Sudan. BERP included (i) improving learning environment through classroom construction and provision of school grants; (ii) increasing the availability of textbooks and learning materials; and (iii) strengthening the monitoring and management mechanisms of the education system. 42. BERP, therefore, is a follow-on project, taking BEP key interventions to larger scale as well as addressing systemic sector challenges not covered by BEP. Specifically, BERP scales up the school construction program to 10 states in Sudan with ten-fold increase in the number of classrooms to be constructed. In addition to classroom construction, BERP will pilot a school grant component to provide schools with additional resources to improve their learning conditions. BERP will go beyond curriculum review to textbook development and distribution, a key constraint to improved learning conditions in primary schools in Sudan. BERP will continue the English language textbooks development that was initiated under BEP. Taking the experience of in-service teacher training in BEP, BERP will help to develop system-wide teacher management database than can inform more efficient deployment as well as professional development needs of teachers. Finally, BERP will take the M&E culture developed in BEP to system level by operationalizing the sector Education Management Information System (EMIS).

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Relevance of objectives 43. BEP’s objective of increasing access to improved basic education in four selected states in northern Sudan remains highly relevant with Sudan’s current context. Access to basic education had increased in the last five years. However the conditions of learning in schools were very constrained and limited. The Second Service Delivery Survey (SDS) conducted in 2010/2011 showed that: (i) Test scores in mathematics among Grade 5 students were weak and lower in rural areas, for children from poorer background and for boys; (ii) School catchment area was vast - a majority of rural students lived far from schools; (iii) School infrastructure was generally poor or missing: 12 percent of rural schools had no school building and students sat under trees. Another 13 percent of rural schools had a roof but no walls. Many schools were inaccessible during the rainy season; (iv) Many schools lacked

18

drinking water and toilets, particularly in rural areas; (v) There were severe shortages in schools of basic inputs such as desks, usable chalkboards and textbooks; (vi) The average actual school year was much shorter than the official school year, indicating a substantial loss of instructional time; (vii) more than 70 percent of teachers had no pre-service teacher training; and (viii) A significant share of the non-salary operational costs was funded by parents or other members of the community. These challenges demonstrated that the objectives and the interventions of BEP stayed highly relevant and will continue to serve Sudan’s aspiration to improve its educational outcomes. Scaling up these interventions will hopefully enable Sudan to move toward achieving the Education for All and the Millennium Development Goals. 44. The new WB Sudan Interim Strategy Note (covering the period 2014-2015) pointed to the principle of partnership and focused the Bank’s engagement with the Government of Sudan and the donors in two pillars: (i) Manage the economic transition; and (ii) Address socio-economic roots of Conflict. The second pillar supported national and local programs for service delivery focusing on key human development outcomes. BEP’s emphasis on improving access to quality education fit within this pillar. The project was also consistent with the World Bank Africa Regional strategy focusing on fragile states. Relevance of design and implementation 45. BEP’s relevance of design is assessed as Substantial. The design team had chosen the relevant objective which remains appropriate even in the current context. The team had also designed appropriate targeting mechanism to ensure that the project cater to the hard to reach locations and focus on girls education and the disadvantaged. The selection of the project interventions was also suitable. School construction, teachers’ pre-service and in-service training and improving curriculum and learning materials are important to improving Sudan education sector outcomes. The chosen interventions in BEP complimented other projects administered by the WB and other development partners in the education sector. The design process also built community engagement to ensure that the project outputs and outcomes can be sustained. Finally, making M&E an explicit function of the project management structure was also valuable in ensuring that eventual project implementation focus on results. The main short-coming in the design was the choice to outsource major construction-related activities to CDF whose capacity and incentive to delivery for BEP was later shown limited. 46. The project design was adjusted during the implementation to reflect the changes in the project environment (Sudan/South Sudan separation and the conflict in Blue Nile and South Kordofan) and the project management capacity and performance. This adaptation helped to improve the project performance by the project end.

3.2 Achievement of Project Development Objectives 47. BEP was expected to increase access to improved basic education in four selected states in northern Sudan. As the project was restructured in 2011 (no change in PDO but clarification and revision of two of the PDO indicators), a split assessment of achievement of PDO before and after restructuring was conducted. 48. Overall the project achievement of PDO was assessed as Moderately Satisfactory (MS). This was the combined assessment for two periods, before and after the restructuring. The achievement before restructuring (42 percent disbursement) was Moderately Unsatisfactory (MU). The achievement after restructuring (58 percent disbursement) was Satisfactory (S).

19

49. Before restructuring – The achievement of PDO was assessed as Moderately Unsatisfactory (MU) because while there was a modest increase in the enrolment of students in project schools and a significant improvement in knowledge of teachers, several project activities had not been completed. Specifically: For the objective of increasing access, at the time of the restructuring in 2011, there was a

modest increase in enrolment (4 percent) in the project schools but there was yet an increase in the average number of functional days of the project schools. The main reason was that construction of the project schools had not been completed.

For to the objective of improving basic education, the project had provided training for 80 percent of teachers in the four states in the areas of core curriculum, mathematics, English and Arabic. A sample of 2520 teachers took part in the pre and post training assessments. The result of the post training assessment, available after the restructuring, showed a significant improvement in the knowledge of teachers across all four domains. The construction and upgrading of the four TTIs had not been completed and the work on curriculum review and revision was limited.

Table 7 documented the achievement of PDO indicators at restructuring Original indicators

Original targets by project end

Baseline Actual at restructuring (March 2013)

Comments

1. Enrolment in selected schools (by gender)

20-30% increase

8599 8979 (4%) 4341 (B) 4348 (G)

Enrolment increase was on track but no change in the average number of days functional for project schools. The main reason was that construction of schools had not been completed 2. Average

number of days functional for schools selected

Annual increase of 15% from baseline


No change

3. Percentage of teachers evaluated positively following in-service training in 4 states


Pre-training test result Core: 25.52 Math: 26.13 English: 21.57 Arabic: 39.08

2520 teachers tested Post-training test result Core: 32.7 (28% increase) Math: 31.74 (19% increase) English: 23.42 (9% increase) Arabic: 51.9 (33 % increase

Pre & Post-training tests were done for 2520 teachers in four states. The results of these post-test became available after the restructuring and showed significant positive improvement in teachers’ knowledge.

51. After restructuring – the achievement of PDO after the restructuring was assessed as Satisfactory (S) due to (i) the impressive enrolment increase in the project active schools; (ii) a significant increase in the average days active Category B schools were open; and (iii) improved teachers’ knowledge; (iv) improved Pre-service B.Ed. program; (v) improved curriculum and

20

textbooks in English language; and (vi) improved capacity for teacher training of two TTIs. Specifically: 52. For the goal of increasing access to education in project schools, the active project schools recorded impressive enrolment increase, in particular in the lower grades. The increase was particular strong for girls who registered 16.8 percent increase. The increase for boys was also high, 9.6 percent, but certainly not as remarkable as for girls. One reason for the lower enrolment for boys was the rush of boys and even male teachers to the areas with newly found gold. 53. The number of days that Category B project schools stayed open was also encouraging. Category B schools are not open during rainy and extremely hot months (depending on where the State is located) due to the quality of school infrastructure. Since construction of the Category B schools was only completed in 2012 for most schools, data was only collected in last school year for six remaining Category B schools which were located in Blue Nile (3 schools) and Red Sea (3 schools) (2012/2013). On average, the six remaining Category B schools were open for 149.6 school days missing the PDO target of 160.3 days. Five of the seven schools were open between 159-160 days with the exception of Shindoday School in Red Sea which was only open 91 days of the school year due to major challenges around teacher retention. 54. With regards to the goal of improving basic education, 11,380 teachers in the four states received this in-service training during the project life. Most of these teachers only finished secondary schools and did not have any teacher training qualification before becoming teachers. They also did not receive any in-service training prior to the project. The pre training test showed very low level of knowledge across the four training domains. Teachers lacked knowledge of the core curriculum, mathematics and English and the only subject that they seemed most knowledgeable was Arabic. The post-training test results showed marked improvement in their knowledge for the core curriculum, mathematics and Arabic. Improvement in their English is less significant. This is to be expected given English is not the main medium of communication for these teachers. While improved teachers’ knowledge cannot be equated automatically with improved quality of basic education, ensuring that teachers have adequate knowledge for the subjects that they teach is an essential step towards improving quality. Teachers in Sudan lacked basic knowledge and opportunities to receive professional development. BEP, therefore, targeted the right audience by providing un-qualified teachers with the basic in-service training program. 55. Overall, BEP has made significant achievements towards the PDO. Across all project-supported schools that are active, enrolment increased for both boys and girls and these students now have access to better school infrastructure. Schools have been constructed with permanent structure with adequate furniture, water supply and other learning materials. Teachers, albeit facing many challenges in their job, received timely in-service training in the areas where they were not confident and needed help. In particular, for teaching English (the subject that most teachers identified as most difficult for them), BEP has helped to developed the new English curriculum and new English textbooks and teachers’ guide for Grade 3. These new textbooks focused on enhancing the English speaking and listening skills and include many activities with communicative and child-friendly approaches. Future teachers can also benefit from the newly developed Bachelor of Education pre-service training program, supported by BEP and the improved facilities at the two TTIs in North Kordofan and Red Sea.

21

Table 8 presented the achievement of key PDO indicators after the restructuring. Original indicators

Original targets by project end

Revised indicators Revised targets Achievement

Enrolment in selected schools (by gender)

20-30% increase

Enrolment in selected schools (by gender)

5% increase 13.2% (Group 1 schools against 2009 baseline, Group 2 schools against 2011 baseline) Boys: 9.6% increase Girls: 16.8% increase

Average number of days functional for schools selected


Average number of days Category B schools are open annually

Category B schools (160 days)

149.6 (when one outlier school is taken out, the average for the remaining Category B project school is 159.3 days)

Percentage of teachers evaluated positively following in-service training in 4 states


Improvement in teachers’ subject knowledge as measured by pre and post training tests

Average increase of 15% in the scores of the post-test evaluation

Pre-training test result Core: 25.52 Math: 26.13 English: 21.57 Arabic: 39.08 Post-training test result Core: 32.7 (28% increase) Math: 31.74 (19% increase) English: 23.42 (9% increase) Arabic: 51.9 (33 % increase

3.3 Efficiency

56. The economic analysis at the appraisal stage was not specific to BEP but covered broad fiscal projections for the education sector expansion. As BEP interventions were very small (30 schools amongst more than two thousands schools in the four states and approximately fifteen thousand schools in Sudan), this exercise was not repeated at the ICR stage. Rather, a project-specific unit cost analysis was done in late 2012 for the school construction component (which involved more than 40 percent of project financing). Cost analysis allowed for cost-comparisons across projects with respect to their implementation arrangements and technology. The methodology provides disaggregate information of both the direct cost of construction, i.e. the cost of the contractor’s contract, and the total project cost, resulting from the addition the direct cost and all the indirect costs, for instance, the management and supervision costs. BEP costs were compared with projects administered by UNICEF, Non-governmental organizations (NGOs) and CDF in various states of Northern Sudan.

57. The average unit costs for BEP were USD189 per square meter; with South Kordofan registered lowest unit cost of USD158 and Blue Nile registered highest unit cost of USD213. Unit costs of projects managed by UNICEF were USD326. Unit costs of the CDF project were USD213. The analysis therefore showed that with comparable quality of construction, BEP schools registered

22

lowest unit costs, comparable with projects managed by Non-governmental organzations (see Annex 3). This result is very encouraging given the schools locations are in the hard to reach areas.

58. The costs for teacher in-service training (about 20 percent of project financing) stayed within the budget. The number of teachers trained by the project resources was slightly higher than the target, yet the total costs of training remained the same as planned. The costs for curriculum review and revision was also comparable to the plan. The costs for TTI upgrading were only 65 percent as the resources planned for two TTIs were reallocated to additional school construction.

3.4 Justification of Overall Outcome Rating 59. The overall rating of the project outcome is Moderately Satisfactory. The project PDO remains highly relevant in today’s Sudan education sector context. The relevance of project design was substantial. The project made significant achievements toward the PDO as evidenced by the improved values of the key PDO indicators at the project end. Finally the project costs were reasonable. 60. The main short-coming in the progress towards the achievement of outcomes was the fact that some of the project outputs were only partially delivered. 61. Some factors were beyond the project control, in particular the separation of South Sudan and the ethnic conflicts in Blue Nile and South Kordofan. Internal factors such as capacity constraint of CDF and the federal and state MoGEs in project management also held the project back from fully satisfactory outcome achievement.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 62. A number of social issues formed the basis for the project design. These were: a) gender equity in access to basic education services; b) the challenge of providing quality education to nomadic communities; c) the importance of supporting education in schools operating under the Sudan People’s Liberation Movement (SPLA) in South Kordofan; and d) ensuring community involvement in project implementation. Poverty 63. BEP provided direct support to four of the states with high poverty incidence in northern Sudan, contributing to improving enrollment and reducing inequality and disparity across the states. As the project resources were limited, the selection of the localities within these four states to receive support was also pro-poor and geared towards the educationally disadvantaged: Localities with the lowest enrolment rates. Localities with the lowest girls’ enrolment rates. Localities that have schools where learning is on-going and is lacking the proper school

facility. Localities where no other development partners were operating. Localities where some school feeding programs were running. In North Kordofan provisions were made to cater for localities with nomadic population.

23

Gender 64. The project has a special focus on girls’ education aiming at reducing the access gender gap in the four states. A number of the schools selected were girls only schools, other schools were supported with latrines for girls to ensure that factors inhibiting girls’ enrolment are reduced, and a number of schools were supported by female teachers’ housing that would ensure the sustainability of female teachers in remote areas. The project M&E system paid particular emphasis on monitoring this aspect and indeed the improvement of girls’ enrolment in the project schools is higher than that of the boys. Social development 65. With the CDF involved at the project start, approach to community engagement similar to that of the CDF project was used to ensure communities were consulted at every step of the project implementation. In fact across the project-supported schools, communities embraced the responsibility for monitoring the quality and utilization of the infrastructure and especially the financing of the schools’ operating costs. This showed great commitment to their children education and to the sustainability of the schools in their communities. (b) Institutional Change/Strengthening 66. With BEP experience in school construction, some of the State MoGEs have now decided to strengthen their engineering office with the view to adopt standard construction guidelines and coordinate all school construction projects in their localities. 67. In addition to strengthening teachers’ capacity through in-service training, BEP had supported the development of the new curriculum in English language. The process for developing the curriculum and the associated textbooks and teachers guides, with support from the British Council, will likely serve as benchmark for further development of learning and teaching materials in higher grades. (c) Other Unintended Outcomes and Impacts (positive or negative)

68. The officials from the Ministry of General Education confirmed that MDTF-N support has been instrumental during a time where there had been a decrease in government revenue and an increase in population. BEP marked the return of the Bank to the education sector in Sudan and paved the way for a larger scale support to the sector through the follow-on Basic Education Recovery Project.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Interviews with school directors, teachers and parents in several project schools in North Kordofan state showed: (i) an overwhelming sense of appreciation from all three groups (school directors, teachers and

parents) for the support that BEP brought in terms of adequate school facilities (ii) a significant increase in the student enrolment, in particular in the lower grades of primary

education (iii) that retention of students up to Grade 8 remained a challenge as that students lack textbooks

and learning materials, making learning in higher grades difficult

24

(iv) that teachers need more professional development to help them upgrade knowledge in key subjects as well as acquire effective pedagogical skills

(v) that in remote schools teacher dormitories help to attract and retain teachers; and that schools rely on parents to support for their additional teachers’ salaries and operating expenses

4. Assessment of Risk to Development Outcome 70. The project development outcomes are significant and observable, but the risk to sustain these outcomes is significant. The maintenance of the physical assets such as school infrastructure and the related facilities will not likely face substantial challenge. Unless conflict happens, parents are committed to making the schools sustainable for their children. The schools will likely continue to attract students, in particular for the beginning grades as they are safe and well-built. The challenges to sustain the enrolment and increase the retention of students in the higher grades are to do with: (i) capacity to attract and retain teachers; (ii) capacity to provide teachers with much needed professional development to enable them to teach effectively; (iii) capacity to provide other learning conditions (textbooks and learning materials) to students to help them acquire basic skills needed to attain in higher grades; and (iv) external factors such as family conditions and wider country socio-economic development. 71. The risk to making the two TTIs in North Kordofan and Red Sea to become effective agents for teacher training reform is also significant. The two TTIs, one upgraded and the other newly built, will need to have a clear strategy to use the new facilities for their teacher training mission. Since the two TTIs were only completed in February 2013, there was insufficient time to assess the capacity of these two TTIs in this respect. A simple analysis of the business strategies of these two TTIs showed that the El Obied TTI in North Kordofan has a clear three year strategic plan which takes advantage of the newly upgrade facilities. By August 2013, more than 2000 teachers have received training in the newly upgraded facility. The TTI in Port Sudan, Red Sea, while becoming a massive infrastructure, has yet got a sustainable development strategy and may run a risk of having costly maintenance expense.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 72. The Bank’s performance in ensuring quality at entry was assessed as Moderately Satisfactory. The team had selected relevant objective for the project. The targeting mechanism ensured that the project catered to the hard to reach areas. Improving girls’ education had been made an explicit objective of the project. In general the selection of the project activities was also appropriate. School construction, teachers’ pre-service and in-service training and improving curriculum and teaching and learning materials were essential for Sudan education sector. The team had paid due attention to designing a detailed monitoring and evaluation framework and to making M&E an explicit function of the project management structure. This was valuable in ensuring that eventual project implementation focused on results. 73. The project design contained some short-comings such as: (i) overly high target for PDO indicator on expected increase in student enrolment; (ii) vague definitions of the two PDO indicators for number of days school are functional and positive evaluation of teacher in-service training; (iii) over-reliance on CDF as a delivery institution for the school and TTI construction components; and

25

(iv) optimism for a unified North/South Sudan curriculum reform which later fell through with the political separation of the two countries. (b) Quality of Supervision 74. The Bank’s performance in supervision of the project implementation is assessed as Satisfactory. 75. At the task team level, the supervision team took over from the design team without any interruptions to the Bank’s providing technical assistance and fiduciary supervision. The supervision team conducted intensive supervision missions in the first year of the project implementation. The supervision missions always focused on: (i) monitoring the PDO indicators’ values and discuss the likelihood of the project meeting its objectives; (ii) updating the outputs by components; (iii) providing technical assistance and capacity building on project management and M&E; (iv) providing support on the ground including intensified supervision of the construction components; (v) identify key challenges for improving the project implementation; (vi) raising these issues to higher levels of management (government and Bank); and (vii) identifying strategies to address the challenges, including the two project restructurings. 76. At the CMU level, regular supervision of the project performance was also recorded. CMU had been active in: (i) encouraging task teams to be candid on assessing the project PDO achievement and its challenges; and (ii) ensuring that task teams focus on key issues pertaining to the project implementation. At the portfolio level, the MDTF-N had engaged a Monitoring Agent (PriceWaterhouseCoopers) to provide additional on-site supervision of the 15 projects in the MDTF-N. The Monitoring Agent provided regular quarterly assessments of all the projects including BEP, using the same six domains in the WB’s system for assessing Implementation Progress (overall implementation progress; financial management; project management; counterpart funding; procurement and monitoring and evaluation). By March 2013, the Monitoring Agent rated BEP satisfactory on all the six domains and considered BEP as among the top three best-performing projects in the MDTF-N portfolio. 77. The SMU also provided active support to the task team during supervision. The recruitment of the local education specialist strengthened the direct support to the client. The SMU also ensured a smooth transition and back-up of Task Team Leaders during 2011 and 2012. Advice was given to the task teams to (i) sustain the M&E focus; (ii) intensify project supervision; (iii) strengthen project management; (iv) re-arrange the project implementation structure; and (v) adjust the project activities that became stalled. The project assessment by the task team and SMU through-out the project life was MS for both PDO and IP, suggesting that the teams remained cautious and candid. (c) Justification of Rating for Overall Bank Performance

78. The overall Bank performance is assessed as Moderately Satisfactory. This includes the Moderately Satisfactory rating for the design stage. The design of project objectives and interventions was relevant but the choice of the implementation arrangement and the risk assessment had some short-comings. Bank performance during supervision was Satisfactory with consistent focus on the monitoring of PDO achievement and pro-active restructuring to ensure that project activities were completed and key outputs were delivered.

5.2 Borrower Performance

26

(a) Government Performance 79. The performance of the Grantee at the government level is assessed as Moderately Satisfactory. The Government showed its commitment to the project through active participation in the project design, mobilizing the federal and state ministries of general education. Several Director Generals (DG) across the states committed state finances towards the project despite the fiscal constraints they were facing. 80. During implementation, the government appointed the project personnel according to the project structure. The Project Steering Committee did meet to review the project progress and deliberate on the project implementation issues. Resolutions to these issues did come, but with delays. For example, the decision to reallocate resources that stalled due to the conflict in Blue Nile and South Kordofan states to new localities/states was challenging and required an extended period of time. The new FMoGE minister provided support and leadership in resolving this difficult situation. 81. The BEP was one of the first projects to be funded by the MDTF without counterpart funding. During project preparation, the GoNU agreed to support the project coordination units with office space, equipment and vehicles (and other items for the schools such as desks, chairs, textbooks etc.) while the BEP would fund operating costs related to the project. This was a verbal agreement with no written documentation of what exactly would be required of the GoNU/MoGE in terms of funding to support the project. As a consequence, the MoGE did not have the required budget to support the project at inception which, in turn, constrained effective implementation of the project. The issue was later resolved under the leadership of the state and federal DGs as well as Undersecretary of the MoGE. In October 2012 there was another issue of counterpart funding related to the payment of taxes for the TTIs in Red Sea and North Kordofan States which was only resolved by the time of the project closing. (b) Implementing Agency or Agencies Performance 82. The performance of the Implementing Agency is assessed as Moderately Satisfactory. The project implementation structure was set up with the Central Coordination Unit (CCU) and four State Coordination Units (SCUs) as per the project design. These units were staffed with managers, procurement, financial and M&E personnel. Implementation of construction-related components was challenging as CDF did not have full ownership of the BEP activities and did not deliver satisfactorily, leading to the first restructuring that limited CDF’s role to 15 schools only. 83. Amongst the CCU and SCUs, there was at times insufficient communication and collaboration. In 2011, internal management issues within the Central Coordination Unit (CCU) occurred and needed to be raised to the political leadership of the MoGE. By June 2012, a consolidation of the five implementing agencies was done, with the centralized implementation unit being responsible for the remaining construction (schools and TTIs) as well as the work on English curriculum development. 84. Management of the project had greatly improved since the PIU was fully staffed and functional. PIU teams had been proactive in implementation of all aspects of the project and coordination with the State level officials has been good. The PIU team led by a dynamic project manager was key to the successful completion of this project. The consolidated PIU team successfully disbursed and implemented over $4 million in less than one year, which was an incredible given the country context. By the project closing, the team had delivered all expected results and disbursed 99% of the grant proceeds.

27

(c) Justification of Rating for Overall Borrower Performance 85. The combined rating for the overall grantee’s performance is Moderately Satisfactory. Government performance was Moderately Satisfactory because although the commitment to the project objective was high, government could not timely provide the parallel financing for project management and was slow in making decision with respect to project resource re-allocation. The Implementing agency performance was Moderately Satisfactory because the implementation capacity was weak during the first two and a half year of the project an only picked up during the last year with the change in the PIU leadership.

6. Lessons Learned 86. Community engagement is powerful, even in conflict situations – BEP experience showed that communities’ engagement led to the ownership of the new school facilities. Communities are committed to ensuring their school sustainability. Communities also understand the challenges facing the schools and are well positioned to contribute to the identification of appropriate interventions. With technical and managerial support, communities can be empowered to take on the responsibility of some school level project activities. 87. Targeting and resource allocation requires that bottom-up need assessment be balanced with top-down planning and monitoring – BEP allocated approximately the same amount of resources to the four states, even though these states have very different education system characteristics (different size of student population, GER, number of schools and teachers etc.). A closer look at Red Sea and North Kordofan, the two states that were not affected by the conflict, illustrated very different resource allocation approaches within BEP. Red Sea, which had the GER of only 37 percent and had almost four times less teachers than that of North Kordofan, chose to allocate the majority (almost 90%) of BEP funding for the building of the new TTI in Port Sudan. In contrast, North Kordofan, with GER at 78%, chose to dedicate two third of BEP resources for school construction and used the remaining fund to only upgrade the existing TTI in El Obied. 88. Data is the cornerstone for project targeting, implementation and result monitoring – BEP efforts in collecting and updating data at the school level helped to inform key stakeholders whether the project was on targets and what were the key challenges facing the implementing entities. In addition to reporting (paper-based and by telephone), site visits to verify data were also instrumental in ensuring that project decision making was based on valid data. For example, project teams should be cautioned against targeting school construction in very remote areas. Some of these areas will not have access to other services such as water and adequate roads, which makes it difficult to attract teachers and students alike. 89. Curriculum reforms are difficult, requiring (i) political wills; (ii) capacity to articulate vision for reform; and (iii) capacity to embrace best international practices and integrate them with the country’s social, economic, cultural and education realities. The separation of Sudan and South Sudan superseded the efforts to develop a unified school curriculum as part of BEP. Even after the separation, the revision of the curriculum and textbooks of one subject, English, was challenging and required external technical support. 90. Interventions to improve the teaching force are also complex, involving teacher preparation, recruitment and remuneration, professional development, and deployment, especially in a decentralized education system. Strategies to address the weaknesses of the teaching force can be organized into short, medium and longer term measures. Adequate time will be required for measures

28

aiming at institutional change within the system. BEP focused on provision of short-term remedial training to unqualified teachers. To sustain the motivation of teachers towards professional development, more opportunities for joint learning can be provided through developing communities of practice amongst teachers at the local level. 91. Interventions to provide education infrastructure need to take into account the local market situation and the capacity of contractors to deliver – As project school sites tend to locate in more remote areas, it is not easy to attract competent contractors that can deliver. BEP works contracts were medium-sized. A combination of low post-qualification financial and technical criteria and of poor procurement capacity of local bid opening and evaluation committees could lead to contract being awarded to poorly qualified contractors. The works can be organized in smaller bid packages awarded through local competitive bidding, thus opening business opportunities to a larger number of smaller contractors, some of them closer to the concerned communities, therefore increasing competition. 92. Institutional assessment needs to be integral part of the design, taking into account both capacity and incentive to deliver of the proposed implementing agencies. Assigning the implementation responsibility to the agency that has ownership of the project outputs increases the chances of project activities being implemented satisfactorily. 93. Ensuring high quality leadership is critical for the project success - Leadership quality was perhaps the most important factor in BEP’s improved performance. 94. Project institutional arrangements need to be clear, with (i) job descriptions and expected performance level benchmarks for each positions/units, (ii) communication channels to be set up and maintained; and (iii) capacity building program appropriate to the roles and responsibilities and capabilities of each teams. 95. Projects, especially projects in more fragile states, are subject to external and sometimes unforeseen factors – In this environment outcome target setting can benefit from tools such as sensitivity analysis to make sure that external factors are taken into consideration and that caution can be exercised. 96. Last but not least, project teams including both the government and the Bank need to stay flexible and agile to make necessary adjustments when the project context and situation so command while maintaining the focus on the project development objective(s).

7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies (b) Cofinanciers/Donors (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

29

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD millions)

Percentage of Appraisal

1. School construction & rehabilitation

5.01 6.21 124%

2.1 Curriculum 0.33 0.35 106% 2.2 In-service teacher training 3.10 3.15 101% 2.3 Construction of Teacher

Training Institute 4.07 2.66 65%

3. M&E expenses & Operational cost

2.35 2.48 106%

Contingency 0.16 0.00 Total project costs 15.00 14.85 99% Project Preparation Costs 0.00 0.00 .00 0.00 0.00 .00 Total Financing Required 0.00 0.00

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD millions)

Percentage of Appraisal

Multi-donor Trust Fund for North Sudan

15.00 14.85 99%

30

Annex 2. Outputs by Components Component 1: Enhancing Access to Basic Education Targets at appraisal: In total, the component was to finance construction and rehabilitation in 30 schools for around 220 classrooms, 29 teachers housing units, 9 school fences, 60 offices, and 47 toilets. Actual implementation: The outputs of construction are reported in two groups of schools: (i) schools in four states in the original design; and (ii) new schools in NK and BN added after (the second) restructuring/reallocation of funds from conflicted areas: For Group 1, although 30 schools were identified at appraisal, by the project end only 25 of these schools were constructed and rehabilitated because the remaining 5 schools were located in the conflict areas in BN and SK and their construction was suspended due to the conflict. For Group 2, after the second project restructuring in June 2012 whereby funds originally allocated for TTI upgrading in BN and SK were reallocated, five new schools in NK and BN were identified for construction/rehabilitation. Table 9: Final output of Component 1- School construction and rehabilitation Intermediate Results Indicators BEP Group 1

Progress up to Aug 2013

BEP Group 2 Progress up to Aug 2013

Total Comments

Additional classrooms built and/or rehabilitated at the primary level resulting from project intervention

162(18 rehab) 40 202

Schools with toilets 19 5 24 The remaining 6 schools have toilets but not funded by BEP

Schools with dormitories built/repaired

20 5 25

Fences built 16 5 21 The remaining 9 schools are in conflict areas in BN & SK, these schools funds are reallocated

Offices built/repaired 74(1 rehab) 10 84

Number of classrooms with blackboards

162(18 rehab) 40 202

Number of schools with on-site water source/storage

13 5 18 5 schools in BN and 3 in SK are done by other partners (Unicef & Muslim Hands).

In addition to construction activities, the project also used remaining funds (around $870,000) to supply classroom furniture (student and teacher desks and chairs), dormitory furniture, water pumps and water tanks as well as learning materials for students (pens, pencils, notebooks and teaching materials) to the project schools.

Overall assessment: ACHIEVED

31

Component 2: Contributing to enhancing the quality of basic education. This component has three sub-components Sub-Component 2.1: Curriculum review. Target at appraisal: The project design foresaw an evaluation of the basic education curriculum and suggestion of changes necessary to enhance quality and relevance of the curriculum to post Comprehensive Peace Agreement (CPA 2005) era, i.e. to include the multi-religious, multi-ethnic and multi-lingual reality of Sudan. Based on this evaluation a draft curriculum framework will have to be developed and discussed in a national forum. Actual implementation: The Comprehensive Peace Agreement of 2005 necessitated a joint curriculum between North and South Sudan. This process, however, has been mired by political issues and developments that culminated in the separation of South Sudan in July 2011. In retrospect the desire to have a unified curriculum was out of sync with the countries’ political dynamics. The uncertainties during 2010/2011 held back the direction and consensus for curriculum reform. For example, there was no real agreement for a fundamental issue such as language of instruction. The policy in North Sudan was to use Arabic as the medium of instruction but in regions in the South English was deemed to be the medium of instruction. In this uncertain environment, the project did help to organize ten workshops at the federal level. The workshops, while at the times of their organization seemed random, can be grouped into three themes: Curriculum related

School curriculum Mathematics education Teaching and vocation subjections Language in education (Arabic, English and French) School curriculum activities

Education system related:

School supervision The pattern of school system Education of internally displaced and out of school children Literacy and adult education School environment Counseling and guidance

Teacher education related:

Basic School Teacher Education & Training (B.ED) Program Following these workshops at the national level, discussion at State level on curriculum issues was held in all States, with the exception of South Sudan.

32

In addition, the project also helped to adapt Arabic language materials for speakers of “Thittire” in South Kordofan basic schools. Specialists in teaching Arabic for non-Arabic speakers were commissioned to carry out contextual analysis of Arabic readers for the first and second graders. Two outputs came out of this process: (i) Book One for teaching and writing “Thittire” using Arabic characters to be used as companion; and (ii) Book Two introduces broad but simple reading materials in "Thittire" written in Arabic characters to facilitate transition from the native language to the use of Arabic as the medium of instruction. This sub-component also produced the new pre-service Bachelor of Education program. This B.Ed program, developed by a committee of experts in consultation with Sudanese deans of faculties of education and teacher training institutions and teacher education specialists, comprised of the following: (i) rationale; (ii) aims and objectives; (iii) expected learning outcomes (expressed as knowledge, skills, competencies, values and attitudes of student teachers; (iv) content (course topics, themes, units and modules – both compulsory and elective); (v) content-appropriate teaching and learning strategies; (vi) strategies for assessing students; (vi) references and guidelines for program improvement

Development of English curriculum Following the separation of South Sudan in July 2011, this sub-component was again refocused solely on the development and writing of basic education English language textbooks. The Ministry of General Education decided that English language will be introduced at Grade 3 from the academic year of 2014/2015. Currently English language is taught from Grade 5 so the existing textbooks, SPINE, only cover Grade 5-8. The project supported the National Center for Curriculum and Educational Research (NCCER) to work closely with the British Council to develop course books for Grades 3 and 4. The result of all these efforts has been the completion of Grade 3 Manuscript: a Pupil’s Book, an Activity Book and a Teacher’s Guide. A Sudanese artist has been selected by the NCCER to produce the pictures/illustrations for the design of Grade 3 books. The syllabus of Grade 4 has also been developed and the manuscripts will be developed in BERP. The publishing of Grade 3 as with all other phases of its production can be treated as a pilot and can be used as a model of good practice for the future production of Grade 4 and all subsequent books all the way up to Grade 8.

Overall assessment: PARTIALLY ACHIEVED Sub-Component 2.2: In-service teacher training Target at appraisal: In-service training was to be provided in Arabic, English, mathematics and core programs to a selected group of teachers across the four states. The modules to be used for the training, developed by ISETI with support from UNICEF, were common across states. The target population to receive in-service training included teachers with only secondary education (without a diploma) and university B.Ed teachers. The target number of teachers to be trained was 11,350 (1200 in Red Sea, 3760 in North Kordofan, 1920 in Blue Nile and 4500 in South Kordofan). Actual implementation:

33

100 percent of the target numbers of teachers were trained on Mathematics, English, Arabic and the Core Program. 69 percent of the teachers trained were female. The majority of teachers (40 percent) were trained in the core program (which is for Grade 1-3 students), 20 percent in Arabic, 20 percent in Mathematics and 20 percent in English. Table 10: Actual Number of Teachers Trained by State and Subject

State Arabic Core Program English Mathematics Total Trained

M F Total M F Total MM F Total M F Total M F Total Blue Nile 69 211 280 219 941 1160 107 133 240 78 162 240 473 1447 1920 Red Sea 65 275 340 159 245 404 68 152 220 60 176 236 352 848 1200 North Kordfan 168 709 877 182 916 1098 339 571 910 226 649 875 897 2836 3733 South Kordfan 297 467 764 695 1361 2056 448 392 840 375 465 840 1815 2685 4500 BEP 599 1662 2261 1255 3463 4718 962 1248 2210 739 1452 2191 3555 7825 11380

The trainees were divided into cohorts of 40 participants subject-wise and trained simultaneously over a period of one year and a half. The length of training varied between 15 to 21 days according to the training modules of each subject. Trainers were from INSETI and TTIs. Although the training started later than planned, by mid-2012 the project completed the target training programs. The training content was based on modules which were developed in the mid-1990s when the curriculum was last revised, and moderately updated in the early 2000s. Feedback by the teachers attended these training include: (i) the duration of the training was short; (ii) the content is basic and introductory in nature; (iii) wishes to received longer training that can lead to certification; and (iv) time of the training to be during summer. The results of the pre training test showed low levels of subject knowledge across all four domains and that trainees are in desperate need of help in how to deal with the subject matter, teaching skills, planning and evaluation. The post-training test registered significant improvement in the participants’ knowledge (28% increase in the Core program, 19% increase in mathematics, 9% increase in English and 33% increase in Arabic). This improvement is encouraging, albeit from a very low base.

Overall assessment: ACHIEVED Sub-Component 2.3: Physical upgrading to a number of TTIs in the four states Targets at appraisal: This sub-component aimed to increase the capacity of training and enhance the learning environment in four TTIs in the four states. The TTIs to be supported by the were Al-Dalang TTI in South Kordofan (rehabilitations and additions), Ganoub Al-Roseirus TTI in Blue Nile (new), Al-Obied TTI in North Kordofan (additions and rehabilitation), and Port Sudan TTI in Res Sea State (new). Actual implementation: The TTI in Ganoub Al-Reseirus in Blue Nile was not constructed because the location presented a challenge given that the soil conditions are unsuitable for foundations. Moreover, the location is not well connected by a road, which raises the question of whether teachers can feasibly be trained there. In addition, given the existing TTI in Damazine is not being adequately utilized; there is insufficient justification to build another TTI in the Blue Nile state, in a more remote location. The TTI in Dalang in South Kordfan was also not constructed due to security reasons. The funding planned for these two TTIs were reallocated to Component 1 for additional school construction.

34

The project, therefore, only funded the construction of two TTIs: (i) building of a new TTI in Port Sudan (Red Sea) and (ii) upgrading of an existing TTI in El Obied (North Kordofan). Construction of these two TTIs experienced significant delays due to (i) the inability of the CDF and an independent consultant to provide good quality schematic designs for the TTIs; and (ii) poor performance of on-site supervision. With intensified contract management and on-site supervision, by February 2013 the construction of these two TTIs was completed and the new facilities were handed over to the States. Table 11: TTIs Construction Completed May 2013

Port Sudan (Red Sea)

El Obied (North Kordofan)

Training Halls 8 4 Offices 10 6 Dormitories 0 2 Storage 1 2 Cafeterias 2 1 Fences 1 1 Latrines 42 10 Library 1 1 Labs 4 4 Meeting rooms 1 1

In addition to construction, the project also provided furniture computers and audio-visual equipment, laboratory equipment, photocopies and kitchen equipment and utensils to the two TTIs.

Overall assessment: PARTIALLY ACHIEVED

Component 3: Monitoring, Evaluation and Project Management Project management: The component financed personnel/units responsible for the implementation at the central (CCU) and states’ levels (SCUs). Those personnel were not to come from the public service and recruited on competitive basis. Operational costs also covered the management costs of the CDF and the Engineering Office in Red Sea States, operational travel expenses and the costs of workshops organized during the project. The project Operations Procedures Manual included procedures for staff compensation as well as travel and workshops. The project structure (PSC, CCU and 4 SCUs) was established and staffed according to the design.

35

The Project Steering Committee met once every quarter with the participation and leadership the state DGs of MoGE, the DG for Planning at the Federal level, the Undersecretary of the MoGE, the BEP team and other stakeholders. With this organizational structure, project management in terms of planning, procurement and contract supervision was moderately unsatisfactory. In the three states where procurement is delegated to CDF, clearing of award recommendations was often delayed by incomplete and/or incorrect Bid Evaluation Reports. The procurement plans were submitted late by the project and only approved in June 2010, seven months after project effectiveness. The design and procurement of the TTIs were also held back as CDF lacked the capacity to provide technical specifications for TTI design. The financial management arrangements were acceptable. The Project installed the accounting software. The CCU drafted FM policies and procedures manual. The issuance of financial reports was generally timely. With respect to safeguards, the CCU coordinated a training workshop on the environmental and social management framework for all the SCUs. The CCU submitted quarterly reports in a regular and timely manner. The main constraint in managing the project initially was the lack of parallel co-financing from the MoGE to support the CCU and SCUs with office space, equipment and transportation. These issues were largely resolved by the second half of 2011. There is room for increased communication, collaboration and sharing of knowledge and resources across implementing agencies and SCUs. In the first restructuring (June 2011), the project had limited the scope of CDF to implementing 15 schools only and engaged other entities (State Ministry of Urban Planning and State MoE’s Engineering office) to implement the remaining schools and the TTIs.

Project Steering Committee

Central Coordination

Unit

4 State Coordination

Units

State MoE

Component 1 CDF/EO

Component 2.3 CDF/EO

Component 2.2 TTIs & INSETI

Component 2.1 NCCER

Targeted schools and TTIs

36

In the second restructuring (June 2012), the project organizational structure was simplified by centralizing all remaining project implementation activities to the Central PIU (as major state-led activities has been completed). Management of the project has greatly improved since the PIU was centralized in Khartoum. The team has been proactive in implementation of all aspects of the project and coordination with the State level officials has been good. By the project closing, the team had delivered all expected results and disbursed 99% of the grant proceeds. Monitoring and Evaluation: the M&E system was designed to focus on the components with highest in investment, the access component and the teachers’ training sub-component. Given that the number of schools to be supported under the project is small compared to the number of schools in each locality and state, monitoring of output and outcomes will be assessed at the schools’ level. The project would use the existing capacities of M&E in the CDF in the three states to be supported where the CDF would be involved in implementation. In accordance with the project M&E design, the project collected data on students, teachers’ and school characteristics of all project schools. The baseline was done by CDF but with some delays and some of the data was questionable. The project then instituted M&E specialist in each of the four states to help with data collection, monitoring and school/community visits rather than relying on CDF. The WB team provided thorough technical assistance to the project M&E team to develop and maintain the monitoring framework. The MTR confirmed that all schools have been visited at least one time by State Coordinators and multiple times by State M&E Officers, which has been essential in monitoring outputs as well as responding to the changing needs of stakeholders. The mission proposed incorporating information on site visits within the BEP quarterly reports. The expiration of staff contracts in Dec. 2011, without a plan for recruitment or retention of key staff, left major holes in implementation and M & E, causing delays in project activities and reporting. It was these delays that caused the downgrading of project implementation from MS to MU during the April 2012 ISR submission. This situation was subsequently rectified with the World Bank sharing the backup data and tables on file and the Monitoring Agent providing informal input as needed. With the second restructuring in June 2012, the M&E function was also centralized to the PIU. The M&E team continued to collect data from each of the schools and visit all the target schools under the project as well as continue supervision of new construction. The PIU M&E team had also been proactive in communicating with all the schools under the BEP project and raising issues/challenges to State and Federal Ministry officials. The team had been working closely with communities to better understand reasons why children were dropping out and ways to improve teacher retention, recording available resources for schools and how the resources were utilized to enhance the learning environment for students.

Overall assessment: ACHIEVED

37

Annex 3. Economic and Financial Analysis (including assumptions in the analysis)

In November 2012, a stocktaking exercise of school construction project was undertaken. The exercise covered: old World Bank-, IDB-financed and UNESCO-supported projects in the Khartoum area; UNICEF-financed projects in Kassala, Red Sea and Darfur States; MTDF-financed MoGE’s project in Blue Nile, South and North Kordofan, and Red Sea States; MTDF-financed CDF project in Blue Nile, Kassala and both Kordofan States; and projects by NGOs such as ZOA and CRS in Darfour States. This exercise allows compiling and comparing resulting outputs in terms of cost-efficiency, in particular vis-à-vis the Basic Education Project. Table 12: BEP Construction Unit Costs (actualized 2012) Location of the project

State Blue Nile South Kordofan North Kordofan

Red Sea Average

Year 2010 2010 2010 2010 2010 Implementor MoE CDF MoE

RS Engin eering Const.Off

All Unit

MoE's Engineering Office

CDF's Locality Implementation Unit

Min of Urb Planning

School

Aghro Rahmat Allah

Average Kadbat Kangar Average Alkobra Marafeet All sites

Technology

Walls Red bricks Red bricks

Stones Stones Cement block

All techno logies

Roof shape Single Slope Double slope

Single slope

Single slope

Single slope

Frame materials iron pipes iron pipes iron wood

Roof materials Corrugated iron sheet Corrugated iron sheets (CIS) CIS zinc

2012 US$/m2 - Unit Cost of construction of a 2 classroon+office room - Actualized 2012

Construction US$/m2 201 225 213 166 150 158 164 203 185

Total cost US$/m2 201 225 213 173 156 164 167 212 189

To assess the efficiency of BEP, various cost analysis allows for cost-comparisons across projects with respect to their implementation arrangements and technology. The cost comparisons are focusing on the blocks of classrooms, and only take into account the tax-free gross square meter. The methodology provides disaggregate information of both the direct cost of construction, i.e. the cost of the contractor’s contract, and the total project cost, resulting from the addition the direct cost and all the indirect costs, for instance, the management and supervision costs, as far as they are known. The four groups of projects are the following.

UNICEF project with Italy Funds implemented by Kassala and Red Sea State administration; UNICEF-funded projects implemented by NGOs in Darfur States; NGO projects using SSB technology in Darfur Sates; BEP projects financed by MTDF and implemented through by different agencies, in Blue Nile,

North and South Kordofan and Red Sea States, CDF-managed projects financed by MTDF in North and South Kordofan and Blue Nile States.

39

Table 13: Project costs and construction unit costs of 6 projects in 2012 US$ per gross m2

Project Unit Cost 383 269 326 151 174 182 171 169 N/A N/A 213 173 156 167 212 189 302 224 216 Const Unit Cost 243 171 207 86 130 130 126 118 153 155 213 166 150 164 203 185 290 215 173

Technology

RB- JA

RB- CIS

Ave

rage

RB-CIS

RB-CIS

RB-CIS

RB-CIS

Ave

rage

SSB-CIS

SSB-CIS

RB- CIS

RB- CIS

St- CIS

St- CIS

CB- CIS

Ave

rage

St-CIS/JA

St-CIS/JA

Glo

bal

Ave

rage

Procure- ment

method NCB NCB LCB LCB LCB LCB LCB LCB NCB NCB NCB NCB NCB LCB agency UNICEF CRS RDN Autash WVI ZOA CRS MoE CDF MoE MoE CDF Com

Location

Kas sala

Red Sea

Dar fur

Dar fur

Dar furDar fur

Dar fur

Dar fur

Blue Nile

South Kordofan

Nth Kord

Red Sea

NK, SK, BN, Kassala

Project name WASH-CFS CFS Schools in Dafur N/A N/A Basic Education Project (BEP) CDF Finanncing agency Italy UNICEF ZOA CRS MDTF MDTF

Implement. agency

name UNICEF thrgh States

UNICEF through NGOs ZOA CRS FMoGE/PIU through SMoE and CDF

PM

type UNO UNO NGO NGO Goverment SF

Notes: RB = red bricks; JA = Jack Arch; CIS = corrugated iron sheet; St = stones; SSB = Soil Stabilised Blocks; LCB = local cpmpetitive bidding; NCB = national competitive bidding; CFS = Child Friendly School; CRS = Catholoc Releif Service; RDN = Rural Development Network; WVI = World Vision International; CDF = Community Developmemnt Fund; Com = comminities; MTDF = Multidonor trust fund; N/A = non applicable.

The lowest global unit costs – around 170-190 US$/m2 -- are achieved through both NGOs and BEP, i.e. through totally different approaches: a) NGO managing projects by delegation from UNICEF and b) the BEP’s PIU supervising projects managed by various managing entities such as States, and CDF

CDF global unit cost is the second higher – 300 US$/m2 -- because its management costs are low, therefore, global unit cost of NGO working for UNICEF/Italy, have surpassed CDF’s one

The highest global unit costs – average 325 US$/m2 -- is achieved by States implementing projects with UNICEF/Italy funds, because high indirect costs are topping already medium-high construction costs.

Source: Technical Note on School Construction November 2012 – BERP preparation mission

40

Comparison of unit costs of different projects (in 2012US$ per gross m2)

41

Annex 4. Grant Preparation and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending/Grant Preparation Jamal Abdulla Abdulaziz Senior Procurement Specialist MNAPR Afrah Alawi Al-Ahmadi Senior Human Development Speci SASSP

Ahmed Chabchoub Consultant AFTH3 - HIS

Prema Clarke Senior Education Specialist HDNFT Therese M. Diomi Temporary AFTSP Yasser Aabdel-Aleem Awny El-Gammal

Lead Social Protection Special AFTSP

Selma Ahmed Siddig Team Assistant AFMSD Isabel Cristina Soares Senior Operations Officer AFMSD Yasmin Tayyab Senior Social Development Spec AFTCS Frederick Yankey Sr Financial Management Specia AFTFM

Supervision/ICR Mohamed Yahia Ahmed Said Abd El Karim

Financial Management Specialis AFTFM

Afrah Alawi Al-Ahmadi Senior Human Development Speci SASSP Kaliope Azzi-Huck Operations Officer AFTHD Antonio J. Cittati Consultant AFTED Prema Clarke Senior Education Specialist HDNFT Omer Nasir Elseed Consultant AFTED Endashaw Tadesse Gossa Senior Operations Officer AFTSP Abdelmonem Osman Kardash Environmental Spec. AFTEN Selma Ahmed Siddig Team Assistant AFMSD Isabel Cristina Soares Senior Operations Officer AFMSD Enas Mohammed Suleiman E T Temporary AFMSD

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending Total: 0.00 Supervision/ICR

Total: 0.00

42

Annex 5. Beneficiary Survey Results – the ICR writer visited North Kordofan State (where highest number project schools are located) and conducted interviews with school directors, teachers and parents School No. 1 – Um Derisa Distance to state capital: two and a half hour’s drive Distance to closest paved road: 10 km Project interventions: building of 4 blocks, each

with 2 classrooms and 1 office; separate latrines for girls/boys; rain water capturing system, fences; teacher dormitory; furniture & school supplies; & teacher training

2013 enrolment: 344 students

Grade Classes Total Boys Girls G1 1 73 35 38 G2 1 52 32 30 G3 1 48 27 21 G4 1 44 22 24 G5 1 38 15 23 G6 1 34 16 18 G7 1 26 14 12 G8 1 29 22 7

Compared with 2012 enrolment: increased by 95

students For some students, it takes one hour to get to school

each day Retention is a challenge (approximately only 40%

of cohort survival) Teachers – 7 regular teachers and 3 volunteer

teachers Volunteer teachers are paid by the community, they

are not yet recognized by the Ministry hence have no training, career path and pension

School finance – Ministry covers salaries of regular teachers and ocasionally chalks. Parents contribute SDG4-9/month to cover additional teacher salaries and school operational costs. Grade 1 students each pay SDG25 registration fee, 80% of which will be transferred to the education administration office, not retained for school spending needs

Parents source of income: farming and some livestocks

Seven students share one textbook

Classrooms/offices and water system

A crowded class

Students’ houses

43

School No.2 - Um El Badry Distance to state capital: three hours’ drive Distance to closest paved road: 25 km Project interventions: building of a 1000m2 school

compound with 4 blocks, each with 2 classrooms and 1 office; separate latrines for girls/boys; rain water capturing system, fences; teacher dormitory; furniture & school supplies; & teacher training

2013 enrolment: 654 students

Grade Classes Total Boys Girls G1 1 102 60 42 G2 1 116 63 53 G3 1 86 43 43 G4 1 87 50 37 G5 1 70 40 30 G6 1 85 40 45 G7 1 57 27 30 G8 1 52 31 21

Compared with 2012 enrolment: increased by 50 students

Because of crowded classrooms, some students have to study outside

School day starts at 8.30AM, grade 1 and 2 students leave at 12.00PM while the other grades stay up to 4.00PM

Some students live around 7 to 10 km from the schools and it takes one and a half hour for them to get to the school each day

Retention is a challenge: around 50% cohort survival

Textbook costs range from SDG40 for grade 1 to SDG100 for grade 8

Only around 20 percent of students have full sets of textbooks

Teachers: 7 regular teachers, 3 volunteer teachers and 1 kindergarten teacher. Given very high enrolment and extremely crowded classrooms in the early grades, the locality has promised to provide more teachers but this has not been done

Teachers feel most confident teaching Arabic language and Islamic studies. They find teaching English, mathematics and science difficult and the training they received inadequate for their job

School finance: Parents contributed SDG15,000 annually to cover non-regular teacher salaries, stationary, school maintenance and water collection costs

Teachers’ dormitory

After meeting with BEP/WB team, parents stayed on for their PTA meeting

Theater and water system

44

School No. 3 Um Sedeira Distance to state capital: two and a half hour’s drive Distance to closest paved road: 15 km Project interventions: building 4 blocks, each with 2

classrooms and 1 office; separate latrines for girls/boys; rain water capturing system, fences; teacher dormitory; furniture & school supplies; & teacher training

2013 enrolment: 604 students Grade Classes Total Boys Girls

G1 2 115 64 51 G2 1 84 44 40 G3 1 109 59 50 G4 2 95 50 45 G5 1 74 38 36 G6 1 54 28 26 G7 1 65 42 23 G8 1 28 12 16

Compared with 2012 enrolment: increased by 90

students 10% of the students live 8 to 10 km from the schools

and during rainy season they tend to miss 2 periods each day

Retention is a challenge: less than 30% cohort survival

Teachers: 10 regular teachers (6 with bachelor degrees and 4 with high school degrees but are currently enrolled in the Open University upgrading courses). Of these teachers, only one can teach math and science. Two of the teachers indicated their intention to leave the school but now decided to stay until the next school year

One volunteer teacher is recruited to teach science subjects (physics and chemistry) due to high load. The school has requested for this volunteer teacher to be formally recruited by the National Teaching Service but this was not confirmed hence the community pays for this teacher remuneration

School finance: Grade 1 students pay SDG25 but the funds stay at the school, other grade students pay SDG2/month to cover water costs, maintenance, teacher salary, building of extra room in teacher dormitory and desks and chairs

The school opened a kindergarten class for more than 100 children in response to parents’ need and willingness to send their children to school early

Students proud of their new school

Teacher office turned into classroom

Parents contributed money to build this kindergarten classroom for more than 100 students

45

Annex 6. Stakeholder Workshop Report and Results (if any)

46

Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR The project had engaged a consultant to write a project completion report, following the WB ICR format. Below is the outline of the grantee’s ICR. 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.2 Rational for MTDF involvement 1.3 Original Project Development Objectives (PDO) and Key Indicators 1.4 Revised PDO and Key Indicators, and reasons/justification 1.5 Main Beneficiaries 1.6 Original Components 1.7 Revised Components 1.8 Other Significant Changes 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.2 Implementation 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.4 Post-completion Operation/Next Phase 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 3.2 Achievement of Project Development Objectives 3.3 Overarching Themes, Other Outcomes and Impacts 3.3.1 Poverty Impacts, Gender Aspects, and Social Development 3.3.2 Institutional Change/Strengthening 3.3.3 Other Unintended Outcomes and Impacts 3.4 Summary of intermediate results 3.4.1 Meeting Objectives 3.4.2 Impact 3.5 Lessons Learned 4. Project Costs and Financing 5. Recommendations Annexes Annex 1: Intermediate Results and Indicators Annex 2: Female Students Enrollment Annex 3: Based Education monitoring as of 30th March, 2012 Annex 4: Signed Contract Register from Inception to Date Annex 5: Persons Interviewed The report concluded that BEP has delivered tangible results towards its development objectives, but at different levels in each component, where, the improvement in access to basic education is most evident compared to the second (curriculum reform) and third (upgrading of TTIs). The report recommended that:

The state government to fulfill its obligations of providing seating, books and exercise books to all school children and secure running costs

The Ministry of education continue to reach out for all teachers in the states for training with emphasis on longer and more intensive training for teachers with secondary school education

The ministry should adopt the model of building teacher dormitories to encourage qualified teachers to stay in rural schools

The level of coordination and information dissemination needs to be improved within the ministry of education, with training institutions and the federal education authorities

47

The federal ministry, state MoEs and TTIs establish mechanism to develop training policies and an expert be assigned (at least for one year) to set up the administrative structure of TTI and define its relationship with other state and federal institutions and supervise the selection and training of the permanent staff of the TTIs

The report did not assess in details the implementation performance of the government, implementing agencies and of the Bank. The report was completed in May 2013 but by the end of August 2013 it has yet been formally discussed with MoGE and Ministry of Finance.

48

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders

49

Annex 9. List of Supporting Documents Aide-Memoires of project supervision missions_World Bank various years

Basic Education Project Final Project Paper_World Bank 2009

Basic Education Recovery Project Appraisal Document_World Bank 2013

English Language Curriculum Reform _British Council Report: September 2012_ May 3013

Implementation Support and Result (ISR) reports_World Bank various years

Multi-Donor Trust Fund – National (MDTF-N)_Implementation Completion and Result Report_May 2013

Strategic Business Plan for Teachers Training Institutes - Red Sea and North Kordufan_May 2013

Sudan Service Delivery Indicator Surveys_World Bank 2009 and 2010/11

Technical Note on School Construction_World Bank November 2012

The Evaluation of In-service Teacher Training in Four States in Northern Sudan_Bring up Teachers’ Knowledge and Skills through In-service Training_ May 2011

The Status of the Education Sector in Sudan_World Bank 2012