Independent Evaluation of the African Development Bank’s ... · Bank (hereafter referred to as the Bank) is undertaking an evaluation of the Bank’s strategy and programmes in

Independent Evaluation of the African Development Bank’s Strategy and Programme in Egypt 2009-2018

Inception Report

23 July 2019

Rue de Clairvaux 40, bte 101 – B 1348 Louvain-la-Neuve – Tel +32 10 45 45 10 – Fax +32 10 45 40 99

E-mail [email protected] – Website www.ade.eu

ADE has prepared this report at the request of the Independent Development Evaluation (BDEV) Unit at the African Development Bank

mailto:[email protected]

http://www.ade.eu/

INDEPENDENT EVALUATION OF THE AFRICAN DEVELOPMENT BANK’S STRATEGY AND PROGRAM IN EGYPT 2009-2018 ADE

Inception Report – July 2019 Table of Contents

Table of Contents

LIST OF ACRONYMS AND ABBREVIATIONS .................................................................... 5

1. INTRODUCTION ................................................................................................. 1

1.1 OBJECTIVES AND SCOPE ................................................................................................. 1 1.2 EVALUATION PROCESS .................................................................................................... 3 1.3 RISKS AND LIMITATIONS ................................................................................................. 4

2. COUNTRY CONTEXT ........................................................................................... 5

2.1 COUNTRY PROFILE SUMMARY ....................................................................................... 5 2.2 MAIN CHALLENGES ........................................................................................................ 6

3. METHODOLOGICAL APPROACH .......................................................................... 9

3.1 STRATEGIES AND THEORIES OF CHANGE ................................................................... 9 3.2 THEORY-BASED AND PYRAMIDAL APPROACHES ....................................................... 12 3.3 SAMPLING PROCESS ...................................................................................................... 18 3.4 PORTFOLIO REVIEW..................................................................................................... 23 3.5 EVALUATION MATRIX .................................................................................................. 23 3.6 DATA COLLECTION TOOLS ........................................................................................... 25 3.7 REPORTING AND DISSEMINATION PHASE .................................................................. 26

4. ROLES AND RESPONSIBILITIES .............................................................................. 27

5. EVALUATION CALENDAR .................................................................................. 29

LIST OF TABLES: Table 1- Main challenges of the implementation of the evaluation and mitigation strategy 4 Table 2 – Framework for project analysis .................................................................................. 19 Table 3 – Overview of Cluster 1 (PRA) and Cluster 2 (desk review) projects ..................... 20 Table 4 – Evaluation Questions .................................................................................................. 24 Table 5 – Evaluation Calendar ..................................................................................................... 29

LIST OF FIGURES: Figure 1 – Evaluation purpose and scope (2009-2018) .............................................................. 1 Figure 2 – Methodological process, tasks and deliverables ....................................................... 3 Figure 3- Overview of GDP growth rate 2008-2018 ................................................................. 5 Figure 4- Macro-economic indicators 2008-2018 ....................................................................... 6 Figure 5 – Evaluation team organizational structure ................................................................ 27

VOLUME II - ANNEXES ANNEX 1 TERMS OF REFERENCE ANNEX 2 EVALUATION MATRIX ANNEX 3 BANK’S PORTFOLIO SUMMARY IN EGYPT (2009-2018) ANNEX 4 PRAS – GEOGRAPHICAL DISTRIBUTION ANNEX 5 OUTLINE: TECHNICAL REPORT AND SECTOR SUMMARY NOTES ANNEX 6 BIBLIOGRAPHY ANNEX 7 PROJECT REVIEW TEMPLATE ANNEX 8 PORTFOLIO REVIEW TEMPLATE ANNEX 9 LIST OF PEOPLE CONSULTED DURING THE SCOPING MISSION ANNEX 10 PROPOSED COMMUNICATION AND DISSEMINATION ACTIVITIES AND PLAN


Inception Report – July 2019 Acronyms

List of acronyms and abbreviations

AfDB African Development Bank

ADF African Development Fund

AFD Agence Française de Developpement

AGTF Africa Growing Together Fund

AWF The African Water Facility

AU African Union

CAPMAS Egypt Statistical Bureau

CBE Central Bank of Egypt

CO AfDB Egyptian Country Office

CODE Committee on Operations and Development Effectiveness

COMESA Common Market for Eastern and Southern Africa

CSO Civil Society Organization

CSP Country Strategy Paper

CSPE Country Strategy and Programme Evaluation

DP Development Partner

EBI Egyptian Banking Institute

EBRD European Bank for Reconstruction and Development

EIB European Investment Bank

EQ Evaluation Question

FDI Foreign Direct Investment

FGD Focus Group Discussion

GEF Global Environment Facility

GIZ German Agency for International Cooperation

GoE Government of Egypt

GDP Gross Domestic Product

BDEV Independent Development Evaluation - African Development Bank

IFC International Finance Corporation

ILO International Labour Organisation

IMF International Monetary Fund

IPR Implementation Progress and Results Report

IsDB Islamic Development Bank

MCIT Ministry of Communication and Information Technology

MENA Middle East and North Africa

MIC Middle-income Country

MIIC Ministry of Investment and International Cooperation

MOP Ministry of Planning, Monitoring and Administrative Reform

MOU Memorandum of Understanding

MTR Mid-Term Review

NCW National Council for Women


Inception Report – July 2019 Acronyms

PAR Project Appraisal Report

PBO Programme-Based Operation

PCR Project Completion Report

PFM Public Finance Management

PPP Public-Private Partnership

SFD Social Fund for Development

ToC Theory of Change

TOR Terms of Reference

UA Unit of Account

UN United Nations

UNDP United Nations Development Programme

USD United States Dollar

WB World Bank


Inception Report – July 2019 Page 1

1. Introduction

The Independent Development Evaluation function (BDEV) of the African Development Bank (hereafter referred to as the Bank) is undertaking an evaluation of the Bank’s strategy and programmes in Egypt over the period 2009-2018. The Country Strategy and Programme Evaluation (CSPE) for Egypt is part of BDEV’s work plan as approved by the Bank’s committee on Development Effectiveness (CODE). The CSPE for Egypt is intended to inform the development of a new CSP for the country due in 2020 and provide useful lessons for both the country and the Bank to inform future decision-making for addressing the development needs of the country. The final report of this study should be disseminated by BDEV to key stakeholders in and outside the Bank and the public. The present document, the Inception Report, is the first main deliverable of the evaluation.

1.1 Objectives and Scope

The evaluation purpose and scope are described in detail in the ToR (Annex 1) and are summarised in the figure below.

Figure 1 – Evaluation purpose and scope (2009-2018)





Interventions to be covered are framed by three Country Strategy Papers (CSPs): (i) CSP 2009-2011 (ii) CSP 2011-2013 (and extended to 2015) (iii) CSP 2015-2018

The outputs of the assignment are the following:

(i) three sector notes including an analysis of the portfolio of the 55 interventions and the sector-specific Theory of Change (ToC);

(ii) a technical report including an overall analysis and synthesis of the Bank’s activities in Egypt based on the sector notes, and an examination of the overall Theory of Change and of specific strategic issues;

(iii) contribution to the BDEV summary of the technical report.

1.2 Evaluation process

The evaluation process was structured in three phases: inception phase, data collection and analysis phase, and reporting and quality assurance phase. Figure 2 summarizes for each phase the tasks to be carried out in relation to the methodological approach, and the corresponding deliverables. This intended process has, however, been disrupted by a number of unplanned issues and relating difficulties, which are being discussed with BDEV and the Bank’s office in Cairo. As a matter of example, the scoping mission undertaken in Egypt from 21st to 24th April 2019 didn’t allow to meet expected stakeholders. A second scoping mission was then conducted from 9th to 16th June 2019, directly followed by the field phase which started on 17th June and is planned to end on 4th July 2019.

Figure 2 – Methodological process, tasks and deliverables



1.3 Risks and limitations

Various risks and limitations were identified in the methodological proposal and during the 1st scoping mission. They are summarized below along with the ways in which the evaluation team intends to mitigate them.

Table 1- Main challenges of the implementation of the evaluation and mitigation strategy

Challenges Mitigation strategy

The width and depth of the scope of a CSPE, which covers ten years and a variety of sectors

CSPEs are well known to the Team Leader and to ADE, which have conducted many such studies, including five of them for BDEV. Our robust methodological approach is designed to cope with this complexity including through the definition to a set of structured evaluation questions and a variety of evaluation tools and methods.

A profound evolution of the context during the evaluation period, requiring strong adaptation by the Bank of its strategies and operations, and affecting institutional memory

ADE has extensive experience in CSPEs in countries with a quickly evolving context. The proposed methodological approach addresses these issues, notably as follows:

The need to acknowledge the instable situation and requirements for AfDB to adapt rapidly its strategic approach and operations. The evaluation approach hence goes beyond a theory based evaluation: it will need to evaluate not only against stated objectives in CSPs but also against the changes in the context.

We will aim at meeting key stakeholders active during the different sub-periods over the last 10 years.

We will also ensure a proper balance in our analysis of AfDB’s support to Egypt during these different sub-periods.

The two scoping missions will make it possible to better understand the context of AfDB support to Egypt and data availability, and on this basis to fine-tune the envisaged methodological approach.

The need for active collaboration from national authorities to conduct successfully this evaluation

We will seek together with BDEV the support of the national authorities where needed (e.g. for data collection, interviews). We will discuss mitigation measures with BDEV whenever necessary. Close collaboration has been established with the MIIC to facilitate an effective implementation of the evaluation. Efforts are made by ADE to plan ahead best possible by assigning specific tasks to one team member to ensure that meetings are not delayed or cancelled- or if so occurring, re-arrange suitably new meetings accommodating the parties involved.

Lack of data

This inception report is based on an initial review of documentation and scoping phases, which has sought to explicitly address weaknesses in data availability and design accordingly.

Evaluation / review / assessment fatigue among interviewees means that there may be limited willingness to grant us time for interviews and meetings

The evaluation team will arrange its work to minimize the ‘contact time’ required of stakeholders by preparing for efficient interviews and focus groups.

Source: ADE technical proposal, updated BDEV ToR (May 2019)



2. Country context

2.1 Country Profile Summary

Egypt with its approximately 100 million inhabitants is the third-most populous country in Africa (after Nigeria and Ethiopia), and the most populous in North Africa, the Middle East, and the Arab world. The country is of major geopolitical importance given its strategic position at the crossroads of Africa and the Middle East. A timeline of key facts of development during 2009-2018 is presented in Fig. 3.

Figure 3- Key developments over the period 2009-2018

Source: ADE technical proposal

Over the last 10 years, Egypt has been undergoing a transformation into a market-oriented economy. The Egyptian economy, which had previously shown resilience to the global financial crisis with an average growth rate of 5% over the period 2009-2010 has suffered from the internal political crisis and revolutionary uprising of 2011. During this period, GDP growth slowed to between 1.8 % in 2011 and 2.9% in 2014. Nonetheless, the economy has recovered since 2015 and further accelerated in the 2016-2017 fiscal year despite the large fiscal deficits and a record of increased public debt (See figure 4).

Figure 3- Overview of GDP growth rate 2008-2018

Source: Egypt Results Brief | AfDB

0

2

4

6

8

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020

Real GDP, % increase on a year earlier

Egypt Timeline

Global recession

Post-uprisingturmoil Recovery

Projected growth



Figure 4- Macro-economic indicators 2008-2018

Indicators 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

GDP per capita (current US$)

2270 2578 2921 3077 3383 3400 3524 3731 3686 2495 2573

Inflation rate (%) 11.7 16.2 11.7 11.1 8.6 6.9 10.1 11 10.2 23.5 20.9

Fiscal balance (%GDP) -6.8 -6.9 -8.1 -9.9 -10.7 -13.3 -12 -11.6 -11.2 -9.5 -9

Public debt (% GDP) 74.3 73.3 73.7 76.2 80.3 87.1 90.5 85 92.3 101.2 92.5

Foreign direct investment, net inflows (BoP, current million US$)

9494 6711 6385 -482 2797 4192 4612 6925 8106 7391

Source: World Bank, IMF, 2018

2.2 Main Challenges

A review of the evolution of development challenges Egypt faces points to the following key areas, some of which have been persistent over the period under review. They include the following:

a) A large population with growing basic needs and high expectations in terms of social, spatial and economic inclusion: in 2017, Egypt has a population of about 97 million that grows annually at about 2% with more than 50% of the population composed of young people (under 24 years), and a life expectancy of about 70 years. These demographic trends are putting increasing pressure on the Government of Egypt (GoE) to ensure adequate provision of basic public services such as food, education, transportation and health. This is further characterized by a high youth unemployment rate of about 34%.

b) Macroeconomic challenges and constrained fiscal space: The high inflation rate has hit hard on the poor and middle class. Over the past two years, the movement has managed to steer the country away from a crisis; foreign reserves are up, the exchange rate is flexible, and the capital account is open. However, the fiscal deficit, public debt to GDP ratio and current account deficit are ail on a downward trajectory. Despite the current tight monetary policy, the real GDP growth is up. For the medium to long term and given Egypt current deep structural reforms, real GDP growth is expected to reach a higher growth rate of 5.3%‘° for FY 2018/19.

c) Governance and corruption levels: Egypt has recorded weaknesses in some key governance indicators. A major concern over the period is the high levels of corruption noted in the strategy documents and review reports, which has constrained the GOE’s capacity to effectively implement its development projects, and in a timely manner for better results. For example, according to the corruption perception index established by Transparency International, Egypt was ranked 117 out of 180 countries in 2017.

d) Insufficient infrastructure: Though the country has seen improvements over the review period as noted in recent economic reviews of the Bank, infrastructure is still considered insufficient and inefficient and need to be enhanced, especially in transport and service provision, to sustain the ambitious economic development plans and achieve structural transformation. The country was ranked 56th in infrastructure and 10Oth in



the information and communication technology adoption out of 140 countries according to the 2018 Global Competitiveness Index (GCI)."

e) Underperforming private sector and potentials: The private sector is characterized by several constraints such as non-conducive business environment, inadequately skilled workforce particularly among the youth and limited entrepreneurship development in rural areas, particularly in agricultural sector. These are noted in recent reviews as limiting its competiveness, contribution to economic growth and job creation. Globally Egypt is ranked 120 among 190 economies in the ease of doing business, according to the 2019 World Bank Doing Business Report.

National security concerns: The Government continues to fight terrorism within a challenging regional context. It has intensified efforts to provide security through increased arrangements throughout the country and in particular the Sinai Peninsula.



3. Methodological approach

This chapter starts with a description of the Bank’s strategies in Egypt and of the relating Theory of Change. It is followed by a description of the overall methodological approach followed (based theory-based and pyramidal approaches) and a presentation of the overall evaluation questions defined in the ToR. It then presents the sampling process, the portfolio review, the evaluation framework, the data collection tools, and a description of the reporting and dissemination phase.

3.1 Strategies and Theories of Change

3.1.1 Country strategies

The Bank’s programme in Egypt for the period under review touches on three strategy periods:

1. The 2007-2011 Country Strategy Paper (CSP) for which an MTR and completion report were respectively concluded in 2009 and 2011.

Two pillars were developed: (i) private sector development, and (ii) social development and protection. The MTR of the 2007-2011 CSP in 2009, and the 2011 Combined CSP Completion Report and CPPR concluded that the Bank’s support during this period focused mainly on infrastructure, which accounted for 78.8% of active operations, compared with 21.2% for social development and protection.

2. The 2012-13 Interim Country Strategy Paper (I-CSP), which was also subsequently updated and extended to June 2015 was to ensure continuity in the Bank’s support to Egypt during the transition phase following the political turmoil.

Two pillars were developed: (i) stabilization and economic recovery, and (ii) inclusive growth to reduce poverty. Lending was suspended until April 2015. Support was refocused on providing grants to support capacity development during the transition phase in various sectors, notably multi-sector, agriculture, water and power.

3. Two pillars were developed (i) infratsructure, and (ii) governance. The 2015-2019 Country Strategy Paper, approved in April 2015 for which a mid-term review has been conducted and approved by the board in May 2019 and the strategy period ending in December 2019. It has recently been decided (June 2019) to extend the CSP till the mid/end of 2020.

Through this CSP the Bank seeks to achieve (i) high and well-diversified growth that creates jobs and higher value added, and (ii) social justice and inclusion while ensuring sound macroeconomic management.

Thus the three CSPs mainly focused on infrastructure development, macroeconomic stabilization, and inclusive growth. The programme has evolved under the three different strategies, but maintained notable continuity with a focus on governance projects channeled mainly through 3 PBOs. The governance programme sought to provide support to the energy sector and enabling activities to boast private sector development. Based on the interventions made by the Bank during the evaluation period the BDEV/team reconstructed a preliminary theory of change which has been further elaborated upon by the ADE team. The latter is shown in Fig. 6

An overview of the evolution of the Bank’s strategies and intervention during the evaluation period 2009-2018 is presented in Fig. 7.

AFDB COUNTRY STRATEGY AND PROGRAM EVALUATION – EGYPT ADE


Figure 6 – Draft Reconstructed Bank’s Intervention theory of change in Egypt- 2009-2018

AFDB COUNTRY STRATEGY AND PROGRAM EVALUATION – EGYPT ADE


Figure 7 – Evolution of AfDB strategies and interventions in Egypt during the period 2009-2018



3.2 Theory-based and pyramidal approaches

A Theory of Change (ToC) approach will be employed. This will address the results that were achieved and also how and why those results were, or were not, achieved. Therefore it will combine both quantitative and qualitative methods using multiple data collection processes. The ToC approach will be coupled to a pyramidal approach. The latter aims to provide specific inputs at three levels of analysis: (i) Strategic level (CSP) (ii) Sector/portfolio level in which sector overviews are provided (iii) Project level, in which an analysis of the portfolio of 53 interventions is conducted.

It places strong emphasis on the articulation and integration between the outputs obtained (Figure 8) in order to provide an overall analysis and synthesis of AfDB support to Egypt during the period under evaluation. Although all projects will be assessed, including non-lending analytical and advisory work (ESW), mainly through the use of secondary data and key informant interviews, three sector summary notes will be produced covering power/energy, water and agriculture. They will be produced following the proposed template, provided by BDEV, included as an Annex of the report. These assessments will overall also help to determine the extent to which cross-program synergies were achieved or not.

Figure 8 – Pyramidal approach- inputs & outputs by level of analysis

There are five key principles in the proposed Pyramidal Approach: clustering, scoring, aggregation, triangulation and sourcing:

1 Clustering and Scoring serves for the analysis of each of the 53 interventions;

2 Aggregation will facilitate the sector-specific analysis and the strategic review: country and/or programmatic;



3. Triangulation and sourcing are transverse principles used throughout the evaluation work at each level of analysis.

1. Clustering and Scoring

Each criterion included in the intervention assessment grid will be scored. The rating scale includes four levels (from 1 "very unsatisfactory" to 4 "very satisfactory"). This scoring scale aims at ensuring sound qualitative assessment based on evidence derived from the documentary analysis and individual or collective exchanges with stakeholders. This scale, used for the public sector operations, is illustrated below for the main criteria. The PBO’s will however, have appropriate scoring system adjusted to assess the efficiency and effectiveness criteria.

2. Aggregation

The aggregation of results and scoring derived from intervention assessment grids is necessary and useful for establishing findings and analysis at both sector and strategic levels. Answering EQs requires more than a simple compilation of individual intervention assessments. The aggregation will facilitate the sector-specific analysis and the strategic review.

Score 1 2 3 4

Scale Very unsatisfactory Unsatisfactory Satisfactory Very satisfactory

Relevance

Very weak, not at all appropriate design and very limited / non-existent alignment

Weak, improper design and limited alignment / several gaps

Rather strong and appropriate design, strong alignment

Very solid and appropriate design, complete alignment

Effectiveness

Expected outputs not achieved in most cases

None or very few of the intended results are achieved

Expected outputs achieved with significant gaps

Few targeted results are achieved

Expected outputs achieved in most cases

Most of the targeted results are achieved

Expected outputs fully achieved

All targeted results are achieved

Efficiency

Very significant difference between the ERR and the opportunity cost of capital

Very significant gap between planned and actual timing of implementation

Significant difference between the ERR and the opportunity cost of capital

Significant gap between planned and actual timing of implementation

Moderate difference between the ERR and the opportunity cost of capital

Moderate gap between planned and actual timing of implementation

Little or no difference between the ERR and the opportunity cost of capital

Little or no difference between planned and actual timing of implementation

Sustainability

Mechanisms guaranteeing sustainability (technical, economic, financial, institutional, partnership, environmental and social) are not assured

Mechanisms guaranteeing sustainability (technical, economic, financial, institutional, partnership, environmental and social) are hindered by significant risks

Mechanisms guaranteeing sustainability (technical, economic, financial, institutional, partnership, environmental and social) are generally ensured, with certain minor risks

Mechanisms guaranteeing sustainability (technical, economic, financial, institutional, partnership, environmental and social) are fully assured



3. Triangulation and sourcing

Evaluation analysis is valid if it derives from several sources of information. It requires cross-verification and demonstration of the evidence on which an assessment is based. Triangulation is a key factor in an evaluation to the extent that it fosters quality and soundness and ensures that the findings are supported by clear and multiple lines of evidence; it entails confirmation, enrichment and explanation. Three levels of triangulation are envisaged for this evaluation: the evaluator’s level, the methodological level, and the data level.

Figure 9 – Main sources of information and tools

Transversal themes - gender and inclusiveness

The proposed method seeks to maximize a systematic evidence-based assessment and includes four main steps:

1. Preliminary analysis, framing and understanding of specific country issues for each crosscutting theme.

2. Screening of the importance of the discourse and perspective(s) with a view to taking

account of the expectations for each crosscutting theme as well as of specific objectives included both in the CSPs and in Egypt’s national strategies.

3. Based on the specific objectives and indicators identified previously, semi-structured

interviews with different actors will aid understanding and analysis of key trends in the evolution of each cross-cutting domain/theme - progress recorded, risks and issues encountered during the evaluation period.

4. Specific analysis of the implementation of planned activities by the Bank, their achievements and demonstration of those achievements through the monitoring and documentation analysis and interviews with project stakeholders.

Non-lending activities

Non-lending activities mainly include analytical work for supporting programme decision-making processes, the related advisory role (often in the form of TA), and policy dialogue.



These knowledge-deepening activities can be linked and reinforce each other but they also concern each level of intervention and analysis within the pyramidal approach (intervention, sector and strategic levels). With regard to the analytical work, the following approach will be used in establishing performance:

establishing the list of analytical work, studies and inputs explicitly envisaged at strategic or sector levels as well as in the context of specific interventions;

collecting evidence on the implementation, publication and dissemination of planned works and studies, contextualizing their specific success factors and barriers;

understanding the level of use and usefulness of analytical work and studies in the light of perceptions and feedback from the stakeholders encountered.

As for policy dialogue activities, the evaluation team will proceed by:

mapping intended domains and themes for dialogue within the Bank’s strategies (CSPs);

analyzing the human resources and technical capacities mobilized to lead and follow this dialogue;

understanding the nature and importance of achievements, and of factors and issues that constitute a hindrance or promotion, based on discussions with the Bank's in-country representatives as well as with strategic stakeholders interviewed in the key national ministries.

Beyond these specific issues non-lending activities will be assessed on the basis of the criteria, EQs and indicators specified in the evaluation matrix.

Factors associated with performance

During the evaluation period, several contextual socioeconomic and political factors influenced the way the Bank designed and implemented its strategy. Therefore it is necessary to systematically identify the contextual factors that improve or hamper the Bank’s and country’s performance. The evaluation team will collect and categorize these factors at the project, sector and strategic levels in order to respond to specific EQs. The contextual issues will be considered in analyzing performance of the overall program.

Further issues for investigation identified during the inception period includes:

A set of non-exhaustive questions needing further analysis through interviews has been identified during the scoping and inception period. These include: Reform process

To what extent has it been possible for Egyptian ministries and State agencies/ companies to adapt to the administrative reform process that has been embarked upon, guided and managed by the Ministry of Planning? What challenges have been observed or have emerged so far? And what mechanisms have been set in motion to comply with the process?

What mechanisms/steps are set in motion by the government/ministries to ensure that the new mindset for improved performance within the ministries and State agencies/



companies are effectively put into action? What mechanisms are used to ‘break down’ old habits and what mechanisms to promote new ways of working – have the top- and middle-level managers knowledge of and the aptitude to adopt ‘change management’?

Coherence and flexibility of operations/interventions

To what extent is the program logic coherent? Are pathways logically established and are linkages between stages in the project cycle strongly anchored, i.e. are outputs linked to outcomes and impact. For example, are baselines established for measuring impact on food security, income and health improvements in water and power infrastructure operations? Is coordination between ‘technical’ and ‘social’ ministries satisfactorily established to ensure that impact can be measured?

With the changes occurring and the reforming process rigorously in motion it is hugely important that capacity-building is being targeted not only at project level but also at organizational level. Mindsets and new work procedures imply new ways of working and capacity and support are required in this area. New units, such SDS/Vision 2030, SME and M&E units, are established in every ministry and governorate, and planning mechanisms are changing to focus on impact. This hugely important institutional and human resource challenge should be addressed to make the reform process successful (emphasized by several of the ministries interviewed, e.g. MoPlanning and MoPetroleum) – how would various Egyptian institutions see this rolled out in practice, at central/ministerial/State agency levels and at Governorates level?

What is the mandate/role of the 33-member economic group of ministers, in relation to the development partners (apart from overall coordination) and on decisions on loan uptake? What does this mean for how the Bank will approach the government and facilitate conditions for loans to be attractive for the government?

Collaboration with the Bank on projects has been excellent overall, especially where expertise related to a loan/grant has been present in the Bank’s office yet more problematic where expertise presence is absent. What is a cost-effective solution to this?

From interviews there appears to be mixed messages as regards flexibility of the Bank in relation to addressing problems emerging prior to or during implementation of the interventions, including the design phase. How best could this be improved to improve the Bank’s reputation in this regard, including discussion of issues such as continuous presence of intervention experts in the CO, the level of decision-making of TMs/CO, management of constructively different procedures between Bank and authority, compliance with Bank’s strategies/policies, etc.?

Have the gender and other crosscutting concerns been satisfactorily addressed at all levels: project, sector and strategic? Are data and design in projects sufficiently disaggregated to provide information on gender issues and inclusiveness, progress, etc. adequately?

Private sector and FDI

To what extent has the Bank been able to contribute to improving the investment and business climate in Egypt?

To what extent have infrastructure investments facilitated the development of the private sector and its needs for accessing services (examples to be provided)



To what extent has the Bank facilitated, through its loans/grants, the development of private enterprises?

Has private entrepreneurship been mainstreamed in all aspects of the CSPs and projects implemented?

What does the government plan to attract foreign development investments (FDIs)?

Is the current selling off of energy-producing entities a viable solution?

In light of the government's pledge in the coming period to focus on improving the standards for the middle-income group, how do you plan to do that, considering that FDI is still very low?

Macro/socio-economic issues:

Income per capita has been stagnant over the evaluation period. What steps should be taken to reverse this situation?

The Egyptian Pound has seen (sharp) appreciation recently due to many factors related to improved tourism, remittances, Central Bank policies, but also the IMF loan which is coming to an end this year; how does the government plan to compensate for this, in the light of the low FDI?

How is Egypt’s plan for repayment of its debts proceeding? It has increased significantly: from 74% to 92,5% of total GDP (a 25% increase over the period 2009-2018). Is the current plan to sell off the energy-producing infrastructure part of a larger programme? Does the Debt Management Committee have a clear plan?

Youth unemployment has increased significantly over the period from about 27% in the late Mubarak years to a stagnant 35% over the last five years. What are the main mechanisms the government is applying to lower this rate? And what are the main barriers in that process?

The poverty head count (as a percentage of the population) has decreased significantly over the period (see statistics below). What is the explanation for that, considering the higher inflation rate and the stagnation of per capita income following the devaluation of the EGP? Are the data contradictory?

Poverty rate head-count 1,90usd (2011 PPP) % of population

Decreased significantly from 3,9% in 2008 to 1,3% (2015)

Poverty rate head-count 3,20usd (2011 PPP) % of population

Decreased significantly from 30,7% in 2008 to 16,1% (2015)



3.3 Sampling process

In accordance with BDEV practices, a set of criteria was applied to the 53 operations approved during the evaluation period to identify a sample of projects to be assessed. The criteria include the following:

projects approved in the period 2009-2018;

coverage of the different evaluation situation periods: Bank’s operations before the

political crisis (2009-2011), Bank’s operations during the transition period (2011-2015)

and the post-transition period (2015-2018);

coverage of both public and private sector operations;

coverage of the key sectors of the Bank’s interventions (power/energy, social,

multisector, governance, water and agriculture);

a balance between investments projects, technical assistance and studies (ESW)

government /sector ministries proposals for PRA/in-depth assessments including field

visits.

Based on these criteria, 25 out of 53 operations (almost half of the Bank’s operations during the review period) have been identified in the initial sampling. This is disaggregated into 10 investment projects out of 11, 11 standalone grants out of 39 (7 studies, 4 technical assistance), 1 line of credit and 3 programme-based operations (PBOs) covering all the main sector of the Bank’ interventions in the country, namely energy/power, water supply and sanitation, agriculture/irrigation, social and multi-sector. The line of credit (Egypt franchising sector support programmes) comprises a technical assistance component to support core project implementation. For the 3 PBOs assessment, the evaluation will make good use of the previous evaluation findings of PBOs conducted by BDEV in 2018, notably the case study on Egypt, and will proceed to a validation of the findings and an update, taking account of the fact that the last tranche of the PBO (EGESP III) released in 2018 has not yet been reviewed. Overall, the sample also entails three non-sovereign operations and 22 public sector operations (total of 25) with 14 completed operations, nine on-going operations and two operations at the approval stage (total of 25). The analysis will cover different sources of funding such as ADB windows, MIC Fund, MENA, Korean TF, AWF, AGTF, GEF, FAPA and take into account the different size of projects in terms of net loan amount. A three-level analysis of the sample is proposed (Table 2):



Table 2 – Framework for project analysis

Evaluation Matrix

Cluster 1

Cluster 2

Cluster 3

Project data - Y Y Y

Logic of intervention - Y Y N

Relevance Y Y Y Y

Effectiveness Y Y Y N

Sustainability Y Y Y N

Crosscutting Y Y Y N

Efficiency Y Y Y N

Design Y Y Y Y

Management for results Y Y Y N

Borrowers performance Y Y Y N

Lessons learned Y Y Y N

Bank’s performance Y Y Y N

Interviews - Y N N

Site visits - Y N N

Legend: Y= Yes; N=Not applicable

The PRA Analyses (Cluster 1) include 7 projects

The Desk Reviews (Cluster 2) include 18 projects

The Basic Assessments (Cluster 3) include 30 projects The Evaluation Matrix sets the stage for the overall analytic framework of the projects carried out during the evaluation period (1st column of Table 19). All key questions outlined in the ToR and related indicators developed by the evaluation team, as well as all questions contained in the Banks’ PRA questionnaire, have been incorporated into the Evaluation Matrix presenting the overall analytical framework for the evaluation. This makes it a heavy tool; the evaluation team may propose simplification in forthcoming stages of the evaluation. Cluster 1 entails the following analytical elements and steps:

define project data (name, country, year approved, year closed (if open), level of disbursement, sector, PCR date)

reconstruction of the theory of change (ToC) of the project

an analysis/assessment of all indicators under the various evaluation criteria and evaluation questions

interviews with the Bank’s Task Managers (TMs) and Egyptian partners and beneficiaries

visits to a selected number of project sites for field observations and acquisition of beneficiary perceptions of project development and achievements

This cluster involves Bank operations that are closed, either completed or near to completion (with at least 82% disbursement rate). These in-depth reviews include all projects having a Project Completion Report (PCR/PCREN) during the evaluation period. As such, of the total of seven projects, two (Abu Quir 1300 Mw Steam Power Project and Ain Soukhna



Thermal Power Generation Project) - approved before 2009 but with their PCRs prepared during the evaluation period - are included. Cluster 2 entails the following analytical elements and steps:

definition of project data (name, country, year approved, year closed, level of disbursement if open, sector, PCR if produced)

reconstruction of the Theory of Change of the project

an analysis/assessment of all indicators under the various evaluation criteria and evaluation questions

The Project Review template for Clusters 1 and 2 are the same, noting that the desk review contains the same exercise, while the PRA/in-depth approach to Cluster 1 projects includes field visits to project sites, unlike Cluster 2 projects. Cluster 3 entails the following analytical elements and steps:

Define project data (name, country, year approved, year closed, level of disbursement if open, sector, PCR if produced)

An assessment of indicators under the Relevance and Design evaluation criteria and evaluation questions, while noting key issues or developments observed in the project document(s) and that may have had / will have impact on the overall assessment of the project in question.

Table 3 shows the breakdown of the seven projects in Cluster 1 (PRA/in-depth analysis) and

the 18 projects in Cluster 2 (desk review)

Table 3 – Overview of Cluster 1 (PRA) and Cluster 2 (desk review) projects

Type of

operations

Sector/Project

name

Status

as at

Dec

2018

Desk

review

PRA

/

deep

dive

Net loan

(million

UA)

Source

of

funding

Category

Power-Energy:

Investment projects

Suez Thermal Power

Project

Ongo

(82.60%) 315.5 ADB Public

Shapoorji Pallonji 50

Mw Solar Pv Project

- Egypt Fit Round

APVD

(22.26%) 13.62 ADB+

GEF

Private

Abu Qir 1300 Mw

Steam Power Project

Comp 214.49 ADB Public

Ain Sokhuna

Thermal Power

Generation Project


Egyptian Refining

Company

Ongo

(100%) 143.38 ADB Private

Alcazar Solar APVD

(24.76%) 18.49 ADB Private



Type of

operations

Sector/Project

name

Status

as at

Dec

2018

Desk

review

PRA

/

deep

dive

Net loan

(million

UA)

Source

of

funding

Category

Studies Studying Integration

Wind Power

Comp 0.45 MIC

Fund

Public

Improv. Oper. Eff.

Exist. Power Plants

Comp 0.42 MIC

Fund

Public

Sub total 4 4

Water and Agriculture 156.31 UA

Investment projects

Gabel Elasfar

Wastewater

Treatment Plant

Comp 4.40 ADB L Public

Sustainable Abu-

Rawash Wastewater

Treatment Project

Phase

Ongo

(0.25%) 107.53 ADB

+AGTF

Public

National Drainage

Programme (NDP)

Ongo

(13.19%)

41.49 ADB L Public

Studies+

Monitoring Of

Water MDG In N.

Africa (MEWINA)

Clsd

1.70 AWF Public

Feasibility Study For

Integrated Sanitation

Technologies In

Some Rural Areas In

Upper Egypt

Ongo

(19%)

0.20 KOREA

TF

Public

Feasibility Studies

For The Zefta

Barrage

Comp

(100%) 0.59 MIC

Fund

Piblic

Technical

assistance

National Drainage

Technical Assistance

Ongo

(13.86%) 0.40 MIC

Fund

Grant

Public

Sub total 6 1

Multi sector-Governance 1099.36 UA

Program Based operations

Economic

Governance And

Energy Support

Program (EGESP)


Economic

Governance and

Energy Support

Program II


Economic

Governance and




Type of

operations

Sector/Project

name

Status

as at

Dec

2018

Desk

review

PRA

/

deep

dive

Net loan

(million

UA)

Source

of

funding

Category

Energy Support

Program III

Line of credit and Technical assistance

Egypt Franchising Sector Support Program

Comp 21

ADB

Loan

Public

Technical assistance

Capacity Building for

Egyptian Agency Of

Partnership for Dev.

Comp

(95,44%) 1.2 MIC

Fund

Grant

Public

Studies

Strengthening the

Rule of Law:

Effective and

transparent delivery

of justice and rule-

making

Ongo 1.78 MENA Public

Sub total 5 1

Social and Finances 56.6 UA

Study Update Of Actuarial

Study For National

Health Insurance

Comp

(97,40%)

0.18 MIC

Fund

Grant

Public

Technical assistance

Green Growth:

Industrial Waste

Management and

SME

entrepreneurship

Hub

Ongo 1.42 MENA Public

Support to Micro,

Small & Medium

Enterprises in

Organic Clusters

Project

Clsd 3.50 MENA Public

Investment & Technical assistance

Rural Income And

Economic

Enhancement

Project (RIEEP)

Comp

(100%) 51.50 ADB+

MIC

TAF+

FAPA

Public

Sub total 3 1

Total 18 7



3.4 Portfolio Review

The portfolio review (see Annex 7) will assist the evaluation in better understanding the Bank’s operations in the country over the last ten years. It will also contribute to providing a response to some key EQs, especially those relating to compliance with the Bank’s operational rules and the extent to which the interventions have achieved their expected results as outlined in the CSPs. This exercise will cover the overall portfolio and will be complementary with other lines of evidence on which it can be built, including the Cluster analyses presented in section 4.2. The content of the portfolio review within the CSPE should mainly address:

portfolio overview: present the trends and evolution of the portfolio in the country;

quality of entry: examine the performance of the portfolio in terms of time-lines and quality of project design;

quality of implementation: analyse the performance of the portfolio during the implementation of the projects/programme;

quality at exit: assess the extent of achievements of the development objectives of the project/programme in the portfolio.

3.5 Evaluation matrix

The ToRs define a set of key issues and criteria that frame the evaluation. Key issues include the following:

Development Results: this set of issues will focus on what the Bank has achieved on the ground, in particular assessing the extent to which the AfDB contributed to development results, and will examine relevance, effectiveness and sustainability of the Bank’s operations. Results in terms of evolution of crosscutting issues will also be analysed.

Managing the Bank’s intervention: in terms of assessing management of the Bank’s engagement, the evaluation will focus on design (including coherence) and efficiency, as well as knowledge work, policy dialogue with the country, partnerships, harmonisation and leverage, and the degree of management for development results.

Borrower’s performance: the borrower’s (Government’s) performance is analysed as a contextual element which may both support or complicate the Bank's action; the contribution of project executing agencies, sector ministries and other actors responsible for project/programme design and implementation will be assessed.

Success factors and lessons: This issue is one of the main objectives of the evaluation. It examines and identifies the key facilitating or constraining factors for achieving development results - i.e. drivers of success - and provide lessons for the future.

The ToR associate the key criteria and issues presented above with 25 Evaluation Questions (EQs) to be addressed in this evaluation.



Table 4 – Evaluation Questions



Identification questions, criteria, indicators and sources of information constitute the evaluation framework, i.e. the Evaluation Matrix (See Annexes).

3.6 Data collection tools

The following data collection and analysis methods will be used:

Literature and documentary review: to ensure that the evaluation takes into account the relevant contextual elements, an analysis of the country context will be carried out, based on collected documentation. Analyses published by international partners will also be reviewed. The contextual analysis is essential for addressing the criterion of relevance. Such an analysis will enable the evaluator to structure a timeline with key milestones of the cooperation and country policies during the period. The types of documents to be included in the documentary review are as follows:

(i) Government of Egypt’s national and sector strategies; (ii) documents from development partners; (iii) Bank’s strategy and policy documents; (iv) documents relating to the Bank's strategies and interventions specifically targeted on Egypt; and (v) project documentation. The Annexes volume includes a list of the available information for each of the 53 projects and highlights the missing information for each project according to its level of implementation. It is important to mention that no information was provided for projects 16, 18, 24, 40, 43, 51 and 52.

Interviews / Focus group discussions (FGD): The evaluation team will conduct interviews with representatives of different stakeholders. These interviews will be semi-structured and adapted according to the interviewees and the missing information. Where useful, group interviews, i.e. focus group discussions, will also be conducted. Also, and most importantly, open-ended questions will be conducted so that the interviewee will not be limited only to responding to questions determined by the evaluation team, but also feel free to express concerns and issues that are found of relevance to the person in question. Main stakeholders for interviews include: Strategic interlocutors: (i) Government of Egypt authorities; (ii) AfDB Task Managers/ Staff (Country Divisions), and (iii) Development partners - Technical and financial partners.

Operational interlocutors: (i) Executing and Implementing Agencies; (ii) Civil Society (i.e. NGOs, private sector actors); and (iii) Beneficiary groups.

Project results assessments (PRAs/in-depth assessments) / Field visits: in addition to the interviews that will take place at strategic level, the field mission will also make it possible to visit targeted interventions (see Annexes for the listed interventions) and thus make a first cross-comparison of the information collected during the documentary phase with the observed realities on the ground. Site visits will be selected and organized in consultation with the main stakeholders, particularly MIIC, in order to better cover the information needs of the evaluation. A plan will be developed and be included in the final inception report. The



Annexes volume includes the geographical distribution of the projects subject to a PRA / in-depth treatment.

A debriefing workshop: At the end of the field mission the evaluation team plans to organize a workshop in Cairo with the Bank’s reference group and a government representative. The workshop will focus on the following:

report on the evaluation work completed at this stage (end of field mission);

presentation and discussion of key information collected and initial draft EQ responses;

At this stage thet objective is to open a discussion with the stakeholders and potential users of the evaluation so that they can actively participate in the evaluation exercise..

3.7 Reporting and dissemination phase

The collection of information and filling in the evaluation framework will be completed with the integration of the results of the field phase and the processing of the data collected from observations and interviews. This consolidated framework will be the starting point for the preparation of the three sector summary notes and the technical report. During the synthesis phase the information gathered during the field phase will be used to enrich, balance and amend the data gathered during the documentary phase so as to allow formulation of the final answers to the EQs. On this basis the conclusions and recommendations of the study will be drafted. The conclusions and recommendations will emerge from exchanges within the team conducted during the field phase. This will benefit both from a multidisciplinary approach and from a comparison of sector analyses and the perspectives of different evaluators (local, international). This phase will culminate in the drafting of a draft technical report to be presented to the Bank for discussion in accordance with the proposed outline included as Annex. Based on these exchanges a final version will be prepared. Once this final comprehensive technical report is approved, a summary report will be prepared. As with previous reports based on exchanges and comments received, a final version (with executive summary) will be submitted to the Bank. In addition, the final versions of the three sector summary notes will be presented in a standard format provided by BDEV (see Annexes volume). As per BDEV practices, the evaluation will hold a capitalization workshop to discuss the key recommendations of the evaluation with national stakeholders and Bank staff. Most importantly, the workshop will be carried out in close collaboration with the outcome of the parallel evaluation performed by the M&E department in MIIC. A proposed plan for dissemination is presented in the annexes.



4. Roles and Responsibilities

The organization of the team includes a clear distribution of responsibilities and tasks, which is summarized in the Figure below. It enables to cover all sectors and themes as well as crosscutting issues. All members of the evaluation team are exempt from any conflict of interest for conducting this CSPE.

Figure 5 – Evaluation team organizational structure1

1 To be updated based on changes in the team composition

ADE

Preliminary Inception Report – July 2019 Page 29

5. Evaluation calendar

The main elements of the calendar are presented in the table below and in detail in the flow chart.

Note that the proposed agenda takes into account the time needed for internal and external reviews and revision by the evaluation team. The duration of each activity and deliverable conforms to the provisional calendar set out in the Terms of Reference but deadlines have shifted owing to the delays for the inception missions.

Table 5 – Evaluation Calendar

Activities Starting date End date

Inception phase February June 15th

1st Scoping mission 21 April 24 April

Preliminary Inception report 22 May 31 May

2nd Scoping mission 10 June 17 June

Final Draft Inception report Mid-June June

Data collection and analysis phase June 17th July - August

Data collection mission 17 June 4 July

Reporting and Dissemination phase July September

Sector notes draft TBC TBC

Sector notes final TBC TBC

Technical report draft 12 August 24 August

Technical report final 28 August 14 September

Summary report draft 23 September 27 September

Summary report final 7 October 17 October

ADE

Preliminary Inception Report – July 2019 Page 30

4 11 18 25 4 11 18 25 1 8 15 22 29 6 13 20 27 3 10 17 24 1 8 15 22 29 5 12 19 26 2 9 16 23 30 7 14

Team meeting and information session (teleconference)

Literature and documentary review; portfolio analysis

Development of the Evaluation framework (EQs)

Context analysis and Theory of Change

Portfolio review and analysis

Development of data collection, analysis, synthesis and coding tools

First Scoping mission in Egypt

D1 Preliminary Inception report &

Comments on the inception report

Second scoping mission

D1 Inception report-Final &

Approval of the inception report

In-depth document and literature review (desk work) - including project

portfolio review

Field mission including data collection at strategic and intervention level,

interviews of key informants and beneficiaries, PRAs, field visit, focus groups,

briefing and debriefing IDEV and team meeting

Emerging findings workshop (To be confirmed with Egypt Country Office)

Data analysis by EQs

D2 Technical report - Draft &

Comments on the Technical report

D2 Technical report - Final &

Approval of the Technical report

Presentation and workshop with evaluation reference group

D3 Summary Report (Draft) - IDEV leads with consultant's support &

Comments on the Summary report

D3 Summary report - Final &

Approval of the synthesis report

Phases and activities

August September OctoberFebruary March April May JulyJune

INCEPTION PHASE

DATA COLLECTION AND ANALYSIS PHASE

REPORTING PHASE

Documents

Independent Evaluation of the African Development Bank’s ... · Bank (hereafter referred to as the Bank) is undertaking an evaluation of the Bank’s strategy and programmes in