Angola -Market Oriented Smallholder Agriculture Project ...€¦ · Web viewGovernment counterpart funds, which were originally anticipated at US$4.4 million at appraisal were US$2.1

Document ofThe World Bank

Report No: ICR00003886

IMPLEMENTATION COMPLETION AND RESULTS REPORT(COFN-C1270, IDA-45020 TF-57303 TF-57845)

ON A

IDA CREDIT

IN THE AMOUNT OF SDR 13.0 MILLION(US$20.0 MILLION EQUIVALENT)

TO THE

REPUBLIC OF ANGOLA

FOR A

MARKET ORIENTED SMALLHOLDER AGRICULTURE PROJECT

September 28, 2016

Agriculture Global PracticeAfrica Region

CURRENCY EQUIVALENTS

Exchange Rate Effective March 31, 2016

Currency Unit = Angolan Kwanza (AOA)AOA 160.54 = US$1

FISCAL YEARJanuary 1 – December 31

ABBREVIATIONS AND ACRONYMS

ADI Agriculture Development InstituteADP Agricultural Development ProgramCDD Community Driven DevelopmentCMC Community Multimedia CenterCSA Climate Smart AgricultureDPA Direção Provincial da Agricultura (Provincial Directorate of

Agriculture)EDA Estação de Desenvolvimento Agrário (Agricultural Development

Office of ADI at the Municipal Level)EMRP Emergency Multisector Recovery ProjectESMF Environmental and Social Management FrameworkESMP Environmental and Social Management PlanFAO Food and Agriculture Organization of the United NationsFFS Farmers’ Field SchoolFM Financial ManagementFMR Financial Management ReportGoA Government of AngolaGDP Gross Domestic ProductGEPE Gabinete de Estatística Planificação e Estudos (Division of

Statistics Planning and Studies)ICR Implementation Completion and Results ReportIFAD International Fund for Agricultural DevelopmentIPM Integrated Pest ManagementLICUS Low Income Country Under StressM&E Monitoring and EvaluationMINAGRI Ministry of AgricultureMIS Monitoring Information SystemMOSAP Market Oriented Smallholder Agriculture ProjectNDP National Development PlanNGO Nongovernmental OrganizationO&M Operations and MaintenancePAD Project Appraisal Document

PAPAGRO Programa de Adquisição de Produtos Agropecuários (Program for Acquisition of Agriculture Products)

PDO Project Development ObjectivePHRD Japan Policy and Human Resource Development FundPISC Project Implementation Sub-CommitteePIU Project Implementation UnitPPISC Provincial Project Implementation Sub-CommitteePPIU Provincial Project Implementation UnitPRSP Poverty Reduction Strategy PaperSADCP Smallholder Agriculture Development and Commercialization

ProjectSOE Statements of ExpenditureTOR Terms of ReferenceTTL Task Team LeaderUN United NationsUNITA União Nacional para a Independência Total de Angola (National

Union for the Total Independence of Angola)

Senior Global Practice Director: Juergen VoegelePractice Manager: Dina Umali-Deininger

Project Team Leader: Aniceto BilaICR Team Leader: Melissa Brown

REPUBLIC OF ANGOLA

Market Oriented Smallholder Agriculture ProjectContents

A. Basic Information...........................................................................................................vB. Key Dates........................................................................................................................vC. Ratings Summary............................................................................................................vD. Sector and Theme Codes...............................................................................................viE. Bank Staff.....................................................................................................................viiF. Results Framework Analysis.........................................................................................viiG. Ratings of Project Performance in ISRs.........................................................................xH. Restructuring (if any).....................................................................................................xiI. Disbursement Profile.....................................................................................................xii

1. Project Context, Development Objectives and Design....................................................12. Key Factors Affecting Implementation and Outcomes...................................................73. Assessment of Outcomes...............................................................................................124. Assessment of Risk to Development Outcome.............................................................225. Assessment of Bank and Borrower Performance..........................................................236. Lessons Learned............................................................................................................257. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...............26

Annex 1. Project Costs and Financing...............................................................................28Annex 2. Outputs by Component......................................................................................31Annex 3. Economic and Financial Analysis......................................................................41Annex 4. Bank Lending and Implementation Support/Supervision Processes.................46Annex 5. Beneficiary Survey Results................................................................................47Annex 6. Stakeholder Workshop Report and Results.......................................................56Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR..........................59Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders............................60Annex 9. List of Supporting Documents...........................................................................65MAP...................................................................................................................................66

A. Basic Information

Country: Angola Project Name:AO-Market Oriented Smallholder Agriculture Project

Project ID: P093699 L/C/TF Number(s):COFN-C1270,IDA-45020,TF-57303,TF-57845

ICR Date: 09/16/2016 ICR Type: Core ICR

Lending Instrument: SIL Borrower:REPUBLIC OF ANGOLA

Original Total Commitment:

XDR 18.50M Disbursed Amount: XDR 12.95M

Revised Amount: XDR 13.00MEnvironmental Category: BImplementing Agencies: Ministry of Agriculture Cofinanciers and Other External Partners: IFAD

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 11/28/2005 Effectiveness: 06/30/2010 09/20/2010

Appraisal: 05/06/2008 Restructuring(s):

02/15/201309/15/201412/15/201503/15/2016

Approval: 07/31/2008 Mid-term Review: 02/25/2013 Closing: 09/30/2014 03/31/2016

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately

v

Unsatisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

YesQuality at Entry (QEA):

None

Problem Project at any time (Yes/No):

YesQuality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing) Agricultural extension and research 9 9 General agriculture, fishing and forestry sector 60 60 General water, sanitation and flood protection sector 8 8 Irrigation and drainage 9 9 Public administration- Agriculture, fishing and forestry 14 14

Theme Code (as % of total Bank financing) Rural markets 50 50 Rural policies and institutions 10 10 Rural services and infrastructure 40 40

vi

E. Bank Staff Positions At ICR At Approval

Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Elisabeth Huybens Michael Baxter Practice Manager/Manager:

Dina Umali-Deininger Karen Mcconnell Brooks

Project Team Leader: Aniceto Timoteo Bila Aniceto Timoteo Bila ICR Team Leader: Melissa Brown ICR Primary Author: Melissa Brown

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)The project development objective is to increase agricultural production through provision of better services and investment support to rural smallholders in selected comunas and municipios of the Recipient's provinces of Bie, Huambo and Malange.

Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of agricultural production increase based on crop production index of participating smallholder farmers

Value quantitative or Qualitative)

100 (base year) 25% 10% 66%

Date achieved 12/31/2011 09/30/2014 09/30/2014 03/31/2016

Comments (incl. % achievement)

This indicator attained 560%. Project restructuring in February 2013 revised targets. The crop index provides an aggregated measure of the increase in crop production (volume). The baseline year was 2011/12 and the crop index was measured annually.

Indicator 2 : Number of clients who have adopted and improved agricultural technology promoted by the project.

Value quantitative or

0% 20,000 32,300

vii



approval documents)

Formally Revised Target Values



Qualitative) Date achieved 12/31/2011 09/30/2014 03/31/2016


Target exceeded by 155%. The project was set to reach a rate of at least 40 percent (20,000 farmers). The project reaching 32,300 farmers (61.5%).

Indicator 3 : Number of clients who adopted an improved agricultural technology promoted by project – female.

Value quantitative or Qualitative)

0 10,000 13,621

Date achieved 09/30/2014 03/31/2016


The number female farmers that adopted technology was determined based on sample 461 farmers that adopted technology and determined that 42.7 percent were female and 57.3 percent were male.

(b) Intermediate Outcome Indicator(s)



approval documents)

Formally Revised

Target Values



Indicator 1 : Number of smallholder farmers that benefitted from training.

Value (quantitative or Qualitative)

0 126,000 50,000 54,982

Date achieved 12/31/2011 09/30/2014 09/30/2014 03/31/2016Comments (incl. % achievement)

Number of farmers trained was combined at project Restructuring in February 2013. The adjusted target number was fully achieved.

Indicator 2 : Participating smallholder farmers in the Project areas who belong to farm organizations.


20% 50% 100%

Date achieved 12/31/2011 09/30/2014 03/31/2016Comments (incl. % achievement)

Fully achieved - all project beneficiaries are members of associations.

viii



approval documents)

Formally Revised

Target Values



Indicator 3 :

Number of smallholder farmers in associations that benefit from grants and completed their activities under the Project’s agricultural investment component.


0 7,200 8,000 12,344

Date achieved 12/31/2011 09/30/2014 09/30/2014 03/31/2016Comments (incl. % achievement)

The indicator attained 154%. This indicator was downgraded from outcome indicator to intermediate indicator in Feb. 2013 restructuring.

Indicator 4 : Ratio of smallholder farmers’ organizations to an extension officer.


8 associations/extension officer

12 12 associations/extension officer

Date achieved 12/31/2011 09/30/2014 03/31/2016


The indicator attained 100%. The policy of the ADI is to have 3 technicians per commune. At the project commencement, in 2011/2012, there were 61 technicians in the project target area. Currently, there are 91, amounting to an increase of 50%.

Indicator 5 : Increase in average yield of major crops (maize, cassava, beans and potato) of participating smallholder farmers. - MAIZE


0.4 t/ha 0.6 t/ha 0.66


Target exceeded by 10%. Final value measured in impact evaluation survey.

Indicator 6 : Increase in average yield of major crops (maize, cassava, beans and potato) of participating smallholder farmers. – CASSAVA.


10.0 t/ha 11.0 t/ha 18.1



ix



approval documents)

Formally Revised

Target Values



Indicator 7 : Increase in average yield of major crops (maize, cassava, beans and potato) of participating smallholder farmers. – BEANS.


0.3 t/ha 0.4 t/ha 0.45 t/ha



Indicator 8 : Increase in average yield of major crops (maize, cassava, beans and potato) of participating smallholder farmers. – POTATO.


4.0 t/ha 5.0 t/ha 7.1 t/ha



G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements(USD millions)

1 11/30/2008 Satisfactory Moderately Satisfactory 0.00 2 05/29/2009 Unsatisfactory Unsatisfactory 0.00 3 12/10/2009 Unsatisfactory Unsatisfactory 0.00

4 05/21/2010 Unsatisfactory Moderately Unsatisfactory 0.00

5 02/08/2011 Unsatisfactory Unsatisfactory 0.30

6 07/07/2011 Moderately Unsatisfactory

Moderately Unsatisfactory 0.30





9 01/11/2013 Moderately Unsatisfactory Moderately Satisfactory 4.23

10 07/09/2013 Moderately Satisfactory Moderately Satisfactory 6.31 11 04/06/2014 Satisfactory Satisfactory 13.26 12 11/01/2014 Satisfactory Satisfactory 15.68

x

13 05/12/2015 Satisfactory Satisfactory 17.03 14 11/06/2015 Satisfactory Satisfactory 18.72 15 03/30/2016 Satisfactory Moderately Satisfactory 19.23

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes MadeDO IP

02/15/2013 N MU MS 4.66

(i) partial cancellation of the IDA Credit in the amount of US$10 million; (ii) adjust results indicators and targets; (iii) reallocated Credit proceeds; and (iv) increase IDA credit percentages of disbursements made under Category 4

09/15/2014 S S 14.44

Extension of closing date September 30, 2014 to December 31, 2015

12/15/2015 S S 18.93

Extension of closing date from December 31, 2015 to March 31, 2016

03/15/2016 S S 19.23

Amendment of IFAD credit closing date from June 30, 2016 to March 31, 2016

xi

I. Disbursement Profile

xii

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. At the time of appraisal Angola was considered a Low Income Country Under Stress (LICUS), largely because of the after-effects of the 40 year civil war following independence from Portugal in 1975. The civil war, which only ended in 2002, resulted in the death of 750,000 Angolans, displacement of about 4.5 million, and 450,000 refugees. The country’s physical and social infrastructure was devastated and Angola had some of the world’s worst human development indicators, weak governance, and fragile human and institutional capacity.

2. At the same time, Angola was (and remains) one of Africa’s most resource-rich countries, with tremendous economic potential, endowed with mineral resources and large stock high potential agricultural land. Prior to independence, agricultural and food production was high and the country was a major exporter of maize and coffee. Smallholders in the Central Highlands, the agricultural heartland of Angola, were firmly embedded in the market economy and were the main producers for both local and export markets. During the war agriculture fell to an almost subsistence level in most areas.

3. The Project was designed to respond to objectives of Angola’s Poverty Reduction Strategy (PRSP), which highlighted rural development with a focus on the improvement of food security and the re-vitalization of the rural economy. The PRSP called for: (i) the strengthening of the production capacity of the traditional sector, particularly food crops and fisheries; (ii) the re-launching of rural commerce; (iii) a sustainable development of natural resources; and (iv) the reorganization of the legal framework and public institutions. Basic underlying principles outlined in the PRSP were a focus on smallholders; the importance of community participation; the concentration of planning, implementation and monitoring at the municipality level; complementary activities with donors, the private sector and NGOs; specific targeting of women, including access to land; and HIV/AIDS as a cross-cutting concern.

4. Both the World Bank’s Interim Strategy for 2007-2009 and International Fund for Agricultural Development (IFAD) Country Strategy Update of 2008/09 identified the need for interventions to restore productivity and revive economic activity in rural areas. IFAD’s strategy focused on two pillars: reduction of rural poverty, and creation of a conducive environment for private sector development. The World Bank’s Interim Strategy focused on three main areas: (i) strengthening public sector management and institutional capacity; (ii) rebuilding critical infrastructure and support delivery of public services for poverty reduction; and (iii) promoting growth of non-mineral sectors.

1

5. The rationale for Bank and IFAD involvement in the project was based on a recognition that that improving agricultural productivity and competitiveness would help to both reduce rural poverty and promote economic growth. The goal of the Market Oriented Smallholder Agriculture Program (MOSAP) was to enable smallholders to increase their productivity and competitiveness and to benefit from improved access to market by focusing on the establishment of rural-urban commercial market linkages. MOSAP was also seen as a way to build on work started under the Bank’s Emergency Multisector Recovery Project (EMRP), which was designed to help reestablish institutional capacity of Angola’s two agriculture research centers and rural extension services in Malanje and Bie and in phase two was to invest in feeder road rehabilitation in the MOSAP areas.

1.2 Original Project Development Objectives (PDO) and Key Indicators

6. The project development objective was to increase agricultural production through provision of better services and investment support to rural smallholders in selected comunas and municipios of the Recipient’s provinces of Bie, Huambo and Malange.

7. The original key indicators selected to measure the progress towards achieving the PDO were: (i) increased agricultural production of participating smallholder farmers based on crop index; (ii) improved smallholder access to market through contractual arrangements with agribusinesses or traders; (iii) improved smallholder vulnerable groups access to market with secured access to market through contractual arrangements with agribusinesses or traders; and (iv) communities and/or associations that benefited from grants and completed their sub-projects under agricultural investment support component.

1.3 Revised PDO and Key Indicators

8. The PDO was not revised during implementation, however, project indicators were revised during a February 2013 restructuring, which also included a partial cancellation to the Credit. Three PDO level indicators and two intermediate indicators were dropped based on the need to strengthen the focus on the PDO which was more narrowly focused on the improvement of agriculture production. The following PDO indicators were dropped:

(i) Percentage of participating smallholder farmers with secured market access through contractual arrangements with agribusinesses or traders;

(ii) Percentage of participating smallholder vulnerable groups with secured access to market through contractual arrangements with agribusinesses or traders;

(iii) Percentage of communities and/or associations that benefited from grants and completed their subprojects under agricultural investment support component.

9. The first and second indicator were dropped as they were judged not relevant to the PDO and the third was changed to intermediary indicator. The restructuring also introduced a new core PDO indicator:

2

(i) Target clients who have adopted and improved agricultural technology promoted by the project (disaggregated by gender).

1.4 Main Beneficiaries

10. Primary beneficiaries. The primary target group for MOSAP were smallholder farmers who would benefit from technical assistance to form or strengthen smallholder groups and associations, manage productive agricultural investment activities funded through the second component, and improve agricultural and marketing skills. The project originally intended to target 126,000 farmers to be direct project beneficiaries in 25 comunas within 12 municipios located in Bie, Huambo and Malanje. Each farmer was assumed to represent a single household of five members. Project interventions were designed to target smallholder farmers who cultivated between 1 and 2 hectares of land under rain-fed conditions with the potential for expanding the size of their holdings to 2.5 hectares. Following restructuring the target number of direct beneficiaries was scaled down to 50,000 farmers.

11. The three provinces included in the project area were heavily affected by the civil war. The project area included the past headquarters of the União Nacional para a Independência Total de Angola (UNITA) movement and portions of the population had been displaced. 12. Indirect beneficiaries. The Project was also designed to generate benefits for government institutions and service providers, particularly at the decentralized level. Government institutions including the Ministry of Agriculture and its Agricultural Development Institute (ADI), as well as local municipalities received training in introducing or strengthening participatory processes that support project implementation, focusing on building skills of staff in institutions involved in the project, and improvement of office space and accommodations for extension officers. For private and non-governmental agricultural service providers, benefits included: (a) strengthening the capacities of private sector agencies, NGOs and CBOs to support smallholder groups and associations to prepare development plans and investment proposals (sub-projects) for funding under component two; and (b) strengthening the managerial and business capacities of local micro-enterprises that support farming activities (small processors, craftsmen and local traders).

1.5 Original Components

Component 1: Capacity Building (Total Financing: US$11.26 million; of which IDA: US$3.42 million; IFAD: US$3.2 million; PHRD Grant: US$4.0 million and GoA: US$0.64 million)

13. The objective of the first component was to strengthen the technical, institutional, managerial and marketing skills of smallholders and their organizations, as well as of services providers and other stakeholders involved in agricultural production and value chains, to more effectively operate in a market-driven environment and to prepare for the

3

agricultural investment support opportunities under component 2. This was to include investments in the following three areas: (i) building and strengthening capacities of smallholder groups and associations through the provision of training and information; (ii) capacity building support in the form of information, training, and hardware investments (office, equipment, transport) for to IDA offices at central and local levels as well as local government; and (iii) strengthening the capacities of private and non-governmental agricultural service providers through the provision of training.

14. Expected results under this component were: (i) improved technical skills of beneficiary stakeholders involved in agricultural production; (ii) improved participation of smallholder farmers in farm organizations; (iii) improved community capacity to prepare agricultural investments support opportunities; and (iv) improved government capacity to support smallholder farmers in the project area. The component was also designed to take a gender and vulnerable group sensitive approach to capacity building.

Component 2: Agricultural Investment Support (Total Financing: US$29.34 million; of which IDA: US$23.46 million; IFAD: US$2.85 million; and Beneficiaries: US$3.03 million) 15. The objective of the second component was to improve the productivity, competitiveness and market access of 24,000 smallholders in the project area (later revised to 8,000 with partial cancellation and increased focus on service delivery). This component was designed to provide demand-based support, in the form of matching grants, to rural communities and smallholders groups, for small-scale agricultural infrastructure, production, processing and marketing sub-projects. The design at appraisal distinguished between three different types of sub-project investments: (i) public goods or community assets such as small-scale irrigation, drinking water, dip-tanks, stores, spot improvement of feeder roads; (ii) productive assets that had an element of multiplication (seed, livestock) within communities or groups; and (iii) productive investments that were primarily capital investments in items such as tractors, animal traction equipment (ploughs), sprayers, pumps, grain mills, oil presses, milk chillers, and transport (ox-cart, trailer).

16. The sub-projects were to be screened for technical, financial, social and environmental feasibility, before approved for funding. Matching grant ratios and funding ceilings initially required 10 percent beneficiary contribution. Rural communities/villages were eligible to receive support, provided that the sub-project proposals result from a community-level participatory planning exercise with a cross section of the population, including women and youth. Smallholders groups or associations were also be eligible for support, provided that they were either registered or informally recognized by the local community and authorities, that they had a democratic structure and have the ability to keep records.

17. Expenditures under this component were subject to a disbursement condition that stipulated that subprojects would only be financed once a number of performance triggers were met, including: (i) at least 30 qualified service providers and public agents were

4

certified and equipped under the project’s Capacity Building program; (ii) the baseline survey had been completed; and (iii) the administrative capacity of Provincial Project Implementation Units were in compliance with Bank fiduciary requirements.

Component 3: Project Management (Total Financing: US$8.75 million; of which IDA: US$3.12 million; IFAD: US$2.15 million; and GoA: US$3.48 million)

18. The third component provided financing for project management and M&E activities. The Ministry of Agriculture (MINAGRI) was responsible for project implementation and had delegated ADI to be accountable for the management and oversight of the project. At the specific request of MINAGRI, a Project Implementation Unit (PIU) was established within the ADI framework and the Director General of ADI to be the executive level public manager of the project. The PIU included a Project Coordinator, a financial manager, a procurement specialist, communication specialist and an (ADI-staff) accountant, as well as a monitoring and evaluation officer. At the provincial level, MINAGRI was to establish three Provincial Project Implementation Units (PPIUs) within ADI, to conduct the operational management of the project. Each PPIU was to include a Provincial Project Coordinator, a financial accountant, a procurement office, communication specialist and a monitoring and evaluation officer. A team of three technical staff, including an agronomist, an agribusiness specialist and a rural infrastructure specialist, was to be based at provincial level (one based in each province), and was to be supplemented by short term consultants according to needs.

1.6 Revised Components

19. Reallocation among components and reduction in the scale of component 2. The fundamental design and the implementation arrangements for the project components did not change during the implementation period, however adjustment was made to the scale of the activities to accommodate the partial cancellation of the credit and reallocation of funding toward greater investment in capacity building and project management. Reallocation of additional resources to component 1 and 3 was significant – 143 percent and 154 percent of appraisal estimates – and were driven in part by higher than expected costs associated with service provision and project coordination and management in the Angolan context. The consumer price index in Angola was 161 by the end of 2015 (2010 = 100) –a 61 percent increase over a five year period. In contrast, appraisal cost estimates allocated only 5.6 percent for additional price and physical contingencies over the project life.

5

Components Appraisal(USD millions)

Actual(USD millions)

Percentage of Appraisal

Component 1: Capacity Building 11.26 13.70 143Component 2: Agricultural Investment Support

26.31 7.74 26

Component 3: Project Management

8.75 11.61 154

Total Baseline Cost 46.32 33.05 67

20. Another factor in reallocation occurred due to delays in starting the agricultural investment support activities under component 2, which were only initiated in 2013 and left limited time for implementation. The reduction in scale in component 2 resulted in a revision to the target number of beneficiaries and productive investment sub-projects. The original targeted of 126,000 farmers at appraisal stage was reduced to 50,000 to reflect this change.

1.7 Other significant changes

21. Extension of closing date. The closing date was extended for a total of 18 months beyond the original closing date.

22. Summary of restructuring. Changes to the project were captured in four Level Two restructurings:

(i) The first restructuring took place in February 2013 with the objective to partially cancel the IDA Credit in the amount of US$10 million in line with a requested by the Government of Angola on December 31, 2012, to match the project’s implementation capacity more realistically with the time available to complete Project activities prior to the Closing Date of September 30, 2014.

(ii) The second restructuring took place in September 2014 and the main objective to extend the project Closing Date for a period of 15 months, from September 30, 2014 to December 31, 2015.

(iii) A third restructuring was processed in December 2015 to extend the project closing date for a period of three months, from December 31, 2015 to March 31, 2016.

(iv) In March 2016 the project was restructured to advance the Closing Date of IFAD co-financing Loan Agreement from June 30, 2016 to March 31, 2016 and to match the IDA and PHRD Co-financing closing dates.

6

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

23. The project preparation and design were affected by the larger country context, which led to delays in approval and impacted availability of technical staff. While the initial identification mission took place in 2005, Board approval occurred only in mid-2008. The project had a relatively long preparation period prior to approval due to a number of factors. Angola’s country reclassification and shift away from concessional borrowing terms in 2007 led to a review of all proposed lending in the country by the Ministry of Finance. This led to delays in the final negotiation of the credit for MOSAP on hard IDA terms. The uncertainty around terms of the Credit also led to a halt in the processing of project preparation funds, which had been proposed to initiate key preparatory activities and finance the initial core set of coordination and fiduciary staff of the PIU. In addition, legislative elections, which were held in September 2008 and represented the first elections since 1992 and the outbreak of the second phase of the civil war, also affected availability of key Government counterparts during final stages of preparation and declaration of effectiveness.

24. Design drew on lessons learned within the country and in other Community Driven Development (CDD) and smallholder agricultural development projects. The design of the project took into account lessons learned from the implementation of the Transitional Support Strategy from 2003-2005 and the first Interim Strategy from 2005-2006, as well as from IFAD’s experience in implementing projects in the rural sector from 1991-2007. The core of project design centered around the need to re-capitalize smallholder producers and re-activate market linkages to support broad-scale and sustainable agricultural production among smallholder farmers who comprised the majority of the population. Following the end of hostilities many returnees in the proposed project area were engaged in farming but had little access to services, markets or improved technologies. At the time of appraisal it was estimated the agriculture sector had re-absorbed 4.4 million people following the end of the civil war. The project’s design focused on technical training, producer organizational development and community driven development modalities in order to address the need for increased productivity and expansion of cultivated land.

25. Project design relied up on earlier background analyses which demonstrated that continued economic growth for Angola, as the country emerged from post-conflict period, would have to be based on supporting a diversified economy in which agriculture could play a small but important role. Other projects supporting smallholder production emphasized the need to support the development market linkages and coordinated value chain interventions.

26. Implementation capacity was considered a major risk to the project and efforts were made to reduce project complexity and address capacity constraints. Although Government ownership for the Project was considered to be strong particularly within MINAGRI, implementation experience from previous Bank and IFAD projects

7

indicated that implementation capacity was extremely weak and that substantial technical support would be required through technical assistance and supervision. Implementation capacity risk was rated substantial. At the start of preparation, extensive discussion took place around the capacity within MINAGRI and ADI at central, provincial and municipal level to implement project activities. Design was adjusted to simplify and reduce the number of sub-components and institutional arrangements put in place to bring in technical and fiduciary staff and contract service providers to provide key technical services.

2.2.1 Implementation

27. Delays in effectiveness and implementation led to a slow start. The project was declared effective 26 months after Board approval in September 2010. At the time, the average time for project to become effective in Angola was 19 months. Initially delays were caused by the length of time required to approve the financing agreement within the Council of Ministers, delays in the provision of the legal opinion, finalization of the project’s operations manual, and hiring of key fiduciary staff. Lack of implementation capacity in MINAGRI, compounded by weak leadership within the PIU contributed to slow progress, which was reflected in low ratings of implementation progress assigned by the supervision team beginning soon after Effectiveness (implementation progress was consistently rated Unsatisfactory or Moderately Unsatisfactory).

28. Implementation under the second component was also affected by delays in meeting disbursements conditions. This included establishing the baseline values for key performance indicators and finalizing the data collection needed to guide sub-project investments. The baseline survey was as a disbursement condition for the second component and delays occurred in launching the selection process for a specialized firm to conduct the planned surveys and undertake the socio-economic profiling of farmers’ organizations needed to design further investments. The baseline study was completed in January 2013, meaning disbursements under the second component took place two years after Effectiveness. Similarly, detailed procedures for funding sub-projects under Component 2, including criteria and procedures for the preparation, appraisal/approval, and implementation of each of the three financing windows under the Project were also disbursement conditions and were submitted for Bank clearances only in December 2012.

29. Availability of service providers was at times challenging but ultimately drove implementation under the Project. The project experienced challenges in mobilizing service providers due to initial contracting delays but also due to the supply and quality of available providers from civil society and the private sector. The number of qualified service providers able to work in Angola were not as numerous as anticipated. The challenge of the overall business environment in Angola, which was characterized by an extremely high cost of doing business and barriers to entry in terms of difficulty obtaining work visas – made it difficult to attract qualified international service providers.

8

30. Service providers were contracted for a range of activities including building and strengthening capacities of smallholder groups and associations; government institutions to support smallholder production and marketing; and private and non-governmental agriculture services providers. Service providers brought a range of skills and perspective that would have likely not been available within MINAGRI, particularly at the decentralized levels.

31. Although start-up was slow, effective project management turned around an underperforming project and implementation in the latter years was effective. Implementation improved significantly after the Government took action by appointing a new Project coordinator at the end of 2011. From that point onward, the project started to make progress, and as a result disbursements increased from US$0.3 million in February 2012 to about US$4.2 million by December 2012 and near total disbursement by project end. The change in project management was likely one of the more significant factors leading to a satisfactory rating and near full disbursement of the remaining Credit.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

32. Although initial M&E implementation was unsatisfactory due to the significant delays in commissioning the baseline assessment, the overall M&E system was sound. The project’s monitoring and evaluation systems were designed to be focused on data collection and reporting on key performance output and impact indicators, including targeted data collection, surveys, participatory assessments and mid-term and final evaluations. Relevant data was gender-disaggregated. Two evaluations of project output and impact indicators were planned at mid-term and at completion. The project was to finance M&E costs, including the mid-term review and project completion review.

33. Although finalization of the baseline assessment was later than expected the project focused on improving the M&E system and several activities were undertaken to ensure effective implementation in terms of M&E design and implementation arrangements. These included: (i) convening workshops with various stakeholders to build consensus and also included building capacity and common understanding of the M&E issues; and (ii) training of M&E provincial officers in Monitoring and Evaluation Systems: Design. On training workshops were held in each province for Estação de Desenvolvimento Agrário (EDAs- Agricultural Development Offices of ADI at the Municipal Level) field staff to: a) expose participants to the principles of participatory monitoring and evaluation; b) jointly design and develop participatory tools for internal monitoring and evaluation of the effectiveness of FFS as an approach for up scaling adoption of technologies; and c) impart participants with analytical and reporting skills for participatory measurement and evaluation. A total of 75 participants attended the workshop;

34. The relationship between project design and impact evaluation was seen as a good practice. The project contracted a service provider to carry out an impact assessment of the project that included establishing a baseline in 2012 and a follow up in

9

2015. The development of baseline data also was used to set project targets and included a review of the M&E framework and reformulation of targets to be achieved by the Project. Each year the PDO indicators were collected to monitor progress and adjust the actions when progress appears to be stalling. An independent assessment was done in 2015 to review progress on the Matching grants (subprojects) implementation; processes involved, track the performance, provide specific recommendations for SADCP). The methodology was robust (described in Annex 5) and efforts made to rigorously capture yield data and achieve a representative sample.

35. The M&E unit (national and provincial level) conducted routine monitoring and supervision of the execution of project activities, regularly reviewing implementation progress. Data collection efforts were carried out at municipal level by EDAs decentralized units and reported to the provincial level for consolidation.

36. The M&E unit helped to disseminate program results through bi-annual and annual conferences with stakeholders. The project also prepared success stories through newsletters, annual reports, brochures, press releases, and video.

2.4 Safeguard and Fiduciary Compliance

37. Safeguards. There were no major negative environmental social impacts and conflicts anticipated as a result of project implementation, however it was acknowledged that sub-projects may generate negative environmental or social impacts. The project triggered OP 4.01 Environmental Assessment and OP 4.12 Involuntary Resettlement. An Environmental and Social Management Framework (ESMF) was prepared in 2008 as part of the design of MOSAP and was updated in July 2012 to include several mitigation measures that had not been finalized in 2008.

38. The ESMF included a screening checklist which was subsequently included in the project’s Operational Manual for subprojects and which required an environmental and social screening assessment be conducted for each sub-project and that corrective or mitigating measures be identified. The screening checklist was used in implementation and applied by technicians of EDAs, with the support of municipality administrations. Approval of subprojects, both by municipal governments and the provincial subcommittees, generally followed established procedures and were based on verification that the subproject has no environmental impact, or in the case of adverse and significant impact, contained measures or actions to mitigate or prevent these impacts. Implementation of the plans were intermittently followed and not always reported on, however, and supervision missions noted there was a lack of technical capacity on environmental management.

39. Safeguard compliance was rated moderately satisfactory for most of the implementation period and no land acquisition and loss of livelihoods, either temporary or permanent has been recorded. The overall recommended Safeguards rating in ICR is also moderately satisfactory.

10

40. Financial Management. The Project was guided by a comprehensive financial management (FM) manual; and financial management report (FMR) based reporting was generally timely and accurate; though the project used statement of expenditures (SOEs) as a basis for disbursement. The financial management (FM) arrangements of the project including accounting, reporting, budgeting and staffing, internal controls and external audit were assessed to be adequate and acceptable to the Bank through the life of the Project. Such assessments were conducted on a regular basis during the FM implementation support missions. Project activities were periodically reviewed by the Internal Auditors. The audit reports were considered timely and were satisfactory in terms of scope.

41. Initial delays in the project led to an unsatisfactory rating which was upgraded to moderately satisfactory following improvement in fiduciary capacity. Delays in disbursements led to a moderately satisfactory rating for the remainder of implementation. The overall recommended FM rating in ICR is also moderately satisfactory.

42. Procurement. The Procurement activities of the Project were led by a team of five staff, two at PIU level and three at PPIU level, who followed the World Bank procurement procedures and the Project Procurement Implementation Manual. The procurement management, in general, which included the leading of procuring consultants, works and goods, reporting, and external audit were assessed to be adequate and acceptable to the Bank in the course of the Project implementation period. The assessments were conducted on a regular basis both by the Bank’s procurement staff, from both country and regional levels, during their support missions to the project. A key challenge noted by the project, both at PIU and PPIU levels, related to the limited knowledge of the World Bank procurement procedures among most of local providers of goods and services, which led to the need for greater oversight on the part of the PIU and PPIU procurement team.

43. Project procurement activities were periodically reviewed by the Bank’s procurement staff from both Country and regional level and an external procurement audit was carried once. Initial delays in the project led to an unsatisfactory rating which was upgraded to moderately satisfactory following improvement in fiduciary capacity. Delays in disbursements led to a moderately satisfactory rating for the remainder of implementation. The overall recommended procurement rating in ICR is also moderately satisfactory

11

2.5 Post-completion Operation/Next Phase

44. Despite its slow start, the project was perceived by Government and stakeholders as contributing significantly to agriculture capacity development, institution building and knowledge transfer in the three provinces where it operated. It became one of the more visible projects for agricultural development in the Central Highlands and at the request of the Angolan Government, the Bank prepared a follow up project - The Smallholder Agricultural Development and Commercialization Project (SADCP) – which was approved by the Board on July 5, 2016.

45. SADCP will scale up the geographic scale of MOSAP to reach additional beneficiaries in more municipios and comunas within the same three provinces: Bié, Huambo, and Malanje. Additionally, the project will include rehabilitation of 1,000 ha of smallscale gravity-fed irrigation schemes as a pilot program for smallholders. Building on lessons learned from MOSAP, the SADCP strongly emphasizes capacity building, institutional development, and sustainability by strengthening MINAGRI’s capacity for statistics, policy analysis, market information, irrigation development, and agricultural extension. Also building on lessons learned, the SADCP aims to mainstream environmental considerations and climate smart agriculture (CSA) practices into the project design through investments in more efficient use of water resources, promotion of soil conservation techniques, and integrated natural resource management.

46. At the request of the Government and based on MOSAP’s successes, IFAD has initiated the design of a Smallholder Agriculture and Marketing Access Project (SAMAP) - a MOSAP Follow-up Project. The SAMAP will be implemented in two neighboring provinces of the recently approved WB-SADCP. Discussions between the two institutions are pointing out towards harmonizing as much as possible SAMAP with SADCP and ensuing co-financing.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

47. The focus on smallholder agriculture was relevant to the country’s post-conflict food security, and poverty reduction priorities. During the civil war agricultural production declined significantly and the country relied on commercial food imports and food aid. In 2000/01 some 420,000 tons of cereals were imported on a commercial basis and an additional 330,000 tons were received as food aid. By 2005/06 commercial imports had risen to 780,000 tons and food aid had decreased to 60,000 tons. With large numbers of persons displaced, and many farming systems and agricultural value chains destroyed by the war, social stability in the project area was a key concern. Ensuring stability, a minimum level of food security, and better integration of rural based populations within the national economy were of critical long term political importance.

48. The project continued to be relevant to the country’s priorities and the objectives of agricultural commercialization and diversification of the economic

12

base. The project continued to be relevant to the country’s current development priorities around expanding and diversifying the export base, which remains dominated by the oil sector (which currently accounts for around 30 percent of Gross Domestic Product (GDP) and, 95 percent of total export value). Following a fall in the price of oil in 2014 and a significant drop in oil revenue the Government of Angola initiated a crisis plan that placed a strong emphasis on agriculture as an immediate way to increase domestic production and reduce imports.

49. Angola’s National Development Plan (NDP) 2013-17 and the Medium-Term Agricultural Development Program 2013-17 (ADP) also place a strong emphasis on agricultural productivity and diversification. The overall objective of the ADP is to promote the sustainable transformation of subsistence agriculture to a commercial orientation, with the aim of achieving food security in the country. Similarly the Bank’s Country Partnership Strategy for FY13–16 was aimed at: (a) deepening diversification for inclusive growth; (b) enhancing the quality of decentralization for services delivery; and (c) building human resources capacity.

50. A CDD approach was an effective way to ensure project beneficiaries had a say in determining project activities to meet their needs. Angola faces a number of challenges that impact social development and economic growth. The civil war resulted in a virtual collapse of market-oriented production and infrastructure suffered from widespread destruction of roads, bridges and warehouses. A community demand-driven (CDD) approach was seen as an important mechanism to support pro-poor agriculture programs for smallholders, and target income and employment generating activities for rural households in a way that empowered communities. Lessons learned from countries emerging from conflicts such as Angola provided examples of channeling investments to grass-roots level when institutional capacity was limited. The adoption of a community CDD approach could ensure that project beneficiaries had a say in determining the project activities that best meets their needs and could be sustained.

51. The project helped facilitate a harmonized framework among donors for supporting smallholder agriculture in Angola. The project offered a framework to attract and work with other development agencies active in smallholder agriculture in Angola. The International Fund for Agriculture Development (IFAD) agreed to support the project and work within the same framework to reduce possible duplication and improve the impact of development assistance. The Bank’s involvement led to the mobilization of additional donor support through Poverty and Human Resources Development (PHRD) grant. The project was also seen as a way to consolidate gains of the EMRP and of other IFAD programs in the same provinces. The EMRP helped reestablish institutional capacity of Angola’s two agriculture research centers and of rural extension services in Malanje and Bie while the second phase supported the rehabilitation of some 500 km of feeder roads in Malanje and Bié, most of which were located in the same comunas and municipios selected by project.

3.2 Achievement of Project Development Objectives

13

52. The degree to which the PDO is achieved is based on assessment of the main inter-related outcomes articulated in the project design – increased agricultural production, improved access to productive assets, and improved capacity of farmers to increase productivity and access markets.

Increased agricultural production of participating smallholder farmers

53. The main PDO indicator on agricultural production was considered fully achieved. The primary PDO indicator for the project measured changes in agricultural production among beneficiaries. The original end of project target at appraisal stage was a 25 percent increase in the crop index. This was subsequently revised in the 2013 restructuring to a 10 percent increase. Analysis of the baseline survey and end of project survey estimated the crop index – an aggregated measure of the increase in crop production across four commodities –showed positive trend in real terms and by 2015 was 166 (or 66 percent), exceeding the overall project target of 10 percent. 54. The crop index was constructed based on yield data collected for maize, cassava, bean and potato production in a baseline survey in 2012, annual follow up surveys in 2013 and 2014 and a final end of project survey in 2015.

Figure 1. Crop Production Index

Source: MOSAP End of Project Impact Evaluation Survey Report

55. Yields among farmers increased among both project and non project beneficiaries during implementation. While the project surveys showed relatively small increases in the control group of non-project beneficiaries, difference in difference analysis was also conducted to capture the net increase in yields among project beneficiaries discounting for the overall yield increase experienced by all farmers. This net increase (the difference-in-difference number – or DD in the table below) showed the highest net increase in yield occurred in maize – where the net increase for project beneficiaries was 0.14 t/ha (or 33 percent higher than baseline yield values) followed by potato at 1.27 t/ha (32 percent higher than baseline), beans at .06 t/ha (20 percent higher than baseline) and cassava 2.19 t/ha (13 percent higher than baseline). These are all still above target levels established by the project.

14

56. Although positive, gains in agricultural production in absolute terms are modest. While the project realized gains in production that exceeded targets in percentage terms, the very low baseline levels of production mean that absolute gains were small in some cases. The net increase in production of maize averaged only 142 kg/ha and average yields post-project were still relatively low compared to potential. It should be noted, however that this level of gain is sufficient to allow a household of five to cross the threshold for subsistence production (with average per capita maize consumption in the region assumed to be 80 – 100kg/person/year based on FAO estimates).

Table 1. Mean Crop Yields across Control and Target Populations (t/ha)


57. The targets for adoption were considered fully achieved. The adoption indicator was added in the 2013 restructuring and established an end of project target of 40 percent. Based on the baseline and end of project survey the project has achieved an adoption rate of 62 percent at the end of the project. Adoption was defined as whether the farmer applied the full package of improved farming practices and which commonly included: (i) seed selection, (ii) sowing/seed placement; and (iii) right plant spacing for the crops promoted by the project (maize, cassava, potato, and bean). For cassava, selection and planting of cuttings were also included. Adoption rates were calculated for each crop and aggregated for an overall adoption rate. Individual rates ranged from a low of 47 percent for beans to a high of 72 percent for cassava.

15

Figure 2: Adoption rate for improved farming practices


58. Beneficiary coverage. In terms of beneficiary coverage the project provided training to over 50,000 smallholder farmers organized in groups or associations of which about over 22,000 farmers were on 726 farmers’ field schools. Additional training outside the farmer field school methodology brought the total number of farmers benefitted from training to 32,300. In addition, the project trained more than 210 technicians. At the time of the ICR all farmer field schools were considered functional as training continued up to the closing date of the project which coincided with the end harvest period for the 2015/16 season.

59. Training covered a range of topics including agricultural techniques and diversification of crops (maize, beans, cassava, potato, and vegetables), crop husbandry, methods of production and conservation of potato, seed multiplication and marketing channels. Farmer fields schools followed a three cycle curriculum that culminated in graduation and movement towards cooperative development and more advanced agribusiness training. By project end more than 100 of the 726 farmer fields schools had graduated from the third cycle and nearly 250 completed the second cycle.

Improved access to productive assets

60. The project financed 257 sub-projects and exceeded targets for beneficiary coverage. The project financed sub-projects in 12 municipios in the form of matching grants for production, processing and marketing related assets. Sub-projects primarily focused on animal traction, mechanization, improved seeds and fertilizer, grinding mills, and small scale irrigation pumps. Sub-projects covered a 12,354 smallholder beneficiaries against an initial target of 8,000. There was no target established for number of sub-projects at the time of appraisal.

Table 2. MOSAP Sub-Projects

Province

Type of Sub-projectAnimal traction Mechanization and seeds Mills

Approved Financed Approved Financed Approved Financed

16

Huambo 92 86 0 0 0 0Bié 28 28 33 33 17 17Malanje 4 4 76 76 13 13Total 124 118 109 109 30 30

Source: MOSAP

61. The majority of sub-projects were considered operational at the time of the ICR mission and the functionality of sub-project investments were assessed as part of the project’s M&E systems. By the time of the ICR mission 90 percent of sub-projects had been completed and the remainder were still under the final stages of delivery or construction. Of the project that were completed around half of all sub-projects were fully operational and another third were operational with some minor deficiencies in management or maintenance. Relatively few of the completed sub-projects (5 subprojects or about 2 percent) were considered non-operational and only one sub-project had been abandoned.

62. Beneficiaries have noted, however, that implementation and procurement of sub-projects could have been more decentralized to allow the menu of options available for sub-projects to be more diversified and operations and maintenance arrangements could have been better identified. In some cases field visits identified challenges in maintenance arrangements, particularly with grinding mills which require more complex and sustained O&M.

Table 3. Sub-Project performanceMalanje Huambo Bié Total

Total subprojects financed 93 86 78 257Completed and fully operational 55 32 45 132Completed and operational with weaknesses 34 45 16 95Completed and not operational 3 0 2 5Still under finalization 0 9 15 24Dropped 1 0 0 1

Source: MOSAP

Improved capacity of farmers to access markets

63. The project supported market oriented training and technology adoption to increase farmers’ long term capacity to engage in markets. The core of the project focused on improving the productive capacity of smallholders and to enable farmer to better develop market linkage and market orientation. The selection of commodities as well as the capacity building and training provided to farmers were driven by a market orientation and the opportunities to increase marketable surplus. The selection adoption of commodities – maize, cassava, bean and potato – represented a move towards increased commercialization as many have both food security and cash crop potential.

17

64. Farmer institutional development was an important element of capacity building in addition to building technical knowledge and skills. For smallholders, better access to markets is often achieved through greater farmer organization which can facilitate learning and exchange of experience, economies of scale in marketing, and provide an entry point for service delivery. The project generated a clear benefit in farmer organizational development compared to non-project beneficiaries. The project supported 726 farmer field schools and trained nearly 52,000 farmers. Evidence from baseline and end of project evaluation indicates the Project changed farmer perceptions around the utility of joining farmers associations, which were seen to provide little benefit.

65. Training on marketing and commercialization formed a core part of the farmer field school curriculum and is the main focus of the training in the third year cycle. It is expected that as farmer organizations become more mature, they will serve as a tool for increased group marketing. Already some associations have explored higher level organizational development and have registered as cooperatives in order to put in place more coordinated output marketing and access to credit.

Table 4: Percentage of Farmers who are Members of a Local Association2012 Baseline 2015 end of project

Survey% Change

Project target area 27.9 87.7 212Control 17.7 35.0 98


66. Investments in productive assets – particularly mills – represented a step towards greater value addition and commercialization. Project investments in small scale maize and cassava processing represented a step towards greater market integration and value addition. Investments in animal traction and mechanization will allow associations to build on the initial successful technology testing phase supported by the project and scale up production in coming seasons.

67. The project could have developed an additional focus on input market access to complement sub-project investments and training, however, this may be appropriate for the second phase. While project beneficiaries increased production of all commodities (maize, bean, cassava, and potato), the availability of inputs was often cited as a constraint. Seed and fertilizer were provided in limited quantities as part of sub-projects but broad access to inputs is extremely limited within the project area. Increased production will be limited so long as improved seed, fertilizer and other inputs are not available. The project did provide training to 90 service providers and agro-dealers within the project area and in in the last year of the project was able to source more seed locally from the more mature farmers’ organizations. Greater attention to linkages to supply chains and input markets, however, may have been useful.

3.3 Efficiency

68. Appraisal stage economic and financial analysis estimated the project would extend capacity building, including group formation and strengthening, as well as basic

18

agricultural and or marketing knowledge to 126,000 smallholder farmers out of which, some 40,000 smallholders would benefit from more intensive participatory farmer group-based agricultural extension support and an estimated 24,000 smallholders would benefit directly from productive investment subprojects.

69. The ICR economic financial analysis follows the same logic as the appraisal stage analysis and assess the quantifiable financial and economic benefits generated by increased production and productivity among project beneficiaries. Data collected through the project’s baseline survey and impact provides evidence on yield increases and changes in total production at the farm level among both beneficiary and control populations. Ex ante economic and financial analysis of the follow on SADCP project provided an updated baseline of farm and crop production models for farmers in the project area and have been used in the ICR analysis as average production models at MOSAP project end.

70. The project’s economic rate of return is calculated at 13 percent based on a relatively conservative calculation of net benefits related to the value of increased production observed at project end. This level is higher than the opportunity cost of capital and therefore indicates the project generated a positive rate of return. It is lower than appraisal ERR estimates of 21 percent, however, which were based on a shorter time period for project implementation, lower costs of for service provision and goods, and larger beneficiary coverage.

71. The ICR rate of return calculation also likely represents a lower estimate of the project’s rate of return as additional benefits would be expected from income generated through productive assets such grinding mills, higher value crop production under irrigation sub-projects, and spillover effects among non-project beneficiaries. The project’s investment in capacity building is also difficult to quantify and higher positive returns are likely when if the planned second phase is included in the analysis.

3.4 Justification of Overall Outcome Rating

Rating: Satisfactory

72. An important outcome of the project is that farmers in the project target areas have demonstrated the potential to raise crop productivity beyond historically low levels as long as they are provided with the appropriate supportive infrastructure and incentives. Starting from a very low base, the project achieved its core objective and generated benefits that will serve as a foundation for further commercialization.

73. The project is rated satisfactory based on an assessment of the relevance of the project to Angola’s post conflict, economic and poverty reduction goals, satisfactory achievement of its stated development objective in raising agricultural production and evidence of the generally positive impact generated by the project, all within a challenging country context.

3.5 Overarching Themes, Other Outcomes and Impacts

19

(a) Poverty Impacts, Gender Aspects, and Social Development

74. Poverty. The project’s design was poverty focused given its focus on raising production and ultimately incomes among smallholder farmers. Poverty data remains extremely limited in Angola making any assessment of poverty impact difficult. Rural poverty across Angola is currently estimated to be 58 percent compared to 30 percent in urban areas. At the time of appraisal extreme poverty within the project area was considered significant and in 2005 it was estimated that 68 percent of the population central highlands were classified as food insecure. Detailed analysis on poverty impacts was not undertaken as part of the project, however the baseline level of production captured in project surveys indicate that many beneficiaries were operating at basic levels of subsistence production. The demonstrated increases in yield experienced by project participants while modest, indicate that project interventions have likely contribute to greater food security.

75. Gender. The project was successful in targeting and supporting female participation in training and as members of farmers associations. While there were gender imbalances in farmer associations leadership, with men usually being in a leadership position and women being the bulk of the members in the groups – the project was generally well balanced in terms of gender. In total 43 percent of farmers participating in project activities were women. Farmer field school participants were an estimated 56 percent female. Design and planning of farmer field school activities made it possible for women to participate more in the planning of agricultural activities together with their male counterparts.

76. Sub-project investments also had clear gender impacts in that many focused on introducing labor saving technologies –such as grinding mills, animal traction, and irrigation pumps. Beneficiaries cited positive gender impacts around such investments and a reduction in time required to take on tasks.

77. Literacy training. Early in the preparation period it was determined that the low levels of educational attainment among project beneficiaries presented a challenge to provision of technical training. As a result, the project included and ultimately expanded training under the first component to include literacy skills in a selected number of farmer associations. Basic literacy skills allowed more farmers to participate in farmer field schools and generated additional benefits beyond the project.

(b) Institutional Change/Strengthening

78. Farmer organizational development. A key achievement of the project was to strengthen farmer associations to empower farmers and develop organizational and leadership skills needed to exercise ownership over project decision-making processes and outcomes. Very few farmers had previous experience in associations and in many cases, the implementation and management of sub-project or training activities represented something new, which required a learning and adaptability. The project also supported a few exchanges between associations, which provided a model for joint

20

learning and the potential for greater collective action in the future. Organized networks of farmers’ organization also provided Government with an entry point in providing support in the sector.

79. Because eligibility for project activities required farmer association membership, there is a risk that associations may not be sustainable. MOSAP associations, however, have demonstrated good group cohesion, which is expected to ensure longevity and the follow on project is expected to support further institutional development and value addition to collective activities.

80. Extension agents and EDAs. The project contributed to revitalization of service delivery through EDAs in participating municipios. EDAs, which represent the lowest administrative level in the extension system, had generally suffered from under-investment and under-staffing. Project investments in infrastructure (housing, offices and transport) allowed new staff to be posted to the EDA level and increase the outreach ratio for extension staff to number of farmers reached. The project built capacity of extension agents and provided training in technical skills as well as the principles of community-organizing and group management skills in order to help facilitate farmers associations and communities. These changes are expected to be sustained in the follow on project with staff and operational costs gradually absorbed in to the Government budget.

81. ADI and MINAGRI. The project provided a useful framework for ADI to deliver technical services and support to farmers which had in the past been relatively ad hoc and constrained by both resources and service delivery models. The framework of farmer association development, technical training and learning coupled with investment in productive assets were perceived to be much more effective than previous efforts. The project became popular at the provincial and municipal levels as a result of its grass roots impact in communities. ADI has expressed interest in incorporating many aspects of the farmer field school approach and farmer training more broadly in the country.

(c) Other Unintended Outcomes and Impacts

82. Testing of alternative extension methodologies. One byproduct of the project was the testing of variety of extension methods and the scaling up of the farmer field school methodology in particular. Given the capacity constraints within the public extension system the project utilized a number of service providers to fill gaps in institutional capacity. This involved local and international NGOs as well the Food and Agriculture Organization (FAO), which had developed specialized expertise in farmer field schools. The use of the farmer field school approach in the project was expanded over time and by the end of the project 726 farmer fields schools had been launched.

83. Originally started as tool for learning integrated pest management in Asia, the farmer field school approach evolved into a group-based learning process characterized by participatory learning approaches carried out over multiple seasons. Studies of impact are varied in their findings on the effectiveness of the approach. Because the project likely represented one of the larger scale uses of farmer field schools in Africa, the

21

project could have (and in its second phase should plan for) more systematic and rigorous impact assessment on the approach.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

84. Beneficiary and stakeholder consultations were held as part of the closure of MOSAP and preparation for the follow on project with a focus on taking stock of implementation experience and to identify lessons learned. Stakeholder perceptions of positive and negative aspects are summarized the table below.

Table 5: Stakeholder perceptions on project implementation and impactPositive impacts or implementation experiences Negative impacts or implementation

experiences Reducing the time spent on some essential activities of farmers (grinding corn, cultivation of farms); Increase in the cultivated area; Introduction of more effective cultivation techniques, and thus increasing productivity and product quality; Development of associations, consolidation of associations; development of planning capacity of the associations (project planning, feasibility simplified calculations of viable alternatives analysis,) Stimulation of other farmer associative initiatives, and the emergence of new projects; Development of cooperation activities between associations; Increase in negotiating capacity with public authorities, improving the knowledge of the resources and the limits of the bodies involved; Better market knowledge, is the amount of production (input suppliers, equipment, animals) or downstream (marketing channels, transport,.

Lack of previous experience with associations in villages ; Shortcomings in the field of technical staff Slow pace of project implementation in the early years and highly limited time to reach the goals after the changes introduced in 2013; Difficulties arising from the incipient network equipment providers ( grindings , motor pumps ) , animals, inputs (seeds, fertilizers ) , and limits of programs Government supply of these inputs ;

4. Assessment of Risk to Development Outcome

Rating: Substantial

85. Overall risk to the development outcome is considered substantial – although approval of the follow on project SADCP could be expected to provide a mechanism to mitigate some of the risks to development outcomes.

22

86. Institutional risks. Institutional risk is considered between moderate to substantial. There are risks around integrating project successes into the larger institutional structures of Government. The project increased the technical and operational capacity of MINAGRI and ADI to better manage programs and improved systems, manuals and technical training within the institution at provincial and municipal level. The project created potential for ADI to continue to support interventions around improving the productive capacity of small farmers even after project closing. One challenge, however, will be in adapting models and approaches developed largely outside its systems by external service providers into its own structures over the medium to long term. Strong leadership within ADI and in the PIU was critical to achieving project success but may be more difficult to replicate if personnel changes take place. The current low revenue environment which has reduced government budgets will also likely impact ADI’s ability to support future service delivery.

87. Risks to productivity gains at farm level. Risks at farmer level are twofold: at the production level and at marketing level. Farmers in the project had shown potential to adapt, adopt, and innovate, provided that they receive timely and adequate support. By demonstrating the efficacy and value of project activities and building awareness at both the national and local levels of the benefits of project interventions, there is a high likelihood of maintaining production levels achieved beyond the life of the project. Although access to inputs were a challenge, increased capacity has been demonstrated through community seed multiplication but bears further monitoring.

88. A major risk to sustaining project achievements will be whether investments in improving capacity and skills can be leveraged into greater commercialization and profits for farmers. This will be influenced by Angola’s larger social and economic context, which presents a difficult environment for smallholder farmers to boost production significantly. The country’s weak supply chains and high margins at the retail level favor cheap agricultural imports and result in low producer prices for smallholder farmers. In such an environment, yield and production increases by smallholder farmers may not always result in improved incomes.

89. Environment and Social development risks. Environmental risks are considered low. Project investments in sub-projects have been largely small scale and created no cumulative negative impacts. Maintenance plans have been put in place as part of the sub-project training. Social development risks are also considered low, as larger country risk around potential conflict and violence are minimal and the project’s use of farmer association and community institutions have created ownership over project activities are considered sufficient.

23

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Rating: Moderately Satisfactory

90. MOSAP was the first agricultural development project supported by the World Bank in Angola. Considerable effort went into design of the project, building relationships with counterpart institutions and understanding a complex country and sector environment. Delays experienced in approval and effectiveness affected project startup although there may have been few options to avoid them. Effectiveness and disbursement conditions led to extensive delays but reflected a desire to put in place the minimum conditions needed to ensure sound implementation capacity was in place. At times the two year delay reinforced each other - in some cases conditions had to be met twice (recruitment of staff had to be done twice when in some cases recruited personnel were no longer available once other conditions had been met).

91. Engagement with IFAD on project design also took place and both the IFAD program document and the Bank’s Project Appraisal Document (PAD) were consistent and well aligned. Nonetheless, additional efforts was required to align World Bank and IFAD funding, which required amending/harmonizing financing agreements.

(b) Quality of Supervision

Rating: Satisfactory 92. The Bank undertook regular supervision with task team leader (TTL) and staff based within the region in neighboring countries. The team displayed flexibility and was proactive in addressing implementation bottlenecks. Potential restructuring was discussed early on when it became clear the time available for implementation was lapsing. The team was pro-active in engaging Government on possible cancellation of the credit and adjusting results framework.

93. The supervision and mid-term review process allowed for course corrections and adjustments to design. Supervision was candid in ratings of project shortcomings and constraints. The project was rated unsatisfactory at the start due to delays in establishing implementing capacity and revised upwards when performance improved. Development of implementation modalities for sub-project and service provision for training took time to finalize and were reviewed in detail during early supervision missions. Supervision of the IFAD funds proceeded smoothly and supervision was often undertaken as joint-Bank-IFAD missions.

(c) Justification of Rating for Overall Bank Performance

24

Rating: Satisfactory

94. The satisfactory rating is based on a greater weighting for quality of supervision.

5.2 Borrower Performance

(a) Government Performance

Rating: Moderately unsatisfactory

95. Although achievement of project development objectives would not have been possible without the relatively strong ownership over the project, particularly at provincial and municipal government levels, the 26 month delay in effectiveness had a negative impact on the project. While lack of familiarity with Bank modalities contributed to some delays others were due to factors within the control of Government. Government counterpart funds, which were originally anticipated at US$4.4 million at appraisal were US$2.1 million by project end.

96. Larger policy and institutional issues also impacted project performance, particularly as they related to improving the competence and incentive structure for government staff, at lower levels; the accuracy and timeliness of government statistics to allow for more informed decision making; and the ability of senior government staff to address structural constraints through informing and promoting agriculture sector policy dialogue. Angola’s overall business environment also affected project performance. Timely entry visas for consultants, contractors and Bank staff to support implementation of the project were difficult to obtain and led to higher costs and implementation delays.

(b) Implementing Agency or Agencies Performance


97. Implementing agency performance is rated moderately satisfactory. While the project took some time to fully launch resulting in an initial unsatisfactory rating for the project, project coordination did improve and remained satisfactory for the remaining project period. A key factor in the change was replacement of key project staff that provided the needed boost to manage and coordinate project activities.

98. The PIU oversaw project implementation by PPIUs and performed central functions while PPIU implementation units provided main frontline implementation support. The project had a reporting system based on monthly and quarterly progress reports prepared. Progress reports were prepared for submission to the Bank and were provide on a timely basis throughout the project implementation period.

(c) Justification of Rating for Overall Borrower Performance


25

99. Overall borrower performance is rated Moderately Satisfactory based on a balance of the moderately satisfactory ratings for Implementing Agency performance and moderately unsatisfactory for Government performance.

6. Lessons Learned

100. MOSAP was successful in reaching women, although more needs to be done to ensure that women play an equal role in farmers’ organizations and influence project investment decisions. Women had only limited influence on the choice of subprojects, an issue that will need extra attention under the proposed project. Women’s participation in farmers’ organizations was lower than that of men and women usually had a secondary role, with only a few occupying leadership positions. More women could be encouraged to get training as FFS facilitators and become community leaders, for example by arranging training sessions to favor women’s participation.

101. The Farmer Field School (FFS) approach to agricultural extension was effective in enhancing smallholder farmers’ capacity to generate and use new knowledge and adopt improved agricultural practices and technology. Although the performance of FFS needs to be evaluated in more depth, it was considered to be the appropriate extension methodology and was found to be an effective platform for farmer organization and empowerment, where smallholder farmers with a common interest can gain increased production, productivity and access to market. This platform served an important role for farmers both in terms of social and technical support. Farmers who participate in the first phase of training of trainers become Master Farmers who facilitate or replicate the training to new farmers, thus achieving an increase in the number of farm families with strengthened capacities. The chances of effective FFS network to grow, can be strengthened with greater focus on farmer-driven network development. In order to build on best practice of using farmers as extension agents whilst ensuring their sustainability post project, it is recommended that SADCP and SAMAP projects, should facilitate farmer promoters to become agricultural input suppliers at the village level. Within SAMAP, and SADCP outstanding farmers should be able to earn service fees in delivering technological advices to their client farmers.

102. A capable and internationally experienced service provider will be required to help farmers’ organizations develop and implement investment proposals, especially for value chain investments. The use of small, local service providers to assist farmers’ associations in the preparation and implementation of MOSAP subprojects was problematic because of their limited technical and organizational capacity. Improving the quality of technical and commercial assistance and making it available to a much larger number of beneficiaries will require recruitment of a highly capable and internationally experienced service provider who will also train the local NGOs and private sector subcontractors with whom they will work.

103. Financing demand-driven subproject investment proposals promotes their sustainability. The modalities for preparing and implementing demand-driven

26

subprojects under MOSAP were generally successful, with over 70 percent of subprojects judged sustainable by the independent evaluation. The key elements of success were (a) creation of ownership through community participation in all stages of subproject preparation and implementation, as well as direct community contributions (cash or in kind); (b) farmers’ organizations’ commitment to engage in operation and management (O&M) before disbursement of project contribution; (c) training of farmers’ organizations and assistance with establishment of user rules and cost-sharing agreements; and (d) awareness raising and encouragement for farmers’ organizations to identify and contract the technical assistance they need.

104. Procurement, FM, and M&E teams need to be involved in the early stages of project design. During MOSAP implementation, the project had difficulty finding local qualified professionals to effectively undertake the fiduciary and M&E functions. The lack of qualified procurement staff was in part responsible for the delayed start of project implementation. While hiring was possible within a reasonable period, the selected consultants were not familiar with World Bank procedures.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies

105. Borrower comments generally concur with the ICR and highlight the following three lessons that will be incorporated in SADCP: (i) improved farmer organization through groups or associations increased provision of assistance to smallholders and their access to inputs, but continued assistance is required for increased sustainability; (ii) development of smallholder capacity is key for increased sustainability; and good communication and (iii) participatory planning was key for smooth implementation. Full comment are contained in Annex 7.

(b) Cofinanciers

106. Comments from IFAD on the draft ICR have been incorporated into the final text including planned follow up financing to MOSAP through a new planned operation SAMAP. Full comments are found in Annex 8.

(c) Other partners and stakeholders

N/A

27

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)


Component 1: Capacity Building 9.59 13.70 143%Component 2: Agricultural Investment Support 29.45 7.74 26%Component 3: Project Management 7.56 11.61 154%

Total Baseline Cost 42.45 33.05Physical Contingencies 0.25Price Contingencies 2.50

Total Project Costs 49.35 33.05 67%

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)


Borrower 4.12 4.40 107% Local Communities 3.03 0.00* N/A International Development Association (IDA) 30.00 18.17 61% International Fund for Agriculture Development 8.20 6.27 76% JAPAN: Ministry of Finance - PHRD Grants 4.02 3.61 90%Sub-Total (w/o local community contribution) 46.34 32.45

Total 49.37*not measured

28

(c) Detailed breakdown by financier1

Table 1.1 At Appraisal

Categories IDA %IFAD %

PHRD % Gov %

Total %

Component 1

(1.a)

Consulting services and training 0.62 7% 3.2 38% 4 47% 0.64 8% 8.46 100%

(1.b) Goods and works 2.8

100% 0 0% 0 0% 0 0% 2.8 100%

Component 2 Matching grants23.4

6 89% 2.85 11% 0 0 0 0%26.3

1 100%

Component 3Goods, consulting services, training& operating costs 3.12 36% 2.15 25% 0 3.48 40% 8.75 100%

Total 30 8.2 4 4.12 46.3

2

Table 1.2. February 2013 with the Cancellation of US$10 million


PHRD % Gov % Total %

Component 1

(1.a) Consulting services and training 0.62 7% 3.2 38% 4 47% 0.64 8% 8.46 100%(1.b) Goods and works 8.26

100% 0 0% 0 0% 0 0% 8.26 100%

Component 2 Matching grants 8.00 89% 2.85 11% 0 0 0 0% 10.85 100%

Component 3

Goods, consulting services, training& operating costs 3.12 75% 2.15 25% 0 1.50 0 6.77 100%

Total20.0

0 8.2 4 2.14 34..3

4

1 The following tables are sourced from the individual restructuring papers and there are some minor difference with the overall summary of project costs in the annex due to of the differences in the exchange rate between SDR and USD at time of calculation.

29

Table 1.3. September 2014


PHRD % Gov % Total %

Component 1

(1.a) Consulting services and training 0.62 7% 3.2 38% 4 47% 0.64 8% 8.46 100%(1.b) Goods and works 7.70

100% 0 0% 0 0% 0 0% 8.26 100%

Component 2 Matching grants 6..78 89% 2.85 11% 0 0 0 0% 10.85 100%

Component 3

Goods, consulting services, training& operating costs 4.90 75% 2.15 25% 0 1.50 0 6.77 100%

Total20.0

0 8.2 4 2.14 34..3

4

Table 1.4. December 2015

Categories IDA % IFAD %PHRD % Gov % Total %

Component 1

(1.a) Consulting services and training 0.31 5% 1.56 5% 4 90% 0.64 0% 6.51 100%(1.b) Goods and works 7.70 100% 0 0% 0 0% 0 0% 7.7 100%

Component 2 Matching grants 6..78 20% 4.29 80% 0 0 0 0% 11.07 100%

Component 3Goods, consulting services, training& operating costs 5.21 75% 2.35 25% 0 1.50 0 9.06 100%

Total20.0

0 8.2 4 2.14 34..34

30

Annex 2. Outputs by Component

Component 1 - Capacity Building

1. This component was designed to strengthen the technical, institutional, managerial and marketing skills of 50,000 smallholders and their organizations, as well as the skills of services providers and other stakeholders involved in agricultural production and value chains, to allow them to operate more effectively in a market-driven environment and prepare for the agricultural investment support opportunities under Component 2. This component was also designed to strengthen the capacity of the Agricultural Development Institute of the Ministry of Agriculture at local and central level by providing periodic training and means (such as offices, equipment, transport, and housing for extension officers at the municipal level) to be able to assist smallholders in the Project area.

2. The Project developed six training modules that were used by the service providers to deliver training programs for smallholder farmers. The Government also contracted the FAO to provide training of smallholder farmers through the Farmers’ Field School.

Summary of Component 1 outputs

3. Overall, the project achieved the following results: (i) training of 54,982 smallholder farmers in community organization and leadership, organization of production and agricultural techniques for maize, beans and Irish potatoes, and cassava; of which over 22,000 farmers through FAO Farmers’ Field Schools; of which 1,497 graduated as facilitators and 96 as community leadership, and about 2,252 benefitted from literacy and agribusiness; (ii) training of 210 Government technicians in community leadership, agronomic, extension methods among others; (iii) construction of 24 staff houses and 8 offices; (iv) provision of equipment to local agricultural offices to improve assistance to smallholders that included 13 vehicles and 40 moto-bikes; and (v) creation of a database of associations, service providers, and EDA technicians.

Table 2.1 Component 1 outputsIntermediate indicator for component 1

Targets Actual Percentage

Number of smallholder farmers that benefited from training

50,000 54,982 110

Percent of participating smallholder farmers in the project areas who belong to farm organizations

40 100 250

Ratio of smallholder farmers’ organizations to an extension officer

12 12 100

31

i) Capacity Building for Farmers and Community Mobilization and Sensitization

4. MOSAP hired four service providers to build capacity among farmers and local services providers and to provide training for local government personnel.

5. Smallholder farmers were trained on improved production technologies for major crops (maize, cassava, beans and potato) and management practices. Demonstration plots were also established to make the farmer aware of the benefits of new technology and to show the farmers how to practically use the new technology. As a result, a portion of the trained farmers adopted the new technologies in their farming land area. A total number of 54,982 smallholders farmers in Associations out of a target of 50,000 were trained in all project areas, with around 42.4 percent of them being females. The training focused on, leadership skills, finance management skills, farmers field schools, operation and maintenance of mills, agribusiness, illiteracy, and business planning and processes.

6. One of the main activities of the first subcomponent was the use of farmer field school (FFS) methodology developed by FAO. Farmer field school methodology aimed to build farmers’ capacity and promoted adoption of better practices, and consequently improve farmers’ lives in terms of agricultural outcomes, health, environment and empowerment. A total of 726 FFS were created in three provinces, comprising of 22,432 farmers’ members, 66 trained extensions facilitators and 307 trained farmers facilitators.

7. FFS were formed either from existing farmer’s groups/associations or from scratch through community mobilization activities, in which problems were identified and possible solutions floated. The FFS group sizes range from 25-30 farmers. The groups elect their own leaders; developed their own norms and rules and met once every week from 8.00 am-12.00 pm. The FFS is run for 30 learning weeks on grants provided by FAO. The farmer led field schools receive backstopping from EDA extension workers from time to time. During the learning period an exchange visit was organized for the group members. The participants of FFS also receive a certificate at the end of the 30 learning weeks.

8. The group members perceived that the farmer field schools have contributed to creating strong farmer-based structure at the grass root level. It also developed farmer’s capacity to understand the activities they are carrying out. However, strong facilitation skills were required for the success of the FFS in addition to monitoring and evaluation systems. On sustainability, it was perceived that the development of FFS networks could help sustain the process of learning. The farmers also felt that FFS has helped to bring Ministry of Agriculture extension workers closer to farmers as previously they were not frequenting the locality.

9. The following were perceived by farmers to be the main components of a field school:

Presence of a study field;

32

Payment of registration fees; Committed members; Willingness to learn; Presence of trained facilitator; Norms and rules; Capable leaders; and Clearly set objectives.

10. Extension workers felt that FFS has helped make them reach more farmers. The FFS has contributed to building up a close working relationship with the extension workers as some of the farmers had started looking for services by frequenting EDA offices. The extension workers also perceived that their roles had gradually been changing to that of a facilitator instead of being a teacher.

11. However, they recognized that there were some conflict with their present work description and the demands of FFS as they were required to attend to implement other extension methodologies of the EDAs besides the FFS. Currently, the EDA Office has allocated five or six FFS to an extension worker.

12. The integration of FFS with a subproject helped to rationalization of resources of EDAs, as well as, to take advantage of the best way the knowledge being taught in FFS could be applied to implement subprojects, mainly crop production subprojects.

ii) Strengthening capacities of relevant government institutions

13. The project provided support to departments within the Ministry of Agriculture to improve the capacity of the public sector to provide core public services required to support enhanced smallholders’ to improve their productivity and product quality.

14. The project has expanded activities with a focus on intensively increasing the quality of support to beneficiaries (farmers and extension workers). This was being done by working EDAs offices in municipalities, improving the quality of support to the beneficiaries, increasing the number of front level staff like Farmers Field Facilitators, who interact directly with the beneficiary farmers and extension workers.

15. At municipality level, a total of 88 government extension workers including, veterinary field who were designated to directly support the farmers were provided training on improved production technology and management training.

16. Identified topics included refresher training on improved agricultural practices that project was working on; how to reach and interact with target farmer associations/groups (facilitation skills); incorporation of gender and diversity aspects into extension activities (FFS); participatory rural appraisal formation, governance and leadership of smallholder farmers associations/groups and health, environmental and nutrition messages.

33

17. The smallholder farmer associations were linked to the respective extension worker through FFS and subprojects. The extension workers provided training to the farmer groups. In this way, an enhanced extension service relation had been developed.

18. The project provided motorcycles to extension workers and to farmers’ facilitators. In order to increase outreach smallholder farmers in remote communities, the project constructed and equipped offices and residences for extension workers.

19. The 3-day workshops on agricultural policies was conducted for the benefit of the central ADI, MINAGRI and other stakeholders. The aim was to give general notions about the role of farming family on rural development and present known and validated methodologies, by the international agencies of the United Nations, that enable partners to obtain intervening concrete tools for their works in their regional and rural realities.

20. Training of ADI Staff on M&E. The FAO M&E expert has provided training to the field staff on FFS data collection method and quality measures in gathering information using the project introduced revised templates.

iii) Strengthening capacities of private and non-governmental agricultural service providers.

21. As a means to develop a private avenue for increasing farmer access to extension, the MOSAP Project also developed capacity of input retailers and local service providers who conducted business with project beneficiaries in order to strengthen or increase their capacity to provide extension services. Assistance included business planning and processes and leadership skills. 90 service providers were trained.

Component 2 - Support to Agricultural Investment

22. This component was designed to provide demand-based support, in the form of matching grants, to 10,000 smallholder farmers organized in groups, for small-scale agricultural infrastructure, production, processing and marketing sub-projects. The key results expected from the implementation of this component include: (i) enhanced agricultural productivity and (ii) enhanced access to agricultural markets.

23. Component 2 was designed to be implemented after capacity building activities supported under Component 1 had been initiated, to ensure that capacity exists at local level to implement sub-projects successfully. For this reason, and given the delays incurred in implementing the capacity building activities supported under Component 1, only in February 2013 farmer’s associations started preparing the sub-projects, with support from the Agricultural Development Office of Agricultural Development Institute (EDA) and the service providers. The main activities carried out up to date include: i) provision of financial resources to 257 sub-projects involving over 12,354 smallholder agricultural farmers; that involved (a) provision of seeds for potatoes; maize, beans and fertilizers; (b) support in provision of assistance for mechanized land preparation covering over 1,500 ha and (c) provision of animal traction. This resulted in: (i) Increased

34

agriculture productivity in most crops supported under the project, (ii) increased agro-processing of cassava; and (iii) increased income of beneficiary farmer as repotted on the beneficiary assessment undertaken in the project.

Table 2.2 Component 1 outputsIntermediate indicator for Component 2 Targets Actual Percentage

Number of smallholder farmers financed under agricultural investment component

10,000 12,531 125

Number of smallholder associations that benefit from grants and completed their activities under the Project’s agricultural investment component

8,000 12,344 154

Average yield for major crops participating smallholder farmers:

Maize 0.6 0.77 128Cassava 11 18.2 165

Beans 0.4 0.47 117Iris potatoes 5 6.9 138

Subprojects details

24. Implementation began in late 2012. Part of the delay in the implementation of Component 2 is related to the delay in the start of training activities under the first component only started in March 2012; and the other was related to the disbursement conditions for component 2. The disbursement conditions were: (i) At least 30 (10 in each province) service providers and certified public officers appointed under the Project Training program; (ii) The basic survey completed; and (iii) the administrative capacity of the Provincial Project Implementation Units are consistent with the fiduciary requirements of the Bank, as certified by World Bank. These conditions were met by the end of July of 2012.

25. Although project activity was limited between 2010 and 2012 some activities around Institutional Development, dissemination of information around the project and community mobilization activities took place with associations and local authorities. During this time he operating procedures of the subprojects and conducting technical training seminars of EDAs in "Developing Productive Subprojects of MOSAP" was prepared.

26. EDA technicians supported small producers to develop their proposals for funding and in consultation with communal administrations to determine the relevance of such associations, and environmental safeguards ensured. Municipal administration/CMCS also gave its opinion to ensure coordination with the Project MOSAP. Subprojects made with budgets ranging from $50,000 to $100,000, an amount which included the 10 percent community contribution. Sub-projects were evaluated and approved by the provincial and national Subcommittees.

35

Table 2.3 Eligible types of sub-projects Sub-Project Types ExamplesHigh public interest and / or public use Agricultural infrastructure of small-scale

processing - Grinding millsEquipment forms the dominant component Animal traction equipment, plows, sprayers.

Agricultural mechanizationPotential for further production and redistribution Animal tractionImproved agricultural technologies Inputs

Table 2.4 Subprojects prepared, approved and financed by year

27. A total of 335 subprojects were prepared in which the vast majority (68 percent) were prepared and financed in 2013. Almost all developed subprojects were funded (97 percent). Of the 257 subprojects financed, 46 percent subprojects were animal traction, 42 percent of mechanization and seeds and 12 percent of mills. Reduction the number of new subprojects occurred after the first and was considered due to the fact that the more effort was given in the first year to the preparation of proposals, due to delays in the project implementation of activities, which were soon balanced out in the last two years.

Tracção animal Mecanização e sementes

Moagens Total0

50

100

150

200

250

300

Subprojectos financiados por tipo

Núm

ero

Ano Prepared Approved Financed2012 54 2 02013 227 221 1522014 48 16 812015 6 24 24Total 335 263 257

36

28. Types of subprojects varied by province. The vast majority of livestock projects (animal traction) were requested by small producers of Huambo and Bié, areas with a history of animal traction usage.

Table2.5 Type subprojects financed by province

Provincia

Tipo de subprojectos Animal Traction Mechanization and seeds Mills

Approved Financed Approved Financed Approved Financed

Huambo 92 86 0 0 0 0Bié 28 28 33 33 17 17Malanje 4 4 76 76 13 13Total 124 118 109 109 30 30

Project beneficiaries Summary

29. The following table presents a summary of the beneficiaries of the subprojects reaching a total of 12,354 beneficiary families. Taking into account an average of 5 people per family in the Project area, it is estimated that the project directly benefited approximately 60,000 people.

Table2.6: Number of beneficiaries by type of subproject

ProvinceSubproject type Total

Animal Traction Mechanization and seeds

Mills

Huambo 3396 - - 3396

Bié 1530 1438 1034 4002

Malanje 199 3959 798 4956

Total 5125 5397 1832 12354

Table: Subproject Data

Província

Municípios Número de subprojectos

Famílias beneficiárias

Numero de Ecas

Técnicos das EDAs

Bié

Andulo 22 1001 107 9Catabola 16 756 104 8Camacupa 23 1398 80 8Chinguar 17 847 71 6

Subtotal Bié 78 4002 362 31Huambo Bailundo 42 1517 51 6

37

Londuimbali 27 1434 55 6Mungo 17 710 58 2

Subtotal Huambo 86 3661 164 14

Malanje

Caculama 19 992 32 8Cacuso 23 1238 46 9Malanje 13 701 23 7Kiwaba Nzoji 22 1144 28 7Kalandula 16 881 67 11

Subtotal Malanje 93 4956 196 42Total Geral 257 12619 722 87

Table 2.7: Subproject expenditures (USD)Tipo 2013 2014 2015 Total Mills Huambo 37.800,00 0,00 1.850,37 39.650,37 Bié 323.798,20 270.806,25 113.947,09 708.551,54 Malanje 195.253,09 2.447,25 126.055,69 323.756,03Total parcial 556.851,29 273.253,50 241.853,15 1.071.957,94Reprodução Natural Huambo 486.995,82 322.659,61 197.495,23 1.007.150,66 Bié 123.202,75 186.271,82 424.246,67 733.721,24 Malanje 68.239,17 274.531,72 115.839,59 458.610,48Total parcial 678.437,74 783.463,15 737.581,48 2.199.482,37Inovação Tecnologica Huambo 274.555,74 310.042,39 598,07 585.196,20 Bié 557.723,46 422.432,88 45.252,28 1.025.408,62 Malanje 1.321.160,89 451.595,37 16.605,66 1.789.361,92Total parcial 2.153.440,09 1.184.070,64 62.456,01 3.399.966,74

Total Geral 3.388.729,12 2.240.787,29 1.041.890,64 6.671.407,05

Operation and maintenance of subprojects

30. The project tried to put in place systems to support the preparation and analysis of productive enterprises as well as systems for the provision of technical assistance for maintenance and operation of investments and development of viable business plans. Municipal EDAs were required to have technicians directly responsible for monitoring and evaluation in order to continue to monitor and evaluate the continuity of subprojects especially those who were operating regularly, the subprojects running and operating with weaknesses.

31. Subprojects supporting mills and/or other recurring source of regular cash flow were targeted for monitoring of income, account services, account opening, deposits and expenses as a way of ensuring that the services of the mills were for all members instead

38

of benefit a few people within an association. For the maintenance, mills were encouraged to create a spare parts store sales for the mills, with direct connection to Sotecma, whose location and supply could provide a response to demand for maintenance services of existing in the province in general mills. Training for technicians to service mills were also discussed.

32. For livestock, the Veterinary Provincial Services was tasked with support for the herd of animal traction cattle introduced by the project in line with their mandate for provision of livestock animal health services in rural communities. A protocol of performance between the IDA, Services veterinarians, municipal administrations and the beneficiary communities was signed to facilitate service delivery. The EDAs were to identify in rural communities, staff experienced in livestock training for animal traction to increase the cultivable area by animal traction in rural areas, given that the actual cattle introduced by the project is not the current area prepared.

Sub-project status and sustainability

33. An estimated 90 percent of the financed sub-projects were already completed at the ICR stage, of which 51 percent are in full operation. Among the remaining 9.4 percent were in the final stages of completion, while only 2.0 percent have stood or abandoned.

Table 2.8: Status of Sub-projectsMalanje Huambo Bié Total

Subprojects financed 93 86 78 257 Subprojects completed and operating normally 55 32 45 132Subprojects completed and operating with deficiencies 34 45 16 95Completed and non-operational 3 0 2 5Still under execution 0 9 15 24Abandoned 1 0 0 1

34. Factors that influenced positively the implementation of subprojects:

• The strategy of giving the associations the identification, implementation of sub-projects has created real opportunities for empowerment;• Functional organizational leadership in existence at both the Association and the monitoring level of the project;• Technology Adoption through field schools, replicating in their own lessons learned;• Existence of a system to account on revenues acquired through meetings between the members and the subproject management group;• Supervision and support of sub-projects by the technicians of EDAs.

39

35. Sub-projects completed and operating with deficiencies include some examples mechanization subprojects. In some cases communities did not fulfill their obligations clear the area, a situation that affected the mechanization of work. Mechanized service providers in the province also had limited capacity to meet increased demand during growing seasons. For mill subprojects, the main constraint is the weak organizational leadership of the association and the management group, changing the operators of mills and / or inefficient operators. As well as the constant interference of some community leaders.

Component 3 - Project Management

36. This component was designed to support Project management to use the funds in accordance with the objectives and procedures and monitoring and evaluation. The main activities carried out to date include: (i) project management including preparation of the procurement plan to procure all goods, works and equipment for the project; (ii) preparation of the mitigation measures for the sub-projects; (iii) verification of all indicators intermediary indicators; (iv) completion of three surveys on outcome indicators (baseline, 2013 and 2015 and end line data); (v) completion of the audits; and (vi) sub-projects, (vii) organization of training for staff, including for monitoring and evaluation, the project had produced two databases: Monitoring Information systems (MIS), training database for Government staff, and Database on farmer Field Schools.

40

Annex 3. Economic and Financial Analysis

Appraisal stage assumptions

1. Appraisal stage economic and financial analysis was based on analysis of the anticipated financial returns to farmers participating in the project and the overall economic rate of return generated by increased crop production. It was assumed the project would extend capacity building as well as basic agricultural and or marketing knowledge to 126,000 smallholder farmers out of which, some 40,000 smallholders would benefit from more intensive participatory farmer group-based agricultural extension support (farmer field schools) and an estimated 24,000 smallholders would benefit directly from productive investment subprojects.

2. The project was expected to increase crop production as a result of: (a) higher productivity per unit of labor and land in the crop and farm models as applied to smallholders with project as compared to smallholders without project; and (b) an increase of the cropping intensity (i.e. area cropped) for smallholders with project as compared to without project.

Table3.1: Cropping pattern of farm models Beneficiary type Area in ha (Huambo/Bie model) Yield ton/ha, year 3 with project

Maize Beans Potato Total Maize Beans Cassava TotalWithout project 1.20 0.15 0.15 1.50 0.50 0.30 2.50 2.80Capacity building

& sub-projects1.80 0.25 0.25 2.30 1.50 0.40 7.50 3.10

Capacity building w/FFS

1.60 0.20 0.20 2.00 1.00 0.35 5.00 3.00

Capacity building - basic

1.20 0.15 0.15 1.50 0.65 0.32 3.25 2.90

Table 3.2: Financial profitability of the farm models (Huambo/Bie model)With/out project Without

projectCapacity building

& sub-projectsCapacity

building w/FFSCapacity

building - basicNet income in US$

118 1 009 445 205

Increase in income in US$ 0 890 327 87Increase in income in % 0 755 277 73Net income per day in US$ 1.5 6.4 3.7 2.6Increase in income per day in US$

0 4.9 2.2 1.1

Increase in income per day in %

0 326 147 73

NPV in US$ 5 037 1 897 524FIRR in % 156 217 9 101

41

3. The appraisal stage analysis showed high financial rates of return to increased crop production and an Economic Rate of Return (ERR) of 21.1 percent.

Table 3.3: Total estimated impact on production

Groups of ProductsAnnual productionw/o the project - mt

Annual productionwith the project - mt

Increase%

Cereals (maize) 63 000 148 300 135Pulses (beans) 5 198 7 458 43Tubers (potatoes) 35 438 86 419 144Roots (cassava flour) 88 200 111 850 27

Table 3.4. Net incomeTotal Project

(126,000 families)W/o ProjectUS$ per year

With ProjectUS$ per year

Increase%

Gross production value 62 165 000 111 061 000 78Production Costs 29 478 000 40 795 000 38Net financial Incomes 32 687 000 70 266 000 115

ICR Economic and Financial Analysis

4. Approach. The ICR economic financial analysis follows the same logic as the appraisal stage analysis and assess the quantifiable financial and economic benefits generated by increased production and productivity among project beneficiaries.

5. Data collected through the project’s baseline survey and impact evaluation is sufficient to undertake a basic economic and financial analysis. The baseline and impact evaluation data provides evidence on yield increases and changes in total production at the farm level among both beneficiary and control populations. Ex ante economic and financial analysis of the follow on SADCP project provided an updated baseline of farm and crop production models for farmers in the project area and have been used in this ICR analysis as average production models at MOSAP project end. The sample size of the impact evaluation is considered to be sufficient to generate a reliable picture of production impact and average production values.

6. Outputs achieved by project completion stage. MOSAP beneficiaries experienced higher average yield increases across all crops targeted by the project. Annual surveys and the final impact evaluation also assessed increases in overall production due to both increased yield and expansion of area under cultivation. Because farming systems and production patterns differed across provinces data remains disaggregated for purposes of the economic and financial analysis. While beans and potato were grown across all three province cassava was promoted in Malanje, a drier area where maize was less viable.

42

Table 3.5 Yield levels achievedBeneficiary

typeYield ton/ha, baseline Yield ton/ha, year 3 with project

Maize Beans Potato Cassava Maize Beans Potato CassavaTarget 0.43 0.28 3.37 15.4 0.52 0.37 5.20 15.0Control 0.43 0.30 3.99 16.4 0.66 0.45 7.10 18.2

Source: MOSAP Impact Evaluation Report

Table 3.6 Average production in target and control beneficiariesBeneficiary

typeAverage production per farmer –

baseline (tons)Average production per farmer –year 3

with project (tons)Maize Beans Potato Cassava Maize Beans Potato Cassava

MalanjeTarget na 0.06 0.87 14.9 0.06 1.05 20.4Control na 0.04 0.39 6.4 0.05 0.48 8.6HuamboTarget 0.59 0.24 0.7 na 0.83 0.42 1.59 naControl 0.62 0.20 0.9 na 1.08 0.26 3.02 naBieTarget 0.51 0.15 0.6 na 0.77 0.36 1.28 naControl 0.36 0.11 0.4 na 0.55 0.18 0.90 na

na= data not available or production level negligibleSource: MOSAP Impact Evaluation Report

7. Only the incremental production generated by project beneficiaries is used in the analysis in order to capture the portion of production increases that can be attributed to MOSAP rather than the general rise in production experienced by both project and non-project beneficiaries. In most cases both target and control groups increased production although MOSAP beneficiaries increased production to levels higher than the control group. Table 3.7 shows the “before and after” change in production within the target group (project beneficiaries) and control populations as well as the net change experienced by the target group above the control. These amounts are then applied to beneficiary population within the provinces to generate an overall picture of the total volume of incremental production generated by the project.

43

Table 3.7 Net production attributable to MOSAP interventions

Beneficiary type

Net increase in production per farmer (end of project average production minus

baseline average production) (tons)

Incremental production per farmer beneficiary attributable to MOSAP

(target group net increase in production minus control group net increase in

production) (tons)

Maize Beans Potato Cassava Maize Beans Potato CassavaMalanjeTarget na 0 0.18 5.5 na 0 0.09 3.3Control na 0.01 0.09 2.2HuamboTarget 0.24 0.18 0.89 na 0.24 0.12 0.89 na Control 0.46 0.06 2.12 NaBieTarget 0.26 0.21 0.68 Na 0.07 0.14 0.18 na Control 0.19 0.07 0.5 na

na= data not available or production level negligibleSource: calculated based on MOSAP Impact Evaluation Report

Table 3.8: MOSAP Beneficiary coverage# benefitting from Sub-Projects

# benefitting from FFS

Total # benefitting from project: Sub-projects, FFS

or general training/extension support

Total # of farmers in project area

comunes

Malanje 4,956 5,703 17,786 49,864Huambo 3,661 5,491 14,705 47,787Bie 4,002 11,238 22,491 103,134

12,619 22,432 54,982 200,785Source: MOSAP Project Reports, MOSAP PAD

8. Economic rate of return. The stream of net income generated by the project is calculated by applying the cost of production and current output prices to the incremental production levels generated from the data above. The value of family labor is included to incorporate the opportunity cost of labor. Because the productivity for both maize and beans are low and labor costs are high, this latter calculation renders economic benefit positive only for potato and cassava. Nonetheless, the stream of net benefits remains positive and the net economic value of production is used to calculate the rate of return.

Table 3.9: Economic Value of incremental MOSAP Production Maize Beans Potato CassavaTotal volume of incremental annual production Malanje (tons) 0 0 1,601 58,694Huambo (tons) 3,529 1,765 13,087 0Bie (tons) 1,574 3,149 4,048 0Total incremental production (tons) 5,104 4,913 18,737 58,694

44

Average farmgate price/kg (AOA) 26 92 61 20

Variable Production Cost /kg (family labor not valued) AOA *

25.7 53.7 18.0 3.0

Variable Production Cost / kg (family labor valued) AOA *

55.3 129.5 33.0 9.3

Net value of production Net Economic Value of production (family labor valued) AOA

-149,464,386 -185,017,421 528,683,550 650,861,568

Net Economic Value of production (USD)**

-931,010 -1,152,469 3,293,158 4,054,202

* Variable production costs estimated from updated farm budgets developed under SADCP baseline scenario** March 2016 official exchange rate: 160.5 AOA/USD

Table 3.10: Net benefit stream generated by MOSAP 2010 2011 2012 2013 2014 2015 2016 2017-30

MOSAP Disbursements (USD)

300,000 26,344 3,339,512 11,346,389 7,090,182 5,789,532 3,362,857

Net value of production (USD)

5,263,880 5,263,880

Net income stream (USD)

-300,000 -26,344 -3,339,512 -11,346,389 -7,090,182 -5,789,532 1,901,023 5,263,880

9. The project’s economic rate of return is calculated at 13 percent based on a conservative calculation of net benefits related to the value of increased production observed at project end. This level is higher than the opportunity cost of capital and therefore indicates the project generated a positive rate of return. It is lower than appraisal estimates, however, which were based on a shorter time period for project implementation, lower costs of for service provision and goods, and wider beneficiary coverage.

10. The ICR ERR also likely represents a lower bound estimate of the project’s rate of return as additional benefits would be expected from income generated through productive assets such grinding mills, higher value crop production under irrigation sub-projects, and spillover effects among non-project beneficiaries. The project’s investment in capacity building is also difficult to quantify.

Table 3.11 Economic Rate of Return and NPVICR stage Appraisal stage

ERR 13.1% 21.1%NPV $5,347,126

45

Efficiency in achieving other project outcomes

11. The efficiency of service delivery and capacity building was not assessed but the project’s indications are that despite the challenges of Angola’s high cost environment made cost effectiveness was prioritized to the extent possible through the use of competitive service providers and in the case of FAO, some cost-sharing for service delivery of farmer field school technical assistance.

46

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Specialty

Lending Arbi Ben Achour Consultant GSU11 Slaheddine Ben-Halima Consultant GGO05 Serigne Omar Fye Consultant GENDR Luisa Moises Matsinhe Senior Executive Assistant AFCS2 Domingas de Fatima Rego Pegado Program Assistant AFMAO

Franco Russo Senior Operations Officer GED02 Daniel Liborio Da Cruz Sousa Consultant GFADR Joao Tinga Financial Management Specialist GGO13Supervision/ICR Pedro Arlindo Agric. Economist GFA07 Andrew Osei Asibey Consultant SACKB Cary Anne Cadman Senior Environmental Specialist GEN02 Antonio L. Chamuco Senior Procurement Specialist GGO07 Meseret Kebede Senior Program Assistant LEGES Mohinder S. Mudahar Consultant GWA01 Domingas de Fatima Rego Pegado Program Assistant AFMAO

Joao Tinga Financial Management Specialist GGO13

(b) Staff Time and Cost

Stage of Project CycleStaff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY05 77.79 FY06 190.35 FY07 407.48 FY08 176.03

Total: 851.65Supervision/ICR

Total: 0.00

47

Annex 5. Beneficiary Survey Results

(MOSAP End of Project Impact Evaluation Survey Report: Executive Summary)

Objectives of the final evaluation

1. As the Market Oriented Smallholder Project (MOSAP) draws to an end, the Project’s Management Unit (PMU) commissioned an end-line study, which would assess and document the final project outcome in 2015 against the baseline values of 2012. It is essentially a comparison of pre- and post-project intervention.

2. The study also intends to identify promising practices; challenges faced during implementation and how those were overcome and addressed. As the final monitoring and evaluation exercise, the report combines the annual performance indicator monitoring together with the impact evaluation of the project. The study includes qualitative and quantitative approaches with a variety of primary data sources including participatory methods, which were the main focus on the identification and classification of promising and best practices.3.4. The project's development objective is to increase the agricultural production through provision of better services and offer investment support to rural smallholder farmers in selected districts of Malanje, Huambo, and Bié provinces. The outcome performance indicators that reflect the impact the project aims to produce, include, (i) 40% of the target smallholders adopting a set of improved agricultural technology promoted by the Project; and (ii) an average increase of 10% of smallholder agricultural production in the target areas. Thus, the report focuses on four key project indicators: (i) percentage of project beneficiaries who have adopted an improved agricultural technology promoted by the project; (ii) average yield of the main crops (maize, cassava, beans and potato) of participating smallholder farmers; (iii) percentage increase in agricultural production based on the crop index for the project area; and (iv) the promising and best practices (PBP) promoted by the project.

Methodological overview5. The methodology employed in the survey was developed with the view of ensuring that the dataset structure for key project indicators is consistent with the baseline survey. The current survey therefore applied the sample frame of 1,500 smallholder farm families, distributed in proportions corresponding to the size of the intervention per province as follows: 48% in Bié, 32% inMalanje and the remaining 20% in Huambo. The total sample was divided into 30 clusters, spread in 25 wards (comunas) in the project target areas and 25 areas identified as the control. Splitting the sample between target and control areas has enabled the application of the Double Difference method of impacts evaluation.

6. The data for this assessment was collected through a combination of quantitative and qualitative methods, so as to provide the most accurate reading of the indicators. The

48

data quality concerning accuracy, reliability and validity was assured by a team of senior agriculturalists with a vast experience in administration of field surveys.

7. Adoption rate in the context of MOSAP is defined as the proportion of beneficiary farmers applying a given set of recommended, improved farming practices (IFP) per crop during the growing season. A farmer was considered an adopter in Year 1, 2, and 3 if he or she was found applying all three of the following practices grouped together in a bundle, that comprise the technology adoption indicators: (i) seed selection; (ii) seed placement i.e. the right amount of seeds per hole or the correct placement of cassava cuttings on the ridge; and (iii) the proper plant spacing, both between and within rows.

8. Crop yield and production data were collected through a combination of cutting and weighing methods together with the farmer reported harvest. The combination of the two methods is justified because it balances off the drawbacks of either one.

9. A set of qualitative data collection instruments was used primarily for the identification and classification of promising and best practices, promoted by the project. These include focus group interviews with farmers organized in associations, and the Discussion Oriented Self-assessment (DOSA) conducted together with the project’s office and field staff.

Key project achievements

Adoption of improved farming practices – technologies

10. The project’s target was set to reach a rate of at least 40% adoption of improved agricultural technology among project beneficiaries. The objective was met successfully, in quantitative terms, reaching 61.5% adoption at the end of the project.

11. The adoption of maize seed selection practices has been reportedly high since the first year of implementation. The final survey results show a 97.5% rate of adoption for seed selection in Huambo and Bié, but it reaches universal levels in Mungo and Londuimbali. In Bié, the districts of Chinguar and Andulo fall behind by only one percentage point. All other districts reported seed selection adoption rates well above 85%. Maize planting (sowing) produces very little adoption with only one in five farmers reporting the adoption of the recommended practices.

12. The adoption of cassava farming practices includes: the selection of the right size of cutting; placing the cutting on the ridge in the right position; plant spacing, both within and between rows; ensuring that the cutting has the right number of nodes; and checking its humidity content. On average, only one in three farmers applied the correct plant spacing. The adoption rates for the size of cassava cuttings is also low (37.9%), indicating variations on the type of vegetative material available for planting. The number of nodes has been adopted by a reasonably high proportion of farmers (59.6%). The data shows that the humidity content and mode of planting are quickly adopted by farmers, with the adoption rates situated above the 90% mark.

49

13. The adoption of seed selection practices on beans has reached almost universal rates in all three provinces, but sowing and plant spacing fall below the average target of 40% in Huambo and Bie. Farmers in Malanje have done better on bean sowing rates (47.2%), but plant spacing is below 6% adoption in all the districts.

14. The adoption of improved potato farming practices shows a clearly distinct and rather unusual pattern. As with the other crops, the adoption of seed selection practices has reached universal levels in a significant number of districts, particularly in Malanje. Potato planting methods have reached high rates of adoption in all three provinces with figures ranging from 64% in Bié to 91.1% in Malanje. The adoption of plant spacing practices in potato production is less problematic that in the other three crops. In Malanje, nearly 30% of the farmers have adopted the right spacing, whereas rates of 42.8% and 48.5% have been reached in Huambo and Bié respectively.

Factors influencing rates of adoption

15. Having a functional association with effective leadership has been a determinant for the adoption of IFP, as well for the overall success of the project. Both the quantitative and qualitative data provide supporting evidence for the fact that the project’s strategy of organizing farmers in associations has enhanced the provision of a comprehensive training package on Improved Farming Practices (IFP) as well as on leadership skills.

16. This report shows the distinctive effect of the project, regarding reaching out to farmers in the target areas through the associations. In 2012, not all farmers were keen to participate in agricultural organizations – a behavior which seemed to be determined by the perception of a lack of tangible benefits to be derived from the participation. The situation has changed in the target areas over the last four years. The proportion of villages with functional associations has grown from 44.5% in 2012 to 85.6% in 2015 in the target areas. This is consistent with the percentage of farmers with association membership, which rose from 27.9% in 2012 to 87.1% in 2015. This has had a positive influence on the adoption rates which grew from 34% in 2013 to the current level of 61.5%.

17. A Decision Tree Analysis (DTA) model was used to identify key variables that had a significant influence on the rate of adoption. Results have shown that, as in the previous DTA model, the training provider is a strong determinant of adoption. It is found that farmers who received training in Farmer Field Schools, in association with the local Agricultural Development Stations (EDA) extension officers, were highly likely to adopt the technology. Having previous knowledge of the improved farming practices was the second-highest predictor of adoption. The results show that farmers who had some knowledge of the improved agricultural practices before training were more likely to become full adopters. The innovation factor was also found to be significant. Farmers, who considered the IFP as an innovation to them, were highly likely to adopt the prescribed recommendations. Farmer-to-Farmer extension is beginning to play a role in

50

adoption. Farmers who have received the training on improved farming practices from other farmers (including FFS facilitators from a different village) had high adoption rates with a significant predictive value for the model.

Crop Production Indicators

18. The crop production index is the key performance indicator, which provides an aggregated measure of the increase in crop production in real terms. The 2011/12. The three-year index trend shows a progressive increase in production of maize, cassava, beans and potatoes. At the end of the project in 2015, the overall CPI was calculated at 166. However, CPI varied largely across individual crops.

Maize crop outcome indicators

19. The maize crop production Index at the end of the project is 142, which is primarily attributed to production increases in Huambo and Bié. In Malanje, despite the potential for high yields, maize cultivation remains limited to a few square meters per farmer and the production is sold for the most part as green mealies. The Index for the control areas is calculated at 102 representing a very modest growth from the year of the baseline. The difference of 40 percentage points between the target and control areas represents the impact of the project on maize production

20. Farmers in Huambo and Bié have not only recovered from years of poor crop yields, but they have experienced growth in real terms. In 2012, the mean yield was estimated at 0.43t/ha for farmers in both target and control areas. The current mean yields are estimated at 0.75t/ha for Huambo and 0.77 t/ha for the province of Bié, an increase of 53.6%. The average yield in control areas is calculated at 0.52t/ha., an increase of 20.5% over the same period. As a result, of the project’s intervention, farmers in the target areas of Huambo and Bié have harvested on average 142kg more maize hectare than their counterparts in the control areas – and this is the real impact of the project as far as maize yield is concerned.

Cassava crop outcome indicators

21. Due to the self-commitment of farmers in Malanje, the cassava crop production Index reached 151 at the end of the project. Farmers in control areas have also recovered from poor crop years, yielding a crop Index of 130. This performance results in a 21 percentage point difference between the two areas, which considered the impact of the project on cassava production. Although the impact is positive, this index level does not reflect the full potential of cassava farmers in the target areas of Malanje. As a semi-perennial crop, the project could not develop a comprehensive strategy to improve both the yields and production to regional standards within the limited timeframe of the project. And for the same reason, the efforts to improve cassava production was limited to the target areas in Malanje province, whereas in Huambo and Bié the crop maintained its marginal status for the entire life of the project.

51

22. The final assessment data for cassava has estimated the yield of 18.2t/ha – a 10.9% increase which was achieved only towards the end of the project. However, cassava yields have fluctuated greatly during the project across all the target areas with the district of Kiwaba Nzoji reporting a declining trend. A similar trend has been observed in control areas with yields maintaining close to the baseline value of 15t/ha. Due to the intervention of the project, farmers in the target areas have realized a real increase of 2.2t of cassava per hectare on average.

Bean crop outcome indicators

23. The 2015 bean crop Index is calculated at 223, with farmers in Malanje contributing the largest share of real growth. However, a bean crop production index of 166 in the control areas does indicate that farmers, in general, did make an effort to increase their production. The difference in the crop production indexes between the two areas is of 57 percentage points, representing in real terms, the impact of the project on beans production.

24. It is irrefutable that the increase in bean production in the target areas is a major consequence of the project’s intervention. The average area under cultivation per farmer increased from 0.47 ha in 2012 to 0.64 ha in 2015. The previous monitoring and evaluation reports indicate that many farmers became more seriously involved in bean cultivation for the first time in 2013.

25. During the project, farmers in the target areas of Huambo and the Bié not only recovered from the consecutive years of poor bean yields but they have also experienced a reasonable increase in real growth. At the end of the project, the mean bean yield is 0.45t/ha, an increase of 46.9% from the base year. Without the project intervention – i.e. in the control areas – the average bean yields is estimated at 0.37 t/ha, representing an increase of 25.6% from a weak base year. As a result of the project, farmers in the target areas have harvested on average 69.7Kg more beans per hectare than those in the control areas.

Potato crop outcome indicators

26. Potato growers have realized the highest real growth, with a crop production Index calculated at 394. This was also true for control areas where the Index is calculated at 576. Farmers in Huambo were largely responsible for the steep growth in potato production in both areas. In fact, it is worth noting that the real growth of potato production in Huambo is nearly twice as higher in the control areas as compared to target areas – an outcome that was already predicted in the midterm monitoring and evaluation performance surveys. The differential growth is 182 for the control areas, implying no real impact of the project on potato production. However, the proportion of farmers participating in potato production, as a result of the project, has doubled, contributing further to the high volume of production recorded in the target areas. Even so, this was not enough to outperform the good farmers in control areas who have brought large areas under cultivation.

52

27. On a rather positive note, potato yield in the target areas has grown from an average of 3.9t/ha in 2012 to 6.9t/ha in 2015, whereas in the control areas yields have grown from 3.2t/ha to 5.0t/ha over the same period. The overall differential impact of the project is 1.3t/ha –an incremental growth that would not have been achieved without the project.

Promising and Best Practices

28. Promising and best practices were identified through a combination of methods i.e. the PBP Classification Scale and the Discussion-Oriented Self-assessment framework (DOSA). The leading practices under review include: technical staff development; staffing levels to ensure adequate monitoring of field activities; timely and participatory planning of subprojects; women’s participation in project planning; delivery of extension services through farmer field schools; and strengthening organizational capacity through farmers training. The project’s model (design and implementation), though it is not a practice as such, was also included in the PBP assessment.

29. The PBP classification scale ranges from 2 to 10 points, whereby a 2-point scale means that a given practice is sufficiently innovative, and benefits have been identified. The highest level of 10 means that a particular practice has reached full maturity, it is deemed essential, and value has been proven.

30. As far as staffing requirements and project delivery are concerned, the PBP overall classification is 7.9. This indicates that the project has made resources available for skills development and enhance best technical practices. However, staffing levels were kept low, to the point of compromising the practical implementation of monitoring and evaluation activities. It did not reach the stage of a best practice but it is highly promising, and it is close to maturity.

Another high-rank practice includes the provision of training to technical staff to enable them to work more efficiently towards the strategic goals of the project. This practice earned the classification of 7.0.

31. Staffing levels, to allow a timely monitoring of field activities, was largely compromised during the project and therefore, the resulting PBP classification for this practice is 5.5. Staffing levels have affected the monitoring of the utilization of inputs distributed to farmers, a practice which is classified at 6.9. The monitoring of factors hampering the adoption of the improved farming practices carries the lowest classification score of 4.0.

32. On project programming practices, the classification of 8.1 was achieved for the efficient delivery of extension services through farmer field schools. Participatory identification and planning of projects were second with a classification of 7.0. The practice of making project management decisions based on results from regular

53

monitoring exercises began to develop late in the project cycle hence a low classification of 5.1.

33. The creation of active farmers associations is considered an important practice of the project, but it carries a low classification of 5.6. However, results of the focus group interviews indicate that there is potential to advance into a best practice.

34. The implementation of the project through farmer’s associations has made it possible for women to participate more in the planning of agricultural activities, together with their male counterparts. The fact that women took seats in the subprojects selection and approval committees, was considered as one of the best practices on gender awareness, earning a PBP classification of 6.9. But the project did not go very far regarding developing specific messages to improve women's farming practices, yielding a low PBP classification of 5.5. Nonetheless, there is agreement that more female-headed households had access to production resources than never before, a practice, which produces a PBP classification of 6.6.

35. With a PBP classification of 8.5, it is agreed that the project’s design and implementation model is sufficiently solid and consistent to be replicated in other parts of the country.

Concluding Remarks

Despite the variations in the crop production index across districts, the project’s target objectives were largely achieved. In fact, farmers have shown potential to reach higher goals.

36. The first two performance evaluation surveys already provided an indication that the project was on the right path to meet its target objectives. Looking at the 2015 crop data, one could say that the project management team was conservative in setting its goals – at least as far as crop production is concerned. Although smallholder crop production is somewhat dependent on the rainfall distribution, these results indicate clearly that this sector has the potential to reach much higher levels of production, provided that farmers have the right commercial incentives. Farmer skepticism about increasing crop production (cassava and potato in particular) beyond their current marketing capability was also noted and documented. It suggests that any attempt to increase the production of what is considered locally as a cash crop, with no provisions for a secure market outlet, is likely to yield poor results.

The project has achieved a high rate of adoption given a four-year implementation time frame. However, the regional pattern of adoption is highly pronounced, which calls for a review of the current approach.

37. The adoption of farming practices of 61.5 percent is well above the project target of 40%. However, one of the lessons that we take from monitoring the adoption rates over the years is that the adoption of a given IFP is determined primarily by the weight of

54

the crop in the farming system. This was very clear in the behavioral pattern of adoption of maize and cassava IFP within Malanje and Huambo respectively. Therefore, measuring the adoption rates for cassava in Huambo and Bié would make sense if the project had a particular strategy to promote the production of this crop in the target areas.

38. A strategy to encourage the production of crops should go beyond the provision of inputs; but include a service which supports the farmers to help them develop storage, processing, and overall marketing strategies. Farmers need to understand the competitive advantage of producing a new crop in order to adapt and adopt improved farming practices. Since it was not the case, cassava kept its marginal status in a maize dominated farming system, and vice versa. And this led to the second major lesson in adoption indicators: unless the project has a comprehensive strategy to promote “foreign” crop within a given farming framework, this crop should not be included in the training for the adoption of IFP.

The adoption of some but not all of the IFP will be maintained in the years ahead.

39. The adoption rates for the IFP has shown a somewhat consistent pattern over the past three years of project implementation. The 2015 final assessment data shows more clearly that the adoption of seed selection techniques for all crops (including the selection of cassava cuttings) has the potential to reach almost universal rates. It is most certain that the newly acquired practices of seed selection will be maintained long after the phase-out project stage. But the same cannot be said for the adoption of sowing (or seed placement) and plant spacing practices.

The question regarding whether local farmers will be able to maintain the association function after the project is complete, remains to be addressed. However, the presence of strong leaders and the absence of multiple hierarchy structures at village level are necessary conditions for the associations to remain functional

40. Perhaps the most important outcome of this project is that farmers in the project target areas have demonstrated the potential to raise crop productivity far beyond the historically low levels, as long as they are provided with the appropriate supportive infrastructure and incentives. But the development of organizational and leadership skills is as important as the supplying the production inputs to farmers. Effective membership has grown substantially in the last three years in response to the increased supply of production inputs – predicted in 2012. This type of supply-driven participation prompts one to question the sustainability of the associations in the absence of the project.

41. The organizational and leadership skills training that farmers received in 2013 should continue, in order to reach members of the associations that did not have the opportunity to participate. Refresher sessions could be introduced in Farmer Field Schools (FFS) during the lean season. This may not be a sufficient condition to ensure the associations continue to function, but a necessary element, reinforcing that the association is not just a platform through which farm inputs are provided. In addition, the number of members per association is critical for ensuring functional sustainability.

55

Associations with strong leadership have limited the number of members to levels between 30 and 50 and they have achieved better results than the larger associations. In the event of a second project, MOSAP-2, these associations should be given an opportunity to continue strengthening the internal organizational capacity and generate the resources to enable further growth.

The relationship between project design and impact evaluation, developed by the project is a lesson worth documenting and recommending for adoption across the sector in Angola.

42. One of the most important attributes one of MOSAP, which deserves the classification of a Best Practice, is the relationship the project has established between the baseline survey and the post-project evaluation. The project implementation design took into serious consideration the need to conduct a baseline study and an end of project impact evaluation, which would assess, not only the performance of the impact indicators over time, but it also attempt to provide answers to questions like what would the productivity outcome of farmers in the target areas have been without a structured intervention? This approach to project evaluation implies, in principle, that without the intervention the impact of the variables (productivity and production) would be the same for farmers in both control and targets areas. This dimension of evaluation is often taken for granted in agricultural development projects in Angola. It is unquestionable that MOSAP provides a valuable experience worth recommending for wide scale adoption in the agricultural sector in this country.

The monitoring and evaluation of process, and impact indicators were one of the main challenges of the project. Low staffing levels and reduced capacity to align the planning and implementation of subprojects together with the M&E framework were the main limitations, leading to an inadequate flow of data to inform programming decisions.

43. The three years of field data collected from standard survey design does not produce the best set of data for an accurate monitoring of crop productivity indicators. The design methodology has to take into account the fact that the four crops promoted by the project follow different growing cycles. Also, with the probable exception of some areas in Malanje, most of the beans and potato crops are produced in two growing seasons. Depending on the rainfall pattern, the crop grown in the two seasons produce different yields, and much of the second season crop data is lost.

44. The 2015 survey provides more accurate estimates of potato yields because the researchers made a unilateral decision to conduct the field data collection in two stages. This approach is appropriate for the assessment of by-seasonal crops, but is not the case with cassava, as the monitoring of production indicators poses a much greater challenge. The fact that there is not a single cassava harvest season and uprooting follows specific consumption and marketing needs, makes crop harvest estimates less accurate. Therefore, the introduction of a cassava harvest diary needs to be introduced to improve measurement of production indicators.

56

Annex 6. Stakeholder Workshop Report and Results

(Report of stakeholder workshop in identifying lessons from MOSAP, translated from the Portuguese)

1 Positive aspects of MOSAP

1. MOSAP was the first community investment project in the areas where it was implemented and had several positive impacts:

• Increased the efficiency of using of time for productive purposes by reduction of time spent on some activities such as grinding maize mail as the project provided grinding meals;

• Increased in the cultivated area;• introduction of more effective technologies of cultivation and therefore increase

the productivity and quality of the products; • development and consolidation of associations;• development of planning capacity of associations (preparation of project

proposals for funding, simplified feasibility studies among other activities); • stimulated new development initiatives and emergency of new projects;• emergency of some activities in cooperation with other associations• increased capacity for negotiations with public support, improved responsibility• improved knowledge of the market for acquisition of inputs and

commercialization such as PAPAGRO

2. In summary the project has achieved the following:

• increased income of the beneficiaries;• increased the technical empowerment of the beneficiary farmers• increased the capacity development of the associations and beneficiary farmers• development of associations • increased social capital

3. It will up to the SADCP project to capitalize and increase the positive impacts achieved under MOSAP.

2 Negative signs and delayed opportunities

a) Negative signs

4. MOSAP was executed under specific conditions which explain the negative points mentioned below which are worth to register:

• Lack of previous experiences with producer associations in the villages;

57

• Lack of field technical team;• Slow start up of the project of the project and unable to achieve any result before

changes in 2013;• Lack of services providers to supply goods such as grinding mills and water

pumps, animals and inputs such as seeds and fertilizers and limitation of the government programs to complement the project with its own resources.

5. However, some of the project norms and strategies may have provided negative signs and stimulate negative behavior which contributed to the difficulties observed by technicians and leaders.

6. The associations had very little participation in the preparation of the sub-projects and most instances was enough to present a desire (“we want grinding mill”), without a requirement to full formulate the project with a budget, implementation timeframe, objectives and targets, projection of revenues and costs of production. Could be said that the association were not stimulated to do this exercise.

7. During the implementation of the sub-projects, the associations had very little participation. For most items (inputs and equipment), meanly those that were procured at the central level, the associations were not asked to look for their own suppliers, examine different options for procurement, etc. In some cases the associations never got in contact with the supplier, which create problems for maintenance when the equipment required technical assistance which create complete dependence on MOSAP intermediation.

8. Also during implementation, the associations were not stimulated to organize and take part on the utilization of resources and look for solutions that would decrease the cost. This was lost opportunity to participate in the management of a project that would have enable them to participate in the registration of revenues and learn that they have to be accountable.

9. The project also had a very small menu of sub-projects which in some cases limited the support that the community was looking for from the project. In most cases the project had informed that the sub-project that they were looking for is not part of the list of the sub-projects support by MOSAP.

10. In some cases the beneficiaries reduced the cultivated area due to difficulties in obtaining seeds and fertilizers at competitive prices, or they had reduced productivity as the inputs arrived late. In addition the distribution of seeds and fertilizers by the government bringing dependency and insecurity: the associations were not certain if they will continue receiving the inputs, not sure about the criteria used to determine the quantities for each associations and even if they want to pay not sure how payment could be made to government.

11. With the objective to help the associations, MOSAP made a tremendous effort to resolve the problem faced by associations (repair of equipment, acquisition of spare parts, mobilization of technical staff and suppliers to resolve the problem). In most cases the

58

project effort was critical for the success of the investment. But there was no clear guidelines in the project to the beneficiaries to solicit this support.

12. This form of support in Brasil is called “dar um jeitinho-- giving a favor” which in general increase the recognition of the people responsible for the project but does not increase the growth of the association.

b) Training: Some delayed opportunities

13. The training provided by MOSAP may have left some opportunities:

14. First there is some disconnect with various capacity building initiatives. In the Farmers Field Schools there is training, at the startup of the project there were other trainings and also many other trainings provided by private consultants.

15. It is difficult to understand the training provided at various stages with various service providers, how they link together. It is difficult to understand from the project this link and how it contributed to the consolidation of the associations.

16. On the other side, apparently the training is done using traditional training methods, teaching, a trainer providing training without knowledge of the real difficulties of the target group and without promoting exchange among the groups. In this way it was lost an opportunity to create space for cooperation between associations for knowledge exchange.

17. Last MOSAP paid for the transport of participants. While the groups are low income this may create dependency and if not paid do not participate.

59

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

(MINAGRI Comments)

1. The ICR reflects well the main challenges faced by the project since its inception to completion, including the results achieved. We would like to add the following lessons learned as will be important for the implementation of SADCP project:

2. Improved Farmers organization through groups or associations increased provision of assistance to smallholder and access to inputs, but continued assistance is required for increased sustainability. It is a known fact that effective membership in the farmers organizations and associations has grown substantially in the last three years in response to the increased project assistance specifically on access to production inputs but the farmers associations seem to be still in development stage and much more work is required, particularly to ensure that all the members see the association as theirs. Farmers have begun to realize that the development of organizational and leadership skills is as important as access to the production inputs. In addition, the presence of strong leaders and the absence of multiple hierarchy structures at village level are necessary conditions for establishing and mainlining functional associations.

3. Development of smallholder capacity is key for increased sustainability. The development of new capacity was the key to the success and sustainability of the actions supported by the project. It allowed the strengthening of small farmers associations, incorporate technological innovations, start new activities, manage production units in a more efficient way and helped to reach markets in better conditions, using, for such, new capabilities that remain after the project end. The experience demonstrates that only associations consisting of small strong and solid agricultural farmers can identify their own problems and solve them and, on the other hand, no poverty can be reversed without generating income in a sustainable manner

4. Good communication and participatory planning was key for smooth implementation. Meetings organized by the project implementation team with different stakeholders involved in the implementation of the project during the implementation of the project proved to be valuable opportunities for reflection by the various actors on their own experiences and learning. People involved could speak openly about their concerns; raise problems and dilemmas they were facing; and discussions emerged. Several meetings were undertaken in order to identify communication bottlenecks in the documentation system. These meetings also helped identify who was responsible for the different information flows and established necessary frequency and deadlines for report submission. By discussing and planning for these issues, the documentation system was more likely to operate smoothly. Also, promoting networking by exchanging knowledge and information, thereby increasing cooperation among different organizations. The sharing of the information makes all involved in the project realize their participation and feedback is indispensable in making needed improvements.

60

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

(IFAD Comments and Lessons Learned)

1. The two IFAD and World Bank missions held consultations and interactions to learn more about MOSAP’s achievements and lessons that could serve in shaping the SAMAP design and fine-tuning the design of SADCP. In addition, A VC meeting was organized between the IFAD’s Country Director, Ms. Abla Benhammouche and the WB Senior Agriculture Specialist, TTL, Mr. Aniceto Bila.

2. The implementation of MOSAP was slow initially, with field activities having started around mid-2012, some 2.5 years after effectiveness. Nonetheless, its implementation progress has been encouraging since February 2012, when a new coordinator and project team was designated to take over the project coordination and implementation.

3. A field visit was organized in the three provinces where MOSAP was implemented, i.e. Malanje, Bié and Huambo. The field visit lasted full three days and included meetings with IDA, Municipal Administrators, Director Provincial Department of Agriculture, Department of Agricultural Extension and a large number of beneficiaries at several locations in different municipalities. These meetings and field visits were extremely valuable in reviewing the achievements under MOSAP as well as drawing important lessons for the current design of the IFAD’s financed SAMAP (follow up project under preparation by IFAD).

4. The main achievements were noted in the strengthening of the capacity of famers’ groups and associations, the strengthening of IDA technical staff through provision of training in Participatory planning, Farmers’ Field Schools (FFS) and support to investment for farmers through provision of financial resources to 282 sub-projects benefiting 14,439 smallholder farmers. The activities financed include: Animal traction; support to mechanized land preparation; and agricultural processing mainly grinding mills.

5. The main results include: Increased in productivity in most crops supported under the project, increased agro-processing of cassava and increased income of beneficiaries.

6. The main challenges after the project completion remain with the lack of exit strategy, poor maintenance of the equipment provided to producers mainly grinding mills, lack of veterinary services in places where farm animals have been provided, weak ownership with farm associations and therefore overall sustainability.

7. Looking forward, for the design of SAMAP and SADCP projects, important lessons should be considered, based on the preliminary findings of the joint WB and IFAD ICR mission. Based on the MOSAP impact assessment report, PCR and

61

discussions held with the MOSAP management team, the Government, IFAD, the World Bank and other stakeholders, a number of important lessons learned emerged that are particularly relevant for the design of the SAMAP and SADCP projects.

Lessons learned from MOSAP

8. Performance of the PCU. Effective project implementation requires good project management and intensive supervision, including functioning and sound monitoring evaluation system. MOSAP was not able to deliver until 2013, when the management team was replaced. Managerial and organizational problems seriously undermine the implementation of MOSAP. For this reason, building coordination and management capacity is crucial for delivering services of value to the beneficiaries.

9. It is critical that contracts allow for regular review of performance and opportunities for renewal or termination, according to performance. There has also been insufficient collaboration and learning between projects and till recently, inadequate follow-up and support from IFAD.

10. Special attention should be devoted to improved monitoring and evaluation systems that facilitate and document progress towards sustainability. Effective M&E of field operations supports sustainability in multiple ways. First and foremost, it identifies strengths and weaknesses in project implementation, which makes possible needed adjustments in response to changes in the operating environment. Second, it can highlight potential linkages among individual project components that enhance the overall impact of programme interventions. Finally, it can establish reliable indicators of project sustainability, which is a critical step in gauging progress towards key benchmarks and formulating effective exit strategy.

11. Opportunities for knowledge management and learning are lost when M&E is not used as a project management tool. Overall, Project M&E systems was not used as project management tools. While the project regularly tracked outputs (training, infrastructure projects, agricultural demonstrations, FFS, etc.), they were not actively engaged in tracking outcomes or impact of project activities. Thus opportunities for knowledge management and learning were often lost.

12. Exit Strategy. Among the most common and detrimental findings regarding the sustainability of MOSAP is the prevailing lack of an exit strategy in the design document and implementation plan. SAMAP (MOSAP follow up project under preparation by IFAD) and SADC should lay out a comprehensive exit strategy linked to IFAD’s Angola COSOP and WB Country Strategy framework. The Exit Strategy should include identification of institutional capacity-building needs, a description of benchmarks for measuring progress towards project objectives, specific action plans for achieving the benchmarks, and a reasonably flexible timeline within which benchmarks will be met. It is also important that an exit strategy draw on a range of specific exit criteria. In addition, in order to put SAMAP and SADCP in the path of sustainability, the designs should incorporate context-specific risk management measures into all activities.

62

13. Sustainability. Sustainability relates to the likelihood that the benefit streams generated by the project would continue after the project closure. While the project embarked on important initiatives there are a number of issues and concerns with regard to sustainability. One of the main issues with sustainability is that the participatory approach has not been mainstreamed into government's regular development planning and budgeting processes: in other words, the efforts for identification of community needs based on a bottom-up approach are undertaken only when there is a "project". Furthermore, the rushed implementation in the latter years prevented MOSAP from giving enough time and attention to consolidate their support to farmer organizations and thus their sustainability. However, the seven year-period of implementation cannot be considered a reasonable time frame to ensure the sustainability of all the results produced by the project.

14. Despite the benefits and positive views by the communities as well as the government staff, the participatory development approach has still remained "project-centered" and has not been mainstreamed into government planning. in addition, a number of institutional and technical factors influenced the sustainability of MOSAP component 2 i.e. Agricultural Investments. Sustainability of sub-projects assisting farmers with agricultural production, processing and marketing should be ensured by promoting community ownership and by building capacity of farmer groups to operate and maintain equipment prior to hand-over.

15. In order to improve the SAMAP sustainability, the temptation to focus solely on the achievement of physical and financial targets should be avoided. As seen during the field visit, this has led to a failure to promote community ownership and to the premature granting of responsibility for equipment maintenance to beneficiaries. It is also important that the design and implementation of sub-project investment in agriculture be based on an analysis of the technical capacity of beneficiaries. This may help define capacity-building needs that will ensure that beneficiaries are able to adequately operate and maintain equipment’s once the project ends.

16. Capacity building. Institutional and organizational support is often mistakenly equated with training. Training is a part of institutional and organizational support but cannot be identified with it exclusively. Institutional and organizational support should be understood as a complex set of different activities, including the provision of incentives, equipment, infrastructure and training, and also the implementation of policy-dialogue activities aime at promoting an enabling environment for broader institutional innovations and organizational expansion to facilitate scaling up.

17. Within the SAMAP and SADCP capacity strengthening should be demand-driven, gradual and dynamic. Flexibility should always be integrated into project design in order to allow room for learning and adjusting to changing circumstances. Trained individuals need good leadership, support tools, equipment and operational budgets to enable them to perform within the established procedures (planning, reporting and accountability), rules and by-laws. However, even a farmer association, that has acquired the ability to plan,

63

implement and monitor its own development plan, may become frustrated and feel abandoned in the medium run if no financing is available to implement the plan.

18. Farmer Field School (FFS). Although the performance of FFS needs to be evaluated, they are considered to be the right extension methodology and it was found that the Master farmers as well as some members of the associations have grasped the techniques for different crops. FFS is an effective platform for farmer organization and empowerment, where smallholder farmers with a common interest can gain increased production, productivity and access to market. This platform served an important role for farmers both in terms of social and technical support. Farmers who participate in the first phase of training of trainers become the Master Farmers (FFS-MF) who facilitate or replicate the training to new farmers, thus achieving a geometric increase in the number of farm families with strengthened capacities.

19. The chances of effective FFS network to grow, can be strengthened if care is given in the SAMAP and SADCP to farmer-driven network development. Furthermore, in order to build on best practice of using farmers as extension agents whilst ensuring their sustainability post project, it is recommended that SADCP and SAMAP projects, should facilitate farmer promoters to become agricultural input suppliers at the village level. Within SAMAP, and SADCP outstanding farmers should be able to earn service fees in delivering technological advices to their client farmers.

20. Value Chain. SADCP and SAMAP projects, should support a demand pull strategy for agriculture. In order for farmers to meet consumer preferences with higher returns and improved household incomes, future initiatives should be value chain anchored as well as production-oriented. By focusing on both production and consumption, it is possible to work for a “double win” scenario where the emerging global market is taken into consideration.

21. Therefore, future operations in Angola should support market linkages and value chain approach. Marketing under the MOSAP project was touched upon very slightly, mostly in connection with the component 2. Analysis of market opportunities should be carried out before investing in production systems, and training on business and marketing aspects should complement production-oriented training. Long-term success requires not only improved on-farm productivity but also opportunities for farmers to have access to, and compete in, output markets. SADCP and SAMAP projects need to provide institutional support for various marketing activities at several levels, including assistance to farmer groups, members of groups or entrepreneurs for establishment and initial operation via credit of marketing associations of agricultural produce or purchase of inputs, private small and medium scale processing plants, equipped with storage facilities and quality testing. Development interventions that support market linkages and the value chain approach can stimulate diversification and investments that would lead to availability of market produce and the strengthening of rural enterprises. They would, further, bring closer the rural entrepreneurs to the market and would contribute to the development of efficient schemes that will incorporate technical support, financing, management and quality control.

64

22. Consolidation. There is a need to strengthen the MOSAP sustainability, through a consolidation phase to be considered in SADCP and SAMAP projects. This phase should include: (i) improving the capacity of farmer associations, to manage their own sub projects, while continuing to strengthen and supervise the recently established FFS which are at different stages. More advanced training in managerial competency should be provided, to strengthen farmer technical capacity, managerial competence; marketing skills and organizational structures.

23. The organizational and leadership skills training that farmers received should continue, in order to reach members of the associations that did not have the opportunity to participate. Refresher sessions could be introduced in Farmer Field Schools (FFS) during the lean season. This may not be a sufficient condition to ensure that the associations continue to function, but a necessary element, reinforcing that the association is not just a platform through which farm inputs are provided.

24. In addition, Sub-projects that largely consists of equipment requires close examination to increase sustainability. The SADCP and SAMAP should increase the support to farmers in preparation of business and management plans to increase sustainability of these investments.

25. All these activities are key to increase the sustainability of the project after closing.

65

Annex 9. List of Supporting Documents

1. Project Appraisal Document (July 2008)2. Restructuring paper (February, 2013)3. Operations Manual 4. IFAD Credit Agreement 5. Aide Memoire (October 2008)6. Aide Memoire (October 2009)7. Aide Memoire (Oct-Nov. 2010)8. Aide Memoire May/2010) 9. Aide Memoire (August 2011)10. Aide Memoire (Jan – Feb 2012)11. Aide Memoire (June 2012)12. Aide Memoire (April 2013)13. Project Progress Reports (2010-2015)14. Government ICR (2016)

Project and M&E studies

15. Baseline Survey (2012)16. Resumo das Lições Aprendidas e Sugestões para o SADCP (2015)17. End of Project Impact Evaluation Study (2015)18. Relatorio Fnal do Acordo IDA/MOSAP/FAO (2016)

66

MAP

67