Document of The World Bank

Report No: ICR00001421

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-34630)

ON AN

IDA CREDIT IN THE AMOUNT OF SDR 35.3 Million (USD 45.0 MILLION EQUIVALENT)

TO THE

REPUBLIC OF UGANDA

FOR THE

NATIONAL AGRICULTURAL ADVISORY SERVICES PROJECT

JUNE 24, 2010

Agricultural and Rural Development Unit Sustainable Development Department Country Cluster AFCE1 Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate in original PAD) Currency Unit = Uganda Shilling (USh.)

US$1.00 = USh. 1,350 US$1.3613 = SDR 1

FISCAL YEAR July 1 to June 30

ABBREVIATIONS AND ACRONYMS

AEP Agricultural Extension Project ARTP Agricultural Research and Training Project ATAAS Agricultural Technology and Agribusiness Advisory Services CAADP Comprehensive Africa Agricultural Development Program CBF Community Based Facilitator CBO Community Based Organization CDO Community Development Officer DANIDA Danish International Development Agency DCA Development Credit Agreement DFID Department for International Development DPs Development Partners DSIP Development Strategy and Investment Plan EMP Environment Management Plan EU European Union FF Farmer Forum FG Farmer Group FID Farmer Institutional Development FM Financial Management GAC Governance and anti-Corruption Arrangements GDP Gross Domestic Product GoU Government of Uganda HH Household HLFO Higher-Level Farmer Organization IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IDA International Development Agency IFAD International Fund for Agricultural Development IFPRI International Food Policy Research Institute ISFG Integrated Support to Farmers Groups ISR Implementation Status Report ITAD Information , Training and Agricultural Development KPI Key Performance Indicator LG Local Government M&E Monitoring and Evaluation MAAIF Ministry of Agriculture, Animal Industries and Fisheries MFI Microfinance Institution MIS Management Information System

MOU Memorandum of Understanding MTEF Medium-Term Expenditure Framework MTR Mid-Term Review NAADS National Agricultural Advisory Services NARO National Agricultural Research Orginasation NDP National Development Plan NGO Non-Governmental Agency NPV Net Percent Value PAD Project Appraisal Document PCC Parish Coordinating Committee PDO Project Development Objective PEAP Poverty Eradication Action Plan PFA Prosperity for All PLWA People Living with HIV/AIDS PM&E Participatory Monitoring and Evaluation PMA Plan for Modernization of Agriculture PMP Pest Management Plan PPF Project Preparation Facility PPP Public Private Partnerships PPR Post Procurement Review PWD People with Disability RDS Rural Development Strategy SFF Sub-county Farmer Forum SNC Sub-county National Agricultural Advisory Services Coordinator SPN Supervision SP Service Provider T&V Training and Visitation TDS Technology Development Site

Vice President: Obiageli Katryn Ezekwesili Country Director: John Murray McIntire Sector Manager: Karen Mcconnell Brooks

Task Team Leader: Christine E. Cornelius ICR Team Leader: Malathi S. Jayawickrama

REPUBLIC OF UGANDA NATIONAL AGRICULTURAL ADVISORY SERVICES

TABLE OF CONTENTS

DATA SHEET

A. Basic Information i 

B. Key Dates i 

C. Ratings Summary i 

D. Sector and Theme Codes ii 

E. Bank Staff ii 

F. Results Framework Analysis ii 

G. Ratings of Project Performance in ISRs v 

H. Restructuring (if any) v 

I. Disbursement Profile vi 

1.  PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1 

1.1 Context at Appraisal 1 1.2 Original Project Development Objectives (PDO) and Key Indicators 1 1.3 Revised PDO and Key Indicators, and reasons/justification. 2 1.4 Main Beneficiaries 2 1.5 Original Components 2 1.6 Revised Components (after restructuring) 3 1.7 Other significant changes 4 

2.  KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 4 

2.1 Project Preparation, Design and Quality at Entry 4 2.2 Implementation 5 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 9 2.4 Safeguard and Fiduciary Compliance 9 2.5 Post-completion Operation/Next Phase 11 

3.  ASSESSMENT OF OUTCOMES 12  

3.1 Relevance of Objectives, Design and Implementation 12 3.2 Achievement of Project Development Objectives 12 3.3 Efficiency 14 3.4 Justification of Overall Outcome Rating 14 3.5 Overarching Themes, Other Outcomes and Impacts 14 

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 15 4.  ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 16  

5.  ASSESSMENT OF BANK AND BORROWER PERFORMANCE 16  

5.1 Bank Performance 16 5.2 Borrower Performance 18 

6.  LESSONS LEARNED 19  

ANNEX 1. PROJECT COSTS AND FINANCING 20 

ANNEX 2: OUTPUT BY COMPONENTS 22 

ANNEX 2 APPENDIX 1. KEY PERFORMANCE INDICATORS 32 

ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS 33 

ANNEX 3 APPENDIX 1. AN ASSESSMENT OF THE HOUSEHOLD SURVEY DATA 42 

ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES 4 4 

ANNEX 5. BENEFICIARY SURVEY RESULTS 46 

ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS 54 

ANNEX 7. SUMMARY OF BORROWER’S ICR AND/OR COMMENTS ON DRAFT ICR 55 

ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 60 

ANNEX 9. LIST OF SUPPORTING DOCUMENTS 61 

i

A. Basic Information

Country: Uganda Project Name: National Agricultural Advisory Services Project

Project ID: P044695 L/C/TF Number(s): COFN-04440,IDA-34630,IDA-3463A

ICR Date: 06/25/2010 ICR Type: Core ICR

Lending Instrument: SIL Borrower: THE REPUBLIC OF UGANDA

Original Total Commitment:

XDR 35.3M Disbursed Amount: XDR 35.3M

Revised Amount: XDR 35.3M

Environmental Category: B

Implementing Agencies: NAADS Secretariat

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 06/15/1998 Effectiveness: 11/27/2001 11/27/2001

Appraisal: 09/11/2000 Restructuring(s): 06/30/2008

Approval: 02/15/2001 Mid-term Review: 06/15/2005 06/03/2005

Closing: 06/30/2008 12/31/2009 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Unsatisfactory

Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately SatisfactoryOverall Borrower Performance:

Moderately Satisfactory

ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Agricultural extension and research 70 70

Agricultural marketing and trade 7 7

Central government administration 23 23

Theme Code (as % of total Bank financing)

Gender 14 14

Infrastructure services for private sector development 14 14

Other rural development 29 29

Rural markets 29 29

Technology diffusion 14 14 E. Bank Staff

Positions At ICR At Approval

Vice President: Obiageli Katryn Ezekwesili Callisto E. Madavo

Country Director: John McIntire James W. Adams

Sector Manager: Karen Mcconnell Brooks Sushma Ganguly

Project Team Leader: Christine E. Cornelius David J. Nielson

ICR Team Leader: Malathi S. Jayawickrama

ICR Primary Author: Malathi S. Jayawickrama

Varun Kshirsagar F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) To assist poor male and female farmers to become aware of and to be able to adopt improved technology and management practices in their farming enterprises so as to

iii

enhance their productive efficiency, their economic welfare, and the sustainability of their farming operations. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Value of total agricultural income per capita of NAADS beneficiaries Value quantitative or Qualitative)

195,000 (2000 USh) 314,000 (2000 USh)

Date achieved 12/31/2004 12/31/2007 Comments (incl. % achievement)

Baseline data source and composition of housholds sampled is unknown (see Annex 3 Appendix 1)

Indicator 2 : Difference in value of total agricultural production per capita of NAADS beneficiaries and non-NAADS beneficiaries

Value quantitative or Qualitative)

Not measured

155,678 (2000 Ush) in 2007 for NAADS and 97,834 USh (2000 USh) for Non-NAADS

Date achieved 12/31/2001 12/31/2007 Comments (incl. % achievement)

The unadjusted difference is 57,843 (2000 USh), or 59.1 %. The adjusted difference is 51,350 (2000 USh) or 46.1%.

Indicator 3 : Value of total agricultural productive assets (equipment and livestock) per capita for NAADS beneficiaries

Value quantitative or Qualitative)

162,690 (2000 USh) 230,600 (2000 USh)

Date achieved 12/31/2004 12/31/2007 Comments (incl. % achievement)

The unadjusted difference is 67,910 (2000 USh) or 41.7%

Indicator 4 : Percentage of NAADS beneficiaries using improved seeds Value quantitative or Qualitative)

52.5% for NAADS and 23.5% for non-NAADS

68.3% for NAADS and 53.3% for non-NAADS

Date achieved 12/31/2004 12/31/2007 Comments IFPRI NAADS Impact Evaluation, 2007

iv

(incl. % achievement) Indicator 5 : Percentage of NAADS beneficiaries using improved breeds Value quantitative or Qualitative)

24.5% for NAADS and 11.3% for non-NAADS

30.9% for NAADS and 29.8% for non-NAADS

Date achieved 12/31/2004 12/31/2007 Comments (incl. % achievement)

IFPRI NAADS Impact Evaluation, 2007

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Greater access to information on improved technologies and practices

Value (quantitative or Qualitative)

no difference

NAADS members were 17.8 percent more likely to have access to information on how better to utilize crop production equipment and 13 percent more likely to have access to information on improving production practices

Date achieved 12/31/2001 12/31/2009 Comments (incl. % achievement)

Indicator 2 : Value (quantitative or Qualitative)

Date achieved Comments (incl. % achievement)

v

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 12/27/2001 Satisfactory Satisfactory 0.00 2 04/30/2002 Satisfactory Satisfactory 0.00 3 11/28/2002 Satisfactory Satisfactory 0.13 4 05/30/2003 Satisfactory Satisfactory 0.24 5 11/26/2003 Satisfactory Satisfactory 0.36 6 06/03/2004 Satisfactory Satisfactory 1.77 7 12/15/2004 Satisfactory Satisfactory 3.12 8 06/15/2005 Satisfactory Satisfactory 3.12 9 12/22/2005 Satisfactory Satisfactory 8.24

10 06/29/2006 Satisfactory Satisfactory 8.24 11 12/21/2006 Satisfactory Satisfactory 11.66 12 06/23/2007 Satisfactory Satisfactory 16.10 13 12/01/2007 Satisfactory Moderately Satisfactory 20.66 14 05/31/2008 Moderately Satisfactory Moderately Satisfactory 21.39 15 12/22/2008 Satisfactory Satisfactory 31.36 16 06/09/2009 Satisfactory Satisfactory 40.08 17 12/22/2009 Satisfactory Moderately Satisfactory 52.98

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

06/30/2008 N MS MS 21.39

NAADS was restructured to better align components to project activities and outputs; and to reflect implementation experience. Key changes made were: (i) refining of KPIs to reflect the new component structure; (ii) reallocation of project cost savings of about 20 percent to several categories of expenditure including the advisory services grants; (iii) adoption of simplified community-based procurement procedures; (iv) increasing the special account ceiling from

vi

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

US$6m to US$9m; and (v) extending the project closing date by eighteen months to December 31, 2009.

I. Disbursement Profile

1

1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.1 Context at Appraisal

1. Uganda’s GDP grew steadily at over 6 percent per year in the 1990s compared to 3 percent in the 1980s, yet, in 2000, about 35 percent of the national population and 38 percent of the rural population lived in poverty and depended primarily on subsistence farming (Poverty Reduction Strategy Credit Progress Report 2001, Poverty Headcount Index). Annual agricultural growth in the 1990s surpassed 4 percent boosted by Uganda’s economic liberalization and complementary institutional reforms, including in the agricultural sector. Key factors in agricultural growth were area expansion and improvements in farmers’ incentives to produce. Despite these advances, many of the rural poor did not benefit from growth and remained outside the monetary economy. Food crop production accounted for at least 65 percent of agricultural GDP, and agriculture was characterized by low productivity. Sustained growth, rural economic transformation, and poverty reduction were thus linked to advances in agriculture--realizing higher productivity and shifting production from low-value staples to higher-valued products.

In response to these challenges, the Government of Uganda (GoU) developed its Plan for Modernization of Agriculture (PMA), to transform agriculture from subsistence to commercial farming. The analysis underlying the PMA had suggested that Uganda’s low agricultural productivity was partly due to the lack of an adequate agricultural technology system. Farmers’ needs were not sufficiently driving research and extension efforts (causing low relevance), while the know-how and the technologies being produced, even when relevant were not being widely taken up by farmers (suggesting lack of effectiveness in the transfer of technologies). Hence, the PMA’s strategy was to enhance the relevance and efficiency of agricultural research and extension. Efforts to address agricultural research were pursued under the Agricultural Research and Training Projects I & II (ARTP I & II). GoU requested the Bank’s help in 1997 to design a different kind of extension program—one that would be more responsive to farmers’ needs. At that time, the Bank was financing the Agricultural Extension Project (AEP), which was based on the Training and Visitation (T&V) approach, and was due to close on June 30, 1998. The Bank invested nearly three years examining the lessons learned and working with counterparts in Uganda and other Development Partners (DPs) to meet Uganda’s request for a more innovative approach. The National Agricultural Advisory Services (NAADS) program, prepared by a GoU Extension Task Force, was one of five national programs identified to execute the PMA. NAADS’ fundamental aim was to develop a decentralized, farmer-owned, demand-driven, and pluralistic extension system. The entire Program was to span twenty-five years. Its first phase, NAADS (IDA US$45 million, of a total US$108 million pledged by DPs), covered seven years (2001-07) and was approved on February 15, 2001. Key features of the program were codified in the NAADS Act, a national piece of legislation passed by Parliament in 2001 that created legally binding provisions of the program, and was included in project design as a disbursement condition.

1.2 Original Project Development Objectives (PDO) and Key Indicators The PDO: to assist poor male and female farmers to become aware of and to be able to adopt improved technology and management practices in their farming enterprises so as to enhance their productive efficiency, their economic welfare, and the sustainability of their farming operations.

2

The Key Performance Indicators (KPIs): (i) the number of male and female farmers and the percent of client farmers of NAADS who are adopting improved technologies and management practices as a result of NAADS; (ii) inventory of measures taken by male and female farmers as a result of NAADS to improve the economic outcomes and the environmental sustainability of their farming operations; (iii) men and women farmers’ own opinions about their own productive and economic progress each year; (iv) productive and economic efficiency of male and female farmers participating in NAADS (raw farm-level and aggregate data and corresponding descriptive statistics); and (v) cash income earned by farm families participating in NAADS (raw farm-level and aggregate data and corresponding descriptive statistics).1

1.3 Revised PDO and Key Indicators, and reasons/justification. NAADS was restructured (see Project Paper dated June 30, 2008 approved by the Board). The restructuring included changes in the component structure and intermediate indicators to: (i) better align components to project activities and outputs; and (ii) to reflect implementation experience, which had shown the need for greater emphasis on certain elements (i.e. agroprocessing) of the project design than originally envisaged. These changes represented a realignment of project activities and did not alter the PDO. The KPIs were refined to reflect the new component structure, but not notably changed in substance. The Revised KPIs (see updated Results Framework, Project Paper Annex 1): (i) Measures of productivity in farm enterprises of NAADS participants exceed Ugandan averages, measured by: (a) value of total agricultural income per capita of NAADS beneficiaries; and (b) difference in value of total agricultural production per capita of NAADS beneficiaries and non-NAADS beneficiaries; (ii) Measures of household well being improve over time for the majority of participating farmers, measured by: (c) value of total agricultural productive assets (equipment and livestock) for NAADS beneficiaries; and (iii) Number of men and women farmers and percent of client farmers who are adopting improved technologies and management practices as a result of NAADS, measured by: (d) percentage of NAADS beneficiaries using improved seeds; and (e) percentage of NAADS beneficiaries using improved breeds.

1.4 Main Beneficiaries These were: (1) the public sector, through a more efficient agricultural advisory service; (2) farmer groups (FGs) and farmers, through an advisory service that was more relevant and responsive to helping them raise productivity, participate in markets, and raise incomes in a sustainable way; (3) the private sector, through agricultural growth and employment opportunities in rural areas as service providers under NAADS and in the re-invigorated private sector; (4) consumers, from greater and more reliable food production; and (5) rural communities and local governments, from greater ownership of local programs and higher growth in rural areas. While NAADS was anticipated to benefit a variety of actors, its main clients were relatively poor farmers, who were expected to capture a large proportion of benefits. 1.5 Original Components Component 1: Advisory and Information Services to Farmers (US$76.4m) sought to support initiatives by men and women farmers working in groups with their sub-county governments to contract agricultural advisory services. Matching grants were to be channeled from the national level of government through the districts for farmers (through farmers’ fora and local governments) to use in

1 The KPIs in the PAD text, PAD Annex 1 and the DCA differ. These KPIs are from the PAD text. They represent the same outcome areas as in the DCA but are more specific on listing the measures to be used to evaluate the PDO.

3

financing such contracts. Services to be contracted included program orientation and FG mobilization, participatory planning, advisory services, and information and communications. Component 2: Technology Development and Linkages with Markets. (US$7.0m) aimed to foster strong linkages among farmers, farm advisers, and researchers, and between farmers and markets, by making funds for technical support available at the district level to farmers and their farm advisors to contract the services of researchers and others with relevant expertise to work with farmers in farmers’ fields on specific technology, market development and adaptation. Component 3: Quality Assurance - Regulations and Technical Auditing of Service Providers (US$1.5m) was to support the NAADS Board and Secretariat, in conjunction with MAAIF, to set standards and provide a regulatory framework for service providers, by setting and enforcing standards for qualification and performance, and developing a model contract. Component 4:Private Sector Institutional Development (US$4.8m) aimed to set up a program to assist establishments to become eligible for award of contracts to provide services to farmers within the NAADS program. Specific activities included local service provider development and support to national representative organizations. The program was also to provide a package of benefits including training, to allow existing public sector extension workers to transition to employment in the private sector. Component 5:Program Management and Monitoring (US$18.3m) aimed to establish and support institutional entities at the national, district and sub-county levels of government to administer the program. At national level, this included setting up and maintaining the NAADS Board and Secretariat. At local government levels, it included help to district and sub-county NAADS coordinators to facilitate the program’s participatory planning process. In addition to supporting the co-ordination, financial management and reporting, a Management Information System (MIS) was to be set up to monitor the program, conduct a baseline survey and gather data for impact evaluation.

1.6 Revised Components (after restructuring)2

Component 1: Farmer Institutional Development (FID). Provision of training and capacity building services for: (a) program orientation and group mobilization for farmers; (b) participatory planning between farmers and service providers, including: on the terms of reference and annual work programs for future activities of farm advisors; and on on-going field activities of farm advisors; and (c) carrying out training events for farmers, farm advisors and district and sub-county officials. Component 2: Advisory and Information Services to Farmers. Provision of technical, advisory and information services for: (a) carrying out of farm advisory services; (b) information and communications services to FGs; and (c) on-farm adaptation and testing of agricultural technologies. Component 3: Agribusiness Development and Market Linkages. Implementation of technical support programs for: (a) facilitating partnerships between market actors and farmers; (b) market research and information; and (c) developing linkages between farmers and input/output markets. Component 4: Local Service Provider Institutional Capacity Development. Provision of technical advisory services to aid individuals and institutions to provide agricultural advisory and farmer

2 See Annex 1 for Project Costs by Component before and after restructuring. See Annex 2 for details on how the new components relate to the original components.

4

institutional development services under the NAADS program, including provision of training to public sector extension workers to acquire skills for work in the private sector. Component 5: Planning, Monitoring/Quality Assurance and Evaluation. Provision of technical advisory services and support to assist the NAADS Secretariat to: (a) develop and adopt minimum standards for regulating service providers, including setting standards for their qualifications and performance; (b) develop model contracts of services to be entered into with FGs; (c) conduct technical audits of service providers’ activities; (d) monitor the NAADS program; and (e) conduct baseline surveys and data gathering procedures for impact evaluation. Component 6: Project Management and Coordination. Provision of goods, equipment, technical advisory services and training for the operation of the NAADS Secretariat at the national level, and of the NAADS coordinators at the district and sub-county levels to enable them to conduct their activities of facilitation, coordination, financial management and reporting, auditing of financial flows, oversight of service contracts, and setting up the MIS.

1.7 Other significant changes The restructuring also allowed to: (a) reallocate project cost savings of about 20 percent to several categories of expenditure including the advisory services grants; (b) adopt simplified community-based procurement procedures, which had not been available at the time of project approval in 2001; (c) increase the special account ceiling from US$6 million to US$9 million; and (d) extend the project closing date by eighteen months to December 31, 2009, to accommodate initial slow disbursements. The Development Credit Agreement was amended on July 11, 2008.

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES

2.1 Project Preparation, Design and Quality at Entry NAADS represented the first deliberate application in the Africa Region of a new set of guiding principles for designing agricultural extension programs. It drew on lessons from previous extension experiences, globally and in Uganda. In Uganda, the predecessor, AEP, was roundly criticized for its centralized and mechanistic approach, its failure to empower farmers, and its exclusive reliance on public sector extension agents, high cost, and mixed record of benefits. NAADS’ basic principles were pluralism in the delivery of services, farmer voice in choice and evaluation of service providers, cost-sharing, decentralized responsibility for extension, and public sector financing. These principles were consistent with approaches also being followed in several successful extension programs throughout the world, especially in Latin America. Project preparation was wide-ranging and participatory, and NAADS was generally viewed enthusiastically by most Ugandan stakeholders as a system that would empower farmers to become more productive and to transform their livelihoods. Preparation spanned several years and included meetings in districts, sub-counties and at village level with farmers, local government officials, local agribusiness representatives and NGOs. In Kampala, it covered several 2 to 3-week working sessions with DPs, civil society, GoU officials, NGOs and farmer representatives. In GoU, there was a core team solidly committed to the NAADS principles although support outside this base was mixed. The then Minister of Agriculture (Uganda’s Vice President) strongly supported an alternative approach to T&V. Others with recognized interest in the status quo, however, were less enthusiastic and expressed reservations. During NAADS negotiations, GoU agreed to a

5

conversion of the existing extension service as NAADS rolled out to create arrangements consistent with the new system. This was referred to as “delayering,” and it was envisaged to involve the retrenchment of several thousand extension workers who would receive a package of benefits including voluntary training to establish businesses as private extension service providers under NAADS and with other clients. In hindsight, and with the benefit of subsequent experience with retrenchment programs in other sectors, one might conclude that the expectations that delayering would unfold smoothly and on schedule were unrealistic. The NAADS design acknowledged the difficulties in developing the capacity and increasing the willingness of NGOs and the private sector to address a wide range of agricultural technology issues and related services in rural Uganda. Previous work had identified the private sector’s limited involvement in agricultural sector service delivery (including the AEP ICR, which recognized this as affecting outcomes), and the inadequate number of competent service providers available at national, district and sub-county levels relative to the demand that NAADS was expected to generate.3 The design included a training component to increase capacity and numbers of private service providers, but its size was modest and this component experienced difficulties in implementation. When the expected flow of service providers from the public sector into the private sector under delayering was deferred, the constraints on the supply side of service provision became limiting. The supervision process agreed to in the Memorandum of Understanding (MOU) reflected recognition that this was a program (rather than a project) supported by a consortium of partners.4

This approach had the advantage that the Bank showed flexibility in adapting to the expressed wishes of the client and partners. It also had the disadvantage that not all of the normal processes of Bank project supervision were followed, such as individual and dedicated Aide Memoires drawn up by Bank teams at the end of twice-yearly supervision missions and transmitted to the GOU. Implementation support took place continuously through consultation and coordination between and among the project staff and field-based representatives of the DPs including the World Bank. Supervision normally comprised an annual high-level joint DP-GoU meeting preceded by a one or two- day review of field activities. Implementation Status Reports (ISRs) were reported to the Bank’s management. On the GoU side, interaction was primarily with the NAADS Secretariat, supplemented by regular discussions with the Ministry of Finance in the context of the annual budget allocations. MAAIF was less involved in the discussions, since NAADS was a parastatal entity, and hence possibilities to address issues of coordination with the legacy extension service were not fully developed. The joint supervision strategy resulted in a likely diminution of attention to deficiencies in implementation that might have received greater prominence had the Bank’s template for supervision been adhered to strictly; i.e., the evident weakness in monitoring and evaluation (M&E) that was never adequately addressed within the project’s implementation systems. In the joint supervision strategy, all contributors to the basket funding mechanism had approximately equal weight in evaluating implementation performance and identifying issues. This approach may have created risks, since some of the smaller donors did not have significant experience in implementing large and demanding programs.

2.2 Implementation

3 AEP ICR July 2000, p. 10. Also see the NAADS Pre-appraisal document, August 24, 2000, p. 14. 4 The MOU was not legally binding but rather set out the working modalities between the signatories. It was signed in October 2001 between GoU and Participating Partners--IDA, the International Fund for Agricultural Development (IFAD), Ireland Aid, Department for International Development (DFID), the Netherlands, the European Union (EU) and the Danish International Development Agency (DANIDA).

6

Project implementation and performance can be divided into two periods: (i) from approval until September 2007, when the overall experience was quite positive; and (ii) from September 2007 until closing, when NAADS came under significant stress associated with domestic political developments. (i) Project Implementation and performance from approval until September 2007

As mentioned above, the NAADS design was innovative and therefore involved difficult implementation choices. Despite this, prudent implementation decisions led to strongly favorable outcomes until 2007 (see Section 3). For example, under original Component 1 (Advisory and Information Services to Farmers), NAADS mobilized and strengthened FGs in order to allow farmers to better articulate their needs for effective service delivery. By 2007, NAADS had provided services to 770,300 farmers in 38,515 FGs, and approximately 500 sub-counties had issued over 5000 service provision contracts by 2007/08 (see Annex 2 for details). Service delivery introduced improved technologies and the introduction of market oriented, high value crops. NAADS also focused on linkages to markets, facilitating FGs in establishing contractual arrangements with agroprocessors and other actors up the value chain. By 2007, NAADS members experienced a 27.4 percent increase in real wealth per capita compared with non-members (see Annex 3 for details). Medium-Term Expenditure Framework (MTEF) budget ceilings and delayed release of funds constrained the NAADS rollout and delivery of benefits to a large number of farmers. The GOU maintained macroeconomic stability through strict management of fiscal relations. Although resources were available from IDA and other partners for the NAADS program, they could not be released until the GOU made space in the budget allocations. Within the allocated ceiling the GOU chose first to draw down grant funds from bilateral partners, and only later to draw on credits. By the time of the MTR, only about 6 percent of total IDA/IFAD credit amounts had been disbursed. Despite the slow disbursement of IDA funds, NAADS expanded to close to the expected geographical coverage at appraisal, although the planned intensity of coverage (the number of farmers reached) within districts was not attained at the time of the MTR. The timeliness of flow of funds remained a problem throughout the period of implementation. Releases was not timed well with the agricultural seasons, and this often affected district and sub-county work plans and implementation activities, such as farmer training and technology development (Value for Money Audit Report, December 2008). Although MTEF ceilings constrained expansion, overall implementation as reported in the ISRs continued to be satisfactory. NAADS’ dynamic leadership and its many innovations (such as involving private sector partners and harmonizing with other agricultural projects) allowed the project to show the potential of the key concepts embedded in its design. There was an enthusiastic response from both existing and newly-formed FGs eager to accept training, and the project did well in empowering farmers, i.e. by creating awareness, and building their capacities to demand and use advice and technologies and take control over funds and processes in their own development (2003/04 NAADS Annual Report).

NAADS made substantial investments in capacity of farmers to articulate their needs, but was slow in assuring the supply side of service provision. The program suffered from lack of clarity on what kind of training was required and who should provide it. The original design was for NAADS to provide basic skills training on teaching approaches and on financial and business management, with the assumption that former public extension agents would already have technical knowledge and/or that private experts would respond to opportunities under the program. The lagging supply side and slow delayering (see below) resulted in a shortage of trained extension workers with relevant skills to meet farmers’ demands (Value for Money Audit Report, 2008, p. 22). The GoU implemented the agreed delayering, which was a legal covenant, at the end of NAADS and to an extent other than originally envisaged. Many MAAIF extension staff remained on the

7

GoU’s (MAAIF’s) payroll, many as NAADS sub-county coordinators. The remaining extension agents had limited funds for mobility and hence could not interact well with farmers, resulting in an ineffective parallel structure. The perceived duplication was cited as one of the reasons for the President announcing a suspension of NAADS in 2007. The issue of duplication was not straightforward, however, since the commodity focus of the NAADS program provided little attention to cross-cutting technical issues, such as farming systems, soil fertility management, and natural resources management. When the delayering was implemented toward the end of the program, some of the remaining extension workers were retained under contractual terms to cover cross-cutting needs not specific to technologies selected by common interest groups. In 2005/06, NAADS created a subset of activities under its normal program to implement GoU’s Rural Development Strategy (RDS) and utilize Ush8 billion under GoU’s Integrated Support to Farmers Groups (ISFG) program. Sentiment within the Africa region generally (i.e., the Abuja Fertilizer Summit in June of 2006) and within Uganda shifted in favor of a greater public role in assuring access to inputs, particularly fertilizer. The NAADS program faced the options of incorporating input provision coupled with advisory services or coexisting with a new GoU program that would do so. NAADS management, in consultation with DPs, opted to incorporate a mechanism for enhanced access to inputs through revolving funds operative at the level of groups. The design of ISFG was based on the advice provided by the World Bank financial services experts based in the Kampala office to ensure its compliance with best practice. The ISFG was to respond to demands for a material inputs/micro-finance facility to match the advice and training being provided, and its introduction did not require a major restructuring. NAADS, however, was not well suited to manage micro-credit activities, and a number of problems arose that became more severe after the stress to the program in 2007. The performance of the ISFG was mixed. In some FGs, especially in newer NAADS sub-counties, there was limited understanding of how ISFG operated. Credit was to be paid back into a group account that served as a revolving fund for other members. Although there is evidence of repayment, repayment rates are low, i.e. for source of loan--ISFG (47.7 percent), other government (49.1 percent), other formal--bank, MFI etc. (73.2 percent) and informal (76.3 percent). (Performance Evaluation of NAADS, March 2008, p. 57-69). The implementation arrangements for ISFG note that ISFG was to: target sub-counties who had participated in NAADS for at least two years; sensitize stakeholders on ISFG objectives and implementation modalities; and allocate ISFG funds to sub-county farmers fora who would select groups according to an eligibility criteria based on demonstrated interest in participation in the program and prior participation in NAADS (including successful establishment of group enterprises, demonstrated on-going group savings mechanism, ability to pay 10 percent co-financing requirements etc.). These agreed processes were not always followed. Even in the best performing sub-counties performance of ISFG was mediocre (Aide Memoire, Fifth Annual GoU-DP Review Meeting, June 1-2, 2006). NAADS was a visible and popular program of the GOU with broad outreach in late 2006, and as such attracted political attention. The Bank team recognized the potential risks associated with this. The ISR of December 21, 2006 noted that, “NAADS may be loaded with politically motivated agendas that may undermine the objective, principles and approaches that had been successful in the first five years,” and additionally records “the potential use of ISFG funds for political purposes”. The team indicated the need to try to manage this risk by establishing a regular dialogue with key officials to ensure that the program objectives and implementation were not compromised unduly. (ii) Project Implementation and Performance from September 2007 until Closure

On September 10, 2007, President Museveni suspended the NAADS program and indicated that changes in design would be incorporated. The suspension was announced during a tour of rural Uganda, without prior consultation with the DPs. The President cited three objectives: namely, the need for further value addition, the desire to use “demonstration”, “model” and “nucleus” farmers to improve

8

technology uptake and market linkages, and greater use of public extension agents within the NAADS program. He also proposed that funds be used for the direct provision of inputs. Concurrently with the suspension of the NAADS program, the GOU under the President’s direct oversight was preparing a new development initiative, the Prosperity for All (PFA) program, in which agriculture was a central pillar. The suspension and the proposed changes reflected new priorities of the GoU but were not consistent with the core principles of the NAADS design and adversely affected NAADS implementation. In October 2007, Cabinet assigned MAAIF/NAADS as key players in executing the Prosperity for All (PFA) program. Consequently, the NAADS program was asked to expand its mandate in value addition and agro-processing, and to support model farmers to act as demonstration and learning centers to encourage commercialization. NAADS, through the ISFG, substantially increased amounts spent under the technology development and demonstration activities.5

In March 2008, after long and detailed negotiations, GoU and the DPs agreed on measures that would accommodate many of the GOU’s desired changes in the program, but still remain within the agreed core principles of NAADS and the related legal commitments. Under the agreements, GoU directed the conversion of existing government extension workers to NAADS to work under 3-year performance-based contracts for improved service delivery, thus acting on the commitment to delayering made during project design. The changes were agreed to be implemented over a one-year transition period. The amended DCA (June 2008) increased agricultural advisory services grants to farmers (including for inputs) from 53 percent of the IDA Credit to 74 percent. GoU added a new budget line to allocate an extra USh34 million of its own resources to each sub-county under the category “support to model farmers in their groups” increasing the total sub-county allocation to USh118 million for FY2009/10 (MAAIF 2008 Press Release.).

Multiple guidelines issued to inform stakeholders, i.e. the 2008 “new NAADS” guidelines, the ISFG procedures, and several “corrections” caused confusion for NAADS implementers and among the ultimate clients (ICR interviews in March 2010). Disparate interpretations of the guidelines, in some case with a possible political objective at the district level necessitated a second round of training and dissemination in connection with the new 2008 guidelines. The changes introduced starting with the ISFG in 2006 and subsequent amendments altered the initial vision of NAADS. The program conceived as an innovative approach to provision of advisory services incrementally incorporated activities in input supply, micro-finance, and value addition. Flexibility and responsiveness to changing priorities were positive and the NAADS team was probably technically more skilled than other potential implementers of programs, and hence best placed to deliver the programs that the GoU chose to launch. At the same time, the additional demands overstretched the implementation team, and the pace of change and confusion in rules brought deterioration in performance after 2007. Public debate on whether, how, and why the program should change was heated, and coverage in the press was extensive. Much of the coverage was highly critical of the program, in part highlighting genuine weaknesses, and in part in pursuit of advocacy by parties with interests in the outcome of the debate.

In February 2009, the President announced that NAADS resources would be concentrated on six farmers per parish, and that those farmers would be chosen through an appointed committee rather than by Farmers’ Fora (GoU Position Paper, March 2009). This decision was conveyed through

5 The costs of private sector service provision fell in relative and absolute terms from 2003/04, while the costs of technology promotion including ISFG rose substantially over 2005-2008. (See the “Performance Evaluation of NAADS, ITAD, Draft Report of March 3, 2008 on the balance between advice and material inputs/credit funds.)

9

a notice in the daily press and in large stakeholder fora without prior consultation with the DPs. The initial directives governing this set of changes in the NAADS program would have taken it out of conformity with the agreed PIM and very likely with the NAADS Act of 2001. The Bank team accordingly drew up a letter of suspension of disbursement and secured approval of the RVP for the pending suspension. The team concurrently continued discussions with the GOU to explore ways to bring the program back into conformity with the legal commitments. In late March, the DPs and GoU reached an agreement under which the program would remain compliant with existing legal documents including the NAADS Act for the remainder of the project. New guidelines were issued again. The process of selection of the nucleus farmers may not have been in accordance with the agreement in all cases, but the withdrawal of support from the large number of farmers benefiting through NAADS groups and redirection to a very small number of specially selected recipients was forestalled.

The increase in relative emphasis on credit and input provision and resulting lesser relative role of advisory services within the program increased vulnerability to distortion through rent seeking, elite capture, and political patronage. This demoralized and discouraged farmers and FGs and created the impression that the program is now for the “privileged few” (ISR, December 22, 2009). At the same time, the decision to seek a common ground with those who sought to change the program and arrive at a compromise consistent with the legal foundation for continued disbursement placed bounds around the rent seeking, elite capture, and political patronage. Had the proposed programs proceeded outside of the NAADS framework, the adverse consequences for large numbers of Ugandan smallholders would most likely have been severe.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The NAADS design is predicated on the assumption that participatory M&E (PM&E) will ensure that the quality of service provision meets the standards required to achieve the program’s intended outcomes (PAD, Annex 11). While the program undertook M&E, the work was not part of a clearly defined M&E framework with program objectives and the relevant indicators of outcome, output/result, and activity level clearly set out (ITAD 2008, p. 2008, p. 117). The ICR team has not been able to find documentation to show that NAADS did conduct regular, broad, and systematic PM&E. The ITAD report concludes that the PM&E system was not routinely followed (ITAD, 2008, p. 116). There are no baseline indicators until 2004. The NAADS team conducted household surveys in 2004 and 2007 and impact evaluations using this survey data. These efforts provide more rigorous and detailed information at two points in time than is available for many projects, but do not provide the more mundane times series of key indicators, particularly for the period between 2007 and closing. The project did track and report annually on the number of districts and sub-counties participating, the number of registered groups and their members disaggregated by gender, the number of farmers’ fora at various levels, and the recruitment of service providers. See Annex 2 on M&E weaknesses (from GoU ICR, p. 34-35). 2.4 Safeguard and Fiduciary Compliance Safeguards The project correctly identified the two environmental safeguard policies triggered at preparation: OP/BP 4.01 (Environmental Assessment) and OP/BP 4.09 (Pest Management) and, based on a participatory approach, prepared an Environment Management Plan (EMP) and a Pest Management Plan (PMP). The Plans specified measures to mitigate any adverse impacts of agricultural intensification (i.e. contract specification to ensure that service providers promoted best practices in land and water management), and risks of pesticide use (i.e. training of input suppliers for the handling and application of agro-chemicals). The NAADS guidelines addressed safeguard concerns adequately. The service provider assessments of

10

enterprises were checked against guidelines and approved by the district environmental officer working closely with the NAADS Coordinator. Service providers, NAADS Coordinators and farmer forums at all levels received training to update them on new issues. In May 2004, NAADS completed a study on the implementation of the pest management policy, and the results were integrated into the NAADS guidelines (ISR, June 2004). The ‘satisfactory’ safeguard compliance rating in the first several years was downgraded to ‘MS” in the May 2008 ISR due to poor reporting.

Financial Management (FM) Project funds were channeled into GoU’s Consolidated Fund Account (as stipulated in the MOU), and were transferred as follows, using existing GoU mechanisms: (i) to the NAADS Secretariat; and (ii) to the districts for (a) core district-level activities, and (b) for sub-county program implementation. A notable achievement of NAADS was that about 80 to 88 percent of funds went directly to the roughly 80 districts and 1066 sub-counties.6 This large amount of resources sent to many independent accounting entities for the benefit of thousands of farmers posed huge challenges of accountability. In the first five years, the FM system seems to have performed well, as the FGs increased their strength and were cohesive, willing and able to monitor both the service providers and the officials at the district and sub-county levels. Following the integration of the ISFG with the NAADS program, the introduction of revised implementation guidelines (that were unclear on the selection criteria, incentives, management of the revolving fund, repayment etc.) and the transformation of NAADS into a system focused on providing inputs, FGs were less able to oversee financial management and assure transparency. This may have been in part because the lead farmers were not in all cases selected by the groups, and did not feel accountable to the groups (MAAIF, April 2010. Also corroborated in ICR field interviews in Feb./March 2010). The refocusing of the program meant that about 59 percent of the NAADS budget, particularly at the sub-county level, went to procurement of goods and services, compared to the 30 percent initially anticipated (MAAIF, April 2010, p. 2). This change escalated accountability challenges, especially at the lower implementation levels. This was compounded by the rapid roll out of the NAADS program, in some cases without full assessments of local governments to implement the program and/or due diligence to assure competence on procurement skills and financial management. The FY2006/07 Audit Report (filed only in May 2008 due to the suspension) noted a number of irregularities in five districts (Iganga, Mbarara, Kanungu, Buliisa and Luwero) where expenditures were subsequently suspended. MAAIF put in place a NAADS Probe Task Force following repeated complaints of mismanagement of the program. The Task Force identified resource abuse and/or misappropriation of NAADS funds in the five districts noted in the audit report (see MAAIF, April 2010 for detailed findings and recommendations). Steps have been taken to identify funds for recovery in four districts, while investigations are ongoing in the Buliisa district. A number of officials have been prosecuted and where farmers are involved, disciplinary actions have been recommended by GoU. Procurement Implementation of several activities in the procurement plan (i.e. selection of consultants and conclusion of partnerships) generally remained behind schedule mainly due to several in-year changes in priorities. Delays in handling procurements under public-private partnerships, which constituted a major set of procurements under the NAADS Secretariat are attributed to: (i) protracted negotiations that were required with potential partners who had no experience in preparing the kind of proposals required; (ii) the selection of eligible partners, which was done mostly through single sourcing and required clearance from the Bank; and (iii) delays in the preparation of procurement plans and commencement of several

6 For example, in the 2009/10 financial year, out of GoU’s provision of USh134 billion for NAADS implementation, the Secretariat, district local government and sub-counties obtained 12.7, 6.3 and 81 percent of the funds respectively (Information Paper: Funding and Levels of Accountability in NAADS Implementation, April 2010).

11

procurements. Supervision missions provided advice to the Secretariat on strengthening guidelines for the selection of private sector partners to ensure equal opportunity for eligible partners. The objective of community procurement (aimed to empower farmers) was largely met with contracting through sub-county procurement committees. This was weakened in the project’s final years due to frequent changes in guidelines. The resulting confusion on procedures undermined farmer participation, provided opportunities for elite capture of sub-county procurement committees, and led to cases of overpricing and delivery of substandard products. This was mitigated by piloting of community procurement committees composed of farmers at parish level. Anecdotal evidence from supervision missions showed increased farmer empowerment in the pilot districts, with fewer cases of elite capture as farmers managed the procurement themselves. The project provided training to strengthen the procurement capacity of FGs. The project experience suggests that the NAADS Act may require amendment to correct a situation of potential conflict of interest, wherein the NAADS sub-county Coordinator and Chairman of the farmers’ forum are appointed members of the sub-county procurement committee. Post Procurement Reviews (PPRs) that were focused on the Secretariat at the national level were completed for 2007, 2008 and 2009, and were complemented by limited procurement audits at sub-country level by consultants. The reviews identified weaknesses in procurement planning and adherence to procurement plans, poor record keeping, inadequate procedures for receipt and opening of bids, and inadequate attention to contract management. These weaknesses are being addressed in revised implementation procedures reflected in the documents defining the next phase of implementation.

The MAAIF Task Force in a second report found several irregularities in procurement in all the sampled districts (MAAIF, 2010). These covered instances of lack of advertisements, falsification of documents, procurement of incomplete packages and substandard technologies, failure to follow technical specifications, and payment before supply or payment without verification. Others including the dominance of supply by a few firms and/or bid prices higher than those available through normal shopping (which was widely reported in all sampled districts) likely resulted in inefficiencies and lower profits at farm level. The report indicates that community procurements were interfered with and mismanaged, and that cases of conflict of interest and noncompetitive supply were noted. The report’s recommended anti-corruption measures and lessons to improve community procurement have been taken into account in the follow-up ATAAS project.

2.5 Post-completion Operation/Next Phase The GoU has taken on and achieved significant reforms in the institutional arrangements for agricultural services and research. These reforms were implemented concurrently but largely separately, and were supported through stand-alone projects; one for advisory services and another for agricultural research. The Government has decided in the future to bring advisory services and research together into an integrated technology system supported through the proposed Agricultural Technology and Agribusiness Advisory Services (ATAAS) project (scheduled for Board approval on June 24, 2010). The new project incorporates lessons learned from the previous projects and aims to establish more effective institutional linkages at all levels to improve technology generation and dissemination. ATAAS plans to re-engineer business processes at National Agricultural Research Organization (NARO) and NAADS and create a single, well-coordinated operation that includes a shared ICT infrastructure with joint workflow management and M&E systems. ATAAS will work towards a shared objective using a unified results framework and performance indicators, and will also provide for a joint baseline and impact evaluations. ATAAS has incorporated measures for accountability through its Governance and anti-Corruption

12

Arrangements (GAC) program, which includes elements for improved information disclosure and transparency of decision making at national and community levels (ATAAS PAD, Annex 9).

3. ASSESSMENT OF OUTCOMES

3.1 Relevance of Objectives, Design and Implementation The project’s objectives are relevant to GoU’s National Development Plan (NDP) and MAAIF’s new Development Strategy and Investment Plan (DSIP). The DSIP envisages a comprehensive approach consistent with the aspirations of the Comprehensive Africa Agricultural Development Program (CAADP) that seeks to facilitate a shift from subsistence farming into commercial agriculture. NAADS is also consistent with the FY2011-14 Uganda Country Assistance Strategy, which recognizes the need for IDA to support GoU’s efforts to strengthen the demand-driven extension approach, foster the adoption of better technology and raise real farmer incomes. The central element of the NAADS design—creation of a pluralistic advisory service with public funding, both public and private delivery of services, and empowerment of farmers to articulate their needs-- remains very valid. The revised results framework was well aligned to the PDO and relevant to measuring improvements in productive efficiency and farmer welfare. The design was unrealistic with regard to the expectation of the timing of retrenchment of public extension agents, but the recognition of the need for this was relevant. With respect to implementation, a substantial portion of NAADS funds flowed to the district level, consistent with the Uganda’s policy of decentralized administration of government programs.

3.2 Achievement of Project Development Objectives7

This section provides evidence of the mechanisms through which NAADS interventions resulted in successful final outcomes between 2004 and 2007.8 For clarity of presentation, the ICR looks at the PDO in two stages (Figure 1): (i) assisting poor male and female farmers to become aware of and to be able to adopt improved technology and management practices in their farming enterprises (i.e. intermediate outcomes); and (ii) this leading to enhanced productive efficiency, economic welfare and sustainability of their farming operations (final outcomes). Annex 3 provides detailed econometric evidence to support all the results presented in this section. Annex 3 Appendix 1 explains why real crop revenue and real wealth per capita are the best indicators of productive efficiency and economic welfare given data constraints. NAADS had three primary outputs that directly affected rural households: 1) provision of advisory services; 2) support to buy plant and seed material and physical inputs; and 3) assistance to form and strengthen FGs in the form of Farmer Institutional Development. Jointly, these outputs were expected to produce the intermediate outcomes, and subsequently enhance crop income and household wealth.

Figure 1: The Causal Mechanism Connecting NAADS Outputs to Final Outcomes Out

7 See Annex 2 for Outputs by Components and Annex 2 Appendix 1 for KPI (revised) achievements. 8 The ICR team used data from two Household Surveys in 2004 and 2007 (done by the International Food Policy Research Institute--IFPRI) to perform the analysis presented in this ICR.

Outputs a) Advisory Services b) Support to Farmers c) Farmer Institutional Development

Intermediate PDO Outcomesi) Greater access to information on improved technologies and practices ii) Enhanced adoption of improved technologies and practices

Final PDO Outcomesi) Increased productive efficiency (real crop revenue per capita) ii) Increased economic welfare (real wealth per capita)

13

I. NAADS’ Outputs (Measured at the Household Level) Output Indicator #1: Provision of demand-driven advisory services A significantly larger proportion of NAADS members demand advisory services compared to non-members. In the available sample, 31.7 percent of NAADS households demanded advisory services (25.7 percent of these from NAADS providers), while just 10.0 percent of households demanded advisory services in non-NAADS sub-counties. Further, 66.1 percent of NAADS members report receiving advisory services in contrast to just 23 percent of non-members in NAADS sub-counties and 25.4 percent of households in non-NAADS sub-counties. (Annex 3 Table 6 describes the demand and supply for advisory services) Households receiving services other than through the NAADS program may have accessed them through NGOs, the residual public service, private providers, or through outgrower schemes. Output Indicator #2: Support (provision of credit, grants and subsidies) to farmers. A much larger proportion of NAADS members (64.5 percent) received support when compared to the support received by non-members in NAADS’ sub-counties (31.8 percent) and households in non-NAADS (22.8 percent) sub-counties. (Annex 3 Table 7 presents the composition of incidence of support to rural households by NAADS member status). Output Indicator #3: Farmer Institutional Development—the strengthening of existing FGs and the formation of new groups. Annex 3, Table 3 describes the incidence and composition of group membership in sub-counties that were part of the NAADS program compared to sub-counties that were not. 62.9 percent of the rural households surveyed were part of a collective organization in NAADS sub-counties, while only 43.9 percent of households in non-NAADS sub-counties were part of a collective organization. Further, there were significantly more women and smallholders who were members of groups in NAADS sub-counties, possibly suggesting that these groups may have been more inclusive. Finally, as Annex 3 Table 5 shows, NAADS members were 44 percent more likely to receive training and 20.5 percent more likely to receive support compared to members of other groups. This also suggests that NAADS groups may have been stronger. II. Intermediate PDO Outcomes Intermediate Outcome #1: Greater access to information on improved technologies and practices. Relative to non-member households, NAADS members were 17.8 percent more likely to have access to information on how better to utilize crop production equipment and 13 percent more likely to have access to information on improving crop production practices. Compared to members of other groups, NAADS members had greater access to crop practices and technologies but not prices. However, because groups were more efficient in accessing information, it is likely that NAADS also had a significant impact on improving access to price information. (Annex 3 Table 8 describes the informational advantage of being a NAADS group member relative to 3 different comparison groups). Intermediate Outcome #2: Enhanced adoption of improved technologies and practices. NAADS member households were more likely to adopt improved crop practices and also to experiment with growing new crops. After controlling for several confounding socioeconomic and regional factors, NAADS group members were 10.7 percent more likely to adopt improved crop practices than non-

14

members and 5.4 percent more likely to experiment with new crops. (Annex 3 Table 9 describes the greater adoption of improved agricultural practices by NAADS group members). III. Final PDO Outcomes Outcome Indicator #1: Increased productive efficiency (real crop revenue per capita). NAADS members experienced a 42.1 percent increase in real crop revenue per capita compared to non-members in NAADS sub-counties and a 30.9 percent increase relative to other group members in non-NAADS sub-counties after controlling for socioeconomic and regional confounding factors. Annex 3 Table 10 describes changes in productive efficiency of NAADS member households relative to different comparison groups. Annex 3 Appendix 1 discusses the data constraints that prevent the measurement of other indicators of productive efficiency. Outcome Indicator #2: Increased economic welfare (real wealth per capita). NAADS members experienced a 27.4 percent increase in real wealth per capita compared to non-members in NAADS sub-counties and a 70.9 percent increase relative to other group members in non-NAADS sub-counties. (Annex 3 Table 10 describes changes in real wealth per capita of NAADS member HHs relative to different comparison groups.)

3.3 Efficiency For the baseline scenario, the ICR estimates a Net Present Value (NPV) of 80.8 million (2000, US$) and a benefit/cost ratio of 2.17. This estimate, however, does not reflect the impact that NAADS may have had on livestock productivity and incomes (due to data limitations). See Annex 3 for details.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory NAADS pioneered an innovative approach to extension provision in Uganda. The project was very relevant then and continues to be relevant. It achieved its PDO satisfactorily until 2007; in particular, NAADS empowered farmers to demand relevant extension services, provided greater access to crop and livestock technologies and practices, and raised per capita real crop revenues and real wealth. The ICR team could not directly measure the achievements of the PDO after 2007 due to lack of M&E and the fact that the household surveys covered only the period from 2004 to 2007. ICR field interviews, the Bank’s 2006/07 Audit Report and several other reports (MAAIF Task Force Reports, March 2010 and April 2010; ITAD, March 2008; IFPRI, March 2010; and the DFID-funded Accountability Working Group paper, January-February 2010) provide qualitative evidence since 2007 of instances of misuse and waste of resources, misdirected inputs, depressed farmer morale, and the weakening of FGs and farmer fora. These factors suggest that the impact of NAADS after 2007 is likely to have been less than in the earlier period, and thus diminishes the case for a satisfactory rating, but does not reverse it. Hence, the achievement of the PDO is rated moderately satisfactory.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development NAADS membership had a significant impact on crop revenues of female-headed households, smallholders (those households who own less than one hectare of land) and households that were poor (i.e. lived in a house with a primitive roof) (Annex 3 Table 11). The wealth impact, however, was statistically significant only for the households with a primitive roof. Since the period covered by the data

15

is short, it is likely that all vulnerable segments that benefited from an increase in income would gradually experience an increase in wealth over a longer period. The project has also had a major impact on social development through the formation and strengthening of FGs and higher-level farmer organizations (HLFOs) as indicated below. (b) Institutional Change/Strengthening NAADS was successful in substantially strengthening farmer institutions. In all participating districts, NAADS started with facilitating the strengthening of existing groups and forming new FGs at parish and sub-county levels. Over 55000 groups in 1066 sub-counties with a total of more than 1 million members had been registered at the sub-counties by end 2009. Broad feedback from FGs in many districts confirms that farmers felt that their level of empowerment and capacity to demand specific advisory services in areas such as improved crop varieties, livestock breeds, management and post-harvest practices, and marketing information had substantially increased. However, this outcome was most pronounced for larger farms and those close to good roads and/or near markets. As noted above, after the changes in the NAADS focus (on demonstration, model and nucleus farmers) in 2007, farmers’ confidence in the power of their FGs is reported to have diminished. The FGs formed the base for the establishment of farmer fora at sub-county (each with their own Technical Team and Procurement Committee), district and national levels. In order to strengthen communications and improve the interface between the sub-county fora and their FGs, from 2005/06 onwards, Parish Coordination Committees were established. At district level and beyond, NAADS also facilitated FGs to form HLFOs to support agro-processing and marketing. This was followed by the establishment and/or strengthening of successful public-private partnerships in several commodities (i.e. tea, coffee, sweet potatoes, rice, citrus, goats and dairy). At national level, NAADS supported the establishment and effective operation of the NAADS Board and the NAADS Secretariat. The Board, which acted in an interim fashion until 2009, when after the formal establishment of the National Farmers Forum in 2007, the full NAADS Board with three sub-committees (Finance and Administration, Marketing, and Technical), was constituted. The NAADS Board provided timely and effective oversight and guidance to the Secretariat and played a key role in monitoring NAADS activities in the field. The NAADS Secretariat coordinated program implementation including sourcing, accounting and audit, and the annual GoU/DP review missions in an effective manner. (c) Other Unintended Outcomes and Impacts (positive or negative) Negative: ICR field interviews and anecdotal evidence suggest that in some cases increased supply engendered through the program could not be absorbed on local markets, and prices fell. This can be attributed to insufficient attention to the marketing elements of the program, and can be remedied through more careful selection of technologies and prior assessment of absorptive capacity. Positive: While Uganda had moved towards a more decentralized administrative system before this program, NAADS was one of the first agricultural programs to channel a large portion of its expenditures to the district level. In fact, 80-88 percent of NAADS expenditures were disbursed to local administrative units at the district level. The program may have consequently demonstrated institutional weaknesses at the local government level that were previously untested. In the short run, these weaknesses may have prevented more favorable outcomes. However, in the long run, the constant monitoring by empowered FGs should help to foster a culture of accountability and thus better align the incentives of local bureaucrats and politicians with those of the (intended) beneficiaries of any decentralized program.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

16

The main beneficiary impact evaluation using two rounds of FG and HH surveys from 2004 and 2007 is summarized in Annex 5.

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Rating: Moderate The institutional risk to development outcome is moderate. NAADS will be financed under the Agricultural Technology and Agribusiness Advisory Services (ATAAS) project. Weak linkages between NARO and NAADS have been explicitly addressed under the new project and it is likely, although not certain, that the design of the new institutional framework, along with institutional reform at MAAIF, will lead to robust gains in the future.9 The Secretariat has had very skilled leadership through a challenging period, and potential turnover in key staff represents a risk. The economic and financial risk to development outcome is moderate. The design of the ATAAS program includes new measures to lower the likelihood of recurrence of governance and accountability issues experienced in the last two years of NAADS. Sub-county committees will be required to publish the details of procurement transactions. All members of the sub-county committees will be members of farmer for a. The NAADS Secretariat will publish indicative prices of commonly procured inputs. Additional resources will be allocated towards FID. Pressures leading to the stress on the program during the last two years remain, albeit at diminished levels, and still pose a risk to the development outcome.

The environmental risk to development outcome is low. NAADS’s environmental rating was ‘B. The project supported training in natural resource management, efficient pest management, and in disease control, and this will likely result in favorable development outcomes.

The social risk to development outcome is low. It is likely that by empowering FGs, and increasing agricultural productivity and incomes of smallholder farmers, NAADS will improve the socioeconomic status of the poor and other vulnerable sections of Uganda’s rural population. The gender impact of the program has been positive, as reported in Annex X.

5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory The Bank team brought knowledge of global good practice in advisory services and facilitated understanding of the model on the part of a wide array of stakeholders in Uganda. The team worked closely with the client to develop an innovative and relevant project to support the NAADS program. The team brought additional partners in at a time when shared financing through a basket mechanism was not common. The project was mainstreamed into the systems of the GOU, including the budget process and the intergovernmental fiscal flows. The Bank simultaneously supported needed reforms in agricultural research, with an appropriate intent to bring the two pieces of the technology system together. Although the two reforms proceeded largely in parallel during the implementation of the NAADS program, the

9 NARO and NAADS will merge priority setting and annual review processes at the zonal and national levels. This should improve the inclusion of NAADS’ (zonal/district) goals for agricultural commercialization and of MAAIF’s (national) development objectives in the NARO agenda.

17

objective of closer alignment was appropriate. The NAADS program was designed for a phased roll-out. The design underestimated the resistance to retrenchment of the traditional extension service, and hence was overly optimistic about the supply of service providers. The design was visionary in providing legislative codification of key features of the program design in the NAADS Act and negotiating passage of the act as a disbursement condition. When the program came under severe stress in the period after 2007, it was the strictures of Ugandan law in the form of the NAADS Act more than the threat of suspension of disbursement of the IDA credit that preserved the core design. (b) Quality of Supervision Rating: Moderately Satisfactory The Bank provided good continuity in team composition throughout the implementation period, including a strong presence in Uganda and regular interaction with the client and partners. Because of the good working relationship between the Bank’s and the GOU’s teams, the Bank’s role was more in the nature of implementation support than classic supervision. This close working relationship was useful during the period when the program was young and facing the challenges of the roll-out. It became vital in the later period when very astute and well coordinate response was required as the program took on a broader mandate and faced pressure for fundamental changes in design. The basket funding arrangement stipulated in the MOU required flexibility on the part of the Bank, as one of several partners, and occasioned a supervision strategy different from that of most other projects at the time. Supervision missions were joint and Joint Review Aide Memoires were shared documents with language agreed through consensus. The consensus led in at least one important case to exclusion of an issue that the Bank team felt important (improved M&E) because other partners did not agree. 10 Donor coordination was not always smooth despite good leadership and efforts from the Chair of the donor group and the Bank’s own staff (see MTR Office Memorandum, May 6, 2005).

Although the write-ups in the ISRs detailed issues, the project ratings were not candid in all categories. Although progress toward meeting the PDO warranted ratings in the satisfactory range, the difficulties in implementation, particularly in the later period, were not recorded in lower ratings. ISRs continued to rate financial management, procurement, M&E and implementation progress as ‘MS’ or ‘S’ until project closure.11 In these categories lower ratings would have properly captured the difficult challenges that the project faced. That the problems were acknowledged, remedial measures undertaken, and pressures on the project design successfully countered perhaps warranted the positive overall ratings, but in the disaggregated categories on the implementation side less than satisfactory ratings would have been appropriate. The deficiencies in M&E were particularly damaging, especially in the period after the detailed survey and prior to closure. With respect to the reported misuse of funds, the Bank should have conducted a supplementary audit. Many of the reports were artifacts of the highly charged public debate at the time, but they still warranted attention. Audits of project were conducted and reviewed by FM and Procurement staff; however, it is not clear if any recommended actions were followed up on. The GOU undertook its own supplementary investigation and took remedial measures. Additional design features to address the identified risks are included in the follow-on operation. The weakness has thus been recognized and addressed, but the

10 On M&E, the Bank proposed an Action Plan in May 2004 to follow up on discussions with NAADS and IFPRI to ensure routine data collection, mainstream Beneficiary Assessment into FID and to mainstream evaluation into Service Provider Contracts. This was not acted upon as all donors did not understand the importance of strong M&E (NAADS TTL interviews. Also see note on Agreements Reached, Discussions on Monitoring and Evaluation, May 24, May). 11 ISR #13 of Dec. 1, 2007 to the final ISR #17 of Dec. 22, 2009 rate the PDO and IP as either “S” or “MS”.

18

supervision team should have reacted more quickly and strongly when the public charges surfaced, and following the Post Procurement Reviews of 2007, 2008 and 2009 (described in Section 2.4 above).

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory

The rating reflects the ‘S’ rating for quality at entry and the ‘MS’ rating for supervision.

5.2 Borrower Performance (a) Government Performance Rating: Unsatisfactory

Shortcomings during project implementation justify a rating of unsatisfactory for borrower performance. The GoU did not until the final years of implementation increase the MTEF ceilings to create the agreed space for allocations to the program. The GOU did not release funds on time, and thus delayed project start up and expansion, and forced curtailment of activities during peak agricultural periods. Disbursement was often later and less than expected. In the later period, after the program had taken on the wider mandate, MTEF ceilings were raised significantly and the contribution of the GOU increased. MAAIF did not act, as agreed in the project’s legal covenant, to implement delayering until the end of the implementation period. The GOU at several points announced changes in design without prior consultation with financing partners. The issuance of successive and conflicting guidelines confused participants and beneficiaries, and undermined confidence in the program. These were serious deficiencies in the Government’s performance in support of the program. At the same time, the Government remained in dialogue with the partners and the implementation team and negotiated compromises on many difficult issues. The willingness to find common ground allowed the program to continue and achieve many of the intended objectives. The Government retained a competent implementation team. The Ministry of Finance had a keen interest in the program throughout and the annual budget discussions served to keep the knowledge of key officials current. When confronted with arguments about risks to changing the program and presented with alternative approaches to reach the stated objectives, senior officials, including the Head of State, adjusted their approaches. Thus, although the Government’s performance was unsatisfactory, it was not highly so. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory The NAADS Secretariat has executed its tasks effectively with a lean structure. The Secretariat and the staff of other implementing entities at the district and sub-county level largely comprised of well-qualified and highly motivated individuals. The new approach to agricultural extension was effectively transmitted to all stakeholders resulting in unprecedented involvement of farmers through their FGs and FFs. Evaluations of NAADS’ impact during the period before the MTR show positive results in terms of achievement of most of the main outcome indicators notwithstanding the perceived key weaknesses discussed earlier (IFPRI, 2008, ITAD, 2008). Implementation of the ISFG program presented new challenges for the Secretariat that they discharged less effectively than the core program of advisory services. On M&E, the NAADS Secretariat did not collect systematic data that the project design envisaged, and hence information vital for management and assessment of outcomes was missing. The Secretariat interacted very effectively with other counterparts within the GOU when changes to the program were proposed, and consistently helped find acceptable common ground. The performance of the Secretariat was in some respects exemplary, but the deficiencies in M&E serious, justifying an overall rating of moderately satisfactory.

19

(c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory This rating reflects the ‘U’ rating for GoU performance, the ‘MS’ rating for Implementing Agency performance, and the ‘MS’ rating for project outcome.

6. LESSONS LEARNED A pluralistic model of agricultural services incorporating elements of global best practice can be implemented in Uganda and shows positive results. The NAADS project was the first in Africa to incorporate public funding and pluralistic provision of services as a national program, with farmers empowered to select their service providers. Prior to introduction of the NAADS program, questions were raised as to whether such a program shown to work elsewhere, particularly in Latin America, would be suitable and feasible in Africa. The NAADS experience offers many lessons about how best to adapt such a model, but a fundamental conclusion can be drawn that the programs are suitable and superior to the centralized traditional model of public funding and public provision of services. Government ownership of a program is very positive, but it can also lead to tensions when governmental priorities change and views of the client and partners diverge. The fact that the GOU saw NAADS as a public program, rather than an enclave project of the donors, gave the program weight and an opportunity to be transformative, through scale and mainstreaming into governmental systems. The Government’s high degree of ownership also obscured the need for close consultation when changes in the program were contemplated. Management of this risk can be done only when key partners have strong technical staff on the ground and ready to respond quickly.

Codification of key provisions of programs in legislation can provide safeguards against policy reversal. Few countries have the equivalent of the NAADS Act to safeguard design of programs. This can be a very constructive measure when programs require long duration to reach impact, but governments and priorities may change frequently.

Advisory services appear to have had a greater impact than subsidized provision of inputs under the program. This finding is very provisional and warrants more careful investigation, since it is critical to policy dialogue in African agriculture at present. Many governments are of the view that widespread distribution of subsidized fertilizer will achieve rapid gains in productivity and incomes. The NAADS experience suggests that the most impressive gains were achieved in the period when the program concentrated on provision of advice and demonstration of technologies. The returns when inputs became relatively more important in the program appear to be more mixed.

89. In pluralistic technology systems, attention to the supply of services is as important as assurance of mechanisms to support demand and accountability. Significant investment is needed to train, certify, and monitor providers of services. The elements of a technology system both nationally and regionally, particularly educational institutions and research organizations, should be explicitly engaged for this purpose. Innovative use of ICT can also help in this regard.

20

ANNEX 1. PROJECT COSTS AND FINANCING

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD millions)

Percentage of Appraisal12

For the period 2001 to 2005 For the period 2001 to 200513

1. Advisory and Information Services to Farmers 68.79 11.78 17.12

2. Technology Development and Linkages with Markets 6.33 3.46 54.66

3. Quality Assurance-Regulation and Technical Auditing of Service Providers 1.33 0.49 36.84

4. Private Sector Institutional Development 4.38 0.11 2.51

5. Programme Management and Monitoring 16.46 5.99 36.39

Original Financing Estimates in PAD 107.91 21.83 20.22 Sub-Total

Additional Financing Available due to US Dollar depreciation against SDR

13.0

Total Project Costs 120.91

For the period 2006 to 2009 1. Farmer Institutional Development 0.00 16.89 2. Advisory and Information Services to

Farmers 0.00 90.28

3. Agribusiness Development and Market Linkages 0.00 11.57

4. Local Service Provider Institutional Capacity Development 0.00 0.41

5. Planning, Monitoring, Quality Assurance and Evaluation 0.00 12.69

6. Project Management and Coordination 0.00 8.99 Sub-Total 140.83

Total Cost 0.00 162.66

Physical Contingencies 0.00 0.00 0.00

Price Contingencies 0.00 0.00 0.00

12 Not applicable because the components changed at the MTR. 13 Expenditures for the period 2001-2005 cover the pre-MTR period based on original components in the PAD.

21

Total Project Costs 0.00 162.614 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00

Total Financing Required 0.00 162.6

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal (%)

Borrowing Agency 0.00 0.00 Borrower 9.12 51.9 569 International Development Association

(IDA) 45.00 53.5 118

International Fund for Agriculture Development 17.50 19.9 113

Local Govts. (Prov., District, City) of Borrowing Country 10.80 2.6 24

Bilateral Agencies (unidentified) 23.34 33.6 143 Local Farmer Organizations 2.16 1.1 51 Total Financing 107.9 162.6 151.1

14 During the FY 2008 and 2009 years government contribution amounted to about Ush 77 Billion compared to an average of about 3.5 Billion in the previous six years. As a result, total funds available to the project were about USD 42 million more than expected.

22

ANNEX 2: OUTPUT BY COMPONENTS Project description (from NAADS Project Paper of June 30, 2008, p. 8). The following changes were made to the NAADS component structure at restructuring: (i) move farmer institutional development activities from AAS to a new stand alone component to more clearly distinguish between service provision and institutional development activities at the district level and below; (ii) move technology development activities into AAS to reflect the close and complementary role technology development plays in the delivery of advisory services; (iii) rename the Technology Development and Linkages with Markets Component to the Agribusiness Development and Market Linkages Component to reflect the focus on market and agribusiness linkages and the removal of technology development activities; and (iv) move M&E activities from the Program Management Component to the Quality Assurance, Regulation and Technical Auditing of Service Providers Component and rename the component to the Planning, Monitoring, Quality Assurance and Evaluation Component in order to aggregate all planning, M&E functions together in one component, and rename the Program Management and Monitoring Component to the Program Management and Coordination Component to reflect the removal of M&E.

Component 1: Farmer Institution Development (FID)(Formerly: Advisory and Information Services to Farmers)

Achievements under this component are rated satisfactory.

Three key indicators were specified under this component15:(i) Total number of districts that have signed participation contracts to enter the NAADS program (target = 83 districts); (ii)Total number of sub-counties with farmer fora formally established and effectively operating (target of 1,033 sub-counties); and (iii) Total number of sub-counties with advisory service providers contracted and operating in the field (no target). Indicators (i) and (ii): The program grew rapidly from its inception in 2001 until closure. By December 2009, 79 districts had signed participation contracts and 1,066 sub-counties in these districts had established farmer fora. Only the 4 new districts that were recently formed had not yet signed NAADS participation contracts. Table 1 describes the rapid expansion of the program into new districts and sub-counties.

Indicator (iii): Approximately 500 sub-counties had received advisory service contracts by 2007/2008. There was a high degree of uncertainty about the privatization of some private service providers and the “delayering” of others. The ICR team isn’t clear about the number of sub-counties with advisory service providers contracted and operating in the field, but field visits suggest that there was a shortage of qualified service providers in remote sub-counties. Further, the uncertainty surrounding the contracts may have resulted in the more qualified potential candidates moving to urban areas with greater job opportunities. Overall, this component aimed to increase the farmers’ capacity and visibility, and enhance farmer empowerment. FG formation and farmer institution (including individual farmers, associations,

15 The Project Paper lists the “the number of farmers being served by NAADS” under component 2 (Agricultural Advisory Services). This indicator more accurately reflects objectives associated with component 1.

23

cooperatives etc.) development were the main approach and entry point for delivery of Agricultural Advisory Services. At program design, FID was to be provided mainly by service providers, many of whom were to be NGOs and Community Based Organizations (CBOs) due to their experience in working with participatory methodologies in communities. While this remained the case in older districts, NAADS guidelines were later revised and FID was then undertaken by the local government staff, i.e. Community Development Officers (CDOs). In the older districts, program roll out was more systematic and a comprehensive process of FID at the sub-county level was undertaken. In newer districts roll-out was done quickly, and the processes of group formation, formation of farmer institutions and enterprise selection were undertaken almost at the same time. Achievements/Outputs:Farmer Group Development: In all the districts, NAADS started with orientation of key stakeholders at district level, as part of the process to introduce the program to the stakeholders. This was followed by formation of FGs and institutions at the sub-county and lower levels. All NAADS groups registered at sub-county level and received certificates after payment of membership fees as stipulated by the NAADS Act. As a result, farmers were mobilized in large numbers. At project closure, over 55,000 FGs had been registered across the country. Membership of FGs was made up of a range of farmer types, the majority of whom were women (Table 2). Creation of Farmer Institutions: The institutions which were established include the NAADS Board, National Farmer Forum, District Farmer Forum (FF), Sub-county Farmer Forum (SFF), and FGs as stipulated in the guidelines. All farmer fora started with interim farmer leadership committees for one year, to allow time for groups to be established with minimum registration requirements. Farmer fora at sub-county level and procurement committees worked with the sub-county NAADS coordinator (SNC) and sub-county technical team on matters of enterprise selection, development, monitoring, and operational matters. NAADS revised the guidelines in 2004/05 financial year and introduced Parish Coordinating Committees (PCCs) in order to fill the communication gap and to facilitate interface between the sub-county farmer fora and the FGs. The Community Based Facilitator (CBF) was also introduced as an agent of implementation. By the closure of the project in December 2009, 8,126 CBFs had been recruited, 2,765 in 2008/09; and more than 70 percent of them facilitated with bicycles. At this time, there was an average of two CBFs per parish. Seventy percent of the CBFs were male. By December 2009, 731,340 farmers had been reached by CBFs, with 248,849 reached in 2008/09. Group Strengthening: FID focused on governance systems and building farmer’s skills for groups and included training on cross-cutting issues, i.e. HIV/AIDS, gender, and natural resources management. Groups in NAADS manifest some positive results such as regular elections, collective savings, labor and management of Technology Development Sites (TDSs). Older groups formed before NAADS manifested stronger elements of group cohesion compared to those formed as a result of NAADS. Farmer institutions provided opportunities for empowerment through participation and in decision-making in the program.

Table 1: Expansion of NAADS into new districts and sub-counties

FY # Districts # Sub-counties 01/02 9 24

24

02/03 16 75 03/04 10 56 04/05 10 127 05/06 13 67 06/07 6 18807/08 15 211 08/09 0 27809/00 0 40Total 79 1066

Source: Government ICR Component 2: Agricultural Advisory Services(Formerly: Technology Development and Linkages with Markets)

Achievements under this component are rated satisfactory.

Three indicators were specified under this component: (i) Number of agricultural advisors employed by NAADS (target = at least 1,033 contracts; (ii) Number of farmers being served by NAADS (by gender) (target = 55,000 FGs); and (iii) Number of farmers who are satisfied with service (target = maintain level of satisfaction of 86 percent in new districts and sub-counties). Indicator (i): Between 2001 and 2007, a total of 4,942 service provider contracts were issued by NAADS. Indicator (ii): Table 2 describes the rapid increase in NAADS group membership from 2001 until 2009. By the end of 2009, there were 55,000 FGs that were affiliated with NAADS. There were 1.1 million farmers that were members of these groups and around half of all members were women.

Table 2: Extent of NAADS membership by Gender Members FY Number of

Groups Total Number Male (%) Female (%) 01/02 206 4,120 1,747 (42.4) 2,373 (57.6) 02/03 4,911 98,220 39,779 (40.5) 58,441 (59.5) 03/04 10989 219,780 95,385 (43.4) 124,395 (56.6) 04/05 15,900 318,000 146,598 (46.1) 171,402 (53.9) 05/06 25,000 500,000 227,000 (45.4) 273,000 (54.6) 06/07 38,515 770,300 368,203 (47.8) 402,097 (52.2) 07/08 42,000 840,000 413,280 (49.2) 426,720 (50.8) 08/09 55,000 1,100,000 531,300 (48.3) 568,700 (51.7) Source: Government ICR Indicator (iii): The IFPRI impact evaluation reports that 86 percent of HHs surveyed in 2007 rated the NAADS training as being satisfactory (compared to 79 percent for government extension workers). Field visits by the team and other agencies and media coverage suggest that the level of satisfaction was lower after 2007. It is very likely that the evidence from the HHs surveys conducted in 2007 (and evidence from field visits before 2007) is accurate and, between 2001 and 2007, NAADS was a popular program. Technology development included introduction, multiplication, adoption and dissemination of recommended practices and approaches. The Technology Demonstration Sites (TDSs) were used to contribute to awareness creation and capacity development, and provide stocks through multiplication of

25

planting material. For this purpose, NAADS implementation guidelines, which included clear steps on accessing technology development funds, management of a TDS, and the roles of both service providers and district technical teams were developed and launched. There is consensus that, generally, these guidelines were followed. The TDSs were universally popular among farmers as they provided opportunities for on-site learning of farming practices and technologies, and also as a source of inputs. Achievements/Outputs:NAADS provided technologies for demonstration on the fields--TDSs of FG members (host farmers) to support the selection of enterprises. Host farmers were chosen by fellow members of the group, and private service providers were contracted to carry out demonstrations and to train farmers at these TDSs. By the close of the project in December 2009, up to 1,100,000 farming households in 1,066 sub-counties had benefited from NAADS, with a 100 percent outreach to all districts. People with Disability (PWD), People Living with HIV/AIDS (PLWA), youth and women participate with varying degrees of success in the program. Low income farmers were involved in the program hence the potential for the NAADS interventions to contribute to poverty alleviation. Overall, the major crops promoted included banana, groundnuts, rice and maize, with banana and groundnuts topping the scale, in terms of the number of TDSs that were established or demonstration held, as well as the period they were in existence. Banana was promoted in more than 10 sub-counties (or more than 35 percent of the total NAADS sub-counties in any year) since 2002, and reached 23 sub-counties (or 60 percent) by 2007. Comparing rice and groundnuts, although they were both promoted in 16 sub-counties (or 43 percent of the total number of sub-counties) by 2007, rice was not promoted at all in 2001 and 2002, and in less than 6 sub-counties (or 18 percent of the total) until 2005. Groundnuts, on the other hand, were promoted since 2001. Regarding livestock enterprises, the major ones promoted were goats (in 35 sub-counties, poultry (27), beekeeping (24), cattle (23), and piggery (18) With respect to the number of TDSs established, groundnuts and rice were the most favored crops, with 540 and 535 TDSs respectively being established by 2007. For livestock enterprises, goats, beekeeping and poultry were the most favored, having 441, 458 and 478 TDSs respectively set up by 2007. In terms of the number of registered NAADS FGs directly benefiting from the TDSs and demonstrations, banana, groundnuts and rice were the most favored crop enterprises, with 1,512, 1,247 and 1,132 FGs benefiting by 2007, respectively. As far as livestock enterprises were concerned, goats and poultry were the most favored with 2,144 and 2,133 FGs, respectively, benefiting under each enterprise by 2007. Demonstration sites and sources of inputs: For many farmers, the TDSs served as sites for practical demonstrations and as sources of inputs. Farmers benefitted more from practical demonstration and hands on rather than theoretical training. TDSs were also used for multiplication of technologies. In Kyenjojo district, the Kyakatume Banana Technology Development Centre at Kyagegwa showed a range of banana husbandry techniques and also provided an important source of improved banana suckers for all group members. Goat TDSs in several districts revolved around the provision of improved male (Boer) goat, enabling the production of hybrid (local-improved) goats for local farmers (primarily members of the FGs hosting the TDS but also non-NAADS farmers). Generally, where farmers did not have the resources to apply the knowledge learned from Agricultural Advisory Services, stocks (crop or livestock) from the TDSs served to ‘kick-start’ new enterprises. Farmers indicated that training sessions at TDSs were better attended than those at central (i.e. parish) locations. At TDSs farmers learned about best practices covering for example, land preparation, crop husbandry, breeding, feed and management of livestock and improved management in other enterprises such as fish farming and bee keeping. At central locations, only theoretical training was available.

26

Management of TDSs: NAADS was flexible regarding the ways in which FGs managed the TDSs. Generally, TDSs were managed by the FGs themselves, with some guidance from PCCs. The FGs set up TDS management committees to oversee management of sites. While some TDSs were managed mostly or entirely by one host farmer, others are managed collectively. The TDSs were generally hosted by a member of one FG, nominated by that FG, under terms agreed within the group. The extent to which the host farmer as opposed to the group members, provided labor and inputs for the enterprise is agreed amongst the group as is the sharing of the benefits. The benefit of this type of management framework is in the experience gained by the manager. Component 3: Agribusiness Development and Linkages with the Market

Achievements under this component are rated satisfactory.

Four indicators were specified under this component (i) Number of technology development activities undertaken in participating sub-counties with the help of scientists and technicians contracted under the direction of NAADS advisors and farmers with funding from NAADS (target = 73,000 demonstration sites established in 2008/09 (constituting conversion of some ISFG beneficiaries to demonstration sites and addition of new demonstration sites); (ii) Number of marketing workshops and marketing oriented projects undertaken by farmers and advisors in participating sub-counties with the help of agri-business advisers (target = 40 public private partnerships in 2008/09); (iii) Number of information and communication initiatives undertaken in participating sub-counties with the help of technical assistance contracted; and (iv) Increased Overall demand for improved technologies (no target set). Indicator (i): There were about 15,000 Technology Demonstration Sites established by May 2008, and 78,400 farmers benefited out of ISFG. Indicator (ii): 47 public private partnerships were completed or were in progress, with five others in the pipeline by May 2008. Table 3 provides details on 11 out of 40 public-private partnerships undertaken by NAADS. In addition, there were 9 market surveys conducted by the NAADS secretariat and these findings were widely disseminated. Indicator (iii): All districts now have regular radio programs that provide information on market prices of key commodities and other information that is intended to help farmers better market their produce. In addition, there are 17 newsletters produced by the Secretariat and the districts and there were also 20 flyers produced by the Secretariat. Indicator (iv): As described in detail in Table 5 (in section 3 of the text), a much higher proportion of NAADS members demand advisory services (31.7) compared to non-members in NAADS sub-counties (6.2) as well as HHs in Non-NAADS sub-counties ( 10.0 ). Achievements/Outputs: Information Outreach: All districts in the Program established and broadcast regular marketing related and other radio programs, 17 newsletters were produced by the secretariat and districts, and 20 flyers were produced by the secretariat. Higher Level Farm Organizations (HLFO): NAADS encouraged the development of enterprise based HLFOs at district and sub-county levels to provide a means for farmers to play an active role in the value chain. As at December 2009, there were more than 200 HLFOs formed with the aim of developing crop-based, tree-based or livestock-based enterprises as agri-business ventures. Deliberate effort was also made to encourage formation of HLFO for bulking, collective marketing and value addition.

27

Public Private Partnerships (PPP): The NAADS Program developed a strategy for agro-processing and marketing based on the PPP principle. PPPs provided a stepping stone for FGs and HLFOs moving into commercial production to reach a market and become more involved in the commodity value chain. The PPPs encouraged private sector companies to work with NAADS on a larger and more comprehensive scale than district SPs, and at the same time provided some capacity building for SPs. Similarly, with private sector actors, NAADS supported FGs and associations in value addition and agro-processing right from household level upwards. Through PPP, NAADS tapped existing potential in both the public and private sectors to: complement the delivery of improved technologies; agricultural advisory services; value addition through agro-processing; market linkages; and financial resources. As at December 2009, NAADS had implemented over 25 PPPs (Table 3).

Table 3: NAADS Public and Private Partnerships

No Enterprise: Partners Objective Amount (USh

Mn) 1. Tea: Good African Tea,

Kayonza Tea, Igara Tea, Mpanga Tea, Mityana Tea, Mabale Tea

Promotion of primary and secondary production through supply of planting materials, advisory services, value addition through agro-processing and market linkages

2,500

2. Coffee: Ankole Coffee Processors Ltd, Mountain View Farm

Promotion of primary and secondary production through supply of planting materials, advisory services, value addition through agro-processing and market linkages

250

3. Grapes: Uganda Grape Growers Dev Association

Promotion of primary production through supply of planting materials, advisory services, and market linkages

80

4. Horticulture: (Citrus, mangoes, spices, etc) : Bukonzo Farm, Mudusu Projects, etc

Promotion of primary production through supply of planting materials, advisory services, and market linkages

350

5. Rice: Upland Rice Millers Promotion of primary production through supply of planting materials and advisory services; and value addition through agro-processing, and market linkages

200

6. Oil Seeds: A. K. Oils (Mukono)

Promotion of primary and secondary production through supply of planting materials, advisory services, value addition through agro-processing and market linkages

300

7. Dairy: Tooro Dairy Promotion of secondary production – value addition through processing, and market linkages

400

8. Goat: Sembeguya Farm Ltd, Katebe Farm Ltd

Promotion of goat breeding and supply of improved goats to out growers, extending technical advisory services and offering market linkages

260

9. Market Information: UCE & MPAIS, Media Centre, Fit Uganda

Provision of market and other relevant information to key stakeholders, and market linkages

150

10. Finance: Micro-finance Support Centre, DFCU

Linkage to source of finances and capacity development in financial mobilization and

Not Applicable

28

No Enterprise: Partners Objective Amount (USh Mn)

Bank management 11. Institutional:

NARO,ICRISAT, NAGRC & DB

Sources of clean planting and stocking materials, disease & pest management, and quality assurance

450

Source: Government ICR

Component 4: Service Provider and Institution Capacity Development

Achievements under this component are rated moderately unsatisfactory.

Three indicators were specified under this component: (i) Number of contracted service providers participating in periodic training, facilitation and coordination events (target = trainings to be held in all 83 districts for new modalities of service provision); (ii) Number of potential service providers meeting required standards (taret = at least 1 service provider per sub-county); and (iii) Number of contracted service providers who performance is assessed as satisfactory (target = at least 86 percent level of satisfaction). Indicator (i): The government ICR reports that, by 2008, 1,510 private service providers were trained in various fora and events. The frequency and quality of this training is uncertain. The Government ICR also states that In the early NAADS districts, the number of SPs was high. For example, as at 2008, there are 21 SPs in Luwero and in Kabale. The numbers were such that the district carried out a pre-qualification process. Some newer districts suffered from acute shortage of SPs, the majority of who were individuals. Where there were only a limited numbers of SPs there was less competition in the selection process and this resulted in a lowering of standards of service delivery. Field visits by the ICR team also gathered that there was a shortage of qualified service providers in remote areas. Indicator (ii): The 2008 project paper reports that, as of 2005, 383 service providers were approved by NAADS. The government ICR states that at least one service provider per district has been approved. The success of the program is contingent on the availability of qualified service providers. This lack of capacity is perhaps a reflection of the rapid expansion of the program. Indicator (iii): This indicator is identical to indicator (iii) of component 2 and has been discussed above. Challenges:Limited availability and capacity of Service Provider (SPs): There was insufficient capacity in terms of quantity and quality of SPs at the lower levels. The initial assumption that many SPs would be qualified and experienced technical or extension staff who had been de-layered in each district and sub-county was not borne out as, for the most part, they remained employed under the LG. Second, the one sub-county extension staff member who remained employed by the LG and yet worked exclusively on NAADS was the sub-county NAADS coordinator. Third, while some capacity building of SPs has actually taken place in the field, it was both minimal and relatively unsystematic. Some SPs (firms and individuals) indicated that they had received little or no capacity building from NAADS. This is confirmed by the assertion by many local government employees, politicians and some farmers across districts who argued that SPs were not adequately qualified or sub-contracted the work to young graduates with no relevant experience or background in agriculture, thereby undermining the quality of service delivery. Even for experienced, ex-government technical and extension staff, given the wide scope of advisory services, further capacity building of SPs remains a critical factor (Government ICR, p. 33).

29

Individuals vs. Firms: The registration of all potential SPs as individuals (rather than members of a firm) seemed to be the preferred policy option. Principally it provided a means of preventing sub-contracting –the individual who wins the contract is the person who must deliver the service. This approach is viewed by the majority of stakeholders, particularly by farmers, as appropriate. However, the question of whether firms are more effective than individuals, and the logic for using firms, remains unanswered.

Component 5: Planning, Monitoring and Evaluation (M&E)

Achievements under this component are rated unsatisfactory.

Two indicators were specified under this component: (i) Percentage of districts where training to project staff and district and sub-county/village administrators in economic and social impact monitoring have taken place (target = training in all new districts (4 districts); and (ii) Implementation of M&E activities. While there were regular reports (including some technical audits, quarterly reports at all levels and semi-annual and annual reviews), the content and quality of these reports were not consistent with a truly participatory M&E system that would have held both service providers as well as local administrative officials and politicians accountable to the farmers. The Government’s own ICR highlights the following challenges of the M&E system: Lack of Overall M&E Framework: A lot of program information was collected using Implementation Guidelines (2007), which describe the main M&E activities. However, there was no overall framework for M&E describing the objectives, roles, responsibilities and periodicity of M&E activities. Therefore, although the program carried out a number of relevant M&E tasks, they were not part of a well defined M&E framework and Program objectives and the indicators at impact, outcome, output/result and activity levels were not comprehensively set out. Understaffed M&E team: The PME team at the NAADS Secretariat is small with only five full-time staff. The effect was a system defaulting to a progress reporting system aimed more at triggering payments to districts than to providing management information. Given the size and importance of the role of PM&E in the program ranging from the national, district and sub-county levels, this needs to be reviewed and enhanced. Incomplete Information Database: A database was developed to capture quantitative and qualitative data submitted from lower level units. Additionally all members of technical staff were, while in the field, obliged to monitor program components and collect relevant data with view to continuously updating the database. However, the database did not contain up-to-date and complete program implementation information. Further work will be required to rectify this situation and to continuously update the database. Insufficient Baseline Indicators: Some baseline surveys were undertaken in Soroti, Kabale and Mukono districts in 2001 on a pilot scale. It is reported that, because of flaws in the methodology and approach, the information could not be replicated. No further surveys were carried out to enhance the baseline indicators. This lack of comprehensive baseline information on the program undermined efforts to assess the realization of planned outcomes.

30

Inadequate Monitoring by Sub-county Farmer Fora (SFF): The SFFs established M&E teams to monitor the program at the sub-county level. However, the frequency of monitoring was irregular as it was determined by the availability of resources. Where the SFFs were unable to undertake monitoring visits, they depended on the reports by PCCs and on the semi-Annual or Annual Review meetings. Complicated Participatory Monitoring and Evaluation (PME) Tool: In some districts and sub-counties, the PME form was seen as too complicated. It was also perceived as a top down monitoring tool although the Secretariat indicated that the form was designed in consultation with local teams and had their approval. While some progress was made to rectify the situation locally with some DNCs and SNCs taking the initiative to develop an abridged version, this needs to take place across the whole country. Inadequate Linkages with local governments: As local governments are autonomous of the NAADS Secretariat, the latter does not have a strong (outside the participation MOU) framework to enforce timely collection of and reporting on field data. As a result the flow of information from lower governments to the Secretariat was slow. The result is the NAADS Secretariat was not always up to date, especially after the re-structuring. Component 6: Program Management

Achievements under this component are rated satisfactory.

Four indicators were specified under this component: (i) NAADS board established and functioning effectively; (ii) NAADS secretariat fully staffed and functioning effectively; (iii) National Farmer Fora established and functioning effectively; and (iv) Farmer fora established and functioning in all participating districts and sub-counties (target = establishment of farmer fora in all new districts (4) and sub-counties (285). Indicator (i): The NAADS Board was established in 2002 and has conducted quarterly meetings. Indicator (ii): The NAADS Secretariat was established and has been ably managed by a dynamic Executive Director. The ICR team believes, however, that the secretariat is understaffed. Indicator (iii): A National farmer organization was established in 2007. Indicator (iv): Farmer fora have been established and function in all participating districts and sub-counties. Achievements/Outputs:NAADS Secretariat: The NAADS governance structures aimed to ensure the program was guided within the national planning and implementation framework, that there was adequate harmonization and therefore reduced duplication of efforts, and that the arrangements for national responsibility for the program were clear. During this phase, NAADS had two Boards; an interim board which operationalized the program at the start of Phase 1 and the current full Board which was inaugurated in 2009. The Board met regularly and provided guidance to the Secretariat to become innovative and to address farmers’ bottlenecks, especially in complementary areas such as market access and ISFG, where the PMA implementation was lagging behind. The Board also played a key role in monitoring NAADS activities directly in the field. It designated its members to oversee at least three districts each, and to report back to the full Board on key achievements and challenges. In this way, the Board kept abreast with developments in the program throughout the country.

31

The NAADS Board established three sub-committees, namely: Finance and Administration; Marketing; and Technical. These committees helped the Board to process in much more detail key issues for easier decision–making, and provided a means for management to improve on its utilization of members’ expertise and experience in the agriculture sector. As most decisions and functions of routine management were responsibilities of the appropriate farmer fora, sub-county and district personnel, the major tasks of the Secretariat remained to provide technical guidance and operational oversight and to facilitate outreach and impact. The NAADS Secretariat was therefore deliberately kept lean to cover the functions of management and administration, planning M&E; and backstopping on technical and social services. Where additional support was required beyond the capacity of the Secretariat, it was outsourced to private professional firms and individuals. The Secretariat also coordinated the annual joint GOU/Donor planning and review missions and the implementation of the program, including sourcing and accounting for the required financial resources in accordance with the Credit and financing agreements. Local Governments: In each participating district, the program financed the contracting a District NAADS Co-coordinator (DNC) and a sub-county NAADS Coordinator. The DNC was responsible for ensuring the smooth operation of NAADS program and integration of NAADS into the district annual work plan and budget. At sub-county level the program finances the incremental operating costs of a staff member to be assigned as sub-county NAADS Co-coordinator. This officer coordinated the implementation of program activities at this level National Farmers’ Forum: The National Farmers Forum was inaugurated in August 2007. This was the final step in establishing farmers’ institutions. It enabled farmers to link and participate directly, influence the national agricultural development agenda, and to voice their concerns on issues affecting the program at national level. The district farmer associations successfully introduced the concept of membership fees, which by far exceeds the individual farmer co-financing of NAADS. It is estimated that about 17 percent of farmer association members are also members of NAADS groups.

32

ANNEX 2 APPENDIX 1. KEY PERFORMANCE INDICATORS

Key Performance Indicators

Baseline Values

Progress till 2007

Unadjusted Difference (2007 values)

Adjusted Difference Comment

Measures of productivity in farm enterprises of NAADS participants exceed Ugandan Averages. Value of Total Agricultural Income per capita

195,000 Ush (2000) in 2004

314,000 Ush (2000) in 2004 NA NA

Value of Total (Net) Agricultural Income per capital (Alternative indicator)

This value was not measured in the 2004 IFPRI survey

117,374 USh (2000) in 2007 for NAADS and 75,988 USh (2000) for Non-NAADS

41,386 Ush (2000)

33504 Ush (2000)

Baseline data source and composition of households sampled unknown (see annex 3, appendix 1).

Difference in Value of Agricultural Production per capital based on NAADS member Status

This value was not measured in the 2004 IFPRI survey

155,678 Ush (2000) in 2007 for NAADS and 97,834 USh (2000) for Non-NAADS

57,843 Ush (2000) : 59.1 %

51,350 Ush (2000) : 46.1%

Alternative Indicator I : Value of (Net) Crop Income per capita

This value was not measured in the 2004 IFPRI survey

100,762 Ush (2000) in 2007 for NAADS and 66,098 USh (2000) for Non-NAADS

34,664 Ush (2000) : 52 %

35,489 Ush (2000) : 47 %

The adjustments accounts for cross-sectional socioeconomic and regional differences in households that received program benefits. However, since we do not have values for the baseline we are unable to eliminate all fixed unobervables that engender treatment bias.

Measures of household wellbeing improved over time for majority of participating farmers Value of Total Agricultural Productive Assets per capita for NAADS Beneficiaries

162,690 Ush (2000) in 2004

230,600 Ush (2000) in 2007

67,910 Ush (2000) NA

Alternative Indicator II : % Increase in real value of total wealth per capita (2004-2007)

54.5 % for NAADS and 38.2 % for Non-NAADS 16.2% 17. 8%

The relative real wealth per capita impact of the program is large and statistically significant and is similar for both double difference estimation with propensity score matching (reported here) as well as double difference regressions controlling for initial wealth and household socioeconomic characteristics and district dummies (see Annex 3).

Number of men and women farmers and percentage of client farmers who are adopting improved technologies and management practices as a result of NAADS

% of beneficiaries using improved seeds

52.5 % for NAADS and 23.45% for Non-NAADS

68.3% for NAADS and 53.3% for Non-NAADS

IFPRI NAADS Impact Evaluation

33

% of beneficiaries using improved breeds

24.5% for NAADS and 11.3 for Non-NAADS

30.9 for NAADS and 29.8 for Non-NAADS

ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS

A. Economic Analysis in the Project Appraisal Document The analysis in the PAD suggests that a moderate improvement in productivity is consistent with a positive NPV. Specifically, the PAD states that an annual increase US$7 in farm net incomes or a 0.4 percent increase in agricultural GDP is consistent with the project being economically and financially viable. Further the PAD suggests that, if by year 7, NAADS reaches 14 percent of the farmers in Uganda and 33 percent of these adopt improved crop practices, NAADS will be economically viable. While the ICR team does not have the requisite information to properly evaluate these statements, this annex attempts to identify, and attribute to the project, the benefits to farm HHs, and consequently, determine the ex-post economic viability of the project. B. What is the Relevant Counterfactual? To determine the causal impact it is essential to assess the outcomes experienced by NAADS member HH against those experienced by a comparable control group. The matrix below describes the strengths and weaknesses of alternative control groups against which the ICR assesses project outcomes. While it is straightforward to control for relevant HH characteristics like education and gender, it is harder to control for social capital (non-NAADS group membership takes on a different meaning in NAADS and non-NAADS counties and, in any event, its effect on outcomes is unlikely to be simply additive), locational factors (with respect to comparison HHs that are in non-NAADS sub-counties), and the NAADS-induced general equilibrium effects and other spillovers within the sub-counties in which it was active. Table 1: Costs and benefits of comparing outcomes against alternate control groups Control Group Relative Strengths and Weaknesses Non-NAADS Group Membership Location

Strengths : Controls, Size

Weaknesses : Confounding Effects , Sample Size

Members/Non-Members All Sub-counties large sample location, social capital and NAADS GE effects

Non-Members NAADS Sub-counties location

social capital, NAADS GE, sample size

Members All Sub-counties social capital location factors, NAADS GE

Members NAADS Sub-counties

social capital, location NAADS GE , sample size

Members Non-NAADS Sub-counties

social capital, NAADS GE location, sample size

As Table 1 shows, there isn’t a clear counterfactual that dominates the others on all dimensions. Therefore, the ICR presents evidence that compares NAADS outcomes against different control groups. Each comparison embodies the particular strengths and weaknesses discussed above. C. Econometric analysis of the causal mechanisms connecting NAADS’ outputs to final outcomes The following section demonstrates the causal impact by answering a series of questions that address the key issues that are essential for the success of a demand-driven agricultural extension service.

34

1. Which HHs became NAADS members and consequently benefited from the program? At least until 2007, NAADS contributed to the formation of FGs that were more inclusive.Although HHs that joined NAADS-affiliated FGs were better educated, they were not necessarily wealthier than HHs in other groups. Table 2 shows that relative to members of other groups, NAADS members were 8.5 percent more likely to have a female HH head and 10 percent more likely to have a primary education.

Table 2: Determinants of NAADS membership relative to different comparison groups Likelihood of being a NAADS Member (%)

All NAADS Non-members

Other Group Members

NAADS-Non Members in NAADS Sub-counties

Female HH Head 7.1 8.5* 7.6 Smallholder -15.7*** -8.9 -16.8*** Primary Education 12.8*** 9.9** 15.1*** Secondary Education -2.3 2.5 -4.7 Age of Head 4.6 -1.0 5.4* Size of HH 10.0 14.3 11.8 Observations 646 429 585

Notes: The covariates are estimated from regression models that include controls for district dummies. *, **, and *** denote significance at 10, 5 and 1 percent levels based on robust standard errors clustered at the district level. 2. Are there more groups in NAADS sub-counties and are these groups more inclusive? Table 3 describes the incidence and composition of group membership in sub- counties that were part of the NAADS program and sub-counties that were not. 62.9 percent of the rural HHs surveyed were part of a collective organization in NAADS sub-counties, while only 43.9 percent of HHs in non-NAADS sub-counties were part of a collective organization. Further, there were significantly more women and smallholders who were members of groups in NAADS sub-counties.

Table 3: Comparison of membership incidence in NAADS and non-NAADS sub-counties Incidence of Group Membership (%)

NAADS Sub-county Non-NAADS Sub-county Total Group Women Smallholder All Women Smallholder All

NAADS 41.6 31.1 42.5 0.0 0.0 0.0 30.3 Non-NAADS 26.9 19.4 20.6 42.3 32.6 43.9 23.8 Any Group 68.2 50.5 62.9 42.3 32.6 43.9 54.1 Observations 151 265 840 52 150 340 1180 Note: The measures are calculated using simple averages from the sample collected using the HH survey. 3. Did NAADS members have a greater voice in evaluating service providers? Farmers in NAADS groups felt more empowered in terms of evaluating the performance of service providers (Table 4). This was the main rationale behind the program.

Table 4: Accountability of NAADS service providers compared to other service providers Percentage Satisfied with their Voice in Evaluating Service Providers

NAADS Sub-county Non-NAADS Sub-county Total Group Women Smallholder All Women Smallholder All NAADS 50 56.2 49.2 - - - 49.2 Non-NAADS 0 33.3 27.2 - 50.0 43.4 32.3 Any Group 44.4 53.7 47.1 - 50.0 43.4 46.8

35

Observations 45 54 227 0 4 23 239Note: The measures are calculated using simple averages from the sample collected from the HH survey. 4. What were the primary benefits from joining a NAADS group? HHs received two primary benefits from joining a NAADS group: (i) greater access to advisory services; and (ii) support. Table 5 shows that NAADS group member HHs were over 40 percent more likely to benefit from training and advisory services. Further, NAADS group members were much more likely to benefit from support in the form of credit, grants and subsidies compared to most HHs. The only exception was when the control group was restricted to members of other groups in non-NAADS sub-counties (2.3 percent). This difference was not statistically significant. This may be because members of these groups had the necessary social (and political) capital to receive support. However, it is worth noting that even compared to this control group of (other) agricultural HHs with social capital, NAADS members were much more likely to have access to agricultural training and advisory services.

Table 5: Benefits from being a NAADS group member relative to different controls Control Group (%) Benefit of NAADS Group Membership

Non-NAADS Group Membership

Location Training & Advisory Services

Credit, Grants & Subsidies

Members/Non-Members All Sub-counties 46.1*** 25.3*** Non-Members NAADS Sub-counties 46.2*** 25.7*** Members All Sub-counties 44.8*** 19.4** Members NAADS Sub-counties 44.0*** 20.5** Members Non-NAADS Sub-

counties 34.4*** 2.3

Notes: The covariates are estimated from regression models that include controls for household and regional characteristics. * significant at 10%; ** significant at 5%; *** significant at 1% are based on robust standard errors

5. Was the service provided to NAADS members driven by farmers’ needs? The nature of the advisory services that NAADS members received was different from non-members. NAADS members were more proactive in demanding visits from extension workers. 31.7 percent of NAADS members demanded advisory services (Table 6), and it is likely that these services were more useful. This represents a significant break from the old T&V supply-driven extension model.

Table 6: Demand for agricultural extension services by NAADS member status Percentage of HHs Requesting Advisory Services in NAADS Sub-counties

Service Provider

Women Smallholder NAADS member

Non-member Full Sample

NAADS 9.2 8.2 25.7 1.8 11.9 Other Govt. 1.3 1.5 2.5 1.2 1.8 Group Member 2.6 3.7 7.6 0 3.6Other Farmer 1.9 5.2 7.3 2.1 4.2 NGO 1.9 2.9 7.1 1.6 3.8 Other 0 0 0 0 0Any Provider 12.5 13.8 31.7 6.2 16.9 Observations 152 268 353 477 851

Percentage of HHs Requesting Advisory Services in Non-NAADS Sub-counties Service

Provider Women Smallholder NAADS

member Non-member Full Sample

NAADS 0 0 - 1.3 1.3Other Govt. 1.9 3.3 - 4.3 4.3

36

Group Member 0 0 - 0 0Other Farmer 0 0 - 1.9 1.9NGO 0 2.6 - 3.2 3.2Other 0 0 - 0 0Any Provider 1.9 6.6 - 10 10Observations 52 150 0 368 368

6. Did NAADS members receive more support and was this support equitably distributed? Also, did NAADS support crowd out or stimulate other forms of support? In addition to receiving advisory services, NAADS members were also more likely to receive support in the form of credit, grants and subsidies, and, this support was fairly equitable until 2007 (Table 7). More than half the women and smallholders who were NAADS members received some support, significantly more than comparable HHs who were non-members and HHs in non-NAADS sub-counties.16

NGOs in NAADS sub-counties did tend to favor NAADS member HHs over other HHs in the sub-county (Table 7). It may be that NAADS groups were able to attract greater attention from the NGOs. In addition, smallholders and female non-members in NAADS sub-counties were much more likely to resort to informal (perhaps more expensive) forms of support than similar NAADS members or similar HHs in non-NAADS sub-counties. For example, 10.4 percent of non-NAADS smallholder HHs in NAADS sub-counties received informal support compared to 3.3 percent of smallholders in non-NAADS sub-counties and just 3.6 percent of NAADS members. It is possible that this is a reflection of the decreased attention these HHs received from NGOs due to the focus on NAADS groups. Further, as Table 7 shows, a comparable proportion of HHs in NAADS and non-NAADS sub-counties received support from other government sources (4.7 and 4.1 respectively). This suggests that, at least in terms of incidence, NAADS did not reduce the proportion of HHs receiving support from other government sources. Taken together, the composition and incidence of support from Table 7 suggest that it is very likely that NAADS may have attracted more support in those sub-counties in which NAADS was active, while some disadvantaged non-members in those sub-counties may have been forced to resort to commercial and informal sources of support due to decreased attention from NGO’s.

Table 7: Composition and inclusiveness of support by NAADS member status Percentage of HHs Receiving Support in NAADS Sub-counties

NAADS Member NAADS non-member Source Women Smallholder All Women Smallholder All Full Sample NAADS 24.1 32.9 35.1 1.1 2.2. 1.8 15.7 Other Govt. 8.1 6.1 5.9 6.8 4.4 3.7 4.7 NGO 20.9 12.2 15.6 5.7 4.4 6.9 10.5 CBO/MFI 8.1 13.4 14.4 16.1 12.7 13.6 13.9 Informal 4.8 3.6 7.3 9.1 10.4 6.4 6.8 Any Support 54.8 59.7 64.5 37.9 32.6 31.8 45.4 Observations 62 82 353 87 181 477 851

Percentage of HHs receiving support in non NAADS sub-counties NAADS Member NAADS non-member

Source Women Smallholder All Women Smallholder All Full Sample NAADS - - - 0 2 1.1 1.1Other Govt. - - - 3.8 2.6 4.1 4.1

16 The table only provides evidence for incidence of support, not the extent of support conditional on incidence. The “optimal” distribution of support depends on the nature of the support, its purpose and the relevant HH characteristics (including land under cultivation) and consequently is harder to interpret in normative terms.

37

NGO - - - 17.3 8.6 8.6 8.6 CBO/MFI - - - 13.4 7.3 6.3 6.3 Informal - - - 1.9 3.3 3.5 3.5 Any Support - - - 36.5 22.6 22.8 22.8 Observations - - - 52 150 368 368 Note: The measures are calculated using simple averages from the sample collected from the HH Survey.

7. Did NAADS members have greater access to information on improved crop and livestock technologies and practices? Table 8 describes the relative advantage NAADS group’s members had with respect to access to information on agricultural prices, technologies and practices relative to three control groups – all HHs that were not members of NAADS groups, members of other groups and finally HHs in NAADS sub-counties that were not members of NAADS groups. For example, relative to non-member HHs, NAADS members were 17.8 percent more likely to have access to information on how to better utilize crop production equipment and 13 percent more likely to have access to information on improving crop production practices. Compared to members of other groups, NAADS members had greater access to crop practices and technologies but not prices. However, because it helped to form groups, it is likely that NAADS had a significant impact on improving price information. The significant difference between NAADS members and non-members in NAADS sub-counties speaks to this issue.

Table 8: Informational benefits from being a NAADS group member % HH with Improved

Information Control I:

All NAADS Non-Members

Control II: Other Group Members

Control III: NAADS-Non Members in

NAADS Sub-counties Crop Production Equipment 17.8*** 15.9*** 18.4***

Crop Production Practice 13.0** 10.2** 13.7*** Livestock Production

Equipment10.0* 4.8 11.1*

Livestock Production Practice

9.1* 7.7 10.0*

Crop Market Prices 10.5* 5.0 10.9* Livestock Market Prices 11.0** 5.5 11.6***

Notes: The covariates are estimated from regression models that include controls for household and regional characteristics. * significant at 10%; ** significant at 5%; *** significant at 1% are based on robust standard errors

8. Did NAADS members adopt better agricultural production techniques? As Table 9 shows, NAADS member HHs were more likely to experiment with new enterprises and new crop practices. For example, controlling for covariates that would bias the estimates, NAADS members had a 10 percent greater likelihood of using improved crop practices.

Table 9: Adoption of improved agricultural practices by NAADS group members % HH with Improved Practice Control I:

All NAADS Non-members

Control II: Other Group Members

Control III: NAADS-Non Members in NAADS Sub-counties

Crop New Use 5.4* 9.4** 5.1 Used Improved Crop Practices 10.7*** 16.3*** 10.8*** Notes: The covariates are estimated from regression models that include controls for household and regional characteristics. * significant at 10%; ** significant at 5%; *** significant at 1% are based on robust standard errors

38

Finally, these improved intermediate outcomes were expected to increase farm crop and livestock incomes and ultimately HH wealth. The available evidence suggests that this was indeed what happened, at least until 2007. As Table 10 shows, NAADS group members experienced large gains in real crop revenue per capita and real HH wealth per capita when compared to several different control groups.

9. Did NAADS members experience larger gains in agricultural productivity and household welfare relative to comparable farmers that were not members of a NAADS affiliated group? It is worth noting that members of other groups in NAADS sub-counties also experienced wealth gains that were probably the result of greater support, but even relative to this group, NAADS members experienced large gains in per capita crop revenue.

Table 10: Final outcomes relative to different comparison groups Control Group (%) Change in Outcome Variables : 2004-2007

Membership in Non-NAADS Groups

Location Real Crop Revenue per capita

Real Wealth per capita

Members & Non-Members

All Sub-counties 42.1*** 27.4***

Members & Non-Members

NAADS Sub-counties 40.7*** 25.8**

Members All Sub-counties 28.7*** 4.7 Members NAADS Sub-counties 26.9*** 0.8 Members Non- NAADS Sub-counties 30.9*** 70.9***

Note: The coefficients are estimated from double difference regressions controlling for HH and regional characteristics in the base year, in addition to the base year values of the relevant outcome variable. These variables are controlled for in order to reduce the selection bias arising from the fact that differences between NAADS members and non-members in the base year may have caused different growth rates in the outcome variable and consequently, without an adjustment, may bias the final impact estimates. *, ** and *** denote significance at 10, 5 and 1 percent levels based on robust standard errors.

Table 11: Final impact of NAADS membership on females and poor Percentage Change: 2004-2007

Household Type (2004) Crop Revenue per capita Wealth per capita Female head 59.1*** 31.4 Smallholder 42.3*** 9.8 Primitive roof 55.0*** 45.5*** Notes: The impacts were estimated from regression models that in addition to controlling for initial outcome, HH and district characteristics in the base year include an interaction term between the NAADS dummy and the relevant household type. The coefficients and significance reported are based on the combination of the relevant coefficients. *, ** and *** denote significance at 10, 5 and 1 percent levels based on robust standard errors. D. Types of new information clusters within the set of NAADS households A key rationale for government intervention in the agricultural sector involves the provision of new information. Markets for information are often missing because information is a public good. As shown above, NAADS members were more likely to receive training as well as support (in the form of credit, grants and subsidies). Table 12 (below) demonstrates the composition of the types of new information made available to NAADS members and how these relate to the incidence of training and support. Table 12 allows us to draw three key conclusions on the composition of information provided by NAADS:

39

i) As expected, new information on crop equipment and practice are strongly correlated, as is information on livestock equipment and practice. This, perhaps, speaks to the need for different types of information based on soil, climate and culture. ii) Also, market information on crops and livestock are much more strongly correlated with each other than they are with information on practice and equipment. This correlation is a bit harder to explain on the basis of agro-ecological conditions. It is likely that HHs closer to markets have greater access to information (in addition to the fact that the information is worth more to them) and also that some NAADS groups are more active in collecting market information. iii) While NAADS training is significantly correlated with the information variables, the correlation between NAADS support and the information variables is quite low. This casts some doubt on the argument that support was provided to make better use of new information.

Table 12: Clusters of new information available to NAADS households

Crop Equip

.Crop Practice

LS Equip.

LS Practice

Marketing (Crop)

Marketing (LS)

NAADS Support Training

Crop Equipment 1.00 Crop Practice 0.76 1.00 LS Equipment 0.46 0.37 1.00 LS Practice 0.38 0.43 0.79 1.00 Marketing (Crop) 0.20 0.19 0.21 0.22 1.00 Marketing (LS) 0.17 0.14 0.28 0.26 0.74 1.00 NAADS Support 0.11 0. 09 0.10 0.09 0.12 0.05 1.00 NAADS Training 0.25 0.19 0.31 0.28 0.23 0. 25 0.21 1. 00 Note: The table shows pair-wise correlations of information variables, and training & support incidence. E. Heterogeneous impacts across agro-ecological zones NAADS’s impact was not uniform across agro-ecological zones. The project had an impact on crop production in all regions except the West Nile (tables 13a and 13b). It is possible that the significant relative wealth impact to NAADS members in the West Nile region was due to program-stimulated increases in livestock productivity. However, it must be noted that there were several other developments in the region during the 2004-2007 period; peace was gradually established during the period, in addition, a new road to Kampala was built and finally there was a rapid increase in exports of agricultural commodities to Southern Sudan. The absence of a substantial relative wealth impact in the Lake Victoria region may be due to the fact that there are many non-agricultural channels through which wealth is determined in this region. The Southwest Highlands may have benefitted from an unusual degree of government support during the last decade. Therefore, it is difficult to attribute wealth gains to NAADS within this region, during this period. However, the impacts across several other variables suggest that NAADS did have a positive influence on rural HHs in this region. Finally, the large relative crop income and wealth impact in the North (East) perhaps speaks to the fact that initial levels were very low and consequently the impact (relative to non-member HHs) may have been large in the period between 2004 and 2007. Taken together, Tables 13a and 13b suggest that, between 2004 and 2007, it is likely that NAADS had an impact on crop production, revenues and HH wealth across all agro-ecological zones in Uganda.

Table 13a: Regional variation in the impact of NAADS on household productivity NAADS Impact Across Agro-Ecological Zones (%)

40

Agro-Ecological Zone Training Crop Practice Crop Equip.

Crop Revenue per capita Wealth per capita

Eastern Highlands 52.2 2.6 NS 47.2 17.2 Lake Albert Crescent 32.1 NS 5.7 15.5 32.9 Lake Victoria Crescent 40.3 15.3 11.1 45.6 NS Northern 11. 6 25.6 NS 63.4 69.6 Southern Dry-lands 38. 4 17. 7 26.7 39.2 35.3 Southwest Highlands 56 15.9 7.1 46.3 NS West Nile 10.2 NS NS NS 36.9 Note: Coefficients that are not statistically significant at 5% are labeled “NS”. The level of significance is calculated by estimating the joint significance of the NAADS dummy with the relevant interaction term. Table 13b provides details on the how these coefficients were estimated.

Table 13b: Heterogeneous impact by agro-ecological zone

Received Training

Received Information on Crop Practice

Received Information on Crop Equipment

(%) Change in Crop Revenue pc

0.403*** 0.154*** 0.111*** 45.678*** NAADS Member in 2007 [0.009] [0.012] [0.010] [3.634]

0.119*** -0.127*** -0.098*** 1.505 NAADS Member X Eastern Highlands [0.008] [0.016] [0.025] [2.955]

-0.082*** -0.149*** -0.053** -30.190*** NAADS Member X Lake Albert Crescent [0.020] [0.014] [0.015] [4.220]

-0.287*** 0.103*** -0.104** 17.715 NAADS Member X Northern [0.029] [0.021] [0.029] [12.792]

-0.018 0.024 0.156*** -6.489 NAADS Member X Southern Dry-lands [0.038] [0.029] [0.035] [6.058]

0.157*** 0.006 -0.041 0.643 NAADS Member X Southwest Highlands [0.022] [0.020] [0.021] [6.324]

-0.301*** -0.195*** -0.144*** -35.112*** NAADS Member X West Nile [0.029] [0.023] [0.022] [8.907]

0.115** -0.017 -0.016 14.095 Received External support [0.039] [0.035] [0.024] [10.474]

0.02 0.033 0.022 10.292 Engaged in Livestock Production [0.032] [0.025] [0.034] [9.299]

0.022 0.016 0.032 -77.408*** Log(Total Crop Revenue per capita (2007 USh) :

2004) [0.020] [0.021] [0.024] [5.285] -0.02 -0.02 -0.022 3.023 Log(Land Area under

Cultivation per capita (Acres) :2004) [0.014] [0.016] [0.023] [5.520]

0.047 0.032 0.034 16.822* Log(per capita Household Wealth (2007 USh) : 2004) [0.031] [0.017] [0.020] [8.268] Constant -0.619 -0.061 -0.093 703.544***

41

[0.401] [0.323] [0.458] [149.642] Observations 473 473 473 467 R-squared 0.269 0.199 0.183 0.451 Note: The regressions also included controls for agro-ecological zones and base-year HH socioeconomic characteristics. * significant at 10%; ** significant at 5%; *** significant at 1% are based on robust standard errors clustered at the zonal level. F. Efficiency Analysis I. Assumptionsi) The discount rate is 12 %. ii) Economic and financial prices are the same in Uganda, an economy not characterized by serious distortions in agricultural prices. iii) The team used the HH panel survey to estimate separate crop revenue impacts (that can be attributed to the program) for those NAAD members who received training and those who received both training and support. For the baseline scenario, the analysis assumes that all members had access to training, while only 50 percent had access to NAADS support until 2007 and just 10 percent had access to support after 2007. While there is no hard evidence to support these guesses, ICR team field visits as well as independent studies funded by DFID and IFPRI point towards some elite capture after 2007.The ICR team believes that this is a conservative estimate. iv) The analysis assumes constant average real prices. v) The analysis includes expenditures from all sources (including the GoU and other donors) when estimating total program costs. The analysis also includes expenditures incurred by NARO and the World Food Programme. II. EfficiencyFor the baseline scenario, the ICR estimates a Net Present Value (NPV) of 80.8 Million (2000, US Dollars) and a Benefit/Cost Ratio of 2.17. It is worth noting that these estimates do not reflect the impact that NAADS may have had on livestock productivity and incomes. Since the available panel data cannot be used to estimate these impacts, the team decided to exclude possible livestock income impacts. On the other hand, the baseline scenario ignores the possibility that some of the gains in crop revenue merely reflect increases in input intensity.17 The team tested this outcome against alternative scenarios that were possible. III. Sensitivity AnalysisAs discussed above, the program was subject to a degree of elite capture. If the program was not disrupted in 2007, the (simple) model suggests that the NPV would have been 129.6 percent higher than the baseline scenario and the benefit/cost ratio would have been 2.9. On the other hand if 50 percent of the crop revenue gains were due to higher input intensities, the NPV would have been 33.6 percent lower than the baseline scenario, the NPV would have been lower (but still positive) and the benefit-cost ratio would have been lower (1.77). Taken together, these calculations suggest that the project was economically viable and the costs incurred were justified.

17 See appendix 1 of this annex for a detailed description of the strengths and weaknesses of the data available to the ICR team.

42

ANNEX 3 APPENDIX 1. AN ASSESSMENT OF THE HOUSEHOLD SURVEY DATA

18

Question that the Team Would Like to Answer

Data Constraint(s) Question that the Team is Able to Answer

What were the (household) characteristics of NAADS beneficiaries?

The team did not have access to a nationally representative household survey. The organization that collected the data has informed the ICR team that before they undertook the survey, they were given a list of districts from which they were supposed to collect the sample of member and non-member households.

What were the (household) characteristics of NAADS beneficiaries in the non-representative sample surveyed?

How did the program change after the distribution of inputs became less equitable (i.e. post 2007)?

No household survey data is available after 2007. The last official (nationally representative) household survey is from 2005.

Reports from field visits provide limited (i.e. anecdotal), but useful evidence.

How is the nature of group support (both the type of support given to the group and the use of that support) related to final outcomes?

This data is available in the FG module, but this module is not linked with the rest of the survey, so we cannot use this to infer outcomes.

We cannot answer this question.

Did NAADS’ interventions cause an increase in Household Wealth?

The team has access to panel data with detailed information on household wealth in 2004 and 2007. We do not have community price indices, so we cannot control for changes in local inflation that were probably influenced by the program interventions and may have consequently biased the “true” real wealth impact outcomes.

The team had the requisite data to answer this question. Importantly, data existed for several household characteristics, group membership and farm-level confounding factors that would have been responsible for time-variant changes that may have biased the results from a simple double difference calculation.

Did NAADS’ interventions cause an increase in crop and livestock incomes?

i) No data on producer prices in the base year ii) No data on input quantities or prices in the base year

Did NAADS-related interventions cause an increase in crop revenue? ICR Approach I: The team used the price in the nearest market. ICR Approach II: Used cross-sectional variation in crop and livestock incomes – this is less

18 This is an assessment of the Household Survey data that was available to the ICR team.

43

accurate since time-invariant unobervables (e.g. soil quality) are not eliminated. IFPRI Solution: i) Used national prices from 2007 adjusted downwards by the national CPI. ii) Assumed (implicitly) that both

input costs and input use did not change.

How strong were the links between research and extension and did this change over time?

Insufficient Data. If the team had data on the enterprises prioritized by both a NAADS district and the relevant Zonal Agricultural Research Center (in a given year) it would have been able to make reasonable inferences on the strength of these linkages. Adoption rates estimated from a nationally representative survey (discussed in the NARO ICR), suggest that the linkages were poor in 2004-2005.

We cannot answer this question.

Did NAADS’ interventions cause an increase in input intensity?

i) No data on input quantities or prices in the base year

ICR Approach: Examine the extent to which NAADS interventions explain cross-section variation in input intensities.

Did NAADS’ interventions cause an increase in crop yields?

i) Insufficient (panel) data to do an analysis, disaggregated by commodity that adjusts for agro-ecological conditions as well as household and plot level characteristics. ii) No data on input quantities or input prices in the base year.

None. It is very difficult to convincingly separate out spillovers from location factors in a relatively small sample.

Did NAADS’ interventions cause an increase in the total value of marketable surplus?

i) No data on producer prices in the base year

ICR Approach: Examine the extent to which NAADS interventions explain cross-section variation in marketable surplus. IFPRI Approach: Examine changes, by commodity, in the proportion of output marketed. The ICR team believes that these results are hard to interpret in a normative manner (e.g. if the fraction Maize marketed decreased and the fraction Vanilla marketed increased, is this a positive outcome for the household?)

44

ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending David Nelson Task Team Leader – Senior Economist AFTR1

John Jaramoga Oloya Rural Development Specialist AFMUG Christine Cornelius Senior Operations Officer AFMKE Lawrence Salmen Senior Sociologist W. Graeme Donovan Lead Economist AFTS2 Edgardo Quisumbing Senior Agriculturalist AFTR1 Almaz Teklesenbet Team Assistant AFTR1 Marie-Helene Collion Senior Agricultural Service Specialist Pierre Rondot Produces’ Organizations Specialist

Anthony Hegarty Senior Financial Management Specialist LOAG2

Modupe Adebowale Senior Financial Management Specialist LOAG2

Joseph Kizito Financial Management Specialist LOAG2 Rogati Kayani Senior Procurement Specialist AFTPC Aberra Zerabruk Senior Counsel LEGAF Yisgedullish Made Poverty Targetting Pamela Khumbah Gender Specialist (Consultant) Mohammed Taqi Sharif Consultant AFTR1 Jacob Kampen Consultant AFTAR

AFTR1 Supervision/ICR Melissa Brown Junior Professional Associate AFTS2-HISHoward Bariira Centenary Procurement Specialist AFTPC

Guy Roger Evers FAO Staff RDVCG-HIS

Madhur Gautam Lead Economist AFTAR Malathi Jayawickrama Sr. Operations Officer, ICR TTL AFTAR Jacob Kampen Consultant AFTAR

Varun Kshirsagar Consultant, ICR Co-Author AFTAR Matthew A. McMahon Consultant LCSAR David J. Nielson Lead Agriculture Services Spec EASER Wilson Onyang Odwongo Senior Rural Development Speci AFTAR Michael Okuny E T Consultant AFTFM Richard Olowo Senior Procurement Specialist AFTPC

45

John Jaramogi Oloya Senior Rural Development Speci AFTAR Mohammed Taqi Sharif Consultant AFTAR Patrick Piker Umah Tete Sr Financial Management Specia AFTFM Johannes Woelcke Senior Economist AFTAR Sergiy Zorya Economist AFTAR

Almaz Teklesenbet Program Assistant AFTAR Karen M. Brooks Sector Manager AFTAR

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks USD Thousands (including

travel and consultant costs) Lending

FY97 28.10 FY98 61.79 FY99 56.93 FY00 35 195.43 FY01 34 121.49 FY02 17 73.03 FY03 0.00 FY04 0.02 FY05 0.00 FY06 8.80 FY07 0.00 FY08 0.00

Total: 86 545.59 Supervision/ICR

FY97 0.00 FY98 0.00 FY99 0.12 FY00 0.00 FY01 6 6.74 FY02 12 53.63 FY03 30 116.87 FY04 22 97.80 FY05 24 169.45 FY06 21 111.80 FY07 37 163.18 FY08 39 181.65 FY09 0.00

Total: 191 901.24

46

ANNEX 5. BENEFICIARY SURVEY RESULTS Methodology and Executive Summary of the Impact Evaluation and Returns to investment of the National Agricultural Advisory Services (NAADS) Program of Uganda, International Food Policy Research Institute, Washington, DC, USA. October 27, 2008. METHODOLOGY Surveys and Data Collection

This study uses data from two rounds of FG and household surveys conducted in 2004 and 2007. The 2004 data served as the baseline on which a stratified sampling is based according to the NAADS rollout phases: 1) sub-counties where the NAADS program was first established in 2001/02, hereafter referred to as “early NAADS sub-counties”, 2) sub-counties where the NAADS program began in 2002/03, hereafter referred to as “intermediate NAADS sub-counties”, 3) sub-counties where the NAADS program began in between 2005 and 2007, hereafter referred to as “late NAADS sub-counties”, and 4) sub-counties where there has not been NAADS program, hereafter referred to as “non-NAADS sub-counties”. The late NAADS stratum was created during the second round of the survey from the non-NAADS stratum of the first round survey. 19 This was necessary since the NAADS program had begun operating in some of the areas where they were not operating in 2004, i.e. at the time of the first round survey. Therefore, new FGs and households were also surveyed to increase the sample in the non-NAADS stratum. Table 1 shows the number of households and FGs sampled from each stratum in each survey year. All the six early NAADS districts and the 24 corresponding sub-counties were selected for survey. In the case of the intermediate NAADS group, four of the nine districts and 18 of the 72 sub-counties were sampled. The districts and sub-counties from the intermediate and late NAADS group as well as from the non-NAADS group were purposively sampled such that they had similar agricultural potential20 and market access21 as the corresponding early NAADS districts and sub-counties. For each of the early NAADS districts, a matching district, i.e., one with similar market access and agricultural potential setting, from the other strata was selected. This was done to minimize the across group variation in agricultural potential and market access, which are likely to greatly influence agricultural production, income and other outcomes of interest that will be analyzed in this study. From each selected sub-county, two parishes were randomly selected, and then from each selected parish one village (LC1) was randomly selected. From each of the selected villages, 6-13 households were randomly selected. For the FG survey, one group was randomly selected from each of the selected villages. Together, 902 households and 116 FGs were surveyed in 2004, and 1200 households and 150 FGs were surveyed in 2007, with a panel of 719 households and 110 FGs (Table 1). The data collected from the household survey include the demographic and socio-economic characteristics of the household. To understand the impact of the NAADS program on adoption and productivity of new technologies and enterprises, data on awareness and use of improved production practices and new enterprises adopted after 2000 in the 2004 survey and after 2004 in the 2007 survey

19 See Benin et al. (2007) for details on the sampling in the first round survey. 20 A gricultural p otential is a n abstraction o f many factors—including r ainfall le vel and d istribution, a ltitude, soil type and depth, topography, presence of pests and diseases, presence of irrigation, and others—that influence the absolute (as opposed to comparative) advantage of producing agricultural commodities in a particular place. 21 Market access is measured as the potential market integration (estimated as travel time to the nearest five markets, weighted by their population (Wood, et al. 1999)) and distance to an all-weather road.

47

were collected at household level. The information is differentiated according those introduced by NAADS and non-NAADS service providers. The household survey also collected data on participation of households in the market and their access to advisory services and other institutions. Other data collected include agricultural production data, income strategies, and perception of change in wealth and food and nutrition security. Production cost data was also collected, but only in the 2007 survey. To get a sense of what the situation was before the program started in 2001, the 2004 survey was also used to collect information on key factors and outcomes in 2000 (particularly on perception of change between 2000 and 2004) in assets and agricultural production, as well as in wealth and food and nutrition security. Table 1: Number of districts, sub-counties, villages, farmer groups and households sampled in each NAADS rollout phases

Survey year/sampling units Early NAADS (2001/02)

Intermediate NAADS (2002/03)

Late NAADS (2005-07)

Non-NAADS

Total

2004 Districts 6 4 0 4 14Sub-counties 24 18 0 16 58Villages 4 8 36 0 32 116Farmer groups 48 36 0 32 116Households 402 300 0 200 902

2007 Districts 6 4 2 6 18Sub-counties 24 18 4 27 73Villages 4 8 36 6 60 150Farmer groups 48 36 6 60 150Households 402 300 81 417 1200

Panel (2004 and 2007) Districts 6 4 2 2 14Sub-counties 24 18 4 12 58Villages 4 8 36 6 26 116Farmer groups 48 36 6 20 110Households 323 202 81 113 719

The FG survey collected data related mainly to empowerment of farmers to organize, to demand and manage advisory services and how advisory services of different types have influenced livelihoods of female and male farmers. Other data collected in the FG survey include access of group members to advisory services, their participation in development of institutions and their perception on the quality and availability of advisory services. In the second round of data collection, secondary data was also collected at the sub-county level on infrastructure and public service provision, NAADS’ processes on farmer institutional development, service provider contracts, technology development sites and demonstrations, and NAADS’ expenditure. In the analysis, all monetary values are converted into 2000 constant prices using the consumer price index as the deflator. This helps to exclude the influence of inflation and other temporal monetary and fiscal trends. All statistics are also corrected for stratification, clustering, and weighting of sample. The clusters were the villages and sampling weights were calculated using parish level human population data. Sample weights are inverse of the probability of a household being selected in the sample, which was calculated as (the number of selected parishes divided by the total number of parishes in the sub-county) multiplied by (the number of selected households divided by the total number of households in the parish). Since population data were only available at the parish (not village) level, random selection of

48

households at the parish level was assumed in the calculation. Due to the nature of the sampling, the results are representative only of the selected sub-counties, since these were purposively selected. EXECUTIVE SUMMARY Uganda has for a long period of time experienced strong economic growth. In the 1990s, gross domestic product grew steadily by more than 6% per annum from a low rate of 3 percent in the 1980s, and the proportion of the population living under the poverty line declined from 56.4 percent in 1992 to 31.1 percent in 2006. This remarkable turnaround from the depression associated with the political turmoil and economic mismanagement of the 1970’s until the mid-1980s has been achieved through sound policies linked to investments and economic liberalization undertaken by the Government of Uganda (GOU) with support from the donor community. Despite the substantial progress made, several challenges remain in sustaining the momentum by way of increasing productivity, ensuring sustainable use of natural resources, and reducing poverty, hunger and human disease. Recognizing the importance of a multi-sectoral approach to accelerating growth and reducing mass poverty, the Government of Uganda has since 2000 been implementing the Plan for Modernization of Agriculture (PMA) as a key policy initiative aimed at reducing mass poverty to a level below 28 percent by 2014. The PMA, which is situated within the country’s vision of Prosperity for All and is supported by the broader Rural Development Strategy,has an ambitious agenda of policy and institutional reform across seven pillars, a key one of which is improving delivery of agricultural extension through the National Agricultural Advisory Services (NAADS) program. Since its inception in 2001, NAADS has devised an innovative extension service delivery approach, that targets the development and use of farmer institutions and in the process empowers them to procure advisory services, manage linkage with marketing partners and conduct demand-driven monitoring and evaluation of the advisory services and their impacts. NAADS was initiated in 2001 in six districts (Arua, Kabale, Kibaale, Mukono, Soroti and Tororo), within which the NAADS program began working in 24 sub-counties. By end of 2006/07 financial year, the program had been extended to 545 sub-counties (about 83.1 percent of the total sub-counties), and is expected to cover the entire country by end of the financial year 2007/08, ending the first phase (Phase I) of the program. By the end of the 2006/07 financial year also, UGX 110 billion (in 2000 value terms) had been spent on the program. Building on the mid-term evaluation of NAADS (Benin et al. 2007 and others), the overall objective of this study was to undertake a rigorous end-of-Phase I evaluation of the NAADS program to analyze and document the outcomes and the direct and indirect impacts of the program, and assess the return on investment. This was done using data from two rounds of FG and household surveys conducted in 2004 and 2007; where 900 households and 120 FGs were surveyed in 2004, and 1200 households and 150 FGs were surveyed in 2007, in addition to obtaining secondary data on NAADS expenditures and provision of public services in all the surveyed sub-counties. Most of the households surveyed in 2004 were also surveyed in 2007, giving a panel of 719 households. The surveys were based on a stratified sampling according to the NAADS rollout phases: 1) sub-counties where the NAADS program was first established in 2001/02 referred to as early NAADS sub-counties, 2) sub-counties where the NAADS program began in 2002/03 referred to as intermediate NAADS sub-counties, 3) sub-counties where the NAADS program began in between 2005 and 2007 referred to as late NAADS sub-counties, and 4) sub-counties where there has not been NAADS program, hereafter referred to as non-NAADS sub-counties.

Various quantitative and analytical approaches, such as trends, descriptive, double difference, propensity-score matching, and two-stage weighted regression methods, were used to: 1. Assess the incidence of rural public services among farming households 2. Estimate the impacts of the program on various indicators associated with the objectives of the

program, including: a. Empowerment of farmers to organize themselves and demand and manage advisory services;

49

b. Farmers’ perception of the availability, delivery and quality of advisory services; c. Farmers’ awareness, incidence of adoption and intensity of adoption of improved technologies and

practices; d. A gricultural productivity; e. M arket participation; and f. Income, assets, food and nutrition security, and welfare

3. Analyze and quantify the contribution of other factors that influence participation in the program and realization of the outcomes, including household demographics and access to other rural public services;

4. Assess the return on investment made so far in the program. Incidence of rural public services among farming households In general, availability of various rural public services (extension services, input supply shops, markets, roads, schools, health centers, etc.) has improved over time, especially between 2001 and 2007, in terms of presence of physical infrastructure as well as farmers’ perceptions of their access to the services. However, availability and access were generally better in the NAADS sub-counties than in the non-NAADS sub-counties. To the extent that these services enhance the impact of the NAADS program, the impact of NAADS is expected to decline as it rolls out to the entire country. Basically, the impacts may tend to be overestimated if differences in availability of services between NAADS-participating and non-participating sub-counties are not adequately controlled for. The results suggest the need of improving rural services in remote areas in order to enhance the returns to NAADS and other investments. For example, one of the weaknesses of the PMA was its failure to strengthen other key rural services such rural financial and marketing services. This has weakened the impact of the NAADS program, especially in the Northern and Eastern regions of Uganda where such services are poor. Notwithstanding their weaknesses, the current efforts through the rural development strategy and prosperity for all to strengthen provision of rural financial and marketing services through cooperatives (MAAIF 2005; GOU 2008) will definitely enhance the impacts of the NAADS program and other rural investments. Farmer institutional development and empowerment Overall, we found that the most common areas that farmers had received training in included development of group constitution and bye-laws, leadership skills, growth and development, planning, record keeping, and savings mobilization; with more than 70 percent of the groups in NAADS sub-counties reporting positively. Also, more groups were trained in the early NAADS sub-counties than in the other sub-counties where the program started later or had not been implemented at all. These suggest that the NAADS program is gradually helping to strengthen the human resource skills and institutional capacity of farmers that will potentially improve natural resource management, agricultural productivity, marketing, etc. This is substantiated by the finding that majority (about 90 percent) of the farmers found the various areas of training to be very useful or useful; although more groups in the early NAADS sub-counties found the training very useful or useful than those in intermediate NAADS sub-counties. NAADS service providers, compared to others, were rated very high on their methods used in the training and on their performance. In terms of farmers’ participation in FG activities, we found increasing performance participation to be strongly correlated with period of entry into the NAADS program, with newly NAADS-participating groups reporting better performance. Although false farmer expectations of receiving cash funds and inputs rather than knowledge and advice could have played a significant role in weak group activities and poor farmer empowerment in certain areas, the above finding also suggests a shift in interest among the older NAADS-participating groups from basic agronomic and production practices to higher value-chain activities, such as marketing and value addition. Thus, operationalization of NAADS’ strategy regarding

50

gradual introduction of activities along the entire value chain is critical. The higher value chain activities such as marketing should be given more attention, especially among the older groups, as it has consistently featured low in the implementation of the program (Benin et al. 2007). In general, farmers felt an increase in their level of empowerment, especially toward technical public officers in general. This is very encouraging since technical public officers are well placed to address farmers’ agricultural production problems or concerns. However, technical public officers also received the worst performance rating in terms of their response to farmers’ requests. While this finding raises concern about the ability to meet farmers’ demands, it suggests a weakening of advisory services as the NAADS program rolls out to more districts and sub-counties. A complementary increase in resources to maintain or increase advisory services to those already participating in the program, as well as to new farmers to be taken on board, is therefore critical. Supply of advisory services NAADS is an enterprise-based program requiring that FGs prioritize their enterprises, which can be a crop, livestock, fishery or beekeeping activity, or a mix of activities, on which advisory services would be focused. As expected, not all enterprises were promoted by NAADS or demanded by farmers everywhere. The major crops promoted by NAADS (and/or requested by farmers) in terms of the number of sub-counties promoted in, or the number of technology development sites (TDSs) established, or the number of FGs directly benefiting from the TDSs included banana, groundnuts and rice, followed by vanilla and maize. Regarding livestock and other enterprises, the major ones were goats, poultry and beekeeping, followed by cattle and piggery. Comparing the major crops and livestock enterprises that were promoted by NAADS (and/or requested by farmer), livestock enterprises seemed to be more widely favored. Decision to participate in the NAADS program Although the NAADS program is a public investment intervention, farmers have to decide to participate in the program through membership of a NAADS-participating FG. Then, together with other groups they demand specific technologies and advisory services associated with their preferred enterprises. We find that the longer the NAADS program has been in a sub-county, the more likely farmers in that sub-county are to directly participate in the program. This means that continued engagement with farmers is critical. One-shot or stop-and-go approaches are not likely to yield any sustained pay-offs. Membership in other organizations, which is a measure of social capital, was also strongly associated with greater likelihood of participation in the NAADS program. This is consistent with the NAADS approach in terms of targeting existing groups to maximize the payoffs from efforts to build farmers’ capacity to demand advisory services. Gender, age, education, and income source were not significant factors in determining group membership, which implies that the NAADS program is targeting all socio-economic groups. Demand for advisory services and adoption of new enterprises and technologies We find that NAADS has had significant success in increasing the capacity of farmers to demand improved crop varieties, crop management practices, soil conservation, livestock breeds, post-harvest practices and marketing information. This demonstrates that the NAADS demand-driven approach is working. Participation in the NAADS program, however, seem to have lowered the probability to demand soil fertility and agroforestry practices, suggesting low capacity of farmers to demand these technologies and/or weakness of NAADS to provide them. NAADS spent relatively low resources in conducting demonstrations on soil fertility management practices, compared to, for example, acquiring improved planting material. In order to ensure sustainable productivity, NAADS needs to increase the capacity of farmers to demand soil fertility management practices. For example, it may be necessary in the initial

51

stages of demand-driven approaches to supply soil fertility and agroforestry practices in order to build farmers’ capacity to demand them. Crop and livestock productivity and commercialization of agriculture Consistent with the positive impact on capacity strengthening, demand for advisory services, and adoption of improved technologies, the NAADS program has had significant impact on crop productivity, with the value of gross crop output per acre having increased by up to 29 percent for those participating directly in the NAADS program more than for their non-participant counterparts. The impact of the NAADS program on livestock productivity is surprising, as the results show that the program has contributed to a decline (about 27-45 percent) in the value of gross livestock output per unit of animal among NAADS participants compared to their non-participant counterparts. We find that NAADS has had a small impact on proportion of agricultural output sold by farmers. The impact of the program is estimated to be up to 6 percent increase in crop sales for the NAADS participants over their non-participant counterparts, and up to 4 percent more sale of total agricultural production. The impact on sale of livestock products is negligible. The NAADS program needs to shift activities away from basic agronomic and production practices towards higher value-chain activities. Income, consumption expenditure, food and nutrition security, and welfare NAADS participants were associated with 42-53 percent average increase in their per capita agricultural income compared to their non-participant counterparts. The impact of the program on total agricultural income was more favorable than the income obtained from crops and livestock. This is due to additional substantial income for NAADS participants from high-value agricultural activities such as beekeeping and aquaculture, and demonstrates the effectiveness of the program in diversifying income and promoting more high-value activities. The results also show that significantly larger proportions of NAADS participants than non-participants perceived that their situation had improved, while larger proportions of the non-participants than participants perceived that their situation had not changed or it had worsened. For example, 41–58 percent of all NAADS participants perceived that their average wealth, access to adequate food, nutritional quality of food, ability to meet basic needs or overall wellbeing had improved between 2000 and 2004 and between 2004 and 2007, compared to 27–44 percent of their non-participant counterparts. These results are consistent with the positive impacts of the NAADS program on adoption of improved technologies and agricultural productivity and income, and they suggest the NAADS program has helped farmers to improve their households’ standard of living. Factors affecting realization of impacts Several factors significantly influenced farmers’ demand for advisory services, and changes between 2004 and 2007 in adoption of improved technologies and practices, crop and livestock productivity, sale of output, and income. The main factors include gender and age of the household head, education, income sources, land and non-land productive assets, and access to all-weather roads. There are two main implications of this. First, the impacts of the program tend to be overestimated when these factors are not controlled for. Second, these are the factors that should be considered for targeting to maximize payoffs from the program. Distribution of impacts Regional distribution of impacts show that the largest impact of the NAADS program has so far occurred in the Central and Western Regions, where the per capita income of NAADS participants rose by 65-165 percent between 2004 and 2007 compared to their non-participant counterparts, suggesting that the impact of the NAADS program has been more pronounced in the well-off regions. The NAADS program has

52

also had more impact women, when looking at the total impact as opposed to the direct impacts, when men have benefitted more. With regards to the total impacts, women participating in the program were associated with 16 percentage points more increase in their average per capita incomes than men participating in the program. This suggests that NAADS is only partially achieving its objective of targeting women, a group that has attracted little resources and experienced limited access to agricultural extension in SSA. The distribution of impacts by asset-endowment tercile shows that the NAADS program has been more successful at raising income among the poorest households, where the per capita income of those participating in the program more than doubled between 2004 and 2007 compared to their non-participating counterparts. This was followed by the richest group, where per capita income of participants grew by nearly 36–40 percent within the same time period. These suggest that NAADS is achieving its objective of targeting the economically-active poor. However, the fact that the richest participants benefited more than those in the middle asset class underlies the importance of household’s capacity to acquire the improved technologies and related advisory services being promoted by the NAADS program. Returns to investments Based on the gross agricultural income per capita, the total benefits of the NAADS program in the 37 NAADS-participating sub-counties that were surveyed was estimated at UGX 49.7-54.8 billion. The total cost was estimated at UGX 14.4 billion, indicating a benefit-cost ratio of about 5. This means that every UGX 1 spent on the NAADS program so far has yielded UGX 5 in terms of its contribution to agricultural income. On accounting for the cost of agricultural inputs and operations, which were estimated at about 35 percent of the gross income, the discounted total benefits dropped to UGX 32.1-35.4 billion. Similarly, accounting for the interest payments on the loans acquired to finance the program led an additional cost of UGX 2.4 billion (or 16.8 billion total), which together with the reduced benefits reduced the benefit-cost ratio to about 2.5. Issues for future research Although the study tried to capture many important issues regarding the benefits and costs of the program to assess the economic returns, a few issues remain that future research can improve upon. These relate to general equilibrium effects, complementarity (or substitutability) of public investments, and other benefits. For example, the scaling out of the NAADS program to all parts of the country is likely to affect relative prices and may require additional taxes to pay back the loan obtained to finance the program. Both effects mitigate the impact of the program, potentially leading to an overestimation of benefits based on partial equilibrium analysis. Similarly, strengthening the capacity of farmers and service providers also will affect the skill composition of the labor force and service providers, which in turn will affect the wage structure and cost of advisory services. Thus, including economic modeling techniques in future analysis will prove useful. Another issue is the complementarity (or trade-offs) between the NAADS program and other different types of public investments. For example, we would anticipate complementarity between investment in the NAADS program and investments in agricultural R&D and education. This is because agricultural technologies tend to be highly complex, knowledge intensive, and location specific, and so technologies that are profitable under local conditions and knowledge and skills are required for the success of the program. Typically, interaction terms among the relevant investments can be included in the regression model to capture these effects; and to the extent that complementarity (substitutability) exists, the benefits would be overestimated (underestimated). Due to the small sample size of the survey data used in the study, interaction terms could not be included in the regressions as doing so can introduce severe multicollinearity, which would then cause the regression parameters to be estimated imprecisely.

53

The NAADS program can be expected to generate a range of other benefits that have not been considered here nor assessed quantitatively. These include the improved human resource skills associated with training and strengthening of local institutional capacity. For example, training on technical and managerial areas that are provided to private service providers, extension staff, subject matter specialists, and research staff will develop improved skills, which would contribute to productivity improvements not only on the farm but off it. Training of village groups, community-based facilitators, farmer contact groups, and farmer fora at the local level will strengthen local institutional capacities and empower farmers to effectively demand advisory services. The improvements in both the human resource skills and institutional capacity will generate benefits when also used in non-agricultural economic and non-economic activities.

54

ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS Not Applicable

55

ANNEX 7. SUMMARY OF BORROWER’S ICR AND/OR COMMENTS ON DRAFT ICR

EXECUTIVE SUMMARY (Government of Uganda ICR, June 9, 2010)

The NAADS Program was developed as a twenty five year program to be implemented by several projects. This project, representing the first phase of the program, was initially costed at USD 107.02 million to be funded by the Government of Uganda with the support of several development partners including among others; the International Development Association, the International Fund for Agriculture Development, bilateral agencies, Local Governments, and Farmers’ Organizations. The objective of the project, which comprised of five components, was to assist poor male and female farmers to become aware of and to be able to adopt improved technology and management practices in their farming enterprises, to enhance their productive efficiency, their economic welfare, and the sustainability of their farming operations. Project implementation lasted nearly nine years instead of the originally planned seven years as a result, in part, of a suspension of the Program by government. This suspension was due to concerns about the modalities of service delivery. As a result, at mid-term, it was decided to restructure the project into six instead of the original five components, focusing the project on farmer empowerment and advisory services delivery, implementation of some pillars of the Program for the Modernization of Agriculture that had remained largely dormant, and availing more resources for the provision of inputs. The design of the project anchored it within the overall national and sector development policies and strategies as the NAADS goals and objectives, to which the project subscribed, are as relevant today with the new National Development Plan as it was with the Poverty Eradication Action Plan (PEAP) which was the national development strategy when it was designed. In the agriculture sector, the PMA and the evolving National Agriculture Policy ensure strong linkages between the project and the NAADS program and the national development strategies. The project objective responded to the Program’s main goal of “decentralized, farmer owned and private sector serviced extension system contributing to the realization of agriculture sector objectives” thereby contributing to the Government’s efforts to achieve rapid economic growth and poverty reduction. The project was implemented by relevant institutions in the sector and the creation of a special purpose vehicle, the NAADS Secretariat, to manage the program ensured sufficient focus and coordination of the project. An evaluation of project performance in 2005 concluded that it had demonstrated positive achievements regarding farmer empowerment and adoption of technologies and on the overall outputs of farmers. Over 1,100,000 farmers were reached and they were found to be better able to articulate their needs. As a result, the technologies for demonstration on the fields of FG members to support the selection of enterprises are valued immensely and there is widespread demand for and adoption (on average 74% for NAADS farmers) of technologies such as improved groundnuts, rice, citrus and poultry. The evaluation also concluded that project benefits, which were largely transmitted through Technology Development Sites and demonstrations, were not restricted to only farmers who participated in the Program. Therefore, the Program has generated substantial indirect or spillover effects regarding adoption of technologies and practices, particularly use of crop improved varieties, recommended spacing and planting practices, and pesticides.

56

The establishment of farmer institutions at national, district, sub-county and parish levels in the country facilitated the empowerment of farmers and led to increases in demand for advisory services. Agri-business activities which were implemented at the sub-county and district levels enhanced marketing linkages (including through Private Public partnerships) for crops such as barley and sorghum. The project established management systems and, while some challenges remained, a strong framework has been created for the implementation of the subsequent phases of the program and for the overall management of agriculture advisory services in the country. As far as financial management of the project is concerned, a review undertaken in 2009 revealed that, although initially, there was a slow rate of implementation mainly due weaknesses in internal controls, the flow of funds eventually gained momentum. . The total Government of Uganda contributed at total of UGX 105 billion (approximately USD 53 million at an average exchange rate of 1USD/2000 UGX) and the development partners approximately USD 107 million. Thus, the estimated total funding to the program amounted to approximately USD 160 million as compared to the original estimated project cost of approximately USD 108 million in the PAD. Impact studies indicated that, overall, the project has delivered vital advisory services to the farmers and has had positive impact and that crop productivity had improved (on average by 27%) for NAADS farmers. They also indicate that there was increased profitability of household farm activities and of farmer incomes (on average 64%) for NAADS farmers. However, in spite of the important achievements of the project, significant lessons were learned and areas for strengthening identified. Such lessons include the importance of clarity in the lines of accountability and reporting needs, clarity and consistency in application of advisory messages and modes of their delivery, prior definition and establishment of monitoring and evaluation systems to ensure the capture of all implementation outputs and lessons, etc. As the demand for agricultural advisory services in the country is high and will remain so in the foreseeable future, there is therefore need to review the lessons learned and to utilize them to enhance the implementation of the subsequent phases of the program. KEY FINDINGS AND RECOMMENDATIONS The reviews and studies undertaken during project implementation highlight a number of findings that need to be taken into consideration in designing further interventions for Agriculture Advisory Services in the country. The following are some important findings and recommendations: Broad Implementation Structure: The NAADS implementation structure was broad and unwieldy comprising of institutions at many levels of government (national, district and sub-county) and the private (FF, SPs, farmers) sector. While roles and responsibilities in the Program were defined, because of its developmental nature, the Program attracted diverse comments, suggestions and directives from stakeholders and leaders. The lines of accountability became fuzzy and governance issues abounded, especially at the local government level, giving the program a poor image. It is recommended that the implementation structure be reviewed with a view to enhancing it in advance of the implementation of the second phase of the Program. Seeming Changes in Delivery Models: With the introduction of the “six farmers per parish” implementation strategy for the “Prosperity for all Program”, farmers got the impression that Government had abandoned the original approach of development through FGs. This undermined the stability of FGs as individual ordinary farmers saw their chances of benefitting from the Program as more remote. Government should take steps to ensure that conflicting messages are kept to a minimum. It is therefore important that the Board develops and implements a robust communication strategy which

57

would include, among others, active engagement with policy makers at all levels to ensure that a balanced and accurate image of the Program is delivered. Implementation Capacity: On declaration of project effectiveness in 2001, the Ministry of Agriculture, Animal Industry and Fisheries had just recently been restructured, through the Post Constitutional Restructuring of 2000, to implement its new roles of policy evolution, standards and monitoring and evaluation and coordination. As it was grappling with the implementation of the directives of the restructuring process, the ministry was stretched in terms of resources (human and financial) to undertake tasks assigned to it under the Program. There was weak capacity (quantity and capacity) and lack of motivation of staff at the local government and farmer levels and continuous demands for and the creation of districts did not help the situation. Frequently, some public sector frontline staff of districts also presented themselves as private sector actors in others district, undermining their roles in their own districts of fulltime employment. The planned conversion (de-layering) of front line staff to employment under NAADS’ better terms did not take place during the phase thereby eroding further the motivation of staff. The farmers’ fora created at all levels did not have ample time to build adequate capacity, to understand the Program concept and to become valuable partners in project implementation. At the NAADS Secretariat level, implementation started out with a lean secretariat to reflect the coordination, planning, monitoring and evaluation roles. However, with the rapid country-wide roll-out, the need for more implementation support increased. Members of staff of the Secretariat were required to support local governments in areas such as PM&E, Procurement. New roles emerged in areas such as PPP management and district financial auditing as the requirements and the complexity of the Program increased. The speedy roll-out of the Program to the continuously increasing number of districts stretched the human resources capacity of Program implementers at all levels. The planned further increase in the number of districts will continue to present this challenge. Going forward, it will be important to undertake a thorough review of the capacity needs of the Program at all levels and to evolve and implement a strategy for ensuring that the capacity requirements are fulfilled. Periodic reviews will be required throughout out the implementation of the next phase to ensure that emerging trends are captured and dully addressed. Farmer Institutions Development: Farmer institutions are important forums for mobilizing farmers around a common objective, delivery of services as well policies that support agricultural development. They form key entry points for service delivery to individual households or communities. However, in general, farmer institutions are still characterized by low capacity to effectively perform their roles and to demand for delivery of agricultural advisory/extension services. Government efforts should focus on strengthening the capacity of these institutions to fully participate in the commodity value chain development and resource mobilization and management. Low demand for some services: There was low demand for soil fertility management practices which could have been due to low capacity of service providers to provide soil fertility management and agro-forestry practices. This could also be due to low investment that NAADS may be putting in soil fertility management practices in the technology development and demonstration sites. These results raise concern on the sustainability of NAADS strategy since it leads to soil fertility mining and could lead to lower productivity in the long-run. It is necessary to review the issues and to provide appropriate response. Future role of PPPs: PPPs are an innovative way of involving the private sector on a larger and more comprehensive scale. Together with recent lead farmer and the nucleus farmer initiatives, they provide a gateway for farmers to move beyond production into further stages of the value chain at the same time as

58

serving as a source of inputs. If NAADS is to continue with and expand its PPP activities, it will need to conduct a thorough cost benefit analysis in order to assess the financial efficacy and sustainability of thiscomponent. GoU will also need to review the legal and policy issues to enhance the working environment for PPPs. PM&E System: There is an awareness that further improvements to the system are required to move further from an activity oriented system, to one which focuses on progress towards outcomes and impact and indeed the annual reports do provide information on outcomes and impact. However, at district level, the Progress Report for the Annual Review, for example, is firmly rooted in reporting outputs and implementation issues rather than higher level results or impact on beneficiaries. The framework for collection, analysis and reporting of Program data at all levels is not clear. This needs to be reviewed and enhanced. Incidence of rural public services on farming households: In general, availability of various services has improved over time, especially between 2001 and 2007, in terms of presence of physical infrastructure as well as farmers’ perceptions of their access to the services. However, availability and access were generally better in NAADS sub-counties than in non-NAADS sub-counties. To the extent that these services enhance the impact of the NAADS Program, the impact of NAADS is expected to decline as the Program is rolled out further to the entire country. To achieve sustained Program impact, it is necessary to ensure the availability of these services in all the districts/sub-counties of roll-out. Farmer empowerment: In general, farmers felt an increase in their level of empowerment, especially towards technical public officers. This is very encouraging since technical public officers are well placed to address farmers’ agricultural production problems or concerns. However, technical public officers also received the worst performance rating in terms of their response to farmers’ requests. While this finding raises concern about the ability to meet farmers’ demands, it suggests a weakening of advisory services as the NAADS Program rolls out to more districts and sub-counties. Complementary increase in resources to maintain or increase advisory services to those already participating in the Program, as well as taking on new farmers, is therefore critical. Demand for more advisory services and technologies: NAADS has had significant success in increasing the capacity of farmers to demand improved crop varieties, crop management practices, soil conservation, livestock breeds, post-harvest practices and marketing information. Farmers with larger farm size and those closer to all-weather roads and markets are more likely to demand advisory services than those with smaller farms and those living in remote areas. Therefore, there is need to target small farmers in remote areas in efforts to increase the capacity of farmers to demand advisory services. It was also noted that households headed by females were associated with about 25 percent lower crop yield and 19 percent lower agricultural income per capita. As this result suggests that food insecurity and poverty may be more problematic in female-headed households it is necessary to enhance the gender dimension of the NAADS Program. Future plans of the Program The NAADs Program has delivered vital advisory services to the farmers in participating districts and sub-counties in a manner which is seen to be better than available modes of delivery. From the implementation of NAADS, the general conclusion is that a positive impact has been made on the ground. However, the demand for AAS by the farmers remains high. Yet, given the large demands on government, GOU may not be in a position to fully finance this endeavor for a while to come. As already indicated, the NAADS Program was designed as a 25 year Program to be implemented in five phases. The design of phase two of the program is already in advanced stages. The challenges and therefore lessons learned in the implementation of the first phase will be taken into consideration in the

59

design and implementation of the subsequent phases of the Program.

60

ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

Comments from GoU and Development Partners will be filed in the Bank’s IRIS when received.

61

ANNEX 9. LIST OF SUPPORTING DOCUMENTS World Bank DocumentsNAADS, Project Appraisal Document, Report No. 21743-UG, January 22, 2001. NAADS, Project Information Document, Report No. PID8313, November 9, 2000. NAADS Pre-appraisal document of the Extension Task Force and Joint Donor Group Mission, August 24, 2000. NAADS Pre-Appraisal Report, Draft, Volume II, Working Papers I to IV, 2000. Draft Environmental Analysis for Uganda NAADS, July 21, 2000. ARTP II, Project Appraisal Document, Report No. 19144-UG, April 9, 1999. ATAAS, Project Appraisal Document, Report No. 54504-UG, May 25, 2010. NAADS, Development Credit Agreement, Credit Number 3463-UG, May 7, 2001. NAADS, Amendment to the Development Credit Agreement, July 11, 2008. NAADS, Implementation Status Report, Number 1-17, December 27, 2001 to December 22, 2009. NAADS, Mid-Term Review, Office Memorandum (Statement of Mission Objectives), May 6, 2005. NAADS, Aide Memoire and Back-to-Office Reports of Implementation Support Missions, FY01-FY09. Fifth NAADS Annual GoU-Development Partner Review Meeting, Aide Memoire Action Points, June 1-2, 2006. NAADS Post Procurement Review Reports May 22, 2007, July 17, 2008, and June 25, 2009. Agricultural Extension Project Implementation Completion Report, July 28, 2000. ARTP II, Implementation Completion and Results Report, April 29, 2010. OPCS, Guidelines for Implementation Completion and Results Report, August 2006. Government DocumentsGovernment of Uganda, NAADS Final Implementation Completion Report, June 2010. MAAIF, NAADS Report of the Fifth NAADS GoU-Donor Mid-Year Planning Meeting, December 2005. MAAIF, NAADS Quarterly Progress Reports, 2001 – 2009.NAADS, Operationalization of the Integrated Support to Farmers’ Groups (ISFG) in Sub-Counties, Implementation Guidelines, December 2005. NAADS, Report of the Limited Audit Review (4th Quarter), November 2005. Republic of Uganda. Prosperity for All Policy, Tackling Poverty at Household Level, February, 2008 Value for Money Audit Report of NAADS: Advisory and Information Services Component, Office of the Auditor General Uganda, November 2008. NAADS Indicative Planning Figures, FY 2009/10, Memo from NAADS Executive Director to The Chief Administrative Officer dated May 5, 2009. Government of Uganda, Position Paper on Extension Service Delivery, Farmer Commercialization and Financing under NAADS, March 2009. MAAIF, Comments on Government of Uganda, Position Paper on Extension Service Delivery, Farmer Commercialization and Financing under NAADS, April 2009. Investigation of NAADS Implementation, A Consolidated Report for Iganga, Mbarara, Kanungu and Luwero, MAAIF Task Force Report, March 2010. An Assessment of the Scale, Nature and Extent of the Misuse of NAADS Funds and Actions Taken to Address the Issue, MAAIF, April 2010, MAAIF, Summary of Agreements Reached at the GoU-Development Partners’ Mid-Term Review of NAADS, May 30 to June 2, 2005. Other Documents

62

NAADS Memorandum of Understanding among GoU and Participating Partners, October 2001. NAADS ICR Mission Report on Key Findings and Issues Arising from the Field Trips undertaken between February 21, 2010 and March 9, 2010. NAADS Document Research on corruption and accountability concerns (January-February 2010), May 26, 2010. Benin, S., Nkonya, E., Okecho, G., Pender, J., Nahdy, S., Mugarura, S., Kato, E. and Kayobyo, G. Assessing the impact of the national agricultural advisory services in the Uganda rural livelihoods. IFPRI Discussion Paper 724. IFPRI, 2007, Washington, D.C., USA. Impact Evaluation and Returns to investment of the National Agricultural Advisory Services (NAADS) Program of Uganda, International Food Policy Research Institute, Washington, DC, USA. October 27, 2008. Rebuilding Agricultural Livelihoods in Post-Conflict Situations: What are the Governance Challenges? The Case of Northern Uganda, Regina Birner (IFPRI), Marc J. Cohen (Oxfam America) and John Ilukor (Makerere University), March 2010.

63

64