EMERGENCY OPERATIONS AND REHABILITATION DIVISION (TCE)

M&E Toolkit

Macro Irrigation Schemes

VERSION 1, DRAFT 1

October, 2006

by Andy Lyons, Consultant

REGIONAL INTERAGENCY COORDINATION SUPPORT OFFICE (RIACSO)

JOHANNESBURG, SOUTH AFRICA

ii

iii

The FAO M&E Activity Toolkit Series This M&E toolkit is one of a series that has been developed by the FAO Regional Interagency Coordination Support Office (RIACSO) in Johannesburg, South Africa. The goal of these toolkits is to strengthen and harmonize monitoring and evaluation of supported emergency activities throughout the southern Africa region. To achieve this goal, these toolkits provide a core set of indicators and templates that enable common data collection and the aggregation of results across multiple project sites within the region. The focus of this toolkit is on macro irrigation schemes, which are broadly defined as medium to large sized irrigation systems that transport water vertically and horizontally from a source to a cultivation area shared by a group of farmers usually with individual plots. This toolkit does not cover the monitoring of smaller irrigated garden projects, which are included in the toolkit for gardens. The audience for this M&E toolkit is anyone who is involved in implementing macro irrigation schemes. This includes FAO staff at country and regional levels, implementing partners at specific project sites, government technical and extension officers, NGO staff, and consultants. These toolkits are a work in progress. To be successful they must address regional cross-cutting information needs on project impact, while also accommodating a diverse set of working conditions, operational resources, and information requirements of numerous national and local users. This is not an easy task, hence feedback from users is essential. Please channel all feedback through your local FAO Emergency Coordination Unit (ECU). Comments should be sent to the Regional Information Officer at RIACSO, Phil Fong <phillip.fong@fao.org>.

How to use this Toolkit

1. Review the core output and outcome indicators (page 5)

2. Review the sample data collection forms (page 24)

3. Develop definitions for “low”, “medium” and “high” levels of vulnerability appropriate for your country, based on the questions asked in the baseline household survey (page 8)

4. Develop a workplan for data collection, processing, and analysis that will work for your specific context (page 21)

5. Implement the workplan.

Development of this toolkit has been made possible by the generous support of The Government of the Republic of South Africa under OSRO/RAF/510/SAF.

iv

v

Table of Contents An Introduction to Macro Irrigation Schemes ...........................................................................................1 FAO's Regional M&E Strategy .................................................................................................................2

Basic vs. Preferred Standards for M&E ................................................................................................3 Monitoring Macro Irrigation Schemes ......................................................................................................4

Components of a Macro Irrigation Project ............................................................................................4 Core Indicators for Macro Irrigation Schemes ......................................................................................5 Measuring Household Vulnerability and HIV/AIDS Impact ................................................................8

Nuts and Bolts of M&E Data Collection for Macro Irrigation Schemes...................................................9 Overview of Information Flow..............................................................................................................9 Data Collection Workplan...................................................................................................................10

Pre-activity planning .......................................................................................................................12 Selection of beneficiaries ................................................................................................................13 Baseline household survey..............................................................................................................13 Training...........................................................................................................................................15 Planting, production, harvesting, and sales .....................................................................................15 End-of-season review......................................................................................................................16 Terminal evaluation ........................................................................................................................17 Post-project evaluation....................................................................................................................20

Macro Irrigation Scheme M&E Database Software ................................................................................20 Making a M&E Workplan for Macro Irrigation Schemes.......................................................................21 References................................................................................................................................................22 Annex I – Sample Data Collection Forms ...............................................................................................24

Adapting the forms..............................................................................................................................24 Beneficiary Selection Worksheet ........................................................................................................25 Training Log........................................................................................................................................26 Baseline Household Survey.................................................................................................................27 Garden Diary .......................................................................................................................................29 Seasonal Production and Sales Summary............................................................................................30 End-of-Season Household Survey.......................................................................................................31 Irrigation Scheme Seasonal Summary.................................................................................................33

Annex II – Macro Irrigation Schemes M&E Database............................................................................35 Annex III – Data format for sharing indicator data .................................................................................36

Why standards for data formats are needed ........................................................................................36 Table Definitions for Output and Outcome Indicators........................................................................36 Other Summaries.................................................................................................................................38 Summaries for Making Custom Maps.................................................................................................39

Annex IV – Supplemental Data Capture Forms ......................................................................................40 Baseline Status of the Scheme before Non-Technical Rehabilitation .................................................41 Baseline Survey Scheme Level Questionnaire....................................................................................43 Baseline Survey Individual Farmer Questionnaire..............................................................................50

Table of Tables Table 1. Macro Irrigation Scheme Indicators ............................................................................................6 Table 2. Data collection tasks for each phase of implementation............................................................12 Table 3. Number of SSI schemes, households, marketing outlets, and cropping intensity .....................36 Table 4. Training indicators.....................................................................................................................37 Table 5. Inputs .........................................................................................................................................37 Table 6. Vulnerability profile, quality of dietary intake, and number of months of production..............37 Table 7. Production and income ..............................................................................................................38

vi

1

AN INTRODUCTION TO MACRO IRRIGATION SCHEMES Despite a growing commercial agricultural sector, the majority of farmers in the southern African region are still small-holders practicing rain-fed agriculture. Although some areas in the region have consistently adequate rainfall for agriculture, many areas are semi-arid and/or drought prone. These areas produce moderate yields even in “good” years, and are prone to total crop loss during periods of drought. There is also strong evidence that precipitation in some areas has experienced a long term decline and is becoming more irregular, possibly due to global climate change and/or local loss of forest cover. Not surprisingly, semi-arid and drought-prone areas also tend to suffer the greatest levels of food insecurity. Although loss of soil fertility, the AIDS pandemic, and weakened capacity within government are all contributing factors, the “straw that breaks the camel’s back” and leads to a humanitarian emergency is often inadequate rainfall. In 2002 alone, an estimated 13 million people in the region required food aid to prevent starvation. In response, governments and donor agencies have been supporting small-scale irrigation projects throughout the region. Irrigation offers the promise of increasing production for thousands of small-scale farmers by removing one of the key constraints to plant growth – access to water. Production can increase both in terms of higher crop yields as well as the number of cropping cycles that can be produced in one year. Many irrigation schemes are centered around vegetable and livestock production, two commodities which can 1) improve the level of dietary nutrition and 2) increase incomes. Irrigation projects are therefore well-suited to support a broad range of household livelihood strategies and mitigating the impacts of shocks such as unemployment and HIV/AIDS. Irrigation projects tend to fall in one of two categories, irrigated gardens and macro irrigation schemes. While there is some overlap in these categories, the main differences lie in the size of the irrigated area, the source of water, type of equipment used, and amount of permanent infrastructure.

Definition of Macro Irrigation Schemes For the purposes of M&E, irrigation projects have been divided into irrigated gardens and macro irrigation schemes. Although there is a certain amount of overlap between these categories, macro schemes tend to be larger projects that serve multiple households and involve the use of permanent infrastructure for the storage and transport of surface water, such as dams, weirs, canals, and pipeline. Schemes may also involve mechanical pumps or gravity fed systems for water lifting.

Although many irrigation projects have dramatically increased production in the short term, the sustainability of irrigation schemes has been disappointing overall. Irrigation systems demand relatively high levels of maintenance that requires high levels of social organization, financial management, and linkages to support services. Setting up local institutions and creating linkages can be difficult and time consuming, and hence are often neglected when it comes to the implementation of projects.

2

FAO’s Emergency Operations and Rehabilitation Division (TCEO) has supported macro irrigation schemes throughout the southern Africa region, especially in Lesotho, Zimbabwe, Zambia, and Mozambique. Many projects focus on the rehabilitation of schemes that have for one reason or another have stopped functioning properly. The focus of many macro irrigation projects includes a mixture of technical rehabilitation, improving access to inputs, strengthening the capacity of local management institutions, and marketing. FAO'S REGIONAL M&E STRATEGY

Monitoring versus Evaluation monitoring: the process of collecting information in a structured manner,

based on a specific set of goals and objectives

evaluation: the process of summarizing and analyzing information to answer specific questions about a project

Monitoring and evaluation in general supports three functions:

• accountability – M&E allows donors, partners, and beneficiaries to see whether a project is being managed responsibly, is on course, and achieving the desired impacts.

• implementation – M&E facilitates implementation of an activity. For

example, irrigation scheme managers need to know about production problems being experienced by scheme members as soon as possible in order to take remedial measures.

• improving project design – M&E allows project staff to make

improvements in both the design and implementation of projects, for example targeting, the sequencing and timing of tasks, design of training, complementary services, etc.

FAO TCEO and other humanitarian relief agencies are trying to strengthen M&E for all of the above reasons. To strengthen activity monitoring in the southern Africa region, FAO TCEO is using a three-pronged approach (Lyons, 2006):

• a conceptual framework outlining a regional M&E system based on the information needs and operational resources of FAO and its partners

• a set of core outcome indicators which cut across multiple activities and project sites, in order to show aggregated impact at national and regional levels

• a series of M&E toolkits like this one providing guidelines and tools for monitoring different types of activities

3

Why all this new fuss over M&E? What's wrong with the way we've been reporting?

M&E in emergency interventions is frequently a challenge due to time constraints, limited staff resources, and the short timeframe of most projects. M&E is therefore often delayed until the project is nearly over, or focuses only on the immediate outputs of an activity. A recent review of terminal reports has shown that most project reporting has not met the basic information needs of project planners and donors, who have a strong interest in project impact, lessons learned, and the linkages between activities. Without evidence to demonstrate that activities are producing results, the credibility of the entire programme is questionable. This toolkit represents one mechanism to fill in some of these gaps by providing guidelines and templates for a more systematic approach toward data collection and analysis.

Basic vs. Preferred Standards for M&E One of the challenges of trying to harmonize monitoring in multiple project sites across multiple countries is that the operational context and resources for M&E vary tremendously. To accommodate the varying levels of staff and funding, and not over-burden offices that are poorly resourced, the M&E guidelines that follow are classified as Basic and Preferred . The Basic level of monitoring covers all of the key information needs, including the core outcome indicators, using a minimum amount of time and staff resources. The Preferred standards also capture all of the core information needs, but go slightly beyond to get a better understanding of an activity’s impact relative to the baseline conditions, a more thorough understanding of cause and effect, and an assessment of the longer-term impact and sustainability issues. All FAO ECUs and their implementing partners are expected to implement a monitoring system which at a minimum meets the Basic M&E standards, and as many of the Preferred standards as resources permit. Some ECUs may also wish to borrow from the Preferred standards to meet reporting obligations under funding arrangements from other donors or joint-programming agreements.

General Differences Between Basic and Preferred M&E Standards (specific differences outlined below)

Basic Preferred • data collection is mostly incorporated into

routine operational tasks, such as beneficiary selection and training, thus minimizing separate stand-alone tasks such as household surveys

• sample sizes for surveys are kept as small as possible

• comparison data can be gathered at the same time as impact assessment, for example from memory recall or non-supported production from within the same household, instead of a control group

• only direct beneficiaries are monitored

• in addition to the collection of data during normal operational tasks, additional focused data collection activities are encouraged

• larger sample sizes, including some non-beneficiaries, are used when picking samples for more in-depth data collection (e.g., surveys)

• participants in focus group discussions should be stratified (e.g., by gender and/or age) to encourage non-biased expression of viewpoints

• a post-project assessment is encouraged to determine whether results are sustainable

4

MONITORING MACRO IRRIGATION SCHEMES M&E systems for macro irrigation schemes serve multiple purposes. On the one hand, we want to know the impact the activity is having on livelihoods, as reflected by output and outcome indicators. Equally important, if not more so, is the information needed for designing and implementing a macro irrigation scheme. Macro irrigation schemes are complex undertakings that depend on numerous technical, institutional, and economic conditions being met. If any of the underlying conditions is not met, the entire scheme may cease to function well, which unfortunately is not uncommon (Dzvurumi, 2006). Thus in addition to measuring impact, M&E should serve as both a diagnostic tool (especially in the case of rehabilitation projects) as well as an early warning system for emerging problems. To develop an M&E system which captures both impact as well as the technical, social, and institutional performance, the following general steps are needed:

1. Make a list of the key components of a macro irrigation scheme, and identify which variables can be measured at each level.

2. Define core output and outcome indicators 3. Develop criteria for identifying food insecure households and households impacted by

AIDS 4. Design a plan for the logistics or “nuts and bolts” of data collection and analysis 5. Implement the plan 6. Review the results and adjust the M&E plan as needed

Components of a Macro Irrigation Project Macro irrigation schemes are among the most organizationally complex projects for strengthening food security. Scheme involve management structures, households, and infrastructure, all of which must work together for the project to produce the desired outcomes. This hierarchical structure of an irrigation scheme in turn requires monitoring at different levels. Hence one of the first steps in developing an M&E system for an irrigation scheme is identifying the components and key variables at each level. While there is some variation among projects, Figure 1 below illustrates a generic structure that represents most irrigation schemes.

5

Figure 1. Components of macro irrigation schemes Level Description Measurable characteristics

country level

the ECU is the main coordinating body at the country level

number and geographic distribution of irrigation schemes

macro irrigation scheme

specific geographic areas where macro irrigation has been developed

new or rehabilitation location area equipped for irrigation area actually irrigated number of registered plot holders inputs and implements distributed training provided

physical

infrastructure

the physical infrastructure of the scheme, including the water delivery system as well as fences, drinking water points, toilets, access roads, etc.

water delivery and storage structures pumping equipment condition of infrastructure stability of water source toilets, drinking water points

management

institutions

the organizations and policies for managing the scheme, including the local Irrigation Management Committee (IMC), Water User Association, bylaws, and supporting institutions

date established demographic composition meetings & record keeping finances management capacity

households

A household which is a registered member of a scheme and maintains a plot either individually or with others

household demographics level of vulnerability impacted by HIV/AIDS training received inputs received income other fields, livestock, assets distance to scheme

irrigated plots

One or more irrigated plots being cultivated by a household

size production & yield farming practices problems encountered

Core Indicators for Macro Irrigation Schemes During an M&E workshop in February 2006, FAO Emergency Coordinators from southern Africa developed a set of core indicators for small-scale irrigation projects which include macro irrigation schemes (Lyons, 2006). These are presented in Table 1 below.

Variables measured at the household level

Variables measured at the

plot level

6

The difference between output and outcome indicators

Monitoring is often divided into looking at the process of conducting an activity, and its ultimate impact. Process is measured by describing the immediate outputs, such as the number of inputs distributed or people trained. Impact is evaluated by defining and measuring the medium and long-term outcomes of an activity. In order to claim that an activity made a difference, we need evidence of both the process and the final impact. Hence most M&E systems simultaneously keep track of both by defining separate indicators for outputs and outcomes.

Table 1. Macro Irrigation Scheme Indicators Indicator Interpretation

Output Indicators 1. Number of macro irrigation schemes supported, disaggregated by type of support (new vs. rehabilitated), size, and water lifting method (gravity fed surface, gravity fed overhead, pressurized surface, pressurized overhead)

The number of schemes supported provides a measure of the scope of the interventions as well as the types of irrigation strategies used. This information can be presented in map form to show the spatial distribution of supported schemes.

2. Number of beneficiary households, disaggregated by gender and level of vulnerability

This indicator captures the overall breadth of irrigation schemes in terms of beneficiaries, as well as a profile of the vulnerability levels. This information can also be presented in map form.

3. Number of beneficiaries trained, disaggregated by type of training and gender

Training is a key element of most irrigation schemes. This indicator will capture both the types of training offered as well as number of beneficiaries reached by gender.

4. Quantity of inputs distributed, disaggregated by type of input (e.g., seed, fertilizer, pesticides, tools)

Irrigated cultivation is input-intensive, hence input distribution is a common element of many irrigation schemes. Although inputs do not automatically translate into improved food security, they are an important intermediate step toward increasing production.

Basic

5. Total irrigated area planted. The total amount of irrigated land provides another measure of the scope of irrigation projects, and will capture the up-scaling of projects from demonstration projects to mainstream production.

Outcome Indicators 6. Quantity of production disaggregated by crop

Increased production for consumption or income is one of the desired outcomes of irrigation schemes and is a core outcome indicator (COI) for other activities.

Basic

7. Percent increase in yield Yield is a good measure of agricultural intensification which is a common theme in many emergency projects supported by FAO TCEO. This is also one of the COIs at the regional level.

7

Indicator Interpretation 8. Quality of dietary intake (Food Consumption Score)

The Food Consumption Score is an index of the consumption of major food groups in the past week, weighted by their nutritional value. It is a good outcome indicator for access to food and the ability to transform raw food into a nutritious diet.

9. Number of months of production per year

This indicator captures one of the main goals of irrigation schemes which is to smooth out food production throughout the year, particularly during the ‘hungry season’

10. Cropping intensity Cropping intensity is the ratio between the total amount of irrigated area summed over all growing cycles in a year, divided by the physical area equipped for irrigation. It is a measure of irrigation intensity and captures whether irrigation infrastructure is being fully utilized.

11. Income generated from sales of crops

Income is an important asset for livelihood security and one of the desired outcomes of irrigation schemes. It is listed as a preferred indicator due to the difficulty in measurement.

Preferred

12. Number of marketing outlets used by scheme members, disaggregated by type of outlet

Many irrigation schemes aim to improve access to markets so that beneficiaries can convert increased food production into other livelihood assets.

Good indicators are SMART

Specific A good indicator measures one and only one characteristic or quality, clearly and unambiguously. A common mistake is to try to put too many characteristics into one indicator. For example, if we are told the indicator “quantity of diversified crop production” has dropped, we wouldn't know if the indicator fell because production was low, or production was good but it just wasn’t diversified.

Measurable It is important that an indicator be measurable in a consistent manner in different places and at different times, so that results can be compared over time and across sites. If the indicator is very subjective or not well-defined (e.g., the 'effectiveness' of a training program), then it will be difficult to interpret or compare results.

Attainable It is important that an indicator be measured easily and reliably. If the measurement is very complicated, requires a lot of resources, or is dependent on many factors falling into place, there is a high risk it will not be measured at all.

Relevant Indicators are supposed to tell us whether we are reaching our goals and objectives, so a good indicator must clearly relate to the condition being measured.

Time-bound We need to know the time period of the measurements to be able to interpret an indicator. The time interval and frequency of measurement should be explicit in both the indicator definition and reporting.

8

Measuring Household Vulnerability and HIV/AIDS Impact Like many other food security interventions, the primary target group for irrigation schemes are households which are vulnerable to food insecurity but are still capable of production (i.e., vulnerable but viable). There are numerous factors which can cause agricultural households to become food insecure, but one factor which is of particular importance in southern Africa is the HIV/AIDS pandemic. Many of the household level indicators for irrigation schemes (e.g., production) require results to be disaggregated based on their level of vulnerability in general, and HIV/AIDS in particular, to show whether vulnerable households are being reached and if so whether the intervention is producing benefits appropriate to their needs. This poses three challenges for monitoring conservation schemes, 1) how do we identify appropriate households for participation, 2) how do we measure food insecurity vulnerability in general, and 3) how do we determine whether a household has been impacted by HIV/AIDS? There are many methods for selecting beneficiaries for macro irrigation schemes. Obviously geographic proximity to the project site is one of the main criteria, however in most schemes the population of the catchment area far exceeds the number of irrigated plots, so additional selection is needed. Some targeting methods involve household-level surveys followed by a quantitative analysis to determine thresholds of vulnerability. Other methods draw on local knowledge through community-based discussions to identify households which are vulnerable but still have a capacity for farming. Other methods involve self-selection mechanisms (e.g., requiring in-kind contributions of work) to filter out all but the targeted households. Sometimes methods are mixed, for example combining community-based targeting with an objective validation process that randomly selects names from a community-generated list and validates whether they meet the stated criteria. All methods have their strengths and weaknesses, and no single method is appropriate for all contexts. In many emergency situations, community-based methods are used due to constraints of time and data. It may not be very practical to record detailed information for every beneficiary selected for an irrigation scheme, however it is feasible to ask the groups involved in targeting to at least record the criteria they used in selecting beneficiaries. This has generally not been done in the past, leaving the targeting process a “black box” open to criticism. The Beneficiary Selection Worksheet (page 25) has been designed to make the targeting process more systematic and record the criteria used in beneficiary selection. This worksheet is designed to be used at the community level and is not meant to be entered into the M&E database. The Preferred M&E standard however recommends entering a sample of randomly selected worksheets for analysis of the targeting process. A second M&E task is validating the effectiveness of our targeting methods and determining whether targeted households are actually benefiting from the activity. While it isn’t possible to evaluate the vulnerability of all participants, if we randomly select households for the baseline and end-of-season household surveys (see below), our results should be approximately correct for all scheme members. Measuring the vulnerability of dozens of households is certainly easier than trying to measure hundreds, but this still leaves us with a big methodological challenge: how does one measure vulnerability to food insecurity? The short answer is that it varies from place to place, depending on the type of farming system, soils, access to employment and social services, and so on. The characteristics of a vulnerable household living on rocky soils in the highlands of Lesotho will be

9

quite different than the characteristics of a vulnerable family in the Zambezi valley. The amount of land needed for food production, dependence on draught power, labour requirements, accessibility to fertilizer, opportunities for alternative employment, and many other factors will vary enormously from country to country and even within countries. For any one context, multiple criteria will probably be needed to measure vulnerability to food insecurity. The same is true for determining whether a household has been impacted by AIDS. In some countries, there are village-based AIDS support groups and one can ask direct questions to assess whether a household has been affected by AIDS. In other places, the stigma of AIDS is still quite strong and indirect methods or proxies must be used to avoid offending people or worsening the stigmatization. Each ECU will therefore need to select criteria for determining how to measure household level vulnerability to food insecurity. The end result should be a vulnerability index that ranges from 1 to 3, where 1 = low vulnerability, 2 = moderately vulnerable, and 3 = highly vulnerable. The vulnerability score should be based on the responses to a series of simple yes/no questions on the baseline household survey (page 27). For example, the ECU in Malawi may decide that a highly vulnerable household is one that has only one working-age adult with five or more children, or has less than 1 ha of land. The Zambia ECU may decide that a highly vulnerable household in Southern Province is one that has no cattle or draught power, regardless of the family size or land holdings. Similarly each household visited during the baseline and subsequent surveys must be assigned a value of “yes” or “no” as to whether the household has been impacted by AIDS. This could be based on whether an adult member of the household has died, someone in the household is chronically ill, or the presence of orphans and vulnerable children (OVCs). Like vulnerability, it will be up to each national ECU to determine how to use the responses on the baseline household survey to identify households impacted by AIDS, with RIACSO providing assistance when needed.

Each Emergency Coordination Unit needs to decide how to interpret the responses on the baseline household

survey to determine the thresholds for high, moderate, and low levels of vulnerability appropriate for the local farming systems, and how to determine whether a household has been impacted by AIDS.

NUTS AND BOLTS OF M&E DATA COLLECTION FOR MACRO IRRIGATION SCHEMES Overview of Information Flow Monitoring data from a macro irrigation scheme is generally recorded in the field by a scheme manager, individual plot holders, and extension staff from FAO or an implementing partner. Next, some of this information may be copied from the paper forms into a database and shared electronically with other offices, including the national ECU and FAO regional office (RIACSO). Other data collected from the field level may be used internally at the local level, and/or shared only in summary form. This general flow of information is illustrated in Figure 2 below.

10

Figure 2. Information Flow from the Field to Regional Level

Country 2ECU

Country 3ECU

reportspresentationsmaps

ECU field staffw/ paper forms

RIACSORegional Office

reportspresentationsmaps

coreindicators

coreindicators

soft copy ofcore indicators

regional summaries& feedback

updates fordonors & partners

country-levelfeedback

Scheme B

Scheme C

Macro Irrigation Scheme ANGO & gov't

extension officersw/ paper forms

Field officeM&E database

SchemeManager

site-levelfeedback

NationalECU

soft copy ofsummary data

Plotholders

At each level, information will be used for different purposes. Some offices will use M&E data to develop workplans or prepare monthly progress updates. Others will need it for preparing mid-term reviews and terminal reports, and still others will use the data for more in-depth assessments or lessons learned briefs. The key to an effective vertically integrated information system is to know what information needs to be shared and in what format. In general, higher organizational levels need only summaries of information collected at lower levels. Annex III (page 36) describes the format for sharing the core output and outcome indicators for macro irrigation schemes. Data Collection Workplan The most challenging component of many M&E systems is data collection. A data collection workplan is used to plan the amount of time, human resources, transport, and other resources needed to collect data. A good data collection workplan recognizes the tradeoffs between the information needs, available resources for fieldwork, and the project context. The most important characteristic of a data collection workplan is that it is viable, even if that means collecting less data than you would ideally like.

11

info needs &indicators

staff &operationalresources

projectcontext

data collectionworkplan

A data collection workplan takes into account the information needs, operational resources, and project context to come up with a plan for data collection, processing, and analysis that is viable.

Table 2 next page describes the main stages of implementation in a macro irrigation project, and the accompanying data collection tasks at each step of the process. The sections that follow describe the data collection tasks in greater detail. This table can serve as a guide for a data collection workplan, but is incomplete because it doesn't specify individual personnel who will be assigned for each task, geographic areas, transport requirements, amount of time needed, budget requirements, etc.

At a glance: Basic level data collection for macro irrigation schemes

• Pre-activity planning: conduct a baseline assessment of the physical characteristics of the irrigation site and management structures

• Selection of members: groups involved in identifying scheme members record the criteria used for selection

• Before the start of the season conduct a baseline household survey on a sample of plot holders, assessing demographics, vulnerability, and participation in the scheme.

• Training: extension officers and community-based trainers maintain a simple training log, recording the numbers of people trained and topics covered

• Planting, harvesting, and sales: selected plot holders that were interviewed for the baseline are invited to keep a garden diary of planting, harvesting, and sales activity, with assistance from the scheme manager or extension staff.

• End of season: the scheme manager and extension staff compile a summary of the season, including training provided, inputs distributed, and crops planted. An end-of-season household survey is repeated, summarizing production and sales data for those plot holders who maintained garden diaries. This review is repeated for each growing season covered by the project.

• End of project: a terminal evaluation includes additional focus group discussions and interviews with major stakeholders and partners.

12

Table 2. Data collection tasks for each phase of implementation Data Collection Tasks Implementation Phase

Basic Preferred (in addition to basic) pre-activity planning & scheme baseline assessment

• discussions with stakeholders • conduct a baseline assessment

of the physical characteristics and management structures of existing irrigation schemes (page 33)

• focus group discussions with scheme members

selection of beneficiaries • record criteria used for scheme membership using the beneficiary selection worksheet (page 25)

• 2% (maximum 50) of the Beneficiary Selection Worksheets are randomly selected for entry into the M&E database for analysis of the selection process

baseline household assessment

• administer baseline household survey to a sample of scheme members (page 27)

• random selection • sample size 5% (maximum 200)

• sample size 10% (maximum 400) • 25-30% of sampled households should

be non-members • GPS coordinates recorded

training • training log maintained (page 26) • additional knowledge assessments and evaluations

planting & production • selected individual plot holders record planting and production events in a garden diary (page 29), with biweekly or monthly support visits from extension staff or the scheme manager

harvesting and sales • selected plot holders record harvesting and sales in a garden diary (page 29)

• monthly summary of sales and harvesting is compiled (page 30)

end-of-season review • scheme manager or extension staff fills in seasonal summary (page 33)

• GPS coordinates for each plot recorded • additional focus group discussions

terminal evaluation • stakeholder interviews • focus group discussions

• separate focus groups for men/women • external consultant contracted for

evaluation post-project evaluation • revisit some of the sampled plot holders

for semi-structured interviews • stakeholder interviews • focus group discussions

Pre-activity planning During the pre-activity planning phase, the details of the project design are finalized, sites identified, and roles and responsibilities of partners defined. For rehabilitation projects (as opposed to new schemes), this is also an opportunity to collect some baseline data on the existing scheme infrastructure and management structures using the Irrigation Scheme Seasonal Summary (page 33). The focus of this assessment is on the physical characteristics of the scheme – area equipped for irrigation, infrastructure, water source, and management etc. These data will be used as a baseline comparison for physical and institutional improvements to the scheme. All baseline Seasonal Summaries should be entered into the M&E database for analysis (page 20). If the number of supported schemes in the country exceeds the capacity of project staff to visit them all, then a sample of schemes can be selected for the baseline, otherwise all schemes should be recorded during the baseline.

repeat for each season

13

The Preferred M&E standard also calls for some focus group discussions with scheme members to learn more about their production practices, experience with irrigation, access to markets, availability of inputs, etc. Such discussions normally take place at the beginning of a project anyway, a focus group discussion is merely a more structured format for focusing and recording the discussion. Selection of beneficiaries The selection of scheme members could be a complex process which is frequently community-based. These factors put limits on the amount of data that can be feasibly collected. The information required at this stage is therefore not detailed information on each and every member, but general information on the process or criteria by which beneficiaries are selected. The Beneficiary Selection Worksheet (page 25) is designed to help the groups involved in the selection to be systematic about the criteria they use, but is not intended to be collected and entered into the M&E database at the Basic level of M&E. However the Preferred standard calls for a random sample of beneficiary selection worksheets (2% with a maximum of 50) to be entered so that a profile of the targeting criteria used can be generated. Baseline household survey The Baseline Household Survey (page 27) is the first of two household-level surveys in which more detailed information is collected on plot holders, their participation in the scheme, and dryland farming practices. Data from the baseline survey is also used to estimate the level of vulnerability of scheme members, which will say something about the effectiveness of the targeting methods, and the percentage of members that come from households affected by HIV/AIDS.

Best Practices in Implementing a Survey: Training and Field Supervision

As much as we try to ensure that a survey form is clear and easy to use, there has never been a survey that didn't cause some confusion in the field. Hence an important component of survey work is providing training for enumerators. This doesn't have to be extensive, however training is essential to ensure that all enumerators understand the questions, translate them into the local language in a similar manner, record answers similarly, measure the size of plots using the same method, etc. It’s also helpful to pilot-test the survey beforehand to catch any mistakes or confusion in the way the questions are written. A second best practice for survey work is field supervision. Particularly in the early stages of data collection, it is important for survey forms to be reviewed by a supervisor as they come in, so any problems or confusion can be quickly identified, diagnosed, and corrected before they cause any data to be thrown out.

The baseline survey should be conducted near the beginning of the project, for example during the selection of members or during the first planting activities. The same households that are selected for the baseline survey will be revisited during the end-of-season survey, hence it is very important that the households for the baseline survey are selected randomly so that they represent the entire population of scheme members. A good way to randomly select households for the baseline survey is to:

14

1. Decide the total number of members to be surveyed, based on the needs and resources

available for fieldwork, data entry, etc.

2. Decide which schemes will be monitored (if not all), making sure that there is representation from each agro-ecological region, type of irrigation infrastructure, etc.

3. Decide how many members will be surveyed in each of the schemes (either divide the total desired sample size by the number of schemes evenly, or make the sample size in each scheme proportionate to the number of members)

4. Take the list of members from each of the selected schemes, and select the every nth name on the list, where:

n = total number of members in the scheme desired number of members for surveying

The Basic standard of M&E calls for 5% of members to be selected for the baseline survey, up to a maximum of 200 for the entire project (country wide). If the Preferred standard is used, the total sample size should be 10% of the total number of members, up to a maximum of 400, of which ¼ to ⅓ may be non-participant households for comparison purposes. Non-member households should also be selected as randomly as possible, from the same areas as the member households, and preferably be involved in other gardening activities. The Preferred standard also recommends recording GPS coordinates of sampled households in order to facilitate relocating the same household during the end-of-season household survey and visualizing the spatial patterns in vulnerability and project outcomes.

Our office staff is small and our implementing partners don’t have the capacity to do survey work.

What should we do?

In the real world, operational constraints and institutional dynamics can make it difficult to conduct a survey for an even moderate sample like 5%, particularly when a project has a late start or field staff have no experience or interest in survey work. In these cases, you should still survey as many households as resources permit, and concentrate on making the exercise a learning experience for project staff and partners. Although the small sample size may make it difficult to make definite conclusions about the project as a whole, some interesting lessons can still be learned even from small surveys. If nothing else, you are guaranteed to learn some lessons on how to implement a household survey, such as the amount of time required, training needed for enumerators, improvements to the survey forms, resources needed for data entry, etc. This can help you plan better for the following season. In addition to downsizing the sample, other options may also exist for conducting a household survey. Many organizations share the costs of data collection by partnering with other organizations. The Swaziland ECU for example uses university students as enumerators when they come for attachment. In Zimbabwe, FAO coordinated 27 NGOs to implement a survey to collectively measure the impact of a number of input provision projects supported by different donors.

15

Training Most irrigation projects include a set of training activities in areas such as vegetable production, marketing, health and nutrition, or life skills. The Basic level of M&E requests training staff to maintain a simple log book of training activities (see Training Log, page 26). This template is simple enough to be used by all trainers, including community-based trainers. At the end of each growing season, training logs should be collected and summarized and the total number of trainings, participants, and contact hours recorded on the Irrigation Scheme Seasonal Summary (page 33). The wide array of training topics makes it difficult to recommend a more in-depth assessment tool for the many types of training, however the Preferred M&E standard calls for the use of training assessments that are appropriate for the topic. Training assessments can focus on measuring awareness, knowledge, values, or behavior. A variety of assessment methods are used in training evaluations, including interviews, observation of behavior, garden inspections, and focus group discussions.

We do a lot of training in our project, so why isn’t there an outcome indicator for the impact of training?

Many irrigation projects include a training component to improve participants’ skills in areas such as production, marketing food processing, and irrigation operations and maintenance. The number of topics and people trained is one of the core output indicators for irrigation schemes (Table 1), and a template for recording the outputs of training is included in Annex I (page 26). However there are no core outcome indicators for training, for two reasons. First, measuring the impact of training properly is very difficult and beyond the capacity of many implementing partners. You have to give knowledge assessments and observe behavior over a period of time that often exceeds the duration of the project. Second, it is very difficult to aggregate information on training impact from different projects and sites, especially when the topics of training are so varied. This does not mean that it isn’t important to assess the impact of training, because without assessments we really have no way of knowing whether the training is relevant to the participants or if the skills are really sinking in. However this type of evaluation is better suited for a terminal evaluation or post-project assessment. Some sample questions for a training assessment are provided in the section on terminal evaluations (page 19). In addition, projects with a heavy training emphasis are encouraged to define additional indicators to measure specific training outcomes for specific topics.

Planting, production, harvesting, and sales Monitoring planting and production in garden situations is difficult, because unlike field crops cultivation and harvesting take place almost continuously. Hence effective record-keeping must take place at the scheme level and be maintained preferably on a daily basis. Plot holders and scheme managers can keep track of important events such as planting, harvesting, and sales by maintaining a garden diary using a template such as the one shown in Annex I (page 29). If a scheme manager is absent, extension staff can work with individual plot holders to maintain diaries for their own plots. It is recommended that the some of the same scheme members that were interviewed for the baseline survey should be invited to keep diaries on production and income.

16

Experience has demonstrated that community-level monitoring is more likely to be effective if the information collected is relevant to local needs. Maintaining production records is useful for many purposes other than project M&E. Garden diaries can be used for various training purposes and demonstrating the effectiveness of specific production methods or watering schedules. Record-keeping however might be a new skill that will require initial training and extension support. When garden diaries don’t seem viable, or the needed extension resources for training and support are lacking, the following alternative strategies might be helpful:

• community-based extension workers, lead gardeners, or local training staff can provide support in record keeping to plot holders

• scheme-level record keeping systems can be pilot tested at a sample of sites • NGO or government extension workers can take charge of record keeping, for example

through biweekly or monthly visits To help compile member-level production and sales data, it is also helpful to have intermediate summaries that can then be aggregated into a final seasonal summary. The Seasonal Production and Sales Summary sheet (page 30) is a template for compiling daily records of production and sales into a seasonal summary. Training will also be needed for this form, including how the information can be used to improve planning and management. End-of-season review The end of every production cycle warrants a review regardless of whether project support is coming to an end or will continue for an additional season(s). The Irrigation Scheme Seasonal Summary (page 24) is designed to guide an end-of-season review by summarizing the physical properties of the scheme, management structures, market outlets, and project activities in infrastructure development, training, and input provision. The seasonal summary, which was also complete during the baseline, can be filled out by scheme managers with assistance from extension staff as needed. The seasonal summary requires the training log(s) to create the training summary, and input distribution records for the section on inputs. If data are not available, these sections should be skipped and the rest of the summary filled in. All seasonal summary sheets should be collected and entered into the M&E database as several of the core indicators are derived from this data. In addition to the seasonal summary for the entire scheme, the End-of-Season Household Survey (page 31) needs to be administered to the same members that were selected for the baseline survey. This is where demographic data is captured as well as participation in the scheme and quality of dietary intake. If the member has been maintaining production and income records, for example with a garden diary, data from the Seasonal Production and Sales Summary should be transferred onto the survey form. The Preferred standard also recommends conducting some focus group discussions with scheme members at the end of each season to understand issues that were missed or too complex to capture on surveys. Good topics for a focus group might include the impact of training, gender issues, problems with the management structures, and planning for the next season.

17

The other half of the story: Interpreting indicators

Indicators are useful in telling us what happened, but that's only one-half of the story. The other half is why the observed results occurred. To enable the reader to interpret our indicators, we must describe the context during the reporting period. For example if repairs to irrigation facilities were late then the production and income might be low even if inputs were available and interest from participants was high. Our description of the results should describe those factors which affected the success of the activity both positively and negatively.

Terminal evaluation Terminal evaluations typically occur at the end of a project’s funding period, even if the activity continues on its own or through some other form of support. Emergency projects usually encompass only one growing season but in some instances could include several seasons, particularly if production is year-round which is typical in irrigation schemes. If a project only spans one growing season, then the End-of-Season Review and Terminal Evaluation would be merged into a single exercise. Terminal evaluations are often a contractual obligation, but should really be seen as an opportunity to share the results achieved with partners, review experiences, document lessons learned, and make recommendations for future irrigation projects. If funds are available, an outside consultant Preferred can be helpful by bringing a wider set of evaluation skills and a broader perspective on irrigation schemes. However even when funds are tight, an internal evaluation is highly worthwhile for packaging, interpreting, and communicating the results of the M&E system.

We haven’t got all of the M&E data from our implementing partners, and the terminal evaluation report is due next week!

What should we do?

Missing or delayed data is a fact of life in M&E, especially when you rely on implementing partners to collect and/or enter data for you. When you’re up against a deadline and some data hasn’t been collected and/or entered into the M&E database, you can still produce summaries with what you have. Although more data usually produces better estimates, preliminary estimates can still be generated in order to meet a reporting deadline. Delayed data should also trigger a lesson learned for future M&E. Was the data collection workplan too ambitious? Were the expectations for implementing partners clearly articulated and incorporated into Letters of Agreement? Were there technical problems with the fieldwork or data entry? By diagnosing the problem you can adjust the M&E system for the following season to ensure that field data are ready when needed.

18

The elements of a final evaluation often include:

• analysis and interpretation of project M&E data • physical assessment of the performance of the irrigation infrastructure, irrigation schedule,

maintenance, etc. • interviews with major stakeholders • interviews with beneficiaries • assessment of the broader context, such as climate, macro-economy, etc.

Most terminal evaluations also include focus group discussions and interviews with additional stakeholders including participants, community leaders, partners from government, other NGOs, and donors. The Preferred M&E standard calls for separate focus groups for men and women to ensure that gender issues come out clearly and without bias. Separate discussion groups may also be needed if there are other sub-groups represented among the scheme members, such as youth or individuals affected by HIV/AIDS. Questions that should be addressed in a terminal evaluation are usually outlined in a Terms of Reference, and should be based on the issues that emerged during the course of the season. It is generally preferable to articulate 3-5 key issues for a project evaluation to focus on, rather than ask every question in the book and risk getting an evaluation that is extremely general or based on scanty evidence. Sample questions for a terminal evaluation of macro irrigation scheme project are provided on the next page. See also FAO (1998) for guidelines on planning a terminal evaluation.

19

Sample Evaluation Questions for a Terminal Evaluation of a Macro Irrigation Scheme1

Some typical issues addressed in terminal evaluations include:

• Summarize the achievements of the macro irrigation scheme (output and outcome indicators).

• How did beneficiaries perceive the project? What did they like and not like? Were the participation requirements for members realistic in terms of their available capital, labour, skills, and opportunity costs?

• Assess the institutional capacity of Irrigation Management Committees, Water User Associations, and any other groups involved in planning or managing the scheme. Describe the frequency of meetings, effectiveness of record keeping, gender representation, interactions with government, collection and management of user fees, enforcement of bylaws, etc.

• Assess the technical performance of the irrigation scheme, including the adequacy of the water supply, infrastructure design, equity in water distribution, efficiency of equipment and transport systems, operations and maintenance, etc.

• How did implementing partners from government and other stakeholders perceive the project? Describe any institutional barriers to implementation.

• Describe any gender issues that emerged during the project, including the role of women in decision-making, planning, the beneficiary selection process, allocation of costs and benefits, provision of toilets and drinking water, cultural barriers to participation by women, etc.

• Describe the marketing side of the irrigation scheme for both inputs and production, including the strategies used to improve access to markets, strengthen weak markets, improve production quality and scheduling, marking information sharing, etc.

• Describe the patterns in vulnerability of the beneficiaries, including impact from HIV/AIDS.

• What impacts did the irrigation scheme have on livelihoods, and through which mechanisms? If production was increased, what did farmers do with the additional harvest?

• What were the benefits of the scheme on AIDS impacted households? Were households affected by AIDS able to benefit as much as non-affected households? How was AIDS mainstreamed into the project?

• What will be needed to sustain any gains in livelihood, and what additional assistance could be needed to maintain the progress made?

• What training was provided, and what were the observable outcomes? • If the project sought to rehabilitate an irrigation scheme, describe why

the scheme deteriorated in the first place, and what the project did to ensure the same problems won’t threaten the scheme again in the future.

• Describe any significant patterns changes that took place in the broader context, such as climatic patterns, organizational or political changes, trends in the macro economy, etc., and how these changes might have affected the implementation and outcomes of the project.

• Describe the policy environment as it relates to macro irrigation schemes, including land and water tenure, service delivery, and coordination. Did any policy issues create barriers for the project?

• Describe the monitoring system used by the scheme, and evaluate its strengths and weaknesses.

• Based on the findings, make recommendations for future projects.

1 see References section for other examples of irrigation evaluation guidelines and case studies

20

Post-project evaluation Output and outcome indicators do a reasonably good job in tracking the short and medium term impacts of irrigation schemes, however many other results will only emerge a year or more after an emergency project is officially over. Post-project evaluations are especially important in macro irrigation schemes because of the huge investment of time and finances in developing the infrastructure, and a known pattern of problems with sustainability. Post-project evaluations are rare in emergency programmes because there is usually no requirement nor funding to conduct them. Nevertheless, the Preferred M&E standard recommends a post-project evaluation because there is no other way to determine whether an irrigation scheme is effective and sustainable technically, economically, and socially. Two strategies for conducting a post-project evaluation with few resources is to make it a joint exercise with other organizations supporting irrigation projects, and to incorporate evaluations of past schemes in the M&E workplan of current projects. A post-project evaluation is similar in focus to a terminal evaluation, with particular emphasis on institutional, technical, and economic sustainability. The best way to prepare for a post-project evaluation is to ensure that the terminal evaluation is well-documented, and data are archived in an appropriate format for re-analysis. MACRO IRRIGATION SCHEME M&E DATABASE SOFTWARE The accompanying CD (Annex II, page 35) contains a customized database application which has been designed to accompany the data collection templates presented in Annex I (page 24). The macro irrigation scheme database, which is in Microsoft Access format, features a custom menu system with user-friendly data entry screens for each of the main data collection forms. The database will allow you to:

• enter M&E data on multiple computers and then merge the records into a single master database

• open ready-made tabular summaries (queries) and charts for analysis of the latest data entered

• generate a printed report of output and outcome indicators • export some or all of your raw data to Excel for further analysis and presentation • export indicator data in a standard format so you can send it electronically to another project

office (see Annex III, page 36) • import updates sent by the regional office with fixes and enhancements • supply the tabular information needed to make maps of your M&E data using an external

GIS program such as Dynamic Atlas or ArcView. The macro irrigation scheme M&E database will not interpret your data for you, nor write your report, but the statistical summaries and charts produced will greatly facilitate the process of analyzing impacts and lessons learned. For additional information on installing and using the M&E database, please see the users manual or select 'Help' from the main menu.

21

Reporting Problems and Desired Features

Users of the M&E database are requested to report problems or confusing error messages so that an update can be prepared and sent out. Problems that affect one user are likely causing trouble for other users as well. In addition, please forward any requests for additional summaries (tabular queries or charts) for data analysis. For example, if you would like to see a summary showing the breakdown of irrigate area by crop, but don't see an appropriate query on the menu system, please forward a description of the desired summary so a query can be designed and an update sent to you and other users via email. (In the interim, you could export the raw data to Excel and create the analysis there). In this way, the database will gradually become much more useful over time. All feedback on the database should be forwarded through the national Emergency Coordination Unit to the Regional Information Officer at RIACSO (Phil Fong <phillip.fong@fao.org>).

MAKING A M&E WORKPLAN FOR MACRO IRRIGATION SCHEMES Indicators and data collection templates won't collect any data or produce any analyses by themselves. For that you need an M&E workplan which operationalizes the details of an M&E system. Each ECU needs to take the lead on developing its own workplan for irrigation schemes as part of its overall M&E workplan in light of the local resources, information needs, and operational constraints. An M&E workplan specifies:

• which data will be collected, when, and by who (i.e., a data collection workplan, see page 10)

• how, when, and where paper forms will be entered into the database • who is going to do the analysis and write-up • estimated costs and time requirements associated with each of the above tasks

This toolkit is not an M&E plan, but the tools and recommendations presented provide many of the core elements of an M&E plan. To produce an M&E plan, the following additional steps are recommended in consultation with implementing partners:

1. Read through this toolkit to get an overview of what is needed and ideas for your specific M&E system. Start small and then expand after you’ve had some success!

2. In addition to the information needs at the regional level, which have already been defined, make a list of the most important national and local level information needs and questions that you would like to address through M&E.

3. Look at the list of Basic and Preferred indicators on page 5, and decide which ones will be adopted for your country.

4. Think about the specific factors that make households vulnerable to food insecurity in the local context, and come up with an index that defines three levels of vulnerability, based upon the responses to questions in the baseline household survey (see discussion on page 8).

5. Think about the indicators of HIV/AIDS impact, and how to determine whether a household is classified as impacted by AIDS based on the responses to questions in the baseline household survey (page 27).

6. Look at the sample data collection forms in Annex I, and decide which ones will be used and whether they need to be modified.

22

7. Modify the data collection forms as needed (soft copies can be found on the CD in Annex II). Contact RIACSO and other users of the irrigation scheme M&E database if changes to the database are needed.

8. Develop a draft workplan which outlines all of the needed M&E tasks, including the sampling methodology, field work, data entry, analysis, and writing. The workplan should specify the dates and staff involved in each task.

9. Discuss the M&E plan with implementing partners, revising it as needed. 10. Integrate M&E responsibilities into job descriptions, individual workplans, and Letters of

Agreement. REFERENCES Chancellor, FM and Hide, JM, 1997. Smallholder irrigation: Ways forward. Volumes 1 and 2. Report no. OD136. HR Wallingford, UK. [http://www.dfid-kar-water.net/w5outputs/small_scale_irrigation.html] Cornish, G. and Skutsch, J. 1997. A procedure for planning irrigation scheme rehabilitation. Report no. OD/TN84. HR Wallingford, UK. [http://www.dfid-kar-water.net/w5outputs/electronic_outputs/odtn84.pdf] Dzvurumi, F. 2006. Evaluation of emergency small scale irrigation projects in southern Africa. FAO Regional Interagency Coordination Support Office. Johannesburg. FAO. 1998. Evaluation Mission Brief. FAO Evaluation Service, Rome. [http://www.fao.org/pbe/pbee/common/ecg/233/en/MISBRIEF.doc] Field, W.P., and Collier, F.W. 1998. Checklist to assist preparation of small-scale irrigation projects in sub-Saharan Africa. Project No. R6518. International Commission on Irrigation and Drainage. [http://www.dfid-kar-water.net/w5outputs/small_scale_irrigation.html] Gorantiwar, S.D. and Smout, I.K. 2005. Performance assessment of irrigation water management of heterogeneous irrigation schemes: A framework for evaluation. Irrigation and Drainage Systems. Vol. 19, pp. 1-36. Hasnip, N., Mandal, S., Morrison, J., Pradhan, P., and Smith, L. 2001. Contribution of irrigation to sustaining rural livelihoods: Literature review. KAR Project R 7879. Report no. OD/TN 109. HR Wallingford, UK. [http://www.dfid-kar-water.net/w5outputs/electronic_outputs/odtn109.pdf] Lyons, 2006. A monitoring and evaluation framework for FAO emergency and rehabilitation activities in southern Africa. FAO Regional Interagency Coordination Support Office. Johannesburg. Makombe, G., and Sampath, R.K., 1998. An economic evaluation of smallholder irrigation systems in Zimbabwe. Water Resources Development. Vol. 14, No. 1, pp. 77-90. Ntsonto, N.E. 2005. Economic performance of smallholder irrigation schemes: A case study in Zanyokwe, Eastern Cape, South Africa. Masters Thesis. University of Pretoria. [http://upetd.up.ac.za/thesis/available/etd-02092006-102025/unrestricted/00dissertation.pdf] Shumba, E.M. and Maposa, R. 1996. An evaluation of the performance of six smallholder irrigation schemes in Zimbabwe. Irrigation and Drainage Systems. Vol. 10, pp. 355-366.

23

Skutsch, J.C. 1998. Maintaining the value of irrigation and drainage projects. TDR Project R 6650. Report no. OD/TN 90. HR Wallingford, UK. [http://www.dfid-kar-water.net/w5outputs/large_scale_irrigation.html] Small, L.E. and Svendsen, M. 1990. A framework for assessing irrigation performance. Irrigation and Drainage Systems. Vol. 4, pp. 283-312.

24

ANNEX I – SAMPLE DATA COLLECTION FORMS The sample data collection templates in this Annex include:

Recommended Data Processing Data Collection Template Basic Preferred

1. Beneficiary Selection Worksheet none 2% sample entered into M&E database

2. Training Log summarized at end of season summarized at end of season 3. Baseline Household Survey entered into M&E database entered into M&E database 4. Garden Diary summarized at end of season summarized at end of season 5. Seasonal Production and Sales

Summary summarized on the end-of-

season HH survey summarized on the end-of-

season HH survey 6. End-of-Season Household Survey entered into M&E database entered into M&E database 7. Irrigation Scheme Seasonal

Summary entered into M&E database entered into M&E database

Adapting the forms These sample data collection templates below have been designed to be as generic as possible, however additional adaptation will be needed to suit the local context. To modify the forms, soft copies in MS Word format can be found on the M&E database CD, see Annex II. Minor design changes that won't affect data entry can be made fairly easily, such as the addition of a logo, signature lines, labels, explanatory text, etc. Adding new fields, columns or sections is also possible, however this could also require modification of the accompanying database if the additional information needs to be captured. If new fields, columns, or questions are needed, it is recommended that you consult with the RIACSO office and other ECUs using the macro irrigation M&E database to see if other users would also like to adopt the new fields. Another type of adaptation needed are the labels for the geographic references. Most countries have administrative areas called “provinces” and “districts”, but in some countries administrative areas are known by other names. These labels can be modified on the paper versions of the data collection forms as needed. The database software has a setup feature to customize the labels and drop-down boxes on data entry forms to ensure they contain the appropriate lists.

We already have a monitoring system for our irrigation scheme, why do we need to start using these new forms?

The short answer is you don't, provided that your current monitoring tools can produce the required numeric summaries for the core outcome indicators outlined in Annex III. The data collection forms in this toolkit are provided as sample templates that can be adopted when nothing comparable is currently available. These templates have been designed to be as generic as possible, but can be adapted at the country level to capture additional information as needed. Another advantage of adopting the sample templates is that the accompanying database software can be used to simplify data entry and analysis.

25

Beneficiary Selection Worksheet

InstructionsThe purpose of this form is to record how people are chosen to be a beneficiary for an irrigation scheme. If there is no selection process because a scheme benefits the entire population of the catchment area, this form can be disregarded. However if irrigated plots, inputs or other services are provided by the scheme, this form applies.

This form can be used by any person or group involved in the selection of beneficiaries, including community groups, extension officers, or project staff. To use the form, put a check or 'x' in each column that was used to select the person. Column headings can be changed as needed. This form can be photocopied or copied into a notebook.

Name of beneficiary m/f Location Reason(s) for selection as a beneficiary

(check all that apply) Explanation or

comments

Elderly headed

HH

Widow headed

HH

Orphan headed

HH

Recent death in

HH

Sick-ness

No one employed

Many children

Poor harvest last yr

No one to send money

No small livestock

In-sufficient

land

No draft

power

Active in a group or

programme

Other (specify)

Selection by: Date:

26

Training Log Instructions. This template is designed for a community trainer or extension officer to record all training activities falling under a macro irrigation scheme. This form can either be photocopied or copied into a notebook. Additional columns can be added as needed. At the end of the season, the information from the training log should be totaled and summarized on the Seasonal Summary sheet. Scheme name or location: District: Training Activities

Date(s) Training Topic(s) Instructor Males Females Time (hrs) Comments

27

Baseline Household Survey Introduction. This survey is designed to learn about how members of irrigation schemes grow crops so we can improve irrigation projects. This is not an exam, and there are no right or wrong answers. Your responses will be used to prepare a report, but neither your, nor any other names, will be mentioned in any report and all the responses will be mixed together so there will be no way to identify that you gave this information. We appreciate your participation in this survey, but participation is completely voluntary and optional. Do you have time to participate? Name: m / f GPS coordinates: S:

participant non-participant E:

Monitoring officer: [District]:

Date of visit: [Village]:

Participation in the irrigation scheme

Do you have a plot in the irrigation scheme? yes no (non-members can skip the rest of this section)

Size of irrigation plot: m2

Plot number:

What is the distance from your house to your plot? hours km (circle one)

Were you of member of the irrigation scheme last season? yes no How did you become a member of the irrigation scheme?

Do you pay fees to be in the irrigation scheme: no yes, amount: per What other responsibilities do you have as a member of the scheme?

Which crops will you plant this season?

What training topics do you feel you need, if any?

Would you be interested in learning how to keep a garden diary where you record things like planting dates, harvests, and sales? yes no

Dryland farming

Do you also have any dryland (non-irrigated) gardens or fields? yes no

If yes, which specific crops and other products do you produce?

Homestead Details

Are you the head of household? yes no

People living in homestead Children (0-17) Adults (18-59) Elderly (60+) for the last three months: M F M F M F Is the head of the homestead: elderly widow orphan1 chronically ill2 none of these

Number of orphans living in homestead (aged 0-17):

Does this household have access to animal draught power for ploughing? yes no

28

Baseline household survey, page 2 of 2 Does anyone in this household generate income other than by selling crops? yes no if yes, how:

Have any of the adult members of the household been sick and unable to work for a total of 3 months over the last 12 months? yes no

Have any of the non-elderly adult members of the household died in the 12 months? yes no

Is anyone in this household currently receiving home-based care, ARVs, or active yes no in an HIV support group?

List all forms of food or cash assistance received in the last six months, including school feeding programmes and home based care:

In the past 12 months, in which months did you harvest from all your gardens combined?

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec (circle all that apply)

Dietary Intake

Over the last seven days, on how many days did you consume the following foods?

maize, maize porridge: fruits: other cereal (e.g. rice, sorghum, millet): beef, goat, or other red meat:

cassava: poultry: potatoes, sweet potatoes: pork: sugar or sugar products: eggs:

beans and peas: fish: groundnuts and cashew nuts: oils/fats/butter:

vegetables/ relish /leaves: milk / yogurt / other dairy bread, pasta: fortified blended food (e.g. CSB, HEPS):

Form reviewed by: Approved by: _____________________________________________

1 an orphan is defined as a child 17 years old or less, with at least one parent deceased 2 definition of chronically ill: ‘sick and unable to work for a total of 3 months over the last 12 months’

29

Garden Diary

Instructions. This template is designed to be used by plot holders of an irrigation scheme to record activities in maintaining a garden, including planting, fertilizer application, spraying, harvesting, and sales. This form can either be photocopied or copied into a notebook. Units should be reported in kg (preferred) or standard units of

volume, such as “5 liter tin”. Additional columns can be added as needed. At the end of each month, the information from the Garden Diary should be copied onto the Monthly Garden Production and Sales summary sheet.

Garden name or location: Owner/Manager:

Date Crop Section1 Action2 Quantity Units Total Price3 Individual4 Intended Use5 Comments6

1 section of the garden, if applicable 2 action should be one of the following: planted, fertilized, sprayed, harvested, sold 3 total price received for crops sold (leave blank if not applicable)

4 name of an individual if the garden is a community garden with individual plots, (leave blank if not applicable)

5 for harvested crops only: sell, consume, barter, unknown, (leave blank if not applicable) 6 indicate area if the entry is a planting

30

Seasonal Production and Sales Summary

Instructions. This form is used to summarize the production and sales from a single plot holder in an irrigation scheme for an entire gardening season. It should be completed at the end of each season by the plot holder and/or extension officer. The information on this summary sheet can come from a Garden Diary or other record keeping system. If the amount harvested or sold is not known, write “unknown” in the ‘total amount’ column. This form can be photocopied or copied into a notebook. Copies should be sent to the office to enter into the M&E database. Plot holder: GPS Coordinates: (if known)

S: E:

Reporting period: From: To:

Crop or Product

Area Planted

Total Amount Harvested

Total Amount Sold

Comments

Prepared by: Name Position Date

31

End-of-Season Household Survey Introduction. This survey is designed to find out how members of the irrigation scheme did this season, and how the scheme can be improved. Your responses will be used to prepare a report, but neither your, nor any other names, will be mentioned in any report and all the responses will be mixed together so there will be no way to identify that you gave this information. We appreciate your participation in this survey, but participation is completely voluntary and optional. Do you have time to participate? Name: m / f GPS coordinates: S:

member non-member E:

Monitoring officer: [District]:

Date of visit: [Village]:

Date of baseline visit (if known): Production - General Which crops did you harvest this season from your plot in the scheme?

Which crops did you harvest from your dryland garden?

In the past 12 months, in which months did you harvest from all your gardens combined?

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec (circle all that apply) Participation in the Macro Irrigation Scheme (non-members can skip this section) The plot in the scheme was:

the only garden in the household the primary garden of multiple gardens a secondary garden Which inputs did you receive from the scheme? (check all that apply)

seed grant / credit tools grant / credit fertilizer grant / credit other grant / credit specify:

Did you receive any training through the irrigation scheme? yes no

If yes, please fill specify the topics covered:

Topic Training provided by Date(s) Comments

Describe any benefits that were made possible by the scheme:

Do you have any recommendations for improving the irrigation scheme?

32

End of season household survey, page 2 of 2 Irrigated Production – Crops Grown (non-members can skip this section) For each of the items you produced in your plot in the scheme, please describe how have used it or plan to use (e.g., consume, sell, gift). Also please describe any problems experienced (e.g., production problems, storage, marketing)

Crop or Item Usage(s)1 Problems2

1 usage(s): 1=consumed, 2=given away, 3=sold, 4=stored, 5=other (specify) multiple answers are OK 2 problems(s): 1=poor germination, 2=weeds, 3=insufficient watering, 4=no fertilizer, 5=too much rain, 6=harvest problems (specify), 7=storage problems (specify), 8=marketing problems (specify), 8=other (specify) multiple answers are OK

Irrigated Production – Amounts Harvested and Income Did you keep a diary of production and sales? yes no (skip this section)

Crop or Product

Area planted

Unit Quantity Harvested

Quantity Consumed1

Quantity Sold1

Income Generated1

Comments

1 if known, else write “unknown”

Dietary Intake

Over the last seven days, on how many days did you consume the following foods?

maize, maize porridge: fruits: other cereal (e.g. rice, sorghum, millet): beef, goat, or other red meat:

cassava: poultry: potatoes, sweet potatoes: pork: sugar or sugar products: eggs:

beans and peas: fish: groundnuts and cashew nuts: oils/fats/butter:

vegetables/ relish /leaves: milk / yogurt / other dairy bread, pasta: fortified blended food (e.g. CSB, HEPS):

Form reviewed by: Approved by:

33

Irrigation Scheme Seasonal Summary

Instructions. This two-page Seasonal Summary should be filled out by the scheme manager or extension officer at the beginning of the project (baseline) and end of each production cycle (e.g., end of April, end of July, end of October). A copy of the Seasonal Summary should be forwarded to the office for entry into the M&E database. Scheme name / location: District:: GPS Coordinates: (if known)

S: E:

Reporting period: From: To: baseline Project type: new scheme rehabilitation

PHYSICAL CHARACTERISTICS

Area of scheme: Total area equipped for irrigation: ha Area irrigated this season: ha If irrigated area was less than the total area, explain the shortfall:

Water: source: distance to irrigated blocks: Reliability of water: was never a problem water level got low water source dried up Comments on water supply:

continue on back if needed Water lifting: gravity fed surface pressurized surface gravity fed overhead pressurized overhead

Water delivery: lined canals pipeline unlined canals bucket other:

Scheme structures: sheds: (number) toilets: other (specify): Describe any problems with physical infrastructure:

continue on back if needed

SCHEME MANAGEMENT Registered plot holders: ____ men ____ women Management responsibility: government farmer joint other: Management committee established: yes no

If yes, composition: men women

Crops grown:

continue on back if needed Number of growing cycles (seasons) this garden was used in the past 12 months:

Markets. List the marketing outlets that members of the scheme used this season to sell their production. Markets can be the name of a local market, the name of an agro-processing company, a government program, etc.:

continue on back if needed

34

Macro Irrigation Scheme Season Summary, page 2 of 2 PROJECT ACTIVITIES Infrastructure Improvements Please summarize any improvements made to the infrastructure of the scheme during the reporting period:

continue on back if needed

Training Activities Conducted During the Reporting Period Training Topic Num Males Num Females Comments

Comments on training activities:

continue on back if needed Summary of Inputs and Implements Distributed

Distribution Input/ Implement

Quantity Start Date End Date

Comments

Comments on input and implement distribution:

continue on back if needed

Other Activities Please list any other activities undertaken during the reporting period: Recommendations Please write any other comments and recommendations below:

continue on back if needed Prepared by: Name Position Date

35

ANNEX II – MACRO IRRIGATION SCHEMES M&E DATABASE The CD below contains a Microsoft Access database that has been designed to accompany the data collection forms and required outputs described above. To use the database, simply copy the “Irrigation Schemes M&E” folder to your hard drive and then open the database file with MS Access version 2000 or later2. See page 20 for more information about the features of the database. The CD also includes soft copies of this toolkit and the data collection forms in Annex I. These templates can be modified to suit local requirements as described above.

TAPE CD ENVELOPE

HERE

2 If you get an error message when trying to open the database, saying that the database is read-only, first make sure you’re opening the copy of the database you put on the hard drive, and not the copy on the CD. If this doesn't solve the problem, close Access, select the database file in Explorer, choose Properties from the File menu, and then uncheck the ‘read only’ checkbox. The database should now be able to open without the error message.

36

ANNEX III – DATA FORMAT FOR SHARING INDICATOR DATA Why standards for data formats are needed One of the primary reasons for harmonizing how activities are monitored is to facilitate aggregation of results across project sites and over time. Without the ability to aggregate M&E results, it is difficult for donors and project managers to determine whether a national or regional programme is a good investment. Merging M&E results is conceptually simple, but requires M&E data to be captured in a uniform manner and formatted in the same way to allow data from multiple sites and years to be merged together. This toolkit and the accompanying M&E database have been designed to streamline this process. The table below shows the required format for the core output and outcome indicators that have been agreed upon for macro irrigation schemes (see page 5). Although most of these indicators are defined as a single number (e.g., number of plot holders), many require some type of breakdown (e.g., by gender and level of vulnerability). Furthermore, because indicator data will be merged across multiple project sites in multiple countries and over multiple time periods, additional columns are needed to indicate where and when the data originated from. Hence indicator data should be reported in the tabular formats outlined below, not as single numbers. The accompanying M&E database is equipped to generate the tabular summaries below in a format than can be electronically sent to other offices and easily merged. Any ECU that chooses to use its own system for data processing must be able to generate the tables below in a common file format, such as Excel, Access, DBF, or CSV. The regional office may be able to provide technical support in this effort as needed. Table Definitions for Output and Outcome Indicators

Table 3. Number of SSI schemes, households, marketing outlets, and cropping intensity Column Label Value

CountryID one of the values from tlkp_Countries ProjectID one of the values from tlkp_Projects (e.g., 1= OSRO/RAF/510/SAF) SchemeLocation name of the scheme location Admin3 the ID number for the 3rd level of administrative areas (in most countries this is the

district), taken from the table tbl_GeoCodes GPS_X the X GPS coordinate for the scheme, if known, in decimal degrees (positive

values for eastern hemisphere) GPS_Y the Y GPS coordinate for the scheme, if known, in decimal degrees (negative

values for southern hemisphere) ReportPeriodFrom date ReportPeriodTo date SeasonID one of the values from tlkp_Seasons, e.g., 2=2005 off season; 3=2005/06 main

season; 4=2006 off-season; 5=2006/07 main season, etc. SchemeSupportTypeID 1=new, 2=rehabilitated WaterAbstractionMethodID one of the values from tlkp_WaterAbstractionMethods (e.g., 1=gravity fed surface,

2= gravity fed overhead, 3=pressurized surface, 4=pressurized overhead) AreaTotal total area of the scheme (ha) AreaIrrigated total irrigated area in the scheme (ha) RegPlotholdersMale number of registered male plot holders RegPlotholdersFemale number of registered female plot holders NumMarketingOutlets number of marketing outlets

37

Notes: This table captures the number of SSI schemes supported, number of households, and marketing outlets. Summing up all the records for one year for each scheme will also allow calculation of the cropping intensity. Data for this table come from the Seasonal Summary Sheet. Table 4. Training indicators

Column Label Value CountryID one of the values from tlkp_Countries SchemeLocation name of the scheme location SeasonID one of the values from tlkp_Seasons, e.g., "2005 off season" = 2; "2005/06 main

season" = 3; "2006 off-season" = 4; "2006/07 main season" = 5, etc. ProjectID one of the values from tlkp_Projects (e.g., 1= OSRO/RAF/510/SAF) TrainingTopicID one of the values from tlkp_TrainingTopics, e.g., 1=food storage, 2=nutrition

requirements, 3=maintenance of pumps, etc. NumMaleParticipants number of male participants trained NumFemaleParticipants number of female participants trained

Note: this table will give an accurate representation of the number of scheme members trained per topic, but it will not be possible to add these together to get an accurate total of the number of participants trained in all topics, because duplication has not been eliminated (people who attend more than one training). It is therefore important to not aggregate the number of people trained when reporting. Table 5. Inputs

Column Label Value CountryID one of the values from tlkp_Countries SchemeLocation name of the scheme location SeasonID one of the values from tlkp_Seasons, e.g., "2005 off season" = 2; "2005/06

main season" = 3; "2006 off-season" = 4; "2006/07 main season" = 5, etc. ProjectID one of the values from tlkp_Projects (e.g., 1= OSRO/RAF/510/SAF) InputTypeID one of the values from tlkp_InputTypes, e.g., seed =1; fertilizer=2; pesticide=3;

herbicide=4; livestock drug=5; tools=6 CropID one of the values from tlkp_Crops, e.g., maize=1; sorghum=2; beans=3;

groundnuts=4; n/a=0 QuantityDistributed quantity distributed QuantityDistributedUnitID a value from tlkp_MeasUnits (kgs, pieces, litres, etc.)

Table 6. Vulnerability profile, quality of dietary intake, and number of months of production

Column Label Value CountryID one of the values from tlkp_Countries ProjectID one of the values from tlkp_Projects (e.g., 1= OSRO/RAF/510/SAF) SeasonID one of the values from tlkp_Seasons, e.g., "2005 off season" = 2;

"2005/06 main season" = 3; "2006 off-season" = 4; "2006/07 main season" = 5, etc.

SchemeLocation name of the scheme location HouseholdID randomly generated number for linking tables together VulnerabilityLevelID one of the values from tlkp_VulnerabilityLevels, 1 = low, 2 = moderate, 3 =

high ImpactedByAIDS 1 = yes, 2 = no FCS_Baseline food consumption score at the baseline FCS_PostSeason food consumption score at the end of the project NumMonthsProduction_Baseline number of months of production per year prior to the project NumMonthsProduction_PostSeason number of months of production per year at the end of the project (leave

blank if not captured)

38

Notes: that data in this table come from the baseline and end-of-season household surveys. Each record in the table represents one of the sampled households. This data will give a profile of the vulnerability of participants and non-participants, the quality of their dietary intake, and number of months of production. Table 7. Production and income

Column Label Value CountryID one of the values from tlkp_Countries ProjectSite name of the project site, or “national” if summary is for all project sites in the

country SeasonID one of the values from tlkp_Seasons, e.g., 2=2005 off season; 3=2005/06 main

season; 4=2006 off-season; 5=2006/07 main season, etc. ProjectID one of the values from tlkp_Projects (e.g., 1= OSRO/RAF/510/SAF) HouseholdID randomly generated long integer CropID one of the values from tlkp_Crops AreaPlanted area planted (ha) Harvested_Quant amount harvested Harvested_UnitID one of the values from tlkp_MeasUnits (usually kg) IncomeGenerated_Value the total income generated through sales IncomeGenerated_MonUnitID one of the values from tlkp_MonetaryUnits

Notes: this table is derived from the end of season household survey. Each record in the table represents one crop from one of the plot holders. The household ID allows cross variable analysis. Other Summaries The five tables described above capture the core indicators that have been defined for irrigation schemes, however, by no means exhaust the types of analyses that can be generated from the M&E data. There are many more summaries that can be generated at the site or national level to help derive lessons learned and make improvements in the design and implementation of projects. Some of the additional analyses include summaries of:

• selection criteria used for selecting beneficiaries • correlations between changes in food consumption score and yields • an comparison of yields broken down by vulnerability status • crop preferences • other food security assistance received by households

Many of the above summaries can be broken down by gender, vulnerability level, family size, geographic area, etc. Additional summaries such as the ones above will be available in the M&E database and can be used when preparing reports and analyses, however only the five summaries of output and outcome indicators need to be forwarded to the RIACSO office. The M&E database is a work in progress and although it will have an initial selection of summaries available, user input will be needed to develop additional ones.

39

Creating new summaries for the M&E database

Summaries like the ones described above are generated by designing a query object in MS Access. A query in Access can be thought of as a “window” on your data. Queries can be saved and each time you open the query, it summarizes the latest data. Emergency Coordinators and their staff are in the best position to know what type of summaries or analyses would be most useful for analyzing an irrigation scheme, and so are encouraged to think about how the M&E data can be summarized to answer different questions. New queries can be added to the M&E database by skilled users, or by sending a request to the RIACSO office. Alternately, the raw data can be exported to Excel or a statistics program and analyzed there. MS Access is a very flexible program and EC’s and their project staff are encouraged to gain some instruction on using MS Access for data analysis.

Summaries for Making Custom Maps In addition to numerical summaries, the M&E database for macro irrigation schemes can be used as a source of data for making maps with GIS software. For example, maps could show the location of each scheme, where the color or size of the symbols represents a variable such as the number of members, cropping intensity, number of training hours, etc. To make such maps, the first step is to construct a query in Access that includes either the coordinates of each point or the name of a place that corresponds to a GIS layer. The second step is to link the GIS software with the query. Most common GIS programs, including Dynamic Atlas, ArcView, and ArcGIS, have the ability to link to tabular data stored in Access.

40

ANNEX IV – SUPPLEMENTAL DATA CAPTURE FORMS The following monitoring forms, developed by the Zimbabwe ECU, supplement the data entry forms found in Annex I. These supplemental forms capture more detail on the performance of an irrigation scheme, in particular the physical characteristics, institutional structures, and household-level variables. These supplemental forms go beyond basic M&E requirements for demonstrating impact, and are designed to guide a comprehensive diagnostic and planning process. They are provided here as examples of what can be developed when greater information needs and resources are present.

41

Baseline Status of the Scheme before Non-Technical Rehabilitation Name of Irrigation Scheme: ……………………………………………………….. Date …………………………………………………. Respondent …………………………………………. ____________________________________________________________________

1. Is IMC in place? 1=Yes 2=No 2. Composition of IMC : Men ……………… Women ……………….. 3. Period of tenure ………………………………………………………………

4. How often does IMC meet ? ……………………………………………………

5. When did they last meet ? ……………………………………………………

6. How often is a general meeting held?.......................................................................

7. What are the functions of the IMC?

8. Are scheme bi-laws in place? 1=Yes 2=No.

9. Who enforces the bi-laws?

10. How are they enforced? 11. What are the sources of funds for running of scheme?

12. If irrigators pay subscriptions state how much (this year)?

13. Where is the money from subscriptions kept?

14. How is Operation, Repair and Maintenance of Scheme Infrastructure done

(mechanisms that are in place)?

15. What is your scheme cropping programme? State it. If no programme, why?

16. How often is the cropping programme reviewed? How is it decided?

17. What crops are in the scheme right now?

18. How is crop rotation done by farmers?

19. Do farmers experience any serious problems of pests and diseases? 1=Yes 2=No

20. If yes, state crop, pest/disease and how they are solved by farmers?

21. How do farmers use crop budgets?

22. What proportion of farmers own dry land plots?

42

23. What is the average landholding for dry land plots? ………………..ha

24. To what extent does dry land farming activities affect irrigation?

25. To what extent do farmers utilize their irrigation plot?

26. What is the current cropping intensity?

27. Where do farmers get their inputs for irrigation?

28. How are inputs acquired? 1=Individually 2=as a group?

29. Where do farmers sell their produce?

30. How are produce sold? 1=Individually 2=as a group?

31. Have farmers ever grown crops under contract? 1=Yes 2=No

32. If yes state contractor, crop grown under contract and when this was done? Crop Contractor When …………………………………………………………………………………………

33. What is your experience with contract growing?

34. What marketing problems for your irrigation produce do you experience?

35. What problems do you have with your water source? Reliability, adequacy?

36. Since inception of the scheme what courses have farmers undergone? (state course ,

when it was attended, who trained?

Course when attended Who trained 37. What are your major challenges in operating and maintaining the irrigation scheme? 38. How can we jointly address them?

43

SMALLHOLDER IRRIGATION REHABILITATION PROJECT

Baseline Survey Scheme Level Questionnaire (To be administered to Key Irrigation Informants)

DATE________________________________________ A SCHEME IDENTITY

1. Name of Irrigation Scheme ……………………………………………..…………

2. Name of Enumerator ………………………………………………………………

3. District : ……………………………………………………………………………

4. Ward ……………………………………………………………………………..

5. Province : ………………………………………………………………………….

6. Grid Reference ……………………………………………………………………

7. Catchment Area ……………………………………………………………………

8. Sub-Catchment Area ………………………………………………………………

9. Farming Sector : 1 = Communal 2 = Resettlement

10. Agro-ecological Zone 1 = NR 1 2 = NR IIa 3 = NR IIb 4 = NR III 5 = NR IV 6 = NR V

B. SCHEME CHARACTERISTICS 11. Total Area of Scheme : ……………………………………………………………….. 12. Area Under Irrigation : ……………………………………………………………….. 15. Registered Plotholders

Total : …………… Males : …………… => Married ……………………… => Divorcees…………………….. => Single ………….…..………… => Widowers ………..…………..

44

Females:………….. => Widows …………………… => Divorcees…………………... => Single ……………………… => With husbands away ……… 16. Scheme has been 1 = Gazetted 2 = Not gazetted 17. Management Responsibility 1 = Government 2 = Farmer 3 = Joint 4 = Other ….. 18. Full time or Part time : 1 = Full time

2 = Part time C. DEVELOPMENT HISTORY OF SCHEME 19. Year (final) of scheme (i) development …….…………………………….

(ii) extension ...…………………………………….

(iii) rehabilitation …..……………………………. 20. First year of operation of scheme ………………………………………………… 21. Who initiated the Project …………………………………………………………. 22. Source of development funds ….…..……………………………………………… 23. Who constructed the project ? (gvt, farmers, contractor) …..…………………….. 24. Construction Period ………………………………………………………………. D. WATER SOURCE 25. Source of Irrigation Water : ………………………………………………………. 27. Type of abstraction Method e.g Pump, Gravity fed,.. : ………………………….. 28. Who owns the abstraction facilities ? : ……………………………………………. 30. Amount of committed water …………………………………………………….. 31. Does scheme has Water Right/Water Permit 1 = Yes 2 = No 32. If Yes (i) Size of Water Right/Permit ………………………………………

(ii) Water Right/Permit Number ……………………………………..

45

33. How far is the Water Source from the Irrigated Blocks ….……………………... E. IRRIGATION SYSTEMS 34. Irrigation method Used : …………………………………………………………….. 35. Method of Water delivery to the Irrigated Blocks (Circle option): 1 = Pipeline 2 = Canal [ 1 = Lined 2 = Unlined ] 3 = Bucket 4 = Other specify __________________________ 36. Distribution Efficiency : 1 = Poor

2 = Fair 3 = Good

Irrigation system Technical Performance: What is the irrigation cycle? _______ days. What is the irrigation set time? ______ hrs/day Is the irrigation water supply normally good, i.e enough water and regular supply? During the past three seasons? Yes ___ No ___ If the supply is not good then what, in the farmer’ opinion is the cause? Not enough water from the source Yes ___ No ___ Poor infield distribution uniformity of irrigation water Yes ___ No ___ Bad control/operation of the main irrigation supply / distribution system. Yes ___ No ___ Bad condition of main irrigation supply system Yes ___ No ___ Bad condition of the infield irrigation system Yes ___ No ___ Breakdown of irrigation equipment e.g pumping units etc. Yes ___ No ___

Others (specify) ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

46

F. CONDITION OF INFRASTRUCTURE . INFRASTRUCTURE CONDITION

1 = Good 2 = Fair 3 = Working but needs repair 4 = Out of order, needs replacement

Dam Reservoir Overnight Storage Dam Wall Main Gates Pump House Pumps Main Canal Roads Drains Field Canals Field Structures Pipes Valves Hoses/Laterals Sprinklers Hydrants Other (specify) G. OPERATION AND MAINTENANCE 38. Who is responsible for operation and maintenance of the following : i) Dam (Water Source) ………………………………..…

ii) Headworks …………………………………….…….

iii) Conveyance …………………………………………..

iv) Infield Works ………………………………. ………....

v) Drainage …………………………………………..

vi) Contribution of plotholders :

Average monthly maintenance fees - $ ……….. Average monthly energy costs - $ .............. Average monthly water charges - $ ................ Average monthly Labor costs - $…………. Other (specify) …………….. Are the bills manageable for the farmers? Yes ............. No ...................

47

If NO, what are the reasons? ................................................................................................................................................................................................................................................................................................ 39. Who is responsible for repair and maintenance of the following : i) Dam (Water Source) …………………………………

vii) Headworks …………………………………………

viii) Conveyance …………………………………………

ix) Infield Works ……………………………… ………..

v) Drainage …………………………………………

vi) Contribution of plotholders : vii) Average monthly maintenance fees - $ ………..

Average monthly energy costs - $ .............. Average monthly water charges - $ ................ Average monthly Labor costs - $…………. Other (specify) …………….. Are the bills manageable for the farmers? Yes ............. No ................... If NO, what are the reasons? ................................................................................................................................................................................................................................................................................................ Do farmers have scheme repair and maintenance fund/bank account? Yes ....... No ......... Do farmers contribute money for to repair and maintenance at the same time? Yes ........ No .......... How often do farmers contribute to the fund? Monthly ........... Seasonally ............ Annually ............ Are the contributions enough for the repairs and maintenance? Yes ....... No ........ If No, give reasons: ........................................................................................................................................................................................................................................................................

48

H. Social and Gender 40 Irrigation scheme membership by Gender : Male .............. Female .............. 41 How many men, women and youths are in the Irrigation Management Committee (IMC) Men .......... Women ........... Youths ............ 42 What roles/responsibilities do IMC members have by gender? Chairman F/M V/Chairman F/M Secretary F/M Treasurer F/M 43 What is the current ratio of male to female headed households in the scheme? Number of male headed HH ........... Number of female headed HH ........... Ratio .............. 44 is there a Main Irrigation Management Committee (IMC)? Yes ........ No ......... 45 Is there an Irrigation scheme Constitution? Yes ......... No .......... 46 List major scheme problems and conflicts: ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ I. HEALTH AND ENVIRONMENTAL ISSUES 44. Do you have public toilets in the scheme 1 = Yes 2 = No 45. If Yes How many toilets are there ? ……………………………….. 46. What type of toilets do you have ? 1 = Separate compartment for men and women 2 = Single compartment for both men and women 47. Where do irrigators get their drinking water whilst in the scheme ? ………………………………………………………………………………………

49

48. Major human diseases Type of Disease

What causes the disease

Where do people get the treatment

Periods when diseases suffered

49. Do you have any salinity problems ? 1 = Yes 2 = No 50. If Yes what hectarage is affected …………………………………… 51. Do you have any erosion problems ? 1 = Yes 2 = No 52. If Yes what hectarage is affected …………………………………… 53. Do you have any drainage problems ? 1 = Yes 2 = No 54. If Yes what hectarage is affected ……………………………………

Farmer And Extension Staff Training Name of course No. of farmers trained Course duration How effective were the course? poor _____ Good _______ very good ______ If poor, give reasons.

50

Baseline Survey Individual Farmer Questionnaire 1. Date…………………………………………………………………………………… 2. Name of Irrigation Scheme …………………………………………………………… 3. Province ………………………………………………………………………………… 4. District …………………………………………………………………………………. 5. Name of Respondent……………………………………………………………………. 6. Name of Enumerator ……………………………………………………………………. 7. Sex of Respondent 1 = Male 2 = Female 8. Name of Registered Plotholder …………………………………………………………. 9. Sex of registered plotholder 1 = Male 2 = Female 10. Plot Number : ………………………………………………………………………... 11. Relationship (What is Respondent to Registered Plotholder) ……………………….. 12. Marital Status of Plotholder 1 = Married 2 = Single 3 = Divorced 4 = Separated 5 = Widowed 13. Age of Respondent …………………………………………………………………… 14. Age of plotholder …………………………………………………………………….. 15. Education : Number of years in school ……………………………………………… 16. Professional Courses Done ……………………………………………………………. …………………………………………..…………………………………………………. 17. Size of Irrigation Plot ………………………………………………………………… 18. Area currently being irrigated ………………………………………………………… 19. Do you have dryland plots 1 = Yes 2 = No 20. If Yes State total size of dryland plot ……………………………………………..ha 21. CROP PRODUCTION DATA

51

Crop name

Variety

Month of planting

Area planted (ha)

Amount of seed used (kg)

Basal or initial fertilizer

(Type)

Quantity (kg)

Cost

Top dressing fertilizer

(Type)

Quantity (kg)

Cost

Pesticides

Type 1

Type II

Type III

Type IV

Cost of transport of

inputs to farm

Land preparation

Weeding

Harvesting

Other labour costs

Total produce

52

Produce consumed (kg)

Produce retained (kg)

Produce sold (kg)

Market 1

Market 2

Market 3

Average price

Cost of transport to

market

Cost of transport of

farmer (fares)

Other market costs

22. Do you have a cropping programme ? 1 = Yes 2 = No 23. If Yes What is it ? ……………………………………………………………………………………………… ……………………………………………………………………………………………… ..…………………………………………………………………………………………… 24. Source of cropping programme 1 = Farmer 2 = Agritex 3 = IMC 4 = Joint (specify) ……………………………… 25. Are you satisfied with the Cropping Programme ? 1 = Yes 2 = No 26. If Yes/No please explain why ? ……………………………………………………………………………………………. …………………………………………………………………………………………….. 27. When was it produced ? ……………………………………………………………..

53

28. Do you rotate your crops ? 1 = Yes 2 = No 29. If Yes How do you rotate your crops ……………………….…………………… ………………………………………………………………………………………… ………………………………………………………………………………………… 30. What was your source of draft this year ? ……………………………………………. 1 = Own draft animals 2 = Own tractor 3 = Hired draft animals 4 = Hired tractor 5 = Borrowed draft 6 = Sharing 31. Do you have any crop budget for the crops that your grow ? 1 = Yes 2 = No 32. If Yes Please state the crop budgets that you have ? ……………………………………………………………………………………………… ……………………………………………………………………………………………… ..…………………………………………………………………………………………… 33. Who prepared the budgets ? …………………………………………………………… 34. Do you use the budgets ? 1 = Yes 2 = No 35. If Yes Please explain how you use them ……………………………………………………………………………………………… ……………………………………………………………………………………………… ..…………………………………………………………………………………………… 36. Do you have any labor shortage 1 = Yes 2 = No 37. If Yes, State operations for which you face labor shortages ……………………………………………………………………………………… ……………………………………………………………………………………… 38. Do you experience any problems in irrigation crop pests ? 1 = Yes 2 = No

54

39. If Yes Please state types of pests/diseases, the crops in which you experience the pest problems and how you control them

Crop Type of Pest/Disease Method of Control Used

40. Do you rotate the use of pesticides ? 1 = Yes 2 = No MARKET SYSTEMS 41. Source of inputs (seed, fertilizer, chemicals)

Inputs (Please state type)

Source (e.g. Name of Town, Growth Point etc.)

Approximate Distance to Input source

42. How do you buy inputs 1 = Individually

2 = In groups 3 = Other (specify)…………………………………

43. Do you get your inputs in time ? 1 = Yes 2 = No 44. If No explain why ? ……………………………………………………………………………………… ………………………………………………………………………………………

55

45. Please fill in the relevant marketing information Crop Market where

sold3 Quantity sold

Distance to market

To whom do you sell at the market4

Reason for using that market

46. How do you sell your produce at the market

1 = Cash 2 = Barter 3 = Contract 4 = Other

47. State the alternative markets where you can sell the produce ? ……………………………………………………………………………………… ……………………………………………………………………………………… 48. How do you get marketing information ? ……………………………………………………………………………………… ……………………………………………………………………………………… 49. Is the information useful to you 1 = Yes 2 = No 50. If Yes, how do you use marketing information ? ……………………………………………………………………………………… ………………………………………………………………………………………

3 Check Markets with those in Crop Production Table 4 1 = Individuals 2 = Wholesalers 3 = Schools 4 = Supermarkets etc

56

51 . If No, Why ……………………………………………………………………………………… ……………………………………………………………………………………… POST HARVEST/ HANDLING FACILITIES 52. Crop Storage facility Handling facility Processing facility 53. Do you have any storage related problems 1 = Yes 2 = No 54. If Yes Please state them ? ……………………………………………………………………………………… ……………………………………………………………………………………… 55. Do you have any harvesting problems ? 1 = Yes 2 = No 56. If Yes state the harvesting problems ……………………………………………………………………………………… ……………………………………………………………………………………… SOURCES OF FUNDS 57. Please state current and past sources of funds to finance irrigation production in the table below

Source Amount Received

Year Received

Use Year Repaid All if Credit

Terms

57

58. If you changed sources please explain ? ……………………………………………………………………………………… ……………………………………………………………………………………… 59. Any credit constraints ? (state them below) ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… 60. Rank your income sources for the last 12 months in terms of dollars Income Source Annual Income ($) ……………………………………………………………………………………… ……………………………………………………………………………………… ……………………………………………………………………………………… ……………………………………………………………………………………… ……………………………………………………………………………………… LEVIES 61. Do you pay any irrigation charges ? 1 = Yes 2 = No 62. If Yes please state the type of charges, amounts that you pay and rate (frequency)

Type of charges Amount Paid Frequency (How often ? )

RECORD KEEPING 63. Do you keep any records of what you do in the scheme ? 1 = Yes 2 = No 64. If Yes please state the type of records kept ……………………………………………………………………………………………..

58

65. What are the records used for ? ……………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… HOUSEHOLD ASSETS 66. State the types of livestock owned and their numbers

Number Type of livestock Irrigation Acquired

Other Total

Cattle Goats Donkeys Sheep Chickens Ducks Turkeys Rabbits Pigeons 67. State the types of farm implements owned and their numbers

Number Type of implement Irrigation Acquired

Other Total

Plough Scotchcart Harrow Cultivator Planter Wheelbarrow Hoes Ridger 68. State the types of household goods owned and their numbers

Number Type of household good Irrigation

Acquired Other Total

Car Bicycle TV Radio Sewing Machine Welding machine

59

69. Type of main house ? (i) Floor type 1 = Cement 2 = Dagga (mud) (ii) Wall type 1 = Brick 2 = Mud 3 = Poles 4 = Thatch 5 = (iii) Roof type 1 = Thatch 2 = Asbestos 3 = Zinc 4 = Tiles 5 = 70. Do you have; (i) Electricity 1 = Yes 2 = No (ii) Solar power 1 = Yes 2 = No (iii) Telephone 1 = Yes 2 = No 71. Source of drinking water ……………………………………………………….. 72. Distance of source of drinking water …………………………………………….. 73. Who fetches water ……………………………………………………………….