Food and Nutrition Technical Assistance III Project (FANTA) FHI 360 1825 Connecticut Ave., NW Washington, DC 20009 Tel: 202-884-8000 Fax: 202-884-8432

Food and Nutrition Technical Assistance III Project (FANTA)FHI 360 1825 Connecticut Ave., NW Washington, DC 20009Tel: 202-884-8000 Fax: 202-884-8432 Email: [email protected] Website: www.fantaproject.org

FFP Annual Monitoring Indicators

Food for Peace Monitoring and Evaluation Workshop for

FFP Development Food Assistance Projects

Session Objectives

Understand:1. The importance of annual monitoring2. The different categories of annual monitoring

indicators3. Which indicators are applicable to your

program4. How data can be collected for the indicators


PART 1 OF 2

Introduction to the FFP Annual Monitoring Indicators


Updated Food for Peace (FFP) indicator list

More standardized and quantifiable

More aligned with Feed the Future (FTF) and State Department (F) indicators

New FFP program-specific indicators introduced

Indicator Revision Process


CATEGORY

Agriculture and LivelihoodsMaternal and Child Health and Nutrition (MCHN)

Resilience

Gender

TOTAL

Indicator Categories


TOTAL NUMBER

Output 16Outcome 16

TOTAL 32

Output Versus Outcome Indicators


TYPE TOTAL NUMBER

Required (R) 2Required if Applicable (RiA) 30

Total of 32annual monitoring indicators

Indicator Types


“Required for all development programs”

Required (R) Indicators


1. Number of vulnerable households benefiting directly from USG assistance

2. Proportion of female participants in USG-assisted programs designed to increase access to productive economic resources (assets, credit, income, or employment)

Required (R) Indicators


“Required if applicable for all development programs that have relevant interventions”

Required if Applicable (RiA) Indicators


• Read applicability criteria• Ask yourself: Should we collect this

indicator?

RiA: When Are They Applicable?


Are you working on value chains?

Example: Gross margin per hectare, animal or cage of selected product

Example of an RiA Indicator


Example: Value of agricultural and rural loans

Are you increasing access to credit?

RiA: Another Example of Applicability


1. Revised Food for Peace (FFP) Indicator Handbook (forthcoming early 2015)

2. Feed the Future (FTF) Indicator Handbook: Definition Sheets3. FFP Annual Indicator Performance Indicator Reference

Sheets (PIRS)4. State Department (F) Performance Indicator Reference

Sheets (PIRS)5. Feed the Future (FTF) Agricultural Indicators Guide*6. FFP and FTF Beneficiary-Based Survey Guide*

(forthcoming Q4 2014)

* Covers only four “challenging” agricultural indicators

Where to Find Guidance


Data Collection for the FFP Annual Monitoring Indicators

PART 2 OF 2


Data collection for annual monitoring indicators is the

responsibility of the Awardees

Who’s Responsible?


1. Gross margin per hectare, animal or cage of selected product2. Value of incremental sales (collected at farm level) attributed to

USG implementation3. Number of hectares under improved technologies or

management practices as a result of USG assistance4. Number of farmers and others who have applied improved

technologies or management practices as a result of USG assistance

All four indicators are RiA

All four indicators are for agriculture program components

Four Challenging Agriculture Indicators


Complex

Changes in beneficiary base

Measurement challenges

Why Challenging?

Five components of gross margin

Incremental sales compares two years with different number of beneficiaries in each year

Measuring hectares through farmer recall or farmer estimation is unreliable


How to collect data for the four challenging FFP

indicators (if applicable to program)

How to collect data for the remaining FFP

indicators

DECISION #1

DECISION #2

Decision Process


How to collect data for the four challenging FFP annual monitoring

indicators

Decision #1


The four challenging indicators require special consideration during

data collection

Four Challenging FFP Indicators: Impact on Data Collection


1. Routine monitoring2. Beneficiary-based survey

How to collect data for the four challenging annual monitoring indicators:

Four Challenging FFP Indicators: Methods of Data Collection


• Continual process• Part of routine

program activities (often the case for FFP projects)

• Managed by internal program staff

• Infrequent activity• “Stand-alone” activity• External technical

expertise needed (usually)

Routine monitoring Beneficiary-based survey

Four Challenging FFP Indicators: Two Data Collection Options


Time Resources Data quality

Difficulty to implement

Routine monitoring

Beneficiary-based survey

Four Challenging FFP Indicators: Advantages and Disadvantages


Use routine monitoring (whenever possible)!

Four Challenging FFP Indicators: Recommendation


1. Appropriate scenario

2. Sufficient resources

3. Sufficient time

Four Challenging FFP Indicators: When to Consider Beneficiary-Based Surveys?


Large program size

Farmer recall/estimation is considered unreliable; direct measurement is preferred

Lack of direct contact between program and beneficiaries

Four Challenging FFP Indicators: Appropriate Scenarios for Beneficiary-Based Surveys

Difficult to achieve ideal of visiting all beneficiaries with routine monitoring

Example: measuring hectares

Example: contact with outside agricultural organizations under a market facilitation approach


Please fill in all cells in yellow. Please fill in all cells in yellow.

SURVEY INFORMATION SURVEY INFORMATIONSample size 255 Sample size 255 Days of data collection 20 Data collectors 10 Interviews per day (rounded up) 13 Interviews per data collector per day 2

Interviews per day 20 DATA COLLECTORS

Interviews per data collector per day 2 Days of data collection (rounded up) 13 Data collectors 7

FIELD TEAMSData collectors per team 3 Teams (rounded to nearest integer) 2 Supervisors per team 1 Field data entry per team 1 Drivers per team 1 Other staff per team -

FIELD STAFFData collectors 7 Supervisors 2 Field data entry 2 Drivers 2 Other staff -

TOTAL FIELD STAFF 13

HEADQUARTERS STAFFSurvey manager 1 Data entry 2 Other staff -

TOTAL HEADQUARTERS STAFF 3

TOTAL STAFF 16

SOLVE FOR NUMBER OF STAFF REQUIRED SOLVE FOR NUMBER OF DAYS REQUIRED

Four Challenging FFP Indicators: Sufficient Resources for Beneficiary-Based Surveys

Task #

Task description Estimated number of

days 1 2 3 4 5 6 7 8 9

1

Develop and finalize sampling plan (sample size calculation, development of sampling frame, survey design)

10 days

2 Develop and translate draft survey instrument(s) (in relevant local languages)

10 days

3

Develop, translate and finalize survey training agenda and materials (in English and in relevant local languages)

Concurrent with data collection

instruments

4 Recruit data collectors 30 days

5 Train data collectors, and pre-test and finalize survey instrument(s)

10 days

6 Data collection 15-20 days

7 Clean and analyze collected survey data

10 days

8 Prepare and present final results

5 days

Four Challenging FFP Indicators: Sufficient Time for Beneficiary-Based Surveys

Within the option of “beneficiary-based survey,” there are several different possible approaches.

Four Challenging FFP Indicators: There Is No “One” Survey Type


Four Challenging FFP Indicators: Different Survey Approaches

Feed the Future (FTF)Agricultural Indicators Guide

(September 2013)

Food for Peace (FFP)/Feed the Future (FTF)

Beneficiary-Based Survey Guide(forthcoming Q4 2014)

Definitions What data to collect

How to collect data How to analyze data

Four Challenging FFP Indicators: Resources to Help Decide


How to collect data for the four challenging

FFP indicators (if applicable)

How to collect data for the remainingFFP indicators

DECISION #1

DECISION #2

Decision Process


How to collect data for theremaining FFP annual monitoring

indicators

Decision #2


1. Indicators related to entities: private enterprises, stakeholders, loans, MSMEs, and communities/villages

2. Indicators related to infrastructure3. Indicators related to MCHN

Remaining FFP Indicators: How to Categorize Remaining Indicators


Indicators related to entities (private enterprises, stakeholders, loans, MSMEs, and communities/villages):• Several indicators (e.g., number of food security

private enterprises (for profit), producers organizations, water users associations, women's groups, trade and business associations, and community-based organizations receiving USG assistance)

Remaining FFP Indicators: Indicators Related to Entities


• How this is done will vary by program• Will usually need a mechanism that is

separate from the program monitoring system

Can collect data through direct contact with the entities

or Can collect data via site observation (for a

few of the indicators only)

Remaining FFP Indicators: Indicators Related to Entities


Three indicators:1. Total increase in installed storage capacity (m3)2. Kilometers of roads improved or constructed3. Number of market infrastructures rehabilitated

and/or constructed

Can collect data through program records

Remaining FFP Indicators: Indicators Related to Infrastructure


IndicatorsCan collect data through direct contact

using community health volunteers or outreach workers via routine monitoring

ORCan collect data through direct contact via

beneficiary-based surveys

Note: Health Clinics do not tend to distinguish program beneficiaries from non-beneficiaries, so not recommended to collect data from HCs

Remaining FFP Indicators: Indicators Related to MCHN


In most cases, you will need a combination of data collection activities, occurring at

different times.

All Indicators: Overall Recommendations


• No simple answer that works across all programs

• Spend time during program design to figure out what is best for your program



If considering a survey, remember...• Surveys are resource-intensive and require

specialized technical expertise!• Make sure sufficient money in budget before

deciding to use a beneficiary-based survey… and expect to use external contractors

• Best to use routine monitoring whenever possible instead



• Food for Peace (FFP) Indicator Handbook (forthcoming early 2015)

• Feed the Future (FTF) Indicator Handbook: Definition Sheetshttp://feedthefuture.gov/sites/default/files/resource/files/ftf_handbook_indicators_sept2013_2_0.pdf

• FFP Annual Indicator Performance Indicator Reference Sheets (PIRS)http://www.usaid.gov/sites/default/files/documents/1866/PIRS%20for%20FFP%20Indicators.pdf

• State Department (F) Performance Indicator Reference Sheets (PIRS)http://www.state.gov/documents/organization/101764.pdfhttp://www.state.gov/documents/organization/101765.pdf

• Feed the Future (FTF) Agricultural Indicators Guidehttp://www.fsnnetwork.org/sites/default/files/ftf_agriculture_guide_0.pdf

• FFP and FTF Beneficiary-Based Survey Guide (forthcoming Q4 2014)

References


http://feedthefuture.gov/sites/default/files/resource/files/ftf_handbook_indicators_sept2013_2_0.pdf




http://www.usaid.gov/sites/default/files/documents/1866/PIRS%20for%20FFP%20Indicators.pdf




http://www.state.gov/documents/organization/101764.pdf






http://www.fsnnetwork.org/sites/default/files/ftf_agriculture_guide_0.pdf



This presentation is made possible by the generous support of the American people through the support of the Office of Health, Infectious Diseases and Nutrition, Bureau for Global Health, United States Agency for International Development (USAID); and the Office of Food for Peace, Bureau for Democracy, Conflict and Humanitarian Assistance, United States Agency for International Development (USAID), under terms of Cooperative Agreement No. AID-OAA-A-12-00005, through the Food and Nutrition Technical Assistance III Project (FANTA), managed by FHI 360. The contents are the responsibility of FHI 360 and do not necessarily reflect the views of USAID or the United States Government.


Documents

Food and Nutrition Technical Assistance III Project (FANTA) FHI 360 1825 Connecticut Ave., NW Washington, DC 20009 Tel: 202-884-8000 Fax: 202-884-8432