34
Food Security Information for Action Food Security Policy Formulation and Implementation Lesson 3 Policy Monitoring and Evaluation Learners’ Notes This course is funded by the European Union and implemented by the Food and Agriculture Organization of the United Nations.

Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Food Security Information for Action

Food Security Policy Formulation and Implementation

Lesson 3

Policy Monitoring and Evaluation

Learners’ Notes

© FAO, 2009

This course is funded by the European Union and implemented by the Food and Agriculture Organization of the

United Nations.

Page 2: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course - Food Security Policies - Formulation and ImplementationLesson 3 - Policy Monitoring and Evaluation

Table of Contents

Learning objectives......................................................................................................2

Introduction..................................................................................................................3

M & E in the cycle of policy formulation and implementation.......................................4

Purpose of FSP Monitoring and Evaluation.................................................................5

Monitoring and Evaluation concepts............................................................................6

Approaches and Methods............................................................................................7

Evaluation criteria......................................................................................................12

Setting up an FSP Monitoring and Evaluation system...............................................13

Linking M & E and FS Information to other Policy Spheres.......................................15

Summary................................................................................................................... 16

If you want to know more...........................................................................................17

Annex 1: Case study - M & E concepts applied to a food security policy and related projects and programmes..........................................................................................18

Annex 2: Example of a basic LogFrame Matrix for a Food Security Policy...............20

Annex 3: Common Food Security Indicators and possible Data Sources..................22

Annex 4: Options and criteria for setting up a monitoring and evaluation unit at central policy level...................................................................................................23

Learners’ Notes 1

Page 3: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Learning objectives

At the end of this lesson you will be able to:

• define the purpose of a monitoring and evaluation system for Food Security Policies; and

• identify the approaches and methods to be applied in monitoring and evaluating the implementation and impacts of Food Security Policies.

Learners’ Notes 2

Page 4: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Introduction

Monitoring and Evaluation (M & E) is an essential element of a policy cycle, i.e. of the process of policy formulation and implementation.

M & E helps to ensure that a policy is effective in achieving its objectives, by reviewing policy implementation and finding out whether and to what extent a policy is implemented as planned and working towards its planned objectives to improve the food security situation.

This lesson presents the concepts and approaches applied in establishing and operating an M & E system for Food Security Policies (FSP).

The food security situation is not only influenced by the implementation of specific food security policies but also by:

other policies (e.g. macro-economic and sector policies); and changing macro-economic,political and social conditions (e.g. currency exchange

rates, world market prices of export and food commodities,civil unrest, war).

Therefore, those factors with implications for food security need to be taken into account in policy M & E.

The M & E results are communicated to the policy makers and other stakeholders, so as to trigger, if required, necessary adjustments in policy design or implementation.

Difference between monitoring and evaluation

Monitoring refers to a continuous observation of the process of policy implementation and of the progress achieved.

Evaluation is a specific and in-depth review done after certain time intervals, when substantive steps of policy implementation have already been made and tangible changes and policy impacts are expected to have materialized and can be assessed.

Learners’ Notes 3

Page 5: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

M & E in the cycle of policy formulation and implementation

The following graph presents an overview of the role of Monitoring and Evaluation in the cycle of policy formulation and implementation:

Learners’ Notes 4

Food Security Policy FORMULATION

Food Security PolicyIMPLEMENTATION

MONITORING & EVALUATON of policy implementation,

results & impacts on food security

Adjustments in policy design

Adjustments in policy implementation

Feedback

Other policyrealms

Other policies, relevant external factors / changed

conditions

FeedbackFeedback

Page 6: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Purpose of FSP Monitoring and Evaluation

Monitoring and Evaluation of food security policies serves to inform government and other stakeholders of the state of implementation of the policies and the progress towards achieving the intended food security objectives.

M & E provides answers to the following questions:

Are the policy measures implemented as planned? Is the implementation of the policies bringing about the desired changes and

working towards attaining the planned food security objectives/results? If there are delays in implementation and/or divergence between the planned policy

objectives and the actual results achieved, what are the reasons for that divergence? Possible reasons to be considered: Flaws in policy design or implementation, impacts of other policies on the food security situation or changing external conditions which affect policy implementation or results.

What needs to be done to remedy the deficiencies (adjustments in policy design and/or implementation, integration of food security considerations into other policies and programmes)?

Learners’ Notes 5

Page 7: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Monitoring and Evaluation concepts

A comprehensive system for monitoring and evaluating food security policies incorporates elements of different monitoring and evaluation concepts, as described below.

Table 1: Different Monitoring and Evaluation Concepts

M & E of policies

Policies are set at aggregate levels (macro, national, sectoral) and aim at wide-spread impacts (country-wide). The subjects of policy M & E are the policy objectives, strategies for and means of implementation and the policy impacts. A policy M & E system is organized at central level, compiling and complementing the results of programme and project M & E.

M & E of programmes & projects

Programmes and projects are set at lower aggregate levels and are more narrowly defined in terms of objectives set, issues and/or target groups addressed and/or geographic areas covered. Monitoring at P & P level is organised by the P & P management. Ideally, P & Ps form part of a strategy for policy implementation, and the results of P & P M & E feed into the policy M & E system.

Monitoring of the implementation process

Monitoring of implementation of policies / programmes / projects covers:

financial and other inputs used (accounting / input monitoring); activities performed (activity monitoring); outputs achieved (output monitoring);

It compares actual accomplishment with plan.

M & E of results and impacts

It traces and assesses the results of policy / programme / project interventions in comparison with the ex ante situation before the intervention (baseline) and in view of the overall objectives set and possible unforeseen side effects.

Monitoring

It is a continuous exercise, internally organized by the organization in charge of policy / programme / project management, to generate real-time data and information on policy / programme / project performance regarding implementation and results as basis for management control, for being able to trace possible diversions between reality and plan in time and to make necessary adjustments as early as possible.

Evaluation

Specific reviews of policies, programmes, or projects. A singular or sequential exercise, often externally initiated and organised by high level decision makers, supervising bodies and financing agencies, in order to provide evidence on the overall performance of projects / programmes / policies in terms of relevance, efficiency, effectiveness, sustainability. Evaluations typically serve as basis for decisions on extension, expansion, modification and/or replication of projects and programmes respectively necessary adjustments of policies.

See Annex 1: Case study - M & E concepts applied to a food security policy and related projects and programmes

Learners’ Notes 6

Page 8: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Approaches and Methods

Next we will describe different approaches and methods which can be applied in monitoring the implementation and impacts of Food Security Policies. They refer to:

Logical Framework analysis; Food Security Indicators; and Data and data sources.

Logical Framework analysisLogical Framework (Logframe) analysis is widely applied in project and programme planning, management, monitoring and evaluation.

It is also a suitable method for policy analysis, monitoring and evaluation because it helps to:

set out clearly defined objectives for the different levels of policy implementation (project / programme and policy level) against which progress in implementation and actual achievements can be monitored and evaluated;

select relevant indicators for measuring progress and achievements; reveal the sources of data to be used for measuring the indicators; and identify conditions which are important for the achievement of the policy objectives.

Though these conditions cannot be influenced by the policy, they need to be monitored and responded to, if they change (e.g. world market prices).

Ideally, a Logframe analysis is done at the early planning stage of a policy, programme or project, in order to ensure that implementation is guided by a clear and consistent framework of objectives, and to allow for the establishment of a baseline situation against which the changes induced can be monitored and evaluated.

A Logframe analysis culminates in the establishment of a Logframe matrix.

Levels of objectives Indicators Data sources Conditions

Overall Policy Objective

Specific Objectives

Results

Policy Measures

Learners’ Notes 7

Page 9: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

The Logframe matrix provides a clear and comprehensive framework of the hierarchy of: interlinked and consistent objectives (1st column); related indicators for measuring objective achievements (2nd column); data sources of the indicators (3rd column); and important assumptions and conditions (4th column).

See Annex 2: Example of a basic LogFrame Matrix for a Food Security Policy

Food Security IndicatorsIndicators are measurements of change which serve as sign posts, milestones and benchmarks to measure progress in policy implementation and of the results achieved so far.

Through the use of suitable indicators it is possible to find out:

whether policy implementation is progressing as planned (process monitoring); which changes policy implementation has brought about already (actual

esults/outcomes achieved compared with baseline); and whether it is on track towards reaching the planned objectives (actual

results/outcomes achieved compared with objectives).

Process monitoring indicatorsTypical indicators for process monitoring are:

the number and type of food security initiatives launched; funds and other resources mobilized for food security interventions; food security expenditures; geographic areas covered (geographic targeting); vulnerable population groups and number of people reached (social targeting); food security concerns considered in other programmes and policies.

See Annex 3: Common Food Security Indicators and possible Data Sources

For the sake of clarity of M & E results, and for reasons of cost- and time-efficiency in collecting the necessary data and analysing them, the selection of indicators should be confined to one or a few indicators that are most suitable for tracking the changes of the phenomena to be observed.

Good and suitable indicators are Sensitive, Measurable, Attainable, Relevant, Time-bound and trackable, summarized as “SMART”, as described in the table below. Let’s read the table considering, as an example, that the policy objective is to increase food supplies by increased domestic food production. Indicators will be: Food production and Import figures:

Learners’ Notes 8

Page 10: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Table 2: “SMART” Quality Criteria for IndicatorsCriteria Meaning Example

Sensitive

Specific to the issue that is intended to be changed, and sensitive to the changes induced.

Both indicators measure the two factors which contribute to food supplies and are sensitive to changes induced by the policy (food production and supplies expected to increase, imports to stagnate or go down).

Measurable Measurable, objective and unambiguous, not easily manipulated.

Indicators are easily measurable, derived from agricultural production and trade statistics.

Attainable Attainable by the policy measures and applicable to measure progress towards achieving objectives.

Increase of food supplies is attainable by increased production brought about by policy measures to promote domestic food production.

Relevant Measuring factors which are relevant, i.e. directly related to the policy objectives to be achieved.

The indicators are relevant, because food production is - apart from food imports - the major source of food supplies

Time-bound and trackable

Varying over time, reflecting at what point in time changes can be expected to happen; trackable on the basis of available data.

Indicators vary over time and can be easily tracked, since the required data are already and frequently available from agricultural and trade statistics.

The application of the “SMART” criteria gives preference to quantitative indicators that are measurable and, therefore, objectively verifiable.

However, sometimes it is necessary to also consider qualitative indicators, if, for example, meaningful quantitative data are not (yet) available, when participatory approaches to monitoring and evaluation (key stakeholder and target group participation) are applied, or for cross-checking (triangulation) of the results of a quantitative analysis.

Examples of quantitative and qualitative indicators Quantitative indicators: income, growth rates, production figures, land use data,

food consumption and expenditure data, prices, marketing volume, imports, food stocks, nutrition status based on anthropometric measurements, etc.

Qualitative indicators: perception of households of their economic status or food situation (better/ worse/same as before); observations, experience and/or expectations of stakeholders and target group regarding changes induced by a policy, programme or project, etc.

In choosing the most suitable indicators, the following words of Albert Einstein should be kept in mind: "Not everything that counts can be counted, and not everything that can be counted counts."

Data and data sourcesThe selection of indicators and their use in monitoring and evaluation is closely linked to the issues of data availability and quality.

Only indicators for which valid data are readily available or that can be generated on time and in a cost-efficient way should be selected.

Learners’ Notes 9

Page 11: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

The data quality largely determines the validity of the monitoring and evaluation findings. Criteria for data quality are Accuracy, Preciseness and Timeliness (“APT”), as presented below.

Table 3: “APT” Quality Criteria for DataCriteria Meaning ExamplesAccurate Data matching, as much as

possible, the actual values or properties of the phenomena being studied.

Food production figures, prevailing market prices, number of malnourished children, number of people below the poverty line.

Precise Data reflecting the exact (and not just approximate, broad, vague) values or properties of the phenomena being studied.

Exact (instead of approximate) production costs, exact market prices (instead of wide price ranges), specific income groups (instead of households with wide income ranges considered in one group)

Timely Data on current situation are available on time.

Data for relevant reference periods (baseline, certain periods, most current) are available on time.

Data collection can be a laborious, cost-intensive and time-consuming exercise.

Reasons of cost- and time-efficiency call for a maximum use of data that are available through statistics and/or are the results of relevant studies and surveys done by different organizations (e.g. research institutions, ministries and government departments, development organizations, NGOs).

Once the indicators are defined and clarification is achieved on the data required, the existing information and data sources should be reviewed, so as to see which meaningful data are available and routinely collected.

If the required data are not available, or cannot be obtained in appropriate quality and time, the following options for data generation are to be considered:

Learners’ Notes 10

Page 12: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Table 4: Options for data generationOptions Description Example

PIGGYBACKING ON / UPGRADING OF EXISTING DATA COLLECTION SYSTEMS

Existing statistical services and data collecting systems may not generate exactly the type of data required but similar kind of data. By complementing or upgrading ongoing data collection exercises, the type and quality of data generated can possibly be adapted to the monitoring and evaluation requirements.

Data on household food expenditures are routinely collected through economic household surveys, but not differentiated by different income groups. If a differentiation of food expenditures by income group is introduced in the survey, this will provide evidence on the effect of food security policies on food consumption of low income households.

Conduct own particular surveys for data collection.

If the data required on particularly important food security issues cannot be obtained from existing data sources, there might be the need to conduct own specific surveys

An assessment of impacts of policies on different vulnerable population groups (e.g. rural poor, urban poor, jobless, small farmers, HIV/AIDS affected households) may require special surveys to be conducted.

Selection of alternative / proxy indicators

If there is a lack of suitable and current data on the subject to be studied, and if a special survey to collect such data is not feasible, alternative indirect or proxy indicators may have to be used on which data are available or can easily be generated. Although not directly an expression of the phenomena to be observed, proxy indicators are related to such phenomena and change in the same direction, thus allowing conclusions on the main subject of concern.

A direct indicator to measure the impact of policies on food security would be a sustainable improvement of nutritional status of a population. If respective data are not available, proxy indicators can be used, such as the nutritional status of children under five in selected sample areas; prevalence of nutrition related diseases; the frequency of appeals for food assistance; the phenomenon of 'hunger migration'; the number of meals consumed per day, etc.

Learners’ Notes 11

Page 13: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Evaluation criteria

To assess the overall performance of development interventions, a set of evaluation criteria has been developed1 which is widely applied in project and programme evaluations, and can analogously be applied to evaluate the overall performance of policies.

The evaluation criteria are the following:

Table 5: Evaluation criteriaRelevance The extent to which the objectives of a development intervention are

consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies.

Effectiveness The extent to which the policy intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.

Efficiency A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.

Impact Positive and negative, primary and secondary long-term effects produced by a policy intervention, directly or indirectly, intended or unintended.

Sustainability The probability of continued long-term benefits of an intervention, even after the intervention as such is being phased out.

1 A set of evaluation criteria has been developed by the by the Development Assistance Committee (DAC) of the Organisation of Economic Cooperation and Development (OECD). The document is available at http://www.oecd.org/document/22/0,2340,en_2649_201185_2086550_1_1_1_1,00.html]

Learners’ Notes 12

Page 14: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Setting up an FSP Monitoring and Evaluation system

Institutional set-upIn order to ensure accountability and transparency, it is essential to ensure the participation of key stakeholders in setting-up and operating an M & E system for food security policies, for example, in:

indicator selection; definition of related benchmarks; and monitoring formats to be applied.

Monitoring should be performed at all levels of policy implementation (project, programme and policy).

While the implementing organizations are responsible for monitoring at the project and programme level, the monitoring and evaluation results of all food security-related interventions should be compiled and analysed by a central M & E unit attached to the institution in charge of overall coordination of food security policies.

There are different options for establishing a policy monitoring unit:

at supra-ministerial government level; as a special unit in a line ministry (e.g. Ministry of Agriculture); as a separate project-type unit; outsourcing to an external institution (e.g. research institute, NGO, consulting firm).

See Annex 4: Options and criteria for setting up a monitoring and evaluation unit at central policy level

Example: central monitoring unit for food security – Sierra LeoneIn Sierra Leone, the Right to Food Secretariat has been established within the Office of the Vice-President. The objective of the Secretariat is to facilitate joint planning, and coordinate, monitor and evaluate efforts among all line ministries and stakeholders in Sierra Leone with the aim of achieving food security for all Sierra Leoneans.The tasks of the Right to Food Secretariat comprise:

monitor and evaluate progress towards the realization of food security in Sierra Leone for the attention of the Committee;

ensure that information about all food security programmes is centrally stored and regularly updated;

ensure that progress towards the realization of food security is properly monitored; ensure that progress reports are regularly prepared for the attention of the

President and Parliament through the Minister of Agriculture, Forestry and Food Security; and

ensure that reports on progress towards the realization of food security are regularly published for the information of the public.

In order to facilitate the compilation and analysis of the M & E results, the activities of the different organizations and at the different levels will have to be harmonized with regard to

Learners’ Notes 13

Page 15: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

indicators used, M & E approaches applied and reporting formats. Such harmonization will also allow the M & E results to be fed into a Food Security Information System (FSIS).

Learners’ Notes 14

Page 16: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Linking M & E and FS Information to other Policy Spheres

Food security information is needed at various stages of food security policy and programme cycles to:

identify problems that need to be addressed; prompt timely and suitable actions; guide the design of policies and intervention strategies; monitor and evaluate the effects of the interventions; and trigger necessary adjustments in policy design and implementation.

In general, a multitude of information systems and data banks are operated by different organizations.

Though related to different policy spheres (e.g. poverty alleviation, rural and agricultural development, trade, social and health policies), such policies have implications for FS and the related data banks, and information systems frequently contain relevant food security information.

However, such information systems are often – if at all – poorly linked, with duplication of efforts and inconsistencies in data generation, data analysis and conclusions on actions to be taken. In order to rationalize the process of policy formulation, implementation, monitoring and evaluation, and to track the impacts of different policies on FS, it is imperative to harmonize and link the various data banks and information systems.

Such linkage helps to:

put policy decisions on a sound information basis; facilitate information and data sharing among the different organizations concerned

with FS interventions; provide relevant data for M & E of the implementation and results of FSPs and

related interventions; and track the impacts of different policies on food security.

Thus, this linkage contributes to overall increased efficiency in policy implementation and effectiveness in reaching the planned FS objectives.

Since poverty and food security issues are closely linked and partly overlap, an integration of the M & E system for both poverty alleviation and FSPs is also a valid option to be considered.

Learners’ Notes 15

Page 17: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Summary

M & E of food security policies serves to inform government and other stakeholders of the state of implementation of the policies and the progress towards achieving the intended food security objectives.

Different approaches and methods can be applied in monitoring the implementation and impact of FSP:

Logical Framework (Logframe) analysis; Food Security Indicators; and Data and data sources.

The policy implementing organizations are responsible for monitoring at the project and programme level. The M & E results of all FS-related interventions should be compiled and analysed by a central unit attached to the institution in charge of overall coordination of FSP.

In order to rationalize the process of policy formulation, implementation, monitoring and evaluation, and to track the impacts of different policies on FS, it is imperative to harmonize and link the various data banks and information systems.

Learners’ Notes 16

Page 18: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

If you want to know more

Online resourcesMetz, Manfred, Monitoring Policy Impacts (MPI) – The Eight Methodo-“logical” Steps for MPI, FAO, EASYPOL Module 57, November 2005 http://www.fao.org/docs/up/easypol/383/8-methlgcl-stps_057EN.pdf

Metz, Manfred, Monitoring Policy Impacts (MPI) – The Application of the Logframe Method, FAO, EASYPOL Module 58, November 2005http://www.fao.org/docs/up/easypol/384/logframe_058EN.pdf

Metz, Manfred, Monitoring Policy Impacts (MPI) – Setting-up and Organising MPI, FAO, EASYPOL Module 59, November 2005 http://www.fao.org/docs/up/easypol/385/set_up_orgnzng_MPI_059EN.pdf

USAID Center for Development Information and Evaluation, 2000, Performance Monitoring and Evaluation, Measuring Institutional Capacity, TIPS, No. 15 http://pdf.usaid.gov/pdf_docs/PNACG612.pdf

Additional reading World Bank, Operations Evaluation Department, 2000, Designing Project Monitoring and Evaluation, Lessons and Practices, Number 9, Washington D.C.

World Food Programme, Office of Evaluation (OEDE), Monitoring and Evaluation Guidelines, draft, Rome, October 2001

Learners’ Notes 17

Page 19: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Annex 1: Case study - M & E concepts applied to a food security policy and related projects and programmes

In pursuance of the Millennium Development Goals (MDG), the government of country X has designed a food security policy with the objective of eradicating hunger and malnutrition of its population.

Based on an analysis of the present state of food insecurity, its causes and effects, it was found that the food security policy needs to address the issues of insufficient access to food for poor and vulnerable population groups, of insufficient food supplies, and that provisions have to be made to cope with food emergencies in the case of natural disasters.

Accordingly, the following policy interventions have been defined under the FSP:

A programme to promote agricultural and food production, composed of several sub-programmes (e.g. research and extension, input supply, marketing); the programme aims at the dual objectives of increasing domestic food supplies and increasing the income of poor smallholder farmers.

A national safety net programme for specific vulnerable groups. A project to set up a food security reserve for market stabilization and

emergency response.

Specific objectives and targets have been defined for each programme, sub-programme and project, and different governmental and non-governmental organizations have been entrusted with their implementation.

Each of these organizations establishes a monitoring system to record the inputs (financial, material, manpower) used, the activities performed and the outputs achieved.

In the case of the safety net programme, the system ofimplementation monitoring covers, for example, the following issues:

Input monitoring - staffing, office and office equipment, cars, financial inputs, etc. Activity monitoring - staff recruitment and training, organizational set up and

management, identification of vulnerable groups, screening and registration of beneficiaries, management of resources, distribution of assistance (e.g. food aid or cash assistance), etc.

Output monitoring - number of (different groups of) beneficiaries actually reached, contribution of assistance to household food supplies and the state of household food security, etc.

A monitoring system for the food security policy, to be established at the national aggregate level, will have to take account of the progress and results achieved in the implementation of all programmes and projects implemented under the policy. Apart from reviewing the progress of policy implementation, it will be particularly important to assess whether and to what extent the bundle of all policy interventions is working towards achieving the overall policy objectives and brings about a significant improvement in the food security situation.

Learners’ Notes 18

Page 20: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Complementary to the continuous monitoring exercises, the individual projects, programmes and the policy as a whole may become subject to in-depth evaluations after a certain period of implementation. The focus of a policy evaluation is the overall performance of the policy in attaining the planned food security objectives.

Learners’ Notes 19

Page 21: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Annex 2: Example of a basic LogFrame Matrix for a Food Security Policy

The table below presents an example of a basic Logframe matrix for a Food Security Policy, implemented through the following three food security interventions (programmes/projects):

1) Promotion of agricultural and food production with a special focus on small farmers. 2) A national safety net programme for specific vulnerable groups.3) A project to set up an emergency food security reserve.

Consider that in a real Logframe matrix, benchmarks/targets should be set, i.e. indicators are to be valued and timeframes for the planned achievement of the benchmarks/targets are to be defined.

Learners’ Notes 20

Page 22: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Objectives Indicators Data sources ConditionsOverall Policy Objective:

Food security improved

Nutritional status of children.

No. and share of undernourished population.

No. and share of households below (food) poverty line.

National and international statistics.

Nutrition and household surveys.

Good governance.

Conducive macro- economic environment and decent economic growth.

Peaceful development.

Specific Objectives of Food Security Programmes:

1) Increased income and household food supplies through increased production by smallholder farmers.

2) Prevention of food shortages of vulnerable groups.

3) Stabilization of food supplies in the case of disasters and production shortfalls.

As to 1) e.g.: Income of smallholder

farmers. Overall and household

food supplies.

As to 2) e.g.: Economic and nutritional

status of vulnerable groups.

Household food supplies of vulnerable groups in comparison with needs.

As to 3) e.g.: Variation of market prices

of food. Food supplies in the case

of production shortfalls.

National statistics.

Food security data bank.

Special surveys.

Donor support for food security programmes.

No major natural disaster (drought, flood, earthquake).

Planned Results of food security policy measures, e.g.:1) Increased agricultural

and food production by smallholder farmers.

2) Food deficits of vulnerable groups covered by public employment and income generation and transfers.

3) Food stocks established to cover temporary production and supply shortages.

Data on agriculture and food production of smallholder farmers in programme area.

Income generated through public employment schemes.

Contribution of incomes generated and transfers to household food supplies of target groups.

Volume of food stocks. Releases of food stocks

in the case of supply shortages.

Project and programme records and M & E results.

Special surveys.

To be defined in project and programme planning.

Policy Measures:1) Promotion of

smallholder agriculture and food production.

2) Establishment of a national safety net programme for social protection.

3) Establishment of a national food security reserve stock.

All types of activities (projects/programmes) performed and amounts of manpower, material and financial resources allocated and used for implementing the policy measures.

Programme and project accounting and records.

To be defined in project and programme planning.

Learners’ Notes 21

Page 23: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Annex 3: Common Food Security Indicators and possible Data Sources

Common Food Security Indicators Data sources Baseline M & E updatePeriod Data Period Data

For measuring ACCESS TO FOODGNP per capita (US $ / annual growth rate) StatisticsGDP per capita in PPP (purchasing power parity) US $ StatisticsPopulation below national poverty line (%) Stat./ surveysPopulation below poverty line of US$ 1 PPP/day (%) Stat./ surveysPoverty gap at US$ 1 PPP/day (%) Stat./ surveysIncome distribution (Gini coefficient) StatisticsFood expenditures by different income groups SurveysShare of household income spent on food (av. %) SurveysShare of own production in household food supplies SurveysHFIAS (Household Food Insecurity Access Scale) SurveysHDDS (Household Dietary Diversity Score) SurveysNo. of people in need OF transfers/food assistance Surveys

For measuring FOOD AVAILABILITYFood production index (1989-91=100) Agric. stat.Agricultural production growth per annum/capita Agric. stat.Average yield food grain production (kg/ha) Agric. stat.Share of food imports in food grain supplies Trade stat.Share of food aid in annual food grain supplies (%) Trade / food

aid statisticsDaily per capita supply of calories (kcal) Food balanceFood calorie availability as percentage of requirements Food balanceDaily per capita supply of protein (g) Food balanceDaily per capita supply of fat (g) Food balance

For measuring STABILITY OF FOOD SUPPLIESAnnual variation food grain production (metric tons, %) Agric. stat.Variation of grain yields (metric tons, %) Agric. stat.Annual variation of food imports / exports (m. tons, %) Trade stat.Annual variation of food supplies (metric tons, %) Ag. & Trade

stat.Seasonal variation of food supplies (metric tons, %) SurveysVolume / variation of (public, commercial, HH) food stocks

Surveys

Variation in food aid deliveries Records

For measuring FOOD UTILIZATIONPopulation (no. %) without access to - safe water

- health services - sanitation

Statistics. /surveys

Prevalence of water borne diseases Medical. stat.

For measuring MALNUTRITIONUndernourished population (number, %) Stat./surveysUnderweight children under five (number, %) Nutrition surv.Stunted children under five (number, %) Nutrition surv.Wasted children under five (number, %) Nutrition surv.Low birth weight (< 2,500g) (number, %) Medical surv.Pregnant women with anaemia (%) Medical surv.Prevalence of diseases related to malnutrition Nut./med.

surv.

Learners’ Notes 22

Page 24: Food and Agriculture Organization · Web viewLinking M & E and FS Information to other Policy Spheres 15. Summary 16. If you want to know more 17. Annex 1: Case study - M & E concepts

Course – Food Security Policies – Formulation and ImplementationLesson 3 – Policy Monitoring and Evaluation

Annex 4: Options and criteria for setting up a monitoring and evaluation unit at central policy level

The table presents the options and the criteria for selecting the appropriate organizational set-up for Monitoring and Evaluation at policy level, and the pros and cons of the different options.

Options Criteria Advantages DisadvantagesM & E unit at supra-ministerial level

Monitoring of broad economic reforms.

Monitoring cross-cutting policies.

Links to policy reforms in different sectors (e.g. PRSP).

Close lines of communication to policy decision makers.

Trust and reputation. Easier access to

official data.

Long communication channels to technical staff, project and programmes in the field.

Thin line between independence and political interference.

Line ministry

Ongoing (long term) monitoring of sector policies.

Monitoring of policy impacts on a specific sector.

Access to own technical expertise.

Acquaintance with subject matter.

Close link between decision-makers, actors and monitors.

MPI might be seen as a “side activity”.

Sometimes no clear division of labour and responsibilities.

Project-type monitoring unit

Timely limited monitoring

Pilot phase for institutionalized unit.

Policy monitoring as condition for external funding

Predetermined planned budget, often with foreign assistance.

Can be tailored to the very specific M & E requirements.

No capturing of long-term implications of policy changes.

Less commitment for time-bound activity.

Possible dependence on external funding.

Outsourcing to external institution

Option, if monitoring and analytical capaci-ties are weak in go-vernment institutions.

Very specific expertise required.

Scaleable from partial involvement to complete outsourcing .

Best and special expertise can be chosen.

Labour division according to needs and expertise.

Greater independence from political interference.

Financial dependence can cause twisted results.

Difficult access to official data and information.

Less support due to distance between monitors and decision makers.

Learners’ Notes 23