EVALUATION THEORY, APPROACHES AND PRACTICES IN THE PHILIPPINES 1 st M&E Network Forum, 7-8...

EVALUATION THEORY, APPROACHES AND PRACTICES IN THE PHILIPPINES1st M&E Network Forum, 7-8 November 2011, Crown Plaza, Manila, Philippines

Since its inception 35 years ago, IFAD focused on rural poverty reduction. IFAD programmes/projects aim to increase incomes of rural poor

Measurement of incomes has traditionally proven to be a challenge, even for well-qualified research outfits

Project managers, often operating in remote areas, do not have access to human & technical resources necessary for results measurement

Performance of M&E systems was habitually one of the weak spots in IFAD projects

In 2001, IFAD developed its highly referenced Guide for Project M&E -- http://www.ifad.org/evaluation/guide/ providing essential tools and guidance (methodology, templates for ToRs, logframe applications, etc.)

But, the MDGs set out poverty/hunger reduction as Goal Number 1

IFAD faced a need to report on results & impact in terms of MDGs– harmonisation and aid effectiveness agenda

Standardised methods were therefore developed- “RIMS” Comprehensive system for results

and impact measurement in use by all IFAD-funded projects

Based on a standard list of indicators (output/outcome/ impact)

Mandatory, periodic reporting to IFAD-HQ

Weaknesses persisted in collecting information required to report on aid effectiveness

2nd LEVEL RESULTS

[OUTCOMES]

IMPACT

Annual reporting to IFAD from PY1 onwards

Annual reporting from PY2/PY3 onwards

Reporting three times in project cycle: baseline, mid-term and completion

Project M&E system RIMS Impact Surveys

1st LEVEL RESULTS

[OUTPUTS]

IFAD’s RIMS mandatory, proxy measures to demonstrate increasing income:◦ Asset ownership, using

Principal Components Analysis◦ Child nutrition

(anthropometrics) RIMS also includes measures for

other MDG-related indicators Impact-level indicators are

measured at baseline, mid-term and completion (normally a 5 to 7 year span)

From IFAD pre-defined standard indicators, projects select only relevant output/outcome measures

Standardisation means data can be aggregated at province, agency, national, regional or global level

RIMS as minimum requirement: projects expected to develop more extensive M&E base

Impact indicators focused on goal and objective level: contribution rather than attribution

Impact assessment focuses on most essential questions: minimalist survey

Harmonisation: Anthropometric data in accordance with WHO and UNICEF global standards

Avoid duplication in efforts: Where UNICEF or national agencies provide data for project area, nutrition surveys not required

RIMS was introduced in the PH in 2006. Most projects conducted baseline impact surveys only, but are regularly reporting 1st and 2nd level indicators.

Project RIMS progress reporting focuses on quantitative data. Qualitative reviews are undertaken during supervision

RIMS level 1 & 2 data are validated through joint supervision missions with NEDA, and annual country programme reviews

Learning approach: involving project and agency staff

Standardized indicators permit to track project progress as well as aggregate measures across all projects;

RIMS is low cost and can be implemented by project teams or its partners. Results can be validated by other exercises like supervision mission, annual country programme reviews;

Some impact indicators are difficult to interpret; e.g. food insecurity are highly sensitive to annual variations in food production;

Outcome surveys may be needed to show trends in short/medium-term outcomes, & explain results chain (links from outputs to impact);

RIMS has ability to report across agencies, provinces and projects on overall country programme. However, it requires:

Computerized MIS that can facilitate and validate data entry, accessible to stakeholders;

Improved data collection at grassroots level (timely, complete, accurate way).