2nd Results- Based M&E Workshop

ADB Grant 0133-CAM: Public Financial Management in Rural Development Ministries for Component 1

2nd Results- Based M&E Workshop

Overview in Institutionalizing

M&E System in Cambodia By

Rebecca V Polestico, PhD 19-20 May 2011

2nd Results-Based M&E WorkshopSalon II Raffles Le Royal Hotel

Phnom Penh, Cambodia

04/19/23 Institutionalizing M&E System in Cambodia 1


2nd Results- Based M&E Workshop Discussion Points Objectives Principles of Monitoring and Evaluation -Results-Chain Roles of Monitoring and Evaluation Monitoring and Evaluation System

Description Attributes

Public Sector Management Components Attributes Results-Based

Assessing Capacity Development Skills




Objective (s) To differentiate Monitoring from EvaluationTo examine the generic components, features and functions of M&E SystemTo identify the capacity skills needed in developing M&E SystemTo adopt the processes in institutionalizing M&E System in RDMs



Definitions

• Monitoring is the continuous assessment of project implementation vis-à-vis schedules and of the use of inputs and services by beneficiaries– Provides managers and

stakeholders with continuous feedback on implementation

– Identifies actual or potential successes and problems early to facilitate timely adjustments

• Evaluation is the periodic assessment of a project’s relevance, performance, efficiency and impact– Managers undertake interim

evaluations during implementation as a review of progress and a prognosis of likely effects.

– Necessary adjustments to project design may be identified. Mid-terms reviews have become popular for the same reason

– Terminal evaluations are conducted at the end and include an assessment of a project’s effects and their potential sustainability




Monitoring

Reports progress to managers and alerts them to problems

Routinely collects data on these indicators, compares actual results with targets

Translates objectives into performance indicators and set targets

Links activities and their resources to objectives

Clarifies programme objectives

Provides lessons, highlights significant accomplishment or program potential, and offers recommendations for improvement

Explores unintended results

Examines implementation process

Assesses specific causal contributions of activities to results

Analyses why intended results were or were not achieved

Evaluation

Complementary Roles of Results-Based Monitoring and Evaluation




Monitoring

Analysis and reporting specified

Data Management Systems specified (Data encoding, programming, processing and reporting)

Institutional responsibilities defined

Indicators from planning

Stakeholders engagement

Results dessiminated or shared


Methodology formulated or designed

Evaluation

With reference to APCOP MfDR Framework Attributes of Results-Based Monitoring and Evaluation


A paper or computer-based tracking mechanism which is indicator-driven, outcome-led and impact-based derived from developed results of any specified intervention strategy, capturing changes contributing to the effectiveness and efficiency of any program development initiatives and setting.

M&E System Defined


M&E System: Description

• Consists of eight components with stakeholders involvement and beneficiary-centred

• Focused on the planning, management, budgeting and resources availability within a timeframe

• It’s the process of getting the results developed (RBM) and managing for developed results (MfDR)

ADB Grant 0133-CAM: Public Financial Management in Rural Development Ministries for Component 199 04/19/23 9Institutionalizing M&E System in Cambodia

1.Review of Project Goals and Objectives

2. Clear statements of measurable project performance objectives for which indicators can be defined

3. Structured set of indicators, covering outputs and their impacts on beneficiaries using the Results-Based Management Logframe

4. Consensus development of evaluation tools for structured and non-structured data gathering techniques

5. Provisions of collecting data and managing project records so that data required for indicators are compatible with existing statistics and available at reasonable cost

6. Reporting and Feedbacking on the data quality assessment, validation and process checks

7. Institutional arrangements on the Formative and or Summative Evaluation (Interim)

8. Information Exchange on the M&E findings for feed back into decision making

Eight Components of M&E System Design


2.

Indicators

Or Perfo

rmance

Measures

1. Project Goals, Purposes, Objectives, and Specific Objectives

3. Results-Based Management Logframe

8. Information

Sharing and

Exchange or

Feedback Mechanism

Results

4. Development of tools

and forms

5. Data Management : Data Collection, Encoding and Programming Information Processing and Analysis

7. Interim Evaluation:Formative or Summative

6.In

stitu

tional

Arrangem

ent

M&E System and Capacity Building Framework

M&E System


Level of Results

1111

MfDR in Rural Development Workshop MfDR in Rural Development Workshop 6/7 December 20106/7 December 2010

ADB Grant 0133-CAM: Public Financial Management in Rural Development Ministries for Component 104/19/23 Institutionalizing M&E System in Cambodia 12

Cambodia Millennium Development Goal (MDG)

To reduce poverty……



Brief Look at the Rectangular Strategy

SDPFrameworkProgramsProjectsActivities


Government Documents

1515 04/19/23 15Institutionalizing M&E System in Cambodia


KEY INDICATORS Unit2008Actual

2009 Estimated

2010Projected

2011Projected

2012Projected

2013Projected

Land under crops (cash and industrial crops)

000 Ha 596 645 774 930 1000 1000

Rice: cultivated area million Ha 2.61 2.63 2.65 2.65 2.65 2.65

Rice Yield per hectare tons 2.74 2.77 2.80 2.83 2.87 3.00

Irrigated land area 000 Ha 1.120 1.145 1.170 1.195 1.220 1.245

Reduction of animal morbidity % 6.0 5.5 5.0 4.5 4.0 3.5

Production of cattle Million heads 4.20 4.29 4.32 4.38 4.44 4.50

Cultivated area of rubber000 Ha 111.4 128.4 143.4 158.4 173.4 188.4

Yield per hectaretons 0.95 1.10 1.10 1.15 1.20 1.30

Rubber production 000 tons 26.6 38.5 49.5 63.3 78.0 97.5

Fishing Lots Sq. Kms 415 415 415 415 415 415

-Released to Fishery Community % 56.4 56.4 56.4 56.4 56.4 56.4

Fish: Production tons 471 515 617 668 726 788

cover -- % of land area % 59.00 57.59 57.99 58.39 58.79 59.19

Reforested 000 Ha 10.81 18.92 73 73 73 73

Forest Boundary Demarcation km 321 228 413 500 500 500

Forestry Community No. 124 210 300 350 400 450

NSDP: Key Indicators for Agriculture, Fisheries, and Forestry


MAFF M&E System and Indicators

MAFF has 5 programs Enhancement of Agricultural Productivity and Diversification Enhancement of Market access for Agricultural Production Strengthen Institutions, Legal, HRD Sustainable Fisheries Resource Management Sustainable Fisheries Resource Management

Each program is broken down into sub-programs – e.g for P1, S1 = soil improvement, S2 = increasing crop production, etc.

Each sub-program broken down into activities – e.g. for S1, A1 = dissemination of compost fertiliser; A2 = crop zoning in communes.

1717


MAFF M&E Information System

Planning Forms Monitoring Forms Evaluation Forms

P1 - Ministry P1 - Ministry Programme Budget Programme Budget SummarySummary

M1 - Quarterly M1 - Quarterly Monitoring ReportMonitoring Report

E1 - Sub-Programme E1 - Sub-Programme EvaluationEvaluation

P2 - Programme P2 - Programme ProfileProfile

M2 - Annual M2 - Annual Monitoring ReportMonitoring Report

E2 - Programme E2 - Programme Evaluation SummaryEvaluation Summary

P3 - Sub-Programme P3 - Sub-Programme ProfileProfile

M3 - Site Visit ReportM3 - Site Visit Report

P4 - Activity ProfileP4 - Activity Profile


Session 3 - Outline of MAFF M&E System and Indicators

Form P1 – Form P1 - Ministry Programme Budget Summary.docForm P2 – Form P2 - Programme Profile.docForm P3 – Form P3 - Sub-Programme Profile.docForm P4 – Form P4 - Activity Profile.docForm M1 – Form M1 - Quarterly Monitoring Report.docForm M2 – Form M2 - Annual Monitoring Report.docForm M3 – Form M3 - Site Visit Report.docForm E1 – Form E1 - Sub-Programme Evaluation.docForm E2 – Form E2 - Programme Evaluation Summary.doc

1919

MfDR in Rural Development Workshop MfDR in Rural Development Workshop 6/7 December 20106/7 December 2010


ADB Grant 0133-CAM: Public Financial Management in Rural Development Ministries for Component 12104/19/23 Institutionalizing M&E System in Cambodia

PBMIE INDICATOR

Activities OutputOutcome

Input

Agencies: MAFF, MRD, MOWRAM, MEF, MOP, MOE, MOL

Impact

Target or Timeframe

Indicator Tracking and Results-Chain

ADB Grant 0133-CAM: Public Financial Management in Rural Development Ministries for Component 12204/19/23 Institutionalizing M&E System in Cambodia

Target: By 2015 Crop production improved by 20%

(Planned-Budgeted-Implementation-Monitoring-Evaluation)

Indicator

At least

20 acres

of land

Irriga-ted

Activities Purchase of Equipment

Output48 Tractors purchased

OutcomeIrrigated lands increased 10% annually

Input Acquisition of farm 48 tractors

Implementing Agency : MAFF, MRD, MOWRAM, MEF, MOP, MOE, MOL

Impact Improved crop harvest

Level of Results

Indicator Tracking

Questions: Indicator linked to planning?Resource allocated?Implemented? By who?Systems used to track results Established ? Evaluated? Feedback mechanism?


Core Attribute 1: What to consider


S/N M&E Core Attribute 1: Indicator Identification, Mapping and Tracking

1 Indicators linked to planning

2 Identified a focal point responsible for collecting the information

3 Defined the indicators using SMART

4 Categorized the indicators as Input, Output, Outcome, or Impact

5 Allocated resources to the activities in achieving the indicator(s)

6 Followed up planned implementation of activities

7 Used customized forms and or tools in tracking the progress of the indicators

8 Reviewed the indicator progress regularly

9 Reported the status of the indicator to the

10 Cascaded the indicators from National or sectors to provinces, districts or agencies


Core Attribute 2: What to consider in keeping relevant information?


S/N M&E Core Attribute 2: Tools and Forms

1 Information provided what the indicators needed

2 Developed tools and forms using proper constructs

3 If new- Pilot-tested the tools prior to formal use; or if recurrent- has it been standardized

4 Used generic forms e.g survey questionnaire, checklist, interview format, mission forms, Monthly / quarterly or Annual Forms

5 Duplicated forms and tools

6 Reviewed forms and tools

7 Updated the forms and tools

Others


Data Collection Methods

Conversationwith concernedindividuals

Community Interviews

Informal/Less Structured Methods

Fieldvisits

Reviews of official records (MIS and admin data)

Participant Observation

Key informantinterviews

FocusGroupInterviews

One-Time Survey

Questionnaires

PanelSurveys

Fieldexperiments

Census

More Structured/Formal Methods

Direct observation


Core Attribute 3: What to consider in managing information?


S/N M&E Attribute 3: Database Management

1 Identified a focal point responsible for managing the information

2 Computer-based database software installed (EXCEL, SPSS, SAS, Dev Info, etc)

3 Paper-based database using form files

4 Data entry or encoding

5 Data programming

6 Data entry validation, controls and checks or Data Quality Assessments

7 Data processing

8 Statistical analyses

9 Data interpretation

10 Data presentation

11 Output reporting of results by indicator


Core Attribute 4: What to consider in strengthening ownership?


S/N M&E Core Attribute 4: Institutional Arrangement

1 Identified a focal point responsible for institutional partnerships

2 Involved in the development of Monitoring Plan

3 Involved in the development of evaluation Plan

4 Agreement done in form of MoUs, MoAs, or government directives and mandates

5 Defined roles of institutions or partners in monitoring and evaluation activities

6 Respect of confidentiality and non-disclosures

7 Feedback mechanism established (reporting structures established)

8 Others


Opening Session: Introduction I



Horizontal and Vertical Integration

Focus on Common Results

Inter-dependent PSM components

Informed Decisions Continuous Learning

Core Results Attributes

Results-Based Public Sector Management

Plan

Budget

ImplementMonitor

Evaluate

ResultsResults

Plan

Budget

ImplementMonitor

Evaluate

Plan

Budget

ImplementMonitor

Evaluate


30

PLANNING BUDGETING IMPLEMENTATION (Institutional)

MONITORING EVALUATION

Linkages between Levels of Results defined

Results Based Budgeting system established

Institutional priorities aligned to planning outcomes

Indicators from Planning

Evaluation Methodology formulated

Indicators with targets specified for each level of results

Budget based on results from planning

Policies,People, Processes orientatied to deliver intended activities


Institutional Responsibilities defined

Indicators satisfy S.M.A.R.T principles (Specific, Measurable, Achievable, Relevant , Time Bound)

Budgets have medium term horizon linked to planning

Service delivery standards established

Data Processing Methodology specified

Dissemination Methodology formulated

Planning targets aligned to available budgets

Tracking and reporting methodology for budget execution established

Analysis and Reporting Methodology Specified

Stakeholder engagement defined

Core Results Attributes: RDMs


Country Setting Results-Based Management

PLAN FOR RESULTS• President’s Guidepost (16 areas

for transformative leadership)• PDP• RM• PIP BUDGET FOR RESULTS

• MTEF• OPIF• GAA-NEP• ABM• ICC Programming- only FAPs

consistent with plan are approved

IMPLEMENT FOR RESULTS

• Agency PAPs• ICC Reevaluation

MONITOR RESULTS• ODA Review• BPAR• RM/ OPIF Monitoring• OPES• COA Audit Reports

EVALUATE RESULTS

• SER• MDG Reports ResultsResults



Results-Based M&E has two handles:

One of Fear and the other is Opportunity

04/19/23 33Institutionalizing M&E System in Cambodia

Results-Oriented M&E Workshop