Upload
xaviera-ferguson
View
44
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Common monitoring and evaluation framework. Jela Tvrdonova, 2010. Strategic approach to rural development. Focus on limited number of objectives Competitiveness – Environment – Quality of life Axes connected with strategic objectives Strong and dynamic agri-food sector - PowerPoint PPT Presentation
Citation preview
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
1
Common monitoring and evaluation framework
Jela Tvrdonova, 2010
Strategic approach to rural developmentFocus on limited number of objectivesCompetitiveness – Environment – Quality of life
Axes connected with strategic objectivesStrong and dynamic agri-food sectorAgriculture and forestry with high added valueEmployment and growth in rural areasImprovement of the governance in rural areas and
mobilisation of the endogenous potential
Common approach to monitoring and evaluationBased on: Exact definition of objectives in Regulation, Strategic
Guidelines for RD and rural development programs EU Strategic monitoring a national strategies Definition of baseline indicators at the program start
period Suitable combination of output, result and impact
indicators Single framework for all program intervetion
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
4
Content of the CMEF
CMEF – Common monitoring and evaluation framework – Handbook, Annexes, Guidance notes
Indicators and their role in the intervetion logic of rural development programs
Common and additional/program specific indicators
CMEF orientation in relation to indicators The role of indicators in particular evaluation
tasks
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
5
CMEF – legal base
EC Reg. 1698/2005 and 1974/2006 and its Annex VIII
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
6
Monitoring and evaluation
Monitoring – measurement of inmediate outcomes and results at measure and axis level
Evaluation - measurement of long-term and rural areas effects/impacts of intervention, comparissons and lessons learned for next interventions
Monitoring
On-going process which monitors the gradual implementation of the program at the level of financial inpiuts, physical outputs and axes results
It subject of annual reporting on the program implementation
Instruments – common and additional indicators of output and results,
Strategic monitoring
Since 2010 and every other two years Focus of the EC on the progress of
national strategic plans and its objectives implementation
Assesment of the contribution of national plans to the EU Rural Development Strategy
Evaluation
On-going evaluation is based on CMEF and country´s own methodology to carry on the evaluation during the program implementation
Process based on annual reporting of result/impact indicators including periodical exercises:
Ex – ante: evaluation of relevance of the planned intervention and optimizing of the RDP budget
Mid-term and ex-post: for the assessing the progress related to the specific and overall/program objectives,
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
10
Monitoring and evaluation in program logic
Good program is basic ground for the high quality of monitoring and evaluation
The evaluation and the monitoring strats with the program intervention logic
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
11
Intervention logic of RDP
Needs
Change observed in reality during and/or
after intervention
Envisioned change before intervention
Impact
Result
Output
Inputs of the RDPMeasures
Overall/horizontal objectives of RDP
Specific/Axis objectives of the RDP
Operational/measure objectives of the RDP
Change
Overall baseline socio-economic and environmental situation of rural areas, at baseline period
Development context - all intervening
factors
Hierarchy o objectives and indicators 5 types of indicators coresponding with the hierarchy of
objectives Hierarchy of objectives: - instrument helping to show
how local activities contribute to the overall objectives, It is composed of:
- operational objectives (measure level)- specific objectives (Axes/priority areas level)- overall – cross cutting/strategic/horizontal objectives (at the program level)
Indicators copy objectives in the hieararchy
Indicators in intervention logic of RDP
Hierarchy of indicators
Hierarchy of objectives
Impact ndicator
Result indicator
Output indicato
Input indicators
Measures
Overall/horizontal objectives of RDP
Specific/Axis objectives of the
RDP
Operational/measure objectives of the
RDP
Context related baseline indicators
Objectives related baseline indicators
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
14
Role of indicators
Indicators – used as tools to assess how far the expected
objectives have been achieved by measures or whole programmes
– should be specific, measurable, available / achievable in a cost effective way, relevant for the programme, and available in a timely manner (SMART)
– Indicators can not always be filled with quantitative statistical data; in some cases, indicators might also include qualitative assessments or logical assumptions
The quality of indicators - SMART approachS – simple, specific
M – measurable - data exist
A – achievable – available – at low cost
R – relevant – selected well for given program
T - timely oriented
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
16
Types of indicators
Baseline indicators: they relate to general socio-economic context of the programme area (context-related baseline indicators) and to the state of the economic, social or environmental situation in direct relation with the wider objectives of the programme (objectives-related baseline indicators)
Financial execution (input) indicators: they refer to the budget or other resources allocated to the programmes
Output indicators: measure activities directly realized within programmes
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
17
Types of indicators
Result indicators: measure the direct and immediate effects of the intervention and provide information on changes that have taken place
Impact indicators: refer to the benefits of the programme both at the level of the intervention but also more generally in the programme area. They are linked to the wider objectives of the programme
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
19
Common indicators
A common set of baseline, output, result, and impact indicators for the RDPs (Art. 62 Reg. 1974/2006) “shall form the Common Monitoring and Evaluation Framework (CMEF)’’ (Annex VIII lists the common indicators)
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
20
Additional indicators
Since common indicators may not fully capture all effects of programme activities, it is necessary to define additional indicators within the programmes (see Guidance notes A & K)
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
21
Why additional indicators?
The CMEF provides the Common Framework to be applied throughout the EU, thus ensuring comparability
The specificities of each single programme cannot be fully reflected by this Common Framework
It is therefore important to complement the Common Framework by additional indicators in order to capture the full range of intended effects of a given programme
As general rule, a thorough analysis of the programme intervention logic can drive the choice of relevant additional indicators
Considering the limited number of common impact indicators and their broad scope, additional indicators are essential to overcome attribution gaps
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
22
CMEF structure
HandbookAnnex 1: Guidance notes - A: Choice and use of indicators, B:
On-going evaluation , C: Mid-term evaluationAnnex 2: Guidance notes – D: Hierarchy of objectives, E –
Measure FichesAnnex 3: Guidance notes – F – K, Common indicators FichesAnnex 4 Guidance J – O Other guidances
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
23
CMEF guidance on indicators
CMEF provides guidance for MAs in setting up indicators at each level of intervention and baseline indicators
Annex 3 of the CMEF Handbook provides detailed description of all common indicators (Guidance notes F – K):
– F: COMMON INDICATOR LIST (overview of all common indicators)
– G. BASELINE INDICATOR FICHES (detailed description of indicators)
– H. OUTPUT INDICATOR FICHES
– I. RESULT INDICATOR FICHES
– J. IMPACT INDICATOR FICHES
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
24
Indicator Fiches
Each indicator fiche contains the following elements: Type of indicator Related measures Measure Codes Definition of the indicator Subdivision Unit of measurement Level of collection Responsible actor for collection Collection method/good practice Sources Registration frequency
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
25
Measure Fiches
Guidance on practical use of indicators in monitoring and evaluation of RDPs
Guidance note E provides the following on use of indicators within each particular measure:
– Measure Code– Rationale of the measure– Target group– Target area– Common indicators – Link rationale of the measure and indicators– Evaluation questions
4/5 June 2009 Challenges of the CMEF & Ongoing Evaluation
26
Setting up the evaluation framework at the stage of Ex-ante and use of indicators
Evaluation Question Indicators
Evaluation questions – cross-cutting
Evaluation questions Axis/sector specific
Output indicators
Impact indicators
Result indicators Axis based baselines, objective related, contextual
Baselines and their quantification,
Target levels
Horizontal baselines, objective related, contextual
RDP objectives
Overall strategic objectives
Axis specific objectives
Measure/operational objectives
Baseline values
of output indicators
Targets for impact indicators
Targets for results indicatiors
Targets for output indicators
Common and program spec. Eval. questions