Poverty Reduction Information Systems (PRIS)

2

TOPICS FOR DISCUSSION IMPORTANCE OF MONITORING &

EVALUATIONDEFINITION OF TERMS: Goals, targets, indicatorsTYPE OF INDICATORSMONITORING WHY AND HOW TO MONITORWHY EVALUATIONHOW AND WHAT TO EVALUATE

CAPACITY BUILDING IN M&E

3

Poverty Information systems are part of the Poverty Reduction Strategies

1. POVERTY MEASUREMENT & DIAGNOSTICSUnderstand the features of poverty and the factors that determine it.

2. POLICY ACTIONSChoose priority actions to reduce poverty, in the short- and longer-term.

3. INDICATORS AND MONITORINGSet targets and identify indicators of progress, and systematically monitor and evaluate results and feedback into decision-making.

4

Poverty analysis, monitoring and evaluation at the four following

steps:

Step 1. Policy selection : Situation DescriptionGeneral AdvocacyDetailed Analysis

Step 2. Policy elaboration: Setting GoalsSetting TargetsSelecting Indicators

Step 3. Policy implementation: MonitoringEvaluation

Step 4. Feedback: Transparency and Accountability

Improved Interventions and Future Policies

5

What do we mean by poverty?

Poverty can be seen as deprivation in both material and non-material goods and assets:

Material poverty: Income poverty Consumption poverty

Non-material poverty: Poor health Low education or illiteracy Social exclusion Insecurity Lack of freedom and voice/lack of empowerment

6

Goals, Targets, and Indicators: Definitions

GOALS are broad social objectives set in the PRSP documents or MDG declarations in general terms, such as reducing poverty and improving educational attainments.TARGETS are particular values of goals/objectives that the governments and donors set to achieve by their actions in a specified period of time.INDICATORS are specific empirical measures required to monitor progress towards achieving a goal (such as proportion of population living below the poverty line) or a target (such as increasing literacy rate by certain percentage).

7

Types of indicators

INPUTSResources committed to a project or program.

OUTPUTSGoods and services produced by a project/program.

OUTCOMESVariables that policymakers expect to be directly influenced by project/program interventions

IMPACTSVariables that are likely to be affected by project/program interventions.

8

An example

IF GOAL: Increase well-being of rural women. INPUTS: Financial, human and technical

resources. OUTPUT: Number of rural primary schools,

number of trained female teachers in regular attendance, introduction of better gender-balanced textbooks, etc.

OUTCOMES: Higher enrollment rate of girls in primary schools. Higher literacy rate/mean academic achievement scores among girls in rural primary schools.

IMPACTS: Reduced fertility, higher incomes and increased empowerment of rural women.

9

Another example

IMPACT

OUTPUTS

OUTCOMES

INPUTS

Effect on living standards - infant and child mortality, - prevalence of specific disease

Financial and physical resources - spending in primary health care

Goods and services generated - number of nurses - availability of medicine

Access, usage and satisfaction of users - number of children vaccinated, - percentage within 5 km of health center

10

Monitoring

MONITORING is concerned with tracking implementation of policies or programs, and with observing how the values of different indicators against stated goals and targets change over time.

11

Monitoring

MONITORING questions may relate to monitoring inputs, outputs, and outcomes: questions may include:

Are the public resources received by health clinics in a given region in line with the amounts agreed in the budget?Is net primary school enrollment of girls in rural areas rising?How is consumption-poverty changing over time in urban areas?Is the perception of the poor about their access to public services changing over time?

12

Why monitor?

MONITORING provides basic information as to whether a program, policy, or development strategy is (i) being implemented as planned and (ii) achieving its objectives.

MONITORING is a crucial tool for four purposes:

1. Effective management: If actual implementation diverges from planned implementation, monitoring provides evidence of problems that need to be identified and solved.

2. Policy transparency: Citizens and civil societies are entitled to information concerning policy formulation, financing, and implementation.

13

Why monitor?

3. Democratic accountability: Citizens are entitled to know whether the actual allocation of public expenditure matches the allocation as promised by the government. If there is a mismatch, they are entitled to know why.

4. Feasible target setting: It is very difficult to know what is possible to achieve in the future without knowing what has been possible to achieve in the past. It is only possible to know what has been achieved in the past through monitoring.

14

What to monitor?

It is more useful to identify a small number of indicators, measure them well and use the results for policy-making than to enumerate a long list of indicators, measure many of them badly and not use the results for policy decisions at all.A priority list of outcome indicators might consist of a set of income poverty indices together with some measures of health status and educational attainment.

15

What to monitor?

Once such a list is prepared and disaggregated by sector (rural and urban), region and gender, the number of indicators rises very quickly.Some countries include a long list of indicators in their PRSPs, e.g., Tanzania has more than 100 indicators listed in its PRSP.

16

How to monitor?

Once targets are set and the indicators selected, the challenge is to manage a monitoring system that integrates a variety of different types of information.Types of information relevant to such a system may include:Management Information System (MIS)Surveys and censuses: Quantitative dataParticipatory monitoring exercises: qualitative dataIt is useful to draw up a poverty monitoring matrix which identifies the data sources for each indicator, the frequency of measurement and the organization responsible for collecting the information.

17

How to monitor?

Given the wide varieties of information required, it is important to plan ahead so that information is collected in a sequence and maximizes the complementarities between different types of data.Outputs of the Poverty monitoring System: Annual poverty reduction progress report (required for PRSP process), creation of poverty database, disaggregated spatial poverty map, ad-hoc studies. It is important that these outputs are actually used to monitor policy and feed into the policymaking process.

18

Monitoring system: What limitations?

There are many actors in the information system, performing many activities. BUT, there is typically a series of issues:  

Roles badly defined Lack of coordination between the different actors:

duplication, competition, gaps… Lack of reliability of some information Difficulty in accessing information Lack of relevance of some information Long delays in production of information Lack of use of the data by the users (don’t know about

them, receive them too late, receive a format which is not adapted, lack of confidence, etc.)

19

Monitoring system: What components?

Actors: Data producers Analysis producers Data and analysis users, including: Decision makers Stakeholders (beneficiaries of services, civil society,

media, parliament, etc.)

Activities: Collecting and processing information Data analysis Dissemination and feedback

 

20

Monitoring system: Where to start?

1. Stock-taking: Who are the actors? What are there activities? What is their capacity? What is their role (officially, in practice), what are the limitations, the barriers? What are their needs? Legislative constraints?

2. Review the actions currently undertaken to build or reinforce capacity (projects, international initiatives, etc.).

3. Review of needs in information (in light of the matrix of policies and interventions).

4. Review of the institutional framework

21

Why evaluation?

IMPACT EVALUATION helps identify the causal link between a policy intervention and an outcome to be identified and measured. In contrast, monitoring helps track down whether a policy or program intervention is being implemented as planned and whether the program or policy is achieving its objectives. Assessing impact indicators is more difficult than monitoring inputs, outputs and outcomes. Monitoring and evaluation (M&E) helps policymakers better design programs/policies and enhance their effectiveness. Better M&E also promotes social debate on poverty reduction strategies.

22

Why Evaluation?

Impact evaluation questions include: § Did a labor training program contribute to job

placement? § Did the provision of textbooks contribute to

students’ improved test scores?§  Did a micro-finance project improve household welfare?§ Did the construction of a rural road improve rural welfare?

23

How to evaluate?

Good practice:Impact assessment of a program or policy intervention requires a comparison of outcomes before and after intervention with outcomes with and without intervention. The problem is often with finding an appropriate comparison group to identify the counterfactual.

24

As impact evaluations are demanding activities, these should be done only if the program intervention warrants an impact assessment. Three questions can help decide when to conduct an impact evaluation.Is the policy or program considered to be of strategic importance (such as having highest poverty impacts) for poverty reduction? Will the evaluation of a program or policy contribute to our knowledge gaps of what works and what does not in poverty reduction?Is the policy or program testing an innovative approach to poverty reduction?

What to evaluate?

25

Concluding remarks on poverty information systems

Monitoring POVERTY indicators (inputs, outputs, and outcomes) must be well-defined and smaller in numbers. Both quantitative and qualitative methods may be used to monitor input-output- outcome indicators of interventions Monitoring is easier and less expensive than evaluating impact indicatorsNo method of impact assessment is perfect -- conceptualize the problem and use the most appropriate method, given resources and time.Impact evaluation is more difficult with economy-wide interventionM&E is a tool for learning whether and how programs matter, and requires planning and resource allocation for its better use.

26

Development profile Public interventions

National Development Strategy

Monitoring outcomes

Demand for information

Supply of Information

LegislationStructure Management

National Statistical System

Resources Technical capacity

Evidence based policymaking

Effective Poverty Reduction

Accountability Transparency

Setting Priorities

Measurement/Diagnostics

Poverty Analysis

Monitoring Evaluation

Supply of data

Demand for data

Analytical support

Information Strategy for Effective Poverty Monitoring

and Evaluation

27

Identify capacity building needs

Data management in the MIS of line ministries.Household survey design and implementation.Poverty measurement and diagnostics.Skills for PPAs and participatory monitoring.Management skills to coordinate overall poverty monitoring and evaluation systemPoverty impact evaluation.