Anti-corruption and Result-Based Management (RBM)UNDP

2012

In plain language…

RBM helps us to connect what we do to what we want to achieve

RBM also tells us how we’ll know if we’ve achieved it

What is results based management?

A management strategy that aims at ensuring:• that activities achieve desired results

(Performance monitoring is a critical element)• How well results are being achieved• What measures are needed to improve the process

Results

Goal

Impact

Outcome

Outputs

Activity

Indicator

RBM Terminology (UNDG Approved)

Changes in a state or condition which derive from a cause-and-effect relationship. There are three types of such changes (intended or unintended, positive and/or negative) which can be set in motion by a development intervention – its output, outcome and impact.

The higher-order objective to which a development intervention is intended to contribute

Positive and negative long-term effects on identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental, technological or of other types

The intended or achieved short-term and medium-term effects of an intervention’s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact.

The products and services which result from the completion of activities within a development intervention

Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs

A tool to measure evidence of progress towards a result or that a result has been achieved.

Advantages of RBM

• Improved focus on results instead of activities

• Improved transparency

• Improved accountability

• Improved measurement of programme achievements (performance rather than utilization)

• Enhanced strategic focus

Challenges

• Difficult to apply causal logic

• Difficult to learn

• Difficult to integrate

• Difficult to revise (... or reluctance to revise? )

• Difficult to measure

• Difficult to ‘attribute’ (at outcome level, the UN is partly accountable but not wholly

responsible)Go to typology

Linking analysis with results

Impact: (e.g., corruption reduced)

Outcome: (e.g., improvements in the performance of government institutions as well as monitoring my citizens)

Output: (e.g., tangible changes in any elements in the capacity of government institutions and population (information, attitudes/motivation, risks, resources)

Linking Analysis to results:

Translating the problem tree into results

Causality Analysis Results chain

(Problem tree) (UNDAF matrix)

Immediate causes Impact (long-term)

Underlying causes Outcomes (medium-term)

Root causes Outputs (short-term)

1. To effectively analyze problems and their causes, using evidence and lessons including key institutions/capacity gaps

4. To establish performance monitoring and evaluation systems (including risk management)

5. To capture and monitor actual results vis-à-vis targets

6. To clearly identify issues/bottle necks, generating lessons regarding what works/ what doesn’t work

2. To develop right indicators that measure capacity changes and performance improvements

3. To determine baseline and setting targets

Linking analysis with results: Why results frameworks are relevant for anti-corruption in UNDAF?

UNDAFs?

efficiency

effectiveness

Impacts

Outcomes

Outputs

Activities

Inputs

RESULTS

INFLU

ENCE

Performance indicators

Pla

nn

ing

Mo

nito

ring

Linking Analysis with Results: The Results Chain

Result Chain – Quick Recap

Inputs

Activities

Outputs

Outcomes

Impact

Indicator

Resources available - money, staff, facilities, equipment, technical expertise

What is done with the inputs - holding seminars, producing manuals

Services or products produced as a result of activities – 100 staff trained

s Changes, effects due to the activities and outputs – higher skill levels

Long-term results – quality of government services, less corruption

Measure progress towards a result over time that indicates positive, negative, no change with respect to progress towards a stated target

Corruption and Anti-Corruption Indicators

What is an indicator?

A tool to measure evidence of progress towards a result or that a result has been achieved.

Types of Indicators

Quantitative statistical measures:

• Number of• Frequency of• % of• Ratio of• Variance with

Qualitative judgments or perceptions:

• Alignment with• Presence of• Quality of• Extent of• Level of

Considerations

• Indicators do not exist in a vacuum – they should always be related to results

• Need a balance of quantitative and qualitative• Some results are more suitable for indicators than

others• It takes time to get indicators right

A Key Challenge:

Is it possible to meaningfully measure corruption, which has no universal definition, but has different forms, typologies, manifestations, determinants, causes and symptoms?

“What gets measured, gets managed.” - Peter Drucker. So, the UN should demonstrate results, results and results.

The mushrooming industry of indicators: Trying to measure perception, impact,

existing gaps, integrity, enabling environment, etc.

80 90888682 84 96 98 00 02 0492 94 06

International Country Risk Guide

Corruption Perception Index

GovernanceMatters

7876

CPIA (WB)

1974

Freedom in the World

Afrobarometer

Bertelsmann Transformation

Index

Bribe Payers Index

BEEPS

CIRIHuman Rights

Database

Commitment to Development

East Asia Barometer

GAPS in Workers’ Rights

Gender Empowerment

Measure

Eurobarometer

Global Accountability Report

Global Competitiveness

Index

Global Integrity Index

Index of Economic Freedom

Journalists killed

Media Sustainability

Index

Opacity Index

Open Budget Index

Polity

Press Freedom

World Governance Assessment

Are there enough tools and methodologies?

YES

Are these existing corruption and anti-corruption indicators helpful?

It depends -

Recall considerations

Understanding the tools and indicators

Global Indices:

CPI; WB Governance Indicators, Freedom House Indicators

Regional indices: Latin Barometer; Afrobarometer

Country level indices

Understanding corruption and anti-corruption tools and indicators (contd.)

Corruption Transparency/Accountability/Integrity

Diagnostic Assessments

Institutions

Processes

Sectors

Local level

Compliance monitoring

Perception

Experience/ victimisationPublic

opinion

Experts

Public sector

General population / vulnerable groups

Public sector

Private sector

Mapped by Transparency International

“Not everything that counts

can be counted, and

not everything that can be

counted counts.”

– Albert Einstein

Challenge is to find the right indicators!

Use and misuse of indicators for RBM

1. No indicator is perfect and thus standard; all have advantages and disadvantages. However, global perception or cross-country indicators are not useful for tracking progress at the country level.

2. In terms of quality and usefulness of data, it is advisable that the country-based and nationally-owned corruption measurement/assessment indicators be used.

3. Similarly, experience-based indicators (victimization surveys) are advisable over the perception indices. However, victimization surveys data are expensive, too.

1. The perception data (e.g., score cards) at the country level has been often used as a proxy.

2. If the country level data/indicator are not available, regional surveys such as Afrobarometer or Latin Barometer can also be useful.

3. The selection of indicator mainly depends on anti-corruption outcomes and outputs, availability of data and the cost associated with gathering data.

DOs and DON’Ts of capturing adequate or satisfactory results

Bad Good

No supporting indicators, programme logic or data

Increase evaluability of anti-corruption projects, prospective evaluations

Weak, activity-oriented indicators Focus on a few good results oriented (outcome and output) indicators, and disaggregate data

Major cause-effect (attribution) leaps from output to impact level

Stop reliance on macro-level global indices (CPI) and develop country level indicators

No evaluation/logical framework to capture results

Mixed methods. Pragmatic approach to statistical design (best fit) and strong qualitative support

Worst: Claiming impact when there is no result chain or framework

Adapt impact evaluation methods to characteristics of anti-corruption projects

Performance Indicator Selection Criteria

• Validity - Does it measure the result?

• Reliability - Is it a consistent measure over time and, if supplied externally, will it continue to be available?

• Sensitivity - When a change occurs will it be sensitive to those changes?

• Simplicity - Will it be easy to collect and analyze the information?

• Utility - Will the information be useful for decision-making and learning?

• Affordable – Do we have the resources to collect the information?

Illustrative example of the results chain of an Anti-corruption initiative (adapted from a country example)

Stakeholder dialogue regarding corruption issues, review of current audit mechanism and gaps, assessment and development of the capacities of supervisory body, budgetary and funding support; etc.

(money, staff, facilities, equipment, technical expertise)

Government supervisory functions and audit processes are established

Anti-corruption institutions/ supervisory bodies are enforcing the law

Media has channels to publish corruption related information

Information regarding incidences of corruption is accessible to the public

Public government officials, citizens, etc. have reduced their engagement in corruption practices

Reduced corruption and increased public trust towards

public institutions

Citizens have mechanism to report on corruption incidences

Government agencies are exchanging information about corruption incidences

Corruption data management system is put in place

Knowledge, Innovation and Capacity Group, BDP, UNDP (April, 2013)

Inputs

Activities

Outputs

Outcomes

Impact

UNDAF’s Results Matrix

• Currently many UNDAFs restrict the aspect of anti-corruption to the objective of governance / human rights / democracy. Anti-Corruotion is hardly an output in case of MDGs, environment, conflict prevention, disaster reduction, etc.

• Although corruption is identified in some of the UNDAFs documents, the RBM is very weak.

Anti-corruption in UNDAF’s

Anti-corruption in UNDAF’s

Option 1 A: UNDAF RESULT MATRIC WITH OUTCOME LEVEL ONLY (Mongolia)

Strategic priority - Governance and Human rights

Outcomes Indicators, Baseline, TargetOutcome 1: Representation, accountability and transparency of governing institutions strengthened

 

Output 1.1. Enabling policy environment created for effective decentralization and increased functional capacity of local governments to deliver service.

 

Capacity of local governments to deliver services

Output 1.2.

Increased capacity to implement the UN Convention Against Corruption

 

Compliance with UNCAC provisions on corruption prevention

Output 1.3. Increased civil society participation in key national processes and strengthened state-citizen engagement for accountable and responsive governance

 

Feedback mechanism of state-CSOs-state Social dialogue between government, workers and employers

Bad or good?

What is the underlying vision behind the reform/policy? What are the objectives pursued? Which results do we expect to see?

What is the logic underpinning the reform/policy design? (from inputs to outputs, outcomes and impact)

What are the preconditions for success? Which behavioural assumptions are being made? How will political, cultural or economic factors affect potential for change?

Key questions you need to consider when formulating the results matrix

Results Matrix

Reduction in corruption

Corruption cases are

investigated to required standard

Substantiated cases are sent

to court and prosecuted

Other law enforcement

inst. cooperate.

Witnesses are willing to come forward

Sufficient resources

and mandate

Donor demands

for outputs

Legislative

obstacles

Result Chain with Risks and Assumption

Group Work 2 (15 minutes)

1. Put together a coherent, logically sequenced result chain to introduce a Code of Conduct (CoC) in ministries (yellow pieces)

2. Identify where indicators fit at the right level of results in the chain.

Inputs Activities Outputs Outcome Impacts

Output indicators

Outcome indicators

Impact indicators

Key messages

• Corruption and anti-corruption measurements and assessments provide useful information for country analysis, which is an integral part of UNDAF process.

• Corruption and anti-corruption measurements and assessments also strengthen results by providing indicators to measure, monitor, and report on change.

• However, it is also important to make sure that the appropriate indicators are used to measure progress and results.