Budget support training

Module 8Implementation, monitoring and evaluation

of a BS programme

Version October 2013

2

Outline

1. BS monitoring: a continuous process

2. Budget support policy dialogue

3. Coordination

4. Evaluation

3

Monitoring the implementation of BS contractsRegular monitoring and policy dialogue are key elements of all BS operations, they should:

Be conducted according to the new governance mechanisms Take into account pro-actively the fundamental values

Focus on: Promoting domestic ownership and accountability Strengthening team work within EU Delegations Strengthening coordination across all budget support providers Paying attention to appropriate communication and visibility

activities

4

Budget support cycle of operations

Programming

Identi-

fication

Formulation

Implementation

Evaluation and Follow-up

Evaluation: focus on joint evaluations

Decision process for tranche release:

Monitoring and dialogue, assessment of payment conditions

oQSG2 - Action Fiche:

focus on eligibility + context+ conditions. Supporting documents, financing agreement and TAPs

oQSG1 -

Identification Fiche:

focus on eligibility criteria + next steps

EC Development Policy

Partner Government Policy

National Indicative Programme:

identify sectors of engagement

BSSC: i) agreement in principle on aid modality; ii) assessment of fundamental values

Budget Support Steering Committee (BSSC): may review budget support financing proposals and disbursement decisions wherever there are substantial or high political and policy implications.

Follow up and data collection

Notes from Central Bank, Min. Fin.,..

Economic & social indicators

IMF/WB Reports Enacted budget &

annexes Report on budget

execution Assessment of PFM

(PEFA, PER) OBI Monitoring & eval. of

reforms Human rights strategy International sources

Running Head 12-Point Plain, Title Case

Analysis

Macroeconomic framework Progress in implementing

policies Relevance/credibility of the

policies Progress of PFM reform Budget execution Availability/accessibility of

budget documents Monitoring of performance

indicators, achievement of results, specific objectives.

Evolution of risks Monitoring of risk mitigating

measures

Payment file/annual report

Verify continued relevance/credibility of policies

Macroeconomic assessment

Annual report on PFM

Assessment of progress in transparency

RMF Matrix of

performance indicators

Monitoring a BS programme:

a continuous process

POLICY DIALOGUE5

What to monitor?

For GGDC: the continued commitment to the fundamental values (pre-condition for GGDC)

For all contracts: The continued eligibility under the 4 eligibility criteria The progress towards the specific objectives of the contract and

the results achievedThe evolution of the risks The progress in the implementation of the mitigating measures The needs and progress of the partner country in terms of

capacity development in the areas of importance for the BS programme

•

6

Capacity development

Capacity development needs must be assessed systematically:Is the existing capacity sufficient to absorb the support programme? Is capacity development based on adequate demand, ownership and

commitment? Is it provided through harmonized and aligned approaches?

Areas of particular importance for capacity development:Public Finance ManagementTax reforms, tax administrationPublic expenditures reviews and tracking surveysPublic sector reforms and public administrationQuality and capacity of statistical systems National control bodies (Parliament, Supreme audit institution, judiciary bodies,

internal audit and control institutions, civil society organisations, local authorities and media)

•7

Decision process (6 steps) for tranche release

Partner country 1) Submits its request for tranche release, with justification (may be cover letter + annual review)

Running Head 12-Point Plain, Title Case

EU Delegation 2) Analyses the request and prepares “payment dossier”.

Are conditions for disbursement fulfilled? Clear recommendation of HoD on whether and how much

should be disbursed

DEVCO

3) DEVCO Geographical services prepares dossier for approval of the relevant Director

in line with the procedures agreed in the BS potentially involving BSSC

4) Geographical Director signs « payment decisions » and sends it to DEU with key messages to be relayed to partner country

EU Delegation 5) pays, with formal letters with key messages6) verifies payment (and exchange rate)

8

9

Outline

1. BS monitoring: a continuous process

2. Budget support policy dialogue

3. Coordination

4. Evaluation

10

BS Policy dialogueIs a key input of BS programmes and the centre piece of mutual accountability.It provides a framework for:

taking stock of partner country’s policies and reforms as well as of the donors’ commitment

assessing progress on both sides on the basis of different information, criteria and indicators and of extensive discussion with the government

It can be used as a forward looking tool for: identifying policy slippages timely reaching common understanding with the authorities on corrective

measures and refining objectives

11

BS Policy dialogue: coverageThe BS policy dialogue should cover:the process of the programme:

Formal meetings with government and stakeholders, reviews of programme, etc.

Informal contacts the content:

National and sector policies Performance indicators Capacity building

Should go beyond the specific elements of the BS operation and address the wider development context

12

BS Policy dialogue: adapted to each type of contract (1)

GGDC: Dialogue on wide range of policies and reforms:

Mobilisation and allocation of resources across the whole government

Efficiency and effectiveness of core government systems

Aimed at: Improving formulation and implementation of national policies Improving accountability Improving service delivery Promoting good governance

13

BS Policy dialogue: adapted to each type of contract (2)

SRC

Dialogue on sector specific issues Key performance indicators of the sector (input, process, outcome) Access and quality of sector delivery

Aimed at Improvement of formulation and implementation of sector policies Improvement of efficiency and effectiveness of Government sectoral

delivery structures and systems

SBC

Dialogue on Overall, coherent and coordinated strategy as regards vital state

functions Provision of basic services to the population Transition toward development and democratic governance

14

An informed dialogue

Policy dialogue should be an informed dialogue:Drawing on the assessments of the eligibility criteria and the risks; Requiring an understanding of the actors, rules and

mechanisms governing the decision making process in the partner country ….

…… and an understanding of the country-donor coordination framework at national and subnational levels of the government.

15

Outline

1. BS monitoring: a continuous process

2. Budget support policy dialogue

3. Coordination

4. Evaluation

16

Donor coordination Nature of donor coordination: Government-led, formal/informal,

participation and roles Existence of common framework and its functioning:

harmonisation without unanimity How does the donor-partner country coordination work?

Existence of a Budget Support Group Leading role of Government Predictability of donor funding Existence of a common Performance Assessment Framework

(PAF) and use of disbursement criteria Existence of complementary initiatives (training, TA,...)

How does the donor-donor coordination work?

17

EU CoordinationObjective of EU coordinated approach to BS: Increase effectiveness of BS Provide coherent and consistent response to challenges

Guiding principles: Sovereign decision making Systematic information exchange Supportive of wider donor coordination and country-led

coordination Involves both HQ and field levels Builds on existing bodies, mechanisms of cooperation, and

existing legal frameworks

18

Outline

1. BS monitoring: a continuous process

2. Budget support policy dialogue

3. Coordination

4. Evaluation

4.1 Objectives and methodology of a BS evaluation

To assess to what extent and under which circumstances the BS has

successfully enhanced the policies, strategies and spending actions of

the partner government so as to achieve sustainable national and/or

sector level development outcomes and positive impacts on poverty

reduction and economic growth.

Joint exercise of BS donors and recipient country Improved methodological approach just been developed. Evaluations need to be prepared right from the start of a BS

contract: collection and recording of baseline information and

data, and setting up a good monitoring and reporting system.

19

The comprehensive evaluation framework

Based on intervention logic of BS: hypothesised sequence of effects. Central notion: BS is a contribution to the implementation of the

policy and spending actions of a partner government. The Comprehensive Evaluation Framework (CEF) consists of five

levels: See page 10 of BS Guidelines. Many external (non-BS) factors determine induced output, outcome

and impact. Broad assessment of outcomes and impact of government policies

needed. Specific CEF needed for a joint GBS programme and each joint SBS

programme.

20

21

22

Government strategy

Impacts Sustainable and inclusive growth Poverty reduction Budget support

Outcomes

Positive responses by service users and economic actors to government policy management and service delivery

Outputs

Execution of the budget Monitoring of results Improved macro-economic management Improved public services Etc.

Induced outputs

Improved macro-economic management Improved public services Strengthened PFM Improved policy formulation Strengthened public sector institutions, Etc.

Inputs

National and sector policies Budgetary resources Human resources Institutional structures Strategies and operational programmes

Outputs

More aid provided through the budget More funds available for discretionary spending Increased predictability of disbursement of

external funds Policy dialogue, conditionalities, TA and

capacity building better coordinated and more conducive for government strategies

Improved harmonisation and alignment Reduced transaction costs

Inputs

Transfer of funds to the national Treasury and disbursement conditions

Policy dialogue and performance indicators Capacity building and TA

Logical frameworks of the partner government action and of the BS programme

Main driving forces

Flow of funds effect: Effects of additional government spending; Effects of pushing donor funds through the PFM

system. Policy and institutional effects, via:

Policy dialogue;Monitoring performance indicators;Disbursement conditions, and;Capacity development activities.

23

The three step approach.

Analysis of direct causality possible up to level 3. At level 4 and 5 too many other influencing factors. Step 1: assessment of inputs, direct outputs and induced

output, and analysis of causality. Step 2: assessment of outcomes and impact of government

policies and spending. Step 3: Exploration of contribution of BS to the government

policies and spending, which have produced the outcomes

and impact.

24

25

Government strategy

Impacts Sustainable and inclusive growth Poverty reduction Budget support

Outcomes

Positive responses by service users and economic actors to government policy management and service delivery

Outputs

Execution of the budget Monitoring of results Improved macro-economic management Improved public services Etc.

Induced outputs

Improved macro-economic management Improved public services Strengthened PFM Improved policy formulation Strengthened public sector institutions, Etc

Inputs

National and sector policies Budgetary resources Human resources Institutional structures Strategies and operational programmes

Outputs

More aid provided through the budget More funds available for discretionary spending Increased predictability of disbursement of

external funds Policy dialogue, conditionalities, TA and

capacity building better coordinated and more conducive for government strategies

Improved harmonisation and alignment Reduced transaction costs

Inputs

Transfer of funds to the national Treasury and disbursement conditions

Policy dialogue and performance indicators Capacity building and TA

Step 1: Evaluation of the chain from inputs to induced outputs of the BS

Step 1

26

Government strategy

Impacts Sustainable and inclusive growth Poverty reduction Budget support

Outcomes

Positive responses by service users and economic actors to government policy management and service delivery

Outputs

Execution of the budget Monitoring of results Improved macro-economic management Improved public services Etc.

Induced outputs

Improved macro-economic management Improved public services Strengthened PFM Improved policy formulation Strengthened public sector institutions, Etc

Inputs

National and sector policies Budgetary resources Human resources Institutional structures Strategies and operational programmes

Outputs

More aid provided through the budget More funds available for discretionary spending Increased predictability of disbursement of

external funds Policy dialogue, conditionalities, TA and

capacity building better coordinated and more conducive for government strategies

Improved harmonisation and alignment Reduced transaction costs

Inputs

Transfer of funds to the national Treasury and disbursement conditions

Policy dialogue and performance indicators Capacity building and TA

Step 2: Evaluation of the results and impacts of the Government action

Step2

27

Government strategy

Impacts Sustainable and inclusive growth Poverty reduction Budget support

Outcomes

Positive responses by service users and economic actors to government policy management and service delivery

Outputs

Execution of the budget Monitoring of results Improved macro-economic management Improved public services Etc.

Induced outputs

Improved macro-economic management Improved public services Strengthened PFM Improved policy formulation Strengthened public sector institutions, Etc

Inputs

National and sector policies Budgetary resources Human resources Institutional structures Strategies and operational programmes

Outputs

More aid provided through the budget More funds available for discretionary spending Increased predictability of disbursement of

external funds Policy dialogue, conditionalities, TA and

capacity building better coordinated and more conducive for government strategies

Improved harmonisation and alignment Reduced transaction costs

Inputs

Transfer of funds to the national Treasury and disbursement conditions

Policy dialogue and performance indicators Capacity building and TA

Step 3: Exploration of the links between the BS and the results and impacts of the Gov. action

Step 3

4.2 Implementing a BS evaluation (1/2): preconditions Objectives, strategies and performance indicators of the BS need to

be clearly defined at the start of the BS programme. Implementation needs to be well monitored and documented, and

reporting well organised. Align timing of BS evaluation with other supportive studies and vice

versa. Start the preparation of the BS evaluation timely: 2 years before the

report has to be finalised. Involve and strengthen evaluation departments of MinFin and – in

case of SRC – sector ministries. Align scope and level of detail of the evaluation with availability of

data and information (and available budget for the evaluation).

28

4.2: Implementing a BS evaluation (2/2) : Phases and activities

Preparation: drafting ToR, establishment of management and

reference groups, recruitment of consultants. Five phases: Inception, data collection and preliminary analysis, in

depth analysis and verification, synthesis and dissemination. The phases will take 12-18 months and require 350 to 450 person

days. Alternatively: steps could be implemented at different times. Step 1 could be carried out more often than steps 2 and 3.

29

4.3 Key points

Evaluation of BS should be carried out every 5-7 years Preparation of a BS evaluation should start when the BS

programme is being formulated. Data collection and background studies and surveys should

become part of the implementation process. Partner government and donors should take joint

responsibility for the BS evaluation. Agreement to carry out a joint evaluation (and its timing)

should be mentioned in the TAP/financing agreement.

30