Document of The World Bankdocuments.worldbank.org/curated/en/523401468030256322/pdf/No… · Capacities and procedures for strategic expenditure planning, budget presentation and

Document of The World Bank

Report No: ICR2445

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H2050 TF-26116)

ON A

CREDIT

IN THE AMOUNT OF SDR 2.1 MILLION (US$ 3.0 MILLION EQUIVALENT)

TO THE

GOVERNMENT OF GEORGIA

FOR A

PUBLIC SECTOR FINANCIAL MANAGEMENT REFORM SUPPORT PROJECT

November 27, 2012

Poverty Reduction and Economic Management Unit Europe and Central Asia

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective as of November 13, 2012)

Currency Unit = Lari (GEL) GEL 1.00 = US$ 1.6643 US$ 1.00 = GEL 0.60

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APST Administration and Procurement Support Team

AWP Annual Work Plan BCC Budget Classification Codes BDD Basic Data and Directions CCG Chamber of Control of Georgia CFAA Country Financial Accountability

Assessment CM Component Managers CoA Chart of Accounts CPS Country Partnership Strategy DFID UK Department for International

Development DP Development Partner ECA Europe and Central Asia

EDPRP Economic Development Poverty Reduction Program

EU European Union FAS Financial Analytical Service GDP Gross Domestic Product GoG Government of Georgia GTZ Geselleschaft fur Technische

Zusammenarbeit (German Technical Assistance)

HRMIS Human Resources Management and Information System

ICR Implementation Completion Report IDA International Development Association IPSAS International Public Sector

Accountability Standards ISR Implementation Status Report IT Information Technology IUFR Interim Unaudited Financial Report M&E Monitoring and Evaluation

MDG Millennium Development Goals MOF Ministry of Finance MTEF Medium-Term Expenditure Framework NBG National Bank of Georgia OED Overall Evaluation Department PDO Project Development Objective PEFA Public Expenditure and Financial

Accountability PFM Public Financial Management PFMIS Public Financial Management

Information System PHRD Policy and Human Resources

Development (Grant) PIP Project Implementation Plan PIU Project Implementation Unit PIT Project Implementation Team PLE Public Legal Entity PMC Project Management Committee POM Project Operational Manual PRSO Poverty Reduction Support Operation PSB Public Service Bureau PSFMRSP Public Sector Financial Management

Reform Support Project SAO State Audit Office SIDA Swedish International Development

Cooperation Agency SWAP Sector-Wide Approach TA Technical Assistance TIS Treasury Information System TSA Treasury Single Account UNDP United Nations Development Program USAID United States Agency for International

Development

Vice President: Philippe Le Houerou

Country Director: Henry Kerali

Sector Manager: William Dorotinsky

Project Team Leader: Elene Imnadze

ICR Team Leader: Elena Nikulina

COUNTRY

Georgia

Public Sector Financial Management Reform Support Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ................................................... 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 4 3. Assessment of Outcomes ................................................................................................ 9 4. Assessment of Risk to Development Outcome ............................................................. 14 5. Assessment of Bank and Borrower Performance ......................................................... 14 6. Lessons Learned............................................................................................................ 16 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............... 17 Annex 1. Project Costs and Financing .............................................................................. 19 Annex 2. Progress in Achievement of Intermediate Results ............................................ 20 Annex 3. Economic and Financial Analysis ..................................................................... 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 29 Annex 5. Beneficiary Survey Results ............................................................................... 31 Annex 6. Stakeholder Workshop Report and Results ....................................................... 31 Annex 7. Summary of Borrower's ICR ............................................................................. 32 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 37 Annex 9. List of Supporting Documents .......................................................................... 37 MAP

A. Basic Information

Country: Georgia Project Name: Public Sector Financial Management Reform Support

Project ID: P063081 L/C/TF Number(s): IDA-H2050,TF-26116

ICR Date: 11/27/2012 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF GEORGIA

Original Total Commitment:

XDR 2.10M Disbursed Amount: XDR 0.90M

Revised Amount: XDR 0.90M

Environmental Category: C

Implementing Agencies: Ministry of Finance State Audit Office Cofinanciers and Other External Partners: Government of the Netherlands SIDA DFID

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 09/23/2005 Effectiveness: 08/03/2006 08/03/2006

Appraisal: 12/19/2005 Restructuring(s): 02/25/2010 02/29/2012

Approval: 02/16/2006 Mid-term Review: 01/20/2009

Closing: 03/01/2010 03/01/2012

C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory

Government: Moderately Satisfactory

Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:

Moderately SatisfactoryOverall Borrower Performance:

Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing)

Central government administration 100 100

Theme Code (as % of total Bank financing)

Administrative and civil service reform 29 14

Other accountability/anti-corruption 28 32

Other public sector governance 14 14

Public expenditure, financial management and procurement

29 40

E. Bank Staff Positions At ICR At Approval

Vice President: Philippe H. Le Houerou Shigeo Katsu

Country Director: Henry G. R. Kerali D-M Dowsett-Coirolo

Sector Manager: William Leslie Dorotinsky Deborah L. Wetzel

Project Team Leader: Elene Imnadze Elene Imnadze

ICR Team Leader: Elena Nikulina

ICR Primary Author: Elena Nikulina

William V. Mayville

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The development objective of the Public Sector Financial Management Reform Support Project (PSFMRS) is to enhance governance, particularly in the public financial management domain, through: i) strengthening the institutional capacity of key agencies to more effectively and efficiently use public resources; and ii) improving accountability in the use of public resources. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : National expenditure priorities fully defended in budget. MTEF increasingly based on fully costed, results-based sector strategies consistent with fiscal forecasts (PEFA I-12 (iii))

Value quantitative or Qualitative)

All executive branch spending units are currently providing MTEF submissions, but of varied quality and based on varied quality sectoral strategies.

Expenditure strategies for executive branch spending units exist covering at least 70% of executive branch expenditures, including fully costed recurrent and investment expenditures and results indicators, and in line with fiscal framework

Costed expenditure strategies included in 2011-2014 BDD covered about 88% of executive branch expenditures as approved by 2011 Budget Law; 2012 State budget was prepared following program budget format

Date achieved 12/19/2005 03/01/2012 03/01/2012 Comments (incl. % achievement)

Achieved PEFA I-12 (iii) rating measuring existence of costed sector strategies is expected to be upgraded from C in 2008 to B in 2012.

Indicator 2 :

Ministry ceilings contained in Budget Guidelines consistent with multi-year function/sector estimates approved by BDD. Differences between actual estimates and approved BDD explained in annual budget presentation (PEFA I-12)


MTEF process has only recently started so no opportunity for these explanations.

Progressive reduction in variance between BDD function/sector allocations, related ministry ceilings and allocations contained in annual budget. Explanation of variance in BDD and annual budget presentation.

Ministry ceilings for FY11 budget consistent with respective BDD estimates. Starting 2012 annual budget law, the annex to the budget contains multi- year (1+3) estimates per budget program. Annual budget planning is in full compliance with midterm f-work

Date achieved 12/19/2005 03/01/2012 03/01/2012

Comments (incl. % achievement)

Achieved Multi-year perspective in fiscal planning, expenditure policy and budgeting improved, as measured by PEFA I-12. The respective rating is expected to improve from C+ in 2008 to B+ in 2012.

Indicator 3 : Consolidated government financial statement published annually based on submission of fiscal reports from all major public entities (PEFA I-25)


Government does not receive regular or complete reports from major public entities for inclusion in budget or compilation of consolidated government financial statement.

Consolidated government financial statement published annually based on submission of fiscal reports from all public entities included in the general government sector.

Consolidated government statement is published annually in the form of annual budget execution report. There are some deficiencies in consistency of presentation and there is no clarity on the accounting standards used.


Not achieved. No improvement in PEFA I-25 is expected in 2012 (2008 rating was D+).

Indicator 4 : Measurable improvement in HR management function of Ministry of Finance


Inadequate personnel management capabilities, policy and procedures. Existing computerized HRMIS does not satisfy increasing demand of HR Management Department and is not connected with the payroll.

Revised HR procedures and mechanisms in place. Fully functional automated HRMIS with personnel system updated monthly and reflected in the payroll.

Comprehensive personnel database for all staff of the MOF system (5,000 records) is created and supported through the functioning software. The database is updated monthly and linked to the payroll module.


Achieved

Indicator 5 : External audit share of central government spending on which on-site audits are completed annually and disclosed to legislature within 4 months of end of period covered (PEFA 1-26)


On-site annual audits cover 15% of central government entities.

On-site annual audits cover 30% of central gov. entities.

For 2011, the financial on-site audits covered 37% of the central government annual expenditures.


Achieved The scope and nature of external audit improved, sa measured by PEFA I-26. The respective rating is expected to be upgraded from D in 2008 to B in 2012.

Indicator 6 : Timely, transparent and user-friendly provision of public financial information to public (PEFA I-10)


Ad hoc publication of public financial information for general public.

Web publication of annual budget, mid-year budget execution reports, consolidated government financial statement and Chamber of Control audit reports in a user-friendly format for

All the types of informaton envisaged are published. The government now discloses to the public five out of six types of fiscal information required by PEFA I-10.

public information.


Public access to main fiscal information improved, as measured by PEFA I-10. The respective rating is expected to be upgraded from B in 2008 to A in 2012

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Capacities and procedures for strategic expenditure planning, budget presentation and budget management are established in Mof and strengthened in ministries. [PEFA I-11(ii)]

Value (quantitative or Qualitative)

Ministries' annual submissions follow 2 our of 7 forms approved by MoF circular; new circular prepared by MoF

Comprehensive and clear budget circular passed to MDAs, reflecting ceilings approved by BDD prior to circular's distribution

MTEF established and MTEF elements function through BDD. Ministry ceilings for FY11 were consistent with BDD estimates. Program budget methodology developed and applied to 2012 State Budget.

Date achieved 12/19/2005 03/01/2012 03/01/2012

Comments (incl. % achievement)

Achieved. PEFA I-11 A rating maintained in 2012 for all dimensions. NOTE: Intermediate indicators included in this table present a subset of intermediate indicators included in ISRs. ICR annex 2 contains full information on intermediate results.

Indicator 2 : Modern accounting standards adopted and in use. [PEFA I-25]


Accounting Reform Strategy under development.

IPSAS or corresponding international accounting standard (GFS2001/ESAF)applied for all public entities.

Based on analysis undertaken, timeframe for developing accounting standards aligned with IPSAS extended. Temporary regulation on financial reporting aligned with

modified cash based IPSAS piloted in 2011.


Not achieved. No improvement in PEFA I-25 expected in 2012(2008 rating was D+)

Indicator 3 : Cash management procedures in Treasury and spending agencies. [PEFA I-16]


TSA established but coverage and content of the system underdeveloped. Cash management procedures are underdeveloped.

Cash flow forecast is prepared for the fiscal year and updated at least quarterly on the basis of actual cash inflows and outflows.

Cash flow forecast prepared by the Treasury based on a contract payment schedule, fixed expenditure pattern and empirical data. No frequent changes made in the plans.


Achieved. Predictability in the availability of funds for commitment of expenditures improved, as measured by PEFA I-16. The respective rating is expected to improve from B+ in 2008 to A in 2012.

Indicator 4 : An integrated Public Financial Management Information System (PFMIS) is operational.


No PFMIS in place.

An integrated PFMIS is implemented (see PAD Annex IV Component 2 for detailed list of modules to be implemented).

PFMIS is being developed in-house. More than half of the system modules are implemented.


Not achieved. Initial results of in-house development are encouraging, however there is uncertainty whether the expected quality and level of integration are achievable through the current approach.

Indicator 5 : Supreme Audit Institution in place with supporting legal framework, and functional.


Chamber of Control Corporate Development and Reorganization Implementation Plan endorse by the Presidium.

Chamber of Control converted legally and operationally to Supreme Audit Institution.

New legislation on SAO adopted on December 26, 2008.Law on SAO amended on December 24, 2011 to bring it further in line with INTOSAI and Lima

Declaration. New auditing procedures in place.


Achieved.

Indicator 6 : Staff capacity for audits: (a) % CoC auditing staff have been adequately trained: (b) % of CoC non-auditing staff have completed relevant courses.


(a) Approx 10% (b) Less than 5%

(a) 90% (b) 15%

In September 2011, 105 Tbilisi-based auditors trained and tested on new financial audit methodology of whom only 45 passed exam and retained and remainder let go. Further 35 staff from Kutaisi and Batumi trained and tested. 20 new staff recruited.


Partially achieved. Calculation of the required percentages is impractical, taking into account high staff turnover during the recent years,

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 05/25/2006 Satisfactory Satisfactory 0.00 2 10/11/2006 Satisfactory Moderately Satisfactory 0.00 3 01/26/2007 Satisfactory Moderately Satisfactory 0.00 4 06/25/2007 Satisfactory Moderately Satisfactory 0.16 5 06/27/2008 Moderately Satisfactory Moderately Satisfactory 0.16

6 05/05/2009 Moderately

Unsatisfactory Moderately

Unsatisfactory 0.16

7 06/24/2009 Moderately Satisfactory Moderately Satisfactory 0.16 8 01/14/2010 Moderately Satisfactory Moderately Satisfactory 0.36 9 10/25/2010 Moderately Satisfactory Moderately Satisfactory 0.36

10 06/30/2011 Moderately SatisfactoryModerately

Unsatisfactory 0.36

11 12/28/2011 Satisfactory Moderately Satisfactory 0.36

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

02/25/2010 MS MS 0.36 Closing date extension

02/29/2012 S MS 0.48 Cancellation of IDA grant balance

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative)

1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) Country background Following the declaration of independence from the Soviet rule in 1991, the Republic of Georgia undertook a series of structural reforms to move toward a new state model. These included measures aimed at trade liberalization and legal and regulatory reform. The reforms also included encouraging privatization and competition, banking sector restructuring, and judicial and health sector reforms. However, weak fiscal management and lack of public sector capacity hindered the effects of these reforms. Moreover, governance instability severely constrained broad-based economic growth and poverty reduction, despite a rebound in the late 1990s. The ‘rose revolution’ of 2003 gave a new impetus to the reform process. Large popular protests during Parliamentary elections in November 2003 forced the resignation of the country’s president and ushered in a reformist government. The newly elected government initiated measures to reinvigorate economic performance. Results were positive: GDP growth at about 6.3 percent in 2004 and 8.2 percent in 2005; inflation at about 6.2 percent end December 2004, averaging 8.3 percent throughout 2005. Tax/GDP ratio improved dramatically, up 3.5 percent in 2004 over the previous year. On the expenditure side, arrears from the previous administration were cleared as a priority and minimum pensions were doubled from January 2005. Despite these results, more than 50 percent of the population still lived in poverty with inadequate public service delivery. Sector background The Government recognized the urgent need for a better functioning public sector to perpetuate economic gains and meet growing social inequities. In 2004, the Ministry of Finance prepared the Strategic Vision for Public Financial Management (PFM). This became the cornerstone for public finance reform from 2004 onward. Objectives of this reform were: (i) maintain fiscal discipline; (ii) support a strategic approach to the management of public finances; (iii) ensure that resources are used efficiently and effectively; and (iv) ensure accountability. Parallel donor efforts supported timely provision of information on transfer of resources in synchrony with the Georgian budget cycle to improve predictability of budget resources. The Government took strong ownership of the reforms but more efforts were needed to coordinate them across the public sector. Critical path areas were procurement, payroll, internal auditing, and control systems. The Economic Development Poverty Reduction Program (EDPRP) Annual Progress Report of 2004 similarly identified weakness in Georgia’s public sector management as the major impediment to implementation of its ambitious state reform program. More accountable and efficient public institutions and creation of a public service to meet medium-term economic and social goals were viewed as priorities. Rationale for Bank assistance Bank assistance supported actions identified in the Government’s Strategic Vision for Public Financial Management Reform. This included anticorruption measures in the National Anti-Corruption Strategy, endorsed by the Government in June 2005. The Project was also consistent

2

with the Bank’s FY06-09 Country Partnership Strategy (CPS). Improving efficiency in public services was one of three overarching objectives to enhance Georgia’s prospects for poverty reduction and attainment of MDGs The Public Financial Management (PFM) reform agenda was incorporated into a coordinated donor program. The program included: an ongoing World Bank PRSO (Poverty Reduction Support Operation) program; the European Neighborhood Policy Initiative (ENPI); and PSFMRSP. Strengthening public sector accountability, efficiency, and transparency was the first pillar of the PRSO program, which directly linked it to the PSFMRSP.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

The PDO focused on enhancing governance. This was to be achieved through: (i) strengthening the institutional capacity of key agencies to more effectively and efficiently use public resources; and (ii) improving accountability in the use of public resources. Key Indicators. PRSO indicators were used as key indicators for the project. This was based on the clear linkage between Pillar 1 of the PRSO and the PSFMRSP. The PAD included the following eight key indicators:

1. Clear national expenditure priorities identified and allocations to national priorities defended in the budget;

2. Fiscal forecasts prepared for at least three years on a rolling annual basis, links between multi-year estimates and annual budget ceilings are clear and differences explained;

3. All major public entities submit fiscal reports to central government at least annually, and central government consolidates overall fiscal risk issues into a report;

4. Routine data collection or accounting systems provide reliable information on resources received in cash and in kind, and expenditures;

5. Timely information is available about the size, composition, and cost to the civil service;

6. Payroll and nominal roll data cover all of Government and are directly linked, with monthly reconciliation, updates, and reports on adjustments;

7. Improved scope, nature, legislative scrutiny and follow-up on external audits, all central government entities are audited using (primarily) compliance audits which adhere to international auditing standards; and

8. Provision of timely and transparent public financial management-related information to Parliament and civil society groups.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

PDO was not revised. The key indicators were revised in November 2006 based on guidance from OPCS and ECSPS to provide a more focused results framework. The number of key indicators was reduced from eight to six, new baselines were established and the indicators were

3

linked to PEFA indicators that became available by that time as an internationally recognized set of indicators for measuring PFM performance. The list of revised key indicators follows below:

1. National expenditure priorities fully defended in budget. MTEF increasingly based on full cost, results-based sector strategies consistent with fiscal forecasts (PEFA I-12 (iii))

2. Ministry ceilings contained in Budget Guidelines w/multi-year function/sector estimates approved by Basic Data and Directions (BDD). Differences between actual estimates and approved BDD explained in annual budget presentation (PEFA I-12)

3. Consolidated government financial statement published annually based on submission of fiscal reports from all major public entities (PEFA I-25)

4. Measurable improvement in the HR management function of the Ministry of Finance

5. External audit share of central government spending on which on-site audits are completed annually and disclosed to legislature within four months of the end of period covered (PEFA I-26)

6. Timely, transparent and user-friendly provision of public financial information to the public (PEFA I-10)

1.4 Main Beneficiaries, (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project)

The direct beneficiaries were central government ministries responsible for public financial management. These included the Ministry of Finance, especially the Treasury and Budget functions, as well as the line ministries expected to be linked to the public financial management information system. Another direct beneficiary was the Chamber of Control (State Audit Office). The project was also expected to provide important benefits for a broader government and general public through improving the timeliness and predictability of public expenditures, especially in areas related to combating poverty, and increasing transparency in the use of public resources. The availability of information on the use of public money to civil society was expected to strengthen the link between governance and public accountability across the executive.

1.5 Original Components (as approved) There were four original components:

(1) MTEF and Budget Management, intended to address the need for a strategic medium term budget planning cycle (estimated cost – US$2.27 million);

(2) Treasury Reform and Budget Execution, intended to develop a core Treasury information system and set standards for line ministries regarding resource management planning and processes. This was the biggest project component with an originally estimated cost of US$8 million. It included two subcomponents (sub-component 2a –support to Treasury, sub-component 2b – support to Budget Execution);

(3) Human Resources Management Information System, intended to create a civil service HRMIS to address the lack of comprehensive data on size, composition, and remuneration of civil

4

servants and to thereby address a part of the Government’s anti-corruption strategy (estimated cost – US$2.14 million);

(4) Public Accountability and Oversight of Public Financial Management, intended to strengthen the Chamber of Control’s operations and structure, and the development and implementation of training program in external audit; as well as institutionalize budget process mechanisms from the executive to foster public accountability (estimated cost – US$1.92 million).

In addition to the main four components, US$0.67 million was allocated to support project management.

1.6 Revised Components

The scope of component 3 (HRMIS for the public service) was revised in the first year of project implementation. The reorganization of the Public Service Bureau in 2006 made it impossible to implement the component as designed. It was decided to retain the component but narrow the scope of the HRMIS activity to focus on HR requirements in the MOF only. 1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations)

The project underwent multiple changes, including reduction in scope and scale, extension of the closing date, and partial cancellation of the funding.

After the mid-term review, the Project closing date was extended for two years to March 1, 2012. The rationale for extension was to enable the implementation of the biggest project activity, the public finance management information system (PFMIS), which was expected to include the core treasury system and additional PFMIS modules supporting budget preparation and execution.

The MOF subsequently decided to cancel the ICB process for the implementation of PFMIS and moved to develop the new information system with its own resources. It was agreed to redirect part of the funds allocated for PFMIS development to other needs (in particular, for equipment for the Finance Academy and the Procurement Agency).

As a consequence, the Project work plan and budget estimates were revised and decreased from an initial US$15 million to US$11.78 million. Given the sizeable balance in the Single Treasury Account as of September 2011, and the major part of the IDA contribution still undisbursed, the Netherlands Embassy and SIDA decided to cancel their final disbursements of US$730K (of US$2.1 million) and US$1.3 million (of US$4.5 million), respectively. In February 2012, USD1.87 million of the IDA grant funds (of US$3 million) was also cancelled per the MoF request, with the funds to be recommitted to another project under the Georgia portfolio.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable)

The Project was initiated to support the new government that had demonstrated strong political will to undertake an ambitious governance reform strategy. The moment was seen as a window of

5

opportunity for moving with long-awaited modernization changes in Georgia. In this context, the timing of the donor response was of the essence. The Bank moved swiftly with designing the package of interventions explicitly to support the government reform agenda.

For a project of such complexity, the preparation period was relatively short (nine months). It was designed as part of a broader program supported by several donors. An important design feature was the close link to the PRSO.

The Bank team followed the standard procedures for project preparation. Lessons learned from previous operations in Georgia and more generally were reviewed. Alternatives were considered. Numerous stakeholders were consulted during the preparation process.

IDA grant financing limits determined the need for partnership with other donors. The Bank played an important role in mobilizing significant co-financing and providing the framework for donor coordination. In the final design, the IDA grant accounted for only 20 percent of the total project financing envelope. Significant attention was paid at the preparation stage to considering alternative instruments to pooling donor funding. The pooling arrangement proposed was new for Georgia. Based on experience with projects using a similar approach to pooled funding, the necessary assessments were carried out and disbursement arrangements discussed in detail with the counterparts. In line with the Government decision to channel all donor funds through the national treasury system, it was decided to use country systems and disburse the project funds through the treasury.

The PCN and Quality Enhancement Reviews both identified the scope of the project as ambitious and suggested that the capacity of the government to implement the project as designed might be inadequate. The particular aspects that made the scope ambitious and demanding for the implementation capacity of the counterpart institutions were: (i) involvement of three implementing agencies (MOF, Public Service Bureau, Chamber of Control), which were not in subordinate relationship to each other; in addition, one of the implementing agencies did not report to the Government (Chamber of Control); (ii) scope of activities under components 1 and 2 involved multiple inter-linkages between the activities and required parallel implementation of several big activities in a relatively short period of time; (iii) inclusion of two large IT activities under components 2 and 3 that involved development of highly complex information systems that were expected to be inter-linked. It is noteworthy that the project was designed soon after the Government decision to dismantle stand-alone PIUs and integrate project implementation structures into implementing agencies. At the same time, the lead implementing agency for the Project, the Ministry of Finance, lacked prior experience in implementing Bank-funded projects.

The final PAD recognized that project implementation capacity was identified as one of the key determinants of success for all Bank-financed projects in the country and region. The main instrument to mitigate associated risks was the project implementation arrangements that envisaged the primary responsibility for project components would be assigned to the government officials representing respective beneficiary agencies. These agencies were to receive support for fiduciary functions by the staff of an experienced PIU recruited by the MOF as consultants. Funds were also reserved for capacity-building activities under the project.

The risk assessment in the PAD was comprehensive and covered two groups of risks: (i) those within the project and its immediate environment, and (ii) risks that fell outside the scope of the project. The first category included: (i) uneven institutional capacity across Government could hinder implementation; (ii) the current fragmented approach to civil service reform and the ongoing, uncoordinated initiatives at individual agency level could be counterproductive and

6

create negative expectations about continued reforms; (iii) resistance to investing excessively in external technical assistance; (iv) sustainability risk of capacity building efforts, as the civil service was fluid and staff turnover common; (v) the substantial IT dimension of the Project in the context of relatively local low implementation capacity in this technical area; and (vi) obstacles to the smooth functioning of the pooled funding arrangements with other development partners.

Risks outside the scope of the Project included: (i) an operating environment in which sometimes haphazard political will and lack of incentives for decisive policy implementation provide inadequate continuity of support or sustain public sector reforms; (ii) support could fade because reforming the public sector was viewed as a demanding and long-term task with few immediate tangible outputs; and (iii) territorial integrity-related problems could delay or undermine timely decision-making that needed to take place to move the menu of reforms forward.

Mitigation measures were designed to address primarily internal risks, while it was also acknowledged that mitigations of external risks were largely outside the donor control.

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) RESTRUCTURING and RISK, subsequent actions

The implementation progress during the first two years was slow and the project was classified as “at risk” by the mid-term review. The IP rating was downgraded to MS in ISR#2, dated November 2006, and further to MU in ISR#6 (May 2009), produced after the Mid-term Review (MTR). The PDO rating was downgraded to MS only in ISR#5 (June 2008) and to MU in ISR#6.

The MTR that took place in January 2009 acknowledged some progress toward achieving the PDO, but pointed to critical and persistent deficiencies in the implementation process, despite concerns expressed by the Bank repeatedly during supervision missions (and recorded in the respective Aide-Memoires (AMs)). These included a perceived lack of commitment and shift in focus in the Strategic Vision for PFM Reform; a severe lag in procurement and IT system development at the center of the PFMIS reform; weak arrangements for managing project components (turnover and lack of capacity in PIU); and unsatisfactory project monitoring and reporting. The MTR concluded with a very critical assessment of the project implementation progress and formulated the recommendations for improvement that were fully accepted by the Government.

On that basis, the Project closing date was extended for two years. It was recognized that by the original closing date a significant number of outputs under components 1, 2 and 4 would have been achieved. However, those pertaining to the Core Treasury System and additional PFMIS modules supporting budget preparation and execution required more time to complete. The implementation of project activities accelerated after the MTR and project management issues were largely resolved.

However, in 2010, the MOF decided to cancel the ICB process for the implementation of an integrated PFMIS solution and moved to develop the PFMIS system with its own resources. The decision was made after the first stage of the ICB process and was preceded by lengthy internal and external consultations. The Project work plan and budget estimates were subsequently revised in coordination with the Bank and other developing partners, and decreased from an initial US$15 million to US$11.78 million. Part of the funds allocated for PFMIS development

7

was redirected to other needs (in particular, for equipment for the Finance Academy and the Procurement Agency). The majority of the project activities not directly dependent on PFMIS were successfully completed by the new project closing date of March 2012.

Factors that affected implementation Positive:

- Continuity of the TTL on the Bank side throughout the design and implementation, as well as continuous presence of the TTL in the field were positive factors for the project. The Bank TTL played a very important role in building counterpart capacity for project management and assuring donor coordination.

Negative:

- The first years of implementation were affected by the military conflict with Russia and the regional financial crisis. These extraordinary circumstances required a crisis management approach from the Government and the longer–term objectives of PFM reform became a lower priority.

- Reorganization of the public service bureau made implementation of the original design of component 3 impossible.

- The lead implementing agency (MOF) did not have prior experience in implementing Bank funded projects.

- The complexity of the project design exacerbated capacity challenges. Multiple project activities were designed to occur in parallel, which led to capacity challenges that the government and its agencies did not manage to overcome. In addition, involvement of multiple implementing agencies determined the initial design of implementation arrangements that centered too much Project responsibility at the Ministry level. This caused lines of accountability at lower levels to become unclear.

- The implementation of component 4 was also affected by multiple changes in the leadership of the implementing agency (Chamber of Control/State Audit Office).

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

The M&E design was sound. PRSO benchmarks were used in the PSFMR as key indicators. This linkage was expected to assure access to the required capacity to deliver PRSO commitments. Attention was also paid to fine-tuning the indicators based on guidance from OPCS and ECSPS to provide a more focused results framework. In this context, new baselines were established and PDOs referenced to PEFA indicators. Key indicators were also supplemented by a comprehensive matrix of intermediate results indicators. In combination, these provided a solid basis for monitoring and evaluation of the project implementation progress. Monitoring was to take place through quarterly performance reports prepared by Component Managers and submitted to PIT and PMC; an annual report was also to be prepared within five months of end of the financial year against the plan of work. The time-bound targets were to be closely monitored.

8

However, the use of M&E was seriously compromised by protracted delays and lagging periodic and annual reports by the Project implementation teams during the initial years. Delays in submission of the Annual Work Program began in 2007. Even with close monitoring, delays related to the integrated PFMIS compromised achieving benchmarks. By June 2007, it became clear that the Project would not achieve its objectives by the original closing date. The PDO rating was downgraded in June 2008 (ISR) and a major deficiency in project management and implementation monitoring cited through May 2009. By June, project management had improved but further improvements in monitoring and evaluation were still called for. After Project extension, the AWP activities began to be monitored and evaluated more effectively, with most activities on track by July 2010. Overall, there was an improvement in the M&E in the course of implementation, but it remained insufficiently strong and underutilized through project closure. This is evident from the quality of the annual progress reports and the final project completion report.

2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) The financial management (FM) arrangements under the project, including accounting and financial reporting, funds flow, and external audit, were acceptable to the Bank during Project implementation. The FM arrangements were rated “highly satisfactory” at the beginning of Project implementation, and “satisfactory” thereafter through Project close. The internal control procedures also were adequate. The level of the government counterpart funding (GCF) overall was adequate; however, it remained below the agreed annual level until the end of the project, when it reached the agreed percentage (6%) of total project cost. Planning and budgeting arrangements were adequate and acceptable to the Bank. However, Annual Work Plans (AWP), which should had been approved no later than May 1 of each year (as per the Project Operational Manual), were usually approved with delays by the Project Management Committee (PMC). All payments were made through the Treasury Service and closely monitored (Bank practice in Georgia since 2006). The quarterly IUFRs were generally received on time and acceptable to the Bank, as were the audited project financial statements with the auditor’s unmodified (clean) opinion. Delays occurred in the submission of audited financial statements for FY 2006, FY2007, and FY2010 (slight delay). The reason was mainly the donors’ preference for a five-month submission period instead of the Bank’s six months for projects in Georgia. There were no major procurement violations in the course of project implementation. However, IT procurement confronted a number of typical issues for large-scale projects, the most important being lack of a general concept or strategy for IT implementation that took into account the long-term nature of the investment for the country. Among the specific issues were: (i) Not all bid submissions were technically sound. Technical specifications in both large and small ICB procurements revealed instances of specifications copied directly from specific manufacturers; (ii) technical requirements/specifications were prepared by the purchaser with specific solutions/equipment or vendors in mind, including requests for specific brands and Direct Contracting; and (iii) bids evaluation was not fully documented for technical compliance, requiring additional clarification.

9

2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable)

While no follow-up operation is anticipated, there continues to be ongoing in-house development of the PFMIS to complete the system requirements initiated under the Project. The preliminary results of in-house development are promising; however, the recommendations of the independent assessment that preceded the MOF decision to cancel the ICB tender for PFMIS were not fully implemented. This raises concerns about the feasibility of achieving a reliable quality of PFMIS based on in-house development.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) Project objectives and achievements remain highly relevant to current country priorities. The project is also consistent with the current CPS for Georgia (FY10-13), which continues to support good governance, resting on the two main strategic pillars: (i) meeting the post-conflict and vulnerability needs, and (ii) restoring growth and competitiveness. The outcome identified under the first strategic objective—improved public resource management-–is linked directly to the PSFMRS project objective and outcomes.

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) The Project provided an important contribution to strengthening the institutional capacity of the key agencies to more effectively and efficiently use public resources and improving accountability in the use of public resources; however, it could not fully achieve the PDO. As reflected in the analysis of results section of the data sheet and the table below, the project contributed to significant improvement in several dimensions of PFM performance, including stronger multi-year perspective in fiscal planning and budgeting; improved predictability in the availability of funds for commitment of expenditures; improved public access to key fiscal information; and improvement in the scope, nature, and follow up of external audit. Five out of six key indicators were achieved. Regrettably, the biggest project activity related to development of the public finance management information system could not be implemented as planned. This did not allow achievement of the respective intermediate results1 and limited the project outcomes.

1 Information on the subset of intermediate indicators regularly included in ISRs is provided in the ICR data sheet. Full information on achievement of intermediate results is presented in annex 2.

10

Achievement of Project Development Objectives

PDO Outcome Indicators Project Achievement

To enhance governance through: 1. Strengthening the

institutional capacity of key agencies to more effectively and efficiently use public resources; and

2. Improving

accountability in the use of public resources.

1. National expenditure priorities fully defended in budget. MTEF increasingly based on full cost, results-based sector strategies consistent with fiscal forecasts (PEFA I-12 (iii))2

2. Ministry ceilings

contained in Budget Guidelines consistent with multi-year function/sector estimates approved by BDD. Differences between actual estimates and approved BDD explained in annual budget presentation (PEFA I-12)

3. Consolidated government financial statement published annually based on submission of fiscal reports from all major public entities (PEFA I-25)

4. Measurable

improvement in HR management function of Ministry of Finance

5. External audit share of

central government

Achieved Costed expenditure strategies included in 2011-2014 BDD covered about 88% of executive branch expenditures as approved by 2011 Budget Law; 2012 State budget was prepared following program budget format. PEFA I-12 (iii) rating measuring existence of costed sector strategies is expected to be upgraded from C in 2008 to B in 20123. Achieved Annual budget planning is in full compliance with medium -term framework. Ministry ceilings for FY11 budget consistent with respective BDD estimates. Starting 2012 annual budget law, the annex to the budget contains multi- year (1+3) estimates per budget program. Multi-year perspective in

fiscal planning, expenditure policy and

budgeting, as measured by PEFA I-12

improved. The respective rating is expected

to improve from C+ in 2008 to B+ in 2012. Not achieved Consolidated government statement is published annually in the form of annual budget execution report. However, there are some deficiencies in consistency of presentation and no clarity on accounting standards used to produce the statement. No improvement in PEFA I-25 is expected in 2012 (2008 rating was D+). Achieved Comprehensive personnel database for all staff of the MOF system (5,000 records) is created and supported through the functioning software. The database is updated monthly and linked to the payroll module. Achieved The scope and nature of external audit

2 PEFA (Public Expenditure and Financial Accountability) framework includes a set of indicators measuring various dimensions of PFM performance. The rating scale used by PEFA is from D to A, where A is the best possible rating. 3 At the time of the ICR report completion, the final 2012 PEFA report was not available. However, the self-assessment was completed and the expert validation mission was completed. Referenced data for expected 2012 ratings is based on the results of the validation exercise.

11

spending on which on-site audits are completed annually and disclosed to legislature within 4 months of end of period covered (PEFA I-26)

6. Timely, transparent and

user-friendly provision of public financial information to public (PEFA I-10)

improved. For 2011, the financial on-site audits covered 37% of the central government annual expenditures. PEFA I-26 measuring scope, nature and follow up of external audit is expected to be upgraded from D in 2008 to B in 2012. Achieved Public access to main fiscal information improved. The government now discloses to the public five out of the six types of fiscal information required by PEFA I-10. The respective rating is expected to be upgraded from B in 2008 to A in 2012.

Causal Linkages between outputs and outcomes (please see Annex 2 for more details).

The objectives of the MTEF and Budget Management Component (1) were achieved (Satisfactory) based on completion of most of the activities/outputs. The MTEF has been fully integrated and institutionalized within the budget planning process, and planning now occurs within a more accurate macro-fiscal framework. The budget process has also become more transparent and open to public scrutiny.

To enable the achievement of these results, the Project supported the development of the new Budget Code that institutionalized the MTEF process and established the equalization transfer formula to regulate the relationship between the central and local budgets. Important support was provided for introducing program-based budgeting. Program budget guidelines with unified terminology were developed and finalized in consultation with line ministries. The 2012 State Budget was prepared using the new program-based budget structure. Through capacity building provided by the Project, the MOF strengthened its strategy and annual budget planning processes and improved arrangements for monitoring and oversight of budget implementation. Additionally, capacity and procedures for strategic expenditure planning, budget preparation, and budget management were strengthened in line ministries. Improved processes for financing the budgets of territorial units were also put in place. An internal budget policy and public expenditure review process was introduced and capacity-building and training strategy for staff in MOF Budget Department and line ministry budget units developed and implemented through the Finance Academy. The objectives of the Treasury Reform and Budget Execution Component (2) could only be partially achieved (Moderately Satisfactory). Based on the analysis undertaken, it was decided to revise the timeframe of implementation of the national accounting standards aligned with IPSAS. The respective key indicator and intermediate indicators therefore could not be achieved. The Ministry of Finance oversaw preparation of IPSAS training methodology and rollout of training based on standards for all central government spending units, including training of trainers. The accounting methodology and analysis department was established in the Treasury to lead the accounting reform process, including defining accounting standards and regulations. Temporary regulation on financial reporting aligned with modified cash based IPSAS was piloted in 2011. Resource management processes (including cash and procurement management) and budget accounting and reporting products were improved in line ministries, PLEs and other general

12

government entities. Cash management procedures were also improved in the Treasury and spending agencies based on longer horizons for cash plans and more reliable access to financial resources. The project contributed to these improvements mainly through training opportunities provided to the Treasury staff. Following an independent assessment of the Ministry’s current applications software solution, which support the financial management practices, and the available IT capacity, the Ministry cancelled the ICB process and instead decided to develop it on its own. A significant part of the system functionality has already been developed in-house, including web-based solutions for submission of payment requests by the spending units (e-treasury) and submission of budget requests by the line ministries (e-budget). The treasury payment system was integrated with the e-procurement system and electronic exchange of data was established with the revenue service, the National Bank, as well as with the civil service and public entity registers.

Progress with automation of the treasury functions resulted in improved efficiency of its services, with the treasury operations now taking only a few minutes to execute. The switch to electronic exchange of information also allowed a radical reorganization of the treasury service and closing all 11 regional treasury offices. Effectiveness of public resource allocation decisions improved significantly because of availability of real-time data on cash balances and spending position. Although initial results of in-house development of the new information system are encouraging, it is too early to assess the overall quality of the development, as a critically important part of the core treasury system is still being designed, including the general ledger. Also, some essential methodology issues still remain unresolved (for example, the method of integration of the budget classification with the chart of accounts).

Project funds not used for PFMIS were partially reallocated to support capacity building of the Financial Analytical Service (FAS) staff as well as make selected investments in ICT infrastructure. The FAS was established in mid-2010 to support existing and new information systems and strengthen information and communications technology used by the Ministry of Finance. The original objectives of the Human Resources Management Information System Component (3) could not be fully achieved, which became apparent with the reorganization of the Public Service Bureau in 2006, thus closing the possibility of developing a government wide HRMIS, as planned. The HRFMIS scope was reduced and focused on the internal needs of the MOF only. The results achieved under the revised scope of this component are satisfactory and include: (i) the development of a database including all MOF personnel; (ii) a payroll system operated centrally by the Treasury system, with monthly salary payments made directly to individual banks; (iii) development of an HRMIS module for the MOF; and (iv) the establishment of the training academy of the MOF. Part of the funds not used for development of information systems were re-directed to purchase equipment for the training academy. The objectives of the Public Accountability and Oversight of PFM Component (4) were achieved (Satisfactory). The Chamber of Control was legislatively transformed into the Supreme Audit Office. The human resources capacity of the Supreme Audit Office was significantly improved by an intensive training program and selection of staff based on merit criteria to meet the needs of the organization and to implement an audit methodology consistent with international best practice. The project supported training for SAO staff, some of whom were trained at the Finance Academy, to improve staff performance in applying the new Financial Audit Manual. The training program led to the certification of auditors successfully completing

13

the course. ITC equipment was purchased to complement the ICT infrastructure and software development needs assessment carried out with assistance from the Swedish National Audit Office. Greater transparency was achieved through frequent stakeholder consultations and public announcements of the SAO findings.

3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return)

This was an institutional development project and standard computations of economic and financial rates of return are not applicable.

3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Rating: MS

Taking into account that the biggest project activity was not fully implemented, and the related key and intermediate indicators could not be fully achieved, the overall rating is Moderately Satisfactory

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development N/A (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development)

Institutional change and strengthening achieved under the project were significant. In particular, reorganization of the treasury function through centralizing and automating the operations, and transformation of the treasury into a service provider represent significant achievements and put the treasury of Georgia among the leaders of treasury reforms in the ECA region. Similarly, progress achieved in transformation of the Chamber of Control into a Supreme Audit Office is very impressive. The Supreme Audit Office received capacity development assistance that enabled it to restructure and undertake a staff rationalization program and subsequent training specifically designed to improve the effectiveness of its function. In addition, a Finance Academy was established (on a quasi-proprietary basis) to provide capacity building for MOF, its affiliates, and the public sector in accounting and auditing in particular. The institutionalization of the training mechanism is especially important for ensuring sustainability of the project results. Development of the MOF internal IT capacity through establishment of FAS is also an important institutional change to which the project contributed. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) N/A

14

4. Assessment of Risk to Development Outcome Rating: Moderate

The project resulted in significant institution-building impact and this created a solid basis for sustainability of its results.

However, the MOF has opted for a risky solution by deciding to develop the PFMIS in-house. Although the initial results of in-house development are promising, the recommendations of the independent assessment that preceded the MOF decision to cancel the ICB tender were not fully implemented. In particular, the assessment noted that “a system with the level of integration and engineering quality required from PFMIS is unlikely to result from piecemeal building of its parts without guiding data, process and technology architectures.” To assure the quality of the system, the assessment recommended bringing in high-quality international expertise to develop data, process, and technology architectures. There was no immediate follow up to the recommendation from the MOF side, and with the project closure it lost access to the donor resources to finance such expertise. There is therefore a significant risk that the quality of the PFMIS system being developed in-house could be below the expected standard, and that the desirable degree of system integration would not be achieved.

Another important risk is the loss of capacity developed because of personnel changes (quite probable now in light of the recent political developments). In the case of the State Audit Office, this risk is exacerbated by the planned move of the main office of the institution to another city.

The latter risk was partially mitigated by establishment, with project support, of an institutionalized training mechanism through the Finance Academy, and the State Audit Office training center. Also, the ongoing twinning arrangement between the SAO and the Swedish National Audit Office provides hope for continuity and sustainability of results achieved under component 4. There are several examples of positive results of recent bilateral assistance provided by SNAO to the counterpart institutions in the ECA region (e.g., Croatia, Moldova).

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: MU

The rating for Bank performance in ensuring quality at entry is Moderately Unsatisfactory, taking into account that capacity constrains flagged by the PCN and QER meetings were not fully addressed in the final design and became an issue within the first project year. The complex and ambitious design did not match the existing capacity on the counterpart side and was one of the reasons for unsatisfactory implementation progress during the first years of implementation.

Both the PCN and QER reviews noted the risk assessment was unrealistic in light of the complex and ambitious nature of the operation. In particular, the QER Panel called for much more elaboration of PFMIS procurement strategy up front to mitigate substantial IT implementation risks, which later materialized. The panel also noted the limited implementation capacity of the government and the importance of incorporating specific benchmarks against which component

15

progress could be measured. Sequencing was also singled out for comment regarding components 1 and 2 as well as HRMIS-related interventions to ensure they properly reflected the actual capacity on the ground, and recommended tempering expectations, especially concerning the PFM and civil service components. In particular, the design of the Project called for implementation of numerous activities in parallel without sufficient regard for the capacity challenges inherent in simultaneous implementation or the need for sequencing. This quickly became an issue during the first years of implementation and resulted in significant delays. The need for strong coordination across Project components was also not adequately taken into account in the design. The risks with project management were assessed but risk mitigation measures proved to be inadequate. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: MS

A Moderately Satisfactory rating is assigned for the quality of supervision.

Problems with capacity of the project implementation entities and the Project management structure emerged at the beginning of implementation and continued through the mid-term review. The Bank continually attempted to draw counterpart attention to these issues during the supervision missions, as recorded in the aide-memoirs. However, the IP rating in the ISR was downgraded to MU only after the MTR. The persistence of the observed implementation deficiencies might have justified a much earlier downgrading.

The MTR was used efficiently by the team to draw Government attention to the fact that project implementation fell well below expectations. Following the mid-term review, the Government acted to strengthen the capacity of the Administrative and Procurement Support Team (APST), and coordination by the MOF improved significantly.

The situation with the ICB tender required flexibility and openness from the Bank team to consider alternatives to proven approaches to developing PFMIS. The in-house development preferred in the end by the Government is a solution the quality of which is hard to assure. Given that the tender for the biggest project activity was also the main reason for the project extension, and in the face of cancellation, the rationale for further involvement of donors in the project became problematic. The TTL deserves credit for managing effectively the dialogue both internally and externally in a very difficult situation. The external assessment of the counterpart IT capacity proposed by the Bank was a constructive solution to facilitate resolution of a critical project situation and avoid further disruptions to implementation. Regrettably, the recommendations of the independent assessment could not be implemented within the project timeframe, which left unaddressed the risks related to the quality of the PFMIS system now being developed in-house.

Fiduciary supervision was adequate throughout the project implementation period.

(c) Justification of Rating for Overall Bank Performance Rating: MS

Based on the evaluation of the Bank performance during the lending phase and supervision, the overall Bank performance is rated as Moderately Satisfactory.

16

5.2 Borrower Performance (a) Government Performance Rating: MS (Moderately Satisfactory)

In the critical initial stages of Project implementation, the Government waivered in its commitment to the agreed Project objectives as well as to its own Strategic Vision, which had provided the blueprint and rationale for the Project. (b) Implementing Agency or Agencies Performance Rating: MU Protracted delays during the first years of implementation, and the lack of action from the MOF on the Bank observations about critical deficiencies in the implementation process made during the supervision missions and reflected in the Aide-Memoires, are the main reasons behind the Moderately Unsatisfactory rating. Following the mid-term review, the Bank insisted that the Government remedy the project management and M&E issues before the Project could be extended. Implementing agency performance improved after the extension, and the PMC and APST teams were able to provide an Annual Work Plan and monitoring and evaluation updates on a more routine basis. However, M&E remained insufficient and continued to be underutilized. (c) Justification of Rating for Overall Borrower Performance Rating: MS Based on the evaluation of the Government performance and the implementing agency performance, the overall Borrower performance is rated as Moderately Satisfactory.

6. Lessons Learned (both project-specific and of wide general application) Government ownership of the reform agenda is essential to ensure effective implementation and future sustainability of the Project. Priority interventions were identified for the Project after extensive consultations with counterparts as well as main stakeholders within the framework of the Government’s Strategic Vision and National Anti-corruption Strategy. A key condition for Board presentation was the endorsement by decree of the Minister of Finance of the reform direction proposed in the Strategic Vision document, and the specific reform dimensions in the Government Accounting Strategy. Yet the Government did not follow through on the strategic vision after effectiveness. In Georgia, with a modernizing but relatively inexperienced government, the possibility of dramatic changes in government agencies, such as the reorganization of the Public Service Bureau, in the first year of the Project, essentially eliminating one component of the project, may be more common than in other countries in or outside the Region. The lesson is that agreements with the Government may subject to unexpected changes given its rapidly evolving and dynamic reform agenda. This suggests that Bank lending approaches may need to be highly adaptable but more tightly focused, especially on institutional development, instead of comprehensive and heavily oriented to ITC. It also suggests that the Government generally takes a more pragmatic than programmatic approach to development solutions, which should be taken into account in subsequent operations.

17

Project implementation capacity is a key determinant of success for all Bank-financed projects in the country and region and needs to be carefully assessed during project implementation. The project was the first experience of the Ministry of Finance in dealing with the Bank procedures. It was also designed soon after the Government decision to move towards integrating project implementation structures into Government entities and dismantling the stand-alone PIUs. These circumstances were compounded by a very complicated and broad project design that did not match the beneficiary capacity. If the capacity of the Government and its implementing agency is not strong, and the project is identified as complex, it is inadvisable to move forward until such time as the requisite capacity is in place or the project is simplified to reflect the counterpart’s actual capacity to manage implementation dynamics to ensure expected project outcomes can be achieved and safeguarded. Decisions on the type of IT solution to be used are critically important for the projects that involve significant investments in complex information systems. This project is one of several cases in the ECA region where governments were hesitant about procuring an off-the-shelf software solution for PFMIS. The MOF of Georgia was interested in having an assessment of possible alternatives at an early stage of the project, but was not fully satisfied with the quality of the consultant analysis it received. This later led to re-opening the issue during the ICB tender and eventually led to the tender cancellation. The Bank might consider investing in development of recommendations for its clients on how to approach their respective decisions in a more systematic manner. The methodology used for the 2010 independent assessment of the existing systems and IT capacity in Georgia could be useful in this context. The assessment of this nature would be most beneficial during the project preparation stage. It also might be advisable to recruit reputable companies for implementation of such assessments, to mitigate the risk associated with reliance on individual consultant opinions. Innovative funding mechanisms, such as “pooled funding” used in this Project, need to be carefully evaluated with respect to suitability if there is probability of weak implementation capacity. The pooled account was expected to provide flexibility to the client. It was also designed to accommodate the partner donor requirements for visibility of their contributions to the project. However, there were challenges in accounting contributions from multiple donors in various currencies and meeting the donor reporting requirements. The pooled account also limited flexibility of donors in adjusting the disbursement patterns to the changing circumstances. In particular, when the project had to be extended, adjustments in disbursement targets and end dates were not easy to implement, (i.e., some donors had to freeze their funding until others expedite disbursements to meet their internal requirements and limitations). In the end, the arrangement did not assure equal treatment of the donors with respect to the funding shares that were to be cancelled. IDA had to cancel the biggest share of its grant, compared to the shares of funding undisbursed by other donors. It is therefore not obvious that the pooled funding arrangement used under the project was optimal under the circumstances. The unintended effects of the funding arrangement used for this project should be analyzed carefully to avoid similar situations in other projects.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Borrower noted in its ICR disagreements with the Bank team on the approach to developing PFMIS solution. In particular, it is mentioned that the MOF questioned from the very beginning the relevance of commercial software for the Georgia context and that it took long time to convince the donors to change the methodology defined in the original project appraisal

18

document. In the MOF view, if they could have started in-house development of PFMIS earlier, the project resources would have been used more efficiently. Donors are requested to be more flexible when the recipient government approaches them with requests to change work direction if the outcome is expected to be the same. The FAS has also disagreed with the ICR team assessment of the risks associated with ongoing approach to development of PFMIS in-house. The ICR team views on these issues are reflected in section 4 (Assessment of risks to development outcomes), section 5.1b) (Bank performance in assuring the quality of supervision), and section 6 (Lessons learnt). In particular, it is acknowledged that the situation with the ICB tender required flexibility and openness from the Bank team to consider alternatives to proven approaches to developing PFMIS. All the available analysis of experiences with implementation of similar systems indicates that in-house development preferred in the end by the Government is a solution the quality of which is hard to assure. The external assessment of the counterpart IT capacity proposed by the Bank was a constructive solution to facilitate resolution of a critical project situation and avoid further disruptions to implementation. Regrettably, the recommendations of the independent assessment could not be implemented within the project timeframe, which left unaddressed the risks related to the quality of the PFMIS system now being developed in-house The main lesson learnt is that decisions on the type of IT solution to be used are critically important for the projects that involve significant investments in complex information systems. The Bank might consider investing in development of recommendations for its clients on how to approach their respective decisions in a more systematic manner. The methodology used for the 2010 independent assessment of the existing systems and IT capacity in Georgia could be useful in this context. The assessment of this nature would be most beneficial during the project preparation stage. It also might be advisable to recruit reputable companies for implementation of such assessments, to mitigate the risk associated with reliance on individual consultant opinions. (b) Cofinanciers Donor partners did not raise issues that require additional comments. Comments from the donor partners can be found in annex 8. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) NA

19

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1. Introduction of MTEF 2.27 3.00 132 2. Treasury Reform and Budget Execution

7.83 5.22 67

3. Development of the HRMIS 2.11 0.123 6 4. Improvement of Public Audit and oversight of PFM

1.92 1.79 92

5. Project Management 0.67 0.63 93

Total Baseline Cost 14.79 10.76 73

Physical Contingencies

0.21

0.00

0.00

Price Contingencies

0.00

0.00

0.00 Total Project Costs 15.00 10.76 71

Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00

Total Financing Required 0.00 0.00 0.00

(b) Financing

Source of Funds Type of Co-

financing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 0.90 0.64 72 UK: British Department for International Development (DFID)

4.50 4.10 91

IDA Grant 3.00 1.38 46 NETHERLANDS: Min. of Foreign Affairs / Min. of Dev. Coop.

2.10 1.64 78

SWEDEN: Swedish Intl. Dev. Cooperation Agency (SIDA)

4.50 3.01 67

Total 15.00 10.76 71

20

Annex 2. Progress in Achievement of Intermediate Results

Intermediate results by components &

subcomponent

Result Indicators

Progress as at MTR

Progress as at projects completion

Component 1: MTEF and Budget Management 1. The MTEF becomes fully integrated and institutionalised within the budget planning process.

MTEF understood by key stakeholders and seen as leading to better budgeting.

MTEF approach is widely disseminated in the Ministry of Finance and line ministries and it is acknowledged as the way to improve strategic and transparent budgeting

MTEF established and MTEF elements function through BDD. Starting 2012 annual budget law, the annex to the budget contains multi- year (1+3) estimates per budget program. Annual Budget Planning is in full compliance with midterm framework defined by the BDD.

2. The planning of the MTEF and Budget takes place against a realistic macro-fiscal framework.

Improved accuracy of fiscal forecasts contained in BDD. Budgets developed consistent with medium-term fiscal forecasts.

Related activities are still in the inception phase and as such results are foreseen to be achieved in the next 2 years

The working group established with Fiscal Forecasting department and budget department staff that they produce macro-fiscal framework based on the in-house developed forecasting model. The forecast for 2011 was close to outturn, although it was on more conservative side in terms of revenue forecast.

3. Strengthened strategic and annual budget planning processes are established within the MOF.

Consistency between expendit-ure priorities identified in BDD, resource ceilings, and budget allocations. Consistency between outer year resource ceilings and budget allocations in subsequent years.

Consistency has been improved, but direct linkage between priorities and budget allocations is still in the pilot phase Outer year resource ceilings are included in the BDD, but not yet in the Budget Circular.

Ministry ceilings for FY11 budget consistent with the BDD estimates. Costed expenditure strategies included in 2011-2014 BDD covered about 88% of executive branch expenditures as approved by 2011 Budget Law; 2012 State budget was prepared following program budget format and its annex includes financing projections for the 1+3 year per State Budget program.

4. Improved arrangements are put in place in the MOF for monitoring and oversight of budget implementation.

In-year and annual budget monitoring reports that highlight budget implementation issues.

Actions taken to address issues identified in monitoring reports.

Six months and annual budget execution reports are prepared

Budget execution bi-annual and annual reports are prepared and issued on time

21

5. Capacities and procedures for strategic expenditure planning, budget preparation and budget management are strengthened in ministries.

The majority of discretionary spending covered by strategic plans by 2009.

Quality of strategic expenditure plans. Ministry budget allocations consistent with MTEF strategic expenditure plans. Greater consistency between budget outturns and budget plans.

Strategic planning by line ministries has improved, though linkages between the strategic priorities and budget allocations are not explicit Aggregate expenditure outturn differs significantly from the original approved budget (more than 15%)

Program budget methodology endorsed by the Ministerial decree, applied to 2012 State Budget. The annual budget law contains annexes on budget programs with multiyear estimates and list of capital projects with estimates of financing for the lifespan of the projects. The ration of total actual outturns to approved budget allocations was 98.5% in 2011.

6. Improved arrangements are put in place for the financing of the budgets of the territorial units.

Improved legislative framework for intergovernmental finance.

Transfers to territorial units based on transparent criteria and formula that address equalisation requirements.

Law on Local Self-Government Budget, determining the rules for budgeting of local self-governments adopted in 2006. Equalization formula developed in 2007 and operational

The Budget Code adopted by the Parliament in 2009 regulates Budget Process of the State Budget, Budgets of Autonomous Republics and local self-governing units. The Budget Codes also includes a Chapter for Equalization Transfer Formula.

7. An internal budget policy and public expenditure review process is introduced.

Reports of budget policy and public expenditure related studies. Study findings are incorporated into BDD proposals and budget planning and management reforms.

Public expenditure review studies not developed yet

While public expenditure review studies were not developed as separate exercise, the budget execution report for 2011 introduced analytical section which not only reports on execution of expenditures, but also on results achieved according to sector plans and analyses reasons for successes and failures. This is expected to further strengthen as a result of program based budgeting introduced starting 2012 budget

8. A capacity building and training strategy for staff in MOF Budget Department and line ministry budget units is developed and implemented.

Annual programme of MTEF/Budget-ing related workshops and other training events prepared and implemented.

Improvements in quality of analysis in budget documentation.

Annual training programme developed, though need to be updated for 2009; workshops conducted for the staff of 5 pilot ministries

Academy of the MOF established in 2011 and tasked with ensuring continuous training and professional development for the government agencies, with primary focus on PFM topics. Capacity building of the Academy is underway and the major part of it is already completed.

22

9. There is greater transparency public accountability in the budget and public expenditure planning process.

Better informed discussion of the Budget and budget related issues in Parliament and Parliamentary committees. Higher quality analysis of budgetary issues in press reports and publications of Civil Society Organisations.

Summary versions of the BDD and Budget Law are being prepared regularly since 2006; Parliament is the primary recipient of these documents

BDDs, Government financial statistics and information about Central Government debt is published on the MOF website and updated; Citizens' Guide on State Budget and BBD was published regularly since 2006 and disseminated to Parliament and other stakeholders; Citizens Guide on Tbilisi 2009 Budget as well as brief review; 2012 State Budget published on the MOF website. The 2011 budget execution report with analytical part of the results achieved and reasons for successes/failures is published on the MOF web-site

Component 2: Treasury Reform and Budget Execution

1. Treasury develops appropriate role in leading accounting reform process, including playing the role in defining accounting standards, regulations, etc.

Treasury defines accounting standards

% of certified accountants in Treasury

IPSAS and its pre-requisite GFS 2001are identified as accounting standards. GFS 2001 Classification is implemented, while methodology will be gradually delivered by 2015. Accounting Policy TA mobilization to assist in IPSAS implementation is in process.

There are no certified accountants in the Treasury. Certification process is not yet defined.

Accounting methodology and analysis department established in the Treasury to lead the accounting reform, to prepare the accounting methodology and guidelines according to the international standards. Georgian Public Sector Accounting Standards Board (GPSASB) established to provide supervision on Public sector accounting standards implementation with Treasury Head as chair. GPSASB website being developed.

Initial IPSAS training delivered to 18 Treasury staff in 2011 of whom 7 people were sent to the next stage training and given comprehensive IPSAS course in 2012. Six people were certified in IPSA Standards.

2. A professionalized public accounting community is introduced in Georgia and IPSAS cash and then accrual accounting standards are adopted gradually.

% of certified accountants employed by the government entities

IPSAS or corresponding national standards are applied for all statements

Certification process to be defined with the assistance of the Accounting Policy TA (mobilization is in process).

Currently National

Training and certification program and all materials for the initial and detailed IPSAS trainings prepared. Trainings are being delivered starting from the Fall 2012 through the MOF academy to public sector accountants. Six certified people from the Treasury is used as trainer resources.

Temporary regulation on financial reporting based on modified cash basis IPSAS was piloted in 2011. Consolidated cash statement and reports for the Pilot Exercise prepared. Regulations revised and rolled out in 2012 for the whole central budget organizations. IPSAS being translated

23

Standards (not compliant with IPSAS, though with some approaches from IPSAS) are applied to all State Budget Statements

into Georgian with EC financing.

3. Cash management procedures are significantly improved in Treasury and spending agencies.

A cash flow forecast is prepared for the fiscal year and updated at least quarterly, on the basis of actual cash inflows and outflows.

Spending units are providing reliable information on commitment ceilings.

Cash Flow Forecasts are prepared on a monthly basis. There are no forecasts for fiscal years.

Starting from December 2008 Spending units prepare and submit to the Treasury the Information on the entire signed contract prior to submitting the commitment document. The contract information includes the total amount and the payment schedule set by the contract. This measure will allow Treasury for better understanding on commitment ceilings position and help the spending unit to improve their cash planning capacities.

Cash flow forecast prepared by the Treasury based on a contract payment schedule, fixed expenditure pattern and empirical data. No frequent change made in the plans

BOs are allowed to download public procurement contract data from the Electronic Tender System in a timely manner. Payment schedules are also entered in the TIS by the spending units which give on-time access to the reliable data on cash requests to the Treasury Service

4. The quality of resource management and of public accounts and reports is progressively improved.

Routine data collection or accounting system provide reliable information on resources received

Consolidated government statement is prepared annually providing near complete

Routine data collection provides reliable information on resources received.

The local authorities are to comply with the GFSM2001 as of January 1, 2009 as per the new

Routine data collection is based on real-time regime and complete information on central budget receipts and payments are available in real time and provides reliable and on-time information for fiscal decision making.

Consolidated government statement is prepared quarterly and annually providing near complete information on revenue, expenditure, and bank account balances. However, there are inconsistencies in presentation and the standards applied to produce the statements are not disclosed.

24

information on revenue, expenditure, and bank account balances

classification becomes effective from January 2009

5. Procurement related information sharing is more efficient

Accurate data on the value of public contracts and methods used to award these contracts exists.

Public procurement data is promptly shared among the spending units, the MOF,

Due to postponement of the development of centralized e-Procurement system by State Procurement Agency (SPA), this indicator no longer valid and shall be cancelled

Accurate data on the value of public contracts exists in the Public procurement database which is integrated with TIS, so that these data shared with the TIS

6. An integrated, modern Public Financial Management System is operational.

PFMS is implemented by the end of 2012 and fully operational in 2013.

Foreign TA to assist with PFMS functional design and development of technical requirements mobilized, consultants started to work January 2009

PFMIS is being developed in-house. More than half of the system is implemented. All information gets shared electronically; Eleven regional Treasury offices abolished; the staff in the central treasury office and in spending units downsized. Material costs economized roughly amounts to ten mllion GEL annually. Treasury Operations take couple of minutes to execute. Effectiveness of public resource allocation decisions improved significantly because of the availability of the real time data on government accounts cash balances and spending position.

A countrywide network connecting all central and local units of the MOF agencies, as well as the National Bank, Supreme Audit Office and other related government agencies established

FAS ICT infrastructure and security systems upgraded;

Component 3: Human Resources Management 1. An automated Human Resources Management Information System (HRMIS) provides a comprehensive and accurate database of all staff employed by the MOF, including essential personnel data, such as employment status, salary and entitlements, educational background, job classification, etc.

Fully functional automated HRMIS in place. Required changes to the personnel records are updated monthly and respectively reflected in the payroll.

Foreign TA to assist with PFMS functional design and development of technical requirements mobilized, consultants started to work January 2009

Central personnel database for all staff of the MOF system (over 5,000 records) is supported through the software developed in-house by the MOF FAS and became fully functional as of January 1, 2010. The database is updated monthly and is linked to the payroll module.

2. An automated payroll provides direct inputs into budgeting and expenditure management functions’ of the MOF and forms the basis for further roll-out to other line ministries.

All MOF employees receive salary payments on time in accordance with their employment contracts.

Foreign TA to assist the MOF with development of the effective Human Resources (HR) management system identified, contract expected

Payroll is operated centrally by the Treasury Service with monthly salary payments made directly to the individual bank accounts.

25

Improvements achieved in pay and incentive systems. Annual payroll audit report published.

to be signed in March, 2009

3. Improved access to accurate and timely information about the size and the cost of personnel in each structural unit of the MOF and respective subordinated agencies provides necessary inputs to targeted restructuring and retrenchment operations.

MOF Personnel database and payroll are directly linked. Retroactive adjustments are rare.

Not done, pending introduction of the automated HRMIS module

The scope of information system, introduced in January, 2012, includes the human resource management (HRM) function comprising payroll, personnel management, benefits administration and manpower analysis modules. The personnel management module provides for maintaining personnel records and database for each budgetary organization. The staffing lists are monthly reconciled with the payroll module under the E-Treasury system. Personnel records are updated in time for the following month payment. Retroactive adjustments are not practiced

4. Training system in place to continuously upgrade capacity of the MoF staff.

% of MOF staff. regularly trained on PFM specific topics Annual training plans exist and are updated

PFM related training offered to MOF staff as well as budget staff of other line ministries. MOF Human Resources Strategy, including training strategy drafted and presented to the Minister for approval.

The Academy of the Ministry of Finance established in 2011 as a legal entity of public law offers continuous professional development and training opportunities to the MOF and other government staff in the areas of budgeting, taxation, accounting, financial management, public and municipal management and finance. About 40% of the MOF core staff was trained on PFM topics. More than 1400 people received training in 2012, of them about 50% MOF system employees and about 37% staff of other government agencies.

Component 4: Public Accountability and Oversight of Public Financial Management 1. Strengthened framework for organization and management of the Chamber of Control.

Organization and structure of the CCG that reflects the desired new service delivery culture.

Support structures in the CCG more responsive to the organisation’s needs.

New draft Regulation of the SAO covers organization, structure, and operation of the SAO

The organisational structure of the State Audit Office reflects the public sectors covered by the National Budget. The Audit Departments are set in a functional way. Each of them are responsible for their sector. It also promotes to form an institutional memory. The Quality Assurance Department and Performance Audit Coordination unit are added to support new tasks. The operational procedures are detailed and regulated in the new Rules of Procedures of the State Audit Office, in audit guidelines such as Financial Audit (including Compliance) manual, Performance Audit manual, Quality Control manual and other institutional policies.

2. Strengthened legal framework governing the CCG

CCG’s governing laws clarifying the powers of the CCG and confirming the CCG’s focus on auditing as distinct from ‘controlling’

New legislation on SAO adopted on December 26, 2008, as well as draft Regulation specifically refers to audit as opposed to

Law on SAO amended on December 24, 2011 to bring it further in line with INTOSAI and Lima Declaration.

26

control. Audit function will be further strengthened as new audit methodology (compliant to the international standards) is drafted and adopted by the SAO

3. The Human Resources capacity of the CCG is responsive to the needs of the organization.

90% of the auditing staff of the CCG (SAO) is trained to perform effectively.

Out of 429 persons enrolled in ACCA training, only 51 remain in the program (at different stages) This is about 25% of audit staff

ACCA training program abandoned. In September 2011, 105 Tbilisi-based auditors trained and tested on new financial audit methodology of whom only 45 passed exam and retained and remainder let go. Further 35 staff from Kutaisi and Batumi trained and tested. 20 new staff recruited. Later in May 2012 newly recruited people and those staff who wished to improve last exams were enrolled in next cycle of Financial audit trainings. In December of 2012 43 auditors were trained in Performance audit manual. The State Audit Office has elaborated 2 training courses in Financial and Performance Audit that will be delivered to staff on an annual basis.

4. An audit methodology for the CCG is implemented.

Annual desk-top audits at minimum continue to cover 70% of total central government expenditures.

On-site annual audits of central government expenditures at least double from current 15% .

96% for CY 2007, and 99% for CY 2008 2% for CY 2007 (due to reorganization of the SAO), and 28% for CY 2008

The State Audit Office conducts annual financial audit of the central government agencies. As reflected in SAO 2012 report (covering FY 2011), the financial on-site audits covered 37% of the central government annual expenditure. The remaining 63 % of expenditures were covered through carrying out substantial analytical review. Overall, the SAO covered 79% of central government expenditures of the fiscal year of 2011." With the introduction of the new financial audit methodology, the practice of desk-top audits has been replaced with "the analytical review", in line with INTOSAI good practice.

Audit methodologies developed, including: Financial Audit Methodology adopted 9/2010, piloted late 2010 in three audits and now being rolled out following September 2011 training; Quality Assurance Policy and Manual adopted 11/2011 and being rolled out with Quality Assurance unit to be established in 2012; Performance Audit Methodology adopted 12/2011 and being piloted.

5. Strengthened external relations capacity.

Audit reports are submitted to the legislation within 8 months of the end

Audit reports are submitted on time

Audit reports are submitted on time

27

of the period covered. Annual consolidated audit reports are published

Audit report on 6 months implementation of 2007 State Budget is published on the SAO web-site

SAO Annual Report on implementation of 2010 and 2011 State Budget is published on the State Audit Office Homepage at http://sao.ge/?action=page&p_id=225&lang=geo

28

Annex 3. Economic and Financial Analysis (including assumptions in the analysis)

The Project supported public sector financial management reforms and related capacity building, so standard computations of economic rates of return are not applicable. Technical assistance and investments envisaged under the Project aimed at improving performance of key government institutions at the central level in implementing specific budget management, execution, accounting, auditing as well as human resources management tasks. Therefore, return of investments under the Project was expected to be, to a large extent, in a form of better performing public institutions capable of employing financial and human resources in an effective, transparent and accountable way. The major benefits of the Project are difficult to quantify and measure. These include providing the means to ensure effective implementation of budget allocations and greater transparency in government’s financial management as well as in strengthening instruments of fiscal control. Implementing the new treasury system and streamlining budget preparation and implementation procedures resulted in efficiencies in government budgetary transactions. Better cash management reduced idle balances in government accounts. More efficient execution of budgetary transactions reduced delays in payments and should have eventually reduced the cost of goods and services to the government.

29

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Specialty

Lending Elene Imnadze Sr. Public Sector Specialist ECSP4 Task Team LeaderMatthew Andrews Consultant AFTPR PFM SpecialistMikhail Bunchuk E T Consultant WBIRP Sophie Devnosadze Operations Analyst ECCGE Tamuna Namicheishvili Program Assistant ECCGE Craig R. Neal Consultant MNSEG IT Specialist

Friedrich Peloschek Consultant LEGEC-HIS

Country Lawyer

Gurcharan Singh Senior Procurement Specialist TWICT Petrus Henricus Van Heesewijk Senior Program Officer PRMPS

Public Sector Specialist

Arman Vatyan Sr. Financial Management Specialist

ECSO3

Andrew Bird Consultant DFID MTEF/Budget Planning

Supervision/ICR Elene Imnadze Sr. Public Sector Specialist ECSP4 Task Team LeaderMatthew Andrews Consultant AFTPR PFM SpecialistOleksiy Balabushko Economist ECSP4 Cem Dener Sr. Public Sector Specialist PRMPS IT Specialist Sophie Devnosadze Operations Analyst ECCGE Gurandukht Elashvili Buyer GSDPR Procurement Spec.

Ranjan Kumar Ganguli Consultant ECSPEAccounting and Audit Specialist

Ahmet Gokce Consultant ECSO2Procurement Specialist

Tatyana Kandelaki Financial Specialist ECSF2 Tamuna Namicheishvili Program Assistant ECCGE Craig R. Neal Consultant MNSEG IT Specialist

Gurcharan Singh Senior Procurement Specialist TWICTProcurement Specialist

Petrus Henricus Van Heesewijk Senior Program Officer PRMPS PFM Specialist

Arman Vatyan Sr Financial Management Specialist ECSO3 Virginia S. Yates Program Assistant ECSP4 Sandro Nozadze Procurement Specialist ECSO2 Irakli Zakareishvili Consultant ECSO3 FM SpecialistGhada Youness Senior Counsel LEGLE Country Lawyer

30

Giovanni Mascarenhas Head CBTAT IT Specialist Ireneusz M. Smolewski

Senior Procurement Specialist ECSO2

Andrew Bird Consultant DFID MTEF/Budget Planning

(b) Staff Time and Cost UPDATE

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY99 12.62 FY00 6.37 FY01 9.31 FY02 14.03 FY03 51.52 FY04 5.33 FY05 83.79 FY06 181.64

Total: 364.61 Supervision/ICR

FY06 24.35 FY07 135.93 FY08 67.11 FY09 134.34 FY10 127.55 FY11 106.09 FY12 80.83 FY13 30.00

Total: 706.20

31

Annex 5. Beneficiary Survey Results (if any)

N/A

Annex 6. Stakeholder Workshop Report and Results (if any)

N/A

32

Annex 7. Summary of Borrower's ICR and Comments on Draft ICR

a. Comments on draft ICR4

We reviewed the report on PSFMRSP implementation results. On behalf of all beneficiaries of the project we would like to note, that we don't have any major concerns and in general, the report is acceptable. FAS provided comments regarding the risks identified in section 2.5 - Post-completion Operation/Next Phase and section 4 - Assessment of Risk to Development Outcome (the respective text is colored in red in the attached file). Financial Analytical Service does not agree with that statement and asserts that data processing procedures and system technology architecture is robust enough to achieve the desired integrated in-house system. The comment is given below: "MOF technical staff developed the technology architecture fully in accordance with data and process model provided by the State Treasury, implementing the best practices widely accepted for the complex financial transaction systems worldwide. Not going into the deep technical details, key characteristics of the system in development are:

§ - Distributed, modular structure of all software components, allowing them to function independently in case of other systems failure;

§ - N-Tier systems architecture, implying moving business logic totally out of database and user interface layers and developing separate system core incorporating business logic there, thus implementing stable and flexible architecture.

§ - Web services (XML based) standard integration model across all systems, providing uniform method for integration of existing systems and for future ones.

§ - - Modern technologies for development environment: MS SQL Server 2008 R2, Microsoft .NET framework 4 and Microsoft Entity Framework, using C# for development language and compiling each layer as a separate assembly block. Communications Layer utilizes Microsoft Windows Communications Foundation, compatible with almost any client technology, based on open standards.

- Virtualized environment for deployment, providing high availability and flexible hardware resource allocation. Business continuity is provided by the disaster recovery center with full capacity hardware and configuration, mirroring the main site.

§ - - Microsoft Operational Framework, ITIL standards and project management methodology was used in the process of development, together with source and code development control implemented through Team Foundation Server.

The aspect of developing system has to be taken into account, meaning that the systems is still under development and certain, though relatively small percentage of features and improvements will be added during the next couple of years. However, core functionality is

4 Comments received from the Ministry of Finance of Georgia by e-mail dated November 26th, 2012

33

already available and stably functioning. The strategy of systems architecture is providing reliable basis for integration of any system, planned according to the standards defined. MOF has decided to advance towards in-house development of PFMS project taking into account a large number of argument on both pro and con sides. One of the key influence factors for such decision was MOF technical staff confidence into its technical capabilities that was confirmed by the independent audit conducted by the World Bank. Technical assistance from the side of international experts in software architecture was asked for additional increase of capabilities of IT application development team, but it should not be considered as a significant factor affecting quality of the system in development."

b. Summary of Borrower’s ICR Component 1 Observations: MOF acknowledges PFMRSP valuable assistance in the reforms in the sphere of Budgeting especially in capacity building:

The project allowed to have a wider access to the best practices of other countries

which have gone through the same stages of reforms by providing the necessary material and consultations related to MTEF, Budget Classification Standards, Program Budgeting etc;

The project assisted in sharing the experience of other countries by organizing study-visits, supporting participation of Georgian side in different conferences, seminars and working-groups;

Project supported capacity building of the respective staff within the MOF and other Spending Units (e.g. New Budget Classification Implementation, Program Budgeting, English Language Courses,)

Project supported equipping the MOF Academy, which is gradually becoming a center for the capacity building of the current and future staff of the organizations related Public Finance.

The project implementation process should be assessed positively in overall. Though the main points of activities under the project needed time before reaching the right

solution the project support to the reform was significant. MTEF – Introducing MTEF was not one of the easiest tasks and it took years before the Basic Data and Directions Document (BDD) obtained its current format and content. Of course the document will be further improved but to reach the current point certain experience was needed. Consultants, material and recommendations under project were of great help through this difficult task. As a result of the experience which was gained with time and years of preparing the document and with help of the Project support MTEF has become functional

34

Program Budgeting – Introducing Program Budgeting was a real challenge. Timeframe for its implementation was quite tightening and a lot of aspects had to be decided and agreed upon while piloting it. The project was helpful in sharing the experience of other countries and giving recommendations of the best ways. Public Finance Management and especially MTEF and Budgeting is the sphere which is under constant reforming, it always needs to be updated and best fit to available resources and expenditure priorities. Thus reforms in this field will be continued in the future as well and any support which will be based on real needs will be welcomed. Component 2 Observations: Overall evaluation of the project implementation results is positive but we observed several issues during the project implementation that was not adequately considered and assessed during the original appraisal of the project goals and objectives. IPSAS implementation – Since the PSFMRSP provided an excellent opportunity to fund the expensive technical assistance and other project related costs, we attempted to design the tight implementation schedule. This was very risky decision because international practice proves that similar projects take decades to implement. The lengthy procurement procedures and deficit of qualified accountants were regarded also as one of the obstacles during the project implementation. Internal capacity was an issue as well. At the very starting point of reforms people are not aware of the scopes and details, therefore identifying the real needs, drafting the right TORs, selecting the best suitable candidate would take weeks and months. WB experts were a huge helping hand in this. They provided useful feedbacks and comments during the preparation of ToRs and evaluation of proposals. Regardless of these problems we are on track and have substantially advanced with the IPSAS implementation project, however some important activities could not be completed, such as training of BOs staff, design the methodology of defining norms for assets depreciation, etc. PFMS implementation – This is extremely both - time and resource consuming project as well worldwide. Besides, rapid development of information and communication technologies and systems in general make it difficult to identify the right solution for the specific assignment. We’ve lost several years while trying to identify implementation methodology. At the very starting point we questioned the relevance of commercial software for our context, after the demonstration of proposed systems our skepticism even increased, but it took couple of years to convince the donors to change the methodology defined in the original project appraisal document. Should have the MoF started in-house development of PFMS earlier, the project resources would have been used more efficiently. Surely. Donors need to be more flexible when the recipient government approaches them with the request to change work direction if the outcome is expected to be the same. Overall result in this field is also notable: FAS infrastructure is the best available as of today. Capacity of Technical and functional staff increased, assessment of systems done.

35

This assessment is very positive and promising and we are confidently heading to the goal.

Component 3: Due to the reorganization of the Public Service Bureau (PSB) in 2006, there is no possibility to develop a government wide HRMIS system for the public sector within the scope of the project now. Instead, the MOF decided to develop a HR management module (as a simplified version of the HRMIS) to cover their requirements and add a payroll calculation module as a part of future PFMS, to create an interim solution. Therefore, PFMS scope was reduced and ICT related Component 3 activities were merged in Component 2. Later MOF HR Department was reorganized and the Component 3 activities brought to an end.

Component 4: Public Accountability and Oversight of PFM. The objectives of this component were to: (i) facilitate the strengthening of the State Audit Office of Georgia’s (formerly Chamber of Control of Georgia) operations and structure and the development and implementation of training program on external audit; and (ii) help with institutionalizing budget process mechanisms within and from the executive that will enable public accountability.

The State Audit Office of Georgia has been involved in fruitful cooperation with the Public Sector Financial Management Reform Support Project. Through the last 6 years of project term the external audit role, mandate and function have significantly improved thanks support of the project and other partners. The Project set specific goals and objectives to be achieved through comprehensive reform activities within the Public Finance Management System of Georgia. The external audit component included all the necessary areas to be improved to build capacity of the State Audit Office. In particular improved scope, nature, legislative scrutiny and follow up on external audits were set as overarching goals of the project. The audit activities in accordance with international auditing standards are central element of the strategic objectives.

During the year of 2011 the State Audit Office of Georgia implemented core

portion of its Strategic Goals set in Plan for 2010-2012. The Year 2011 should be distinguished for introduction of risk based modern audit methodologies. The results of institutional reforms have significantly increased the efficiency of the work to be performed by institution, the auditors and experts with impartially measurable financial and non-financial impacts. All the aforementioned promote improvement of management of public finances as the results of the issued recommendations and more efficient delivery of public service. The sustainable institutional development process covered wide range of the aspects of the functioning of the SAO: Mandate, Governance, Audit quality, Human Recourses, Information Technologies, Communication etc. Below are described all the important strategic steps taken by the SAO. State Procurement Agency Institutional Support

The Objective of this sub component was to make the procurement related information sharing more effective. To ensure the Georgian State Procurement e-System full and

36

efficient functioning IT infrastructure has been established under PSFMRSP. As a result, the Georgian Electronic Government Procurement (GeGP) is linked to the Treasury’s information system, enabling sharing all procurement related information, inter alia information regarding contracts and phases of their fulfillment. GeGP is also linked to the Public Registry (NAPR) of firms, which assists at recognition of the entity while registering and to the Civil Registry Agency to facilitate registration process of physical entities, also to establish e-signature tool. GeGp linking to the State Audit Office is also planned, that aims at effective and systematical control of public expenditure.

37

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Swedish International Development Cooperation Agency 5: Sida’s overall results assessment of Public Sector Financial Management Reform Support Project is considered satisfactory. Sida mainly agrees with all findings presented in the ICR report and will highlight some of weak areas of the project that could be additionally emphasized. Government of Georgia was not considering the public finance reform as one of the most important priorities during the Country’s democratic development process. The implementation of the PSFMRSP was delayed due to internal and external risks. The mitigation measures were not adequately taken by the Project Implementation Team. The Project Implementation Team had no Project Manager who could lead the project’s timely implementation and coordination among GOG and between implementing entities of the PSFMRSP. Accordingly, the disbursement of Sida funds has been lower then decided in the Sida Decisions on Contribution. Cancelling the ICB process and PFMIS systems in house development was one of the main reason withdrawing Sida funds (1.3 Million USD out of total 4.5 Million USD). According to Sida assessment PSFMRSP has built the capacity of Ministry of Finance, Finance Academy, Chamber of Control (State Audit Office), Procurement Agency and improved the key Governmental entities performance. The overall public finance system of Georgia became more transparent and accountable to public. The embassy of Netherlands responded that they had no comments on the draft ICR. DFID program for Georgia has been closed and no representatives with institutional memory of the project were identified.

5 Comments received from the Embassy of Sweden in Tbilisi by e-mail dated November 20s, 2012.

38

Annex 9. List of Supporting Documents World Bank. Republic of Georgia: PEFA Assessment, 2008. World Bank. Final Supervision Mission, ISR No. 11, December 16, 2011 World Bank. Georgia Public Sector Reform Support, Project Concept Note, September 29, 2005, and Minutes World Bank. Georgia: Project Appraisal Document, Public Sector Financial Management Reform Support Project, January 18, 2006 World Bank. Implementation Review Report, January 13-20, 2012 (Final) ICR Mission. Interviews with TTL, Government counterparts during ICR field mission, and ECA Management ISRs 2006-2012 World Bank. Mid-term Review, Aide-memoire, January 20-30, 2009 Ministry of Finance, Republic of Georgia, Project Unit, PSFMRSP Implementation Completion Report, July 14, 2012, Draft World Bank. Quality Enhancement Review, Georgia Public Sector Reform (P063081), October 16, 2005

Ca

uc

as

us

M

ou

nt

ai

ns

K o l k h i da

L ow

l an

d

MqinvartsveriMqinvartsveri(5047 m)(5047 m)

A J A R AA J A R A

A B K H A Z I AA B K H A Z I A

TkvarceliTkvarceli

DzvariDzvari

SenakiSenaki

SamtrediaSamtredia

TkibuliTkibuliSachkhereSachkhere

KhashuriKhashuri

MestiaMestia

OniOniKazbegiKazbegi

MarneuliMarneuliTsiteli-Tsiteli-TskaroTskaro

LagodehiLagodehi

AhalkalakiAhalkalakiKazretiKazreti

NinocmindaNinocminda

AkhmetaAkhmeta

GurjaaniGurjaani

OzurgetiOzurgeti

KutaisiKutaisi

TelaviTelavi

RustaviRustavi

MtskhetaMtskheta

AmbrolauriAmbrolauri

GoriGori

ZugdidiZugdidi

AkhaltsikheAkhaltsikhe

T'BLISIT'BLISI

SouthSouthOsset iaOsset ia

Gagra

Ochamchira

Tkvarceli

Dzvari

Senaki

SamtrediaPoti

Kobuleti

TkibuliSachkhere

Khashuri

Mestia

OniKazbegi

MarneuliTsiteli-Tskaro

Lagodehi

AhalkalakiKazreti

Ninocminda

Akhmeta

Gurjaani

Ozurgeti

Kutaisi

Telavi

Rustavi

Mtskheta

Ambrolauri

Gori

Zugdidi

Akhaltsikhe

Suhumi

Batumi

Chinvali

T'BLISIA D J A R A

SouthOsset ia

A B K H A Z I A

R U S S I A N F E D E R A T I O N

T U R K E Y

A R M E N I A A Z E R B A I J A N

Rioni

Inguri

Rion

i

Kvirilk

Iori

Iori

Mtkvari

(Kura)

Iori

Mtkvari (Kura)

Tske

nisc

kali

Alazani

B lack Sea

MingechevirReservoir

LakeSevan

To Soai

To Vladikavkaz

To Zagatala

To Yevlax

To Armavir

To Erzurum

To Erzurum

To Trabzon

Ca

uc

as

us

M

ou

nt

ai

ns

K o l k h i da

L ow

l an

d

Mqinvartsveri(5047 m)

40°E 42°E 44°E

40°E 42°E 44°E 46°E

44°N44°N

42°N 42°N

GEORGIA

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

0 20 40

0 10 20 30 40 50 Miles

60 Kilometers

IBRD 33410R

JULY 2012

GEORGIASELECTED CITIES AND TOWNS

AUTONOMOUS OBLAST CENTER

AUTONOMOUS REPUBLIC CENTERS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

AUTONOMOUS OBLAST BOUNDARY

AUTONOMOUS REPUBLIC BOUNDARIES

INTERNATIONAL BOUNDARIES