Document of The World Bankdocuments.worldbank.org/curated/en/... · PDO Project Development Indicators . ... implement the measures under the rogram outlined in the GSAC Program Document

Document of The World Bank

Report No: ICR00003033

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37570)

ON AN

IDA CREDIT

IN THE AMOUNT OF SDR5.80 MILLION (US$7.77 MILLION EQUIVALENT)

TO THE

KYRYGZ REPUBLIC

FOR A

GOVERNANCE TECHNICAL ASSISTANCE PROJECT

June 24, 2014

Public Sector Institutional Reform Cluster Poverty Reduction and Economic Management Unit Europe and Central Asia Region

1

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective May 31, 2014)

Currency Unit = Som Som 1.00 = US$ 0.0191 US$ 1.00 = Som 52.271

FISCAL YEAR

ABBREVIATIONS AND ACRONYMS

AC Accounts Chamber CAS Country Assistance Strategy CC Coordinating Council CFAA Country Financial Accountability Assessment CoA Chart of Accounts CSMA Civil Service Management Agency DFID Department for International Development (United Kingdom) ECA Europe and Central Asia EU-TACIS European Union TA program for CIS countries FM Financial Management FMIS Financial Management Information System FSU Former Soviet Union GDP Gross Domestic Product GSAC Governance Structural Adjustment Credit GTAC Governance Technical Adjustment Credit HRMIS Human Resource Management Information System IBRD International Bank for Reconstruction and Development IBS Information Business Systems ICB International Competitive Bidding IDA International Development Association IDF Institutional Development Fund IMF International Monetary Fund LCS Law on Civil Service M&E Monitoring and Evaluation MOF Ministry of Finance MOU Memorandum of Understanding NPRS National Poverty Reduction Strategy NBKR National Bank of the Kyrgyz Republic PCL Pharm Control Laboratory PCN Project Concept Note PDO Project Development Indicators PER Public Expenditure Review PIU Project Implementation Unit PMG Project Management Group PPP Purchasing Price Parity PRGF Poverty Reduction and Growth Facility Arrangement PSRMAC Public Sector Resource Management Adjustment Credit SDR Special Drawing Rights SES Sanitary and Epidemiology Service SPS State Personnel Service TGL Treasury General Ledger TMIS Treasury Management Information System TSA Treasury Single Account UNDP United Nations Development Program USAID United States Agency for International Development

Vice President: Laura Tuck (ECAVP) Country Director: Saroj Kumar Jha (ECCU8) Sector Manager: Adrian Fozzard (ECSP4) Project Team Leader: David Nummy (ECSP4) ICR Team Leader: Zachary Mills (ECSP4)

2

KYRGYZ REPUBLIC Governance Technical Adjustment Credit

Contents Data Sheet ........................................................................................................................... 4

A. Basic Information ....................................................................................................... 4 B. Key Dates ................................................................................................................... 4 C. Ratings Summary ....................................................................................................... 4 D. Sector and Theme Codes ........................................................................................... 5 E. Bank Staff ................................................................................................................... 5 F. Results Framework Analysis ...................................................................................... 6 G. Ratings of Project Performance in ISRs .................................................................... 9 H. Restructuring (if any) ............................................................................................... 10 I. Disbursement Profile ................................................................................................ 10 1. Project Context, Development Objectives and Design ............................................. 11 2. Key Factors Affecting Implementation and Outcomes ............................................ 17 3. Assessment of Outcomes .......................................................................................... 24 4. Assessment of Risk to Development Outcome ......................................................... 32 5. Assessment of Bank and Borrower Performance ..................................................... 32 6. Lessons Learned ....................................................................................................... 37 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 40 Annex 1. Project Costs and Financing .......................................................................... 41 Annex 2. Outputs by Component ................................................................................. 42 Annex 3. Economic and Financial Analysis ................................................................. 48 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 49 Annex 5. Beneficiary Survey Results ........................................................................... 52 Annex 6. Stakeholder Workshop Report and Results ................................................... 53 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 54 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 71 Annex 9. List of Supporting Documents ...................................................................... 72 MAP .............................................................................................................................. 73

3

Data Sheet A. Basic Information

Country: Kyrgyz Republic Project Name: Governance Technical Assistance Project

Project ID: P071063 Credit Number: IDA-37570 ICR Date: 06/04/2014 ICR Type: Core ICR

Lending Instrument: TAL Borrower: THE KYRGYZ REPUBLIC

Original Total Commitment:

XDR 5.80M Disbursed Amount: XDR 4.14M

Revised Amount: XDR 4.14M Environmental Category: C Implementing Agency: Ministry of Finance of the Kyrgyz Republic Co-financiers and Other External Partners: N/A B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 05/02/2002 Effectiveness: 08/28/2003 08/28/2003

Appraisal: 03/26/2003 Restructurings:

05/12/2008 05/28/2009 12/20/2010 02/14/2011 05/28/2013 12/18/2013

Approval: 05/15/2003 Mid-term Review: 02/23/2009 03/03/2012 Closing: 12/31/2008 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory

4

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Satisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing Agency: Moderately

Unsatisfactory Overall Bank Performance:

Moderately Unsatisfactory

Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

Highly Satisfactory

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100

Theme Code (as % of total Bank financing) Administrative and civil service reform 23 25 Public expenditure, financial management and procurement

22 75

Other accountability/anti-corruption 22 0 Other public sector governance 22 0 Poverty strategy, analysis and monitoring 11 0 E. Bank Staff

Positions At ICR At Approval Vice President: Laura Tuck Johannes Linn Country Director: Saroj Kumar Jha Dennis de Tray Sector Manager: Adrian Fozzard Helga Muller Project Team Leader: David Michael Nummy Jit Bahadur S. Gill ICR Team Leader: Zachary Mills ICR Primary Author: Zachary Mills

5

F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project objectives are to strengthen the Borrower’s institutional capacity to implement the measures under the program outlined in the GSAC Program Document and to assist with the Treasury modernization. Revised Project Development Objectives (as approved by original approving authority) The objectives were not changed. (a) PDO Indicators

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target Years

Indicator 1: Establishment of a modern computerized treasury system.

Most treasury operations carried out manually.

Automated and modernized treasury system.

N/A Not achieved. The introduction of the automated treasury system could not be completed within the extended closing date. The Government canceled the contract with the vendor on December 16, 2013.

Indicator 2: Public perception of transparency, responsiveness and integrity of public sector as measured by annual surveys has improved. No baseline

surveys were conducted.

Undefined improvement as a target value was not defined.

N/A Not achieved. No surveys were conducted under the project.

(b) Intermediate Outcome Indicators



approval documents)



Indicator 1: Number of declarations of high state officials published annually.

No declarations or provision for declarations.

Increase in number of declarations.

Achieved. All high state officials present and publish declarations.

Indicator 2: Improved access, reliability and timeliness of information about budget execution, to public officials and civil society.

6



approval documents)



Budget reports fragmented and not widely available.

Greater access to and quality of budget execution reports.

N/A Achieved. Comprehensive summary budget reports made publicly available within 6 months of budget execution. Public procurement information more readily available.

Indicator 3: Access to legal database of the Ministry of Justice available to the public online. Legal information

not readily available.

Legal information accessible to the public online.

N/A Achieved. Online access to the centralized database of legal information is available through the website of the Ministry of Justice: www.miniust.gov.kg.

Indicator 4: Government Public Information Center established and functioning. Limited

accessibility of public information to private sector and civil society.

Increased accessibility of Government information.

N/A Achieved. Center fully established and significant number of users from the general public. (Target assumed based on formulation of indicator).

Indicator 5: Number of draft laws and regulations discussed with the private sector and civil society, prior to their adoption. Limited input

from private sector and civil society on Government legislation.

Target value not defined.

N/A Achieved. Public steering councils were established comprising representatives from civil society, think tanks, professional and sector unions, and experts to provide public opinion on formulating and implementing government policy (Target assumed based on formulation of indicator).

Indicator 6: Number of public grievances addressed annually by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC. No data is kept

on the number of public grievances addressed.


N/A Dropped. Activity dropped from project due to availability of grant financing from other donors.

Indicator 7: Coverage, quality and timeliness of audit reports of (sic) Chamber of Control.

7



approval documents)



Reports of the Accounts Chamber (AC) confined to compliance with regulations


N/A Achieved. Improved quality of audit reports, particularly on the annual budget over the course of the project (Target assumed based on formulation of indicator).

Indicator 8: Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation. Budget poorly

presented; allocations managed on a monthly basis according to cash availability.


N/A Partially Achieved. Transparent budget prepared on basis of medium-term framework. Revisions to budget typically take place through due process and not on basis of cash available to the Treasury. Nonetheless budget allocations are revised at mid-year based on cash availability. (Target assumed based on formulation of indicator).

Indicator 9: Improved financial accounting and compliance with laws and regulations in three ministries, due to strengthened internal control systems. Not measured. Target value not

defined. N/A Dropped. Activity dropped from

project due to availability of grant financing from other donors.

Indicator 10: Kyrgyzstandard reorganized to separate regulatory and enforcement functions. Kyrgyzstandard

carried out both regulatory and enforcement functions.

Clear separation of functions.

N/A Achieved. Regulatory and enforcement functions fully separated.

Indicator 11: Percentage of private firms participating in public procurement.

Not measured. Target value not defined.

N/A Achieved. 100 percent of companies currently participate in public procurement. (Target assumed based on formulation of indicator).

Indicator 12: Improved allocation, predictability, availability of resources for health and education services. Not measured. Target value not

defined. N/A Achieved. New financing

formulas have been agreed for both sectors and are currently applied (Target assumed based on formulation of indicator).

Indicator 13: Percentage of recruitments and promotions in the Civil Service made on a competitive basis.

8



approval documents)



Not available as the old civil service law did not require mandatory competitive recruitments or promotions.

Target values not defined.

N/A Partially Achieved. Most chief and senior positions in the government are filled through a competitive selection process. (Target assumed based on formulation of indicator).

Indicator 14: Independent Civil Service Agency established and operational. Independent Civil

Service Agency did not exist.

Fully functional and independent civil service agency.

N/A Achieved. State Personnel Service established and authorized to develop, implement and ensure the sustainable functioning of state personnel policy in both central and local government bodies.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/25/2003 Satisfactory Satisfactory 0.00 2 12/11/2003 Satisfactory Satisfactory 0.08 3 05/24/2004 Satisfactory Satisfactory 0.10 4 06/29/2004 Satisfactory Unsatisfactory 0.10 5 11/17/2004 Satisfactory Satisfactory 0.26 6 12/29/2004 Satisfactory Satisfactory 0.30 7 06/23/2005 Moderately Satisfactory Moderately Satisfactory 0.57 8 06/28/2006 Moderately Satisfactory Moderately Satisfactory 1.43 9 06/29/2007 Moderately Satisfactory Moderately Satisfactory 1.58

10 06/24/2008 Moderately Satisfactory Moderately Unsatisfactory 1.85

11 03/17/2009 Moderately Satisfactory Moderately Satisfactory 2.00 12 12/30/2009 Moderately Satisfactory Moderately Satisfactory 2.46 13 05/21/2010 Moderately Satisfactory Moderately Satisfactory 2.81

14 12/15/2010 Moderately Satisfactory Moderately Unsatisfactory 3.16

15 03/12/2011 Moderately Satisfactory Moderately Satisfactory 3.96 16 05/24/2011 Moderately Satisfactory Moderately Satisfactory 3.96

17 12/28/2011 Moderately Unsatisfactory

Moderately Unsatisfactory 5.03

18 07/16/2012 Moderately Moderately 6.02

9

Unsatisfactory Unsatisfactory







H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

05/12/2008 No MS MU 1.80 Closing date extension

05/28/2009 No MS MS 2.08

To align the PDO text in the PAD with that in the legal agreement (part of a Bank-wide exercise).

12/20/2010 No MS MU 3.00 Closing date extension 02/14/2011 No MS MS 3.50 Closing date extension 05/28/2013 No MU MU 6.00 Closing date extension

12/18/2013 No

MU MU 6.35 Canceling the undisbursed credit amount of $2.5 million due to contract termination

I. Disbursement Profile

10

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal The Kyrgyz Republic is a small, land-locked country with a population of 5.6 million. Following independence in 1991, income levels declined by about 60 percent as budgetary transfers from the Soviet Union ended, traditional trading relationships were disrupted, and the central planning system collapsed. At appraisal, agriculture accounted for about 40 percent of GDP and gold and electricity were the two most important exports. The country undertook major reforms beginning in 1993 to create the basic foundations of a market-oriented economic system, including a market-based price system, substantial private ownership of key productive assets, and an open international trade regime (the Kyrgyz Republic was the first CIS country to join the World Trade Organization). A large number of small and medium enterprises emerged during this period. The regional financial crisis in 1998 halted economic recovery. It also highlighted continuing vulnerabilities caused by large financial imbalances in the energy sector, fragile public finances, inefficient safety nets, and a poorly developed banking sector. The primary purpose of the Governance Technical Assistance Credit (GTAC) approved in May 2003 was to support the implementation of the reforms included in the two-tranche Governance Structural Adjustment Credit (GSAC) which was approved by the Bank at the same time. These projects were launched in an improved macroeconomic context as the country had made significant progress in attaining macroeconomic stability and restarting economic growth, averaging 4.8 percent in 1999-2001. The fiscal deficit had fallen from about 11.3 percent of GDP in 1998 to 5.2 percent of GDP in 2002, mostly as a consequence of expenditure cuts. The current account deficit declined from an average of 18.4 percent of GDP in 1996-98 to 3.2 percent of GDP in 2002. As a result of the fiscal and monetary tightening, inflation fell to 2.3 percent in 2002. Despite the improving economic and fiscal situation, the Kyrgyz Republic remained the second poorest country of the Former Soviet Union (FSU) republics and one of the poorest countries in the world. In 2001, per capita income was less than US$300. Severe poverty affected 13.5 percent of the population and 47.6 percent were considered poor with incomes below US$4.30 per day (PPP). Poverty was highest in rural areas, but there were large regional disparities. Access to public services such as running water, public sewerage, reliable electricity, district heating and telephone service was very low. The country also had a heavy burden of external debt, which stood at 477 percent of Government revenues in net present value terms at the end of 2001. Civil servants were poorly paid and often sought alternative sources of income. Macroeconomic Policy Framework: A 3-year arrangement under the Poverty Reduction and Growth Facility (PRGF), approved by the Executive Board of the IMF in December 2001, provided the basic underpinnings for the macro-policy framework. The

11

Government’s reform efforts had also been supported through adjustment lending by the World Bank. Seven prior Bank adjustment operations provided support for wide-ranging and ambitious policy and institutional reforms in the following areas: privatization and enterprise reform; agricultural privatization; improvements in the business environment; financial sector reform; public sector resource management; pension reform; and power and gas sector reform. These operations helped the Kyrgyz Republic to orient its policy and institutional framework to the principles of a free market economy. Governance. The National Poverty Reduction Strategy (NPRS) recognized that weak governance and an ineffective state were serious hurdles to achieving development goals. At the level of central and local administration, policy making capacity was extremely weak, implementation of policies was poor and often non-existent, transparency and accountability were inadequate, and Government interference in business activities was excessive. The civil service also suffered from political interference. Its professional capacity and accountability for performance were low as real salaries declined and motivated staff became demoralized and left for other opportunities. Surveys of households and firms confirmed that corruption was a systemic problem in public agencies. Moreover, the judiciary and law enforcement agencies faced serious capacity and integrity challenges that reduced their ability to uphold the law. Political parties were weak, politics was personalized and characterized by populist measures to mobilize political support and rent-seeking behavior. NPRS noted that the legal framework suffered from lack of public discussion of proposed laws and regulations and inadequate public access to normative acts. For these reasons, the NPRS accorded a high priority to measures that strengthened governance and accountability. Public Financial Management. Economic and sector work in 2002-2003, such as the Public Expenditure Review (PER) and the Country Financial Accountability Assessment (CFAA), identified a moderate to high degree of fiduciary risk in the use of public resources. Critical weaknesses included the absence of effective cash planning, weak institutional arrangements for assessing the effectiveness of budget implementation, weak internal controls further compounded by the absence of an internal audit function, insufficient accountability arrangements for state-owned enterprises, weak external audit, and weak capacity in the legislative body to provide effective parliamentary oversight over the executive. The Government recognized the need to modernize its treasury operations and was strongly encouraged to do so by the IMF. The country already had the requisite legal and regulatory framework for treasury operations but all treasury functions were carried out manually. Government and its external partners believed that computerization would help to address shortcomings of the manual system related to transparency, informality, control of funds and inefficiency. Rationale for Bank Involvement. The reforms included in the Government’s program were urgently needed to successfully implement the poverty reduction agenda. The Bank had been actively involved in supporting reforms to key public sector institutions through various operations, including the Public Sector Resource Management Adjustment Credit (PSRMAC), projects to reform the health sector and Institutional Development Fund (IDF) grants to strengthen public procurement. GTAC’s primary objective was to provide

12

technical assistance to design and implement the policy and institutional reforms supported by the GSAC (see Table 1). Extensive consultations with the Government and the donor community helped to build consensus around the GSAC-supported policy actions and the technical assistance activities to be provided through the GTAC. The Bank was identified as the lead partner given its global experience in implementing public sector reforms, its country knowledge and strong relationships with counterparts and stakeholders. The Bank was only one of several partners supporting a complex, multi-sector reform program and the GTAC was only one of several Bank-financed projects supporting these reforms. However, the Bank was identified as the only donor with the expertise and resources needed to support the computerization of treasury operations. Table 1. Technical Assistance for the GSAC GSAC Objective Source of Technical

Assistance I. Enhancing Transparency, Openness and Accountability of the Public Sector to External Stakeholders (i) Improving Personal Accountability of Political Officials and Civil Servants WB-GTAC

(ii) Strengthen Voice and Participation by Improving Access to Official Information WB-GTAC

(iii) Improving the Interaction between Regulatory Agencies and Business Enterprises

USAID; WB-FINBUS; WB-Health II Project

(iv) Strengthening External Audit DFID II. Improving Efficiency and Effectiveness within the Public Sector (i) Strengthening Public Expenditure Management

(a) Establishing a more Strategic and Transparent Budget Formulation Process DFID

(b) Strengthening Accountability and Transparency in Budget Execution IMF, WB-GTAC

(ii) Improving the Internal Control Environment DFID (iii) Improving Transparency, Value for Money and Accountability in Public Procurement WB-GTAC, WB-IDF

(iv) Improving Service Delivery in the Health Sector WB-GTAC; WB-Health II Project

(v) Enhancing the Effectiveness of the Civil Service WB-GTAC; EU-TACIS (vi) Streamlining the Structure of Government Ministries and Agencies DFID; UNDP; EU-TACIS

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The objective of the GTAC was to strengthen the Borrower’s institutional capacity to implement the measures under the Program outlined in the GSAC Program Document and to assist with the Treasury modernization. The original results indicators spanned several sectors and were designed support implementation of GSAC program measures:

13

• Number of declarations of high state officials published annually • Number, quality and timeliness of key pieces of official information, specified in

the GSAC Policy Matrix, published annually • Number of draft laws and regulations discussed with the private sector and civil

society, prior to their adoption • Number of public grievances addressed annually, by the President's

Administration, the Office of the Government and the regulatory agencies covered by the GSAC

• Coverage, quality and timeliness of audit reports of Chamber of Control (COC) • Public perception about the transparency, responsiveness and integrity of the

public sector • Increased usefulness of the budget as a strategic policy instrument, as a result of

improvements in its formulation process, content and presentation • Improved access, reliability and timeliness of information about budget execution,

to public officials and civil society • Improved financial accounting and compliance with laws and regulations in three

ministries, due to strengthened internal control systems • Percentage increase in the number of private firms participating in public

procurement • Improved allocation, predictability, availability of resources for health &

education services • Percentage of recruitments and promotions in the Civil Service made on a

competitive basis. Only one of these indicators was related to the second element of the PDO to assist with Treasury modernization: improved access, reliability and timeliness of information about budget execution, to public officials and civil society. Text in the table at PAD Annex 1 (Project Design Summary) identified ‘computerized Treasury system implemented’ as an output-level rather than an intermediate indicator. No indicators were formulated to track progress in the implementation of Treasury modernization or the impact of these reforms. The structure of the intermediate indicators bore little relation to the relative weight of the project components, focusing on the activities supporting GSAC implementation rather than the longer-term needs of Treasury modernization. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, Although the GTAC was extended several times, no revisions were made to the PDO. Relevant project results indicators were updated to reflect new closing dates and two results indicators were dropped, because the Borrower (during implementation) gained access to grant funds from other development partners to implement the corresponding activities. No key indicators were formulated to track progress in the implementation of Treasury modernization or the impact of these reforms despite repeated project restructuring, even when the Treasury modernization became the only active project component.

14

1.4 Main Beneficiaries The initial project design anticipated that benefits of the project would accrue to:

• The people of the Kyrgyz Republic through more effective and transparent use of public sector resources and improved access to information and grievance mechanisms, both critical to the delivery of public services key to addressing poverty reduction and other developmental goals;

• The private sector, as a result of improvements in regulatory policies and practices and public procurement;

• Civil servants, on account of improvements in applicable policies, processes and actual practices; and

• The political executive, due to the availability of a more effective state machinery to assist in policy-making and implementation.

Poverty would be reduced indirectly through improved public services resulting from better public financial management and improved business confidence resulting in increased investment and faster economic and employment growth. 1.5 Original Components GTAC was intended to support a wide range of reforms, including but not limited to those supported by GSAC program. GTAC had three components: Component 1: Technical assistance to support the design and implementation of the reforms included in the GSAC (US$1.98 million). GTAC provided TA for key reforms supported by the GSAC, complementing grant assistance provided by other donors. GTAC assistance covered a five year implementation as compared to the two years for the GSAC in order to enable institutionalization and sustained implementation of difficult public sector reforms. The key reforms to be supported by GTAC were: strengthening voice and participation by the private sector and civil society in public decision-making; increasing transparency in public procurement; improving service delivery in education, social protection and health services; strengthening the professionalism and effectiveness of the civil service by increasing its independence and merit-based personnel actions; streamlining the structure of three ministries via functional reviews; refining the NPRS and implementing a monitoring and evaluation (M&E) system to measure progress in achieving NPRS objectives. Component 2: Modernization of the treasury (US$6.37 million). This component was to finance consulting services, goods and training for the modernization of treasury operations through the implementation of a Treasury Management Information System (TMIS), restructuring for Treasury General Ledger (TGL), and implementation of a new Chart of Accounts (CoA) in full compliance with the new GFSM2001. It was also to provide assistance in (a) preparation of user requirements, conceptual design of the system, system architecture and software and hardware specifications for the TMIS; (b) selection, procurement, adaptation and implementation of an off-the-shelf software

15

package; (c) procurement and implementation of hardware and communication equipment; (d) development of the regulations for the implementation of the system; and (e) training of staff. It was envisaged that the IMF would be closely involved in the implementation of the TMIS and would also provide technical assistance. The system would first be piloted in the Central Treasury and two or three pilot oblasts before being replicated to all regional treasury offices. Component 3: Project management (US$ 0.66 million). This component was intended to finance consulting services, equipment and training for the Project Management Group and Project Implementation Unit. It was also intended to finance the communication of project objectives and activities to stakeholders, stakeholder surveys and project audits. GTAC combined two distinct sets of activities: public sector and institutional reforms supported through GSAC; and a treasury modernization project comprising ICT hardware and software, associated regulatory reforms and capacity building. The modest credit (SDR5.80 million) could only finance an initial set of activities within the vast public sector institutional reform agenda of the time, but it was considered prudent to proceed with a project of this size initially so that the Government could draw on the experience gained from implementing this TA project in addressing its wider TA needs for supporting public sector reform. 1.6 Revised Components There were adjustments in the structure of Component 1 during implementation. Two subcomponents were dropped in their entirety because the Government was able to access grant financing from other donors: sub-components 6, streamlining the structure of Government ministries and agencies; and sub-component 7, refining NPRS and monitor and evaluate progress. The two intermediate indicators applicable to these subcomponents were also dropped. In addition, specific activities were dropped from the subcomponents that remained under Component 1:

• Strengthened Voice and Participation: regulatory impact assessment of proposed laws and regulations; support to public grievances by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC.

• Improving Service Delivery in Health and Education: improvement of inspection regimes of Sanitary and Epidemiology Service and Pharm Control Laboratory PCL; funding for health insurance for pensioners and the unemployed accordance with budget allocations.

• Improving the Internal Control Environment: improvement in financial accounting and compliance with laws and regulations in three ministries through strengthened internal control systems.

The specific reasons for these adjustments in the scope of the project are not clear from the project record: some activities were addressed through complementary projects, others were dropped because priorities changed.

16

1.7 Other significant changes There were four extensions of the project closing date with the project finally closing – without concluding project activities – on December 31, 2013 five years beyond the original closing date of December 31, 2008. The final project restructuring completed on December 18, 2013 cancelled the remaining US$2.5 million of the credit. There were no significant changes in implementation arrangements.

2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project preparation commenced in December 2000. Support for preparation and capacity-building was also partly financed with a Policy and Human Resource Development (PHRD) Grant (TF026448) for US$729,250 from the Government of Japan which was formally associated with the preparation of the GSAC program but covered preparation of several GTAC activities. These included technical assistance for:

• an Anticorruption Strategy (US$56,350); • Public Administration Reform (US$460,600) including the preparation of an

administrative and civil service reform program; a draft law on public service; an assessment of budgetary employment and remuneration; and an analysis of the institutional environment within which three to four key ministries or government institutions function; and

• Public Expenditure Management (US$48,550) including development of proposals for reforming budget management in the health and education sectors.

GTAC was grounded in diagnostic work, including but not limited to the analysis financed by the PHRD Grant, which had supported the development of the GSAC program. There was strong national ownership of the institutional reform program, an extensive consultation process with the government and with society more broadly. The context appeared very conducive for reform. Against this background, and although the size of the credit was small, the project scope and activities were thought to be achievable. Indeed, the project scope and objective were also reviewed by a Quality At Entry (QEA) exercise which rated the project under preparation as Highly Satisfactory. However, in hindsight, the Treasury modernization component of the project was inadequately prepared and the MOF far from ready for implementation. The design of the Treasury modernization component was similar to other Bank-financed treasury modernization projects prepared in the early 2000s. A detailed review of treasury operations had been completed by a joint IMF-WB team during GTAC appraisal. The review concluded that Government was now ready for a successful transition, within a period of about three years, to a fully functional modernized treasury that employed contemporary information technology. However, this timeline proved to be unrealistic. Preparatory work on Treasury modernization, in particular the detailed specifications for

17

the TMIS, the business process re-engineering requirements and implementation of related institutional reforms, only began after project became effective. 2.2 Risks Assessment and Mitigation Most of the significant risks were correctly identified and recognized during preparation, and risk mitigation measures put in place, although these risk mitigation measures were not always effective. However, the risk of major political upheaval was one not identified at appraisal: indeed, such a risk in the case of the Kyrgyz Republic in 2003 would have been considered a ‘black swan’ event and inconceivable at appraisal. The risks identified at appraisal, and what actually happened, are summarized in Table 2 below. The PAD failed to identify risks specifically related to the implementation of Treasury modernization. Three risks stand out, all of which subsequently materialized: first, the lack of technical and management capacity to design and manage a complex TMIS contract and the business reengineering requirements of Treasury modernization; second, internal resistance to reform from those with vested interests in the existing system and the flexibility and informality that it facilitated; and third, differences in perspectives regarding systems requirements across institutions and within the Ministry of Finance. GTAC’s failure to manage these risks contributed to the failure of the project’s Treasury modernization component.

Table 2. Risks Identified at Appraisal and Outcome

Risk Rating at appraisal

Outcome

Political commitment to implement the NPRS and improve governance and public sector performance is weakened

H Despite the voice, accountability and transparency gains from the implementation of GTAC, there was some backtracking on politically sensitive reforms such as the de-politicization of the civil service.

Demand for good governance and improved public sector performance amongst the private sector, civil society and the public is not strong

S Strong demand for good governance was sustained. Weak governance, high-level and administrative corruption and an ineffective public sector were underlying factors behind the 2005 and 2010 revolutions.

Civil society, the private sector and the public do not contribute to policy debates, or make optimal use of publicly disseminated official information to enforce accountability in the public sector

H Some elements of the publicly disseminated official information (e.g. income and asset declarations, budget information) have proven to be useful instruments to strengthen accountability.

Government commitment to implement reforms included in the GSAC is not strong.

S Government commitment to implement GSAC reforms was strong at the outset but diminished: the GSAC ICR rated GSAC performance as MU.

The Government is unable to overcome opposition to reforms from vested interests

H This was true in the periods before the two revolutions, when vested interests stalled or pushed reforms.

Government lacks the capacity to H Weak institutional capacity has been an issue,

18

Risk Rating at appraisal

Outcome

implement the reforms effectively exacerbated by losses of trained civil servants across ministries (including the MOF) both before and after each of the revolutions.

There is a lack of coordination of reform efforts between the President’s Administration, Office of the Government, MOF and concerned line agencies

S The turnover of high-level officials led to a non-functional GSAC/GTAC Coordination Committee. Although this intensified coordination challenges, project decisions continued to be taken, to the extent possible, by Technical Committees.

Relevant managers and staff of the President’s Administration, Office of the Government, MOF and concerned line agencies are not actively involved in the design, development, quality control and implementation of project initiatives

S Engagement of officials in design and implementation of project activities varied across agencies and was not always constructive. Vested interests opposed to greater transparency, control and accountability in the management of public finances have resisted the Treasury modernization.

Donor coordination in parallel financed project activities or in implementing independently executed activities is weak

S In practice, donor coordination remained fairly strong even in the challenging environment.

Adequate allocation of funds for the project is not made in the national budget and availability of counterpart funds is inadequate

H This was not a problem. Indeed, the opposite was the case: government spent its own funds to out in place the ICT backbone and infrastructure for the TMIS while the World Bank refused to agree to additional financing at a critical time in 2009 when this was needed to meet the unforeseen increase in the estimated cost of the TMIS/HRMIS.

Overall risk rating H Risk rating – H (high risk), S (substantial risk), M (modest risk), N (negligible or low risk) Source: GTAC PAD 2.2 Implementation By the time of the Mid-Term Review (MTR) in March 2008 the majority of Component 1 activities had been completed, with only a small allocation left for work on strengthening capacity in the civil service reform. The MTR assessment at that time was that the reforms envisaged by GSAC and supported by GTAC had progressed at a varying pace during different periods due to both political developments and the complex nature of the institutional reforms. Nonetheless, the MTR considered the reforms as largely successful and prospect of sustainability for these reforms to be high. Implementation of the Treasury Modernization component proved to be problematic. Procurement issues and the political events of 2005 led to delays in the first three years of the project. As a result, GTAC had only disbursed 23 percent of the credit proceeds at MTR. A project extension was needed to launch procurement of the TMIS/HRMIS software (ICB-1) and hardware (ICB-2), which only started in 2009. Although the installation of the ICT hardware (ICB-2) was successfully completed, subsequent implementation delays relating to the implementation of the TMIS/HRMIS software

19

(ICB-1) led to three additional extensions of the closing date, more than doubling the original implementation timeline. In the end, this component could not be satisfactorily completed. The factors that affected implementation of the Treasury modernization component are outlined below. Delays in the preparation of technical specifications for TMIS and HRMIS. The decision to submit the GTAC for Board approval alongside the GSAC operation meant that the GSAC preparation schedule was driven by the country’s urgent budget support needs. Preparation and the implementation readiness of GTAC activities were a secondary consideration. Consequently, preparatory work on the Treasury modernization component was only undertaken after the project became effective in August 2003 when a consultant firm was contracted to define the functional requirements and prepare the technical specifications for the TMIS and HRMIS. Although a bidding document for this tender was issued to a shortlist of three qualified companies in February 2004, the evaluation committee became deadlocked and opted for an external evaluation of the proposals. An external expert was selected in December 2004 to evaluate the proposals, and an international firm selected. The TMIS and HRMIS design reports were finally completed at the end of 2006, more than three years after effectiveness. These delays had significant implications for procurement, contracting and contract implementation. In the absence of detailed specifications, there was no proper basis for estimation of the cost of the software and related hardware at the time of Project appraisal, and these costs turned out to have been underestimated. The specifications, scope and business re-engineering requirements of the Treasury modernization were not thoroughly reviewed during project appraisal and agreed during negotiation of GTAC financing. Lack of agreement on these details contributed to delays during implementation and ultimately to the failure of the Treasury modernization component. Deficiencies in the TMIS tender process. The TMIS as described in the Treasury modernization component of the PAD did not envisage modules for human resource management, budget preparation, debt servicing, and agency accounting on an accrual basis. However, these modules were included in the technical requirements issued to bidders. This added to the system cost and complexity. When the first international competitive bidding (ICB-1) process for the TMIS and HRMIS application software and related servers was evaluated by the Government’s Evaluation Committee, the lowest evaluated bid was found to be nearly US$2 million above the budget allocation. The Evaluation Committee decided to rebid, hoping that the participation of more companies would lower the bid price. When the tender was relaunched, specifications were not updated to incorporate advances in information technology, changes in the country’s pay system, the preferred structure and content of the single CoA, or the MOF’s approach to adopting international accounting and reporting standards. Some of the TMIS functional requirements were not clearly defined, including terms such as ‘preparation of a draft national budget’ and ‘preparation of a medium term financial plan’. The expectations of MOF’s Treasury and Budget departments differed on the extent to which budget preparation functionality would be included in TMIS. The vendor agreed to produce a customized module, which it demonstrated to the Government in mid-2012, but the MOF was unwilling to accept this module. The MOF and the vendor had very different

20

expectations regarding the scope of system customization required to satisfy the requirements of the contract signed in 2009. These differences in expectations regarding the scope of TMIS functionality and system customization were compounded by delays on the part of the authorities in finalizing two system design elements: the CoA and the interface between the Treasury Single Account (TSA) and the banking system. Delays in the approval of the Chart of Accounts. A unified CoA was expected to provide the backbone for classification, accounting, transaction processing, reporting and control within the TMIS. However, the development of the CoA took much longer than anticipated. The revisions to the budget classification and CoA structures only started in 2006. A unified CoA was expected to be introduced in 2009 and shared immediately with the winning bidder of the ICB-1 second stage. In reality only a partial version, relating to Treasury, was ready at this time. The selected vendor stated that it could provide a unified CoA as part of its software package. After a few months, however, it became clear that the vendor’s standard CoA did not meet the needs of Government operations. After a year of negotiations, the vendor eventually agreed that a new unified CoA was needed. The CoA was finally completed in September 2012, nearly two years later than scheduled. Lack of clarity with regard to the design of the Treasury Single Account and NBKR interface. In 2007, before launching bidding on TMIS/HRMIS, the MOF signed an agreement with the National Bank of Kyrgyzstan (NBKR) on managing gross payments in online mode and establishing direct access for the Treasury to the central bulk clearance arrangements. The NBKR did not have the capacity to perform transactional banking tasks for the Government and expected that the existing practice of using commercial banks, or establishing access to the interbank payments system for Treasury, would be adopted for payment processing, detailed settlements, and transfers between the Government’s accounts. The NBKR maintained a simpler structure for Government accounts and had revised its CoA prior to the 2007 agreement. The NBKR was not consulted about the technical specifications for the TMIS contract. These specifications included a general provision for the design of interfaces with the NBKR and agent (commercial) banks for the transfer of information on accounts, payments, and other matters, and for the receipt of payments data to commercial banks. It also included an outline of the proposed operations of a TSA system that the Treasury expected to be open with the NBKR. After the TMIS contract was signed, disagreements arose on how the TSA should function. The Treasury and the vendor were unable to resolve the details of the TSA arrangements due to the difficulty in defining the mode of TSA operations and the lack of legal/operational framework (such as a TSA agreement or protocol) between the Treasury and the NBKR. In 2012, the vendor agreed to develop an NBKR interface to work within the bulk clearance system. Ultimately, the vendor was not able to produce this software before the contract was terminated. Delays in contract implementation. In January 2012, the Bank was informed of the Government’s intention to terminate the contract with the vendor due to the continuous delays in contract implementation. At this stage, the contract was supposed to close by the end of December 2012. Following discussions with the vendor, the Government

21

agreed to continue with the contract. The MOF and the vendor signed an MOU on June 8, 2012 setting out explicit deliverables and timelines. However, progress in TMIS/HRMIS implementation continued to be unsatisfactory as both the MOF and vendor missed key deadlines, and the parties were not able to agree on detailed system specifications for TMIS/HRMIS software, testing and operational acceptance. In early 2013, a new action plan for full implementation was agreed between the parties with the vendor agreeing to demonstrate a fully functional module by December 2013. Delays and communication difficulties persisted, however, and the Government decided to terminate the contract with the vendor on November 27, 2013 citing the vendor’s failure to complete TMIS/HRMIS implementation activities within the agreed timeline. The Government also notified the Bank on December 16, 2013 that it had elected to cancel the undisbursed amount of the credit (approximately US$2.5 million). The MoF and the vendor are expected to be in discussion over the issues in contention. 2.3 Monitoring and Evaluation Design, Implementation and Utilization The GTAC results indicators and results framework reflected the practice in the Bank in 2003. As a TA project supporting the achievement of key policy and institutional reforms under an adjustment credit, the results framework was aligned with the GSAC policy matrix. The results framework was reviewed during the Quality At Entry (QEA) exercise and considered as representing best practice. In retrospect this assessment seems unwarranted: the monitoring and evaluation framework was weak in design and implementation. The use of public perceptions of transparency, responsiveness and integrity of the public sector as the PDO indicator for Component 1 presented problems of attribution since a wide range of factors outside the project would have impacts on those perceptions. Many of the GTAC intermediate outcome indicators measured processes rather than the outputs or outcomes of reforms. Many of the indicators lacked baselines, others lacked target values. In constructing the datasheet, the ICR team has presumed target values based on assumptions of the kinds of outcomes that would be associated with the indicator. Attribution problems also arise because of the design of GSAC and GTAC as parallel operations. Component 1 indicators were monitored by joint GSAC and GTAC teams, focusing on the Government’s progress in achieving GSAC triggers. Close alignment of the GSAC and GTAC results frameworks made it difficult to distinguish which results can be attributed to each of the financing instruments. A significant shortcoming of the M&E original design was the lack of an appropriate monitoring framework for Treasury modernization, the largest of the two components in terms of value. This component featured as a PDO indicator (“Establishment of a modern computerized treasury system”). However, the intermediate indicators were formulated in such a way as to assess the performance of the Treasury and Budget systems as a whole rather than specific components (“Improved access, reliability and timeliness of information about budget execution, to public officials and civil society”; “Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation”). The intermediate indicators provided

22

little guidance to stakeholders on the intended scope and functional capabilities of the proposed Treasury system and related reforms. Implementation of the M&E framework was also deficient. The PDO indicators required the use of periodic surveys of stakeholders of households, firms, and public officials to assess perceptions about transparency, responsiveness, and integrity of the public sector. No baseline survey was undertaken and the periodic surveys were dropped without explanation. Data was gathered in relation to intermediate indicators, though several of these indicators were dropped as the scope of Component 2 was adjusted in the course of implementation. It is unclear to what extent M&E information was used to feed back into management decisions and guide GTAC implementation. Monitoring of the indicators related to Component 1 ceased following the MTR in 2009, by which time most of the Component 1 activities were completed. Consequently, it is not possible to assess the status of these indicators at the time of project closing. These shortcomings in the GTAC M&E design were noted in the MTR and several subsequent ISRs. However, no action was taken to adjust the M&E framework to better align project objectives, scope, content and results indicators. Perhaps the most obvious missed opportunity was the 2009 restructuring undertaken to align the PDO text in the PAD with that in the legal agreement as part of a Bank-wide exercise. Other opportunities to restructure the results framework arose during subsequent project extensions. Indeed, the need for restructuring of results indicators was discussed in ISRs as late as 2013. 2.4 Implementation Arrangements A GSAC Coordination Council (CC) was set up to provide strategic guidance to both the GSAC and GTAC projects. The CC’s role was to: oversee and track the implementation of policy changes; approve and submit draft legislative acts and regulations to the Parliament and the President; help to resolve inter-agency coordination issues; and assist in overcoming resistance to some of the reforms. The CC was chaired by the Deputy Head of the President’s Administration and included the Vice Minister for Economic Development, Trade and Investment, the Minister of Finance and other members drawn from different branches of government. Between the 2005 and 2008 when the GSAC closed, there were at least eight changes in the CC Chairman together with other changes in CC membership. Loss of continuity undermined the CC’s effectiveness and the coordination function fell largely to the Ministry of Finance, which became the central government institution with relatively more institutional continuity. Once Component 1 activities had been completed, the multi-sector CC oversight arrangements were no longer relevant to GTAC implementation, since Component 2 was solely the responsibility of the Ministry of Finance.. Originally, a Project Management Group (PMG) was established to manage GTAC project implementation. PMG was led by a director with a manager to run day-to-day project operations and deal with coordination and reporting. The PMG was financed by the project. Funds were provided for local consultants as thematic coordinators to assist

23

the government agencies to push forward the GSAC agenda. These coordinators covered: civil service reform and public administration; public expenditure management and related areas; treasury modernization; and information technology. Separately, a Project Implementation Unit (PIU) provided procurement and financial management services. These arrangements proved dysfunctional. The PMG was dissolved in the early stages of GTAC implementation and the responsibilities were taken over by the Project Implementation Group; individual consultants continued working to support the government’s efforts to implement GSAC working directly with the respective institutions, while the PIU functions were transferred to another existing PIU in the Ministry of Finance. 2.5 Safeguard and Fiduciary Compliance The project did not activate any safeguards. No fiduciary compliance issues were flagged during implementation and the fiduciary ratings were Satisfactory through most of project implementation. 2.6 Post-completion Operation/Next Phase There are currently no concrete plans for a follow-up project to GTAC. The joint donor trust fund for public finance capacity building, which is not linked to GTAC, has provided post-GTAC support for PFM reforms and will continue to do so. Other institutional reforms are supported by the government’s own resources with assistance from other development partners such as the EU, USAID and DFID. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The project objectives were strategic and remain relevant. The Country Partnership Strategy for 2013-17 includes a pillar on governance, which includes actions related to public administration and public service delivery. Improvements in governance are seen as essential for achieving poverty reduction and boosting shared prosperity. 3.2 Achievement of Project Development Objectives The PDO was formulated in two parts: first, to strengthen the Borrower’s institutional capacity to implement the measures under the program outlined in the GSAC Program Document; and second, to assist with the Treasury modernization. In relation to the first part of the PDO, the 2009 GSAC ICR concluded that “the objectives for Pillar 1 on public sector transparency and responsiveness were essentially achieved”. Pillar 1 of GSAC contained most of the elements included in Component 1 of GTAC. The overall ICR rating of the GSAC was ‘Moderately Unsatisfactory’ because at the time of writing, 2009, there was evidence of backtracking in areas such as depoliticizing the civil service, in procurement, and in overall transparency. However,

24

after the GSAC ICR, the 2010 revolution led to a change in Government and many of the reforms proved to be resilient. The independent Civil Service Agency continues to operate, competition-based appointments are more common – though political patronage and other considerations are still important in many selections – and procurement and transparency reforms have been followed-up by subsequent Governments. The per capita health and education financing models developed with GTAC support are still in use today; so too are the income and asset reporting system for public servants, the improvements in budget presentation and reporting, the mechanisms for public consultation around the policy process and the public information centers. GTAC played an important role in the implementation of all of these reforms. Given the impact and resilience of the GTAC supported reforms a rating of Moderately Satisfactory is justified. Figure 1. Governance Indicators in Kyrgyz Republic, 2003-2012

Source: World Governance Indicators (2014) The PDO sought to assess the impact of these reforms through surveys of public perception of transparency, responsiveness and integrity of public sector. Since the surveys were never undertaken, it is not possible to assess performance against this indicator. An alternative source of data from the World Governance Indicators shows that ‘Voice and Accountability’ and ‘Government Effectiveness’ indicators were volatile over the project period (see Figure 1). It is unclear what impact the GTAC project had – or could have had - on these indicators or public perceptions. Broader governance concerns related to political stability are likely to have been a far more important consideration of public perceptions than any of the GTAC-supported governance reforms no matter how successful they may have been. Progress against the second part of the PDO, Treasury modernization is considered Unsatisfactory. At GTAC closing, the MoF did not – and still does not – have a functioning ‘modern computerized treasury system’, the PDO performance indicator. This investment has not been a complete failure. By the MOF’s own account the Ministry

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0 2003200420052006200720082009201020112012

Inde

x R

ange

(-2.

5 lo

w to

2.5

hig

h)

Voice and Accountability

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0 2003200420052006200720082009201020112012

Inde

x R

ange

(-2.

5 lo

w to

2.5

hig

h)

Government Effectiveness

25

now has the foundations needed for the implementation of TMIS system. The data centers (ICB-2) were successfully completed. This has improved the network connectivity for the whole MoF structure. The legal basis for a modern automated treasury system is now in place. The Ministry has gained valuable experience and is now better placed to supervise future system development. Work in this area continues, financed by a recipient-executed trust fund for public financial management. 3.3 Achievement of Results by Component Component 1. TA to support implementation of the reforms included in the GSAC Access to information on income and assets of high state officials (political appointees) has increased. The government adopted legislation on income and assets declarations of high state officials and their immediate family members. GTAC-financed consultants helped the government to draft and introduce necessary by-laws regulating the declaration process for this category of civil servants and make the whole process operational. As a result, currently about 98 percent of political appointees declare their incomes and assets on an annual basis and submit the declarations to the Civil Service Agency. According to the law, the CSA has the right to verify the information provided by communicating with the Tax Inspection and to report to the President, Parliament, Constitutional and Supreme Courts in case incomplete or false information was provided. The agency was also authorized to publish information on incomes and assets of politicians (except details such as addresses and telephone numbers) on the agency magazine and web-site. While the information on income and assets of senior officials was available in numerous publications, it is not widely used by civil society as an instrument for strengthening accountability and integrity in the public sector as originally expected. Media and NGO awareness on how to use such information has improved over time. A Public Information Center was established and is now functioning. The PIC was opened in late 2005, under the President’s Administration, with GTAC financial support for procuring equipment and software, as well as for training of staff. The PIC provides a wide range of services for the general public, including free access to internet-based information databases and a PIC library, IT training for different groups of citizens, public discussions on various topics. The government has established regional mini-PICs. This success became possible due to strong support from the President’s administration and dedicated PIC staff. The PIC successfully collaborates with many donors working in the country. Access to legal information was improved. The GTAC helped the MOJ create a database of laws, treaties, legal acts and judicial decisions, publicly accessible through the MOJ web-site (www.minjust.gov.kg) which in 2007 won first prize as the most popular government site. A Law on access to public information was adopted at end-2006. The Law was drafted by a Working Group, which included representatives from the government and civil

26

http://www.minjust.gov.kg/

society with active participation of GTAC expert consultants. According to the law, all government entities (including local self-governance bodies) are obliged to make all information related to their performance public other than so called “state secret information”, either by publishing this information in mass media, or upon requests of citizens in written format. The law also provides tools for individuals to appeal against denials of requests for information. Improvements in private sector and civil society engagement in decision-making. Despite the modest financial resources allocated a number of activities were implemented to facilitate citizens’ access to public information and their participation in drafting of laws and regulations. Public steering councils were established comprising representatives from civil society, think tanks, professional and sector unions, and experts to provide public opinion on formulating and implementing government policy Improved transparency and value for money in public procurement. GTAC helped strengthen capacity of the State Procurement Agency and line ministries, improve public procurement rules and increase transparency of public procurement process. About 4,500 staff of procuring agencies were trained in the new procurement legislation and rules, many through a Public Procurement Regional Training Center. New tender documents conforming to the 1997 Public Procurement Law, guidelines and manuals were developed and adopted. The Government initiated publication of a monthly Public Procurement bulletin. Working with the local branch of Transparency International, the Government raised awareness of new procurement rules among business circles. An internationally recognized company undertook an audit of selected items procured from the republican budget and made recommendations for improvement of the procurement process to achieve greater value for money. Improved service delivery in health, education and social protection. GTAC provided technical assistance to strengthen financial management in support of service delivery. This included the development of more equitable formulae for grant transfers for health and education which were then under local government jurisdiction. The health sector financing reforms were not implemented owing to the transfer of responsibility for all health institutions to the republican budget. However, a more transparent and objective geographic allocation of health resources was achieved through the application of the new financial rules for the sector based on per capita and caseload criteria. In the case of education, the GTAC-supported capitation-based grants were adopted in March 2008, when the government decided that reforms in education sector had made sufficient progress to use per capita financing for resource allocation. Improved transparency, accountability and effectiveness of the civil service. GTAC supported civil service reforms that sought to eliminate patronage by ministers and their appointees in designating civil servants, introduce merit and rule based appointment and promotion and improve security of tenure for professional civil servants. Following the approval of the 2004 Civil Service Law (CSL), GTAC helped establish the CSA. GTAC financed advisory services developing a rule based system of appointments and dismissals which separated political appointees from career civil servants and introduced

27

State Secretaries to head the permanent civil service in all ministries. A massive training program for civil servants in Human Resource Management was conducted throughout the country, including special programs for CSA staff. During the revolution of March 2005, when there was considerable turnover of political appointees, the newly established CSA managed to protect some professional civil servants from illegal dismissals. The system of appointments was significantly improved with the reported number of appointments made through due process being close to 100 percent at the time of the MTF, compared with about 50 percent at the end of 2005. To reduce the personal interface with candidates for posts, the CSA, with GTAC financial support, established a Testing Centre, where all candidates undergo computer-based assessments of their general competencies. Most government bodies now have State Secretaries who manage human resources and budget of the ministries to meet their objectives. While the responsibilities and capacity of State Secretaries need to be further developed, this reform has had a positive impact on the quality of civil service and agency management. Strengthening budget preparation, reporting and oversight. GTAC financed consultants to help the Treasury develop the GFS-2001 compliant budget classification and provided technical support to implement the Medium Term Budgetary Framework (MTBF). The budget is now presented based upon this new classification structure and is presented with two years of historical data and indicative ceilings for the next three years. The linkages between the MTEF and the annual budget is, however, extremely weak. GTAC provided technical support for the development of the unified CoA and the creation of the TSA. These reforms contributed to the development of the MoF’s capability to submit comprehensive budget execution reports to the Prime Minister’s Office, the Parliament, NBKR, the National Statistics Committee, and other stakeholders within six months of budget execution. These reports are posted on the official website of the MoF for public access. However, in the absence of an operational TMIS, data from regional treasuries is still compiled from summary reports, and MOF is unable to access all budget execution data at a transaction level in real time. In addition, to work on budget reform and treasury reform, GTAC financed consultants to help the Government draft and pass the law on CoA that stipulated performance audits and an evaluation of the efficiency and effectiveness of public spending in June 2004. Capacity building to the Accounts Chamber was provided through IDF grant. The Parliament now receives the progress report of the Accounts Chamber every September for the previous budget year as well as the opinion on the draft Budget Law for the subsequent fiscal year and the results of the audit reports are widely discussed and referenced in the mass media. The reports, however, are mostly related to the legality, not the effectiveness or efficiency of public expenditure. Component 2. Modernization of the Treasury Establishment of a modern computerized Treasury system. GTAC supported the modernization of treasury operations through the development of a Treasury Management Information System (TMIS). It was originally intended that the TMIS would be piloted in the Central Treasury and two or three pilot oblasts, before being rolled out to all 8 oblasts and 57 rayon treasury offices. TMIS development was

28

interrupted in 2005, resumed and then ran into a series of technical and operational problems discussed at length in section 2.2 above. Ultimately the project closed without delivering a functioning computerized Treasury system. The TMIS will require significant additional work to be made fully operational. Intermediate outputs: the regulatory framework for Treasury automation. GTAC has provided assistance to the authorities in developing the legal and regulatory instruments needed for the implementation of a TMIS. These include: instructions on functioning of the Unified Treasury Account; adoption of a unified CoA and a unified CoA Manual for Users; and a protocol between the MOF and the NBKR on the functioning of the TSA. Delays in the adoption of these regulatory instruments contributed to the delays in TMIS development. Intermediate outputs: the ICT infrastructure for Treasury automation. The Bank had identified the key Information and Communications Technology infrastructure requirement needed for successful implementation of TMIS/HRMIS. The Government put this infrastructure in place using its own budgetary resources, including: reconstruction of the Data Processing Center (DPC) and the Reserve Data Processing Center (RDPC) and laying of fiber-optic communication lines (FOCL) between KyrgyzTelecom Joint-Stock Company, the Central Treasury of the Ministry of Finance, the State Personnel Service, and the Data Processing Center. GTAC financed system-specific infrastructure including: equipment for the central TMIS/HRMIS system, 22 servers, data storage systems and management and monitoring systems; equipment, software and main and local networks installed at regional units of the Treasury; and computers for the Central Treasury and its regional units. While the ICT infrastructure cannot be used to its full operational potential without the TMIS software, the MoF has used these platforms to support the daily operations of legacy systems and automate parts of the budget execution, reporting and support functions using software developed in-house. Establishment of a Human Resource Management Information System. The HRMIS was not identified as a project activity or output in the PAD. However, the HRMIS was discussed in detail in the first aide memoire dated September 2003 as a tool that would support the automation of human resource management functions with the Civil Service Agency as the primary user. Notwithstanding the fact that the HRMIS and TMIS had distinct institutional arrangements, the Bank team advised the authorities that the TORs of the TMIS and HRMIS should be included in a single package bid package to generate sufficient interest from international firms. The HRMIS was included in the TMIS contract signed by the client in 2009. The HRMIS implementation also met significant delays due to disputes over the level of customization and system functionality. In March 2013, the CSA and the vendor had identified a list of 44 changes to be made in the HRMIS component to finalize the software. At project closing, the vendor had completed 22 of these changes and the Civil Service Agency stated that the system did not meet their requirements owing to problems in calculating the payroll and in generating the reports they needed.

29

Component 3. Project Management Project is managed efficiently, implementation progress monitored and evaluated on time. Component 3 financed project management, implementation, coordination and the monitoring arrangements. Project management was geared to the complex coordination needed for Component 1 with its multiple stakeholders. Its monitoring framework was designed to track implementation of Component 1 activities. However, GTAC never implemented the perception surveys that were required to monitor and evaluate the achievement of PDO level outcomes. Moreover, the project management arrangements failed to provide adequate support to Component 2. Inadequate attention was given to the management requirements for TMIS design and implementation with its associated business process reengineering and change management activities. In the end, most of the management and oversight over TMIS was undertaken by the MOF structures with limited technical support from the project. 3.3 Efficiency The PAD anticipated that GTAC would enhance the efficiency and effectiveness within the public sector by improving transparency and accountability. However, as a TA project, no cost-benefit analysis was undertaken at appraisal. Component 1 of GTAC has indeed contributed to significant, sustainable reforms that have delivered improvements in transparency and accountability. However, these benefits cannot be precisely mapped to improvements in efficiency and effectiveness in public spending. Furthermore, attribution of benefits to Component 1 of GTAC is not straightforward: the reforms supported by GTAC were also supported by the Bank-financed GSAC and projects financed by bilateral partners. In relation to Component 2, preparatory work has contributed to improvements in transparency – notably through work on the regulatory framework – and will facilitate follow-up work on Treasury modernization. In addition, IT infrastructure has been significantly strengthened throughout the country due to the US$3.5 million for the establishment of regional data centers and hardware upgrades. However, given that the project failed to deliver a functioning TMIS, the investment of the US$5.5 million (70 percent of project costs) in this component was not an efficient or effective use of funds. Finally, the operational efficiency of financing was very low. The costs of supervision at $1.2mn represents 15 percent of the original credit amount and 20 percent of disbursements. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory While a single rating is required, GTAC is best understood as two distinct projects: Component 1, a multi-sector technical assistance program designed to support a development policy operation; and Component 2, a standalone investment project in

30

Treasury automation and information systems. GTAC’s implementation arrangements and monitoring framework were designed to support the first component. The second component was inadequately prepared. Furthermore, at project design, the project’s implementation arrangements and monitoring framework were not designed to support management of a complex IT contract and associated business processing and institutional change management. The broad objectives supported by GTAC remain highly relevant today just as they were in 2003. The National Sustainable Development Strategy (2013-17) considers governance and state-building as its overarching theme. The Bank’s FY14-17 Country Partnership Strategy also focuses on governance as a means of reducing extreme poverty and promoting shared prosperity. The first of four areas of engagement under the FY14-17 CPS supports efforts to create a robust public administration and improve the efficiency and quality of essential services. Building on the achievements of earlier reforms, including those under GTAC Component 1, the Bank’s priorities shifted slightly: transparency and accountability of the civil service and in decision making are less prominent in the reform agenda, since the basic structures have been put in place, the management of public resources continues to figure prominently, though with greater emphasis on internal audit and external oversight, and the justice sector has been identified as a new area of engagement. GTAC’s performance in terms of achieving the PDO is rated as Moderately Unsatisfactory. Most of the activities and outputs foreseen under Component 1 – other than those that were dropped because alternative sources of financing were identified – were completed successfully and have proved sustainable: the independent Civil Service Agency continues to operate, procurement and transparency reforms have been followed-up by subsequent Governments. Component 2, in contrast, failed to deliver a functioning ‘modern computerized treasury system’ as expected. While many of the elements needed for the implementation of a TMIS system have now been put in place, it will still take several years for the government to put in place a TMIS with the broad system functionality originally proposed if contracts have to be tendered to identify a new vendor. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The PAD anticipated that GTAC would have a positive impact on poverty reduction through the reduction in opportunities for corruption and more effective use of public funds. These impacts cannot be assessed. No specific gender perspective was addressed in the PAD or during project monitoring. (b) Institutional Change/Strengthening Institutional change and strengthening was a central theme of GTAC and so pervades much of the discussion of development objectives and outcomes. The public sector and governance reforms supported by Component 1 have proved resilient. A large proportion of civil servants are now recruited competitively following a process approach overseen

31

by the State Personnel Service. Asset disclosure by political appointees and civil servants has been institutionalized and further reforms are being supported by other donors. Per capita financing for the education and health sectors are now used to allocate resources. The Government has implemented an access to information law and created opportunities for the engagement of citizens in the decision making process. GTAC also contributed to the introduction of a unified CoA, a TSA and timely audit of the state budget. The widespread increase in computer use through all levels of the civil service, and especially in the Treasury Department of the MOF, has enhanced capacity, as have the expanded training efforts. Indeed, Component 2 has also proved more successful in strengthening the institutional framework for Treasury automation than it has in delivering a functioning TMIS. The institutional and technological foundations for Treasury automation are now in place and MOF has gained valuable experience in contract management. MOF is now well-placed to proceed with Treasury modernization. (c) Other Unintended Outcomes and Impacts (positive or negative) No unintended positive or negative outcomes have been identified. However, the failure of Component 2 and the lack of a modern, computerized TMIS have significantly limited the benefits that the Government has been able to realize from its PFM reforms. The existing Treasury information systems have been adjusted to accommodate the new CoA, the TSA has been established and is operational, however the existing is not integrated across functions, institutions and levels of government. Consequently, MOF is not able to drill down in real time to transaction level data across the whole of government and budget execution reports are compiled using aggregate level data submitted periodically by the regional treasuries. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops No beneficiary survey or stakeholder workshops were conducted as part of this ICR. The public perception surveys foreseen under the project were not implemented.

4. Assessment of Risk to Development Outcome Rating: Substantial The resilience and institutionalization of key public sector reforms under Component 1 indicates that their sustainability is high. However, the failure of the TMIS contract does pose a substantial risk to project outcomes. The vendor has revoked MOF access to any of the systems in development. It is unlikely that MOF will be able to proceed with Treasury automation until this issue is resolved.

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance

32

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory The fundamental flaw in the design of the GTAC project was the combination of two distinct projects with very different institutional and implementation requirements under a single financing instrument. Project design focused on the needs of Component 1, the multi-sector technical assistance program supporting a broad range of institutional reforms under the associated development policy operation, GSAC. Preparation of this Component was supported through a PHRD Grant, officially tied to the GSAC, which provided useful capacity-building and analytical support to the MOF. The implementation arrangements focused on the requirements of the multi-sector reform program and the project results framework focused on these activities. A Quality At Entry review undertaken before approval rated the project preparation Highly Satisfactory. While this assessment may be merited for the design of Component 1, it ignored the poor preparation of Component 2. Component 2, Treasury automation, was unprepared and certainly not implementation ready at effectiveness. The functional scope, technical specifications and business process change requirements of the TMIS were not agreed by effectiveness. There is no evidence that the project had undertaken an assessment of the change management implications of Treasury automation, identifying the winners and losers, reform champions and resistors and adjusting project institutional arrangements to address implementation risks. Indeed, the project implementation arrangements were unsuited to TMIS implementation which requires clear lines of vertical accountability to ensure prompt and authoritative decision-making. These design shortcomings combined led to significant delays in the launch of the TMIS procurement and undermined effective TMIS contract management. Furthermore, the results framework did not include indicators that could be used to monitor the performance of the TMIS modernization and provided no guidance on the TMIS functional scope. Given these design failures Component 2 was poorly equipped to achieve the PDO. (b) Quality of Supervision Rating: Moderately Unsatisfactory GTAC has benefitted from intensive supervision for much of the implementation period, albeit with interruptions during periods of political instability in 2005 and 2010. Implementation support missions have taken place at least twice each year, complemented by additional support through remote VC consultations. During the early stages of implementation supervision activities covered by Components 1 and 2 and was combined with supervision of the parallel GSAC. During the period up to the 2005 revolution the project benefitted from intensive supervision, with substantial technical assistance and advisory services to support implementation. Supervision subsequently tailed-off to the time of the mid-term review in 2009, when activities under Component 1 had largely been completed. Thereafter, supervision focused exclusively on Component

33

2 and ramped up significantly as problems arose in the Treasury modernization. Supervision was particularly intense in the last two years of project implementation. Over the ten year project implementation period, there have been numerous changes in key Bank staff: including four country managers and six project team leaders. Nonetheless, throughout the project there has been a strong mix of technical skills on the team. Throughout TMIS procurement, contracting and subsequent project implementation the Bank’s Task Team included specialists in the area of Treasury reform, financial management information systems and procurement of IT systems. Recognizing the challenges in implementation of Component 2 and broader governance reforms, the Bank, with support from bilateral partners under a parallel capacity building project, fielded a resident TTL in Bishkek from September 2011 to March 2013. After this field assignment was terminated and the TTL changed, from mid-2013 through to project closing, the Washington, D.C.–based TTL organized tripartite video conferences – Bank, MOF and vendor – every three weeks to track progress in implementation and an effort to resolve implementation problems. The Bank made repeated efforts to help the MOF and TMIS vendor come to an agreement on the TMIS functional and customization requirements. Aide Memoires of implementation support missions provided detailed assessments of the implementation challenges and clear recommendations on solutions. Ultimately, in December 2013, four years after contract signature, MOF cancelled the contract. The Bank continues to provide technical support on Treasury modernization issues after the project closed, financed through a trust fund. Despite the high technical quality of the supervision of both components and the close implementation supported offered by the Bank, the overall quality of supervision is rated Moderately Unsatisfactory for two reasons. First, the Bank’s supervision failed to use the opportunity of project restructuring to address the design flaws in Component 2, particularly as regards the lack of a clear decision-making structure for the resolution of technical problems related to the design and implementation of the TMIS and the lack of an adequate results framework to monitor and evaluate the Treasury modernization component. Second, the Bank’s supervision did not place sufficient weight on the delivery of technical advice to the MOF on TMIS design and contract management. GTAC did contract an international advisor to provide this support, but the advisor was insufficiently engaged in the discussions within the MOF to guide the authorities in managing a complex TMIS contract and relations with the vendor. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory Overall Bank performance in relation to Component 1 was satisfactory. GTAC Component 1 complemented a governance reform program supported by the GSAC. At negotiations, Component 1 met the criteria for a successful project: high level of national ownership and commitment to reform, close coordination between external partners and a package of technical support for implementation. The Bank provided close supervision during the implementation phase, adjusting the scope of Component 1 to take account of

34

alternative sources of financing and addressing challenges during implementation. Component 2, in contrast, was poorly prepared and certainly not implementation ready at effectiveness. The Bank failed to ensure clarity over TMIS specifications during procurement and failed to address design flaws when opportunities for project restructuring arose. Despite the high quality of technical supervision of Component 2, the serious shortcomings in terms of the implementation support to the management of this component justify an overall supervision rating of Moderately Unsatisfactory. With the benefit of hindsight, the Bank could have been more decisive in cutting its losses and closing the project when faced with disagreements between the MOF and its vendor and repeated failure of the vendor to meet production deadlines. The task team and Bank management discussed alternative project restructuring options during 2012 and early in 2013. However, the Bank’s technical team believed that the contract could have been completed successfully if the Government and vendor agreed on system specifications and the vendor put adequate resources at the disposal of the project. Furthermore, the Bank recognized that there was a significant downside risk to project failure since the Treasury would have to restart the process of procuring an automated Treasury system, and delivery would be delayed by several years. Bank management decided that the risks and costs to the client from prematurely closing the project outweighed the risks and costs to the Bank.

35

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The Government of the Kyrgyz Republic managed to deliver a broad program of governance reforms with support from GSAC and GTAC through a period of political instability. During and in the immediate aftermath of the 2005 and 2010 revolutions, the Government was understandably preoccupied with restoring political and fiscal stability, and the reform agenda was a secondary consideration. The CC established by Presidential Order played a role in coordinating these reforms during the early years. After 2005, Ministry of Finance assumed this role. The performance of Government institutions – the Ministry of Finance, the Procurement Administration, the Civil Service Agency, the Ministry of Health, and the Chamber of Accounts – has inevitably been affected by changes in Government and institutional leadership and the capabilities of different leadership teams have varied. Nonetheless, despite these changes, the broad direction and institutional framework created by these reforms under Component 1 of the project has been sustained. (b) Implementing Agency Performance Rating: Moderately Unsatisfactory The MOF was the lead implementing agency for Component 1 and the sole implementing agency for Component 2. There have been frequent changes in MOF management during the course of project implementation: with a succession of six Ministers of Finance, four State Secretaries and five heads of Treasury between 2010 – when the TMIS vendor began activities – and project closing at the end of 2013. These changes have inevitably impacted the capability to manage a complex Treasury modernization process. Leadership change entails a steep learning curve as managers were brought up to speed on project design and implementation issues. This inevitably contributed to delays in decision making and lack of continuity. This problem was compounded by the lack of familiarity with Bank procedures. Protracted procurement procedures and time-consuming management processes requiring heavy involvement of senior officials contributed to delays in respect of ICB-1 and cancellation of ICB-2. This delayed implementation of Component 2 and the pursuit of early results. Two particular shortcomings in implementing agency performance stand out. First, the Ministry of Finance was slow to agree on and implement the institutional reforms needed to support the Treasury modernization, notably the unified CoA and the requirements for the TSA interaction with the banking system. Second, the Ministry failed to articulate its expectations of the TMIS clearly at the start of implementation and subsequently changed system requirements during implementation, this was partly due to differing expectations within the Ministry and the Ministry’s failure to put in place an adequate decision making structure to resolve these differences in views. These shortcomings contributed to delays

36

in implementation, delays in procurement and the disputes between the MOF and the vendor. Both of these issues reflect the Ministry’s limited technical and managerial capacity and its inexperience in dealing with complex Treasury modernization. After all, the Ministry had never gone through this process before. The Ministry of Finance recognized its limited capacity in this area and established an in-house IT organization to develop solutions that were outside the capabilities of MOF staff. However, this organization was inadequate to address the technical and change management requirements of a Treasury modernization process. Both the Bank team and the vendor had considerably more experience in Treasury automation, and both had the opportunity to highlight these problems and ensure that they were addressed. The Bank team should have done more to ensure that the Ministry had adequate access to a technical specialist embedded in the Ministry of Finance with experience on Treasury modernization and automation to support decision making throughout TMIS design and implementation. Opposition to the Treasury modernization process from within the Ministry of Finance contributed to these delays and the difficulties in reaching agreement with the vendor. When many of the high level officials who were most critical of the modernization process were replaced starting in late 2012, the prospects for successful implementation improved markedly. The vendor, however, did not respond to the improved operating environment and did not deliver on commitments made to supply a functioning system by November 2013. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory The successful management of the governance and institutional reforms under Component 1 has to be offset against the poor management of the Treasury modernization process under Component 2.

6. Lessons Learned These lessons learned focus on the TMIS modernization component. The Bank recognizes that complex IT projects are particularly difficult to implement. However, the potential rewards are significant, and it is now inconceivable to undertake institutional reforms without the use of information technology. Based on review of the project documentation and discussions with stakeholders, including Government, Bank staff and suppliers, the main lessons learned from the failure of GTAC Component 2 are: The Bank should avoid combining complex ICT projects with development policy operations. The time frame for DPO preparation tends to be driven by budget financing requirements and the government’s readiness to complete prior actions; DPO

37

implementation tends to be relatively short term; and the operations tend to include a range of interventions across sectors and thematic issues requiring complex, multi-institution implementation arrangements. All of these characteristics are incompatible with the requirements of complex ICT projects. ICT preparation schedules are much more extended and determined by project implementation readiness; implementation schedules are much more extended; and implementation arrangements have to be simple, ensuring clear lines of decision-making and accountability within a lead institution. ICT projects are first and foremost institutional reform and change management projects. Project design and the early stages of implementation focused on the IT and PFM technical requirements of Treasury modernization, with little consideration given to the institutional and change management requirements of Treasury modernization. A better understanding of the winners and losers within MOF and more active steps to manage potential resistance to change could have helped in identifying and managing risks before and during implementation. Adequate resources should be devoted to assist MOF staff and other internal and external stakeholders understand the institutional reforms and potential impacts. Resistance to change should be anticipated and the project should retain consensus building and communications arrangements throughout implementation. Project design should build in ‘circuit breakers’ to allow disengagement during implementation if certain key assumptions are not fulfilled or there are major delays. In the case of the GTAC, such a ‘circuit breaker’ could have been inserted during restructuring in the form of dated covenants to enable the Bank to cancel credit proceeds if the contracts for associated activities and TMIS implementation had not met specified implementation deadlines. The institutional requirements for Treasury automation – and other complex IT systems - should be in place and detailed system specifications should be agreed before project effectiveness. Changes in the institutional arrangements and system specifications during implementation are a source of delays and a cause of disagreements between governments and vendors. This has important implications for the sequencing and financing of system design work, which needs to be undertaken during project preparation rather than project implementation. Typical project preparation budgets are insufficient to cover these costs and so preparation financing is required, whether from trust funds or project preparation facilities. The expectations of vendors should be made explicit through the tender process and during contract negotiation, in particular as regards the degree of customization and the presence of technical staff in the field. Customization and field presence are areas where the vendor has strong incentives to make cost savings, while the client has a strong preference for more vendor input. Specification of these requirements up front allows the vendors to budget appropriately in the tender process and provides a basis for monitoring contract performance.

38

Implementing agencies should retain advisers with expertise in managing large and complex ICT systems implementation projects. It is possible that the problems in communications and mismatched expectations could have been avoided had the initially strong team been retained or replaced with equally effective experts. Contract management capacity is particularly important when disputes arise. ICT projects should retain ICT experts to provide strategic advice on integration of ICTs into the sector specific domain and examine the domain for business architecture and business process re-engineering needs. The resultant requirements need to be planned and executed by a client PIU that possesses in-house expertise to prepare technical bidding documents and ensure appropriate vendor and contract management The expected results and impacts of ICT projects should be specified in project documentation. Delivery of an automated system is not an end in itself. Specification of the results and expected impacts of ICT projects helps stakeholders prioritize, informs system design and adjustments during implementation, and allows assessment for effectiveness and efficiency. A realistic time frame for the design and implementation of a complex FMIS project is seven to eight years (based upon a World Bank review of 55 closed and 32 active FMIS projects from 1984 to 2010). Implementation time frames can be reduced by completing FMIS and business process reengineering design work during project preparation. However, design adjustments and resulting delays are not uncommon. A clear understanding of the nature of IT projects at their start, their implementation challenges and unusual back-ended disbursement profile can help reduce the frustrations and disagreements among key stakeholders. Complex IT projects require a significant investment in change management. Since the technical specifications were not written by the Treasury, the key internal stakeholders had limited understanding of the technical details developed by the consultant and the implications for business processes. The tender documents, for example, were hundreds of pages long and many of the terms were not properly translated into Russian. More resources should have been devoted to assist the MoF staff to understand the inherent changes in business processes that would result from implementing the new system and a more comprehensive change management process should have been incorporated into the project design to prepare and manage the change process. Capacity building activities should begin early, ideally during project preparation. The procurement of ICT systems in areas such as Treasury modernization requires substantial involvement of senior government officials, both in terms of system design and the organization and implementation of related business process reengineering and change management activities. It is unlikely that senior officials will have these skills, since these systems tend to have a life expectancy of a decade or two. Consequently, capacity has to be built for senior agency management if they are to assume the leadership functions effectively. Where Bank procedures are to be used for the procurement of these systems, senior management needs to be informed of the

39

implications of these procedures for system design and implementation in order to facilitate timely and informed decision making.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Borrower highlighted many of the issues contributing to the failure of the project discussed earlier in this report, for example, the multiple changes of key personnel, the generality of contract requirements and failure to reflect reform progress in the contract provisions, the communications difficulties between the Borrower and the vendor. The Borrower also emphasizes the need for continuous and consistent support from the World Bank and the importance of suitable team leaders. These comments reinforce the points raised earlier about the importance of appropriate and sufficient implementation support from the Bank to the Borrower, particularly in low capacity environments. The Borrower provides added explanation for the difficulties encountered in linking TMIS to the NBKR and the difficulties that the vendor had in understanding and adapting to the Borrower’s needs. The Borrower also provides more details on the chronology of progress with systems development, illustrating the multiple interchanges between the parties. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable.

40

Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Total Baseline Cost 9.41 7.98 78.16 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.80 0.00 0.00 Total Project Costs 10.21 7.98 78.16 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00 Total Financing Required 10.21 7.98 78.16

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower N/A 2.44 1.63 66.60 International Development Association (IDA) N/A 7.78 6.35 81.62

41

Annex 2. Outputs by Component Indicator Output Indicators Status Component 1: Technical Assistance to Support Implementation of Reforms included in GSAC (US$ 1.98 million) (i) Strengthened Voice and Participation (US$ 0.38 million) Information about income and assets of high state officials, their close family members and civil servants free available.

Number of declarations of high state officials published annually.

Submission of declarations on income and property by government officials in political and special positions has taken place since 2005, in accordance with the Law on Declaring and Publishing the Information on Income, Liabilities and Property of Persons in Political and Special Civil Servant Positions, as well as their close relatives, dated August 7, 2004. The cumulative data on the income, liabilities and property is posted on the website of the State Personnel Service of the Kyrgyz Republic. In 2013, the compliance rate was 97.2 percent.

Declarations of income and assets of civil servants available to the public.

Declarations filed by civil servants and their close family members regularly scrutinized for conflict of interest or illicit enrichment.

Number of declarations of civil servants and their close family members scrutinized annually.

Government's capacity to disseminate information improved.

Government Public Information Center established and functioning.

The public information center under the President’s Administrative Unit was established through the Presidential Decree dated July 10, 2004 and is fully accessible to the public. Number of non-government

organizations and individuals using GPIC services.

Increased availability, timeliness and quality of key items of official information available to the public.

Number, quality and timeliness of key pieces of official information, specified in the GSAC Policy Matrix, published annually.

Summary budget execution reports are submitted to the government officials, i.e. the Prime Minister’s Office, the Parliament (Jogorku Kenesh), the National Bank of the Kyrgyz Republic, the National Statistics Committee, and other stakeholders. They are also posted on the official website of the Ministry of Finance for public access. The State Agency for Public Procurement was established through the President’s Decree in 2009. Approximately 99% of the participants are the commercial organizations, the rest are public entities. The information of procurement is posted on Okmot.kg website under the public procurement section. The normative legal support is ensured by the Public Procurement Law enacted in 2004 with recent amendments made in 2013.

Access to legal database of the Ministry of Justice available to the public online.

Online access to the centralized database of legal information is available through the website of the Ministry of Justice: www.miniust.gov.kg.

42

Indicator Output Indicators Status Conflict of interest created by jurisdiction over regulatory and enforcement functions in Kyrgyzstandart removed.

Kyrgyzstandart reorganized to separate regulatory and enforcement functions.

The Presidential Decree on the “Institutional and Structural Reforms in the Technical Regulation of the Kyrgyz Republic” was issued on April, 30 2005 to separate the regulatory and executive functions.

Inspection regimes of SES and PCL improved.

Number of inspections conducted by SES and PCL annually.

Dropped.

Rules and regulations of SES and PCL available to the public. Number of grievances addressed annually by SES and PCL.

Regulatory impact assessment of proposed laws and regulations initiated.

Regulatory impact assessments of three new laws and three new regulations agreed with IDA, conducted.

Dropped.

(iii) Improve Transparency in Public Procurement (US$ 0.15 million) Transparency and accountability of public procurement processes increased.

Performance indicators as specified in the CPAR.

Based on the CPAR recommendations, an action plan was developed and the following activities were implemented: • A set of new tender documents meeting requirements of the new Public Procurement Law, as well as different types of guidelines and manuals have been elaborated; • Funds for monthly Public Procurement bulletin publication (at early stage) and procuring of printing equipment for bulletin publishing were provided; • Audit of selected items procured from republican budget was conducted by an internationally recognized company, and recommendations for improvement of the process were provided to the government; • A project aiming at raising awareness of new procurement rules among business circles and assessing efficiency of public procurement in selected sectors was conducted by Transparency International (local branch).

Results of procurement audits of selected, high value items.

Dropped.

43

Indicator Output Indicators Status Number of procurement staff trained. Appropriate staff of procuring agencies has been trained in the new

procurement legislation and rules, including through a Public Procurement Regional Training Center (about 4,500 in total).

(iv) Improving Service Delivery in Health and Education (US$ 0.21 million) Improved financing of health and education services.

Revised categorical grant formulae for health and education sectors implemented.

A local company hired under the project, elaborated per capita categorical grant formulas for both health and education sectors, which were agreed with the Ministries of finance, health and education. Nevertheless, because of the government’s decision to transfer all health institutions and their financing to the republican budget, implementation of the categorical grant formula for health became irrelevant. At the same time, a more transparent and objective geographic allocation of health resources throughout the country was achieved through the application of the new financial rules for the sector based on per capita and caseload criteria. In the case of education, the capitation based categorical grants formula (that was elaborated several years previously under GTAC) was adopted in March 2008, when the government decided that reforms in education sector had made sufficient progress to use benefits of per capita financing for resource allocation.

Functional and expenditure responsibilities in the health and education sectors defined clearly.

Resources for health insurance for pensioners and the unemployed provided on time in accordance with budget allocations.

Dropped.

(v) Improve Effectiveness of the Civil Service (US$ 0.95 million) Institutional framework for the civil service and the capacity for effective civil service management improved.

Independent Civil Service Management Agency established and operational.

The Presidential Decree of September 18, 2004 established the Agency of the Kyrgyz Republic on the Civil Service. A subsequent Presidential Decree dated October 26, 2009 reorganized and renamed it the State Personnel Service of the Kyrgyz Republic. Currently, the State Personnel Service is authorized to develop, implement and ensure the sustainable functioning of state personnel policy in both central and local government bodies, and reports directly to the President of the Kyrgyz Republic.

Civil Service Law amended. The Civil Service Law was amended several times over the course of project implementation.

44

Indicator Output Indicators Status Percentage of "Main" and "Lead" civil servants recruited on competitive basis.

As of January 1, 2013 all chief and senior positions in the government are filled through competitive selection in accordance with the legislation on the “State Service” of the Kyrgyz Republic dated August 11, 2004.

Average number of applications for a civil service position.

Not tracked.

Percentage decrease in the number of civil servants.

Not tracked.

Percentage increase in civil service salaries.

Not tracked.

(vi) Streamline the Structure of Government Ministries and Agencies (US$ 0.13 million) Organizational structure of Ministries of Health and Education, Ministry of Labor and Social Protection and one other ministry to be agreed with IDA, streamlined.

Number of ministries reorganized based on functional reviews.

After project effectiveness, it was agreed that other donors (UNDP, DfID, and TACIS) would conduct the functional reviews from their own funds.

(vii) Refine the National Poverty Reduction Strategy (NPRS) and Monitor and Evaluate Progress (US$ 0.16 million) Capacity for monitoring and evaluating progress in implementation of the NPRS increased.

M&E system implemented. Dropped upon request of the implementing agency (due to available grant funding from other sources). Funds were reallocated among other components.

Not Specified Public perception of transparency, responsiveness and integrity of public sector as measured by annual surveys has improved.

No surveys were conducted under the project.

Number of public grievances addressed annually by the President's Administration, the Office of the Government and the regulatory agencies covered by the GSAC.

Dropped.

Number of draft laws and regulations discussed with the private sector and civil society, prior to their adoption.

Public steering councils were established through the President’s Decree dated September 29, 2010 and comprise representatives from civil society, think tanks, professional and sector unions, and experts. Their main objective is to provide public opinion on formulating and implementing the government policy.

45

Indicator Output Indicators Status Coverage, quality and timeliness of audit reports of Chamber of Control. The Jogorku Kenesh (the Parliament) receives the progress report of the

Auditing Chamber every September for the previous budget year as well as the opinion on the draft National Budget Law for the subsequent fiscal year.

Increased usefulness of the budget as a strategic policy instrument, as a result of improvements in its formulation process, content and presentation.

Budget amendments are conducted on the basis of the macroeconomic indicators and forecasted revenues. In the current legislation, budget amendments are only allowed two times per year. The format of budget presentation has been revised in accordance with GSM2001 methodology and reflects both financial and non-financial assets operational costs. The Ministry of Finance has conducts public hearings of the draft budget in the cities of Bishkek, Osh, Karakol, prior to its submission. A civilian budget is also posted on the Ministry of Finance’s website and published in brochures that are distributed among NGOs, educational institutions, and other stakeholders.

Improved financial accounting and compliance with laws and regulations in three ministries, due to strengthened internal control systems.

Dropped.

Percentage of private firms participating in public procurement. 100 percent of companies participate in public procurement. The State Agency for Public Procurement was established through the President’s Decree in 2009. Approximately 99% of the participants are commercial organizations, the rest are public entities. The procurement information is posted on the Okmot.kg website under the public procurement section. The normative legal support is ensured by the Public Procurement Law enacted in 2004 with recent amendments made in 2013.

Component 2: Modernization of the Treasury (US$ 6.37 million) Computerized Treasury System Implemented.

Computerized Treasury System Implemented.

The introduction of both the TMIS and the HRMIS were not completed within the provided timeframe.The technical infrastructure for the automated treasury system was developed through: • Reconstruction of the server centers for the Main and Backup Data Processing Center; • Delivery of 22 servers, information storage facilities and management & monitoring systems for the central TMIS/HRMIS network; • Delivery of equipment and software, and installation of the main and local networks for the regional treasury departments; • Training conducted in information technologies.

46

Indicator Output Indicators Status The normative legal base has also been developed through: • The temporary instructions for the TSA of the Central Treasury approved by a Government resolution on August 30, 2012; • The Unified Chart of Accounts and the Guidelines on the use of the Unified Chart of Accounts were approved by the Decree of the Ministry of Finance on September 4, 2012; • The Protocol of Cooperation between the Ministry of Finance and the National Bank of the Kyrgyz Republic on the Unified Chart of Accounts was approved by both parties on October 23, 2012.

47

Annex 3. Economic and Financial Analysis The PAD (p.18) states “The project seeks to increase the transparency and accountability of the State to external stakeholders and enhance the effectiveness and efficiency within the public sector. Given the nature of the project, it is not possible to quantify its benefits.”

48

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending

Supervision/ICR David Nummy Senior Public Sector Specialist ECSP4 Current TTL Zhanybek Ybraiym Uulu Public Sector Specialist ECSP4 Team member Irina Goncharova Procurement Specialist ECS02 Procurement

Galina Alagardova Financial Management Specialist ECSO3 Financial Management

Nargiza Tynybekova Program Assistant ECCKG Team member

Cem Dener Senior Public Sector Specialist PRMPS IT Specialist (2005 to April 2011)

K. Migara de Silva Senior Public Sector Specialist ECSP4 TTL until late 2012

Svetlana I. Proskurovska Senior Public Sector Specialist LCSPS TTL until early 2011

Natalia Pisareva Senior Economist ECSP1 TTL until late 2010 Roland N. Clarke Lead Economist LCSPR TTL until late 2006 Jit B. S. Gill Lead Public Sector Management

Specialist ECSPE TTL until early

2004

Henry Forero Senior Public Sector Specialist LCSPS IT Specialist (April 2011 to close)

Lilia Saetova Consultant ECSP4 Former team member

Nurbek Kurmanaliev Procurement Specialist ECSO2 Procurement Kathy Lalazarian Senior Public Sector Specialist LCSPS Team member

John Otieno Ogallo Senior Financial Management Specialist OPSOR Financial

Management Afsaneh Sedghi Senior Economist ECSP1 Team member Petrus Henricus Van Heesewijk Senior Program Officer PRMPS Former team

member

Olga Sipka Consultant ECSP4 Former team member

Aliya Kim Financial Management Specialist ECSO3 Financial Management

Afsaneh Sedghi Senior Economist ECSP1 Former team member

Asel Almanbetova Program Assistant PRS Former team member

Takuji Yano Public Finance Specialist ECSP4 Former team member

Dominique de Roquefeuil Senior Informatics Specialist IT Specialist (2003-2004)

Naushad Ali Khan Lead Procurement Specialist Procurement

49

Gurcharan Singh Senior Procurement Specialist TWICT Procurement

Azamat Abdymomunov Economist Former team member

Daniyar Aitimbetov Team Assistant Former team member

Dinara Djoldosheva Senior Country Officer ECCKG Former team member

Cholpon Ibraimova Team Assistant TWITR Former team member

50

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands

(including travel and consultant costs)

Lending FY01 11.16 53.35 FY02 25.75 158.53 FY03 28.93 185.63 FY04 0 7.65 FY05 0 0 FY06 0 0 FY07 0 0 FY08 0 0 FY09 0 0 FY10 0 0 FY11 0 0 FY12 0 0 FY13 0 0 FY14 0 0

Total: 65.84 405.16

Supervision/ICR

FY01 0 0 FY02 0 0 FY03 0 0.17 FY04 49.86 194.33 FY05 20.25 89.28 FY06 17.44 97.42 FY07 20.87 79.65 FY08 21.65 78.06 FY09 30.96 83.78 FY10 21.51 60.60 FY11 21.52 136.95 FY12 36.04 98.49 FY13 44.64 230.69 FY14 15.85 63.85

Total: 300.59 1,213.27

51

Annex 5. Beneficiary Survey Results Not Applicable

52

Annex 6. Stakeholder Workshop Report and Results There were no stakeholder workshops.

53

Annex 7. Summary of Borrower's Comments on ICR

Project Completion Report Governance Technical Assistance Credit (GTAC) Project

Donor: The World Bank (WB) WB Credit Amount: 5.8 mln. SDR (7,775 mln. USD) The Financial Terms of the Credit 0,75% a year for 40 years, grace period of 10

years Signing Date: 28 May 2003 Effectiveness Date: 28 August 2003 Closing Date: 31 December 2013 Executing Agency: Ministry of Finance of the Kyrgyz Republic Disbursements made as of 31 December 2013:

IDA: 6,350,588.41 USD GOKR: 1,625,343.34 USD

I. Project Management: 1. From the World Bank’s Side Since January 01, 2013, Mr. David Nummy has been GTAC Project Manager from the World Bank. During this time there have been regular monitoring activities (once in 3 weeks) conducted to follow the progress with the involvement of all project stakeholders. Besides, the IT-related matters of TMIS/HRMIS components have been coordinated by Mr. Henry Forero since April 2011.

2. From the Ministry of Finance of the Kyrgyz Republic

The Ministry of Finance of the Kyrgyz Republic approved the Project Charter in April 2011. Based on the Charter, revisions were made in the project structure and management. Thus, in accordance with the Project Charter the Stats Secretary of the Ministry of Finance is the Project Director. Also, there were Working Groups/Task Forces established under the leadership of respective deputy ministers.

Besides, through the Ordinance of the Government of the Kyrgyz Republic dated 26 July 2011, no. 299-р, the Government approved a new Regulation on the Coordination Council for GTAC Project chaired by the Minister of Finance of the Kyrgyz Republic.

3. From FreeBalance Side

Since the contract came into force the FreeBalance Company assigned Vice President, Ms. Suan Dorion and, later, Mr. Kamran Khan to manage the project on behalf of the company. The company opened a local office run by the country managers. II. Treasury Management Information System and Human Resources Management Information System (TMIS/HRMIS).

54

Implementation of the contract with Free Balance Ltd. for Application software Development (ASD) for TMIS/HRMIS included delivery and installation of the central servers and equipment for data processing in TMIS/HRMIS (ICB-1). The contract came into force on 28 Dec 2009. The contract amount is $2,590,063. As of 31.12.2013 under ICB-1, the disbursement volume amounted to $388,509.15 (i.е. 15% of the total contract amount). The FreeBalance Ltd. delivered equipment in accordance with Section 4.0 of the TMIS/HRMIS Project Implementation Plan: Supply and Installation of Equipment for MSC and BCC, including delivery and installation of the central servers, data storage and backing-up/recovery equipment and relevant software. Payment for equipment delivered had not been made as the Buyer (Ministry of Finance) has not yet signed a transfer/acceptance document (to be signed based on testing of the software loaded on equipment).

1. Hiring an International Consultant (Contract Manager) to Support in TMIS

and HRMIS Development and Introduction.

The contract with Mr. Aare Laponin was signed on 13 October 2010 for providing support in TMIS implementation. The contract amount is $109,793. The contract with Mr. Laponin was revised in view of including the support in HRMIS implementation and the respective revisions were approved by the World Bank on 7 June 2011. Based on the revised contract there were new tasks and functions added to the Terms of Reference providing for HRMIS implementation. On 10 June 2011, there was an Addendum to the above contract with Mr. Laponin signed covering the period ending 28.12.2012. As the World Bank did not approve extension of the contract with Mr. Laponin, the Ministry of Finance initiated hiring another international consultant in April 2013, to provide professional consulting services to the Ministry of Finance and the State Personnel Service of the Kyrgyz Republic in managing the contract signed with FreeBalance. Thus, the Ministry of Finance prepared the draft Terms of Reference for the international consultant and sent to the World Bank for approval. Due to the project completion, the selection of consultant was terminated on 09.12.2013.

2. The Project Extension and TMIS/HRMIS Project Implementation Plan. The contract with Free Balance Ltd. was extended 3 times:

• On 31.12.2010 a Memorandum of Understanding (MOU) no. 1 was signed approving the updated Project Schedule and Deviations List. MOU also set the new completion date - 30.06.2012. The reason for extension is the lack of Budget Module and module for accounting at budget-funded institutions in the system design.

55

• Due to delays in implementation and introduction of Budget Module Software of TMIS/HRMIS, a decision was made at the Meeting of the Coordination Council chaired by the Ministry of Finance on 09.02.2012 on adopting a new MOU and revised schedule. On 08.07.2012 a new MOU no.2 was signed with the new Project Schedule and Deviations List. There was also the new project completion date set for 28.12.2012. The reason for extension is the poor quality of TMIS and failure to submit a report on Budget Module by FreeBalance.

• Given the end of GTAC Project period the Ministry of Finance initiated holding a meeting of the Coordination Council which took place on 12 December 2012. The decision was made on this meeting to extend GTAC Project for 2013 and complete already launched activities by FreeBalance to introduce TMIS/HRMIS. The final project completion date is 31 December 2013. On 31.12.2012 the World Bank extended GTAC Project till 31 December 2013. However, a decision was made this time to sign an Addendum no. 1 to the Contract through which revisions would be made in the contract terms. On 01.02.2013 the Ministry of Finance and FreeBalance agreed on the updated Project Schedule with the World Bank’s participation which was proposed to be adopted when signing Addendum 1. The reason is in the incorrect banking interface that FreeBalance had been providing and in more time that would be needed to complete the work in accordance with the requirements of the Central Treasury of the Ministry of Finance. Activities on Budget Module, TMIS, HRMIS implementation were implemented in accordance with the Work Schedule agreed by the Parties on 1.02.2013.

Given the failure to timely complete activities on TMIS/HRMIS implementation, there was a decision made on 27.11.13 at the Coordination Meeting to terminate the contract with FreeBalance effective 31.12.2013. 2.1. Budget Module Designing Process:

The revised report on SRS for designing the Budget Module of version 1.3 was submitted on 27.03.2013 by FreeBalance. The report was finalized based on discussions at the seminar on 5-7 March 2013 and comments/additions from the Main Department for Budget Forecasting of the Ministry of Finance of the Kyrgyz Republic were forwarded to FreeBalance on 18.03.2013. In April – June 2013 FreeBalance submitted two versions of the Supplementary Report on SRS to designing of the budget module based on the comments from the Ministry of Finance. According to FreeBalance, the Supplementary Report on SRS to designing of the budget module would be submitted in early July 2013 based on the recent comments from the Ministry of Finance dated 7 June 2013. The available comments of the Ministry of Finance were taken into account in the final version of the report approved by the project director on 26.07.2013. Since then, no activities were conducted by FreeBalance on the Budget Module. On 30.09.2013 Vice President of FreeBalance, Mr. Kamran Khan, informed that in December 2013, there would be a demonstration of Budget Module held. Meanwhile, FreeBalance plans to work on Budget Module only in 2014. According to FreeBalance, they planned to have a

56

demonstration of Budget module in December 2013. These activities were never conducted during the period ending 31.12.2013. The timeframe set by the updated Project Schedule dated 01.02.2013 was not followed.

2.2. Designing of TMIS: In spite of the continued joint activities, there were delays from FreeBalance’s side in following the Work Schedule. In order to solve the arising issues FreeBalance meets with the Central Treasury of the Ministry of Finance. It should be noted that according to the decision of the Coordination Council Meeting on 12 December 2012, the Parties were to prepare a draft MOU to reflect the interaction between the InfoSystem State Enterprise under the Ministry of Finance with FreeBalance in TMIS application software implementation and introduction. To implement the given decision of the Coordination Council it was recommended to GTAC PIG to consider the need and feasibility of preparing this MOU and in February 2013 a meeting was held between the Ministry of Finance, FreeBalance, InfoSystem State Enterprise under the Ministry of Finance, which did not result in any agreements on potential cooperation. Since granting GTAC Project an extension for 2013, the Central Treasury of the Ministry of Finance together with the National Bank of the Kyrgyz Republic conduct activities on testing the interface with the interbank pay system for direct participation of the Central Treasury in the given pay system. TMIS Status: Report In September 2013, 2 reports were submitted on TMIS marking the completion of the designing stage:

• Supplementary report on “TMIS System Requirements Analysis” (SRA or SRS – system requirements specifications), with a detailed description of future TMIS system design with functioning of the Unified Treasury Account and use of the new Unified Chart of Accounts.

• Report on “Functional Compatibility Analysis” (FCA), with a detailed description of technical solutions to ensure direct interaction of TMIS with interbank pay system (Gross System of Settlements in Real-time Mode and Package Clearing System), and interaction of TMIS with information systems of the Tax and Customs services of the Kyrgyz Republic. The National Bank of the Kyrgyz Republic provided any kind of assistance in this process, the clarifications/explanations were provided to FreeBalance specialists on formulation of program codes for banking interface and typical test scenarios were offered. Also, there were 3 reports signed on TMIS, submitted by the Supplier:

57

• “TMIS Training Plan” describing the strategy and methodology for training of all categories of TMIS users.

• “Information Protection and Electronic Digital Signature Application” describing the key measures on information protection and use of the electronic digital signature in TMIS.

• The report on “Data Architecture and Hierarchy” showing the architecture of the TMIS software (application software) platform with description of data categories.

It should be noted that approval of the above reports by the Ministry of Finance was given without the expert opinion of the contract manager A. Lapynin, because of having no approval from the World Bank for a contract extension with him. In communications with FreeBalance, the Ministry of Finance has been expressing the commitment in successful implementation of TMIS/HRMIS indicating the delays in Work Schedule, i.e. not completed activities on system design reports within the given timeframe, failure to conduct testing etc. The Ministry of Finance had concerns on failing to conduct the planned testing. In spite of the above, the parties have been conducting joint activities on achieving the project goals to introduce TMIS/HRMIS and continuing the work on finalizing the system design reports. In order to finalize and approve two major reports: “Supplementary Report on SRS to designing TMIS application software system of 4.1 version” and “Supplementary Report on SRS to TMIS of 6.0 Version”, FreeBalance and the Central Treasury of the Ministry of Finance together with the National Bank of the Kyrgyz Republic, State Tax Service, State Customs Service conducted joint events in April-June 2013 to determine the functionality of current accounts mode and development of interface with the interbank pay system to ensure direct participation of the Central Treasury in this pay system, and also interfaces with the information systems of the State Tax Service and the State Customs Service. The National Bank of the Kyrgyz Republic provided various types of support in this process, the clarifications/explanations were provided to FreeBalance specialists on formulation of program codes for banking interface and typical test scenarios were offered. As a result of joint activities by the Central Treasury and FreeBalance specialists the Supplementary Report on SRS to designing TMIS application software was thoroughly finalized, particularly, the subsection on description of TMIS interfaces with external system. This subsection of the report was finalized in accordance with the requirements of the Protocol of Interaction between the Ministry of Finance of the Kyrgyz Republic and the National Bank of the Kyrgyz Republic dated 23.10.2012 on the banking interface, and also the proposals from Sinam Company – developer of the State Tax Service Information System.

58

On 17 May 2013, the Ministry of Finance of the Kyrgyz Republic sent comments by the Central Treasury and the National Bank of the Kyrgyz Republic on functional compatibility analysis for FreeBalance’s review and finalization of the report. Ministry of Finance also sent the Terms of Reference on "Automated System of the Main Book of the National Bank of the Kyrgyz Republic", approved by the Central Treasury of the Ministry of Finance and the National Bank of the Kyrgyz Republic dated 27.05.2013 to finalize the Supplementary SRS Report. In the process of discussing the above reports, the parties have agreed on simultaneous approval of these reports as they were interrelated. As a result, the Supplementary Report on SRS to System Design of TMIS was approved on 26.07.2013, while the report on analysis of the functional compatibility (IDS) was approved on 30.08.2013. TMIS: Comprehensive Testing and Banking Interface Testing After the approval of the Supplementary Report, FreeBalance has been conducting acceptance testing of TMIS modules in September, except for the interface of the National Bank of the Kyrgyz Republic, as this system, according to the Supplier was planned to be provided in November 2013. Based on the tests conducted, it should be noted that they were of ad hoc character, rather than comprehensive, as stipulated in the contract (Section 2.7.2.2.7). In tests conducted it was not possible to follow the full chain of money flows in TMIS. Besides, because the banking interface was not ready, the testing of it was not conducted (Free Balance promised to conduct testing of banking interface in November 2013). Individual modules of TMIS were also missing with reference to the lack of the banking interface (e.g., revenues modules, the Main Book, Budget Reporting Module). Based on the testing conducted, the comments were sent to FreeBalance with a request to revise the application software. TMIS: Training In spite of the above, to ensure the progress, the Ministry of Finance decided to shift to the next project stages – training stage and TMIS piloting stage. FreeBalance submitted the methodology, plan and schedule of testing, which were approved by the Ministry of Finance on 01.10.2013. During the period of October 3-18, FreeBalance has been conducting training of the treasury specialists (the total of 9 days on individual modules of TMIS), which further would become trainers on TMIS application software.

TMIS: Pilot Testing TMIS Pilot Testing was pledged by the Supplier to take place after 10.10.2013. TMIS Pilot Testing (contract term 2.7.2.4) according to the Schedule would start in April 2013, yet these tests were postponed till October 2013. On 11-12 October 2013, Free Balance specialists together with the Central Treasury and InfoSystem State Enterprise have visited the pilot regional treasury units (Leninskoye and Chuyskoye regional treasury units) to start testing, where the assignments were given to prepare initial data entry. On

59

12 October 2013, all requested data was prepared and sent by FreeBalance to budget-funded enterprises (listed in the National Budget of the Kyrgyz Republic). Since then, no other activities on pilot testing were conducted on TMIS application software, i.e. pilot testing has not started yet. The Ministry of Finance has repeatedly communicated this issue to Free Balance and on 4 November 2013, the company requested filling out additional tables on entry data for pilot units. As of today, the regional treasury units have filled out the tables, yet the pilot testing has not been started. Meanwhile, Free Balance informed that the tests will lack the module for local budget implementation. In accordance with the contract terms (2.2.2.9.1), the local budget implementation module should be part of this stage. This was communicated to Mr. Nummy during the meeting on 26.11.2013 at the World Bank Office. Activities on pilot testing were not completed by the Supplier, as well as the works on interface with the National Bank of the Kyrgyz Republic.

TMIS: Accounting Module FreeBalance submitted a report on the system design phase for the accounting module. On 13 June 2011, the Department for Financial Reporting and Internal Audit Methodology communicated to FreeBalance that they had no comments to the report submitted and sent a request to share information on operations for accounting of funds in the cashier’s office of a budget-funded entity; operations for accounting of reporting entities, on banking transactions, on maintenance of the Main Book at each budget-funded organization. Besides, the specialists have conducted a thorough expert appraisal of reports submitted by FreeBalance, particularly for the financial reporting and accounting modules. Afterwards, the comments and suggestions were sent to FreeBalance. Also, the Department for Financial Reporting and Internal Audit Methodology of the Ministry of Finance sent an official request to FreeBalance to share additional information on accounting module meeting the technical specifications of IBS. 2.3 HRMIS Designing: The joint work of FreeBalance, State Personnel Service and GTAC on HRMIS implementation has been conducted in accordance with the revised Schedule of remaining activities for 2013. There were some deviations from the schedule, yet, overall, there was a certain progress in some of the activities under HRMIS implementation. Loading of master files into the system for 32 state bodies out of 36 is complete. Loading of terminology was conducted (however, the loaded terminology did not meet the requests of module managers: terms were not translated from English, it was not possible to determine the meaning and format of information being entered) and mass calculation of the salaries for all loaded state bodies (the state bodies have noted mistakes in calculation of base salary rates; recalculation of salaries was not conducted because the module was not ready).

• FreeBalance provided electronic carriers of HRMIS application software and package list of licenses on 10.02.2013. The transfer & acceptance protocol was

60

signed by the State Personnel Service and approved by GTAC Project Director on 12.03.2013.

• FreeBalance submitted the Plan and Methodology for Testing the HRMIS Interface for review and approval on 15.02.2013. The Plan and Methodology was signed by the State Personnel Service approved by GTAC Project Director on 19.02.2013.

• FreeBalance submitted the Schedule of the Second Cycle of Comprehensive Testing dated 15.02.2013, which was then signed by the State Personnel Service and approved by GTAC Project Director on 19.02.2013.

• FreeBalance submitted the Preliminary Testing Report on HRMIS Interfaces dated 15.02.2013 (company’s internal testing report) for acceptance testing in accordance with the Schedule of HRMIS Implementation, para. 3.2: “Interfaces of HRMIS with the webportal and software for testing the State Personnel Service”. The State Personnel Service communicated on 05.04.2013 that had no comments or suggestions on this document.

• FreeBalance submitted the Training Plan for TMIS/HRMIS, version 1.2 dated 20.02.2013 which, because of no comments from the State Personnel Service, was approved on 04.03.2013. The partial training on this plan was conducted in October 2013 (some of the modules were presented for the first time, i.e. “Assessment/Attestation” Module. It should be noted that training was done on user level).

• The second cycle of Comprehensive Testing of HRMIS modules based on the schedule approved by the deputy director of the State Personnel Service/head of the working group on HRMIS dated 19.02.2013. Testing cycles, both the first and the second one were of ad hoc mode. The proposed testing scenarios did not cover the functional requirements. As of 01.04.2013 the report on the second cycle of comprehensive testing of HRMIS modules has not been submitted by FreeBalance.

• FreeBalance submitted the Supplementary Report on HRMIS implementation, version 4.2 dated 05.03.2013, for approval by GTAC Project Director. The report was reviewed by the State Personnel Service and comments were communicated to FreeBalance. FreeBalance addressed the above comments and the State Personnel Service approved, while the Ministry of Finance confirmed the Supplementary Report on HRMIS implementation, version 4.2 on 6 Mary 2013.

• FreeBalance submitted the Methodology and Testing Schedule of HRMIS Users’ Accounts dated 22.03.2013, which was approved after receiving no comments from the State Personnel Service on 02.04.2013. However, the HRMIS users’ report, in accordance with the legislation of the Kyrgyz Republic was not submitted for testing until end-2013.

Besides, to expedite activities related to HRMIS implementation on 28.03.2013 there was a discussion held of unresolved issues and a decision made to finalize the work based on

61

mistakes found. However, the mistakes were not eliminated by the time of the next training in October 2013. There were regular meetings held to discuss issues related to HRMIS implementation with the participation of stakeholders (Ministry of Finance, GTAC PIG, the State Personnel Service and FreeBalance: on Income Declaration, HR Expenditure Accounting and Salary Calculation, data entry on 8 state bodies, terminology issues related to all modules etc.) Supplementary Report on SRS to HRMIS, version 4.2, was submitted by FreeBalance. It was approved on 06.05.2013 by GTAC Project Director, i.е. the Ministry of Finance. On 12 June 2013 GTAC PIG sent the information to Vice President of FreeBalance, Mr. Kamran Khan on the current situation and introduction of HRMIS as of 12.06.2013. On 20 June, 2013, FreeBalance submitted the information “On Updating the Current Activities on HRMIS”, with the milestones indicated to address earlier found comments and on overall progress in HRMIS implementation. The works on HRMIS on users' accounts development were not completed. Therefore, FreeBalance did not submit results of users’ accounts testing. The mistakes reflected in the Journal to Register Mistakes were not fully eliminated. No training was conducted for the system administrators, although the administrator resource was submitted for a review in the report on application software development. In particular, the training did not cover the documentation processes, generation of new types of reports etc. Later, with agreements reached during Mr. Kamran Khan’s visit to Bishkek on 23-30 September 2013, training of users was conducted on 8-18 October 2013), administration issues were not considered. The full functionality of the system was not implemented. Due to the delays in the implementation, and as a result of FreeBalance’s request, there were priority tasks identified, which also were not implemented. During the implementation process, no logical sequence of actions was observed. For example, training of trainers and users had been conducted before the system was finalized, i.e. the system faced significant changes after the training in 2012. The standard software application of the company was never adapted to requirements of the business processes of the Kyrgyz Republic. The Report on HRMIS Configuration submitted by FreeBalance on 11.09.2013, the Report on Completion of HRMIS Testing, the Report on the Second Cycle of the Users’ Acceptance Testing of HRMIS were not approved by the State Personnel Service because of a number of comments. The listed reports were returned to the Supplier for finalization. As a result, HRMIS system, as of 2013, had significant flaws and did not ensure proper processing and formulation of exit information.

62

3. Activities to Implement ICB-2 Contract Implementation of the Contract for Delivery and Installation of Computer, Network Equipment and Engineering Systems (ICB-2) with ALSI (Kazakhstan) that came into force on 9 March 2010. The contract amount is $3,517,426.43. As of 30.12.11, under ICB-2 the disbursements amounted to $2,603,247.31. This includes an advanced payment, payment for system design, payment for equipment and software delivery under Phase I (Installation and Commencement at DPC and RDPC), and payment for training of technical specialists and system administrators of InfoSystem State Enterprise in charge of supporting the TMIS/HRMIS systems and payment for equipment and software delivery under Phase 2 (Delivery and installation of equipment, installation of SCS at regional units of the Treasury and territorial representative offices of the State Personnel Service). Over this period, ALSI conducted reconstruction of RDPC and partial works at DPC. Also, ALSI completed Phase 1 of the contract on delivery of equipment for DPC, RDPC, telecommunications equipment and computers. Testing of systems supplied under Phase 1 is complete. Under Phase 2 of the contract, ALSI completed delivery and tuning of the computer and telecommunications equipment and establishment of cable networks at 59 regional units of the Central Treasury and 3 branch offices of the State Personnel Service. ALSI completed works under the contract and submitted three reports for review and approval by the Ministry of Finance of the Kyrgyz Republic. These reports on Testing of the Introduced Systems in the Field under the second stage of General Type Hardware and Software Provision and on Testing with the Full Loaded System, and also the final report on the System Roll-Out were agreed with the working group on ICT and heads of TMIS/HRMIS component, and approved by the TMIS/HRMIS project director. Based on the contract terms, after approval of the report on Testing, the introduced systems in the field under the second stage of General Type Hardware and Software Provision, a payment of $210,693.84 was made; after approval of the report Testing with the Full Loaded System, a payment of $351,742.64 was made. Also, upon approval of the final report on the System Roll-Out, a payment of $351,742.64 was made to ALSI Company. The contract was completed and the guarantee period has started. III. Supplementary Reforms in Public Finance Management: 1. Development of a Unified Chart of Accounts (UCA) To assist the Ministry of Finance in UCA development, contracts were awarded to Mr. V. Redburn, the international consultants under GTAC (the contract amount is $100,000) and 2 local consultants (А. Аlapaev, Е. Tsoi) (the contract amount is $8,000). In response to a request from the Central Treasury and upon approval from the World Bank, contracts with the local consultants were extended by additional 6 months. The total contract amount is $12,000. The unified chart of accounts was fully developed. The draft Manual

63

on its use was prepared. The local consultants presented UPS to the chief accountants of leading ministries and agencies. Given the significant volume of work for the local consultants, the Ministry of Finance initiated an extension of contracts with the local consultants by additional 4 months (from September till December 2012). Because of a late request from the Ministry of Finance, i.e. the letter was sent to PIG GTAC after the contract completion date (following 10 days after the completion date), the World Bank rejected the extension. Thus, through the initiative of the Central Treasury a new tender was announced to select local consultants based on the open competitive bid. As a result of summing-up the scorecards and interviews with bidders, А. Аlapaev and Е. Tsoi were selected. The contract amount is $12,000.

2. Transition to UCA and Connecting the Central Treasury to the Package

Clearing System

In the process of TMIS application software development, FreeBalance faced difficulties in understanding the structure of interbank pay systems (Gross System of Settlements in Real-time Mode and Package Clearing System) of the National Bank of the Kyrgyz Republic. In accordance with the final official report on TMIS functionality by FreeBalance dated 9 March 2012, «…The Central Treasury is becoming a direct participant of SRS in PCS of the National Bank and will use both systems according to the regulation of the National Bank». However, when specialists of the Central Treasury and FreeBalance moved from the reporting phase to practical actions, i.е. preparation of commissioning testing, including the interface with the National Bank, FreeBalance proposed a totally different principle of building interface with the National Bank. To obtain the model of UCA functioning, provided in the contract between the Ministry of Finance and FreeBalance, in August-October 2012, there were videoconferences and series of meetings conducted with participation of the World Bank’s leading experts, engaged in the project for treasury modernization since the very beginning of the project, and also with participation of the regional representative of the Department for Budgeting Issues of the International Monetary Fund, Mr. D. Zohrab. Based on the meetings of FreeBalance there was a rigid clarification given on the need to develop and introduce TMIS interface and application software in accordance with the requirements of the Central Treasury and regulated standards of the National Bank. Procedures for interaction of the Central Treasury with Interbank Pay Systems (Gross System of Settlements in Real-time Mode and Package Clearing System) of the National Bank and step-by-step business processes in the treasury system with UCA functioning, including functions of the financial control an information security were jointly worked out by both parties and reflected in the Protocol of Interaction between the Ministry of Finance, in person of Minister Olga Lavrova and the National Bank of the Kyrgyz Republic in person of Chairperson Zina Asankojoeva, signed on 23.10.2012.

3. Development and Promotion of Normative Legal Acts of the Kyrgyz Republic

Providing for Improvement of the Public Finance System Functioning to

64

Implement the State Budget through Introduction and Use of TMIS/HRMIS Integrated Systems.

In order to implement the above activities, 2 local consultants were hired under GTAC Project, to develop functionalities and provide legal support. The contract with the Functional Specialist (J. Kadyrova) was signed on 19.08.2011 and its amount was $6,000. The contract with Legal Consultant was signed on 13 September 2011. In response to the request of the Ministry of Finance and approval from the World Bank, contracts with the above consultants were extended by 2 more months.

4. Introduction of the Unified System of Accounting with Use of UCA and

Development of Reporting in accordance with СООС of the Ministry of Finance

In the context of reforms being conducted to improve the public finance management system, the Ministry of Finance of the Kyrgyz Republic, in accordance with the Decree of the President of the Kyrgyz Republic dated 22 October 2009 no. 396, plans to shift to the international standards of financial reporting for the public sector (IFRSPS). IFRSPS will be initially launched on cash basis with further transfer to accrual basis. In this regards, the Central Treasury initiated hiring an international consultant. The terms of reference were prepared by the Ministry of Finance and approved by the World Bank. Following the review of CVs submitted, the best candidate meeting criteria was selected. Given that financing from GTAC of this consultant is over, the further financing will be supported from the trust fund. IV. Other Activities Financed under GTAC Project 1. Matters Related to Network Infrastructure of TMIS/HRMIS

1.1. To implement TMIS/HRMIS, there was an agreement signed between the Ministry of Finance and KyrgyzTelecom Joint-Stock Company on provision of communication services to the Ministry of Finance (connection through VPNL2 channel) for FreeBalance Software with throughout capacity above 128 Kb/s. Today, these lines are being used by the Central Treasury for the current automated system called “IS:Kazna.Budget”. At present, the Ministry of Finance is conducting works on connecting regional treasury units to VPNL2 channels and to territorial financial units; this measure will enable increasing the speed of throughout capacity on these channels significantly.

1.2. Prior to the start of TMIS/HRMIS Project implementation, the World Bank set a number of preliminary conditions for the Government of the Kyrgyz Republic related to establishment of information and telecommunication infrastructure. These conditions were fully met by the Government, i.e. the InfoSystem State Enterprise was established, the Data Processing Center and the Reserve Data Processing Center were reconstructed (prepared), and the fiber-optic lines of communication (FOCL)

65

between KyrgyzTelecom Joint-Stock Company, the Central Treasury of the Ministry of Finance, State Personnel Service, DPC, and RDPC were laid. For laying the above FOCL the Ministry of Finance has signed contracts with KyrgyzTelecom Joint-Stock Company and Aknet ltd. for the total amount of KGS 1,424,293, financed from the state budget.

2. Activities Being Implemented and Planned under the “Civil Service Efficiency Improvement” Component 2.1. Training of the civil servants. Budget - $98,157.37. 2.1.1. Develop capacity of the civil servants and the State Personnel Service of the Kyrgyz Republic (conduct conferences, seminars, and study tours). The funds remained unused. 2.2 Consulting Services 2.2.1. Develop and implement the Training Program for Stats Secretaries of Government Agencies. Budget – $33,188.90. Funds remained unused. 2.2.2. Hire local consultants to develop drafts of normative legal acts and methodic instructions to implement the Concept of Modernized Civil Service of the Kyrgyz Republic. Budget – $73,600. The draft terms of reference were developed by the State Personnel Service and approved by the World Bank. Besides, there was recruitment of the relevant consultant announced and CVs collected. Yet, since no candidate met the qualification criteria and given the end of GTAC project, this selection was terminated. Funds remained unused. 2.2.3 Hire an IT Consultant to Assist in Implementation of the Information System Component. Budget – $8,000. All procedures on hiring a consultant were conducted in accordance with the World Bank requirements. The approval was received. Contract no. CS-GTAC/INDV-32/2013 was signed between the Ministry of Finance of the Kyrgyz Republic being the Executing Agency for GTAC Project and Mr. Kalybekov on 05.04.2013. 2.3 Goods and Commodities 2.3.1 Procurement of computer and office equipment for the State Personnel Service. Budget: $144,500. The announcement on re-tender was published on 31 August 2012. Bids were opened on 12 October at the Ministry of Finance. The technical and financial evaluation processes were suspended for the period of conducting negotiations between the Government of the

66

Kyrgyz Republic and the World Bank on GTAC Extension until end-2013. Following the decision on the project extension, bid evaluation process resumed. The tender committee conducted the following procedures of technical and financial evaluation in accordance with the World Bank requirements: evaluation report on procurement of the computer and software for the State Personnel Service was sent to the World Bank for approval on 22.02.2013. The World Bank had comments of technical character related to some of the equipment types dated 20.03.2013. The comments were reviewed by the procurement specialist and experts. The information was sent to the World Bank on 29.03.2013. Approval from the World Bank was received in April 2013. Contracts were signed between the Ministry of Finance of the Kyrgyz Republic and Logic Company (Lot 1,3, Contract dated April 2013) and Atexis Company (Lot 2, Contract dated 3 May 2013) for delivery of computer and telecommunication equipment for the State Personnel Service under GTAC Project where all activities were completed and contract amounts fully disbursed. 3. Other activities 3.1. Under the GTAC Project there was office furniture procured for the Central Treasury (chairs). 3.2. Under the GTAC Project office equipment and computers were procured for the Central Treasury staff. 3.3. Under the GTAC Project office equipment and computers were procured for the subordinate units of the State Personnel Service.

Findings

In spite of the long implementation period of the GTAC Project, the Project did not achieve its primary goal: implementation of a modern information system for accounting of public funds and human resources based on the international standards failed.

In light of the situation developed, we consider the following potential reasons.

• Frequent changes in the project management and of project participants: since the date of signing the contract with FreeBalance for implementation of TMIS\HRMIS (2010) there were the following changes observed: project heads from the World Bank’s side - 4, Ministers of Finance - 6, Project Directors – 4, vice presidents of FreeBalance – 3, country managers – 4. In spite of this, changes in management had a minor impact on the project implementation progress, as every manager tried to make every effort for successful implementation.

• High level of aggregation and generality of contract requirements which allowed different understanding by each of the parties. It should be noted that the contract was signed in 2009 based on specifications developed in 2005. When signing the contract, the changes in the pay system of the Kyrgyz Republic and objectives of the Ministry of Finance to shift to international accounting and reporting

67

standards were not taken into account. All of these aspects were reflected in the contract, yet in a generalized manner. Therefore, clarifications in the design of the future system took a long time.

• Systematic failure to fulfill the requirements by the Supplier of TMIS/HRMIS application software. This resulted in extending the project 3 times. Moreover, the last Work Schedule was also not fulfilled within the given timeframe (Attachment 1 gives detailed monitoring information as of 25.12. 2013).

• Lack of qualified functional and technical specialists from FreeBalance. In order to timely resolve the issue, the Ministry of Finance repeatedly communicated to FreeBalance a request to mobilize highly qualified experts on the functionality of TMIS, Budget Module and a Specialist on interfaces for the entire period of project implementation. However, specialists have been coming for 1 to 2 weeks, attending 2-3 meetings and going back. In 2-3 months new specialists would be coming with the same questions as their predecessors. There is no consistency of operations. A more permanent presence has been ensured by managers, yet they were more focused on addressing the conflicting issues related to contract terms. Staff of the local office are not fully aware of the possibilities and products of the mother company.

• When conflicts would arise related to contract terms, it took a long time to finally agree on some of them. For example, the language issue took 2 to 3 months (because different sections of the contract stipulated different terms), UCA discussion took more than 8 months, and discussion of the UCS functioning principle took more than 4 months. While the Ministry of Finance was the Client of the System, it had to prove to FreeBalance the properness and correctness of its requests, compliance with the international accounting and reporting standards, and information safety requirements. For that, the Ministry of Finance of the Kyrgyz Republic appealed to the International Monetary Fund and the World Bank, thus the relevant consultants supported the decision of the Ministry of Finance and, only after that, FreeBalance would commence the next stages of project activities.

• The needed level of support from InfoSystem State Enterprise under the Ministry of Finance to FreeBalance was not achieved in implementation and introduction of TMIS/HRMIS application software.

• The psychological tiredness of staff of the Ministry of Finance from receiving reports by FreeBalance with no practical results (e.g., in a form of separate modules demonstration). Practically, all of the reports submitted by FreeBalance had to be revised many times and it took a long time always, some of the reports were approved after the 14th or 15th submission.

The methodological and consultative support from the World Bank and the International Monetary Fund in addressing the aforementioned issues related to understanding of the contract terms should be particularly noted. The relevant experts were engaged by the World Bank and IMF which helped to correctly determine the project objectives in accordance with its goals and in compliance with the international standards and

68

requirements. However, we think it would be more efficient if the experts would be engaged on permanent basis and closely cooperate with the project, i.e. not only when problems would arise, but when moving from one stage to another in view of identifying the similar difficulties and working more closely with the executing parties in charge of each component.

We’d like to emphasize the continuous monitoring and support from the World Bank Project Team Leaders. With the appointment of a new Project Team Leader, Mr. David Nummy, there has been regular monitoring of the project progress conducted (once in 3 weeks) through video conferences, where each party could express the thoughts and a unanimous decision would be made and further reflected in the minutes. We think the same practice should have been adopted from the very start of the project. We also would like to emphasize the following positive aspects in GTAC Project implementation: 1. Technical infrastructure for the automated treasury system functioning was developed:

• Reconstruction and refurbishment of the server centers for the Main and Reserve Data Processing Center;

• Equipment delivered for the central part of TMIS/HRMIS system, 22 servers, data storage systems and management and monitoring systems in total;

• Equipment and software delivered and main & local networks installed at the regional units of the Treasury of the Ministry of Finance;

• Computers of the Central Treasury of the Ministry of Finance and its regional units were renewed;

• Training has been conducted on information technologies. 2. The Ministry of Finance of the Kyrgyz Republic developed the normative-legal base for the improved treasury system:

• In accordance with the Resolution of the Government of the Kyrgyz Republic no.597 dated 30 August 2012, there was a Temporary Instruction on Functioning of the Unified Treasury Account approved for the Central Treasury of the Ministry of Finance;

• The Unified Chart of Accounts (UCA) and the Manual on UCA Use were developed and approved by the Order of the Ministry of Finance no.177-П dated 04 September 2012 in accordance with the Resolution of the Government of the Kyrgyz Republic “On Delegation of Authorities to the Ministry of Finance of the Kyrgyz Republic to Approve the Unified Chart of Accounts and the Manual on Its Use by the Public Administration Sector” no. 605 dated 04 September 2012.

• Based on the recommendation of the World Bank, the Protocol of Interaction between the Ministry of Finance of the Kyrgyz Republic and the National Bank of the Kyrgyz Republic on Functioning of the Unified Treasury Account of the Central Treasury of the Ministry of Finance was approved by both parties on 23.10.2012.

69

• Design of future TMIS and HRMIS developed.

70

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable

71

Annex 9. List of Supporting Documents GTAC Program Document, April 2003 Aide Memoires for GTAC ISRs for GTAC GTAC mid-term review Contracts and Consultant Reports GSAC Program Document, April 21, 2003 CAS-2004-07 General Correspondence

72

MAP

I N S E R T

M A P

H E R E

AFTER APPROVAL BY COUNTRY DIRECTOR

AN ORIGINAL MAP OBTAINED FROM GSD MAP DESIGN UNIT

SHOULD BE INSERTED

MANUALLY IN HARD COPY

BEFORE SENDING A FINAL ICR TO THE PRINT SHOP.

NOTE: To obtain a map, please contact

the GSD Map Design Unit (Ext. 31482)

A minimum of a one week turnaround is required

73