ASIAN

CAREWe

This Commaccor

N DEVE

PROJE

EC Transpstern PRC

report has municationsrdance with

ELOPME

ECT PROC

port CorridC Internat

been redac Policy (PCPparagraph 1

Office

ENT BA

CUREMENLoan

dor I (Zhamional Tran

P

cted in accoP). In partic126 of the PC

of Antico Issue

ANK

NT-RELATn 2503-KAmbyl Oblansit CorridProject I

rdance withular, it excluCP, items 9,

orruption aed 20 July 201

TED REVIAZ: ast Sectiodor 1] Inve

h the Asian udes confide, 11, 12, 14

and Integ1

EW REPO

on) [Westeestment P

Developmeential and oth& 15.

rity

ORT

ern EuropeProgram –

nt Bank’s Pher informati

e – –

Public ion in

2

A Project Procurement-Related Review (PPRR) is

a review undertaken by OAI on ongoing ADB-financed projects, to confirm compliance with applicable ADB’s policies, guidelines, and the loan agreement, with a focus on preventing

and detecting integrity violations (http://www.adb.org/Integrity/integrity-violations.asp) involving ADB-related activities as defined under ADB’s Anticorruption Policy as amended (http://www.adb.org/Anticorruption/policy procedures.asp) and ADB’s Integrity Principles

and Guidelines (http://www.adb.org/Documents/Guidelines/Integrity-Principles-Guidelines/2010/integrity-guidelines-procedures-2010.pdf).

ADB’s Anticorruption Policy requires all parties, including borrowers, beneficiaries, bidders, consultants, suppliers, contractors, and ADB staff to observe the highest ethical standards

when participating in ADB-related activities. The Policy supports ADB’s obligation, in accordance with Article 14(xi) of the Agreement Establishing the Asian Development Bank,

to ensure that the proceeds of ADB financing are used only for intended purposes.

The PPRR assesses internal controls in place, identifies irregularities and instances of non-compliance, inspects the project outputs, and recommends enhancements to mitigate or eliminate opportunities for fraud, corruption, or abuse of resources and to help improve

development effectiveness of future projects.

A Project Procurement-Related Review is not an evaluation to assess development effectiveness of ADB-funded projects. It does not review project outcomes or development impact, which can only be assessed after the

finalization of a project.

CURRENCY EQUIVALENTS (as of 31 March 2010)

Currency Unit = Kazakhstan Tenge (KZT) 1.00 = $0.0068038782

$1.00 = KZT 146.975000

ABBREVIATIONS

ADB - Asian Development Bank BOQ - bill of quantity CAREC - Central Asia Regional Economic

Cooperation COR - Committee of Roads CV - curriculum vitae CWRD - ADB Central and West Asia Department CWTC - ADB Transport and Communications

Division, CWRD EA EOI

- -

executing agency expression of interest

FFA - financing framework agreement IA - implementing agency ITS - intelligent transport system MFF - multitranche financing facility MOTC - Ministry of Transport and Communications OAI - ADB Office of Anticorruption and Integrity PRC - People’s Republic of China RDP - road development plan TOR - terms of reference

NOTE

In this report, $ refers to US dollars.

CONTENTS

Page

EXECUTIVE SUMMARY I 

I.  PROJECT BACKGROUND 1 

II.  REVIEW OBJECTIVES, SCOPE, AND METHODOLOGY 3 

III.  REVIEW FINDINGS AND RECOMMENDATIONS 5 A.  Procurement 5 B.  Deployment/Replacement of Experts 10 C.  Asset Verification 11 D.  Financial Management 12 E.  Weaknesses in Internal Control & Capacity Issues 12 

IV.  CONCLUDING COMMENTS 13 

i

EXECUTIVE SUMMARY

1. The Office of Anticorruption and Integrity (OAI) conducted a project procurement-related review (PPRR) of Asian Development Bank (ADB) Loan 2503-KAZ: CAREC Transport Corridor I (Zhambyl Oblast Section) [Western Europe – Western PRC International Transit Corridor 1] Investment Program – Project I (Project) in Kazakhstan (KAZ) from June to September 2010.1 This report documents findings and makes recommendations as a result of the PPRR. 2. The Project is the 1st tranche of the Multitranche Financing Facility (MFF) agreed under the Financing Framework Agreement signed between the Republic of Kazakhstan and Asian Development Bank on 13 January 2009.2 The MFF is intended to finance projects under the Investment Program aimed to support the Republic of Kazakhstan’s objective to develop its transport sector as outlined in its Transport Strategy and Road Development Program 2006-2012.

3. The CAREC Transport Sector Strategy for 2008-2018 prepared under ADB assistance has three overarching goals: (i) establishing competitive transport corridors across the CAREC region; (ii) facilitating efficient movement of people and goods across borders; and (iii) developing safe, people-friendly transport systems that are environmentally friendly. This strategy also identified six strategic corridors for full completion by 2018. The Investment Program is in line with the 2006-2008 ADB’s Country Strategy and Program Update for Kazakhstan that includes support in transport sector focusing on national networks that are critical links in the regional network to reduce physical and non-physical barriers to regional trade and movement of goods and people.3

4. Total investment cost was estimated at $6,700 million, of which ADB finances $700 million through the MFF. Loan 2503-KAZ or Tranche 1, with a total estimated project cost of $400 million was declared effective on 31 July 2009. ADB’s funding of $340 million comprised two components: (i) Road Development for construction of a 125 km highway i.e., from km 404 to 483 and km 214 to 260 in Zhambyl Oblast; and (ii) Road Operations and Maintenance for improvement of the road maintenance system and development of an Intelligent Transport System. 5. The PPRR Team examined the Project’s procurement and disbursement documentation, inspected road construction works, and performed a review of the financial management system. The main PPRR objective is to identify and mitigate internal control weaknesses that may impair development effectiveness or allow integrity violations to occur in ADB-financed projects as defined under ADB’s Anticorruption Policy4 as amended and ADB’s Integrity Principles and Guidelines (May 2010).

6. The findings and recommendations in this report are intended to further enhance development effectiveness of the Project and subsequent tranches of the Investment Program. Comments on development effectiveness throughout this report are limited to PPRR observations, and how recommendations made here - with respect to anticorruption measures, strong internal controls and compliance with applicable guidelines - can increase development 1 OAI staff members and engaged consultants comprised the PPRR Team. 2 The Loan Agreement for the Project was signed on 30 March 2009. 3 ADB. 2009. Financing Framework Agreement – Multitranche Financing Facility, KAZ – CAREC Transport Corridor

(Zhambyl Oblast Sections) [Western Europe to Western PRC International Transit Corridor] Investment Program. 4 ADB. 1998. Anticorruption Policy. Manila. para. 67.

ii

effectiveness. This report does not and cannot comment on the overall development effectiveness of the Project. Major Recommendations 7. The key recommendations are for Ministry of Transport and Communications (MOTC) and Committee of Roads (COR), in coordination with ADB Central and West Asia Department (CWRD) to:

• reinforce the evaluation process and ensure selection of the most responsive and qualified bidder/consultant;

• strengthen controls and ensure the procurement/selection process is transparent and conforms fully to ADB Procurement Guidelines/Guidelines on the Use of Consultants by ADB and Its Borrowers;

• institute measures to ensure key proposed experts are available for deployment during their engagement term prior to finalizing the contract with winning consulting firm;

• identify relevant trainings to reinforce its capacity on Project procurement and disbursements;

• revisit the design of future tranches and institute measures to minimize costly variations; and

• jointly work with ADB Kazakhstan Resident Mission (KARM) to ensure that procurement integrity and transparency are established at the commencement of every development project and observed throughout implementation.

8. Strong procurement, financial management, and internal controls mitigate the risk of improper use of Project funds and assets, maximize development effectiveness, and deter fraud and corruption. The Borrower is encouraged to continue taking the lead in addressing this promptly, and collaborate with ADB to strengthen Kazakhstan’s capacity to manage for development results. 9. The PPRR Team also recommends for COSO to continue taking steps to integrate the needed information for publication of contract awards in its database and system enhancement in compliance with ADB’s Procurement Guidelines and Guidelines on the Use of Consultants, and to achieve enhanced transparency in the procurement process. Results in Brief 10. Red flags. The PPRR Team noted instances that ADB’s Anticorruption Policy may have been violated by contractors and consultants. OAI intends to investigate these indicators in relation to possible misrepresentation in accordance with ADB’s Integrity Principles and Guidelines. 11. Procurement. The PPRR Team determined that the Project generally complied with ADB’s Procurement Guidelines, Guidelines on the Use of Consultants, Framework Financing Agreement, and Loan Agreement except for a few deviations and areas for improvement. These include (i) questionable shortlisting of consultants and evaluation of bids/proposals; (ii) lack of due diligence in examining documents submitted by contractors/consultants; (iii) inconsistencies in applying shortlisting criteria; (iv) contracts awarded after expiry of bid validity; and (v) inadequate information in the publication of contract awards.

iii

12. Deployment of Experts. The PPRR Team noted instances where proposed key experts did not participate in the Project, with delayed, or no replacements. While the delays in mobilization and replacement or non-replacement of experts did not significantly impact the project progress as of review date, recurrence of deployment issues may in future invariably affect project implementation progress.

13. Asset Inspection. The PPRR Team inspected three construction works projects. While the construction progress was generally on track with no significant work delays, a number of variations to two contracts, with significant cost implications are anticipated. 14. Financial Management. The PPRR Team examined the financial management system at COR. Based on review of processed claims at the time of the review, which only included mobilization advances and an interim progress payment, the PPRR Team determined that these claims were disbursed properly and complied with ADB Disbursement Handbook. COR maintained sound financial records on ADB accounts. Concluding Comments 15. The PPRR Team recognizes the challenges faced by MOTC and COR inherent in implementing an ADB project for the first time – considering that the project is part of an extensive investment program through MFF. Greater collaboration may be required between CWRD, KARM, and MOTC/Government of Kazakhstan to ensure procurement capacity and transparency are enhanced. 16. Despite the weaknesses noted, there were positive indications that the Project will deliver the expected outputs through the commitment and dedication of MOTC and COR, such as construction work being generally on track with no significant delay. Implementation of recommendations in this report can only augment results achieved to date. The PPRR Team encourages CWRD, KARM, MOTC, and COR to implement lessons learned from this Project to the succeeding tranches of the investment program.

17. Firms or individuals who violated ADB’s Anticorruption Policy by engaging in corrupt, coercive, collusive, or fraudulent practices are not eligible to participate in ADB-financed, administered, or supported activity during their sanction period. The PPRR Team encourages MOTC and COR to sign up for access to ADB’s full sanctions list (http://sanctions.adb.org) and check the sanctions list before engaging any entities to protect project funds and project outcomes. 18. The PPRR Team thanks Project management and staff, especially MOTC and COR, as well as CWRD and KARM for their cooperation and assistance to the PPRR requirements and valuable inputs. ADB values the courtesy and support that Project officials extended to the PPRR Team.

1

I. PROJECT BACKGROUND 1. The Asian Development Bank (ADB) Loan No. 2503-KAZ: Central Asia Regional Economic Cooperation (CAREC) Transport Corridor I (Zhambyl Oblast Section) [Western Europe – Western People’s Republic of China International Transit Corridor I] Investment Program – Project I (the Project) is the 1st tranche of the Multitranche Financing Facility (MFF) agreed under the Financing Framework Agreement (FFA) signed between the Republic of Kazakhstan and Asian Development Bank on 13 January 2009. The Loan Agreement for the Project was executed on 30 March 2009. 2. The MFF is intended to finance projects under the Investment Program which aims to support the Republic of Kazakhstan’s objective to develop its transport sector as outlined in its Transport Strategy and Road Development Program 2006-2012 (RDP). The RDP comprises both physical and non-physical investment plans. The physical investments aim to reconstruct and/or expand about 7,000 km of national roads, perform small-scale rehabilitation works on 18,000 km of the network, as well as significant rehabilitation and upgrade of 18,000 km local roads. Non-physical investments cover institutional reforms, management training, improved technology and systems, and promotion of private sector participation in selected road sections. 3. The Investment Program aims to improve and expand the Western Europe – Western People’s Republic of China (PRC) International Transit corridor, which runs from Khorgos at the border of PRC, through Almaty and Shymkent, to the Western Border of the Russian Federation. The Project was designed to provide sustainable economic development and increased domestic and international trade growth within Kazakhstan, and between its trading partners, through reduced vehicle operating costs; travel time; and traffic accident rates of domestic, regional, and transit traffic on the Regional Highway in Zhambyl Oblast, connecting cities of Almaty and Shymkent.

4. The Investment Program is an integral part of a huge project that aims to remove existing bottlenecks to achieve better connectivity, competition, and trade by building 14 strategic road corridors including 6 international corridors that span to 2,715 km; 2,237 km of which are in the pipeline for construction and rehabilitation. Total investment cost was estimated at $6,700 million and sourced from the following: (i) ADB and other international financial institutions5 ($3,570 million or 53%), private sector ($2,221 million or 33%), and the Government ($909 million or 14%).

5. ADB and its co-financing partners, Japan International Cooperation Agency and Islamic Development Bank finances $1,264 million of the $6,700 estimated cost where ADB finances $700 million through the MFF. The Table below illustrates the allocation of the funding share of ADB and its co-financing partners to the Investment Program, including the allocation for the subject Loan 2503-KAZ, and the subsequent tranches:

5 Funding international financial institutions include ADB, Islamic Development Bank, Japan Bank for International

Cooperation Agency, European Bank for Reconstruction and Development and the World Bank.

2

Summary Financing Plan of the Investment Program

Financier and Financing Modality Project I

(Loan 2503-KAZ)

Subsequent Tranches Total % to Total

Share

International Financial Institutions

ADB Multitranche Financing Facility 6 340 360 700 47% IDB – Collaborative / Parallel Financing 0 414 414 28% JICA – Joint Financing 0 150 150 10% Total IFI Share 340 924 1264 85% Government 60 156 216 15% Total Project Cost 400 1,080 1,480 100

6. Loan 2503-KAZ or Tranche 1, with a total estimated project cost of $400 million was declared effective on 31 July 2009. ADB’s funding of $340 million comprised two components: (i) Road Development for construction of a 125 km highway i.e., from km 404 to 483 and km 214 to 260 in Zhambyl Oblast; and (ii) Road Operations and Maintenance for improvement of the road maintenance system and development of an Intelligent Transport System (ITS). The loan allocation as of 31 March 2010 (the review cut-off date) was as follows:

Category Name Cat. Ref. Allocation (In million $) % of Financing

Works7 01 297.0 85% of total expenditure claimed Consulting Services 02 14.0 100% of total expenditure claimed* Unallocated 03 29.0

Total 340.0 Basis: ADB Loan & Grant Financial Information System * Exclusive of taxes and duties imposed within the territory of the Borrower 7. The Ministry of Transport and Communications (MOTC) is the Project executing agency (EA) and is responsible for the overall project implementation, as well as in liaising with ADB and other IFIs involved in the Investment Program. The Committee of Roads (COR) is the Project implementing agency (IA), headed by a Project Director who oversees the day-to-day activities, including project preparation, implementation, monitoring, and reporting. The Project Director, and its support team from MOTC, the engaged project management consultants and construction supervision consultants comprised the IA – ADB team, and is primarily in charge of the administration of Project funds sourced from ADB and its co-financing partners. 8. Roads are a key element of Kazakhstan’s transport system. They play an important role in providing access to rural areas, facilitating transit traffic and in-country transport movement. However, much of the road network is in poor condition, requiring major

6 Tranches may be provided in sequence or simultaneously and some may overlap in time with each other. The

facility will be provided in individual tranches from ADB’s Ordinary Capital Resources in the following manner: Tranche 1: $340 million, tranche 2: $210 million, and tranche 3: $150 million.

7 Works would mean construction or civil works to be financed from the proceeds of the loan, and including services such as drilling or mapping, and project related services that are provided as part of a single responsibility or turnkey contract, but excluding consulting services.

3

rehabilitation and proper maintenance. It is expected that the Investment Program will benefit an array of people and business in Kazakhstan and neighboring countries. The main benefits revolve around better, safer, cheaper, and faster connectivity. Better connectivity can yield important externalities, including access to health and education centers. Urban centers will become better connected to isolated rural areas, thus improving mobility and job opportunities in the city.8

9. Transport development is one of the six topics for dialogue and intervention included in ADB’s strategy, focusing on “supporting road transport development(s) that improve and maintain existing infrastructure, facilitate improvements to transport services through support for intermodal development and creation of competitive transport markets.” As part of the large support for the whole corridor from international financial institutions, the development of this corridor will also help “ensure greater regional economic linkages by improving regional trade links.” The Investment Program is in line with ADB’s country strategy and program update (2006-2008) for Kazakhstan, which includes support in the transport sector focusing on national networks that are critical links in the regional network. It is consistent with the agreement, where support to road transport development is one of the main focuses of the medium-term strategy.9

II. REVIEW OBJECTIVES, SCOPE, AND METHODOLOGY

10. The overall PPPR objective is to prevent and detect fraud and corruption as defined under ADB’s Anticorruption Policy10. Corrupt behavior is a serious impediment to the development process, severely reducing development effectiveness and jeopardizing successful delivery of development benefits. The Anticorruption Policy, alongside with the Procurement Guidelines11 and Guidelines on the Use of Consultants by Asian Development Bank and its Borrowers12 require all parties, including borrowers, beneficiaries, bidders, consultants, suppliers, contractors, and ADB staff to maintain the highest ethical standards for ADB-financed activities.13 11. To ensure such ethical standards, ADB’s Anticorruption Policy prohibits fraudulent and corrupt practices in ADB-financed, administered, or supported operations, and defines the following as integrity violations:

• a corrupt practice as the offering, giving, receiving or soliciting, directly or indirectly, anything of value to influence improperly the actions of another party;

• a fraudulent practice as any action or omission, including a misrepresentation, that knowingly or recklessly misleads, or attempts to mislead, a party to obtain a financial or other benefit or to avoid an obligation;

8 ADB. 2008. Proposed Multitranche Financing Facility and Administration of Loan CAREC Transport Corridor I

(Zhambyl Oblast Section) [Western Europe-Western People’s Republic of China International Transit Corridor] Investment Program (Kazakhstan). Manila.

9 Ibid. 10 Paragraph 67, ADB’s Anticorruption Policy (2 July 1998). Manila. 11 Paragraph 1.06, Guidelines for Procurement Under Asian Development Bank Loans (November 2004); Paragraph

1.14, Procurement Guidelines (February 2007 and April 2010). Manila. 12 Paragraph 1.23, Guidelines on The Use of Consultants by Asian Development Bank and Its Borrowers (February

2007 and April 2010). Manila. 13 Paragraph 14(iii), ADB’s Anticorruption Policy (2 July 1998). Manila.

4

• a coercive practice as impairing or harming, or threatening to impair or harm, directly or indirectly, any party or the property of the party to influence improperly the actions of a party; and

• a collusive practice as an arrangement between two or more parties designed to achieve an improper purpose, including influencing improperly the actions of another party.

12. In addition,

• abuse is theft, waste, or improper use of assets related to ADB-related activity, either committed intentionally or through reckless disregard;

• a conflict of interest is any situation in which a party has interests that could improperly influence that party’s performance of official duties or responsibilities, contractual obligations, or compliance with applicable laws and regulations, and that such conflict of interest may contribute to or constitute a prohibited practice under the anticorruption policy; and

• an obstructive practice is (i) deliberately destroying, falsifying, altering, or concealing of evidence material to an ADB investigation; (ii) making false statements to investigators in order to materially impede an ADB investigation; (iii) threatening, harassing, or intimidating any party to prevent it from disclosing its knowledge of matters relevant to the investigation or from pursuing the investigation; or (iv) materially impeding ADB’s contractual rights of audit or access to information.

13. Other integrity violations include:

• violations of ADB sanctions; • other violations of ADB’s Anticorruption Policy, including failure to adhere to the

highest ethical standards; and • retaliation against whistleblowers or witnesses, which is any detrimental act,

direct or indirect, recommended, threatened, or taken against a whistleblower or witness or person associated with a whistleblower or witness in a manner material to a complaint because of the report or cooperation with an ADB investigation by the whistleblower or witness, which shall be investigated in accordance with Administrative Order 2.10.14

14. The PPRR is intended to:

• identify whether the procurement of project goods, works, and services complied with ADB’s Procurement Guidelines, Guidelines on the Use of Consultants, and the related Financing Framework Agreement and the Loan Agreement;

• determine whether project contracts were implemented according to the terms; • ensure ADB’s funds were used for their intended purposes; and • recommend improvements to internal controls that mitigate opportunities for

fraud, corruption, or abuse in this project and future ADB-financed projects.

14 Paragraph 2, Integrity Principles and Guidelines (May 2010). Manila; and ADB Administrative Order 2.10:

Whistleblower and Witness Protection.

5

15. PPRRs are conducted in line with ADB’s efforts to manage for development results. PPRRs identify internal controls that should be in place or strengthened to mitigate or eliminate instances of fraud and corruption, or abuse of resources. Effectively applied internal controls give assurance that project funds are directed towards intended uses for intended beneficiaries. PPRRs thus contribute to ensuring ADB-financed projects are managed to produce intended development results. 16. To achieve the review objectives, the PPRR Team conducted a detailed review of documentation at the IA, inquiries with IA staff and engaged consultants, and ocular inspection at the project sites, and performed the following:

• reviewed the Project’s procurement and disbursement documents; • evaluated the Project’s procurement processes, internal controls, and financial

management practices; and • inspected existing works at Project sites.

17. The PPRR Team examined the procurement documentation and processes for all 6 contracts awarded as of 31 March 2010, amounting to a total of $187.83 million. These comprised three civil works contracts procured through international competitive bidding, and three consultancy contracts awarded under the quality-and-cost-based selection. The PPRR Team also examined the processed claims, including mobilization advances under the foregoing contracts as of 31 March 2010.

III. REVIEW FINDINGS AND RECOMMENDATIONS

18. The PPRR findings pertain primarily to Project procurement, deployment/replacement of experts, asset verification, financial management, and internal control and capacity issues, as highlighted below. A. Procurement

19. Based on the review of the Project’s procurement procedures and documents, the PPRR Team determined that the Project generally complied with ADB’s Procurement Guidelines, Guidelines on the Use of Consultants, Project’s Framework Financing Agreement, and the Loan Agreement except for a few noted deviations and areas for improvement. 20. The PPRR Team also noted instances where ADB’s Anticorruption Policy may have been violated by contractors and consultants. OAI intends to investigate these indicators in relation to possible misrepresentation in accordance with ADB’s Integrity Principles and Guidelines. Violations to ADB’s Anticorruption Policy invariably results in ineffective and/or improper use of the Project funds, thereby reducing development effectiveness.

1. Unsupported Bid Data

21. The PPRR team noted that the information in the required financial bid form did not match the financial data provided during prequalification by a bidder for a construction contract. The bidder’s net worth for the three-year period CY 2006-2008 and reported revenues for 2007 and 2008 were higher in the actual bid submission compared to the figures reported during prequalification. Reported figures in the actual tender were also inaccurate as the sum of total liabilities and total net worth did not tally with the total assets.

6

22. Current assets and current liabilities information which are critical to gauge the bidder’s available working capital resources were missing from the bid submission. These information were subsequently provided to the IA by the bidder in its five-year financial position form in response to IA’s request for clarification. The PPRR Team, however noted that the financial information submitted to establish the bidder’s available working capital appeared to be unreliable, particularly for 2008 as reported current assets amounting to $71,273,803 far exceeded the reported total assets of $58,735,783. In addition, the current liabilities for 2006 and 2008 amounting to $34,925,032 and $50,764,841, respectively were higher than the reported total liabilities for the same period.15 23. The bid evaluation report confirmed the non-submission of the bidder’s financial statements during actual tender, and that reference was made to the bidder’s balance sheets during prequalification to assess its financial condition.16 The balance sheets claimed to have been submitted during prequalification were nonetheless not seen by the PPRR Team. The lack of source documents to validate the reported figures in the bid form posed a limitation on the PPRR review scope. 24. The bid document also requires all bidders and each partner to a joint venture to provide information on their current commitments on all contracts that have been awarded, or for which a letter of intent or acceptance has been received, or for contracts approaching completion, but for which an unqualified, full completion certificate has yet to be issued.17 The PPRR Team noted inconsistent information on the same bidder’s prequalification and bid submissions, particularly on its reported contract commitments. 25. Review of the prequalification and bid documents disclosed that the contract commitments declared in the winning bidder’s bid submission was less than the number of projects shown in its prequalification document. The PPRR Team was informed that the undeclared contract was put on hold after prequalification stage due to funding constraints faced by the project owner, and thus was no longer reflected in the bid submission. This was not disclosed in the bid submission and the evaluation report. However, it should be noted that pending any termination, the contract still remains in force, and therefore should have been reported among the bidder’s current commitments, with full disclosure of its current status. Should the contract that was put on hold becomes implemented at the time the contractor is pre-occupied with the ADB Project contract, delays and poor quality of works may occur as a result of overlapping commitments.

2. Questionable Bid Evaluation

26. The bid document requires the bidders to submit their last three years audited financial statements to support the financial information they provide in the prescribed financial bid forms.18 Upon request, the PPRR Team obtained information and documents such as the bidder’s current assets and current liabilities, bank credit line statement, and detailed mobilization and construction schedule.

15 Total liabilities reported: 2006: $33,321,289; 2008: 50,691,379. 16 The Bid Evaluation Report (Appendix 3 – Examination of the Completeness of Bids) confirms the non-submission

of the financial statements, among others in the actual tender. 17 Section 4 – Current Contract Commitments/Work in Progress 18 Form Fin 1 – Financial Situation as per Bid Data Sheet Sec 4; and Fin – 3.1 Financial Situation, as per Sec. IV of

the PQ document. These documents require the bidder to provide its Total Assets, Total Liabilities, Current Assets, Current Liabilities, Total Revenues and Profit before Taxes.

7

27. Closer examination of the credit line statement provided by the bidder disclosed that the issuing bank merely indicated considering the possibility of providing a credit facility to the bidder for execution of the contract in the amount of $1.48 million for a period of three years. The letter from the issuing bank further indicated that a final decision will be rendered by the bank’s credit committee upon completion of the tender and award of the bid. The availability of the credit line was therefore not firm at that time as this was conditional and subject to the issuing bank’s appropriate approvals.

28. In its response letter dated 5 November 2009 to ADB the IA indicated that “we have examined their (the bidder’s) financial statements thoroughly, considering their 2008 cash reports, and reconfirm that they present a solid financial capacity required for the contracts.” The letter also indicated that “combining the other liquid assets, lines of credit, and cash into financial resources, they proved to meet the cash flow requirements of the contracts.” It is worthy to note that this confirmation of the bidder’s financial capacity cannot be substantiated, given the foregoing observations. 29. The equipment indicated in the bid of the same bidder was mostly leased items being utilized for ongoing projects. However, there were no accompanying lease agreements to establish the lease term, or to assess the availability of the equipment for the project under bid, which is not compliant to the bid requirement.19 30. The foregoing observed discrepancies in financial information for the same period in the submissions of the bidder during prequalification and actual tender, as well as during clarification stage. Questionable reported financial figures and lack of validation of submitted financial figures against source documents give the appearance of misrepresentation of these information and questionable evaluation process. The consideration of the bidder for further evaluation and its eventually winning the contract award despite non-compliance to critical requirements of the bid during bid submission is a violation of the ADB Procurement Guidelines.20

3. Lack of Due Diligence in the Bid Evaluation 31. The purpose of bid evaluation is to determine the cost to the borrower of each bid in a manner that permits a comparison on the basis of their evaluated cost. The bid with the lowest evaluated cost, but not necessarily the lowest submitted price, shall be selected for award.21 The PPRR Team noted that for construction contracts (Lots 1-3), price evaluation was limited to comparison of the total engineer’s estimate with the total bid prices of technically responsive bidders, and did not entail a rate analysis or comparison of the BOQ line item prices. The PPRR Team reviewed and compared unit prices of the BOQ line items against those in the engineer’s estimates, which disclosed abnormally high and low rates. 32. The IA explained that they have reviewed unit rates submitted by the bidders against the bill of quantities, and took into account the total cost of the project and price proposed by the bidder. Apart from the total proposed prices, comparison of the individual line-item prices of

19 Bid Data Sheet, Section 4 – Equipment Details. The bidders are required to provide details about the equipment

which would be used for the project execution. For all leased equipment, the bidder has to provide lease agreement, which indicate the period of availability of the equipment with the bidder.

20 ADB Procurement Guidelines (February 2007 and April 2010), paragraph 2.48. 21 ADB Procurement Guidelines(February 2007 and April 2010), paragraph 2.49.

8

competing bidders against the related individual engineer’s estimate facilitates an in-depth price evaluation. As a result, the bid evaluation committee may better assess the reasonableness of the bidders’ prices, analyze unusually high and low-priced items and its impact to the bidder’s evaluated cost, as well as identify areas for price negotiation. 33. In the evaluation of the expressions of interest (EOI) for a consulting contract, a participating firm was given the equivalent bonus points for being ISO certified on environment and traffic safety. However, there was no proof or certification submitted. The IA rationalized that the additional points were given based on the consultant’s EOI indicating that the company and its partners have the said certificate. Such consideration resulted into the consultant meeting the required score to be shortlisted and invited to submit its technical and financial proposals. The IA also indicated that had the consultant won the contract, they would have requested a copy of the ISO certificate. 34. Another consulting firm that expressed interest for the same consulting services was also given the corresponding points for an expired ISO certificate which otherwise would have failed them to be shortlisted. Similarly, a participating firm in another consulting contract was also awarded 150 points for an expired ISO certificate, which the IA attributed to oversight due to the certificate’s small format and unclear expiry date. In both instances, the additional points were instrumental in shortlisting the consulting firms. 35. The lack of due diligence affects the evaluation process and raises concerns on awarding contracts to contractors and consultants who may have inadequate qualifications.

4. Inconsistent Bid Evaluation 36. The ADB Guidelines on The Use of Consultants require that in preparing the shortlist, the borrower shall give first consideration to those firms expressing interest that possess relevant qualifications.22 The PPRR Team noted several instances of inconsistencies by the evaluation committee in applying the EOI shortlisting criteria among consulting firms that submitted EOIs. The lack of detailed attention to shortlisting requirements during evaluation may lead to questionable shortlisting process and inappropriate ranking of consultants to determine the shortlist. 37. A participating firm that expressed interest for consulting services for road management system was provided 300 points for regional experience - a mark that was above the allocated point score of 200 points. In the same shortlisting, it was also noted that another consultancy firm that garnered highest points for criteria on “specific work experience” and “specific regional experience,” was rated “zero” under “nature of firm.” If the firm was properly evaluated and classified, it should have received a qualifying mark and may have been considered along with other qualified consulting firms to submit technical and financial proposals. 38. The shortlisting criteria defined the maximum score allocation for six similar projects at 450 points, or an equivalent of 75 points each per similar project. The PPRR Team noted that two participating consulting firms who reported more than six similar projects were rated invariably for 420 and 405 points, respectively, and failed to get shortlisted. The IA explained that scores were flexibly adjusted depending on its semblance to the project for bidding (i.e., 75 points for similar projects, and 60 points for partially similar projects). These considerations, 22 ADB Guidelines on The Use of Consultants (February 2007 and April 2010), paragraph 2.8.

9

however, were not documented and therefore basis for the allocated scores could not be validated by the PPRR Team. 39. The project experience of an associated firm was not considered in rating the consulting firm’s regional experience, resulting in the under-evaluation of its score by 60 points that otherwise would have qualified the firm for the shortlist. The Evaluation Committee considered only the similar regional experience of the lead firm. ADB’s Guidelines on The Use of Consultants stipulates that consultants may associate with each other in the form of a joint venture or of a sub-consultancy agreement to complement respective areas of expertise, strengthen technical responsiveness of their proposals, and make available bigger pools of experts, provide better approaches and methodologies, and in some cases, offer lower prices.23 40. It was also noted that in terms of criteria on overseas experience, higher marks were given to two experts of the winning consulting firm in one of the consultancy contracts, when in fact their curriculum vitae (CV) did not show any experience working with international organizations apart from not being able to speak the English language. 41. The foregoing inconsistent evaluations indicate questionable evaluation process, which may result in shortlisting and awarding contracts to consultants with inadequate qualifications. The IA may also need to ensure adequate information is reflected in the relevant evaluation reports to ensure transparency in the consultant’s selection process.

42. In their response to the draft report, ADB Transport and Communications Division (CWTC) indicated that they have taken steps to ensure merits of the lead firm and the associated firm(s) or JV partners were appropriately considered in determining shortlisted firms for road maintenance system improvement consulting services, among others. CWTC also indicated that they have implemented a measure in coordination with COR to facilitate verification/validation of qualification of experts in the technical evaluation reports.

5. Contracts Awarded after Expiry of Bid Validity 43. The PPRR Team noted that the three consultancy contracts were awarded after the bid validity period stipulated in clause reference 1.12 of the Data Sheet. The bid validity period, however was not extended, and this poses a risk in ensuring availability of experts. Furthermore, the determination of the final ranking and request for approval from ADB were made after the bid validity period. 6. Publication of Contract Awards 44. ADB Procurement Guidelines and Guidelines on the Use of Consultants both require the publication of contract awards.24 For procurement contract awards, the following information are required to be published: (i) name of each bidder who submitted the bid; (ii) bid prices as read out in the bid opening; (iii) name and evaluated prices of each bid; (iv) name of bidders who were rejected and reasons for rejection; and (v) name of the winning bidder, the price it offered as well as the duration and summary scope of the contract awarded. As for consulting contracts, the following information is required to be provided to ADB for publication in its website: (i) the names of all the consultants who submitted proposals; (ii) technical points

23 ADB Guidelines on The Use of Consultants (February 2007 and April 2010), paragraph 1.14. 24 ADB Procurement Guidelines (February 2007 and April 2010), paragraph 2.60; and Guidelines on The Use of

Consultants (February 2007 and April 2010), paragraph 2.18.

10

assigned to each consultant; (iii) offered prices of each consultant; (iv) overall ranking of the consultants; and (v) winning consultant and the contract sum. 45. The PPRR Team noted that information on contract awards published in ADB’s website were limited to: (i) loan number; (ii) project name; (iii) executing agency; (iv) name and address of supplier; (v) contract nature, description and date; and (vii) project sector.25 As discussed with the Central Operations Services Office (COSO) staff, the disclosure of the other required information has not been facilitated yet in view of the continuing enhancement of COSO’s system and database. They are, however, mindful of this requirement and are taking steps to integrate the needed information into the database to facilitate its publication, and thus enhance transparency of the procurement process. Recommendations 46. It is recommended that MOTC and COR, in coordination with ADB Central and West Asia Department (CWRD) ensure that the evaluation committee:

• stringently assess the completeness of documentation submitted by contractors and consultants as prescribed in the prequalification/bid document and expression of interest/request for proposals, respectively; exercise due diligence in the evaluation process, and apply consistent selection criteria to ensure selection of the most responsive and qualified bidder/consultant;

• conduct a thorough and detailed analysis of price proposals to include among

others, line-item rate analysis of technically responsive bidders’ prices against engineer’s estimates, and price consistency analysis for similar line items quoted by same bidders bidding for different lots in similar bid packages, with due consideration of other factors affecting unit prices like material source, geographical attributes peculiar to the bid, etc. to determine the lowest evaluated bid; and

• document other evaluation considerations, including deviations or consensus as

agreed to by the evaluation committee in the assessment of bids and selection of consultants, to establish audit trail and fully achieve transparency.

47. The PPRR Team also recommends that MOTC and COR observe bid validity period set in the request for proposal, and seek extensions when necessary to ensure availability of proposed experts.

48. The PPRR Team further recommends that COSO continue to take steps to integrate the needed information for publication of contract awards in its database and system enhancement in compliance with the requirements of the ADB Procurement Guidelines and Guidelines on the Use of Consultants. B. Deployment/Replacement of Experts 49. In one of the consultancy contracts, the PPRR Team noted that during contract negotiations, MOTC requested substitution of a Transport Economist whose CV received less 25 Contract awards are also published in the MOTC’s website but extent of information disclosed was not verified by

the PPRR Team as these are no longer available online.

11

than 70 points of the total 100 score allocation. The firm agreed to submit a replacement but this never materialized, and an expert was never deployed. Instead, MOTC completed the supposedly Transport Economist’s work. 50. In the same contract above, the PPRR Team also noted that four experts named for various posts by the engaged consulting firm and whose CVs were signed by the firm and not by the individual experts were never deployed. These posts were for: (i) the Deputy Team Leader/Highway Engineer, (ii) Financial Management Specialist, (iii) Environment Specialist, and (iv) Procurement Specialist. It was noted that the four originally proposed experts subsequently expressed regrets not to participate in the project due to personal reasons. The Deputy Team Leader and Financial Management Specialist were replaced after a month of delay, and the related replacements each assumed the tasks supposedly to be performed by the Environment Specialist and Procurement Specialist, respectively. 51. Similarly, the proposed project managers for all of the civil works contracts were not deployed, and later replaced by the engaged contractors. The PPRR Team noted that two of these replacements were documented only through endorsement letters from concerned contractors to either the COR Deputy Chairman, or the supervision consultants with a copy to the COR Chairman. There was no related documentation seen on one of the replacements. 52. As of review date, the delayed mobilization and replacement of experts did not significantly impact the project progress as it was still on track. However, recurrence of deployment issues may later invariably affect implementation progress. In this regard, the PPRR Team emphasizes the importance of ensuring availability of the proposed experts as well as the process of validating reasons for replacements and qualifications of experts proposed therefore. Recommendations 53. The PPRR Team recommends that MOTC and COR reinforce negotiation proceedings by ensuring that key proposed experts are available for deployment during their related engagement term. Contracts should not be signed pending resolution of such issues. Non-compliance with contractual provisions should also be dealt with accordingly. 54. To ascertain the authenticity of the CVs and availability of individual experts for the project, the PPRR Team recommends for the evaluation committee to consider requiring CVs duly signed by the respective experts prior to contract signing. Also, changes or replacements should go through a formalized process to ensure prior validation of the qualifications of the proposed experts. Appropriate documentation should be maintained. C. Asset Verification 55. The PPRR Team inspected the three ongoing construction sites and confirmed that the construction progress is on track and there are no significant work delays as of the review date. However, discussions with construction supervision consultants and contractors disclosed that several variations with cost implications aggregating about KZT283 million and KZT846 million for contracts pertaining to Lots 2 and 3, respectively are anticipated. These are attributed to upgrade of transmission cable and equipment, relocation of power transmission lines, and inaccurate estimates of BOQ quantities and related costs. The PPRR Team was informed that approvals by MOTC and ADB are yet to be sought. Land acquisition issues were also noted for Lots 1 and 2 of the civil works contracts, and are pending resolution as of review date.

12

56. Considering the anticipated contract variations, it is prudent that these variations and cost implications are monitored from the outset. Lessons learned from the design of this tranche should be carried forward to the next tranches to minimize further costly variations.

Recommendation 57. It is recommended that MOTC and COR, in coordination with ADB CWRD:

• conduct a detailed review of the potential variations, including the resulting cost implications and address the same issues in future tranches;

• arrange site visits and encourage contractors to visit the project site before submitting their bids to assess actual requirements vis-à-vis project design, and conduct pre-bid meetings to discuss any clarifications on project design, among others; and

• work together to resolve the land acquisition issues to avoid implementation delays.

D. Financial Management 58. The PPRR Team, based on its review of processed claims which only included mobilization advances and an interim progress payment, determined that these were in accordance with ADB’s Disbursements Handbook. The EA maintained sound financial records on ADB accounts. E. Weaknesses in Internal Control & Capacity Issues 59. As the project is the first ADB project being implemented by MOTC, the PPRR Team noted that the IA was in the process of building its capacity during the procurement process for this tranche. During the early stages, a Procurement Specialist consultant was deployed by ADB to assist the IA in preparing the request for proposals for consulting services, prequalification documents for recruitment of contractors, technical specifications for goods and works to be procured, bidding documents, and related invitation notices. The Consultant’s terms of reference (TOR) also included assistance in the review and evaluation of bids. 60. While the Consultant’s TOR indicate training of the EA staff on procurement aspects, the PPRR Team noted that current members of the IA’s procurement team have not undergone training on ADB procurement procedures and guidelines. One of three finance staff who joined the IA recently to assist the Project has not attended the training on ADB disbursement guidelines either. Such capacity issues may have partly contributed to the foregoing weaknesses noted on the procurement process and evaluation. The engaged project management consultants and construction supervision consultants are expected to contribute in improving the institutional capacity for the subsequent tranches. 61. A bid opening record is kept which lists the names of the bidders or their representatives, contract amounts, bid security amounts, and duly signed by the participating bidders/ representatives and the bid evaluation committee members. This does not, however fully capture what transpired during the bid opening (e.g., the Chair of the meeting; who actually opened the bids; relevant information raised by the attendees; and any discussions and clarifications during the bid opening).

13

62. The filing system was not very organized. During the review, the IA maintained a date-wise filing system which made it difficult to retrieve required documents without asking the file owners. As the IA will be implementing more ADB-financed projects when subsequent tranches under the MFF are implemented, a filing system on a per contract basis will be much more helpful to facilitate retrieval. 63. The PPRR Team noted that the IA maintained a bid collection/registration log for bids and proposals received. However, this did not include EOIs received for consulting contracts. A late bid that was returned unopened to the bidder was not noted in the said register. Recommendations 64. The PPRR Team recommends that MOTC/COR, in coordination with CWRD identify relevant trainings to reinforce its capacity on procurement and disbursement aspects, which include seminars on ADB Procurement and Disbursement Guidelines. This is to prepare the team for the next tranches and ably cope with the huge requirements of the Projects under the Investment Program. 65. To enhance the IA’s capacity, MOTC/COR, in coordination with CWRD should also consider establishing a procedural manual to outline the detailed procedures for the different activities in the implementation of the Investment Program in general, and the different tranches in particular, indicating therein staff responsible for each activity. The manual is also a helpful tool not only for new staff, but for all the IA staff to properly carry out the required processes and perform their respective tasks in relation to ADB-funded projects. 66. In order to adequately capture what transpired during the bid opening, it is recommended that MOTC/COR improve the bid opening documentation by supplementing the bid opening record with minutes to include information on, among others the Chair of the meeting; who actually opened the bids; relevant information raised by the attendees; and any discussions and clarifications during the bid opening. 67. Proper filing and archiving of project documentation is essential in tracking the Project operations and establishing audit trail. In this regard, the IA should maintain a contract-based filing system to ease retrieval, and log completely and accurately all bid submissions, including related actions taken thereon in the bid register.

IV. CONCLUDING COMMENTS

68. The PPRR Team recognizes the challenges faced by the MOTC and COR in implementing a complex and multi-donor funded project as their first ADB project. Capacity of the procurement and finance staff of the IA should therefore be continuously strengthened to enhance project monitoring and implementation, and effectively cope with the requirements of subsequent tranches. 69. Considering the size and value of the contracts under this Project, it is imperative to exercise due diligence during bid evaluation and consistently apply the selection criteria to ensure the transparent selection of the most qualified bidder. Full documentation of the evaluation process is of equal importance to establish trail of the bid evaluation committee’s decisions. Bidding integrity and transparency should be established at project commencement and observed throughout project implementation in order to maximize development effectiveness.

14

70. While physical progress is on track, variations with significant cost implications are anticipated. Weaknesses in project design and inaccuracies in related BOQ requirements and estimates should be reviewed carefully, and improvements thereon should be identified to avert similar huge variations in future tranches, and to effectively manage the use of Project funds. 71. OAI encourages ADB CWRD, in coordination with Kazakhstan Resident Mission to conduct limited compliance reviews (spot reviews) during review missions to ensure bidding integrity and transparency in the procurement process. OAI plans to conduct seminars in 2011 to assist the Operations Departments in conducting this type of review. 72. The PPRR Team extends its thanks to MOTC and COR for their cooperation and responsiveness on the PRRR requirements; and the ADB Transport and Communications Division, CWRD; and Kazakhstan Resident Mission for their full support and assistance to the team during the course of the review. ADB values the courtesy and support that the Project officials extended to the PPRR Team.