Marshall Islands - First ICT Sector Development Policy ...€¦ · Web viewKey Factors Affecting Implementation and Outcomes. 2.1 Program Performance. The . First DPO. was. appraised

Document ofThe World Bank

Report No: ICR00003905

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IDA-H8380)

ON A

GRANT

IN THE AMOUNT OF SDR 2 MILLION

$3 MILLION EQUIVALENT

TO THE

REPUBLIC OF THE MARSHALL ISLANDS

FOR A

FIRST ICT SECTOR DEVELOPMENT POLICY OPERATION

February 21, 2017

TRANSPORT AND ICTEAST ASIA AND PACIFIC REGION

CURRENCY EQUIVALENTS

(Exchange Rate Effective August 15, 2016)

Currency Unit = $1.00 = $ 1.00$ 1.00 = $1.00

FISCAL YEAROctober 1 to September 30

ABBREVIATIONS AND ACRONYMS

$ All dollars are in United States dollars unless otherwise indicatedCPS Country Partnership Strategy CTF Compact Trust FundDPO Development Policy OperationGbps Gigabits per secondGDP Gross domestic productGNI Gross national incomeICT Information and communication technologiesMoF Ministry of FinanceMTC Ministry of Transportation and CommunicationNTA National Telecommunications AuthorityRMI Republic of the Marshall IslandsSOE State-owned enterpriseTA Technical Assistance

Senior Global Practice Director: Jose Luis Irigoyen Country Director: Michel Kerf

Sector Manager: Jane TreadwellProject Team Leader: Natasha Beschorner

ICR Team Leader: James L. Neumann

- i -

REPUBLIC OF THE MARSHALL ISLANDS

FIRST ICT SECTOR DEVELOPMENT POLICY OPERATION

CONTENTS

Data Sheet .......................................................................................................................i

1. Program Context, Development Objectives and Design

2. Key Factors Affecting Implementation and Outcomes

3. Assessment of Outcomes

4. Assessment of Risk to Development Outcome

5. Assessment of Bank and Borrower Performance

6. Lessons Learned ........................................................................................................

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

Annex 1. Bank Lending and Implementation Support/Supervision Processes

Annex 2. Beneficiary Survey Results

Annex 3. Stakeholder Workshop Report and Results

Annex 4. Borrower's ICR and/or Comments on Draft ICR

Annex 5. Comments of Cofinanciers and Other Partners/Stakeholders

Annex 6. List of Supporting Documents

MAP: IBRD 33444

A. Basic Information

Country:Republic of the Marshall Islands

Program Name:MH: First ICT Sector Development Operation

Program ID: P128013 L/C/TF Number(s): IDA-H8380ICR Date: 02/23/2017 ICR Type: Core ICR

Lending Instrument: DPL Borrower:MARSHALL ISLANDS

Original Total Commitment:

XDR 2.00M Disbursed Amount: XDR 2.00M

Revised Amount: XDR 2.00MImplementing Agencies: Ministry of Finance

- ii -

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 12/19/2011 Effectiveness: 06/03/2013 Appraisal: 07/05/2012 Restructuring(s): Approval: 03/19/2013 Midterm Review: Closing: 12/31/2013 12/31/2013C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Unsatisfactory Risk to Development Outcome: High Bank Performance: Moderately Unsatisfactory Borrower Performance: UnsatisfactoryC.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Not Applicable

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies: Not Applicable

Overall Bank Performance:

Moderately Unsatisfactory

Overall Borrower Performance: Unsatisfactory

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if any) Rating:

Potential Problem Program at any time (Yes/No):

NoQuality at Entry (QEA):

None

Problem Program at any time (Yes/No):

NoQuality of Supervision (QSA):

None

DO rating before Closing/Inactive status:D. Sector and Theme Codes

Original ActualMajor Sector/Sector Information and Communications Technologies Telecommunications 100 100

Major Theme/Theme/Sub Theme Private Sector Development Business Enabling Environment 63 63

- iii -

Regulation and Competition Policy 63 63 ICT 12 12 ICT Solutions 12 12 Urban and Rural Development Rural Development 25 25 Rural Infrastructure and service delivery 25 25E. Bank Staff

Positions At ICR At Approval Vice President: Victoria Kwakwa Axel van Trotsenburg Country Director: Michel Kerf Franz R. Drees-Gross Practice Manager/Manager:

Jane Lesley Treadwell Randeep Sudan

Program Team Leader: Natasha Beschorner Natasha Beschorner ICR Team Leader: James L. Neumann ICR Primary Author: James L. Neumann

F. Results Framework Analysis

Program Development Objectives (from Project Appraisal Document)

The objective of the Program is to increase the availability of ICT services and enable the more widespread application of ICT services supporting improvements in economic and social development in the Marshall Islands. The objective of this first Operation is to support policy development and prepare the foundation for the legal, regulatory and institutional reforms needed to support sector liberalization.

Revised Program Development Objectives (if any, as approved by original approving authority)

Not applicable.

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Increase in the level of competition in the ICT sector to two licensees (assumes increase in the number of Internet service providers)

Value (quantitative or

No competition for the supply of ICT

Two licensees (assumes

No new licenses issued.

- iv -

Qualitative) services.

increase in the number of Internet service providers)

Date achieved 03/19/2013 12/31/2013 12/31/2013Comments (incl. % achievement)

Not achieved.

Indicator 2: Percentage of population with access to ICT services on Outer Islands (without coverage under the Baseline scenario) increases.

Value (quantitative or Qualitative)

Without ICT services (Program Document, para. 4.03)

No data available.

Date achieved 03/19/2013 12/31/2013Comments (incl. % achievement)

Not measurable.

Indicator 3: Reduction in prices of core ICT services (mobile and fixed local and international calls; Internet services).


Not defined No data available.


Not measurable.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: An open and competitive ICT market


No. No.


- v -

G. Ratings of Program Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements(USD millions)

No ISRs were completed during the DPO Program.

H. Restructuring (if any)

The DPO was not restructured.

- vi -

1. PROGRAM CONTEXT, DEVELOPMENT OBJECTIVES, AND DESIGN

1.1 Context at Appraisal

Country Development Context

1. At the time of appraisal of the First ICT Sector Development Policy Operation (DPO) in 2013, one of the main challenges facing RMI was its remoteness and dispersed geography. Limited connectivity imposed high business and social costs, including the isolation of Outer Island communities and missed opportunities for economic and social development. Key reasons for the limited and costly service included the high costs of connecting remote and sparsely populated islands, the monopolistic market structure, and the constrained financial position of Marshall Islands National Telecommunications Authority (NTA), which limited access to funds for new infrastructure. ICT sector reforms implemented in similar countries, including elsewhere in the Pacific (e.g., Fiji, Samoa, Solomon Islands, and Vanuatu), demonstrated linkages between market-based reforms to attract new private sector investment and improved economic and social indicators.

Sectoral and Institutional Context

2. RMI was one of the least “connected” countries in the world. There was a single service provider, the majority state-owned NTA. Survey data from 2013 indicated that only around 35 percent of the population subscribed to ICT services. Mobile phone penetration was around 26 percent of the population. Less than 2 percent of the population subscribed to an Internet connection. Mobile broadband was not yet available. Total broadband Internet take-up was approximately 520 subscribers, or around 1 percent penetration of the population. This was despite the fact that, in 2010, Majuro and Kwajalein were connected to an 80 Gbps capacity fiber-optic cable system linking RMI to Guam via the HANTRU-1 cable system (including 10 Gbps of “lit” capacity). Submarine fiber optic cable capacity in RMI was heavily under-used, in part due to the monopolistic market environment.

3. NTA presented a significant fiscal risk for the government. The RMI section of the HANTRU-1 cable system was installed at a cost of $21.5 million. It was financed by a loan of $18.5 million from the United States Department of Agriculture’s Rural Utilities Service to NTA, unconditionally guaranteed by the Government of RMI. NTA’s financial situation was deteriorating due to the burden of repayments under the Rural Utilities Service loan. The total level of debt and loan obligations of NTA was approximately $26 million and comprised one of the largest fiscal risks to the government of all state owned enterprises (SOEs). The company was generating insufficient cash flow from its operations to meet its repayment obligations without additional financial support. The government was making annual transfers to NTA of approximately $1.5 million. An analysis of its financial position determined that NTA was unable to undertake the investments in new infrastructure that were required to improve access and services in line with demand.

4. A new legal and regulatory framework was needed to attract new investment. At the request of the government, in 2010/2011 the World Bank provided

- 1 -

technical assistance in relation to ICT sector development options and strategy. This included an assessment of existing network infrastructure, a supply and demand analysis, and an initial review of the current ICT policy and legal environment. The nation’s remoteness and small size were recognized as significant challenges to attracting foreign investment, and the government agreed that it would need to undertake policy and legislative reforms to improve the investment climate and to attract new investment into the ICT sector. ICT Sector Policy was the responsibility of the Ministry of Transport and Communications (MTC), which undertook primarily technical regulation (spectrum management and numbering). At appraisal, it was identified that the capacity of the MTC to lead an engagement on ICT sector policy was constrained and would need to be enhanced.

5. Rationale for World Bank involvement. The Program was intended to build upon the analytical and advisory technical assistance that the Bank provided in 2010. The Program recognized and built upon the extensive experience of the Bank in the field of telecommunications policy, regulatory reform and rural access issues. The appraisal identified the role that the Bank had performed in providing advice to governments in the Pacific and globally on ICT issues. The proposed Program was also designed to draw upon lessons learned from the implementation of similar reforms in the Pacific region and in other comparable countries. It was consistent with the Country Partnership Strategy (CPS) for the Marshall Islands that was presented to the Board with the First DPO, and which focused on strengthening economic governance and promoting the effective use of public resources to enhance living and service delivery standards. Support for the liberalization of the telecommunications sector, with the aim of connecting more Marshallese to social and economic opportunities, was also identified in the CPS as a key component in helping the Marshall Islands to mitigate the effects of economic isolation and to take advantage of opportunities from closer regional and global integration.

Economic Context

6. At the time of appraisal, RMI had an undiversified economic base and persistent current account deficits. With limited opportunities for exporting and domestic production, public administration and social services constituted the largest share of the economy at around 40 percent of gross domestic product (GDP). Since 2007 the current account deficit excluding grants averaged 47 percent of GDP, which was financed largely by grant inflows. Aid and fiscal transfers, primarily from the United States, supported reasonable standards of living for the majority of the population, with GNI per capita of $3,910 in 2011. On-budget grant income from various sources was equivalent to 49 percent of GNI, and official development assistance per capita was over $3,450 in 2010. The combination of economic recovery and expenditure control measures allowed the government to avoid cash-flow problems in FY2011 and to generate a fiscal surplus equivalent to about 3.7 percent of GDP. However, the FY2012 budget deficit was estimated at about 1.1 percent of GDP.

7. RMI was struggling to address key vulnerabilities in the management of SOEs. SOEs were economically significant, holding assets of $116 million as at the end-

- 2 -

FY2008 (about 76 percent of GDP). Performance of many SOEs was poor, with government subsidies to the SOE sector reaching an average of $7.2 million per annum between FY2007 and FY2010 (around 4 percent of GDP). Poorly performing SOEs represented a key fiscal risk for the government while also constraining growth through poor service delivery and high prices. The government was continuing to carry out key economic and public sector reforms, but challenges remained. A review of the status of the Public Sector Program at appraisal indicated that, while some reforms were on track (energy sector and tax reform), others were not (generation of a sustained fiscal surplus and expenditure compression).

8. The prospective end of funding from the United States Government through the Compact of Free Association (“Compact”) presented a key challenge to fiscal sustainability. A Compact Trust Fund (CTF) had been established to replace Compact grants from 2024 onward. However, expenditure and revenue trajectories, including international donor support, showed that current contributions to the CTF would be inadequate to assure a smooth transition. Public sector and structural reform measures were expected to help to significantly increase accumulation in the CTF if successfully implemented. However, for the CTF to reach adequate capitalization by 2023, additional grant support would be needed under any realistic scenario.

9. RMI was at high risk of debt distress. A joint Bank-Fund debt sustainability analysis was not available for RMI at the time of appraisal. However, an informal Bank assessment carried out using the joint framework showed RMI exceeding several policy thresholds under baseline and stress-test scenarios. External debt at appraisal was around 67 percent of GDP, mostly in the form of concessional Asian Development Bank loans and government guaranteed debts in the SOE sector. Debt servicing obligations were equivalent to 16 percent of exports and around 20 percent of revenue, although both were expected to decline. Previous arrears to the Asian Development Bank had been cleared, but the government had yet to develop a clear debt management strategy on how to meet the looming service payments.

10. At the time of appraisal, the budget support associated with the proposed Program was expected to assist RMI to move towards long-term fiscal sustainability. Short to medium-term risks to the macroeconomic framework were assessed as manageable. However, a sharper than expected rise in global commodity prices would have exacerbated inflation and led to further contractions in the RMI economy. The macroeconomic policy framework also depended on the continued availability of grants from development partners over the medium-term, especially the United States. Any change in Compact arrangements, or in RMI’s capacity to access federal grants, was expected to have a significant and negative impact on macroeconomic stability. Uncertainty regarding the United States’ plans for the Kwajalein military base presented some risk in terms of potential job losses, but was expected to be offset by new projects. RMI was considered vulnerable to a variety of natural disasters that presented an additional point of potential stress for the macroeconomic policy framework.

- 3 -

1.2 Original Program Development Objectives (PDO) and Key Indicators

11. The First DPO was part of a planned programmatic series of three development policy operations. The objective of the Program was “to increase the availability of ICT services and enable the more widespread application of ICT services supporting improvements in economic and social development in the Marshall Islands”. The completion of the series of three DPOs was expected to contribute to overall improvements in the economic and social development in RMI through ICT sector reform and development. The objective of the First DPO grant was “to support policy development and prepare the foundation for the legal, regulatory, and institutional reforms needed to support sector liberalization.”

12. The DPO Program Document’s Policy Matrix specified three end-of-program indicators. These were:

(a) Increase in the level of competition in the ICT sector to two licensees.

(b) Percentage of population with access to ICT services on Outer Islands increases (no baseline or target specified).

(c) Reduction in prices of core ICT services (mobile and fixed local and international calls; Internet services) (no baselines or targets specified).

13. The Policy Matrix also contained a list of “medium term outcomes”1 (see Section 3).

14. The sequential logic of the Program was as follows: The first operation ($3 million IDA Grant) focused on the basic policy and regulatory foundations for ICT sector reform. The second DPO ($5 million indicative IDA Grant) was to focus on the enactment of key enabling legislation, the establishment of the new sector regulator, the adoption of a plan for restructuring of NTA, and the allocation of spectrum for new mobile operators. The third DPO ($5 million indicative IDA Grant) was to focus on new licensing and market entry, the commencement of regulatory functions, and further strengthening of the ICT sector’s enabling environment, with a particular emphasis on facilitating access to ICT in Outer Islands.

1.3 Revised PDO and Key Indicators, and Reasons/Justification

15. Neither the Program Development Objectives nor the PDO of the First DPO were revised during the period of implementation. No key indicators were introduced, revised, or dropped.

1 Refer to Annex 2 of the Program Document.

- 4 -

1.4 Original Policy Areas Supported by the Program

16. The DPO contained three main policy areas: (a) introduction of a pro-competitive ICT sector policy; (b) strengthened legal and regulatory framework; and (c) restructuring and liberalizing the ICT Sector. The Program drew on analytical work undertaken in 2010 and 2011 by the World Bank at the government’s request, which examined options for ICT sector development and reform. Beginning with the introduction of a new pro-competitive policy and draft legislative framework under the First DPO, the Program was expected to build support for the development of an enabling environment and investment climate that would increase the attractiveness of the ICT sector in RMI and lead to new private sector investment in ICT services.

17. The first policy area focused on introducing a pro-competitive ICT sector policy. The new National ICT Sector Policy established the government’s commitment to introducing a competitive market, while also recognizing that the small market size and challenging business environment in RMI may mean that new entry does not materialize, and that NTA may continue as a sole provider to be monitored and supervised by an independent regulator. The Policy provided a roadmap for the transition to competition and implementation of the new market structure. The policy development phase included a consultative process with key stakeholders and reflected accepted international practices regarding telecommunications policy and regulation. The policy framework also set out the government’s expectations regarding the strengthening and restructuring of NTA in order to position it more effectively in a competitive market structure, including the need to address the high level of debt owed by NTA and to put NTA back on a sound financial footing so that it could make new investments to improve its level of service. The guiding principle was to improve access and affordability of telecommunications services, recognizing the extreme challenges in connecting remote and sparsely populated islands.

18. The second policy area focused on strengthening the legal and regulatory framework. The existing Marshall Islands National Telecommunications Authority Act 1990 (40 MIRC Ch. 1) did not (and still does not) allow for the introduction of competition and reserves certain monopoly rights over telecommunications services in favor of NTA.2 Accordingly, a new legal framework was required to give effect to the government’s new ICT sector policy and to permit new investment in the ICT sector by private sector operators. The new legal framework would also provide for an independent regulator to carry out economic and technical regulation of the sector and promote the long-term interests of users of telecommunications services in RMI. Also identified was the possibility to collaborate with the Federated States of Micronesia and the Republic of Palau to reduce costs and better leverage scarce resources, through the creation of a subregional regulatory agency.

2 The Marshall Islands National Telecommunications Authority Act (40 MIRC Ch. 1 §107) reserves to NTA “[…] the exclusive right to engage in the erection, construction, installation, maintenance, operation, and management of domestic and international telecommunications services in the Republic […]”.

- 5 -

19. The third policy area focused on restructuring and liberalizing the ICT Sector. The sustainability of the reforms depended upon restructuring and strengthening NTA to compete effectively in a liberalized market, which included providing the fiscal support needed for the government to carry out a restructuring of NTA’s debt and to finance a recapitalization. The financial position of NTA posed the most significant barrier to implementing sector reforms and needed to be resolved prior to any form of market liberalization. The Program proposed to focus on options for greater private sector involvement in NTA to strengthen its operations and also to reduce or eliminate the government’s direct fiscal exposure to NTA. This focus was to transition to new institutional arrangements that would expand significantly the macroeconomic benefits of developing the ICT sector as a whole. The final step of the Program would have been satisfied when the regulator offered the right to another operator(s) to supply telecommunications services in RMI pursuant to arrangements that ensured a level playing field for all market participants.

1.5 Revised Policy Areas (if applicable)

20. The policy areas were unchanged.

1.6 Other significant changes

21. There were no other significant changes.

2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES

2.1 Program Performance

22. The First DPO was appraised in July 2012 and approved by the Bank’s Board of Executive Directors on March 19, 2013. All prior actions listed in Table 1 for the First DPO were completed by the time of Board approval, and the operation was fully disbursed before the closing date. On this basis, the First DPO was effective in supporting the government’s efforts to develop a new ICT sector policy to promote private sector investment and improve access to ICT in RMI, as well as to support the preparation of a draft legislative framework, which was the first step in the law reform process needed to give effect to the new policy.

- 6 -

Table 1. Completed Prior Actions for DPO 1

Policy Areas and Expected Prior Actions Completed ActionsPro-Competitive ICT Sector PolicyThe Recipient has endorsed a new ICT sector policy and committed to liberalize the ICT sector, introducing a modern legal and regulatory framework in the Recipient’s territory and restructuring NTA.

Government adoption of the ICT Sector Development Document dated April 23, 2012, through a Cabinet Resolution endorsing said ICT Sector Development Document embodied in the Cabinet Minutes No.072 (2012), dated May 5, 2012.

Ministry of Finance press release dated July 13, 2012, as published in the Recipient’s mass media (including the Marshall Islands Journal).

Strengthened Legal and Regulatory FrameworkThe Recipient has, though the Office of its Attorney General, prepared, satisfactory to IDA, a draft Communication Bill backstopping the new ICT policy framework adopted by the Recipient.

Copy of the Cabinet Meeting 184, Reference – CP 6696 (2012), dated December 31, 2012, furnished to the Association by the Recipient’s Ministry of Finance, approving and endorsing the Communications Bill 2012, and authorizing the Minister of Finance and Minister of Transportation & Communications to introduce jointly the Communications Bill 2012 as a Government bill to the Nitijela (Parliament) during the January 2013 session.

23. Approximately 10 months lapsed between appraisal and the Board approval of the First DPO. A Board date was originally scheduled for July 2012, but was postponed until March 2013 to accommodate delays by the government in preparing the new legal framework document. This delay and length of time needed to complete the prior actions for the First DPO prior to Board approval was an early indication of the difficulties that the government was to experience in executing the reforms from the outset of the Program. The government continued to confirm its commitment to the Program and to reassure the Bank that it could complete all agreed actions, notwithstanding opposition to ICT sector reforms from key stakeholders within the government and from NTA. However, implementation actions on the ground were slow, and the government repeatedly requested additional time to overcome severe capacity constraints that impeded the commencement of the analytical work required to support the Program.

24. A companion technical assistance project was approved but not implemented. In July 2013, shortly after the disbursement of the First DPO grant, the Bank approved an ICT Sector Technical Assistance Project (TA Project, P132119) of $1.25 million (financed by a grant from the Pacific Region Infrastructure Fund) aimed at supporting the government in implementing the Program. The government attempted unsuccessfully to complete the procurement processes for the selection of advisors, financed under the TA Project, on two separate occasions in 2013 and 2014. In 2013 there was an extended delay in completing the request for proposal stage that led to the procurement process becoming stale. It was restarted in August 2014, but formally abandoned again in March 2016 until the government and the Bank could determine

- 7 -

whether to restructure and extend the TA Project. It was eventually restructured in August 2016, at the government’s request, to extend the closing date to August 2018, and to narrow the focus on options for introducing private sector investment, with a view to easing NTA’s financial dependence on the government, reduce the public finance risk, and enhance competitiveness. The inability of the government to complete one international recruitment during three years of implementation of the TA Project, from August 2013 until August 2016, despite extensive technical support from the Bank, including extensive assistance of a procurement specialist deployed by the Bank during 2015 to help build capacity and provide in-country training on procurement procedures and processes, highlights the deep capacity limitations encountered during the period of this Program.

25. The government continued fitfully with the law reform process throughout 2014 and 2015. The government initially indicated that the Communications Bill (No. 44), dealing with sector liberalization and regulatory reform, would be introduced into the Nitijela and would receive its first reading in March 2014. However, the Nitijela adjourned on March 31, 2014 without the Bill having been tabled. An indicative Board date of May 29, 2014, for the Second DPO was therefore dropped due to the lack of progress in satisfying this key prior action. The Bill was subsequently introduced and sent to the Select Committee for consultation and review. It was expected that the Bill would be reported back to the Nitijela during the August/September 2014 session. However, this step was deferred to allow more time for the Committee to complete its work. The Bill was reported back from the Select Committee and received its Second Reading in February 2015. In March 2015, during an interim Bank supervision mission, the government again confirmed its intention to expedite the recruitment of international expert advisers to provide essential support for the law making process, including consultation and engagement with stakeholders and NTA as part of the Select Committee processes. These advisors were never retained and, while the Bill was debated during the August 2015 session, it never completed its Third Reading.

26. The caretaker period leading up to and immediately following the national elections in November 2015 also delayed Program implementation between September 2015 and February 2016. No supervision missions were undertaken during this period, and work by the government on the Program was suspended. A new government was formed in early 2016, and the Bank sought immediate guidance regarding the allocation of IDA 17 resources for the DPO Program and its commitment to using the TA Project. In its engagement with the government, the Bank recommended that the DPO be dropped from delivery under IDA 17 and that these funds be reprogrammed. The Bank also emphasized that the government could continue to use the TA Project to support work on telecommunications sector reform, but that strong commitment would need to be demonstrated before the Bank would consider an extension of the TA Project’s closing date of August 31, 2016.

27. In May 2016, the new government reconfirmed its commitment to ICT sector reform but requested a deferral of the Second ICT Sector DPO, at which point the Bank dropped the operation, terminating the Program. The new government communicated to the Bank its intention to examine options for addressing the fiscal risk

- 8 -

posed by its guarantee of NTA debt. It also indicated its willingness to examine new market arrangements to promote private sector led investments in new infrastructure and improved services. It requested the postponement of the DPO Program until the work under the TA Project was further advanced and indicated that it would review IDA 18 programming in light of the strategic options and recommendations for ICT sector reform and development that would be identified under the TA Project, including funding requirements needed to implement any reforms.

2.2 Major Factors Affecting Implementation:

28. Five major factors affected implementation: (a) a lack of borrower readiness for a development policy operation in the ICT sector; (b) sector readiness for the introduction of competition was overestimated, and ICT sector reforms proved highly controversial; (c) the failure to implement the TA Project left the government without the expert advice needed to articulate the objectives of reform and support implementation, especially to develop specific recommendations and options for dealing with the debt and loan obligations of NTA; (d) program coordination was weak, and the government failed to collaborate and share information effectively among responsible ministries and agencies; and (e) the Bank failed to understand the political economy and internal political dynamics affecting the government commitment to ICT sector reforms.

29. There was a lack of borrower readiness for a development policy operation in the ICT sector. As mentioned earlier, RMI maintains a structural fiscal deficit, which is financed by external aid. Weak oversight of development assistance, combined with weak budget monitoring and execution procedures, created incentives and opportunities for ad hoc budget management and expenditure practices. These factors presented a substantial risk to good policy planning processes and fostered reactive decision-making driven by the need to fill a fiscal deficit. The DPO instrument and the prospect of fungible grant financing focused the government's attention on the benefits of the budget support and weakened the focus on the Program objectives. In other words, the government was too quick to agree on the DPO Program in order to assure itself of a flow of budget support funds, without taking into account equivocal stakeholder support and weak capacity to deliver on the obligations of the Program. The government was also under significant pressure to use its funding allocation under IDA 16, which further incentivized the government to agree to the Program quickly, notwithstanding that the fundamental political support was insufficient and the underlying sector analyses had not been completed. The government had committed to programming only the DPO Program during IDA16 and elected not to reallocate during the IDA period. To the extent that the initial efforts by the government to meet the prior actions for DPO 1 were done for budget reasons, this may have contributed to a misleading impression regarding the underlying difficulties with the reform program. Overall, it suggests a less ambitious program would have been more appropriate in the given circumstances, but both the Bank and the borrower were highly driven by IDA commitment factors.

30. Sector readiness for the reform was overestimated. The introduction of competition proved controversial, and the level of opposition from MTC and NTA was not anticipated. The groundwork for the reforms depended upon the availability of robust

- 9 -

technical, economic, financial, and market analyses that were not completed by the time the First DPO went to the Bank’s Board for approval. While the government and the public broadly supported the Program’s objectives of improving quality and increasing access to ICT services, there was little experience in undertaking liberalization or privatization processes. No local expertise was available, and the country was and remains wholly dependent on international technical assistance to execute these reforms. NTA recognized that it would need significant new capital in order to ready itself for the impact of competition. It advised the government that its support for the reforms was conditional upon it receiving the $13 million of financing under the Program. When such commitments were not forthcoming, NTA moved quickly to mobilize opposition to the reforms. While agreeable to market-based reforms in general terms, MTC opposed any changes to the status quo without the consent of NTA.

31. The failure to implement the TA Project left the government without the expert advice needed to articulate the objectives of reform and support implementation. The TA Project aimed to provide support in the following areas: (a) policy, legal, and regulatory functions, including capacity building for MTC and establishment of an independent regulator; (b) advising the MoF on NTA restructuring; and (c) capacity building for NTA to prepare for the transition to a competitive market. The availability of this technical assistance was identified during the appraisal of the DPO as one of the key risk mitigation measures. The possibility that the TA Project would not be implemented successfully was not expressly contemplated, nor was any provision made to link performance under the TA Project with the prior actions required under the First DPO. Despite the Bank’s sustained guidance, the selection of consultants to provide key advisory services was never completed, and no technical assistance activities were implemented. The failure to mobilize technical support under the TA Project reinforced impressions among sector stakeholders that insufficient attention was being paid to the underlying analysis needed to support the reform process.

32. Program coordination was weak, and the government failed to collaborate and share information effectively among responsible ministries and agencies. MoF was responsible for leading the implementation of the policy actions and coordinating with relevant government agencies and stakeholders, most importantly MTC and NTA. A working group comprised of representatives of MoF, MTC, and NTA was established and tasked with the responsibility to lead the procurement of consultants. However, this group met infrequently and was ineffective. MoF was unable to manage key aspects of the political process, and the main elements of the Program were not well integrated. MoF and MTC, for the most part, did not communicate, share information, or collaborate to advance the Program in a cohesive and coordinated manner. The support of NTA’s management and shareholders to implement a key component of the reforms (NTA’s restructuring and liberalization of the market) also was uneven and was not sustained throughout implementation. Without specialist advice and support from MTC or NTA, MoF did not have the necessary expertise or mandate to lead on the ICT sector-specific aspects of the Program, and lacked credibility to champion the reform process. This created political tensions within the government due to a misalignment of incentives and responsibilities, and inadequate information sharing. These factors ultimately combined to paralyze political decision making by the government.

- 10 -

33. The Bank failed to understand the political economy and internal political dynamics affecting the government’s commitment to ICT sector reforms. Following the disbursement of the First DPO, the Bank and the government conducted an extended dialogue about whether to proceed with the Program, including options to use the IDA 16 allocation for other purposes. The government repeatedly stated its commitment to the Program and provided assurances regarding its capacity to implement it. The Bank took these assurances at face value while lacking a necessary deeper understanding of the factors driving opposition to reforms by NTA and MTC and of the effects of the lack of common understanding by key stakeholders of the purposes and implications of the reform program. Based on the government’s ability to carry out (albeit with delays) the First DPO’s prior actions—which were, in retrospect, unchallenging and more aspirational than substantive—the Bank continued to overestimate the government’s commitment and ability to follow through on the more difficult sector reforms planned under the second and third operations. The Bank’s knowledge gaps about local interests and dynamics were exacerbated by a lack of any full-time in-country presence.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Use

34. The Program’s monitoring and evaluation had significant shortcomings of design, implementation, and use. The three end-of-program key indicators (see Section 1.2) were adequate to measure the first key Program outcome of increased availability of ICT services, but only one of these indicators (increased number of licensees) had baselines and target values. There were no indicators for the second PDO outcome of enabling the more widespread application of ICT services. The provisions for monitoring and evaluation in the Program Document were vague, referring only in very general terms to monitoring by MoF and the Bank. MoF provided no formal reports on implementation progress or results. The Bank did not file Implementation Status and Results Reports during the life of the Program. No restructuring was carried out to address the gaps in the M&E framework. There are no indications that an M&E system for the Program was implemented or used.

2.4 Expected Next Phase/Follow-up Operation (if any)

35. The ICT TA Project has been extended until August 2018 and restructured to support a sector analysis and readiness assessment of NTA. This next phase of work will provide support aimed at strengthening the financial performance of NTA, promoting its financial sustainability, including options for reducing its substantial debt burden, and reducing the call on government funds for activities that may be financed commercially. The TA Project will help assess the financing required to recapitalize the sector and place it on a strong footing to deliver access to services demanded by users. It will assist the government to explore the possibility of introducing private sector investment and participation in the ICT sector. The approach to these issues also will no longer be on an atomized, single-sector basis. The government has determined to link ICT sector development with its wider program of reforms for SOEs and public financial management reform, which appears to have broad support. Finally, the TA Project was restructured to include more dedicated support ($0.2 million) for implementation and

- 11 -

management activities, including on the application of World Bank Group policies, procedures, reporting requirements, contracting, procurement, and coordination.

36. The operating environment for implementing sector reforms in RMI remains challenging. However, a number of significant changes have taken place recently. A dedicated unit has been established with MoF to coordinate and support the implementation of World Bank financed projects, along with projects funded by other development partners. Initial experiences with this new unit have been promising, and fiduciary capacity has increased. In addition, MoF is also completing a comprehensive review of its aid coordination processes and procedures and is expected to implement new aid coordination mechanisms at the national level to improve implementation effectiveness, transparency, and accountability. At the project level, MoF has convened a steering committee to involve key stakeholders (including MTC and NTA) to guide implementation. An “independent” expert advisor will also be retained directly by the steering committee to help the government manage and act on the recommendations arising from the sector analysis and readiness assessment of NTA. The formation and engagement of this working group from the beginning of implementation activities is a key lesson learned.

37. Following the completion of the TA Project, a new ICT sector project may be developed. Any new project for ICT sector development could support the implementation of the strategy and recommendations adopted by the government from the NTA readiness assessment. This may include introducing new market rules, providing for private sector investment, and implementing associated legal and regulatory reforms including, e.g., the creation of a new independent sector regulator. A key focus for any funding for the sector may be to improve the financial viability and sustainability of NTA, especially developing options to reduce its debt burden to sustainable levels, reducing the government’s exposure to NTA debts and operational costs, and support for public sector investments in areas that cannot attract and sustain infrastructure investment on purely commercial terms, e.g., on the Outer Islands. The government will review the programming of IDA in line with the strategic options and recommendations identified under the TA Project.

3. ASSESSMENT OF OUTCOMES

3.1 Relevance of Objectives, Design and Implementation

Rating: Modest

Relevance of objectives: Substantive

38. The DPO’s objectives of increasing the availability of ICT services and enabling the more widespread application of ICT services supporting improvements in economic and social development in RMI remain highly relevant. The objectives were closely aligned with the government’s priorities under the Vision 2018 National Development Plan, and continue to reflect the urgent needs to address the fiscal risks

- 12 -

posed by NTA and establish a more efficient ICT sector. In a letter to the World Bank dated May 2, 2016, the new government confirmed its commitment to sector reform and development consistent with the objectives of the DPO Program and the companion TA Project. The Program’s objectives remain fully aligned with the Bank’s CPS for the Marshall Islands FY13-16,3 which continues to be applicable. The new CPS is scheduled for Board discussion on February 28, 2017.

Relevance of Design and Implementation: Modest

39. Program design was relevant to its objectives. The design of the Program followed established good practice principles regarding the structuring, sequencing, and implementation of ICT sector reform. All prior actions and triggers were consistent with the development objectives. The Program followed a logical sequence, beginning with the basic policy and regulatory foundations for ICT sector reform, the enactment of key enabling legislation, the establishment of the new sector regulator, the adoption of a plan for restructuring of NTA, the issuing of new licenses and market entry, the commencement of regulatory functions, and the further strengthening of the ICT sector’s enabling environment, with a particular emphasis on facilitating access in the Outer Islands.

40. The debt obligation of NTA. The identification of the need to strengthen and reposition NTA financially, especially its debt obligations to RUS, prior to the liberalization of the market and the introduction of competition, also followed good practice. Specific measures to prepare NTA for the introduction of competition were also important to build a consensus and manage the risk of opposition to the reforms. However, while the design may have been sensible in theory, for a new engagement in a fragile environment, the timeframe to deliver the policy and institutional reforms was overly ambitious. The pace of reform under the Program caused anxiety amongst NTA and other stakeholders, which ultimately slowed progress as the government’s attention was diverted towards opposition to the reforms. A program with a less ambitious series of deliverables may have been more appropriate and may also have helped to avoid suggestions that the sector restructuring and liberalization program had been predetermined or stipulated by the Bank.

41. Implementation was inadequate. The pairing of the Program with the TA Project was an astute and proactive design feature that was well attuned to the capacity issues in RMI, which were identified and anticipated during appraisal. Unfortunately, implementation was not well managed. Information flows were poorly coordinated and the stakeholder engagement process was not managed satisfactorily within the government. With the benefit of hindsight, the Program’s start might have been postponed until the TA Project could be implemented and produce results. Such a deferral, however, while ideal in principle, probably would not have been feasible, given the government's urgent needs for budget support and the pressure to commit IDA funds

3 International Development Association and International Finance Corporation, Country Partnership Strategy for the Republic of the Marshall Islands for the Period FY13-FY16, February 19, 2013. “Increased economic competitiveness and enhanced regional and global integration through ICT reform and market liberalization” was one of the four medium-term goals of this CPS.

- 13 -

in FY13 that was the last year of IDA16. The Bank might have done more in the first year of the DPO Program to support the government to get the consultant advisers mobilized and have them produce an initial scheme for restructuring NTA. The First DPO might also have included a prior action specifying Cabinet consideration and endorsement of a report on market restructuring options for NTA, for example, which would have helped to ensure the DPO Program was implemented with the benefit of expert technical assistance. The up-front presence of recipient executed technical advisors to provide independent guidance on policy and legal reforms would also have helped to counter suggestions that the Bank was forcing the reforms.

3.2 Achievement of Program Development Objectives

Rating: Negligible

42. The First DPO met its objective to support policy development and prepare the foundation for the legal, regulatory, and institutional reforms needed to support sector liberalization. Achievement is measured against two indicators: (a) a pro-competitive ICT sector policy; and (b) a strengthened legal and regulatory framework. As to indicator one, the baseline value at appraisal identified no policy outlining the government’s vision for sector development. Satisfying prior action one, the government formally endorsed a new ICT sector policy through a Cabinet Resolution and its publishing in the official government journal, which committed it to liberalizing the ICT sector and to restructuring NTA. As to indicator two, the baseline value at appraisal identified an inadequate ICT sector legal and regulatory framework that prohibited competition and provided inadequate mechanisms for oversight of service outcomes. Satisfying prior action two, the government prepared a draft Communication Bill consistent with the new ICT policy framework, and the Ministers of Finance and of Transportation & Communications authorized the joint introduction of the Communications Bill 2012 to the Nitijela (Parliament).

43. The DPO Program did not achieve its stated objectives to increase the availability of ICT services and enable the more widespread application of ICT services. The government did not implement any of the subsequent, more challenging, and substantive policy and institutional actions called for in the Program’s Policy Matrix. The Communications Bill was not enacted. Consequential to the completion of the law reform process, an independent ICT sector regulator was not established. Spectrum allocations for new wireless mobile services were not carried out, and no new operators were licensed. A restructuring plan for NTA was not prepared or approved. The burden and fiscal risk posed by NTA to the government remains unaddressed. No steps were carried out regarding open access to international connectivity. The limited development of the ICT sector due to the failure to implement the reforms is also illustrated by the lack of improvement in sector indicators. Mobile penetration in 2015 was 32 percent of the population, which represents only 3 percent annual growth since 2013, which is

- 14 -

significantly lower than the reported annual growth rate of 12.5 percent for the Pacific region between 2009 and 2014. Mobile broadband is also yet to be introduced.4

44. The Program’s medium-term outcomes, as expressed in the Policy Matrix of the Program Document, remain unrealized: (a) an open and competitive ICT market; (b) a universal access program for supporting the services that are not viable on purely commercial terms, including on the Outer Islands; (c) an increase in access to ICT services offered on Outer Islands; (d) reduction in prices of core ICT services (local, international calls, and Internet services); (e) government subsidies towards NTA’s operating costs reduced or eliminated; (f) increased availability of value-added services such as mobile phone-enabled banking (e.g., for remittance transfers), e-commerce, and online government services; and (g) improved communications facilities for schools, clinics, and government offices.

45. The key end-of-program indicator targets were not achieved. No new ICT sector licenses were issued. There was no increase in access to ICT services in the Outer Islands. Prices of core ICT services (mobile and fixed local and international calls; Internet services) did not decrease. The prospect of imminent competition, where none existed before, may nevertheless have triggered some behavioral changes in NTA and led to more visible marketing, the introduction of a limited range of new services in Majuro, and lower prices.

3.3 Justification of Overall Outcome Rating

Rating: Unsatisfactory

46. For all the reasons outlined above, it is unambiguous that the Development Objectives of the programmatic series of three DPOs were not achieved. While the First DPO was successfully completed and the policy gains achieved under the First DPO have proved resilient, the subsequent operations were not triggered and consequently the Program’s key milestones were not achieved relating to the enactment of new legislation introducing competition, the establishment of an independent ICT regulator, government approval of a restructuring plan for NTA, and offering of new license(s) for a second operator.

3.4 Overarching Themes, Other Outcomes and Impacts(a) Poverty Impacts, Gender Aspects, and Social Development

47. None under the Program. If implemented, the Program would likely have had a positive impact on the fiscal sustainability of the government, the financial performance of NTA, and the availability and affordability of ICT services throughout RMI, with corresponding benefits for lower-income people and women, who are major users of ICT services, especially on Outer Islands. The Bank team, during a supervision mission in March 2014, consulted with the Ministry of Internal Affairs and Women United Together

4 The GSMA, The Mobile Economy: Pacific Islands 2015, accessed on February 15, 2017: https://www.gsmaintelligence.com/research/?file=23485245295f02524925b2bd3aeec6de&download

- 15 -

Marshall Islands (a nongovernmental organization) on gender issues and the potential for ICT to empower women. Specific issues identified included high rates of domestic violence, teen pregnancy, limited access to education, and a lack of professional and business opportunities for women. Increased access to mobile phone and Internet may help raise awareness of these issues among women and men especially in the more remote islands, and help to create income generating opportunities for economic empowerment (e.g., e-commerce for traditional handicrafts). These social and economic develop gains are readily apparent in countries that have liberalized markets and facilitated new entry and investment over the last decade, including in the Pacific region, which have seen dramatic increases in access particularly to basic telecommunications and data services, and falling prices (see the World Bank’s ICT for Development Report [2010]). However, the Program was not implemented and these outcomes were not achieved.

(b) Institutional Change/Strengthening

48. None under the Program. The post-Program reengagement under the restructured TA Project has contributed to improvements within MoF including capacity building focused on fiduciary responsibilities. A moderately satisfactory relationship between key institutions has also developed following a reengagement between MoF, MTC, and NTA as part of the process to restart the sector diagnostic and analytical work. A working group among these three key stakeholders has been reestablished and reinvigorated under the guidance of the Division of International Development Assistance within MoF. Specialist project management and fiduciary capacity has also been prioritized to strengthen institutional capacity. Longer-term capacity building will depend on actions that: (a) promote trust and confidence among key institutions, ensuring transparent consultations process and effective information management; and (b) minimize the risks of expedient decision making, especially in the context of budget support operations, and emphasize the underlying objectives of substantive and sustainable sector reform and development.

(c) Other Unintended Outcomes and Impacts (positive or negative, if any)

49. The failure to complete the ICT sector reform program has contributed to number of unintended positive outcomes. First, the government has established a unit within the MoF to manage international development and aid programs. This has improved coordination within the government and improved information flows. Project management has also been prioritized, especially procurement and financial management. Second, policy engagement and coordination between ministries, stakeholders, and political decision makers has been reformed and enhanced. Procedures have been implemented to govern public/private consultations, especially formal record keeping, which have reduced the risks of miscommunication, improved accountability, and strengthened the robustness of decision making. Third, the relationship between the government and the Bank has developed and strengthened. The Bank has remained engaged in supporting the government to pursue reforms at its own pace and with access to advisors who will assist in identifying and analyzing strategic reform options that are

- 16 -

responsive to the government’s needs. The Bank’s continued constructive engagement has helped to maintain its position as a trusted development partner.

3.5 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

50. Not applicable.

4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Rating: High

51. The risks that the development outcomes will not be maintained or realized are rated as High, recognizing that the Program’s policy and institutional actions were not implemented and outcomes were not achieved. In terms of actions implemented under the First DPO, the government’s commitment to introducing competition has proved resilient. In May 2016, notwithstanding a change in the government and the failure of the law reform to progress in the Nitijela, the government confirmed its commitment to the National ICT Sector Policy and its willingness to examine new market arrangements in order to promote private sector led investments in new infrastructure and improved services. This is a positive reflection on the significant investments in knowledge transfer and capacity development activities undertaken by the Bank as part of the engagement under the Program, which emphasized the importance of introducing modern regulatory arrangements to promote new investments in ICT services and reduce ad hoc fiscal risks to the government. The longevity and robustness of the policy gains that were supported and achieved under the First DPO also highlight the appropriateness of the Program design, which was highly relevant to the national need for urgent improvements in the performance of the ICT sector.

52. The sustainability of the policy gains under the First DPO are crucial to future engagement in ICT sector reform and development in RMI. Without the government’s ongoing commitment to the objectives under the Program, specifically to examine the fiscal risk posed to the government by NTA debt and the potential to introduce legal reforms to promote private sector led investments in new infrastructure and improved services, there would have been no basis to continue with the implementation of the ICT TA Project. The recommitment by the government (in May 2016) to the policy gains made under the First DPO provided an important part of the rationale for the decision to extend the TA Project until August 15, 2018, and to restructure its focus to strengthen the financial performance of NTA, promote its financial sustainability, and reduce the call on government funds for activities that may be financed commercially. The government has not given up on the prospect of ICT sector reform and still intends to explore the possibility of introducing private sector investment and participation in the ICT sector. The continuation of the TA Project also demonstrates that the Bank remains the government’s trusted development partner, and provides an opportunity to develop the government’s project implementation capacity, building on the institutional strengthening already undertaken at the government’s own initiative within the MoF.

- 17 -

5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE

5.1 Bank Performance(a) Bank Performance in Ensuring Quality at Entry

Rating: Moderately Unsatisfactory

53. The design of the programmatic series of three DPOs was appropriate and reflected strong alignment with broader priorities of RMI and the Bank’s country strategy. The supported policy actions struck an appropriate balance between short-term actions and longer-term reforms. The Program evidenced a clear intent to maintain focus on structural reforms that would improve the investment climate and increase fiscal and financial sustainability. It also provided a mechanism for improving outcomes for the ICT sector, which needed additional capital to reduce its debt load to sustainable and commercially viable levels, while supporting policy dialogue and specific key policy reforms. Coordination was undertaken with other development partners, especially the Asian Development Bank, to ensure a cohesive approach.

54. The Bank’s assessment and mitigation of the risks to the Program were deficient. The two key risk factors that caused the Program to fail—weak capacity of the public sector and opposition from the incumbent operator to sector reforms—were identified at appraisal as posing a high risk to the successful implementation of the Program. In engaging in ICT sector reform in RMI, the Bank faced an extremely challenging situation and took a calculated risk in supporting the preparation and implementation of the Program at the urging of the Borrower. Ultimately, the magnitude of these risks were misjudged and the Program’s development outcomes were not achieved. The Bank did not have sufficient understanding of the political and governmental context, which emphasizes shared decision-making. An in-country presence by the Bank would have enabled it to better comprehend the relevant governmental policy making processes, political economy dynamics, interests supporting the incumbent monopoly (including private shareholders, management and workers), and limitations of capacity. Understanding of these factors would have allowed the Bank to design the Program in a more realistic way or perhaps to have not undertaken a DPO operation at the time. Better knowledge of the local situation might also have enabled the Bank to predict, ex ante, that the use of the DPO instrument—with its conditionality subject to policy and institutional actions—would be cited by opponents to claim that the Bank was forcing the reforms. The lack of any previous lending relationship with the Bank, together with knowledge of the prior Bank-executed sector diagnostic work in 2010 and 2011, carried out at the government’s request (under TA-P128013-TF012483), accentuated these difficulties.

(b) Quality of Supervision


55. The Bank maintained a sustained engagement through supervision missions, but these efforts were ineffective to overcome the lack of implementation capacity and Borrower readiness. The team carried out four formal supervision missions and an

- 18 -

undocumented number of technical support visits between the approval of the Program and the decision to drop it in May 2016. These helped keep the need for reforms on the government’s agenda and reduced the risk that even the modest policy gains under the First DPO would be unwound. Interviews carried out as part of the ICR process also highlighted the extensive and sustained efforts undertaken by the Bank to build capacity, transfer knowledge and ensure that all relevant stakeholders were consulted and accurate information was disseminated. Due to some apparent gaps in record keeping, these significant additional efforts put in by the team during implementation may not be fully reflected. No ISRs were completed.

56. The Bank maintained a regular in-country engagement, and sustained the intensive, ongoing and regular dialogue that is needed for DPO. Consistent with good practice, the Bank engaged widely with the government and stakeholders to build a complete understanding of perspectives and gather all relevant information, including relevant Ministers (Finance and MTC), Ministries (Office of the Chief Secretary, MoF, Attorney General, and MTC) and senior management to NTA. Briefings were also delivered directly to the Cabinet, which helped to overcome information asymmetries and improve transparency. The rationale for the Program and the Bank’s engagement supporting ICT sector development in RMI was explained thoroughly and thoughtfully. At the request of the Chairman of the Nitijela Committee on Resources and Development, the Bank team met with the Chairman and several members of the Committee during the March 2014 supervision mission to address issues around ICT sector reform, the draft Communications Bill and the rationale for the Bank’s engagement.

57. The Bank proactively identified and attempted to resolve risks to the achievement of the Program's development outcomes. The possibility of dividing the proposed Third DPO into two operations was identified proactively by the government during the Bank’s mission of June 2014. The Bank moved quickly to consider possible restructuring options that would move the opening of the ICT market into a standalone Fourth DPO. This was a creative option to signal to stakeholders that work on NTA strengthening and restructuring, including to restructure NTA’s debt to put it in a strong and sustainable financial position, would precede any moves to introduce competition. However, these changes to the Program were not implemented due to the government’s lack of readiness. The Bank also maintained a sustained engagement with NTA, based on international best practices, to share information and explain the objectives of the DPO Program and the support available under the TA Project. The sequencing of reforms, including to strengthen NTA to ensure that it was well placed to succeed in a liberalized market, was also explained. In all engagements, emphasis was placed on the importance of the government and stakeholders collaborating closely to implement the reforms.

58. The Bank continually and regularly highlighted the lack of progress on the TA Project, and provided detailed procurement guidance and support. The Bank also carried out a review of the TA Project’s financial management arrangements, which had been performing unsatisfactorily due to very poor contract management and a lack of oversight of the project financial management requirements by MoF. The Bank’s

- 19 -

supervision mission in September 2014 included not only the core team, but also the country officer, procurement specialist, and financial management specialist and focused on the unsatisfactory status of implementation of the TA Project; particular focus was placed on procurement actions, reporting, and financial management issues. Again to better align progress under the TA Project and the DPO Program, the possibility of moving activities relating to the strengthening of NTA into a new standalone DPO 3 was discussed further, although ultimately no actions were taken to remedy the shortcomings of Program design. The mission in March 2015, again, attempted to resolve longstanding procurement difficulties recruiting advisors under the TA Project. The Bank highlighted proactively its concerns that slow progress under the Program would risk RMI failing to use its entire allocation of IDA 17 funds, as was the case under IDA 16.

(c) Justification of Rating for Overall Bank Performance


59. Considering all the factors mentioned above, overall Bank performance is deemed to have been Moderately Unsatisfactory, reflecting in particular: (a) the deficiencies in risk assessment and mitigation at entry, which caused the Bank to approve a DPO Program that was unrealistic for the client; and (b) the Bank’s failure to restructure the Program in the face of three years of nonperformance.

5.2 Borrower Performance

(a) Government Performance


60. At the outset of the Program, albeit with some delay, the government prepared and approved a new national policy and submitted a draft Communications Law. The National ICT Sector Policy reflected good practice and was acceptable to the Bank. The policy commitments have proved resilient, and the new government that took office in 2016 confirmed its agreement with the policy, demonstrating the longevity of the initial steps supported under the First DPO, consistent with the observed national need for urgent improvements in the performance of the ICT sector identified under this Program.

61. In implementing the Program, the government demonstrated major shortcomings in policy coordination, stakeholder support, and implementation capacity. As discussed earlier, the government never completed the underlying analytical studies that were needed to provide a foundation for the government’s decision-making for ICT sector reform and to support implementation of the Program, including work on restructuring NTA’s debt that posed the most significant barrier to implementing sector reforms and needed to be resolved prior to any form of market liberalization. MoF did not engage effectively with the key sector stakeholders, especially MTC and NTA. Information flows were poorly coordinated, and stakeholder concerns regarding the reform process were not dealt with satisfactorily, resulting in substantial confusion, misunderstanding and misinformation at all levels of the

- 20 -

government. These contributed to a perception by leading actors of an overriding lack of transparency and accountability in the management of reform processes. Without adequate recordkeeping, the government was unable to demonstrate during the law reform processes that it had consulted effectively and addressed all stakeholder concerns. This failing left the government open to criticisms, whether or not justified, around the process and rationale for reforms. It also contributed to a lack of trust among key stakeholders and undermined support for the Program at key moments during implementation, especially the opposition to the reforms that was mounted by NTA during the Nitijela’s consideration of the Communications Bill in 2015. Without proper and timely mitigation of these risks and the effective management of concerns raised by stakeholders, the policy and institutional actions that comprised the triggers for the Second DPO could not be implemented, and the expected project objectives could not be achieved.

(b) Implementing Agency or Agencies Performance


62. Implementing agencies did not carry out the reforms supported by the Second DPO, leading to the cancellation of the remainder of the Program. MTC, NTA and MoF (which led the overall engagement on the DPO) did not interact in an effective or coordinated way. There was limited or no regular attendance at the working group level to collaborate on the broader policy aspects of the reforms, share information, identify a common perspective, plan for implementing the program, or make progress on the selection of specialist advisors under the TA Project. The principal reason for this weak performance was a lack of trust in the overall reform agenda and a lack of understanding regarding the rationale for the reforms proposed. The implementation of the TA Project was designed to address these issues and to mitigate these risks, but the specialists were never retained and the strategic sector policy advice was never delivered. The implementing agencies did not work together to give effect to the government’s policy for ICT sector reform and to implement the Program.

(c) Justification of Rating for Overall Borrower Performance


63. Considering all factors mentioned above, overall borrower performance is deemed to have been Unsatisfactory, reflecting unsatisfactory government performance, and unsatisfactory performance by the implementing agencies.

6. LESSONS LEARNED

64. The delays and political opposition to reforms in the ICT sector are a reminder of the difficulties of sector reforms and the challenges of opening monopolies to competition. A thorough political economy assessment is an essential tool to support ambitious reform programs. The interests of incumbent monopolies, especially private shareholders, management and workers, capacity issues, and

- 21 -

government readiness all play a role. The robustness of the government’s consultation and decision making processes also need to be evaluated carefully, especially to avoid situations where a lack of information, or misinformation, may be used by opponents to undermine the credibility or desirability of reforms. Political economy issues or concerns may be aggravated if the government’s decision-making processes are weak and consultation processes are ineffective, especially in countries where shared decision-making is favored. The Bank needs to undertake realistic assessments of the political economy situation and ensure that operations include actions to resolve likely areas of opposition or resistance.

65. Proper risk identification and mitigation, including to guide the choice of assistance instruments (IPF, DPO, nonlending TA, etc.), is essential. A DPO depends upon a deep and sustained level of engagement and trust between the Bank, the government and among stakeholder interested or affected by the proposed activity. With this in mind, it is doubtful that RMI was ready for this type of operation as its first substantial lending activity with the Bank, especially in view of the government’s lack of familiarity with World Bank Group processes and procedures. The two key risk factors that caused the Program to fail—weak capacity of the public sector and opposition from the incumbent operator to sector reforms—also point against the use of a DPO instrument, with its basis of policy and institutional actions as conditionality. In this context, close implementation support or capacity building by the Bank may risk adding to misconceptions on the part of opponents that reforms are being forced by the Bank. In countries with high budget deficits and related macroeconomic weaknesses, DPOs can create incentives to commit to unattainable reforms in order to trigger the release of budget support, notwithstanding a lack of readiness or consensus among key stakeholders. Especially for new borrowers, the nature of the Bank’s role and its reason for engagement needs to be made very clear to all stakeholders throughout the entire engagement, beginning with project identification.

66. Public sector capacity constraints are especially challenging to deal with in the context of a DPO. While it is critical to obtain full commitment and buy-in from the client, as well as a clear indication of the demand for Bank support for identified sector and objectives, the capacity and incentives to deliver on reforms also need to be present. In this case, there was full commitment from the MoF to address the fiscal, financial, and ownership issues posed by the ICT sector and NTA. However, MoF did not have the mandate within the government, or the technical credibility, to lead an ICT sector reform process. For its part, MTC lacked internal capacity and expert technical assistance to sustain an ongoing and meaningful engagement under the Program. This highlights the importance of maintaining credible and effective engagement with relevant sector ministries that are properly resourced to implement activities. Institutional capacity constraints, challenges of remoteness and spatial dispersion for projects in RMI were also underestimated. Project preparation needs to be thorough, detailed, and advanced before project approval. Options for dealing with capacity constraints need to be practical, realistic, and enforceable.

67. The Bank must ensure that its operations are supported by robust analysis and understanding of political economy factors. This is especially pertinent for

- 22 -

complex, risky operations such as the DPO Program assessed in this ICR. In this case, all the conditions seemed to be in place to support the reform objectives. There were high levels of dissatisfaction with the low quality and high price of ICT services. Experience internationally and from around the Pacific region had demonstrated the benefits of liberalization and the potential for market-based reforms to attract new private sector investment, which would lead to substantial improvements in sector performance. The evident interest of MoF to address the substantial financial and fiscal risks arising due to the debts owed to RUS by NTA and guaranteed by the government, together with the initially encouraging adoption of the new sector policy framework by Cabinet and presentation of the draft Communications Bill to the Nitijela (satisfying the disbursement conditions for the First DPO), obscured the underlying weakness of the political backing for the overall reform program. This demonstrates the importance of thorough due diligence by the Bank on stakeholder support risks. Such efforts ex ante, in the case of this Program, would have revealed the likely resistance by the incumbent monopoly (including private shareholders, management and workers). In turn, this knowledge would have allowed the Bank to select the right set of instruments and sequencing to achieve the intended objectives.

7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES

AND PARTNERS

(a) Borrower/Implementing agencies

68. Not applicable.

(b) Cofinanciers

69. Not applicable.

(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

70. Not applicable.

- 23 -

ANNEX 1. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES

(a) Task Team members (by alpha surname)

Names Title Unit Responsibility/Specialty

LendingDouglas M. Addison Senior Economist GMF02 EconomicsNatasha Beschorner Senior ICT Policy Specialist GTI09 Task Team LeaderDoyle Gallegos Lead ICT Policy Specialist GTI11 Technical Specialist

Stephen Paul Hartung Financial Management Specialist GGO20 Financial Management

Piers E. Merrick Senior Operations Officer MNADE OperationsJunko Narimatsu ICT Policy Specialist GTI09 OperationsJames L. Neumann Senior Counsel GTI11 Technical SpecialistCarlo Maria Rossotto MENA Regional Coordinator GTI11 Peer ReviewerAndrea Ruiz-Esparza Senior Program Assistant GTI09 OperationsDavid Satola Lead Counsel ICOIO Technical SpecialistJinan Shi Senior Procurement Specialist GGODR ProcurementDouglas Webb ConsultantJoyce Miriam Denise Witana Procurement Specialist GGODR Procurement

SupervisionNatasha Beschorner Senior ICT Policy Specialist GTI09 Task Team LeaderRosanna Chan Economist GTI09 EconomicsJunko Narimatsu ICT Policy Specialist GTI09 Technical SpecialistJames L. Neumann Senior Counsel GTI11 Technical SpecialistAndrea Ruiz-Esparza Senior Program Assistant GTI09 OperationsDavid Satola Lead Counsel ICOIO Technical Specialist

(b) Staff Time and Cost

StageStaff Time and Cost (Bank Budget Only)

No. of staff weeks ($ thousands) Travel and Consultant costs)

LendingTotal: 2.3 22.3

Supervision/ICRTotal: 11.3 22.8

- 24 -

ANNEX 2. BENEFICIARY SURVEY RESULTS (IF ANY)

Not applicable.

- 25 -

ANNEX 3. STAKEHOLDER WORKSHOP REPORT AND RESULTS (IF ANY)

Not applicable.

- 26 -

ANNEX 4. BORROWER'S ICR AND/OR COMMENTS ON DRAFT ICR

Republic of the Marshall IslandsICT Sector Development Policy Operation (First)

Implementation Completion Results ReportSelf- Assessment

December 19, 2017

The Program consisted of three ICT Sector Development Policy Operations (ICT Sector DPO) covering the period 2012- 2016. The first operation (US $3 million IDA Grant) focused on the basic policy and regulatory foundations for ICT sector reform and on an investigation of the feasibility of splitting the submarine cable assets and the related RUS loan out of the incumbent operator. The second operation (US$5 million IDA Grant) would focus on key enabling legislation, the establishment of the new sector regulator, the adoption of a plan for restructuring of NTA, and the allocation of spectrum for new mobile operators. The third operation (US$5 million IDA Grant) would focus on new licensing and market entry, the commencement of regulator functions, and further strengthening of the ICT sector-enabling environment, with a particular emphasis on facilitating access to ICT in outer islands.

The rationale for Bank’s engagement in the ICT sector and the economic benefit as the result of reform was well outlined. The Bank has had extensive experience in the field of telecommunications policy, regulatory reform, and rural access issues. It has advised several governments on ICT issues in the Pacific region and globally. The Marshall Islands has been facing several challenges in the ICT sector, especially the fiscal challenge associated with the RUS loan. With the instrument of Development Policy Operation (DPO), the fungible nature of the DPO funds has enabled the Government of RMI to the flexible use of funds to address its fiscal constraints. The purpose and development objective of the DPO was clear and valuable.

However, assessing the outcome of the operation up to this point in time, it was realized that sector readiness was not sufficient to progress from the first operation to the subsequent operations. Technical Assistance, economic analysis, market analysis, and other strategic studies/plan should be completed at the first phase to well inform the Government and the ICT sector stakeholders’ decision making. The Government and the ICT sector itself has been limited with thin capacity to carry out these important studies but has been depending on technical assistances provided by development partners. All of these strategic studies and technical assistances would lay a significant foundation for decision-making and planning the sector reform. Solid, accurate, and valid information and results generated from the relevant studies and analysis could play a key role for

- 27 -

proper judgment. Together with continuous consultation and an awareness-raising campaign, the increase of support and political commitment could probably be foreseen.

Moreover, lack of proper engagement and coordination contributed to the poor result of the first operation. It was observed that the Ministry of Finance has not engaged the key sector stakeholders, including the National Telecommunication Authority (NTA) and the Ministry of Transportation and Communications (MTC) to a satisfactory extent. The aid coordination and management structure was not established and the coordination platform and channel was poorly managed. The information was poorly coordinated and communicated with the key stakeholders which could create additional risks related to confusion, disconnection, misunderstanding, misinterpretation, and misrepresentation at all levels. To a more significant extent, this type of misunderstanding could contribute to lack of trust and resistance from key stakeholders. Without proper and timely mitigation of these risks, the expected project objectives could not be achieved.

Lack of transparency was also observed in the process. For example, the stakeholders and the decision makers (such as the Cabinet members) did not receive comprehensive information but only partial information. Information was disseminated among selected members. After-fact notification and dissemination was commonly observed and was problematic. It resulted in that some decision makers did not understand the project objectives, the project structure, and other important elements and thus were not on board to support the project.

The ICT Sector DPO was the first project from the World Bank in the Marshall Islands. The understanding of the role, procedures and requirements, instrument, and development objectives and expectations of the World Bank was limited. To achieve the objectives of DPO, it involved strategic and sophisticated planning and implementation. The degree of complexity implied in the actual implementation could be high and should not be overlooked nor underestimated. In this context, the introduction of DPO as the development instrument should be dealt with at even higher level of due diligence to ensure readiness and acceptance.

Learning from the assessment, the Ministry of Finance is currently taking action to improve its aid coordination and management structure by first establishing a designated division (Division of International Development Assistance, DIDA) to undertake these tasks. In addition, a review on current aid coordination structure is taking place at the national level to develop an aid coordination mechanism and a development assistance policy. It is expected that the development assistance policy will be finalized in January 2017. At the project level, the Ministry of Finance is adopting

- 28 -

a working group and/or steering committee to involve key stakeholders in the process. It is recognized that the formation of this type of working group or steering committee be established at the early stage of the project preparation. Continuous and honest engagement and consultation is highlighted as the key guiding principle. To strengthen the transparency and information dissemination, the Ministry of Finance official website is currently under development and can be expected in early 2017. This official website will be served as the platform for information dissemination, such as reports, studies, assessments, project documents, etc. A Facebook page will also be created to share information and updates.

To conclude, lessons learned is the key for further improvement in planning and implementing the development projects in the Marshall Islands. The Ministry of Finance has recognized the importance of better aid coordination and management in the Public Financial Management Reform Roadmap. The Ministry of Finance will continue working closely with development partners to address the gaps and strengthen the capacities.

- 29 -

ANNEX 5. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS

Not applicable.

- 30 -

ANNEX 6. LIST OF SUPPORTING DOCUMENTS

Project Document 65985-MH Legal document: Financing Agreement Grant Number H838-MH Program-related correspondence

Concept Note documents Appraisal documents Approval documents Supervision documents

- 31 -

- 32 -