Performance Evaluation

Report

Promoting Energy Efficiency in the Pacific

Raising development impact through evaluation

EvaluationIndependent

Technical Assistance Numbers: 6485 and 7798 Independent Evaluation: TE-71

Performance Evaluation Report June 2021

Promoting Energy Efficiency in the Pacific

This document is being disclosed to the public in accordance with ADB’s Access to Information Policy.

NOTE In this report, “$” refers to United States dollars.

The guidelines formally adopted by the Independent Evaluation Department on avoiding conflict of interest in its independent evaluations were observed in preparing this report. To the knowledge of the management of the Independent Evaluation Department, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area.

Director General Marvin Taylor-Dormond, Independent Evaluation Department (IED) Director Nathan Subramaniam, Sector and Project Division, IED Team leader Sherine Ibrahim, Principal Evaluation Specialist, IED Team members Lawrence Nelson Guevara, Senior Evaluation Officer, IED

Christine Grace Marvilla, Senior Evaluation Assistant, IED

Abbreviations ADB – Asian Development Bank CFL – compact fluorescent lamp CO2 – carbon dioxide DMF – design and monitoring framework ESCO – energy service company GEF – Global Environment Facility GIEO – GEF Independent Evaluation Office GHG – greenhouse gas IED – Independent Evaluation Department kWh – kilowatt-hour LED – light-emitting diode M&V – measurement and verification MEPS – minimum energy performance standard MWh – megawatt-hour PARD – Pacific Department PDMC – Pacific developing member country PMF – program management firm PNG – Papua New Guinea REG – regional

SDG – Sustainable Development Goal TA – technical assistance TCR – TA completion report tCO2 – tons of CO2 TPER – TA performance evaluation report

Contents

Acknowledgments vii Basic Data ix Executive Summary xi Chapter 1: Introduction 1 A. Evaluation Purpose, Scope, and Methodology 1 B. Technical Assistance Project Objectives 2 C. Technical Assistance Completion Reports 3 D. Structure of the Evaluation Report 4 Chapter 2: Design and Implementation 5 A. Rationale 5 B. Time, Cost, Financing, and Implementation Arrangements 7 C. Role of Consultants 11 D. Changes in Design 12 E. Monitoring and Reporting Arrangements 12 Chapter 3: Performance Assessment 14 A. Relevance 14 B. Effectiveness 17 C. Efficiency 22 D. Sustainability 23 Chapter 4: Other Assessments 25 A. Development Impact 25 B. Performance of the Asian Development Bank 27 C. Performance of Implementing Agencies 27 Chapter 5: Overall Assessment, Lessons, Issues, and Recommendations 29 A. Overall Assessment 29 B. Lessons 30 C. Issues 31 D. Recommendations 32 Appendixes 34 1. Summary Design and Monitoring Frameworks of Individual Technical Assistance Projects 35 2. Summary of Energy Efficiency Pilot Projects Implemented under Technical Assistance 39

for Promoting Energy Efficiency in the Pacific (Phase 2) 3. Overview of Energy Efficiency Measures Adopted in Participating Pacific Developing 41

Member Countries

Acknowledgments This report was prepared by the Independent Evaluation Department (IED) of the Asian Development Bank (ADB), led by Sherine Ibrahim, with Lawrence Nelson Guevara and Christine Grace Marvilla as team members. The evaluation was supported by an international consultant (Anil Terway) and three national consultants (Boyd Ellison for Cook Islands, Chris Solomona for Samoa, and Hikaione Loumoli for Tonga) who provided valuable input and support. The evaluation was conducted under the supervision of Nathan Subramaniam, Director, Sector and Project Division, and under the overall guidance of Marvin Taylor-Dormond, Director General, IED. The evaluation team appreciates the support of the governments of Cook Islands, Papua New Guinea, Samoa, Tonga and Vanuatu. The team is grateful to the ADB Pacific Department for providing project information and coordinating with the governments for the mission. Comments received were considered and incorporated as deemed appropriate. The internal peer review comments by Alfredo Baňo-Leal (at the evaluation approach stage), Garrett Kilroy and Benjamin Graham (at the evaluation stage) and external peer review by Priyantha Wijayatunga and David Corderi-Novoa were also appreciated. IED retains full responsibility for the report.

Basic Data

Key Information TA 6485-REG TA 7798-REG Title Promoting Energy Efficiency in the

Pacific Promoting Energy Efficiency in the Pacific (Phase 2)

Sector Energy Energy

Subsector Energy sector development Energy efficiency and conservation

Strategic Agenda/ Themes (subtheme)

Sustainable economic growth (promoting economic efficiency and enabling markets), environmental sustainability (global and regional trans-boundary environmental concerns and issues), capacity development (institutional development)

Economic growth (promoting economic efficiency and enabling business environment), environmental sustainability (eco-efficiency, environmental policy and legislation), capacity development (institutional development)

Executing Agency Pacific Department, ADB Pacific Department, ADB

Key Figures TA 6485-REG ($ million)

TA 7798-REG ($ million)

Source of TA funds CEF and GEF TA Special Fund, ACEF, GEF, and Government of Australia

Approved fund TA Special Fund 0.00 1.00 CEF 1.20 0.00 ACEF 0.00 1.50 GEF 0.20 5.25 Government of Australia

0.00 1.00

Total 1.40 8.75 Actual disbursed 1.35 7.15

ACEF = Asia Clean Energy Fund, ADB = Asian Development Bank, CEF = Clean Energy Fund (under the Clean Energy Financing Partnership), GEF = Global Environment Facility, REG = regional, TA = technical assistance, TCR = technical assistance completion report. Sources: Asian Development Bank Technical Assistance Reports and Technical Assistance Completion Reports.

Key Dates TA 6485-REG TA 7798-REG Board approval date 12 September 2008 31 March 2011 Effective date 12 September 2008 31 March 2011 Fielding of consultants 27 April 2009 6 November 2011 Original completion date 31 December 2009 31 March 2015 Actual completion date 31 August 2011 23 October 2015 Original closing date 31 December 2009 1 July 2015 Actual closing date 31 August 2011 23 October 2015 TCR circulation date 15 September 2011 14 September 2016

Executive Summary

The Asian Development Bank (ADB) undertook capacity support and implemented pilot projects to promote energy efficiency in the Pacific region under two technical assistance (TA) grants. This technical assistance performance evaluation report evaluates these TA projects, which were provided to Cook Islands, Papua New Guinea, Samoa, Tonga, and Vanuatu. The total value was $10.15 million.

The evaluation assessed the overall performance of the TA projects less than successful. The first TA project included mostly diagnostic and policy advisory aspects and was relevant, effective, efficient, and most likely sustainable. The follow-on TA project included substantive investments in pilot energy efficiency projects and was less than relevant because of a reduction in the scope, less than effective as targets were not achieved, efficient and less than likely sustainable. Taken together, the Promoting Energy Efficiency in the Pacific program (the combination of the two projects) was less than relevant, less than effective, efficient, and less than likely sustainable.

The evaluation makes five recommendations for ADB’s future support in the Pacific and similar small developing member countries: (i) improve TA administration, (ii) identify winners and losers when implementing policy reforms, (iii) run campaigns to change the purchasing behavior of households and businesses regarding energy efficiency products, (iv) consider using the small expenditure financing facility for energy efficiency programs, and (v) promote low-carbon development for reducing energy consumption in buildings.

This report evaluates the performance of two regional technical assistance (TA) projects for promoting energy efficiency in five Pacific developing member countries (PDMCs): Cook Islands, Papua New Guinea (PNG), Samoa, Tonga, and Vanuatu, approved by the Asian Development Bank over 2008–2011. The TA projects were Promoting Energy Efficiency in the Pacific—referred to as the first project; and (ii) Promoting Energy Efficiency in the Pacific(Phase 2)—referred to as the second project. Theenvisaged outcomes of the two TA projects were(i) agreement on a project to be jointly funded byPDMCs, Asian Development Bank (ADB) andGlobal Environment Facility (GEF), which will fundthe development of capabilities and energyefficiency improvements in industrial, commercial,residential, and public sector buildings; and(ii) end consumers using power efficiently in theparticipating countries.

Evaluation Purpose and Scope

This technical assistance performance evaluation report (TPER) involved a desk review of the documentations and data related to the

Promoting Energy Efficiency in the Pacific program (i.e., the two TA projects) and a virtual evaluation mission in November 2020. The mission held discussions with government officials, staff of power utilities, and representatives of key development partners in four of the participating PDMCs. Representatives of Cook Islands preferred to discuss and share project information in face-to-face meetings with the national consultant engaged by the evaluation mission instead of participating in virtual meetings with the full evaluation mission.

Design and Implementation

The first project was implemented from September 2008 to August 2011. ADB provided $1,200,000 from the ADB-administered, multi-partner Clean Energy Fund under the Clean Energy Financing Partnership Facility and GEF provided a project preparatory grant of $200,000 to identify and develop the energy efficiency program under the follow-up TA, Promoting Energy Efficiency in the Pacific (Phase 2)—the second project.

xii Promoting Energy Efficiency in the Pacific

The expected outputs of the first project, according to its design and monitoring framework (DMF), were: (i) an assessment of the energy efficiency policy, regulatory framework and past assistance; (ii) recommendations for an energy efficiency policy and regulatory framework in five PDMCs; (iii) recommendations for a structured energy management system for sustained energy efficiency initiatives, including possible establishment of energy service companies; (iv) assessment of training and assistance needed for regulatory and institutional reforms for promoting viable market for energy efficiency services; (v) preparation of a pipeline of energy efficiency projects for funding by ADB, GEF, or other financing sources (referred as the “energy efficiency program”); and (vi) preparation of a strategy for raising public awareness and education. During the inception mission for the first project, it was discovered that the participating PDMCs lacked suitable databases for energy demand-side information and that the existing policies and regulations did not give due importance to an energy efficiency strategy. The governments, utilities, and ADB agreed to focus on four outputs: (i) establishment of energy use baseline, (ii) recommendations for energy efficiency policy and institutions, (iii) recommendations for future energy efficiency programs, and (iv) implementation of a demonstration energy efficiency project in each of the five PDMCs. The energy efficiency program outlined seven energy efficiency measures: energy efficiency in government buildings, street-lighting, hotels, water pumping, use of compact fluorescent lamps in residences; application of energy labelling and minimum energy performance standards (MEPS); and power factor correction. If these measures were implemented country-wide, the potential energy savings in the five PDMCs were 48,453 MWh per year, or about 4.5% of total electricity sales, by 2013.

The second project was implemented from April 2011 to October 2015. ADB provided $1,000,000 from the TA Special Fund, the Government of Australia provided $1,000,000, the Asian Clean Energy Fund provided $1,500,000, and the GEF provided $5,254,545.

The expected outcome of the second project was more efficient use of electricity by end consumers. The planned outputs were (i) comprehensive information on energy use and appliances that stakeholders could access; (ii) energy efficiency practices mainstreamed into government processes, procedures, and policies; (iii) energy efficiency program implemented effectively and sustainably; and (iv) energy efficiency information disseminated and public awareness of the benefits of energy savings raised. However, instead of country-wide energy efficiency programs, the second project implemented 34 pilot projects in the five PDMCs with a view to sharing the experience of using new technologies that would help achieve national energy efficiency targets. The estimated cost of the pilot projects was $2,726,482 and the expected annual energy saving was 3,411 MWh.

Performance Assessment Relevance. The Promoting Energy Efficiency in the Pacific program was aligned with ADB’s Clean Energy Program, Energy Policy and Pacific Strategy, and with the PDMCs’ desire for resilient development that included lowering energy use and reducing dependence on imported fossil fuel. Lowering energy use continues to be a priority for ADB and the governments of the participating PDMCs. The PDMCs made a negligible contribution to global carbon dioxide (CO2) emissions (0.012%) but the costs of adapting to climate change in these countries are high. Development partners have generally supported implementation of highly visible renewable energy projects but few that had measures for energy demand side, except for the assistance in formulating the Pacific Appliance Labelling Standards. The International Energy Agency considers energy efficiency to be the “first fuel” for a transition to a clean energy future. The first project covered a review of energy use, policies, institutions, and regulations. It implemented energy efficiency demonstration projects to provide in-country learning experiences and to build support for energy efficiency. The potential energy savings from the seven energy efficiency measures would guide further assistance for the PDMCs. This evaluation assessed the first project relevant. The second project built on the outputs of the first. It was appropriately designed to overcome the

Executive Summary xiii

barriers to energy efficiency adoption. It covered preparation of a database for energy use, deliverables for energy efficiency practices, and information dissemination. The TA design required implementation of an energy efficiency program, which was reduced to the implementation of 34 pilot projects during TA implementation. There was no record of the justification for or approval of this major change in scope and realignment of the DMF outcome and output targets. The energy efficiency policy-related analysis, together with the demonstration and pilot projects, provided the knowledge and experience sharing for adopting suitable energy efficiency technologies in the five PDMCs. However, design weaknesses of the second project prevented the achievement of the quantitative energy use reduction targets. These weaknesses were: use of the TA modality as significant investment was included in the project, a reduction in the scope of the investment component, and a failure to realign the quantitative targets. IED assessed the second project and the Promoting Energy Efficiency in the Pacific program as less than relevant. Effectiveness. The envisaged outcome of the first project was an agreement between the PDMCs, ADB, and GEF on extending support for the second project. This was achieved. Under ADB’s supervision, the consultants produced four outputs: (i) a database on energy use; (ii) energy efficiency policy recommendations; (iii) implementation of five energy efficiency demonstration projects, one in each PDMC; and (iv) a pipeline of energy efficiency projects for implementation in the second project. This evaluation assessed the first project effective, as the outcome and three outputs were achieved (the database was inadequate for bottom-up estimation of energy efficiency benefits, so that output was considered not achieved). The second project outcome targets included a 10% reduction in average monthly power consumption by residential and commercial customers and in public buildings. These were to be achieved by the country-wide adoption of the energy efficiency measures identified in the first project. The outcome targets were not achieved. The consultants implemented four outputs that built on the work carried out under the first project: (i) an internet web-based energy use database, (ii) deliverables for energy efficiency

practices to be mainstreamed into government processes, procedures, and policies, (iii) 34 energy efficiency pilot projects, and (iv) information dissemination and public awareness raising. The reductions in monthly consumption by various customer categories could not be assessed because of the lack of disaggregated energy sales data, and the savings from the 34 energy efficiency pilot projects could not be confirmed because the measurement and verification (M&V) protocol was not implemented. Although the knowledge and support aspects were high, only one of the four outputs could be assessed as having been achieved: information dissemination and public awareness raising. IED assessed the second project and the overall Promoting Energy Efficiency in the Pacific program as less than effective. Efficiency. The consultants for the two TA projects were engaged on a competitive basis. Procurement of equipment for the energy efficiency demonstration and pilot projects was within budget, in fact the price of the lights that were procured was 20%–30% below estimates because of competitive bidding and a worldwide decline in prices. Closing of the second project was delayed by 7 months, but 12 pilot projects were incomplete, and the M&V protocol could not be implemented until completion of the pilot projects. Since costs were incurred for the 34 pilot projects, a conservative quantitative analysis showed an overall internal rate of return of 33% and a pay-back period of 1.9 years. IED assessed the first project efficient as the TA funds were well utilized and implementation delays were attributed to the preparation of the GEF grant application. Although there was underutilization of TA funds in the second project, the indicative internal rate of return and pay-back period of the pilot projects were reasonable. IED assessed the second project and the Promoting Energy Efficiency in the Pacific program as efficient. Sustainability. The Promoting Energy Efficiency in the Pacific program was innovative as it provided knowledge and support for country-wide energy efficiency programs, compared with earlier externally assisted projects which contributed limited inputs to just two of the five participating PDMCs. The transformation of the energy efficiency market was ongoing, inefficient incandescent lamps were no longer available, and appliance traders were stocking 3- and 4-star

xiv Promoting Energy Efficiency in the Pacific

rated refrigerators and air conditioners. Electricity tariffs remained high in all countries so households with high electricity consumption had an incentive to select energy efficient products. IED assessed the first project most likely sustainable because it was innovative and identified energy efficiency technologies for energy saving. The second project achievements were unlikely to last until all the governments start enforcing MEPS, energy efficiency markets are fully transformed, and the public is more aware of the availability of energy efficiency

products. IED assessed the second project and the overall Promoting Energy Efficiency in the Pacific program as less than likely sustainable.

Other Assessments Development impacts. The five PDMCs have committed themselves to resilient development and their Intended Nationally Determined Contributions submitted to the United Nations Framework Convention on Climate Change included targets for lowering greenhouse gas (GHG) emissions, with support from the international community. Development partners implemented two more projects, which included supply of some energy efficient products and conducting energy audits for continuing the reduction in energy use. The long-term environmental impact of the energy efficiency measures was mostly positive as the pollution due to electricity generation decreased. The estimated annual diesel saving from the 34 pilot projects was 871,463 liters; the consequent carbon dioxide (CO2) emission reduction was 3.2 tons. The private sector enterprises that managed the trade of energy efficiency products and appliances were provided considerable knowledge by way of demonstration and pilot projects, reports and brochures to highlight the benefits and short pay-back period of energy efficiency measures. The DMF impact targets were not achieved and were not realigned when the scope of the second project was reduced. The evaluation assessed the development impact of the Promoting Energy Efficiency in the Pacific program’s knowledge and support to be satisfactory. Knowledge products for promoting energy efficiency were produced and there was significant capacity building of government staff and other stakeholders. Energy efficient lights, refrigerators, and air conditioners are becoming

readily available at competitive prices and the domestic appliance market is gradually transforming. However, overall, the development impact of the Promoting Energy Efficiency in the Pacific Program was less than satisfactory because the DMF targets were not achieved. ADB performance. ADB led the initiative and obtained considerable support from other development partners for implementing the Promoting Energy Efficiency in the Pacific program in the five PDMCs. During program implementation, ADB fielded regular field missions and reviewed consultants’ progress reports. TA administration was effective and ADB provided timely approval of several minor changes in scope, time extensions, and over 22 contract variation orders. ADB submitted the required reports to GEF, including a detailed supplementary appendix with the TA completion report for GEF projects. However, TA administration records did not include information on an approval of a major change in scope and ADB did not investigate options for using surplus GEF funds or for further extension of the second project closing date to allow implementation of the M&V protocol for the benefit of the pilot projects. IED assessed ADB’s overall performance less than satisfactory mainly because of unexplained design changes, underutilization of TA funds, and because the consultant for the second project was not required to carry out the M&V protocol for the pilot projects. Performance of implementing agencies. The five implementing agencies in the PDMCs supported the implementation of the Promoting Energy Efficiency in the Pacific program. They provided the consultants with administrative support, although there were some issues with the availability of reliable energy use data. Overall, IED assessed the performance of the implementing agencies to be satisfactory.

Overall Assessment, Lessons, Issues, and Recommendations Overall assessment. IED assessed the first project successful as it was relevant, effective, efficient, and most likely sustainable. The second project was assessed less than successful as it was less

Executive Summary xv

than relevant, less than effective, efficient, and less than likely sustainable. With the committed TA amount for the second project being about seven times larger than the first one, the success of the Promoting Energy Efficiency in the Pacific program (the combination of the two projects) had depended mainly on a successful implementation of the second project. However, the five participating PDMCs had not fully embraced the energy efficiency concept, therefore, the expected energy savings was far less than the target and the TA implementation could have been improved. Accordingly, the Promoting Energy Efficiency in the Pacific program was assessed less than successful. The program was less than relevant, less than effective, efficient, and less than likely sustainable. Lessons Support for energy efficiency programs can make significant contributions toward savings and low-carbon development. An energy efficiency program is a cost-effective alternative for low-carbon development. The first project under the Promoting Energy Efficiency in the Pacific program estimated that an investment of $48 million would save 48,453 MWh per year through seven energy efficiency measures to be implemented across the PDMCs. The energy saving was equivalent to the output of a solar photovoltaic-based power generation capacity of 38.3 MW, considering 15% transmission and distribution loss and a capacity utilization factor of 17%. The capital cost in 2015 for such a solar project would have been more than double the investment in the energy efficiency program. The high capital costs of renewable energy-based power generation projects translate into high tariffs for consumers, and in PDMCs consumers already pay high tariffs for conventional power generation. An energy efficiency program would provide savings in energy costs over the lifetime of the appliances. In general, the PDMCs and their development partners acknowledged that the Promoting Energy Efficiency in the Pacific program had built support for adoption of energy efficient technologies. Reliable measures of completed costs for TA projects should help with the preparation of future projects. A proper cost analysis was not possible after completion of the Promoting Energy Efficiency in the Pacific program because

all the inputs were not fully accounted for. ADB kept a record of the disbursements, so reliable information was available for that part of the program cost. However, audited project accounts should have been kept for the demonstration and pilot projects to confirm the major program costs. The consultants could have also used reasonable assumptions to establish the in-kind contributions of counterpart staff and power utilities. A reliable measure of the costs of the completed TA projects would have helped in the preparation of the cost estimates and scope of future TA projects. Regional institutions can play a strong role in preparing and delivering knowledge products. The terms of reference for both the first and the second project required the consultant to work closely with the Pacific Center for Renewable Energy and Energy Efficiency, the Pacific Regional Data Repository for Sustainable Energy for All, and the Pacific Community. These organizations helped to make up for the lack of domestic experts and to lower the cost of data acquisition and analysis. The consultants’ reports indicated that these regional institutions helped in sharing the experience of the energy efficiency pilot projects, although they could have played a greater role in the preparation and maintenance of an energy efficiency appliances database. TA designs are more efficient if they are based on local conditions. Since the Pacific Appliance and Labelling Standards had already been developed, the output of the second project need not have included development of the standards and instead just focused on developing legislation, instruments and institutions to enforce them. Similarly, since energy service companies (ESCOs) were not commercially viable, the output of the first project could have identified alternative arrangements for incentivizing the private sector to retrofit energy efficiency measures. The consultant’s terms of reference should have explicitly stated that the technical specifications of energy efficiency equipment should consider the local weather conditions for avoiding early deterioration, like corrosion in coastal regions. Issues Inadequate TA administration records. Preparation of this evaluation was made more difficult because of the lack of proper TA administration records. There was no paper trail

xvi Promoting Energy Efficiency in the Pacific

for the changes in scope that took place during contract negotiations and the inception mission. There are records of extensions of TA completion dates and the variation orders for consultancy services contracts but no explanations for them. The e-operation records could not be accessed so the evaluation team could not easily understand TA implementation progress, procurement delays, and other issues. Hard copies of TA administration records that are held in storage could not be retrieved because of restricted staff movement as a result of the coronavirus disease (COVID-19) pandemic. The TCRs for the two projects were prepared before the December 2019 revision of the staff instruction and so contained less information and fewer ratings than are now required. The reasons for the delay in implementing energy efficiency policies and strengthening institutions were not clear. The reports by the TA consultant and the review mission do not explain the remaining barriers for implementing energy efficiency measures in the PDMCs, including those related to the decisions taken by individual households and businesses, which are influenced by the political environments in the PDMCs. The TA projects did not review the political and societal support for energy efficiency measures that apparently benefited everyone. Subsidies and the dissemination of energy efficiency information may be insufficient for changing behaviors. Energy efficiency measures are still not mainstreamed. The program showed a lack of understanding of customer behavior and did not design suitable triggers to shift customer choice toward energy efficiency products. Traders lacked incentives for actively promoting the sale of appliances with a higher efficiency. Commercial establishments did not have clear pathways for retrofitting existing high energy consuming systems or mandatory energy efficiency codes. The TA modality may not be suitable for projects that include both knowledge support and investments. The TA approach allowed ADB to design a project that assisted more than one DMC, but its basic purpose was to provide knowledge and advisor support. It lacked the instruments needed for large-scale procurement of goods, civil works, and installation services. Detailed cost and financing plans, audits of

project expenditures, and economic and financial analyses were not prepared. Recommendations ADB should: 1. Improve TA administration records. Performance evaluation of large TA programs, with over $1 million worth of consulting services implemented over a duration of more than 2 years, requires a complete set of TA administration records, including back-to-office reports of TA review missions, quarterly updates of TA implementation progress, reviews of consultants’ proposals for changes, memos seeking approval of minor and major scope changes, and time extensions. Such records can be used to validate the information obtained from the TA consultant’s report. 2. Identify winners and losers and better manage the impacts of policy reforms, especially on losers. Implementation of a policy reform requires change management, which is influenced by the political environment in each country. The Promoting Energy Efficiency in the Pacific program had both winners (e.g., a ministry of finance that benefited from lower electricity costs for public buildings) and losers (e.g., utilities whose power generation capacity expansion plans and revenues from electricity sales were affected by energy savings). Negotiations between these winners and losers are needed when implementing energy efficiency policies and procedures. The PDMCs have committed themselves to low-carbon growth targets, and their high electricity tariffs provide consumers with incentives to switch to energy efficiency appliances. The environment was therefore favorable for the implementation of the Promoting Energy Efficiency in the Pacific program, but the outcome was not achieved. Future ADB assistance in support of policy reforms should consider engaging a social science specialist in the consultancy team to assess the political environment and suggest appropriate approaches to support negotiations between winners and losers for the successful implementation of new energy efficiency policies and regulations. 3. Design and run more targeted campaigns that aim to change the purchasing behavior of

Executive Summary xvii

households and businesses towards energy efficiency products. Policy and regulations can provide the enabling environment, but it is individual customers and business entities that will take the decision to shift to energy efficiency appliances and products. Enforcing MEPS requires an understanding of triggers and incentives to convince appliance dealers to import and provide customer services for efficient products. Future operations should include a marketing specialist in the consultant’s team to study purchasing behavior in the domestic appliance market and suggest campaigns that would shift consumer choice towards more efficient appliances and equipment. The marketing specialist could identify triggers and a suitable subsidy program to accelerate a shift to energy efficiency products.

4. Consider using the small expenditure financing facility for future energy efficiency programs. The envisaged reduction in energy use and consequent reduction in GHG emissions enabled the Promoting Energy Efficiency in the Pacific program to attract significant support from ADB’s development partners. The activities included consulting services and implementation of a very large number of small subprojects. Future ADB assistance for energy efficiency programs in PDMCs, or in other developing member countries that depend on imported energy, should consider using the small expenditure financing facility modality. However, since implementation capacity is weak in some developing member countries, the facility may need a minor modification to allow ADB to be the executing agency of an energy efficiency program as is the practice for a TA project. Future energy efficiency investments should be structured as a public sector project because of the limited size and competitiveness of the private sector-led appliance and electrical equipment markets in the PDMCs; extending support to a group of PDMCs would improve procurement efficiency by increasing the lot sizes and lowering logistics costs.

5. Promote low-carbon development by reducing energy consumption in buildings. In addition to supporting renewable energy and electricity storage projects, ADB should support the lowering of energy consumption in buildings. Support needs to include specific activities that build on the status of the energy efficiency measures that have been adopted in the PDMCs. For example, projects could subsidize the higher up-front costs of energy efficiency products, and the retrofitting of energy efficiency measures in buildings. They could create an energy efficiency fund to offer medium-term loans based on energy savings. This would lower dependence on imported fossil fuel, improve energy security, increase household and enterprise savings because of lower energy bills, and lead to a cleaner environment because of a reduction in emissions.

S O U T H P A C I F I C O C E A N

N O R T H P A C I F I C O C E A N

Erikol

Rongerik

Likiep

Bikini

Ujelang

Enewetak

AilinginaeWotho

Taongi

WotjeMaloelap

ArnoMili

Jaluit

Ebon

AilinglaplapNamu

KwajaleinUjae

RongelapUtirikAiluk

KiliKnox

Butaritari

Marakei

South Tarawa

AroraeTamanaOnotoa

TabiteueaNonouti

Abemama

BeruNikunau

Kuria

Abaiang

Teraina

Tabuaeran

Kiritimati

Malden

Starbuck

MilleniumVostok

Flint

Winslow

Rawaki

Kanton

Orona

Carondelet

Nikumaroro

Mckean Birnie

Manra

Ederbury

Kosrae

Senyavin

Pohnpei Pingelap

Mokil

Yap

Ngulu

FaisUlithi

WoleaiIfalik

Sorol

Eauripik SatawalChuuk

Mortlock

Nukuoro

Kapingamarangi

Minto Reef

Oruluk

NamonuitoGaferutFaraulep

PulapLosap

Ngatik

PikelotLamotrek

Elato

Fayu

Nanumea

Nukufetau

Nui

NanumagaNiutao

Nukulaelae

Niulakita

Vaitupu

Fogafale

Rotuma

Viti Levu

Taveuni

Vanua Levu

Kadavu

Ceva-i-Ra

Lau

Yasawa

Choiseul

Ontong Java

Santa Isabel

MalaitaGuadalcanal

New Georgia

Santa Cruz Islands

San CristobalRennell

AnutaFatakaTikopia

Helen Reef

MerirPulo Anna

Sonsoral Islands

Hatohobei

Peleliu

Banaba

Hall

Rarotonga

Suwarrow

Pukapuka Manihiki

Penhryn

Palmerston

AitutakiManuae

MaukeAtiuTakutea

Mangaia

Mitiaro

Nassau

Rakahanga

Tongatapu

Niuafo'ou

NiuatoputapuTafahi

Lifuka

'Ata'Eua

Vava'u

Savai'iUpolu

Vandua LavaVetaounde

Espiritu Santo

Torres

Malekoula

Aniwa

Aneytioum

MaewoPentecost

Ambryn

Futuna

Efate

NukumanuNew Britain

New Ireland

Bougainville

Buka

Manus

New Hanover

TrobiandWoodlark

Melekeok

Honiara

Majuro

Suva

Tarawa

Yaren

Funafuti

Palikir

Alofi

Canberra

Wellington

Nuku'alofaAvarua

Port Vila

Port Moresby

Apia

REPUBLIC OF PALAU

SOLOMON ISLANDS

FEDERATED STATES OF MICRONESIA

K I R I B A T INAURU

MARSHALL ISLANDS

FIJI

TUVALU

NIUE

PAPUANEW GUINEA

VANUATU

TONGA

COOK ISLANDS

SAMOA

NEWCALEDONIA

HAWAII

WALLIS ANDFUTUNA

AMERICANSAMOA

TOKELAU

FRENCH POLYNESIA

GUAM

NORTHERNMARIANA ISLANDSPHILIPPINES

BRUNEI DARUSSALAM

MALAYSIA

A U S T R A L I A

N E W Z E A L A N D

This map was produced by the cartography unit of the Asian Development Bank. The boundaries, colors, denominations, and any other information shown on this map do not imply, on the part of the Asian Development Bank, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries, colors, denominations, or information.

National Capital

Reef/Atoll

Country Groupings

International Boundary

Asian Development Bank

Boundaries are not necessarily authoritative.

ADB

0 500 1000

Kilometers

N

180°00140°00 E 160°00 W

180°00140°00 E 160°00 W

0°0°

20°N

20°S

20°N

20°S

160°00 E

160°00 E

40°S40°S

ADB'S DEVELOPING MEMBER COUNTRIES IN THE PACIFIC

PROMOTING ENERGY EFFICIENCY IN THE PACIFIC

210022 2

0P

acific

AB

V

CHAPTER 1

Introduction

1. Pacific developing member countries (PDMCs) face unique energy challenges. Fossil fuel resources are available in only a couple of these countries, but a lack of technical and financial capacity limits their use. Power generation is dependent on imported diesel fuel; the small size of Pacific economies weakens their negotiating position, and countries have few instruments available to manage the price volatility of international oil markets. Outdated power infrastructure, geographical dispersion, and scattered small island grids also push electricity tariffs higher. Improvements in energy efficiency would lower electricity demand and make Pacific economies more resilient, while increasing disposable household incomes and making private businesses more competitive. 2. The Asian Development Bank (ADB) is empowering people and communities by financing additional capacity for renewable energy integration, battery storage, and enhanced transmission and distribution infrastructure. ADB’s Clean Energy Program seeks to increase efficiency in energy, transport, and urban development. It is aligned with Sustainable Development Goal (SDG) 7ensure access to affordable, reliable, sustainable and modern energy to all. Improving energy efficiency is part of SDG 7; in order to meet the SDG target of doubling the global rate of improvement in energy efficiency, the annual improvement rate needs to be at least 3% per year till 2030.1 ADB has also established a donor-funded Clean Energy Financing Partnership.

A. Evaluation Purpose, Scope, and Methodology 3. Evaluation purpose. The Independent Evaluation Department (IED) assesses the performance of technical assistance (TA) activities for their contribution to learning and accountability, and to improvements to project designs and effectiveness. The findings of this evaluation will feed into an ongoing evaluation of ADB support for action on climate change. An evaluation of a completed TA program for promoting energy efficiency also builds on one of the recommendations of a recent sector-wide evaluation of the ADB energy policy that ADB should support demand-side energy efficiency by improving policies and developing incentive mechanisms for consumers, utilities and other market players to become more energy efficient.2

4. Evaluation scope. This TA performance evaluation report (TPER) evaluates two regional TA projects for energy efficiency and conservation implemented by ADB in five PDMCs: the Cook Islands, Papua New Guinea (PNG), Samoa, Tonga, and Vanuatu. The two regional capacity building TA projects (which have been reclassified as knowledge and support TA) were: (i) Promoting Energy Efficiency in the Pacific—referred to as the first project;3 and (ii) Promoting Energy Efficiency in the Pacific (Phase 2)—referred to as the second project.4 The two TA projects are jointly referred to as the Promoting Energy Efficiency in the Pacific program. The first project identified a significant pipeline of activities for promoting energy efficiency that were implemented in the five PDMCs under the second project. The overarching question identified in the evaluation approach paper was: to what extent has ADB, through

1 United Nations Statistics Division. 2020. Goal 7 Affordable and Clean Energy. New York; accessed on 23 November. 2 ADB. 2020. Sector-wide Evaluation of ADB Energy Policy and Program 2009–2019. Manila. 3 ADB. 2008. Technical Assistance to Cook Islands, PNG, Samoa, Tonga, and Vanuatu for Promoting Energy Efficiency in the Pacific

(Promoting Energy Efficiency in the Pacific). Manila. 4 ADB. 2011. Technical Assistance to Cook Islands, PNG, Samoa, Tonga, and Vanuatu for Promoting Energy Efficiency in the Pacific

(Phase 2) (Promoting Energy Efficiency in the Pacific (Phase 2). Manila.

2 Promoting Energy Efficiency in the Pacific

the Promoting Energy Efficiency in the Pacific program, contributed to the long-term objective of improving energy security, reducing life-cycle costs, and reducing greenhouse gas (GHG) emissions in the Pacific? The evaluation matrix in the evaluation approach paper listed a number of key evaluation questions.5

5. Evaluation methodology. The evaluation is based on the relevance, effectiveness, efficiency, and sustainability of the TA projects, in accordance with ADB’s evaluation guidelines.6 It examines the extent and sustainability of the outcomes of the TA projects and the institutional capacity development they supported, their development impacts, and the performance of ADB and implementing during TA implementation. 6. The evaluation evidence was gathered through: (i) a desk review of project documents, including TA reports, TA completion reports (TCRs), consultant reports, published reports and other deliverables under the projects, analytical reports published by ADB, IED evaluation studies; (ii) virtual interviews with ADB project officers, representatives, and staff of implementing agencies, and development partners;7 and (iii) the collection of energy efficiency data.

7. The evaluation mission had to conduct virtual meetings with government officials and counterpart staff because of travel restrictions imposed by the coronavirus disease (COVID-19) pandemic. This saved mission costs and reduced the effort provided by all participants of the virtual meetings. However, interviews were sometimes affected by technology-related issues, and, more importantly, participants were unable to provide all the information requested. The evaluation mission communicated a country-specific questionnaire in advance, but participants were not always prepared with answers, particularly for the data requirements, such as annual electricity consumption for various consumer categories and counterpart contributions for pilot projects. The mission could not interview the counterpart staff who had supported the TA projects because of staff turnover since the completion of the second project in 2015. Representatives of Cook Islands preferred to discuss and share project information in face-to-face meetings with the national consultant engaged by the evaluation mission instead of participating in virtual meetings with the full evaluation mission and no meeting could be held with the PNG government because the evaluation mission could not receive a timely concurrence.

B. Technical Assistance Project Objectives 8. Consultations for the regional Promoting Energy Efficiency in the Pacific program began in 2007. The objective was to address Pacific countries’ high dependence on imported petroleum for power generation and their vulnerability to adverse impacts of climate change. Implementation of the first project started in 2008 with the goal of improving energy security in the five participating PDMCs by promoting energy efficiency policies and strengthening project implementation capacity. A reduction in demand-side energy would avoid diesel-based power generation and reduce GHG emissions. A summary of the intended impact, outcome and outputs of the two TA projects is provided in Table 1. The design and monitoring frameworks (DMFs) are included in Appendix 1. The Global Environment Facility (GEF) support was based on the DMF for the second project, and had the objective of improving energy security, making energy more affordability, and reducing GHG emissions from the power sectors of the five PDMCs.

5 ADB. 2020. Evaluation Approach Paper for Technical Assistance Performance Report for Promoting Energy Efficiency in the Pacific.

Manila. 6 ADB. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila. 7 Development partners included Australian Government Department of Foreign Affairs and Trade, Japan Bank for International

Cooperation, New Zealand Ministry of Foreign Affairs and Trade, and The World Bank.

Introduction 3

Table 1: Intended Impacts, Outcomes, and Outputs Promoting Energy Efficiency in the Pacific Promoting Energy Efficiency in the Pacific (Phase 2)

Impact Mature energy efficiency applications reduce growth of commercial energy consumption and improve energy security.

Impact Reduction in fossil fuel use by the power sector without a corresponding reduction in energy services in the five participating PDMCs.

Outcome PDMCs, ADB, and GEF agree on joint ADB and GEF energy efficiency project, which will fund the development of capabilities and energy efficiency improvements in the industrial, commercial, residential (including poor urban and rural households), and public sectors.

Outcome Consumers use power efficiently in the five participating PDMCs.

Outputs 1. Assessment of the energy efficiency policy and

regulatory framework and past energy efficiency assistance in the Pacific, with dissemination of lessons.

2. Energy efficiency policy and regulatory frameworks recommendations for adoption in the five PDMCs.

3. Establishment of structured energy management systems to sustain energy efficiency initiatives, including possible ESCOs.

4. Training needs assessments and services and other assistance with regulatory and institutional reform to promote a viable market for energy efficiency services in each country.

5. Pipeline of assistance projects for funding by ADB, GEF, or other relevant financing sources.

6. A strategy for public awareness raising and education.

Outputs 1. Stakeholders have access to comprehensive

information on energy use by sector and appliance. 2. Energy efficiency practices mainstreamed into

government processes, procedures and policies in the participating countries.

3. Energy efficiency programs implemented effectively and sustainably in each participating country.

4. Information on energy efficiency shared and public awareness of the benefits of energy saving improved.

5. Effective project management established.

ADB = Asian Development Bank, ESCO = energy service company, GEF = Global Environment Facility, PDMC = Pacific developing member country. Sources: Asian Development Bank Technical Assistance Reports.

C. Technical Assistance Completion Reports 9. Promoting Energy Efficiency in the Pacific. The TCR for the first project self-assessed it successful. The TA inputs were satisfactory for the prescribed activities. The participation of national energy efficiency experts helped capacity development and knowledge transfer. TA consultants assessed the policy and regulatory framework and made recommendations for improvements. A lack of baseline data for demand-side energy indicated the need for surveys to reveal which electrical appliances were being used by households and commercial establishments. During the inception mission, it became evident that the small market size of the five PDMCs meant that it would not be viable for commercial and industrial consumers to outsource the design and implementation of energy efficiency measures to energy service companies (ESCOs). The scope of the first project was expanded to include an energy efficiency demonstration project in each participating PDMC. The TA also developed a pipeline of energy efficiency projects for the PDMCs, which led to the approval of the second project with cofinancing by the Asian Clean Energy Fund established by the Government of Japan under the Clean Energy Financing Partnership Facility, GEF, and the Australian Government.8

8 ADB. 2011. Technical Assistance Consultant's Report: Promoting Energy Efficiency in the Pacific (Phase 1). Manila.

4 Promoting Energy Efficiency in the Pacific

10. Promoting Energy Efficiency in the Pacific (Phase 2). The TCR for the second project self-assessed it successful.9 The GEF Independent Evaluation Office (GIEO) reviewed the completion report and its overall rating was moderately unsatisfactory.10 In the absence of data on appliance use, surveys helped create an energy demand-side database that met minimum requirements (output 1); national energy efficiency targets were established and suitable knowledge products were prepared for commonly used appliances (output 2); 34 energy efficiency pilot projects were implemented instead of the country-wide energy efficiency measures that had originally been envisaged (output 3); and public awareness was raised and knowledge disseminated (output 4). The TCR for the second project recommended that ADB should carry out measurement and verification (M&V) of energy savings from the pilot projects, explore policy instruments to address energy efficiency market failures, and experiment with innovative financing mechanisms for commercial investments in energy efficiency measures.11

D. Structure of the Evaluation Report 11. The TPER has five chapters. Chapter 2 summarizes the design and implementation features of the Promoting Energy Efficiency in the Pacific program. Chapter 3 assesses the performance of program based on its relevance, effectiveness, efficiency, and sustainability. Chapter 4 assesses the program’s development impacts and the performance of ADB and the implementing agencies. Chapter 5 summarizes the evaluation, identifies issues, highlights lessons learned, and provides recommendations.

9 The rating in the TCR for the second project was partially successful based on earlier ADB guidelines. 10 According to the GEF rating scale, “moderately unsatisfactory” meant that the level of outcomes achieved was somewhat lower

than expected and/or there were significant shortcomings. GEF. 2017. Guidelines for GEF Agencies in Conducting Terminal

Evaluation of Full-sized Projects. Washington. 11 ADB. 2016. Technical Assistance Completion Report: Promoting Energy Efficiency in the Pacific (Phase 2). Manila.

CHAPTER 2

Design and Implementation 12. This chapter provides information on (i) the rationale for the Promoting Energy Efficiency in the Pacific program, (ii) implementation arrangements in terms of time, costs, financing, and executing and implementing agencies; and (iii) the role of the consultants. It also summarizes changes in the design during implementation and arrangements for monitoring and reporting.

A. Rationale 13. Regional context. The PDMCs are among the world’s smallest and most remote states, with small and isolated populations. These states have generally not been gifted with the physical resources, size, location, or populations that would enable them to achieve and maintain inclusive growth without external support. Their high exposure to the impacts of climate change and natural disasters, and their small size and narrow production base make them extremely vulnerable to economic shocks. These circumstances increase the cost of production, limit private sector development, diminish the potential for export-led growth and job creation, and increase the cost of public administration and service delivery. 14. Energy sector context. The preparation of the first project coincided with the international price of crude oil reaching $145 per barrel in July 2008. Some PDMCs had indigenous commercial energy resources, but only a few had a well-developed energy supply chain and distribution network. Most PDMCs were heavily dependent on imported petroleum fuels for commercial energy and power generation, despite the increasing use of renewable energy resources. Consequently, electricity services based on sustainable resources were available to only a minority of the urban populations. For people with access to electricity and motorized transportation, imported petroleum products accounted for approximately 80% of primary energy consumption. Half of the imported petroleum products were consumed by transport (land and maritime) while 40% was used by diesel-fired power generation plants. 15. The cost of providing electricity in the region is still very high, because consumers are geographically dispersed over several small islands and domestic markets are small, in terms of consumer numbers and electricity use. Electricity tariffs in PDMCs’ capital cities are significantly higher than in Asia and neighboring industrialized countries, such as Australia and New Zealand, which affects the competitiveness of businesses, including tourism. In rural and outer island areas, the cost of the electricity supply is even higher.12 Despite the high cost, electricity consumption per capita is likely to increase further as incomes rise and Pacific economies modernize. In developing economies, there is a positive correlation of commercial energy use with real income and lifestyle quality; higher energy use is associated with a lower poverty rate. Lowering energy costs will promote higher use but, since nearly all energy is imported and fossil-fuel dependent, this will adversely affect the balance of payments. Increasing demand-side energy efficiency offers an alternative strategy as it allows consumers to gain the same service while lowering energy use. For example, two similar sized refrigerators providing identical

12 According to the benchmarking report, the average cost for commercial consumers was about $4,560 for consuming 12,000

kWh in 2017. (Pacific Power Association. 2018. Pacific Power Utilities Benchmarking Report for 2017 Fiscal Year. Suva). The estimated average cost of electricity for a small and medium enterprise in South Australia, which has the highest electricity tariff in Australia, based on October 2019 retail tariff, was AUD4,764 ($3,334) for consuming 12,000 kWh (Energy Consumers Australia. 2019. Analysis of Small Business Retail Energy Bills in Australia, Final Report, December 2019. Sydney). That is, businesses in PDMCs, were paying 37% higher than the counterparts in South Australia, or 81% higher than those in New South Wales or Victoria.

6 Promoting Energy Efficiency in the Pacific

service may be either 2-star rated (consuming 542 kilowatt-hour per year [kWh/year]) or 4-star rated (consuming only 318 kWh/year), 59% less electricity. 13 An increase in the use of renewable energy resources, in particular solar and wind, is another strategy for reducing dependence on imported fuel for power generation. 16. Resilient development. PDMCs are extremely vulnerable to climate change and climate hazards. Climate change predictions include very hot days and warm nights, extreme rainfall events, higher intensity of tropical cyclones, sea level rise, and ocean acidification. PDMCs are highly exposed to natural disasters of hydro-meteorological origin (e.g., droughts, cyclones, and landslides), and geological origin (e.g., volcanoes, earthquakes, and tsunamis). Actions in response to climate change and disaster risks have to be taken in numerous sectors, including health, education, transport, energy, fisheries, tourism, environment, and transport. The Framework for Resilient Development in the Pacific produced by the Pacific Community in 2016 identified priority actions for enhancing resilience to climate change and hazards while contributing to sustainable development. Its three interrelated goals were (i) strengthened integrated adaption and risk reduction for enhancing resilience; (ii) low-carbon development; and (iii) strengthened disaster preparedness, response and recovery.14 In the context of low-carbon development, the Pacific region contributes only 0.012% of world’s total carbon dioxide (CO2) emissions yet it is among the most vulnerable regions to the effects of climate change. Increasing energy efficiency is cost-effective as the pay-back period of investments for most measures is 2–5 years. Lowering energy consumption in households, buildings and transport, increases energy security and self-sufficiency. Priority actions by PDMC governments in the Framework for Resilient Development in the Pacific included the development and enforcement of energy efficiency codes for buildings, energy efficiency standards for imported electrical appliances and goods, and the development and adaptation of technical standards for new technologies for promoting energy efficiency by the private sector. 17. Policies and regulations in participating PDMCs to promote energy efficiency. In Cook Islands, the National Energy Policy, 2003 included articles to promote energy efficiency and conservation practices for all consumers and the National Sustainable Development Plan, 2007 included the objective of decreasing energy consumption per capita by 20% by 2010. However, there was little reduction in energy use, as the energy efficiency program relied on donor funding and regional assistance; government departments and the power utility lacked the capacity to deliver such a program. In PNG, less than 10% of the population had access to electricity services, so the main focus has been on expanding the power sector. ADB advisory TA supported the preparation of a national electrification roll-out plan but energy efficiency was given a low priority.15 However, the National Energy Policy, 2017–2027 recognized the importance of energy efficiency and outlined suitable strategies for promoting it.

18. In Samoa, the National Energy Policy, 2007 incorporated energy efficiency components, but little practical progress was made. Excise duties and other taxes did not differentiate between efficient and inefficient appliances, which discouraged changes in purchasing habits. In 1996, Tonga started to intensify efforts to promote the use of renewable energy resources and energy efficiency. The Tonga Energy Road Map, 2010–2020 included options for reducing energy demand. The Electricity Act, 2007 restructured the power sector and empowered the electricity commission to regulate the sector. Vanuatu prepared a National Energy Policy Framework in 2007 that specifically required energy providers and equipment suppliers to raise public awareness of the benefits of energy efficiency measures and provided duty exemptions for imports of energy efficient appliances.

19. Strategic context. Both the ADB Strategy 2020 and the ADB Energy Policy, 2009 called for the promotion of energy efficiency through supply-side and demand-side measures under the core operational area of infrastructure. At the regional level, the ADB Pacific Strategy, 2004 identified ADB’s

13 The Energy Rating Label. How the Stars Help You; downloaded on 26 November 2020 from https://www.energyrating.gov.au 14 Pacific Community (SPC). 2016. Framework for Resilient Development of the Pacific. An Integrated Approach to Address Climate

Change and Disaster Risk Management (FRDP) 2017–2030. Suva. 15 ADB. 2012. Technical Assistance to Papua New Guinea for Implementation of the Electricity Industry Policy. Manila.

Design and Implementation 7

most important contribution and primary mode of assistance to be knowledge products, provided mainly through grant-financed TA. It also identified energy efficiency, together with the integration of renewable energy, as a priority response to Pacific environmental issues.16 The ADB Pacific Approach, 2009 included the development of energy sectors in PDMCs as an operational priority. Its framework recognized the adverse environmental impacts and high costs of diesel-based electricity supplies and suggested the use of more energy efficient technologies for achieving the twin objectives of economic growth and environmental sustainability.17 Upscaling energy efficiency in the Pacific (the second project), with GEF co-

financing, was included in the indicative assistance pipeline in the regional operations business plan:

Pacific, 2010–2013.18

20. Instruments for assistance. The regional TA grant modality was used for the Promoting Energy Efficiency in the Pacific program as ADB’s objective was to extend consulting services for knowledge and support.19 ADB also wanted to carry out activities in five participating PDMCs, which was not possible if the project modality was used. A design change was made during the inception of the first project to allow implementation of five demonstration projects to show the benefits from actual energy efficiency projects. A limit of about $250,000 was agreed for the grant assistance for the demonstration projects. During preparation of the second project, support from GEF, Asia Clean Energy Fund, and the Government of Australia became available for the implementation of the pilot projects. The second project also used the TA grant modality, possibly because funds from cofinancing partners could cover most of the equipment costs of the energy efficiency projects, with utilities and PDMC governments paying the balance. While it addressed TA disbursement issues, the modality was a compromise that allowed implementation of energy efficiency projects across five participating PDMCs.

B. Time, Cost, Financing, and Implementation Arrangements

21. The total cost and sources of TA funding for the Promoting Energy Efficiency in the Pacific program is given in Table 2.

Table 2: Total Cost and Funding Sources of Both Technical Assistance Projects

Item Promoting Energy Efficiency in the Pacific

Promoting Energy Efficiency in the Pacific

(Phase 2) Total Estimated Cost $1,900,000 $12,421,545 TA Amount $1,400,000 $8,754,545 Sources of Funding: TA Special Fund 0 $1,000,000 Clean Energy Fund a $1,200,000 0 Asian Clean Energy Fund b 0 $1,500,000 GEF $200,000 $5,254,545 Government of Australia 0 $1,000,000 Counterpart Resources c $500,000 $3,667,000 Sources of Counterpart Resources: Participating PDMCs $500,000 $2,047,000 Power Utilities in Participating PDMCs 0 $1,620,000 Total Actual Disbursement $1,346,283

(96%) $7,149,507

(82%) Executing Agency PARD PARD

ADB = Asian Development Bank, GEF = Global Environmental Facility, PARD = Pacific Department, PDMC = Pacific developing member country, TA = technical assistance. a Contributed by governments of Australia and Norway under the Clean Energy Financing Partnership Facility. b Established by the Government of Japan under the Clean Energy Financing Partnership Facility.

16 ADB. 2004. Responding to the Priorities of the Poor: A Pacific Strategy for the Asian Development Bank 2005–2009. Manila. 17 ADB. 2009. ADB’s Pacific Approach (2010–2014). Manila. 18 ADB. 2010. Regional Operation Business Plan for Pacific 2010–2013. Manila. 19 Known as policy, advisory and capacity development when the TAs were approved.

8 Promoting Energy Efficiency in the Pacific

c In-kind contributions in the form of counterpart staff, office accommodation, communication, local transport, consultations, awareness facilitation, and other energy efficiency initiatives. Source: ADB. 2011. Technical Assistance Completion Report: Promoting Energy Efficiency in the Pacific. Manila; and ADB. 2016. Technical Assistance Completion Report: Promoting Energy Efficiency in the Pacific (Phase 2). Manila.

1. Promoting Energy Efficiency in the Pacific

22. International crude oil prices passed $100 per barrel in early 2008, threatening PDMCs with adverse trade balances as result of the imported petroleum fuel they needed for power generation. The cost of fuel crowded out other imports and lowered real incomes, intensifying poverty. Given the importance of sustainable development, the Pacific state leaders committed themselves in 2005 to implementing national policies and programs to promote an optimal energy mix and energy efficiency.20 23. The expected outcome of the first project was an agreement for the implementation of a joint ADB, GEF, PDMC project to developing energy efficiency capabilities and improvements in the industrial, commercial, residential (including poor urban and rural households), and public sectors of PDMCs. The expected outputs of this knowledge and support TA, according to its DMF, included: (i) an assessment of energy efficiency policy, regulatory framework and past assistance; (ii) a recommendation for an energy efficiency policy and regulatory framework in the five participating PDMCs; (iii) a recommendation for a structured energy management system for sustained energy efficiency initiatives, including the possible establishment of ESCOs; (iv) an assessment of the training and assistance needed for regulatory and institutional reforms for promoting energy efficiency services; (v) preparation of a pipeline of energy efficiency projects for funding by ADB, GEF and/or other development partners; and (vi) preparation of a strategy for raising the public awareness. 24. The estimated cost of the first project was $1,900,000 and it was funded by Clean Energy Fund under the Clean Energy Financing Partnership Facility ($1.2 million) and a GEF project preparation grant ($0.2 million).21 The Clean Energy Fund is a multi-partner trust fund that is administered by ADB; its resources are used to improve energy security in ADB developing member countries and to decrease the rate of climate change. The governments of Australia and Norway had contributed to the Clean Energy Financing Partnership Facility. In addition to the TA funds, it was estimated that the five PDMCs (Cook Islands, PNG, Samoa, Tonga, and Vanuatu) would provide an in-kind contribution of $100,000 equivalent each towards the cost of counterpart staff, office accommodation, communication, and local transportation. The amount of TA funds disbursed was $1,346,283 (96% of the approved amount). Of this amount, the disbursement for consultant remuneration and out-of-pocket-expenses was $1,090,837 (81.0% of total disbursement), equipment for demonstration projects was $253,680 (18.8%), and training was $1,766 (0.1%). The budget of $50,000 for TA administration remained unutilized. The in-kind contribution by PDMCs was not assessed at TA completion.

25. In addition to the TA fund, implementation of the five demonstration projects involved cost sharing by the beneficiaries and power utilities. 22 In Cook Islands, Rarotonga residential consumers contributed 50% of the cost of compact fluorescent lamps (CFLs), totaling about $19,200; in PNG, the power utility provided $339,000 for power factor correction equipment; in Samoa, the power utility provided $25,400 for power factor correction equipment; in Tonga, the power utility provided $32,500 and the installation cost for light-emitting diode (LED) street lights; and in Vanuatu, a hotel owner provided $23,508 for the energy efficiency measures. The total counterpart financing for the demonstration projects was $440,608 (in addition to the $253,680 disbursed for equipment from the TA Special Fund).

20 Pacific Islands Forum Secretariate (PIFS). 2005. The Pacific Plan for Strengthening Regional Cooperation and Integration. Suva. 21 The GEF grant supported the preparation of pilot EE projects; GEF approved it after ADB’s approval of the first project (para. 26). 22 ADB. 2011. Technical Assistance Consultant’s Report for Regional: Promoting Energy Efficiency in the Pacific (Phase I) Prepared

by Econoler International, Canada. Manila.

Design and Implementation 9

26. The first project was approved and became effective on 12 September 2008.23 The consultants were mobilized in April 2009 and the inception report was completed on 14 August 2009. A large pipeline of energy efficiency projects was identified in the inception report and this was the basis for the approval of the GEF project preparation grant ($200,000) on 1 March 2010. Consequently, a minor change in TA scope and implementation arrangements, together with an extension of the TA completion date from 31 December 2009 to 10 September 2010, was approved on 13 April 2010. The original project design included the preparation of an energy efficiency upscaling project, but the resources for this only became available after the approval of GEF’s project preparatory grant, 2 months after the originally planned 16-month implementation period. The same consultant was awarded the additional task of energy efficiency program preparation on a single source selection basis. The completion date was further extended to 22 April 2011 to enable the complete preparation of the energy efficiency program and to manage the GEF approval process (for the second project). A final extension to 31 August 2011, which was the actual completion date, was approved to complete the required revisions in the draft final TA consultant’s report and closing the contract. 27. The executing agency for the first project was the Transport, Energy and Natural Resources Division in the ADB Pacific Department (PARD). In each participating PDMC, an implementing agency was designated to facilitate the collection of the country’s energy-efficiency-related information. A steering committee provided overall supervision of the TA activities to ensure there was adequate cooperation between governments, power utilities, regional organizations, development partners and other stakeholders.

2. Promoting Energy Efficiency in the Pacific (Phase 2) 28. PDMCs are highly vulnerable to the impacts of global warming and climate change and they have supported a global reduction of GHG emissions and of their own emissions. Development partners have supported the implementation of highly visible renewable energy projects but have seldom endorsed energy efficiency projects. The same five PDMCs (Cook Islands, PNG, Samoa, Tonga, and Vanuatu) participated in the second project after the completion of the first. The second project was designed to promote the use of new energy efficiency technologies and to help PDMCs overcome the barriers that compromised energy efficiency adoption, such as insufficient awareness of energy efficiency technologies, a lack of local energy efficiency projects to demonstrate the benefits, inadequate institutional capacity, insufficient supporting policies and regulations, and limited political commitment and financial support. 29. The TA impact was envisaged to be a reduction in fossil fuel use by the power sector while sustaining the same level of energy services in the five participating PDMCs. The envisaged outcome was that end consumers would be using power more efficiently. The planned TA outputs were: (i) comprehensive information made available on energy use for sector and appliances that stakeholders could access; (ii) energy efficiency practices mainstreamed into government processes, procedures, and policies; (iii) an energy efficiency program implemented effectively and sustainably; and (iv) energy efficiency information disseminated and public awareness of the benefits of energy savings improved.24 The DMF identified TA performance indicators and target dates. Completion of the knowledge products—outputs (i) and (ii)—was scheduled for 31 March 2013; implementation of the energy efficiency program and its information sharing components—outputs (iii) and (iv)—and consumers using electricity efficiently—the project outcome—were also to be completed by 31 March 2015; and a reduction in fossil fuel use—the project impact—would be completed by the end of 2018. For monitoring progress, the DMF mentioned several data sources that were not being prepared at the time the TA project was approved,

23 TA approval and effectiveness dates are same for regional TA as separate endorsements are not required from the governments

of participating developing member countries. 24 The DMF mentioned effective project management as a separate output and the target was timely and within-budget

implementation of the TA program. A separate assessment of achievement of this output was unnecessary as the quality of project management is reflected in the assessment of other outputs.

10 Promoting Energy Efficiency in the Pacific

including a survey of monthly electricity bills, energy efficiency road map implementation reports, and annual household appliance surveys.

30. The estimated cost of the second project was $12,421,545. The approved TA amount was $8,754,545 to be funded by the TA Special Fund ($1,000,000), GEF ($5,254,545), Government of Australia ($1,000,000), and the Asian Clean Energy Fund under the Clean Energy Financing Partnership Facility ($1,500,000).25 The five PDMCs were to contribute a total of $2,047,000 equivalent in the form of in-kind support,26 power utilities were to provide $1,620,000 in the form of in-kind parallel financing for energy efficiency program implementation and information dissemination—outputs (iii) and (iv). The TA report (footnote 4) stated that the five power utilities in the PDMCs had issued letters of support stating they would provide an in-kind contribution on a parallel basis. This arrangement was unusual as the power utilities were responsible only for the supply side of energy meters, whereas the energy efficiency program was implementing energy efficiency measures on the energy demand side, so the consumers had to meet part of the cost of the energy efficient products. 31. The disbursed amount of TA funds was $7,149,507 (82% of the approved amount). Of this amount, disbursement was as follows: consultant’s remuneration and out-of-pocket expenses, $6,071,725 (84.9% of the total disbursement); equipment for energy efficiency pilot projects, $771,754 (10.8%); training and seminars, $216,752 (3.0%); studies, $89,101 (1.2%); and TA administration fees $176 (less than 0.1%). Details of the studies were not included in the TCR or the TA consultant’s report. The budget of $1,962,070 for equipment, contingencies and TA administration remained uncommitted during TA implementation. According to the GIEO review report, the unutilized GEF fund was about $1.60 million, i.e., nearly all the undisbursed TA amount.27 According to the TA consultant’s report, the total cost of the 34 pilot projects was estimated to be $1,836,184, which suggested that the counterpart funding was 238% of the TA disbursement for equipment.28 The basis for cost sharing between the TA financing and beneficiaries was not explained in the TA consultant’s report except that a cap was provided for the amount of TA financing to be allocated to each pilot project. However, the evaluation mission was unable to obtain validated reports for the counterpart funds provided by the consumers, so the actual share is unconfirmed. 32. The second project was approved and became effective on 31 March 2011. The consultant was mobilized in November 2011. The implementation was originally planned to take place over a 4-year period ending on 31 March 2015, but the actual TA completion and account closing date was 23 October 2015. According to the DMF, procurement of lighting equipment for energy efficiency pilot projects was to be carried out from April 2012 to April 2013 and installation of the lights from April 2013 to April 2014. The energy audit of public and commercial buildings was scheduled to be completed by April 2013, but the time period for selection of beneficiaries, procurement, and installation of the equipment was not specified. The TA report did not specifically provide the time needed for M&V of energy savings; the consultant’s bid proposal included an M&V protocol in the work plan but the delay in procurement and installation did not allow sufficient time for it to be implemented for the energy efficiency pilot projects. 33. The ADB PARD, through its Energy Division, was the executing agency. The in-country implementing agencies of the participating PDMCs were: (i) Energy Department, Cook Islands; (ii) Department of Petroleum and Energy, PNG; (iii) Ministry of Natural Resources and Environment, Samoa; (iv) Tonga Energy Road Map Implementation Unit, Prime Minister’s Department, Tonga; and (v) Energy Unit, Ministry of Lands and Natural Resources, Vanuatu. In each country, a national steering committee

25 Asia Clean Energy Fund was established by the Government of Japan. It was administered by ADB on a parallel basis for the TA

costs incurred in PNG. 26 For cost of counterpart staff, office accommodation, communications, local transport, consultants, awareness facilitation, and

other EE initiatives. 27 GEF. 2016. Terminal Evaluation Report form, GEF Independent Evaluation Office. Washington, DC. 28 ADB. 2015. Technical Assistance Consultant’s Report for Promoting Energy Efficiency in the Pacific (Phase 2). Manila.

Design and Implementation 11

was set up to guide the implementation of the second project activities and to ensure proper coordination among implementing agencies and other organizations and stakeholders.

C. Role of Consultants 34. Promoting Energy Efficiency in the Pacific. An international consulting firm was engaged using a quality- and cost-based selection process to implement the first project.29 The estimated consulting time required was 30 person-months of international consultants and 52 person-months of national consultants. Provision was made for an assistant project manager to support PARD in the management and coordination of project activities. 35. Two international experts (an energy efficiency economist and policy specialist, and an energy efficiency institution and capacity-building specialist) were identified in the original contract to provide a total of 16 person-months consulting services (14.75 person-months in the field and 1.25 person-months home office). Five national experts were also identified to provide 40 person-months consulting services. Following the approval of the GEF project preparation grant, the contract amount was increased in April 2010 to include another two international experts (an energy efficiency specialist and an environmental specialist) to provide 6 person-months consulting services. A contract variation in January 2011 provided a total of 28.3 person-months consulting services (20.18 person-months in the field and 8.12 person-months home office) for the four international experts. 30 This indicated that there was no change in the team of international experts, but the ratio of field to home office work was lowered from 11.8 to 2.5. There was only a small change in the contribution by national expertsof the five experts identified initially, one was substituted, and the consulting services were increased from 40 to 42 person-months. 36. Promoting Energy Efficiency in the Pacific (Phase 2). The estimated consulting services to implement the second project was 191 person-months of international experts and 313 person-months of national experts. A program management firm (PMF) would provide 114 person-months of international experts and 240 person-months of national experts. Short-term technical specialists were recruited on an individual basis to provide 77 person-months of international and 25 person-months of national experts. ADB was to engage a national program coordinator for 48 person-months to assist in project and procurement management. 37. An international consulting firm was selected as the PMF to implement the second project using quality- and cost-based selection procedures. 31 The weight of the cost was reduced from 20% to 10% in order to give greater emphasis to the technical proposal. The initial contract was for 88.5 person-months in the field and 37.5 person-months in the home office; a total 126 person-months for eight international experts.32 The original provision included another 180 person-months for five energy efficiency technical national experts. A final contract variation order on 7 July 2015 elaborated the changes in the PMF’s team. Over the 4.6-year implementation period, the field effort had been expanded, with the addition of another four international energy efficiency experts (under the budget for short-term individual experts), two international experts were engaged to replace one of the original experts, and the team leader was replaced. In addition to the experts providing consulting services in the field, two more international experts provided consulting services in the home office. The total in the field was 49.4 person-months and the total in the home office was 200.5 person-months. The evaluation mission noted that the ratio of field to home office work of the international experts was lowered from 2.4 to 0.2. Consulting services by the

29 Econoler International, Canada. According to the firm’s website, it had extensive experience in design and implementation of EE

and renewable energy projects. 30 The evaluation mission could not access the breakdown for field and home office inputs for the international experts added in

April 2010. Further, the final effort at contract closing is expected to be the same or very close to that mentioned in variation order of January 2011.

31 International Institute for Energy Conservation, Thailand. According to the firm’s current brochure, the firm was founded in 1984 to dramatically increase the use of energy efficiency in developing countries.

32 Energy efficiency technical experts (6), energy policy and regulation expert (1), and energy survey and database expert (1).

12 Promoting Energy Efficiency in the Pacific

national experts marginally increased to 181.9 person-months and the number of national experts was increased. Three national consultants were replaced. It was unclear whether the services of short-term international experts, selected on an individual basis, were utilized according to the provision in the TA cost estimate.33 The low disbursement for TA administration suggests that a national program coordinator was not engaged by ADB.

D. Changes in Design 38. Promoting Energy Efficiency in the Pacific. During the inception mission for the first project, it was discovered that the participating countries lacked suitable databases covering the energy demand side. Existing policies and regulations did not give due importance to an energy efficiency strategy. The governments, utilities, and ADB agreed to focus on four adjusted outputs, (i) establishment of energy use baselines, as there was very little to assess in terms of energy efficiency policy, regulatory framework, past assistance and lessonsoutput (i) of the DMF, para. 23; (ii) recommendations for energy efficiency policy and institutions, as the PDMCs were at very similar initial stages of introducing energy efficiency measures, so a common framework was needed instead of country-specific detailed action plansoutput (ii); (iii) recommendations for future energy efficiency programs based on an analysis of energy use, identification of consumer categories and potential energy savings from a future energy efficiency programoutput (v); and (iv) implementation of a demonstration energy efficiency project in each of the five PDMCs to help the countries to understand the steps for implementing energy efficiency projects, including energy audits, energy efficiency technology selection, design, implementation, cost–benefit analysissubstituted outputs (iii) and (vi). Three approvals made minor changes in scope, 34 implementation arrangements and extensions of closing dates. However, the TA administration documents do not explain the approval process for narrowing the focus to just four outputs instead of the six mentioned in the DMF.

39. Promoting Energy Efficiency in the Pacific (Phase 2). Although the second project outputs were rephrased in the PMF’s report, the activities were carried out mostly as envisaged. The PMF (i) developed an energy use databaseoutput (i), para. 29; (ii) developed energy efficiency policies and proceduresoutput (ii); (iii) implemented energy efficiency pilot projects instead of country-wide energy efficiency programsoutput (iii) with significantly less coverage;35 and (iv) disseminated energy efficiency information and helped raise public awarenessoutput (iv). Two activities were not required. Under output (ii), the use of Pacific Appliance Labelling Standards meant there was no need to develop minimum energy performance standards (MEPS). Under output (iii), the pilot projects were not registered for support from Clean Development Mechanism because the small GHG emission reduction from pilot projects would not justify the high transaction costs. The decision to implement pilot projects instead of the energy efficiency program should have been properly explained because the DMF’s quantitative targets were linked to country-wide application of energy efficiency technologies; however, no explanation or record of Management approval could be found in the TA administration documentation shared with the evaluation mission.

E. Monitoring and Reporting Arrangements

40. PARD fielded regular TA review missions that visited the participating PDMCs in turn. At times, staff of the Regional and Sustainable Development Department (now the Sustainable Development and Climate Change Department) also joined these missions to assess the use of GEF funds. During the review

33 The evaluation mission compared the “statement of disbursements” against 112 vouchers for the second project and the “status

of TA by cost category (summary),” although the total of disbursed amount was the same, it could not be reconciled against contract numbers.

34 The change in scope was the inclusion of demonstration projects. ADB. Pacific Department. TA No. 6485-REG: Promoting Energy Efficiency in the PacificMinor Change in TA Scope and Implementation Arrangements. Memorandum. 29 September 2009 (internal).

35 Table 4 lists the seven energy efficiency measures included in the energy efficiency program identified in the first project.

Design and Implementation 13

missions, ADB staff guided the PMF, helped in securing government support for the activities and resolving implementation issues, and met some beneficiaries of energy efficiency pilot projects to obtain their feedback. The PMF submitted regular progress reports, which were generally good and described the completed and ongoing activities, work plans for the following reporting period, summary of meetings and implementation issues. Use of GEF funds added another layer of reportingfrom ADB’s Sustainable Development and Climate Change Department to GEF.36 The two consultants’ reports (together with appendixes) were elaborate and informative. COVID-19 related international travel restrictions prevented the evaluation mission from visiting the participating PDMCs so the TA-related information in this report is mostly based on the consultants’ reports.

36 ADB is one of the 18 agencies that can access GEF resources. Application for and use of GEF funds is centralized in ADB and

managed by the Sustainable Development and Climate Change Department.

CHAPTER 3

Performance Assessment 41. This chapter assesses the performance of the Promoting Energy Efficiency in the Pacific program based on four core criteria: (i) relevance to the development strategies of the governments of the five countries involved and of ADB, and to achieving the envisaged objectives; (ii) effectiveness of outcomes and outputs; (iii) efficiency of the utilization of resources; and (iv) sustainability of outcomes and outputs. 42. The evaluation generally considered the first project to be a feasibility study and the second to be assistance for the implementation of suitable policy and regulatory measures, and investment in the subsidized installation of energy efficiency equipment. Both projects supported knowledge transfer on energy efficiency technologies. The same five PDMCs participated in the two projects and shared a common goal: to mainstream energy efficiency measures in order to lower consumption of imported fossil fuel for power generation. This would lead to savings because of the lower lifetime costs of energy efficient lighting and electrical appliances. Accordingly, this chapter discusses the performance of the first project, the second project, and the Promoting Energy Efficiency in the Pacific program (the combination of the two projects). The achievements against the targets identified in the DMF are summarized in Appendix 1. The assessment also considered the conclusions in the GIEO review of the terminal evaluation report (footnote 27), ADB’s TCRs (footnotes 8 and 11) and the TCR supplementary appendix for GEF projects.37

A. Relevance 43. The Promoting Energy Efficiency in the Pacific program was aligned with the ADB Clean Energy Program, Energy Policy, and Pacific Strategy (para. 19). The five PDMCs prioritized resilient development that included lowering energy use and dependence on imported fossil fuel (para. 16). The results framework of the Pacific Approach adopted in 2009 sought greater use of renewable energy resources and envisaged support for climate change mitigation through energy efficiency improvements.38 Since energy use in the five participating PDMCs is a fraction of global consumption, the contribution of the project to CO2 emissions and the consequent impact on global warming would be negligible. The CO2 emissions of the five participating PDMCs in 2019 were estimated as 4.5 megaton (4.1 megaton from PNG), which was just 0.012% of the total global CO2 emission (38 gigaton).39 Notwithstanding the negligible contribution of the program to global warming, the PDMCs need to adapt to climate change because of the projected 40–80 centimeter rise in sea levels and the increase in the frequency of extreme weather conditions caused by global warming.40 Greater energy efficiency in the Pacific would: reduce fuel import costs, lower the risks associated volatile international oil markets, increase disposable household incomes, and decrease operating costs for businesses. Greater energy efficiency in the Pacific would support PDMC leaders in their demands that the global community should move toward net zero carbon emissions by 2050. Various regional organizations signed a framework for resilient development in the Pacific (footnote 14), which recognized that increasing energy efficiency measures would be more

37 ADB. 2015. Technical Assistance Completion Report: Promoting Energy Efficiency in the Pacific (Phase II). Manila. Supplementary

Appendix on GEF Projects. 38 ADB. 2017. Pacific Approach, 2016–2020. Manila. 39 Joint Research Center (European Commission). 2020. Fossil CO2 and GHG emissions of all world countries 2020 Report.

Luxembourg. https://edgar.jrc.ec.europa.eu/ downloaded on 23 September 2020. The annual country-level estimates include CO2 emissions from five sectorstransport, power, other industrial combustion, buildings and others (industrial processes, agricultural soils, and waste).

40 The Economist. 2020. A Grim Outlook. How Worse Weather will Disrupt Businesses and their Supply Chains. 17 September.

Performance Assessment 15

cost-effective than changing the source of energy for a low-carbon development path.41 Development partners have generally supported implementation of highly visible renewable energy projects but few that had measures for energy demand side, except for the assistance in formulating the Pacific Appliance Labelling Standards. 44. The International Energy Agency considers energy efficiency to be the “first fuel” for the clean energy transition and a resource that all countries possess in abundance. In the International Energy Agency member countries, 30 of the world’s large economies, it was estimated that the energy efficiency measures since 2008 had reduced nearly 20% of their energy use by 2018, an amount greater than the total energy consumption of India.42 When the first project was designed, the importance of energy efficiency was not sufficiently recognized. A review of 18 completed and ongoing projects by development partners in the Pacific region showed that, while most projects targeted an expansion of renewable energy to reduce fossil fuel use and half included energy efficiency elements, none provided stand-alone assistance for energy efficiency policy formulation or provided significant investment in energy efficiency measures. The main energy efficiency output was the development of energy labelling standards for Fiji, which are now available to other PDMCs. Of the five participating PDMCs in the Promoting Energy Efficiency in the Pacific program, Samoa was included in six previous projects with energy efficiency components and Vanuatu in one. For the remaining three (Cook Islands, PNG, and Tonga), this was their first project to promote energy efficiency. The Promoting Energy Efficiency in the Pacific program filled a critical gap in knowledge sharing and support.

45. The preparatory work under the first project covered a review of energy use, policies, institutions, regulations, and implemented energy efficiency demonstration projects as an in-country learning experience and to build support. The reports of three demonstration projects (para. 58) were useful for preparing pilot projects under the second project. The impact of an energy efficiency program would come from the potential for lower electricity use in government buildings, streetlights, residential lighting, energy efficiency measures in hotels, energy labeling and MEPS. The electricity saving from a complete country-wide roll out of the energy efficiency program in the five PDMCs was estimated to be 48,453 megawatt-hour (MWh) per year, while GHG emissions would be reduced by 31,177 tons of CO2 per year (Table 3). The TCR and this evaluation found the formulation of TA inputs (consulting inputs and budget) to be satisfactory, although the first project had an issue of quality at entry as the focus of the outputs had to be modified during inception. 46. The second project was built on the outputs of the first. It was appropriately designed to address the barriers for energy efficiency adoption (para. 28). An energy use database for each participating PDMC was created using available electricity supply data and appliance surveys, which helped identify the pilot projects. The second project also recommended energy efficiency policies and procedures, including building codes and energy audit practices. Information was disseminated through training programs and workshops. This evaluation found parts of the TA design and implementation of the supporting policy, regulatory and training structures for rolling out the energy efficiency program to be satisfactory. Regarding involvement from partners, although GEF and the governments of Japan and Australia supported the program, there was little evidence of regular coordination during implementation. 47. The critical design change during implementation of the second project was the decision to limit the project to 34 energy efficiency pilot projects instead of the country-wide energy efficiency program that had been originally envisaged. The potential energy savings identified in the DMF were ambitious (para. 45). The energy efficiency program under the first project had covered a substantial portion of residential, and public and commercial buildings, but under the reduced scope of the second project the estimated energy savings were only 3,411 MWh and 3,204 tCO2 or only 7% of the of the country-wide

41 The organizations that co-signed the framework were: Pacific Islands Forum Secretariat (PIFS), Pacific Community (SPC),

Secretariat of the Pacific Regional Environment Programme (SPREP) and United Nations Office for Disaster Risk Reduction (UNISDR).

42 International Energy Agency . 2020. Energy Efficiency Indicators Highlights 2020 Edition. Paris.

16 Promoting Energy Efficiency in the Pacific

energy efficiency programs. The M&V protocol was not carried out for the pilot projects. The exclusion of activities related to MEPS may have also slowed down the application of energy efficiency standards and labelling in some PDMCs. Another design shortcoming lay in the identification of the pilot projects: the second project could have included a pilot project for industrial energy use in PNG, such as waste heat recovery for power generation or process heat (the industrial sectors in the other PDMCs were too small to make significant contributions to energy savings).

48. The TA report for the second project (footnote 4) mentioned that the energy efficiency program would cover streetlighting; residential, commercial, and industrial lighting; and energy audits for hotels and nonresidential private and government buildings. The PMF and governments had flexibility in the selection of specific energy efficiency projects during TA implementation. However, the intended outcome and output targets were substantially reduced when only 34 pilot projects were selected. There is no record of the justification for this reduction in scope and there was underspending of TA funds. This was a significant lapse in TA administration: approval of a major change in scope is required if the scope of a program is significantly reduced. DMF outcome and output targets should have been realigned to the activities that were now being carried out by the PMF.

49. From an ADB perspective, the low quantitative outcome was not a major weakness. By design, TA helps developing member countries to implement projects, improve capacity, promote technology transfer, and stimulate regional cooperation. It does this by providing studies, surveys, workshops, seminars and training. When procurement of equipment is included in a TA, the intent is to support knowledge transfer and not to deliver stand-alone results. The design of the second project was innovative as support from the GEF (for Cook Islands, Samoa, Tonga, and Vanuatu) and the Asian Clean Energy Fund (for PNG) was combined to implement a large energy efficiency program across the five PDMCs. Although the TA project’s format was changed to include implementation of small energy efficiency pilot projects, the administration and implementation processes remained the same. The TA implementation procedure led to some procurement inefficiencythe second project consultant’s report mentioned that some large equipment packages were not used because of the cap for procurement using single source selection procedure. 50. The evaluation team was unable to assess the extent to which energy efficiency measures had been adopted in the participating PDMCs, as the governments had not adopted energy efficiency indicators nor tracked the purchase of energy efficiency lights and appliances. Similarly, the power utilities did not prepare electricity demand projections for different scenarios while adopting energy efficiency measures. The feedback during meetings with the governments and development partners was that energy efficiency lights, refrigerators, and air conditioners are now readily available in local markets, so the improvements that had been sought by the Promoting Energy Efficiency in the Pacific program happened anyway. However, the success of the program could not be quantified in the absence of supporting data. 51. The electricity tariff in PDMCs is higher than in most other countries (footnote 12), so customers have a strong financial incentive to save energy. The second project design was unable to address two barriers to the expected country-wide roll out of the energy efficiency program. First, the TA funds allocated to energy efficiency equipment were only about 5% of the investment needed for the energy efficiency program ($2.4 million allocated against an estimated cost of $48 million), so consumers were not offered enough incentive for behavioral change; they continued to base their purchase decisions on front-end costs instead of on life-time costs. Second, there was an expectation that the PDMC governments would announce energy efficiency policies and regulations to transform the light and electrical appliance market by March 2013, or within 2 years of approval of the second project. However, these policies and regulations have taken much longer to be implemented than anticipated. Vanuatu’s Energy Efficiency of Electrical Appliances, Equipment and Lighting Products Act, 2016 became effective on 1 January 2018. Samoa adopted legislation for energy efficiency in August 2020; the government informed the evaluation mission that its enforcement (confiscation of non-standard appliances) was

Performance Assessment 17

expected to start in 2021. Tonga has prepared an Energy Efficiency Master Plan and a draft Energy Bill with actions to improve energy efficiency, which is expected to be submitted to the parliament in 2021. The design of the second project should have adopted a more realistic timeframe for these reforms. 52. The Promoting Energy Efficiency in the Pacific program filled a knowledge gap for promoting energy efficiency in the five PDMCs. Other development partners endorsed the design and provided cofinancing; ADB’s TA Special Fund contribution of $1 million was exceeded by commitments of $7.8 million from other sources. However, the choice of TA modality was a compromise so knowledge support and implementation of small energy efficiency projects could be combined in five PDMCs. The DMF indicators and targets were not helpful in measuring the extent of knowledge transfer as they required successful implementation of country-wide energy efficiency projects. The TCR for the first project assessed the formulation to be satisfactory, which is understood to represent a rating of relevant. The TCR for the second project did not explicitly assess relevance but the language suggested a highly relevant rating. During its review, GIEO downgraded its relevance rating to satisfactory. This evaluation assesses the first project relevant, but the second project and the Promoting Energy Efficiency in the Pacific program as a whole as less than relevant. The energy efficiency policy-related analysis, together with the demonstration and pilot projects, provided knowledge and experience sharing that were helpful for the adoption of suitable energy efficiency technologies in the five PDMCs. However, the quantitative energy use reduction targets identified in the DMF could not be achieved because of some design-related factors, e.g., use of TA modality, the reduction in scope of the investment component, and a failure to realign the quantitative targets.

B. Effectiveness 53. The TA outputs were modified during implementation of the Promoting Energy Efficiency in the Pacific program (paras. 38 and 39). ADB did not explicitly approve the change but agreed to contract variations and acknowledged periodic progress reports. Therefore, this evaluation has focused on the achievements of actual outputs delivered by the consultants. The targets mentioned in the DMF were used as reference. 54. The outcome of the first project was achieved. The two outcome targets were: submission of draft final consultant report by January 2010, and signing of a memorandum of understanding on further assistance by ADB and the five PDMC governments by March 2010. The draft consultant report was submitted a few months before September 2011, which was acceptable as the scope had been expanded to include the application process of GEF funds for the second project (approved in February 2011). A memorandum of understanding was considered unnecessary to finalize the proposal for ensuing the second project. The first project outcome was achieved. 55. The outcome of the second project was not achieved. The DMF target was a 10% reduction by 31 March 2015 in the average monthly electricity consumption in public buildings and by residential and commercial consumers. A baseline was given only for residential consumers (125 kWh per month). This target was inappropriate for the 34 pilot projects implemented under the second project as it required an assessment county-wide energy sales data. The power utilities did not publish data that would allow a change in monthly consumption to be calculated. Electricity sales data were not shared with the evaluation team (Appendix 3) to allow the trend to be determined. The savings from the 34 pilot projects were very small compared with the potential saving under the original country-wide implementation of the energy efficiency program. In the absence of an M&V protocol, the monthly energy savings could not be determined. Since energy efficiency policy and regulatory changes are yet to take effect in four of the five participating PDMCs, it is unlikely that the consumer purchasing behavior has changed sufficiently in favor of energy efficient lights and appliances to reduce the overall monthly electricity consumption by 10%. The issue of poor data collection was known but the energy use database (para. 60), which was one of the second project outputs, was also not achieved. Therefore, on the whole, the second project outcome was not achieved.

18 Promoting Energy Efficiency in the Pacific

56. The first project’s output 1 on establishing an energy use baseline in the five participating PDMCs was not achieved. The first project consultant’s report (footnote 22) provided an overview of key parameters across the five participating PDMCs for fiscal year 2009.43 Electricity was mostly used for lighting and air conditioners (although in PNG it was also used by large industrial consumers). Data were obtained from power utilities after persistent effort, but there was a lack of accurate, comparable and consistent time-series energy supply and demand data. An estimate of appliance use could not be backed up by data on appliance sales, and it was not possible to know how many appliances had been brought in by returning overseas workers. The output was not achieved since a bottom-up approach for estimating energy efficiency benefits still required extensive surveys, energy audits and measurements. 57. The first project’s output 2 on energy efficiency policy recommendations was achieved. The key capacity development component of the Promoting Energy Efficiency in the Pacific program was supporting PDMC governments to prepare and implement energy efficiency policies and regulations. The first project consultant reviewed the status of energy policies, regulations, institutions, and stakeholders’ capacity. The scope of past and ongoing assistance (generally training and capacity building) by development partners was also reviewed. The Australian Greenhouse Office began providing assistance in the late 1990s for Fiji energy standards and its labelling scheme, the precursor for MEPS. Generally, past projects had devoted limited time and effort to promoting energy efficiency and any improvements achieved during implementation were lost within months of project completion because of changing priorities and staff movements. The role of appliance importers was highlighted, their participation in formulating and implementing energy efficiency policy was essential if local markets were to be transformed. Barriers to energy efficiency implementation were discussed in the consultant’s report, which noted that, even when an energy efficiency framework was in place, its implementation was limited to a couple of activities by the power utility. With all participating PDMCs at a similar initial stage of promoting energy efficiency on the energy demand side, the consultant prepared generic policy recommendations, which included (i) formation of a suitable government organization to design and manage a well-structured energy efficiency program, (ii) provision of demand-side management services by utilities and a reward mechanism for persuading consumers to use energy efficient products that would compensate for the revenue shortfall because of reduced electricity sales, (iii) energy efficiency awareness campaigns and sharing information on energy efficiency appliances, (iv) inclusion of energy efficiency in education curricula and training programs for energy audits, (v) elaboration and enforcement of MEPS, (vi) inclusion of energy efficiency in building codes and efforts made to ensure compliance, and (vii) including energy efficiency aspects in public procurement. Institutional changes were also recommended, including the establishment of an energy efficiency center for carrying out research, analysis and transformation of products and services, the appointment of energy managers for government buildings, and the establishment of an energy efficiency fund to lower the cost of energy efficiency products. These policy and institutional recommendations have not yet been fully implemented in the five PDMCs. In view of the small size of all the PDMCs other than PNG, it was recommended that the responsibility for promoting energy efficiency be shared with the private sector through associations of construction industry so that codes for “green” buildings, labelling, and MEPS for appliances could be adopted. The first project consultant’s report could have included a discussion on how the regional institutions had helped bridge the knowledge gap and lowering resource costs for small PDMCs. The TCR for the first project noted that the policy recommendations were generic, which reflected the similar conditions in the PDMCs. This evaluation agrees that generic policy recommendations were appropriate because of the similarity of the countries and the low penetration of energy efficiency measures in the five PDMCs. The first project consultant provided training for conducting investment grade energy audits in Vanuatu. The output was achieved. 58. The first project’s output 3 on demonstration projects for energy efficiency technologies was achieved. The original scope required the consultant to recommend structured energy management

43 For example, electricity generation and use, electricity tariffs, electrification rate, per capita electricity energy intensity of the

economy, economic structure, GDP per capita, share of electricity sales and average monthly electricity consumption by different consumer categories.

Performance Assessment 19

systems to sustain the energy efficiency program. This would generally involve a contract between an energy consumer and a professional ESCO, which would include an energy audit, recommendations for appropriate energy efficiency measures, preparation of necessary documentation for financing, procurement, implementation of the energy efficiency measures, and adoption of the M&V protocol. The ESCO’s profits would be linked to a share of the reduction in consumers’ energy costs. The transaction cost of such a system was high and commercial banks needed to price the risk appropriately. While the energy efficiency literature has discussed the ESCO model for many years, there are only a few professional service providers in developing economies. During the TA inception mission, as an alternative learning experience, it was decided that a demonstration energy efficiency project would be implemented in each of the five participating PDMCs. This would enable domestic experts and local agencies to understand the process and share the experience of preparing the documentation required. These energy efficiency demonstration projects would also give additional visibility to energy efficiency measures. Five demonstration projects were identified in consultation with the governments on the basis of their potential to save energy and on whether they used relatively new energy efficiency technologies. Three demonstration projects involved energy-use technologies: (i) replacement of 18,000 incandescent lamps with CFLs in residences in Cook Islands; (ii) installation of 109 light emitting diode (LED street lights) in Tonga; and (iii) reduction of energy use in a hotel in Vanuatu by replacing all incandescent lamps with CFLs, installing room key tag switches to avoid lights and appliances being left on when room was not in use, and installing six rooftop solar water heaters to supplement the existing gas water heaters. The projected annual energy savings were determined using billing data (where available) and by measuring electricity consumption. The demonstration projects implemented in PNG and Samoa involved the installation of power factor correction equipment (this measure was excluded from the second project). The consultant’s report for the first project estimated the aggregate annual saving from the three-energy demand-side demonstration projects to be 948 MWh of electricity, about 90% of which was from the distribution of CFLs in Cook Islands. The demonstration projects had some issues, including weak documentation. Training material could have been included for the demonstration project in Vanuatu, including specific formats for accessing financing from commercial banks, procurement, installation, and the adoption of M&V protocol. The TCR for the first project did not assess the demonstration projects and the evaluation team received mixed feedback on their present status. Residential energy efficiency lights are now mainstreamed in Cook Islands. Tonga used the experience from its demonstration project to roll out a country-wide LED streetlight program. However, some pole-mounted LED streetlights had water ingress in the sensor enclosures because of rust; technical specifications should have considered protection against the salty ambient conditions. The evaluation team acknowledges that the demonstration projects were useful for knowledge sharing and considers that this output was achieved. 59. The first project’s output 4 on a pipeline of energy efficiency projects for subsequent support was achieved. A pipeline of a future energy efficiency program was prepared using available data, information on energy use from suppliers of appliances, and discussions with stakeholders. The pipeline covered seven energy efficiency measures: energy efficiency in government buildings, streetlighting, the hotel sector, water pumping, use of CFLs in residences, and power factor correction (Table 3). The enforcement of MEPS would transform the market for lighting, refrigerators and air conditioners and contribute about 37% of potential annual energy savings. Power factor correction and water pumping were excluded from the second project as the potential savings were low. The potential total savings from the pipeline were 48,453 MWh per year, or about 4.5% of total electricity sales in the five PDMCs in 2013. The corresponding reduction in diesel consumption was estimated for each energy efficiency measure included in the energy efficiency program, and it was assumed that implementation of the program would displace diesel-based power generation in the PDMCs. The proposed energy efficiency program was included in the application of GEF funds for the second project. The consultant’s scope of work was expanded to include support for the successful processing of the application of GEF funds. The TCR for the first project considered that the output was achieved, and the evaluation team concurs.

20 Promoting Energy Efficiency in the Pacific

Table 3: Estimated Total Energy Savings and Greenhouse Gas Emission Reduction Per Energy Efficiency Measure for Participating Pacific Developing Member Countries

Energy Efficiency Measure

Total Investment ($ million)

Annual Energy Saving (MWh)

Annual Emission Reduction

(tCO2)

Simple Payback Period (years)

Government buildings 7.3 7,121 36,690 3.5 Streetlighting 8.1 4,396 23,011 5.5 Residential lighting 0.8 9,315 53,830 0.5 Hotel sector 24.6 8,786 44,190 4.0 MEPS and labelling 6.0 18,145 93,400 1.0 Power factor 0.8 423 2,480 5.7 Pumping stations 0.5 267 1,650 4.7 Total 48.1 48,453 255,251 …

… = data not available, MEPS = minimum energy performance standard, MWh = megawatt-hour, tCO2 = ton of carbon dioxide. Source: ADB. 2011. Technical Assistance Consultant’s Report for Regional: Promoting Energy Efficiency in the Pacific (Phase I). Manila.

60. The second project’s output 1 on energy use databases was not achieved. The PMF offered countries the design of a comprehensive energy-use database with many functions. However, power utilities had limited energy consumption data so extensive surveys had to be conducted for energy use by households and large consumers. The survey results were used to establish the key parameters of the energy efficiency program. The second project website included an energy-use database that provided a breakdown of energy use for lighting, air conditioning and household appliances. However, there were several shortcomings in this database: (i) it did not cover the policies and action plans of the five participating PDMCs; (ii) reporting and user interfaces were not included; (iii) a list of 34 pilot projects was included, but not project-specific details such as design, implementation and M&V protocol; and (iv) a data search function was not included. The TCR for the second project stated that the output had not been achieved. The second project website and the database were available only during TA implementation, as no provision was made for hosting the website or populating the data after TA closing. IED agrees that the output was not achieved. 61. The second project’s output 2 on energy efficiency policies and procedures was not achieved. Continuing the preparatory work carried out during the first project, the PMF provided several specific deliverables for energy efficiency practices to be mainstreamed into the governments’ processes, procedures, and policies. These included (i) preparation of national energy efficiency targets; (ii) energy efficiency building codes and best practices; and (iii) energy audit training and capacity building.44 An audit training program (four modules over 11 days) was implemented in all five PDMCs and was attended by 159 trainees. Several deliverables identified building energy efficiency technologies and best practices. However, two deliverables—developing MEPS and support for ESCOs—were not needed as Pacific appliance and labelling standards already existed and ESCOs were not commercially viable given the small size of energy efficiency markets in the PDMCs. However, the expectation was that the second project would enable the five participating PDMCs to introduce and enforce building codes and MEPS, which would make the highest contribution to the energy efficiency program’s overall energy saving and GHG reduction (Table 3). The coverage and quality of the deliverables were adequate, but their recommendations are yet to be mainstreamed in government processes. Accordingly, in the view of the evaluation the output was not achieved.45

44 The knowledge products are part of the PMF’s final report (footnote 28). The deliverable for codes and best practices was based

on global practices that were relevant to the PDMCs, and included an assessment of energy efficient technologies, best practices for new constructions (building codes), a green hotel rating scheme, a green buildings scheme and energy efficiency assessment guidelines (using assessment frameworks developed by the World Bank, the American Council for Energy Efficient Economy, and the International Energy Agency).

45 The evaluation considered the exclusion of deliverables related to ESCOs and MEPS to be a TA design issue.

Performance Assessment 21

62. The second project’s output 3 on pilot energy efficiency projects was not achieved. Instead of rolling out country-wide energy efficiency programs, as originally planned, the second project implemented 34 pilot projects in the five participating PDMCs. It was intended that these pilot projects would provide experience of using new technologies that would help achieve national energy efficiency targets (Appendix 2 lists the pilot projects in the five PDMCs). The pilot projects were selected after a review of energy consumption data and consultations with customers and utilities; the envisaged energy saving from the 34 pilot projects was only 7% of the projected savings from the country-wide adoption of an energy efficiency program. The estimated overall costs and benefits are presented in Table 4. Since 12 pilot projects had not been completed by the time the second project closed, the benefits could not be verified. The evaluation team was informed by the respective governments that most pilot projects had been successfully completed but that the M&V protocol of benefits had not been carried out. Feedback was provided on some of technical issues faced by the projects. Post-mounted solar hybrid LED lamps were installed in Rarotonga airport and Punanga Nui market in Cook Islands; at the time of the evaluation, five of the 16 lamps were not working, and they could not be repaired because of high rental cost of a long-boom crane. Another two pilot projects required the installation of T5 fluorescent lamps in public buildings and residences; however, the technology was superseded by LED tubes so the T5 fluorescent lamps were not used. There was a negative feedback on the solar water heaters installed in Rarotonga hospital, where the metal parts had rusted in the high-salt ambient conditions. Rusting was also observed in some inverter-type air conditioners installed in a resort. In Samoa, the solar hybrid air conditioners were not used because the glass panels of the solar water heaters broke. In Vanuatu, air conditioners using variable refrigerant flow were installed in the Ministry of Finance building but were destroyed by tropical cyclone Pam in March 2015 and all the fittings were scrapped. None of the participating PDMCs confirmed that the bulb eaters provided for safe disposal of used fluorescent lamps were being properly used, mostly because waste segregation was not practiced; Cook Islands informed the evaluation team that a local appliance distributor had installed a bulb eater that was regularly used. The estimated investment in the 34 pilot projects was $2.7 million, and the benefit was an electricity saving of 3,411 MWh per year, corresponding to an annual energy cost saving of $1.5 million and a reduction of GHG emissions by 3,206 tCO2 (Table 4). The indicative payback period was within 2 years, but in the absence of an M&V protocol, benefits could not be confirmed. The TCR for the second project assessed that the output had not been achieved, based on a comparison of the energy savings of the pilot projects to the target savings of 45,000 MWh that were estimated for the country-wide roll out of the energy efficiency program detailed in DMF.46 By the second project closing date, 66% of the pilot projects had been completed and the PMF provided comprehensive documentation of an ex-ante cost–benefit analysis, procurement, and installation. The estimated amount of diesel saved by each pilot project was included in the report, which will be useful for determining priorities for future resilient development projects. The output was a tangible contribution to a reduction in energy use in PDMCs, and the pilot projects were valuable as a learning and knowledge sharing product. However, seven pilot projects had maintenance issues or were no longer functioning. It is conservatively assessed that the achievement was less than 80%, so in the view of the evaluation mission the output was not achieved.

Table 4: Estimated Costs and Benefits of the 34 Pilot Energy Efficiency Projects

Item Estimated Cost

($)

Energy Saving

(MWh/year)

Energy Cost Saving ($/year)

Annual Diesel Saving

(liter/year)

Annual Emission Reduction (tCO2/year)

Cook Islands 671,100 525 321,285 153,312 471 Papua New Guinea 470,325 428 122,494 108,190 685 Samoa 528,567 954 350,024 250,987 762 Tonga 478,250 639 232,021 109,973 582 Vanuatu 578,240 865 433,098 249,001 706 Total 2,726,482 3,411 1,458,922 871,463 3,206

MWh = megawatt-hour, tCO2 = tons of carbon dioxide. Source: ADB. 2015. Technical Assistance Consultant’s Report for Promoting Energy Efficiency in the Pacific (Phase 2). Manila.

46 For an objective assessment, the same shortfall need not be linked to several outputs.

22 Promoting Energy Efficiency in the Pacific

63. The second project’s output 4 on information dissemination and public awareness was achieved. The activity included printing energy saving tips on the reverse of electricity bills (Cook Islands) and in brochures (PNG). Home energy guides provided information on the electricity costs of various appliances. Comprehensive energy efficiency guidelines were prepared and distributed in hotels as well as in commercial and public buildings. Three workshops were held in Vanuatu (2013), Tahiti (2014) and Samoa (2015) to share energy efficiency knowledge and experience. The TCR for the second project did not assess the achievement of this output. In the view of the evaluation the output was achieved.

64. The TCR for the first project assessed the first project effective while the TCR for the second project did not provide a rating for effectiveness. The GEF review gave a high weight to the shortfall in the GHG emission reduction target and assessed the second project “moderately unsatisfactory” as several critical outputs did not materialize (footnote 10). This evaluation assessed the first project effective as the outcome and three out of four outputs were achieved. It assessed the second project less than effective. Although the knowledge and support aspects were generally achieved, the quantitative outcome targets were not. Three out of four outputs were not achieved and the PDMCs have yet to mainstream all the energy efficiency supporting policies and regulations (Appendix 3). The evaluation also assessed the overall Promoting Energy Efficiency in the Pacific program less than effective.

C. Efficiency 65. Consultants were engaged for TA implementation on a competitive basis. The contract was awarded after an evaluation of technical and financial proposals submitted by firms with requisite experience; and ensuring that the technical proposal was of a high standard. For ranking the proposals, the weight of the financial bids was 20% (default) for the first project and was reduced to 10% for the second project as the scope of the policy advice and knowledge transfer was high. Both TA projects were implemented within budget: TA fund disbursement was 96% of the TA amount for the first project and 82% for the second. The procurement of energy efficiency lighting was very competitive and there was also a worldwide decline in prices, so the bids were 20%–30% lower than estimated. Funds allocated for equipment were underused: only $0.8 million was utilized out of the allocated $2.4 million. The evaluation team concluded that the lower fund utilization of the second project reflected the limited capacity of PDMCs to implement the Promoting Energy Efficiency in the Pacific program.

66. Implementation of the first project took 20 months longer than scheduled but this was mostly because the GEF grant was approved 14 months after the original TA completion date. The delay in the second project implementation was 7 months (15% of the 4-year scheduled period). The TCR for the second project did not mention the reasons for the delay. According to the PMF’s report, there was a 6-month delay because the responsibility for procurement of equipment was transferred from ADB to the consultant after the first 10 pilot projects. There was some delay in the allocation of government counterpart funds for the installation of pilot projects and the public awareness program. The major consequence of the energy efficiency equipment procurement delay was that 12 pilot energy efficiency projects could not be completed, and the M&V protocol was not carried out to confirm the corresponding energy savings. 67. There was no change in the team of international experts in the first project. The ratio of field to home office work was reduced from 11.8 to 2.5 (para. 34), which partly reflected the additional effort in preparing supporting documents for processing the application for GEF funds. There was only a minor change in the team of national experts. Implementation of the second project started with eight international experts and a provision to add four more international experts on an individual basis. Three substitutions were required over the 4.6-year implementation period, which was considered acceptable (para. 37). However, the significant reduction in the ratio of field to home office work (from 2.4 to 0.2) lowered the knowledge transfer output.47 There were three replacements in the team of national experts.

47 The number of return travel changed from 88 to 113 (28% higher), which also indicated more frequent shorter field trips.

Performance Assessment 23

The in-country implementing agencies were adequately staffed and the steering committees guided the TA activities. 68. The TCR for the first project assessed the first project efficient while the TCR for the second project did not explicitly rate its efficiency. GIEO valued the investment (TA funding disbursed, $7.15 million) against the GHG emission reduction with a replication factor of 1.2 over a 15-year period (105,732 tCO2). This was translated to a CO2 abatement cost of $67.5 per tCO2, a significantly high cost.48 Accordingly, GIEO assessed the efficiency of the second project unsatisfactory. The evaluation team prepared a conservative estimate of the internal rate of return (IRR) using the cost–benefit approach for the 34 pilot projects (Table 4). The key assumptions were: operation and maintenance cost to be 10% of the capital cost, an implementation period of 1 year between delivery of energy efficiency products and installation, and a life of 5 years for the energy efficiency products (mostly light bulbs and air conditioners), since they might become obsolescent. The calculated IRR was 33%. A more conventional approach to assessing viability would be to calculate the pay-back period, which was 1.9 years for the same investment. Unfortunately, the quantitative analysis of investment efficiency is based on forecasts of costs and benefits, as the actual completed costs of the 34 pilot projects, applicable taxes and duties, and verified benefits were not recorded at the completion of the second project. ADB guidelines differ from those of GEF and require only an assessment of process efficiency for TA projects (footnote 6). Therefore, the evaluation assessed the first project efficient as the TA funds were utilized well and the implementation delays were attributed to the implementation of the GEF grant. There was underutilization of TA funds in the second project, but the indicative IRR of the pilot projects was expected to be high, and the pay-back period was low. The evaluation assessed the second project and the Promoting Energy Efficiency in the Pacific program as a whole efficient.

D. Sustainability 69. The Promoting Energy Efficiency in the Pacific program was innovative as it provided knowledge and support for national energy efficiency programs. Previous externally assisted projects had contributed limited inputs to just two of the five participating PDMCs (para. 44). Energy use analysis showed the program introduced new energy efficiency technologies for lights, air conditioners, and refrigerators. The TCR for the first project noted that counterpart agencies in the PDMCs strongly supported the energy efficiency program and were satisfied with the inputs. Counterpart staff were involved in implementing the second project, the TCR for which noted that they considered the deliverables to be of an acceptable quality. In-kind support (footnote 26) was provided by the PDMC governments during the implementation of the program. The cost of the demonstration and pilot projects was shared by the beneficiaries, who contributed $0.4 million for the first project and $1.1 million for the second. 70. During implementation of the Promoting Energy Efficiency in the Pacific program, the participating governments showed a strong political commitment to promoting energy efficiency. Cook Islands prepared draft legislation that required sales of all appliances to comply with MEPS. The PNG National Energy Policy, 2017–2027 included a separate chapter for challenges and strategies for promoting energy efficiency.49 Samoa enacted energy efficiency legislation in 2017, while the Tonga Energy Road Map, 2010–2020 included action for promoting energy efficiency. Vanuatu enacted the Energy Efficiency of Electrical Appliances, Equipment and Lighting Products Act in 2016. The two main Pacific regional organizations, the Pacific Islands Forum and the Pacific Community, are also strong advocates of energy efficiency and renewable energy integration as means for resilient development (footnote 14). However, the energy efficiency program envisaged in the first project has not been implemented across the PDMCs because of the lack of regulatory action, particularly the enforcement of MEPS, which is being done only in Vanuatu. Despite the lack of regulations, the energy efficiency market is being transformed, inefficient incandescent lamps are no longer available, and appliance traders are

48 ADB guidelines for economic analysis value the global cost of carbon emission as $25 per tCO2 equivalent. 49 Department of Petroleum and Energy. 2017. National Energy Policy 2017–2027: Harnessing Energy for Life. Port Moresby.

24 Promoting Energy Efficiency in the Pacific

stocking 3- and 4-star rated refrigerators and air conditioners. As electricity tariffs remain high, households with high electricity consumption have a strong incentive to select energy efficiency products based on lifetime costs. 71. The TCRs for the two projects did not assess their sustainability. However, the TCR for the second project stated that the sustainability of its four outputs was positive in terms of their contribution to knowledge sharing and information dissemination. GIEO assessed the Promoting Energy Efficiency in the Pacific program’s sustainability to be “moderately likely” for four categories: financial, sociopolitical, institutional framework, and environmental. This evaluation assessed the first project most likely sustainable because it was innovative and identified energy efficiency technologies for energy saving. However, it rated the second project less than likely sustainable until all the governments start enforcing MEPS, energy efficiency markets are fully transformed, public awareness of the need for energy efficiency is increased, and energy efficiency products become more available. The evaluation’s overall assessment of the Promoting Energy Efficiency in the Pacific program was less than likely sustainable, based on the low level of adoption of recommended energy efficiency measures.

CHAPTER 4

Other Assessments 72. This chapter assesses the noncore evaluation criteria—the development impact of the Promoting Energy Efficiency in the Pacific program and the performance of ADB and the implementing agencies. The ratings for these criteria do not form part of the overall assessment.

A. Development Impact 73. The five PDMCs have committed themselves to resilient development (para. 16) and their Intended Nationally Determined Contributions submitted to the United Nations Framework Convention on Climate Change in 2015 included targets for lowering GHG emissions. These targets are conditional on countries receiving further support from the international community. Cook Islands generated 50% of its electricity from renewable energy sources in 2015 and committed to increase this to 100% by 2020. PNG committed itself to 100% electricity generation from renewable energy sources by 2030. Samoa established a target of generating 100% of its electricity from renewable energy sources by 2017, which it later extended to 2025 because of increasing electricity demand. Tonga has committed itself to generating 50% of its electricity from renewable energy sources by 2020 and to increase this to 70% by 2030. Vanuatu plans to approach 100% renewable energy for electricity generation and an energy saving of 15% through the implementation of energy efficiency measures by 2030. The evaluation noted that there was high access to electricity in three of the participating PDMCs: Cook Islands (99%), Samoa (100%), and Tonga (89%). By contrast, access to electricity was low in PNG (12%) and Vanuatu (33%).50 74. Although promoting energy efficiency in electricity demand is very important, the general approach of development partners has been to implement renewable energy integration projects that lower fossil fuel consumption and reduce GHG emissions in the Pacific, as the cost of most available renewable technologies, solar and wind power generation, has become more competitive. Recent ADB projects in the Pacific have included battery energy storage systems to address the intermittent nature of installed renewable energy sources. There are critical differences between the implementation of renewable energy projects and energy efficiency measures that reduce demand-side energy use. Renewable energy projects are generally implemented by a state or corporate agency, often the state-owned power utility, whereas the implementation of energy efficiency measures requires actions by households, commercial establishments, and industry. Further, a drop in a consumer’s bill will not necessarily reflect greater energy efficiency; an elaborate M&V protocol is necessary to account for external changes such as weather conditions, operating hours, and changes in air conditioner temperature settings. The transaction cost of implementing energy efficiency measures can be high and the development community has not yet established a suitable model for financing energy efficiency equipment costs that is based on future energy savings. While governments and consumers understand the benefits of energy efficiency measures, progress has been slow because few projects supported by development partners have included investments in energy efficiency measures. Appendix 3 contains a summary of electricity use in the five participating PDMCs (policies, regulations, institutions supporting energy efficiency) and the status of pilot energy efficiency projects under the Promoting Energy Efficiency in the Pacific program. Information was gathered through a questionnaire circulated to stakeholders.

50 ADB. 2019. Pacific Energy Update 2019. Manila.

26 Promoting Energy Efficiency in the Pacific

75. Energy efficiency measures are progressively implemented by transforming the energy market. For example, the cost of energy efficiency technologies has been dropping and energy efficient LED lamps have become more cost-effective than inefficient incandescent lamps in most countries. Customers have been purchasing more 4- and 5-star efficiency refrigerators and air conditioners as shop assistants explain the lifetime savings to them. The evaluation concluded that energy efficient refrigerators and air conditioners are being sold in Cook Islands, while in Tonga, the power utility used the experience of the pilot projects under the second project to install about 28,000 energy efficiency lights. The Vanuatu government reported that only energy efficiency lighting is now being sold in the domestic market and that the installation of solar water heaters is easier when new houses are built. Enforcement of MEPS, which was part of the policy recommendations of the second project, would have accelerated the transformation of domestic appliance markets in the PDMCs that are yet to transform. 76. The evaluation team learned that two projects for promoting energy efficiency had been financed by other development partners since the Promoting Energy Efficiency in the Pacific program. In 2016, the People’s Republic of China provided 28,500 energy efficient lights and 500 air conditioners for residences and public buildings in Tonga. An ongoing United Nations Development Programme project, to be completed in 2021, will include an energy audit of large industrial consumers in PNG. 77. Environmental impact and climate change. The long-term environmental impact of implementing energy efficiency measures was mostly positive as these changes reduced the pollution caused by electricity generation. However, old fluorescent lamps contained mercury, which had to be safely disposed of to avoid water contamination. The five PMDCs’ annual CO2 emissions of 4.5 megatons from fossil fuel use was insignificant in relation to global emissions (38.0 gigatons) so the program’s contribution to climate change was negligible. However, the program was anchored on reducing diesel use for power generation. The estimated annual diesel saving from the 34 pilot projects was 871,463 liters, leading to a consequent CO2 emission reduction of 3.2 tons CO2 (Table 4). Based on the National Pollutant Inventory, Australia, the reduction in emissions of local pollutants was estimated to be 11.4 tons of carbon monoxide, 26.9 tons of oxides of nitrogen and 1.5 tons of particulate matter with size less than 10 microns.51 A roll out of the full energy efficiency program could save over 10 times more diesel, but this would also depend on the penetration of renewable energy-based power generation in the future. 78. Energy efficiency measures implemented by the private sector. Demand-side energy efficiency measures are implemented mostly by the private sector, i.e., by households and by commercial and industrial consumers. Governments and utilities are directly involved in implementing demand-side energy efficiency measures only in public buildings and streetlights. Furthermore, private sector enterprises manage the trade in energy efficiency products and appliances. The Promoting Energy Efficiency in the Pacific program provided considerable knowledge by way of demonstration and pilot projects and through reports and brochures to highlight the benefits and short pay-back period of energy efficiency measures. The evaluation mission was provided with anecdotal evidence that the markets for lighting, refrigerators and air conditioners are being transformed. Electrical contractors had developed the capacity to carry out energy audits but, even when commercial enterprises engaged them to do this, the recommendations were not implemented, possibly due to weak incentives. Regulatory changes, including enforcement of MEPS and building codes, would encourage greater use of energy efficiency measures. 79. The DMF of the first project included an impact target of a 30% reduction in growth of energy imports in the five participating PDMCs. The impact targets of the second project were: (i) a 10% reduction in fossil fuel imports for power generation (2008 baseline, 135 million liters per year); (ii) a total saving of 45,000 MWh per year; and (iii) GHG emission reductions of 30,000 tCO2 per year. These targets had not been achieved at the closing of the program. This was because only 34 pilot energy efficiency projects with a total estimated energy saving of 3,411 MWh/year were implemented, instead

51 Department of Planning, Ausgrid. 2008. Air Quality Impact Assessment: The Integrated Generators Project. Sydney.

Other Assessments 27

of the original country-wide roll out of energy efficiency programs. The results were 13 times lower than if the original proposal had been implemented. Furthermore, the governments of the five countries did not provide data to the evaluation mission that would have enabled it to assess the achievement in fiscal year 2019. The main reason why fuel and electricity savings are expected to be far less than the targets is because MEPS and building codes were not being enforced in the participating PDMCs. 80. The evaluation assessed the Promoting Energy Efficiency in the Pacific program’s knowledge sharing and support satisfactory. Knowledge products to promote energy efficiency were provided and there was significant capacity building of government staff and other stakeholders. The quantitative targets identified in the DMF were not achieved because of delays in the implementation of energy efficiency policy recommendations. In the context of energy policy reforms, world oil prices probably influenced government priorities as the oil price dropped from a high of $145 per barrel in 2008 to $60 per barrel at the end of 2014. Energy efficient lights, refrigerators and air conditioners are becoming readily available at competitive prices and the domestic appliance markets is gradually transforming. Overall, IED assessed the development impact of the Promoting Energy Efficiency in the Pacific program less than satisfactory because of the non-achievement of the DMF impact targets.

B. Performance of the Asian Development Bank 81. The Promoting Energy Efficiency in the Pacific program was designed to fill a crucial knowledge gap on lowering fossil fuel use and supporting resilient development. ADB led the initiative and obtained considerable support from other development partners for the program. The design of the second project was innovative as 27% of the TA financing was allocated for equipment. However, the design under-estimated the time and effort needed to enforce the MEPS that were essential if the quantitative DMF targets were to be achieved. During program implementation, ADB fielded regular field missions and reviewed the consultants’ progress reports. The PMF report stated that there had been procurement delays because of ADB’s approval procedure, which was needed for the first 10 energy efficiency pilot projects. However, ADB’s TA administration was effective in the timely approval of several minor changes in scope, time extensions, and over 22 contract variation orders. ADB submitted the required reports to the GEF, including a detailed supplementary appendix for GEF projects (footnote 37). However, TA administration records did not make it clear when it became evident that the quantitative outcome targets would not be achieved; ADB did not investigate options for using surplus GEF funds or for further extensions of the second project closing date to allow the PMF time to carry out an M&V protocol of the benefits. The TCRs were based on earlier staff instructions so they did not include specific ratings for the evaluation criteria (the current staff instructions were issued in December 2019). The TCRs stated that ADB showed flexibility and assessed its performance satisfactory. This evaluation lowered the assessment of ADB overall performance to less than satisfactory, mainly because of the unexplained design changes, the large underutilization of TA funds, and the fact that the PMF was not required to carry out an M&V protocol of the pilot projects. A separate assessment of the executing agency was not needed as ADB’s PARD was the executing agency of the program.

C. Performance of Implementing Agencies 82. The participating PDMCs appointed in-country implementing agencies. The consultant’s report for the first project stated that the governments and power utilities lacked accurate and reliable data on energy use and had limited staff resources to promote the energy efficiency program. The TCR for the first project assessed the overall performance of the implementing agencies satisfactory, despite their lack of support for data collection. Counterpart funds were provided for the identification and implementation of the demonstration projects. The consultant’s report for the second project mentioned that the support received from the respective implementing agencies was adequate, although this was not assessed by the TCR. Since this evaluation was prepared over 5 years after completion of the program, the staff at the implementing agencies had changed. The evaluation mission was not able to meet any staff to discuss the TA projects, even virtually. However, informative discussions were held with

28 Promoting Energy Efficiency in the Pacific

government officials and power utility staff in Samoa, Tonga, and Vanuatu. Cook Islands had other ongoing priorities, so no virtual meetings were held; however, staff of the power utility and the government officials provided responses to the questionnaire through the national consultant. In PNG, the mission could have virtual meetings only with two development partners (the World Bank and Japan International Cooperation Agency). IED assessed the overall performance of implementing agencies satisfactory.

CHAPTER 5

Overall Assessment, Lessons, Issues,

and Recommendations

A. Overall Assessment 83. This chapter provides an overall assessment of the Promoting Energy Efficiency in the Pacific program. It highlights issues and lessons and provides recommendations derived from the evaluation. The assessments of the first and second projects and of the Promoting Energy Efficiency in the Pacific program as a whole are presented in Table 5.

Table 5: Assessments of Promoting Energy Efficiency in the Pacific, Promoting Energy Efficiency

in the Pacific (Phase 2), and the Overall Program

Criteria Promoting Energy

Efficiency in the Pacific Promoting Energy Efficiency

in the Pacific (Phase 2) Promoting Energy Efficiency in

the Pacific Program Relevance Relevant Less than relevant Less than relevant Effectiveness Effective Less than effective Less than effective Efficiency Efficient Efficient Efficient Sustainability Most likely sustainable Less than likely sustainable Less than likely sustainable Overall assessment

Successful Less than successful Less than successful

Source: Asian Development Bank (Independent Evaluation Department).

84. The TCR for the first project self-assessed the project successful. It did not rate the four performance criteria separately. The TCR for the second project self-assessed the project less than successful. The first project was relevant, effective, efficient, and most likely sustainable. It identified a program of seven energy efficiency measures that were cost-effective and had the potential of saving about 4.5% of total electricity sales in the five PDMCs in 2013. The second project was less than relevant, less than effective, efficient, and less than likely sustainable. The unexplained design change during implementation significantly lowered the potential to reduce energy use, the database could not be used, several energy efficiency pilot projects had not been completed by the time the second project was completed, the M&V protocol to measure savings was not carried out, and there was a low level of enforcement of MEPS.

85. Promoting Energy Efficiency program was implemented as 2 projects. The first project focused on establishing energy efficiency policy measures in five PDMCs. The second project attracted wide support of development partners for implementing energy efficiency measures that were identified successfully in the first project for addressing the high dependence on fossil fuels and sustainable energy sector development. The committed TA amount for second project was about seven times that of the first project, and the planned implemented period was 48 months (March 2011 to March 2015) compared to only 16 months (September 2008 to December 2009) for the first project. The first project had a budget for consulting services and knowledge sharing only, whereas the budget of the second project included $2,310,000 of TA funds and additional $1,100,000 of counterpart funds from PDMCs

30 Promoting Energy Efficiency in the Pacific

governments for the procurement of energy efficient equipment. As planned, the approval of second project followed the successful implementation of the first project. The DMFs confirm that the assistance was not limited to implementing energy efficiency policy measures in the five PDMCs, but included, a tangible reduction of energy use in key economic sectors through the installation of energy efficient equipment, activities prioritized in the second project. Therefore, the overall success of the Promoting Energy Efficiency in the Pacific program had depended mainly in a successful implementation of the second project. However, the five participating PDMCs are yet to fully embrace energy efficiency in their respective national energy policies and in their guidelines for energy use. The direct energy saving from the pilot projects was not verified nor was it estimated from the replication of the pilot projects. There was insufficient evidence that the actual energy saving was even close to the DMF targets. Underutilization of TA funds also implied that there was room for improvement in TA implementation. Accordingly, IED assessed the whole program of Promoting Energy Efficiency in the Pacific less than successful; the program was less than relevant, less than effective, efficient, and less than likely sustainable.

B. Lessons 86. Support for energy efficiency programs can make significant contribution towards savings and low-carbon development. An energy efficiency program is a cost-effective alternative for low-carbon development. The first project under the Promoting Energy Efficiency in the Pacific program estimated that an investment of $48 million would save 48,453 MWh per year through seven energy efficiency measures to be implemented across the PDMCs. The energy saving was equivalent to the output of a solar photovoltaic-based power generation capacity of 38.3 MW, considering 15% transmission and distribution loss and a capacity utilization factor of 17%. The capital cost in 2015 for such a solar project would have been more than double the investment in the energy efficiency program.52 The high capital costs of renewable energy-based power generation projects translate into high tariffs for consumers, and in PDMCs consumers already pay high tariffs for conventional power generation. An energy efficiency program would provide savings in energy costs over the lifetime of the appliances. In general, the PDMCs and their development partners acknowledged that the Promoting Energy Efficiency in the Pacific program had built support for adoption of energy efficient technologies.

87. Reliable measures of the completed costs for TA projects should help with the preparation of future projects. A proper cost analysis was not possible after completion of the Promoting Energy Efficiency in the Pacific program because all the inputs were not fully accounted for. ADB kept a record of the disbursements, so reliable information was available for that part of the program cost. However, audited project accounts should have been kept for the demonstration and pilot projects to confirm the major program costs. The consultants could have also used reasonable assumptions to establish the in-kind contributions of counterpart staff and power utilities. A reliable measure of the costs of the completed TA projects would have helped in the preparation of the cost estimates and scope of future TA projects. 88. Regional institutions can play a strong role in preparing and delivering knowledge products. The terms of reference for both the first and the second project required the consultant to work closely with the Pacific Center for Renewable Energy and Energy Efficiency, the Pacific Regional Data Repository for Sustainable Energy for All, and the Pacific Community. These organizations helped to make up for the lack of domestic experts and to lower the cost of data acquisition and analysis. The consultants’ reports indicated that these regional institutions helped in sharing the experience of the energy efficiency

52 International Energy Agency. 2020. Attracting Investment to Fund Sustainable Recoveries: The Case of Indonesia Power Sector.

Paris. A chart in the report indicates that the capital cost of a utility-scale solar photovoltaic-based project (over 50 MW capacity) in emerging economies (Brazil, Indonesia, Mexico and South Africa) was about $2,050 per kW in 2015. By 2019, the range was $1,000–$1,400 per kW, except in India and Mexico. Because of economies of scale, the capital cost of the smaller grid-connected solar projects that can be implemented in PDMCs (around 0.2 MW capacity), would be more than double the cost of utility-scale solar projects.

Overall Assessment, Lessons, Issues and Recommendations 31

pilot projects, although they could have played a greater role in the preparation and maintenance of an energy efficiency appliances database. 89. TA designs are more efficient if they are based on local conditions. Since the Pacific Appliance and Labelling Standards had already been developed, the output of the second project need not have included development of the standards and instead just focused on developing legislation, instruments and institutions to enforce them. Similarly, since energy service companies (ESCOs) were not commercially viable, the output of the first project could have identified alternative arrangements for incentivizing the private sector to retrofit energy efficiency measures. The consultant’s terms of reference should have explicitly stated that the technical specifications of energy efficiency equipment should consider the local weather conditions for avoiding early deterioration, like corrosion in coastal regions.

C. Issues 90. Inadequate TA administration records. Preparation of this TPER was made more difficult because of the lack of proper TA administration records. TA team leaders shared the records held in their computers. There was no paper trail for the changes in scope that took place during contract negotiations and the inception mission. There were records of extensions of TA completion dates and the variation orders for consultancy services contracts but no explanations. The e-operation records could not be accessed so the evaluation team could not easily understand TA implementation progress, procurement delays, and other issues. Hard copies of TA administration records that are held in storage could not be retrieved because of restricted staff movement as a result of the COVID-19 pandemic. The TCRs for the two projects were prepared before the December 2019 revision of the staff instruction and so contained less information and fewer ratings than are now required. 91. The reasons for the delay in implementing energy efficiency policies and strengthening institutions were not clear. The consultants provided inputs for appropriate energy efficiency policies and the pilot projects helped to explain the economic and financial benefits of energy efficiency. However, it was unclear why some PDMCs have not yet enacted legislation or established suitable institutions to promote energy efficiency measures. The reports by the TA consultant and the review mission do not explain the remaining barriers for implementing energy efficiency measures in the PDMCs, including those related to the decisions taken by households and businesses, which are influenced by the political environments in the PDMCs. The TA projects did not review political and societal support for energy efficiency measures that apparently benefited everyone. 92. Subsidies and the dissemination of energy efficiency information may be insufficient for changing behaviors. The pilot projects distributed energy efficient lighting to households and shared the costs of the energy efficiency measures implemented by businesses. Brochures informed customers how energy efficiency products could be identified and the benefits they offered. However, these measures have not enabled energy efficiency measures to be mainstreamed. The program showed a lack of understanding of customer behavior and did not design suitable triggers to shift customer choice toward energy efficiency products. Traders lacked incentives for retraining their sales staff, running advertisement campaigns, or otherwise actively promoting the sale of appliances with a higher efficiency. Commercial establishments did not have a clear pathway for retrofitting existing low-efficiency air conditioning systems and building codes did not include mandatory energy efficiency standards. 93. The TA modality may not be suitable for projects that included both knowledge support and investments. The TA approach allowed ADB to design a project that assisted more than one developing member countries, but its basic purpose was to provide knowledge and advisor support. It lacked the instruments needed for large-scale procurement of goods, civil works, and installation services. Detailed cost and financing plans, audits of project expenditures, and economic and financial analyses were not prepared.

32 Promoting Energy Efficiency in the Pacific

D. Recommendations

94. When designing knowledge and support TA projects in future, ADB should:

95. Recommendation 1: Improve TA administration records. Performance evaluation of large TA programs, with over $1 million worth of consulting services and implemented over 2 years or more, requires a complete set of TA administration records, including back-to-office reports of TA review missions, quarterly updates of TA implementation progress, reviews of consultants’ proposals for changes, memos seeking approval of minor and major scope changes, and time extensions. Such records can be used to validate the information obtained from the TA consultant’s report. 96. Recommendation 2: Identify winners and losers and better manage the impacts of policy reforms, especially on losers. Implementation of a policy reform requires change management, which is influenced by the political environment in each country. The Promoting Energy Efficiency in the Pacific program had both winners (e.g., a ministry of finance that benefited from lower electricity costs for public buildings) and losers (e.g., utilities whose power generation capacity expansion plans and revenues from electricity sales were affected by energy savings). Negotiations between these winners and losers are needed when implementing energy efficiency policies and procedures. The PDMCs have committed themselves to low- carbon growth targets, and their high electricity tariffs provide consumers with incentives to switch to energy efficiency appliances. The environment was therefore favorable for the implementation of the Promoting Energy Efficiency in the Pacific program, but the outcome was not achieved. Future ADB assistance in support of policy reforms should consider engaging a social science specialist in the consultancy team to assess the political environment and suggest appropriate approaches to support negotiations between winners and losers for the successful implementation of new energy efficiency policies and regulations. 97. Recommendation 3: Design and run more targeted campaigns that aim to change the purchasing behavior of households and businesses towards energy efficiency products. Policy and regulations can provide the enabling environment, but it is individual customers and business entities that will take the decision to shift to energy efficiency appliances and products. Enforcing MEPS requires an understanding of triggers and incentives to convince appliance dealers to import and provide customer services for efficient products. Future operations should include a marketing specialist in the consultant’s team to study purchasing behavior in the domestic appliance market and suggest campaigns that would shift consumer choice towards more efficient appliances and equipment. The marketing specialist could identify triggers and a suitable subsidy program to accelerate a shift to energy efficiency products. 98. Recommendation 4: Consider using the small expenditure financing facility for future energy efficiency programs. The envisaged reduction in energy use and consequent reduction in GHG emissions enabled the Promoting Energy Efficiency in the Pacific program to attract significant support from ADB’s development partners. The activities included consulting services and implementation of a very large number of small subprojects. Future ADB assistance for energy efficiency programs in PDMCs, or in other developing member countries that depend on imported energy, should consider using the small expenditure financing facility modality.53 However, since implementation capacity is weak in some developing member countries, the facility may need a minor modification to allow ADB to be the executing agency of an energy efficiency program as is the practice for a TA project. Future energy efficiency investments should be structured as a public sector project because of the limited size and competitiveness of the private sector-led appliance and electrical equipment markets in the PDMCs; extending support to a group of PDMCs would improve procurement efficiency by increasing the lot sizes and lowering logistics costs.

53 ADB. 2018. Proposal for ADB’s New Products and Modalities. Policy Paper. Manila.

Overall Assessment, Lessons, Issues and Recommendations 33

99. Recommendation 5: Promote low-carbon development by reducing energy consumption in buildings and transport. In addition to supporting renewable energy and electricity storage projects, ADB should support the lowering of energy consumption in buildings. Support needs to include specific activities that build on the status of the energy efficiency measures that have been adopted in the PDMCs. For example, projects could subsidize the higher up-front costs of energy efficiency products, and the retrofitting of energy efficiency measures in buildings. They could create an energy efficiency fund to offer medium-term loans based on energy savings. This would lower dependence on imported fossil fuel, improve energy security, increase household and enterprise savings because of lower energy bills, and lead to a cleaner environment because of a reduction in emissions.

Appendixes

APPENDIX 1: SUMMARY DESIGN AND MONITORING FRAMEWORKS OF INDIVIDUAL TECHNICAL ASSISTANCE PROJECTS

Table A1.1: Technical Assistance for Promoting Energy Efficiency in the Pacific (TA 6485-REG, approved September 2008)

Design Summary Performance Targets

and Indicators TA Completion Report

(September 2011)

TA Performance Evaluation Report

(May 2021) Impact Mature energy efficiency applications reduce growth of commercial energy consumption and improve energy security

Growth of energy imports in selected PDMCs reduced by 30%

Not assessed

Not assessed. The baseline (projected growth in energy imports by 2020 for power generation and transport) was not available. Energy efficiency measures have not significantly reduced energy imports but the COVID-19 linked economic slowdown (particularly in tourism) will have reduced energy imports.

Outcome PDMCs, ADB, and GEF agree on joint ADB and GEF energy efficiency project(s), which will fund the development of capabilities and energy efficiency improvements in the industrial, commercial, residential (including poor urban and rural households) and public sectors

Draft final report submitted to governments and ADB and other agencies by January 2010 Memorandum of understanding agreed on by ADB and the governments for further processing of energy efficiency related assistance by March 2010

Not assessed. Final report submitted in September 2011. Memorandum of understanding was not needed for approval of Promoting Energy Efficiency in the Pacific (Phase 2).

Achieved. Delay was for an extended scope of work for assistance in preparation for the application for GEF funding for Promoting Energy Efficiency in the Pacific (Phase 2)—referred to as the second project; a memorandum of understanding was not necessary for approval of the second project.

Outputs 1. Assessment of the energy efficiency policy and regulatory framework and past energy efficiency assistance in the Pacific, with dissemination of lessons

Consultant reports acceptable to the Government and ADB: Inception report – month 1.5 Mid-term report – month 7 Draft final report – month 12 Final report – month 14

Achieved. Policy and regulatory framework assessed and recommendation for improvement (generic) were submitted. The achievements were not assessed against the target dates.

Output 1 was not achieved. The output was modified based on the inception report. Energy use baselines were prepared for five participating PDMCs. However, a bottom-up approach for estimating energy efficiency benefits required further detailed surveys.

2. Energy efficiency policy and regulatory frameworks recommended for adoption in five selected PDMCs

Output 2 was achieved. The consultant’s report included generic policy recommendations and identified institutional arrangements; training was provided.

3. Recommended best way of promoting and setting up structured energy management systems to sustain

Achieved after output was modified. Energy efficiency markets were analyzed. With the exception of PNG, they

Output 3 was achieved. Five demonstration projects were developed and implemented.

36 Summary Design and Monitoring Frameworks of Individual Technical Assistance Projects

Design Summary Performance Targets

and Indicators TA Completion Report

(September 2011)

TA Performance Evaluation Report

(May 2021) energy efficiency initiatives, including possible ESCOs

were considered too small for the commercial operation of ESCOs. Instead, demonstration energy efficiency projects were prepared and implemented

- compact fluorescent lamps in Cook Islands

- power factor correction in PNG and Samoa

- light-emitting diode streetlamps in Tonga

- solar water heaters in Vanuatu

4. Training needs assessments and services and other assistance with regulatory/ institutional reform to promote a viable market for energy efficiency services in each country.

Achieved. A training needs assessment was carried out and a trial training program for an investment grade energy audit was conducted in Vanuatu

Achievement was not separately assessed. This was assessed as part of output 1. Training needs identified and training given in Vanuatu for an investment grade energy audit.

5. Pipeline of assistance projects for funding by ADB, GEF, or other relevant financing sources.

Achieved. Energy efficiency program developed for the preparation of the proposal for the second project.

Output 4 was achieved. Future energy efficiency program developed with seven energy efficiency measuresenergy efficiency in government buildings, streetlighting, hotels, and water pumping; use of CFLs in residences; MEPS and labeling; and power factor correction.

6. A strategy for public awareness raising and education

Strategy for raising public awareness was developed, and contributed to the existing program in Cook Islands

Output was not included in modified scope, not assessed separately.

ADB = Asian Development Bank, CFL = compact fluorescent lamp, COVID-19 = coronavirus disease, ESCO = energy service company, GEF = Global Environment Facility, GWh = gigawatt hour, MEPS = minimum energy performance standard, PDMC = Pacific developing member country, PNG = Papua New Guinea, REG = regional, TA = technical assistance, TPER = TA performance evaluation report. Source: ADB TA reports, TCR for Promoting Energy Efficiency in the Pacific, and TA performance evaluation report.

Appendix 1 37

Table A1.2: Technical Assistance for Promoting Energy Efficiency in the Pacific (Phase 2) (TA 7798-REG, approved March 2011)

Design Summary Performance Targets

and Indicators TA Completion Report

(August 2016)

TA Performance Evaluation Report

(May 2021) Impact Reduction in fossil fuel use by the power sector without a corresponding reduction in energy services in the five participating countries

By end 2018 Reduction of fossil fuel use by 10% relative to projected growth (baseline 135 million liters per year) Total energy savings from power sector (all five DMCs) of 45 GWh per year. Greenhouse gas emission reduction from power sector (all five PDMCs) of 30,000 t CO2 per year

3.4 GWh per year (estimated) 3,204 t CO2 equivalent (estimated)

Not achieved. However, the knowledge products for promoting energy efficiency were produced. Achieving the quantitative targets would require implementation of the recommendations for energy efficiency policy and procedures.

Outcome End consumers use power efficiently in the participating countries

By 31 March 2015 Reduction of average monthly electricity consumption by 10% for residential, commercial and public buildings; baseline electricity consumption for residential consumers was 125 kWh per month

Quantitative targets not achieved.

Not achieved. The design change reduced the scope to implementation of only 34 pilot projects with an estimated electricity saving of 3.4 GWh per year and a reduction of 3,206 tCO2 per year.

Outputs 1. Stakeholders have access to comprehensive information on energy use by sector and appliance.

By 31 March 2013 Comprehensive database of major energy consuming equipment and appliances prepared in each participating country.

Achievement not assessed. Energy end-use database could carry out only basic functions; partly due to the limited availability of data in the PDMCs.

Output 1 was not achieved. An energy use database was prepared but with limited capabilities; further, access to the database was available only during implementation of the second project.

2. Energy efficiency practices have been mainstreamed into government processes, procedures, and policies in the participating countries.

By 31 March 2013 National energy efficiency targets incorporated into national energy policies. Schemes to reduce importation of energy-inefficient equipment and appliances adopted in the five countries. Energy efficiency building codes for residential, commercial,

Achievement not assessed. However, it is recognized that (i) national energy efficiency targets were established, (ii) the task of preparing minimum energy performance standards was abandoned as Pacific Appliance Labelling Standards already existed,

Output 2 was not achieved. However, specific deliverables were provided for: energy efficiency practices, national energy efficiency targets, building codes and best practices, energy audit training, and capacity building. However, energy efficiency practices have not yet been mainstreamed in the PDMCs

38 Summary Design and Monitoring Frameworks of Individual Technical Assistance Projects

Design Summary Performance Targets

and Indicators TA Completion Report

(August 2016)

TA Performance Evaluation Report

(May 2021) and public buildings established.

(iii) building sector energy efficiency technologies, best practices and rating schemes were developed, (iv) a comprehensive energy audit training program was implemented, (v) tasks related to ESCOs were abandoned

3. Energy efficiency programs implemented effectively and sustainably in each participating country.

By 31 March 2015 50% of all public street lighting upgraded using light-emitting diode or high-pressure sodium technology 90% of incandescent bulbs installed residentially replaced by compact fluorescent lamps. Reduction in monthly energy consumption of major public and commercial buildings by 10%

The scope was reduced to implementing 34 pilot energy efficiency projects. Quantitative targets were not achieved.

Output 3 was not achieved. 34 energy efficiency pilot projects were implemented to enable knowledge transfer; 12 of these were incomplete at the closing date of the second project and measurement and verification not carried out to confirm the benefits. It was reported that seven pilot projects had maintenance issues or were not functioning.

4. Information on energy efficiency has been shared and public awareness of the benefits of energy saving has improved.

By 31 March 2015 At least 50% of population aware of energy efficiency initiatives and benefits of energy conservation

The quantitative target was not assessed. Energy efficiency information dissemination was achieved through various public education programs; regional workshops were held to share energy efficiency best practices and lessons

Output 4 was achieved. Energy saving guides and information brochures were prepared and distributed. The coverage was not measured.

5. Effective project management has been established.

By 31 March 2015 Program of activities implemented on time and to budget.

It was part of the assessment of performance of consultant, executing agency and implementing agency.

CO2 = carbon dioxide, ESCO = energy service company, GWh = gigawatt hour, PDMC = Pacific developing member country, t = ton, REG = regional, TA = technical assistance, TPER = TA performance evaluation report. Source: ADB TA reports, TCR for Promoting Energy Efficiency in the Pacific (Phase 2), and TA performance evaluation report.

APPENDIX 2: SUMMARY OF ENERGY EFFICIENCY PILOT PROJECTS IMPLEMENTED UNDER TECHNICAL ASSISTANCE FOR PROMOTING ENERGY EFFICIENCY IN THE PACIFIC (PHASE 2)

Table A2.1: Summary of Energy Efficiency Pilot Projects Implemented No. Pilot Project Name Energy Efficient Technologies

Cook Islands

1 Energy Efficient Lighting at Rarotonga AirportSolar-powered LED luminaires for area illumination, poles suitable for category 5 cyclone

2 Energy Efficient Lighting at Punanga Nui Market

Solar-powered LED luminaires for area illumination

3 Energy Efficient Refrigerator/Freezer Replacement

Refrigerators/freezers with 3- or 4-star efficiency rating to replace 1- or 2-star refrigerators/freezers

4 Energy Efficient Lighting in Marina and WharfSolar-powered LED luminaires for area illumination, poles suitable for category 5 cyclone

5 Energy Efficiency Measures in Public BuildingsT5 FTLs with integrated electronic ballast, timers for air conditioning units

6 Energy Efficiency Measures in Edgewater Resort

Split inverter-type air conditioning, T5 FTLs and luminaires, key tag switches, insulation

7 Residential Energy Efficient Lighting T5 FTL with electronic ballast 8 Energy Efficient Street Lighting in Outer

Islands High-efficiency LED luminaire

9 Solar Water Heating in Rarotonga Hospital Thermosyphon solar water heater

Papua New Guinea 1 Energy Efficient Street Lighting in Port

Moresby White LED luminaires for streetlights in residential areas

2 Energy Efficient Lighting in PPL Head Office T5 FTLs with electronic ballast and highly reflective luminaire, zone control and motion sensors, programmed kill switches, lamp disposal,

3 Energy Efficient Street Lighting in Alatau White LED luminaires for streetlights in residential areas

4 Energy Efficient Lighting in Port Moresby Hospital

T5 FTLs with electronic ballast and highly reflective luminaire, photo and motion sensor

5 Energy Efficient Lighting and Ventilation in UPNG

T5 FTLs with electronic ballast and highly reflective luminaire, photo and motion sensor

Samoa

1 Energy Efficient Street Lighting in Apia LED luminaires 2 Energy Efficient Lighting in Government

Buildings T5 FTL with electronic ballast, run-hour meter

3 Energy Efficient Luminaires in Government Buildings

T5 FTL with electronic ballast

4 Demonstration of Energy Efficient Air Conditioning

Inverter type air conditioner and solar hybrid air conditioner, smart controller

5 Energy Efficient Lighting in Commercial Buildings

LED batten

6 Energy Efficient Air Conditioning in Hotels High-efficiency inverter air conditioner

7 Residential Energy Efficient Lighting T5 FTL with electronic ballast Tonga

1 Energy Efficient Street Lighting in Outer Islands

White LED luminaire

2 Energy Efficient Street Lighting in Tongatapu White LED luminaire 3 Residential Energy Efficient Lighting in Outer

Islands T5 FTL with electronic ballast

4 Energy Efficient Lighting in Public Sector Buildings

LED tubes

5 Residential Energy Efficient Lighting in Tongatapu

T5 FTL with electronic ballast

40 Summary of Energy Efficiency Pilot Projects Implemented Under Technical Assistance for Promoting Energy Efficiency in the Pacific (Phase 2)

No. Pilot Project Name Energy Efficient Technologies Vanuatu

1 Energy Efficient Air Conditioning Using VRF Technology

Variable refrigerant flow air conditioning

2 Energy Efficient Lighting in Public Sector Buildings

LED tube lights and T5 FTL with electronic ballast

3 Energy Efficiency Street Lighting in Loganville High-efficiency LED luminaire

4 Residential Energy Efficiency Lighting in Loganville

T5 FTL with electronic ballast and CFL

5 Energy Efficiency Street Lighting in Port Vila High-efficiency LED luminaire

6 Residential Energy Efficient Lighting in Port Vila

T5 FTL with electronic ballast and CFL

7 Energy Efficient Lighting in Public Sector and Schools

T5 FTL with electronic ballast

All Pacific Developing Member Countries 1 Lamp Waste Treatment Equipment Bulb eater

CFL = compact fluorescent lamp, FTL = fluorescent tube light, LED = light emitting diode, PPL = Papua New Guinea Power Limited, T5 = tubular shape with 16-millimetre diameter, UPNG = University of Papua New Guinea, VRF = variable refrigerant flow. Source: ADB. 2015. Technical Assistance Consultant’s Report for Promoting Energy Efficiency in the Pacific (Phase 2) prepared by

International Institute for Energy Conservation. Manila.

APPENDIX 3: OVERVIEW OF ENERGY EFFICIENCY MEASURES ADOPTED IN PARTICIPATING PACIFIC DEVELOPING MEMBER COUNTRIES

A. Regional Knowledge Hub 1. The Pacific Centre for Renewable Energy and Energy Efficiency (PCREEE) was developed as a regional resource in 2014–2015 with the support of the international community. It is part of a global network of regional sustainable energy centres. It functions under the Pacific Community (SPC) and operates jointly with Pacific Regional Data Repository for Sustainable Energy for All (PRDR for SE4ALL) as a knowledge platform for overcoming barriers to the uptake of sustainable energy technology markets.1 2. Development partners have extended wide-ranging investment and knowledge transfer support to the participating Pacific developing member countries (PDMCs). In addition to the Asian Development Bank (ADB), the other major contributors are Australian Department of Foreign Affairs and Trade, European Union, Global Green Growth Institute, International Renewable Energy Agency, New Zealand Foreign Affairs and Trade, United Nations (several agencies), and the World Bank.

B. Cook Islands

3. Electricity demand. Only limited data on electricity consumption in Cook Islands were provided to the evaluation mission. During the 10-year period starting FY2009, electricity demand increased by 13%, a compound average growth rate (CAGR) of 1.2%; during the same period, GDP grew at a CAGR of 3.2%. The electrification ratio was nearly 100% in 2019. Electricity consumption in FY2020 was 29,387 GWh; 15% higher than the forecast for the business-as-usual case based on the modelling done under the technical assistance (TA) for Promoting Energy Efficiency in the Pacificthe first projectusing actual data for FY2011. 4. According to Te Aponga Uira (TAU) staff, energy consumption gradually declined from FY2012 to FY2016 because of energy efficiency measures used by consumers. However, consumption gradually increased during FY2017–FY2018 because of the higher rate of economic growth. Consumption dropped in FY2020 and was expected to further drop in FY2021 because of the coronavirus disease (COVID-19) downturn. The decline in the 2 years was projected to be nearly 20% from the FY2019 level. 5. Energy efficiency regulations. Draft legislation for minimum energy performance standards (MEPS) was prepared in 2014 but has not been enacted; 2 the scope included refrigerators and air conditioners. 6. Institutions for promoting energy efficiency. The Renewable Energy Development Division (REDD) under the Office of the Prime Minister provides strategic directions through policy planning and the regulatory framework; it is responsible for reviewing the renewable energy policy and the implementation of renewable energy projects. The Office of the Energy Commissioner, under the Minister for Energy, provides guidance to REDD and is responsible for coordination and progress in the power sector (at present the Prime Minister is also the Minister of Energy). TAU is a government-owned vertically integrated power company that is responsible for providing electricity services in Rarotonga. It is under REDD. Local governments generally engage private sector companies to provide electricity services outside Rarotonga.

1 Members of the international community that support PCREEE include Austrian Development Agency, Royal Norwegian Ministry

of Foreign Affairs, Small Island Developing States Sustainable Energy and Climate Resilience Initiative, and the United Nations Industrial Development Organization.

2 Pacific Community. 2019. Pacific Appliances Labelling and Standards Programme, Final Evaluation. Suva.

42 Overview of Energy Efficiency Measure Adopted in Participating Pacific Developing Member Countries

7. Public awareness of energy efficiency. In 2014, SPC and the University of the South Pacific carried out a training needs assessment for sustainable energy and adaptation. This identified a need for awareness raising for energy efficiency. 3 The consultants also prepared resources for training local electricians and raising public awareness; training courses were implemented during training needs assessment study but not since then. TAU trained its own staff to carry out energy audit and load analysis for determining the most suitable size of rooftop solar photo-voltaic and battery installations. Such training courses were open to local electricians. However, TAU does not provide energy audit services to its customers. There are four local electrical firms that offer such services to large customers. 8. Energy efficiency appliances have become readily available, and prices are dropping. Only 3.5- or 4-star refrigerators and air conditioners are now being sold in the domestic market, so customers have had to switch to more efficient appliances as replacements, or as original appliances in new households and commercial establishments. The government has a public awareness program for influencing customer behaviour, but its efficacy had not been assessed. Electricity bills provided by TAU include advice on using energy efficient appliances. 9. Cook Islands Tourism Industry Board, which represents over 150 businesses, has noted that several energy efficient technologies are being used in hotels, e.g., solar water heaters with electric or gas boosters, key cards to switch off air conditioners in vacant rooms, light emitting diode (LED) lights, and solar pool pumps. However, there has been no assessment of the extent of such installations nor any quantification of energy savings. 10. Status of demonstration and pilot projects. The demonstration projects included the distribution of 10,000 compact fluorescent lamps (CFLs) to households through a local trading company. This was considered to have been successful in supporting the transition to energy efficient lighting, although a quantitative evaluation was not carried out. 11. Solar-powered LEDs were installed in the Rarotonga Airport car park (8 units) and Punanganui Market (8 units); a special feature was the use of imported poles suitable for a category 5 cyclone. It is reported that three lights in the airport car park and two lights in the Punanganui Market were not working (a failure rate of over 30%). Some lights were missing from the Punanganui market. The reasons given was the prohibitive rental cost of long-boom cranes (NZ$500 per hour) needed to access the faulty lights and the lack of spare control boards. The Ports Authority also installed 16 LED area lights; all were functioning. High winds had damaged 2 solar panels, which had been replaced. LED streetlights (30 units) were installed in outer islands; there was positive feedback regarding cost savings and the illumination level. However, the sensor protection was inadequate, so there was water ingress in four fixtures. Two pilot projects (T5 fluorescent lamps) were not implemented; procurement was carried out but the T5 lamps have been kept in the stores of the Cook Islands Investment Corporation. The reason given by the counterpart staff was that the light intensity was inadequate, and the technology has been superseded by LED tubes. Solar water heaters (6 units) were installed in Rarotonga Hospital. There was very negative feedback about these, mostly because of the inferior materialthe metal parts had soon rusted, one had been replaced with a gas-based water heater, and elements of another three were replaced in 2019. Local technical staff had recommended that all solar water heaters be replaced. Split-type inverter air conditioners (75 units) were installed at the Edgewater Resort. According to resort staff, some air conditioners had to be replaced because the salty air caused rusting of the body. The energy savings had not been monitored. Two large retailers (Cook Islands Trading Corporation [CITC] and Motor Centre) participated in a program for the subsidized sale of refrigerators, a total of 325 energy efficient refrigerators were distributed at subsidised cost.

3 Pacific Community. 2014. European Union Pacific Technical and Vocational Education and Training Project (EU PacTEVT) on

Sustainable Energy and Climate Change Adaptation, Cook Islands, Training Needs and Gap Analysis. Suva.

Appendix 3 43

12. Environmental impact. A bulb eater was provided under Promoting Energy Efficiency in the Pacific (Phase 2)—the second project—for environmentally safe disposal of mercury in damaged fluorescent lamps. It is currently with the Waste Management Division of Infrastructure Cook Islands, but is not being used. Waste segregation is not being practiced, so it was likely that the damaged lamps are going to waste dump sites with other household garbage. However, CITC had installed its own bulb eater, which was being used. 13. Development partners. Other development partners did not have any ongoing assistance projects for energy efficiency. 14. Issues for promoting energy efficiency. The Cook Islands Tourism Industry Board mentioned that the higher initial investment in energy efficiency appliances was still a barrier. There was also a fear of obsolescence when more efficient technologies became available. Local technicians needed training to attend to operating problems or to carry out routine maintenance.

C. Papua New Guinea

15. The evaluation mission was unable to obtain energy consumption data for Papua New Guinea (PNG). 16. Energy efficiency regulations. One of the principles of PNG’s National Energy Policy, 2017–2027 was that promoting energy efficiency meant that action was required across all sectors (industrial, residential, agriculture, and transport). 4 According to media reports and information provided by development partners during meetings with the evaluation team, the National Energy Policy was formally launched in August 2020.5 Of the PDMCs, PNG had the largest potential for savings through energy efficiency because of the significant energy consumption by commercial and industrial sectors. A 2019 evaluation of the implementation of appliance standards and the labelling program reported that draft energy efficiency legislation included lighting, refrigerators, freezers, and air conditioners. However, the timeframe for its enactment was not known.6 The Electricity Industry Act, 2002 is the current legislation, but it provides little guidance on promoting energy efficiency measures. 17. Institutions for promoting energy efficiency. The Department of Petroleum and Energy promotes and regulates the development of petroleum and renewable energy resources. A proposal has been made to establish a National Energy Authority that would take over economic regulation and set electricity industry standards. Kumul Consolidated Holding Limited holds the government’s share in all state-owned companies, including PNG Power Limited (PPL). PPL is responsible for developing the power sector and supplying electricity throughout PNG. 18. The evaluation mission was unable to obtain information on the present status of public awareness of energy efficiency measures. The demonstration project under the first project involved installation of power factor correction equipment (capacitors), this was a routine utility practice that was being maintained by PPL. The five energy efficiency pilot projects under the second project involved energy efficiency lights in residences, public buildings, and streetlights. The evaluation mission was unable to determine whether the energy efficiency lighting products were being widely used, the use of the pilot project equipment, or the status of the bulb eater. 19. Development partners. In 2017, United Nations Development Programme (UNDP) approved the Facilitating Renewable Energy and Energy Efficiency Project. This is being implemented using a GEF grant

4 Department of Petroleum and Energy. 2017. National Energy Policy 2017–2027. Port Moresby. 5 The National. 2020. Policy eyes renewable energy. Port Moresby, published on 20 August 2020. 6 Pacific Community (SPC). 2019. Pacific Appliance Labelling and Standards Program, Final Evaluation. Suva.

44 Overview of Energy Efficiency Measure Adopted in Participating Pacific Developing Member Countries

of $2.8 million. It piloted an energy efficiency audit of 11 of the highest energy users in Wewak.7 The World Bank shared a 2016 consultant’s report on developing an energy efficiency plan with the evaluation mission. The report drew heavily on the consultant reports for the first and second ADB projects and the goals included reductions in the share of thermal power generation, peak power demand, electricity use by public services (buildings, water supply, and sewage treatment); and the application of MEPS for appliances in public facilities, electric motors, and all imported appliances. The evaluation mission was unable to obtain a timetable for implementation of the energy efficiency plan.

D. Samoa

20. Electricity demand. Table A3.1 presents the key statistics for electricity consumption in Samoa. During the 11-year period starting FY2008, electricity demand increased by 56%, a CAGR of 4.1%; over the same period, GDP grew at a CAGR of 1.2%. The electrification ratio was 100% by 2018. The electricity consumption in FY2020 was 152 GWh; 15% higher than the forecast for the business-as-usual case based on the modelling done under the first project using actual data for FY2011. It appears that the efficiency of energy use had deteriorated, but there may be several explanations for the large variation, including the very conservative assumptions used in the first project’s demand forecast model. Consumption by government and schools increased by 312%, and by residential customers by 64%; electricity demand grew because of the 100% electrification ratio and the better services offered by the Electric Power Corporation (EPC). The view of EPC staff was that electricity demand had increased because of the high economic growth rate.

Table A3.1: Key Statistics on Electricity Consumption in Samoa

Item FY2019 FY2018 FY2017 FY2016 FY2015 FY2011 FY2008

GDP (ST million, 2013 prices)

2,051 2,006 1,992 2,005 1,934 1,826 1,804

Population 200,900 199,200 197,600 196,000 194,200 187,800 185,600

Total production (MWh)

169,258 153,507 154,376 141,631 132,434 109,030 115,523

Total sales (MWh) 151,745 138,424 138,378 122,311 116,419 89,735 97,110

Domestic 43,657 38,978 38,115 34,910 31,241 26,529 28,771

Commercial 48,201 45,909 45,119 39,229 47,586 39,037 41,787

Government, including schools

33,320 28,917 29,914 29,854 20,301 10,665 13,134

Hotels 7,460 5,594 5,303 3,455 2,814 3,278 4,706

Industry 5,517 5,440 5,916 4,815 4,833 5,299 4,789

Religion 6,809 6,531 6,949 6,009 5,483 4,925 5,115

Streetlights 2,740 2,869 2,271 … … … …

Total customers 40,088 39,107 37,706 36,431 39,922 36,793 … = not available, FY = fiscal year, GDP = gross domestic product, MWh = megawatt hour.

Source: Electric Power Corporation, Samoa.

21. Energy efficiency regulations. Samoa approved the Energy Efficiency Act, 2017 and issued the Energy Efficiency Regulations, 2018. These enabled the government to set standards and required registration and labelling of products that use energy. In August 2020, energy efficiency regulation was further strengthened with the enactment of the Energy Management Act, 2020, which empowered the government to seize any imported product that did not comply with efficiency and labelling standards or was not registered under the regulations.8 Other aspects of the power sector are governed by the

7 UNDP. 2020. Facilitating Renewable Energy and Energy Efficiency Project. New York. Posted on 17 August 2020, accessed on

5 December 2020. 8 Ministry of Finance, Samoa (MOF). 2020. Energy Management Act 2020. Apia. MOF. 2017. Energy Efficiency Act 2017. Apia.

MOF. 2018. Energy Efficiency (Approved Energy Using Products Standards) Regulations 2018. Apia.

Appendix 3 45

Electricity Act, 2010. The theme of the Samoa Energy Sector Plan, 2017–2022 was “Sustainable and Affordable Energy Supply for All.”9 For the electricity sector, one of the key issues to be addressed was energy efficiency improvements on the supply and demand supply. The government expected that new requirements for importing MEPS-compliant lighting fixtures and appliances (refrigerators, freezers, and air conditioners) would transform the market and enable consumers to reduce energy use without sacrificing the level of service provided by these electrical products. 22. Institutions for promoting energy efficiency. The Energy Division, under the Ministry of Finance, is responsible for the planning and development of the power sector. EPC is a state-owned vertically integrated power company that reports to the Ministry of Works, Transport and Infrastructure. The Ministry of Finance regulates the electricity tariff. The Ministry of Natural Resources and Environment is responsible for implementing renewable energy projects and shares the task of promoting energy efficiency. The Energy Division has a section that monitors steps being taken to promote energy efficiency; however, it did not conduct regular training or public awareness programs during the evaluation period. EPC implemented projects for improving supply-side energy efficiency but did not offer energy audit services to help customers lower their energy use. Certified electricians were permitted to install energy efficiency appliances, but they lacked the capacity to carry out energy audits or to recommend measures for energy conservation. 23. Public awareness of energy efficiency. The government is committed to promoting energy efficiency, which is reflected in the Energy Management Act, 2020. However, according to the government, households continued to give higher weight to cost when purchasing appliances rather than to their efficiency or life-time energy cost. This is expected to change when the local market stocked only MEPS-compliant appliances. Government inspectors carry out quarterly market surveys to ensure unregistered products are not being stocked in shops. The national building codes are in the form of “best practices;” they cover energy efficiency measures but are not binding. 24. Status of demonstration and pilot projects. All seven pilot projects under the second project have been completed. However, the design of the solar hybrid air conditioner for the Ministry of Natural Resources and Environment was defective as the glass panels of the solar heater broke under local weather conditions; as a result, the equipment was never used. Due to staff changes, the government was unable to provide further details regarding project costs and actual energy savings. 25. Environmental impact. The bulb eater was used for about a year after installation but has not been functioning since then for want of spares; the government considered the cost of spares to be too high. 26. Development partners. The Department of Foreign Affairs and Trade, Australia (DFAT) supported a TA that halved the energy use for air conditioning of the newly constructed parliament building in Apia that was inaugurated in March 2019. The building was designed by an Australian architectural firm; the addition of the dehumidifiers recommended by the TA significantly lowered the amount of air conditioning needed. The success of this project led to a similar approach being adopted under the follow on DFAT project for the legislative assembly offices in the parliament complex, which is under implementation. The Ministry of Foreign Affairs and Trade, New Zealand (MFAT) cofinanced an ADB project for the implementation of small hydropower projects;10 in 2014, it supported the 2.2 MWh capacity solar photovoltaic capacities at the Faleota racecourse, Salelonga and gym building.11 The representative of the World Bank said that the ADB Promoting Energy Efficiency in the Pacific program had built significant awareness at the Ministry of Finance regarding energy efficiency. The general public

9 MOF. 2017. Samoa Energy Sector Plan 2017–2022. Apia. 10 ADB. 2015. Additional Financing for Proposed Administration of Grants to the Independent State of Samoa for Renewable

Energy Development and Power Sector Rehabilitation Project. Manila. 11 New Zealand Ministry of Foreign Affairs and Trade. 2015. Evaluation of the Samoa Country Programme. Wellington.

46 Overview of Energy Efficiency Measure Adopted in Participating Pacific Developing Member Countries

had also become more aware of power usage by appliances and brands that offer energy labelling. The World Bank did not have any ongoing programs in the energy sector.

E. Tonga 27. Electricity demand. Tonga collected only limited consumer consumption data so a table could not be included. During the 10-year period starting in 2009, when the Promoting Energy Efficiency in the Pacific program was approved, electricity demand increased by 26% in Tonga, a CAGR of 2.3%; the GDP grew at a rate of 2.2%. The electrification ratio was 94.5% in 2019; in Tongatapu, the main island, it was 98.1%. 12 Tonga Power Limited (TPL), the state-owned power company, mentioned that electricity consumption had recently increased after the distribution networks had been upgraded, which provided greater access and improved the reliability and stability of supply. 28. Energy efficiency regulations. The United Nation’s Climate Technology Centre and Network provided consulting services and training in 2017–2018 for the development and implementation of a Tonga Energy Efficiency Master Plan.13 The cabinet of ministers endorsed the plan in November 2020. The government began the process for submitting the Tonga Energy Bill to the parliament in 2021, this legislation will repeal the Electricity Act, 2007 and will include provisions for enforcing MEPS. In 2021, the government will finalise the Tonga Energy Road Map Plus for 2020–2030, which will include actions for promoting energy efficiency in buildings, and transport. 29. The Tonga Energy Road Map, 2010–2020 sought a least-cost approach to reducing the country’s vulnerability to oil price shocks and increasing access to modern energy services in a financially and environmentally sustainable manner.14 It included a target of a 50% reduction in petroleum use for power generation. According to the TPL annual report, the share of renewable energy was only 9.1% in FY2019. COVID-19 related travel restrictions held up implementation of several renewable energy projects so this target will need to be revised. Installation of battery energy storage systems will be essential to addressing the intermittent availability of such renewable resources as wind and solar. Both power generation from renewable energy and improved energy efficiency on the supply and demand sides were identified as necessary to reduce diesel consumption. 30. Institutions for promoting energy efficiency. The Department of Energy, under the Ministry of Meteorology, Energy, Information, Disaster Management, Environment, Climate Change and Communications, is responsible for development of the power sector. It has been restructured and has over 40 staff with offices in the main islands. However, there is no specific division for oversight of energy efficiency plans and activities. The PACREEE local office carried out regular energy efficiency training programs focused on the private sector. In 2020, it implemented an energy efficiency training program for the whaling industry. TPL did not offer energy audit services to customers, but several licenced electrical contractors in Tonga offered energy audit services. However, even when large customers used these services so they could understand the benefits, the recommendations were seldom adopted. 31. Public awareness of energy efficiency. According to feedback from TPL, people were aware of the benefits of efficient lighting. The government informed the evaluation mission that when plans and construction drawings of public buildings required government approval, the opportunity was taken by the Ministry of Infrastructure to review the energy efficiency measures. 32. Status of demonstration and pilot projects. The demonstration energy efficiency project under the first project was the installation of 109 LED streetlights, and the five pilot projects under the second

12 Tonga Statistics Department. 2020. Tonga Multiple Indicators Cluster Survey 2019, Survey Findings Report. Nuku’alofa. 13 The United Nation’s Climate Technology Centre and Network. 2018. Government of Tonga Energy Efficiency Master Plan.

Copenhagen. 14 Government of the Kingdom of Tonga. 2010. Tonga Energy Road Map 2010–2020. Nuku’alofa.

Appendix 3 47

project included a total of 296 LED streetlights. These were operating successfully; the brightness was adequate and energy saving had been significant. The only reported problem was the failure of sensors because of the high salty environment. TPL had rolled out the program and now has about 5,300 LED streetlights in operation. About 21,000 T5 fluorescent tube lights were installed in residences and 1,785 LED tubes in public buildings to replace inefficient lights. Feedback on these initiatives was not provided to the evaluation mission. It was evident that measurement and verification (M&V) of energy saving had not been done. 33. In 2015, the People’s Republic of China gifted 500 air conditioning systems, over 27,000 LEDs, and 1,500 solar streetlights to Tonga.15 These were installed in government offices and churches in 20 towns. The memorandum of understanding for the assistance and the list of public buildings that used the air conditioners were shared with the evaluation mission, but there were no records of M&V of benefits. 34. Environment impact. The bulb eater was used by TPL for the safe disposal of mercury in the florescent lamps that had been removed from homes and public buildings. The evaluation mission was informed that the bulb eater was operated by private electrical contractor. 35. Issues for promoting energy efficiency. The government said that people gave a higher weight to upfront costs when purchasing appliances and that more effort was needed to change their purchasing behaviour. 36. Development partners. Other development partners did not have ongoing or recently completed projects to promote energy efficiency on the demand side. The emphasis was on low-carbon technologies and support for renewable energy. Regarding the low utilization of GEF resources for the second project, the staff of other development partners felt that this could have been because of the limited capacity of the Department of Energy.

F. Vanuatu

37. Electricity demand. Table A3.2 presents the key statistics of electricity consumption in Vanuatu. During the 11-year period starting in FY2008, electricity demand increased by 9%, a CAGR of only 0.8%; GDP grew at a rate of 2.6%. The electrification ratio was 33% in 2018. The electricity demand reduced by 8% because of the devastation caused by tropical cyclone Pam in 2015. The government has not estimated the energy savings from the energy efficiency measures that have been implemented in the country. Demand in 2020 will have been lower than in 2019 because of the economy-wide downturn caused by COVID-19restrictions to international travel have shut down tourism and related services. A report published in September 2020 by the domestic industry association mentioned that electricity sales had been reduced by about 20% in July–August 2020 on a year-to-year basis.16

Table A3.2: Key Statistics on Electricity Consumption in Vanuatu Item FY2019 FY2018 FY2017 FY2016 FY2015 FY2011 FY2008

GDP (Vt million, 2006 prices) … 68,370 66,443 62,495 59,696 56,829 52,810

Population … 285,136 278,726 272,459 268,598 245,245 229,073 Total production (MWh) 68,482 67,019 64,268 64,268 56,822 60,632 62,798

Peak demand (MW)a 12.4 12.6 11.8 11.8 12.0 … …

Total distribution (MWh) 63,490 62,255 59,819 59,766 53,022 55,994 58,105

Total sales (MWh) 58,016 56,923 57,441 54,855 48,636 53,434 55,131

Domestic, less than 60 kWh/m 3,114 3,213 3,263 3,311 3,299 … …

Domestic, prepaid 2,877 2,483 1,965 1,370 761 … …

15 The East-West Center. 2015. Energy Efficiency to Help Tonga’s Economy. Hawaii; media report of 21 September accessed from

the Pacific Island Report website www.pire.org. 16 Vanuatu Chamber of Commerce and Industry. 2020. Vanuatu Economic Outlook Report. Port Vila.

48 Overview of Energy Efficiency Measure Adopted in Participating Pacific Developing Member Countries

Item FY2019 FY2018 FY2017 FY2016 FY2015 FY2011 FY2008 Domestic others 12,223 12,370 12,442 12,382 10,628 … …

Commercial and hotel 14,855 14,201 14,608 14,378 12,872 … …

High voltage, industrial 19,912 19,363 19,901 18,684 15,946 … …

Administration 4,860 5,107 5,081 4,543 3,942 … …

Streetlights 176 187 182 187 188 … …

Total customers 15,295 14,169 14,096 13,385 12,559 … … … = not available, FY = financial year, GDP = gross domestic product, kWh/m = kilowatt-hour per month, MW = megawatt, MWh = megawatt hour. a The peak demand is for Port Villa; the peak demand in Malekula and Tanna is about 2% of Port Vila. Source: Vanuatu Statistics Office. 2020. Statistics Release: Gross Domestic Product 2018. Port Vila; and UNELCO. 2015 to 2020. Annual Technical Report: Electricity. Port Vila.

38. Energy efficiency regulations. The Energy Efficiency Act came into effect on 1 January 2018.17 It required all lighting products, refrigerators, freezers and air conditioners to comply with the requirements of Standards Australia, Standards New Zealand or Fiji Standards for energy performance and labelling. Importers need to register these items with the Department of Energy, with penalties for noncompliance. Incandescent lamps were no longer available in the domestic market; when such lamps failed, they were replaced by efficient tubular, CFLs or LEDs. The Updated Vanuatu National Energy Roadmap, 2016–2030 recognized that energy efficiency was cost-effective and identified specific targets and actions.18 39. Institutions for promoting energy efficiency. The Department of Energy is within the Ministry of Climate Change Adaptation, Meteorology, Geohazards. Energy, Environment and National Disaster Management. The Department of Energy has a dedicated staff with responsibility for promoting energy efficiency and enforcing MEPS. It carries out energy audits of three to five government buildings every year and follows this up with recommendations to relevant departments. A few private sector firms had developed the capacity to carry out energy audits, including a partner of UNELCO, the private power company with a licence to conduct electricity business in Port Vila and another two islands. The level of activities did not warrant a country-level energy centre in Vanuatu. The government used the regional knowledge platformsPRDR for SE4ALL and PCREEE (para. 1). 40. Public awareness of energy efficiency. The Department of Energy targeted the tourism sector for energy efficiency measures, given its large share of electricity consumption. It frequently ran awareness programs and circulated energy efficiency guides. According to the government, it plans to implement an initiative for promoting green buildings with support of PCREEE and the international community. 41. Status of demonstration and pilot projects. The first project supported an energy efficiency demonstration project in a hotel in Port Vila and the second implemented seven pilot energy efficiency projects. The consultant reports for the second project estimated the energy savings from the pilot projects based on normal operations; however, there was no record of the actual measurement or tracking of the performance of the energy efficiency equipment that was installed. Energy efficiency measures in the hotel included the use of CFLs, roof-mounted solar water heaters, and room key tags for switching off the electricity supply in empty rooms. The proper use and upkeep of these measures would yield energy savings, but in the absence of hotel records these cannot be calculated. Air conditioners with variable refrigerant flow were installed in the Ministry of Finance and National Statistics Office; however, the whole building was destroyed by tropical cyclone Pam (March 2015), so the energy saving outcome was not achieved. The other pilot projects implemented efficient lighting in public buildings, streetlights, and residences. UNELCO informed the evaluation mission that, based on the experience of

17 Republic of Vanuatu. 2016. Energy Efficiency of Electrical Appliances, Equipment and Lighting Products Act of 2016. Port Vila. 18 Department of Energy, Vanuatu. 2016. Updated Vanuatu National Energy Road Map. Port Vila.

Appendix 3 49

the pilot projects, most but not all streetlights had been converted to energy efficiency fixtures. The experience was not used to reduce electricity consumption by other consumers because the private sector power company did not have an incentive to invest in measures that would reduce demand side energy. With regard to rooftop solar installations, Vanuatu does not have a buy-back mechanism for excess power. 42. Environment impact. A bulb eater was commissioned under the first project, which was operated by the municipal office. The government expressed concern that the safe disposal of used fluorescent lamps was proving difficult because of the lack of waste segregation. It also mentioned that some private sector companies were engaged in safe waste disposal. 43. Development partners. Other development partners did not provide assistance for energy efficiency on the demand side. However, the Updated Vanuatu National Energy Road Map, 2016–2030 was fully supported by Vanuatu’s development partners and provided a platform for coordination. UNELCO functions well as a concessionaire (an arrangement that will last until 2032); the power supply was restored fairly quickly after tropical cyclone Pam.

44. Issues for promoting energy efficiency. During the evaluation mission, the government and UNELCO staff shared several issues related to implementing energy efficiency measures; (i) higher income households had switched to more efficient refrigerators and inverter-type air conditioners, but their inefficient used appliances were often sold on to other users, so electricity waste continued; (ii) a decoupling mechanism is needed to break the link between the amount of energy sold and revenues because power companies did not have an incentive to engage in energy use side energy efficiency measures; (iii) while efficient appliances were available in the market and customers had easy access to energy efficiency-related knowledge, behavioural changes would take timee.g., hotel guests kept windows open so the electricity consumption of air conditioners remained high, or commercial consumers did not switch off lights even when rooms had ample sunlight; and (iv) energy audit recommendations were not being implemented because the investment had to be made by the landlord but the tenant was the main beneficiary.