Document of The World Bank Report No: …...2010/09/29 · Document of The World Bank Report No: ICR00001420 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-71850) ON A LOAN IN

Document of The World Bank

Report No: ICR00001420

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-71850)

ON A

LOAN

IN THE AMOUNT OF US$30 MILLION

TO THE

DOMINICAN REPUBLIC

FOR THE HEALTH REFORM SUPPORT (APL1)

FIRST PHASE OF THE

HEALTH REFORM SUPPORT PROGRAM

June 28, 2010

Caribbean Country Management Unit Human Development Sector Management Unit Latin American and the Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective: 04/12/2010)

Currency Unit = Dominican Pesos

1.00 = US$ 0.02 US$1.00 = DOP 36.17

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS APL Adaptable Program Loan ARS Health Risks Insurer (Aseguradora de Riesgos de Salud) CAS Country Assistance Strategy CERSS Executive Commission for Health Sector Reform

(Comisión Ejecutiva para la Reforma del Sector Salud) CMDB Minimum Set of Basic Data (Conjunto Mínimo de Datos Básicos) DDEI Directorate of Strategic Development (Dirección de Desarrollo Estratégico DGDF General Directorate of Drugs and Pharmacies

(Dirección General de Drogas y Farmacias) DGHA General Directorate of Authorization and Accreditation (Dirección General de Habilitación y Acreditación) DIDA Directorate of Social Security Beneficiaries’ Information and Advocacy (Dirección de Información y Defensa de los Afiliados a la Seguridad

Social) DIES General Directorate of Health Informatics and Statistics Dirección General de Informática y Estadísticas en Salud DR Dominican Republic EU European Union FMR Financial Monitoring Report GoDR Government of the Dominican Republic IADB Inter-American Development Bank ICB International Competitive Bidding ISO International Organization for Standardization ISR Implementation Status and Results Report MDGs Millennium Development Goals MEPD Ministry of Economics, Planning, and Development (Ministerio de Economía, Planificación y Desarrollo) MoH Ministry of Health PAD Project Appraisal Document PARSS Project in Support of the Health Sector Reform Proyecto de Apoyo a la Reforma del Sector Salud

PASS DPL Performance and Accountability of Social Sectors Development Policy Loan (PASS DPL)

PBS Basic Health Plan (Plan Básico de Salud) PCU Project Coordination Unit (Unidad Coordinadora del Proyecto) PDO Project Development Objective POA Program of Activities PROMESE/CAL Essential Drugs Program/Logistics Support Center (Programa de Medicamentos Esenciales/Central de Apoyo Logístico) PRP Prioritized Regions and Provinces (Provincias y Regiones Priorizadas) QAG Quality Assurance Group QER Quality Enhancement Review RS Subsidized Regime (Régimen Subsidiado) SENASA National Health Insurance

(Seguro Nacional de Salud) SESPAS Secretariat of Public Health and Social Assistance (Secretaría de Estado de Salud Pública y Asistencia Social) SFS Family Health Insurance (Seguro Familiar de Salud) SISALRIL Health and Labor Risks Superintendence (Superintendencia de Salud y Riesgos Laborales) SIUBEN Unique System of Beneficiaries Sistema Unico de Beneficiarios SRS Regional Health Services (Servicios Regionales de Salud) STP Secretariado Técnico de la Presidencia (Technical Secretariat of the Presidency) UAP Primary Health Care Unit

(Unidad de Atención Primaria) UEP Project Implementation Unit (Unidad Ejecutora del Proyecto) UMDI Modernization and Institutional Development Unit USAID United States Agency for International Development

Vice President: Pamela Cox

Country Director: Yvonne Tsikata

Sector Manager: Keith Hansen

Project Team Leader: Fernando Montenegro Torres

ICR Team Leader: Fernando Montenegro Torres

Dominican Republic

Health Sector Reform Support Project (APL1)

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 12. Key Factors Affecting Implementation and Outcomes ............................................ 114. Assessment of Risk to Development Outcome ......................................................... 315. Assessment of Bank and Borrower Performance ..................................................... 326. Lessons Learned ....................................................................................................... 357. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 36Annex 1. Project Costs and Financing .......................................................................... 37Annex 2. Outputs by Component ................................................................................. 39Annex 3. Economic and Financial Analysis ................................................................. 45Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 46Annex 5. Beneficiary Survey Results ........................................................................... 48Annex 6. Stakeholder Workshop Report and Results ................................................... 48Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 49Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 57Annex 9. List of Supporting Documents ...................................................................... 57Annex 10. Key performance indicators as included in the original PAD .................... 58Annex 11. Summary of the Final Report on the Investment Activities of PARSS 1 ... 61Map ............................................................................................................................... 64

i

A. Basic Information

Country: Dominican Republic Project Name: DO: Health Reform Support (APL)

Project ID: P076802 L/C/TF Number(s): IBRD-71850

ICR Date: 06/30/2010 ICR Type: Core ICR

Lending Instrument: APL Borrower: DOMINICAN REPUBLIC

Original Total Commitment:

USD 30.0M Disbursed Amount: USD 29.3M

Revised Amount: USD 29.3M

Environmental Category: B

Implementing Agencies: CERSS

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 09/23/2002 Effectiveness: 01/06/2005 01/06/2005

Appraisal: 05/06/2003 Restructuring(s): 12/13/2006

Approval: 06/26/2003 Mid-term Review: 10/30/2007 10/16/2007

Closing: 12/31/2007 12/31/2009 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Unsatisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Unsatisfactory

Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory

Government: Moderately Satisfactory

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:


Overall Borrower Performance:


ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

Satisfactory

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Unsatisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Health 50 50

Non-compulsory health finance 50 50

Theme Code (as % of total Bank financing)

Health system performance 100 100 E. Bank Staff

Positions At ICR At Approval

Vice President: Pamela Cox David de Ferranti

Country Director: Alan G. Carroll Caroline D. Anstey

Sector Manager: Keith E. Hansen Evangeline Javier

Project Team Leader: Fernando Montenegro Torres Patricio V. Marquez

ICR Team Leader: Fernando Montenegro Torres

ICR Primary Author: Maria R. Puech Fernandez F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The design of the Adaptable Program Loan (APL) was based on three phases, each with an estimated duration of 4 years. The overarching objective of the APL was contributing to the achievement#by the year 2015#of the Millennium Development Goals (MDGs) agreed to by the Government of the Dominican Republic (GoDR) at the Millennium Summit held in September 2000. Specifically, the APL would (i) address three MDG goals: reduce child mortality, improve maternal health, and contribute to the eradication of extreme poverty by protecting the poorest from financial loss due to ill health and disability; (ii) support the implementation of a new legal framework for the health sector, particularly aimed at the institutional strengthening of SESPAS in its new stewardship role, the development of insurance mechanisms, and the creation of regional health

iii

networks; and (iii) support the preparation, validation, and dissemination of norms and regulations supplementary to the General Health Law (42-01) and the Social Security Law (87-01). The original Project Development Objectives (PDOs) of the first phase of the APL, also known as PARSS1 for its Spanish Acronym (Proyecto de Apoyo a la Reforma del Sector Salud Fase 1), were to: (i) contribute to the eradication of extreme poverty by protecting the poorest from financial loss due to ill health and disability through enrollment of the target population (i.e., people living below the relative poverty line in the regions III, IV, VI, VII, and VIII) in the government health insurance plan and making available and promoting the use of basic health services of high-quality standards; (ii) support the implementation of the health sector reform legislation that would provide the legal, financial, organizational, and managerial enabling environment to achieve the MDGs related to health. Revised Project Development Objectives (as approved by original approving authority) A first-order restructuring was signed on February 2007. The new PDOs introduced during this restructuring were to support the Borrower in: (i) improving access of the poor living in the Prioritized Regions and Provinces (Provincias y Regiones Priorizadas or PRPs) to high-quality health services included in the Basic Family Health Insurance Plan (Plan Básico de Salud del Seguro Familiar); (ii) reorganizing and strengthening the health systems in the Prioritized Regions so they can fulfill the roles and functions as established in the General Health Law and the Social Security Law; (iii) strengthening the financial planning of the Subsidized Regime (Régimen Subsidiado or RS), ensuring the effective enrollment of the poor, and improving their knowledge of their rights under the Family Health Insurance (Seguro Familiar de Salud or SFS); and (iv) strengthening the capacity of national health sector organizations to fulfill their roles and functions as established in the General Health Law and the Social Security Law. At the request of a new administration that took office in August 2004 (5 months prior to Project effectiveness), the project was restructured. Extensive discussions and some analytical work were carried out to identify key bottlenecks and priorities. The process was designed to build consensus among key stakeholders and to ensure consistency of Project activities with the new government priorities#accelerating health sector reform and expanding health insurance coverage to the poor. New PDOs and indicators were agreed on, which included some trigger indicators for the next phase of the APL. The indicators used in this ICR to assess the achievements of the Project are those introduced in the Project#s restructuring of December 2006. For a table of the original APL PDOs and the trigger indicators, see Annex 10 of the ICR. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of public primary health care facilities in PRPs that are accredited

iv

with SESPAS Value quantitative or Qualitative)

0 50% 33%

Date achieved 09/30/2007 12/31/2008 12/31/2009 Comments (incl. % achievement)

Not completed due to legal and regulatory changes to human resources laws and regulations not taken by government.

Indicator 2 : Percentage of public health units in PRPs that have tracer drugs that are part of the Basic Health Plan (Plan Básico de Salud) available for the population enrolled in the RS

Value quantitative or Qualitative)

60% 80% 23

Date achieved 01/31/2007 12/31/2008 12/31/2009

Comments (incl. % achievement)

A new inter-agency agreement allowed for a more efficient solution. Decision makers agreed to use not only health centers but PROMESE/CAL discount pharmacies as points of services for those health centers which is measured by this indicator.

Indicator 3 : Percentage of pregnant women who receive prenatal care services in PRPs in accordance with national guidelines


8% > 50% 14%

Date achieved 03/31/2008 12/31/2008 12/31/2009


Government develop evidence-based guidelines with broad stakeholders# consultations taking longer than expected. New multi-sectorial approach supported by a new DPL includes training on new guidelines will be one of the triggers for its second phase.

Indicator 4 : Percentage of infants who receive routine check-ups in PRPs in accordance with national guidelines


> 50% 9%

Date achieved 12/31/2008 12/31/2009 Comments (incl. % achievement)

S

Indicator 5 : Percentage of personnel in PRPs who are familiar with the national guidelines for maternal and child care


26% 80% 32%


S

Indicator 6 : Percentage of managers and administrators of second-level hospitals in PRPs that

v

were hired through a competitive recruitment process over the last 12 months


0

No specific target was defined at the time of restructuring

No activity was carried out under the Project regarding this indicator.


Achievement of this target depended on changes to laws and civil service policies. New law has not been approved by Congress and is not clear final outcome.

Indicator 7 : Percentage of secondary care facilities in PRPs that disseminate quarterly performance reports


0

No specific target was established at the time of restructuring

Date achieved 01/31/2007 12/31/2008 Comments (incl. % achievement)

Percentage of secondary care facilities in PRPs that disseminate quarterly performance reports

Indicator 8 : The design of an integrated financial and performance management system completed, populated, and implemented


0

June 2007: Design of an integrated financial and performance management system completed; December 2007: Integrated financial and performance management system piloted in the Regional Health Services (Servicios Regionales de Salud or SRS); June 2008: Syst

Integrated financial and performance management system designed. Training carried out in Regions VIII and VI. Ongoing training in Region V.

Date achieved 01/31/2007 12/31/2008 12/31/2009 Comments In 2010 the MoH and SENASA agreed to use new governmental system SIGEF

vi

(incl. % achievement)

as a part of a set of tools for management of the new results-based financing mechanisms for the first level of care that APL2 will contribute to finance.

Indicator 9 : Number of annual performance agreements between SESPAS and the SRS that include incentives to improve performance


0 SRS of Regional III, IV, V, and VI

By Project completion, Regions VI and VIII had signed a results-based agreement with SESPAS.


Results based agreements were signed in the last quarter of 2009 between the MoH and SRS of Regions VI and VIII. By March 2010 all health regions signed performance agreements

Indicator 10 : Percentage of the budget for health care services that was transferred to the Prioritized Regions through SENASA


4.4% 8% 17.5%


The successful achievement of this objective exceeds the target. At the time of the Project closing, about one-third of the total target population (1.3 million out of 3.2 million) were enrolled.

Indicator 11 : SESPAS, SENASA, the SRS, and the STP have developed, approved, and implemented a multi-annual financing plan for the RS in the PRPs (2008#2010)


0

June 2007: Plan designed; September 2007: Plan approved; January 2008: the Plan in to be reflected in the 2008 national budget

Not met as described


Indicator 12 : Percentage of poor living in PRPs that are enrolled in the RS who know the benefits of the Family Health Insurance Plan (Seguro Familiar de Salud or SFS)


45% 70% 57%

Date achieved 01/31/2007 12/31/2008 12/31/2009 Comments (incl. %

Success in the rapid expansion of enrolled population outpaced plans strategy for communication that considered a smaller population.

vii

achievement)

Indicator 13 : Percentage of children born alive in public hospitals over the preceding 12 months for which the hospitals have an electronic copy of the birth certificate on file.


0 >80% 14.5%


Logistical and technical difficulties delayed implementation. PCU estimates that 34 hospitals that account for more than 80 percent of deliveries can fully start providing birth certificates over the next 18th months.

Indicator 14 : In SESPAS: monitoring and evaluation system designed, populated, and results disseminated.


0

May 2007: System designed; October 2007: System populated; January 2008: First performance report disseminated

Indicator not achieved.


Government decided to avoid fragmentation and created the new national monitoring and evaluation unit with the objective of integrating dispersed efforts avoid duplicity and overlapping and enhance efficiency.

Indicator 15 : SENASA: monitoring and evaluation system designed, populated, and results disseminated.


0


Indicator not achieved.


Government decided to avoid fragmentation and created the new national monitoring and evaluation unit with the objective of integrating dispersed efforts avoid duplicity and overlapping and enhance efficiency.

Indicator 16 : Percentage of drugs and medical supplies (by value in $) procured by PROMESE/CAL through international public bidding procedures.

Value quantitative or

0 20% Norms to carry out international

viii

Qualitative) bidding process prepared and personnel trained.


Government prioritized acquisition of the most expensive medicines through joint Central America and DR price negotiations and for HIV/AIDS drugs an agreement with the PAHO. PROMESE will conduct distribution

Indicator 17 : CERSS: Monitoring and Evaluation system designed, populated, and results disseminated.


0


The system was designed by Project completion (April 2009). Not fully implemented.

Date achieved 01/31/2007 12/31/2008 12/31/2009


Same comment as indicators 14 and 15. The software and system for health sector reform monitoring was transferred to the MoH to ensure that will be integrated within the new M&E policies and systems. This was a trigger indicator.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 :

An explicit differentiation between final and intermediate outcomes was not made during restructuring given the nature of what they intended to measure. PDOs indicators 5, 8, 13, and 14 were selected as intermediate outcome indicators in ISR reporting.

Value (quantitative or Qualitative)

Date achieved Comments (incl. % achievement)

ix

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 11/17/2003 Satisfactory Satisfactory 0.00 2 06/03/2004 Satisfactory Satisfactory 0.00 3 11/24/2004 Satisfactory Unsatisfactory 0.00 4 03/03/2005 Satisfactory Satisfactory 1.55 5 06/21/2005 Satisfactory Satisfactory 1.55

6 06/23/2005 Moderately

Unsatisfactory Moderately

Unsatisfactory 1.55



Unsatisfactory 2.08



Unsatisfactory 4.07

9 12/21/2006 Moderately Satisfactory Moderately Satisfactory 4.60 10 06/13/2007 Moderately Satisfactory Moderately Satisfactory 6.03 11 12/13/2007 Moderately Satisfactory Moderately Satisfactory 9.65 12 06/30/2008 Moderately Satisfactory Moderately Satisfactory 14.16 13 10/03/2008 Satisfactory Satisfactory 16.91 14 06/23/2009 Moderately Satisfactory Moderately Satisfactory 22.72 15 12/18/2009 Unsatisfactory Moderately Satisfactory 27.44

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

12/13/2006 MS MS 4.60

x

I. Disbursement Profile

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1. At the time of appraisal, mother and child health outcome indicators in the Dominican Republic were considerably lower than those in other countries with comparable income per capita. Infant mortality rates were 31 per 1,000 live births, with persistent levels of neonatal mortality. The maternal mortality rate was higher than that of other countries in the region of comparable per capita income and studies suggested that 93 percent of these deaths were preventable. In 1999 maternal mortality was estimated at 113 per 100,000 live births, while it was 33 and 35 in Chile and Costa Rica respectively. 2. The provision of health care services was highly fragmented. The health system consisted of multiple public and private providers, operating with loose guidelines, minimal regulations, and overlapping functions. There was scarce coordination to guarantee universal access to quality and efficient health services. At appraisal, the poor quality of health services was considered an important factor linked to poor maternal and child health outcomes. Weaknesses identified in public sector health services included lack of planning and gaps in infrastructure and equipment, particularly for delivering primary health care services; and, at the first level of care, irregular supply of pharmaceuticals and medical inputs. Another important area that required strengthening was policy development and implementation, particularly regarding health sector planning and the management and allocation of human resources in the health sector (particularly at the first level of care). 3. The public sector was the largest provider of health services for the majority of the population. However, the financing of health care services was regressive and inequitable, and resources were not allocated efficiently—few funds being provided for the first level of care. Public sector health spending made up 2 percent of GDP, among the lowest in the region. The two largest providers of health services—the MOH and the Dominican Institute of Social Insurances (Instituto Dominicano de Seguros Sociales or IDSS)—accounted for less than a quarter of total health care spending while out-of-pocket expenses represented more than half of total national spending on health. 4. After a decade of discussions, analyses, and consensus building, in 2001, two critical laws were approved: the General Health Law (N. 42-01, passed on May 8, 2001) and the Social Security Law (N.87-01, passed on May 18, 2001). These laws constituted a major health sector reform effort and their provisions aimed to overhaul the financing, organization, management, and delivery of services in the country. The laws envisioned a 10-year period of transition in order to initiate and complete the full development and implementation of operational, regulatory, financial, institutional, and organizational arrangements that are involved in such a complex implementation. 5. The General Health Law introduced a separation of functions between the stewardship over the health system (a role assigned to the MoH) and the provision of health care services (a role assigned to public and private health care providers). This law established a process of decentralization of the management of health care services (SRS), which were to be organized as public and autonomous networks of health care providers within the framework of a primary health care approach. This Law also created the National Health Council (NHC), which was made responsible for the formulation, follow-up, and evaluation of health policy.

2

6. The Social Security Law introduced mandatory health insurance for all citizens, while separating the functions of financing, risk pool management, and provision of health services. Mandatory health insurance, known as SFS, would be implemented through a family insurance system with a unified and explicit package of benefits—the Basic Health Plan (Plan Básico de Salud or PBS). Three regimes were established: a contributory one for workers in the formal sector and their families; a subsidized one for the poor and individuals from families living below the poverty line who are not covered by the formal sector, and a partially subsidized regime for autonomous workers and other individuals outside the formal sector who are not eligible for the RS. 7. The National Health Insurance (Seguro Nacional de Salud or SENASA) was established as an autonomous public entity that would act as a mandatory risk-pooling agency (Aseguradora de Riesgos de Salud or ARS) for the RS, to be financed fiscally via health insurance premiums. The SRS were established as the mandatory providers of health care services for the RS. 8. The financing of the contributory system would be based on a payroll tax system. Individuals in the contributory system could opt either for private risk-pooling agencies (ARS) or for SENASA (financed through the payroll tax system). The Social Insurance Treasury was established and, among other duties, given the responsibility for collecting revenues and managing the financial pools for all regimes. Another public agency, the Health and Labor Risks Superintendence (Superintendencia de Salud y Riesgos Laborales or SISALRIL) was established to audit, monitor, and evaluate public and private ARSs to ensure the soundness and sustainability of their financial management. 9. Through the Provincial Health Services Project (Report N.17199 DO), the World Bank supported the GoDR in its efforts to develop and build consensus on the health sector reform’s legal framework, which ultimately resulted in the 2001 Laws as well as the development of numerous regulations needed to actually implement these Health Sector Reform Laws. In the years prior to the preparation of this Project, other donors and multilateral institutions—such as the IDB, the EU, and USAID—also financed projects to introduce piecemeal changes to the public sector health system.1 10. In line with the GoDR sector priorities and the CAS,2 the GoDR and the Bank agreed on an APL to support the health sector reform. The Program was viewed by the GoDR as a valuable instrument to assist the country in achieving its long-term sector objectives, given the long-term horizon of the sector reform program and the preparation of a 10-year Sector Development Plan. Moreover, the global impetus to set ambitious national goals to reduce poverty and improve health sector outcomes—given by international, multilateral, and national organizations through the development and endorsement of the MDGs—contributed significantly to the engagement of the GoDR. The government clearly saw the new laws and the new Bank Project as an opportunity to accelerate the introduction of changes in the health sector, thereby improving its performance and achieving various key MDGs. The country was one of the signatories of these internationally agreed objectives and public authorities of various sectors had agreed to meet them.

1 Technical assistance was provided by UN agencies (PAHO/WHO, UNDP, and UNICEF). This support had brought about a geographical fragmentation in the implementation of health reforms.

2 CAS – Doc. 19393 –DO. Discussion June 9, 1999.

3

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 11. The APL aimed at: (a) contributing to achieve by the year 2015 the MDGs agreed to by the GoDR at the Millennium Summit held in September 2000. Specifically, the APL was supposed to address three MDG goals: reduce child mortality, improve maternal health, and contribute to the eradication of extreme poverty by protecting the poorest from financial loss due to ill health and disability; (b) supporting the implementation of the new legal framework for the health sector, particularly the institutional strengthening of SESPAS in its new stewardship role, the development of insurance mechanisms, and the creation of regional health networks; and (c) supporting the preparation, validation, and dissemination of norms and regulations supplementary to the General Health Law (42-01) and the Social Security Law (87-01).

12. The APL was to achieve the above mentioned development objectives in three phases over a period of 12 years. There were trigger indicators to move from Phase I to Phase II, and then to Phase III (see Annex 10 for a table of these indicators).

13. Specifically, the Project would: (i) contribute to the eradication of extreme poverty by protecting the poorest from financial loss due to ill health and disability through the enrollment of the target population in the government health insurance plan and by making available and promoting the use of basic health services of high-quality standards. The target population would be people living below the relative poverty line in regions II, IV, VI, VII, and VIII; and (ii) support the implementation of the health sector reform legislation that provides the legal, financial, organizational, and managerial enabling environment to achieve the MDGs related to health.

14. The original Project design included the following key performance indicators: Directly linked to the MDGs:

Reduce Extreme Poverty and Hunger - Target: by 2015, protect the poorest from financial loss due to ill health; - Indicators: percentage of families with health insurance; prevalence of underweight children (under 5 years of age), low birth weight, and nutritional status;

Improve Child Health - Target: by 2015, reduce under-five deaths by two-thirds; - Indicators: percentage of underweight newborns; percentage of children 12–23 months

old fully immunized; infant mortality rate; under-five mortality rate; mortality and morbidity from upper respiratory infections, diarrhea and measles.

Improve Maternal Health - Target: by 2015, reduce by three-quarters the maternal mortality rate; - Indicators: percentage of pregnant women with four prenatal visits; percentage of births

attended by skilled health personnel; maternal mortality rate; contraceptive prevalence rate; total fertility rate; and percentage of cesarean sections.

Indirectly linked but supporting the MDGs:

- Anchoring the MoH’s stewardship role: national health policies issued, consistent communication strategies and public information campaigns organized, performance monitoring and evaluation.

4

- Promoting the decentralization and autonomy process

- Improving secondary and tertiary facilities: physical structures refurbished,

electromechanical and biomedical waste management equipment adequately maintained, productivity incentives developed, staff trained and duly performing, information systems operating.

- Consolidating national essential public health functions

15. The original Project design included the following outcome/impact indicators: PDO 1: Increase the proportion of the target population using health services provided under the Basic Health Plan (PBS). Indicators:

Percentage of pregnant women with at least four prenatal care visits; Percentage of births attended by skilled personnel; Percentage of women in reproductive age who use modern family planning methods; Percentage of children 12 to 23 months old who are fully immunized (DPT, polio, and

measles); and Percentage of children under 5 with diarrhea who have been medically diagnosed and are

being treated. PDO 2: Improve the quality of services provided under the PBS to the target population. Indicators:

Percentage of women receiving prenatal care who are immunized against tetanus; Percentage of women diagnosed with pre-eclampsia and referred to tertiary facilities; Percentage of staff working at Health Service Providers’ facilities present during working

hours; Percentage of target population satisfied with the quality of the PBS services.

PDO 3: Remove financial barriers to access and protect the target population from the financial consequences of ill health. Indicators:

Percentage of the population eligible for government services under the Social Security Law that is enrolled in the National Insurance Plan;

Percentage of private insurance expenditures as a percentage of total health expenditures. PDO 4: Improve the efficiency of the health system. Indicators:

Percentage of SENASA’s financial resources allocated to primary care services; Nurse/midwife per doctor ratio for Health Service Providers (excluding auxiliary nurses); Hospital beds per 1,000 population for Health Service Providers; Percentage of administrative and managerial positions created in the reform process at the

regional, provincial, and municipal level of SENASA that are filled by SENASA employees.

5

(See annex 10 for a table of outcome indicators with baseline values and targets, as included in the PAD) 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and

reasons/justification 16. In 2004–05 health sector authorities of the incoming administration developed a new national strategic agenda for the health sector reform, emphasizing consensus building, improved planning, and cooperation among public sector agencies (“Agenda Estratégica Nacional para la Reforma del Sector Salud 2005”). In that same year, the UN selected the Dominican Republic as one of the “pilot countries”—along with Ethiopia, Ghana, Kenya, Senegal, Tajikistan, and Yemen—to help identify the best ways to integrate MDG targets and timelines into their national strategies to reduce poverty. The aim was for these national strategies to serve as models for similar undertakings in developing countries throughout the world. 17. In early 2005, during the very first meetings with the Bank’s team after the Project had become effective, the GoDR requested that a process be initiated to adjust the Project to ensure its relevance, given the time elapsed between the early stages of preparation and effectiveness and the evolution of all the institutions established by the Health Sector Reform Laws of 2001. In addition, the government sought to use the Project as a tool to leverage the momentum deriving from the political commitment to: (i) expand health insurance coverage among the poor in those areas with the largest concentration of low-income families; and, (ii) accelerate the decentralization of the SRS, while developing primary health care networks of public providers. As a result, the Project initiated a process of extensive consultations, some analytical activities, stakeholder analyses, and consensus building among all key actors across multiple organizations and agencies from the health sector. This restructuring agreement involved first-order changes, which the Board of Executive Directors of the Bank approved on December 13, 2006, and the restructuring documents were duly signed in February 2007. 18. The new PDOs after the first-order restructuring were to: (i) improve access of the poor people entitled to the RS of the SFS living in prioritized regions and provinces to high-quality health services included in the insurance’s benefits package (Plan Básico de Salud); (ii) reorganize and strengthen the health systems in prioritized regions so they can fulfill the roles and functions as established in the General Health Law and the Social Security Law; (iii) strengthen the financial planning of the RS, ensuring the effective enrollment of the poor and improving their knowledge of their rights under the Seguro Familiar; and (iv) strengthen the capacity of national health sector organizations so they can fulfill their roles and functions as established in the General Health Law and the Social Security Law. The objectives of the APL were not modified at the time of restructuring but the outcome indicators as well as the trigger indicators were. The latter are shown in the results framework tables and in annex 10 (trigger indicators).

1.4 Main Beneficiaries, 19. The APL and the original Project focused on activities that ultimately were expected to contribute to gains in the health status of the poorest population groups in selected provinces of the country, which the previous World Bank project had also targeted, most of them in provinces that bordered Haiti. However, the PAD stated that the population covered under the Project, Phase I of the APL, was to be 2.3 million (28 percent of the total population), of which 1.7 million were living under the poverty line, as defined at the time of preparation, which seems to

6

correspond to the total number of individuals expected to be covered by the health insurance for the poor (RS). 20. The restructured Project continued to target the poor population as a whole while the activities in the Project components were prioritized by region (Regions II, IV, and VI). Activities in Component IV and some of the activities in Component III were to benefit the entire population of the country. 21. The second amendment of the Loan expanded the geographical scope of activities of Component I, allowing for the financing of critical improvements in the health delivery system nationwide, introducing the possibility of expanding the population covered under the Project.3

1.5 Original Components (as approved) 22. The original design of APL1 included four components that were meant to be included in all of the three phases of the programmatic lending that supported the health sector reform. The PAD outlined the main activities to be carried out under each component during Phase I. The main components were the following: Component I: Supporting health services extension to achieve universal coverage (total cost: US$25.97 million or 60.8 percent of total Project cost). This component would follow up on and extend the achievements of the Provincial Health Services Project4 and would: (i) complete coverage of the poor population with a basic health plan, emphasizing maternal and child care; (ii) add Region VI, with the provinces of San Juan de la Maguana and Elías Piña; and (iii) add the provinces of La Vega and Monseñor Nouel to complete the coverage of Region VIII. All of these regions had the largest proportion of poor people in the country. Component II: Supporting implementation of the health sector reform legislation (total cost: US$7.20 million or 16.8 percent of total Project cost). This component would: (i) assist in building the institutions to implement the reformed health sector legislation, by supporting the development of regulatory frameworks, organizational structures, financial instruments, training programs, and management information systems; and (ii) assist in providing the legal and financial enabling environment for achieving the objective of Component I. Component III: Improving selected essential public health functions to promote and protect the health of the population (total cost: US$2.66 million or 6.2 percent of total Project cost). This component would finance two essential public health functions: (i) epidemiological health surveillance and control of risk in public health, by strengthening the information system for epidemiological surveillance, and the capacity and quality control of the National Public Health

3 The Resolution N. 43-02 of the Borrower’s National Social Security Council established that the enrollment of the poor in the RS of the Family Health Insurance would be achieved gradually, in the first phase focusing on four of the country’s nine health regions, prioritized by poverty level. The first phase was completed faster than anticipated and in 2008 the poor were enrolling in the RS across the nation. Enrollment nationwide had become a priority for the health authorities. Accordingly, the GoDR expanded geographically its efforts to strengthen the delivery of health services.

4 The Provincial Health Project (Ln. 4272-DO) was still under implementation when the present Project was prepared and approved by the Executive Board. It supported the provision of primary care services to the poorest population, with emphasis on maternal and child care in three regions (III, IV and VII), one province of Region VIII (Sánchez Ramírez), and in one health area of the national district (Santo Domingo Centro).

7

Laboratory, the cold storage facility (cold chain); establishing a national pharmacological surveillance system; and relocating and strengthening the Forensic Pathology Department; and (ii) the development of policies, plans, and management capacity for assuring SESPAS’ stewardship by supporting the design of a 10-year national strategic plan to gradually build up the national health system and the social security system. Support was to be provided to carry out policy and operational studies, including one that would focus on the training of physicians, nurses, and other health personnel. Component IV: Project management and impact appraisal of next phases (total cost: US$6.58 million or 15.4 percent of total Project cost). This component would support Project management, including the strengthening of the monitoring and impact evaluation system to measure the results achieved during implementation.

1.6 Revised Components 23. The first amendment, which was the result of extensive discussions and consultations with health sector stakeholders, was finally signed in December 2006 and involved first-order changes: the original APL1 objectives and outcome indicators were changed as well as the Project’s components. The restructuring replaced the original design with a new one, in which all activities were organized around four components as follows: Component I: Improve access of the poor in prioritized regions and provinces to high- quality health services included in the benefits package (Plan Básico de Salud) of the Family Health Insurance (Seguro Familiar de Salud). (Total cost: US$15.6 million or 38.5 percent of total Project cost). This component would finance activities that would improve access to high-quality health services in prioritized regions and provinces to meet the increasing demand of the poor enrolled in the Régimen Subsidiado. In particular, activities to close gaps in the infrastructure network and improve the condition of buildings and equipment, the geographical distribution and management of human resources for health, the clinical management of patients, the supply and use of pharmaceuticals, and the general management of hospitals. Component II: Reorganize and strengthen health systems in prioritized regions so they can fulfill the roles and functions as established in the general health and social security laws. (Total cost: US$1.2 million or 3 percent of total Project cost). This component would finance: (i) the reorganization of the SRS. This would include consultancies and training to consolidate the model of administration, as well as the management and delivery of health services through the development and implementation of operational plans; (ii) the development of a Health Management Information System, including subsystems for financial management and service delivery; and (iii) the elaboration and negotiation of performance agreements and contracts between the regional health services and SESPAS, and between the former and SENASA, and the design and implementation of mechanisms to enforce them. Component III: Develop a multi-annual financing plan for the Régimen Subsidiado, ensure the effective enrollment of the poor, and improve their knowledge of entitlements. (Total cost: US$1.3 million or 3.3 percent of total Project cost). This component would: (i) strengthen the coordination and planning of financing of the Subsidized Regime; (ii) enroll and empower the population eligible for the Subsidized Regime. Activities under this component would include consultancies, training, the purchase and installation of hardware and software, the acquisition of technical equipment, and the production of informational material to improve the process of enrollment of the eligible population in the Subsidized Regime—among other things, by

8

strengthening the birth registration system and empowering the enrolled population to use the health care services of the PBS. Component IV: Strengthen the capacity of the public health organization to fulfill its new roles and functions as established in the general health and social security laws. (Total cost: US$6.1 million or 15.6 percent of total Project cost). This component would finance activities that build the capacity of national public health sector agencies to fulfill the roles and functions established in the General Health Law and the Social Security Law. Priority would be given to SESPAS, SENASA, PROMESE/CAL, the Technical Secretariat of the Presidency (Secretariado Técnico de la Presidencia or STP), and CERSS, and to activities aimed at establishing monitoring and evaluation systems for the sector, the reform process, and its impact. The component would also include activities to ensure the appropriate management of the Project. Explanation of Changes in the Project’s Components

24. A new administration took office in 2004 and by December of that year, the project finally became effective (Board approval took place in June 2003). From the early stages of preparation in 2002 to the beginning of actual implementation in 2005, important progress was made by the country in financing, staffing, and providing adequate infrastructure for key, central-level institutions as needed to comply with the mandate of universal health insurance. For instance, the Social Insurance Treasury, SENASA, and SISALRIL had already been established and were functioning by then; however, the number of covered individuals was still minimal (less than 70,000 of the total target population estimated at 3.2 million individuals). 25. The new administration placed great emphasis on managing a comprehensive approach to health sector reform, ensuring a nationwide scope of action and prioritization of interventions to benefit the poor, and reaffirming the stewardship role of the MoH in the entire health system. At the same time, the government sought to exploit synergies with other poverty reduction policies by fostering coordination with other stakeholders, public agencies, donors, and international organizations working in other sectors. This represented an important change with respect to past approaches, when various health sector stakeholders, donors, and multilateral agencies had actually exacerbated sectoral fragmentation with their piecemeal approaches, frequently associated with the de facto geographical division of the country. 26. The head of the Executive Commission for Health Sector Reform (CERSS)—who in August of 2005 became the minister of Health—and his team produced a document on the health sector reform entitled National Strategic Agenda for Health Sector Reform 2005, summarizing the vision for health sector reform of the new administration. The GoDR requested that the Bank, in the implementation phase, support focusing on strengthening the stewardship function of the MoH while cultivating an environment conducive to collaboration and dialogue among all public sector agencies involved. 27. At the request of the government, the Bank team launched a series of analytical studies, workshops, discussions, and consensus-building activities during the first 18 months of implementation, leading to the identification of priorities and a sequence of interventions (critical path) designed not only to culminate in the restructuring of the Project but also to ensure ownership of the reform process by all stakeholders.

9

Strategic Lines of the National Sector Agenda for Health Reform

(summary extracted from the document “National Strategic Agenda for Health Sector Reform 2005”)

The key areas of focus of the new administration were the following issues:

Strengthening the stewardship role of SESPAS, with emphasis on the basic public health functions and the reorganization of programs in the framework of the reform.

Development of the Regional Health Networks. Ensuring equitable access to quality medicines. Ensuring health insurance coverage with an emphasis on the poor to:

i. Ensure access to quality health services ii. Enhance the demand side

iii. Accelerate the social protection of the poor via their enrollment into the Régimen Subsidiado (subsidized regime) of the Seguro Familiar (Family Insurance) in prioritized regions and provinces

Moreover, two overarching intervention areas were included as cross-cutting topics for the agenda:

Strengthening Human Resources in the health sector (performance and management) Development and implementation of a management system based on modules

28. The Project was redesigned to reflect new organizational, institutional, and political realities that favored an accelerated reform pace and raising financial resources for the health insurance of the poor. In the process, the GoDR with the support of the Bank, built consensus and established communication bridges among key organizations that previously had been working under a silo approach, in an uncoordinated fashion, which constituted one of the greatest bottlenecks for the reform since the relevant laws had been passed in 2001. 29. As a result, the Bank supported the GoDR initiatives to use the Project activities to jumpstart a collective effort involving all key public health sector organizations (MoH, SENASA, and the Ministry of Economy, Planning, and Development (Ministerio de Economía, Planificación y Desarrollo or MEPD) to re-launch the reform process, with emphasis on social protection for the poorest families. The critical path formed a framework for making selective investments, aimed at laying the basis for a new approach to strengthening the infrastructure, organization, and management of health care. Within this overall framework, emphasis was to be placed on the supply of services at the first level of care and on the long-term goal of maximizing the impact on preventive health care and care interventions for the poor and most vulnerable groups of the population.

1.7 Other significant changes Financial Management

30. The Project components also came to represent the disbursement categories—thereby reducing the total number of categories that had to be documented and monitored as Project components (as opposed to type of expenditures) from seven to four. To simplify financial management and Project execution, the Bank’s financing share was increased to 80 percent across all Project activities. This unified share was higher than the original one (70 percent), but still lower than the maximum allowed by the new Country Financing Parameters (90 percent). These

10

changes were in line with the Country Financing Parameters and Bank Norms, which have been updated since the Project’s approval. 31. The increase in the share of Bank financing to 80 percent sought to enable the implementation of the first 12-month Procurement Plan of the restructured Project (US$14 million), without major changes to the counterpart fund allocations already approved for 2006 and requested for 2007. This change resulted in a reduction of the Project size from US$42.7 to US$39.1 million (see table 1). Despite this reduction, the available resources were believed to be sufficient to achieve the revised PDOs.

Table 1: Financing Plan—Original versus restructured Project

Original Project Restructured Project Borrower

US$12.7 million (30%)

US$9.1 million (23%)

IBRD US$30.0 million (70%) US$30.0 million (77%) Total US$42.7 million (100%) US$39.1 million (100%)

Procurement

32. The Bank’s revised procurement guidelines (May 2004) were adopted at restructuring. This meant: (i) the introduction of additional procurement methods (e.g., Quality-Based Selection, Selection under a Fixed Budget, and Single-Source Selection) and the abolishment of aggregate thresholds for all procurement methods, providing more flexibility and efficiency in the use of loan funds; and (ii) the mandatory use of a procurement plan to support the planning and coordination across implementing agencies. 33. The closing date of the Loan was extended by 12 months, that is, through December 31, 2008. This first extension sought to partially compensate for the 18-month delay in Project effectiveness caused by late congressional approval and subsequent implementation delays owing to Project design problems.

Second Amendment (July 2008)

34. Additional changes to the Project were requested by the GoDR and agreed with the Bank at the time of the Mid-Term Review of the Project. The GoDR had allocated increasing amounts of financial resources to accelerate the expansion of health insurance coverage for the poor, which led to the need to support the GoDR in several activities nationwide. Although great progress had been made at the highest level among public health sector ministries and agencies, important constraints remained within some institutions, in particular the MoH. 35. Keeping the required high levels of internal coordination and joint planning and implementation of activities was a moving target, since the MoH itself had embarked on an ambitious agenda of institutional reengineering. The first restructuring process had contributed to building a broad consensus on the mid- and long-term vision of the reform and on how the Project would benefit the activities in each institution. However, the GoDR considered it important to reinforce the capacity of the MoH to steward the reform process and expand the scope of efforts to ensure that all the key public agencies could keep pace with the rapidly growing population covered by health insurance for the poor.

11

36. The GoDR felt that some constraints in the coordination capacity at the horizontal level (among various branches of the MoH) and vertical level (between the central level and the SRS) were leading to delays in the initiation of key activities meant to strengthen the stewardship of the MoH (at the central level) and the supply of services (at the level of the decentralized SRS). These problems were exacerbated by natural disasters that affected the Dominican Republic at the end of 2007. 37. Therefore, the second amendment extended the Project’s closing date by 12 months, through December 31, 2009, and introduced the following changes in the Project:

An expansion of the geographical scope of the Project activities under Component I so as to cover the entire country. Given the financial constraints at the time of restructuring, it was thought that enrollment in the RS would be gradually achieved, starting with four of the nine health regions. However, the political momentum achieved with the re-launching of health sector reform resulted in the allocation of larger amounts of resources for the expansion of health insurance coverage nationwide. Therefore, the amendment modified the description of Component I, introducing the possibility of financing critical improvements in the health care delivery system nationwide to meet the increasing demand generated by new enrollments in the RS. An expansion of the scope of activities of the Project to complement the government’s ongoing efforts to strengthen the capacity of all key organizations to fulfill their roles and functions as established in the General Health Law and the Social Security Law. First, by increasing the number of activities led by the MoH that were aimed at strengthening the MoH’s capacity to exercise its stewardship role during the process of institutional reengineering. Second, by engaging other key stakeholders. This meant essentially to incorporate two additional agencies into the collective decision making on planning and implementation of Project activities: (i) the Superintendence for Health and Labor Risks (Superintendencia de Salud y Riesgos Laborales or SISALRIL) and the Directorate of Social Security Beneficiaries’ Information and Advocacy (Dirección de Información y Defensa de los Afiliados a la Seguridad Social or DIDA). 2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry Project Preparation 38. Quality Enhancement Review (QER) satisfactory evaluation of the Project’s quality at entry. In September 2003, the Bank’s Quality Assurance Group (QAG) assessed the quality at entry of the Project. Overall, the panel rated the quality at entry Satisfactory. The Project was rated Highly Satisfactory for its Strategic Relevance and Approach. All other dimensions were rated Satisfactory on the whole; nevertheless, the QER highlighted implementation areas that required further elaboration. The Project Appraisal Document (PAD) only outlined clearly the Project Coordination Unit’s (Unidad Coordinadora del Proyecto or PCU) responsibilities for the Project but not for the four Project Implementation Units (Unidades Ejecutoras del Proyecto or UEPs). It also described the functions of other public agencies that were to have their own Implementation Units. However, the PAD did not specify how the implementation was to be carried out by such a large number of Implementation Units. 39. Project Design. Given the ambitious, long-term vision for the sectoral changes and the breadth of the existing legal framework for the health reform, the choice of the APL as financing instrument was appropriate. The selection of the original PDOs for the APL correctly identified

12

the key issues relevant to the reform. Both the Project and the choice of indicators to measure the outcomes attributable to the Project interventions were ambitious. 40. The original Project design had maintained the approach followed by the Provincial Health Service Project, which had preceded the new APL, prioritizing activities in the same provinces. At the time, the Project had reflected the overall agreement on the long-term goals. While the PAD acknowledged that the Project could support the reform efforts, the Bank also realized that its ability to influence the outcomes of the reform would be limited. Nevertheless, this insight was not reflected in the goals, through the outcome indicators set for the Project. The Project activities and the responsibilities for their implementation were not always fully defined, making it very challenging to achieve the selected PDOs within the time-frame of the implementation of APL1. 41. Institutional arrangements: Institutional arrangements were not well defined, especially when considering the complexity of having four UEPs. The failure to clearly delimit the roles of all entities involved in the Project and, at a later stage, the lack of details in the Operational Manual placed a heavy coordination burden on the PCU. 42. Risk Analysis. The PAD did not consider a risk the challenges posed by a project design that involved multiple implementing agencies, thereby raising unrealistic expectations among them regarding their autonomy in developing separate operational and procurement plans and having fiduciary responsibilities over project implementation. As a result, once the Project became effective, frictions caused by discussions and negotiations over how to plan and implement activities emerged. These difficulties made it even more challenging to actually start with the activities.

2.2 Implementation Factors that posed challenges to project implementation 43. The financial crisis and changes in the administration led to delays in the Project’s approval by Congress. The Project was prepared in the midst of a major financial crisis in the final year of an outgoing administration which, after losing the elections, yielded power to a new government that took office in August 2004. Congressional priorities that emerged as a result of the crisis and political negotiations with the opposition in both the outgoing and incoming administration had a negative impact on the approval of the Project by Congress (18 months had passed before it was finally approved in December 2004). 44. Once the Project became effective, the incoming administration requested that the Bank carry out a major restructuring of the Project. The process of restructuring launched in May 2005 included the development of a conceptual framework for a critical path that would prioritize the poorest families and other vulnerable groups (i.e., mothers and children). Substantial background technical work and extensive discussions with the stakeholders, conducted in the framework of the restructuring, catalyzed a high-level agreement that provided the political endorsement of key organizations including the MoH, the MEPD, SENASA and PROMESE/CAL. As a result, the signing in May of 2006 of a mid-term agenda crystallized the commitment of these organizations to relaunch the reform. The shared agenda made it easier to overcome health sector reform bottlenecks stemming from the lack of a comprehensive approach to the development of a mid-term critical path to leapfrog social protection in health. The new agenda focused on the rapid expansion of health insurance coverage for the poor and the implementation of activities aimed at strengthening the institutional capacity of the MoH to exercise its stewardship role, lead the

13

decentralization push, and improve the supply of services with an emphasis on primary care. As a consequence, beyond adjusting the Project to the new realities and governmental priorities, the process of restructuring had itself generated a coordination space that fostered concerted actions by key stakeholders. 45. Rapid expansion of enrollment of the poor. The targeting mechanisms for the health insurance for the poor were aligned with other social programs including the conditional cash transfer program known as Solidaridad. The adoption of a mechanism for identification of the poor (Sistema Unico de Beneficiarios or SIUBEN) through Solidaridad laid the ground for better targeting and more rapidly enrolling persons qualifying for the health insurance for the poor once the government had allocated sufficient resources for that purpose. SENASA focused on boosting enrollment and the demand for health services, which were sometimes hindered by the lack of coordination with the service providers. The implementation of the health insurance at a faster pace than the implementation of the model of decentralization and strengthening of the MoH’s stewardship made it challenging to match the demand and supply of quality services, particularly at the first level of care. 46. Expectations created by organizational arrangements of the original Project design posed challenges for the start of the Project implementation. Project implementation as initially designed proved to be cumbersome. This in turn led to difficulties in reaching consensus on annual operational and procurement plans since each institution sought to finance its own fiduciary unit and cut the Project in five independent operational plans with insufficient coordination among the units involved. The operational manual approved did not provide an adequate operational framework for implementation by four UEPs. Furthermore, the necessary inter-institutional agreements had not been signed. 47. There was a lack of adequate planning, technical assessment, and supervision in the initial stages of implementation of civil works. Civil works was is one of the main challenges to Project implementation. More detailed information is provided in the Annex section. Annex 12 provides a summary of the final evaluation of the implementation of civil works. 48. A survey of the needs for infrastructure and basic services at the first level of health service provision was carried out in 2006. The survey was prepared by different consultants hired by the PCU. The generally inadequate level of expertise in the preparatory and planning stages for civil works, compounded by the insufficient depth of the assessment of needs, posed serious challenges to the implementation of civil works, which represented about a third of the total amount of the loan. 49. The subpar quality of the technical evaluation and the preparatory work done for the civil works manifested itself in different aspects of the implementation. For instance, the unsatisfactory technical evaluation of the various factors that needed to be taken into account and addressed even before the development of blueprints (i.e., site selection, in-depth technical evaluation of the terrain and soil for the type of civil work) led to major shortcomings in the cost estimation. The wide variety of civil works included as part of the Project—ranging from basic refurbishments to the construction of health centers and major civil works for hospitals and a medicines’ warehouse—exacerbated the lack of adequate technical planning and evaluation. 50. All these factors had a negative impact on the planning for procurement and bidding as well as on the actual construction work, particularly at the end of the project, when most of the civil works were executed. Substantial and numerous changes had to be made to the original cost estimates for the small civil works projects and to their final design. This resulted in a highly

14

inefficient process which posed multiple technical challenges; the latter had to be overcome in a very short period of time that coincided with the final year of Project implementation. Moreover, the small refurbishment works were lumped together for the bidding process, which was not really a viable approach given the asymmetries in the diagnostics carried out by different consultants, something that both the CERSS and the Bank supervision team had failed to point out. As a result, civil works proposals already approved had to be altered significantly during the implementation stage and substantial cost increases had to be dealt with. The geographical dispersion of the small civil work projects made implementation even more difficult, although these projects had been grouped together to make them financially more interesting for contractors as well. Regarding the large civil works (such as the PROMESE/CAL warehouse), the technical analysis carried out to determine the investment needs had been subpar, requiring that significant changes be made in Project design and cost estimates at a later stage. 51. PCU’s weaknesses in supervising and managing the implementation of civil works: CERSS showed important weaknesses in its capacity to manage the technical and managerial aspects of civil works, including the procurement processes needed to successfully complete all the civil works contemplated in the Project after restructuring. 52. The lack of adequate planning (both from a procurement and technical perspective), the subpar assessment of needs and supervision of the entire process for civil works led to serious inefficiencies and delays in Project implementation. Sometimes, after civil works had been awarded to the lowest bidding firms, the awardees proved unable to deliver with the quality and timeliness required, particularly when prices contractually agreed to were substantially lower than the market reference prices. 53. For instance, following a lengthy process, the contract for the construction of the Hospital of Pimentel ended up being cancelled once obvious, serious technical problems had been detected. However, some expenses were made to protect and repair the faulty construction that had been developed by the awarded firm. In the end, this project could no longer be financed with loan resources since it would have required a third extension, which in turn would have required a change of the closing date for a period longer than 2 years beyond the original closing date. Although the government decided to complete the hospital with its own resources (as the Bank did not provide the no objection to continue with this large-scale civil work, among other reasons because it was unlikely to be completed by closing date), this did affect the efficiency of the use of resources for this civil work. Factors that facilitated Project implementation: 54. Agreement among the key institutions in the health sector. In July 2006, a new inter-institutional agreement was signed. The agreement included the commitment of all of the institutions involved to work towards making progress with the health reform, set the main objectives and results to be achieved in the medium term, and identified the institution responsible for achieving them. 55. Substantial emphasis placed on reaching consensus during the restructuring process. The Borrower and the Bank dedicated much time and effort to the review and assessment of the progress made with the health sector reform. Rather than working on a single document to reflect the changes and agreements, the teams focused on dialogue, consensus building, and defining the priorities and activities that would lead to reaching the objectives sought. The process, which lasted short of 2 years, was fruitful in building alliances and understanding among the relevant

15

institutions and with civil society. Numerous workshops and information and dissemination events were organized during this period. 56. Flexibility to choose activities within the objectives of the sector reform, without blocking Project implementation. The Borrower and the Bank remained flexible as far as working within the critical path defined during the restructuring was concerned. This was reflected in an unusually wide operational program, which included more activities than could realistically be carried out. Nevertheless, this flexibility gave room to finance and carry out activities under the umbrella of the sector reform that could contribute to the overall reform objectives. The Bank remained flexible—i.e., willing to adapt the Project—as implementation progressed. This was illustrated by the support provided to the discussions and agreements on the implementation of the mandatory health insurance not only for the RS but for the Contributory Regime as well. 57. Strengthening of the supervision of all infrastructure projects. The Bank team strengthened the supervision of civil works in November 2008 and hired a specialized consultant to conduct an in-depth review of the status of the implementation of ongoing civil works, including field visits, with a view to identifying potential technical planning problems and assessing what works were likely to be concluded before the closing date of the Project. To overcome the serious implementation delays and achieve the disbursements of funds, the Borrower and the Bank agreed on a schedule of implementation and supervision. Numerous field visits were carried out through Project completion and this translated into the satisfactory completion of the works and an accelerated pace of disbursements. 58. Government satisfactory allocation of counterpart funds. Despite the global economic crisis and its negative effects on the country, the government sustained the allocation of counterpart funds to finance the Project activities. 59. Changes in teams were beneficial to the rhythm of implementation. Clearly there was an important change in the pace of implementation when the new management team took over in August 2008. At the time the new Bank team took over the Project, half of the Project resources were pending disbursements, largely caused by problems and delays in civil works. The team conducted in-depth discussions with the Borrower and a review of the Project activities with the relevant actors. This approached allowed for an accelerated pace of implementation and disbursement of the remaining Loan amount.

2.3 Monitoring and Evaluation (M&E), Design, Implementation and Utilization 60. The original Project included some indicators linked to the MDGs and some free-standing ones to assess the outcomes achieved in the 2003-06 period. It also included a list of output indicators to track the progress made with Project activities (see Annex 2). The indicators originally selected to assess the impact of the Project focused on final outcomes, which significantly limited their usefulness for measuring the Project’s impact, since several outcomes are known to be affected by many factors beyond the control of the Project. Therefore, it would have been difficult to show a causal relation between the Project interventions and the measurement of target indicators. These indicators might have been adequate for the long-term APL, if the three loan Project components and PDOs had remained unchanged during the 12 years of implementation. However, for the first phase of the APL, they were clearly too ambitious. 61. The first-order restructuring of the Project in 2006 introduced a new set of indicators (17 in total) linked to the new PDOs and the expected outcomes of each of the new components. The Restructuring Board Project Paper included outcome indicators, but did not propose any

16

intermediate indicators. However, as a result of the extensive work done with the Borrower’s team to prepare the restructuring of the Project, intermediate indicators had been defined and used by CERSS to be able to regularly monitor the progress of the Project, as agreed in a document that summarized the critical path and that was used to draft the operational and procurement plans (see Annex No 2); the ISR reflects these indicators. 62. At the time of restructuring no integrated efforts to monitor and evaluate health system performance was in place. There were fragmented data collection systems of uneven development among all key stakeholders of the health sector reform. The process of agreement on health sector reform priorities catalyzed the identification of working on M&E modules as a priority to advance and measure progress towards the objectives of the reform. The large number of indicators that had been agreed with all the institutions at the original design of the project was reduced. Indicators 2,3,4,5 and 12 would include surveys to collect data but given the short period of time for the project left only a baseline and second measurement were agreed to be financed. The framework considered several indicators (indicatrors 1,6,7,9,13 and 16) that would capture progress once policy changes would have been put in place and administrative information systems and procedures strengthened and measure was not collected by surveys but collected by the PCU. Finally the framework included several indicators that would capture qualitative changes in procedures and policy decisions.

63. After project restructuring the Bank had had an important achievement in the development of M&E system in the HIV/AIDS strategy led by COPRESIDA through the support provided within the context of the HIV/AIDS prevention and control project. At the request of the government an ambitious an expanded Demographic and Health Survey was financed through the loans and would provide key information for monitoring mother and child health care. The GAIN grant managed by the Bank was also successful in developing a baseline for nutrition. The successful experience of the construction of these baselines and the strenthtening stewardship and the reengineering process of the MoH resulted in prioritization of strengthening national efforts rather than favoring individual modules.

64. Both in the original design of the Project and after its restructuring, the PCU was all assigned the responsibility for carrying out the main Project monitoring and evaluation activities. External factors (such as tropical storm Noel and procurement problems) delayed the completion of surveys. The targets were set at the moment of restructuring, while the studies had not yet been carried out. Once the results were obtained, it turned out the proposed targets were not realistic, given the time left for the implementation of the Project. However only seven months prior to presidential elections the outcomes were not clear and the possibility of a new administration with changes in leadership of public organizations the Bank team choose to focus instead on ensuring sufficient progress on key aspects of the reform.

65. The rational was to consolidate gains in progress of the reform so that if new authorities or a new administration would come the dialogue would focus on how to the project could contribute to move forward the achievements of the reform agenda in the following four years. It was agreed with the government that a new M&E system would be discussed after elections. This indeed occurred and the decision was to have a more integrate approach. The new Monitoring, Information and Evaluation Unit was established at the MoHs now plays a key role in the development and harmonization of data collection and analysis modules fostering multisectorial and inter institutional agency coordination.

66. To strengthen the capacity of the MoH to exercise its stewardship role and to avoid “silos” and fragmented information modules, the government has created the new National Monitoring

17

Unit (Unidad Nacional de Monitoreo or UNM). This unit is in charge of providing leadership, coordination, and technical assistance and is to ensure that all key stakeholders across the health sector (including the MoH at the central level, SENASA, and the SRS) are on the same page; it is also responsible for the timely implementation of an integrated information system for the sector in accordance with national and health sector priorities.

67. The design of an action plan and new instruments and mechanisms to collect, aggregate, and analyze data to monitor performance and results based agreements (Conjunto Mínimo de Datos Básicos or CMDB) have been agreed with all stakeholders at the central and regional level. The CMDB was designed in such a way that the data collected will be useful for the reform of the CCT Solidaridad Program, particularly to improve the monitoring of the conditionalities of its beneficiaries. The CMDB will be digitalized and automated in Regions VI and VIII over the next 12 months and progressively also in the rest of the country.

2.4 Safeguard and Fiduciary Compliance Safeguards: Moderately satisfactory 68. Compliance with Bank’s safeguard policies was in general satisfactory. The environmental assessment carried out during Project preparation resulted in a category B rating for this safeguard policy. The Borrower complied with the environmental guidelines regarding the civil works carried out under the Project. An assessment of the biomedical waste management practices in the country was carried out, providing recommendations for improvement. Training of personnel on the management of hazardous waste was carried out with resources provided through bilateral agreements with Japan. 69. The original Project allocated resources under Component I to finance improvements in the biomedical waste management of the different participating institutions, the acquisition of equipment, the training of personnel, and the development of guidelines and internal norms. The Project built on the achievements of the previous Provincial Health Services Project 5 and provided resources to continue supporting the strengthening of biomedical waste management, including the development of guidelines and training. The General Health Law of 2001 established the Basic principles regarding health waste management but a specific regulation for waste management in hospitals was developed (Reglamento de Residuos Hospitalarios) and finally approved in June 2009. This norm was revised by the NHC in 2007–08. All of the Project activities were guided by these norms. 70. In October 2007 the NHC approved the regulation on the Management of biomedical waste in health centers (Reglamento sobre los Desechos y Residuos generados por los Centros de Salud y Afines). The regulation was adopted by the executive in 2009 (Decree 126-09). The Project contributed by preparing an operational manual (Manual de Residuos de Servicios de Salud) in 2008, to facilitate the implementation of the new regulation, and by updating the Project’s Operational Manual accordingly, thus complying with the Loan Agreement. While the Borrower had complied with the Loan agreement by Project completion, the process was sluggish due to multiple factors, including the lengthy approval process of the new regulation, and was

5 The Provincial Health Services Project contributed to improve healthcare waste management in the country by financing the assessment of waste management in healthcare facilities under the project, buying incinerators and other equipment for the large hospitals in the country, training of staff in biomedical waste management, and designing and updating the environmental norms and guidelines.

18

finally completed in the context (or perhaps the urgency) of preparing the follow-up operation (APL2). While the norms and regulations are in place there is a clear need to systematically measure progress on its implementation in the future. During discussions for preparation of the APL2, the GoDR initiated a dialogue with the Bank to identify alternatives for mechanisms through which the Bank can assist health sector stakeholders in developing and implementing a monitoring system to support the enforcement of the new regulations. Procurement: Moderately Unsatisfactory 71. The first Procurement Plan submitted after the Project had become effective had been developed to ensure consistency with the discussions regarding the upcoming restructuring of the Project and was considered of low quality—the procurement processes were deemed slow and many errors in decision-making were identified. 72. In 2008, the ex-post review found important weaknesses in procurement planning and the implementation of activities, particularly regarding civil works. It highlighted inconsistent adherence to good procurement practices and the Bank’s guidelines, including inadequate and incomplete information on contracts under implementation and bid evaluation. After discussions with the Bank, the PCU strengthened its procurement team to improve overall procurement performance and agreed to an evaluation in the ex-post review of the following year. 73. In mid-2009, despite procurement staff strengthening, the ex-post procurement review found that the weaknesses previously identified had persisted. The conclusion was that the PCU did have a stronger procurement team but there were other flaws in the process of evaluating bids and awarding contracts. More specifically, in the contracts reviewed, the team found non-compliance with the Bank’s procurement policies and inconsistencies with the bidding documents issued by the PCU. The evaluation also found problems in several procurement processes and, in one case, the Bank declared misprocurement on infrastructure activities and, consequently, cancelled US$555,405 of the total loan amount. 74. Remedial steps were immediately taken to strengthen the technical supervision of civil works. Also it was agreed with the PCU that all the activities will be conducted only with a prior review of all civil works agreed with the Borrower. Secondly, a plan to improve adherence to the Bank’s procurement guidelines was devised, with a focus on the procedures for shopping requirements. The rating was upgraded to moderately unsatisfactory before Project closing. 75. Although the lion’s share of activities were civil works, implementation during the last year represented more than 50 percent of loan disbursements, including a very large number of small activities involving many stakeholders and multiple agencies. These activities posed serious planning challenges. The last ex-post review found that the last year of implementation again showed weaknesses in terms of planning and other stages of the procurement process. Financial Management: Satisfactory 76. Financial management was rated satisfactory throughout the life of the Project. There were no problems with counterpart funding during implementation. The Borrower submitted the audit reports on time and they were, in general, unqualified.

2.5 Post-completion Operation/Next Phase

19

77. Preparation and approval of the next phase of the APL: The 2001 Laws contained the main principles and objectives of the health reform. These were supported by the APL as a whole. The APL1 contributed to substantial progress towards two critical areas of the reform: increase social protection for the poor population through a mandatory health insurance providing a family health package and improve performance and accountability in the health system. Regarding the first area, by Project completion over 1.3 million poor people had health insurance coverage provided by the subsidized regime. By the end of 2010, the government estimates that will be able to cover roughly 2 million people and 2.5 by the end of 2011 out of the total estimated 3.2 million of the target population. At project closing there were roughly 2.2 million people covered by the contributory system (for heads of households in the formal sector and their dependents). Expansion of coverage of health insurance via the subsidized system (for the poor) and the contributory (for individuals in the formal sector and their dependants) are critical components of the system of social protection to which APL1 contributed. 78. The APL1 contributed to the reorganization of the regional health systems. The Project contributed to create regional public health care networks which were beginning to function by Project completion. The Regional Health Services, organized as networks, were the providers of health services, with different levels of assistance. The APL1 contributed to progress in critical areas of intervention required to advance the reform: improving performance of the health system. Strategic interventions that were made possible by APL1 include the signing of performance agreements with SRS;6 the development of operational manuals, information systems and other instruments that facilitated coordination and harmonization of financing mechanisms used by SENASA and the MoH with the SRS; the implementation of new regulatory instruments for the delivery of care based on a new model based on a primary health care approach; the accelerated decentralization of the management of networks of providers, i.e. delegating some important decisions on the management of human resources to the regions—now in Regions VI and VIII. one of the most significant changes in terms of human resources management in the health sector in the last decades. 79. The rationale for proceeding with APL2 with Board Approval was linked to the APL1 outputs and results, laying the ground for the development of a joint MoH-SENASA strategy based on results-based financing for the first level of care. The Bank and the Borrower decided to go ahead with the preparation of the second phase of the APL based on the progress achieved in the first phase (although the targets had not been met) and progress made to health reform agenda in two of the most critical aspects that had been key bottlenecks: health insurance coverage and decentralization of services. These instrumental changes in the sector along with improved definition of roles and coordination among key stakeholders faced new challenges with increased demand. 80. As a result of policy dialogue and assessment of outputs and results of PARSS1 and within the framework of the global economic crisis which reduced fiscal revenues, the government decided to have a more efficient allocation of resources to keep the poverty reduction goals. The Ministries of Health, Finance, Economy, Planning and Development and in coordination with SENASA decided to introduce new harmonized financing mechanisms for the first level of care to generate incentives for improved quality of services with an emphasis on the poor. Results-adjusted capitations was the financing mechanisms identified as the best option to

6 On July 1, 2009, the MoH signed the first two performance agreements with two SRS (Regions VI and VIII), which constitutes a major step towards decentralization. In Maya ll the SRS had signed performance agreements with the MoH.

20

improve quality of spending via separation of financing from delivery of care and to introduce tools for strategic purchasing. 81. The government has worked out in great detail a model for improving the performance of decentralized SRS. The MoH has reached agreement with SENASA on a set of performance indicators to be introduced in performance agreements and contracts between the MoS, SENASA, and the SRS. The MoH has signed management agreements with Regions VI and VIII and, therefore, the prior necessary actions have been taken. The MoH and SENASA have also signed management-by-objectives agreements with the other regions. Furthermore, in two regions (VI and VIII) the MoH has signed performance agreements that contain specific indicators and targets to be measured on a quarterly basis, as the first stage in the use of results-based financing. 82. The APL phase II included in its design results based mechanisms, was approved by the Board in August 2009 and became effective in January 2010. With the support of the Bank, SENASA is signing contracts with SRS and introducing the same system of results-based financing for the first level of care in at least two SRS (VI and VIII) which are harmonized with the performance agreements of the MoH with the same SRS. The goal is to seek expanded impact through a unified system of results adjusted capitation while increasing accountability and providing incentives for delivery of better quality of services for the poor. 3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) Relevance of objectives: High

83. The PDOs for the APL1 were very well aligned with the country’s new legal framework for the health sector reform. The first-order restructuring of 2006 changed the PDOs of the APL1 to reflect the consensus reached on a 3-year agenda for the health sector reform. These were highly relevant not only to the progress of the reform but also to the GoDR’s priorities—they laid the ground work for a multiannual development plan that aims to exploit the synergies resulting from poverty reduction interventions across sector and public agencies. Relevance of design and implementation: Low 84. The original Project design could have contributed to the health sector reform agenda but its design showed shortcomings that made it hard to achieve the PDOs during the life of the Project. The first component involved supporting the extension of health services to achieve universal coverage. The Project design focused its interventions only on selected health regions while the PDOs’ outcomes focused on the entire population. More important, organizational arrangements were not conducive to better coordination and the attainment of highly ambitious targets. 85. The restructuring of December 2006 changed the design and implementation of the Project— refocusing on the process rather than the achievement of targets. Furthermore, the Project was also ambitious in its scope (particularly regarding the activities to be carried out by the MoH), as it sought to give a significant boost to the broader national agenda of expanding social protection; further advancing poverty reduction strategies such as the CCT ; and providing

21

birth certificates to children so they can get access to basic social services. Therefore, while the design of the restructured Project was relevant to the country’s priorities, it was inconsistent with selected indicators and above all the highly ambitious targets. 86. The implementation of the restructured Project shows major flaws, particularly regarding monitoring and evaluation and the strengthening of the supply of services. These flaws became more obvious after the Mid-Term Review. Although the implementation, in the restructuring phase, showed great adaptiveness to the country’s new realities and the new government’s health sector reform agenda, it became less responsive to the changes that occurred after restructuring. This became more evident once the baselines had been constructed. Although external factors—such as the floods that affected various activities to be carried out by the MoH and the national debate on the contributory system—crowded out much of the attention of key policy makers, by the end of the third year of implementation there were signs that suggested it was time for a serious review of indicators and targets and for rethinking the implementation timeline of planned activities. 87. Furthermore, after the Mid-Term Review, the Project implementation failed to fully and timely adapt to the emerging realities, i.e., sluggish decision-making processes within the MoH persisted due to its complex structure and internal reengineering processes. The Project’s implementation in terms of PDO indicators and targets was again discussed only in the Project’s final 15 months, after general elections had given the incumbent administration a second term. However, by then it was clear that some indicators would not be met and this, coupled with the ensuing global crisis, prompted the government to accelerate the preparation of APL2 rather than amend ongoing projects.

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) Original PDOs Rating: Unsatisfactory 88. Soon after the Project had become effective and in the early stages of the analytical work and discussions on the restructuring of the Project, it became clear that the PDOs would need to be changed. During practically the first 2 years of implementation until the restructuring was officially countersigned in December 2006, the bulk of the Loan and counterpart resources were devoted to identifying the main bottlenecks of the health sector reform, building consensus, and defining a critical reform path from which selected activities would be financed by the Project. For these reasons, at the time of restructuring, only approximately US$4.2 million (or 15 percent) of the total loan amount had been disbursed and virtually no progress had been made towards the achievement of the original PDOs. Restructured PDOs Rating: Unsatisfactory 89. The PDOs and Project indicators were revised at restructuring. The amended PDOs reflected a comprehensive approach in order to align the Project with the government’s poverty reduction policies, which translated into a complex operational plan encompassing an extremely broad range of activities. This approach’s main strengths were that: (i) resources were allocated to various key stakeholders to complement government resources allocated to expand health insurance coverage for the poor; and (ii) the Project activities fostered coordination among all

22

parties involved, particularly regarding the MoH initiatives to decentralize towards the SRS. The latter was done in close coordination with SENASA, which prepared the ground for the introduction of new mechanisms for results-based financing. 90. Important shortcomings of the revised Project were the broad scope of planned activities, and the dependence of the impact of some activities on policy decisions that could be negatively affected by the political economy of the public sector reform. Also, it seems unrealistic to assume that some activities financed, such as the development of new protocols and treatment guidelines and training could have such an impact on the medical staff’s behavior to meet agreed targets. In general, a key shortcoming of the Project’s design after restructuration was setting overly ambitious targets and maintaining those unrealistic targets even after the baselines had been constructed or implementation of key activities was delayed. The Project successfully implemented a significant number of activities relevant for the overall advancing of the health sector reform but many of these activities’ outcomes did not meet agreed targets. Even if all the changes had been introduced in a timely fashion, several targets still seem unlikely to have been met during the Project’s implementation, even with the extension of the closing date by 2 years. PDO I: Improve access of the poor in Prioritized Regions and Provinces to high-quality health services included in the Basic Health Plan of the Family Health Insurance 91. The degree of achievement of this objective was to be measured by indicators 1–7 of the results framework. While progress was made towards achieving the targets set, all indicators fell short of the targets agreed at restructuring, Indicator 1 Percentage of public primary health care facilities in PRPs that are accredited with SESPAS 92. As explained in the Borrower’s Final Implementation Report, in order to achieve this indicator, the primary health care centers had to meet the following criteria: (i) have adequate infrastructure and equipment as stipulated in the guidelines of the General Directorate of Authorization and Accreditation (Dirección General de Habilitación y Acreditación or DGHA); and (ii) have an adequate team including a graduated and certified physician (physician with exequatur), a certified nurse, and a health promotion supervisor. The target was not achieved, mainly because most of the physicians were graduated physicians but due to an existing law they do not have final license to practice (exequatur) until they complete the first year in a public health care unit (which is the case for an estimated 70% of facilities). Other reasons include: (a) decisions regarding the staffing of the primary health care centers were beyond the control of the Project since these requires an adequate allocation of fiscal resources (for the financing of new staff) and changes to laws and civil service policies, which were not introduced by the Ministry of Health (MoH) during the life of the Project; and (b) DGHA did not have effective enforcement mechanisms for primary health care centers that continued their operations without accreditation. Within the context of the new Performance and Accountability of Social Sectors Development Policy Loan (PASS DPL) approved by the Board in 2009, nationwide shortcomings in infrastructure, equipment, and staffing were identified and the government allocated funds to bridge these gaps in order to ensure access to basic services for the poor which is expected to boost the supply side gap and exploit synergies with the conditional cash transfer program (CCT) Solidaridad. This indicator was one of the APL trigger indicators. Indicator 2

23

Percentage of public health units in PRPs that have tracer drugs that are part of the Basic Health Plan (Plan Básico de Salud) available for the population enrolled in the RS. 93. At the time of restructuring, the target was to achieve a percentage higher than 50 percent. The baseline was defined in 2007 and the target was then set at 80 percent. Agreements between the MOH, SENASA, and PROMESE/CAL introduced changes to how medicines would be provided and to ensure that not all primary health care centers would have stocks of medicines, but only those that did not have a PROMESE/CAL distributing pharmacy in their vicinity. Moreover, variations in stock levels—due to the current procurement and supply chain design—impede an accurate estimate of the actual availability of drugs when only one measurement is used. Indicator 3 Percentage of pregnant women who receive prenatal care services in PRPs in accordance with national guidelines Indicator 4 Percentage of infants who receive routine check-ups in PRPs in accordance with national guidelines Indicator 5 Percentage of personnel in PRPs who are familiar with the national guidelines for maternal and child care 94. At the time of restructuring, the baseline was not available. The “Estudio de Calidad de la Atención Materno-Infantil,” finalized in March 2008, provided the baseline. The final target was not modified after the study was concluded. This indicator was introduced as a trigger for the APL and its target could not be met since the nationwide training was delayed due to the government’s decision to overhaul its national guidelines in order to enhance the performance of public sector health care networks. The highly complex task of reaching consensus among key stakeholders on the final version of new Guidelines for Mother and Child Health Care (financed with loan resources) took longer than expected—they were finalized only a few months before Project closing in September 2009. However, the new guidelines include evidence-based interventions adjusted to the new primary health care network approach and the reality of the Dominican Republic, and have been endorsed by that authorities at various levels. Although all the materials for training have been finalized, the actual nationwide training is only being organized this calendar year. The Bank is now working on a multisectorial approach will be one of the triggers for the second phase of the PASS DPL; in fact constitutes one of the supply-side strengthening interventions aimed at complementing the improved CCT program. Indicator 6 Percentage of managers and administrators of second-level hospitals in PRPs that were hired through a competitive recruitment process over the last 12 months 95. Decisions regarding the hiring of directors and other high-level staff at hospitals were beyond the control of the Project. Achievement of this target would require changes to laws and civil service policies. Appointments of directors for secondary and tertiary level hospitals currently are centralized decisions taken by high-level authorities at the MOH and remain discretionary. Although originally this was one of the issues that health authorities attempted to address, there was just not a political consensus to introduce major changes to civil servant human resources laws and policies.

24

Indicator 7 Percentage of secondary care facilities in PRPs that disseminate quarterly performance reports 96. A pilot system was developed and tested but not implemented before Project closing. Although a consultancy was financed to develop a first attempt at developing new tools to be used by hospitals to implement a performance monitoring and evaluation system government prioritized the strengthening of institutional records and health information systems since major flaws and weaknesses impeded an appropriate monitoring of vital statistics. The main achievements towards this PDO are: 97. Development of a set of clear accreditation standards for primary health care centers. This was supported by the Project, in the context of strengthening the stewardship role of SESPAS, and was key to improving the quality of provision of health services.

98. Strengthening of the public entity in charge of accreditation (DGHA) through the development of new guidelines and training of its personnel to carry out its core activity of accreditation for health centers. The Project also provided an important tool to monitor the performance of accredited systems and of the overall network of public providers through a geo-referential digital system (Sistema de Información Geográfica en Salud or SIGPAS).

99. Strengthening of the first level of care by improving the infrastructure and equipment of primary health care centers. The Project contributed to closing the gaps in the infrastructure network and to improving the condition of the buildings and equipment. It specifically contributed to the construction, improvement, and refurbishment of 140 primary health care centers and to the equipping of 502 primary health care units (Unidades de Atención Primaria or UAPs) in the Regions III, IV, V, and VI. The Project contributed to equipping 19 secondary and tertiary health units in the same regions. This activity was achieved after carrying out an assessment of the basic infrastructure needs 100. Strengthening of the regulation, procurement, and distribution of medicines

(i) Strengthening of the regulatory framework regarding medicines: Among the main achievements are: (a) passing of a decree on medicines prepared by SESPAS in 2006;7 (b) the standardization of protocols and treatment protocol for the first level of care; (c) the development of a new approach to foster rational selection and prescription thorugh a standardize prescription formulary; (d) the development and dissemination of new pharmaceutical protocols for units delivering primary health care; (e) the training of staff on this protocol at all regional health services and at the UAPs; (f) the training of about 20,000 staff at the regional level in the use of the single prescription to standardize prescriptions for primary health care medicines; (e) the elaboration of guidelines and norms for the management of pharmaceuticals at the primary health care level and (f) the strengthening of the National Health Laboratory of the MoH.

7 A new Ley de Compras y Contratación del Estado was approved in 2006. This law set the basic framework for the acquisition of medicines through international competitive bidding.

25

(ii) Strengthening of the DGDF, by: (a) Overhauling and strengthening the regulatory function through: (a) Development of new regulations and operational procedures for private and public pharmacies were developed (b) supporting the development of operational manuals for this department of SESPAS; (c) defining personnel profiles to improve its technical capacity; (d) hiring new personnel (university graduates) trained on health system surveillance; and (e) improving the DGDF’s physical infrastructure and equipment to improve monitoring and procedures.

101. The Project contributed to strengthening the supply chain of PROMESE/CAL:

Provided in time technical assistance through non lending technical assistance to identify best options for strengthening the supply chains analyzing pros and cons.

Modernization and expansion of warehousing capacity including the construction a second national warehouse and the acquisition of a fleet of refrigerated trucks and specialized transportation to improve the supply chain.

Support for expansion and improvement of the equipment and infrastructure of the network of pharmacies across the country: 426 farmacias del pueblo expand access to medicines to individuals from the SR but also to other individuals that can purchase medicines at discount prices.

102. Development of national guidelines establishing the protocols for the treatment of pregnant women and infants. The guidelines were elaborated in the course of 2009 and finalized in the last quarter of that year. This allowed for enough time to train the regional facilitators but not enough time to see practices consistently change across-the-board in primary health care centers. The Project evaluations showed progress in the use of the guidelines. In the case of the guidelines for the treatment of infants, 74 percent of doctors had conducted a physical exam, 68 percent had checked whether a child was up-to-date on his/her vaccinations and 26.5 percent had checked the child’s card (tarjeta perinatal). The region that presented the best results was Region III, where more training had been given. Progress was definitely made in creating awareness of the existence and content of the guidelines. There was not enough time before Project completion to have a higher impact, given the time it took for the analysis to be completed and the Project activities to be implemented. Only 7 months had passed, which is an important mitigating factor. A possible explanation the analysis gives for the detected improvement is the availability of the guides at the centers and the enhanced supervision at the primary health care centers. Nevertheless, the analysis pointed out there still is a significant gap to bridge to improve the quality of care for infants and pregnant women. 103. The main reasons for these shortcomings are: (i) fulfilling the targets set was not entirely under the Project’s control. This applies, for instance, to the accreditation of health care centers. In order to qualify for a license, the centers would have had to be staffed in accordance with the norms. Most of the health centers are staffed with physicians that just graduated and need to practice for one year after graduation in a public facility (also known as pasantes ). After this requirement most of them leave and a new promotion of young graduates take their place. Not only they are not acknowledged by the accreditation system because they are not yet licensed (without exequatur) but accreditation policy is inconsistent with Law that set up the system as an incentive for young physicians to take a position in health centers and rural areas. Accreditation with licensed physicians other than pasantes would have required the implementation of new human resource policies in the health sector, which was not achieved during the life of the Project.

26

104. Another example is the indicator related to the availability of so-called tracer medicines at health care centers. The indicator focused on the availability of medicines at the health center but a new agreement and policy would allow individuals from the SR to obtain their medicines in PROMESE/CAL pharmacy network reducing duplication of effort and exploiting synergies. Project restructuring and implementation fostered the discussions and agreements that resulted in a rapid expansion of SR and eventually of the new agreements for expanded access to SR so reality moved faster than what could have been foreseen when the original indicator was designed .

PDO II: Reorganizing and strengthening health systems in prioritized regions so they could fulfill the roles and functions as established in the General Health Law and the Social Security Law 105. The main indicators to measure this outcome are indicators 8 and 9 of the results framework. Indicator 8 The design of an integrated financial and performance management system completed, populated, and implemented 106. A model was developed, fine-tuned for implementation on SRS and training was carried out in the prioritized regions. However, government decided to include part of its results for the implementation in the health sector of a national system. The new national system was developed standardize and improve financial management (SIGEF). The activities carried out with Bank financing served to refine the new system for the health sector and are will now become part of the results based financing approach to be introduced not only by the MoH but also in coordination with SENASA as part of the new results agreements signed with two health regions (VI and VIII). This will support the capitated payments management introduced with the support of APL2. Indicator 9 Number of annual performance agreements between SESPAS and the SRS that include incentives to improve performance 107. Results based agreements were signed in the last quarter of 2009 between the MoH and SRS of Regions VI and VIII which constituted the basis for results based financing mechanisms introduced with the support of APL2. Also in the first months of 2010 performance agreements were signed with all the rest of the SRS nationwide. This indicator reflect the success of various activities financed by the Project and its predecessor and that served as springboard for the GoDR to launch a new results based financing for the first level of care that will be introduced in this year with the support of SENASA and with the support of the APL2. PDO III: Strengthening the financial planning of the Subsidized Regime (Régimen Subsidiado), ensuring the effective enrollment of the poor, and improving their knowledge of their rights under the Family Health Insurance 108. The outcome indicators for this PDO were numbers 10 to 13 of the results framework.

27

Indicator 10 Percentage of the budget for health care services that was transferred to the Prioritized Regions through SENASA 109. The successful achievement of this objective exceeds the target. The rapid expansion of health insurance coverage for the poor is at the root of this increase. At the time of the Project closing, about one-third of the total target population (1.3 million out of 3.2 million) were enrolled. Indicator 11 SESPAS, SENASA, the SRS, and the STP have developed, approved, and implemented a multi-annual financing plan for the RS in the PRPs (2008–2010) 110. Although a multiannual financing plan as originally described was not achieved The Borrower succeeded in elaborating a multi-annual budget (Plan Presupuestario Plurianual 2010–2013), which involved SESPAS, SENASA and the SEEPyD. The Budget was approved by the National Health Council (Consejo Nacional de Salud or CNS) and included a national strategy for development. Furthermore, the preliminary discussions and coordination conducted for a multi-annual financing plan for the Subsidized Regime served as an input for a more ambitious multisectorial, multiannual poverty reduction development program. Indicator 12 Percentage of poor living in PRPs that are enrolled in the RS who know the benefits of the Family Health Insurance Plan (Seguro Familiar de Salud or SFS) 111. Success in the rapid expansion of enrolled population outpaced plans strategy for communication that considered a smaller population. The study carried out to elaborate the baseline provided key information for the development of an information and communication strategy designed to reach the enrollees in the RS. This was a trigger indicator. Indicator 13 Percentage of children born alive in public hospitals over the preceding 12 months for which the hospitals have an electronic copy of the birth certificate on file 112. Although the target as measured by December of 2009 was not met the Project financed key activities including the negotiation for a designated space, refurbishments, equipment and training in 34 hospitals that account for 80 percent of the deliveries nationwide. The software required took longer than expected to be developed and adjusted to the actual needs and technological requirements but according to the most of the hospitals are implementing the system in this CY as noted in the country report .

The main achievements towards this PDO are:

113. A significant increase in enrollment in mandatory regimes of health insurance, both in the subsidized and contributory regimes. By Project closing, the number of enrollees had reached around 3.4 million, of which over 1.35 million were enrolled in the RS (up from a total of 70,000 enrollees in 2005).

28

114. The Project supported the efforts of SENASA to (i) identify the target population and its dependants; (ii) disseminate the rights and obligations of beneficiaries including strengthening of specialized units for advocacy in health establishements; and (iii) support for the process and logistics needed to rapidly provide identification cards to the enrolled population. All of these efforts will continue to be supported by PARSS II.

115. To expand rapidly coverage among one of the most vulnerable groups of the population and the poor the project supported simplification and automatization of birth certificates. Progress was achieved in the registration of births in 34 hospitals that account for 80% of the births nationwide The four hospitals that account for 50 percent of all births are still not entering the information electronically. Only two of the selected hospitals have entered data on all births they registered. The application has been installed in all of the selected hospitals. This accomplishment is crucial to achieving greater enrollment of children of parents eligible for the RS.8 A lack of supervision by the DIES and delays in the installation of the software are responsible for these disappointing results.

116. Significant progress has been made in terms of the knowledge of the population enrolled in the Subsidized Regime of the benefits of the family insurance:

117. The DIDA was supported by the Project in it’s the development and implementation of advocacy and information campaigns at the community level.

8 It is important to note that 95 percent of births in the DR are institutional, and therefore the electronic birth certificate

will significantly further the enrollment of the target population in the RS.

-

200

400

600

800

1,000

1,200

Ind

ivid

ua

ls c

ove

red

by

he

alt

h i

ns

ura

nc

e(t

ho

us

an

ds

)

Expansion of Health Insurance Coveragefor the poor (Subsidized Regime)

2003 - 2008

29

118. Substantially larger fiscal resources were allocated to the SR, while the those for SESPAS for health care services grew at a considerably lower rate. This was an achievement in terms of breaking with “traditional” budgetary allocations. More budgetary resources are channeled to the regions through SENASA.

119. Under the leadership of SEEPyD, a multi-year budget for 2010–13 has been prepared and presented to the ministries. This budget represents a national strategy for development and comprises two parts: health and social security. The latter covers three areas: enrollment, primary care, and collective health. The aim is to get the entire target population enrolled by 2013. This is a first step towards changing the historical budget allocation and traditional financing arrangements towards prospective payments that can eventually be used to introduce incentives or performance based financing mechanisms.

PDO IV: Strengthening the capacity of national health sector organizations so they can fulfill their roles and functions as established in the General Health Law and the Social Security Law. 120. Indicators 14, 15, 16 and 17 were designed to measure progress towards this PDO. The main achievements towards this PDO are:

121. Strengthening SESPAS’ stewardship role and the development of the financial and administrative capacities. The Project supported the strengthening of DGHA, DIES, the Directorate of Strategic Development (Dirección de Desarrollo Estratégico or DDEI, formerly known as UMDI), and the creation of a public information center (Oficina de Acceso a la Información or OAI). Key to strengthening the leadership and planning roles of SESPAS are the achievements in the development and use of planning and budgeting tools: multi-annual budgets, the allocation of budget resources based on results, a system to plan acquisitions and an investment plan. Some of these tools are already in use. 122. Design of a monitoring and evaluation system to be used by CERSS to monitor the progress of the health sector reform. The system had been designed by Project completion but was not pursued further under the Project. 123. Strengthening of SENASA’s institutional capacity: the Project contributed to the elaboration of an institutional development master plan for SENASA. It identified the main weaknesses and proposed ways to overcome them. The Project supported the development and implementation of internal rules and regulations, with operational manuals for each of the main departments in SENASA. Manuals for the beneficiaries and providers were developed as well. SENASA is following the process necessary to get the ISO certification. Three information modules for its monitoring and evaluation system were developed with Project support, meant to monitor SENASA beneficiaries cared for in SESPAS facilities. 124. Institutional strengthening of PROMESE/CAL. With support from the Project, PROMESE was strengthened as the central venue to acquire medications of good quality at competitive prices. While PROMESE had not carried out an International Competitive Bidding (ICB) process to buy medications by Project completion, significant progress had been made in this area—the norms to carry out ICB had been approved. As they had been completed late in the Project, there had been no time to carry out an ICB process before completion. Significant achievements had preceded this step: the approval of the Ley de Compras y Contratación del Estado in 2006, the Medications Decree (approved by SESPAS in 2006, establishing the norms

30

regarding imports, exports, registration, etc.). The acquisition process of medicines was also improved with support from the Project: the process became more transparent and efficient, which translated into lower prices for medications at PROMESE’s pharmacies.

125. The table below provides the calculations as per the ICR Guidelines’ weighting systems for projects with formally revised objectives.

Rating Against Original

PDOs Against Revised

PDOsOverall Comments

Description Unsatisfactory Unsatisfactory Value 2 2 Weight 13.82%

(US$4.07/US$29.44)86.17%

(US$25.37/US$29.44)100%

Weighted 0.17 1.7 1.8 Final Rating -- -- 2 Unsatisfactory

3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return)

3.4 Justification of Overall Outcome Rating Rating: Unsatisfactory 126. The Project’s overall outcome is considered unsatisfactory based on the unsatisfactory achievement of almost all of the targets set for the outcome indicators. Though being of high relevance in the selection of PDOs, the relevance of the original design was low and the Project was overly designed at restructuring. However, it deserves to be acknowledged for its efforts and successful support of a wide range of activities of high relevance for the overall progress of the health sector reform. Nevertheless, strategic planning to channel financing towards those activities that were geared to meeting the PDOs and achieving the targets agreed upon was poor.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 127. The contribution of the project to the rapid expansion of SR which constitutes one of the pillars of the new system of social protection in health provides financial protection for the poor from ill health income and consumptions shocks and other negative effects to household consumption and human capital investment. Clearly the government’s commitment to the reform, a context of economic growth are the main causalities but the project played a crucial role in establishing how to move forward with priorities and permitted to move to the next stages of the reform that had been stalled for almost five years.

128. CERSS carried out two studies, one to define a baseline for this indicator and another one at the end of 2009 to assess the progress made in this area. The study showed the following results for the RS: (i) targeting to the poor has improved in recent years after the introduction of the CCT beneficiary identification instrument (also known as SIUBEN); (ii) it has had a positive impact on the gender gap; (iii) the average age of the Regime’s affiliates has increased in the

31

evaluated period and a rise in the adult population enrolled has been observed (from 20.7 percent to 24.7 percent); (iv) the distribution of the enrolled population follows the distribution of the population nationally—64.9 percent are urban residents and 53.1 percent live in rural area; (v) the number of affiliates with a primary education rose with respect to the 2007 level (relative to the population with a secondary or higher education); (vi) the level of unemployed affiliates has increased by 14.3 percent with respect to the 2007 levels; and (vii) 89 percent of the affiliated population knew about the importance of health care interventions (versus 79.8 percent in 2007) and 80.3 percent knew about the importance of preventive care (medical checkups). (b) Institutional Change/Strengthening 129. As mentioned in section 3.2, the Project contributed to the institutional strengthening of SESPAS, SENASA, and PROMESE by supporting the preparation and implementation of norms, regulations and protocols; and providing training and support to changes within the implementing institutions. (c) Other Unintended Outcomes and Impacts (positive or negative) 130. The Project provided critical support to the achievement of one of the most strategic an unintended outcome: the rapid progress made towards the expansion of enrollment in the Contributory Regime focusing on the formal sector (as opposed to the Subsidized Regime for the poor in the informal sector). The support of the Project to governments discussions and policy dialogue made it possible enroll into the contributory system head of households working in the formal sector and their dependants. By the project closing a total of 2.2 million people were enrolled into the contributory system which was the other key pillar of expansion of health insurance coverage to all the population.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops n.a.

4. Assessment of Risk to Development Outcome Rating: Moderate 131. The risk to the development outcome is considered moderate based on the following factors:

The country’s commitment to the long-term goals of the health sector reform as demonstrated by the current enrollment of over 1.4 million beneficiaries in the RS and an estimated 2.2 million by the end of CY 2010; investments agreed in APL2 for continuing decentralization while introducing results-based financing mechanisms.

Support by the Bank and other multilateral and international organizations of the government’s multisectorial approach linking demand-side interventions that foster human capital investment (like the CCT program Solidaridad) through several instruments of investment lending, policy development lending, etc.; Development of the first multisectorial 4-year development plan (Plan Plurianual del Sector Público 2010–

32

13), which includes clear goals, targets, and indicators for social protection in health, and is to be financed through public sector investments and annual budgets.

132. The risks derive from the need for political commitment to tackle certain aspects of the reform, such as the enhancement of human resources. In order to reach the development objectives, such as the full licensing of UAPs across the country, the UAPs have to be staffed with graduated doctors (with exequatur) to be in full compliance with the law.

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Unsatisfactory 133. The Bank collaborated closely with the Borrower in the preparation of the Project within the framework of the implementation of the Provincial Health Project. The Bank engaged the Borrower’s team participation proactively in every stage of the preparation process, including QAG meetings during the internal quality at entry review. Performance is rated moderately unsatisfactory because: (i) the selection of the PDOs for the Project and the key outcome indicators were more appropriate for the APL than for its first stage (APL1). They were valid long-term objectives, be it more in line with the 10-year APL program, but could not be achieved by the Project within its intended timeframe; (ii) inadequate organizational arrangements that included four implementing units and a lack of details on how the various public sector agencies would be involved and what their responsibilities would be. (b) Quality of Supervision Rating: Moderately Unsatisfactory 134. There were three task managers over time for this Project and transitions from one TM to the next went smoothly, with a good degree of continuity. The task manager who led the restructuring was part of the original preparatory team. From the start of Project implementation, once the Project had become effective, the Bank adopted a highly proactive and flexible approach to restructuring the Project to meet the actual needs and support multi-sectorial efforts to reduce poverty through an expanded social protection system. However, after the Mid-Term Review, the supervision changed its focus to processes and after midterm review targets of PDOs were not lowered to more realistic levels. This was, in part because of the need to provide advice and support to the Government and the PCU, which faced multiple challenges trying to implement a vast array of activities, and ensuring adequate coordination with multiple agencies and stakeholders in each of the agencies and finally the upcoming presidential elections that made difficult to get decision makers to focus on discussion of such important changes. 135. Introducing extensive civil works activities can be risky and supervision is highly demanding and time-consuming; the team had to deal with multiple demands and did not had from beginning of project implementation an expanded team with different and complementary skills in order to tackle in a more comprehensive way the multifaceted areas of the ambitious operational and procurement plans. Proactive supervision helped identify fiduciary problems and develop remedy and mitigating risk plans. During the last 15 months of implementation

33

strengthening the supervision team with new consultants who had the necessary training and skills help to strengthen supervision and proactively identify and seek solutions to the many issues and problems that had arisen from the PCU’s implementation weaknesses regarding civil works was a positive development. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 136. Before effectiveness: The Bank carried out frequent and well-staffed missions to support the Borrower with the implementation of the Project. During the first year after approval, it concentrated its efforts on supporting the government in getting the Project approved by Congress. Once the Project had been approved, the Bank team worked closely with the new government to restructure the Project to align its objectives and activities with the government’s vision of the health reform. 137. After effectiveness: The Bank team fostered a lengthy but truly participatory process, engaging key stakeholders and key decision-makers, to ensure that the Project would become an instrument to build consensus on how to move forward the stalled process of health sector reform. It took more than a year to get the Project countersigned, but it was very helpful to assess what the key bottlenecks of the reform process were; and to take stock of the main strengths and building upon them (to develop a strategic path with a mid-term action plan that identified short- and medium-term priorities while keeping in line with the long-term sector goals). 138. One weakness in the first stages of Project implementation was the supervision of the civil works. The Bank fostered south-south support with individuals from a health project from Mexico, which provided a valuable exchange of ideas regarding how to develop an infrastructure plan within the framework of a new accreditation system. However, the skills needed to continuously monitor and evaluate the planning and progress of civil works were just not available. In the last 15 months, the Bank did add an expert consultant to the supervision team but, by then, much time and implementation efficiency regarding civil works had been lost. The Bank focused on assessing major implementation risks and providing technical assistance on numerous civil works project implementation challenges that emerged from the PCU’s weaknesses in planning, bidding, and supervising civil works (a summary of the final report from the civil works consultant is included in Annex 11). 139. The other important weakness identified in the supervision relates to the balance needed throughout project implementation among different supervision approaches. Specifically, the balance between enabling the borrower with tools for strategic planning and proactively providing advice on monitoring progress towards agreed targets and indicators on the one hand and identifying the need to adjust targets on the other hand. The Project clearly was instrumental in overcoming key health sector reform bottlenecks and the progress made in the implementation of activities was measured periodically as agreed. However, reaching the agreed indicators and targets would have required a considerably higher pace of progress overall in the implementation of activities; this is particularly true for those activities related to the supply side and strengthening of the stewardship functions. Furthermore, after the Mid-Term Review, a discussion on the targets and indicators was only reactivated once baseline data and data on progress made had become part of the preparatory meetings for the APL2. 140. In the last 15 months of Project implementation, the onset of the global crisis and its negative impact on fiscal revenues forced the government to focus on an accelerated preparation

34

of the second loan of the APL, building on the lessons learned and focusing on results-based financing (rather than on a potentially lengthy, new restructuring process that would have consumed more time and resources for an amendment that probably would have been signed only a few months before closing the Project). The possibility of actually preparing the APL2 with a focus on results-based financing was possible thanks to the ground prepared by APL1, its outputs, and intermediate outcomes. However, those achievements fell short of the ambitious targets and indicators chosen during the first-order restructuring.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 141. The government’s performance is rated moderately unsatisfactory. The government was committed to health sector reform over the long term and worked closely with the Bank to prepare the Project and, at a later stage, restructure it. Changes in administration, the economic and financial context, and the complexity of the issues involved in health reform sometimes resulted in delays in key decision making (human resource reform, for instance). The government sought and seized the opportunities generated during Project restructuring and implementation to overcome some of the key bottlenecks that had stalled progress on the implementation of health sector reform laws. 142. The main reasons for the moderately unsatisfactory rating are: (i) the delay to get the Project approved in a timely manner; (ii) the lack of political will to implement certain aspects of the reform that are considered critical to reaching the PDOs; and, (ii) changes in administration increased difficulties in taking key decisions that involved political economy considerations. This was the case with the human resource reform, which was not addressed in a timely manner to contribute to achieve the changes sought with support from the Project. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 143. The performance of the PCU was uneven across time and across areas of Project implementation. Major points of weakness were: (i) the poor management and supervision of the civil works activities from the start and over a long period during the life of the Project, which caused unnecessary inefficiencies in Project implementation; (ii) weaknesses in the timely use of monitoring data for strategic decision making, particularly regarding updates of operational plans; (iii) the inability to sustain improvements in procurement and seek timely advice from the Bank before rushing to take decisions, yielding to the pressure of a large number of activities waiting to be implemented. 144. Regarding the other participating entities, the initial impetus for coordinated action and collaboration with the PCU lost steam after the mid-term review. Only after the administration’s second term had started and staff had changed at the PCU and some of the other agencies, a renewed interest in a more hands-on approach to Project implementation emerged among some of the participating entities. (c) Justification of Rating for Overall Borrower Performance

35

Rating: Moderately Unsatisfactory 145. Key aspects of the Borrower’s performance:

- Borrower’s commitment to the health sector reform with adequate allocation of resources

not only as counterpart funding but for expanding SR; - Successful close collaboration among public agencies to implement midterm agenda to

leapfrog health sector reform ; - Proactively developing and implementing an strategic approach to foster collaboration

among public sector agencies and development partners - Carrying out a broad participatory and consensus building process aimed at reaching

identifying a critical path for re-launching the reform; - Delays in approving the Project and achievement of effectiveness ; - Weaknesses in the planning, selection and management of procurement and civil works

activities.

6. Lessons Learned Project design, restructuring and amendments 146. Need to better assess the institutional capacity of the implementing agencies at the design stage. The PAD was not realistic when assessing the capacity of the several institutions involved in the implementation. As a result, even though the Project took 18 months to become effective, when the time arrived, the inter-institutional agreements had not been signed. This was a difficulty that delayed the smooth start of implementation. More emphasis should be placed on institutional strengthening as a crucial component of health reform. 147. Need to be realistic when setting the PDOs’ indicators and the targets to measure the outcomes. While the PDOs were valid and in line with the long-term goals of the reform, they were too broad and complex to be achieved by the Project during the proposed execution time. Moreover, the indicators proposed were also influenced by many factors beyond the control and actions of the Project. This problem was replicated when the Project was restructured. The indicators and targets selected were again very ambitious and influenced by factors outside the control of the implementing agencies. It would be advisable to limit the scope of achievements and targets to the Project description and base the selection of indicators and targets on this. On the other hand, in the case of this Project, the indicators selected did not show some of the progress that had been achieved. This was particularly the case of the civil works activities: the indicators do not show the progress achieved towards closing the infrastructure gap. Implementation 148. When reforms are complex, it is crucial to foster consensus building and the engagement of different key actors. The Bank and the Borrower tried to do this by organizing workshops along the restructuring process. Later on, the financial component of the restructured Project served to distribute the work among the key institutional actors in the health sector. The Project design included a component on financing, which is unusual. It served to gather all actors from different institutions (SESPAS, SENASA, CERSS, SEEPyD) and have them reach consensus on the budget for the health sector in 2007 for the first time. This has continued as a working practice and has served to formulate a multi-year development plan with clear and measurable goals and development objectives.

36

149. If a project supports complex reforms, it is very important to carefully select the activities to be financed by the project. The design and implementation of APL1 suggests that the momentum generated either by adopting a new legal framework or by accommodating an incoming administration tends to foster an overly optimistic environment, in which the actual possibilities for achieving goals in the short and medium term are overestimated. The momentum may be very conducive to setting ambitious, long-term goals and securing political commitments but, if the borrower does not make a clear distinction between the long-term reform goals and what a small loan can achieve, overly ambitious objectives and targets for an investment program may be included in project design, without an appropriate evaluation of the risks involved. Alternatively, the measurement of targets that require very complex and costly studies may be more suitable for an evaluation analysis where a new policy is introduced than for PDO indicators. Targets that can be measured periodically should be closely monitored to determine whether they need to be adjusted, especially if a clear trend of lack of progress towards PDO indicator targets emerges. 150. If it is decided that an investment loan will include civil works, it is critical to ensure from the earliest stages on that both the Bank and the implementing units have sufficient, adequately staffed teams with the right mix of skills. The problems faced in the implementation of the civil works financed by the Project highlight the importance of having the right teams both as part of the PCU and as part of the Bank supervision teams. 151. The early stages are the most critical and investing in adequate needs assessment, a sound technical evaluation, and appropriate development of blueprints can minimize the risks of serious technical errors and inefficient implementation of these types of activities. It is very important to ensure that direct beneficiaries—such as the staff of health care units or the management of public sector agencies—are involved in every step of the process, from the earliest plans to the development of blueprints as well as the beginning, completion, and reception of civil works. It is likewise important that all decisions made be well-documented and not merely translated in verbal agreements. 152. Moreover, the supervision of procurement processes of civil works in the health sector is highly complex due to various special circumstances arising from the geographic dispersion of health care units and the wide range of types of works (from simple refurbishments in numerous but scattered health centers to highly specialized infrastructure projects such as hospitals and pharmaceutical warehouses). Therefore, the bidding, planning, and implementation processes require experienced teams that can take into account the realities of the health sector civil works and of the market. Price considerations may require an analysis of past experiences with public bidding to avoid dealing with local firms that may offer lower-than-market prices but later prove unable to deliver, leading to lengthy processes for the cancellation of contracts and/or costly investments to repair and preserve investments already made.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers. N.A. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

37

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Restructured Estimate

(USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

IBRD Borrower IBRD Borrower

HEALTH SERVICES COVERAGE EXTENSION 25.97

n.a 1.49 0.58

IMPLEMENTATION OF THE HEALTH SECTOR REFORM LEGISLATION

7.20

n.a 1.50 1.14

IMPROVING SELECTED ESSENTIAL PUBLIC HEALTH FUNCTIONS TO PROMOTE AND PROTECT THE HEALTH OF THE POPULATION

2.66

n.a 0.75 0.39

PROJECT MANAGEMENT AND IMPACT APPRAISAL OF NEXT PHASES

6.58 n.a.

2.46 1.4

Improve access of the poor to quality services included in the Plan Básico de Salud

0.00 15.60 3.92 13.27

Reorganize and strengthen health systems in prioritized regions 0.00 1.20 0.29 1.22

Improve the planning and management of the Subsidized Regimen

0.00 1.30 0.321.01

Strengthen the capacity of national health sector organizations

0.00 6.10 1.53 8.66

Total Project Costs 42.41 24.20 6.06 29.27 9.57

Front-end fee IBRD 0.30 0.00 0.30 Total Financing Required 42.71 24.20 6.06

The total project cost was estimated at US$39.1 million; US$38.7 was the total project cost after restructuring, US$0.4 was the front-end fee, US$24.20 million was the cost of the new components and the difference was the amount already implemented: US$4.07 million.

38

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 12.71 9.09 71.5International Bank for Reconstruction and Development 30.00 29.27 97.5

Note: At restructuring, the Borrower contribution was lowered to US$9.1 million and the total Project cost to US$39.1 million. A total of US$555,404.92 was cancelled as a result of misprocurement.

39

Annex 2. Outputs by Component Outcomes, Intermediate Results and Outputs after restructuring Component I PDO: Improve access of the poor to quality health services in prioritized areas

Specific Objective: In PRPs, public primary and first-line secondary health facilities meet the criteria for accreditation with MoH The RHS have a Master Plan for Infrastructure and Equipment Main results and outputs

Diagnostic of the infrastructure needs for the prioritized regions (III, IV,V

and VI); Acquisition of equipment for the first level of care in the prioritized regions; Revision and improvement of the instruments to assess the infrastructure and

equipment needs for the first level of care; Rehabilitation of 162 health facilities in Regions 0 (Office of Sectoral

Analysis in SESPAS), II (Construction of the medicine warehouse for PROMESE/CAL in Santiago de los Caballeros; III (primary health care centers, rural clinics and the RHS office in San Pedro de Macoris, medical residences in hospital Jaime Mota

The MoH has an accreditation system that guarantees minimum quality standards for the national health system. Main results and outputs related

Strengthening of the DGHA in the MoH Acquisition of furniture and equipment for the DGHA; Elaboration of norms for accreditation of health care centers; Elaboration of manuals (Manual de Inspección y Habilitación) and

formularies; Design and implementation of an application and information system; Elaboration of an inspection and supervision plan for the health care centers

in the prioritized regions. The DGHA is carrying out the inspection of the health care centers in the

prioritized regions and there is a process of self evaluation by the Regional Health Services.

Specific Objective: In PRPs, improve the access of the poor enrolled in the Subsidized Regime to essential drugs included in the Basic Health Plan. The RHS have a system for planning, provision and rational use of medicines. Main results and outputs :

Evaluation of the availability of drugs in the primary health care units and Elaboration of a baseline;

Training of personnel in training of personnel on its use; training of personnel in the “farmacias ambulatorias” on basic concepts on pharmaceutical work;

Design and use of the single prescription document (receta unica) and training of personnel in its use;

Design, printing and distribution of the pharmaceutical and therapeutical guides;

40

Workshops to create the pharmaceutical and therapeutical commissions at regional level

PROMESE/CAL has a system to manage medicines and medical input satisfactory to meet the demand of the Subsidized Regime. Main results and outputs:

Strengthening of the Dirección General de Medicamentos in MoH; Strengthening the institutional capacity of PROMESE/CAL: Construction of a new medicine warehouse in Santiago de los Caballeros;

Acquisitions of vehicles for the regional distribution network; improvements in the Farmacias del Pueblo;

Definition of the mechanisms to distribute medicines in the Regional Health Services;

Creation of a thematic group on medicines; PROMESE/CAL has improved its capacity to acquire drugs through ICB. Main results and outputs:

Training of personnel on the acquisition methods of medicines; Revision of the procedures for procurement of medicines, according to

international standards;

Specific objective: In PRPs improve the quality of the services provided in primary and first-line secondary facilities included in the Basic Health Plan. Intermediate result: the RHS use guides and protocols to deliver care for pregnant women and children under one year of age. Main results and outputs:

Printing of guides for monitoring the care and evaluation of the care for pregnant women and children under one year of age;

Training of personnel in monitoring and evaluation of the services provided for these groups;

Design of a monitoring system to follow up the use of the content of the guides;

Printing of the guidelines for prescription of medicines at the primary level of care;

Printing of 1,000 guide books and 300 sets of flip charts for health workers (promotores de salud); 5,000 guidebooks for diagnosis and treatment; 3000 copies of the organizational manual for the UNAPS; 3,000 copies of operation manuals of the UNAP's;

Training of personnel in the primary health care units on the content of these guides.

Specific Objective: In the PRPs, contract hospital executives is based on competitive procedures

Support for Development of regulatory and legal frameworks (Ley de Carrera Sanitaria)

Specific Objective: In the PRPs, management of hospital based on financial and performance management systems. The RHS have a system to evaluate the performance of second level hospitals

41

Design of the automated system for the family health card (Ficha Familiar de

Salud); Acquisition of equipment for connectivity.

Component II PDO: Reorganize and strengthen health systems in prioritized regions to comply with the roles and functions as established in the general health and social security laws

Specific objective: In the PRs, the Regional Health Services operate an integrated financial management system that permits entering into service contracts. The RHS have an administrative and financial system that allows accountability (in accordance with the SIGEF) Main results and outputs:

Proposals for tools for an administrative and financial system in the SRS, among these: a proposal for a single budget and manual for its use and management the single budget, containing the budget structure and a guide to train personnel of its management as well as the draft text for the regulation;

Development of models for data bases and a web-site to manage the administrative and financial system;

Redesign of the procedures and instruments for the administrative-financial area;

Development of a monitoring system for the administrative-financial system; Personnel trained in the SRS; Elaboration of catalogues of services and service providers and criteria for

their classification. The RHS have a Management Information System that allows decision making and accountability towards SENASA, the MoH and beneficiaries. Main results and outputs: Design of the set of norms and regulations necessary for the organization and

functioning of Regional Health Services. A total of 29 products were designed by the Escuela Andaluza de Salud Publica. Some of them were the following:

Proposal of a draft legal text to create the National Health Service; draft legal

text for the Functional and Territorial Decentralization of the Public Health Services;

Draft proposals for texts on the "Cuadros de Mandos (Organizational Framework) for the Regional Health Services", "Compendium of Basic Clinical Documentation" and "Conjunto Minimo de Datos de Alta Hospitalaria” ;

Draft decree establishing the Strategic Plan for the functional decentralization of the provision of health services (Plan Estratégico para la descentralización funcional de la provisión de Servicios de atención a las personas);

Proposal for the Management Agreements between the SESPAS and SRS, including the different administrative levels within the SRS and with the hospitals. These agreements were used as a model for the final agreements reached by SESPAS with Regions VI and VIII;

Regulation for the reference price of essential drugs and supplies of PROMESE/CAL; Operations Manual for public pharmacies (Farmacias del Pueblo and the Red de Botiquines of the UNAP's kits rural), organizational manual for the prescription system;

Signing of the Framework agreement for the management agreements between the RHSs, SENASA and the MoH, which involved an agreement between SENASA and

42

the MoH and agreements between SENASA and Regions VI and VIII. Component III PDO: Develop a multi-annual financing plan for the subsidized regime, ensure the effective enrollment of the poor and improve their knowledge of entitlement

Specific objective: Increase the budget for health care delivery transferred to the PRs through SENASA. The Subsidized regime has a multi-annual investment and financing plan. Main results and outputs:

Creation of an inter-institutional group (MoH, SENASA, PROMESE/CAL, SEEyP) to reach consensus on enrollment on the Subsidized Regime.

Institutional coordination to elaborate the 2007 budget for the health sector; Multi-year budget for 2010-2013 agreed upon, led by the SEEyP;

Prioritized public hospitals have an electronic system to generate birth certificates, installed and data in the system; Delegaciones de Oficialias Civiles (offices of the civil registry) in public hospitals have an automated civil registry system. Main results and outputs:

Elaboration of a baseline of the situation of the information system on vital statistics in the prioritized hospitals;

Support the development of the Information System of Vital Statistics through: (a) implementation of Birth Information System (SINAC); (b) provision of computers and networks in prioritized centers and in DPS in prioritized areas to manage the information modules on births and deaths; (c) preparation of an operational manual on birth information; and (d) updating the format of the live birth certificate. Regarding information on deaths, the following products were prepared: (i) an operational manual on management of information on deaths, (ii) design of a software application and user manual of the new death certificate; (iii) printing of the guide to fill up regular death and fetal death certificates, and (iv) training of regional staff on management of information on vital events;

Rehabilitation of 19 offices of the Central Electoral Board and provision of furniture and equipment in 24 hospitals out of the 34 where 80% of total births occur;

Training workshops for personnel of different institutions working with the MoH birth certificates;

SENASA has a system of enrollment which guarantees that all eligible people can fulfill the requirements to get enrolled Main results and outputs s

Support for DIDA by financing workshops and printing materials for the implementation of an educational information campaign strategy on the rights and benefits of the Basic Health Plan for those enrolled in RS as well as potential enrollees;

Support for the issuance and distribution of cards (carnetización) for 426.821 new affiliates and taking 169.089 pictures for new enrollees in the Family Health Insurance subsidized regime.

Consultancy to enable birth registrations and issuance of identity documents to allow the enrollment of 2.515 people (dependent beneficiaries) in the subsidized regime;

Consultancy to design a methodology for the evaluation, monitoring and indexing of the cost of the Basic Health Plan used by SISALRIL;

Support for the logistics to carry out the field work for the distribution of cards and for the identification and enrollment of families in the subsidized regime.

43

Component IV PDO: Strengthen the capacity of public organizations to comply with its new roles and functions as established in the general health and social security laws.

Specific objective: MoH monitors and evaluates the performance of the health sector and disseminates results annually.

MoH has improved its institutional capacity to carry out its stewardship function.

Main results and outputs:

Strengthening the Department of Accreditation (DGHA) (see Component I);

Provision of furniture and computer equipment to the Directorate of Health Informatics and Statistics (DIES), the Unit of Sector Analysis and Unit for National Health Accounts; and provision of hardware, software and related services for the Office of Access to Information in SESPAS.

Support provided to approve the norm to regulate the management of waste generated by the health centers (Reglamento sobre Deshechos y Residuos Generados for Centros de Salud y Establecimientos Afines), approved by the government by decree No. 126-09 of February 20, 2009;

Workshops for the formulation and dissemination of the Ten-Year Health Plan 2006-2015;

Consultancy for the Institutional Strengthening and Restructuring of the Administrative and Financial System of SESPAS: this supported the development and implementation of new operational procedures for the Administrative and Financial Area, the Purchasing and Contracting Department and the Office of Access to Information (OAI); elaboration of a proposal for a new filing system and management of correspondence for the OAI; the development of planning and budgeting tools. Thanks to these new manuals and procedures, an improved system for managing procurement and contracting of goods started to be implemented, and competitive biding has been used to acquire office and medical supplies;

Strengthening of the Office of Access to Information (OAI): upgrading of its physical space; provision of furniture and computer equipment, training of personnel within and outside the country;

Design of the website for SESPAS;

Design for the system to monitor the performance of the health system, through consulting and workshops;

Development of the draft law to reform human resource policies in health (Ley de Carrera Sanitaria), financing of consultants and workshops; Strengthening of the Modernization Unit (UMDI) in SESPAS, which is at present the Directorate for Strategic Development (DDEI) through the provision of furniture and equipment;

Strengthening of the OAI with acquisition of furniture and equipment, training of its personnel, design of a system of authorizations (Sistema de autorizaciones Sanitarias) under a single point (Modalidad de Ventanilla Unica) in SESPAS. Training of personnel at national and International level. Intermediate result: SENASA has improved its capacity to administer health risks. Main activities:

Institutional development plan (Plan Maestro de Desarrollo Institucional) for SENASA developed, identifying priorities for institutional development weaknesses, gaps and a strategic path to implement the changes;

Acquisition of Hardware and Software as well as the installation of software

44

in health centers in Regions III y IV to allow the timely reporting of data to SENASA on the enrolled population;

Development of three modules of the information system to monitor those users of SENASA cared for in SESPAS facilities;

Training of technical personnel in the country and abroad. Intermediate result: CERSS has improved its rol as a coordinator of the health reform process.

Acquisition of Computers for the PCU; Improvement of the Management of documents: redesign of the data base,

migration to Electronic files of documents and availability on the web, preparation of summaries of key studies for its dissemination.

Upgrade of the facilities of the information center (Centro de Documentación de la CERSS), provision of furniture and equipment; By project completion, there was the Centro de Contacto Gubernamental, a web page, a virtual library on health reform, an on-line training center, a center for documents with more than 7,000 documents, individual cabins with Computers and Internet access for consultation.

Provision of technical and financial resource to carry out the Encuesta Demográfica y de Salud;

Design of the system to monitor the progress of the health sector reform;Support other participating institutions by supporting the training of their staff Workshops on results based Management, monitoring and evaluation of social programs, Management of health insurance, to train staff of other participating institutions.

Strengthening of MEPyD Acquisition of Hardware and Software; Training staff on financial management, in the context of the formulation

and monitoring of social policies with emphasis on social protection specifically in health.

45

Annex 3. Economic and Financial Analysis N/A

46

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Specialty

Lending Patricio Márquez Task Team Leader LCSHD Christoph Kurowski YP/Health Specialist LCSHD Marta Ospina Procurement Analyst LCSHD Maria Colchao Program Assistant LCSHD Juan Carlos Alvarez Legal Counsel LEGLA Patricia de la Fuente-Hoyes Financial Management Specialist LOCA Guido Paolucci Senior Procurement Specialist Joseph Paul Formoso Senior Finance Officer LOCA Cecilia Balchun Financial Analyst

Robert Crown Consultant, Institutional Development and Operations

Oscar Echeverri Consultant, Public Health Specialist LCSHD

Alberto Gonima Consultant, Health Management and Information Systems Specialist LCSHD

Aracelly Woodall Financial/Cost Specialist LSCHD Evelyn Pesantes Institutional Development Specialist Willy de Geyndt Institutional Development Specialist Catherine Abreu-Rojas Operations Specialist LCC3C Massimiliano Paolucci DR Country Officer LCC3C Marco Mantovanelli Country Manager LCC3C William Experton Sector Leader LCC3C Martha Vargas Team Assistant LCSHH Supervision/ICR Zoila Catherine Abreu Rojas Procurement Specialist LCSPT Fabiola Altimari Montiel Sr. Counsel LEGLA Cecilia Maria Balchun Consultant CTRDM

Carmen Brinckhaus Procurement Specialist Consultant

CTRDM

Samuel C. Carlson Lead Specialist SASED Mary A. Dowling Language Program Assistant LCSHE Leanne Farrell Junior Professional Associate LCSSA Patricia De la Fuente Hoyes Senior Finance Officer CTRFC C. Izquierdo-González Finance Assistant CTRDM Verónica Yolanda Jarrin Senior Program Assistant LCSHH

Svetlana V. Klimenko Sr. Financial Management Specialist

LCSFM

Christoph Kurowski Sector Leader LCSHD

47

Claudia Macias Operations Officer LCSHH Natalia Moncada Program Assistant LCSHH Fernando Montenegro Torres Sr. Economist (Health) LCSHH

Xiomara A. Morel Sr. Financial Management Specialist

LCSFM

Alexandre Borges de Oliveira Senior Procurement Specialist LCSPT Gunars H. Platais Sr. Environmental Econ. LCSEN Gustavo Castro F. Raposo Financial Analyst CTRDM Maritza A. Rodriguez De Pichardo Financial Management Specialist LCSFM

Rocío Schmunis Health Specialist ET Consultant

LCSHH

Geraldine Beneitez ET Temporary LCSHD Julie B. Nannucci Language Program Assistant LCSHD

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY02 0.13 2.92 FY03 13.45 115.06

Total: 13.58 117.98 Supervision/ICR

FY04 3.61 24.95 FY05 17.40 64.48 FY06 31.28 112.33 FY07 30.11 101.75 FY08 17.74 68.48 FY09 19.69 111.04 FY10 20.79 89.60

Total: 140.62 808.59

48

Annex 5. Beneficiary Survey Results N/A

Annex 6. Stakeholder Workshop Report and Results N/A

49

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

BORROWER’S COMMENTS ON ICR

We understand that overall the Completion Report focuses on assessing the degree to which the targets set for the Performance Indicators and Project Development Objectives have been met. As we discussed on several occasions through the dialogue of the Bank with the current management team of the PCU, our review of the attainment of the targets set for the Project prior to and after the Project’s restructuring has confirmed that substantial errors were made both in the design and restructuring phase of the Project, mainly related to the nature and scope of the Project indicators and targets. Basically, the planned indicators and targets spanned results that were linked to external factors of a national nature that depended on institutional actions lying outside the Project’s scope. Thus, these indicators measured the degree of progress attained with the health sector reform process rather than the Project’s contribution to the reform process. The draft ICR deems the Bank’s performance in the Project’s design and implementation moderately unsatisfactory. We understand that the factors mentioned above may have adversely affected this performance rating, mainly because of the Bank’s participation in the aforementioned design and restructuring of the Project and, more specifically, the definition of the indicators. As to the financial-administrative and procurement aspects, we understand that they remained within parameters that were deemed satisfactory by the country and in view of the progress made with institutional capacity; in this context, the progress made in the framework of the National Plan for the Strengthening of the Public Institutions, that the Government of the Dominican Republic has embarked on. As far as the operations go, the performance of both the Bank and the Project Coordinating Unit (PCU) should be improved to ensure that the implementation is in line with the Bank’s norms and procedures, but also that the planned time periods are met in form and substance. The problems deriving from the Bank’s late replies to no-objection requests for technical and procurement matters need to be addressed in future operations, such as the PARSS2, which is already in the implementation stage. We would like to highlight that during the drafting of the ICR we tried to ensure that the document would capture the Project’s real contribution to the health sector reform process, in a way that would make visible the progress that has been made as a result of the Project’s implementation, even though it may not necessarily have resulted in the achievement of the targets set for the performance indicators. In fact, the Project did contribute to the attainment of Intermediate Indicators – sometimes in full – and, nonetheless, the failure to take some decisions at the national level and/or external factors outside the scope of the Project are ultimately responsible for the failure to reach the targets set for most of the PDO Indicators. Among the key accomplishments attained by the country through the direct support of the Project are the contribution to the National Health Insurance (SENASA) in the form of the distribution of ID cards to new members of the subsidized regime and the capture of their dependents as well as the institutional strengthening of said organization; the opening of civil registry offices at public hospitals; the design of the norms and dispositions needed for the organization and functioning of the Regional Health Services, a total of 29 products; the support provided to the Ministry of

50

Health in the inspection of public and private health facilities, in the refurbishing of primary and secondary health care facilities, in the formulation and distribution of the 10-Year Health Plan, and in the redesign of administrative and financial procedures to ensure the public’s access to information, as a mechanism for transparency and accountability vis-à-vis the population. All of the above illustrate the Project’s contribution to the reform process. The Project activities have led to the strengthening of the management and regulation functions and, consequently, to an improved stewardship capacity of the Ministry of Health, the strengthening of the regional health services’ administrative management, the improvement of SENASA’s management capacity and of PROMESE’s (Program of Essential Medicines) capacity to manage procurement processes. The Project’s restructuring offered an opportunity to realign the Project objectives with initiatives in the context of the reform process and at the same time identify Project-specific objectives in the broader context of the health sector reform. The Project’s rate of disbursement is evidence of the substantial improvement made in managing the Project, as evidenced by a disbursement rate of 50 percent of the total loan amount during the last year and a half of the implementation. Although efforts were made to close the gap between actual programmatic results and targets, the shortcomings in the Project’s design pointed out above and the lack of time made these tasks particularly challenging. SUMMARY OF GOVERNMENT’S ICR

Background

The Government of the Dominican Republic (GDR) in 1997 embarked on a process of health sector reform with the creation of the Executive Commission for Health Sector Reform (Comisión Ejecutiva para la Reforma del Sector Salud or CERSS), which was made responsible for coordinating and supporting the reform process. In 1998 two projects to support reforms in the health sector were launched: the Provincial Health Systems Project, financed by the World Bank (WB) and executed by CERSS, whose main objective was to strengthen the health sector by focusing its actions on the provincial health care systems. In parallel, a Health Sector Modernization and Restructuring Program was implemented with support from the Inter-American Development Bank (IDB). During the 20002004 administration, the Government and the WB agreed on an Adaptable Program Loan (APL) to support the health sector reform process, to be implemented between 2003 and 2014 for an estimated amount of US$ 126.71 million. The Bank would finance US$90 million and the Government would contribute US$36.71 million. The program (PARSS1) was to be implemented in three phases over a period of 12 years, each phase lasting 4 years. In June 26, 2003, the GDR and the Bank signed a Loan Agreement for US$30 million to implement the first phase of the program (PARSS1). The loan became effective 18 months after its approval by the Bank, due to delays in its ratification by Congress. The closing date was June 30, 2007. The PDOs were to:

(a) Contribute to the achievement by 2015 of the Millennium Development Goals (MDGs) agreed by the GDR at the Millennium Summit held in September 2000. More specifically, the project would focus on three MDGs: reduce child mortality; improve maternal health;

51

and contribute to the eradication of extreme poverty (by protecting the poorest against financial loss resulting from poor health and disability);

(b) Support the implementation of the new legal framework for the health sector, particularly

the institutional strengthening of the Secretariat of Public Health and Social Assistance (Secretaría de Estado de Salud Pública y Asistencia Social or SESPAS) in its role as steward, the development of insurance mechanisms, and the creation of health service networks; and

(c) Support the development, validation, and dissemination of norms and regulations to

implement the General Health Law (1942-1901) and the Social Security Law (1987-1901).

The project originally consisted of four components:

1. Extension of the Coverage of Health Services; 2. Legislation of the Health Sector Reform; 3. Essential Functions of the Public Health System; and 4. Monitoring and Evaluation.

While the implementation of PARSS1 started in January 2005, very few activities were carried out during the first months. Restructuring of thePARSS1 When the new administration took office in 2004, the Government and the Bank thoroughly reviewed the project and identified the following design problems:

1. The MDGs were very ambitious given the project’s time horizon; 2. The project’s design did not adequately take into account the limited capacity of the

participating entities to implement the project; 3. The project components and activities did not correspond entirely with the reform

priorities at that time; 4. The project’s design did not support the administration’s priority to harmonize the

health reform efforts among the different health regions, prioritized by poverty level. The health sector conditions, the progress made with the reform process, and fiscal constraints, led the Government and the Bank to carry out an evaluation, which identified the critical points in the areas of financing, service provision, and stewardship, to help accelerate the pace of the reform process. To this end, three national and one international consultant were hired to work with key technical and other health sector stakeholders in assessing the situation of the health reform at the time. As a result of this exercise, a critical path for accelerating the pace of reform was elaborated, which contained key actions for the short- and medium-term. This evaluation process, which began in the last quarter of 2005 and ended in the second quarter of 2006, was the main input for the reformulation of the Project. The evaluation resulted in the following studies: "Analysis of the Progress of the Financing Function in the Context of Health Sector Reform," "Analysis of the Progress of the Stewardship and Provision of Services Functions in the Context of Reform Health Sector," and "Analysis of the Obstacles to Development of Financing, Provision of Services and Stewardship Functions in the Process of Health Sector Reform."

52

The main challenges identified through the evaluation were: In the area of financing:

• Accelerate the pace of expansion of coverage for the poor to the Family Health Insurance Plan (Seguro Familiar de Salud or SFS) and ensure the program’s financial sustainability;

• Strengthen the operational mechanisms to ensure efficiency and effectiveness of the health

services provided under the subsidized regime;

• Strengthen the coordination of the government agencies involved in the implementation of the subsidized regime;

• Strengthen the participation of civil society in the reform of social protection in health.

In the areas of stewardship and service provision: The challenges regarding these aspects were derived from the overall goal for the regional health services and the resulting priority actions were:

• Redefine and reaffirm the central role of the National Health Council (Consejo Nacional de Salud or CNS) and clearly set out the respective areas of expertise of the CNS and the National Social Security Council (Consejo Nacional de Seguridad Social or CNSS) as well as the areas in which they must coordinate their activities.

• Emphasize activities aimed at enhancing SESPAS leadership role in supervision and control

as well as its support for the decentralization, de-concentration, and regionalization processes;

• Improve the National Health Care Insurance’s (Seguro Nacional de Salud or SENASA)

systems in the areas of logistics, risk management, control of membership, technical and financial risk analysis, costing of service packages and adjustment of capitations.

• Design and implement the tools and procedures for the Provincial Health Directorates

(Direcciones Provinciales de Salud or DPS) to be able to assume their new roles; • Design and implement the tools for the Regional Health Care Service Centers (Servicios

Regionales de Salud or SRS) to be able to assume their new roles;

• Increase the technical and operational capacity of CNSS. The evaluation also identified priority actions for SESPAS and the SRS by area of intervention:

• Strengthen SESPAS’ regulatory function by supporting the development of the Dirección General de Habilitación;

• Develop tools and mechanisms that will allow the decisions regarding infrastructure and

equipment to be based on actual sector needs;

53

• Contribute to building regional health networks in the prioritized regions by improving the infrastructure and equipment in the SESPAS’ health facilities at the primary level of care;

• Strengthen the management of human resources, focusing on the decentralization towards

the SRS; • Strengthen the administrative capacity to support the development of regional health

networks, including the definition of a model of care, the organization of health services within the network, the development and organization of diagnostic support systems, the organization of an emergency network and emergency clinics, the design and implementation of a program of continuous quality improvement, and the design and implementation of a subsystem of information for management and administration;

• Design and implement a financial and accounting system for the regional health services; • Strengthen drug management through the development of complementary legislation a

standard prescription model, a system to manage prescriptions; a system for medicine logistics and medical supplies; and an information subsystem for the management of medicines;

• Develop a model for clinical management for primary health service providers; and • Design and implement methodologies for participation and social mobilization.

As a result of the evaluation, four major objectives were identified to promote a change that would put users at the center of public health policies and in turn serve as the basis for the construction of a joint working agenda for SESPAS, SENASA, the Essential Drugs Program/Logistics Support Center (Programa de Medicamentos Esencial/Central de Apoyo Logístico or PROMESE/CAL), CERSS, and the Technical Secretariat of the Presidency (Secretariado Técnico de la Presidencia or STP, at present the SEEPyD) to accelerate the reform process and make changes to the project’s design.

These objectives were:

• Improve access for the poor to high-quality primary and secondary health services included in the Basic Health Plan Family Insurance in the prioritized regions and provinces;

• Reorganize and strengthen the health systems in the priority regions so they can fulfill the

roles and functions set forth in the General Health Law and the Social Security Law; • Develop a multi-annual plan for the financing of the subsidized regime, ensuring the

effective membership of the poor and improving their knowledge of their rights as members; • Strengthen the capacity of health sector national organizations to fulfill their roles and

functions as provided in the General Law of Health and Social Security Law. In July 2006, a framework agreement for inter-institutional cooperation was signed, making this agenda a reality. The agenda reflected the commitment of all parties involved to accelerate the process of health sector reform and contained specific results—assigned to each of the institutions—to be achieved over the medium term. Moreover, the agreement created a working space that facilitated the definition of objectives and expected results, as agreed by consensus

54

among the institutions involved. Through this process, the trigger indicators and their targets were also agreed on. The PARSS1 was restructured to reflect a results-based management approach. For each PDO, objectives and expected results were defined, as well as the activities to be carried out, thereby establishing three levels of objectives that formed a chain of results. The completion of a set of activities would lead to the achievement of an expected result. Several expected results contributed to the achievement of a second-level objective and reaching several second-level objectives would mean achieving a development objective. The chain of results was not linear, and different expected results were inter-related while not all corresponded to the same second-level objective. Specific indicators were identified and some of them measured the achievement of more than one third-level result. The responsibility for achieving the expected results was distributed among the institutions. It was clear at this point that the activities carried out by each of the institutions would have repercussions, positive or negative, on the activities and achievements of all other participating institutions. As part of the restructuring, the Operational Manual was revised and adapted to reflect the new requirements of the project. The revised manual incorporated a Plan for Management of Environmental Risk and Labor Safety and the general guidelines for the project’s monitoring and evaluation. The PARSS1 institutional structure consisted of the Project Coordination Unit, located in the CERSS, and three Project Implementation Units (PIU), in SESPAS, PROMESE, and SENASA respectively. The latter three teams were established as coordinating bodies for the PARSS1. The responsibility for financial management and procurement remained with CERSS. In order to better manage the project, thematic groups were created. These interagency working groups were composed of personnel from the participating institutions and served as coordinated working spaces for the formulation, implementation, and monitoring of PARSS1 activities, in accordance with the results expected in each project component. Groups were formed on Empowerment, Organizational Development, Infrastructure, Finance, Human Resources, Health Networks, and Drug Administration. These thematic groups failed to consolidate and worked effectively for a short period of time only. The relationship between the teams could be presented schematically as follows:

Thematic Groups Coordinating

Team

Technical Group PARSS

55

A monitoring plan for the project was prepared, which included objectives, activities, information flows, a set of indicators, the structure of reports, monitoring forms, and strategies to promote the use of information. Main Problems Affecting the Implementation

These were the following:

The restructuring process absorbed a full year of the life of the project;

Many of the results sought could just not be reached because they were outside the project’s reach;

The achievement of certain outcomes depended upon political decisions that were beyond the control of the project implementation team;

Many of the project’s indicators could not be realistically attained in the time frame of the life of the project;

Most of the indicators’ targets were too ambitious; moreover, it was anticipated that the indicators would only be measured at the end of the implementation period, rather than at regular intervals—providing timely information on the extent to which targets had been met up to that point;

The project did not include any activities to create awareness at the political level of the importance of taking key decisions and adopting certain measures to achieve the PDOs;

There was over-planning of the activities to be implemented under the project (through the POAs, which were impossible to implement);

The institutions responsible for project implementation actually undertook some activities that were not directly related to the achievement of the outcomes agreed under the project;

The implementing units did not exert the leadership role that had been expected;

The PCU in CERSS faced restrictions to sustain the functioning of some of the mechanisms designed at restructuring with a view to facilitating the coordinated implementation of the PARSS1;

The Procurement Unit was overwhelmed by the volume of activities planned, particularly in civil works; The Bank did not respond timely to many of the non-objections presented by the Borrower, thereby delaying the procurement processes.

Lessons Learned

The project objectives and goals must be aligned with the institutional capacity of the implementing institutions. The objectives of the PARSS1 were beyond the project’s area of intervention, actually proving out of reach for the project;

Good inter-institutional communications is a key factor for the success of a project. Poor communication between the PCU and the implementation units in the participating institutions was one of the project’s main weaknesses;

56

The creation of inter-institutional groups is a good strategy to create ownership among the technical levels around project activities, to reach consensus on the implementation of activities, and to foster their timely execution. The PARSS1 included the creation of thematic groups, which worked well during the first year of implementation but languished and were even dissolved later on;

The coordinating team must ensure that the activities included in the POAs are in line with the implementation capacity of the participating institutions. The project suffered from over- planning of activities, clearly exceeding the capacity of the implementation institutions to execute all planned activities before project completion, especially in SESPAS;

It is critical to identify the risks to a project’s implementation and formulate adequate measures to address these. The PARSS1 faced many risks that had not been identified early in time and, as a result, there was no plan in place to address these;

Good time management is key to a project’s success. The operational plans must be followed up and updated regularly and should be shared with all the participating institutions. Many of the implementation problems in the PARSS1 were caused by a lack of knowledge (on the part of the implementing institutions) of the updated content of the latest POA;

The procurement team must be trained on the use of the procurement documents in the early implementation stages so it can function effectively. In this particular project, despite the training provided, the procurement team took a long time to master the use of the relevant documents, a factor that delayed the procurement processes.

A timely and faster response to non-objections by the Bank’s procurement team significantly improves the efficiency of a project. In the case of the PARSS1, many procurement processes were delayed due tot the slow response by the Bank to the request for non-objections;

The different units in a project should define their financial needs and requirements at the beginning of the implementation so that the team responsible for the project’s financial management is fully aware of them and can duly address them in a timely manner.

57

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A

Annex 9. List of Supporting Documents

Bank Preparation Documents:

Terms of Reference for the Pre-appraisal mission, Aide Memoires and Back-to-Office reports. World Bank.

Terms of Reference for the Appraisal mission, Aide Memoire and Back-to-Office report. World Bank.

Project Appraisal Document. Report No. 25809-DO. World Bank. Quality at Entry Review – 2003. Loan Agreement, dated June 27, 2003. Loan Number 7185-DO.

Bank and Borrower Project Implementation Documents:

Aide Memoires for Bank Supervision missions. World Bank. Amendment to the Loan Agreement, December 2006. Second Amendment to the Loan Agreement, July 2008 Mid-term Evaluation Power Point Presentation. World Bank. Project and Implementation Status Reports (PSRs and ISRs). World Bank. Borrower Quarterly Progress Reports. CERSS. Project Final Report 2002–2009 (Informe Final del Proyecto). CERSS. “Evaluación de la Calidad de la Atención a la Embarazada y al Niño Sano en los

Centros de Primer Nivel de Atención de las Regiones de Salud II, IV, V y VI de la República Dominicana. Resultados de la Segunda Fase.” Informe Final. Diciembre 2009. CENISMI. CERSS.

“Informe Final sobre Disponibilidad de Medicamentos del Plan Básico de Salud en las Unidades de Dispensación de las Regiones II, IV, V y VI.” CERSS. Diciembre 2009.

“Informe Final: Satisfacción de Afiliados de SENASA del Régimen Subsidiado.” Gallup. Dominican Republic. July 2009.

“Avances, Obstáculos y Desafíos de la Reforma del Sistema de Protección Social en Salud de la República Dominicana. Función Financiamiento. Informe Final.” Maritza García. June 2006.

58

Annex 10. Key performance indicators as included in the original PAD The original PDO were to: (i) contribute to the eradication of extreme poverty by protecting the poorest against financial loss due to ill health and disability through the enrollment of the target population in the government health insurance plan and make available and promote the use of basic health services of high-quality standards. Target population would be people living below the relative poverty line in regions II, IV, VI, VII, and VIII; and (ii) support the implementation of the health sector reform legislation that provided the legal, financial, organizational, and managerial enabling environment to achieve the MDGs related to health. Annex 1 of the PAD included the following outcome/impact indicators:

PDOs Indicators Baselines and (Targets)

Increase the proportion of the target population using health services provided under the Basic Health Plan.

Percentage of pregnant women with at least four prenatal care visits

Percentage of births attended by skilled personnel

Percentage of women in reproductive age that use modern family planning methods

Percentage of children aged 12 to 23 months fully immunized (DPT, Polio and Measles)

Percentage of children under 5 with diarrhea that are medically diagnosed and treated

Baseline (target) III: 92.4 (98) IV: 90.2 (98) VI: 90.3 (98) VII: 95.5 (98) VIII91.3 (98)

Baseline (target)

III: 98.2 (98) IV: 91.5 (98) VI: 93.7 (98) VII: 96.8 (98) VIII: 97.6 (98)

Baseline (target)

III: 67.9 (73) IV: 61.0 (66) VI: 65.1 (70) VII: 70.4 (76) VIII: 69.4 (75)

Baseline (target)

III: 42.1 (54) IV: 26.0 (41) VI: 35.3 (48) VII: 33.8 (47) VIII: 33.0 (46)

Baseline (target)

III: 41.4 (53) IV: 47.6 (58) VI: 41.0 (53) VII: 34.0 (47) VIII: 37.2 (50)

Improve the quality of services provided under the BHP to the target population

Percentage of women receiving prenatal care that are immunized against tetanus;

Percentage of women diagnosed with pre-eclampsia referred to tertiary facilities;

Baseline: (target) III:92.7 (100) IV:89.6(100) VI: 92.6 (100) VII: 94.4 (100) VIII: 93.1 (100) Baseline: n.a. Target: 100%

59

Percentage of staff in facilities run by Health Service Providers that are present during working hours;

Percentage of target population satisfied with the quality of the BHP services.

Baseline: n.a. Target: 95%

Baseline: n.a. Target: 75%

Remove financial barriers to access and protect the target population against the financial consequences of ill health.

Percentage of the population eligible for government services under the Social Security Law enrolled in the National Insurance Plan;

Percentage of private insurance expenditure as a percentage of total health expenditure

Target: 80%

Baseline: 73% {2000}

Target: pending

Improve the efficiency of the health system

Percentage of SENASA’s financial resources allocated to primary care level services;

Nurse/midwife per doctor ratio in Health Service Providers (excluding auxiliary nurses);

Hospital beds per 1,000 population in Health Service Providers;

Percentage of administrative and managerial positions created in the reform process at the regional, provincial, and municipal level of SENASA that are filled by SENASA employees.

Baseline: N.A. Target: 30%

Target: 0.8:1.0

Target: 1/1,000

Target: >90%

Triggers for Launching the Second Phase of the APL by Project Development Objective in the Original PAD Readiness for expanding Insurance System to Additional Provinces and Regions (original Component I)

Health Management Information System fully operational in three regions and in SENASA and SISALRIL (including billing, reimbursement, beneficiary identification, and clinical auditing).

The proportion of financial resources flowing from SENASA to the project regions—on the basis of services provided—exceeds 20% of the total financial resources allocated to those regions.

Readiness for expanding Quality Assurance Measures to Additional Regions and Provinces (Component I)

Certification/licensing by SISALRIL and SESPAS of core service providers operating within PARSS in numbers that are sufficient to meet expected demand for primary health care.

Readiness for Establishing Additional RHS (Component II)

Selection of up to five new provinces in the remaining regions (I, II, and V) on the basis of: (i) poverty and vulnerability of the population; (ii) the health status of women and children; and (iii) completed negotiations among SESPAS, the Regional Authorities, and stakeholders regarding the framework and timetable for applying adequate norms for management, fiduciary accountability and operational transparency as indicated by Phase I experience.

60

Readiness for Expanding Deconcentration and Decentralization Process to Additional Regions and Provinces

Two out of five Health Service Providers and more than 1/3 of the hospitals integrated into the five HSP assumed full responsibility and authority to manage human resources (including relocation of staff, contracting and lay-off).

Readiness for Expanding Measures to Strengthen Essential Public Health Functions (Component III)

The 10-year National Strategy Plan specified mechanisms for expanding health sector reforms achieved during the first four years of the Program; action plans for restructuring the national laboratory network and pharmaceutical system are designed, costed and negotiated with principal stakeholders.

Triggers for the second phase of the APL by Project Development Objective After Restructuring Indicator Target (2008) Component 1: Improve access of the poor in Prioritized regions and Provinces to high-quality health services included in the Plan Básico de Salud of the Seguro Familiar. Percentage of primary health care facilities in prioritized regions and provinces accredited by SESPAS.

50%

Percentage of pregnant women who received prenatal care services in prioritized regions and provinces in accordance with national guidelines.

50%

Component 2: Reorganize and strengthen health systems in prioritized regions so they can comply with the roles and functions as established in the General Health Law and the Social Security Law. Number of prioritized regions using an integrated financial and performance management system.

4/4

Component 3: Strengthen the financial planning of the Subsidized Regime, ensure the effective enrollment of the poor, and improve their rights under the Seguro Familiar. Percentage of the poor in prioritized regions and provinces enrolled in the Subsidized Regime who know the benefits of the Basic Health Plan.

>70%

Percentage of infants born in public hospitals that receive health certificates >80% Component 4: Strengthen the capacity of national health sector organizations to fulfill their roles and functions as established in the General Health Law and the Social Security Law. CERSS develops a monitoring and evaluation system for reform progress and disseminates results annually

First report disseminated

61

Annex 11. Summary of the Final Report on the Investment Activities of PARSS 1 Description The investment activities in Component I aimed at strengthening the first level of health care delivery and other strategic areas related to the improvement of the supply side (management, distribution of supplies, and support services). Activities took place in Regions I to VII. While the actual implementation period of the Project was 2005-2009, these activities were implemented mostly during the last 2 years of the Project life. The total number of sub-projects is presented in the table below:

Table 1: PARSS 1 Investment Activities

Sub-project status Number of Sub-projects Completed 37 Started and not completed 5 Cancelled 9

Total 51 A total of 5 subprojects were started and not completed due to technical or procurement problems while 9 sub-projects were cancelled due to inconsistencies between their periods of implementation and the Project completion date. As shown in the table below, the majority of the sub-projects were executed during the last 2years of implementation, indicating that the preparation activities for these investments were not given an adequate priority:

Table 2: Infrastructure Sub-projects by year and amounts ($US millions)

2007 2008 2009 Completed Sub-projects 2 15 20 Investment amount 0.22 1.46 12.57 Accumulated investment amount 0.22 1.69 14.21

Component Performance The analysis of the infrastructure activities shows the following:

The total final cost paid for the 37 completed sub-projects was 25.24 percent higher than the total value of those contracts when they were awarded;

Only 28 of the 37 completed sub-projects presented scenarios of cost variations depending on the duration of the implementation. The final cost of the sum of these sub-projects was 10 percent higher than estimated at pre-investment and the total final cost was 38 percent higher than the value of the awarded contracts.

There were serious delays in the sub-projects’ implementation, as shown below, which

points to a low capacity at the PCU (CERSS, which actually had control over the first two phases) to manage these processes efficiently. The time-lapse for all three phases improved over the last 18 months of the life of the Project.

62

Table 3: Estimated and actual time of Sub-project implementation

Phase Period Planned (days) Actual Period (days) Variation (days) Bid evaluation 60 180 120 Initiation of works 90 222 132 Period between initiation

and completion of works 109 270 161

Preparation of Sub-projects The preparation phase experienced the following: Small rehabilitation works: These activities consisted of the rehabilitation of primary health care centers dispersed around the country. The infrastructure needs were assessed by different groups of professionals, guided by terms of reference that were too generic. As a result, the final products (the assessments) presented wide differences in depth and asymmetries. Furthermore, this was not identified as an issue by the technical team in CERSS who received the results of the assessments. This led to significant modifications to the contracts once the works were actually contracted. Large works: These works, though i small in number, represented a large share of the total investment. There were deficiencies in the elaboration of the designs, the selection of the locations, the monitoring and the initiation of the works. The designs were seriously weak, showed little or no knowledge of the locations selected, and the proposed implementation times were unrealistic. All of this caused significant modifications to the works during their implementation. Management and Implementation of Subprojects CERSS was responsible, through its Infrastructure Unit, for managing the implementation of these activities. Its management was highly centralized, and it did not consult with the health centers in the regions and only alittle with SESPAS. This might be explained by the shortage of human resources in SESPAS. This system did not allow leaving any learning experience in the other institutions, though it was the most advisable to achieve the implementation of activities. Within the CERSS infrastructure unit, there was a comparatively small group of technical staff dedicated to the preparation of sub-projects rather than to the monitoring of their actual implementation. The design of a significant number of sub-projects was concentrated in a reduced number of people, a factor that contributed to their inability to prevent and correct the deficiencies highlighted in the assessments. Technical deficiencies were present during the implementation of sub-projects as well. An example of these weaknesses was the sub-project of the Pimentel Hospital. These deficiencies in supervision were corrected by CERSS, after the Bank team allocated an expert to assess the status of the infrastructure component and placed a strong focus and constant support to overcome the problems and move the implementation of the viable sub-projects. A strong supervision of the “physical” progress of the sub-projects, and monitoring of the implementation times was agreed and implemented until Project completion.

63

Lessons Learned

It is crucial to prepare good quality and reliable diagnostics to provide realistic assessments of the needs. This is key to having a realistic understanding of the problems and to formulate adequate, realistic solutions;

In order to formulate an investment component, sector diagnostic studies should be prepared and they should be framed within the strategic programs of the institutions, which provide guidance on the sector priorities. Besides, it is important to define standards of quality and models for the provision of services;

In order to successfully prepare and implement infrastructure activities, it is key to have

strong institutional capacity to manage these processes; Focusing on the preparation phase is important: allocating enough human resources and

time will avoid problems during implementation, reduce risks, and avoid problems; Importance of focusing on implementation calendars while preparing an infrastructure

project; It is crucial to define during preparation the monitoring practices that will be used to

follow up on a project’s implementation. This allows for identifying milestones, monitoring processes, and drawing lessons from them.

64

Map