Document of The World Bank

FOR OFFICIAL USE ONLY

Report No: 30401

IMPLEMENTATION COMPLETION REPORT(CPL-40860 SCL-40870 PPFB-P2540)

ON A

LOAN

IN THE AMOUNT OF US$ 21.8 MILLION

TO THE

REPUBLIC OF PARAGUAY

FOR A

MATERNAL HEALTH AND CHILD DEVELOPMENT PROJECT

June 22, 2005

Argentina, Chile, Paraguay, Uruguay Country Management Unit Human Development Sector Management Unit Latin America and the Caribbean Regional Office

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective as of June 2004- ICR Mission)

Currency Unit = Guarani (PYG) 1 PY$ = US$ 0.00016

US$ 1.00 = PY$ 5,920.00

FISCAL YEARJanuary 1 December 31

ABBREVIATIONS AND ACRONYMS

AIEPI Atención Integral a Enfermedades Prevalentes en la Primera Infancia (Integrated Management of Prevalent Childhood Illnesses -- IMCI)

AIP Annual Implementation PlanARENA Atención del Recién Nacido (Care for the newborn)ARI Acute Respiratory InfectionsBCG Tuberculosis VaccineBCP Paraguy Central Bank (Banco Central del Paraguay)CAS Country Assistance StrategyCEBINFA Centro de Bienestar de la Infancia y la Familia (Family and Child Welfare

Centers)CEPEP Centro Paraguayo de Estudios de Población (Paraguayan Center for

Population Studies)CHW Community Health WorkerCIRD Centro de Información y Recursos para el Desarrollo (Development

Information and Resource Center)CORPOSANA Corporación de Obras Sanitarias (Water and Sewerage Works Company)CPPR Country Portfolio Procurement ReviewDGBS Dirección General de Bienestar Social (Social Welfare Directorate at the

MSPBS)DGRH Dirección General de Recursos Humanos (Human Resources Directorate at

the MSPBS)DGSS Dirección General de Servicios de Salud (Health Services Directorate at the

MSPBS)DPE Dirección de Promoción y Educación (Education and Promotion Directorate at

the MSPBS)DPT Diphtheria, Whooping Couch (Pertussis) and Tetanus vaccineENDS Encuesta Nacional de Demografía y Salud (Demographic and Health Survey)ENS Encuesta Nacional de Salud (National Health Survey)EPI Expanded Program of ImmunizationFGD Focus Group DiscussionsFOE Funciones Obstétricas Esenciales (Essential Obstetric Functions)GDP Gross Domestic Product

GNP Gross National ProductHC Hospital de ClínicasIADB Inter-American Development Bank (Banco Inter-Americano de Desarrollo)IBRD International Bank for Reconstruction and Development (World Bank) (Banco

Mundial)ICB International Competitive BiddingIEC Information, Education and CommunicationIES Income and Expenditure SurveyINVEC Computerized Inventory System for Pharmaceuticals and Medical SuppliesIPPF International Planned Parenthood FederationIPS Instituto de Previsión Social (Social Security Institute)JICA Japanese International Cooperation AgencyKfW Kredistanstalf fur Wiederaufbau (Germany’s international cooperation agency)LAC Latin America and the Caribbean RegionLCB Limited International BiddingMEC Ministerio de Educación y Cultura (Ministry of Education and Culture)MEF Mujeres en edad fertile (Women of Fertile Age)MH Ministerio de Hacienda (Finance Ministry)MIP Master Investment Plan of an Eligible Health RegionMOF Ministerio de Finanzas (Ministry of Finance)MSPBS Ministerio de Salud Pública y Bienestar Social (Ministry of Health and Social

Welfare)NCB National Competitive BiddingNCHS Unites States National Center for Health StatisticsNGO Non-Governmental OrganizationOBGYN Obstetrics and GynecologyOM Operational ManualOPCI Dirección General de Proyectos con Cooperación Internacional (General

Directorate of Projects with International Cooperation at the MSPBS)PAHO Pan-American Health OrganizationPAI Plano Ampliado de Inmunización (Expanded National Immunization Program)PCU Project Coordinating UnitPHRD Policy and Human Resources Development Fund financed by the Government

of JapanPOA Plan Operativo Anual (Annual Implementation Plan)PPF Project Preparation FacilityPSM Proyecto de Salud Materna y Desarrollo Integral del Niño (Maternal Health

and Child Development Project)RCO Regional Coordination OfficeRO Red Obstétrica (Obstetric Network)SENASA Servicio Nacional de Saneamiento Ambiental (National Service for

Environmental Sanitation)SENEPA Servicio Nacional para la Erradicación del Paludismo (National Service for

the Eradication of MalariaSM

Sanidad Militar (Military Health Service)SP Sanidad Policial (Health Service for the Police)STP Secretaría Técnica de Planificación (Technical Planning Secretariat at the

MSPBS)TFR Total Fertility RateTOR Terms of ReferenceUMT Unidad de Medicina Transfusional (Blood transfusion unit)UNDP/PNUD United Nations Development Programme (Programa de las Naciones Unidas

para el Desarrollo)UNFPA United Nations Population FundUNICEF United Nations Children’s FundUSAID United States Agency for International Development VAT Value-added taxWDR World Development Report

Vice President: Pamela CoxCountry Director Axel van TrotsenburgSector Manager Evangeline Javier

Task Team Leader/Task Manager: Montserrat Meiro-Lorenzo

PARAGUAYPY-Maternal Health and Child Development Project

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 115. Major Factors Affecting Implementation and Outcome 236. Sustainability 247. Bank and Borrower Performance 258. Lessons Learned 269. Partner Comments 2710. Additional Information 27Annex 1. Key Performance Indicators/Log Frame Matrix 29Annex 2. Project Costs and Financing 33Annex 3. Economic Costs and Benefits 37Annex 4. Bank Inputs 38Annex 5. Ratings for Achievement of Objectives/Outputs of Components 41Annex 6. Ratings of Bank and Borrower Performance 42Annex 7. List of Supporting Documents 43Annex 8. Summary of Post Project Evaluation of the Coverage, Quality and Efficiency In the Delivery of Basic Health Care in Project Areas

44

Annex 9. Borrower's Contribution 62

Project ID: P007927 Project Name: PY-Maternal Health and Child Development Project

Team Leader: Montserrat Meiro-Lorenzo TL Unit: LCSHHICR Type: Core ICR Report Date: June 22, 2005

1. Project DataName: PY-Maternal Health and Child Development

ProjectL/C/TF Number: CPL-40860; SCL-40870;

PPFB-P2540Country/Department: PARAGUAY Region: Latin America and the

Caribbean Region

Sector/subsector: Health (88%); Pre-primary education (8%); Central government administration (4%)

Theme: Child health (P); Population and reproductive health (P); Health system performance (P); Decentralization (S); Participation and civic engagement (S)

KEY DATES Original Revised/ActualPCD: 10/20/1994 Effective: 12/05/1996 02/14/1997

Appraisal: 08/13/1996 MTR: 06/15/1999 01/07/2002Approval: 09/05/1996 Closing: 06/30/2003 06/30/2004

Borrower/Implementing Agency: REPUBLIC OF PARAGUAY/Ministry of Health and Social WelfareOther Partners:

STAFF Current At AppraisalVice President: Pamela Cox Shahid Javed BurkiCountry Director: Axel van Trotsenburg Gobind T. NankaniSector Manager: (Acting) Cristian Baeza Alain ColliouTeam Leader at ICR: Montserrat Meiro-Lorenzo Sandra RosenhouseICR Primary Author: Montserrat Meiro-Lorenzo

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: U

Sustainability: UN

Institutional Development Impact: M

Bank Performance: U

Borrower Performance: U

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: Yes

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:Background. In the mid 1990s, Paraguay’s health infrastructure was insufficient and obsolete, reflecting the lowest GDP per capita investment in the region. As a result, the country substantially lagged behind in health services coverage, with immunization coverage well below 70 percent in children under 1 year old, and institutional deliveries at the level of the Andean countries. Paraguay had at the time the second highest estimated maternal mortality in the region, (350 to 386/100,000 live births). Institutional data from the Ministry of Public Health and Social Welfare (Ministerio de Salud Pública y Bienestar Social, MSPBS) indicated that infant mortality was around the Regional average (40.8 per 100,000 life births), and concentrated around the neonatal period. In contrast, the nutritional status was one of the best of the region and probably the underlying reason for a relatively low child mortality rate despite the low quality and accessibility of the health services.

In the decades leading up to the Project launch, investments into health services had not responded to population growth and internal migration. The existing health infrastructure, particularly in the rural areas, was insufficient due to the lack of maintenance. The health network did not have a communication and a transport system making the referral of emergencies extremely difficult. Paraguay was the only country in the region that did not establish a national cold chain for immunization. Medical personnel were concentrated in Asunción and surrounding areas, leaving many departments seriously understaffed. Moreover, most professionals had not received training in over 10 years. Studies of the Pan-American Health Organization (PAHO) showed that about 37% of the physicians did not diagnose and treat pneumonia correctly.

Objectives. The Maternal Health and Child Development Project, the first Bank-financed Project in this sector, was approved by the Board in September 1996. The main objective of the Maternal Health and Child Development Project was to improve the health status of the underserved Paraguayan population, particularly women and children (Staff Appraisal Report [SAR] 15610-PA, p. 19). To achieve this objective the Project was to (a) increase the coverage, quality and efficiency of basic maternal and child health services in six underserved departments in Northeastern Paraguay; (b) increase the population’s knowledge about adequate health practices; (c) pilot-test a strategy to enhance early child development in Asuncion; and (d) strengthen management capability of the Ministry of Public Health and Social Welfare (Ministerio de Salud Pública y Bienestar Social, MSPBS) in support of an eventual decentralization of public health services in Project areas (SAR, p. 19). Hence, for the specific objectives (a), (b) and (d) the Project areas were the country’s six most underserved departments in Northeastern Paraguay. The child development pilot was to be implemented in the low-income areas of Asunción.

Specified targets for the objectives. The Staff Appraisal Report did not specify specific targets for the impact of the Project but did set quantitative targets for the coverage of the population. The SAR specified 84 indicators (SAR, Annex 8) to monitor and assess the impact of progress for the Project. The most pertinent impact indicators for the overall Project objective (improvement of the health status of the underserved Paraguayan population, particularly women and children) are: (i) maternal mortality rate; and (ii) infant mortality rate. For the Project to have met its development objective, at least a positive trend for these impact indicators (in the six Project areas) would be expected. While the SAR did not quantify specific targets for the large number of indicators measuring health service delivery or health knowledge of the population, the SAR did postulate an ambitious coverage rate for essential mother and child health services (SAR, p. 20): the total population which was to be covered by the health component was 500,000 (i.e. the poor) out of which 210,000 were estimated as children under five years of age and women of reproductive age. As shown in Annex 1, Table 4 of the SAR (p. 56), such target implied a 100 percent

- 2 -

coverage of the poor population, especially women and children, with essential health services by Project completion. In the selected regions for Project implementation the private sector played (and plays) a very minor role in service provision so that the specified targets effectively translated into overall population coverage rates of close to 100 percent.

3.2 Revised Objective:While the Project activities were substantially restructured in 2001, the original objectives were not formally revised (compare Section 3.4.).

3.3 Original Components:The Project consisted of four components. Component A was the main component, directly addressing Project objectives (a) and (b), as well as supporting objective (d). Separate components, B, C, and D addressed the remaining Project goals. The implementation period was estimated at six years. The original closing date was June 30, 2003. The closing date was extended to June 30, 2004, in recognition of improved performance following Project restructuring, to finalize some activities and allow for evaluation of results and orderly closing.

Component A- Maternal and Child Health Care. (US$20.1 million equivalent without contingencies or 74.3 percent of total baseline Project cost as appraised). This component aimed to improve the health conditions of poor women and children in six of the poorest, predominantly rural, departments in the Northeast of Paraguay through the following subcomponents:

A.1 Rehabilitation and Maintenance of Infrastructure and Equipment SubProjects (US$11.2 million without contingencies). This subcomponent aimed to strengthen the delivery of maternal and child health care services in six targeted departments by improving existing infrastructure and ensuring the availability of needed equipment and adequate maintenance in regional hospitals and health centers. Financing would support subProjects involving works, basic medical equipment, communications equipment, vehicles (including motorcycles), recurrent maintenance costs and training for maintenance staff. New construction was only to be considered if deemed necessary to improve efficiency of existing infrastructure. A Master Investment Plan (MIP) was to be prepared for each health region [1] to establish the investment priorities for subProjects. The Regional Health Council (RHC) was to present subProject proposals annually for review by Regional Coordination Unit (RCU) before submission to the Project Coordination Unit (PCU). The criteria for the selection of subProjects were the following:

Infrastructure and equipment would be used primarily for maternal and child care;lSubProject addressed needs identified during Project preparation and/or by the study of the lregional health network;SubProject implementation would not exceed 12 months or US$200,000 equivalent;lRecurrent costs generated by the investment in excess of 15 percent would require evidence lof the availability of budgetary funds to cover such costs; SubProjects would provide for maintenance; and lThe facilities benefiting from the subProject would be adequately staffed.l

A.2 Pharmaceuticals and Supplies (US$6.64 million, without contingencies). This subcomponent was to finance procurement and distribution of pharmaceuticals and medical supplies for basic maternal and child health services in eligible health facilities. Financing was also to be provided to train personnel on the operation of a computerized inventory system (INVEC) for pharmaceuticals and medical supplies. Only the health facilities with adequate storage space were to be eligible to benefit from this subcomponent. Procurement of drugs was to be centralized, but to ensure that local needs were met, items were to be

- 3 -

regionally selected by the health regions from a list of essential drugs.

A.3 Information, Education and Communication (US$1.19 million, without contingencies). This subcomponent aimed to promote preventive health care in the communities served by the Project through a program of information, education and communication (IEC). The program aimed to inform the population about the health care services available in the community and promote good health practices. It also aimed to promote the participation of civil society in the provision and management of health services. The subcomponent was to finance consultants’ services and technical assistance to review the existing educational materials, define the contents of media messages, and collect information on existing breastfeeding and weaning practices, child and pregnancy care, attitudes and beliefs regarding disease prevention, family planning and health promotion. Financing was also to be provided to develop and disseminate new printed materials and radio messages, and to cover travel expenses of personnel involved in the program.

A.4 Training (US$1.05 million, without contingencies). This subcomponent aimed to support the design and implementation of an in-service training program for paramedical and medical personnel (including managers) to improve their technical competence to provide health care services, health education, detect health risks, and carry out Project activities. The subcomponent was to finance training and training materials, as well as an assessment of the training needs of paramedical staff, program follow-up and evaluation, and travel expenses for trainers and trainees. A competency-based training methodology was to be used, emphasizing prevention and treatment of key epidemiological problems in each region. Specifically, training would focus on diagnosis, referral and care of: pregnancy and obstetric complications, safe delivery, reproductive health and common childhood diseases. Training would also cover interpersonal skills for relating to patients, community outreach techniques, and -- for administrative staff -- management and organizational skills. In each region, a Training Action Plan was to guide training activities. Approximately 80 percent of the funds were to be allocated to train paramedical staff and community health workers (CHW), and the remainder was to finance training for trainers and other personnel.

Component B- Early Childhood Development (ECD) Pilot. (US$2.12 million equivalent without contingencies or 7.8 percent of total baseline Project cost as appraised). Proposed as a pilot Project, the component was to test a strategy to improve health and psychosocial development of 2-5 year old children in poor areas of Asunción by establishing early childhood development centers (CEBINFAs) and strengthening the Social Welfare Directorate’s (Dirección General de Bienestar Social, DGBS) capacity to supervise ECD services and give technical assistance to communities interested in providing these services on their own. The CEBINFAs were to be located in neighborhoods classified as poor (based on 1992 census indicators of water supply, sanitation and education). The Project was to finance the physical rehabilitation of up to 40 CEBINFAs, educational materials and supplies, furniture and equipment, training for care-giving mothers and for managerial staff, and operating costs including three meals a day for children and stipends for educators. Rehabilitation works were not to exceed US$5,000 equivalent per center and labor would be provided by the community. The operating costs of the centers were to be partly financed by the corresponding communities. The stipend for care-givers would be partly financed from monthly payments made by parents of children served by the CEBINFAs (about US$10.00 equivalent per month, per child).

Component C- Support for Decentralization. (US$1.20 million equivalent without contingencies or 4.4 percent of total baseline Project cost as appraised). This component aimed to improve the management capacity of the MSPBS, in preparation for the eventual decentralization of public social services, including health. The management capacity of health sector staff was to be strengthened at the regional and central

- 4 -

levels through: (a) staff training; (b) establishment of management information systems; (c) pilot activities to increase Project sustainability; and (d) financing recurrent costs to supervise maternal and child health services in the Project areas. Management training would be contracted out to private firms or academic institutions.

Component D - Project Administration, Monitoring and Evaluation. (US$3.17 million equivalent without contingencies or 11.7 percent of total baseline Project cost as appraised). The Project was to be administered by a Project Coordination Unit (PCU), reporting directly to the MSPBS Minister and the General Directorate of Projects with International Cooperation (OPCI). The PCU was to be assisted by Regional Coordination Offices (RCOs) established in each region. All Project resources were to be managed centrally by the PCU, and funds were to be allocated annually to the regions based on Annual Implementation Plans (Plan Operativo Annual), except for small civil works, goods and service contracts which were to be procured by the RCOs. Other institutions to be involved in Project execution included: Regional Health Councils, responsible for setting priorities for investment under Subcomponent A.1; the Direction of Health Promotion, Prevention and Education, Dirección de Promoción, Prevención y Educación de la Salud (DPPES), and the Direction of Community Participation, Dirección General de Participación Comunitaria (DGPC), at the MSPBS, responsible for carrying out Subcomponent A.3; the general Direction of Health Services, Dirección General de Servicios de Salud (DGSS) and the General Direction of Human Resources, Dirección General de Recursos Humanos (DGRH) at the MSPBS, responsible for the implementation of Subcomponent A.4; the General Direction of Welfare, Dirección General de Bienestar Social (DGBS) at the MSPBS, responsible for the implementation of Component B; and finally, the Education Ministry, the Ministerio de Educación y Cultura (MEC) was to provide technical supervision of early childhood stimulation activities under the same component.

The Operational Manual (OM), approved at negotiations, described the objectives, activities, institutional arrangements, monitoring indicators and implementation targets for each Project component, the functions and attributions of each entity involved in Project implementation, and the technical, financial and procurement norms and procedures applicable to the Project.

3.4 Revised Components:The Project was off to a difficult start. By end 1999, the lack of counterpart funding had become chronic and the Bank considered Project management inadequate. After about sixty months of implementation, only six percent of the total Project cost had been disbursed. Successive supervision missions identified the following disconnects between some of the original features of the Project and implementation arrangements, and the country’s situation:

- 5 -

(i) frequent changes of the Ministry and PCU staff (four ministers between 1997-2000) had weakened the MSPyBS ownership and understanding of the project aims. Furthermore, project coordination was poor and the PCU was seen as a separate Ministry by the MSPBS’s line staff, and contributed little to improve the Ministry’s management capacity. The MSPBS’s National Directorate of Social Welfare, responsible for the implementation of Component B, had been substantially weakened by the successive administrations;

(ii) there was little or no ownership of the project by the Government at-large, thus contributing to the project’s continuous lack of counterpart funds;

(iii) sector policies central to Project implementation had been reversed, including that (a) decentralization was no longer on the agenda of the administration. All initial advances in this direction had been reversed and the Ministry remained highly centralized; (b) the policy on community participation in health services management had been reversed; (c) pilot activities to increase the project’s financial sustainability by purchasing pharmaceuticals and creating revolving funds in the health facilities (Components A and C) were contradicted by a policy that eliminated charges for pharmaceuticals; and (d) family planning activities had been stopped by one Minister;

(iv) the capacity of the national Public Health Institute to train personnel was lower than expected , but contracting-out training was not acceptable to the Ministry;

(v) Regional Directorates, that were the key implementers of most activities had a very low implementation capacity, and since decentralization policies had been reversed, no longer had the authority to be strengthened and to carry-out their originally proposed role;

(vi) the CEBINFAs required more effort to open and supervise than expected, were more expensive than predicted, and their potential impact was deemed marginal for the improvement of neonatal and maternal deaths;

(vii) lack of doctors outside Asuncion, which had been one of the Project driving force tofocus on community health workers and traditional births attendants to deliver services, had been overcome by generations of new graduates from medical schools willing to go to the provinces. The new issue was to ensure that those professionals had the adequate skills.

From 1996 to 2000, the Project disbursed only six percent of the loan (about US$1.9 million). Few Project activities were started (among them contracting the design for the health facilities to be built; financing of rehabilitation of a few health facilities; launch of two bids for basic pharmaceutical drugs). In the ECD Pilot, four CEBINFAs were rehabilitated and launched in Asunción. However it soon became clear that the model that called for volunteer mothers to run the centers was not working, and that it was not perceived as a priority, thus further investments were discontinued.

The Maternal Health and Child Development Project was not the only problem Project in the Bank’s portfolio in Paraguay. After the political instability of the late 1990’s, and the subsequent economic crisis, practically the entire Bank portfolio was on hold amid accusations of government corruption and shortage of counterpart funds. There were no Projects in the pipeline and the dialogue with the country was reduced to a minimum. At that point the Bank considered canceling the Loan balance.

By early 2000, the country’s political situation stabilized and the new Minister of Health was committed to

- 6 -

implement most of the Project on an accelerated basis. Also, in contrast to his predecessors, the new Minister was able to secure, most of the time, counterpart funds for the Project, speed-up implementation, and gave substantial support to a newly staffed PCU. With these improved prospects for satisfactory implementation, the Bank agreed not to cancel the loan balance, but to review Project activities.

Project Restructuring

The Project was restructured progressively throughout 2000. It affirmed the validity of Project objectives (a) (b) and (d). Recognizing the difficulties in establishing the CEBINFAS, and their cost, the Bank and Government agreed that the pilot had shown the non-feasibility of the early childhood development scheme. Further, the means of the Project to achieve the objectives and the scope of objective (d) were deemed inadequate.

A full review of the Project’s activities was undertaken based on a Project-financed study on the quality and efficiency of the health services in the Project areas respect to eight essential obstetric and one neonatal functions (8FOE&1N) [2] considered fundamental to reduce maternal and neonatal mortality. The thrust of the restructuring was to focus Project activities on achieving a targeted client-specific reduction in a selected set of acute and frequent health conditions, rather than to continue with efforts to create an environment in which such conditions (and others) might be treated. This was to give the Project a greater sense of focus. It implied changes as to how objectives were to be achieved, who was responsible for the activities, and how many types of activities were to be implemented. As a result, the redesigned Project sought to: (i) reduce the institutional complexity; (ii) align activities with national policies, particularly by adopting centralized management; and (iii) limit activities to those that could have, in the medium term, a direct impact on maternal and child mortality around childbirth.

Although the Project operated following the new orientation from early 2001, the formal amendment to the Loan Agreement(s) to reflect the revisions in the Project financing was not signed until February 4, 2002. The delay in signing was partly due to a six-month gap between the Bank's task managers. After 95% of the funds had been disbursed, there was a one year extension of the Loans’ Closing Date until June 30, 2004, to reflect a four and a half months portfolio-wide suspension of disbursements, and small delays caused by an electoral year, and to permit a smooth closing of Project administrative hand-over to the new administration.

Restructured Component A- Maternal and Child Health Care: The specific objective of this component remained unaltered by the restructuring. In addition to the factors mentioned above, restructuring was necessary due to the Regional Directorates requiring more intensive support to prepare subProjects than had been previously estimated during preparation of the Project. Also, successive administrations had not complied with the preparation of regional implementation plans. Further, while Project-supported training had been initiated for traditional midwives and health workers, the new health policy discouraged them from carrying-out deliveries which posed a problem for Project implementation.

The Borrower and the Bank teams agreed that the key strategy to curb mortality in the target groups was to develop an obstetric network (red obstétrica). The Project financed the 8FOE&1N study that provided detailed data on the health service provision in the Project areas, and was used as a situation baseline and to develop a clear map for Project activities.

The revised activities included: (i) improvement and expansion of primary and secondary health infrastructure in the poorest areas of the country; (ii) creation of a communications and transportation

- 7 -

network to improve referrals and transport of obstetric emergencies to health facilities; (iii) creation of a network of blood banks to improve the supply of safe blood in obstetric emergencies; (iv) pilot new techniques for behavior change among the population in three specific subjects linked to key maternal and child mortality causes (immunization, blood donations and institutional deliveries) through communication campaigns at local level; (v) training of health sector staff in basic obstetric and neonatal care and on AIEPI; (vi) improvement of data collection for maternal mortality through the creation of maternal mortality investigation committees; (vii) strengthening the national immunization campaigns and national health programs; and (viii) strengthening capacity at the regional level to train health personnel in emergency obstetrics, care for the newborn, and immunization; monitor such training; manage immunization campaigns; and manage and implement local behavior change and education campaigns.

Component A activities were further revised to reduce those interventions deemed less effective in curbing maternal and perinatal mortality, such as non-pregnancy-related reproductive health care for women, and traditional midwives and health workers training. Finally, the teams decided that to reduce child mortality it was paramount to focus the Project’s support on some of the most cost-effective interventions for children under age five, namely, the Expanded National Immunization Program (Plano Ampliado de Inmunización, PAI) and the integrated treatment of childhood diseases (Atención Integral a Enfermedades Prevalentes en la Primera Infancia, AIEPI) including the development and introduction of a neonatal AIEPI to reduce neonatal deaths.

Restructured Component B- Early Childhood Development (ECD) Pilot. The Government and the Bank agreed to cancel any remaining activities under the Early Childhood Development Pilot (Component B) since it had become clear that the pilot experience had not been successful. As a result of these changes, the Project became entirely focused on addressing the root causes of maternal and child mortality.

Restructured Component C- Support for Decentralization. The Government and the Bank also agreed to reduce the scope of the ‘Support for Decentralization’ component, narrowing its focus from general capacity building to strengthening regional capacity to carry out personnel training, immunizations, and local communication campaigns in support of maternal and child health care, and to re-focus the training subcomponent (Component A) towards medical personnel and for supporting the implementation of national maternal-child health care programs introduced by the MSPBS at the end of the 1990’s.

Restructured Component D- Project Administration, Monitoring and Evaluation. The PCU was reorganized to reduce the role of departmental sub-units, and integrated into the MOH structure. It was re-staffed. In addition, the 8FOEs& 1N survey was carried-out to be used as a base line on quality and efficiency of health services. It was decided that the Government and the Bank would use the results of this survey (together with those of the Sexual and Reproductive Health Survey) as tools to monitor and evaluate the Project.

3.5 Quality at Entry:Unsatisfactory. Project design reflected the recommendations of a poverty assessment conducted by the Bank in 1994 by targeting interventions to poor areas, strengthening service delivery, emphasizing preventive health care, promoting community participation, and improving sector management capacity. However, by today’s standards, the quality of entry of the Project suffered a number of important shortcomings including: (i) unrealistic assumptions underlying the relevance of parts of the Project’s design; (ii) absence of adequate base-line data and monitoring and evaluation systems; (iii) too ambitious targets for the coverage of the population to be reached; and (iv) an inadequate institutional assessment. Moreover, the social and

- 8 -

institutional instability affecting the country and the MSPBS during the first years of Project implementation did not permit the immediate correction of these issues once identified.

The first and most important shortcoming was the lack of direct relevance of some of the objectives (and corresponding components) to the key sector issues. The introduction of Component B: (Early Childhood Development Pilo), late in Project preparation phase proved to be a miscalculation, as it diffused implementation efforts away from the Project’s main component, and thus from the key health issues confronted by poor mothers and children in Paraguay. Likewise, it is not clear how as designed, Component C (Support for Decentralization) could have supported the health sector without also addressing longstanding structural weaknesses at the national level, particularly in a country with such strong central government. These two components increased the Project’s institutional complexity by adding Regional Directorates and a newly upgraded (and unproven in its implementation capacity) National Directorate of Social Welfare (within the MSPBS) as surrogate implementation units.

Second, although the SAR recognized that vital statistics and health service data in Paraguay were unreliable and thus inappropriate to evaluate Project outcomes, it only partially addressed the issue. The SAR appropriately proposed the use of survey data to evaluate Project outcomes. However, there was no clear guidance on the activities to establish a monitoring system and at the same time, the SAR listed more than eighty monitoring and impact indicators. The level of guidance provided in the SAR did not seem to be conducive to the development of an operational Monitoring and Evaluation (M&E) system. Furthermore, given the identified data problems, an excess results of indicators was specified. Finally, there was no clear connection between all Project objectives, some interventions, and some expected outcomes and indicators.

Third, the SAR specified an – in hindsight – too ambitious target for health service coverage of mothers and children in the Project areas. While the SAR did not establish target values for overall impact indicators of the Project, it did specify target coverage rates of the basic package of mother and child health services which was to reach 100 percent of the poor population by Project completion (SAR, p. 20). Given the difficulties stemming from both the supply side (functioning and governance of health service delivery in Paraguay) as well as demand factors outside of the Project influence (transportation, household income, traditional health beliefs) such complete coverage rates were likely to be overoptimistic.

Fourth, institutionally, the Project implementation arrangements were complex. The arrangements did not establish clear coordinating lines with regular ministry programs and seem to have underestimated the weak implementation capacity of the executing agency. For example, the design included the creation of Regional Coordination Offices and Regional Health Councils with the responsibility to identify and review infrastructure and equipment subProjects, as though such investments were to be managed as a social investment fund. At the same time, however, investments were to be identified based on a comprehensive diagnostic study of the six Project departments with decision-making taking place at the central level. Another example is the lack of experience of the agency that was in charge of implementing the Early Childhood Development Pilot with respect to mobilizing low-income urban communities to run a facility-based integrated ECD program.

Some of the potential strengths of the design became liabilities as the socio-political turmoil in the country developed. Notably, the creation of a strong PCU (a clear result of the lessons learned) became a handicap. The proportionally higher salaries of the PCU, particularly as the country entered an economic crisis, became a magnet for political appointments, and the unit’s staff was changed frequently by the different ministers (four ministers in three years). Moreover, the MSPBS staff perceived the PCU as a parallel ministry that had preferential treatment and direct access to the different Ministers. As a result, there was

- 9 -

little Project ownership in the MSPBS. This was particularly apparent in the design of Component A where the most central set of Project interventions were not coordinated with the MSPBS’s national program, thus reducing Project ownership and fostering animosity.

The adjustments to the Project introduced in the 2001 review addressed most of the perceived design shortcomings. As a result, the pace of implementation accelerated substantially and the loan disbursed in three and a half years what had been Projected to implement and disburse in six years.

- 10 -

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:UnsatisfactoryAlthough the Project achieved an important turnaround in performance after the restructuring in 2001, the Project’s overall performance towards achievement of its stated development objectives needs to be considered as unsatisfactory given that (a) the available survey as well as administrative data on overall impact indicators show either stagnant (infant mortality) or worsening (maternal mortality) overall trends during the lifetime of the Project (it is acknowledged that the time series of administrative data include important measurement errors); and (b) available data on health service delivery from the National Reproductive Health Survey show improvement in coverage rates for the Project areas which, however, fall significantly short of the 100 percent coverage objective as stated in the SAR. This said, and as will be shown below and in Annex 1, the Project has made significant advances to improve efficiency and quality of care in Project areas since 2001 and thereby has partially obtained the specified objectives. Hence, if the rating scale would be available, the Project rate for the achievement of overall development objectives would be moderately unsatisfactory.

Main development objective. The main objective of the Maternal Health and Child Development Project was to improve the health status of the underserved Paraguayan population, particularly women and children (Staff Appraisal Report [SAR] 15610-PA, p. 19). Annex 1 (Key performance indicators), includes both a description of indicators used at various times of Project preparation and execution and specifies available information on the major impact indicators (maternal mortality, neonatal mortality and infant mortality):

Base LatestRecorded maternal mortality, (MOH administrative data) - national 101.2/ 100,000 l.b. (1996)

(182.1)/100,000 l.b.(2002)Neonatal mortality, (MOH administrative data) - National 11.3 (1996) 11.4 (2001)Neonatal mortality, (EDSSR)- National (previous 5 years) 16.0** (previous 5 years) 17.0** (previous 5 years)Infant mortality, (administrative data)- National (registered)- National (estimated)

20.9 (1996) 40.62 (1996)

19.6 (2002) 19.8(2001)38.10 (2000)

Infant mortality, (EDSSR)- National (previous 5 years)- National (previous 10 years)

27.0 (1996)**33.0 (1996)

29.0 (2004)**26.0 (2004)

** not statistically significant changeSource: Annex 1

The above table includes both administrative data as well as data that recently became available from a recent National Reproductive Health Survey. The survey instruments generally provide reliable estimates and, while the survey did not collect data in the six Project regions alone, it can be seen that key outcome indicators (neonatal and infant mortality) did not change significantly over the implementation period of the Project. Regarding administrative data, this cannot be relied on for comparative purposes as the degree of underreporting of mortality estimates changed significantly over the last years in Paraguay, in part due to

- 11 -

Project activities. Hence, the large increases in key outcome indicators as recorded in the above table are very likely not correct. Today, underreporting of maternal deaths it is estimated at 30%. In sum, the available information is incomplete (no geographic information from the survey) and provides no conclusive evidence on developments of key outcomes the Project was to achieve. Survey data suggest, though, that overall outcome indicators did not improve in Paraguay as had been stipulated in the SAR.

Specific development objectives. Given the large number of monitoring and impact indicators determined at the time of appraisal, this ICR focuses on a number of key indicators which have been chosen among the large list of monitoring and impact indicators specified in the SAR. These indicators are: (i) percentage of pregnant women that had at least one neonatal visit during their pregnancy; (ii) percentage of pregnant women who had at least one prenatal visit during the first trimester of their pregnancy; (iii) percentage of institutional deliveries; (iv) Percentage of newborns that had their neonatal controls; (v) immunization coverage in children from 12-23 months; and (vi) percentage of health staff that applies the AIEPI (IMCI) strategy.

Objective 1: To increase the coverage, quality and efficiency in the delivery of basic maternal and child health care services in the Project area.

The results of this component were assessed through: (i) collection of health service coverage data from two comparable nation-wide household surveys (in 1998 and 2004); (ii) specialized quality and efficiency surveys conducted in 2001 (at the time of Project restructuring) and 2004 at the end of Project implementation. The methodology and detailed results of the quality and efficiency survey is presented in Annex 8.

Improvement in Service Coverage: The improvement in two key proxy indicators of service availability and coverage showed that coverage improved substantially over the lifetime of the Project but fell short of the universal coverage target (of the poor population) specified in the SAR. (See also Annex 8).

Using administrative data, Table 1 shows the number and percentage of institutional deliveries between 2001 and 2003. The table compares the Project areas to the rest of the country. Notwithstanding data limitations, institutional deliveries in Project departments increased by 3.8 percent compared to 2.3 percent in the rest of the country and 2 percent in the two neighboring departments, thereby helping to bridge the gap between the Project area and the country as a whole (see Annex 8, Table 1). In addition, the capacity to carry out at-risk institutional deliveries increased in the Project area by 8.7 percent from 2001 to 2003.

- 12 -

Table 1. Number and Percentage of Institutional Deliveries in Project and rest of the Country 2001 2003 Departments NVR Institutional % NVR Institutional % Concepción 3398 2409 70.9 3543 2547 71.9 San Pedro 3745 2692 71.9 4046 3019 74.6 Alto Paraná 8791 6394 72.7 8109 6123 75.5 Amambay 1471 1311 89.1 1263 1128 89.3 Canindeyu 1746 1143 65.5 1439 1100 76.4 Alto Paraguay 119 93 78.2 160 128 80.0 Project Departments 19270 14042 72.9 18560 14045 75.7 Project at risk deliveries/total 18.3 19.9 Paraguay TOTAL 83919 71736 85.5 83000 72616 87.5 Source: Dept. Bioestadistica, MSPBS (2004). Note: Data for 2003 are provisory; NVR=registered live births

Although less reliable, institutional records on child health care visits at the regional hospitals in the Project area also showed improvements in coverage from 55.1 percent in 1997 to 61.5 percent in 2004. Vaccination coverage in the Project areas and in the country also improved during the Project period mainly due to the Project-financed establishment of the cold chain and the constant support for Project-financed vaccination activities.

To summarize, the existing data on trends in coverage rates of key services supported by the Project indicate that the Project did cause overall improvements. However, such improvements fell short of the specified target.

Improvements in Quality: Two types of survey, the “8FOE&1N surveys” (2001-2004) and the AIEPI assessments, reviewed quality and efficiency parameters, measured by the presence and functionality of the infrastructure and equipment, and presence of the required qualified personnel, needed to perform the identified functions and or tasks. It also evaluated the technical competence of the personnel through observation of the tasks performed. As mentioned above, the 8FOE&1N survey (financed with Project funds) was fielded in 2001 (at the time of Project restructuring) and in 2004 (at Project ending). Comparisons to non-Project areas are only indicative and cannot be attributed to the Project since no proper baseline survey had been conducted which would have allowed a proper impact assessment, comparing Project interventions to a properly defined control group of beneficiaries and health institutions.

In the Project areas, quality of service in different types of health institutions improved significantly during the time period from 2001 to 2003 [3]. Using the 8FOE&1N functions as a reference, the quality of services in the Project area hospitals improved by over 15 percent, on average achieving a 71.6 percent satisfactory performance compared to 62 percent in 2001 (see Annex 8, Table 3). The only exception was a slight decline in the score for the referral function, which in fact is good news because it reflects the improved capacity of the Project hospitals to treat obstetric and neonatal patients with more serious conditions. Although the improvements in health centers and posts were more modest (55 percent and 45 percent satisfactory performance respectively), the figures suggest a real improvement in the quality of health services at these levels as well. Likewise, health posts have seen those functions that should not have been performed at their level reduced (with the exception of blood transfusion, which needs to be monitored), and an increase in their essential function, family planning.The 2004 round of the 8FOE&1N included data collection on the quality of prenatal care provided in the Project areas [4]. The survey shows

- 13 -

good quality of prenatal care at all levels, including health posts.

Finally, UNICEF’s AIEPI assessment shows that in 2004, in the Project-supported departments, 58 percent of the health staff applied AIEPI in daily practice. This figure represents an improvement of 9 percent over the 2001 baseline. Most importantly, almost all the improvement was at the level of nursing assistants, which are responsible for 85 percent of pediatric outpatient pediatric visits. In the Project areas, the increase of use of AIEPI in their consultations was 30 percent on average. This increase was greater in the departments that started with lower levels of implementation of this national strategy. Direct observations demonstrated that by using the AIEPI strategy, nurse assistants were able to diagnose and treat/or refer children better.

Improvements in Efficiency: The specialized surveys measured the level of efficiency of health facilities as a function of: (a) the technical/knowledge abilities demonstrated by the staff, (b) the physical characteristics of the health institution, (c) the equipment, (d) the availability of supplies and drugs, and (e) the availability of essential health personnel. Although Project interventions addressed most of these efficiency conditions, the availability of health personnel (and to a lesser extent that of drugs and supplies) were outside the control of the Project. Nevertheless, all efficiency measures improved between 2001 and 2004 in the Project areas and surpassed those in non-Project departments, with the exception of availability of supplies and drugs (see additional Annex 8, Table 4). While such comparisons are informative, they do not represent a regorous impact evaluation since no control group had been selected at the start of the Project. Hospital efficiency reached 71.6 percent in Project-supported departments compared to 71.1 percent in non-Project departments; Project health centers exceed by 20.3 percent points those in non-Project departments; and Project-supported health posts improved to the level of non-Project health posts. The performance of essential obstetric and neonatal functions also improved, depending on the type of health facility. For example, according to the survey :

- 14 -

• Surgical procedures in hospitals improved by 21.2 percent, particularly with respect to physical conditions of the establishment (31.8 percent), supplies (43.5 percent) and availability of health personnel (49.4 percent), all of which are presently comparable to non-project departments.

• In 2003, conditions for the performance of anesthesiology services in Project hospitals were 25.7 percent better than in non-project hospitals – albeit, as stated above, this does not lend itself to a statement about the impact of the project due to the absence of a properly defined control group which would have lent itself to attribute changes in performance to the project itself. The availability of anesthesiology staff in hospitals improved by 54.3 percent. Yet, efficiency losses were observed with respect to supplies and drugs, which declined by 84.1 percent.

• Blood transfusion conditions improved by 35.4 percent in hospitals, but deteriorated in health centers (-54.5 percent) and remained approximately the same in health posts (-1.9 percent). In hospitals, improvements in the availability of equipment (64.8 percent) and specialized health personnel (53.2 percent) largely account for the overall positive result.

• For newborn care, hospitals show a 40 percent efficiency gain, largely due to improvements to the physical plant (50.4 percent), availability of supplies (59.6 percent) and presence of specialized health personnel (54.6 percent). At health centers and posts, conditions did not change significantly.

• Pre- and post-natal care improved in hospitals by 26.2 percent due to better equipment and supplies. Conditions remained basically unchanged in health centers and posts.

Regarding the efficiency of the application of the AIEPI strategy, in the baseline study, only 21 percent of the health facilities had the adequate equipment, and medical and administrative material. Only 3.8 percent of the evaluated services had the required drugs required to treat the basic childhood illnesses according the AIEPI protocols. In the AIEPI evaluation in 2003, 100 percent of the health facilities in the Project areas had the basic equipment and materials necessary to carry out the AIEPI program and 35 percent of the facilities also had the pharmaceuticals to treat the basic illnesses.

These data indicate that the quality of obstetric and neonatal care in the Project areas improved significantly during the active years of the Project. In particular, the Project improved the capacity of hospitals to perform obstetric surgery, provide blood transfusions, care for the newborn, and exercise proper vigilance of patients during labor.

Objective 2: To increase the knowledge of the population about adequate health practices.

Starting in 2000, efforts to strengthen community knowledge of healthy maternal and infant practices were carried out in three of the six Project departments, namely in the Concepción, Alto Paraná, and San Pedro Departments [5]. Together these three departments account for 80 percent of the total Project target population. The IEC activities were focused on: (i) increasing knowledge about the importance of vaccination and increasing vaccination demand; (ii) increasing knowledge about the importance of prenatal care and institutional deliveries and increasing the demand for these services; and (iii) mobilizing communities to donate blood. Prior to the intervention, the Project contracted local experts to carry a survey of knowledge attitudes and practices (KAP) in the three areas where IEC activities were going to be implemented. After the interventions, the same group evaluated the results through focus groups and

- 15 -

in-depth interviews with community leaders and health services service users and non-users. The results indicate that after the activities, the population in general increased its knowledge about the three targeted issues, (Table 2).

Table 2. Knowledge of need for prenatal care, immunization and blood donations by region

Region Results 2002 KAP Results 2004 KAPAlto Paraná 52 % 73%Concepción 47 % 70%San Pedro 32% 67%

There was also an increase in the demand for vaccination and competition among barrios to get high vaccination coverage. Prenatal care attendance also increased. Finally, for the first time ever, the three regional hospitals in the selected areas have a list of voluntary donors that give blood regularly. In addition, the Project created instruments for effective community outreach. Based on a survey of selected communities carried out in 2002, the MSPBS now has the know-how to disseminate critical health information to local communities in Paraguay. Youth were selected as the main health promotion agents in the community, an appropriate approach given their leadership potential and enthusiasm, and the high return of their knowledge throughout their lives. The experience of mobilizing youth was very successful, as their activities resulted in desired results (see outcomes below) and the IEC evaluation showed good acceptance by the community of these volunteers. Moreover, the youth theater groups formed under the Project are continuing with their educational efforts even after the Project’s end. This experience has provided a good model that can be replicated elsewhere in the country (see outputs by components below). By the end of the Project, there were formal requests from several communities and secondary schools to continue and expand the efforts.

Objective 3: To test a strategy to improve early childhood development.

During the 2001 Project review, it was decided that the results of the pilot-testing of the early childhood development strategy in Asuncion were not conducive to provide continued support for this activity. The experiment failed to provide the integrated health, nutrition and education services for preschool children originally envisaged, basically because the CEBINFAs model did not succeed in generating sustainable community capacity to operate the centers. More importantly, the intervention did not respond to the epidemiological profile of Paraguay, which indicates that priority should be given to infants, particularly during the perinatal period of life. For this age group, the ECD strategy was clearly ineffective. Although lessons were learned from the pilot, the Bank concurred with the Government decision to abandon the initiative and reallocate scarce resources to the most cost-effective components of the Project.

Objective 4: To strengthen the management capability of the MSPBS to decentralize the delivery of health services and to carry out Project implementation.

Overall, this objective was not achieved as had originally been planned for. The Project was able to support the functions of staff training, supervision of service delivery, and the planning and implementation of local IEC programs on the basis of statistical information at the regional level which strengthened the management of health regions and districts in the Project area and established the basis for systematic community outreach. However, with the severe scarcity of counterpart funds during the first three years of Project implementation in addition to the administration’s rejection of all advances towards fiscal decentralization, the Project focused only on strengthening those technical skills that did not need

- 16 -

significant additional resources. Decentralization activities – and their impact – supported by the Project were hence significantly more limited than had originally been planned for.

- 17 -

4.2 Outputs by components:During its implementation the Project achieved the following:

(i) construction and equipment of two new district hospitals;

(ii) rehabilitation and equipment of obstetric and pediatric services in five regional and districts hospitals;

(iii) rehabilitation and equipment of 42 health centers and heath post;

(iv) construction and equipment of the national cold chain for immunization composed of a national and three regional depots, and mobile fridges;

(v) equipment and personnel training for five decentralized blood banks;

(vi) purchase of pharmaceuticals for basic mother and child conditions;

(vii) launching, training and supervision of the maternal deaths surveillance system;

(viii) training of health staff on obstetric emergency care, and integrated management of childhood illnesses (IMCI);

(ix) recurrent costs for immunization campaigns;

(x) training and recurrent costs for community-based behavior-change campaigns on prenatal care, blood donation and immunization; and

(xi) studies on diagnostic and proposal of a national blood bank network, the effectiveness, efficiency and quality of health services regarding the basic eighth obstetric and one neonatal functions (8FOE &1N), the implementation of the IMCI strategy, and the knowledge attitudes and practices of the population on prenatal care, blood donation, and immunization, before and after the communication campaigns.

The Project increased the effectiveness, efficiency and quality of the 8FOE&1N health services in the Project areas; improved the detection and recording of maternal mortality; contributed to the improvement of vaccination coverage, increased the implementation of the IMCI strategy particularly among auxiliary nurses that see 80 percent of the children; and launched a movement for blood donation.

Component A- Maternal and Child Health Care. (US$19.0 million equivalent or 79 percent of total Project cost): Moderately Satisfactory

The outcomes of this component are described below by subcomponent with all reported activities pertaining to the second phase of the Project (after restructuring in 2001). During the first years of the Project (from 1996 to 2000), the Project had initiated the rehabilitation of a few health centers and posts only.

A.1 Rehabilitation and Maintenance of Infrastructure and Equipment (US$15.6 million): This subcomponent financed:

- 18 -

• the construction, expansion and/or rehabilitation of and equipment for nine regional and district hospitals, four of which are new facilities;

• the rehabilitation of and equipment for 15 health centers, and 22 health posts;

• the installation of a radio-communications network in 138 health facilities in the project area (this investment, which was not originally planned, effectively expedited referrals, thus significantly improving the performance of this function in Northeastern Paraguay);

• repairs of 40 ambulances and the procurement of four boats to operate as ambulances serving communities along the Paraguay river, and

• support for the extended immunization program (PAI) through the installation of refrigeration facilities in all health establishments in the project area, and in the construction of a central and three regional vaccine warehouses.

All civil works were completed and are in operation, except for regional warehouses for vaccines which, in some cases, are not yet operating. A sample of 14 infrastructure subProjects shows that construction took from 170 to 330 days per facility, averaging 265 days (see Annex 1, Table A.3). According to supervision reports the quality of construction is deemed acceptable and in accordance with the specifications. Implementation of the architectural and engineering design for the new hospitals were contracted to a firm in Spain which caused important problems for program implementation as most of the designs required extensive modifications. Such modifications were time consuming since the firm did not have any local representation.

Medical and industrial equipment was procured during the 2000-2001 period and was subsequently delivered and used. A few of the furniture items and equipment did not match the needs of health facilities, and a database was created to facilitate their reallocation by the MSPBS. Procedures to control equipment guarantees are in place and all directors received a complete inventory of items delivered to their respective facilities. The main concern regarding this component is the possibility of insufficient budgetary allocations to finance recurrent maintenance expenditures associated with the infrastructure and equipment financed by the Project. To minimize this risk, the Project created a maintenance fund to finance basic recurrent expenditures.

A.2 Pharmaceuticals and Supplies (US$2.3 million): The procurement and distribution of drugs, laboratory materials and medical supplies was carried out as planned and was accompanied by training in the use of the items acquired. Adjustments were required with respect to the procurement of contraceptives planned for the Project. Since USAID was already financing all contraceptives required in Paraguay, this item was dropped from the basic list of drugs to be procured under the Project. Another modification to the original plan was the non-implementation of community pharmacies, which were expected to be relatively self-sufficient through the introduction of cost-recovery. As mentioned previously, this idea was tested, but could not be carried out due to impossibility of maintaining a revolving fund at the pharmacies after a Ministerial Resolution in 1999 mandated free drugs for mothers and children under age five. Although the Project contributed to reduce shortages of drugs and supplies in the health facilities of six departments, shortages persist countrywide indicating that this is still a vulnerable area deserving further attention.

A.3 Information, Education and Communication (IEC) (US$0.9 million): This subcomponent financed a baseline survey to determine the prevailing health care attitudes and knowledge in local communities (“Conocimientos, Actitudes y Prácticas en Vacunación, Donación Voluntaria de Sangre y Riesgo del Embarazo, Parto y Puerperio”, May 2004). Based on this information, community outreach models were

- 19 -

developed for the dissemination of information about reproductive risk, blood donations and immunizations. Pilot activities with youth were successfully carried out in three Project departments (Concepción, Alto Paraná and San Pedro).

Among the critical activities completed with community participation, the strengthening of the Epidemiologic Vigilance of Health and Maternal Mortality (Comité Nacional de Vigilancia Epidemiológica de la Salud y la Mortalidad Materna, VESMM) is particularly important. This program was put in place to reduce under registration of maternal death and resulted in substantial improvement in the recording of those events. The strategy established local committees to investigate and report every maternal death. This initiative, tested in the Project area, has now been expanded throughout the country.

Other specific investments made include: (a) establishment of central, regional and local IEC units; (b) leadership training for community youth; (c) communications and social participation campaigns focused on immunizations, blood donation and obstetric risk, together with an evaluation of the results of these campaigns; (d) design, printing and distribution of educational materials under the coordination of the MSPBS Directorate General of Health Promotion designed for community level dissemination and technical use; (e) design and production of a radio promotion campaign targeted to adolescents in three Project departments; (f) social communication promotional campaign covering immunizations, reproductive risk and voluntary blood donations, and (g) organization of community volunteers to help provide transportation to health care facilities for mothers ready to deliver their babies.

A.4 Training (US$0.2 million latest estimate): The revised Project supported training programs designed to strengthen national maternal-child care programs already adopted by the MSPBS. Specifically, the following activities were financed:

- 20 -

• training designed to implement the AIEPI child health care strategy in the project health facilities and evaluation of the effectiveness of the program; 1,156 persons were trained (137 physicians, 152 nurses, 789 auxiliary nurses and 78 voluntary community promoters) and, as a result, the level of utilization of the AIEPI protocol was raised from 9 percent (2001) to 58 percent of the staff in 2004;

• a thorough analysis of the performance of eight essential obstetric functions and one neonatal function in the project-supported health facilities provided information to target training for 1,400 obstetric and neonatal staff focused on emergency care and utilization of a technical manual developed by the project;

• development of instruments to improve blood transfusions at the regional level through rationalization of the services provided, creation of a promotion program to increase voluntary blood donations, and establishment of a rotating fund to guarantee the opportune supply of reactive and other essential items and related training in these areas;

• training on quality improvements in health care services was provided for 156 managerial staff;

• training of 1,400 health professionals on how to improve coordination with midwives and contribute to better community delivery practices; and

• professional training to improve the care of the newborn (Atención al Recién Nacido—ARENA) was carried out in the project area in collaboration with the National Paraguayan Pediatric Association, the National Medical School (UNA) and MSPBS specialized staff, using the strategy of training of trainers. This initiative contributed to marked improvements in the performance of neonatal health care, particularly in hospitals and health centers in the project departments.

Component B- Early Childhood Development Pilot ($0.2 million latest estimate or 1 percent of total Project cost): Moderately Unsatisfactory

The ECD Pilot, as it was implemented, was less than 8 percent of the original plan to create 40 CEBINFAs. As noted above, the experience of the first few CEBINFAs during the initial phase of Project implementation clearly indicated that the model was neither a viable one nor the most appropriate strategy to improve child health in Paraguay. Thus, the decision to discontinue investments under this component is considered appropriate but component’s relevance for the Project was hence also doubtful. The basic problem faced by the CEBINFAs was that the centers did not provide for the proper delivery of integrated health, nutrition and educational services to preschool children and did not address the critical high infant mortality rate observed during the neonatal period. The practical arrangements for the implementation of the centers relied on community organization and cost-recovery through user fees. These arrangements were unsuccessful in installing adequate community capacity to guarantee service sustainability.

The revised Project (since 2001) adopted a comprehensive child health care strategy. First, a new child health care protocol developed by the Pan-American Health Association (PAHO) was introduced for the integrated attention to the most common childhood diseases (AIEPI) and health personnel were trained in its application (see A.4 above). Second, a community outreach program was created to expand the coverage of the child immunization program (Programa Ampliado de Inmunizaciones—PAI) with assistance from PAHO and UNICEF. Training in the application of the AIEPI protocol resulted in an increase of over five times in the application of the integrated strategy by health personnel, a change from 9 percent in 2001 to 58 percent in 2004.

- 21 -

Component C- Support for Decentralization ($1.4 million latest estimate or 6 percent of total Project cost): Moderately Unsatisfactory

With the restructuring, the Project’s support for decentralization was reoriented towards strengthening basic material and child health care services. Specifically, the functions of staff training, supervision of service delivery, and the planning and implementation of local programs were decentralized to regional and local levels, substantially expanding the scope of these activities and contributing to improving community outreach efforts. This component financed: (a) training of regional trainers; (b) technical assistance to help regional directors plan the training and follow-up; (c) technical assistance and training on AIEPI; (d) technical assistance to plan and carry out vaccination campaigns; (e) recurrent cost of vaccination; (f) training of community specialists at the regional level to monitor, supervise, and carry out community health campaigns; and (g) minor goods for use at the regional level.

Component D- Project Administration, Monitoring and Evaluation ($3.4 million latest estimate or 14 percent of total Project cost): Moderately Unsatisfactory

Project administration improved only after country conditions stabilized and the new team, installed in 2000, was able to systematize financial management and procurement, put in place technical expertise to reorient the Project interventions, better link Project activities with the regular policies and program of the ministry, fully gain the confidence and collaboration of central and regional health authorities, and promote the full ownership of the Project within the ministry. The relatively high administrative cost of the Project is indicative of both the turmoil the Project passed through (especially from 1996 to 2000), as well as the frequent supervision needed for the civil works, the transport of Ministry’s personnel for vaccination campaigns and supervision, and the monitoring efforts of the technical team.

Since 2000, the PCU had satisfactory administrative and financial controls over Project activities. The positive Project achievements in the last years of implementation have shown the good potential in terms of technical capacity and also dedication of the PCU team given that they had not received training from the Bank on basic norms and procedures for Project implementation. However, the Project Operational Manual was not updated and a comprehensive computerized monitoring system was not established.

While the Project did carry out two monitoring surveys in 2001 and 2004, the monitoring and evaluation system of the Project was, overall, insufficient. The lack of an effective day-to-day monitoring system was largely due to Project design deficiencies (the original plan had been to use MSPBS’s institutional monitoring system) and there was no priority given to setting up a parallel system. When this problem became apparent, it was too late to establish a separate system. The above problems with judging the overall impact of the Project are in part due to such deficiencies.

4.3 Net Present Value/Economic rate of return:Not Applicable.

4.4 Financial rate of return:Not Applicable.

4.5 Institutional development impact:ModestThe institutional development impact of this Project is considered modest compared to its original objectives. While some progress was made in strengthening the administrative capacity of the MSPBS at

- 22 -

its central, regional and local levels, integration of the Project implementation unit (and, thereby, the very Project activities) as an integral part of the Ministry of Health did not succeed. Consequently, few staff have been hired into the Ministry after loan closing.

However, the Project introduced several innovations in managerial and technical practices that may contribute to sustaining the improvements made in maternal and child health care with regard to coverage, quality and efficiency of care as outlined above. These innovations include: (i) an improved referral system; (ii) the use of radio communications; (iii) the assessment of service performance standards through surveys; (iv) the redirection of training activities to critical areas where services need to be strengthened; (v) the introduction of community outreach and IEC techniques to improve knowledge of healthy practices in local communities; (vi) the improvement of the reporting of maternal and infant mortality; (vii) the introduction of maintenance plans for health facilities and a revolving fund to finance recurrent maintenance costs; and (viii) the Ministry being in the process of standardizing the physical specifications for health facilities. Such positive developments justify a modest rating with regard to institutional development impact.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:There were several external factors that affected Project implementation, including the impact of adverse economic factors in Paraguay’s neighboring countries at the end of the 1990s and the impact of the Argentina financial and economic collapse in 2001 and 2002. Further, probably part of the social unrest which all but paralyzed policy making from 1997 to 2000 was also beyond the immediate Government control and due to a complex set of historical, social and economic (both external and internal) factors.

5.2 Factors generally subject to government control:The largest part of the difficulties the Project faced during its implementation phase can be traced to government system weaknesses, including: (i) the instability of the Government cabinet, leading to four changes of ministers of health from 1998-2000; (ii) the local currency (Guaraní) suffered a cumulative devaluation of approximately 180 percent compared to the exchange rate in May 1996 when the Project was appraised, and this was a problem for the Guaraní-denominated local contracts, and the capacity of the government to provide sufficient counterpart funds; (iii) Loan disbursements were suspended during six months, from January through June 2003, in response to the country’s lack of capacity to honor its debts to IBRD, causing delays in the completion of several Project activities; (iv) lack of counterpart funds in the early years which basically halted Project implementation until 2001; (v) national health policies changed significantly after Project approval, undermining the implementation of important Project components, including the drug and decentralization activities; and (vi) the community outreach strategy introduced by the Project faced significant institutional resistance, particularly at regional level.

5.3 Factors generally subject to implementing agency control:During the first three years of Project implementation, the PCU team did not succeed in integrating Project activities into the structure and programs of the MSPBS, isolating the Project and not making proper use of the available institutional resources. In contrast, the decision to change the PCU team in 2000 and restructure the Project demonstrated a commitment on the part of the MSPBS to install strong leadership and salvage the Project.

5.4 Costs and financing:The Project was financed with two IBRD loans of US$10.9 million each—a currency pool loan (CPL 40860) and a single currency US Dollar denominated loan (SCL 40870), that represented 70 percent of the estimated total Project cost. Project costs were estimated based on 1996 prices with contingencies estimated at 13 percent of total baseline costs. Recurrent costs, including drugs and medical supplies, maintenance of

- 23 -

infrastructure and equipment, and operating costs for early childhood development centers (totaling US$12.0 million equivalent or 38.4 percent of estimated total Project cost), were to be financed by the Bank Loan on a declining basis. The estimated final total Project cost (including IBRD and Government financing) is US$24.0 million, or 77 percent of the US$31.2 million estimated at appraisal. The 23 percent reduction in total Project cost is largely attributable to the devaluation of the Guaraní, which implied that none of the contingency funds were needed. In addition, the cost structure of the Project changed with the reallocation of approximately US$2.0 million from Component B, and US$1.0 million for pharmaceuticals from Component A. As of Project completion (September 30, 2004) US$21.27 million of the loan was disbursed (98 percent of the loan amount) leaving an undisbursed balance of approximately US$350,000 that was cancelled.

Procurement was carried out centrally by the PCU and in total, the Project implemented 58 firm and 68 consultant contracts for an aggregate amount of US$18.67 million. The Project was audited by external auditors annually. A total of 12 audit reports were submitted and reviewed by the Bank, and only two of them indicated minor administrative issues such as filing problems or the lack of appropriate signatures. In the last available audit report (first half of 2004), the financial statements of the PCU were unqualified. In 2003 a detailed FMR inspection on the part of the Bank identified several issues that were promptly corrected.

6. Sustainability

6.1 Rationale for sustainability rating:Unlikely. At this point, we judge the sustainability of a number of important Project activities as questionable, albeit the integration of Project staff and specific programs into the line Ministry has been accelerated over the past year. The key concern – necessary to ensure the continuity of the improved efficiency and coverage in service delivery – relates to the maintenance of purchased equipment and the rehabilitated infrastructure. The currently very high share of personnel spending in overall health expenditures in Paraguay presents a serious problem for maintenance in the overall system, including the activities procured by the Project and the Ministry’s budget for infrastructure maintenance remains inadequate. Maintenance plans exist – and had been supported by the Project activities – but an overall system that ensures the maintenance of the installed capacity is not operational (the IDB and the Bank will support Government in the set-up of such system through future support). The availabilities of drugs and medical and laboratory supplies remains a bottleneck for service delivery given that both the current distribution system as well as the current user cost policy does not allow health centers to retain locally raised revenue but requires them to pass the collected fees to Ministry of Finance. On the positive side, transparent international bidding has reduced the price of essential drugs by over 60 percent over the last year and the IDB is supporting Government’s Essential Drugs National Policy through the development of a purchase and distribution system for drugs and medical materials.

6.2 Transition arrangement to regular operations:Independent of the above concerns regarding maintenance of installed infrastructure, equipment and the supply of inputs, beginning in 2004, transition of the Project to the Ministry is proceeding well. Project activities were increasingly integrated into the regular operations of the Ministry and the MSPBS Directors have assumed the responsibility for Project activities in their respective areas, as mandated by the Minister. The transfer of completed works and installed equipment from the PCU to the MSPBS Infrastructure and Equipment Directorate has been completed. All equipment guarantees provided by the suppliers are in order and maintenance contracts are in place. The MSPBS has prepared a maintenance and operation plan for all health facilities benefiting from the Project and is engaged in establishing a continuous in-service

- 24 -

staff training program through distance learning and other means.

7. Bank and Borrower Performance

Bank7.1 Lending:Unsatisfactory. As indicated in Section 3.5, the quality of Project design was unsatisfactory, particularly with respect to monitoring and evaluation arrangements, but also regarding the design of Project components and the institutional arrangements for Project implementation. This was the first health Project financed by the Bank in Paraguay and the lack of country experience on the part of the Bank team may account for the setting of too ambitious targets coupled with an overestimation of the implementation readiness of Government and a rather weak assessment of potential risks. All those issues negatively impacted Project design (while the Bank had not carried out a sector specific ESW as part of preparation, the team did use sector analysis from the Inter-American Development Bank).

7.2 Supervision:Satisfactory. [Given the mixed performance overtime, we would rate the Bank's supervision performance as moderately satisfactory if that option were available]. The Bank’s initial supervision effort was clearly unsatisfactory. Six task managers were involved in Project supervision. During the startup period from effectiveness to December 2000, there were four different task managers, generating considerable instability in the guidance provided to the Paraguayan Project team. This may be partially explained as part of the reaction of the Bank as a whole to the unstable environment for development lending in Paraguay during the period. However, Bank performance improved considerably during the last four years of implementation. The Bank decided not to cancel the undisbursed loan funds and to re-establish dialogue with the Ministry and Government in general. The Project was assigned two task managers, both with strong technical and operational profiles that fully responded to the Project needs. When in 2001 the Bank team recognized serious design and implementation issues, it worked closely with the MSPBS to refocus and revise the Project and amend it formally. During this phase of Project implementation, Bank supervision improved and could be considered satisfactory.

7.3 Overall Bank performance:Unsatisfactory. [We would rate the overall Bank performance as moderately unsatisfactory, if that option were available]. Overall, the Bank performance is rated as unsatisfactory, in spite of the satisfactory supervision efforts in the last years which were not enough to compensate for lending development performance and supervision performance until 2000.

Borrower7.4 Preparation:Unsatisfactory. The Government, through MSPBS, had relatively little ownership of the Project during the preparation phase. Little or no effort was made to establish actual coordination links between the PCU team and the regular departments of the MSPBS and health authorities in the Project area. This limited the quality of Project design and placed the Project in an isolated position that did not lead to a successful startup phase. As a result, Borrower preparation performance is considered unsatisfactory.

7.5 Government implementation performance:Unsatisfactory. [We would rate the government's implementation performance as moderately unsatisfactory if that option were available]. In spite of a highly unsatisfactory performance during the initial years of Project implementation, the Government made an important effort to rescue the Project and redirect Project activities during the second phase of Project implementation. Still, the ICR rates the government performance in carrying out the Project as, overall, unsatisfactory. Given the acute health

- 25 -

needs of Paraguay and the very high infant and mortality rates, it is regrettable that a long startup period of inefficient Project management delayed the achievement of Project benefits.

7.6 Implementing Agency:Satisfactory. [If we had the possibility, we would rate as moderately satisfactory]. As outlined above, the performance of the implementing agency before restructuring of the Project was clearly unsatisfactory as well. However, once the Project was revised, the MSPBS demonstrated effective support for the Project and actively took measures to expedite implementation, both within the MSPBS itself and in the Finance Ministry. The MSPBS authorities ensured transparent management procedures in the PCU even though in some cases, there were difficulties due to the general inexperience with Bank norms and procedures and there could have been more effective quality control instruments to help improve sector administration, and more support for community participation and outreach activities. Overall, hence, this ICR rates the implementing agency's performance over the entire Project duration as satisfactory, although moderately satisfactory- if that option were available- would be more appropiate.

7.7 Overall Borrower performance:Unsatisfactory. [If we had the possibility, we would rate as moderately unsatisfactory]. Taken the above ratings together, the overall performance of the borrower during preparation and implementation of the Project is rated as (moderately) unsatisfactory.

8. Lessons Learned

Several lessons can be learned from the experience of this Project. In particular, the following are especially important:

• Baseline data and a realistic set (and targets) of monitoring and evaluation indicators need to be defined at appraisal. As the experience with the Project shows, leaving such definition to actual Project implementation is highly risky. The planned baseline survey was not carried out, making not only the evaluation of the Project very difficult but also depriving Project management in the Government and the Bank of an important tool which could indicate that corrective action would need to be taken. Further, the specified target that a basic package of relevant health services would be delivered to the entire poor population in the Project areas was, in hindsight, a too ambitious target. Realistic target setting for investment Project (especially in sectors like health where many factors – many of which beyond health service delivery per se – influence outcomes,) is pivotal. Finally, the selection of indicators should be limited to a rather limited number of relevant and collectable list.

• Project ownership across different levels of the institution is key to ensure continuity of Project implementation and adequate level of implementation: The objectives and strategies of a social development Project succeed only in so far as they are fully supported. To achieve this, concerted efforts to engage key members of the Government, the executing agency and the targeted communities as actors in the Project need to be made from the earliest stages of Project preparation. This will also serve to align Project interventions to existing programs that are familiar to those actors. Even a well designed Project, if carried out in isolation, faces a high risk of not succeeding in reaching its objectives.

• Training and retraining strategies need to be part of a staff development system that considers not only the skill needs by level, but also the adequate methodologies to transmit said skills to different types and levels of personnel, including on the job-training. Any training needs to be followed-up by close supervision as part of a “mentoring/apprenticeship” approach. In terms of procedures, it is also important that there is continuous technical support and training to the members of the PCU from the Bank team, particularly in less developed countries. In Paraguay, the new PCU team (since 2000) did

- 26 -

not participate in Bank training and this made their work much more difficult. Yet, the frequent and well-staffed supervision missions carried out during the second phase of Project implementation helped the new team revise the Project and improve implementation significantly.

• Implementation arrangement need to be carefully tailored to institutional capacity of the client country and closely monitored during Project implementation. The importance of this rather basic lesson cannot be overstressed, as witnessed when analyzing the Paraguay Project at hand which was greatly impacted from a too broad and ambitious design, coupled with an institutional set-up that made the functional integration of the Project (and it’s administration unit) in the line structure of the Ministry difficult. Further, in settings where administrative and implementation capacity is weak, Project activities centered around action-driven implementation of service delivery and their subsequent scaling-up can be more effective than plans of fundamental institutional change. It is also important to constantly monitor the institutional arrangement in such environments during Project implementation and to adopt modifications, if necessary. This holds also for implementation arrangements such as counterpart financing requirements, for example, which – as was the case in Paraguay – might need to be quickly reviewed and changed in situations of crisis.

9. Partner Comments

(a) Borrower/implementing agency:Final comments from the Ministry of Health on behalf of the Borrower were received on June 14, 2005. A copy of the comments is included as an additional Annex 9.

(b) Cofinanciers: Not applicable

(c) Other partners (NGOs/private sector): Not applicable

10. Additional Information

Notes:

[1] The MSPBS has divided the country into health regions based on the departmental structure; the Project focused on six underserved departments.

[2] These include: Surgery, Anesthesia, Control of Complicated Deliveries, Family Planning, Clinical Vigilance During Labor, Pre-and Post-Natal Care, Newborn Care and Referral.

[3] The 8FOE&1N study observed and surveyed all the health facilities in the six regions. Team composition and supervisors were virtually identical.

[4] Specifically the survey measured whether pregnant women had their blood pressure taken and their anti-tetanus vaccination in accordance with their visits.

[5] Given the time available, the MSPBS prioritized the IEC activities in these three departments, as being representative of the universe of the Project areas.

[6] In recent years, poverty increased in Paraguay and particularly in the Project areas. The results presented in this section should be considered with this in mind as it may have had a negative impact on access to basic services by the poor. Moreover, given the unreliability of the institutional data it is very difficult to properly assess service

- 27 -

coverage.

ICR Team:Montserrat Meiro-Lorenzo, ICR Task Team Leader, LCSHHJesko S. Hentschel, Country Sector Leader , LCSHDMartha P. Vargas, Team Assistant, LCSHDNatalia Moncada, Program Assistant, LCSHD

Comments received from:Suzana Campos Abbott, Lead Operations Officer, LCSHDEvangeline Javier, Sector Director, LCSHDCristian C. Baeza, Acting Sector Leader/ Sr. Health Specialist, LCSHHPeter M. Hansen, Country Manager, LCCPYFelipe Saez, Lead Country Officer, LCC7C

- 28 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Evolution of Project indicators and sources of data.

When the Project was first designed the team presented 84 indicators in the SAR (53 output and 31 outcome indicators) which proved difficult to monitor and use in evaluating Project outputs and results. In August 1999, as the Project seemed to start picking up, the Borrower and the Bank agreed to review the numerous indicators and reduce them to 10 (6 health outcome indicators and 4 output indicators). The following were the chosen indicators:

• Reduction in the number of reported cases of induced abortion in regional hospitals.

• Reduction in perinatal mortality rate

• Reduction in infant mortality due to obstetric causes by department.

• Women receiving pre-natal care who also receive full immunization against tetanus for the country and by department as % of total.

• Increase in the % of patients in regional hospitals that were referred by lower-complexity health units.

• Coverage of children less than one year of age in regional hospitals, by department.

• Coverage of children less than 5 years of age in regional hospitals, by department.

• Auxiliary nurses trained in Project Area (six departments) by year for obstetrics and common childhood illnesses: (a) Obstetrics Nurses; (b) Childhood Illnesses Nurses

• Midwives trained by year.

• Health centers that have oral contraceptives.

However, by the end of 2001, as the Project was ready to be restructured, it was clear that these indicators were even more difficult to measure than the previous ones, and could only be measured through administrative data which was known to be unreliable, instead of through surveys as planned in the SAR. The Government and the team decided to revisit the indicators to select some key measures from the original list, using as the main criterion that they could be measured through the national survey, or through the 2001 eight obstetric and one neonatal function survey (8FOE&1N) which had been the bases for the restructuring. The one exception is maternal mortality, which has never been measured in Paraguay in recent years and the figures for which are based on reported deaths. Since reporting has improved, the tendency in this measure is upward.

The final list basically corresponded to the key impact indicators on page 73 of the SAR (Annex 8) and adaptations of the two impact indicators on infrastructure and equipment subcomponent in page 74. The latter two were expanded to correspond with the 8FOE&1N survey.

- 29 -

Table A.1. Key Performance Indicators

Indicators 1996 20041. Infant mortality rate whole country (base line 1996) 1/ 27** 29**2. Neonatal Mortality rate (base line 1996) 1/ 173. Maternal Mortality (baseline 1996, latest estimate 2002) . 2/ 101.2 182.14. Percentage of pregnant women that had at least one neonatal visit during their pregnancy. National. 3/

88.0% 94.2%

5. Percentage of pregnant women who had at least one prenatal visit during the first trimester of their pregnancy. National. 3/

54.0% 68.0%

6. Percentage of institutional deliveries. National . 3/ 56.3% 74.1%7. Percentage of newborns that had their neonatal controls. 3/ 79.0% 90.1% Urban Areas 93.1% 95.3% Rural areas 74.1% 83.4%8. Immunization coverage in children from 12-23 months. National. 3/ 49.8% 65.2% Urban 59.4% 65.8% Rural 42.8% 64.4%9. Percentage of health staff that applies the AIEPI (IMCI) strategy. Project areas4/.

49% 58%

10. Percentage of health facilities that have all necessary equipment functioning to perform deliveries according to their level of complexity, in Project areas (base line 2001)5/

36.8% 51.9%

** Differences are not statistically significant.1/ Data from the 1995-96 and the 2004 National Demography, Sexual and Reproductive Health Surveys.2/ Reported deaths from the Bio-statistics Department of the MSPBS, as reported in the PAHO Core Health Data website. Statistics maternal mortality have improved substantially thanks to the Project-supported maternal mortality vigilance committees. The biggest jump in maternal death reported was between 1999-2001 (114.4- 160.7) just as the PSM and UNICEF instituted said vigilance committees in the different parts of the country.3/ Data from the 1998 and the 2004 National Demography, Sexual and Reproductive Health Surveys.4/ Data from UNICEF and AIEPI Project Survey5/ Data from the 8 FOES and 1 N study.

It is important to signal that national child mortality rates in the ENDSSR refer to a 5 year period (March 1999-February 2004), and have a confidence interval of +/- 5% for the national level figures. Urban v.s. Rural mortality rates refer to a 10 year period (1994-2004), and have a confidence interval of +/- 10%.

The Project financed several studies during its implementation period:

- 30 -

Table A.2: Principal Studies Financed by the Project Completion Date

Name Main Findings and Lessons Learned

1997 Health Network Study Assisted in setting priorities for investment by department. 1998 Culture, Gender and

Fertility Helped to understand determinants of fertility in Paraguay.

2001 “Diagnóstico de las Ocho Funciones Obstétricas Esenciales y Una Neonatal”

The study covered 6 regional hospitals, 27 health centers, and 195 health outreach posts. The results of the study were useful to: (a) design a staff training plan addressing the weaknesses identified, particularly in the areas of AIEPI Red Obstetrica.

2001 “AIEPI baseline study” Established base-line values for MEPI program 2001 “Safe Blood Study” Design of a feasible and effective blood supply system for the six health

departments of the project, including: transfusion units, promotion of voluntary blood donors, and training of human resources.

2001 “Training Impact Study” by the MSPBS

This study shows that most health care providers (91%) do not use the integrated AIEPI approach to child health care, an estimated 21% of the establishments have equipment and supplies necessary for AIEPI practice, but only 3.8% have the necessary basic drugs. These results are difficult to interpret in the absence of baseline data.

2004 “Second Training Impact Study” by the MSPBS

Based on the study of 30% of the child health care personnel who received training, the study found that 58% applied the AIEPI protocol in their daily care of children—more than five times the number who did so in 2001. However, the scarcity of drugs and supplies still affects 65% of the establishments surveyed.

2004 “Diagnóstico final de las Ocho Funciones Obstétricas Esenciales y Una Neonatal”

It gave data about the improvements on efficiency and quality of care in all the Project areas health facilities alter Project implementation.

- 31 -

Table A.3: Paraguay Sample of Infrastructure Subprojects Original Revised/Estimated Total Contract Type1/

Start Date Completion Date Completion Date Execution Period Value (US$)

1 9-Aug-02 9-Feb-03 5-Mar-04 300 88,268 *ER 2 6-Aug-02 6-Feb-03 5-Mar-04 300 184,916 ER 3 1-Nov-02 1-Apr-03 4-Apr-03 complete 325 48,877 ER 4 16-Jul-02 16-Jan-03 5-Mar-04 180 155,462 ER 5 2-Sep-02 2-Feb-03 30-Jul-03 complete 330 49,466 ER 6 3-Oct-02 3-Mar-03 11-Sep-03 complete 300 49,433 ER 7 4-Oct-02 4-Mar-03 13-Aug-03 complete 300 49,833 *R 8 22-Oct-02 22-Mar-03 19-Dec-03 175 47,516 R 9 11-Nov-02 11-Apr-03 16-Jun-03 complete 170 49,271 R

10 13-Aug-02 13-Feb-03 10-Oct-03 complete 390 40,435 ER 11 1-Oct-02 1-Apr-03 17-Sep-03 330 63,524 ER 12 11-Dec-02 11-May-03 11-Sep-03 complete 265 48,630 ER 13 12-Dec-02 12-May-03 18-Sep-03 complete 265 49,103 New 14 8-Nov-02 8-Apr-03 13-Oct-03 complete 330 48,723 R

3960 973,457 Average subproject cost = USD $64,897 Average subproject completion = 264 days Source: Medina, Gabriel (2003). “Informe General: Misión de Supervisión y Evaluación de la Calidad y Progreso de Obras de Infraestructura—Noviembre 28 a Diciembre 9, 2003.” Asunción, Diciembre 10, 2003. *ER: Extensive rehabilitation ; **R: Modest rehabilitation

- 32 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$million equivalent) 1/

AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

ComponentComponent A: Maternal-Child Health Care 20.14 19 94.3% A.l Infrastructure and Equipment SubProjects 11.24 15.64 A.2 Pharmaceuticals and Supplies 6.65 2.33

A.3 Information, Education and Communication 1.19 0.94 A.4 Training 1.05 0.06

Component B: Early Childhood Development 2.12 0.16 7.54%B.1 Management Training 0.12 0.04B.2 Rehabilitation of Centers 0.20 0.05B.3 Training for Caregivers 0.12 0.002B.4 Operation of Centers 1.61 -----

Component C: Support for Decentralization 1.20 1.45 120.8%C.1 Management Training 0.12 0.015C.2 Information System Support 0.81 0.88C.3 Strengthening of Supervision 0.07 0.55C.4 Pilot Fund 0.20 -----

Component D: Project Administration, Monitoring and Evaluation

3.17 3.29 103.7%

PPF 0.45 0.18 40%

Total Baseline Cost 27.05Contingencies 4.17 0.00

Total Project Costs 31.22 24.07 77.09%

Total Financing Required 31.22 24.07 77.09%1/ Figures may not add up due to rounding.

- 33 -

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$million equivalent) 1/

Expenditure Category ICB 2/ NCB 2/ Other 3/ N.B.F. 2/ Total Cost

Works 6.3 0.4 6.7(4.5) (0.7) (5.2)

Goods 5.3 0.8 0.9 7.0(2.3) (0.7) (2.4) (5.4)

Vehicles 0.7 0.7(0.0) (0.0)

Pharmaceuticals and MedicalSupplies 1.7 0.5 2.3

(3.6) (0.2) (3.8)Training and Consulting 4.8 4.8

(1.1) (1.1)Maintenance of Equipment 0.2 0.2

(1.0) (1.0)

PPF Refinancing 0.2 0.2(0.4) (0.4)

Incremental Recurrent Costs 2.0 2.0(0.0) (0.0)

Total 7.0 7.1 7.0 2.7 23.9(5.9) (5.2) (5.9) (0.0) (17.0)

1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.

2/ /ICB = International Competitive Bidding; NCB = National Competitive Bidding; NBF = Not Bank Financed

3 //Includes three quotations for minor civil works (US$0.8 M); National shopping for goods (US$2.7 M); International shopping for goods (US$0.5 M); Limited International Bidding (LIB) (US$1.4 M); National shopping for pharmaceuticals and medical supplies (US$0.5 M); and services of contracted staff of the Project management office, training, technical assistance services, and incremental operating costs related to maintenance for equipment and information systems (US$3.3 M).

- 34 -

Allocation of the Proceeds of IBRD Loans 4086-PY and 4087-PY Original Allocation (US$million)

Loan 40860 Loan 40870 TOTAL 1. Works 2.370 3.329 5.699 2. Goods: 2-A Pharmaceuticals 1.790 1.000 2.790 2-B Equipment 1.735 3.200 4.935 2-C Furniture 0.085 0.030 0.115 2-D Other 0.760 0.180 0.940 3. Consultants’ Services 2.115 1.850 3.965 4. Training 0.520 0.450 0.970 5. Maintenance 0.485 200,000 0.685 6. Other Operating Costs 0.070 0.161 0.231 7. PPF 0.225 0 0.225 Unallocated 0.745 0.500 1.245

TOTAL 10.900 10.900 21.800

- 35 -

Allocation of the Proceeds of IBRD Loans 4086-PY and 4087-PY Latest Estimate (US$million) 1/

Categories

Loan 40860

Loan 40870

Total 2/

Latest Estimate as

Percentage of Appraisal

1. Works 3.201 3.160 6.71 111.6 2. Goods: 2-A Pharmaceuticals 0.846 0.972 1.818 139.2 2-B Equipment 3.440 3.431 6.871 139.2 2-C Furniture 0.015 0.027 0.042 36.5 2-D Other 0.111 1.387 1.498 159.4 3. Consultants’ Services 0.519 0.368 0.887 22.4 4. Training 0.060 0.368 0.428 44.1 5. Maintenance 0.054 0.112 0.166 24.2 6. Other Operating Costs 0.181 0.094 0.275 119.0 PPF SA 0.401 0.383 0.784 63.0

TOTAL 10.570 10.126 20.696 94.9

Available 0.033 0.774 0.807 3.7 1/ As of May 13, 2004. 2/ Taking into account the SA, a total of US$1.591 million remains to be documented, or the equivalent of 7.3% of the total amount of both loans.

- 36 -

Annex 3. Economic Costs and Benefits

Not applicable.

- 37 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Lending02/14/1994 3 TASK MANAGER (1);

CONSULTANTS (2)05/19/1994 3 TASK MANAGER (1);

CONSULTANTS (2)11/1/1994 3 TASK

MANAGER/ECONOMIST (1); ENVIRONMENTAL HEALTH SPECIALIST (1); HEALTH SPECIALIST (1)

03/27/1995 7 TASK MANAGER (1); CONSULTANTS (4); ECONOMIST (1); PUBLIC HEALTH SPECIALIST (1)

04/22/1995 1 REPRODUCTIVE HEALTH SERVICES SPECIALIST (1)

07/30/1995 6 TASK MANAGER (1); HEALTH SPECIALIST (1); CONSULTANTS (4)

09/12/1995 4 TASK MANAGER (1); HEALTH SPECIALIST (1); CONSULTANTS (2)

11/9/1995 2 TASK MANAGER (1); ECONOMIST (1)

01/15/1996 6 TASK MANAGER (1); ECONOMIST (1); CONSULTANTS (2); IMPLEMENTATION ARRANGEMENTS SPECIALIST (1); RESIDENT MISSION STAFF (1)

05/20/1996 2 TASK MANAGER (1); CONSULTANT (1)

Supervision

10/24/1997 4 TASK MANAGER (1); PROCUREMENT SPECIALIST (1); CONSULTANT (1); TASK ASSISTANT (1)

S S

05/22/1998 6 TASK MANAGER (1); ECONOMIST (1); PROCUREMENT SPECIALIST (1); PROCUREMENT ANALYST (1); TELECOMMUNICATIONS

S S

- 38 -

SPE (1); TASK ASSISTANT (1)12/08/1998 3 MISSION LEADER (1);

MID-WIFE NURSE (CONS.) (1); ENGINEER (CONS.) (1)

S S

08/05/1999 1 TASK MANAGER (1) HU U12/08/1999 6 TEAM LEADER (1);

FINANCIAL MGT./ORG. (1); PUBLIC HEALTH (1); PROCUREMENT (1); INMUNIZATIONS (PAHO) (1); SECTOR LEADER (1)

U U

02/11/2000 7 TEAM LEADER (1); PUBLIC HEALTH (2); FINANCIAL MANAGEMENT (1); PROCUREMENT (1); HEALTH INVESTMENTS (1); IMMUNIZATIONS (PAHO) (1)

U U

06/09/2000 6 TEAM LEADER (1); CONSULTANT (4); PAHO (1)

S U

09/15/2000 2 TASK MANAGER/HEALTH SP (1); PROCUREMENT SPECIALIST (1)

S U

03/30/2001 2 TASK MANAGER (1); VACCINES SPECIALIST (1)

S S

05/25/2001 9 TASK MANAGER (1); HEALTH ECONOMIST (1);PROCUREMENT SPECIALIST (1); HEALTH CONSULTANT (1); DISBURSMENTS ANALYST (1);SOCIAL COMUNICATIONS (1); ENGINEER (1); FM SPECIALIST (1); TEAM LEADER (1)

S S

10/25/2001 6 TASK MANAGER (1); DISBURSEMENT SPECIALIST (1) HEALTH ECONOMIST (1);PROCUREMENT SPECIALIST (1); ENGINEER (1);MD CONSULTANT

S S

01/18/2002MID TERM

4 TASK MANAGER (1);PROCUREMENT SPECIALIST (1); IEC SPECIALIST (1); MD CONSULTANT (1)

S S

04/30/2002 4 TASK MANAGER (1); PUBLIC HEALTH CONSULTA (1); FINANACIAL MANAGEMENT (1); COMUNICATION SPECIALIST (1)

S S

- 39 -

09/26/2002 4 TASK TEAM LEADER (1); CONSULTANT(1);COMUNICATION SPECIALIST (1);ARCHITECT CONSULTANT (1)

S S

05/31/2003 3 TASK TEAM LEADER (1); CONSULTANT (1); CONSULTANT-ARCHITECT (1)

S S

09/12/2003 5 TASK TEAM LEADER (1);FM SPECIALIST (1); CONSULTANT-ARCHITECT (1); ECONOMIST (1); EDU. SPECIALIST (1)

S S

12/15/2003 4 TASK TEAM LEADER (1);CONSULTANT-ARCHITECT (1); HEALTH SPECIALIST; ENVIRONMENT SPECIALIST

S S

05/28/2004 6 TASK TEAM LEADER (1); SECTOR MANAGER- LCSHH (1); COUNTRY SECTOR LEADER (1); HEALTH SPECIALIST (1); FINANCIAL MGMT.ANLST. (1); CONSULTANT ENVIR+S.DEV (1)

S U

ICR06/25/2004 4 TASK TEAM LEADER

(1);FM SPECIALIST (1);PROCUREMENT SPECIALIST (1); SOCIAL SECTOR CONSULTANT/ICR (1)

S U

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Lending 0 265Supervision 96.23 728ICR 5.53 44Total 106.76 1,037

*These figures are based in the current information found in SAP. The figures for identification, preparation, appraisal and negotiations have been estimated and added under one category (Lending). Data for FY95 until FY97 is provided by total amounts without disaggregating by number of staff weeks and Project cycle stage.

- 40 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 41 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 42 -

Annex 7. List of Supporting Documents

García Calvente, Dra. María del Mar. 2001. “Consultoría Internacional para el Acompañamiento del Diagnóstico de las Ocho Funciones Obstétricas Esenciales y una Neonatal en las Regiones Sanitarias del PSM: Informe Final.” Escuela Andaluza de Salud Pública, Granada, España.

Medina, Gabriel. 2003. “Informe Final sobre el Estado de las 19 Obras Pendientes de Terminación Relacionadas en la Anterior Misión de Junio, 2003.” Asunción, Diciembre 10, 3002.

World Bank. 1996. “Staff Appraisal Report, Paraguay, Maternal Health and Child Development Project.” Report No. 15610-PA. Washington, DC, August 13, 1996.

_________. 1966, “Loan Agreement (Loan Number 4086-PA).” Washington, DC, September 29, 1996.

_________.1966. “Loan Agreement (Loan Number 4087-PA).” Washington, DC, September 29, 1996.

_________.2001. “Ayuda Memoria Misión del 21 de mayo al 25 de mayo de 2001.” Asunción, Paraguay.

_________. 2001. “Ayuda Memoria Misión del 22 al 25 de octubre de 2001.” Asunción, Paraguay.

_________. 2002. “Ayuda Memoria Misión del 6 al 18 de enero de 2002.” Asunción, Paraguay. (Midterm Review)

_________. 2002. “Ayuda Memoria Misión del 20 al 30 de abril de 2002.” Asunción, Paraguay.

_________. 2003. “Ayuda Memoria Misión del 4 al 12 de septiembre de 2003.” Asunción, Paraguay.

_________. 2003. “Ayuda Memoria Misión del 29 de noviembre al 12 de diciembre de 2003.” Asunción, Paraguay.

_________. 2004. “Ayuda Memoria Misión del 27 de marzo al 2 de abril de 2004.” Asunción, Paraguay.

- 43 -

Annex 8. Summary of Post Project Evaluation of Coverage, Quality and Efficiency in the Delivery of Basic Maternal and Child Health Care Services in the Project Area.

Methodology

As part of the Project revision in 2001, and in order to establish a baseline and be able to measure improvements (given poor institutional data), the MSPBS carried out a baseline survey of the performance of eight essential obstetric and one neonatal functions in the health facilities of the six Project departments. A follow-up outcome survey was carried out in 2004. These surveys consisted of fieldwork that covered all public health establishments in the Project area, or a total of 237 in 2001 and 335 in 2004. In each facility, approximately 15 to 22 health professionals responded to a questionnaire appropriate for the level of service performed in the facility. Both essential and complementary procedures for each function were examined in the surveys. The quality of these surveys is considered excellent in terms of reliability and comprehensiveness.

The original survey in 2001 (baseline) included only the Project-supported departments. However, in 2004, for comparative purposes, the MSPBS decided to include two neighboring non-Project departments (Caaguazú and Caazapá) that were known to have witnessed significant improvements in general indicators from 1992-94. Although it is clear that the two new departments would not really be comparable, it was decided to measure the situation to have some sense of the general trends in the rest of the country during the years of Project implementation (except Central and Asuncion, which have their particular dynamics). The two non-Project departments were selected based on their general comparability to the six Project departments, although they are known to have a lower level of poverty and better services than the Project-supported areas. Both non-Project departments received support through an Inter-American Development Bank (IDB) loan during the same time period. Caazapá was also supported beginning in 2002 by PAHO, through technical assistance to improve personnel skills and a public insurance Project.

In addition to assess infant/child health outcomes, the PCU carried out a baseline, an intermediate and a final assessment (after training and supervision) of the implementation of the AIEPI in the Project areas. UNICEF also carried a global assessment of the implementation of AIEPI in the entire country. Finally, the MSPBS’s National Direction of Social Communication carried out Knowledge, Attitude and Perception (KAP) studies before and after a few months of the implementation of the Project-financed IEC activities.

Overall Findings

The 2004 outcome survey used prenatal care and institutional deliveries as a proxy for health services coverage (the 2001 survey did not include coverage data). Unfortunately, the statistics provided by the MSPSB Department of Biostatistics are based on incomplete and tardy reporting by departments and facilities, and have a large margin of error [6]. For example, the MSPSB estimates that 2002 births and deaths are respectively 48 and 36 percent under-registered. Data on the use of health facilities is similarly unreliable.

Considering these data limitations, institutional deliveries in Project departments increased by 3.8 percent compared to 2.3 percent in the rest of the country and 2 percent in the two neighboring departments, thereby helping to bridge the gap between the Project area and the country as a whole (see Table 1). In addition, the capacity to carry out at-risk institutional deliveries increased in the Project area by 8.7 percent

- 44 -

from 2001 to 2003.

Table 1. Number and Percentage of Institutional Deliveries in Project and Non-project Departments 2001 2003 Departments NVR Institutional % NVR Institutional % Concepción 3398 2409 70.9 3543 2547 71.9 San Pedro 3745 2692 71.9 4046 3019 74.6 Alto Paraná 8791 6394 72.7 8109 6123 75.5 Amambay 1471 1311 89.1 1263 1128 89.3 Canindeyu 1746 1143 65.5 1439 1100 76.4 Alto Paraguay 119 93 78.2 160 128 80.0 Project Departments 19270 14042 72.9 18560 14045 75.7 Project at risk deliveries/total 18.3 19.9 Paraguay TOTAL 83919 71736 85.5 83000 72616 87.5 Source: Dept. Bioestadistica, MSPBS (2004). Note: Data for 2003 are provisory; NVR=registered live births

Since the 2001 baseline survey did not collect coverage data on prenatal care for pregnant women, it is not possible to assess outcomes in the Project areas. However, the 2004 survey did show that the quality of neonatal visits was very good as a substantially high percentage of women were vaccinated and had their blood pressure measured (see section on improvements in quality below). Yet when compared to non-Project departments, not as many women in the Project-supported departments had more than four prenatal consultations (see Table 2). However, it should be noted that international standards indicate that the number of visits needed to impact maternal and neonatal mortality is three.

Table 2: Number of Prenatal Consultations by Region and Type of Facility 2004 Number of prenatal consultations as % of total pregnant women served

3- 4 5 to 9 Project Departments Hospitals 65.0 35.0 Health Centers 57.8 42.2 Health Posts 59.3 40.7 Total Project Departments 62.1 37.9 Non-Project Departments (*) 59.2 40.8

Source: op.cit. (May 2004). (*) Ca’aguazy and Ca’azapa

Regarding health care coverage for children and infants, institutional records on child health care at the regional hospitals in the Project area show improvements in coverage from 55.1 percent in 1997 to 61.5 percent in 2004 (although given the quality of data these figures, should be taken with caution).

Vaccination coverage in the Project areas and in the country improved during the Project period mainly due to the Project-financed establishment of the cold chain and the constant support for Project-financed vaccination activities.

- 45 -

Neonatal mortality declined 11 percent between 1997-2003 in the Project departments. Mortality rates of children under age five also declined in the Project departments (MSPBS institutional data).

Improvements in Quality

Improvements in quality were easier to assess because both the eight FOEs&1N surveys and the AIEPI assessment reviewed quality and efficiency parameters. In both studies, quality was measure by the presence and functionality of the infrastructure and equipment, and presence of the required qualified personnel, needed to perform the identified functions and or tasks. It also evaluated the technical competence of the personnel through observation of the tasks performed.

The 8FOE&1N functions, the quality of services in the Project area hospitals improved by over 15 percent, on average achieving a 71.6 percent satisfactory performance compared to 62 percent in 2001 (see Table 3). The only exception is a slight decline in the score for the referral function, which in fact is good news because it reflects the improved capacity of the Project hospitals to treat obstetric and neonatal patients in situ.

Table 3: Performance of Essential Obstetric and Neonatal Functions by Type of Facility: 2001-2004

Project Departments Hospitals Health Centers Health Posts

Essential Functions

2004 Score

2001-04 % ?

(percent change)

2004 Score

2001-04 % ?

(percent change)

2004 Score

2001-04 % ?

(percent change)

1. Surgery 75.2 16.6 60.8 5.7 46.5 0.4 2. Anesthesia 65.7 4.3 41.4 -3.5 31.9 -34.4 3. Control of high obstetric

risk 74.9 8.4 49.6 17.6 30.3 13.5

4. Family planning 78.4 5.2 70.8 0.0 70.2 7.3 5. Clinical vigilance during

labor 79.9 22.7 62.6 7.6 43.1 -5.1

6. Blood transfusion 57.0 36.0 10.8 -46.8 19.5 101.0 7. Pre- and post-natal care 81.3 8.1 71.8 4.1 64.8 -5.1 8. Care for the newborn 67.9 22.8 71.8 24.7 42.6 -1.4 9. Referral 47.0 -6.2 37.5 7.1 31.0 9.5

Total Project Area 71.6 15.3 55.1 10.9 44.7 5.4 Source: MSPyBS (May 2004) “Seguimiento a la Linea de Base del 2001 sobre el Diagnóstico de las Ocho Funciones Obstétricas Esenciales y Una Neonatal en las Regiones Sanitarias del PSM, además de las Regiones Sanitarias de Caaguazy y Caazapa.” Note: The percent difference [?] is calculated as: (S2004-S2001)*100/S2001 where S is the performance score.

Although the improvements in health centers and posts seem more modest (55 percent and 45 percent satisfactory performance respectively, the figures suggest a real improvement in the quality of health services. Both the anesthetic and the blood transfusion functions have diminished in health centers. A reduction on the anesthetic capacity at this level is somewhat worrisome. However, if we discount Alto Paraguay from the list, the overall anesthetic function has been maintained. Regarding blood transfusion, the national health policy discourages blood transfusion at this level, since there is not yet a functioning blood bank network and most transfusions are done arm to arm. Likewise, health posts have seen those

- 46 -

functions that should not have been performed at their level reduced (with the exception of blood transfusion, which needs to be monitored), and an increase in their essential function, family planning.

The 2004 survey on the quality of prenatal care shows good quality of care at all levels including health posts. The survey measured whether pregnant women had their blood pressure taken and their anti-tetanus vaccination in accordance with their visits. On average, the departments supported under the Project came out ahead (see Graph 1).

Graph 1: Quality of Prenatal Care, 2004

Women that had their Blood Pressure Taken and Correctly Vaccinated in Antenatal Care (%)

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%90.0%

100.0%

San P

edro

Conce

pcion

Amab

ay

Canind

eyu

Alto Para

na

Alto P

aragu

ay

Caagu

azu

Caaza

pa

Blood pressure

Tetanus vaccine

Finally, UNICEF’s AIEPI assessment shows that in 2004, in the Project-supported departments, 58 percent of the health staff apply AIEPI in daily practice. This figure represents an overall improvement of 9 percent over the 2001 baseline. Most importantly, almost all the improvement was at the level of nursing assistants, which are responsible for 85 percent of pediatric outpatient pediatric visits. Overall, nurse assistants applied AIEPI in 20 percent more of their pediatric outpatient consultations compared to the baseline. When measuring the same indicator in the Project-supported departments (except Alto Paraguay), the increase in the use of AIEPI among nurse assistants reaches 30 percent on average. This increase was greater in the departments that started with lower levels of implementation of this national strategy. Empiric observation demonstrated that by using the AIEPI strategy, nurse assistants where able to diagnose and treat/or refer children better. Fewer nurses and doctors applied the strategy because they found it too basic. Thus, the application of AIEPI needs to be reviewed in terms of how best to target training and supervision.

- 47 -

Improvements in Efficiency

The level of efficiency of health facilities is measured as a function of: (a) the abilities demonstrated by the staff, (b) the physical plant, (c) the equipment, (d) the supplies and drugs, and (e) the availability of essential health personnel. Although Project interventions addressed most of these efficiency conditions, the availability of health personnel (and to a lesser extent that of drugs and supplies) are outside the control of the Project. Nevertheless, all efficiency conditions are better in 2004 than in 2001 and surpass those in non-Project departments, with the exception of availability of supplies and drugs (see Table 4). Hospital efficiency reached 71.6 percent in Project-supported departments compared to 71.1 percent in non-Project departments; Project health centers exceed by 20.3 percent those in non-Project departments, and health posts are practically identical (see Annex 1). It is interesting to note that actual performance scores in Project facilities in 2004 improved compared to 2001 at higher rates than efficiency conditions, suggesting high motivation on the part of the staff to deliver better services.

Table 4: Efficiency Conditions in all health facilitiesDimensions of Efficiency 2004 Score 2001-2004 % ·

1. Ability of health personnel 62.3 7.62. Physical plant 50.4 2.53. Equipment 51.9 15.14. Supplies 35.3 -6.95. Availability of health personnel 36.0 7.1

Overall Efficiency Project Departments Non-Project Departments

47.046.8

7.6(*)

Source: op.cit. (May 2004).(*) Ca’aguazy and Ca’azapa The 2001 survey was not carried out in non-Project departments.

Graph 2 highlights the improvements made to the physical plant and availability of medical equipment as the most important Project impact on efficiency levels of health facilities. A 7.1 percent increase in the availability of health personnel is a commendable achievement on the part of the MSPBS and reflects support to and ownership of the Project by the Ministry. However, the unreliable availability of drugs and supplies remains the most challenging problem facing the Project departments and possibly the entire health care system in Paraguay.

- 48 -

Graph 2: Changes in Efficiency Conditions in the Health Facility in the Project Departments 2001-2004

7.6

2.5

20.3

15.1

-6.9

7.1

-10

-5

0

5

10

15

20

25

TOTAL DESTREZAS PLANTA FISICA EQUIPAMIENTO SUMINISTROS RECURSOSHUMANOS

CONDICIONES ESENCIALES

Source: op.cit. (May 2004).

Efficiency conditions affect the performance of essential obstetric and neonatal functions in different ways, depending on the type of health facility. Detailed data on these relationships are shown in Section 10. Highlights of some of the essential functions are provided below.

• Surgical procedures in hospitals improved by 21.2 percent, particularly with respect to physical plant (31.8 percent), supplies (43.5 percent) and availability of health personnel (49.4 percent), all of which are presently comparable to non-project departments.

• Conditions for the performance of anesthesiology services in hospitals are 25.7 percent better than in non-project hospitals. Due to the MSPBS, the availability of anesthesiology staff in hospitals improved by 54.3 percent. Yet, efficiency losses were observed with respect to supplies and drugs, which declined by 84.1 percent.

• Blood transfusion conditions improved by 35.4 percent in hospitals, but deteriorated in health centers (-54.5 percent) and remained approximately the same in health posts (-1.9 percent). In hospitals, improvements in the availability of equipment (64.8 percent) and specialized health personnel (53.2 percent) largely account for the overall positive result.

• For newborn care, hospitals show a 40 percent efficiency gain, largely due to improvements to the physical plant (50.4 percent), availability of supplies (59.6 percent) and presence of specialized health personnel (54.6 percent). At health centers and posts, conditions did not change significantly.

• Pre- and post-natal care improved in hospitals by 26.2 percent due to better equipment and supplies. Conditions remained basically unchanged in health centers and posts.

Data by type of establishment are the most helpful to demonstrate Project impacts. As shown in Table 5, the performance of health centers improved the most, even though more needs to be done to improve the control of high obstetric risk health centers. Regarding hospitals, there has been an overall substantial improvement and the hospitals in the Project-departments are better off than those in the non-Project

- 49 -

departments. However, the care for newborn babies function for the Project hospitals is slightly below non-Project hospitals. In health posts, service quality is uneven: for some functions, performance is much better than average while services are unavailable to perform other essential functions. This unevenness creates vulnerability in the network as a whole. It should be noted however, that the substantial efforts made by the Project to improve referrals through deployment of radio communications equipment and staff training resulted in scores for this function that are significantly above those in non-Project departments. The information by department shows some variation, as indicated in Table 5 below.

Table 5: Performance of Essential Obstetric and Neonatal Functions by Department: 2001-2004

Hospitals Health Centers Health Posts Department 2004

Score 2001-04

% ? (percent change)

2004 Score

2001-04 % ?

(percent change)

2004 Score

2001-04 % ?

(percent change)

Alto Paraná 82.5 13.0 60.1 6.9 50.2 2.4 Alto Paraguay 43.6 -11.2 45.8 (*) 24.4 -30.7 Amambay 71.9 5.7 61.4 16.5 48.4 6.8 Canindeyu 78.3 39.1 66.2 18.0 44.1 10.0 Concepción 75.0 25.4 54.9 27.4 41.9 -6.1 San Pedro 67.4 1.8 50.1 23.7 44.4 10.2

Total Project Departments

71.6 15.3 55.1 10.9 44.7 5.7

Non-Project Departments (**)

71.1 45.8 45.2

Project Departments except Alto Paraguay

75.0

15.9

58.5

17.7

45.8

4.3

Source: op.cit. (May 2004). Note: The formula used to calculate the percent difference is: (Score-04 – Score-01)*100/Score 01. (*) In 2001, the health centers in Alto Paraguay were not surveyed, due to access problems. (**) Ca’aguazy and Ca’azapa. These departments were not surveyed in 2001.

This type of analysis is extremely helpful to guide further action by the MSPBS aimed at future improvements. Since the gaps are clearly identified, appropriate national strategies can be put in place to remedy inefficient operating conditions and more precise regional action plans can be developed.

The only department that does not show improvement is Alto Paraguay, most likely because the Project interventions in that department were completed near the end of the Project and their impact has not yet been felt. As mentioned above, this department has only one percent of the Project area’s population and has the most severe poverty and access problems.

Regarding the efficiency of the application of the AIEPI strategy, in the baseline study, only 21 percent of the health facilities had the adequate equipment, and medical and administrative material. Only 3.8 percent of the evaluated services had the required basic drugs required to treat the basic childhood illnesses according the AIEPI protocols. In the AIEPI evaluation in 2004, 100 percent of the health facilities had the basic equipment and materials necessary to carry out the AIEPI program and 35 percent of the facilities also had the pharmaceuticals to treat the basic illnesses.

- 50 -

The main conclusion that can be drawn from these data is that the Project had a positive impact on the quality of obstetric and neonatal care affecting 1.5 million people, although much remains to be done. In particular, the Project significantly improved the capacity of hospitals to perform obstetric surgery, provide blood transfusions, care for the newborn, and exercise proper vigilance of patients during labor. During the Project period (1997-2004), there were not significant improvements in infant and maternal mortality indicators, but this is largely due to an improvement in reporting in the entire country as supported by the Project.

In summary, overall outcomes are significant, particularly taking into account the extremely deficient conditions prior to the Project. In addition, the Project helped the country take big step forward in the health sector. Paraguay now has precise knowledge about what still needs to be done regarding the delivery of quality health care services to improve maternal and infant health status and lower maternal and infant mortality rates. The country has also gained the instruments to monitor progress properly as it continues its efforts towards this goal.

Detailed findings by Department

It is important to note that the number of health facilities in the Project departments is extremely uneven in relation to the population. While in the case of hospitals, this fact may be less relevant because their capacity varies widely by department, in the case of health centers and posts, which tend to have similar capacity, the lack of proportionality with respect to the demand is striking and may pose problems. For example, a typical health center serves an average of 1,800 people in Alto Paraguay and 18,000 people in Alto Paraná, and a typical health post serves an average of 14,500 in Alto Paraguay, and approximately 4,000 in Canindeyu, Concepción and San Pedro. Many of the health posts in the Project area are in fact dispensaries lacking minimum operating capacity. Thus, a general conclusion to be drawn from the available data is that a review of the regional network configuration is an important task that deserves the attention of the MSPBS. The following paragraphs discuss the main challenges or each department with respect to basic obstetric and neonatal care.

(a) Alto Paraná

The population of Alto Paraná accounts for 44.5 percent of the total population in the Project area. As a result of the Project, obstetric and neonatal care health services in this department have improved with respect to quality, efficiency and coverage. The two hospitals serving the department reached 82.5 percent performance score for all essential functions, showing only one setback related to the provision of blood transfusions (-8.6 percent) compared to 2001. The seven health centers in the department averaged a 60.1 percent performance score, showing an improvement of approximately 7 percent compared to conditions in 2001. In the 37 health posts of the department, the quality of care is still below minimum standards (50.2 percent), but has improved by 21 percent compared to conditions in 2001. In particular, health posts improved significantly with respect to care for high obstetric risk patients and in the provision of blood transfusions. For the obstetric network as a whole, this department is probably placing an excessive burden on its hospitals because the number of health centers and posts seems small compared to the population. This situation tends to increase the cost of health care because many patients who could be served at lower-complexity facilities go to the hospitals for basic care. Among the existing health centers and posts, improvements are still needed for the provision of anesthesiology services (38.8 percent) and control of high obstetric risk patients (46.7 percent), functions that show lower performance levels compared to 2001.

- 51 -

The positive results observed in this department reflect specific gains in efficiency resulting from Project interventions. Yet, it should be noted that in spite of an overall performance that is above average, Alto Paraguay still needs to improve efficiency conditions, particularly those showing scores below 50 percent (shown in bold in the following table).

- 52 -

Table 6: Alto Paraná Department: Efficiency conditions for the performance of essential Obstetric and Neonatal functions in all health establishments in 2004

Efficiency Conditions Essential Functions Total Staff

Ability Physical

Plant Equip. Drugs &

Supplies Available

Staff Surgery 65.3 79.3 69.7 71.8 44.1 53.4 Anesthesia

38.8 33.4 77.1 37.5 11.7 50.8

Control of high OB risk 46.7 80.7 31.9 32.0 22.0 67.1 Family planning 79.4 86.9 90.0 81.4 86.8 55.7 Clinical vigilance of labor 66.8 84.2 69.7 71.8 46.6 61.8 Blood transfusion 43.0 43.0 45.0 54.4 40.0 33.9 Pre- and post-natal care 70.3 91.4 82.8 79.1 56.3 37.2 Care for the newborn 57.8 74.4 70.5 60.5 41.5 42.1 Referrals 39.8 38.1 NA 53.1 38.8 29.1 Total 54.4 64.2 59.2 58.3 41.2 45.3

Source: op.cit. (May 2004).

The infrastructure and equipment investments made in Alto Paraná are described below by type of facility.

(a) Regional Hospital of Ciudad del Este. This hospital was rehabilitated and its functions restructured. Specifically, a maternity and infant care block was improved including areas for children’s therapy, emergency care, consulting rooms, and administrative and service areas. In addition, external areas were rehabilitated to facilitate access to the hospital. Unfortunately, due to less than efficient hospital administration, the surgery room built for at risk deliveries is not yet fully operational.

(b) District Hospital at Hernandarias. This is new hospital of 5,000 m2 and 68 beds. Facilities include an emergency block, X-Ray, blood bank, laboratory, consultation rooms, pediatric and adult internment with 10 and 21 beds respectively, service block including laundry, sterilization room, kitchen, cafeteria, storage space, morgue, and sanitary facilities. The hospital is provided with an air conditioner, medical gases, water tank, and dormitory for physicians on turn, energy generator, vaccination room, administrative areas, social services and education areas, central telephone exchange and radio room. The hospital is fully furnished and equipped. A regional warehouse for vaccines equipped with a freezer and a refrigerator compartment is the only part of the hospital that is not yet operational due to administrative problems at the central and regional level.

(c) District Hospital at Minga Guazú. This existing hospital was expanded by 987 m2 and the remaining 1089 m2 was restructured. New construction and equipment was provided for a surgery block, maternal-child consultation rooms, infirmary, emergency care, recuperation room and pre-delivery care, delivery room, sterilization facilities, administrative and general services area.

(d) Three health centers at Presidente Franco, Santa Rosa del Monday, and Ytaipyté were expanded and improved.

(e) Three health posts located in rural settlements were improved, provided with additional beds and

- 53 -

facilities for simple deliveries, primary care and consultations.

(b) Alto Paraguay

Alto Paraguay is the poorest department served by the Project, has the most difficult access conditions, the smallest health facility network, and the lowest performance rates for essential obstetric and neonatal functions in 2001 and 2004. Its population represents one percent of the total for the Project departments and is in dispersed settlements along the Paraguay River. Compounding these problems, Project activities in the department were the last to be completed with civil works finished in mid-October 2003 and equipment starting to be delivered only after November 2003. These various reasons explain why performance scores in 2004 are low for all essential functions in the network.

The one hospital serving Alto Paraguay is located in the riverside city of Fuerte Olimpo. This hospital shows improvements with respect to blood transfusions and pre- and post-natal control, but experienced setbacks in anesthesiology services and referrals compared to 2001, but overall, the hospital performance (43.6 percent) remains below acceptable levels. The two health centers in the department achieved a combined score of 45.8 percent with marked fluctuations with respect to specific functions. Improvements were observed in pre- and post-natal care, care for the newborn, and control of obstetric risk. For the other functions, scores vary from extremely deficient for blood transfusion and referrals, to medium-low for surgery, anesthesia, family planning, and clinical vigilance during labor. The average performance level of the eight health posts in the department is only 24.4 percent (see Table below). In general, the Project has had only a marginal impact on the efficiency conditions of health facilities in the departments, and the network remains extremely deficient. It should be noted, however, that the lowest score was recorded for availability of health personnel, which could not be corrected by the Project interventions.

Table 7: Alto Paraguay Department: Efficiency conditions for the performance of essential Obstetric and Neonatal functions in all health establishments in 2004

Efficiency Conditions

Essential Functions Total Staff Ability

Physical Plant

Equip. Drugs & Supplies

Available Staff

Surgery 38.6 63.0 31.4 44.1 37.2 17.3 Anesthesia 23.5 26.4 38.8 25.0 8.3 25.0 Control of high OB risk 23.9 60.6 13.6 16.6 23.1 5.6 Family planning 49.1 68.8 42.5 40.8 61.3 30.0 Clinical vigilance of labor 36.8 62.5 31.4 42.9 29.7 17.3 Blood transfusion 10.4 16.2 20.0 5.6 8.3 0.0 Pre- and post-natal care 48.9 52.5 61.3 56.0 56.3 20.8 Care for the newborn 26.8 51.3 29.4 20.0 12.5 20.7 Referrals 19.8 6.9 NA 34.3 23.6 14.4 Total scores in 2004 29.7 42.4 29.8 31.0 26.9 15.1 Source: op.cit. (May 2004).

Investments made to improve the health infrastructure in this department include:

(a) Rehabilitation and maintenance of the Fuerte Olimpo Regional Hospital, including an expansion of 85 m2, restructuring of 590 m2 and repairs to 35 m2. As a result, the hospital now has functional areas for external consultations, laboratories, X-ray, emergency care, maternity and neonatal care, administrative

- 54 -

and general services. A new telephone system was installed as well as sterilization facilities, kitchen, cafeteria, morgue and pharmacy. An energy generator was installed along with complete medical and industrial equipment that permit the full use of existing and new facilities.

(b) The Indigenous Dispensary at Fuerte Olimpio was repaired.

(c) Three health posts (Carmelo Peralta, Isla Margarita and Puerto Esperanza) were improved, with an expansion to accommodate three beds, reception area, infirmary, consulting rooms, and delivery room and general services areas. Two of these health posts are built on the river on stilts.

(c) Amambay

This department is served by one hospital, two health centers and 15 health posts. The quality of obstetric and neonatal health services in these facilities change only a little between 2001 and 2004. There were however improvements in the rest of the functions. Thus overall there was an improvement of approximately 16.5 percent is registered for health centers, followed by 8.5 percent for health posts and 5.7 percent for hospitals. Shortages of health personnel, drugs and supplies constitute the most serious efficiency gaps in this department, and these strongly influence the overall low performance score.

Table 8: Amambay Department: Efficiency Conditions for the performance of essential Obstetric and Neonatal functions in all health establishments in 2004

Efficiency Conditions Essential Functions Total Staff

Ability Physical

Plant Equip. Drugs &

Supplies Available

Staff Surgery 53.7 80.1 41.6 76.0 25.2 45.8 Anesthesia 34.6 42.4 71.9 35.0 8.2 25.0 Control of high OB risk 37.9 83.1 11.1 24.4 8.7 62.0 Family planning 82.7 88.7 90.0 91.1 78.3 67.8 Clinical vigilance of labor 59.1 78.8 64.2 76.0 30.9 45.8 Blood transfusion 37.4 44.1 54.2 44.4 31.3 12.5 Pre- and post-natal care 77.3 93.3 93.3 90.2 66.7 39.6 Care for the newborn 54.6 71.1 80.7 53.9 29.4 37.8 Referrals 41.9 33.7 NA 53.9 34.8 45.0 Total 51.4 53.8 50.1 50.0 27.0 36.6 Source: op.cit. (May 2004).

Infrastructure investments in the Amambay Department comprise:

(a) Regional Hospital at Pedro Juan Caballero: This hospital was built with funds from a donation from Japan. Project investment covered 1,500 m2 to house a dormitory for pregnant women, and high risk infants and lactating babies and their family caretakers, intended for families whose home is located far away from the hospital and therefore cannot commute often. This shelter was fully furnished and equipped. In addition, the existing areas of the hospital were provided with preventive maintenance services and a systematic maintenance program.

(b) The health center at Capital Bado was fully rehabilitated and provided with consultation rooms, laboratory, emergency room, surgery, pre-delivery and delivery rooms, general services areas and morgue,

- 55 -

all fully furnished and equipped.

(c) Three health posts were expanded and given preventive maintenance services (Nandeyara Puente, Aguará and San Roque). These health posts now have 3-4 beds each, waiting room, infirmary, delivery room, consultation rooms and general services areas.

(d) Canindeyu

This department stands out for having experienced substantial improvement in the delivery of essential obstetric and neonatal services compared to 2001. Better performance is observed at all types of facilities in the regional network. Overall, Canindeyu improved by 11.6 percent its performance, reflecting a 39.1 percent improvement in hospitals, 18 percent in health centers and 10 percent in health posts. While the total performance score is still relatively low at 56.7 percent and more needs to be done, it is evident that the Project had an important impact in this department. Project interventions contributed to establish the basis for future improvements in the efficiency conditions of the network. The main gaps are those observed with respect to availability of staff, drugs and supplies, which are largely outside the scope of the Project.

Table B.4: Canindeyu Department: Efficiency Conditions for the performance of essential Obstetric and Neonatal functions in all health establishments in 2004

Efficiency Conditions

Essential Functions Total Staff Ability

Physical Plant

Equip. Drugs & Supplies

Available Staff

Surgery 62.6 77.6 73.4 72.5 41.1 48.2 Anesthesia 52.9 61.1 83.9 70.0 8.9 58.3 Control of high OB risk 38.0 78.7 34.4 16.4 11.1 49.3 Family planning 77.5 86.2 87.5 74.5 80.0 62.8 Clinical vigilance of labor 64.4 78.6 73.4 72.5 49.1 48.2 Blood transfusion 39.3 28.9 75.0 58.3 37.7 2.8 Pre- and post-natal care 63.7 92.0 46.7 84.8 70.0 27.4 Care for the newborn 54.0 65.4 81.3 59.6 29.3 34.4 Referrals 46.1 44.1 NA 52.4 40.0 48.1 Total 51.5 61.3 59.6 56.6 38.7 38.4 Source: op.cit. (May 2004).

Eight health facilities in the Canindeyú Department directly benefited from Project investments. These are:

(a) Regional hospital at Salto del Guairá: this new 4,300 m2 hospital was provided with an emergency block, a laboratory and x-ray block, blood bank, consultation rooms, a surgery block with two operating rooms, a maternity and neonatal block with two delivery rooms, two surgery rooms, and 8 neonatal beds, a 10-bed pediatric internment block, a 24-bed adult internment block, and hospital services including laundry, sterilization room, morgue, maintenance shop, kitchen, cafeteria and sanitary services, air conditioner and supply of medical gases, water tank, energy generator, central telephone switchboard, and radio communications facilities. The new hospital also houses a regional vaccine warehouse fully equipped with two chambers for refrigeration and freezing, office space, deposit and machine room. The hospital has been fully furnished, and equipped with medical, laboratory and industrial equipment.

- 56 -

(b) District Hospital at Curuguaty: This new hospital consists of a 2,600 m2 building housing blocks for maternal/neonatal care, emergencies, maternity/deliveries, laboratory and x-ray, blood bank, consulting rooms, pediatric and adult internment totaling 11 and 12 bed respectively, an a service block consisting of laundry, kitchen, sterilization room, cafeteria, morgue, pharmacy, vaccination room, administrative services, storage room, social services and education room. The building is air conditioned and has built-in supply of medical gases, an energy generator for emergencies, a modern telephone exchange and radio communications room. All facilities were fully furnished and equipped with medical and industrial equipment. Adjacent to the hospital a regional vaccine warehouse was built and equipped with two vaccine chamber, one for refrigeration and another for freezing, offices for vaccination workers, a deposit and a machine room.

(c) Six health posts in the department were expanded and received preventive maintenance. These are located in Yasy Cañy, Manduara, Maracaná, Guyrá Kehá, Villa Ygatymí, and Ybyrárobaná. Improvements consisted of adding emergency beds, consulting rooms, infirmaries, delivery rooms, and other primary health care facilities including equipment to facilitate referrals.

(e) Concepción

The Project had a strong impact in improving the delivery of basic obstetric and neonatal health services in the one hospital and five health centers serving this department. The observed improvements were 25.4 percent and 27.4 percent, respectively. However, because performance at the 41 health posts in this department remains low (41.9 percent) and actually deteriorated between 2001 and 2004, the overall performance score shows only small improvement (5.3 percent). For this reason, the analysis of the efficiency conditions prevalent in the regional network in 2004 is shown below by type of facility

Table 9: Concepción Department: Efficiency Conditions for the performance of essential Obstetric and Neonatal functions in 2004

(a) Hospitals Efficiency Conditions

Essential Functions Total Staff Ability

Physical Plant

Equip. Drugs & Supplies

Available Staff

Surgery 78.0 82.1 81.6 88.9 53.3 68.2 Anesthesia 68.8 87.5 81.6 55.0 18.8 66.7 Control of high OB risk 84.1 100.0 0.0 60.0 66.7 100.0 Family planning 80.0 57.1 100.0 100.0 100.0 100.0 Clinical vigilance of labor 86.0 95.7 81.6 88.9 83.3 68.2 Blood transfusion 54.5 57.1 100.0 66.7 25.0 25.0 Pre- and post-natal care 91.7 88.9 100.0 100.0 100.0 83.3 Care for the newborn 64.1 75.0 88.9 64.6 36.4 13.3 Referrals 66.7 50.0 NA 100.0 66.7 50.0 Total 75.0 83.3 79.1 79.8 48.2 60.4

- 57 -

(b) Health Centers Surgery 60.4 75.2 57.9 71.6 26.9 20.0 Anesthesia 41.7 40.7 57.9 13.9 5.6 22.2 Control of high OB risk 51.7 73.9 9.1 21.2 21.8 34.1 Family planning 70.8 95.8 50.0 88.9 54.2 20.9 Clinical vigilance of labor 63.4 82.8 57.9 71.6 28.3 20.0 Blood transfusion 9.2 10.0 8.3 12.5 6.3 1.4 Pre- and post-natal care 69.9 84.4 66.7 83.3 NA 27.8 Care for the newborn 54.4 69.6 36.4 24.7 11.4 51.5 Referrals 50.5 42.9 NA 64.3 42.9 50.0 Total 54.9 68.5 53.9 60.2 23.0 28.6

(c) Health Posts Surgery 42.2 60.5 40.6 6.8 17.5 50.6 Anesthesia 24.4 NA NA NA 24.4 NA Control of high OB risk 31.0 62.6 16.3 12.2 0.0 49.6 Family planning 65.4 85.4 NA 46.3 59.8 30.5 Clinical vigilance of labor 40.2 47.3 40.6 46.8 16.6 50.6 Blood transfusion 10.4 19.5 NA NA 7.3 NA Pre- and post-natal care 60.7 76.0 69.5 57.1 31.7 4.9 Care for the newborn 40.6 47.2 40.6 19.5 11.4 NA Referrals 31.2 34.2 NA 25.6 46.3 27.6 Total 41.9 54.1 39.5 39.3 20.6 41.3 Source: op.cit. (May 2004).

In order to ensure the consolidation of the progress made in Concepción, it is important that the MSPBS make additional efforts to improve the availability of pharmaceuticals and supplies, and increase the health personnel allocated to the department, particularly in health centers and posts. The Project results in this department are a good example of what can be achieved and should encourage further efforts countrywide.

The Project made extensive investments in the Concepción Department, as follows:

(a) Regional Hospital at Concepción: This existing hospital was substantially expanded with a new 5,500 m2 building for 87 beds and new blocks for the care of emergencies, x-ray and laboratory, blood bank, surgery, maternity and neonatal care, pediatric and adult internment, general services (including laundry, sterilization room, maintenance workshop, and morgue), as well as a block for kitchen, storage, cafeteria, and staff sanitary services. The building is air conditioned and supplied with medical gases and ancillary services including a water tank, generator, and telephone switchboard and radio communications rooms. In addition, the old hospital building—which consists of 3,400 m2 containing 24-bed internment facilities for adults, consulting rooms, administrative services offices, dormitory for on-duty physicians, and sanitary services—was fully refurbished. The entire hospital was provided with medical, laboratory and industrial equipment that fully meets the needs of maternity-delivery rooms, neonatal care and infirmary services.

(b) Four health centers, located at Tbt-yaú, Horqueta, Valle, and Pinasco were expanded by an additional 43 beds, consulting and delivery rooms, neonatal and emergency care facilities, X-ray, laboratories, ondontogy facilities and administrative service areas.

(b) Four health posts in the department—including one in which the building was close to collapsing—were

- 58 -

improved with added bed capacity, infirmary, normal delivery rooms and consultation rooms. These posts are located in Oegyahi Kinam /arritutim /cryce /bekka /vusta abd /cakera Utacúa.

(f) San Pedro

With an estimated population of 352,018 inhabitants, San Pedro is the second most populous department in the Project area, accounting for 23.5 percent of the total, and its public health network is the largest, comprising 2 hospitals, 8 health centers and 72 health posts or a total of 82 facilities. The most salient impact of the Project in this department was a 23.7 percent improvement in the performance of health centers. However, the overall score for the network remains very low (48.9 percent) and practically did not change since 2001. As in the case of Concepción, the large number of health posts calls for analysis of regional conditions by type of facility. These data show that the largest deficits in hospitals relate to shortages of drugs, supplies and medical staff, although total hospital performance is relatively high at 67.4 percent. Health centers have generalized deficiencies except with respect to staff ability. In particular, the very low performance associated with blood transfusions deserves immediate attention. With respect to health posts, the overriding conclusion that can be drawn from these data is that a large number of posts should be closed or re-classified as dispensaries dedicated to pre- and post natal care and family planning, since the performance with respect to all other functions is below minimum standards. Given the current size of the network, it is unlikely that total performance scores can be raised in the near future without re-configuration.

Table 10: San Pedro Department: Efficiency Conditions for the performance of essential Obstetric and Neonatal functions in 2004

(a) Hospitals Efficiency Conditions

Essential Functions Total Staff Ability

Physical Plant

Equip. Drugs & Supplies

Available Staff

Surgery 78.8 98.2 72.4 83.3 75.0 34.1 Anesthesia 68.8 93.8 72.4 87.5 0.0 33.3 Control of high OB risk 76.7 90.0 37.5 45.0 50.0 75.0 Family planning 80.0 71.4 100.0 100.0 75.0 75.0 Clinical vigilance of labor 50.8 88.0 72.4 0.8 95.8 34.1 Blood transfusion 62.6 71.4 86.7 66.7 45.8 41.7 Pre- and post-natal care 82.2 84.4 100.0 95.0 100.0 46.7 Care for the newborn 60.9 70.5 58.3 70.3 34.1 7.0 Referrals 70.0 50.0 NA 87.5 66.7 75.0 Total 67.4 83.5 71.6 58.4 53.4 39.8

- 59 -

(b) Health Centers Surgery 56.0 72.2 52.9 60.7 33.3 21.0 Anesthesia 41.4 46.9 52.9 63 15.6 21.9 Control of high OB risk 43.1 67.8 5.0 13.3 5.0 23.8 Family planning 64.0 93.4 47.4 45.6 79.0 26.3 Clinical vigilance of labor 56.7 74.7 52.9 60.7 31.1 21.0 Blood transfusion 1.5 0.0 0.0 0.0 5.8 0.0 Pre- and post-natal care 66.4 78.3 47.4 89.5 NA 33.3 Care for the newborn 55.3 67.8 53.3 25.7 22.5 50.0 Referrals 30.2 17.3 NA 50.0 23.1 27.0 Total 50.1 64.3 49.6 49.5 24.8 26.1

(c) Health Posts Efficiency Conditions

Essential Functions Total Staff Ability

Physical Plant

Equip. Drugs & Supplies

Available Staff

Surgery 46.7 66.9 37.1 45.4 28.2 58.4 Anesthesia 40.3 NA NA NA 40.3 NA Control of high OB risk 32.9 62.0 18.5 15.6 1.4 51.8 Family planning 62.2 91.1 NA 68.1 18.1 31.3 Clinical vigilance of labor 42.4 51.1 37.1 45.4 20.8 58.4 Blood transfusion 16.7 26.4 NA NA 13.4 NA Pre- and post-natal care 66.4 82.5 68.8 61.7 62.5 2.8 Care for the newborn 40.5 46.2 37.1 23.6 17.6 NA Referrals 33.0 32.4 NA 22.2 36.1 47.7 Total 44.4 55.7 37.6 40.9 24.9 48.9 Source: op.cit. (May 2004).

Project investments in San Pedro Department health infrastructure benefited the Regional Hospital of San Pedro and several health centers and posts. Specifically:

(a) The San Pedro Regional Hospital was expanded by a new block (488 m2) and the refurbishing of 1,917 m2 of existing areas. New facilities built and equipped include a maternal and neonatal surgery rooms, new beds for children and adults, emergency care area, delivery and pre-delivery rooms, kitchen, cafeteria, laundry, morgue, general services area and parking for ambulances.

(b) The District Hospital at San Estanislao was expanded by 3,900 m2 to accommodate 66 new beds, two delivery rooms, emergency facilities, X-ray, a service block including laundry, kitchen, morgue, and pharmacy, and a regional vaccine warehouse.

(c) Seven health centers were improved, including those at Itacurubí del Rosario, General Aquino, Santa Rosa del Aguary, Choré, Capiibary, Nueva Germania and General Resquín. Improvements consisted of additional bed capacity (10 to 15 beds were added to each center), consulting rooms, laboratories, delivery rooms, pre-delivery care and neonatal rooms, emergency and surgery blocks, administrative and general services areas, odontology service areas, and X-rays.

(d) Three health posts were expanded, restructured and received preventive maintenance. These posts,

- 60 -

located at Tacuatí, Aguerito, and Tava Guaraní, were provided with 3-bed additional capacity, waiting room, infirmary, consulting rooms, delivery room, and other primary health care service facilities.

- 61 -

Additional Annex 9: Borrower's Contribution

Asunción, 9 de junio de 2005

SEÑOR PETER HANSEN, REPRESENTANTE RESIDENTE

BANCO MUNDIAL

ASUNCIÓN

Distinguido señor:

Con nuestros atentos saludos, hacemos referencia a su nota de fecha 25 de mayo del año en curso, por medio de a cual nos ha hecho llegar el Informe Final de Ejecución (IFE) del Proyecto de Salud Materna y Desarrollo Integral del Niño/a (PSM).

Al respecto, cumplimos en remitirle adjunto nuestro comentario acerca del referido documento.

Hallamos propicia la oportunidad para exteriorizar nuestro sincero reconocimiento al Señor Representante y colaboradores por a excelente predisposición e interés puestos de manifiesto en favor del pueblo paraguayo; y reiterarle al mismo tiempo la seguridad de nuestra más elevada consideración y estima,

DRA. MARIA TERESA LEON MENDAROMINISTRA

- 62 -

COMENTARIOS

En líneas generales del Informe Anual de Ejecución del Proyecto de Salud Materna y Desarrollo Integral del Niño (CPL-40860) (SCL 40070) por un monto de 21.8 millones a la República del Paraguay por el Banco Mundial. se puede evidenciar cuanto sigue:

• Debilidades en el diseño. Se resalta en el informe que no hubo una conexión clara entre todos los objetivos del Proyecto, algunas intervenciones, algunos resultados esperados y los indicadores.

• Debilidades y retrasos en la Gestión y Ejecución al inicio del Programa

• La no institucionalización y consecuente involucramiento de las instancias de condiicci6n superior tecnopolíticas y, la debida apropiación de las instancias ejecutoras del Ministerio de Salud Pública y Bienestar Social.

• No se estableció pautas claras sobre las actividades para establecer un sistema de monitoreo y al mismo tiempo se veía irracional contar con un número tan grande de indicadores (más de ochenta) de monitoreo e impacto.

• Rediseño del Proyecto. Si bien las actividades del Proyecto fueron reestructuradas en el 2001, los objetivos originales no fueron normalmente revisados.

La Reestructuración no alteró los componentes originales, salvo el componente B de desarrollo infantil temprano que se acordó cancelar y el redireccionamiento del componente C, Apoyo a la Descentralización.

• La evaluación de las actividades del Proyecto se realizó mediante un estudio de ocho funciones obstétricas y una neonatal (8FOE&1N), recomendándose el uso de los resultados de la Encuesta Nacional de Salud Reproductiva y Sexual como herramientas más apropiadas para monitorear y evaluar el Proyecto en lugar de las estadísticas institucionales. entendiéndose por éstas como no confiables.

Con esto se buscó alinear los objetivos y actividades del Proyecto con las políticas nacionales- reducir la complejidad institucional y tener a mediano plazo un impacto directo sobre la mortalidad materno infantil.

En síntesis:La evaluación del informe final califica al Proyecto de Salud Materna y Desarrollo Integral del Niño, como sigue:

• Calidad al inicio: Insatisfactorio• Resultado/logro del objetivo: Insatisfactorio• Impacto en el desarrollo institucional: Modesto• Sustentabilidad: Improbable• Desempeño del Banco y el Prestatario: Insatisfactorio

Resaltando como moderadamente Satisfactorios, la Supervisón y la Gestión ejecutora de la PCU en la Case posterior a la reestructuración del Proyecto. haciendo notar que las autoridades del Ministerio y la PCU aseguraron procedimientos de gestión transparentes.

- 63 -

Lecciones Aprendidas

Es necesario que los datos iniciales y un conjunto realista de indicadores de monitoreo y evaluación y metas sean definidos durante la evaluación inicial.

La institucionalización y apropiación de los Programas o Proyectos con financiación externa por los diferentes estamentos y niveles decisorios y operativos de Ia institución y continuidad de ejecución es clave para asegurar un adecuado nivel tanto como para la sostenibilidad de los mismos.

Es necesario que las estrategias de capacitaci6n y recapacitación sean parte de un sistema de desarrollo del personal que toma en cuenta no solo las habilidades necesarias por nivel sino también las metodologias adecuadas para transmitir estas habilidades a diferentes tipos y niveles de personal, incluyendo el entrenamiento en el lugar de trabajo.

Es necesario que los arreglos de la ejecución sean diseñados cuidadosamente, tomando en cuenta la capacidad institucional del cliente, y monitoreados durante la ejecución del Proyecto.

- 64 -

- 65 -