Document ofThe World Bank

FOR OFFICIAL USE ONLY

Report No: 22959

IMPLEMENTATION COMPLETION REPORT(CPL-35500; SCL-3550A; SCPD-3550S)

ONA

LOAN

IN THE AMOUNT OF US$ 93.5 MILLION

TO THE

REPUBLIC OF INDONESIA

FORA

THIRD COMMUNITY HEALTH AND NUTRITION PROJECT

12/19/2001

Human Development Sector UnitEast Asia & Pacific Region

This document has a restricted distribution and may be used by recipients only in the performance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective as of August 31, 2001)

Currency Unit = Indonesia Rupiah (IDR)IDR 1,000 = US$ 0.11

US$ 1 = IDR 9,150

FISCAL YEARApril I - March 31 till 2000

FY 2000 was 9 months, April 1 to December 31Same as calendar year from 2001 onwards

ABBREVIATIONS AND ACRONYMS

AMP Maternal and Perinatal Audit MOH Ministry of HealthBapelkes Center for Health personnel MOHA Ministry of Home Affairs

TrainingBAPPENAS Ministry of National Planning MONE Ministry of National EducationBidan C Village midwives recruited locally MSA Multi Stream Academy

in Irian Jaya and MalukuDGHE Directorate General for Higher NCB National Competitive Bidding

EducationDIP Project development budget NOL No Objection Letter

allocationFEPS Final Executive Project Summary NTT East Nusa Tenggara ProvinceGOI Government of Indonesia PDAs Program Development ActivitiesHIS Health Information System PKM MOH Center for Health

EducationKfW Kreditanstalt fur Wiederaufbau PMU Project Management UnitIBRD International Bank for Pusat Data MOH Center for Health

Reconstruction and Development InformationICB International Competitive Bidding PUSKESMAS Health CenterIDA Iron Deficiency Anemia QAG Quality Assurance GroupIDD Iodine Deficiency Disorders REPELITA Five year planning cycleIDR Indonesian Rupiah SAR Staff Appraisal ReportIMCI Integrated Management of SKRT, National Household Health,

Childhood Illness SUSENAS Expenditure SurveysIPB Institute of Agriculture in Bogor THP Third Health ProjectLSS Life Saving Skills UGM Gadjah Mada UniverisityMOEC Ministry of Education and Culture UNICEF United Nations Children's FundMOF Ministry of Finance WHO World Health Organization

Vice President: Jemal-ud-din Kassum, EAPVPCountry Manager/Director: Mark Baird, EACIF

Sector Manager/Director: Maureen Law, EASHDTask Team Leader/Task Manager: Samuel S. Lieberman, EASHD

FOR OFFICIAL USE ONLY

INDONESIAID-THIRD COMMUNITY HEALTH AND NUTRITION PROJECT

CONTENTS

Page No.i. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 34. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 206. Sustainability 227. Bank and Borrower Performance 238. Lessons Learned 259. Partner Comments 2710. Additional Information 28Annex 1. Key Performance Indicators/Log Frame Matrix 29Annex 2. Project Costs and Financing 30Annex 3. Economic Costs and Benefits 34Annex 4. Bank Inputs 35Annex 5. Ratings for Achievement of Objectives/Outputs of Components 37Annex 6. Ratings of Bank and Borrower Performnance 38Annex 7. List of Supporting Documents 39Annex 8. Borrower's Evaluation 40

This document has a restricted distribution and may be used by recipients only in theperformance of their official duties. Its contents may not otherwise be disclosed withoutWorld Bank authorization.

Project ID: P003914 Project Name: THIRD COMMUNITY HEALTHAND NUTRITION PROJECT

Team Leader: Samuel S. Lieberman TL Unit: EASHD

ICR Type: Core ICR Report Date: December 19, 2001

1. Project Data

Name: THIRD COMMUNITY HEALTH AND L/C/TF Number: CPL-35500;NUTRITION PROJECT SCL-3550A;

SCPD-3550SCountry/Department: INDONESIA Region: East Asia and Pacific

RegionSector/subsector: HN - Nutrition

KEY DATESOriginal Revised/Actual

PCD: 01/30/1987 Effective: 04/21/1993 07/30/1993Appraisal: 06/15/1992 MTR: 06/15/1996 06/291996Approval: 12/22/1992 Closing: 03/31/1999 03/31/2001

Borrower/lmplementing Agency: GOI/DEPT. OF HEALTHOther Partners: Kreditanstalt fuir Wiederaufbau (KfW)

STAFF Current At AppraisalVice President: Jemal-ud-din Kassum Gautam S. KajiCountry Manager: Mark Baird Nicholas HopeSector Manager: Maureen Law Clifford GilpinTeam Leader at ICR: Juliawati Untoro Susan A. StoutICR Primary Author: Juliawati Untoro; Samuel S.

Lieberman

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U-Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=HighlyUnlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: S S

Project at Risk at Any Time: No

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:

The original goal of the Third Community Health and Nutrition project (CHN3) was to raiseinfant, child, and matemal health status by improving the effectiveness of safe motherhood, child survival,nutrition, and health education programs in the provinces of Irian Jaya (now Papua), Maluku, NusaTenggara Timur (NTT), West Java, and Central Java. This objective was to be achieved by (i) buildingprovincial and district capacity to plan, implement and evaluate programs; and (ii) strengthening thecapacity of the central Ministry of Health (MOH) and selected higher education institutions under theMinistry of Education and Culture (MOEC) to support provincial efforts.

The project's objective, which was consistent with the goals of the Bank's Country AssistanceStrategy (CAS), responded to a critical program priority for the Borrower. This was to reduce existinghigh infant, child, and maternal mortality rates. Moreover, the project's emphasis on building provincialcapacity supported the Government of Indonesia's (GOI's) policy of gradually transferring responsibilitiesand resources to lower level administrative units. In this respect, the project built on the achievements ofthe then ongoing Third Health Project (THP, Ln 3042), in decentralizing some planning, budgeting, andimplementation-related activities in the provinces and districts of East Kalimantan and Nusa TenggaraBarat. CHN3's provincial focus also contributed to GOI's goal of promoting inter-regional equity.Selection of Irian Jaya, Maluku, and NTT was seen as leading to a more balanced distribution of sectoralresources, by working to close gaps between Java and Eastem Indonesia in availability of health facilitiesand staff.

3.2 Revised Objective:

The original objectives were not revised.

3.3 Original Components:

The project design responded to the above objectives through (i) a provincial service delivery andcapacity development component, and (ii) a central support services and capacity building component.

The provincial component comprised interventions through MOH's Safe Motherhood, ChildSurvival, Nutrition and Health Education programs. The emphasis on child and matemal survival was anunderstandable response to concerns and imperatives at the time. High national level estimates for theinfant, child, and matemal mortality rates served as a call-to-action. Health gains in some regions andprogram successes in related areas, e.g., family planning, suggested that progress could be made inIndonesia if the context was favorable. Policy specificity, i.e., tailoring interventions to local needs andconditions, was seen as critical for creating circumstances conducive to significant health status advances.

Two risks were identified during preparation and addressed in the project design. The first wasthe generally limited province level experience in preparing and implementing safe motherhood, nutrition,and child survival interventions. To address this risk, CHN3 added a second component to augment centralcapacity to help the project provinces to improve their planning, targeting and evaluation techniques.Specifically, the central component aimed to strengthen the Directorate General for Community Health, theCenter for Health Education and the Center for Health Information in regard to policy development,provision of technical assistance, organizing and monitoring training, and carrying out surveys andoperational research. The central component also supported staff and curriculum development, researchand training in MOH's Centers for Health Personnel and Manpower Development, and in research and

- 2 -

training in six institutions, five schools of public health and one specializing in nutrition, which were theresponsibility of the Directorate General for Higher Education (DGHE) in MOEC.

The second risk identified during preparation also contributed to the decision to incorporate asizable central component. This was the possibility that project-related results would not be transferableunless MOH decision makers were actively involved in translating project experiences into replicable andsustainable policy packages. In response, the design envisaged an activist role for central units tasked withdigesting and disseminating CHN3 lessons to non-project provinces.

The rationale for and scale of CHN3's central level support were debated during projectdevelopment and remained an issue during implementation. This component was a focal point in thepreparation of this ICR. By including various central activities, the project design counted on changes inmanagement practices and behavior within MOH. These adjustments were seen as needed if MOH was toperform the facilitative and wholesaling functions which had been delineated.

However, by financing a large and diverse Jakarta-based component (thereby departing from theTHP template), CHN3 itself took on risks relating to ineffective or even counterproductive performance bycentral units. THP was less vulnerable to such problems since it financed only a few centrally implementedtasks, which were under a single unit, the Bureau of Planning, where it received high quality managementattention.

The risks entailed in supporting large scale central level activities were recognized to some extentduring preparation--the appraisal report referred to delays and other problems encountered in implementingthe Health Manpower project (Ln. 2542) which had recently closed with an Unsatisfactory rating. It wasanticipated, though, that responsibility for backstopping and overseeing project activities in the five CHN3provinces would provide the purpose and defined framework needed for successful implementation withinMOH. But the expected changes did not eventuate, while implementation in three program dimensions infive provinces and numerous central component activities under two ministries made the project complexand difficult to coordinate (and supervise). As discussed below, what resulted were substantialachievements in the provincial component, but only adequate overall performance in the central componentelements. Judged from the province level perspective, the project yielded significant and durable outcomes.The record was weaker seen in terms of the project's central level capacity building and institutionalaspirations.

3.4 Revised Components:

The above components were not revised.

3.5 Quality at Entry:

QAG assessed CHN3's readiness for implementation and other aspects of quality at entry, asSatisfactory, although with qualifications. This review found QAG's evaluation, including the reservationsexpressed, to be valid-quality at entry in this ICR is Satisfactory. The QAG review, done in April 1999,commented favorably on the project's emphasis on province-level capacity building, and development ofquality assurance and evaluation techniques and on the effort made to integrate lessons from earlierprojects. But QAG thought that these strengths were weakened by design flaws which were attributed inpart to incomplete preparations. QAG referred to the monitoring and evaluation element within the projectdesign which it considered overly complex considering what was seen as the limited capacity of provincialand district level staff. The QAG team found that implementing units relied on only a few indicators with

- 3 -

most of the many markers presented in SAR (Annex 8) not being used. QAG referred as well to theabsence of the operational baseline data needed to measure progress overall and in individual projectcomponents.

QAG also cited changes at appraisal and negotiations which added elements to a project whosescope and complexity already presented supervision obstacles. During appraisal it was agreed that theproject would strengthen staff capacity and support studies within five Schools of Public Health and theSchool of Nutrition in the Institute of Agriculture, Bogor (IPB). These possible project activities had beendiscussed during pre-appraisal and mentioned in the March 1992 Aide Memoire as a matter requiringfurther discussion. Accordingly, they were not "added" at appraisal as indicated by QAG. Unfortunately,these activities were not included in the Final Executive Project Summary (FEPS), and their rationale andassociated risks were not examined during the FEPS meeting which agreed that appraisal could go ahead.

The DGHE component led to problems almost immediately and contributed to a five month periodbetween Board approval (1/21/93) and formal effectiveness (7/30/93) and in a longer de factor delay afterproject start-up. Specifically, there were misunderstandings (see December 1993 Aide Memoire),concerning procedures for research activities and other aspects of the sub- component. There were alsotime consuming internal GOI hurdles in compensating project secretariat staff and releasing funds forprovince level activities. MOF's position was usually at variance with what MOH and the design team hadanticipated, requiring several months to resolve.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

Project outcomes were as Satisfactory on balance.

General Objective: to elevate infant, child and matemal health status by improving the specificityand effectiveness of Safe Motherhood, Child Survival and Nutrition programs, including health andnutrition education, in the provinces of Irian Jaya (now Papua), Maluku, Nusa Tenggara Timur (NTT),West Java, and Central Java.

Objective 1: Building provincial capacity to plan, implement and evaluate programs.

Objective 2: Strengthening the capacity of the central MOH's Directorate General of CommunityHealth, Center for Health Promotion and Center for Health Information System to support provincialefforts.

Project objectives were embodied in a design in which health and nutritional status was addressedthrough interrelated activities at the province, district, and central levels. In the provinces and districts, theproject focused on enhancing service delivery and building capacity through improvements ininfrastructure, staff skills, planning, epidemiology, health education, research, and program managementand monitoring. Provinces in Eastern Indonesia and Java were selected purposefully to demonstrate waysof raising performance in contrasting settings--there are striking differences between densely populatedJava and the sparsely settled, culturally heterogeneous Eastern Islands. Among the key design feature wasthe opportunity created through CHN3 for provinces and districts to leam by doing in developing healthand nutrition programs. During project preparation, each of the five provinces developed locally groundedplans to improve service delivery, staff skills, and related institutional capacities. During implementation,each province went through a rigorous annual planning process which culminated in a yearly workshop

-4 -

with the other provinces. What's more, the project provided a mechanism, the Program DevelopmentActivity (PDA) funding window, that enabled provinces and districts to overcome some of the rigidities inthe national budgeting process while planning and implementing locally specific programs.

Province and district level interventions were to be bolstered through CHN3-strengthenedassistance from the Directorate General of Community Health, other MOH units, and various technicalcenters under MOEC management. Project-induced changes in responsibilities and capacities wereexpected to help the MOH Directorate and Centers for Health Education, Health Information, and HealthPersonnel Training and other participating institutions provide proactive, effective leadership andcoordination as regards policy development and planning. This support was to be organized throughinitiatives which integrated central, and province and district-based activities. For example, PDAs wereexpected to combine province and district level insights, proposed solutions, and resources with specializedskills and experience supplied by MOH and the universities.

Programs and research covering several or more provinces were also expected to focus andintegrate central activities. Of these, perhaps the most crucial were several mutually reinforcingmonitoring and evaluation mechanisms including routine collection and analysis of the survey and otherinformation needed to track project performance. Other instruments included regular qualitativeassessments of PDAs and an annual review of progress and plans for each province by central programunits. The mid term review (MTR) was to be a more in depth version of the latter.

Evaluation vantage points. How indispensable to project results was the support provided tocentral level units? Two perspectives on this project design and performance issue were present in theSAR. The first tasked the central component with "changing the role and capacities of relevant programunits with each of the affected units expected to play a stronger leadership role as regards PolicyDevelopment, Planning, and Coordination" (SAR, page 25). What was envisaged was a tightly integratedprogram in which transformed and energized central level activities were seen as making a criticalcontribution to health advances in the five provinces. Surveys of health behavior and outcomes were anexample of what was anticipated from the central component. Survey results, including findings fromspecial near term rounds of the National Household Health (SKRT) and Expenditure (SUSENAS) Surveys,were understood to be crucial for adequate impact evaluation at project end, and in the design ofwithin-project fine-tuning and course corrections.

However, the central component went away almost immediately, as seen in the failure to deliver themonitoring and evaluation plan agreed to at Negotiations. Ultimately, no start-up survey or equivalentbaseline exercise was ever carried out, and without such information, it was impossible to satisfy literallythe objective, stated on page I of the SAR, of achieving significant, measured improvements in infant andchild mortality and nutritional status during project implementation. Problems in respect of measuringproject results were matched by set backs and delays in other central activities.

A restated concept. Early performance shortfalls provided an opening for an alternative view onthe role of central units which is also contained in the SAR. As just discussed, the first version of theproject concept saw central level initiatives as pivotal--central level changes were expected to lead directlyand promptly to strengthened service delivery efforts in the five CHN3 provinces. In contrast, thealternative project design was not dependent on central level leadership in the alternative perspective--theprovincial component was seen as largely autonomous, with inputs from slow maturing central activitiesregarded as contributing to but hardly determining the direction and pace of project execution. Linkageswere expected to strengthen during implementation as central units became more experienced and effectivein assisting province and district level programs.

-5-

The second version of CHN3's design is discernible in the way health information and relatedmatters were handled. For instance, despite the attention given to measuring project results, the SAR onlymentioned possible baseline survey options and suggestions for performance measures, leaving actualinstruments, indicators, methodology, and implementation arrangements to be worked out aftereffectiveness. For QAG, the absence of adequate baseline data and an agreed monitoring and evaluationplan was evidence of hasty and incomplete preparations (see section 3.5 above). Instead, the issue seems tohave been that of changing views on the necessity and availability of a central "engine" to moveimplementation along--an initial assessment, that a strong central role was both feasible in the near termand indispensable, gave way to a commitment to institutional development in the central component.

It was this more realistic formulation of CHN's goals and processes which actually guidedimplementation and which arguably, should inform performance assessment. The effort to clarify projectgoals and means began when the July 1993 start up mission found little progress in developing the projectmonitoring and evaluation plan agreed at Negotiations (see section 4.2b). Follow up visits pursued issuessurrounding baseline and follow up performance indicators, within increasing attention to addressing thecapacities of Jakarta-based units. The tone and content of mission reports during 1993 and 1994 focusedon what were seen increasingly as formidable, perhaps insurmountable, institutional challenges in theJakarta-based component. Meanwhile, review missions also focused on overcoming various proceduralbarriers which had dissuaded provinces from using the PDA mechanism and interfered with CHN3'slearning-by-doing mechanisms. In January 1995, there was an important breakthrough. GOI and the Bankagreed that almost two thirds of the consultants' services, research, and studies spending categories wouldbe channeled through PDAs, which were incorporated through an Amendment into the Loan Agreement.PDAs could also draw on funds from the equipment, training, and instructional material disbursementcategories if these items were part of the activity being funded. It was decided as well that PDAs would beeligible for 100% project financing if various criteria were met, and that PDA allocations could be carriedover for up to three years if unspent.

Finally, two years after the project became effective, the June 1995 mission reported that the paceof implementation in the provincial component had finally picked up. The Aide Memoire added that with"a critical mass of staff, mechanisms, and procedures in place, it may be timely to revisit the aims andexpectations associated with the project." No definitive restatement was proposed, but directions wereclear--the document dealt mainly with province- level service delivery program planning and supportingPDAs, with ongoing Central activities described briefly towards the end as needing "to be consolidated andlinked to service delivery." Clearly, CHN3 had evolved into a project with two loosely connectedcomponents.

Overall implementation results. How well did this restated project perform? Evaluation ofCHN3 is impeded, of course, by the already cited absence of indicators for the project provinces atstart-up. The lack of a project specific baseline, a clear design shortcoming, makes it difficult to excludeconfounding factors and estimate with any precision the effects of project interventions. Nevertheless, it ispossible to draw on different information sources to put together a credible picture of project achievements.

First, national surveys (SKRT and SUSENAS) showed that there were significant, very likelyCHN3-assisted improvements during the 1990s in service coverage and in child and reproductive healthand nutritional status in NTT, Maluku and Irian Jaya, and in West and Central Java. For example,prenatal care coverage (including TT2 vaccination) and skilled-assisted birth delivery increased sharply inNTT, Irian Jaya, and Maluku, the province's with the largest CHN3 financial support, and to a lesser

- 6 -

degree in West Java and Central Java during the project period (Annex Table 1). There were also strikinggains in all five CHN3 provinces in the proportion of births attended by medical personnel, as well as somesubstantial improvements in the immunization coverage (DPT 1) of young children and in the proportionreaching acceptable birth weight levels. There were also significant reductions in estimates of infant andchild mortality and fertility in most of the project provinces.

Gains in service coverage, and in health and nutritional status occurred during a period ofincreasing access to trained health personnel. For example, the number of health centers and sub-centersexpanded during the project period, along with the number of qualified staff (Annex Table 1). CHN3 wasinstrumental in this process. For instance, the project helped to train and deploy large numbers ofgovernment-employed midwives in the five provinces. In Irian Jaya alone, CHN3 was instrumental indeploying almost 6000 trained midwives, including constructing village health centers in which almost afifth of these women could be based. Moreover, CHN3 contributed to program capacity and effectivenessin numerous other ways depending on specific needs by province. Again, the lack of baseline data makes itimpossible to develop robust estimates of the project's contribution to the large reductions in mortality andmorbidity which took place.

Among CHN3's lasting province-level, "software" contributions were the backing provided forintroduction and improvement of Integrated Management of Childhood Illnesses (IMCI) strategies,adoption of Maternal and Perinatal Audits (AMP) and related community-based and quality of care-relatedapproaches to reducing maternal mortality, and the use of mapping techniques and follow up measures toaddress consequences of micronutrient deficiency disorders in specific areas. These are examples of worksupported through PDAs which became an effective policy instrument and learning-by-doing mechanismafter various procedures and disbursement rules were amended (see discussion above).

PDAs in turn were often most effective when they contributed to and/or were suggested as aconsequence of the annual work plan exercise and related Integrated Planning and Budgeting approachwhich the CHN3 provinces were among the first to implement. The techniques and procedures of theIntegrated Planning and Budgeting methodology were disseminated through a series of training activitiesand workshops which covered related skills, e.g., data collection and analysis. These skills were put to usein regular planning, programming, reporting and other ongoing tasks which were brought together at theannual planning meeting attended by all five provinces. There venue for these meetings shifted each year,enabling CHN3 staff from each province to make site visits and compare experiences. In this way, CHN3helped strengthen province and district level capacity to understand and exploit decentralized planning.And though results for this objective could not be assessed quantitatively, it was generally agreed that theCHN3 provinces and districts activities were more advanced than other provinces in terms of programplanning and management. This achievement was evident when Laws 22 and 25 on decentralizationbecame effective in January 2001. Apart from strife-torn Maluku, CHN3 districts appeared readier to copewith decentralization than those in other provinces.

Implementation results by province are described in the following section. Also reviewed isperformance in a central component initially tasked with providing proactive leadership and seasonedguidance. After effectiveness, some of the pre-conditions for a catalytic central role became clearer.Central units would need to move from a "top-down" to a supportive engagement, allowing provinces anddistricts to take the initiative and responding to their needs. This major change in the "rules of the game"would require systematic changes in incentives, roles and responsibilities, and associated organizationalbehavior. This ambitious goal was not achieved. Most central units persisted with traditional, ratherdirective organizational behavior patterns. Without the anticipated changes in organizational behavior, theprovinces were able to achieve satisfactory gains in health outcomes. However, access to responsive,

- 7 -

technically competent central inputs may have resulted in more striking advances (see discussion below onthe remaining agenda by province).

Conclusion project = S Performance was Satisfactory

4.2 Outputs by components:

(a) A Provincial Service Delivery and Capacity Development Component

Rating: Satisfactory

Province-based activities accounted for almost two-thirds of project spending, with over 70% ofthese outlays directed to the three Eastern Indonesia project provinces (Annex Table 2). This componentaccounted for most project successes and arguably for any lasting impacts due to the project.

The review below summarizes the areas of project emphasis and topics of note for individualprovinces. This discussion draws on the co-financier's assessment of equipment procurement and use. TheBorrower's ICR proved helpful, as did the cofinancier's procurement review, a number of high quality,project-supported evaluations of different sub-components and instruments, e.g., PDAs. Theseassessments, which typically involved surveys and focus group discussions with staff and beneficiaries,contributed to the Borrower's ICR as well.

East Nusa Tenggara (NTT)

Rating = Satisfactory

CHN3 extended service access in this province, and fostered use of monitoring and evaluation todefine interventions and improvements in program effectiveness. One area of support was for constructionof vaccine warehouses in Flores and Sumba Timur which contributed significantly to the logistic system,and purchase of radio systems which improved medical communication in the province. NTT absorbed11% of overall project spending, with these resources accounting for at least a third of the developmentbudget for health in the province.

A distinctive and successful feature of CHN3 in NTT was the priority given to staff capacity at themanagerial and service delivery levels. The project financed in-country degree training for 87 staff, andtraining externally for another 29 individuals, and developed a Multi-stream Academy (MSA) andstrengthened a Bapelkes (Center for Health Personnel Training) in Kupang. Policy makers wanted toaddress: (1) the limited capacity and skill of district and sub-district health staff in planning,implementation and evaluation; and (2) the failure to plan services in an integrated manner. NTT officialsfelt that too much emphasis was given to traditional classroom based, often focused on vertical programsand only those staff working in those programs. NTT wanted a total paradigm shift in training moving thetraining away from vertical programming toward an approach that emphasizes "on-the-job" training toteach practical applications and to change traditional attitudes and practices.

In response, the province developed its own strategy to strengthen planning capacity and staffskills. Key components included: (1) in-service training to strengthen team work not merely individualscapacity; (2) establishment of a team of core-trainers at the province and district level; (3) promotion ofprogram-based rather than project-based planning; (4) support for bottom-up planning based on generalpolicy guidelines from the province; (5) utilization of data to identify local problems and program needs;

- 8 -

and (6) development of planning guidelines and formats to support this process. Participants included arange of district health staff, including most section and sub-section chiefs from both the District HealthOffice and the District Hospital.

An ex post assessment revealed that the data analysis and management training was very importantto building skills in program planning. Staff at both the province and district level felt themselves morecompetent to conduct planning and programming and more responsible for these activities. The emphasiswas on integrated service planning and to eliciting greater support to health services from the non-healthsectors. On-the-job training also encouraged the shift from exclusive responsibility of the health centerdoctor for all management tasks to involving other non-physician staff in planning and management tasks.Contract doctors had more flexibility to focus their skills on medical services rather than administrativetasks. The management training program, which initially covered three districts, was later expanded tocover the whole province and is supported through local government funding.

Provincial health officials were especially concerned about the quality of midwives training inobstetric and neonatal care. Although training combining clinical and classroom based activities, manymidwives had fewer than five cases to manage during their training. Fewer still attended a sufficientnumber of deliveries during a year to maintain competency in many of their clinical skills. Accordingly,training in health centers and hospitals was expanded so that village midwives could be exposed todeliveries and basic diagnosis and treatment in a clinical setting.

Provincial health officials also decided that since village midwives were sometimes the only sourceof health care in many parts of NTT, it was essential to train them in basic health skills besides maternitycare. After completing their clinical training program, the village midwives also attended a two-weekprogram during which they learned social-cultural and community development skills. The project alsoprovided midwifery kits and transportation allowance and in kind support of boots, umbrella and raincoats.

Remaining agenda. The province has already extended and incorporated many CHN3 initiativesinto existing programs through other funding sources. Looking ahead, there were aspects ofimplementation that could have been more effective and others which need to be followed up.

* NTT's use of PDAs lagged behind the other CHN3 provinces. The most successful PDA, onmodifying Integrated Management of Childhood Illness (IMCI) training, was adopted as anation-wide program. But overall, only a few PDAs were accepted due to limited staff skillsand experience in use of local data to identify priorities and prepare proposals.

* Despite training and other support, an evaluation found that many village midwives did notintend to continue their work in their village after their government contracts ended. Moreover,supervision of village midwives was seen as irregular due to limited staff capacity.

Provincial and district staff felt that central program supervision was insufficient, and there wastoo little support from local governments and related sectors.

Irian Jaya (now Papua)

Rating = Satisfactory

CHN3 responded to this province's size, and its small, impoverished, dispersed and often isolated

-9-

population with a major intervention program which accounted for over a fifth of project spending, andmore than half of the province's development budget for health. The overriding challenge was to extend thenetwork of facilities network and staff to increase access to health services, especially for the indigenouspopulation who often lived in very remote areas. The project was entrusted with a specific responsibility,embodied in a covenant, to ensure that the proposed health measures took account of the preferences of theindigenous population.

In response, one policy thrust was to strengthen the capacity of provincial and district staff throughtraining and technical assistance. CHN3 funded fellowships for various in-country and extemal courseswere awarded to over 244 staff. Capacity building also involved construction of seven nursing schools, amedia production center and a Multi Stream Academy (MSA) subsuming Academies of Nursing,Environmental Health, and Nutrition.

But the most striking and important policy measure involved midwives. Since recruitment forregular government midwives (Bidan A), who were required to have completed three years of technicalnursing school plus one year of midwifery training, was insufficient and due to the limited education ofmost indigenous women, a special option was developed, known as Bidan C training. Over 3000 locallymidwives in Irian Jaya. Village midwives were also trained to provide basic health service beyondmaternity care. However, as in NTT, instruction in obstetric and neonatal skills was problematic, sincemany midwives had only few clinical cases to manage during their training.

This initiative involving women from indigenous communities seemed like best way of reaching apopulation which is disproportionately rural, often located in extremely isolated remote areas, anddisadvantaged in term of language and literacy. Such disadvantages were reflected in health services withindigenous groups receiving limited health care from small numbers of trained health workers. The BidanC program was seen by all fulfilling GOI's obligations as regards the indigenous people's covenant.

Field evaluations, including a 1999 study of 48 Bidan C and 150 community members from sixdistricts, found that these women were active, eager, and dedicated, and had generally been deployed inappropriate locations and been successful in getting services for the first time to many mothers andchildren. The study concluded that the Bidan C initiative was a worthwhile program, but could beimproved by offering continuing training, e.g., in primary and emergency care, stronger supervision andreferral arrangements, and gradual improvements in infrastructure. With project support,puskesmas-based supervision was supplemented when a cohort of 26 Bidan A were trained as fieldsupervisors of Bidan C. The study proposed partnership arrangements with NGOs to assist in training andbackstopping the village midwives, and called as well for field ethnographic studies of reproductive beliefsand practices, and compilation of detailed lessons of Bidan C experience good and bad.

Remaining agenda. Irian Jaya used CHN3 to good advantage including launching a daringapproach to improving service delivery in the special rural conditions in the province. However, thismeasure, i.e., Bidan C initiative, and associated policies need to be sustained and extended if healthadvances are to continue in the special and difficult circumstances in the province. The principal follow upissue for CHN3 relates to program sustainability. In this regard, the Bidan C initiative must addressseveral issues, some of which extend far beyond the boundaries of CHN3:

* The acceptability of these service providers in local comrnunities cannot be taken for granted.A study found that local communities welcomed midwives but were disappointed that theydidn't routinely provide medicines or injections. Also many women did not understand theneed for antenatal care nor assistance with delivery. Clearly, there is much to be done to

- 1 0 -

prepare and educate communities preventive health and the role of the Bidan C. Thecommunity development and communications skills of these midwives will need to get attentionas well.

* The willingness of Bidan C staff to continue their work is also at issue. It was estimated thatmore than 75% of the village midwives remain in the same village in which they wereoriginally placed. However, some indicated they may not be willing to stay after theirgovernment contracts ended-a pervasive concem was whether communities would provideadequate support including a place to provide services. In focus group settings, midwivesvoiced concem about their limited skills, equipment, and supplies, and sought training, careercounseling, and access to radio communication.

* The broader health agenda including the spread of HIV and other diseases cannot be ignored.The Bidan C, despite their lack of training, will still be asked to treat victims of traditionalcommunicable diseases as well as individuals with AIDS-this disease is thought to beapproaching epidemic proportions in the province. CHN3 helped put HIV on the province'shealth agenda by supporting an incisive anthropological field assessment of the epidemic.Interventions post CHN3 will likely need to give greater attention to communicable diseases,including HIV, and possibly other priorities. Mrian Jaya faces a vast health agenda.CHN3-financed improvements provided a start, but the province will need not only sustainedsupport for the Bidan C but a major commitment to a sector-wide program.

Maluku

Performance not rated.

As with their counterparts in NTT and Irian Jaya, the govermment of Maluku decided to use CHN3to organize health care in a manner consisting with local circumstances. Maluku is an archipelago withmany remote, lightly populated islands. The strategy was to use CHN3 resources to develop 21 centersserving clusters of islands. This included rehabilitations of health centers, and construction of aMulti-stream Academy (MSA) and a Bapelkes in Ambon. Service delivery in the 21 centers was to bestrengthened by training traditional birth attendants, improving midwife training and deployment includingrecruiting almost 900 Bidan C staff, upgrading transport and referral systems, and introducing maternaland perinatal audits (AMP). An iodine deficiency survey was conducted in all sub-districts to provideprevalence information for better program targeting. Project outlays, over 40% of which was spent onequipment, accounted for 1 1% of total CHN3 expenditures. Unfortunately, project activities in Malukuended in 1998 due to violent civil strife. The MSA and Bapelkes closed, and many health staff trainedthrough CHN3 had to leave the province; most equipment procured through the project was damaged.

Unfinished agenda. Sustained and destructive violence in the province negated what had been apromising start on CHN3-funded program. Access to services of adequate quality and health status itselfhave deteriorated. Meanwhile, with security likely remaining a concern even in areas with a semblance oflaw and order, the province is facing greater hurdles than before in recruiting trained staff. In thesecircumstances, the strategy developed for CHN3 funding, e.g., upgrading, staffing, and equipping 21central service points, remains valid, and hopefully will be supported financially and implemented when thecrisis ends.

-11-

West Java

Rating = Highly Satisfactory

This province was included in CHN3 because infant and maternal mortality rates had remainedhigh even though access to health facilities, along with communications and transportation arrangements,was adequate. Project outlays, which were smaller than for other provinces and modest relative to the thenpopulation of over 32 million people, were large enough to make an impact in specified areas.

CHN3 was used to enhance the quality and effectiveness of infrastructure and staff that werealready in place. CHN3-funded fellowships were awarded to 265 individuals. Specific projectachievements included introduction of verbal autopsies or maternal and perinatal audits (AMP) todetermine the causes of infant and maternal death (see below). Also worth noting is CHN3's modification(in Cianjur district) of WHO's Integrated in Management of Childhood Illness (IMCI) algorithm. Thisinnovative step involved community health volunteers, village leaders, religious figure, traditional healers,as well as local drug vendors. The approach was replicated in other districts in the province. Similarly, theprovincial health office developed standard opeTational procedures for neonatal and infant health services incooperation with Padjajaran University. This became an important tool for monitoring overall infantdevelopment which was replicated throughout West Java, and later in other provinces. CHN3 supported asurvey in West Java and Central Java to define prevalence and identify detenninants of iron deficiencyanemia in pregnant women and children under-five. Results were used at the district level to improveprogram targeting.

Within CHN3, West Java province made the most extensive and impressive use of the PDAmechanism to fund innovative activities. Some interventions were later institutionalized and expandedthrough the local budget and other funding mechanisms. These included a manual for health-postsupervision, a growth monitoring chart, standard health procedures for infant health service and IMCI, andevaluation of Life Saving Skills (LSS) training.

Finally, CHN3 significantly strengthened provincial and district planning and management byintroducing integrated planning and budgeting, and techniques for decentralized community healthdevelopment. It is generally agreed that implementation of CHN3, including support for data analysis andmanagement training, has been very instrumental in speeding up decentralization in West Java. On theother hand, the effective use made of CHN3 reflected strong teamwork at provincial and district levels, theinvolvement of local government, and inter-sectoral collaboration

Sector agenda. West Java use of CHN's resources and opportunities was exemplary and valuablefor its own health development needs and in terms of benefits to other provinces. Health status, includingrisks to mothers and children, remains a matter of considerable concern in this large and comparativelypoor province. Policy makers have responded by building on and going beyond CHN3 to grapple with theopportunities, and needs and risks associated with much expanded local autonomy and responsibilitiesfollowing govermnent decentralization.

Central Java

Performance = Satisfactory

Like West Java, this province was included in CHN3 because of its sizable population (16% ofIndonesia's total) and relatively high infant and mortality rates. The project focused on improving servicequality through staff development and strengthened monitoring and supervision. The capacity of provincial

- 12 -

and district health staff was enhanced through in-country and external program fellowships, which wereawarded to 1135 people, as well as construction of a Bapelkes (Center for Health Personnel Training) inGombong. CHN3 supported iron deficiency mapping to improve targeting in the control program. Thissurvey was the most widely disseminated PDA in Central Java.

Another noteworthy PDA was an effort initiated by the health center doctor in Kepil to reducematernal mortality. The approach adopted in what was called the Tabulin scheme ensured that all womenin the community received a minimum package of maternal health services (four antenatal care visits, adelivery assisted by a midwife, transportation and referral costs, and free emergency obstetric care forwomen who were referred to the hospital). Those women who could not afford the initial fees were allowedto pay their fees in-kind through local agricultural products such as peanuts or vegetables. The villageraised seed money to start up the Tabulin community fund, run by a Board, by determining in collaborationwith the community the fees for membership and the services covered.

A lasting CHN3 achievement in Central Java was the attention given to enhancing the quality ofmaternity care services through LSS, emergency obstetric care management training, and AMPs. CentralJava along with West Java led to the institutionalization of AMP. A typical maternal audit used familyinterviews and other techniques to go beyond the standard clinic-based medical audit approach todetermining causes of death. CHN3 supported preparation and dissemination of training and technicalmanuals for AMP which was introduced in four districts and two in West Java in 1994 and in the rest ofthese provinces in 1996 and later adopted in numerous other provinces and as an element in therecommended reproductive health policy package. With assistance from CHN3, AMP was developed notonly into a valuable diagnostic and reporting tool but an important training instrument and a means forfinding solutions to policy constraints.

Remaining Agenda. While performance was satisfactory, there are indications that the provincecould have exploited CHN3's mechanisms more fully:

* Central Java could have made greater use of PDAs. Assessments showed that staff at districtlevel were not fully informed of opportunities to use PDA funds to test innovative programactivities.

* The co-financier's end-of-project procurement inquiry found instances of underutilization ofCHN3-financed equipment. Purchase of low quality and sometimes irrelevant equipment andlow maintenance and repair funding were cited as the main problems.

* The project made important contributions to implementing bottom-up planning through asystem that stratified health centers. However, some district staff felt that provincialprograms were given priority over local initiatives. In general, Central Java gave lessemphasis than the other provinces to specific health problems. Most interventions were genericactivities which did not reflect provincial or local characteristics.

* Central Java used CHN3 to improve data analysis and utilization. However, the HealthInformation Center which was supported was not linked to most districts and served mainly fordata processing and production of annual health profile reports.

(b) A Central Support Services and Capacity Building Component

Rating = Satisfactory (marginally)

The central component, covering diverse activities in numerous units under two ministries,

- 13 -

accounted for over a third of project spending. The broad goal for each of the sub-components withinMOH and MOEC was the same, i.e., to support provincial and district level efforts to improve servicequality and effectiveness. Achievements included assistance to provincial and district staff in designing andcarrying out PDAs, along with centrally initiated in-service seminars for program managers from eachprovince, and work force and ethnographic studies. Also noteworthy were the IDD mapping in the CHN3provinces and use of results for program formulation, establishment of a longitudinal surveillancemechanism, and support for public health and nutrition faculties via scholarships for advanced degrees.The Borrower's ICR (Section 9a) discusses central component contributions in further detail.

Central component achievements were qualified or offset, however, by design and implementationdifficulties in individual sub-components. All in all, the performance of these Jakarta-based activities wassatisfactory at best. This was a disappointing result especially for a component looked to initially foractive leadership and timely guidance to newly empowered provincial and district teams. Expectationswere lowered in the "rebooted" design which actually guided project implementation. But even in terms ofa more modest outlook, the central component under-performed-the Borrower's ICR reached much thesame conclusion. Outcomes in key sub-components are discussed below.

Health Education

Performance = Unsatisfactory

The objective was to help transform the MOH Center for Health Education (PKM) into a unitcapable of supporting the distinctive health education activities needed as part of the service deliverystrategies adopted in each province. To this end, CHN3 supported education and training for healtheducation staff at central, provincial and district levels. This approach was not very successful. Provincialand district capacity overall remained weak, with no system in place to help newly trained staff implementwhat they had learned. The media production skills of provincial and district health staff were notsufficient to meet the needs of community; good data and research were limited and rarely used forpianning. In addition, many staff trained in health education moved to other units. Health educationdevelopment was also constrained by a lack of accountability and low motivation among provincial anddistrict level staff.

An end-of-project study indicated that many decision makers in the provinces and central programunits felt that PKM was not responsive to their needs. Health education activities remained Jakarta-basedwith provincial and district staff habituated to waiting for guidance rather than taking initiativesthemselves. Central authorities showed little awareness of or interest in provincial and district levelactivities, needs, and difficulties confronted in the field. Indeed, a Plan of Action, developed by theinternational consultant team, dealt with overcoming supposed "threats" to PKM rather than itsresponsibilities towards the provinces and districts. Central supervision and monitoring arrangementsremained weak, especially in the eastern provinces. In retrospect, CHN3 provided a limited initialfoundation in health education and promotion in province and district level units. Further initiatives will beneeded to build effective local capacity, and to work out the appropriate role for central units. A morefundamental reformn of the health education system and substantial changes in the available resources isrequired.

Health Information System (HIS)

Performance= Unsatisfactory

- 14-

The objective stated in the SAR was to enable the Center for Health Information (Pusat Data) tooperate effectively as MOH's multi-functional, technical secretariat for health information. To achieve thisand associated goals, CHN3 financed equipment, software, training, fellowship and technical assistance.This sub-component accounted for almost a quarter of the funds initially allocated for central levelactivities.

The objective of developing Pusat Data into a technically competent and responsive unit was notachieved through the project. Burdensome and redundant data collection continued during and afterimplementation, while the quality of different data collection system remained weak and as did analysis anddissemination of findings from different sources to province and district level officials. Three areas ofunderperfornance stood out:

* As discussed, no baseline survey ever took place despite design provisions and follow upefforts to find ways of proceeding, making it difficult to gauge performance. After years ofdiscussion, the SKRT was finally conducted in a sample of project districts in 1996, and aRapid Assessment was carried out in 1998/99 in each province except for Maluku.

* There was little progress in developing health information systems appropriate for provinceand district needs. CHN3 supported implementation of new ( in West and Central Java) orimproved ( in NTT, Maluku, and Irian Jaya) facility reporting systems. However, thesearrangements were not fully accepted by all program unit staff (who claimed their data needswere not met). This resulted in the limited use of the pilot systems and redundant datacollection by each program. Pusat Data provided very limited backing for this work, retaininga top-down approach, failing to explicate the technical and administrative issues entailed, andproviding negligible leadership, problem solving options, coordination between sectors, andfeedback and advice to the provinces. Conditions deteriorated due to low routine budgetallocations for information systems and weak data processing skills at health center and districtlevels.

* The CHN3-supported longitudinal survey sites in Purworejo (Central Java) and Belu (NTT)were not exploited to improve data use in health and nutrition decision making. Set up andmanaged by Gadjah Mada University (UGM), Purworejo was utilized mainly for academic andscientific purposes by UGM students--the indicators collected did not match the needs ofdistrict and provincial health offices and thus attracted little interest--Pusat Data did notprovide any leadership or informal assistance. Purworejo also was ineffective as a trainingvehicle for district staff. This program has attracted other funding sources and continued as acommunity laboratory for the students even after the project closed. Belu was shut down in1999, due to political instability in Timor. The use of the surveillance sites as centers ofexcellence for program evaluation was not achieved in the project. Research results did notcontribute to provincial and national agendas, due to ineffective communication andcoordination between the surveillance sites and the provincial health office.

This disappointing outcome is attributable, basically, to a faulty diagnosis of Indonesia's healthinformation needs, challenges, and capacities which resulted in unworkable initial and revisedsub-component designs. Inadequate analysis was evident in the original project concept in which thesub-component's role was not only overloaded, but also under-specified. In this design, Pusat Data wasresponsible for generating the information needed to "record significant, measured changes in infant andchild mortality, and in nutritional status" (SAR, p. 1). The text and annexes of the SAR mentioned various

- 15 -

steps, activities, and even performance indicators which would contribute to producing the needed data.But apart from a specific reference to one survey (SUSENAS), the document was non-committal as topriority actions and timing, citing GOI assurances that a project evaluation plan would be presented nolater than six months after loan effectiveness.

Weak analytical underpinnings also explain the lack of a pragmatic alternative when GOI failed tofulfill its commitment to develop a monitoring and evaluation plan. The Bank's response acknowledgedPusat Data's capacity weaknesses but then set goals which were not only as unrealistic and overloaded asbefore but also inflexible. The revised game plan for the health information sub-component was set out ina December 1994 Aide Memoire which took note of progress, but then indicated that Pusat Data's newlydrafted strategic plan was far from adequate. The reasons included inadequate attention to the number andcomplexity of health center registration activities, and to how improvements in the "quality and validity" ofhealth center data would be achieved. This Aide Memoire then recommended activities which would leadto effective monitoring and evaluation. Among these were in-depth analyses of i) the information needsassociated with each provincial service delivery strategy and ii) existing health center recording practices.This work was expected to lead to a detailed proposal to test and evaluate a revised recording system forfacilities in Central Java, and by December 1996, to a training and hardware provision plan extending thisnew system to the other CHN3 provinces.

The December 1994 Aide Memoire acknowledged that the agenda would require Pusat Data tostrengthen its own institutional capabilities as well as those of its province and district level counterparts.The suggestion was made that, with project financing, the Centers for Disease Control in Atlanta, Georgia,could help Pusat Data address its institutional weaknesses. The accompanying (January 1995) Back toOffice Report (BTOR) indicated that the above elements had been agreed and included in a revisedStrategic Framework which "usefully articulates a primary focus on improving the use of data for decisionmaking at local levels, and reinforces the role of Pusat Data in improving the efficiency of MOH "healthinformation systems." Finally, the BTOR made it clear that "Achieving the policy's objectives will be amajor challenge for Pusat Data, which suffers from weak leadership and limited staff capacity."

In any event, this revised approach was not successful--nor could it have been. A follow upmission, in June 1995, reported that "numerous, promising activities" were underway that needed to beconsolidated. However, whatever commitment and momentum which had been created could not besustained; planned steps did not go ahead, and by the July 1996 Mid Term Review (MTR), the Bank andMOH had essentially turned to other options. The MTR's stance was eclectic, with the provinces andcentral program units encouraged to develop core and supplementary performance indicators for inclusionin the SKRT or specialized rapid surveys. Pusat Data was again asked to submit a plan for assuring thequality of different data collection systems, and improving ways of disseminating findings. Subsequentmissions dealt with dissemination of the SKRT "baseline" survey and then with piloting rapid assessments.Interactions continued but Pusat Data no longer loomed large in Bank-GOI project discussions as it had atproject start. By project closing, barely half of the original allocation for this sub-component had actuallybeen spent. The actual "instrument" for these spending cuts were repeated rejections, by BAPPENAS ofPusat Data funding proposals-the Bank team welcomed these decisions, while MOH did not lobbyintensively to overcome objections within Bappenas.

What lessons should be drawn from this experience? First, with the initial enthusiasm for anational approach giving way to encouragement for province-based solutions, CHN3 ended up with anappropriately modest and experimental stance towards health information matters. However, this outcomeresulted from an extended and costly learning process for which Pusat Data bears considerableresponsibility. In effect, this unit did not take advantage of an excellent opportunity to define and

- 16 -

operationalize a key role for itself in an upgraded health information system.

However, the Bank was not blameless. The original concept underestimated the difficultiesentailed in developing a project-wide baseline survey, and in implementing a multi-prong, multi-levelinformation system concept. This complex and sophisticated design did not benefit, it appears, from anassessment of Pusat Data's capacities, e.g., staff skills and motivation, organizational and coordinationcapabilities, resource availability, and formal and informal responsibilities and managerial authority.Moreover, the Bank's response to early implementation problems was also unrealistic. The recommendedstrategic framework was well beyond what clearly were very limited Pusat Data capabilities. To its credit,the design team took on difficult issues for which there were no best-case solutions even in high incomecountries. But the lack of an recommended solution in the SAR and the inconsistent signals in somereports, support the impression that the Bank team was uncertain as to what would constitute a reasonable

approach.

This must account for the eventual decision to move to a more modest engagement in the information field.

Training

Performance = Satisfactory

The goal of the DGHE portion of this sub-component was to strengthen the quality of thetraining, research, and technical assistance carried out in several universities and other institutionswith ties to the five CHN3 provinces. Trainees in the DGHE sub-component were drawn fromthe University of Indonesia's Faculty of Public Health and Center for Childhood Development(assigned to West Java), Gadjah Mada University's Faculty of Public Health (Central Java, NTT),Diponegoro University's Faculty of Public Health (Central Java), Airlangga University's Facultyof Public Health (NTT), Hasanudin University's Faculty of Public Health (Irian Jaya, Maluku),and the Community Nutrition and Family Resources Faculty (West Java) within the AgriculturalInstitute in Bogor (IPB). Also covered in this discussion is the training experience of centralhealth staff, including those drawn from the Nutrition Academy. (Reference is also made totraining achievements and concerns of province and district level staff). Also part of CHN3'straining interventions was the Center for Health Personnel Training which received support atcentral and provincial levels facility construction and renovation, curriculum development, andtraining of trainers.

** As an example, the December 1993 supervision team reported satisfactory progress in developing the methodology for astart-up survey, but noted that the "study on baseline data" had been downgraded and linked to district and sub-districtcapacity development. The Aide Memoire warned that baseline work "should not be undertaken without clarity on theroles and responsibilities of different organizations."

- 17 -

In-country Overseas Overall-tot.PhD Master L dergra Diploma i. Cours Sub-Tota PhD Master S Course Seminar Sub-Total

Provincelnan Jaya 0 19 84 104 10 217 1 12 14 0 27 244Maluku 0 37 70 35 147 289 0 4 20 0 24 313NTT 0 14 73 100 6 193 0 6 23 0 29 222CJava 0 122 107 122 708 1059 0 13 63 0 76 1135W Java 0 98 94 17 9 218 0 18 29 0 47 265

Sub-total 0 290 428 378 880 1976 1 53 149 0 203 2179Central unitDitNutrition 1 8 29 3 147 188 4 3 31 11 49 237Dit Higher Educa!ion (MONE) 26 35 0 0 180 241 9 12 35 2 58 299Center for Health Education 0 3 6 0 0 9 0 5 16 6 27 36DIt Comm Participabon 0 3 7 0 87 97 0 0 6 2 8 105Dit Family Health 0 3 1 0 6 10 0 4 4 3 11 21Center for HlS 0 2 1 5 131 139 0 2 32 2 36 175Center for Health Personnel 0 3 4 9 0 16 0 5 2 2 9 25CenterfbrHealthTraining 0 5 8 5 2 20 0 2 7 1 10 30

cademy ef Nutrition 0 7 6 8 6 27 0 3 8 3 14 41Dit Health Center Promotion 0 2 2 1 2 7 0 1 8 0 9 16Secretrariat Comm Health 0 0 0 0 7 7 1 10 3 0 14 21

Sub-total 27 71 64 31 568 761 14 47 152 32 245 1006

Overall-total 27 361 492 409 1448 2737 15 100 301 32 448 3185

CHN3 made a large investment--US$9. 1 million in the central component and US$9.6 millionspent on staff development in the provinces and districts--in capacity building. This type of investmentdeserves close scrutiny. However, capacity building results, which are difficult to observe except inspecific situations, may take years to emerge in full.

Nevertheless, a rough overview can be assembled, drawing on a project-financed beneficiaryassessment, focus group discussions, and other sources. The picture that emerges is of a sub-componentwhose positive impact is already being noticed. Moreover, this investment, arguably the most successfulelement in the central component, will continue to yield benefits in the medium term as trainees move alongin their careers. This assessment is based on the positive feedback recorded in the survey and otheraccounts. Respondents in the DGHE sample, for example, voiced strong satisfaction with theirexperiences, citing increased personal competence and confidence, and enhanced opportunities to raise theirincomes through promotions and grade increments. Academic staff cited improved teaching-learningtechniques, stronger grounding in curricula and syllabi, widened networks, and greater ability to carry outresearch, present findings, and prepare proposals. A survey of health staff sent for training foundanalogous results, which were generally confirmed through observation.

Feedback and lessons from this sub-component were not, of course, uniformly positive. Criticalobservations from different vantage points include:

* Provincial and district health staff were often unimpressed with the technical assistanceprovided by staff, many newly educated, from the Schools of Public Health and Nutrition. Inparticular, consultation results were sometimes seen as not specific enough and difficult to beapplied as program activities. The most significant contribution was seen as the assistance indeveloping specific PDAs. On other matters, communication and coordination betweenconsultants and provincial or district health staff remained weak.

- 18 -

* A related point pertains to research conducted by staff from the School of Public Health andNutrition. Although much of this work was done in the CHN3 provinces, there was inadequateinvolvement of local staff, and the focus did not reflect the need of the programs. A weaknessin this sub-component was the inability to establish effective professional collaborationbetween the universities and the local government and health staff through the project.

* Some province staff felt that a disproportionate share of trainees came from Central Java andWest Java instead of eastern provinces where the needs were greater. They argued as well thatrecruitment of fellowship recipients especially for short courses was not transparent, and thatqualifications of some participants did not correspond to the prerequisites. The main limitationon participation of staff from NTT, Maluku, and Irian Jaya in external training was their morelimited English language proficiency.

* Some noticed that staff who had been trained through the project often moved to other units.This reduced the impact of the fellowship component on program development. In addition,some staff from Irian Jaya and Maluku moved to positions outside the province after theirtraining.

4.3 Net Present Value/Economic rate of return:

Not Applicable.

4.4 Financial rate of return:

Not Applicable.

4.5 Institutional development impact:

Rating = Substantial

CHN3 pursued and achieved an ambitious institutional goal, that of improving the effectiveness oflocally grounded health programs. However, the way this goal was attained differed from what wasanticipated. The project's design incorporated numerous institutional development measures, many focusedon strengthening central MOH units which were expected to contribute proactively to provincial programs.As discussed, this initial institutional development strategy, i.e., to remake key central functions, faceddifficulties from the outset. On the whole, central level units continued to operate in traditional, directivefashion, with little movement towards a catalytic and opportunistic approach to policy development.

Problems early on in the central component led to changes in emphasis and institutional tactics. Inthe initial 18 months of implementation, PDAs were caught up in unworkable and time consuming researchreview and approval procedures. The problem was recognized and correct via an Amendment, and PDAswere then developed into versatile and effective policy and learning-by-doing instruments, which linked upwell with the annual work planning, and integrated planning and budget techniques and procedures whichprovinces and districts adopted with project-financed assistance. The latter, meanwhile, emerged asvaluable coordination and learning exercises as well, especially when returnees from different national andoverseas training programs, began to participate.

These instruments, i.e., Integrated Planning and Budgeting, PDAs, and extensive staff training,improved local government capacity to plan and manage their own programs, and brought the CHN3 areasgreater autonomy. For this reason, the CHN3 provinces and districts were better prepared than other

- 19-

regions for the substantial decentralization measures which went into effect January 1, 2001. It is thisreadiness for decentralization which comprises CHN3's main institutional development legacy.

5. Major Factors Affecting Implementation and Outcome

5.] Factors outside the control of government or implementing agency:

Indonesia experienced a severe economic crisis starting in 1997 and lasting through 1999. Thisperiod was marked by government funding shortfalls and rapid inflation. GOI's crisis response includedsignificant, publicly disclosed cuts in sector allocations, along with further unannounced controls on actualspending. The latter were especially disruptive, leading to sharp reductions in health service delivery inpublic facilities in some regions. Along with these formal and informal policy measures, the civil strifewhich exploded in Maluku in 1998 resulted in effect in the virtual cessation of project activities in theprovince. Together these different factors led to implementation delays and ultimately cancellation ofalmost US$20 million. All in all, the crisis and associated policy responses dealt a heavy blow to a projectwhich had hit its stride in the previous 24-30 months and was taking on health information and otherchallenges from a province-grounded perspective. The crisis undernined the climate and motivation forpolicy innovations. Behavior within MOH became more risk averse and remained so even after PresidentSuharto left office in mid 1998 and the "reform" era officially began. When the crisis began to ease in1999, CHN3 was in its final phase, and not well placed to provide the reform ideas and energy which MOHneeded.

5.2 Factors generally subject to government control:

Commitment on the part of key policy makers in relevant agencies was the most critical factorsubject to government control as regards CHN3. Backing from the government was especially importantunder the authoritarian, highly centralized governance arrangements of President Suharto's New Orderregime.

Several dimensions need to be distinguished when characterizing the extent and nature of GOI'ssupport for the project. First, the broad policy environment was not conducive generally for policy-focusedprojects such as CHN3. During the Suharto era, stability and continuity in institutions and policies werehighly prized. Officially, policy change was possible only at five year intervals when the REPELITA (theplanning cycle) exercise was initiated. Within each REPELITA period, policy adjustments includinginstitutional changes were discouraged. Second, particular policy avenues were either blocked entirely bythe regime or subject to limited or controlled access. Examples included participation by beneficiaries,changes in governmental responsibilities, any regionally centered steps which seemed to challenge Jakarta'spowers, and civil service reforms. Moreover, MOH had little clout and standing within GOI. It was notlooked to on inter-sectoral matters and was ineffective in addressing health financing issues. This wasevidenced in low and unstable funding levels-salaries were such that government staff relied on privatepractice income with adverse consequences for service quality.

However, there were other considerations which offset to some degree this generally unfavorableclimate for health policy improvement. For instance, comparisons with other countries, intemationalpressures to participate in high visibility meetings and causes, interactions with multilateral and bilateralassistance agencies, and recognition at the technical level that the health system faced serious problems allprepared the way for policy interventions in this sector. There was also concern within BAPPENAS andother parts of GOI about inequality of health results between regions and between poor and non-poor. Infact, health policy was by no means static within each REPELITA. During the 1990-93 period, GOIintroduced a number of policy innovations, e.g., the contract doctor scheme, and the village midwives

- 20 -

initiative. (Moreover, government spending per capita rose in by a third in real terms between the late1980s and mid 1990s). These policy changes were acceptable because they were presented as elaborationsor technical adjustments of existing measures.

Among the permissible policy steps were loan-funded projects, with BAPPENAS, MOF, MOH,MOHA, and MOEC reaching a common view of project content and timing prior to discussions with theBank. Here, another aspect of government cormnitment, especially during the Suharto era, was thedifficulty sometimes entailed in arriving at ajoint GOI position at Negotiations. What made this task moreproblematic in the case of CHN3 was the need for different units within each ministry to come together onan agreed position.

These factors-an unfriendly environment for policy reform, genuine concern to address healthissues, and difficulties in securing robust agreement between and within ministerial bureaucracies--resultedin a characteristic pattern of government commitment, i.e., broad but not necessarily sustained, andimprecise in key respects. This led to policy designs which were open-ended on still undecided matters. Anexample was the health information sub-component for which key aspects of the design were to be workedout after effectiveness.

In summary, commitment to the project was satisfactory, though government support wasmeasured and insubstantial on key matters, and subject to contextual influences and other factors duringimplementation. Government commitment was not sufficient to move ahead with strong and proactivecentral unit activities. However, government backing was more than adequate to underpin the alternative,province-based version of CHN3 which emerged.

5.3 Factors generally subject to imnplementing agency control:

MOH's overall commitment to the project was strong, especially to the revised version of thedesign. Program priorities and means, e.g., deployment of village midwives including Bidan C, had fullMOH backing as did the goal of helping to improve province and district level planning and programming.

MOH's support was qualified somewhat as regards specific central level activities. Clearly, theoriginal project concept with its proactive, lead role for central units did not attract sufficient supportwithin MOH. This is not surprising considering that overall GOI commitment to the project almostcertainly did not extend to backing what would be, in effect, significant reforms within MOH. VigorousGOI support would have been needed to bring about changes within MOH and also in relationships withother ministries, e.g., MOEC. Understandably, MOH was more comfortable with the second formulationof the project concept with its looser, task specific linkages between the provinces and central units. Witheffective coordination provided by the Project Management Unit (PMU), this version of the projectperformed especially well overall in the 1994/5-1997/8 interval.

Nevertheless, even in terms of the reformulated project design, there were disappointing aspects toMOH's commitment to the project, notably its inability to arrive at a viable approach to health information.MOH could not get its own house in order on this topic, nor was it able to capitalize on the project-funded,program-relevant activities that were underway in UGM's longitudinal survey sites and also inparticipating schools of public health and nutrition. As discussed, ineffective comnmunication of resultsfrom the surveillance sites to MOH at the provincial and central levels resulted in limited utilization of datafor program planning and development and lack of replication in other districts and provinces. Work donein the schools of public health was not disseminated well either.

- 21 -

5.4 Costs and financing:

The total size of the loan was US$ 95.3 million from the original total cost of US$ 164.1 million.The Bank financed US$ 72.2 million of total project cost as compared to original Bank loan of US$ 93.5million. Due to the economic crisis, GOI requested cancellation of US$ 16.0 million. This step did notaffect financing of the original components as substantial dollar savings resulted from the devaluation ofthe IDR. Another US$ 3.7 million was canceled by project closing. Detailed project cost and financinginformation is presented in Annex 2.

6. Sustainability

6.1 Rationalefor sustainability rating:

The sustainability of project activities and achievements is Likely (except in Maluku), but notassured. What augurs well for sustaining CHN3-financed advances is that, first, the project has alreadymet a market test. That is, participating provinces have drawn on their own revenues and other donorfinancing to mainstream and extend many CHN3-financed innovations. Second, CHN3-sponsoreddecentralized program planning and management advances not only enhanced efficiency and capacity butcontributed to making districts decentralization-ready. These regions should be able to handle theirbudgeting and health responsibilities competently, making it less likely that health services, includingCHN-introduced measures, would be disrupted and/or dismantled due to spending cuts or publicdisinterest.

Nevertheless, there is no guarantee that specific CHN3-funded innovations and interventions willsurvive, especially in poorer regions which have lost, due to decentralization, access to equity balancing,poverty oriented grants and other sources of concessionary health funding. Indeed, some eastern districtsmay face difficulties in sustaining interventions on even a modest scale without secure access to funds atlow cost. Meanwhile, the organized violence which destroyed health facilities and equipment in Malukuand drove staff away has abated. A resumption of CHN3 type health interventions is not likely in the nearterm because of continuing instability.

6.2 Transition arrangement to regiular operations:

Despite the many CHN3 features which fostered sustainability, a proactive stance towardsprogram continuity overall is warranted currently. What is needed is a package of measures whichfacilitate health decentralization, and adequate, stable health financing rather than interventions aimed atmaintaining specific program activities, e.g., training of Bidan C. In this regard,

* MOH needs to strengthen its dialogue with local authorities and stakeholders, using itscredibility as a technical agency to advocate and lobby for adequate allocations to the healthsector. MOF needs to develop funding instruments which can be use to channel special healthgrants to poor communities. Clarification of the roles of central, provincial and district levelgoverrment in health is essential for effective implementation of decentralization..

• Effective monitoring and evaluation arrangements of suitable scale and scope should beestablished at province and district levels with linkages to a national "grid" to be developedwhen ready.

On behalf of constituent districts, provinces should determine essential specialist and other criticalpublic health skill and work force needs in light of market conditions, the availability of candidates, and

- 22 -

other supply arrangements. The central government should consider providing stipends to midwives andother staff working in remote areas where it is impossible to sustain viable private practices.

7. Bank and Borrower Performance

Bank7. 1 Lending:

Performance = Satisfactory (with reservations)

The project design went beyond the successful, province-focused THP, by focusing on priorityhealth needs, and enhancing the important capacity building, health education, and health informationdimensions of the health system. Project identification was consistent with the government's developmentstrategy and the Bank's country assistance concept strategy. However, the programmatic focus andincreased number of provinces covered (from two to five), and the inclusion of new elements, especially thelarge central component, together resulted in an operation far more complex and institutionally demandingthan the THP template. In this regard, there was insufficient assessment of the Borrower's implementationreadiness and capacity--the project team built on relationships with GOI established during previousimplementation of projects. Relevant lessons then available from recently closed Bank projects should havebeen reviewed more systematically. Apart from the issue of complexity, important features of the design,e.g., the health information and MOEC components, were not fully prepared at appraisal and projectstart-up (see section 3.5). This resulted in a slow start for the project, necessitating in effect a stagedreappraisal of a reinterpreted project design (section 4.1). The loan amount and project implementationperiod for the project were well estimated at appraisal time, while the availability of counterpart funds wasoverestimated. Some elements, e.g., Pusat Data, within the central component were reduced in scalethrough reallocation of funds. The crisis-related cancellation and extension of the Loan occurred forreasons that were neither anticipated nor controllable.

7.2 Supervision:

Performance = Satisfactory

CHN3's design created special challenges for project supervision. As pointed out in the previouslymentioned 1999 QAG assessment of supervision quality, the design came together quickly and involved lastminute changes; defined numerous and onerous objectives; involved too many activities, many of whichwere not well tested; was vulnerable to pressures from a government keen on rapid replication; andincorporated a central component covering almost every MOH Directorate with little consideration ofneeds and absorptive capacity.

In light of these project features, the supervision strategy was to be selective, focusing mostattention on the more viable project elements. QAG praised this approach, noting that the team had a goodskill mix including senior staff based in Jakarta, showed leadership in managing the supervision process inan "organic" way, that is, in letting project components that did not show any promise or seemed to gonowhere quietly die, while promoting others that did show growth and promise, e.g., iodine deficiency, safemotherhood, and village midwives. Health information was an example of a sub-component which gotintensive supervision and support initially and then less attention after disappointing results. Anotheraspect of this triage-like approach was the inclusion, in the supervision context, of technical assistanceinputs directed at specific counterpart units or programs. For example, social anthropologists were hired tocarry out field assessments in Irian Jaya on topics of concern, e.g., the status of the Bidan C program andits acceptability to indigenous communities, and the cultural practices and behavior which were facilitating

- 23 -

the spread of HIV.

The QAG review felt that the supervision team could have reacted more quickly to problems in theHealth Education and Information sub-components, However, Aides Memoire for the 1993-1995 periodsuggested the opposite, i.e., that the team had focused on the health information issues and on facilitatinguse of PDAs-- the MTR was used to inform project stakeholders about the changing status of theseactivities. QAG was critical as well of the intensity of supervision at the provincial level. On this, it isworth pointing out how inadequate the normal supervision budget was in the face of five provinces, andmultiple Central program elements. Other QAG observations are worth citing as well. Reporting ofprogress was found to be adequate, but delays in No-Objection Letters (NOLs) from the Bank werementioned as a problem by central as well as provincial government staff. This was attributable tounfamiliarity with procedures.

7.3 Overall Bank performance:

On balance, Bank performance was Satisfactory under the project.

Borrower7.4 Preparation:

Preparation by the Government was Satisfactory. Involvement of the central program units andprovinces in the development of the project's provincial component was adequate as indicated by the activeparticipation of staff from units at different levels during project identification, preparation and appraisal.Each of the five provinces in the project formulated its own specific program to improve service deliveryand related institutional capacities, including staff development, prior to project appraisal. As discussed,preparation of elements in the central component seem to have been done hurriedly and without in-depthanalysis of needs and capacities.

7.5 Government implementation performance:

The government's performance during implementation was Satisfactory. Commitment to thesectoral goals and to strengthening provincial roles and capacity showed little sign of eroding, while thegovernment worked patiently with the Bank to develop a reformulated project concept. Positive aspectsincluded backing for innovative funding/programmatic instruments such as PDAs; adequate levels ofcounterpart financing up until the economic crisis; generally strong support from and good coordinationbetween the central and provincial PMUs; preparation of a good MTR; a pragmatic and helpful approachtowards implementing the indigenous people's covenant in Irian Jaya; and the strong effort made todocument and disseminate project practices and lessons.

Among the negative aspects of implementation performance were the frequent changes in personnelassigned to PMU especially at provincial and district level which disrupted coordination of projectactivities; and an unwillingness, due perhaps to lack of clear authority, to organize greater CHN3-relatedteam efforts within the Community Health Directorate General, within MOH overall, and within GOI; thevery limited, non-operational role played by the project advisory board and project steering committee,consisting of designated echelon I and II officials; and a pattern of delay in submitting audit reports (mostof which were unqualified), notably for Maluku for which there remain funds not fully accounted for.

- 24 -

7.6 Inplementing Agency:

Overall performance was Satisfactory. The central PMU did an excellent job, responding well tothe needs of provinces and effectively coordinating and supporting project activities within MOH. Thecentral PMU was particularly effective at pulling together and presenting the findings of PDAs and otherproject results. On the whole, the PMU handled procurement matters well, overseeing one Loan-financed,International Competitive Bid (ICB) contract and three funded by KfW, which cofinanced the project. Onthe other hand, the PMU could not overcome weak management in individual units, e.g., Pusat Data, andthus could not organize the decisive action that might have produced agreed performance indicators and anearly baseline survey. The project advisory board and project steering committee, consisting of designatedechelon I and II officials, played a very limited role overall, and were not involved in operational matters.

7.7 Overall Borrower performance:

The overall government's performance is rated satisfactory.

8. Lessons Learned

CHN3 provided valuable lessons for health strategy design and implementation in Indonesia itself,as well as findings with broader applicability. Concerning Indonesia, the project's implementationexperiences demonstrated that i) provinces and districts have or can establish the capacity to take onsignificantly increased responsibilities, and ii) it is risky to rely on an unreformed central ministry to leadthe way in health decentralization. When CHN3 was being designed, the choice was seen in part asbetween i) replicating a province-based model, e.g., THP, with virtually no central component, and ii)proceeding with provincial activities but linking these to a centrally powered "replication" motor andcompass.

Looking back at CHN3's performance in terms of these options, the record suggests makingprovinces and districts the focus of policy and investment attention, with provinces taking on variousbackstopping and cost-saving responsibilities as required and sought by constituent districts and the public.MOH is unlikely to play the sort of effective leadership and facilitation role which is needed withoutsubstantial changes in its culture and orientation. It is not clear what mechanisms could be used toaccomplish this change in world view--CHN3's experience suggests that self-managed, incremental change,i.e., allocating funds directly to central programs, would not work. Instead, the answer may lie in reducingthe MOH budget even further than has taken place with decentralization, transferring the funds to provincebased health councils representing constituent districts. This approach underpins the recently approved,First and Second Provincial Health Projects (Credit No. 338 I-IND and Credit No. 3357-IND, Loan No.4207-IND). For example, responsibility for health education together with resources is assigned toprovinces in these operations, while monitoring and evaluation is contracted out, with procurement handledby the central PMU on behalf of the provinces.

What about replication to non-participating provinces? A large proactive central component wasCHN3's initial, and not fully successful response to this question. Other mechanisms outside of MOH canbe developed, at lower cost. For instance, arrangements could be worked out for regional universities,professional associations, networks of provinces, NGOs, and/or consultant firms to handle differentbackstopping, information sharing, quality control, and cost control functions.

CHN3 also generated program-specific findings, some which have already been applied within theIndonesian setting. For instance, the project provided specific lessons, incorporated in the follow up IodineDeficiency Disease (Loan No. 4125-IND) and Safe Motherhood (Loan 4207-IND) projects, relating to:

- 25 -

Iodine Deficiency Disease:

* The national IDD strategy should differentiate responses according to local situations;

* It is essential to recognize the role of private sector producers, wholesalers, and retailers in thesalt industry and the importance of improving efficiency in salt production and processing;

* Updated information is needed on both the IDD status of the population in different regionsand the iodine content of salt consumed at the household level.

Midwives:

* GOI's village midwives initiative needs to address both supply and demand factors, and to belocally grounded and focused;

* Upgrading the skills of midwives requires strengthening of pre-service as well as in-servicetraining;

* Improving effectiveness will also require intensified supervision and support, directedespecially to the more recent graduates;

* The village midwife's role needs to be widened to a number of other community level healthservices;

* Sustaining the village midwives initiative will require innovations in the terms and conditionsof their employment to take account of diverse work settings and to ensure that activitiesevolve in a manner that fits local needs.

CHN3's lessons of broader applicability pertain to:

Project Design

- Multiple components at central unit and implementation in 5 provinces made the projectcomplex and difficult to be supervised. It is crucial to prioritize objectives in order to developappropriate intervention strategies and performance indicators.

* Human resource capacity at central and provincial/district level should be carefully evaluatedprior to the project implementation. The project objective should accord with the capacity ofthe government particularly the implementing units.

* Supervision of a project involving multi-year systems development should be frequent andrigorous. Potential problems should be reported and effectively addressed.

Decentralization

* A high level of government commitment is crucial for success in any major reform program.This commitment is needed not only from central government officials but also those at theprovincial and district level.

* Reform would work best only if each program unit at central and provincial and district levelhad a clear perception of its new role and the capacity to implement it.

- 26 -

Capacity building

* Developing a strategy which responds to local conditions is more likely to be effective andsustainable.

* The results of studies and PDAs should be disseminated at all levels, so they can be used toimprove program strategies and for replication to other provinces and districts. Utilization ofdata for program planning and management should be incorporated into a system.

* Fellowships should be integrated into a comprehensive human resources development plan inorder to have benefit both recipients and the program. Staff from the eastern provinces shouldhave greater access to fellowships.

* Strong management and coordination are needed to develop a workable health informationsystem, suitable for province and district needs.

* Organizational arrangements for health education should be sufficiently flexible, cross-sectoraland responsive to community need. General management, budgeting, planning, staffmanagement and accountability requirements should not be overly bureaucratic and shouldfocus on specific objectives.

* Management of logistics and the availability of spare-parts locally should be integrated into an

equipment procurement package.

Monitoring and Evaluation

* Poor monitoring and evaluation of project performance resulted in difficulties in assessing theeffectiveness of project interventions.

* Baseline performance indicators are essential to evaluate achievement of project objectives.

9. Partner Comments

(a) Borrowver/implementing agency:

Please see Annex 8.

(b) Cofinanciers:

Summary of KfW's Procurement Evaluation:

Upon completion of the CHN3 project evaluation in September 1999, KfW intended to evaluate thecondition and utiliziation of KfW funded equipment in four CHN3 provinces excluding Maluku. Thelessons learned and recommendations were:

* As has been anticipated earlier, this evaluation was overwhelmed by the process of tracing therecords that would ascertain the source of funding. It was mostly constrained by thenon-functioning logistic system, if there were any.

* The KfW equipment has been felt very useful, particularly those provided to the trainingcenters and referral hospitals. Many of the refrigerators were beneficial for remote sub healthcenters in NTT. Unlike the health center, the hospital has budget allocation for operation andmaintenance. They also have in-house repair unit for simple problems.

- 27 -

* The quality of equipment that were delivered to health centers required special attention,especially because there were no maintenance and repair budget. Only a few health officialsand direct users showed the sense of ownership or felt responsible to maintain the equipments.Part of the reasons could be that many might not need the given equipment, did not know howto operate and did not have the budget for maintenance.

* Equipment provision goes hand in hand with program policy as it supports program operation.The skills and confidence to operate the equipments have to be taken into account besides thenumber of referred cases, the types of cases, the local policies and location of the healthcenters-with bed with regard to patient demands for obstetric emergency referral. Those weresome of the issues that should be evaluated separately under the program not equipmentevaluation.

* Given the scarce resources it would be wise for future procurement to proceed withpreliminary needs assessment.

* To optimize the use of available equipment, relocation of unused equipment to other neededfacilities within the same administrative boundaries will increase efficiencies and improveprogram effectiveness, particularly in the era of decentralization. The new organizationstructure that brings basic health care and referral services under one roof in the DirectorateGeneral for Medical Services should enable the inclusion of hospital in this system.

* While waiting for improvements of the logistic management system, the government mayinstruct 1) one-door policy for incoming and outgoing goods and 2) stick appropriate label onequipment indicating the source of funding.

(c) Other partners (NGOs/private sector):

Not applicable.

10. Additional Information

- 28 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Annex 1. Key Performance Indicators

Selected Indicators 1/ West Java 2/ Central Java NIT Maluku Irian JayaAppraisal Latest Appraisal Latest Appraisal Latest Appraisal Latest Appraisal Latest

Population (million) 35.4 42.4 28.5 30.8 3.3 3.8 1.9 2.2 1.7 2.1Populabon density 765.0 980.0 834.0 954.0 68.0 81.0 25.0 29.0 4.0 5.0

Infant mortality rate 88.4 60.6 51.1 45.2 70.6 59.7 68.0 29.5 61.3 64.7Total fertility rate 3.2 3.0 2.8 2,6 3.9 3.4 3.7 3.3 3.2 3.3Prevalence of underfive underweight(%) 34.0 21.4 34.4 21.3 46.4 33.6 38.6 26.0 29.5 30.1Infant birth weight (>2.5 kg, %) 91.4 89.6 95.9 93.5 86.9 86.9 93.4 93.6 89.9 92.1Children with DPTI vaccinabon (%) 3/ 96.0 100.2 95.0 102.0 89.0 96.3 55.0 81.3 76.0 98.1Pregnant women with TT2 (%) 4/ 70.0 83.2 63.0 86.0 41.0 63.2 20.0 55.2 34.0 41.4Women with prenatal care (%) 81.9 86.1 90.2 93.8 69.1 85.4 65.1 67.4 75.5 87.2Birth attended by medical personnel (%) 27.0 59.3 27.2 64.9 16.8 55.9 22.2 35.6 26.0 34.7Health centers (n) 728 1112 730 852 151 210 92 161 110 200Health centers with beds (n) 87 150 123 181 33 56 22 48 37 74Populaton: health center 48,599 38,156 39,068 36,103 21,656 18,143 20,217 13,665 15,000 10,500Physicians (n) 2,205 4,377 2,336 2,651 164 237 158 286 167 167Population: Physician 16,045 9,694 12,210 11,605 19,931 16,082 11,779 7,704 9,888 12,594Midwives (n) 1,452 8,995 2,771 9,134 321 2,173 273 1,900 132 2,711Population: Midwive 24,365 4,717 10,293 3.368 10,183 1,158 12,530 775

Notes:1/ Source: IDHS 1993 and 1997, Susenas 1992, 2000, Health Profile 1992/1993 and 20002/ Includes Banten3/ DPT1: first dose of diphteria, pertussis, tetanus (DPT) vaccination4/ TT2: two doses (or more) of tetanus toxoid vaccinations

- 29 -

Annex 2. Project Costs and Financing

Annex 2a. Project Cost by Components (in US$ million equivalent)

Compo tActualLatest Percentage ofComponent Appraisal Estimate Estimate Apraisal

A. Provincial Service Delivery and Capacity Building1. Survice DelicerySafe Motherhood 17.9 12.0 67.0Child survival 16.4 8.0 48.8Improving nutrition status 6.5 4.2 64.6Health education 5.6 3.8 67.92. Capacity BuildingPolicy and program development 5.6 4.4 78.6Project organization & management 7.8 5.9 75.6Staff development 10.9 9.6 88.1

Sub-total 70.7 47.9 67.8

B. Central Support Services and Ca acity Building1. Policy Development, Planning andProject Coordination 14.6 11.3 77.42. Technical services for provinces 28.6 11.9 41.63. Interprovincial programs 7.1 4.0 56.34. Capacity building 13.3 9.1 68.4

Sub-total 63.6 36.3 57.1

Overall total 134.3 84.2 62.7

Notes:

1/ KfW grant (Co-financier) is not included2/ Actual/latest estimate of Bank's disburesement as of 76/DJA (November 2001)

- 30 -

Annex 2b. Project Cost by Procurement Arrangements (in US$ million equivalent)

Project Costs by Procurement Arrangements (in US$ million equivalent)

Expenditure Procurement Method 1/ Procurement Method 1/Appraisal Estimate Actual/Latest Estimate

ICB NCB Other 2! NBF 3/ Total ICB NCB Other 2/ NBF 3/ Total1 Works 0.0 27.4 2.0 0.0 29.4 0.0 14.9 0.0 0.0 14.9

_ _________________ (0.0) (10.8) (1.0) (0.0) (11.8) (0.0) (13.6) (0.0) (0.0) (13.6)2 Goods 0.0 6.6 4.5 0.0 11.1 0.0 8.4 0.0 17.3 25.7

________________ (0.0) (5.4) (3.0) (0.0) (11.4) (° °) (6.9) (0.0) (0.0) (6.9)3 Services 4/ 0.0 0.0 22.9 0.0 22.9 0.0 0.0 15.1 0.0 15.1

(O 0) (0.0 (22.9) (0.0) (22.9) (0.0) (0.0) (15.1) (0.0) (15.1)4 Fellowships 0.0 0.0 22.1 0.0 22.1 0.0 0.0 16.5 0.0 16.5

(0.0) (0.0) (0.0) j9.0 (0.0) (16.5) (0.0) (16.5)5 In-country training & 0.0 0.0 28.2 0.0 28.2 0.0 0.0 16.9 0.0 16.9

workshop (0. ) (0.0) (19.9) (0.0) (19.9) (° °) (0.0) (15.4) (0.0) (15.4)6 Project Management 0.0 0.0 5.4 0.0 5.4 0.0 0.0 1.4 0.0 1.4

________________ (0.0) (0.0) (5.4) (0.0) (5.4) (° °) (0.0) (1.4) (0.0) (1.4)7 Project Development 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.0 2.8

Activities (0.0) (0.0) (0.0) (0.0) (0.0) (° °) (0.0) (2.8) (0.0) (2.8)8 Recurrent cost 0.0 0.0 0.0 13.7 13.7 0.5 0.0 0.0 1.5 2.0

(0.0) (0.0) (0.0) (0.0) (0.0) (0.5) (0.0) (0.0) (0.0) (0.5)Total 0.0 34.0 85.1 13.7 132.8 0.5 23.3 52.7 18.8 95.3

(3.0) (16.2) (74.3) (0.0) (93.5) (0.5) (21.0) (50.7 0.0 (72.2)

Notes:1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Included international and national shopping, selection of consultants following3/ NBF= Not Bank Financed (includes GOI and Co-financierlKfW grant)4/ Services include consultants, reseach and study

- 31 -

Annex 2c:Project Financing by Component (in US$ million equivalent)

I ~~~~~~~Percentage of AppraisalComponent Appraisal Estimateni Atual/Latest Estimate

Bank Govt. CoF. Bank Gov Co F. Bank Govt CoF.

1. Service Delivery 14.70 3.20 8.90 3.10 9.70 60.5 96.9Safe MotherhoodChild Survival 10.90 5.50 6.30 1.70 2.80 57.8 30.9

Improving Nutrition Status 4.20 2.30 3.60 0.60 0.20 85.7 26.1

Health Education 3.80 1.80 2.90 0.90 1.40 76.3 50.0

2. Capacity Bldg; Policy & 5.60 0.00 4.30 0.10 2.00 76.8 0.0Program DevelopmentProject Organization and 7.80 0.00 5.60 0.30 0.00 71.8 0.0ManagementStaff Development 10.90 0.00 9.40 0.20 0.00 86.2 0.0

B. Central Suppot andServices and CapacityBuilding;1. Policy Development, 12.80 1.80 10.90 0.40 0.00 85.2 22.2

Planning and ProjectCoordination2. Technical Services for 8.90 19.70 7.90 4.00 0.00 88.8 20.3Provinces3.Inter provincial 4.70 2.40 3.60 0.40 0.00 76.6 16.7Programs4. Capacity Building 9.20 4.10 8.80 0.30 0.00 95.7 7.3TOTAL 93.50 40.80 18.50 72.20 12.00 16.10 77.2 29.4 87.0

Notes:1. Break-down information of co-financier (KfW grant) at appraisal is not available2. Actual/latest estimate of co-financiaer (KfW grant) as of August 20013. Actual/latest estimate of Bank's disbursement as of 76/DJA (November 2001)

- 32 -

Annex 2d. Paroject Financing by Province (in US$ million equivlent)

Actual/Latest Estimate (US$ mili on)Bank Government Co-financier Total

A Provincial Service Delivery andCapacity BuildingWest Java 5.5 1.2 1.8 8.5Central Java 7.3 1.2 2.0 10.5NTT 6.1 1.1 3.6 10.8Maluku 6.5 0.8 4.2 11.5Irian Jaya 15.6 2.6 4.5 22.7

Sub-total 41.0 6.9 16.1 64.0

B Central Support Services and 31.2 5.1 0.0 36.3Capacity Building

Overall total 72.2 12.0 16.1 100.3

- 33 -

Annex 3. Economic Costs and Benefits

Not Applicable

- 34 -

Annex 4. Bank Inputs

(a) Missions:

Stage of Project Cycle No. of Persons and Specialty Performance Rating(e.g. 2 Economists, I FMS, etc.) Implementation Development

Month/Year Count Specialty Progress Objective

Identification/PreparationJune/July, 1991 7 1 Task Team Leader, S S

I Principal Economist,I Population Specialist,I Public Health/NutritionSpecialist,I Operations Officer,I Public Health Specialist,I Health Education Specialist

Appraisal/NegotiationJune/July, 1992 9 1 Task Team Leader, S S

I Principal Economist,I Population Specialist,I Senior ImplementationAdvisor,I Operations Officer, IOperations Assistant,I Public Health & NutritionSpecialist,I Public Health and MCHSpecialist,I Health EducationSpecialist

SupervisionJune 2, 1993 6 1 Health Specialist,

1 Population Specialist,1 Procurement Specialist,I Nutrition Specialist,I Population & HealthSpecialist,I Health & EducationSpecialist

November 8, 1993 6 1 Operation Analyst,I Reprodutive Health Epidemiol,I Population Specialist,I Public Health/NutritionSpecialist,I Health Specialist,I Health Education Specialist

June 12, 1994 3 1 Operations Officer, S S

I Public Health & NutritionSpecialist,I Health Specialist

- 35 -

December 9, 1994 3 1 Operations Officer, S S2 Health Specialists

June 30, 1995 5 2 Human Resources Specialists, I S SImplementation Specialist,I Anthropologist,I Economist

June 29, 1996 4 1 Anthropologist, S SI Operations Officer,I Consultant,I Principal Economist

September 13, 1 1 Senior Health Specialist S S1996

February 6, 1997 3 1 Senior Health Specialist, S SI Anthropologist,I Medical Antrhopologist

August 28, 1997 2 1 Operations Officer, $ SI Senior Health Specialist

June 29, 1996 3 1 Senior Health Specialist, S S2 Operations Officers

April 23, 1999 3 2 Consultants, S SI Operations Officer

December 9, 1999 3 1 Health Economist, S SI Operations Officer,I Anthropologist

April 5, 2000 1 1 Operations Officer S S

ICRAugust 31, 2001 3 1 Operations Officer, S S

I Health InformationSpecialist,I Procurement Specialist

Notes: 1/ Information at identification/preparation is not available2/ Performance ratings for supervision in 1993 are not available

(b) Staff

Stage of Project Cycle Actual/Latest Estimate

No. Staff weeks US$ ('000)Identification/Preparation 614.7Appraisal/Negotiation 172.5Supervision 600.9ICR 13.6Total 1 ,411.7

- 36 -

Annex.5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substanhal, M=Modest, N=Negligible, NA=Not Applicable)

RatingL Macro policies O H OSUOM ON *NAE Sector Policies O H *SUOM O N O NAF Physical O H OSU*M O N O NAO Financial OH OSUOM O N * NA2 Institutional Development 0 H O SU O M 0 N 0 NAO Environmental O H OSUOM O N * NA

SocialLII Poverty Reduction O H OSUOM O N * NALI Gender OH OSUOM ON *NALIO Other (Please specify) O H OSUOM O N O NA

Oi Private sector development 0 H O SU O M 0 N * NAE Public sector management 0 H O SU O M 0 N 0 NALiOther (Please specify) O H OSUOM O N O NA

- 37 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

N Lending OHS OS OU OHUM Supervision OHS OS OU O HUX Overall OHS OS O U O HU

6.2 Borrowerperformance Rating

X Preparation OHS OS OU O HUZ Government implementation performance O HS OS O U 0 HUZ Implementation agency performance OHS OS OU O HU

F Overall OHS OS 0 U O HU

- 38 -

Annex 7. List of Supporting Documents

1. World Bank. Staff Appraisal Report: Third Community Health and Nutrition. Population and HumanResources Division. EAP Region. 1992.

2. Loan Agreement for CHN3 between the Republic of Indonesia and International Bank for Reconstruction andDevelopment. January 21, 1993.

3. World Bank. Project Completion Report: Second Nutrition and Community Health Project (Loan 2636-IND).Population and Human Resources Division. EAP Region. 1993.

4. World Bank. Aide Memoires: Supervision Missions. 1993-2000.

5. Mid Term Review for the Third Community Health and Nutrition (CHN3) project. Report. CEBU, GadjahMada University. 1996.

6. RA Calkins. Project Quality at Entry. Indonesia Discussion Paper Series No 3.

7. Indonesia Demographic and Health Survey (IDHS). Central Bureau of Statistics, National Family PlanningCoordinating Board, Ministry of Health and Demographic and Health Surveys - Macro International Inc.Calverton, Maryland. 1991 and 1997

8. Evaluation on the deployment of midwives in Irian Jaya. Provincial Health Office and YPMD Irian Jaya. 1997.

9. UNICEF. The State of the World's Children 1997. UNICEF. New York. NY. 1997.

10. AED. Health Education Component Evaluation Report. Academy for educational Development Inc. 1998

11. AIHI. Health Education Component Evaluation. Project Completion Report. Australian International HealthInstitute. 1999.

12. Cargill LC. Bidan C Program in Irian Jaya. Consultant Report. 1999.

13. World Bank. Guidelines for Preparing Implementation Completion Reports (ICRs). 1999.

14. P Daly and W Soerojo. Summary of Final Evaluation Activities for CHN3 Project. 1999.

15. Final Evaluation of Higher Education Component of CHN3 project. Diponegoro University. 1999.

16. W Soerojo, S Hartini and S Soetarjo. Summary of PDAs results. CHN3 CPMU. 2000.

17. Longitudinal Surveillance Reports. LPKGM Purworedjo. 1997-2000.

18. Satoto. Evaluation of Implementation of Fellowship Activities of CHN3 Project. Diponegoro University. 2000.

19. P Daly. Best Practices of the Third Community Health and Nutrition. 2000

20. CHN3 Project Evaluation in Irian Jaya. Provincial Health Office and CHN3 CPMU. Report. 2000.

21. CHN3 Project Evaluation in Nusa Tenggara Timur. Provincial Health Office and CHN3 CPMU. 2000

22. CHN3 Project Evaluation in Jawa Barat. Provincial Health Office and CHN3 CPMU. 2000

23. CHN3 Project Evaluation in Jawa Tengah. Provincial Health Office and CHN3 CPMU. 2000

24. CHN3 Project Evaluation in Maluku. Provincial Health Office and CHN3 CPMU. Report. 2001.

25. Physical Achievement of CHN3 Project. Final Report. CHN3 CPMU. 2001.

26. Summary: Health Information System Evaluation of CHN3 Project. SPH University of Indonesia. 2000.

27. S Prasetyo. Implementation Completion Report on Health Information System. Consultant Report. 2001.

28. UNICEF. The State of the World's Children 2001. UNICEF. New York. NY. 2001.

- 39 -

Additional Annex 8. Borrower's Evaluation

Description of the project:

Name of the project Third Community Health and Nutrition

Loan Number IBRD LN.3550 INDKfW Grant 93.65.321

Date of Agreement IBRD Loan, January 21, 1993KfW Grant, October 27, 1993

Total Loan and Grant Original IBRD Loan USD. 93.5 millionAmended Loan as of August 2000: USD. 73.8 millionKfW Grant DM. 30 million

Effective Date IBRD Loan July 30, 1993KfW Grant October 27, 1993

Date of Disposition Fund IBRD Loan: August 31, 1993KfW Grant July 1, 1994

Closing Date IBRD Loan March 31, 2001KfVW Grant : December 31, 2001

(i) The project objective, design, and implementation.

The CHN-3 Project was a large project in term of budget as well as scope. The goal and projectobjectives are to elevate the infant, child and matemal health status through the improvements in theeffectiveness of safe motherhood, child survival, and nutrition program in the provinces of Irian Jaya,Maluku, Nusa Tenggara Timur, Central Java, and West Java. The project supported provincial and localhealth officials in these provinces to prepare and implement policies and program interventions to addresshigh priority local community health and nutrition programs. This was achieved by 1) improving servicedelivery in provinces and building provincial capacity to plan, implement and monitor basic health serviceprogram, and 2) strengthening the capacity of the central level of the Ministry of Health, primarily theDirectorate General of Community Health to support these provincial and district efforts.

At the provincial level, the project focused on improving service delivery and strengtheningcapacity through improvement of infrastructure, staff development, and improvement of technical skills ofhealth planning, epidemiology, management information system, health education, research, management,and monitoring of health program. The supporting system for CHN-3 project were: Health InformationSystem, Health and Nutrition Education, and Health Manpower.

The Budget allocation by category have been amended six times:

First amendment issued on November 11, 1994 due to additional category namely "ProgramDevelopment Activity (PDA)" 100 % paid by loan, this is for innovative activities proposed by districts.Second amendment issued on May 23, 1997 due to reprogramming after Midterm Review. Thirdamendment on October 12, 1998 due to the monetary crisis the loan was canceled amount to USD. 16million. Fourth amendment issued on September 20, 1998 because of reallocation of category where the

-40 -

budget allocation for category 8 (PDA) was increased due to overcome the health crisis and consequencethe project was extended up to March 31, 2001. Fifth and six amendment issued in December 1999 andAugust 2000 were to reduce the loan.

The total amended loan amount to USD.73,800,000.

The total disbursement as of August 2, 2001 amount to USD. 71,818,788.05 (up to 75/DJA) or97.1 %

The budget allocation by Category are as follows:

Category Original Loan Last Amendment(USD) (USD)

1. Civil Work 10,800,000 13,650,0002. Equipment 6,900,000 4,700,0003. Instructional Material 2,400,000 2650,0004. Consultant and Study 22,900,000 15,100,0005. Fellowship 22,100,000 16,500,0006. Workshop and Local Training 19,900,000 15,450,0007. Project Management 5,400,000 1,450,0008. PDA

A. PDA Equipment 0 750,000B. PDA Study, Workshop 0 3,550,000Unallocated= 3,100,000 0Total Cancellation 1+2+3 _ 9350000 _ 19,700,000

TOTAL 93,500,000 93,500,000

(ii) Performance during the evolution and implementation of the project.

The project implementation consist of two part: Provincial and Central Project.

A. Provincial Project

West Java

1. Service Delivery

a) To enhance the Safe Motherhood Program, the project increase the coverage and the quality ofAntenatal Care through the provision of medical equipment for Health Center and Referral Hospital. Thoseequipment mostly funded by KfW. Providing equipment for Health Center, posyandu, district referralhospital will improve the quality of health services. The other activities were training for 1,384 femalenurses/village midwives in Life Saving Skill, Obstetric Service and Basic Essential Neonatal, Obstetric andNeonatal Emergency Services, Supervision to TBA, workshop on data validation of Mother Cohort,Develop the new module of Mother and Child Health Awareness and Cader Training

To increase the quality of referral system the project conducted the training on Obstetric andNeonatal Emergency Services for midwife and medical doctor of Health Center with bed and districthospital, Audit Matemal and Perinatal, on the job training for midwife at district hospital, develop and

- 41 -

disseminate information of Standard Operating Procedure for village midwife.

Strengthening the cross sectoral collaboration through workshop on Community Based MaternityCare.

b) West Java conducted some activities for 1) strengthening Child Survival Program throughtraining on Diarrhea and Acute Respiratory Infection (ARI) for medical doctor, paramedical, and cader,development of cross sectoral working group for Diarrhea and ARI 2) accelerating the coverage of UCIthrough the provision of immunization equipment, training for immunization personnel, and meetings. 3)Improving the neonatal health through training on neonatal health especially for neonatal tetanus and lowbirth weight for medical doctor, paramedical and TBA. 4) Surveillance morbidity through training onsurveillance for health personnel at district and Health Center.

c) Some activities for improving nutrition status West Java province conducted introduction forfermented soya (tempe) formula, Rapid Survey for malnutrition pregnant women, and campaign of balancediet. The studies on IDD, anemia, and malnutrition were conducted as a basic for further programdevelopment.

d) To support Health Education the project provided HE equipment at provincial level as well asdistrict level, conducted training for health personnel, community leader, female religion leader and womenworkers.

e) The component of Health Information System, the project carried out rapid survey to getspecific data for district, develop Local Area Network, and tried to simplify the Reporting RecordingSystem of Health Center (SP3).

2. Capacity Building

a) Staff Development: the project gave fellowships for Dl in Nutrition, SI, S2 short term incountry and overseas, for the total of 151 persons. The fellowships gave improvement health manpowerquality. The provision of educational equipment improve the skill and quality of the graduates,The recruitment of local short term consultant for West Java totaling of 30 MM and Internationalconsultant 21 MM, especially in the field of management, and Health Education could help the province indeveloping program management.

b) Policy and Program Development. Seven studies and 17 PDAs have been carried out The PDAactivities were to 1) improve the capability of the program holder in developing and designing programactivities 2) improve the coordination between program and education/research institution. 3) revise andimprove the on going activities base on PDA findings such as Infant Cohort, Life Saving Skill (LSS)training, Posyandu Supervision System.

Central Java

1. Service Delivery

a) The main activities for Safe motherhood were: 1) improve the quality of health services throughAudit Maternal Perinatal, develop emergency unit at Health Center with bed, provided Health Centerequipment, give training on high risk mother, medical emergency, Life Saving Skill, and on the job trainingin the hospital for village midwife.

- 42 -

2. To improve the program management by training on MCH program, and cohort analysis, Local areamonitoring of MCH program, meetings province and districts staff on monitoring and evaluation of villagemidwife, LSS, and maternity hut.

3. Developing MCH program by publishing MCH book and its training, monitoring the use of MCH book,training on Community Nursing for Maternal and Infant in 7 districts.

4. Improve the community participation through guidance and supervision to TBA, provision of TBA kit(basic drugs, and umbilical cord package). Social marketing for Mothers Friendly Movement (GerakanSayang Ibu) at all level of administration of MCH programAfter the completion of CHN-3 project the follow up activities of Safe Motherhood are funded by the localgovernment.

b) Some activities for Child Survival were:1) improve the quality of health services by the provision of immunization equipment, training on high riskneonatal case finding for midwife and HC medical doctor.2) improve the community participation by cader training on malaria, diarrhea, ARI management,neonatorum tetanus elimination.3) Campaign healthy house in preventing ARI in mountainous area by exhibition of the model of housewith chimney.

c) The program for nutrition were: 1) Program development through training on balance diet,nutrition status monitoring and nutrition surveillance at province and district level. Develop a nutrition unitat health center with bed.2) improve the community participation by social marketing of severe malnutrition management, andchronic energy deficiency.

d) The main benefit of Health Education component was the provision of HE mobile unit and HEequipment which was able to support the successful of the community health education. The otheractivities were:1) improve the quality of health education by developing specific media package based on the local need,and training for improving skill of health educators.2) improve the program management by training on social marketing and intersectoral meeting on Healthand Clean Behavior. The result of short course on health promotion, and the provision of multimediacomputer to each district enable the HE staff to create media package based on their own needs Aftercompletion of CHN-3 project the operational cost will be taken over by the local govemment.

2. Capacity Building

a) Policy and Program Development

1) Improve the quality services by designing waste disposal of HC with bed, rehabilitation of HClaboratorium, and septic tank, development of HC quality assurance2) Strengthening Health Information System by constructing Health Information building, providing ofLAN for SP3, renovating of computer room at province and districts. Develop Provincial and districthealth profile books, health problem area mapping. Rapid Survey and Rapid Evolution methods, trainingon situation analysis and evolution of maternal and infant health program. The benefit of developingInformation System is getting the accurate data, on time, and suitable for the required information. Theoperation cost funded by local government.

- 43 -

The HIS now is expanded to other remaining districts funded by other resources.3) Collaboration with Higher Education through technical assistance, gave the benefit in managing districthealth program, and now is being continued by other project for other districts.4) Conducted 7 studies on MCH, IDD, KAP, midwife role and 8 PDAs mostly on maternal health.The result of some PDAs bring about the midwives skill improvement in communication, more acceptableby the community, and improve the role of religious leader in health community education.

b) Staff Development

Central Java gave fellowships for DI and D3 program on nutrition, nursing and statistics, S1, S2,short course in country and overseas for the total of 1.333 persons. This fellowships gave direct impact forthe staff capacity in term of decentralization preparation. After having reviewed the performance of staffget in the fellowships, the high level manager of Provincial Health Office assured that the fellowship gavebenefit especially in program planning, then Central Java decided to sent more staff to continue theireducation by taking fellowships from other project.

East Nusa Tenggara (NTT)

1. Service Delivery

Number of activities for Safe Motherhood program were: 1) partnership between village midwifeand TBA, Empowering Polindes, Audit Matemal and Perinatal, improving skill in Emergency ObstetricNeonatal Services, Friendly Mother and Infant Movement and also provision of medical equipment such asEmergency Obstetric Neonatal especially for HC with beds, provision of neonatal, perinatal Midwifery andTBA kit, gave contribution in increasing the coverage of MCH program within the period of 5 years project(1995-1999). The birth delivery by health personnel increased from 33 to 56 %,2) complete antenatal care (at least 4 visits to HC) from 44 to 62 %3) total pregnant women get complete iron tablet from 37 to 65 %,4) number village midwife which still stay in the village at least 3 years from 87 to 98 %.

The project activities for Child survival mainly to overcome the problem of diarrhea and AcuteRespiratory Infection. Within 5 years project these activities reduced the diarrhea cases from 35 to 23 %.Other activities are the prevention of infectious diseases through immunization by providing refrigeratorsand vaccine carriers for Health Center, implementing the activity of Integrated Management Child Illness,also Early Detection of Growth and Development of under 5 children.

For the Nutrition program the number of under 5 children possessed the Health Chart ("KMS")increased from 87 to 95 %, and the severe malnutrition reduced from 12 to 4.9 % within the period of 5years (1995-2000). The main useful activities of Health Education was intervention for Health and CleanBehavior Pattem, and community education based on local culture.

2. Capacity Building

As a program for Human Resource Development, the province of NTT has been improving theknowledge and skill of health staff by giving them for continuing education. To date there were 87 incountry fellows consist of 73 persons S1 degree and 14 persons S2 degree, overseas S2 degree totaling 6persons and 23 persons for overseas short courses. Some of them hold the strategic position in provincialHealth Office, Training and Education Institution. Develop Integrated Work Pattern (Team Building) has

- 44 -

successfully implemented in Health Centers and Referral Hospital.

Maluku

The gugus pulau health system paradigm depicted aims to have an impact in reducing morbidityand mortality. During the project, the province of Maluku has procured medical equipment, boats, radiomedic communications, instructional materials for emergency obstetric, but most of them was destroyedduring the riots in Maluku. The building for Training Center, Multi Stream Academy, Data Room are stillexist as well as educational equipment. Since 1998 there is no activity reports from Maluku.

Irian Jaya

1. Service Delivery

As a whole, the project gave significant contribution for Safe Motherhood program by increasingthe coverage of health service delivery and health service facilities among others by renovation 431polindes, providing equipments, and training on MCH.Within 5 years of the project the coverage birth delivery by health personnel increased from 24% to 50 %,the coverage of Ante Natal Care increased from 30 to 50 %, number of HC have the ability for ObstetricNeonatal Basic Essential Services (PONED) increased from 0 to 68 %. Number of Medical Doctor andMidwife with Life Saving Skill training increased from 0 to 1,345 persons, the total number of districtconducted Audit Matemal Perinatal increase from 0 to 10 districts out of 12 districts. Total number oftrained staff on Safe Motherhood program were 1,218 persons

The number of staff trained in diarrhea increased, but did not significantly influenced theprevalence of diarrhea. Total number trained staff on Child Survival were 280 persons. Nutrition programhave not given any significant impact on improving community nutrition, this is due to the geographicalsituation, and many of villages could not covered by nutrition program due to the remoteness. Totalnumber trained staff on Nutrition were 222 persons. For health education program this project providedpackages of media education through the development of Media Production Center, but the qualitativeoutcome have not been measured by the end of this project. Total number trained staff on HealthEducation were 60 persons.

2. Capacity Building

To improve the quality of health staff for strengthening the program development, the projectprovided fellowships for short courses, D-3, S-l, S-2, S-3 program, in-country as well as overseas. Thetotal number of health staff awarded fellowship were 2.763 persons. Now, some of the S-2 fellows wereposted in strategic position.

To achieve the equity of basic health services, Irian Jaya provide midwife education for type A,and C. The total number of midwives type A were 2,477 persons, and type C were 3,244 persons. Themidwives type C were posted in the villages, and help a lot of people in the village for basic health serviceinstead of mother and child care. Based on the community demand in the village, Irian Jaya throughProgram Development Activity develop a program for training to give additional knowledge and skill inPrimary Health Care and Standard Diagnosis and Therapy of Common Illness.

In relating with strengthening the input component for manpower development, the project builtphysical facilities and its supporting facilities for 7 (seven) Nursing Schools, I (one) Multistream

- 45 -

Academy, 1 (one) Training Center. Total number trained staff on Capacity Building were 142 persons.A series of studies and Program Development Activities gave contribution to health management planningand health services.

B. Central Project

1. Directorate of Family Health

The Role of Directorate of Family Health was: intensification and extensification Mother and ChildActivities, improve the cross sectoral and cross program coordinator to get support to overcome the directcause of maternal death. The other activities were to facilitate and supervise the development of provincialMCH program. The development of Integrated Management Child Illness (IMCI) now become a nationalmodel

2. Directorate of Health Center Promotion

This Directorate has successfully improve the health service and referral system through radiomedic communication system especially in NTT, Maluku and Irian Jaya provinces.

3. Center for Health Education

Provision of Health Education Mobile units for districts are very useful for Health Educationoperation in the field. The experiences from this activity, then replicated to other province using otherresources.

4. Directorate of Community Participation

This directorate has developed Community Participation Management (ARIFF) and being usenationally.

5. Directorate of Nutrition

This directorate has successfully leading nation wide for Iodine Deficiency Disease through IDDmapping in CHN-3 provinces.

6. Health Data Center

This component has tried to simplify the Health Center Recording and Reporting System (fromSP2TP to be SP3) in West Java and Central Java, but the result was not fully acceptable by program units.The other activities were developed Rapid Survey Manual and conducted its training for health district levelstaff.

7. Higher Education

The graduates of continuing education had been utilized as a team member of some MOHactivities, and as technical assistants (Bantuan Tehnis) at district level to guide the health staff in problemsolving and health planning.

The development of technical assistant for district were able to motivate the head of District

-46 -

Health Office and the technical staff to continue their education. The Longidinal Surveillance in thedistrict of Purworejo and Belu, were able to improve the capability of the district staff in planning,supervising, evaluating health program based on epidemiology data, and health information system.

In the year of 1999 has been established the Collaboration between University and the localgovernment including health sector. Effort has been made to institutionalize the collaboration by issuingMOU between Ministry of Education, and Ministry of Home Affairs, and developing of CollaborativeBody on Community Food, Nutrition and Health (BKS-PGKM)

The benefit of the project

Safe motherhood

Establishment of Audit Maternal Perinatal (AMP), has been implemented for maternal andperinatal death cause finding. The development of Basic Emergency Obstetric and Neonatal Services(PONED), Standard Neonatal Essential Health Services (PONEK), have been implemented as nationalmodel.

The result of KAP study in Healthy Mother and Healthy Baby Campaign has been used to developHealthy Mother and Healthy Baby Project financed by Australian Grant. Moreover the operational studyon Integrated Management of Child Illness (IMCI) has been used nationally.

The development of MCH book has been successfully used in Central Java and the provision ofMCH equipment were very useful to improve the quality of services for obstetric and neonatal.

Child Survival

The development of big Road to the Health Chard integrated with IMCI and Baby Cohort havebeen successfully implemented in West Java, then continue by ADB project. Unicef supported theimplementation of Baby Cohort in West Java for other non CHN-3 districts. The CHN-3 project providedimmunization equipment (Kerosene Refrigerator, vaccine carrier, syringe and autodestruct syringe) tosupport UCI in project provinces.

Health Education

The provision of Health Education Mobile Unit for districts are very useful for HE operation in thefield. The experiences of these activities has been replicated to other provinces using other donor agenciesbudget.

Nutrition

IDD mapping which can be used to calculate IDD target for IDD prevention intensification hasbeen expanded to 22 other provinces. In addition, computerization of Nutrition Status Monitoring was veryuseful to get rapid and accurate information of nutrition problem in the field.

Health Data Center

Computerization Health Information System was very useful in supporting the implementation ofRecording Reporting System although it is not yet optimal due to the lack of professional manpower and

- 47 -

hardware specifications

Health Manpower

Development of in-house training model for Master Trainer and Training Center staff, has beenreplicated to other provinces financed by HP V. Moreover, development of module in special subject ofnutrition, nursing, and midwifery has been used nationally.

Higher Education

Higher Education has successfully develop and continue partnership between university and localgovernment in food, nutrition, and community health. Develop model of technical assistance from Schoolof Public Health to district level through partnership collaboration which have benefit for both parties. Thelongitudinal Surveillance gave the benefit for district level.

Lessons learned

1. Decentralized planning and management

CHN-3 has been the forefront of promoting decentralized planning and management for health andnutrition services. The capacity of the district level in NTT, Irian Jaya, and Maluku at the start of theproject was much more limited than that in Java. NTT, in particular, give great attention to improvedistrict planning and budgeting capacity and district planners which are now much more able to plan forhealth services based on local needs.With the advent of decentralization, still remains an urgent need to further strengthen district planning andbudgeting skill.

2. Staff development

One of the Objectives of CHN-3 was to strengthen skill at all level of the health and nutritionprogram. In country training courses such as executive training program initiated by CHN-3 project inWest Java and Central Java, and distant learning in NTT, appear to have been particularly successful aswell as appropriate for each environment.

The project financed total of 115 overseas advanced degrees and 880 in country advanced degreesof staffs from the Ministry of Health and National Education. However, there were constrains to thesehigher level degree program such as: 1) overseas training was negatively affected by the insufficientEnglish language skill of many participants. For candidates attending programs overseas appropriatemeasures should be taken to assure that they receive sufficient English language training. 2) the majority ofin country and overseas high level degrees are candidates from the central level or from the provinces ofWest Java and Central Java and not staffs in Eastern Indonesia wheTe the needs are greatest. More effortfrom all parties should be given to identify candidates from Eastern Indonesia for in country higher degrees.

3. Infra Structure Development

The success of the implementation of a services model such as IMCI, PONED, PONEK, InfantCohort, AMP caused by the identification of the community problems, supported by studies and PDA, sothe result were applicable to overcome the health problem and can be replicated nationally.

- 48 -

4. Technical Issues

The project supported a wide variety of discrete health and nutrition activities at the national levelin each province. Quite a number of these innovative activities have been expanded on a much wider scale.A view of the most notable activities are:

1) Integrated Management of the Childhood illness (IMCI) is an innovative initiative in West Javato adopt the WHO model of the integrated approach to the management of childhood illness. This activitywas supported with local government fund and expanded to other provinces.

2) Matemal and Child Health Handbook funding by JICA, evaluated and revised using CHN-3fund and expanded to other provinces.

3) A pilot project to reduce IMR and MMR by enhancing the perfonnance of village midwife andhealth center in three districts in Irian Jaya. This pilot project has been expanded to another district andfinanced by UNICEF.

4) IDD mapping is now being conducted nationwide.

5) Curriculum for multistream Academies for nutrition, nursing and midwifery courses have beenuse nationally.

6) Research surveillance laboratory in Purworejo have generated a wide range of nested researchstudies in priority areas of matemal, child health and nutrition.

7) Various provincial health staff have said that the PDA as program development and policyinformation have been one of the valued parts of the CHN-3 project.

Nevertheless, the capacity to design technically competent studies was widely variable.For some units both at the central and provincial level where there is insufficient planning and researchskill, it was difficult to prepare the proposal. In many cases the quality of these proposals remained lowdespite opportunities for revision.

Project Management and Technical Assistance

1) The Central Project Secretariat was especially important to facilitating communication withinthe program units and provinces and to explain guidelines and procedure for project activities. By allaccount the secretariats were critical for conducting annual planning, preparing achievement reports andproviding administrative support for fellowships, civil work and procurement.

The role of the Project Steering Committee initially was to provide operational guidance to theproject, but because of the time constrains of most of these echelon II officials in the MOH it was notpossible for them to meet on regular basis for CHN-3 matters.

2) One of the more difficult elements of the project for both central level and provincial staff wereprocurement procedures for International Competitive Bidding (ICB). The secretariat took a proactiverole in the procurement process with ICB.

3) Experience in the project has shown than the most value is gained from technical assistancewhere the program unit or province has put in time up front to assure that the terms of reference areappropriate, the candidate is the best qualified to meet their needs, and there is an appropriate counterpartin place.

-49 -

(iii) The performance of the Bank and Cofinancier

The supervision mission conducted twice a year is very useful for the project implementationespecially to overcome project problems. During supervision in 1994 the World Bank mission realized theweakness of Service Delivery implementation, and the mission decided agreed to add one category namelyProgram Development Activity (PDA) with the criteria that PDA proposal would be prepared bykabupaten and or provincial team to meet an identifiable local need, the activity should examine an explicitaltemative or innovative approach to service delivery. The World Bank mission decision is very useful tospeed up the project implementation since the PDA 100 % financing under category 8 of the loan. Themission also agreed to simplify the procedure for getting approval such as the studies, research, and PDA,which is the Bank gave an authority to the Central Project Secretariat to review and approve all those forunder USD 25,000. During the Christmas and New year time usually the approval was delay because mostof the World Bank staff took long holiday.

The KfW grant as cofinancier of CHN-3 project provided Procurement Advisors which is veryuseful for the procurement of equipments. His role as liaison between the project (MOH) and KfW tospeed up the approval of bidding process especially under the International Competitive Bidding. The KfWsupervision missions were very accommodative and helpful to support procurement problems immediately.

Suggestion for future operation of the project

1. Baseline data should be prepared during the preparation of the project proposal.Experience showed, if the baseline data implemented during the beginning of the project it would bedelayed due to the bureaucracy of project implementation.

2. The future project should assure that appropriate support measures including Technical Assistancetraining and manual are in placed if there is to be Intemational Competitive Bidding (ICB).

3. The technical assistance for the provinces for International Consultant should understand BahasaIndonesia, or preferable has an counterpart with domestic consultant, because the province usually has lessexperience in managing technical assistance and sometime failed to provide appropriate support andguidance.

4. The project secretariat should provide technical staff with experience in managing foreign assistantproject and ability in developing of data base for monitoring and evaluating the project achievement.

- 50 -