Document of The World Bank

Report No: ICR00003184

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-42050 TF-56841)

ON A

CREDIT

IN THE AMOUNT OF

SDR46.2 MILLION (U$67.5 EQUIVALENT)

AND A

GRANT

IN THE AMOUNT OF

US$25.3 MILLION

TO THE

REPUBLIC OF INDONESIA

FOR AN

EARLY CHILDHOOD EDUCATION AND DEVELOPMENT PROJECT

June 27, 2014 Human Development Sector/Education Indonesia Country Department East Asia and Pacific Region

i

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 27, 2014)

Currency Unit = IDR IDR 12, 099.21 = US$1 IDR 1.00 = US$0.08265

FISCAL YEAR

January 1 December 31

ABBREVIATIONS AND ACRONYMS

BAN-PT National Accreditation Board for Higher Education

IKIP Teaching Training Institute

BAPPENAS Agency for National Development Planning

IG Inspectorate General

BPS Badan Pusat Statistik or Bureau of Statistics

KB Kelompok Bermain or Playgroup

BSNP Board of National Standards of Education

KDP Decamatan Development Project

CDD Community Driven Development

MDG Millenium Development Goals

CDW Child Development Worker MOEC Ministry of Education and Culture

CT Community Teams MOF Ministry of Finance CPICU Central Project Implementing

and Coordinating Unit NEST National Early Childhood

Specialist Team CSO Civil Society Organization NGO Non-Governmental

Organization or Civil Society and Local Organization

DESP-TF Dutch Education Support Trust Fund-World Bank administered

NMC National Management Consultant

DDTK Child Growth and Development

PAUD Pendidikan Anak Usia Dini or Early Childhood Directorate

DGHE Directorate General of Higher Education

Perda Local Governments Regulation

DGNFE Directorate General of Non Formal Education

POM Project Operational Manual

DGQITEP Directorate General of Quality Improvement of Teacher and Education Personnel

PPCU Provincial Project Coordinating Unit

DIPA Governments Annual Budget P2TK Teachers and Education Personnel Directorate

DPIU District Project Implementing Unit

RA Raudhatul Afthal or Islamic Kindergarten

ii

ECED Early Childhood Education and Development

RENSTRA Medium Term Planning

ECEDP Early Childhood Education and Development Project, the Project

RMC Regional Management Consultant

ECED-TF ECED Netherlands Trust Fund Government administered

RPJM Medium Term Development Plan

EDI Early Development Instrument (School Readiness)

SK Surat Keputusan or Decree

EFA Education for All SUSENAS National Socio Economic Survey

IFR Interim Financial Report TK Taman Kanak-Kanak or Kindergarten

GOI Government of Indonesia Tupoksi Organization roles and function

Vice President: Axel van Trotsenburg Country Director: Rodrigo A. Chaves Sector Manager: Luis Benveniste

Project Team Leader: Rosfita Roesli ICR Team Leader: Rosfita Roesli

ICR Primary Author: Suzana Nagele de Campos Abbott

iii

INDONESIA Early Childhood Education and Development Project

CONTENTS

Data Sheet

A. Basic Information ....................................................................................................... v B. Key Dates ................................................................................................................... v C. Ratings Summary ...................................................................................................... vi D. Sector and Theme Codes .......................................................................................... vi E. Bank Staff ................................................................................................................. vii F. Results Framework Analysis .................................................................................... vii G. Ratings of Project Performance in ISRs .................................................................... x H. Restructuring (if any) ................................................................................................. x I. Disbursement Profile ................................................................................................. xi

1. Project Context, Development Objectives and Design ........................................ 1 2. Key Factors Affecting Implementation and Outcomes ........................................ 6 3. Assessment of Outcomes ................................................................................... 15 4. Assessment of Risk to Development Outcome .................................................. 21 5. Assessment of Bank and Borrower Performance ............................................... 23 6. Lessons Learned ................................................................................................. 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 26 Annex 1. Project Costs and Financing .......................................................................... 27 Annex 2. Outputs by Component ................................................................................. 28 Annex 3. Economic and Financial Analysis ................................................................. 40 Annex 4: Project Impact Evaluation ............................................................................. 42 Annex 5. Bank Lending and Implementation Support/Supervision Processes ............ 62 Annex 6. Beneficiary Survey Results ........................................................................... 64 Annex 7. Stakeholder Workshop Report and Results ................................................... 65 Annex 8. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 66 Annex 9. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 75 Annex 10. List of Supporting Documents .................................................................... 76

MAP

iv

A. Basic Information

Country: Indonesia Project Name: Early Childhood Education and Development Project

Project ID: P089479 L/C/TF Number(s): IDA-42050,TF-538141,TF-56841

ICR Date: 06/16/2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: INDONESIA Original Total Commitment:

XDR 46.20M Disbursed Amount: USD 93.44M2

Revised Amount: USD 92.8M3 Environmental Category: C Implementing Agencies: Ministry of Education and Culture Cofinanciers and Other External Partners: Netherland Minister of Foreign Affairs and Development Cooperation B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/06/2005 Effectiveness: 12/11/2006 12/11/2006

Appraisal: 03/14/2006 Restructuring(s): 05/15/2012 05/15/2013

Approval: 06/27/2006 Mid-term Review: 02/01/2011 08/02/2011 Closing: 12/31/2013 12/31/2013

1 TF 53814 was reported under separate GRM

2 This figure reflects total disbursement under the project: IDA Grant 42050 (USD 70.04M) and Dutch TF 056841 (USD 23.40M). There were exchange rate gains to the SDR:USD exchange rate

3 This figure reflects the total funds made available for the project: IDA Grant 42050 (67.50M) and Dutch TF 056841 (USD 25.30)

v

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory

Overall Borrower Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators

QAG Assessments (if any)4 Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 5 5 Pre-primary education 90 90 Sub-national government administration 5 5

Theme Code (as % of total Bank financing) Child health 17 17 Decentralization 17 17

4 QALP-2 was initiated in late 2009 and completed in July 14, 2010. The rating given for Quality of Design was 2 (Satisfactory/ Likely) and Quality of Supervision was 3 (Moderately Satisfactory/ Likely)

vi

Education for all 33 33 Participation and civic engagement 17 17 Rural services and infrastructure 16 16 E. Bank Staff

Positions At ICR At Approval Vice President: Axel van Trotsenburg Jeffrey Gutman Country Director: Rodrigo A. Chaves Andrew D. Steer Sector Manager: Luis Benveniste Christopher J. Thomas Project Team Leader: Rosfita Roesli Mae Chu Chang ICR Team Leader: Rosfita Roesli ICR Primary Author: Suzana Nagele de Campos Abbott F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the project is to improve poor children's overall development and readiness for further education, within a sustainable quality ECED system. To achieve this objective, the project will: (i) increase poor communities' capacity to engage in participatory planning that will result in new or improved ECED services for their children and families (see Component 1); (ii) prepare the foundation for a sustainable ECED system through budgetary commitments from participating districts, establishment of a national quality assurance and professional development system, and district capacity building (see Component 2); (iii) ensure continuous improvement of service delivery and system building through establishing effective project management, and monitoring and evaluation (see Component 3). Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 :

Increase in Early Development Instrument (EDI) scores of children entering kindergarten or 1st grade of primary school (stated as declines in percent of children vulnerable on each domain). Values are for 4 year old cohort from 2009 and 2013 surveys

Value quantitative or Qualitative)

* Physical: 18.5 * Social: 6.6 * Emotional: 35 * Lang and cognitive:

N/A

* Physical: 7.9 * Social: 8.1 * Emotional: 28.5 * Lang and

vii

88.2 * Communication: 1.1

cognitive: 2.5 * Communication: 1.1

Date achieved 04/01/2009 05/01/2013 Comments (incl. % achievement)

Vulnerability in child development declined especially in physical, emotional, language and cognitive. Social slight increase. No change in communication (already low). Detailed analysis is available in Annex 3.

Indicator 2 : Improvement in children's development. Values are for 1 year old cohort of children from 2009 and 2013 surveys. N/A corresponds to item beginning dropped at endline since cohort outgrew the development measures.

Value quantitative or Qualitative)

Gross motor (out of 20): 13.7 Fine Motor (out of 12): 6.1 Language (out of 5) 2.9, Cognitive (out of 8): 4.4 Socio emotional (out of 2) 1.6 Receptive Language: 37.1 Expressive language: 39.8

N/A

Gross motor (out of 20): 19.9 Fine Motor (out of 12): 11 Language (out of 26): 19.6 Cognitive (out of 8): N/A Socio emotional (out of 2) N/A Receptive Language:N/A Expressive language: N/A

Date achieved 06/01/2009 05/01/2013 Comments (incl. % achievement)

Using child development scores, outcomes improve in all domains and language. 2010 midline data shows increase: cognitive 7, receptive lang 85.5, expressive lang 100. N/A corresponds to item beginning dropped.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of parents/communities who have received information regarding ECED Value (quantitative or Qualitative)

0.0 80.00 99.80

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target exceeded. Community facilitation successfully reached out parents/ communities. Once services are running, they also serve as demonstration to attract more parents.

Indicator 2 : Number of communities submitting an expression of interest to receive grants for ECED service development Value (quantitative 0.0 6000 5990

viii

or Qualitative) Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target 99 percent met. Target not met in a few remote villages

Indicator 3 : Number of block grants awarded to communities Value (quantitative or Qualitative)

0.0 6000 5990

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target 99 percent met. Block grants that were paid in 3 consecutive years, supported village to establish and operate new services. Target not met in a few remote villages.

Indicator 4 : Increased enrollment in ECED services by poor children Value (quantitative or Qualitative)

0.0 738000 673162

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target 91% met. Last monitoring was done in mid-2013. New enrollment was expected starting in Aug 2013.

Indicator 5 : Percentage of targeted districts with regulations supporting ECED positions in district government Value (quantitative or Qualitative)

0.0 80.00 96.10

Date achieved 12/31/2007 12/31/2007 06/17/2013 Comments (incl. % achievement)

Target exceeded by 20 percent. 48 out of 50 districts established permanent ECED position and issued various district and even village regulations supporting ECED with long term sustainability effect.

Indicator 6 : Development of standards and recognition of model ECED centers Value (quantitative or Qualitative)

0.0 36 25

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

National standard for ECED services issued in 2008. Target for establishment of model centers meeting the standard is met in 25 out of 36 target districts (69 percent) due to district financial constraint in providing matching fund.

Indicator 7 : Percentage of targeted districts with qualified and functioning district training teams Value (quantitative or Qualitative)

0.0 80.00 72.20

Date achieved 12/31/2007 12/31/2007 09/12/2013

Comments (incl. % achievement)

Target 90 percent met. All of trainers trained all ECED teachers. However some trainers did not conduct follow-up supervision due to lack of transport support for visiting teachers in remote areas. District trainers continue to serve after project closed.

ix

Indicator 8 : Percentage of villages undertaking annual community assessments Value (quantitative or Qualitative)

0.0 80.00 80.90

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target fully met.

Indicator 9 : Percentage of villages submitting routine reports to districts Value (quantitative or Qualitative)

0.0 80.00 96.00

Date achieved 12/31/2007 12/31/2007 09/12/2013 Comments (incl. % achievement)

Target exceeded. Communication between village (with project supported services) and district improve.

Indicator 10 : Impact study undertaken Value (quantitative or Qualitative)

N/A Impact survey completed Impact survey completed

Date achieved 04/01/2009 05/01/2013 Comments (incl. % achievement)

3 rounds of survey were completed in mid-2013 and preliminary result disseminated

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP Actual

Disbursements (USD millions)

1 09/07/2006 Satisfactory Satisfactory 0.00 2 10/11/2007 Moderately Satisfactory Moderately Satisfactory 0.00 3 12/01/2008 Satisfactory Moderately Satisfactory 8.45

4 06/12/2009 Satisfactory Moderately Unsatisfactory 19.89

5 02/02/2010 Satisfactory Moderately Satisfactory 22.07 6 01/15/2011 Satisfactory Moderately Satisfactory 55.84 7 12/19/2011 Satisfactory Satisfactory 64.75 8 04/03/2012 Satisfactory Satisfactory 69.11 9 12/26/2012 Satisfactory Satisfactory 70.16

10 07/07/2013 Moderately Satisfactory Moderately Satisfactory 70.16 11 12/29/2013 Moderately Satisfactory Moderately Satisfactory 70.23

H. Restructuring (if any)

x

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

05/15/2012 N S S 69.78

Reallocation of Financing Proceeds. To allow more fund for training and workshop under Part A of the Project

05/15/2013 S S 70.16 Extension Closing date to December 31, 2013

I. Disbursement Profile

The graphic represents disbursement profile of IDA Grants only.

xi

1. Project Context, Development Objectives and Design 1. The SDR46.2 million credit for the Early Childhood Education and Development Project (ECED) was approved by the Banks Board on June 27, 2006, signed on September 13, 2006. The project became effective on December 11, 2006 and closed on December 31, 2013. The Project was co-financed by a US$25.3 million grant from the Government of the Netherlands Trust Fund (the ECED-TF), approved by the Bank acting as a Trustee of a Dutch Trust Fund (TF) from the Netherlands Ministries of Foreign Affairs and of Development Cooperation. The Government administered the ECED-TF. The closing date of the Loan and the Grant was December 31, 2013. 1.1 Context at Appraisal. 2. Background. At the time of project preparation in 2006, Indonesia ranked as a lower-middle income country, with a gross domestic product (GDP) per capita of US$4,248. Indonesia had restored its macroeconomic and financial stability and economic growth had increased from 3.8 percent in 2001 to 5.6 percent in 2005 while the percentage of the population living in poverty in the period 1999-2007 had fallen from 27 to below 16 percent in 2007. 3. Despite all of these achievements, Indonesia continued to under-perform compared with its neighboring countries in terms of access to quality health, education and other basic services, as reflected in the millennium development goals (MDGs) indicators. Indonesias centralized development planning and service delivery systems had increased overall access to basic services but have fallen short in ensuring quality. 4. The Government of Indonesia (GOI, the Government) was committed to poverty reduction, and had laid out a comprehensive medium-term poverty reduction strategy in its Medium Term Development Plan (RPJM). The Governments main approach towards poverty reduction was to increase incomes by means of productivity improvements and decreasing expenditures by means of lessening basic needs burden, such as education, health, and infrastructure that eases and supports social economic activities. 5. Decentralization of public service delivery had been an important element for effective implementation of the poverty reduction strategy. Indonesia was one of the more decentralized nations in the world. Districts were responsible for one-third of all government spending, and half of the development budget. Most spending on education, health and infrastructure was district based, and the districts employed three-quarters of the civil service. In 2006, 90 percent of the Ministry of National Educations (MOEC) budget had shifted from the central to the district levels in the form of block grants channeled directly to the schools. Limited experience and capacity at both the district and school levels to manage the funds and lack of knowledge about effective inputs for quality education was a major constraining factor in the implementation of the decentralization laws for the sector. 6. Education was regarded as one of the most important elements in poverty reduction, and early childhood development as an integral part. The education gap between the rich and the

1

poor in Indonesia was pronounced, with poor children in poor districts exhibiting the lowest outcomes. Specifically, research suggested that districts with higher levels of preschool enrollment tended to have lower primary school dropout rates. More directly, childrens participation in ECED programs appeared to improve their readiness for primary school education through improving their overall development across domains. An evaluation of the World Bank-financed Early Child Development Project found that children participating in the project for two years had higher scores for school readiness at age 65. These positive effects for ECED program enrollment were even greater for the less privileged children who were living in poor districts or whose parents had lower levels of education. 7. Based on international evidence that showed the value of ECED, and building on some of its early efforts, the GOI was committed to prioritize ECED, as reflected in several government strategic documents, but it faced important challenges. The strategic documents included the National Education System Act No. 20 (2003), the Education for All plan and the Governments National Medium Term Planning 2004-2009 (RENSTRA 2005). Challenges included: (i) the fragmentation of ECED service delivery and the delivery of health- and nutrition-related services and of parent education; (ii) extremely low participation in ECED, especially in poor communities (enrollment was only 8 percent compared with a 24 percent global average enrollment rate for low-income countries); (iii) lack of government investment in ECED and failure of the private sector to reach the poorest children; (iv) insufficient provision of services for families, infants and toddlers (despite the fact that approximately 90 percent of villages had Posyandus, i.e. village-based health posts); (v) limited progress toward a quality assurance system for ECED, including quality standards and monitoring and supervision; (f) lack of ECED training for managers, teachers and field workers; (vi) fragmentation of responsibility for ECED services across ministries and directorates overseeing different forms of early childhood development services; and (vii) limited district-level capacity to implement ECED programs, especially relevant since districts responsibilities had been expanded to include implementation, financing and supervision of ECED services in their respective areas. 8. The GOIs strategy aimed to provide expansion of access to quality ECED services with equity through major investments in: (i) promoting community-based and decentralized service delivery, and strengthening community capacity in service delivery; (ii) targeting the poorest communities and households for public financing of ECED service provision; and (iii) strengthening system management capacity in planning, quality assurance, and monitoring and evaluation at central, provincial and district levels. The Project was designed to support the GOIs strategy in ensuring that children aged 0 to 6 from economically disadvantaged families and rural areas participate in and benefit from non-formal integrated quality ECED services.

5 World Bank, Early Child Development Project (approved July 28, 1998), Project Appraisal Document Report No. 18151 dated July 7, 1998.

2

1.2 Original Project Development Objectives (PDO) and Key Indicators 9. The Projects Development Objective was to improve poor childrens overall development and readiness for further education within a sustainable quality ECED system. To do this, the Project was to: (i) increase the capacity of poor communities to engage in participatory planning that would result in new or improved ECED services for their children and families; (ii) prepare the foundation for a sustainable ECED system through budgetary commitments from participating districts, establishment of a national quality assurance and professional development system, and district capacity building; and (iii) ensure continuous improvement of service delivery and system building through establishing effective project management, and monitoring and evaluation. 10. The Projects progress towards its PDO was to be measured through the following PDO indicators: (i) increase in early development scores of children entering kindergarten or first grade of primary school; and (ii) increase in Child Growth and Development (DDTK) scores of children 0-3 years. The intermediate indicators were: Component 1: (i) percent of parents/communities who have received information regarding ECED during community mapping exercises; (ii) number of communities submitting an expression of interest to prepare an ECED grant proposal (by completing a form distributed during village pre-conditioning process); (iii) number of block grants awarded to communities; (iv) increased enrollment in ECED services by poor children. Component 2: (i) development of standards and recognition of model ECED centers; (ii) percent of targeted districts with regulations supporting ECED positions in district government; and (iii) percent of targeted districts with qualified and functioning district training teams. Component 3: (i) percent of villages undertaking annual community assessments of existing ECED services and unmet ECED needs; (ii) percent of villages submitting routine reports to districts on project activities and uptake of ECED services; and (iii) impact study undertaken. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 11. The Projects Development Objectives were not revised during the project period.

1.4 Main Beneficiaries 12. The primary beneficiaries of the Project as described in the Project Appraisal Document (PAD) were poor children ages 0 to 6. Through geographic targeting at all levels, the Project was to focus on an estimated 738,000 children living in 50 of the poorest districts of the country and in the poorest kecamatans, villages and communities within those districts. Once a community was identified and received a grant, all children with that community were to benefit from the ECED services implemented, including children with disabilities. The Projects secondary beneficiaries were to include: (i) parents and caregivers, particularly those of children ages 0 to 3 that would receive relevant information and support from trained community members on how to best nurture the development of their children; (ii) the 6,000 teachers and 6,000 child development workers that would receive both systematic training in early childhood care and development and close supervision in their daily practice with poor children; and (iii) civil

3

society and local organizations (NGOs) already active at the community levels whose presence the Project would capitalize upon to be involved in all stages of the process, including training, service delivery and independent monitoring. The Project was also expected to have a demonstration effect on the Government, which could then expand the coverage of ECED services for poor children in Indonesia. 1.5 Original Components 13. The Project objective was to be achieved through three components: 14. Component 1: Increasing Integrated ECED Service Delivery Through Community-driven Mechanisms in Targeted Poor Communities (US$96.65 million). This component aimed at delivering integrated ECED service to poor communities in fifty districts following a Community-Driven Development (CDD) approach. The 50 districts chosen to participate in the Project were selected according predefined criteria, detailed in the Project Appraisal Document. This component comprised four sub-components, as follows: 15. Sub-Component 1.1: Establishing Training Programs and Providing Staff Development (US$12.20 million). This sub-component was to finance the training of 50 members of the National Early Childhood Specialist Team (NEST), 150 district trainers, 600 facilitators (200 three-member teams consisting of an ECED Specialist, A Family Support, Health and Nutrition Specialist, and a Community Development Specialist), 6,000 teachers (for services to children ages 3 to 6) and 6,000 child development workers (CDW) for services to children ages 0 to 3 and their families. 16. Sub-Component 1.2: Preconditioning, Launching and Facilitation of the Community Grant Process (US$18.89 million). This sub-component was to finance: (a) the selection of eligible kecamatans and villages within about 50 participating districts (about 60 villages per district, that is a total of 3,000 villages nationwide); (b) the sensitization of villages about the importance of ECED services (preconditioning phase); and (c) the launching and facilitation of the community grant process. 17. Sub-Component 1.3: Providing Block Grants to Poor Communities. This sub-component was to finance the provision of grants to about 3,000 villages (with an average of two beneficiary communities or dusuns per village) in about 50 districts. Grants were to be awarded twice a year on the basis of a proposal review conducted by the DPIU. 18. Sub-Component 1.4: Recognizing Model ECED Services and Using Them for Demonstration Purposes. This sub-component was to finance additional block grants to one ECED model program per eligible kecamatan (about 10 model programs per district).

19. Component 2: Developing a Sustainable System for ECED Quality. This component was to help ensure quality in the programs financed under Component 1, and create conditions that would support sustainable, scaled-up provision of holistic, quality ECED services.

4

20. Sub--Component 2.1: Quality Assurance. This sub-component was to finance consultancy services, professional development workshops, and materials development to assist the Government in developing standards and recognition procedures to support quality ECED delivery. Assistance was to be directed to relevant units in MOEC and related agencies, in consultation with the Board of National Standards of Education (BNSP). 21. Sub-Component 2.2: Institutionalization of Quality ECED at the District and Provincial Level. This sub-component was to finance facilitation of coordination among and partnership of ECED-related agencies; inclusion of ECED in local governments regulation (Perda) on organizational structure, roles and function (Tupoksi) in Dinas Pendidikan as well as other ECED-related agencies. In addition, it would support expert training and workshops to improve the knowledge bases of the key ECED-related officials; and provide matching grants on a competitive basis to district governments and existing ECED service institutions to establish such resource centers to support the training. 22. Component 3: Establishing Effective Program Management, Monitoring and Evaluation. This component was to help ensure continuous improvements of both ECED service delivery and system building over time. 23. Sub-Component 3.1: Program Management. This sub-component would finance both project management at the central and district levels. For project management at the central level, the Project would finance activities to support MOEC in implementing its mandated roles and functions. For project management at the district level, this Sub Component would finance activities to support participating districts in implementing their mandated roles and functions. 24. Sub-Component 3.2: Monitoring and Evaluation. This sub-component was to finance the development of a monitoring and evaluation (M&E) and supervision system for ECED programs at village/community and district levels, including those ECED programs not supported by the Project. It also would support formal evaluation6 of project benefits. 1.6 Revised Components

25. The Projects components were not revised.

1.7 Other significant changes 26. A project restructuring was approved on May 15, 2012. The purpose of this restructuring was to reallocate funds among expenditure categories in both the Credit Agreement and the Trust Fund Grant Agreement to reflect the larger number of village teachers trained (reflecting greater demand), the reductions in Regional Management Consultants (RMC) due to procurement delays and the need to increase technical support for project management and monitoring. The closing date of the Dutch trust fund was extended from June 30, 2013 to December 31, 2013 to allow for

6 This formal evaluation is what became the Projects Impact Evaluation (Section 2.3).

5

continued financing of activities under sub-component 2.1 and to continue financing consultants supporting project management under Component 3.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 27. Project Preparation. The Project addressed the GOIs priorities and counted upon its commitment and ownership, including at the district and village levels; it was also in line with the World Banks Country Assistance Strategy (CAS). The Government was committed to poverty reduction and to improving services to the poor by strengthening decentralization of service delivery. Expanded provision of ECED services were seen as fundamental for Indonesia to achieve MDG targets in education and health, and those set by RENSTRA and the EFA Plan. Yet, ECED was still a relatively new sub-sector, and there were challenges in the start up phase. Consistent with the CAS focus on areas most crucial to poverty reduction, including the lack of quality service delivery to poor people, the Bank was well placed to assist the GOI in closing both the financial and technical capacity gaps that the Government had identified. 28. The Bank preparation and appraisal teams consisted of technical experts that were appropriate for the development of the project. A Quality Enhancement Review (QER) was also conducted and the project design took into consideration comments which included the need to: (i) provide services for 0-3 year old children; (ii) include indicators that measured overall child development for children 0-3 rather than specific standalone health or nutrition indicators; (iii) provide a simple operations manual for use at the local level; and (iv) establish a mechanism within the project management structure for coordination between various ministries such as a steering committee. The preparation team incorporated lessons from earlier and ongoing Bank-supported projects in Indonesia as well as other Bank-supported ECED projects. However, lessons learned were limited at the time because there were few standalone ECED projects under implementation in the Bank and ECED was a relatively new investment sub-sector in the Bank. The preparation team also identified the appropriate critical risks to the project and proposed mitigation measures within the project design. 29. Project Design. The PDO for the project was appropriate for an outcome-focused project, which was also the view of the 2006 QER and the project decision meeting reviewers. In addition, the results framework, at the time of preparation, was appropriate to measure achievement of the PDO. The framework had two PDO indicators to measure overall development and readiness for further education and appropriate intermediate indicators to monitor progress toward development of a sustainable quality ECED system as well as that of component implementation. The QER and peer reviewers all confirmed this judgment. The DDTK scores of children 0-3 indicator was selected specifically to respond to the QER recommendation. It was also an existing local instrument that could be used to strengthen linkages with the health sector. Baseline data for the PDO indicators was to be determined by a baseline data survey in year 1 of the project due to the fact that the Indonesia ECED system was just being developed and a data system was not yet fully functional. These were areas that the project would support for further development.

6

30. The Project was designed to be implemented within the Governments institutional structure and budget framework, utilizing a system of disbursing a flat rate of block grants. The Central Project Implementing and Coordination Unit (CPICU) reporting to the Director of Early Childhood in MOEC was responsible for monitoring overall implementation and the coordinating activities in 50 poor districts at the village level. 7 The budget framework and disbursement flows that were used had been applied under other Bank-financed projects in Indonesia, including those in the education sector. By targeting poor districts rather than poor families at the village level, the design ensured a manageable implementation approach. This, coupled with knowledge that children benefit most from the ECED experience when they interact with children from more favorable socio-economic backgrounds, made targeting districts more appropriate. Moreover, the project design ensured that the 50 districts had the capacity and were committed to the project as demonstrated by financial contributions and establishment of institutional structures. Participating districts also signed a Memorandum of Understanding (MoU) between MOEC and district governments. The MoU included selected strategic measures that districts needed to adopt to have long-term impact, such as: (i) the incorporation of ECED (as a new sub-sector) in the districts 5-year Development Plan (Renstrada); and (ii) the establishment of an ECED unit within local education office (as a district anchor in the ECED program). Establishing an ECED unit eventually lead to districts allocating local funds to continue advancing the ECED agenda.

31. A more community-driven approach, consistent with the GOIs decentralization framework, was incorporated under the Project, where villages were to be empowered, with support from facilitators, to decide upon those services that best suited their needs. An extensive review was carried out of Bank support to various Community-Driven Development (CDD) models applied in several sectors in Indonesia. Based on experience, the Project adopted an approach that channeled grants from districts to communities, along with direct involvement of local government (especially at the district level) to ensure both technical inputs during implementation and financial support for recurrent costs. 32. The design included components that reached all levels of key stakeholders and strengthened their respective roles, thereby ensuring a holistic approach to building sustainability. Specifically, the Project supported: (i) the MOEC at the central level to develop standards and policy; (ii) provincial level to strengthen their coordination role; (iii) the district level to strengthen the enabling environment, implementation and monitoring capacity; and (iv) the community level to understand the benefit of ECED for their children as well as establishing and managing community based ECED service. Moreover, project support at the community level consisted of the key ingredients, such as facilitation, fund and training for teachers, while requiring villages to contribute to the infrastructure, which was another measure to ensure sustainability. Participatory planning was a central feature in the villages and communities definition of priorities. Through established fora, villagers identified priority communities and their representatives, who would, in turn, through participatory planning processes, identify priority needs for delivering ECED services.

7 The project included 50 poor districts and 6,000 ECED centers in the districts.

7

33. The project design also included a parallel co-financing grant in the amount of US$25.3 million from the Royal Netherlands Government. This grant, which was recipient-executed, supported the development of a sustainable quality ECED system by supporting: (i) the establishment of core for the National Early Childhood Specialist Team (NEST); (ii) the development of materials to ensure the that project would begin with basic capacity for implementation and facilitation of the community grants, project evaluation and capacity building; and (iii) international and most local consultancy services, fellowships and training. This support was fundamental in financing initial activities required for the Project, which included: (i) a decree on participating districts and on the establishment of the CPICU; (ii) expressions of interest by all 50 districts; (iii) MoUs between the MOEC and district governments; (iv) selection of communities consultants; and (v) the design of the impact evaluation and instruments for the baseline survey. These activities ensured that implementation could begin in a timely manner. The donor-financed grant represented an instrumental and flexible vehicle in support of enhanced implementation.8 2.2 Implementation 34. Overall implementation was positive. Implementation began with a 2006 implementation preparation mission pending effectiveness. The CPICU was already preparing implementation plans for FY2006 and 2007; the Bank team and PAUD (Pendidikan Anak Usia Dini or Early Childhood Directorate) held discussions on the development of training materials for the NEST trainers; criteria for NEST national trainers had been confirmed; discussions began on the development of ECED standards; and on the feasibility of building in a randomized designed in a few districts for the impact evaluation. By 2007, implementation of all project activities was underway mostly with support from the Dutch trust fund. Credit disbursements were low due to the fact that a government decree was issued cutting travel costs for government programs, impacting government officials ability to travel to the 50 districts and support project implementation. This was resolved when the government clarified that the cuts did not apply to externally funded projects. Once this was resolved, credit disbursements increased and the project continued to move forward. 35. In 2009 and 2010, project activities continued to be implemented. The baseline survey for the impact evaluation took place in 2009 with a second round of data collection in 2010. In addition, the project had: (i) established community-based ECED services; (ii) distributed the first tranche of block grants; (iii) developed and piloted parenting training modules; (iv) provided district level support for ECED teachers; (v) issued national ECED standards through the 2009 Minister Decree 58; (vi) worked with higher education leaders to identify desired competencies for ECED graduates; and (vii) issued district level financial regulations for supporting ECED within their district budgets. It was at this time that the Banks Quality Assessment of Lending Portfolio (QALP-2) assessed the likelihood of the project achieving its PDO as Moderately Likely, partly because it was too early to assess the overall quality and impact of the ECED

8 Reporting on the trust fund has been through the Banks Grant Reporting and Monitoring (GRM) form.

8

interventions. It found the project well designed, with strong government commitment at all levels. 36. While there were many achievements; there were also challenges. In 2009, due to MoEC reorganization, most of key personnel involved in project design were replaced. This included the Director General, Director of ECED (implementation unit) and CPICU coordinator. At this time, the new Director requested substantial changes to the project design and implementation arrangements by significantly reducing the role of consultants at all levels and replacing them with government staff. The procurement process of key consultant packages (mostly regional management consultants) were put on hold during this time. The Bank team had several meetings with the new Director explaining the importance of the original design which led to an agreement to keep the original project design and implementation arrangements. This however, slowed the implementation process. 37. In late-2010, another reorganization took place. The responsibility for formal and non-formal ECED was merged under the Directorate General of ECED, Non-Formal and Informal Education. This led to an increase in directorates relevant for the project: the Directorate for ECED (formal and non-formal) and the Directorate for Professional Development of Teacher and Personnel of ECED, Non-Formal and Informal Education (P2TK PAUDNI). Although project implementation remained under the Directorate for ECED, the reorganization of staff slowed implementation activities again. One notable change occurred when in 2010, approximately 130 community facilitators resigned for a variety of reasons (i.e., secured permanent civil service employment, ran for local offices, etc.), which required the project to recruit new facilitators and establish procedures to ensure continuity in village support. Financial management during this period presented challenges and was downgraded to moderately unsatisfactory. The QALP-2 also rated implementation as moderately unsatisfactory primarily in response to shortcomings in financial management and procurement processes. 38. By 2009, it was also increasingly evident that there were challenges with the PDO indicators, particularly with the DDTK Scores of children 0-3 years. Although it was selected based on recommendations from the preparation stage QER, it became increasingly obvious that this indicator was more appropriate for determining whether a child had special needs rather than measuring childrens development over time. Moreover, it was not feasible to find and train enough psychologists for the administration of the DDTK considering the scale of the project. The implementation teams reviewed the Early Development Index (EDI) and determined that it was more suitable for the purposes of the project since it is a school readiness instrument. It covered the same aspects of development as the DDTK but was more relevant to the projects aims and did not require formal psychologists to undertake the assessment, making it easy to complete by either the caregiver or teacher of the child. Examples of the proxy indicators used to replaced most of the DDTK indicators included: (i) the number of tasks children are able to complete (covering the domains of gross motor, fine motor, language, cognitive and socio-emotional development), and (ii) the percentage of children able to answer the receptive and expressive language questions correctly. (See Annex 3 for the full comparison of indicators.) At the same time, the team made the decision to use the impact evaluation to gather, analyze and report on achievement of the PDO. This is judged to have been an appropriate decision on the part of the Bank and government implementation teams because of the data challenges and the

9

fact that this approach could provide more direct information on project attribution and PDO achievement. 39. The 2011 mid-term review (MTR) confirmed that the PDO was appropriate and that there was a high likelihood of achieving the PDO. This was important particularly in light of the modification to the PDO indicator that measured child development for children ages 0-3. At the time of the MTR, the project was also on track to complete all project activities with a credit disbursement rate of 91 percent and 76 percent for the Dutch trust fund. The Bank also began to dialog extensively with the government at the central, district and community level about the need to have all necessary policies and institutional measures in place to ensure that the ECED model would be sustained. It was during the MTR that the government requested a reallocation of the Credit funds between disbursement categories. The Bank team agreed to put forth the necessary project restructuring documentation for the reallocation in April 2012. In late 2012, another restructuring was processed to extend the closing date of the TF which was completed in April 2013. At the time of the final restructuring, the implementation unit also made the request to restructure the result framework to formally reflect the use of the proxy indicators as a replacement to the DDTK. The package was submitted to the Ministry of Finance and Bappenas. However, the government felt that the agreements within the project documentation was adequate and was not inclined to pursue the process further. 40. At the time of project closing, the structure of a successful ECED program was in place, with many sustainable features such as training and monitoring, as well as a strong understanding and demand from the beneficiary district governments. Furthermore, the delivery model is replicable and can be scaled-up across Indonesia. This is a substantial achievement. Over the project implementation period, the Bank team conducted six formal implementation support missions and the Banks Indonesia country office team provided almost continuous on-the-ground support from preparation through completion. The make-up of the Banks teams was well balanced with financial management staff well represented on missions. The project closed as scheduled on December 31, 2013 and had disbursed 99 percent of the credit (SDRs 45.9 million or US$70.4 million equivalent) and 92 percent of the Dutch trust fund (US$23.4 million). 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 41. Design. The M&E design was twofold. The first focus of the M&E design was to develop a system to monitor project outcomes. The development of the system was important since there was no formal ECED data gathering system at the time the project was designed. The system would provide information for project outcomes as well as project outputs and was to be managed by the CPICU. The second was an independent evaluation of the ECED community-based interventions would be assessed through an impact evaluation that also looked at trends in some child development indicators. Putting in place a system to monitor project outcomes was seen as critical for sustainability of ECED programs. As such, design focused on developing a monitoring, supervision and evaluation system for ECED programs at village/community and district levels, including those ECED programs not supported by the Project. This system would be developed through annual community assessments comprising: (i) registration of children aged 0-6, including background information on family circumstances, current participation in ECED programs and anthropometric assessments; (ii) an inventory of ECED services and

10

resources accessible to a village/community; and (iii) an assessment of the developmental levels of children entering kindergarten and school using a school readiness instrument9. 42. The development of an ECED monitoring system was judged to be highly relevant for the institutionalization of ECED sub-sector. Moreover, the original results framework was judged to have been appropriate at project design because it was monitoring child development and readiness although no specific target value was assigned for the respective indicators. Although gathering data related to the DDTK scores of children 0-3 proved to be difficult and, more importantly, not the appropriate indicator for assessing child readiness 10 , the project team quickly made the adjustment and adopted new indicator measures that proved to be appropriate. The intermediate indicators were linked to the components and were appropriate to measure intermediate level achievements. The inclusion of the impact evaluation was highly relevant and an important design feature for understanding the effectiveness of project interventions. 43. Implementation and Utilization. During implementation monitoring and evaluation focused on: (i) development of a system that tracked the intermediate outcome indicators, and (ii) an impact evaluation which evaluate the PDO indicators (school readiness and child development. 44. The monitoring system was developed and is user-friendly and reliable, and an improved data entry application was developed to support the system. Monitoring activities were managed by community facilitators (village level), by the DPIUs and the RMC/District Consultants (district level), by the provincial implementing units (provincial level) and the CPICU and NMC (central level). The system is now capable of providing comprehensive data from the field to guide decision-making and project improvement. The system counts on data entry of the 5,990 project-supported village implementing units level, and is capable of aggregating data by village, sub-district, district, and province, and for the whole 5,990 units. Training on use of the monitoring system was provided through a cascade model to the NMC, RMCs and district consultants and field facilitators at the community level. 45. The collected data in the monitoring system encompasses the intermediate indicators in the Projects results framework, including carefully tracking the number of new enrollments in each of the 5,990 project supported centres.

9 School readiness would be measured using the EDI, an instrument correlated with later school performance that looks at physical health and well-being, social competence, approaches to learning, emotional maturity, language development, cognitive development, communication skills and general knowledge. The instrument was considered to provide communities with an easy to understand and psychometrically sound instrument to monitor their efforts to improve early years outcomes over time. This instrument had been used in developed and developing countries, and was adapted to the local context.

10 The DDTK is a diagnostic tool developed based on a Denver test and adapted to Indonesias country context that tracks individual development of children. It also measures whether a child has a special need and, as such, should generally be used as part of an intervention rather than for monitoring and evaluation. There were important issues, including institutional, in terms of the availability and linkages with professionals to track individual children and the consequent financial implications for its usage. This made its use under the Project inappropriate

11

46. Five cycles of annual monitoring were carried out under the Project, and in each of these, improvements were made to revise and improve the monitoring instrument and data entry system, and reduce the bottlenecks in data entry to improve timeliness of information. This was so successful that by the end of the third cycle information was received from districts within two months compared to six months in the previous cycle. The results of the data analysis were routinely discussed with the ECED Directorate and district representatives. The system has developed a rating system for ECED centers using a spider web diagram (Annex Figure 1 in Annex 2) that can provide an indication of the performance of individual ECED centers, and aggregate the data at higher level. 47. This capacity is used to map the centers and the specific aspects in service delivery that need improvement. The data provided from the monitoring system is particularly important for district level supervision as well as informing resource allocation and focusing support to the needy areas. Furthermore, feedback and monitoring from project-supported activities (e.g., professional development, including teacher training) were routinely carried out and used to adjust and improve programs, manuals, and the design of specific activities. Based on experience with the monitoring system supported by the Project, the MOEC expressed interest in a comprehensive monitoring system that would allow them to map the performance of ECED services across Indonesia The M&E system developed in the project is being adopted by Directorate of Teacher and personnel PAUDNI, especially for supporting school supervisor in their tasks in quality assurance. The comprehensive M&E system put in place under the Project is a major accomplishment. It is being utilized to rate ECED centers according to various criteria that reflect the quality of service delivery, identifying areas that need attention to improve overall quality, not only in project-supported centers, but also, by linkage to MOEC monitoring systems to all ECED services across the country. 48. The impact evaluation carried out under the Project has provided valuable information to inform and document the Projects impact. The Banks implementation support team provided just-in-time advisory support to the MOEC in all phases of carrying out the IE, and set up a team in the country office to produce technical and policy papers that make use of the IEs rich data to guide its education sector work.11 For each round of the survey, a joint GOI-Bank team worked closely to ensure the quality of data being collected by the government, through contracted survey firms. In the first two rounds (2009 and 2010), the Bank team provided intensive support to the CPICU to ensure reliability of data as well as build the capacity of the CPICU to analyze the data and produce reports based on the analysis. In the end, the Bank used the data provided by the system to produce the 2013 Directions in Development report Early Childhood Education and Development in Poor Villages of Indonesia: Strong Foundations, Later Success. (See Annex 4 for full details of the impact analysis methodology and data analysis).

11 P110517 (ID-ECD Evaluation)

12

2.4 Safeguard and Fiduciary Compliance 49. Safeguard Compliance. The Project was classified as Category C for environmental purposes so an environmental management plan was not prepared. Nevertheless, the Project Operational Manual (POM) contained appropriate guidance concerning how to address issues of waste disposal, noise, etc., in the event that community projects involved minor construction/renovation. 50. The Project did trigger OD4.20 (now revised as OP4.10), Indigenous Peoples policy, as isolated vulnerable peoples were present in some areas of the 50 districts covered. A Framework for Treatment of Indigenous or Isolated Vulnerable People was prepared that identified activities to be carried out to ensure that the Project addressed the special needs of these groups. The specifics of these activities were included in the POM. Section 3.4 presents a summary of the Social Assessment carried out upon project completion.

51. Fiduciary Compliance. There was a risk assessment of both financial management and procurement as high, in part due to the lack of capacity at all levels, and in part due to the Projects decentralized implementation. To address these risks specific action plans in each of these areas had been agreed, and the Project included financing for identified critical activities (e.g., strengthening of MOEC internal control capacity by financing the development of audit manuals, including audit or community block grants and the training for staff of the Inspectorate General). The project operational manual (POM) documented the financial management system and procedures (including procurement) to be followed by the Projects implementing agencies at all levels. Consultants were contracted to support the CPICU in fiduciary areas.

52. Despite this, financial management and procurement issues surfaced, especially in early implementation, requiring intensive and repeated follow-up by the Banks implementation support team. For financial management, this was evidenced by delays in: (a) appointing a financial management consultant to the CPICU; (b) introducing minor financial management revisions to the POM; (c) contracting the technical assistance to strengthen the MOECs internal audit function; (d) submission of Interim Financial Reports; (e) conducting transaction reviews and payment verifications, appropriate recording of expenditures and filing of supporting documents and addressing expenditures backlogs; and (f) providing follow-up to observations in the Projects external audit. These issues were addressed by early 2012, and consultants at the district level were contracted, significantly improving financial management performance at the district level. External audits were timely and unqualified, although they did occasionally contain observations. 53. Procurement. Procurement also surfaced as a major issue that required attention and adjustment during implementation. The Projects original procurement plan had envisaged the contracting of a National Management Consultant (NMC) and five Regional Management Consultant (RMC) firms to strengthen the capacity of project management in participating districts (including for procurement). Each of the RMCs would contract community facilitators to provide support at the village level. Procurement of the RMCs suffered delays, due in part to weak capacity of the procurement team and the re-organization in MOEC, so in the meantime, the CPICU directly contracted 600 community facilitators individually. After the contracting of

13

two RMCs had been concluded (RMC1 and RMC4), they took over the contract of the community facilitators working in their respective regions. For areas without RMCs, 34 district consultants were contracted directly by CPICU, and the rest of community facilitators continued to be contracted by CPICU on individual basis. These individuals, contracted as non-consulting services, had been contracted by early 2011, resolving an issue that was delaying much needed support to the districts and villages/communities. Although involving a minor adjustment from the Projects original design, the substitution of the three RMCs by individuals represented an effective approach in to support implementation. Other issues that affected procurement processes were the delays in getting the procurement committee together, multiple changes in its members, and persistent issues with the quality of bidding documents and bid evaluation reports. These were the focus of follow-up and technical support and advice by the World Banks specialists. Project procurement plans were updated at least annually, and used effectively to monitor compliance with scheduled activities. 2.5 Post-completion Operation/Next Phase 54. The FY13-15 Country Partnership Strategy for Indonesia reports that the GOI has indicated a preference for not seeking external financing for education at this time. Nevertheless, the CPS proposes that the Bank remain engaged through knowledge services, and linking of education, including ECED, with other operations. 55. Specifically, starting in late 2012, the Project promoted a pilot collaboration between ECED and the Bank-financed National Community Development Program (Program Nasional Penberdayaan Masyarakat Generasi Sehat Cerdas, PNPM Generasi) funded by the GOIs Ministry of Home Affairs12. Both programs share a common CDD approach, and the PNPM Generasis focus on maternal and child health and basic education provides a logical opportunity to link it with ECED, especially by using Posyandu as the entry point. Indicators related to ECED services for children aged 0-6 were added to the Generasi menu using the PNPM Generasi's performance based block grant mechanism to increase participation of the poor in ECED services. 13 The pilot leveraged the experience and materials developed under the ECED to train sub-district and village facilitators in the benefits of ECED, who then raised awareness among rural community members of the importance of ECED. PNPM Generasi block grants were invested in activities aimed at improving access to ECED services, including transportation to ECED centers, procurement of educational materials, rehabilitation of posyandu buildings to make them appropriate for use as ECED centers, and so on. The pilot did not provide fund for the local government. In 2014 the pilot was operating in three districts (Boalemo, Sumbawa, and Cianjur).

12 PNPM Generasi is an incentive-based community-driven development program that focuses on maternal and child health, as well as on universal education. The program provides incentives to communities to meet a number of maternal and child health indicators, including those on ante-natal care, immunizations and growth monitoring. Incentives are also provided for meeting or exceeding enrollment thresholds for 7-15 year olds. The pilot location was Sumbawa and Boalemo districts (where the project overlap) and Cianjur (non ECED project district) 13 These indicators included: every child under 3 years receives joint stimulation/nurture once a week and every child 3-6 years receives early childhood stimulation at a minimum 3 times per week.

14

56. In addition, the Directorate P2TK (Professional Development for ECED Teacher and Teaching Personnel)in 2012-12, requested the Bank to conduct a review in two areas: (a) capacity of ECED supervisors, and (b) training programs for ECED teachers. The review produced a draft of policy recommendations for improving the capacity of ECED supervisors with a specific focus on conducting efficient monitoring of quality of services and quality of teacher training. The review also produced recommendations on improving curriculum and delivery mechanism of the teacher training program. The review and recommendations drew significantly on the training materials and other resources produced under the Project and, therefore, have continued the effort of institutionalizing the Projects achievements.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 57. Relevance of Objectives. The relevance of the Projects objective is high. The Projects objective was relevant at the time it was prepared and continues to be relevant today. It was consistent with the Governments priorities and with the Banks CAS when it was prepared, and is still very relevant to the GOI and the Bank although the GOI is no longer seeking external financing for education. Participation of children aged 0-6 in ECED services remains low, with significant gaps in participation among rural and urban areas, and among income levels. As a result, by the time children from poor families begin primary school they are already behind in essential areas of development and school readiness. Yet, it is these very children from poor families that stand to benefit the most from ECED services. Therefore, the Projects objective of improving poor childrens overall development and readiness for further education and of providing a sustainable quality ECED system continues to be very relevant. 58. Relevance of Design. The relevance of the project design is substantial. The Projects design included activities at all levels of government that were needed to put in place and deliver a quality ECED system in 50 districts, while strengthening capacity to replicate this system in others. The Project established implementation units, embedded in the education office, at the Central level (CPICU), Provincial Level (PPIU) and District level (DPIU). At each level, especially at the district level, budget is allocated to build capacity of the respective government unit, including coordination meetings, study visits as well as technical assistance. At the same time, project activities also sensitized project communities about the importance of ECED and training community members to serve as ECED teachers at the local level. The community-driven nature of the Project, with districts selected according to pragmatic criteria, built upon proven models of CDD projects to incorporate districts and communities needs. The selection of PDO indicators at the time of design was judged to have been appropriate. During implementation there were challenges with the PDO indicators as describe above. However, these modifications made are judged to have been appropriate for measuring achievement of the PDO as well as impact of the project on the ECED system. 59. Relevance of Implementation. The relevance of project implementation is judged to be substantial. As indicated above, implementation of the majority of project activities was successful. The Banks implementation assistance was responsive to emerging needs. While the Project was implemented almost entirely within the parameters defined during preparation, the

15

Banks implementation assistance demonstrated flexibility when issues emerged. The implementation support team responded pragmatically to the MOECs request to contract individual consultants to substitute for the originally planned RMCs in three regions. But it did so after ensuring that a monitoring mechanism was put in place so that the maximum of 34 individual consultants received coaching, and supervision from the national management consultant and the CPICU. Similarly, when it became apparent that the MOEC did not have the experience in M&E, the Bank provided greater than expected support to the GOI in developing the monitoring system and providing just-in-time assistance in the collection of data and evaluation of results of the Projects impact evaluation. The development and institutionalization of the ECED data system that resulted in utilization of the data, mostly by the introduction of the spider web diagram is judged to be a project a success. Finally, the Banks implementation support has played a strong, catalytic role in working with the various ministries at the central level, and districts at the local level to disseminate the Projects achievements, explore options for a continuation of support to districts and communities and pilot alternatives to sustain and expand its model to other districts and communities. 3.2 Achievement of Project Development Objectives 60. The Projects PDO was to to improve poor children's overall development and readiness for further education, within a sustainable quality ECED system. The Projects achievement of its Development Objective is rated moderately satisfactory. The PDO addressed two aspects: first, to improve poor childrens overall development and readiness for further education, and second, to do that within a sustainable quality ECED system - achievements in both of these aspects are addressed separately below. 61. Improving poor childrens overall development and readiness for further education. A randomized impact evaluation (IE) carried out between 2009 and 2013 presents rigorous evidence on the impacts of the project on a number of child development outcomes. The IE tracked the development outcomes of two cohorts of children in 2009, 2010 and 2013. These cohorts were aged 1 and 4 years old in 2009. Using a range of experimental and quasi-experimental methods (See Appendix 3 for details) the IE provides evidence to support the contention that the project improved poor childrens overall development and readiness for further education. Specifically:

The project led to an improvement in the physical health and well-being as well as

the language and cognitive development of the 1-year-old cohort at end-line. This cohort of children outgrew the development measures used to assess them at the time of the baseline. Thus at the end-line, they were assessed using a locally adapted version of the Early Development Instrument. This is the same instrument that has been administered to the older cohort in past rounds of the IE.

On average the data show that the project led to fewer improvements in

development outcomes for the 4-year-old cohort than the 1-year-old cohort. There is an increase in the emotional maturity of the older cohort of children. Other

16

point estimates, though positively signed, are not statistically distinguishable from zero.14

However, when the data are disaggregated, they reveal that these average results

mask substantial improvements for poorer children in the 4-year-old cohort. There is compelling evidence that the project led to improvements in poorer childrens social competence, language and cognitive development as well as emotional maturity.15

62. The two cohorts of children studied in the impact evaluation provide complementary evidence on the Projects impact. While several moderately sized development impacts were found, the strongest evidence of an impact across all analyses is for improvements in childrens language and cognitive development. This domain was found to be the weakest for children in rural Indonesia when baseline assessments of the children were conducted in 2009 and is one of the domains most strongly associated with school readiness. 63. Taken together, the evidence from the IE suggests that exposure to the Project improved poor children's overall development and readiness for further education. The extent of the improvement in childrens outcomes that can be attributed to the Project varies depending on the aspect of school readiness under consideration. It also varies depending on the cohort of children being studied. 64. Developing a Sustainable Quality Community-based ECED System. The project has been moderately successful in meeting the sustainability issue, as substantiated by the achievement of selected intermediate outcome indicators, summarized as follows: 65. With respect to the indicator percentage of targeted districts with regulations supporting ECED positions in district government, 48 out of 50 districts have established permanent ECED positions within their education office, exceeding the target by 16 percent. In addition, the project helped establish technical committees in 47 of the 50 districts to further support coordination of quality ECED services. In addition to supporting ECED positions, the majority of districts and several villages have issued regulations or decrees in support of ECED16. 66. With respect to the indicator development of standards and recognition of model ECED centers, target was partially met. National Standard for ECED services were issued through Ministerial Decree 58 in 2009 consisting of four categories: (i) Developmental Achievements in Early Childhood; (ii) Educators and Education Personnel; (iii) Content, Process and Assessment;

14 This lack of statistical significance is a reflection of the fact that the evaluation was designed to be able to detect a minimum effect size larger than the one obtained in reality. Thus the design does not have enough statistical power to detect the effect sizes actually observed. 15 These results are robust to alternative approaches. Comparing the evolution of outcomes between children whose parents report having poor parenting practices at baselines reveals a similar story. Together these results are consistent with the project having improved outcomes for the most vulnerable groups. 16 Examples of regulations in support of ECED issued by districts and villages are Peraturan Bupati (Decree of District Head) and Peraturan Daerah (District Regulation) as well as Peraturan Desa (Perdes)

17

and (iv) Facility and Infrastructure, Management and Financing. Although not yet fully enforced, they do represent a significant achievement and lay the foundation to improve the quality of programs, teachers and training/professional development. The target for the establishment of the district resource centers, with matching grants from district government, was missed: 25 model centers were established compared to the target of 36. However, the project did meet the target of providing 10 model centers per district with additional block grants.17

67. With respect to the percentage of targeted districts with qualified and functioning district training teams, the target was 90 percent met.18 All of district trainers completed their training. However some trainers did not conduct follow up supervision due to lack of transport support to visit teachers in remote areas. District trainers continue to serve even after the project closed. The extensive training and capacity building activities conducted at the national, district and community levels of the ECED system has undoubtedly raise the quality of ECED services being provided19. 68. In addition to meeting the project intermediate indicators, the following sustainability-related activities have been achieved with project support: (i) as envisaged, all 50 districts signed an MoU prior to their participation in the project that included actions to support the institutionalization of quality ECED at the district and provincial levels; (ii) as part of the MoU, participating district ECED financing increased from US$356,000 in 2006 to US$11.91 million in 2013 which is a considerable achievement considering there was not a legal obligation to allocate ECED funding at the provincial or district levels; (iii) all districts have ECED supervisors20 who are responsible for quality control and supervision of ECED centers; (iv) each district has a Forum PAUD which consists of people from different ECED backgrounds to support and advocate ECED; (v) 36 districts have included ECED in their strategic development plan; 23 districts allocated funds for volunteer teacher incentives, 40 districts to monitoring and evaluation and more than half the districts to infrastructure, coordination and operational program work. 3.3 Efficiency 69. The Project resulted in an efficient use of resources, and was implemented efficiently. A full ex-post economic analysis is provided in Annex 3 and summarized below. The PAD conducted a cost-benefit analysis of the Project, which estimated that the average project benefit cost ratio would be 6:1. This analysis assumed a 5 percent discount rate, rates of return to education ranging from 29.8 percent for primary education to 13.8 percent for higher education. It further assumed that the target of 738,000 children would be reached and that the target for educational attainment in project districts would be the average educational attainment of non-

17 Under the Project, two types of model centers were established: (i) one model center identified within each district that would receive a US$200,000 matching grant from the Project (to be matched by at least US$100,000 from the district budget); and (ii) 10 model centers per district that received additional block grants of US$25,000. 18 The target was for all 50 districts to establish a training team. 19 Please see Annex 2 for the complete list of trainings and capacity building activities 20 57 percent of districts elected their supervisors while 43 percent of districts assigned supervisors.

18

project districts. Lastly this analysis assumed that benefits would accrue in the form of increased wages till age 60. 70. This analysis has been updated using the actual number of children reached (673,162 as of June 2013) and the actual observed increases in educational attainment. These vary depending on the age cohort analyzed. For the four-year old cohort the estimates are 0.1 years on average and 0.4 years for the one-year-old. It also uses more conservative estimates of rates of return: which range from 6.8-10.6 percent as estimated by Duflo (2001) and from 6.1-12.3 percent as estimated by Patrinos et. al. (2006). 71. Assuming a 6.5 percent rate of return to education (averaging the bottom end of the rates of return reported in the papers above), and that children do not begin to realize the benefits of increased wages until age 18 and that they do so for 40 years, a 0.1 year increase in schooling results in a benefit-cost ratio of 1.3.21 Similarly, a 0.4 year increase in schooling results in a benefit-cost ratio of 4.3. Using the lowest rate of return assumed in the PAD (11.2 percent) suggests a much correspondingly higher benefit-cost ratio of 2.1 7.3. Thus even the most conservative cost-benefit estimates would suggest that the Project did better than break even. This is an underestimate of the benefit given the conservative estimates of returns to education used, the shorter-than-usual time horizon for accrual of benefits as well as the fact that these are only private returns. Social returns to education have not been factored in. 72. Efficiency in the Use of Resources. The final allocation of funding under each of the Projects three components is provided in Annex 1. Under component 1, no more than 20 percent of the funds received by each village could be used on physical infrastructure or construction according to the Projects design. Thus to compare efficiency, the amounts other projects have spent on classroom construction is compared with the amount communities could spent on construction under the Project. With one exception, the Indonesia ECED Project spent less on physical infrastructure than other education projects, which had a similar component suggesting a relatively more efficient use of project resources. The Project was also implemented efficiently and completed, as scheduled, with no extensions of the closing date.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 73. The Projects overall outcome is rated Moderately Satisfactory. The objectives were and continue to be highly relevant. The project help develop an ECED mechanism that establish a framework for the delivery of ECED services to poor communities. This mechanism counts on strong institutions at the national level, including training, standards and monitoring and evaluation, developed and implemented a model for efficient delivery of ECED services that can be sustained and expanded using alternative funding mechanisms, including those provided at the district level and through other complementary programs that transfer funds through block grants. Although there were data collection problems with the original PDO indicators, stemming

21 The 2012 GDP per capita in PPP terms was US$4876. In our calculations of rate of return, we assume that rural wages are a third of this number.

19

from capacity constraints and resources for collection of data in 6,000 ECED centers, the proxy indicators are judged to have been substantially relevant to measure progress toward achievement of the PDO. These indicators also show attribution of project interventions to PDO achievement. The efficiency of the project is also judged to be substantial. Project Relevance Achievement of PDO

(Efficacy) Efficiency Overall Rating

Substantial Moderately Satisfactory Substantial Moderately Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 74. By focusing on 50 of the poorest districts in the country, the Project had a poverty focus by design. The IE confirmed that the Projects impacts were greater for children from poor households and disadvantaged groups. 75. A Social Assessment was carried out upon completion to assess to what extent the Project was able to reach the poor indigenous children. The general finding was that ECED services in remote areas were able to reach indigenous children in areas accessible by transport, but that special efforts would be required to cover very remote areas. Although it found that these centers were providing good services to the children mostly due to the enthusiasm of the volunteer teachers that deliver them, it also found that ECED teachers in non-urban areas require more training, especially in the form of real-life examples as opposed to books and guidelines, and supervision to provide them with support and back up. Finally, Indonesias indigenous peoples communities in remote areas cannot simply rely on school management and the communities willingness to payadditional financial support is required.

(b) Institutional Change/Strengthening 76. The Project had a strong institutional focus at all levels of government. The Project succeeded in putting in place a framework for ECED, generating demand at the sub-national level, and challenging districts and communities to explore and test innovative methods to sustain ECED services and increase their impact (by coordinating with complementary services). The model also depended upon integrated actions and activities at all levels, all of which received strengthening and assistance to improve their institutional capacity for implementation. The Project had an important impact on institutional capacity, change and strengthening as described above. (c) Other Unintended Outcomes and Impacts (positive or negative) 77. The Project had an important impact both on district governments and the MOEC that expand beyond it completion. At the district level, districts have considerable flexibility of their constrained budgets, but it was interesting to see that several have adopted innovative strategies to internalize ECED activities in the context of other programs supported by national regulations, thereby promoting synergies and efficiency in delivery. For example, one district has developed

20

its own program---Stimulation, Intervention and Optimization of Services for Young Children (SIOLA)that combines ECED with parenting education, growth monitoring and micro-credit. Another district issued local regulation in support of POSPA-BKB integrating Posyandu (Health Post), PAUD (ECED service) and BKB (Parenting Education Program). 78. At the national level, based on experience with the monitoring system supported by the Project, the MOEC has expressed interest in a comprehensive monitoring system that would allow them to map the performance of ECED services across the country (not just those supported by the Project). The Project supported the piloting of an integrated system designed to integrate the Projects monitoring system into the Governments routine system employed at the ECED Directorate, thereby providing for continued monitoring of the Projects activities through the GOIs system upon completion. The piloting addressed four pillars necessary to support the function of the system, especially at the sub-national level, including readiness and capacity assessments in the area of: (a) institutional and legal framework; (b) human capacity; (c) organizational arrangements for monitoring and evaluation; and (d) infrastructure support, including the use of management information systems. The piloting is testing a web-based monitoring and evaluation system that will enhance accountability and improvements in ECED delivery by making the results widely accessible by relevant stakeholders. The pilot was conducted in 9 districts in 6 provinces, and the findings will be used to strengthen the MOECs ECED monitoring system at the national and sub-national level. In addition, the P2TK Directorate is interested in adopting this piloted monitoring and evaluation system to be applied by school supervisors as tools to implement the role of Penilik in ECED quality assurance. A collaborative activity between the Bank and P2TK Directorate to adopt the system is underway. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable.

4. Assessment of Risk to Development Outcome Rating: Substantial 79. Risk to Development Outcome is assessed as substantial. Risk to development outcome is linked to sustainability, a topic that became a central focus of attention by the Banks implementation support