Document of The World Bank

Report No: ICR00001265

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-43180)

ON A

CREDIT

IN THE AMOUNT OF SDR 65.6 MILLION (US$ 100.0 MILLION EQUIVALENT)

TO THE

THE ISLAMIC REPUBLIC OF PAKISTAN

FOR A

SINDH EDUCATION SECTOR DEVELOPMENT POLICY CREDIT

December 15, 2009

Human Development Sector South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 31, 2007)

Currency Unit = Pak Rupees (PKR) US$ 1.00 = PKR 61.4

FISCAL YEAR

July-June

ABBREVIATIONS AND ACRONYMS BOC Bureau of Curriculum

CAS Country Assistance Strategy

CSR Composite Schedule of Rates

CWIQ Core Welfare Indicators Questionnaire

DEMIS District Evaluation and Monitoring Information System

DLI Disbursement Linked Indicator

DPC Development Policy Credit

DSP Differential Stipend Program

EC European Commission

ED Education Department

EMR Education Management Reform

FD Finance Department

GoS Government of Sindh

ICR Implementation Completion and Results Report

IDA International Development Association

M&E Monitoring and Evaluation

MTFF Medium Term Fiscal Framework

NEAS National Education Assessment System

NER Net Enrollment Rate

P&D Planning and Development Department

PAD Project Appraisal Document

PDO Project Development Objective

PEACE Provincial Education Assessment Center

PESRP Punjab Education Sector Reform Program

PGDP Provincial Gross Domestic Product

PKR Pakistani Rupee

PPRS Public Private Partnership for Rural Schools

PRSP Poverty Reduction Strategy Paper

PSLM Pakistan Social Living Standards Measurement Survey

RSU Reform Support Unit

SBD Standard Bidding Document

SDR Special Drawing Rights

SEDPC Sindh Education Development Policy Credit

SEF Sindh Education Foundation

SEMIS Sindh Evaluation and Monitoring Information System

SEP Sindh Education Sector Project

SERP Sindh Education Sector Reform Program

SMCs School Management Committees

SSAC Sindh Structural Adjustment Credit

TOP Terms of Partnership

TPV Third Party Validation

Vice President: Isabel Guerrero

Country Director: Yusupha B. Crookes

Sector Manager: Amit Dar

Task Team Leader: Reema Nayar

ICR Team Leader: Benjamin Safran

PAKISTAN Sindh Education Sector Development Policy Credit

CONTENTS

Data Sheet A. Basic Information

B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Program Performance in ISRs H. Restructuring

1. Program Context, Development Objectives and Design............................................. 1 2. Key Factors Affecting Implementation and Outcomes............................................... 3 3. Assessment of Outcomes .......................................................................................... 10 4. Assessment of Risk to Development Outcome......................................................... 15 5. Assessment of Bank and Borrower Performance...................................................... 15 6. Lessons Learned........................................................................................................ 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........... 18 Annex 1 Bank Lending and Implementation Support/Supervision Processes.............. 19 Annex 3. Stakeholder Workshop Report and Results................................................... 21 Annex 4. Summary of Borrower’s ICR and/or Comments on Draft ICR ..................... 22 Annex 5. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 29 Annex 6. List of Supporting Documents....................................................................... 30 Annex 7. Operation Policy Matrix and DLI Matrix Comparison ............................... 301 MAP

i

A. Basic Information

Country: Pakistan Program Name: Sindh DPC

Program ID: P100846 L/C/TF Number(s): IDA-43180

ICR Date: 12/23/2009 ICR Type: Core ICR

Lending Instrument: DPL Borrower: GOVERNMENT OF PAKISTAN

Original Total Commitment:

XDR 65.6M Disbursed Amount: XDR 65.6M

Revised Amount: XDR 65.6M

Implementing Agencies: Government of Sindh

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 07/18/2006 Effectiveness: 06/19/2007

Appraisal: 04/07/2007 Restructuring(s):

Approval: 06/07/2007 Mid-term Review:

Closing: 12/31/2007 12/31/2007 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Substantial

Bank Performance: Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Not Applicable

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Not Applicable

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments

(if any) Rating:

Potential Problem Program at any time

No Quality at Entry (QEA):

None

ii

(Yes/No):

Problem Program at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

General education sector 75 75

Sub-national government administration 25 25

Theme Code (as % of total Bank financing)

Decentralization 17 17

Education for all 33 33

Gender 17 17

Public expenditure, financial management and procurement

17 17

Social safety nets 16 16 E. Bank Staff

Positions At ICR At Approval

Vice President: Isabel M. Guerrero Praful C. Patel

Country Director: Yusupha B. Crookes Yusupha B. Crookes

Sector Manager: Amit Dar Michelle Riboud

Program Team Leader: Reema Nayar Reema Nayar

ICR Team Leader: Benjamin Safran

ICR Primary Author: Benjamin Safran F. Results Framework Analysis

Program Development Objectives (from Project Appraisal Document) The objectives of the SEDPC were to support the Government's reform program which aims to: improve participation, retention and transition, reduce gender and regional disparities, and improve quality in elementary and secondary education (grades 1-10). As a result, (i) net enrollment rates in primary, middle and secondary education are expected to increase from 48%, 18% and 12% in 2004/05 to 59%, 22% and 14% respectively in 2009/10; (ii) the share of girls in primary and in middle schools in rural areas is expected to increase from 36% and 28% to 40% and 31% respectively between 2005/06 and 2009/10; (iii) the transition rate for rural girls between primary and middle school is

iii

expected to increase from 47% to 50% between 2005/06 and 2009/10; and (iv) completion rates and learning outcomes will be regularly monitored. Revised Program Development Objectives (if any, as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Net enrollment rates in primary, middle and secondary education. Value (quantitative or Qualitative)

48%, 18% and 12% 59%, 22% and 14%

51%, 18% and 11%

Date achieved 11/01/2004 11/30/2010 11/30/2007

Comments (incl. % achievement)

Note that these targets are measured three years before expected assessment date and early (first full school year after approval), as these are the latest available data. Enrollment data from the annual census suggest continued improvement into 2008/09.

Indicator 2 : The share of girls in primary and in middle schools in rural areas. Value (quantitative or Qualitative)

36% and 28% 40% and 31% 37% and 32%

Date achieved 11/01/2006 11/30/2010 10/31/2008 Comments (incl. % achievement)

25% and 100% of target achieved respectively. Note that these targets are measured two years before expected assessment date, as there are the latest data available.

Indicator 3 : The transition rate for rural girls between primary and middle school. Value (quantitative or Qualitative)

47% 50% 49%

Date achieved 11/01/2006 11/30/2010 10/31/2008 Comments (incl. % achievement)

66% of target achieved. Note that target is measured two years in advance of expected completion date.

Indicator 4 : Completion rates and learning outcomes regularly monitored.

Value (quantitative or Qualitative)

Sindh Education Management Information System (SEMIS) data reliability for monitoring completion rate was issue; National

Gradual strengthening of provincial monitoring systems.

Continued improvements in quality and reliability of information in Provincial Annual School Census.

iv

Education Assessment System (NEAS)/Provincial Education Assessment Center (PEAC) introduced.

PEACE successfully administered the country's first provincial diagnostic examination representative at district level.

Date achieved 01/02/2006 05/31/2010 04/30/2009

Comments (incl. % achievement)

Target achieved. Quality and reliability of monitoring from provincial census improved. Completion rates show improvements according to both survey and census data (section 3.2). System for regular monitoring of provincial learning achievement in place.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Expenditure on education to increase from 1.2% of provincial GDP with improvements in the composition of expenditure and increase in non-salary O&M as budget is aligned with sector strategy.

Value (quantitative or Qualitative)

Expenditure on education was 1.2% of PGDP in 2005-06. Share of salaries in education budget was 74.5%.

Expenditure on education reaches 1.4% in 2006-07; share of salaries in education budget falls to 68%.

Expenditure on education was 1.5% of PGDP in 2006-07; share of salaries in education budget was 67.7% in 2007-08.

Date achieved 06/30/2006 06/30/2007 06/30/2007 Comments (incl. % achievement)

Target achieved and sustained.

Indicator 2 : Conditional grants transferred from provincial to district governments upon negotiation and signature of partnership agreements to support improved service delivery

Value (quantitative or Qualitative)

Not applicable as conditional grants to districts did not exist.

At least 12 out of 23 districts negotiate and sign partnership agreements and receive conditional grants of Rs. 100 million each.

Partnership agreements signed in all 23 districts in 2007/08 and 2008/09. Conditional grants received.

Date achieved 05/15/2006 06/30/2007 04/30/2009 Comments Target achieved and sustained.

v

(incl. % achievement)

Indicator 3 : Revamped delivery mechanisms improve delivery of textbook and stipends program to intended beneficiaries

Value (quantitative or Qualitative)

Not available but several studies established poor delivery record.

(1) Timely delivery of textbooks to at least 80% of grade 1-10 students. (2) At least 80% of intended girls in middle and high schools receive full amount of stipends.

(1) Textbooks delivered to 86% of grade 1-10 students within first month of 2008/09 academic year. (2) Stipend delivered to more than 90% of intended girls in 2008/09 academic year.

Date achieved 05/15/2006 06/30/2006 04/30/2009 Comments (incl. % achievement)

Target achieved and sustained.

Indicator 4 : New merit based policy in place for transparent recruitment of teachers

Value (quantitative or Qualitative)

No policy for merit based transparent contract based recruitment of teachers.

Policy to be approved in 2006-07 and implemented in successive years.

Policy approved on April 25, 2007. 5,819 teachers recruited on merit in 2007/08, as validated by third party.

Date achieved 05/15/2006 06/30/2007 04/30/2009 Comments (incl. % achievement)

Target achieved and sustained.

G. Ratings of Program Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 12/05/2007 Satisfactory Moderately Satisfactory 99.02 H. Restructuring (if any) Not Applicable

1

1. Program Context, Development Objectives and Design

1.1 Context at Appraisal At the time of preparation of the Sindh Education Development Policy Credit (SEDPC), Pakistan’s economy had seen a remarkable turnaround since 2000 compared to the 1990’s. Economic growth averaged more than 7 percent from 2003-2006, reaching 6.6 percent in 2005/06 despite adverse shocks from its worst ever earthquake and sharp increases in oil prices. This turnaround of the economy was attributed to government initiated reforms, debt restructuring, increased remittances and concessional aid-flows. The government’s prudent expenditure and debt management strategies coupled with improved revenue collection resulted in fiscal improvements. Economic growth generated improvements in human development, with estimates of the magnitude of the poverty reduction over the first half of the decade ranging from 5 to 10.5 percentage points. Despite this progress, however, absolute levels of social indicators were low and compared poorly with those of other countries with similar levels of per capita income. The Federal Government embarked on a comprehensive program to reduce poverty and improve human development outcomes including education. Sindh, Pakistan’s second largest province in terms of both population and economic output, had the highest output per capita among the provinces. Despite its strong relative economic position and growth performance, Sindh performed poorly across a range of socioeconomic indicators. Fewer than half the children of the province’s primary school-age population were enrolled in school. The enrollment rate was particularly low in rural Sindh, where gender gaps were significant. Low enrollment rates in rural Sindh reflect primarily low entry rates into primary schools, but also low transition rates from primary to secondary, as fewer than half of rural girls stayed enrolled for some middle or secondary education after completing primary schooling. Lastly, evidence on student achievement showed low quality of education. Major issues that affected the performance of the education sector and service delivery, thereby causing the aforementioned poor outcomes, were: (i) insufficient and inefficient resource utilization for education; (ii) weak education sector management at all levels; (iii) constraints to increasing access to quality education; and (iv) constraints to improving the quality of teaching and learning. Addressing the low human development indicators in Sindh, and in particular rural Sindh, was critical to achieving Pakistan’s growth and poverty reduction objectives. The Government of Sindh, therefore, adopted a medium-term Education Sector Reform Program (SERP) in 2006. The vision for SERP built on the lessons from the sector reform program initiated in 2001, while further deepening and broadening the reforms. In particular, it recognized that education reforms can only be successful when sector specific reforms aimed at improving quality and access are included in a broader framework that improves accountability in the sector. It also acknowledged that education reforms cannot be successful in isolation, but need to be embedded in a broader reform framework that ensures fiscal sustainability and improves the fiduciary environment. SERP aimed to introduce system-wide reforms, such as approving a new teacher recruitment policy, establishing conditional grants under Terms of Partnership (TOP) agreements between provincial and district governments, instituting a Reform Support Unit (RSU) as an integral part of the education department, and implementing a revamped school census to improve data collection and information on outcomes. It also introduced improvements in incentive programs; the provision for free textbooks to all public school students and stipends to all girls in classes VI

2

through X. These sector reforms were coupled with fiscal reforms to improve sustainability and effectiveness of public expenditures, including education sector expenditures. The World Bank was requested to provide technical and financial support to the development of the reform program. A development policy credit (DPC) was an appropriate instrument to support cross-sectoral areas linked to the education reform program. The SEDPC was intended to be the first of a programmatic series of three development policy credits in support of the government’s medium-term reform program. The Board of Executive Directors in the Bank approved the SEDPC in June 2007. International Development Association (IDA) financing was committed through US$100 million single tranche DPC. Follow up credits of US$100 million each were planned for FY08 and FY09 as part of the programmatic series in support of the government’s medium-term reform program. The series, however, was discontinued, although the Bank continued to support the reform agenda through the Sindh Education Sector Project (SEP), approved by the Board in June 2009 with implementation ongoing (see section 1.6 for more details).

1.2 Original Program Development Objectives (PDO) and Key Indicators The objectives of the SEDPC were to support the Government's reform program which aims to: improve participation, retention and transition, reduce gender and regional disparities, and improve quality in elementary and secondary education (grades 1-10). As a result, (i) net enrollment rates in primary, middle and secondary education are expected to increase from 48%, 18% and 12% in 2004/05 to 59%, 22% and 14% respectively in 2009/10; (ii) the share of girls in primary and in middle schools in rural areas is expected to increase from 36% and 28% to 40% and 31% respectively between 2005/06 and 2009/10; (iii) the transition rate for rural girls between primary and middle school is expected to increase from 47% to 50% between 2005/06 and 2009/10; and (iv) completion rates and learning outcomes will be regularly monitored.

1.3 Revised PDO and Key Indicators, and Reasons/Justification Neither the PDO nor the key indicators were revised.

1.4 Original Policy Areas Supported by the Program The program is built on four complementary pillars designed to improve quality, equity and efficiency in education service delivery, and to increase accountability in the education sector:

I. Improving fiscal sustainability and the effectiveness of public expenditures including in education, through fiscal reforms to enable increases in education and other pro-poor expenditures, to improve budgetary processes and the composition of expenditures; as well as financial management and procurement reforms to increase credibility, transparency and accountability in the use of public resources.

II. Improving education sector management through reforms to strengthen the functioning, capacity and accountability of provincial and district education management in line with devolution objectives, to provide incentives to district governments to improve service delivery, and to strengthen the role of school management committees in school development and monitoring.

III. Improving access to quality schooling, with a particular focus on rural areas and girls,

through reforms to improve the quality and utilization of physical infrastructure in schools, to address implementation bottlenecks and improve the effectiveness of

3

incentive programs (free textbooks and girls’ stipends), and to launch partnerships with the non-government/private sector to increase access and quality.

IV. Improving the quality of teaching and learning, through instituting merit-based

recruitment and improved accountability of teachers, establishing a competency-based system of teacher education and continuous professional development, and through strengthening the assessment of student achievement in the provincial education system.

1.5 Revised Policy Areas The original Policy Areas were not formally revised.

1.6 Other significant changes The SEDPC was designed to be the first in a series of three programmatic DPCs to support the government’s own reform program. The series, however, was discontinued after the first education sector credit. The Bank instead continued to support the sector-wide reform program through a sector investment credit, and the SEP was approved by the Board of Executive Directors of the Bank on June 4, 2009. The SEP design reflects the lessons learned from the SEDPC. It builds on the foundation laid under the policy credit, following the design and adoption of significant policies and reform programs. The change in instrument maintained the focus on completing predetermined results for disbursement as in DPCs, but included an enhanced focus on implementation outcomes through the achievement of measurable targets (known as disbursement linked indicators or DLIs). It also provided enhanced focus on strengthening associated provincial systems through technical capacity building and implementation support. Lastly, the new instrument provided continued assistance over a three-year period, disbursing semi-annually, reflecting the Bank’s commitment and predictability of financing, predicated on achievement of results.

2. Key Factors Affecting Implementation and Outcomes

2.1 Program Performance Overall, the implementation of the government’s program, the SERP has been commendable, with achievement of result targets broadly as anticipated. The prior actions supported by SEDPC laid the foundation for strong implementation of the reform program. The reform program was designed around four core pillars to improve quality, equity and efficiency in education service delivery, and to increase accountability in the education sector. SEDPC supported reforms to establish the needed credibility through up-front governance reforms cutting across the pillars, critical to improving quality, equity and efficiency. In addition to the achievements of the prior actions for SEDPC (see Table 2), the Operation Policy Matrix set annual targets for each policy area, providing the framework that guided implementation between project approvals. Although the series was discontinued after the first credit, the Policy Matrix, in particular year two targets, still guided implementation and ultimately set the foundation for the SEP DLI matrix. Annex 7 shows how performance benchmarks/triggers originally anticipated under the second DPC were achieved, and in some cases surpassed by the achievements of year one DLIs in the SEP, while being accompanied by strengthening particularly in fiduciary and safeguard aspects. The program performance is summarized pillar by pillar below: Pillar I: Improving fiscal sustainability and the effectiveness of public expenditures To enhance sustainability of provincial finances while creating additional fiscal space for spending on education reforms, the government approved a Medium Term Fiscal Framework

4

(MTFF) for FY08-10. In addition, to improve the level and quality of education expenditures, the government approved a medium-term sector policy framework for education and an accompanying medium-term financial plan. Table 1 shows the MTFF and education sector policy framework targets were met or exceeded, and have been sustained through 2008/09.

Table 1: Pillar I Fiscal Targets and Achievements

A. Medium Term Fiscal Framework

Indicator Target

(2005/06-2009/10) Actual

(2005/06-2008/09)

Increase in own source revenue 13.5% p.a. 16.5% p.a.

Increase in interest and subsidy expenditure 5% p.a. (maximum) Declined by 4% p.a.

Increase in PRSP expenditure1 16% p.a. 16.6% p.a.

B. Education Sector Policy Framework

Indicator Target Actual

Increase in non-salary recurrent expenditures 30% p.a. through

2009/10 56% p.a. through 2008/09

Education expenditures (as % of PGDP) 1.5% by 2009/10 1.5% in 2008/09

Increase in overall education sector budget 26% from 2005/06 to

2006/07 26.7% from 2005/06 to

2006/07

To increase effectiveness of public expenditures, including on education, financial management and procurement reforms were accelerated. An Executive Order was issued to revitalize Department Accounts Committee meetings, and hence review and resolve audit observations raised against them. By 2007/08, more than 35 percent of the 7,679 outstanding audits were resolved, exceeding the target for the third year of the program after just one year of implementation. In addition, a revised procurement law was passed in April 2009, followed by the notification by SPPRA of procurement rules and notification of quarterly updates of the CSR to align it with market rates. These procurement reforms will ensure the use of (i) revised standard bidding documents (SBDs) in line with international best practice; and (ii) regular quarterly updates of the Composite Schedule of Rates (CSR) to align it with the market, both of which are necessary reforms to increase transparency and efficiency in public procurement. These actions resulted in improved medium-term fiscal policy and created space for spending on education reform. As a result, the education sector budget increased by 84 percent over the three year period of 2005/06 to 2008/09. In addition, SERP was fully financed for 2008/09, in alignment with the MTFF framework, showing the sustainability of these reforms as they continued after implementation of SEDPC.

Pillar 2: Improve education sector management

1 Increase in PRSP expenditures calculated through 2007/08 as this is the latest year with actual, not budgeted, expenditures.

5

Capacity constraints, at all levels of the government, were a systemic issue for service delivery. To strengthen provincial capacity, the RSU was established as an integral part of the education department. Tasked with policy formulation, monitoring and evaluation functions, and donor coordination, the RSU became the primary implementing and coordination agency for most of the reform initiatives. To strengthen monitoring and evaluation activities, a revamped public school census was implemented for academic year 2005/06. This provided key new information (on teachers’ positions and qualifications, school infrastructure, conditions, etc.) that previously was not collected, and provided the data used to establish of baselines for the program’s key performance indicators. In addition, data reliability improved, in large part due to capacity improvements and equipment upgrading at both at the provincial (SEMIS) and district (DEMIS) levels. Census administration has continued to improve both in terms of quality and timeliness since the first ‘revamped’ census. The 2008/09 census was administered in fewer than five months, including implementation of a sample based data verification exercise for the first time. To address service delivery capacity constraints, particular in the districts, the province launched a process to strengthen education management at all levels. A review of roles, responsibilities and organization structures in education management at provincial and district levels was completed for 11 districts in 2008/09. These findings created the basis for an education management reform (EMR) program. Implementation of the EMR is underway, targeting the introduction of education management structure, posts, job descriptions, training, district education development plans and quality assurance of schools in at least 2 pilot districts during the 2009/10 academic year. Another objective of the reform program was to increase community participation through school management committees (SMCs). While not a condition for disbursement under the SEDPC, mobilizing and establishing SMCs in middle and secondary schools was a target activity under the pillar 2 reforms. SMC guidelines on composition, appointment procedures, roles and responsibilities were developed, but the mobilization campaign was delayed beyond the implementation of the SEDPC. While this was a shortcoming of the project implementation, SMC mobilization was a key initiative of the SEP, with a target of 60 percent of all functional primary and secondary schools having active SMCs, including receiving annual budget allocations from the provincial government during the 2009/10 academic year. Thus far, more than 18,000 SMCs have been activated, receiving guidelines and funding via direct deposits into newly established SMC bank accounts. Pillar 3: Improving access to quality schooling To make immediate progress in increasing the availability of minimum physical facilities in government schools, grants of PKR 100 million, conditioned on signing terms of partnership (TOP) agreements, were released to districts to increase the number of schools that meet minimum physical standards. These reforms aimed to improve the quality of school rehabilitation by districts (in line with devolution) and were also designed to improve education sector management by introducing accountability and performance incentives for improved service delivery. Numerous implementation issues in needs-based project identification, documentation, construction quality and completion were found by supervision field visits and third party validation (TPV) reports. Therefore, for the second round of TOP funding, SEP supported the contracting of a third party construction supervision firm to vet the set of proposed projects, provide form designs to ensure minimum levels of quality, supervise construction, and validate completion of all works, ensuring quality and accountability. In addition, in year two of the TOP program, grants to districts were conditioned on performance, with the strong performers

6

receiving more than PKR100 million, and weak performers receiving less, further incentivizing performance for future rounds. The government continued to roll out its programs to reduce costs and encourage families to send their children to school through the expansion of the middle school girls stipends program to include high schools (class IX and X), and through the extension of the policy of free textbooks up to class X. New delivery mechanisms for both these programs were introduced to improve monitoring and efficiency. As a result, more than 80 percent of students in classes I through X received free textbooks in the 2006/07 academic year. In addition, the annual stipend for the 2005/06 school year was delivered to about 80 percent of eligible girls, dramatically improving delivery success, as previously payments often arrived as just a fraction of the intended amount, if at all. Issues of delays still existed, however, as payments were received with a one year lag. The improvements under SEDPC have continued to be built upon, and are supported by SEP. Textbooks were delivered to more than 86 percent of all students within the first month of the 2008/09 academic year. Stipends reached more than 90 percent of intended beneficiaries during the same academic year, a delivery of more than 300,000 girls. Monitoring and evaluating these programs will be further enhanced with TPV reports planned for 2009/10 for stipends delivery and in 2010/11 for both textbook and stipends deliveries. Recognizing the key role that the private sector plays in education, SERP envisioned the establishment of a partnership with the private sector to enhance the access to schooling. A private school census was launched to assess the existing private school network. Ultimately, this commitment first voiced during implementation of the SEDPC lead to the development of the Public Private Partnership for Rural Schools (PPRS) program. Implemented by the Sindh Education Foundation (SEF), the PPRS partners with entrepreneurs in underserved localities (identified as those with the nearest school more than 1 km away) and distributes a per student subsidy to these entrepreneurs to provide schooling as long as they maintain a minimum level of learning and infrastructure quality. The PPRS launched with 200 schools in 2009 and currently supports more than 16,000 students. Plans are for the program to increase participation to as many as 1,000 schools over the next two years. The program is fully funded by the GoS, implemented through the SEF, and supported by the SEP, with an on-going impact evaluation to assess the outcomes of this innovative program. Pillar 4: Improving the quality of teaching and learning Reforming teacher recruitment procedures was one of the most challenging initiatives under SERP because of Sindh’s history with patronage-based appointments. Despite this challenge, the GoS committed to transparent merit-based recruitment of teachers on school specific contracts by approving a new teacher recruitment policy. The policy specifies formula-based selection using a candidate’s test score (in an independently administered test), domicile, qualification (both professional and academic), and gender as the predetermined criteria. In the first round of recruitment, approximately 7,000 of the 130,000 applicants scored above the 60 percent pass threshold, with just over 5,800 teachers ultimately appointed positions. A TPV exercise determined the first round was satisfactory implemented. Candidates were appointed according to merit for the first time in the history of the province, with all newly appointed candidates scoring above 60 percent on the entrance exam, and no government teachers joining the teaching force outside the merit-based recruitment process. To increase accountability for student learning outcomes and strengthen assessments, staff positions for the Provincial Education Assessment Center (PEACE) were created within the Bureau of Curriculum (BOC). The creation of PEACE was the first step toward institutionalizing assessments to provide regular feedback on the provincial education system. A schedule of

7

diagnostic assessments, the first assessments representative at the district level for any province in the country, has been set as part of the SEP, and class IV math diagnostic was implemented in academic year 2008/09. Three additional diagnostic exams are scheduled over the next two academic years, with corresponding analysis and reports to be disseminated subsequently. Table 2 below details the set of prior actions for the SEDPC all of which were met in a timely manner.

Table 2 : Prior Actions for SEDPC Sindh Education Sector Development Policy Credit List conditions from Legal Agreement/ Program Document Status Pillar 1: Improving Fiscal Sustainability and Effectiveness of Public Expenditures Objective: Provincial medium-term fiscal policy is sustainable and creates space for education reforms

• The Government approves a Medium-Term Fiscal Framework (MTFF) for FY08-FY10 which shows at least a 13.5% p.a. increase in own source revenue between 2006/07-2009/10; keeps increases in interest and subsidy expenditure below 5% p.a.; and increases PRSP expenditures by at least 15% p.a.

Completed

Objective: Increase level and quality of education sector expenditures in line with education sector policy.

• The Government approves a medium-term sector policy framework for education and a financing plan aligned with sector policy which projects annual increases in nominal non-salary O&M allocations of 30% p.a. and an increase in education expenditures to 1.5% of PGDP by 2009/10. The 2006/07 education budget is aligned with sector policy and shows an increase of over 26% in overall budget for education.

Completed

Objective: Increase effectiveness of public expenditures (including in the education sector) through financial management and procurement reforms

• The Government issues an Executive Order requiring regular DAC meetings in order to review and settle all ‘advance’ audit observations on the accounts of Sindh’s agencies and departments at both provincial and district levels to progressively clear their backlogs

Completed

PILLAR 2. IMPROVING EDUCATION SECTOR MANAGEMENT Objective: Strengthen devolution by increasing provincial capacity in policy formulation, standard setting, technical support and monitoring and evaluation while strengthening incentives and capacity of districts to deliver on education sector service delivery objectives.

• Reform Support Unit established and fully functional as integral part of the education department, with responsibility for education policy formulation, monitoring and evaluation and donor coordination in education sector.

• Implement a revamped school census in order to establish baselines for key performance and input indicators at provincial and district level.

• GoSindh negotiates and signs terms of partnership (TOP) agreements with at least 12 district governments and releases conditional transfers upon signature. The agreements set out provincial and district government responsibilities in education service delivery together with associated performance indicators, their baselines and annual targets.

Completed

PILLAR 3. IMPROVING ACCESS TO QUALITY SCHOOLING PARTICULARLY IN RURAL AREAS AND FOR GIRLS Objective: Improve the quality and utilization of public school infrastructure and provide incentives for school attendance, particularly for girls from poor families

Completed

8

• Deliver textbooks to at least 80% of grade 1-10 students within one month of the start of the school year, and deliver stipends to at least 80% of intended beneficiaries (girls in middle and high schools), based on revamped delivery mechanism, and launch third party validations of both interventions

PILLAR 4: IMPROVING THE QUALITY OF TEACHING AND LEARNING Objective: Improve recruitment and management of teachers

• GoSindh notifies new policy for transparent, merit-based recruitment of contract teachers under school specific non-transferable contracts, based inter alia on scores on tests administered by a third party, domicile of the candidate, gender, and academic qualifications.

Completed

2.2 Major Factors Affecting Implementation: The substantial implementation progress of SERP was a result of a combination of factors, many of which positively affected implementation, but some of which slowed and even hindered progress: The provincial reform program was closely aligned to major national level initiatives, as well the reform programs in other provinces (Punjab in particular). The alignment of national and provincial reform programs is seen in the national PRSP and comparable Punjab Education Sector Reform Program (PERSP). Thus the reforms were not isolated attempts to improve the education sector in Sindh, rather part of the larger national agenda which helped ensure political commitment and maintain continued commitment to SERP across two political governments. Donor harmonization was critical, ensuring that a single program was being supported by DPs and minimizing the government’s capacity constraints. Donor alignment between the World Bank and EC was exemplary. The use of the same policy matrix focused the reform agenda on a single program and reduced the reporting burden on the RSU. Joint review missions allowed for a more fluid and concentrated transfer of ideas between the development partners, and between the donors and the government. Regular and sustained dialogue between the Bank and senior level Government officials and various stakeholders at the district and school level played a key role in facilitating continuity in SERP implementation by the new government that came in early 2008, particularly in sector governance reforms. At the same time, technical inputs from Bank staff and consultants through, for example, analytical reports and a variety of just-in-time policy and technical assistance notes have supported design and implementation. Risks were well identified and appropriately mitigated. There were a number of risks identified in project preparation that threaten the sustainability of the reform program, such as uncertainty in political stability and capacity constraints. These risks, however, were well identified and mitigated through project design and preparation (see section 4 for more details).

Continuity in the Bank’s support and message were central to guiding implementation. Despite a lag in Bank’s financial assistance after the approval of SEDPC, the framework provided by the Operation Policy Matrix guided implementation after approval and before preparation of SEP. Consistency between Operation Policy Matrix and DLI Matrix provided additional emphasis on key reforms.

Despite these positive factors, numerous factors slowed down implementation. First, in anticipation of elections, the government’s attention was diverted from implementation of the

9

reform program. Politically difficult reforms, such as merit-based teacher recruitment, experienced a loss of commitment as elections drew nearer. Subsequently, the unforeseen political unrest in the wake of Benazir Bhutto’s assassination stalled implementation during this time due to continued political uncertainty, riots, and civil unrest. While the commitment was renewed when the new government took over in 2008, the result was more than six months of lost implementation. Lastly, the macroeconomic growth seen in advance of project preparation was not sustained during implementation, due in part to the various crises (food, fuel and financial) that likely increased the opportunity cost associated with going to school (or sending one’s children to school). The steady progress in achievement of the development objectives is especially commendable when considering these factors. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization:

M&E Design: Adequate performance indicators were provided to monitor progress towards achieving the PDOs. As described in the program document, data reliability for monitoring was historically an issue. Therefore, continued strengthening of provincial monitoring systems was critical in determining baselines for development objectives and key project indicators. Household survey information from the PSLM/CWIQ provided complementary information used to triangulate and corroborate the census data, as well as the primary source for evaluating certain PDOs. Overall, in large part due to improvements during project preparation, M&E design was adequate to establish indicators to monitor project and program performance. M&E Implementation: Monitoring and evaluation during implementation was exemplary, strengthened through multiple channels. First, capacity building of the SEMIS within the RSU strengthened linkages between data collection, monitoring, analysis and policy planning. The core output of this system, the new public school census provided significantly improved information on an annual basis, increasingly reliable due to validation exercises and performance incentives at the district level. Second third party validation exercises were used to monitor performance on certain activities, such as evaluating compliance with the teacher recruitment policy and incentive programs. Third, the RSU commissioned several independent studies on specific policy issues,such as the effectiveness of SMCs or the viability of closed schools, to help inform policy discussions. Fourth, a provincial assessment system of student learning was established to enable feedback to improve learning quality. Establishing the baselines and ability to monitor learning quality is the first step in addressing this critical element of the education system. In addition, a number of impact evaluations for key interventions, such as the public-private partnership initiative, were planned at the time of SEDPC preparation and are now underway as part of the SEP. These will provide valuable information to inform future policies in the province, the rest of the country, and even globally. M&E Utilization: The use of data collected as a result of the strengthened monitoring and evaluation systems began during project implementation and continued through the preparation and implementation of the SEP. For example, the improved census was used to establish targets for the SEDPC implementation, as well as aid in the design of the SEP. As a result of improvements in data quality, the department has begun using these data to plan and implement policies and design programs. For example, information on all public school teachers was collected for the first time, which enabled identification of schools with a dearth of resources (high student-teacher ratios) to be given new teachers under the merit-based recruitment initiative. 2.4 Expected Next Phase/Follow-up Operation:

10

The discontinuation of the DPC series after only the first of three planned credits was done in the context of the preparation of the SEP, which was approved June 4, 2009 by the Board ($300 million over three years). The lessons learned from the SEDPC were incorporated into the design of the SEP, and many of the initiatives build on the momentum initiated under the SEDPC. The SEP, instead of continuing in the series of DPCs, is a specific investment credit, disbursing against achievement of DLIs. As mentioned in section 1.6, this innovative instrument maintained the focus on completing predetermined results for disbursement as in DPCs, through DLIs, which had an enhanced focus on implementation outcomes and also through technical assistance to strengthen provincial systems (e.g. fiduciary, safeguards, and monitoring and evaluation) and for capacity building and implementation support. The Operation Policy Matrix year two targets set the foundation for the SEP DLI matrix. Annex 7 shows how performance benchmarks originally anticipated under the second DPC were achieved, and in some cases surpassed by the achievements of year one DLIs in the new project. In addition, the SEP provides continued assistance over three years, reflecting the Bank’s commitment and predictability of financing, predicated on achievement of results.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives: The objectives of the project remain relevant and consistent with the Bank’s Country Assistance Strategy for 2006-2009. This CAS emphasized strengthened macroeconomic management and resource utilization (CAS Pillar 1), strengthened governance and service delivery (CAS Pillar 2), and improving lives and protecting the vulnerable. These priorities are core elements of the program objectives (PDO 2 and PDO 3) and design (pillar 1 and pillar 2 reforms). The project also contributed to Pakistan’s priority for investing in human development articulated in PRSP II (Government of Pakistan, 2009), finalized in January 2009. PRSP II emphasizes the need (i) to address financing for the sector; strengthen planning and implementation capacity; (ii) improve resource utilization; (iii) enhance governance for greater accountability of education providers to the community; (iv) build capacity of district and local institutions; and (v) strengthen the role of school committees; all of which are central in SERP and SEDPC. Design and Implementation: The design of the project is relevant, reflecting proper diagnosis and plans to meet the development priorities. The program’s focus on (i) implementation capacity, (ii) governance reforms; (iii) strengthening monitoring and evaluation; and (iv) improved service delivery for underserved groups were appropriately identified as key issues for effective service delivery. The Bank’s implementation assistance was responsive to the government’s needs. The Bank team engaged in high-level policy dialogue throughout project implementation, and provided continued technical assistance. In addition, the shift in Bank support from the series of policy credits to investment and results based financing reflected the enhanced focus on implementation outcomes and the need to strengthen associated provincial systems through technical capacity building and implementation support.

3.2 Achievement of Program Development Objectives PDO 1: Improve participation, retention and transition PDO Indicators: (1) Net primary, middle and high school enrollment rates; (2)Total public school enrollment; (3) Transition rate from primary to secondary school.

11

There has been satisfactory progress made toward this objective. SEDPC supported SERP to achieve this objective in two key ways. First, fiscal (pillar 1) reforms resulted in, among other benefits, increased education sector spending and created space to finance interventions in the reform program. Second, SEDPC supported capacity and system improvements to successfully execute the reform agenda which led to improved performance of key sub-programs and therefore improve service delivery. The results were improved inputs, quality and ultimately demand for education sector services. Construction of new and missing facilities under the TOP subprogram resulted in approximately 2,400 additional classrooms, including more than 500 formerly shelterless schools receiving a new school building. Under the PPRS initiative, nearly 200 schools have been established to date with more than 16,000 students enrolled in these schools. Nearly 6,000 teachers were recruited under the new merit-based teacher initiative. Additionally, the improvements in the performance of incentive programs have resulted in the annual delivery of more than 20 million textbooks and more than 300,000 girls’ stipends payments since 2006/07. Improved inputs into education increased accessibility to quality facilities that otherwise would not have been present, especially in disadvantaged areas, which likely induced out of school children into schooling. Incentive programs reduced the direct cost of schooling, which potentially encouraged households to choose to send children to school when they might not have otherwise, of particular value during a crisis. And improvements in quality, both of learning conditions and teachers, could nudge households to decide to send all or some of the children to school if the perceived value of schooling increased. As a result, net enrollment rates in primary school showed steady growth from 48 percent in 2004/05 to 51 percent in 2007/08. Over the same period, net middle school enrollment rates increased from 17 percent to 18 percent and net enrollment rates in high school remained constant at 11 percent. While the achievement is short of the targets set in the program document, there are a number of factors that, when taken into consideration, warrant the satisfactory rating: (i) targets were set to be achieved in 2010/11, and therefore are assessed three years in advance of completion date (and only one year after project approval); (ii) relative to annual targets set in the program document, primary net enrollment rate (NER) exceeded the 2007/08 target, and middle and high NER are 1 and 2 percentage points below the respective annual target; (iii) there was a slowdown in implementation experienced just after project approval (see the final paragraph of section 2.2 for further details), after which implementation progress picked up, and therefore the indicators are expected to show accelerated progress in the coming years; (iv) enrollment targets were considered ambitious when set, and have since been revised downward as part of SEP. The annual school census provides more recent data that shows continued strong improvements in public school enrollment. Total public school enrollment increased by 7 percent from 2005/06 to 2008/09, or nearly 280 thousand students, from just over 4.06 million students to over 4.34 million students. In rural Sindh, public school enrollment increased by 10 percent, or more 320 thousand students, over this period. This achievement is in line with the design of the reform program, which specifically targeted rural areas because of the poor enrollment performance and regional disparities. While there was a decline in public school enrollment in urban areas, this is likely explained by an increased presence of private sector in urban areas, supported by PSLM data showing stagnated and declining NER for public primary and secondary schools in urban areas compared to increased NER in urban private schools at the primary and secondary levels. Lastly, transition rates from primary to secondary school improved in rural Sindh as well, increasing from 54 percent to 56 percent from 2005/06 to 2008/09. These improvements were driven by improvements for rural girls, specifically targeted in the program (see PDO 2 for

12

further detail). There was, however, a decline in public school transition rates in urban areas. This trend, again, in likely explained by the shift from public to private sector in urban areas, as public secondary NER decreased by 3 percent p.a. from 1998/99 to 2006/07 compared to an increase in private secondary NER of 5 percent p.a. over the same period. PDO 2: Reduce gender and regional disparities PDO Indicators: (1) Share of girls in primary schools in rural areas; (2) Share of girls in middle schools in rural areas; (3) Transition rate for rural girls from class V to class VI. There has been satisfactory achievement of this objective. Stronger capacity and budgetary allocations supported by SEDPC (as noted in PDO 1) resulted in successful implementation of initiatives targeting the most disadvantaged groups: rural households and females. Revamped delivery mechanisms for incentive programs that set specific performance targets resulted in significant performance improvements. As a result, free textbooks were delivered to more than 80 percent of all students in class I through class X in academic year 2007/08. This performance was sustained in academic year 2008/09 with more than 86 percent of students receiving textbooks within the first month of the academic year; a delivery of more than 20 million textbooks. There was also improved performance of the delivery of stipends to all girls in class VI to class X. More than 90 percent of intended beneficiaries received the subsidy payment in academic year 2008/09; a total of more than 300,000 beneficiaries. In addition to improved implementation performance of existing incentive programs, SEDPC supported the design and introduction of new innovative programs targeted to reach the most disadvantaged groups. The differential stipend program (DSP) focused on reducing the rural gender disparities. Launched in 2007/08, the program provided a higher stipend amount to girls in low transition rate (rural) talukas. The aforementioned PPRS program also targeted improvements in access in the rural areas. And school rehabilitation through the TOP often brought infrastructure improvements to the rural areas where the need was the greatest. Improved or new facilities, such as construction of missing toilets, are especially critical for girls, particularly towards the end of primary school and at higher levels. Incentive programs reduced the direct cost of schooling, which potentially nudged households to send children to school when they might not otherwise, especially for poor families who often decide to keep some of the children (usually female) out of school to assist in household work. The successful implementation of these programs translated into achievement of the development outcome objective. The share of girls in primary schools in rural areas has increased from 36 percent baseline in 2005/06 to 37 percent in 2008/09. This represents progress towards achievement of the target, and high likelihood of achievement by the target date. In addition, the share of girls in middle schools in rural areas has increased from 28 percent to 32 percent over the same period, exceeding the project target two years in advance of expected assessment date. This is caused, in part, by the increase in the transition rate for rural girls from class V to class VI from 47 percent in 2005/06 to 49 percent in 2007/08, nearly achieving the project target value two years in advance of expected achievement date. Rural boys transition rate over the same period remained constant at 60 percent, consistent with the explanation that the programs that targeted girls transition rates (i.e. stipends) contributed to the achievement of the PDO.

Achievement of this PDO by improving access and equity to the most disadvantaged is a key step to reduce the gender and regional disparities and progress towards achieving the MDGs of universal primary completion and elimination of gender disparity in primary and secondary education.

13

PDO Objective 3: Improve quality in elementary and secondary education PDO Indicators: (1) Introduction of provincial assessment system; (2) Number of teachers hired on merit; (3) Regular monitoring of completion rates. There has been satisfactory progress towards achieving this objective. Improving the measurement of learning is an important part of any strategy to improve quality. SEDPC facilitated the strengthening of PEACE as the agency with the primary responsibility for evaluating learning quality across the province. A diagnostic assessment in class IV mathematics was successfully administered by PEACE in academic year 2008/09 for more than 4,000 students. These results will be representative at the district level, the first time such data will be collected in the country’s history. In addition, PEACE has an ambitious but achievable schedule of diagnostic assessments and associated reports over the next three years. These will establish baseline and targets for improvements in learning quality throughout the province. The approval of the policy for merit-based teacher recruitment, a prior action under SEDPC, has been a significant step in improving teacher quality. Historically, appointment of teachers was done irrespective of merit, often on referral or patronage basis. In the first round of teacher recruitment, more than 5,800 teachers were recruited on merit, as validated by a TPV exercise. Newly appointed teachers passed the entrance exam, and no teachers were appointed outside this process after the policy approval date. The TPV also found lower rates of absenteeism among contract teachers, with an absenteeism rate of 16 percent among new recruits compared to 21 percent among regular teachers. In addition to the enhanced quality of teachers, improvements in the quality of learning facilities (discussed in PDO 1 and 2) also contributed to the overall improvements of quality in elementary and secondary education Lastly, improvements in the quality of the annual school census have translated to improved ability to monitor completion rates. Early census administration showed a contrasting picture on completion rates; which were high according the household survey data, yet much lower in the 2005/06 census. The administrative data showed a high dropout rates within primary, in particular between class I and II. This discrepancy likely reflected a large number of children who register for first grade but never attend for any meaningful length of time. The class I to II ‘dropout rate’ in the census decreased from 35 percent to 26 percent from 2005/06 to 2008/09, improving the reliability of the census data in monitoring completion rates. According to these data, primary completion rates2 increased from 35 percent to 49 percent from 2005/06 to 2008/09. In addition, completion rates increased according to household data3 as well, from 90 percent in 2005/06 to 93 percent in the 2006/07. Completion and transition rates are considered proximate indicators of quality. In addition to the improvements in completion rates, improvements in transition rates from primary to secondary (discussed in PDO 1) provide further indication of improvements in education quality.

2 Primary completion rate is calculated as number of students in class V divided by number of students in class I. While this definition likely over states the true completion rate as it does not subtract the number of repeaters in class V, data on number of repeaters was not collected in 2005/06 census, so consistency of definition is used to match data availability.

3 Calculated using the PSLM data as the share of individuals aged 15-19 completing primary school, conditioned on ever attending school. The difference in definitions accounts for the differences across the completion rate statistics.

14

3.3 Justification of Overall Outcome Rating Rating: Satisfactory

The achievement of the development outcomes is rated satisfactory, reflecting the substantial achievement or progress towards achievement for most of the key performance indicators. While some of the indicators have not fully achieved the targets, this assessment is limited to occur well in advance of target completion date. Nevertheless, there has been substantial progress across intermediate outcome indicators and PDOs, and in many cases achievement of targets well in advance of projected completion dates. The substantial achievement in most of the key project development outcomes has contributed to improvements in the access, equity and quality of the education sector. In addition, the PDOs remained relevant throughout project implementation and continue to remain relevant today. These achievements are expected to have long-term benefits of positive impact on levels of human capital and economic development as the investments are sustained.

3.4 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Numerous elements of SERP, supported by SEDPC, are designed to improve equity across regional and gender disparities. For example the incentive deliveries under SERP helped increase enrollment by reducing the direct cost of schooling and providing incentives for households to send their children to school. As a result, PDO 2, aimed at reducing gender and regional disparities, was achieved (see section 3.2 for more detail). In addition, from 2006/07 to 2007/08, the ratio of (i) female to male primary school NER increased from 77 percent to 84 percent; (ii) rural to urban primary NER increased from 64 percent to 71 percent; and (iii) rural female to rural male primary NER increased from 63 percent to 69 percent. Improvements in gender and regional disparities are illustrated in Figure 1 below.

Figure 1: Gender and Regional Disparities have narrowed

(b) Institutional Change/Strengthening Implementation of SEDPC has resulted in many significant institutional changes, thereby strengthening capacity. Some of the most significant areas of improvement are listed below:

i. Fiscal sustainability and effectiveness of public expenditures have been improved through the approval of a Medium-Term fiscal framework and financial management and procurement reforms.

ii. Education department capacity has been strengthened through the formation of the RSU, which now is the primary implementing agency for policy formulation, monitoring and evaluation, donor coordination of many of the reform initiatives.

15

iii. To address weak capacity at the district level, the EMR initiative was designed and approved on the basis of a review of roles, responsibilities and organization structure in education management. Implementation of this reform, targeting the introduction of a new education management structure, posts, job descriptions, training, district education development plans and quality assurance of schools, is underway.

(c) Other Unintended Outcomes and Impacts SERP implementation with the support of SEDPC contributed to an increased demand for Bank support in Sindh, both in the education sector and outside. The Sindh Skills Development Project is currently under preparation for the technical and vocational education and training sector in Sindh. In addition, high-level dialogue has led to increased requests for technical assistance from the Bank team in implementing sector programs that are outside of SERP. For example, the GoS had engaged the Bank team for consultations on Early Childhood Development and School Based Feeding programs, both in the education department’s portfolio, but not part of the reform agenda.

3.5 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable.

4. Assessment of Risk to Development Outcome Rating: Substantial

There are a number of risks that could threaten the sustainability of the reform program, and ultimately maintaining the progress and achievement of the PDOs. These include: (i) uncertainty in political stability potentially contributing to waning/declining political commitment to the reform program; (ii) capacity constraints, despite dramatic improvements to date; and (iii) questions of national security that may limit Bank supervision and implementation support. There are numerous mitigation measure that have been undertaken as part of both SEDPC and SEP, notably: (i) intensive, high quality and continued dialogue between the Bank team and highest levels of government; (ii) strong emphasis on continued capacity building for the implementation of reform initiatives; and (iii) systematic third party validations and ability for video-conference supervision in the event of restricted mission. Despite these mitigation measures, however, the significant uncertainties justify the substantial rating of the risk to development outcomes.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory

The Bank performance for ensuring quality at entry is rated as satisfactory. The Bank team remained proactively engaged with the government on the program design, including policy dialogue, reinforcing ownership and active participation while consulting various stakeholders, from provincial to district government. In addition, the project design reflected lessons learned from the Sindh Structural Adjustment Credit (SSAC), building on project experience. For example, the dedicated M&E team in the RSU was established by merging the Research Monitoring and Evaluation Cell of the education department initiated under SSAC with the SEMIS to create a central authority responsible for data quality and credibility. In addition, project objectives established at entry remained relevant through implementation and continue to

16

be in line with country priorities because of sound design focused on addressing key poverty, gender and social development aspects. (b) Quality of Supervision Rating: Satisfactory

The quality of supervision is rated as satisfactory. The Bank’s task team was composed of Human Development, Poverty Reduction and Economic Management, Financial Management and Procurement, and Social Assessment staff, complemented by international and national consultants who contributed significantly. The cross sectoral support was exemplary, with seamless integration of reforms across the various sectors in the project. The close collaboration with the government during preparation and negotiation continued through implementation, and ultimately the preparation of the SEP. In addition, strong donor coordination, notably with EC, enabled better alignment of technical expertise and sources while reducing the reporting burden for the government. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory The Bank team worked closely with the government, proactively engaged in design and preparation of the project. Strong dialogue with key policy makers was maintained to ensure quality during implementation. The team, consisting of staff from various sectors of the Bank, maintained efficient supervision and monitoring of the program. The engagement and close collaboration ensured a continued dialogue, resulting in the preparation of the SEP. The consistency of the Bank’s message, in the form of the Operation Policy Matrix and subsequent DLI Matrix allowed for focused implementation between the approvals of the two projects.

5.2 Borrower Performance (a) Government Performance The government and implementing agency are the same so the discussion and rating are given below in section (c). (b) Implementing Agency or Agencies Performance

(c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory

The government showed high ownership and commitment to implement the project at all levels and across departments. This was demonstrated by instituting crosscutting reforms, both in the education sector and in public finance, and making considerable progress to improve service delivery. Capacity issues were addressed with the creation of the RSU which played a key role in developing policy, implementing reform programs and coordinating with donors. Monitoring and evaluation systems were greatly strengthened with the launch of the revamped school census, which provided quality and timely data that aided decision making and performance monitoring. The Borrower ensured quality at preparation and implementation and complied with covenants and agreements, ensuring progress toward the achievement of development objectives.

17

6. Lessons Learned Numerous lessons can be learned from the implementation of SERP, many of which were included in the design of the SEP: Strong political commitment and ownership are critical for successfully steering a sector-wide program and for addressing governance constraints to effective service delivery. Many changes required challenging historical expectations and vested interests. One example is the success in implementing a merit-based teacher recruitment policy in Sindh, in a significant break from past practice. A sector program that combines sector-specific interventions with cross-cutting fiscal and fiduciary interventions is more likely to yield results and to be sustained. It also contributes to increased ownership of the sector program beyond the line department. A strong partnership between key stakeholders – Planning and Development, Finance and Education Departments (P&D, FD and ED respectively) – facilitated increased implementation momentum. Complementary non-lending analytical and advisory support and sustained dialogue is essential.Bank engagement at the highest political levels played a key role in supporting the new government in implementing improvements in sector governance. At the same time, technical inputs from Bank staff through, for example, analytical reports and a variety of just-in-time policy and technical assistance notes supported design and implementation. Given the educational context in Pakistan, it is important to simultaneously address the issues of school access and quality. Experience shows that the quantity of education (enrolling in school) is only one factor which affects the socio-economic prospects of individuals. The level of learning (quality of education) is also a powerful determinant. It is likely that the simultaneous focus in SERP on inputs and reforms to improve quality also improved access.

A good monitoring and evaluation (M&E) system that provides timely and reliable information is essential. A well-designed and comprehensive M&E framework needs to be integrated into the design of any sector-wide program. Establishment of baselines, followed by intensive monitoring, and verification/triangulation from different data sources increases the credibility of and accountability for results. The use of data in planning and monitoring itself helps focus attention on the quality of monitoring data and on improving their reliability. A reliable and comprehensive monitoring and evaluation system can contribute to the political sustainability of programs; it also facilitates mid-course program corrections and refinements. Systematic communication can play a key role in ensuring wider understanding and ownership. Strategic communications can increase understanding among the general public, media and civil society, local communities as well as internal stakeholders such as teachers or other employees. This involves two way communications and extensive consultations. Focused and consistent messages in the government and World Bank dialogue can be critical to ensuring continued implementation progress even after project preparation and approval.Reforms detailed in Operational Policy Matrix provided guidance for implementation between the approval of the SEDPC and SEP. While prior actions listed for year two and year three of the DPC series were not ultimately used for disbursement (as the series was discontinued) the clarity of these messages guided the government reform team, and ultimately were the basis for the SEP DLI matrix year one targets.

18

Changing instrument from DPC to SIC can be an effective way to retain the same focus on results as provided in a DPC, while further providing enhanced focus on implementation outcomes and on strengthening associated provincial systems through technical capacity building and implementation support. The Bank team was effective in transitioning the support to SERP from policy lending instrument to investment lending while continuously supporting the government’s program.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/Implementing agencies See Annex 4 for the Borrower’s own detailed ICR. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable.

19

Annex 1. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Zahid Hasnain Sr Public Sector Spec. EASPR Tahseen Sayed Khan Operations Adviser SACBD Bertha Mburugu Program Assistant SASHD Hanid Mukhtar Senior Economist SASEP Reema Nayar Lead Economist SASED Alan Norley Consultant SASED Uzma Sadaf Senior Procurement Specialist SARPS Sofia Shakil Sr Education Spec. SASED

Supervision Ismaila B. Ceesay Lead Financial Management Spec SARFM Zahid Hasnain Sr Public Sector Spec. EASPR Tahseen Sayed Khan Operations Adviser SACBD Hanid Mukhtar Senior Economist SASEP Reema Nayar Lead Economist SASED Alan Norley Consultant SASED Saeeda Sabah Rashid Financial Management Specialist SARFM Uzma Sadaf Senior Procurement Specialist SARPS Sofia Shakil Sr Education Spec. SASED

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage No. of staff weeks USD Thousands (including

travel and consultant costs) Lending

FY07 83 300.15 FY08 0.69

Total: 83 300.84 Supervision/ICR

FY07 0.30 FY08 22 78.41

Total: 22 78.71

20

Annex 2. Beneficiary Survey Results

Not applicable.

21

Annex 3. Stakeholder Workshop Report and Results Not applicable.

22

Annex 4. Summary of Borrower’s ICR and/or Comments on Draft ICR

Sindh Education Reform Program (SERP)

SINDH EDUCATION SECTOR DEVELOPMENT POLICY CREDIT (SEDPC)

1. Program Objectives, Design and Implementation The Sindh Education Reform Program (SERP) – supported under World Bank’s Sindh Education Sector Development Policy Credit (SEDPC) - was based on a programmatic series of three development policy credits aimed at supporting the implementation of a Medium Term Education Sector Reform Strategy (SERP, 2006), approved by Govt. of Sindh. The historical assessment suggested that the key critical challenges to poor outcomes of education sector in Sindh were mainly due to : (i) Inadequate resource allocation; (ii) low capacity in education managers at all levels; (iii) multiple constraints to increasing access and quality; and (iv) lack of strategic policy direction in teacher education and learning assessment.

1.1. Key Program Objectives Against the identified challenges, Sindh Education Reform Program determined its overarching objectives aimed at improving quality, equity and efficiency in education service delivery including an effort to improve accountability in the education sector. However, the specific objectives of the Reform Program were set to :

- Improve participation, retention and transition; - Reduce gender and regional disparities; and - Improve quality in elementary and secondary education (grades 1-10).

In order to achieve the program objectives, it was expected that the following key development indicators would help achieve program objectives :

i) Increase net enrollment rates (NER) in primary, middle and secondary education from 48%, 18% and 12% in 2004/05 (baseline) to 59%, 22% and 14% respectively in 2009/10;

ii) Increase the share of girls in primary and in middle schools in rural areas from 36% and 28% to 40% and 31% respectively between 2005/06 (baseline) and 2009/10 ;

iii) Increase the transition rate for rural girls between primary and middle school from 47% to 50% between 2005/06 (baseline) and 2009/10; and

iv) Sustain and improve monitoring/evaluation system for data reliability and improve learning outcomes

1.2. Program Design

The program design was conceived in such a way that it comprehensively included all reform

activities and imitative including interdepartmental coordination within the four proposed pillars: • Improving fiscal sustainability and the effectiveness of public expenditures This pillar focused on fiscal and budgetary management in order to create fiscal space for increase in education and poverty reducing expenditures in Sindh. It also included financial management and

23

procurement reforms to increase credibility, transparency and accountability in the use of public resources. • Improving education sector managementThe set of key policy and other activities targeted education sector management by strengthening the functioning, capacity and accountability for improving the service delivery at both provincial and district level in line with district devolution system introduced in Sindh (SLGO, 2001). The key other activities included providing incentives to district governments through terms of partnerships to improve service deliver. The role of school management committees was also brought into focus for institutional development and monitoring of school as a workable unit.

• Improving access to quality schooling In view of massive challenge of lagging indicators for rural areas especially for girls, the program endeavored to provide focus to rural areas by means of reforms to improve the quality and provision/utilization of physical infrastructure in schools. The aim was to help remove implementation bottlenecks and improve the effectiveness of incentive programs such as free textbooks and girls’ stipends including by encouraging public private partnerships to improve access and quality of education. • Improving the quality of teaching and learning Quality of teaching and learning was expected to be improved by introducing and implementing a merit-based recruitment. A comprehensive teacher education and development and student reforms were also focused. In addition, assessment of student learning outcomes was also targeted to identify areas of future intervention in terms of teacher trainings and curriculum development.

1.3. Expected Benefits: Additional benefits expected out of the program activities were as follows: - Sustained commitment by the government in accelerating progress towards poverty reduction

objectives; - Revitalization of sustained engagement of the World Bank in support of the education sector

reform agenda; - Effective development partners’ coordination in education sector - Improvement in fiscal stability and effectiveness public resource management

- Cross cutting governance reforms particularly in education

1.4. Program Implementation and Operational Experience The program implementation has been a success as significant progress was achieved against target development objectives. All prior actions for each pillar for Year I as agreed to with the Bank under the Operational Policy Matrix of the program were completed in time making it possible for the Bank to approve and disburse the first tranche (SEDPC) for US $ 100.00 mil in support of the sector reforms. The SEDPC was intended to be the first of a programmatic series of three development policy credits in support of the government’s medium-term reform program. The reform program was designed around four core pillars to improve quality, equity and efficiency for effective education service delivery, and to increase accountability in the education sector by introducing up-front governance reforms through cutting across

24

interventions. Annual Targets were laid down relating to each pillar within Operation Policy Matrix which guided policy formulation and implementation within agreed time frame. This helped keep the program on track throughout the year in spite of challenges to the program. The program in Year 2 of SEDPC, got side tracked but it still continues to guide the implementation in the form renegotiated Sindh Education Project (SEP)/DLI Matrix. 1.4.1. Performance under Reform Program Pillars

Pillar I: Improving fiscal sustainability and the effectiveness of public expenditures Achievements:

- Medium Term Fiscal Framework (MTFF) for FY08-10; - Medium Term Sector policy Framework for education; - Financing Plan for Medium Term Sector Policy Framework; - Financial Management and Procurement Reforms accelerated; - Department Accounts Committee were activated; - Target set for Resolution of audit observations for three years met ( 35% of the 7,679

outstanding audits); - A revised procurement law was passed in April 2008;

These actions helped in improving management of fiscal policy resulting into fiscal space for enhanced spending in education sector. Operational Experience: MTFF was a new concept in Pakistan, particularly in Sindh. The Bank had the required expertise to build the capacity of the Govt. of Sindh, to undertake such big task. Towards this end, Finance Department played a key role in developing a required MTFF in consultation with stakeholders. During consultative process, the capacity constraint was identified as a major issue, which needs to be tackled in future through a series of specific, specialized trainings/workshops for stakeholders. Departmental Audit Committees were revitalized through an executive order by the Chief Secretary Sindh which was inactive at that time with thousands of audit observations/para pending for resolution. The initiative with intense RSU/ELD follow up, the target was well achieved. Previously monitoring and follow up was missing which played a key role in achieving the target. Close consultation was undertaken with Sindh Public Procurement Regulatory Authority (SPPRA) with the RSU Steering Committee support. The government was aware of the sensitivity of the issue in the wake of expiry of the life of the ordinance. The Bank’s technical assistance was well appreciated. Pillar 2: Improve education sector management Achievements:

- Reform Support Unit (RSU), as an integral part of the Education Department established, overseen by RSU Steering Committee headed by the Chief Secretary Sindh;

- Annual School Census improved and SEMIS data reliability enhanced; - Review of roles and responsibilities of education managers commenced (which later led

to approval of Education Management Reforms Policy in 2009 by Govt. of Sindh);

25

- Measures were undertaken to strengthen the roles of School Management Committees; Operational Experience: RSU continues to play a key role in steering the reform process. The Unit consisted of three wings – Policy, SEMIS and Monitoring & Evaluation. Availability of competent staff at RSU played a critical role in achievement of agreed actions under the program. However, in the formative phase of RSU, it faced resource constraints. RSU/SEMIS brought efficiencies in the Annual School Census process by improving communication amongst key stakeholders both at provincial and district level. In addition, capacity constraint was gradually removed by holding trainings/workshops for district officers for enhancing their technical capacity for conducting school census at district level. RSU/SEMIS team developed data base software. Majority of District Officers (SEMIS) posted on merit. Education Management Reforms was considered a massive challenge at this stage. However RSU/ELD played a key role in initiating a dialogue for determining policy direction. Discussions were held to review roles and responsibilities of the education managers in Sindh in the light of the devolution system introduced in Sindh province. Later, in 2009 (SEP Y1 DLI) an Education Management Reforms Policy was approved by Govt. of Sindh for undertaking piloting of management reforms in three selected districts – Dadu, Naushehro feroze and Mirpurkhas. School Management Committees faced a major challenge in terms of its revitalization mainly due to government ban on release of funds. Pillar 3: Improving access to quality schooling Achievements:

- Terms of Partnership (TOP) Agreements with all 23 district governments providing conditional grants ( Rs. 100 million per district per annum) for provision and improvement of school infrastructure;

- Over four million students enrolled in public school provided free text books from Class I to X;

- Approx. three hundred thousand girls students (Class VI-X) provided stipends Operational Experience: Terms of Partnership (TOP) agreement with district was used as an instrument for effective partnership for implementation of Sindh Education Reform Program activities at district level. For this purpose districts were offered conditional grants for undertaking School Rehabilitation works. Towards this end, a consultative dialogue with districts was conducted to agree on terms of the agreement. Text book distribution mechanism was improved and monitoring strengthened to ensure timely delivery of books before the start of the academic year. Improvement in delivery cycle is consistently being strived for under SEP. Girls’ stipends delivery was revamped by ensuring delivery through customized money orders in partnership with Pakistan Postal Offices. Major increase has consistently been recorded in the number of beneficiaries over the years. Effective monitoring was also conducted.

26

Pillar 4: Improving the quality of teaching and learning

Achievements: - Merit-based Teacher Recruitment Policy approved; - Over five thousand teachers recruited on merit during round one verified by TPV; - Provincial Education Assessment Center was strengthened;

Operational Experience:Formulation of merit-based recruitment policy was a major challenge which demanded consensus of criteria for merit determination for school specific appointments. Extensive consultation was undertaken with political and bureaucratic leadership. Testing was conducted by third party and no weight was given to interview. An intense third party validation exercised confirmed merit-based recruitment. Under SEP, second round of the recruitment is well underway. Provincial Education Assessment Center (PEACE) working under Bureau of Curriculum (BOC), faced shortage of staff. New staff positions were created and capacity was enhanced through trainings. Under SEP, the PEACE Sindh became the first institution in the country to have developed and conducted its own diagnostic test (Class IV Math) for learning assessment of students at district level. The test was administered to 40, 000 students across all districts of Sindh.

2. Assessment of Outcomes

2.1. Improving participation, retention and transition PDO Indicators: (1) Net primary, middle and high school enrollment rates; (2)Share of girls in primary and middle schools in rural areas; (3) Transition rate for girls in rural areas from primary to secondary schools; and (4) Monitoring of learning assessments Overall adequate progress was made under the key indicators as listed above. Positive growth was recorded in most of the indicators. The progress was mainly supported by improving access (TOP/School Rehabilitation), provision of new teachers (Merit-based Recruitment Round One), delivery of girls stipends and distribution of free text books. The incentive programs helped encourage new enrolment, reduce the drop outs and sustain transition. The recruitment of meritorious teachers and their school specific posting would lead to improvement in learning outcomes of students. In addition, the conduct of diagnostic assessments would lead to improvement in curriculum and teacher trainings in future. Resultantly net enrollment rate at primary level recorded positive growth from 48% in 2004/05 to 51% in 2007/08. Similarly net middle school enrollment rate recorded increase by 1% (from 17% to 18%) and net enrollment rates in high school remained un-changed (11%). In addition, the transition rates (rural Sindh) also recorded improvement from primary to secondary school, from 54% to 56% from 2005/06 to 2008/09.

2.2. Reducing gender and regional disparities PDO Indicators: (1) Share of girls in primary schools in rural areas; (2) Share of girls in middle schools in rural areas; (3) Transition rate for rural girls from class V to class VI. Government of Sindh achieved satisfactory results. Owing to increased allocation of budgetary resources as well as enhanced capacity implementation of initiatives met with success. .

27

2.3. Improving quality in elementary and secondary education PDO Indicators: (1) Improvements in availability of timely and valid data; (2) Introduction of provincial assessment system; (3) Number of teachers hired on merit; (4) Regular monitoring of completion rates. Process improvements contributed in Annual School Census contributed in achieving this objective. As a result of revamping, effective monitoring and evaluation was introduced this helped improve data accuracy, reliability and time lag. Capacity of PEACE was enhanced by creation of new staff positions which contributed immensely in timely administration of assessment testing. In addition to it, the approval of the merit-based teacher recruitment policy by Government of Sindh, and consequent teacher recruitment (round one) improved quality of teaching at schools.

3. Performance of Government of Sindh The Government of Sindh showed strong political commitment, sustained support and ownership of the reform program. The Steering Committee headed by Chief Secretary Sindh continued to provide oversight of the program implementation regularly. Education Department however played a key role in coordinating with other departments for steering forward the reform activities in order to meet agreed prior actions. Reform Support Unit (RSU), the program management unit, and provided full back up support in coordinating and facilitating all stakeholders both at provincial and district level. The RSU also played a key role in development partner coordination under the program. The program implementation helped identify critical lessons which have been integrated under renegotiated Sindh Education Project (SEP) design. The critical lessons learnt which were key to successful program implementation are: i. No reform program can be a success without strong political commitment and

ownership; ii. A well coordinated program laying out specific program interventions is more

likely to yield sustainable results; iii. Interdepartmental coordination through a high powered forum plays

instrumental role in guiding and direction the program; iv. Close, regular dialogue with all stakeholders is necessary to avoid delays in

implementation; v. Strong analytical work is essential for negotiations and policy formulation. vi. An effective monitoring and evaluation (M&E) system utilizing services of both

the third parties and project management unit ensure quality outputs; vii. Effective communication with all stakeholders can enhance the efficiency of

outputs; and viii. Understanding of ground realities is more important in policy making and

program implementation.

4. Performance of the World Bank The World Bank Mission’s performance remained satisfactory. The Bank provided continuous support and commitment to the Government of Sindh, in implementing the reform program.

28

Owing to effective monitoring and technical support at each stage of the program – from design to implementation – helped Sindh Government in achieving all prior action agreed under Operational Policy Matrix (Year 1). Govt. of Sindh acknowledged the positive, critical role played by Bank’s Mission throughout the period of implementation. The critical lessons learnt in interacting with the Bank are described as follows:

i. Strong communication and periodic consultation between partners is the most critical link for successful achievement of prior actions;

ii. Strong analytical home work is essential for negotiations with the Bank; iii. Bank can be relied upon for providing technical support on program

design and policy formulation; iv. Bank has the expertise to help evolve an effective monitoring and

evaluation (M&E) framework; v. Ground realities needs to be consistently brought into focus while

interacting with the Bank.

5. Institutional Arrangement for Sustainability of Reform Program Government of Sindh successfully renegotiated the financing of the ongoing reform program after discontinuation of the SEDPC, under Sector Wide Approach (SWAp) financing (US $ 300.00 million including US $ 6.00 million as TA, for the period 2008/09-2010-11) for Sindh Education Project (SEP) which resulted in approval of the financing by the Bank Board in June, 2009. The SEP is a specific investment credit providing for disbursements against achievement of specific disbursement linked Indicators. Under the SEP, Government has already achieved the release of the first tranche for US $ 108.8 mil which included technical assistance for US $ 1.8 million. The SEP incorporated the OPM actions from Year 2 onwards with enhanced focus on implementation effectiveness. It also maintains strong commitment to the policy areas such as Education Management Reforms, Teacher Education Development, SMC strengthening and mobilization, Merit-based Teacher Recruitment (Round Two), Differential Stipends Program along with regular girls stipends program, free text books distribution, terms of partnerships with districts for provision of missing facilities at schools, learning assessments, and so on. The Govt. of Sindh maintains the original oversight of the SEP through RSU Steering Committee headed by the Chief Secretary Sindh. Reform Support Unit, as an integral part of the Education Department, continues to provide project facilitation/implementation support with professional, experienced and competent staff. Availability of WB TA ensures that capacity and other constraints can also be removed in timely manner. Finally, the reform program continues to enjoy the required sustained political and bureaucratic commitment and ownership which is the pre requisite for successful implementation. Both the Bank and the Govt. of Sindh regularly consult each other and maintain strong, effective communication links.

29

Annex 5. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable.

30

Annex 6. List of Supporting Documents Government of Pakistan, Federal Bureau of Statistics (2008). Pakistan Social and Living Standards Measurement Survey (PSLM) 2006/07. Government of Pakistan, Federal Bureau of Statistics (2009). Pakistan Social and Living Standards Measurement Survey (PSLM) 2007/08. Government of Pakistan, Finance Division (2009). Pakistan Poverty Reduction Strategy Paper (PRSP) –II. Government of Pakistan, Ministry of Finance (2003). Poverty Reduction Strategy Paper (PRSP), “Accelerating Growth and Reducing Poverty, The Road Ahead”. Government of Sindh (2009). Provincial Education Assessment Center (PEACE) PC-1. Government of Sindh, Reform Support Unit (2009). Policy Document on Training and Development.

Government of Sindh, Sindh Education Foundation (2008). Promotion of private schools in rural areas of Sindh/urban slums (PPRS). Government of Sindh, Sindh Evaluation and Monitoring Implementation System. Annual Public School Census databases and dissemination reports from 2005/06-2008/09. Government of Sindh, Finance Department. Sindh Medium Term Fiscal Framework 2009-10 to 2011-12. World Bank (2006). Pakistan Country Assistance Strategy (CAS) for FY06-09, April 4, 2006 (Report No. 35718-PAK)

World Bank (2007). Program Document: First Sindh Education Sector Development Policy Credit

World Bank (2009). Project Appraisal Document. Sindh Education Sector Project. 47642-PK

World Bank. Aide-Memories, Back-to-office reports, Management Letters, Project Status Reports/Implementation Status and Results reports of all implementation review missions and project preparation missions from May 2007-July 2009.

1

Annex 7. Operation Policy Matr ix and DLI Matr ix Compar ison

The following table demonstrates how the SERPreforms, results and institutional changes, supported by theSEP, build on and deepen the supportprovided under SEDPC. It also shows how performance benchmarks originally anticipated under year two of the DPC were achieved4, and insome casessurpassed by the achievements embodied in March 2009 DLIs in theSEP.

Issues (at launch of SERP in2006/07)

SEDPC1 (June2007) Exper ienceand lessons SEP - Year 1 results (June2009) andrelationship with SEDPC-I I

Desired Year 3 (June2011) results

1: IMPROVING FISCAL SUSTAINABILITY AND EFFECTIVENESS OF PUBLIC EXPENDITURES

• Weak budgetary processeslacking strategic results focus

• Provincial finances and thereforeadequateand predictablefinancing of education and pro-poor sectors vulnerable toshortfalls in federal revenuestransfers, and lack of discipline indevelopment spending.

• First MTFF (FY08-FY10)introducing medium-term resultsfocus, launching SERPandmedium-term financing plan, andestablishing fiscal targets approved.FY06/07 budget aligned withSERP.

• MTFF not updated in FY08 forFY09-FY11 due to tight time topreparebetween appointment ofnew government and budget.

• FY07 and FY08 budgetsbroadly inlinewith MTFF targets, forprovincial own source revenue(av.annual growth of 13.9%), PRSPexpenditures (av. annual growth of16%) and consolidated fiscaldeficit below 0.6% of GDP) andover 75% of ADP allocated toongoing schemes.

• FY08 and FY09 budgets werealigned with SERP

• FY09 budget prepared by newgovernment reflects increase inprovincial own tax revenueandeducation expendituresbutallocated less than 50% of ADP tonew schemes.

• MTFF process resumed and FY10-12MTFF prepared based on new Chart ofAccounts; integrating updated SERPandfinancing plan; and establishing fiscaltargets for provincial own tax revenuegrowth (10% per year), fiscal deficit(below 1.25%), and shareof allocation ofdevelopment budget to existing schemes(70-75%)

REFLECTS ACHIEVEMENT OF RESULTSAS

INTENDED FOR SEDPC-II

• Gradual institutionalization ofresults based budgeting throughannual preparation andpublication of MTBFs (foreducation and additional sectors)in budget documentsandpreparation of annual budgetsconsistent with MTFF;

• Annual financing of SERPasperupdated resultsbased strategy andfinancing plan

• Achievement of fiscal targets inMTFF/MTBF.

• Largenumber of advanceauditparagraphs in theeducationsector reflect poor internal auditcontrols and financialaccountability

• Issuanceof directive to reduce‘advance’ audit paragraphs in theeducation sector

• Significant reduction of auditparagraphs in the education sector,but large number of auditparagraphs continue to begenerated each year.

• Reduction of over 40% of advance auditparagraphs compared to 2005/06 baseline

SURPASSES SEDPC-II AND SEDPC-III RESULTS

TARGETS.

• Annual reduction of 60% ofresidual audit paragraphs(following last in-first outapproach) through improvedinternal controls andpreventative/systemic actions

• Deficiencies in provincial (andeducation sector) procurementenvironment; no institutionalhome for procurement nor an

• Procurement reform roadmapprepared covering regulatory,institutional and capacity aspects.

• SPPRA established (by ordinance)

• Government continued withimplementation of procurementroad map; SPPRA resourced andcapacity strengthened. Draft

• New procurement law approved byparliament.

• SPPRA approved revised rules;• GoS updated CSR and issued notification

• Procurement law enacted anddefines procurement regulatoryframework for province

• Revised rules, SBDs and

4 SEDPC-II targets wereoriginally intended to beachieved by June2008; however thepolitical changes and security considerationshavecontributed to a longer implementation

period. See the final paragraph of Section 2.2 for further details.

2

Issues (at launch of SERP in2006/07)

SEDPC1 (June2007) Exper ienceand lessons SEP - Year 1 results (June2009) andrelationship with SEDPC-I I

Desired Year 3 (June2011) results

updated strategy for reform;CompositeScheduleof Ratesreflects historical (2004) costs.

and Federal PPRA rules adoptedunder interim arrangementsandpre-registration of bidders forworks procurement abolished

procurement legislation and SBDfor goods works and services(consistent with international bestpractice) prepared.

for periodic updateof CSR to market rates.

REFLECTS ACHIEVEMENT OF RESULTSAS

INTENDED FOR SEDPC-II

implementing regulations in placeand in use

• Province-wideprocurementmonitoring system functional

• Semi-annual/quarterly updates ofCSR implemented

• Inefficient procurement practicesand inadequatesupervision ofeducation civil worksprocurement by districts

• Provisions made for third partysupervision and ex-post review ofschools

• Contracting for third party tosuperviseconstructionrehabilitation works and to carryout ex post procurement review hastaken timeand itsneed hasbeenconfirmed by theexperienceofrehabilitation works under TOP1.Concerns include inadequateselection of rehabilitation works onthebasis of needs; inadequateattention to risks from naturaldisasters or water quality, to schooldesign and construction as well asinefficient procurement practices.

• Third Party contracted and will carry outconstruction supervision and ex-postprocurement reviews. It will also verifypriority lists from districts, carry outinspection of schools in high risk areas,prepareschool siting guidelines, preparemodel designs for cost effective,appealing, structurally safeand energyefficient schools, provideconstructionsupervision and quality control, monitorcompliancewith guidelines for schoolsiting and design and water qualitymonitoring, certify compliancewith ESMF

SURPASSES SEDPC-II TARGETSDUE TO

ENHANCED FOCUSON SAFEGUARD AND

SCHOOL DESIGN ISSUES

• Improved procurement practicesby districts, improved schoolsiting and water quality in highrisk areas, improved safety andquality of design andconstruction.

I I : IMPROVING EDUCATION SECTOR MANAGEMENT

• Weak provincial capacity formonitoring and evaluation ofoutputs, outcomes and programs

• Lack of timely, comprehensive,reliable information; weak

• Policy formulation not evidence-based

• Reform Support Unit established inED with responsibility formonitoring and evaluation,evidence-based policy formulation,coordination of reform efforts andpartner assistance, andimplementation of provincialincentiveprograms

• Annual Censusof Schoolsimplemented with improvements incoverageof information andimproved quality controlprocedures

• Insufficient capacity of ReformSupport Unit hasbeen aconstraintin monitoring and design ofprograms and adequateimplementation and monitoring ofprovincial incentiveprograms.

• Issues remain with regard tocoverageof schools, reliability ofdata, speed and timeliness of dataavailability, useof data for mid-coursecorrectionsand decisionmaking

• RSU capacity strengthened to adequatelycarry out its role in monitoring,implementation and policy formulation (aswell as main implementing agency forSEP.)

• ACS 2008/09 implemented with useofimproved standardized processes for datacollection, entry and verification, andresults availablewithin 3-4 months.

NEW RESULTSTARGETSDEFINED BASED ON

IMPLEMENTATION LESSONS(NOT IN SEDPC-II)

• Improved provincial capacity formonitoring and evaluation ofoutputs, outcomes and programs.

• Improved timeliness, reliability,adequacy, dissemination and useof data in theACS for educationsector monitoring and policyformulation through annualimprovements.

• Weak district educationmanagement capacity for schoolinspection and advice/support

• Lack of clarity in roles andresponsibilities;

• Principles for entry and careerpath in education managementnot articulated

• Managers lack educationmanagement and leadership

• New district education managementreform (EMR) policy prepared andsubmitted for approval sets out principlesguiding entry and progressivedevelopmentalong a career path and envisionsgradualintroduction of new educationmanagement cadre

[REFLECTS ACHIEVEMENT OF RESULTS AS

INTENDED FOR SEDPC II]

• Implementation of new educationmanagement cadre in a fewdistricts with clearly defined jobdescriptionsand responsibilitiesfor each post, training and revisedprocedures for performanceappraisal

• Inspection and advice/supportsystem designed andimplemented in a few districts

3

Issues (at launch of SERP in2006/07)

SEDPC1 (June2007) Exper ienceand lessons SEP - Year 1 results (June2009) andrelationship with SEDPC-I I

Desired Year 3 (June2011) results

skills; no system for training,induction or professionaldevelopment.

• Politically motivated postings,transfers and promotionsbasedon seniority

• Poor quality of schoolinfrastructure in all districts

• Irregular or limited funding forschool rehabilitation

• No incentives for needsbasedprovision of good quality schoolinfrastructureby districts;political interference in selectionof rehabilitation works

• Fundsprovided to all districtgovernments for schoolrehabilitation works conditional onentering performanceagreements(TOP1).

• Rehabilitation grantsused for theconstruction of shelter, provision ofmissing facilities, and additionalclassrooms

• Issues remain with selection ofpriority works, and withconstruction quality and timelycompletion (seealso I above)

• TOP II agreements signed and 30% offunds transferred to districtsbased onperformanceagainst TOP I indicators.

• TOP II indicators includecompletion ofworks asper vetted priority lists, indicatorsof school improvement and contractsrequirecompliancewith safeguard andprocurement considerations

REFLECTS ACHIEVEMENT OF RESULTSAS

INTENDED FOR SEDPC II

• Annual transfer of funds todistricts for school rehabilitationworks with higher sharebased onperformancecontributing toincreased availability and useofminimum school infrastructureand improved school environment

• Weak management at schoollevel

• Limited community participation• No formal SMCs in middleand

secondary schools• Irregular funding to SMCsand

uneven implementation ofcapacity building programs forSMCs

• New guidelines provided toestablish SMCs in middleandsecondary schools

• Lack of sustained effort to ensureeffective involvement of SMCs inschool management

• SMC databaseprepared/completedincluding bank account information

• Issuanceof updated guidelines for SMCswith enhanced discussion of fiduciaryrequirements

• Awareness raising workshopscompletedto disseminateguidelines

• Development of apilot program (andevaluation design) for SMC communitymobilization and capacity building inschool improvement planning (SIP)

[REFLECTS ACHIEVEMENT OF RESULTS AS

INTENDED FOR SEDPC-II]

• Implementation of pilot SMCcommunity mobilization and SIPcapacity building program asperagreed program and impactevaluation parameters

I I I : IMPROVING ACCESS TO QUALITY SCHOOLING

• Incentives programs for publicschool children (girls’ stipendsand textbooks) provision hadlimited impact due to seriousproblems with delivery

• Girls’ stipends were either notreceived at all, received withserious delays, or in significantlyreduced amounts

• Textbooks reached with delaysand shortfalls relative to needs

• Weak monitoring and insufficientcontrols for both programs

• Delivery mechanisms for bothprograms streamlined: role ofdistrict management in deliveryeliminated.

• RSU/ED works with STBB fordistribution directly to sub-districtlevels from where they are collectedby headteachers.

• RSU/ED works directly with GPOfor stipends delivery to girls

• For the first time, about 90% oftextbooks were delivered within thefirst month of the school year

• Stipends delivered in followingacademic year and timing ofdelivery within year variablereducing desired incentive impact

• Delivery performance difficult tosustain due to delays andincompleteness in vitaladministrative data from districts,continued deficiencies inmonitoring

• Controls to ensure full delivery tofinal beneficiaries need to befurther strengthened

• Textbooks delivered to at least 90% ofpublic primary, middle, and high schoolswithin the first month of the 2008-09academic year

• Stipends for two academic years deliveredin 2008/09 to eliminate lag and 2008-09academic year cash stipends successfullydelivered to at least 90% of intendedbeneficiaries in all districts before theclose of the 2008-09 academic year.

• Differential stipends program providinghigher stipends amount to districts withlow girls’ transition rates designed and

• Sustained/improved deliveryperformance of both programs

• Improved monitoring systems andcontrols for both programs enableadministrative verification withinweeks of delivery.

• Administrative monitoringcomplemented by third partyverification

• Roll out of differential stipendsprogram as per agreed programand impact evaluation parameters

• Launch of reforms to introduce

4

Issues (at launch of SERP in2006/07)

SEDPC1 (June2007) Exper ienceand lessons SEP - Year 1 results (June2009) andrelationship with SEDPC-I I

Desired Year 3 (June2011) results

• Need for stronger incentives forrural girls

• Poor (physical and curricular)quality of textbooksduetolimited competition in textbookdevelopment and printingpublishing

• For the first time, about 80% ofbeneficiaries receive the fullamount of thestipend.

introduced.• Improvements in administrative

monitoring; TOP II includes indicators fordistrict compliancewith monitoringrequirements of theprograms

REFLECTS ACHIEVEMENT OF RESULTSAS

INTENDED FOR SEDPC-II WITH SUSTAINED

EMPHASISON DELIVERY AND MONITORING

PERFORMANCE; INTRODUCTION OF DSP AHEAD

OF SCHEDULE SURPASSES SEDPC-II TARGETS

competitivepractices in textbookdevelopment andprinting/publishing.

• Limited mobilization of privatesector in rural Sindh

• SEF has responsibility for PPPsbut has traditionally engaged insmall scalepartnerships;effectiveness in implementinglarger scalePPPs requiresinstitutional autonomy

• Innovativemedium scalePPPprogram designed which would befacilitated by acceleration of SEFautonomy reforms.

• Launch of anew PPP program thatprovides aper student subsidy to privateentrepreneurs for each child they enroll inunderserved rural areas

• SEF autonomy principles approved by theChief Minister

[SURPASSES SEDPC-II RESULTSTARGETSDUE

TO APPROVAL OF AUTONOMY PRINCIPLES]

• Roll out of PPP program in up to1000 schools asper agreedprogram and impact evaluationparameters

• SEF fully functional asanautonomous institutionresponsible for PPPs in theprovince

IV: IMPROVING THE QUALITY OF TEACHING AND LEARNING

• Patronagebased recruitment hasbeen serious issue, compromisingquality and presenceof teachersand contributing to politicallymotivated postings/transfers

• Posting of teachersnotadequately based on needs; largedifferences in student teacherratiosbetween schoolsexacerbated by politicalinterference in transfer/postings

• Concernswith quality andintegrity institutionscertifyingpotential teachers limit usefulnessof relying on these.

• A new policy for recruitment ofteachersusing transparent merit-based criteria (including a testadministered by an independentinstitution) and their placementunder school-specific contractsapproved.

• Pressures for deviating from thispolicy elevated especially in therun-up to theelections. Severaldistricts initiated recruitment usinghigh weight for interviews incontradiction to thespecificationsof thepolicy.

• New government did not supportthesedecisions, recruitment incontravention of policy stoodcancelled and revised guidelinesissued eliminating weight tointerviews

• Inadequateuseof evidence indetermination of vacancies meantthat schools with high studentteacher ratio did not uniformlyreceivehigher priority

• Bank duediligence reveals that first roundof recruitment under thenew policy hasbeen carried out without significantdeviations; this isbeing validated by athird party

• Second round of recruitment initiated withimproved dissemination of recruitmentcriteria, introduction of pedagogicalcompetencies in the test, and assessment ofneeds based on ASC to givepriority inplacement to schools with higher studentteacher ratios

• TOP II agreements include indicators fornumber of schools achieving target studentteacher ratios

REFLECTS ACHIEVEMENT OF SEDPCII RESULTS

TARGETS

• Gradual institutionalization ofmerit-based recruitment throughmultiple rounds of successfulimplementation of policy

• Gradual introduction of newteacher standardsandcompetencies into teacherrecruitment tests

• Posting of teachers on schoolbased contracts in schools withgreater needs reduces teachershortages

• Together thesecontribute toimproved quality of newlyrecruited teachers and improvedteacher and student attendance

5

Issues (at launch of SERP in2006/07)

SEDPC1 (June2007) Exper ienceand lessons SEP - Year 1 results (June2009) andrelationship with SEDPC-I I

Desired Year 3 (June2011) results

• Limited impact of teachereducation and development(TED) programs

• Lack of standardsandcompetencies for theappointment, training,professional development ofteachers

• No coherent policy for TED• Unsystematic and unregulated

provision of CPD not based onsystematic needs assessment andpredominantly used tocompensate for poor quality ITE

• Disparateand overlappingprovision (by BoC and PITE) andinconsistent quality of ITE

• Lack of professional oversight,policy feedback, qualityassuranceand effectivecoordination of TED

• Lack of support and opportunityfor teachers’ professionaldevelopment

• Comprehensive TED policy andimplementation road map developed andapproved which envisions coherent systemfor ITE and CPD.

REFLECTS ACHIEVEMENT OF SEDPC-II RESULTS

TARGETS

• New standards and competenciesadopted and in use in pilot CPDprogram

• Institutional focal pointestablished for strategic oversight,accreditation and qualityassurance

• Roles of BoC and PITE redefinedwith PITE responsible foroperational management of allTED institutions

• Accreditation and qualityassurance evaluation of agreednumber of CPD providers.

• New teaching diploma introducedbased on new standards andcompetencies for certification offuture teachers

• Poor student learning outcomes• NEAS provides information on

learning outcomes at provinciallevel but no reliable informationon learning outcomes at moredisaggregated level (e.g. districts)

• Teachers lack requisite trainingand tools for classroomassessment and adaption ofteaching practices

• Information on learningoutcomes not adequatelydisseminated to stakeholders andnot used to improve teaching andlearning

• First provincial learning assessmentimplemented with district-representativesample for grade 4 mathematics

• New assessment is diagnostic – designedto provide information on specificdifficulties within subject areas usingNEAS findings on curricular areas of poorperformance in Sindh.

[SURPASSES SEDPC-II RESULTSTARGETS,ACHIEVING SEDPC-III RESULTSTARGETS]

• Implementation, analysis anddissemination of findings ofprovincial learning assessment inthree subjects in grade 4 and onein grade 8

• Improved summative assessmentsin grade 4 mathematics andlanguage in use in a sample ofschools

GilgitChitral

Mardan

PeshawarKahat

Rawalpindi

Bannu

Mianwali Khushab

Jhelum

Sargodha

GujratSialkot

Gujranwala

Bhakkar

JhanaLahore

KasurFaisalabad

Sahiwal

MultanD.G.Khan

Bahawalpur

Bahawalnagar

Rahimyar Khan

RohriSukkur

Larkana

Dokri

Nawabshah

Hyderabad

Thatta BadinKarachi

Bela

Gwadar

Panjgur

Kalat

Khuzdar

Quetta

Sibi

D.I.Khan

ISLAMABAD JAMMUand KASHMIR

N.W.F.P.

B A L O C H I S T A N

PUNJAB

SINDH

Jhel

um

R.

R.

Ravi

Sutlej

R.

Indus

R.

Nar

aC

anal

Nal

R.

Mouths ofthe Indus

Zhob R.

A R A B I A N S E A

IndusR.

Kuna

r

R.

ApproximateLine of Control

TAJ IK ISTAN

C H I N A

AFGHANISTAN

I S L A M I C R E P .

O F I R A N

I N D I A

70° 75°

35°

30°

75°

25°

70°65°60°

25°

30°

35°

60° 65°

30°

35°

75°

35°

30°

25°

75°70°65°

25°

70°65°

N.W.F.P.

BALOCHISTAN

SINDH

PUNJAB

••

•••

••

•••

•

•

••••

••

• •••• •• ••• • • • • • • ••

Approx. Line of ControlJAMMU

AND KASHMIR

Arabian Sea

TAJIKISTANCHINA

I N D I A

A F G H A N I S T A N

ISLAMIC REP . OF

IRAN

TURKMENISTAN

UZBEK.

PAKISTANPAKISTAN

TAJIK.

PAKISTAN

Islamabad

0 100 200 300 KILOMETERS

0 50 100 150 200 MILES

DEC

EMBER 2009

IBRD 35451R

PAKISTANSINDH EDUCATION SECTOR

DEVELOPMENT POLICY CREDIT

PROJECT PROVINCE

SELECTED CITIES

NATIONAL CAPITAL

MAIN ROADS

RAILROADS

RIVERS

PROVINCE BOUNDARIES

INTERNATIONAL BOUNDARIES

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shownon this map do not imply, on the part of The World Bank Group, anyjudgment on the legal status of any territory, or any endorsement oracceptance of such boundaries.