PROGRESS EVALUATION OF THE UNICEF … · Liberia Case Study April 2011 ... EMERGENCIES AND POST-CRISIS TRANSITION PROGRAMME (EEPCT): Liberia Case Study ... 3.8 Operational issues

PROGRESS EVALUATION OF

THE UNICEF EDUCATION IN

EMERGENCIES AND

POST-CRISIS TRANSITION

PROGRAMME (EEPCT):

Liberia Case Study

April 2011

Cover page

EVALUATION REPORT

EVALUATION OFFICE

March 2011

PROGRESS EVALUATION OF

THE UNICEF EDUCATION IN

EMERGENCIES AND

POST-CRISIS TRANSITION

PROGRAMME (EEPCT):

Liberia Case Study

EVALUATION REPORT

EVALUATION OFFICE

March 2011

PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 1


Progress Evaluation of UNICEF’s Education in Emergencies and Post-Crisis Transition Programme: Liberia Case Study © United Nations Children‘s Fund, New York, 2010 United Nations Children‘s Fund Three United Nations Plaza New York, New York 10017 March 2011 The purpose of the evaluation reports produced by the UNICEF Evaluation Office is to assess the situation, facilitate the exchange of knowledge and perspectives among UNICEF staff and to propose measures to address the concerns raised. The content of the report do not necessarily reflect the policies or views of UNICEF. The text has not been edited to official publication standards and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on legal status of any country or territory, or of its authorities, or the delimitation of frontiers. All photographs in the evaluation report are the copyright of UNICEF © UNICEF/2010/Columbia Group for Children in Adversity For further information, please contact: Evaluation Office United Nations Children‘s Fund Three United Nations Plaza New York, New York 10017 [email protected]


PREFACE

The purpose of this evaluation was to identify and assess progress of the Education in Emergencies and Post-

Crisis Transition (EEPCT) Programme and to enable systematic reflection towards improving programme

results. The EEPCT Programme was examined at global, regional and country levels through quantitative and

qualitative methods that combined comprehensive coverage with in-depth analysis.

The EEPCT Programme began in 2006 as a four year (later extended to five), US $201 million dollar

partnership between UNICEF and the Government of the Netherlands. The EEPCT Programme aims to ―put

education in emergency and post-crisis transition countries on a viable path of sustainable progress toward

quality basic education for all.‖ EEPCT funds support UNICEF education programming in 39 countries and

territories and are also used to advance the global agenda for education in crisis-affected contexts.

The Evaluation Office commissioned this independent progress evaluation in June 2010. The global

evaluation was overseen by a Reference Group led by internal and external technical experts in evaluation,

education, and emergencies. The evaluation was conducted by Columbia Group for Children in Adversity,

associated with Columbia University. The independent team of consultants was led by Neil Boothby and

Peter Buckland. The evaluation was managed by Silvia De Giuli (Evaluation Specialist), and Ashley Wax

(Evaluation Specialist).

The evaluation methodology included: extensive document review; six in-depth country case studies in

Angola, Colombia, Côte d‘Ivoire, Liberia, the Philippines, and Sri Lanka; extensive interviews; on-line surveys

for key programme staff; comprehensive focus group discussions; review and analysis of data collected

against the programme Logical Framework.

This report presents in-depth analyses and results of the progress of the EEPCT programme in Liberia. Carl

Triplehorn and Evelyn Kandaka lead the case study with support from research associate, Natasha Rothchild,

and national researchers, Willie Benson, Mabel Flumo and Francis T. Wayne.

Special thanks to UNICEF staff across the organization and to the Reference Group who both participated

actively and provided substantive comments on emerging issues and interim reports. We would like to

acknowledge, in particular, the support of Susan Durston, Chief of Education, and Jordan Naidoo Senior

Advisor, Education Section. Likewise, we appreciate the efforts and support by the country office, including

Stella Kaabwe. Genuine thanks to the Government of the Netherlands, European Commission and other

partners who have supported education in emergencies and post-crisis transition, as well as evidence-based

decision-making.

Samuel Bickel Officer-in-Charge Evaluation Office UNICEF New York Headquarters


TABLE OF CONTENTS

PREFACE .............................................................................................................................................. 2

ACRONYMS .......................................................................................................................................... 5

EXECUTIVE SUMMARY ....................................................................................................................... 7

The Way Forward: Recommendations ................................................................................................ 11

1.0 INTRODUCTION ........................................................................................................................... 14

1.1 Overview ....................................................................................................................................... 14

1.2 Country Context ............................................................................................................................ 15

1.3 Educational Context ...................................................................................................................... 15

1.4 EEPCT Programme ...................................................................................................................... 17

2. METHODOLOGY .......................................................................................................................... 19

2.1 Purpose and methods ................................................................................................................... 19

2.2 Field-site and sample selection ..................................................................................................... 20

2.3 Limitations ..................................................................................................................................... 20

3. EVALUATION FINDINGS ............................................................................................................. 21

3.1 Overview: Results Analysis ........................................................................................................... 21

3.2 Goal One: Improved quality of education response in emergencies and post-crisis transition

countries .............................................................................................................................................. 21

3.3 Goal Two: Increased resilience of education sector services delivery in chronic crises, arrested

development, and deteriorating contexts ............................................................................................ 24

3.4 Goal Three: Increased education sector contributions to better Prediction, Prevention and

Preparedness for emergencies due to natural disaster and conflict ................................................... 27

3.5 Goal Four: Evidence-based policies, efficient operational strategies and fit-for-purpose financing

instruments for education in emergencies and post-crises. ................................................................ 27

3.6 DAC Criteria .................................................................................................................................. 30

3.6 Focus Groups Discussion Outcomes ............................................................................................ 33

3.7 Cross-cutting Issues ...................................................................................................................... 35

3.8 Operational issues and management issues ................................................................................ 37

3.9 Partnerships .................................................................................................................................. 38

4.0 THE WAY FORWARD ................................................................................................................... 38

4.1 Lessons learned and conclusions ................................................................................................. 38

4.2 Recommendations ......................................................................................................................... 41

REFERENCES .................................................................................................................................... 44

ANNEXES............................................................................................................................................ 48

Annex 1: Methodology ......................................................................................................................... 48

Annex 2: Daily Schedule ..................................................................................................................... 54

Annex 3. Interviews Conducted: National, Sub National and Community Levels ............................... 60


Annex 4: Liberia EEPCT Evaluation Framework................................................................................ 61

Annex 5: Evaluation Tools ................................................................................................................... 65

EE-DAC Score Card .................................................................................................................... 65

Adequacy Survey Checklist ......................................................................................................... 84

Programme Design ...................................................................................................................... 84

Programme Implementation ........................................................................................................ 84

Programme Learning ................................................................................................................... 84

Financial ...................................................................................................................................... 85

Best Practices/Standards ............................................................................................................ 85


ACRONYMS

ALP Accelerated Learning Programme

BEGE Basic Education and Gender Equality

CEO County Education Officer

CFS Child Friendly School

CGCA Columbia Group for Children in Adversity

CPC Country Programme for Children

CRC Convention on the Rights of the Child

CO UNICEF Country Offices

DEO District Education Officer

ECD Early Childhood Development

ECD/DAC Organization for Economic Co-Operation and Development‘s Development Assistance

Criteria

EC/EU European Community /European Union

EPF Education Pooled Fund

EEPCT Education in Emergencies and Post-Crisis Transition programme

EFA Education for All

EMIS Education Management Systems

EPDF Education Programme Development Fund

ESP Education Sector Plan

FAWE Forum for African Women Educationalists

FGD Focus Group Discussion

FTI Fast Track Initiative

GER Gross Enrolment Ratio

IASC Inter-Agency Standing Committee

IDP Internally Displaced Person

INEE Inter-Agency Network for Education in Emergencies

IRC The International Rescue Committee

LAB4LAB Learning Along Borders for Living Across Boundaries

LACE Liberia Agency for Community Empowerment

LET Liberian Educational Trust

LCO UNICEF Liberia Country Office

LPERP Liberia Primary Education Recovery Programme

NGO Non-Governmental Organization

MDGs Millennium Development Goals


MOE Liberian Ministry of Education

MOF Liberian Ministry of Finance

MSEE INEE Minimum Standards for Education in Emergencies, Chronic Crises and Reconstruction

NER Net Enrolment Ratio

NPSCE National Primary School Certificate Exam

OECD Organisation for Economic Co-operation and Development

OSI Open Society Institute

PTA Parent Teachers Association

PRES Programme Review and Evaluability Study

PREV Progress Evaluation of the UNICEF Education in Emergencies and Post-Crisis Transition

Programme

RTTI Rural Teacher Training Institute

SC/UK Save the Children/United Kingdom

SWAp Sector Wide Approach

TOT Training of Trainers

UN United Nations

UNDP United Nations Development Programme

UNESCO United Nations Educational, Scientific and Cultural Organization

UNGEI United Nations Girls Education Initiative

UNHCR United Nations High Commissioner for Refugees

UNICEF United Nations Children's Fund

UNMIL United Nations Mission in Liberia

UNOPS United Nations Office for Project Services

USAID U.S. Agency for International Development


EXECUTIVE SUMMARY

The Progress Evaluation of the UNICEF Education in Emergencies and Post-Crisis Transition Programme is

an appraisal of the implementation of UNICEF‘s ‗EEPCT: Education in Emergencies and Post-Crisis

Transition Programme‘ (EEPCT). Started in 2006, EEPCT is a four-year, $201 million dollar partnership

between UNICEF, the Dutch Government and the European Union. EEPCT currently supports programming

in 39 countries, as well as global initiatives such as the Inter-Agency Education Clusters and the Inter-Agency

Network for Education in Emergencies. The EEPCT programme aims to ‗put education in emergency and

post-crisis transition countries on a viable path of sustainable progress towards quality basic education for all‘.

The evaluation was global, reviewing EEPCT implementation in 39 countries. However, it also included six

country case studies to support the global analysis. UNICEF Evaluation Office chose Liberia as one of the

case studies. This report examines how EEPCT funds have been used within UNICEF‘s programmes in

Liberia, and discusses ways forward.

Liberia Background

Liberia‘s violent internal conflict ended in 2003. The 14-year conflict devastated the country‘s infrastructure

and caused economic collapse. A significant portion of the population was internally displaced or became

refugees. Seven years later, the country is in the process of recovery and development. To ensure children‘s

access to education and to support a prolonged peace, UNICEF implemented the EEPCT programme.

The Liberian education system was shattered during the country's 14-year war, with the destruction of

schools, loss of staff, and disruption of children‘s education. Schools are currently being built, and teachers

trained and educational materials distributed, but recovery is not keeping pace with public-school enrolment;

between the 2005/2006 and 2008/9 school years, this saw a 48% increase, 33% of which was female.1 This

increase in enrolment was aided by the enforcement of the 2001 Education Act, which made primary school

education compulsory, and the 2006 abolition of school fees.2

The UNICEF Liberia Country Office was active throughout the war and reconstruction period. Within the

education sector, UNICEF responded through its Basic Education and Gender Equality (BEGE) programme.

The office collaborates closely with the Ministry of Education (MoE) and its programmes are closely aligned

with those of the MoE‘s three-year Liberian Primary Education Recovery Programme (LPERP).

The UNICEF country office began implementation of the EEPCT programme in 2007. Liberia is one of 39

countries participating in this US $201 million dollar partnership between UNICEF, the Government of the

Netherlands and the European Union/European Community. The office received a total of US $24,180,000

between 2007 and 2010, making it the largest beneficiary of the 39 programme countries.

In Liberia, the EEPCT programme is implemented through two different avenues, one directly implemented by

UNICEF and the other through the Liberian Education Pooled Fund (EPF). The programmes directly

supported by UNICEF include the continuation and expansion of the Accelerated Learning Programme (ALP),

building the capacity of the District Education Offices, printing supplementary readers, training of teachers and

building a Learning Along Borders for Living Across Boundaries (LAB4LAB) School. Additionally, UNICEF

provided technical assistance to two national school censuses and to development of policies on abolition of

school fees, and supported the creation of long-term educational planning documents such as the Education

Sector Plan.

1 Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009). Liberian Ministry of Education, 2010. 2 Scott, Aldophus, ‘Never to late to learn, 2008. [Access on September 21, 2010 from http://www.unicef.org/infobycountry/liberia_42354.html]

http://www.unicef.org/infobycountry/liberia_42354.html


UNICEF also supported the creation of the US $16.25 million Liberian Education Pooled Fund, which in turn

supported three initiatives: (1) the rehabilitation of the Rural Teacher Training Institutes (RTTIs) through

renovation of teachers housing, logistical support and residential furniture; (2) the purchase and distribution of

1.2 million textbooks; and (3) the construction of 40 primary schools.3

In total, the UNICEF office has implemented roughly nine programmes as part of the EEPCT programme:

educational materials, instructional support, infrastructure support, technical support, Child Friendly Schools,

ALP, LAB4LAB/Talent Academies, EMIS and EPF.

Approach and Methodology

At the country level, the evaluation focused on EEPCT implementation as it relates to beneficiary and

education systems outcomes, as well as the quality and innovation involved in implementation. The evaluation

examined how the following EEPCT programme goals were implemented:

1. Improved quality of education response in emergencies and post-crisis transition countries.

2. Increased resilience of education in chronic crises, arrested development and deteriorating contexts.

3. Increased education-sector contribution to better Prediction, Prevention and Preparedness for

emergencies due to natural disasters and conflict.

4. Evidence-based policies, efficient operational strategies and fit-for-purpose financing instruments for

education in emergencies and post-crisis situations.

The country case studies were designed both to fit into the larger global assessment of the EEPCT

programme and to act as stand-alone studies of EEPCT implementation and outcomes in different national

contexts. This report represents a detailed description and analysis of EEPCT‘s operations, lessons learned

and discussion of ways forward in Liberia. The global report synthesizes findings on country, regional and

global levels.

The Liberian evaluation drew on both quantitative and qualitative methods. Data sources included document

reviews, interviews with key informants, surveys, and field visits; there were also focus-group discussions with

children, youth, parents, and educators. The above information was used to identify and link results in terms of

approaches, processes and outputs.

The evaluation faced limitations due to the timing, logistics, and the decentralization of information between

UNICEF and the MoE. The primary impact of these limitations was to decrease the numbers of schools visited

and the number of beneficiaries consulted. However, some of the challenges were mitigated by ensuring the

communities knew the time and needs of PREV team visits. The team also added an extra research-staff

member from the Columbia Group for Children in Adversity (CGCA), which carried out the evaluation.

Evaluation Findings

EEPCT has significantly improved the UNICEF country office‘s educational response and has already had a

significant impact on the education of Liberia‘s children.

Despite this overall verdict, there are both positive and negative findings from the evaluation. These are set

out below, and are explained in more detail in the main body of the report.

3 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one programme, one process. UNICEF: Monrovia. 2009. Pg.

26.


(1) EEPCT progress was enhanced through the UNICEF Country Office‘s adherence to existing programme

plans.

The strength of EEPCT in Liberia is its integration, where possible, into pre-negotiated plans with the MoE.

The LCO specifically states that it was ‗not implementing EEPCT as a country programme but as a BEGE

supported by other donors as well.‘4 The LCO‘s adherence to these plans is illustrated through its integration

of the LAB4LAB initiative into the pre-existing plans for CFS and its choice not to implement the Talent

Academy because it did not fit within the existing plan.

(2) Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative.

Throughout the evaluation, LCO emphasized that ‗EEPCT was never meant to be a Liberia Country Office

Programme‘ as it was not part of the 2008-2012 Liberia Country Programme. EEPCT was viewed as an HQ-

driven initiative with limited involvement from the Country or Regional Offices.5

This was exemplified by the lack of clarity in the development and dissemination of EEPCT documentation

and understanding of how the programme would be integrated into UNICEF planning structures. As an

example, EEPCT funding came after the approval of the 2008-2012 UNICEF Liberia‘s Country Programme for

Children (CPC) which had been aligned with the MoE‘s LPERP and Education Sector Plan (ESP). The initial

focus on the regional LAB4LAB and Talent Academies initiatives was additionally confusing as it was not clear

whether EEPCT could support programmes outside these two initiatives.

The promotion of EEPCT was also cited as lacking awareness of the realities of field implementation. COs are

judged according to the implementation of the core programmes of Child Survival and Development (CSD),

BEGE and Child Protection, which are derived from government plans. Deviation from these programmes,

such as EEPCT, could affect how offices are evaluated.6

(3) The UNICEF Country Office was able to attract donor support by combining standard UNICEF

programming with LAB4LAB innovation.

The Country Office‘s combination of the innovative LAB4LAB programme with UNICEF‘s standard CFS model

helped leverage support from a significant donor. In 2010, the Japanese government contributed US $8.5

million to support the ‗construction of six LAB4LAB schools, 24 new MoE standard primary schools and the

renovation/expansion of 60 others to increase schooling for 22,500 children.‘7 The office stated that part of the

attraction for the Japanese was the application of CFS principles in this innovative new way.8 School

construction was seen as an entry point to incorporate CFS principles into standard practice.

(4) UNICEF and the Liberian government did not have sufficient staff and technical expertise to implement

the breadth and complexity of EEPCT programmes.

Capacity assessments of COs and implementing organizations should be conducted before programmes are

initiated. As an example, the Country Office stated that in implementing the EPF it ‗lacks enough technical

staff to be able to effectively support the larger macro-processes.‘9 The lack of a monitoring and evaluation

officer in the office has equally affected programme implementation. UNICEF implementing partners, such as

the MoE, should be subject to equal scrutiny. The constant reference to the limited capacity of the government

possibly should have led to more modest interventions with greater monitoring and technical support, as was

4 Email, 16 August 2010. 5 UNICEF, EEPCT Comments on Draft Report, Liberia, 2010. 6 Email, 16 August 2010. 7 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs,

2010. Pg. 9. 8Interview, New York, 17 November, 2010. 9 UNICEF,UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation. Monrovia. 2007.


seen by the additional support provided in the textbook distribution and the finalization of school renovation.

As an example, OSI supported the textbook distribution and will conduct an evaluation of the process within

2010.

(5) Programme transitions should be gradual, with benchmarks to ensure relevance and sustainability.

If programme responsibilities are to be sustainable, there is a need to work within the timeframe and

capacities of beneficiary communities and organizations. For example, the revitalization of the CEO (County

Education Officers) Complexes should have included a longer transition period. As it stands, it was assumed

that the government would supply fuel to CEOs to conduct school monitoring, which it could not do. Equally,

emergency programmes, such as ALP, should have definitive points for review and adjustment. The Country

Office is hoping to begin developing best practices for programme transitions by conducting an ALP

evaluation and an assessment of out of school children in 2011. Results from these studies will inform the

MoE‘s decision regarding the ALP. With regard to the LAB4LAB initiative, UNICEF will support the first year

of the school‘s operation to determine its operating costs.

(6) Monitoring of programmes does not draw upon communities and is poorly supported through programme

transitions.

Program monitoring is the responsibility of all parties involved in program implementation, including children.

Programme planning should include sufficient long-term resources to develop a culture of monitoring and

evaluation internally but also within the MoE and communities which they serve. Children and communities

monitored textbook distribution and maintenance through the implementation of a textbook policy; similarly,

communities were mobilized to monitor ALP programmes in their communities. Building upon these examples,

there should be further community involvement in school construction and supply distribution. Communities

should be involved in the selection of contractors for school construction and should be informed about

upcoming supply distributions via radio transmissions, while materials received should be signed for by a

student, parent and the school principal.

(7) Programmes lack components such as assessments, baselines and strategies to link improvements to

policy change through advocacy.

Few of the EEPCT-funded programmes were based on situational assessments and baselines; will hinder

evaluation the impact of these programmes and advocacy for change. For example, the lack of a situational

assessment and baseline for the LAB4LAB/CFS programme will hinder evaluation and thus the

communication of the programme‘s benefits to communities and government. Advocacy strategies also

need to be included in the development of targeted products. For example, while a documentation exercise for

the EPF was undertaken, the target audience for this product was unclear.

(8) Decentralization supports the implementation of other interventions.

The EEPCT‘s capacity building of the CEO Centres was described by the Country Offices as part of their

government capacity building initiatives. The evaluation team would identify it as supporting the resilience of

the education system. This support is in line with MoE‘s plans for decentralisation, which it hopes will increase

service efficiency, transparency, accountability and responsiveness.10

EEPCT‘s support for the rehabilitation

of CEO complexes and provision of motorbikes was an initial step complemented by training, data collection

and production of the National School Census. The training of 92 DEOs and 14 CEOs to perform the census

built MoE capacity and created the opportunity for links between the community level (schools), the county

level (CEOs and DEOs) and the National MoE.11

This capacity is important because it improves monitoring

10 World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group. [ Accessed on September 18, 2010 at http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html] 11 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010.


and evaluation at the community level, including more efficient and timely data collection and improved

programme management.

(9) Significant participation of children and communities was lacking within EEPCT programming.

A significant portion of EEPCT work was ‗upstream‘ in supporting the MoE. This strong upstream approach

did not bring communities and children into the decision making and dialogue. By incorporating children‘s

voices into programme design and implementation, the efficiency and quality of EEPCT programming could

be enhanced. Children have valuable insights and energy to contribute in post-crisis situations. At both the

community and national level, children can help advocate change on issues such as gender-based violence in

schools. This should be an intentional area of focus in moving forward with CFS schools.

(10) Emergency Education needs a structured approach that takes into account natural disasters and conflict.

With the on-going tensions in Guinea, UNICEF and the MoE would benefit from a more structured approach to

the implementation of emergency education responses. The Country Office chose not to implement any

Disaster Risk Reduction Programmes as part of the EEPCT programme. Conflict in surrounding countries and

the potential resurgence of conflict overshadows the risk of natural disasters in the country. When asked

about disaster preparedness, one community-level key informant expressed that children should be taught to

flee if war came again.12

As mentioned above, Liberia also faces minor threats of natural disasters such as

floods and storms.13

The Way Forward: Recommendations

Government/MoE:

Evaluate the relevance the ALP programme. This can be supported by UNICEF‘s ALP evaluation and

the UNESCO/UNICEF assessment of out-of-school children.

Establish an emergency education/preparedness section in collaboration with the IASC Education

Cluster Leads (UNICEF and Save the Children) based upon the FTI‘s recommendation.

Remain engaged with the implementation of CFS to identify best practices to adopt as national policy.

Establish, and identify resources for, a monitoring plan for CEOs.

UNICEF Headquarters:

Lead the development of programme learning workshops around specific emergency education

programme components, such as ALP. The results of these workshops could then be developed into

a programme plan to make promising practices into best practices through thorough assessment,

baselines and evaluations.

Reinforce the application of the programme cycle at the global, regional and country level through

inclusion in workshops, review of proposals and newly developed programme documents. This

should emphasize the importance of tying evaluations to advocacy.

12 Interview. Liberia. 1 August 2010. 13Preventionweb. Liberia profile (Accessed September 21, 2010 from (http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr) 2010.


UNICEF HQ should lead a lessons learned review of the EPF to identify when such a strategy should

be applied but also the implications for the CO regarding staffing and technical support. If Pooled

Funds become a common intervention UNICEF should develop in-house capacity to provide technical

support.

Adapt the CFS model to be used in emergencies.

Identify resources to continue the LAB4LAB and Talent Academy programmes to an established

transition point. Both of these initiatives are high level, regional/global initiatives. UNICEF is under

some obligation to see them through to commonly identified transition point. This would include

updating the Global Framework of LAB4LAB and Talent Academies in collaboration with the Regional

Offices and COs. Such a process would include a sociological and political survey, survey,

identification of interventions and indicators, and decisions regarding how LAB4LAB/Talent

Academies will proceed as a regional/global endeavour.

Regional Offices

Identify and provide the support needed by EEPCT countries through existing personnel, short term

contracts or consultancies. Participate actively in the development of long-term LAB4LAB and Talent

Academy planning.

UNICEF Liberia Country Office:

Review opportunities for children‘s participation in decision making and programme implementation.

Specifically, children‘s involvement should be integrated into the expansion of the CFS programme.

Initiate a long-term implementation and transition plans for the Ganta LAB4LAB School. The LCO

needs to develop a plan for how continued costs will be covered based upon the first year operational

costs (including electricity, computer maintenance, radio license, etc). New partners, such as Search

for Common Ground/Talking Drum to lead radio programming, should be identified as part of this

planning process.

Support the MOE to make informed decisions regarding the future of the ALP including exploration of

options such as youth/adult literacy programming and early childhood education.

Review the implementation of CFS programme to include situational assessments as well as baseline

and evaluation strategies that will provide data which can be used for advocacy.

Review the staffing that is needed to ensure sufficient monitoring and transitioning of programmes.

Possible areas of focus include the clustering of activities around (1) ALP – conducting the ALP

Evaluation, Out of School Children Assessment and coordination with the government – and (2)

CFS/LAB4LAB programme – implementation of the LAB4LAB School, coordination of additional

schools with Japanese funding and the roll-out of the CFS programme. These positions are important

for the continuation for the programme after EEPCT.

Identify and address missing technical support that has resulted from the absence of a monitoring and

evaluation officer. Short-term consultants could be hired to support assessments, baseline and the

development/application of indicators.

Review UNICEF‘s strategy to support the Ministry of Education to monitor schools, distribution and

data collection.


Develop a multi-year CFS advocacy strategy to accompany the 2008-2012 Education sector plan.

The CFS programme should be a product of a solid assessment, and development of indicators and a

programme baseline. Moreover, this should be coordinated with the development of messages and

products for targeted audiences.

Document the ALP programme including programme structure, guidelines, monitoring structures and

government involvement.

Plan a dissemination and advocacy strategy for the ALP Evaluation and the UNICEF/UNESCO

assessment. Messages should be targeted for communities, the Liberian government, UNICEF and

the international community.


1.0 INTRODUCTION

1.1 Overview

UNICEF‘s Education in Emergencies and Post-Crisis Transition Programme (EEPCT) began in 2006 as a

four- (later extended to five-) year, US $201 million dollar partnership between UNICEF and the Government

of the Netherlands with €4 million (about $5.8 million) additional support contributed by the European

Commission. The EEPCT programme is intended as a strategic intervention in support of the Millennium

Development Goals (MGDs) and the Education For All (EFA) movement, and both increases institutional

capacity and provides direct programme support. EEPCT works in 39 countries, and also supports global

initiatives such as the Inter-Agency Standing Committee‘s Education Cluster and the Inter-Agency Network for

Education in Emergencies (INEE).

Monitoring of the EEPCT‘s implementation was included within the project‘s design. In October 2009, UNICEF

commissioned an independent programme review and evaluability Study (PRES) to assess EEPCT‘s

performance so far. The evaluability component looked at a number of priority areas for future evaluation,

including: access to quality education; education system development; disaster risk reduction (DRR); and

conflict-related impacts—and offered guidance on how these components, as well as EEPCT as a whole,

could be evaluated. This progress evaluation addresses the PRES findings and recommendations. It

examines the implementation of the EEPCT programme to date, globally and in more detail in six case-study

countries. The evaluation‘s objectives include:

1. Taking stock of the first three years of implementation to determine, as systematically and objectively

as possible, the EEPCT Programme‗s relevance/appropriateness, effectiveness, efficiency,

coherence/coordination, and, as far as possible, impact and sustainability in relation to its objectives;

2. Evaluating both the intermediate results achieved and the processes set in motion, to assess

EEPCT‘s added value for the education sector in emergencies and post-crisis transitions, as well as

UNICEF‘s specific contribution to the programme;

3. Gathering relevant and applicable lessons learned on education interventions in emergencies,

transition, and fragility-affected contexts; and,

4. Making recommendations for future programming and for future decision-making by UNICEF

headquarters, regional and country offices, and relevant stakeholders.

The six countries to be examined as case studies of EEPCT implementation were chosen by UNICEF‘s

Evaluation Office. This report examines EEPCT implementation in Liberia, included because it is the largest

beneficiary of EEPCT funding and also has an active fit-for-purpose financing mechanism. Findings of this

case study also contributed to the overall report on implementation of the EEPCT globally.


1.2 Country Context

Liberia‘s violent internal conflict ended in 2003. The

14-year conflict devastated the country‘s

infrastructure and caused economic collapse. A

significant portion of the population was internally

displaced or became refugees. Liberia is one of the

poorest countries in Africa;14

84% of the population

live on less than US$1.25 a day, and 85% of the

population work in the informal sector.15

Of children

under the age of five, 26% are underweight, and 36%

do not use an improved water source.16

Liberia ranks

169 out of the 182 countries in the UN Human

Development Index.17

In 2008, Liberia initiated the

2008-2011 Poverty Reduction Strategy (PRS) to align

itself with the MDGs. Data about the situation in

Liberia is limited but it is clear that only a few of the

MDGs will be met.18

The extended war has made Liberia a challenging

environment in which to implement programmes.

The war had a significant impact on infrastructure

such as power, telephone and internet – and building

maintenance. Additionally, there are too few qualified

Liberian staff and contractors for reconstruction.

Liberia‘s limited road system was especially affected as

bridges were destroyed and roads fell into disrepair. All of this has made communication, travel and provision

of programme support difficult.

1.3 Educational Context

The war shattered The Liberian education system; 48% of schools were either destroyed, or damaged enough

to not be fully useable.19

This was coupled with the looting of roofing, school furniture and educational

materials, and the destruction of the two operational Rural Teacher Training Institutes (RTTIs) in Bong and

Maryland Counties.

14 Government of the Republic of Liberia. 2008 National population and housing census: preliminary results, Institute of Statistics and Geo-Information Services, 2008.pg.2. 15 Department of State, Background Note on Liberia, 2010. U.S. government. [Accessed by on September 18, 2010 at http://www.state.gov/r/pa/ei/bgn/6618.htm] 16 UNDP, Human Development Report, 2010. [Accessed online September 18, 2010 at http://hdrstats.undp.org/en/countries/data_sheets/cty_ds_LBR.html] 17 UNDP. Human Development Report 2009 – Liberia, 2009. [Accessed online September 18, 2010. At http://hdrstats.undp.org/en/countries/country_fact_sheets/cty_fs_LBR.html] 18 Republic of Liberia, Liberia’s progress towards the Millennium Development Goals 2008, Monrovia. 2008. 19 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010.

Figure 1: Map of Liberia


During the war, when and where security permitted, the Liberian Ministry of Education (MoE), UN, NGOs and

donors sought to maintain the education system. At the same time, UN agencies and NGOs also established

education programmes in the refugee camps along the border in Guinea and Côte d‘Ivoire. Students who fled

to urban Montserrado County or the refugee camps in Guinea and Côte d‘Ivoire were able maintain their

education and integrate into the Liberian school system upon their return.

Replacement and rehabilitation of damaged and destroyed schools continues, but slowly. Between the

2006/2007 and 2008/2009 school years, the number of operational schools increased from 865 to 1,282.21

The MoE reports that 59% of schools still require repair or reconstruction, and only 15% have water.22

There

is a lack of qualified teachers; during the war, both teachers

and educational administrators were killed, forced to leave the

profession, migrated to urban areas, or sought refuge in

another countries. In areas and times of stability, volunteer

teachers without formal training or experience entered the

classrooms. These teachers continue to be the foundation of

Liberia‘s education system, in which only 40% of all teachers

are considered qualified.23

The war also affected the ratio of

male to female teachers, with only 11% of the workforce now

women.24

However, between the 2005/2006 and 2008/9 school years,

Liberia saw an increase in schooling at all levels. Table 1

shows this increase across the public school system for this

period. Overall, there has been a 48% increase in public school

enrolment, of which 33% was among girls.25

This increase was

aided by the School Law ratified in 2002, which abolished school fees for primary schools in 2006 and made

schooling at this level compulsory.26

However, the education system is severely stressed and cannot

adequately accommodate this influx of new students. In addition, some children remain out of school as they

are unable to pay unofficial school support ‗fees‘, as well as the costs of school supplies and uniforms.

Overage students are common in primary school classrooms as a consequence of the war coupled with a

culture of late enrolment of children. According to the MoE, primary school enrolment is only for children ages

6-11; but 63% of primary school students in the 2007/2008 school year were over 11.27

Class sizes are

reported to exceed 60 students.28

Yet improvements are being made, as , according to the Fast Track

Initiative (FTI): ‗growth in gross enrolment in primary school has decreased to 89% (85% girls and 93% boys)

from 155% in 2006 and net enrolment ratios are 42%29 up from 25% in 2006.‘

30 Secondary gross enrolment is

33%, with 29% girls and 37% boys.31

A USAID assessment of youth in 2008 found that 85% of those sampled wanted to further their education and

build skills. The greatest request (39%) was for vocational skills, while 27% requested literacy and numeracy

20 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 21 Ibid. 22 Ibid. 23 Ibid. 24 Ibid. 25 Ibid. 26UNICEF, EEPCT Comments on Draft Report, 2010. 27 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 28 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 29 UNICEF, EEPCT Comments on Draft Report, 2010. 30 Fast Track Initiative. Recovery of the Education Sector in Liberia. 2010. [Access on September 18, 2010 from http://www.educationfasttrack.org/newsroom/focus-on/fti-annual-report/recovery-of-the-primary-education-sector-in-liberia/] 31 UNICEF, Self assessment, 2010.

Table 1: Liberia Public Enrolment Data between 2005/6 and 2008/9

20

Total number of student in

public schools

% increase between

2005/6 and 2008/9

Pre-primary 335,600

69%

Primary 340,869 31%

Accelerated Learning Programme

60,405 55%

Junior High 46,176 36%

Senior High 20,335 94%

Table 1

http://www.educationfasttrack.org/newsroom/focus-on/fti-annual-report/recovery-of-the-primary-education-sector-in-liberia/


skills.32

Children whose education was interrupted during the crisis have now grown up illiterate. Although

progress has been made, Liberia is unlikely to achieve the MDG and EFA goal of universal primary

education.33

UNICEF‘s programmes are closely aligned with those of the MoE‘s Liberian Primary Education Recovery

Programme (LPERP). The LPERP is a three-year programme focused on the rehabilitation of primary

education. This programme, which began in 2007, has been the framework for educational reconstruction.

During its first year, Liberia was admitted to the EFA/Fast Track Initiative (EFA/FTI) Partnership but was not

granted financial support. However, with the assistance of the Education Programme Development Fund

(EPDF), the Ministry has been preparing a ten-year Education Sector Plan (ESP) based on an exhaustive

Country Status Report (CSR) and consultation process that included all education stakeholders. This

consultative development has allowed for building consensus between the government and donors.

The ESP outlines the short-, medium-, and long-term government strategies for the education sector and is

accompanied by a three-year implementation plan. A sector policy letter lays out the key government

priorities, main policy reforms and target indicators to be met over the ten-year period of the ESP. This letter

has been endorsed by both the MoE and the Ministry of Finance.

In terms of government spending on education, it is estimated that since 2004, average real spending on

education has increased by 26 percent per year.35

This increase is similar to other areas of government

expenditure. In 2010, Liberia qualified for US $40 million-worth of funding from the EFA/FTI Catalytic Fund.36

1.4 EEPCT Programme

Since 2007, UNICEF-Liberia has received US$24.2 million in EEPCT support (Table 2). In Liberia, the EEPCT

programme is implemented through two different

avenues. First, it is directly implemented within UNICEF

Education Section‘s Basic Education and Gender

Equality (BEGE) programme. The second avenue is

through support of the MOE‘s Liberia Education Pooled

Fund (EPF).

The primary EEPCT programme implemented within the

BEGE is the Accelerated Learning Programme (ALP).

Starting in 1998, ALP was an ‗emergency‘ intervention to

enable over-age children to re-enter or finish primary

school in three years instead of six. With support from

UNICEF, the MoE implements the programme by

providing learning, recreational, and teaching materials,

furniture and refresher courses for the local teachers.

UNICEF is the MoE‘s largest supporter for the ALP. In

2009, the UNICEF-supported ALP was expanded from

11 to all 15 counties in the country.37

In 2010, 30,78538

students were supported by UNICEF within the government‘s ALP programme, which accounted for 41% of

the estimated 76,000 students nationally.39

32 Coyne, Geoff, et al, Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID. 2008. 33 Republic of Liberia, Liberia’s progress towards the Millennium Development Goals 2008. 2008. 34 UNICEF, Self-assessment, 2010. 35 World Bank and Government of Liberia, Public Expenditure Management and Financial Accountability Review (PEMFAR), 2009. 36Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education

recovery. Open Society Institute. 2010. Pg 2. 37 UNICEF, UNICEF Liberia: Annual Report 2009, Government of Liberia-UNICEF Programme of Cooperation, Monrovia, 2010. 38 Ibid.

Table 2: Liberia EEPCT Support34

Year

Receiv

ed

BEGE

Support

Education

Pooled

Fund

(EPF)

Total

EEPCT

Support

2007 2,000,000 2,000,000

2008 1,500,000 12,000,000 13,500,000

2009 1,500,000

2,500,000 1,000,000

2010 2,980,000 3,200,000 6,180,000

TOTAL 8,980,000 15,200,000 24,180,000

Table 2


EEPCT also enhanced the operational capacity of the County Education Offices. The training of the County

Education Officers (CEOs), District Education Officers (DEOs) and enumerators supported the development

of the national Education Management Information System (EMIS) to produce the 2007/2008 and 2008/9

Census Reports.40,41

The data from this report directly supported the implementation of LPERP.

With support from the EEPCT programme, the UNICEF country office chose to participate in the regional

Learning Along Borders for Living Across Boundaries (LAB4LAB) programme. The LAB4LAB programme

seeks to use education regionally as a means of addressing developmental challenges to peace posed

throughout the Mano River countries of Guinea, Côte d‘Ivoire and Liberia. Built on land donated by the

community, the LCO initiated construction of the LAB4LAB school in Ganta, Nimba County, in 2009. The

school contains advanced features such as solar panels, computers and a radio transmitter to facilitate cross-

border communications. The construction of the school has been challenging due to security issues, poor

roads and a long rainy season.

In addition to the above efforts, EEPCT has supported infrastructure development through contributing

consultants to support the design of the pooled fund (2007), development of the -sector-wide approach

(2007), the LPERP implementation plans (2008), the Education Sector Plan (ESP) (2009), and the School Fee

Abolishment Policy (2009). In the case of the ESP, the EEPCT funded an additional consultant in coordinating

inputs from partners for the application for the Catalytic Fund.

EEPCT has also funded the training of 250 school teachers, and helped buy over 1 million textbooks and

readers for grades one, two and three and 20,000 school chairs. Most recently EEPCT funds were used to

support supported the MOE to establish a Child-Friendly Schools programme (CFS). To maintain program

coherence, the LCO integrated the LAB4LAB School under its existing BEGE plans for CFS.42

However, the largest portion of EEPCT support in Liberia has gone to the creation and initial funding of the

EPF (the pooled fund), which is the second avenue for educational support for the LPERP. which was

established with EEPCT and Open Society Institute (OSI) funding as well as UNICEF technical support. The

EPF is intended to assist development of a ‗fit-for-purpose‘ financing system, to implement an education

reform programme while cultivating national capacity (in line with EEPCT Goal Four). However, according to

the 2009 Consolidated Report to the Government of the Netherlands, UNICEF Headquarters saw the EPF as

a programme while the UNICEF country office felt it was another funding mechanism.43

The MoE and Ministry of Finance oversee the EPF, assisted by an advisory board composed of the fund‘s

donors, UNICEF and the Open Society Institute. The EPF was designed with short-term technical assistance

from UNICEF Headquarters working with the MoE from November 2007 until the launch of the EPF on May

22, 2008. Headquarters assistance was also provided to operationalise the Fund through a sector-wide

approach.44

Since its inception, the EPF has supported three initiatives: the re-establishment of the three Rural Teacher

Training Institutes (RTTIs); the distribution of 1.2 million textbooks; and the construction of 40 schools

throughout the country.

39 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 40 UNICEF, Progress Report for UNICEF’s Education in Emergencies and Post-Crisis Transition Programme. Monrovia. 2009. 41 Ibid. 42 Interview, New York, 16 November 2010. 43 UNICEF, EEPCT Comments on Draft Report, 2010. 44 Ibid.


2. METHODOLOGY

The methodology used by the evaluation is described briefly in this section, and in more detail in Annex 1.

The evaluation team consisted of two co-leaders and a biostatistician, a finance administrator, and a

programme officer. Besides the six case-study countries, the co-leaders and support staff were responsible for

implementation of global evaluation activities, as well as final data analysis, synthesis and report writing. The

Liberia case study team consisted of two co-team leaders, a Columbia University Research Associate and

three national researchers.

2.1 Purpose and methods

This evaluation is a progress evaluation, or PREV; its purpose is to determine the impact of EEPCT so far,

identify lessons learned and make recommendations for strengthening the programme in the years to come.

Within this context, the evaluation seeks to achieve three inter-related objectives:

Provide an outcome-indicative impact analysis of the EEPCT Programme (2006-2009)

Examine the programme against DAC evaluation criteria as applied to education in emergencies, transitions

and fragility

Provide evidence-based conclusions, lessons learned and recommendations.

EEPCT was also examined with reference to five cross-cutting issues. These included ender; rights-based

programming; disaster risk reduction; and sensitivity to conflict and fragility. The fifth was accountability,

monitoring, evaluation and learning.

At the country level, the evaluation examined how EEPCT‘s goals were translated and adapted to country

contexts and why. In addition, the PREV sought to determine the extent to which global and regional

strategies and outputs are supporting change in the six case-study countries. The country case studies were

designed to both fit into the larger global assessment of the EEPCT programme and to act as stand-alone

studies of EEPCT in different national contexts. The research plan allowed for six to eight weeks per country.

A GCA-Inter-Agency Network for Emergency Education (INEE) Memo of Understanding also enabled close

collaboration with education agencies and their staff in each country where a case study was conducted.

The evaluation used both quantitative and qualitative methods, using the results of one method to support the

informed development of others. This approach enabled both in-depth exploration of methods and innovation

as well as the collection of comparable data across countries. A literature review ranged from government

documents to publications produced by local and international NGOs to academic literature. The evaluation

also reviewed financial allocations relative to the four programme goals and the cost-effectiveness of EEPCT.

The evaluation included individual interviews with relevant UNICEF staff, government officials, NGO staff and

other key stakeholders at the community, regional and global levels. In case-study countries, the team

consulted key stakeholders; these were identified in consultation with the UNICEF country offices, and

included key government and civil-society actors and others who were most knowledgeable of the education

sector in general and the EEPCT programme in particular.

The research team employed participatory methods with beneficiaries—children, parents, educators and local

communities members—to ensure an upstream flow of findings and recommendations. Field visits also

documented the results at the country level of global and regional contributions (INEE Minimum Standards for

Education in Emergencies, Chronic Crisis and Early Reconstruction, Education Cluster coordination and

Regional Office technical support). There were also focus-group discussions that used different tools to


capture participants‘ perceptions of EEPCT; these tools included the DAC score card, Child-Friendly Schools

checklists, and participative ranking methodology (PRM). These are discussed in more depth in annex 1.

2.2 Field-site and sample selection

Field-visit sites in Liberia were selected both at random and through planned sampling processes. After

consultations with the UNICEF country office and the national research team, Monterrado and Nimba

Counties were chosen for field visits, as the majority of the country‘s students are enrolled in these two

counties. The country office was also consulted on the EEPCT programme components.

When these were identified, the locations of programme sites were determined. Because the textbook

programme that was administered as part of the LPERP and supported by the EPF was nationalized,

comparison groups of public schools not benefitting from the programme were not possible. As the

accelerated learning programme (ALP) was implemented in all counties but not all schools, a random

selection of schools was made from a list of all UNICEF-supported ALP programmes. Sites from this selection

were then chosen based on accessibility and distance.

The sample of stakeholders was selected at national and sub-national level, including MoE staff, government

officials, county and district education officers, and international and national NGO staff. At community level,

the stakeholders included students, teachers and parents. Fuller details of the focus-group participants may

be found in annex 1.

2.3 Limitations

The evaluation team took multiple steps to ensure the methods used in Liberia were consistent with those of

the research teams in the other case-study countries. To achieve this, there were several components

common to each country, including an introductory PowerPoint presentation for stakeholder consultations,

user guides for each tool and instructions for all data collection methods; central training; and a consistent

approach to data entry, including creation of a singular entry form, and to supervision of research teams

during the fieldwork.

However, the lack of global clarity on multiple terms within the sector limited a theory based approach to this

evaluation. This limitation includes the meaning of terms (emergency, post-crisis, transition, quality education,

resilient education system). For these reasons, the CGCA created blogs that sought to clarify these terms.

Despite this, development of definitions, terminology and concepts continued to be difficult for focus-group

participants at the community level, due to differences in education, insufficient English-language abilities or

lack of similar words within vernacular language. Also, no programme baseline was established, which made

it impossible to report on percentage changes in goal-area outcomes. The evaluation team addressed some

of these concerns through the use of retrospective baselines.

In addition to the above, several other factors had an impact on the evaluation at field level in Liberia. Few key

informants or focus-group participants knew of EEPCT. There were also some practical problems; for

example, the evaluation coincided with school holidays, making it harder to gather participants from all cohorts

for focus groups at community level (this was mitigated by contacting regional officials and community

members prior to the evaluation visit). Also, UNICEF did not hold all the information regarding programme

sites, locations and participants, so it had to be obtained from different parties at the MoE, slowing the

planning process. Finally, in one country, the county and district education officers were at a workshop abroad

at the time of the evaluation team‘s mission.


3. EVALUATION FINDINGS

3.1 Overview: Results Analysis

The progress of EEPCT implementation was estimated both overall, and through the individual programmes

against three of the four EEPCT goals (in the case of Goal Three, the UNICEF office was not implementing

any activities). The evaluation found that UNICEF Liberia‘s EEPCT programme was having a significant

impact on reconstruction of Liberia‘s education system. This section presents the evaluation‘s conclusions on

EEPCT in Liberia in detail – first by the four EEPCT goals, and then with reference to overarching areas

(including operations and management, and performance on cross-cutting issues such as gender).

As will be seen, EECPT is having an attributable impact under Goal One (to increase Liberian children‘s

access to quality education). Since its inception, EEPCT has increased the number of textbooks in schools

nationwide; trained 250 teachers to C-level certification; conducted CFS training; supported the completion of

19 schools, including the LAB4LAB School; and provided important technical support to the MoE, such as

supporting the drafting the School-Fee Abolition Policy.

Contrary to what is reported in the EEPCT global donor report, the UNICEF Liberia country office did not see

its programme as contributing to Goal 2, as ‗Liberia does not have chronic emergencies, no arrested

development or deteriorating contexts.‘ 45

All of the programmes attributed to resilience were carried out

under Goal 3. The resilience of the education system was supported through the continuation and expansion

of the ALP to provide direct support to 30,785 students.46

The regional LAB4LAB regional initiative in which

the country office participated is another means of supporting resiliency and peace within Liberia as well as in

the Mano River Region.

The EECPT programme supported two national school census reports and better planning and programme

implementation within UNICEF, the MoE and other education stakeholders. The EEPCT has also supported

the development and continuation of the Liberian Education Pooled Fund. This fund‘s dual purposes are to

support government capacity building and to provide programme support. Overall, the programme has fallen

short of these goals due to an over-estimation of the capacity of the government to implement large projects.

Donors who have conducted external reviews of the fund have decided not to fund it.47

The research team found that while EEPCT has made strides in several of the goal areas, there are still

certain limitations to its work in Liberia. The following sections provide a detailed description and analysis of

the particular goal areas in which EEPCT funds and capacity have been invested.

3.2 Goal One: Improved quality of education response in emergencies and post-crisis

transition countries

The UNICEF country office was able to provide information for four out of the five indicators for Goal 1. One

indicator, regarding primary completion rate, could not be calculated due to lack of information.

UNICEF in Liberia has achieved the EEPCT goal of 80% of the children being enrolled if the indicator is

applied to Gross Enrolment Ratio (GER). The indicator is not achieved if applied to Net Enrolment Ratio

(NER).48

EEPCT contributed directly to the increased number through the construction of the LAB4LAB

School, and more than 800 permanent learning spaces were created under the EFP.

45 UNICEF, EEPCT Comments on Draft Report, 2010 46 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010. 47 Interview. 5 August, 2010. 48

UNICEF, [Results Framework Response], 2010.


Training and educational materials

In 2007, UNICEF distributed 140,000 Grade One and ALP supplementary learners‘ kits.49

In 2008, the EPF

supported the procurement and distribution of 1.2 million English, Math and General Studies textbooks with

teacher‘s guides, and trained principals and teachers on systems for their retention and use. According to

UNICEF, this distribution has reduced the student-to-textbook ratio from 27:1 prior to the programme to 2:1.50

However, as school was not in session during the evaluation, this programme was assessed through literature

review, focus groups and observation. For this reason, the evaluation team was not able identify teachers

trained, observe classes or accurately measure the student-to-textbook ratio. A key informant within the MoE

stated that the textbooks had been procured but could not attest to their distribution as no end-user monitoring

system was in place.51

Other informants reported problems with the capacity of county and district education

officers to monitor the equitable distribution of texts. The Open Society Institute contributed one person to

support the procurement and distribution of the 1.2 million textbooks;52

but it is doubtful this was sufficient.

Assessing impact of the textbooks through the focus-group discussions was difficult as participants did not

distinguish between the textbooks themselves and their effect on the students that used them. However, four

out of the seven discussions conducted with children and youth cited the positive aspects of textbooks by

saying that they were less afraid, their attendance was improved and that the books enabled them to teach

other students. An important finding was that the textbooks were a sustainable input since they were

distributed with training and copies of a Textbook Policy and Instructional Material Management Handbook.

Parents, in one out of the two discussions conducted with them, stated the importance of being involved in the

distribution of the textbooks and signing for them during distribution. However, parents also reported this

process as a weakness, since they were monetarily responsible if the books were damaged or not returned.

Three out of five groups of youth also cited this as a weakness.

Infrastructure

Observation, literature review, key-informant interviews and focus groups were used to evaluate the progress

of this programme. The EPF supported the MoE in constructing 40 schools across the country. This project

was divided between two contactors at the time of evaluation data collection. Only 18 of the schools, which

were all contracted to one contractor, had been completed. The remainder were all between 40-45%

completed due to contractual issues between the MoE53

and the other contractor regarding overheads. The

completion of these schools awaited the MoE‘s selection of an alternative contractor.

The evaluation team visited four out of the 18 completed schools. The school buildings were examined to

evaluate whether or not selected criteria from the UNICEF CFS Manual were met.54

Two of the schools met

four of the 12 selected criteria; the other two met only three criteria. However, the building design was outside

UNICEF‘s control, as MoE designs and standards were used. The UNICEF country office was clear that these

schools were not planned to be ‗Child Friendly‘.

The team also visited one of the five rehabilitated County Education Office (CEO) complexes. EEPCT‘s

revitalization of these complexes assisted the government‘s process of decentralization. The rehabilitation

included, where appropriate, warehouses, seminar rooms with kitchens, wells with water towers and

generators/housing for electricity. This support enabled the complexes to act as hubs for the DEOs and their

49 UNICEF, UNICEF Liberia: G3001 Netherlands, UNICEF Liberia Country Office: Monrovia, 2007. 50 UNICEF. Progress Evaluation of the Education in Emergencies and Post Crisis Programme (PREV). Liberia: UNICEF. 2009. 51 Interview. 12 July, 2010. Liberia. 52 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. pg 16 53 Interview. Monrovia, Liberia. 27 July, 2010. 54 UNICEF, Child Friendly Schools Manual. New York. 2009. Criteria included the following: These criteria included the following: whether or not the school appeared child friendly; if classrooms were easily exited in emergency; all classrooms have windows; separate spaces are provided for teachers and administrators; teachers and administrator location allows for monitoring students; availability of water on school grounds; separate latrines for boys and girls; locks on latrine doors; appropriate ratio of latrines to boy and girl students; separate latrines for teachers; separate space with soap and water for hand washing; and presence of a disaster risk reduction plan.


monitoring. The complexes were also reported to decrease operational costs.55

USAID and the EU have

recognized their importance and have started to provide additional support.56

In the two sub-national

government focus-group discussions conducted, one ranked the change in monitoring and evaluation as

much better (5) while the other ranked it as better (4). The latter group explained that the lower ranking was

partly due to the fact that while motorcycles had been provided, the Ministry decreased the amount of fuel and

maintenance provided, curtailing their monitoring availability.

Under the LPERP, the EEPCT programme combined resources with other partners, including USAID, and

planned to assist the re-establishment of the three Rural Teacher Training Institutes (RTTIs) in Webo, Zorzor

and Kakata (the RTTI in Webo was eventually not supported due to delays in construction). However, the EPF

contributions to the RTTIs were not clear and the team were not able to verify them. According to the UNICEF

2008 donor report, classroom and library furniture and equipment were provided. 57

However, according to

Schmidt‘s report on the EPF, the support included ‗transport, cookers, residential furniture and other

requirements‘58

and the procurement of ‗three buses, three pick‐ups and three jeeps.‘59

Additional learning

materials were provided under the second tranche of EEPCT funding. As school was not in session, the team

could not confirm this support or identify provided teaching and learning materials. However, a USAID mid-

term evaluation noted that these items were non-existent or in poor condition.60

The sustainability of the RTTIs was not considered. From the literature review, the MoE had assumed that the

recurrent costs of US $2 million per year would be covered by project partners; however, the MoE request for

support was turned down by the EPF. USAID and other organizations have come forward to assist the

Government in supporting these institutes. However, USAID‘s mid-term evaluation of the programme found

the RTTIs underused and unable to attract qualified trainees. It also highlighted the lack of female trainees

(only 44 out of 478 candidates in 2009).61

Child-Friendly Schools Programme (CFS)

There has been training on the CFS Framework at the LAB4LAB School, for its 16 teachers as well as those

from the surrounding area. There was also later training for 55 trainers-of-trainers. In 2010, UNICEF held

sensitization workshops to support full implementation in 2011.62

This concept of implementing CFSs from the

start of construction onwards has attracted the attention of the Japanese government and has garnered

support of US $8.5 million to take the concept forward. The proposal will support ‗construction of six LAB4LAB

schools and 24 new MoE standard primary schools, and the renovation/expansion of 60 others to increase

schooling for 22,500 children.‘63

However, no baselines and indicators have been developed to monitor

programme implementation or assist in evaluation of project outcomes and impacts.

Coordination

In relation to coordination, the EEPCT programme established an Education Development Partners

Secretariat, to coordinate donor support to the sector within the sector-wide approach. The Secretariat hosted

bi-weekly partners‘ meetings, ensuring the alignment of partner support to government education priorities.

The Secretariat organized the first Education Sector Review in June 2009, and was instrumental in

55 UNICEF, EEPCT Comments on Draft Report, 2010. 56 UNICEF. Self-assessment. Monrovia. 2010. 57 UNICEF, Progress Report for UNICEF’s Education in Emergencies and Post-Crisis Transition Programme. Monrovia. 2009. pg. 12. 58 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg.

44. 59 Ibid. 47. 60 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID, 2008. pg.9 61 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID. 62 UNICEF. Utilization of Donor Funds and Expenditures –EEPCT. Monrovia. 2010. 63 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs, 2010. Pg. 9.


coordinating partners‘ inputs and appraisal of the 10-year ESP for Liberia. This supported the successful US

$40 million EFA/FTI submission in 2010.64

3.3 Goal Two: Increased resilience of education sector services delivery in chronic

crises, arrested development, and deteriorating contexts

As stated above, the UNICEF country office did not categorize its programmes under Goal Two as Liberia was

not felt to be a country in chronic crisis, arrested

developed or deterioration.65

(Resilience will be

reviewed later in this report with reference to the DAC

Scorecard and Results Logframe.) Some of the four

indicators for Goal Two were not applicable. However,

information was provided for one of them – the

percentage of the national budget dedicated to

education, as a sign of resilience. The data showed

an almost doubling of this percentage from 7.9% to 14

% between 2007-2009. EEPCT can claim some

attribution to this rise due through the formation of the

EPF. The office‘s results framework states that

education remains a high priority.

Accelerated Learning Programme (ALP)

EEPCT directly supports the access of 8% of the country‘s children to primary education and half of the

country‘s total ALP students supported by the government.66

According to the evaluation of the program

conducted by UNICEF prior to the start of EEPCT, the ALP program has many strong features. As with ALP

programmes in other countries, the MoE has strong ownership of the programme, having developed the

curriculum, teaching manuals and structure. Selection of communities for the programme is at county level,

while PTAs are responsible for running the actual classes. MoE Master Trainers provide pre-service and in-

service training to ALP teachers. Monetary incentives are provided for the ALP teachers and the principals to

promote the operation of the programme.67

Since its inception, the ALP programme has benefited more than 60,000 children.68

This large number can be

attributed to UNICEF‘s 10 years of support to the programme, which has continued via EEPCT funds since

2007. From the 2005/2006 to the 2008/2009 school years, the number of ALP students increased by 55%,

compared to a 33% increase in overall primary school enrolment.69

However, this rise coincides with key

informants reporting that the ‗ALP system was being abused‘ due to the large number of under and overage

children in the programme.70,71

The 2007/8 school census shows that parents register their children late for

primary school (see Chart 1), which should start at six years old.72

This late enrolment overlaps with the ALP

64 UNICEF, EEPCT Comments on Draft Report, 2010. 65 UNICEF, EEPCT Comments on Draft Report, 2010. . 66 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 67 Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia Implemented by UNICEF. 2007. 68Wayne, Trokon. and vonHahmann, Gail. Achieving quality in accelerated learning: Student assessment in Liberia" Paper presented at the annual meeting of the 53rd Annual Conference of the Comparative and International Education Society, Francis Marion Hotel, Charleston, South Carolina, 2009 . 69 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 70 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 71 Interview. Monrovia, Liberia. 27 July, 2010. 72 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008.

0

2

4

6

8

10

12

14

2007 2008 2009 2010

% of nationalBudget


programme, which starts at age eight. Additionally, the census shows a substantially older overage population

(see Chart 2). The programme was not designed for this breadth of learners.73

The upcoming ALP evaluation in 2011 will examine why such a significant number of children are enrolled in

ALP. Younger children may enrol to avoid or prepare for taking primary-school entrance exams, but reasons

for older students are less clear. An evaluation of USAID‘s similar ALP+ programme, which also includes

vocational training, highlighted the programme‘s shift from serving those whose school has been interrupted to

serving as an expansion of, or alternative to, formal education.74

This is a potentiality for the MOE

programme.

The efficacy of the ALP programme is difficult to determine as data is available on enrolment and

matriculation, but not for student retention. No government systems are in place to track ALP students‘ entry

and progress in the primary or secondary education systems. This is a global challenge regarding ALP

programmes.75

The 2007 MOE/UNICEF ALP Assessment highlights this and cites informants who note that

ALP students who passed the National Primary School Certificate Exam (NPSCE) were even more likely to go

to grade 7, were in grade 12 or had reached university.76

Also, the programme is completely supported by donor funds – which raises issues of sustainability.77

Support from the MoE has been uncertain; the previous administration favoured phase-out, but the current

73 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 74 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation. USAID. 2008. 75 Johannessen, Eva Marion, A study of former TEP students in Angola and Burundi, NRC, 2005. 76 Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia implemented by UNICEF. 2007 77 UNICEF, EEPCT Comments on Draft Report, 2010.

Chart 2: Distribution of Liberian Students in ALP by Age

Chart 1: Distribution of Liberian Students by Age


administration would like to maintain the programme. Responsibility for UNICEF-supported ALP schools was

due to be transferred to the MoE in 2010 as an EPF initiative. UNICEF will continue to provide technical

support such as joint monitoring visits, facilitating refresher training workshops and the planned ALP

evaluation in 2011.78

LAB4LAB

The LAB4LAB Liberia component is one part of a regional peace and development initiative that is one of the

innovations of the EEPCT programme, and was conceptually linked to the CFS initiative.79

A global

framework was developed for implementation of the LAB4LAB programme in 2008.80

In Liberia, LAB4LAB

schools were viewed as model CFSs. The two-year framework articulated the programmes implementation

into three components: design, construction and equipment; programming and outreach activities; and

capacity-building and development. UNICEF Headquarters has provided limited leadership in the

implementation of the LAB4LAB schools, aside from support for construction and conducting one regional

workshop.

The LAB4LAB School in Ganta, Nimba County has largely been completed. At the time of the evaluation

team‘s field work, registration for the impending school year was underway. However, the LAB4LAB School is

a pilot that cost more than US $1,000,00081

, not including operating costs. None of the contractors who

tendered for the contract had prior comparable construction experience. Nevertheless, lessons have been

learned and future UNICEF-built LAB4LABs will be scaled down for faster and cheaper construction.82

To assess the LAB4LAB School, the evaluation team conducted focus-group discussions and key informant

interviews and completed a school construction checklist. As with the other constructed schools, the team

applied the School Assessment Checklist that it had designed with CFS criteria. The school met 11 out of the

12 criteria (92%) on the checklist. This is notable as none of the six standardized government schools

constructed under the EPF that were visited met more than four of the 12 criteria (see section on EPF, above).

The only component lacking in the LAB4LAB School was an emergency preparedness plan, which has not

been a focus of the MoE or UNICEF in Liberia. The initial LAB4LAB design is being adapted for future

MoE/UNICEF construction and could serve as a source of advocacy for improved government school design.

However, based on key informant interviews, the roles that the MoE, UNICEF and the community will play in

sustaining the LAB4LAB School are unclear or not fully disseminated. Significant support will be required to

maintain the school, radio equipment, computers, programming and teachers. On a broader scale, the

programme has developed minimal regional connections. In 2009, the Côte d‘Ivoire Country Office hosted a

regional workshop on the implementation of LAB4LAB Schools and Talent Academies.83

However, this event

did not promote discussion on how the LAB4LAB programme would be organized regionally, how the peace

agenda would be developed, or the refinement of a long-term plan.84

Key informant interviews revealed that

there is minimal to no communication with LAB4LAB projects in Guinea or Côte d‘Ivoire.85

The regional impact of LAB4LAB is unrealized. No baselines or assessments were conducted to guide and

measure the programme‘s short and long-term outcomes, nor was the goal of peace operationalised in such a

way as to point to specific, measurable outcomes. As post-election tensions in Guinea again threaten the area

with potential refugees,86

the ability of the school to support peace in such a situation is unclear.

78 UNICEF, EEPCT Comments on Draft Report, 2010. 79 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs, 2010. 80 UNICEF, [Global Framework of LAB4LAB and Talent Academies –Draft], New York, 2008 81 UNICEF. Self Assessment. Monrovia. 2010. 82 Interview. Liberia. 1 August, 2010. 83 UNICEF Report of the workshop on the Project “Learning along borders for living across boundaries” and “Talent Academies”. Abidjan. 2009. 84UNICEF Report of the workshop on the Project “Learning along borders for living across boundaries” and “Talent Academies”. Abidjan. 2009. 85 Interview. Liberia. 7 August, 2009. 86 Aghailas, Tamba, Guineas’ post-election unrest threatens fragile peace in Liberia, Voice of Liberia, 2010. [Accessed 17 November, 2010 from http://www.voiceofliberia.org/guineaelectioncrisis.htm]

http://www.voiceofliberia.org/guineaelectioncrisis.htm


Talent Academies

An assessment for the Talent Academies in 2009 included possible strategies for implementation. As noted by

the consultant undertaking the assessment, this was ‗virgin territory‘ for UNICEF.87

Despite a positive

assessment, the UNICEF country office decided not to implement the Talent Academies as the programme

did not fit within their existing BEGE plan.88

The office also concluded that it did not have the capacity to

implement an additional innovative programme; limited technical support was provided by HQ to guide this

activity.

3.4 Goal Three: Increased education sector contributions to better Prediction,

Prevention and Preparedness for emergencies due to natural disaster and conflict

The office does not implement any programmes which contribute to Goal Three. However, it did provide

information for two of the four indicators under Goal 3 in the Results Framework. More specifically, the office

cited that a few activities focused on peace, including an unnamed UN programme for youth and its own

LAB4LAB programme. For the final indicator concerning an increase in international educational assistance,

Liberia had qualified for the $40 million grant from the EFA/FTI Catalytic Fund; and donors such as USSAID

and EU, had committed themselves to extending their programmes.

The two indicators that were not addressed were the adoption of a Disaster Risk Reduction Strategy and

Education for Conflict Prevention. Although the office positioned the LAB4LAB programme under Goal 2,

neither this nor the UN youth employment programmes were cited as strategies to address potential threats to

peace and stability.

3.5 Goal Four: Evidence-based policies, efficient operational strategies and fit-for-

purpose financing instruments for education in emergencies and post-crises.

LCO provided information on all seven of the indicators under Goal 4. Regarding the first two indicators,

coordination increased within the period of EEPCT implementation with the continuation of the Education

Development Partners Group (EDPG) and formation of the Education Cluster. The EDPG was central to

formulation and implementation of the LERP and the ESP as well as the successful application to the IMF

Catalytic Fund. EEPCT can claim some credit for this due to the formation and on-going support of the pooled

fund, the EPF. The second indicator of having a credible education sector plan was fulfilled through the

coordination highlighted above. EEPCT support was vital to Liberia‘s achievement of the third indicator – to

increase the number of countries using EMIS and systematic data collection annually. Since 2007, EEPCT

has supported National School Censuses annually and School Mapping since 2009.

The last two indicators relate to the existence of a pooled fund and its continuing support. UNICEF Liberia is

the only Country Office that has initiated a pooled fund with EEPCT funding. Regarding the indicator of

increased international funding for education, the EPF has not attracted any new donors other than the EC.

However, Liberia has qualified for a $40 million grant from the EFA/FTI Catalytic Fund. Aside from this,

USAID and EU extended their programmes past 2010.89

Assessments, baselines and evaluations are not consistently applied across EEPCT programmes. This may

be partly due to the absence of a Monitoring and Evaluation Officer in the UNICEF Country Office for more

than eight months of EEPCT implementation. However, the lack of baselines precedes this, as no regional or

87 UNICEF. Talent Academy Report: assessment of the talents, potentials, & preferences of out-of-school young people in urban communities in Liberia. Monrovia. 2009. 88 UNICEF, UNICEF, EEPCT Comments on Draft Report, 2010. 89 UNICEF, [Results Framework Response], 2010.


local baselines were conducted for the LAB4LAB programme. In 2009, a process was begun to document the

EPF.90

At the same time, an audit of the EPF was conducted.91

The EEPCT programme also helped the MoE

to develop guidelines for the ALP.92

This uneven application of monitoring has affected the office‘s ability to measure the effect and impact of their

programmes and identify areas for improvement. Meanwhile, six focus groups at the national, sub-national,

and community levels were asked about their perceptions of change in monitoring and evaluation of the

education system. Half reported it to be the same as prior to the implementation of the EEPCT programme,

two ranked it as better, and one ranked it as much better (see Chart 4). This indicates that the participants

are aware of monitoring and evaluation and that there is room for its improvement.

The UNICEF Country Office drew upon comparable programmes to implement EEPCT. Global and regional

workshops seem to be the best means of disseminating information about programmes. EEPCT helped key

staff members participate in many of these workshops, including the aforementioned LAB4LAB workshop in

Abidjan in 2009. In the case of the CFS programme, the government officials who attended the workshop

were motivated by its content and interested in bringing the concepts back to Liberia. By posting the ALP

Evaluation on the UNICEF website, the office sought to share its experience with other UNICEF offices and

implementing organizations. Document sharing can be an effective means of conveying information and

lessons learned.

EMIS

EEPCT‘s support of the MoE‘s Educational

Management Information System (EMIS) assisted

internal and external educational programming and

planning. As noted earlier, two National School

Censuses were completed for the 2007/2008 and

2008/2009 school years. The three focus-group

discussions at the national and regional level all cited

the importance of the census data. The improving

data collection processes contributed to the

Education Sector Plan which was approved for the

FTI/Catalytic Funding in 2010.

To support wider dissemination and use of the

statistics, a MoU between the MoE and the National Statistics Office was signed to link census data with the

Liberian Institute of Statistics and Geo-Information Services (LISGIS). This linkage will improve planning on

issues such as teacher deployment and distribution of teaching and learning materials.93

School mapping and

micro-planning will assist allocation of new schools and resources by the size of the school and the number of

children within its catchment area.94

Liberia Education Pooled Fund (EPF)

The EPF has already been described briefly (see section 1.4, above). It is a US $16.25 million dollar pooled

fund managed by the MoE and Ministry of Finance (MoF).95

The EPF emerged when Liberia was refused

90 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009 91 Ernst and Young and MGI MONBO & Company. Draft Audit report on the Project financial statements of the Education Pooled fund of the Ministry of Education pooled Fund of the Ministry of Education Liberian Primary Education Recovery Program (L-PERP). 2009 92 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 93 UNICEF, Self-assessment, 2010 94 UNICEF, Self-assessment, 2010. 95Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 28.

Chart 4

Change in Monitoring and Evaluation

No change

Better

MuchBetter


support from the EFA/FTI Catalytic Fund in 2007.96

As an alternative, the EPF was set up with EFA/FTI‘s

encouragement. In May 2007 UNICEF agreed to take the lead in the EPF‘s implementation.

The EPF was established using EEPCT funds between June 2007 and May 2008 in consultation with the

LCO, UNICEF HQ, the MoE, the MoF, Open Society Institute (OSI) and the World Bank.97

Due to the limited

capacity of the government, the fund is under dual management; the MoE is the lead organization and

oversees the planning and implementation of EPF activities, but fiscal management and procurement is

overseen by the MoF. 98

The partnership is intended to build the capacity of the MoE so that it can manage

funds directly in the future. The pooled fund began with US $12 million of support from EEPCT and US $4.25

million from OSI. In March 2010, UNICEF contributed an additional US$3.2 million to the fund, enabling OSI

to release its remaining US $750,000 and complete the foundation‘s US $5 million commitment.99

However, the concept of an education pooled fund pre-dates EEPCT and, unlike other initiatives such as

LAB4LAB and the Talent Academies, is driven and owned by the country office. The EPF is unique for several

reasons. From the start, it brought together a wide variety of education actors to inform the fund‘s

development and dispersal of funds.100

One key informant reported that an advantage of the EPF was that it

could move faster and was more flexible than other mechanisms.101

Another informant stated that the EPF

was the best forum for discussion of educational issues in Liberia.102

The fund has also been noted as an

example of public-private partnerships103

and gives the government access to multi-year funding.

The implementation of the EPF was challenging for the UNICEF Country Office as it did not have the in-house

capacity to establish and develop the Fund. UNICEF HQ provided multiple technical support visits, but a

temporary post might have been more useful in providing sustained support. UNICEF Liberia‘s 2007 Progress

Report stated that the ‗UNICEF Liberia education programme lacks enough technical staff to be able to

effectively support the larger macro-processes which have impact on the MDGs and at the same time pilot

viable solutions to inform policy at the national level.‘104

As stated earlier, the limited capacity of the MoE was also a challenge.105

The Country Office has stated that

this was ‗most clearly illustrated by the need for technical support for OSI to support the textbook procurement

and distribution'.106

Regarding school construction, the MoE did not ‗have the capacity to supervise school

construction projects of such magnitude‘.107

This lack of capacity was not limited to the MoE, as the MoF also

requires significant outside technical support to financially manage the fund.108

The EPF is viewed by some key informants as the most poorly managed of Liberia‘s different sectoral

funds.109

Unlike other funds, the EPF is co-managed by the MoF‘s Project Financial Management Unit

(PFMU), which is supported by the World Bank. This does not include a strategy for capacity building – which

is essential, given the high turnover of staff within the Ministry. Nonetheless, the presence of the EPF is

96 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 28. 97 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg vii. 98 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg. vii 99 Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education recovery. Open Society Institute. 2010. Pg 3 100 Interview, Liberia, 4 August 2010. 101 Interview, Liberia, 5 August 2010. 102 Interview, Liberia, 4 August, 2010. 103 Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education

recovery. Open Society Institute. 2010. 104UNICEF, UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation, Monrovia, 2007. 105 UNICEF, EEPCT Comments on Draft Report, 2010. 106 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 16 107 UNICEF, EEPCT Comments on Draft Report, 2010. 108 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 30. 109 Interview, Liberia. 5 August 2010.


believed to have contributed to the successful application to the Fast Track Initiative/Catalytic Fund.110

To

date, the EPF has dispersed three tranches of funds, as shown in Table 4.

The sustainability of the Fund is questionable. The 2010 plans will exhaust much of the remaining EPF

resources. Key informants stated that some donors cannot or have not chosen to contribute to the Fund due

to accountability and reporting issues. In 2009, the EU chose not to contribute; based upon an unshared

capacity assessment of the Fund. Rather a separate fund was created which received the newly awarded US

$40 million contribution from the FTI/Catalytic Fund.112

3.6 DAC Criteria

The analysis below was done by measuring stakeholders‘ perceptions of EEPCT‘s performance against the

OECD/DAC criteria. The charts in the following sections show the ratings given broken down by different

levels of stakeholder – community, sub-national and national.

Relevance/Appropriateness (average 4.5)

Relevance/Appropriateness was assessed through an

analysis of access to education, quality of education, and

girls‘ enrolment. Overall, the stakeholders reported

improvement in these areas (an average score of 4.5).

Chart 5 illustrates the stakeholders‘ perceived changes in

these areas since the beginning of EEPCT. There is no

statistically significant difference in mean rankings of the

different levels of stakeholders. These perceptions are

corroborated by Liberia‘s overall increase in Gross

Enrolment Ratio (GER).113

110 Interview, Liberia, 5 August 2010. 111 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. 112 Interview. Liberia. 5 August 2010. 113 UNICEF [Results Framework Response], 2010.

Education Pooled Fund Disbursements111

Date Tranche Supported Activities

2008

September $1,507,901

Operationalising three Rural Teacher Training Institutes (RTTIs)

including vehicles, rehabilitation and refurbishment of teachers

residences

November $5,510,829

Procurement of 1.2 million textbooks for English, Math, General

Science and Social Studies with teachers‘ manuals (for public

schools only)

2009 March $5,238,268 Construction of 40 six-room Primary Schools in 15 counties

Chart 5

Table 4

4.6667 4.6667 4.1667

0

1

2

3

4

5

CommunityEducators

Subnational National

Relevance


Overall, out of eight focus-group discussions conducted with children, four stated that that their feelings

towards school were much better while the remaining four ranked it as better. Parents were difficult to

assemble, but the one group of 12 organized ranked their confidence in the quality of education as much

better. Out of the eight groups with teachers, four ranked the education as much better, while the remaining

groups ranked it better.

Among the five groups with educators that discussed retention rates in school, four ranked the change in girls‘

retention as much better while one ranked it as better. Moreover, two out of five FGDs with girls and female

youth ranked their attendance as much better. These perceptions are confirmed by comparison of enrolment

rates from the 2006/2007 with those from the 2008/2009 National School Censuses.

Of five focus groups with teachers that ranked their ability to teach, four ranked this as much better while the

fifth ranked the change as better (overall mean rank 4.8). Teacher training and the ALP were two issues that

featured prominently in the discussions. Recommendations varied amongst the groups, with common themes

arising around the issues of teacher training and programme sustainability.

Effectiveness (average 3.75)

Key outcomes that were considered to evaluate the

effectiveness of the programme were the education

system‘s ability to recover from emergencies, government

preparedness and increased early-warning knowledge.

Overall, stakeholders reported improvement in effectiveness

(3.75). Chart 6 shows the changes in the stakeholders‘

perceptions. There was no statistically significant difference

between the mean rankings of three levels. However, the

programmes were viewed as more effective at the national

level. Of the six focus groups at the three levels, two

reported no change in government preparedness and early

warning, one reported it as better, and one as much better while one

didn‘t know.

As Liberia is still emerging from the conflict, the recovery of the education system is seen as a key means of

improving safety and security. The evaluation team found that perceptions of safety and security are a

concern on school premises. Of the five FGDs conducted with children that discussed safety and security

issues, three ranked their sense of security at school as the same or no change since the programme began

while one ranked it as better and one ranked it as much better. This varied from youth perception of the levels

of violence. Of the four youth FGDs, two gave no response while one ranked it as better and one ranked it as

much better.

Efficiency (average 4.4)

Stakeholders also reported improvement in efficiency

(with an average score of 4.4). Key outcomes are

considered under programme performance compared

to similar programmes and child safety in schools.

Chart 7 illustrates the assessment by the national, sub-

national, and community stakeholder groups of the

change in these criteria since the programme began.

4

4.8 4.6667

0

1

2

3

4

5

CommunityEducators


Efficiency

3.5 3.75 4

0

1

2

3

4

5

CommunityEducators


Effectiveness

Chart 7

Chart 6


Coherence and coordination (average 3.75)

Coherence and coordination were assessed through

implementation of the INEE Minimum Standards. In

all other case-study countries, the Education Cluster

support for the MoE coordination role was also

measured as a means to assess coherence and

coordination. However, the conflict and recovery in

Liberia pre-dates that of the Education Cluster, so

the question was not asked. Overall, stakeholders

reported improvement in coherence and

coordination (an average score reported 3.75). Chart

8 illustrates the participants‘ assessments of these

criteria. Of the three FGDs conducted with government

officials, two reported they didn‘t know when asked about the change in implementation of the Minimum

Standards of the Inter-Agency Network for Education in Emergencies (INEE), while one group didn‘t answer

this question.

Sustainability (average 3.8)

Sustainability was assessed based on the sector‘s

perceived ability to respond to another emergency,

its capacity to support education in emergencies

and monitoring and evaluation. Overall,

stakeholders reported improvement in sustainability

(with an average score of 3.8). Chart 9 illustrates the

focus group participants‘ assessment of these

criteria.

Of the six discussion groups conducted, three

ranked monitoring and evaluation of the education

system as the same or no change since the time the

programme began, while two groups ranked it as

better and one group ranked it as much better.

Summary

Across all levels and categories there was an overall average score of 4.4. The lowest overall average ranking

within the five categories was recorded for both the inter-related issues of effectiveness and

coherence/coordination (3.75). Relevance and appropriateness received the highest overall average rank

(4.8). Community level stakeholders‘ lowest scores were on both coherence/coordination and effectiveness

(3.5), while relevance received the highest rank (4.6). Sub-national stakeholders ranked efficiency highest

(4.8), and coherence/coordination and effectiveness lowest (3.75). At the national level, efficiency was ranked

highest (4.6), and sustainability was ranked lowest (3.66).

3.5 3.75 4

0

1

2

3

4

5

CommunityEducators


Coherence and Coordination

3.7 4.1

3.7

0

1

2

3

4

5

CommunityEducators


Sustainability and Connectedness

Chart 8

Chart 9


3.6 Focus Groups Discussion Outcomes

The focus-group discussion tool was used in at the

community level with children (ages 9-13), youth

(ages 14-18), educators and parents. Each group

was asked a set of questions which measured the

intended outcomes of the EEPCT programme at the

particular beneficiary level. Participants were asked

to provide a group ranking of 1 to 5, similar to the

assessment against DAC criteria described in the

section above.

Overall, community-level stakeholders ,including

children, youth, parents and educators, reported

improvements against the various education-related criteria since the EEPCT programme began (see Chart

10).

0 1 2 3 4 5

Feelings about school

Attendance

Girls' participation

Boys' participation

Sense of safety and security

Ability to protect self if another emergency

Amount students help each other

Students' involvement in problem solving

Use of fighting to resolve interpersonaldifferences

Students' Average Ranking of Change

All Students

Youth

Children

0

1

2

3

4

5

Overall Ranking of FGDs

Overallaverage

Chart 10

Chart 11


Children‘s outcomes were measured through change in the following criteria: feelings about school;

attendance; girls‘ participation; boys‘ participation; sense of safety and security; ability to protect themselves in

the event of another emergency; the amount that students help each other; students‘ involvement in problem

solving at school; and the use of fighting to resolve interpersonal differences.

The overall average of five groups of children was four out of five (referring to ‗better‘). The five focus-group

discussions with children were conducted at three different schools in two different counties. The mean

rankings for the three schools were 4.13, 3.93 and 3.88, which all reflected perceptions of improvement since

the programme began. There was no statistically significant difference between the mean rankings of the

children at the three schools (alpha = .05 and p-value = 0.733).

The overall average ranking of four groups of youth was 4.41. Like the groups of children, the four youth-

group discussions were conducted at three different schools in the two counties visited. The mean rankings of

youth at the three schools were 4.50, 4.44 and 4.29, which indicate perceptions of positive change since the

programme began. There is no statistically significant difference between the means of the three schools

(alpha = .05 and p-value=0.819). (See Chart 11)

Educator‘s outcomes were measured through change in the following criteria since the programme began:

ability to teach; provision of teaching and learning materials; quality of training; interaction between children of

different groups; reporting and monitoring procedures; quality of education at the school; retention rate for

girls; ability to respond to future emergencies; amount that students help each other; students‘ involvement in

problem solving; the use of fighting to resolve interpersonal differences; and feelings about school. With these

criteria in mind, the overall average ranking of the five groups of educators was 4.32 (referring to a ranking of

‗better‘). (See Chart 12)

Finally, parents‘ outcomes were measured through the following criteria: community involvement in school

emergency planning; the community‘s ability to address safety and abuse in the school; confidence in the

quality of education; community ownership of school construction; the amount that students help each other;

students‘ involvement in problem solving; the use of fighting to revolve interpersonal differences; and feelings

about school. As parents were largely unavailable during the working day, the evaluation team was able to

conduct only one focus-group discussion with parents. However, the average ranking of all these criteria was

positive (4.38).

0 1 2 3 4 5

Ability to teach

Quality of training

Reporting and monitoring procedures

Girls' retention rate

Amount students help each other

Use of fighting to resolve interpersonal…

Educators' Average Ranking of Change

Chart 12


3.7 Cross-cutting Issues

Rights-Based Approach

EEPCT has supported UNICEF‘s upstream work with the MoE to achieve the Right to Education and

principles set forth in the Universal Declaration of Human Rights. Liberia ratified the Convention on the Rights

of the Child (CRC) in 1993 and has submitted four progress reports to the Committee on the Rights of the

Child. With the entry of a new government in 2006, the Education Law of 2001, which abolished primary

school fees, was applied; also, school was made compulsory.114

However, obstacles still remain to its

enforcement, including protecting schools from charging additional fees and demanding birth certificates. The

ALP guidelines cite the Right to Education as a founding principle and include a component on Peace and

Human Rights. However, the evaluation was not able to confirm how this has been implemented. Overall,

children‘s participation is not strong within EEPCT. Key decisions such as textbook selection and distribution

did not involve children, and they are not involved in major EEPCT components such as the EPF.

Gender

Gender is a central component of UNICEF‘s

BEGE Programme, which includes EEPCT. As

part of this, UNICEF supports the Gender

Education Unit within the Ministry of Education. No

specific gender initiatives were undertaken under

EEPCT, but the EEPCT-supported programme did

provide data regarding the gender disparities

according to age, educational level and

geography.115

This data improved programme

planning and implementation.

ALP also has a significant gender component.

Girls often do not complete their education;

however, participation in ALP is presently near

gender parity. This is positive, but census data is required to understand its significance.

UNICEF staffs were aware of the gender disparity between male and female students and teachers. The

importance of female teachers was viewed as ‗debateable.‘116

This contradicts other sources that state that

their presence can affect the enrolment, attendance and retention of girls and female youth in school.117

This

lack of awareness could be attributed to a lack of a functioning UNGEI forum. UNICEF reported that a forum

had launched in January 2009, but that it was not functional due to capacity constraints in MoE‘s Girls‘

Education Unit.118

No documentation was seen regarding this, and the UNGEI website cites that no

partnership had been formed.119

Disaster Risk Reduction (DRR)

No information was provided on DRR programmes or emergency education programmes within the country.

The LCO chose not to implement any DRR programmes as part of the EEPCT programme. Conflict in

surrounding countries and the potential resurgence of conflict overshadows the risk of natural disasters in the

country. When asked about disaster preparedness, one community-level key informant said that children

114 UNICEF, EEPCT Comments on Draft Report, 2010. 115 Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009). Liberian Ministry of Education, 2010. 116 UNICEF, EEPCT Comments on Draft Report, 2010. 117 UNESCO, The Impact of Women Teachers on Girls Education, Paris, 2006. (Accessed on November 18, 2010 from Link: http://unesdoc.unesco.org/images/0014/001459/145990e.pdf 118 UNICEF, EEPCT Comments on Draft Report, 2010. 119 UNEGI, Liberia: background, 2010. (Accessed on October 2, 2010 from http://www.ungei.org/infobycountry/liberia.html)

Perceptions of Government Preparedness

No change

Better

Much better

Don't Know

Chart 3


should be taught to flee if war came again.120

Aside from conflict, Liberia faces minor threats such as floods

and storms. With this in mind, Liberia did not provide data for any of the Goal Three indicators in the Revised

Logframe. The majority of focus groups did not have knowledge of DRR activities or believed they had not

changed with the implementation of EEPCT. The weakness of the MoE‘s preparedness and response was

noted in the 2010 FTI/Catalytic Fund Approval.121

Sensitivity to Conflict and Fragility

EEPCT support has been largely been used to stabilize the education system. ‗Peace‘ programming exists in

a small way within the ALP programming.122

Peace is stated as a key component of the LAB4LAB

programme, but it is unclear how this aim is to be measured. As the programme lacks any clear indicators, it is

difficult to determine whether LAB4LAB‘s peace component is being achieved or how it will be connected to

other programmes in Liberia or within the region.

Accountability, Monitoring and Evaluation

The global instrument for monitoring the implementation of EEPCT

is the Revised Logframe (right). The UNICEF Country Office stated

that one of the weaknesses of the EEPCT programme was that it

had to create data retrospectively.123

It was able to provide data for

14 out of the 20 indicators; most of the information provided was

qualitative and typically did not cover more than one year. The office

presented information on the two EEPCT goals relating to the

change in the number of children both in and out of school. School

Census data was cited earlier; however, the number of children out

of school was not calculated.

The office‘s donor reports in 2007, 2008, and 2009 show an

increasing clarity on EEPCT aims and goals. However, EEPCT

remains unmentioned within the CO Annual Reports.

Monitoring of programme implementation occurred at different levels within EEPCT programmes. Monitoring

was most intense in programmes directly overseen by UNICEF, such as the onsite monitoring of the

construction of the LAB4LAB School. Most of the monitoring for programmes such as ALP and distribution of

learning materials was done by the MoE. Equally, the MoE was responsible for monitoring its own support of

the RTTIs, textbook distribution and school construction. The MoE has limited capacity to conduct this level of

monitoring and this in turn impairs the UNICEF Country Office‘s capacity to show the progress and quality of

programming being supported.

Due to the lack of assessments, no indicators were created nor baselines conducted for EEPCT programmes.

The PREV sought to establish retro-active baselines within the research methodology but found this

challenging as the programme is nationalized and it is difficult to disaggregate EEPCT and non-EEPCT

programmes.

Within the EEPCT timeframe, the single monitoring initiative was the documentation of the EPF.124

12 Interview. 1 August 2010. 121 World Bank. Appraisal of the 2010-2020 Education Sector Plan. 2010. 122 MOE. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia, 2008. 123 UNICEF, [Results Framework Response], 2010. 124 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009.

Goal LCO

Response

Total Number

of Indicators

Goal 1 4 5

Goal 2 1 4

Goal 3 2 4

Goal 4 7 7

14 20

C


3.8 Operational issues and management issues

Understanding of the programme cycle

The lack of baseline data was a challenge in gauging the progress of EEPCT. For example, no regional or

country assessments had been conducted for the LAB4LAB programme, hindering the development of

indicators and baselines. The Country Office illustrated the achievement of this objective by stating:

―completion of the [LAB4LAB] school has brought hope for more development in Nimba County and in Ganta

city. As the school was being built and when it was completed, some people thought it was a college or

university.‖ The positive spin offs, it was stated, might not be felt immediately, but other counties now also

want such model schools if money becomes available. At any one time there were more than 500 workers on

site at the school. These were mostly young former refugees whose job prospects were limited before school

construction began. The total landscape of the school‘s neighbourhood has changed since construction. Many

people have built houses close to the school so their children can attend it. When it did open its doors,

children could not stop washing their hands with soap at break time. However, while this qualitative

information is important in evaluating the programme‘s effectiveness, it also needs to be accompanied by

quantitative indicators and data.

Staffing and technical support

The Country Office is managing a large education programme with upstream and downstream components.

The BEGE programme is presently being implemented by two international and two national staff members.

This staffing level is not sufficient to oversee the direct implementation of programmes, the EPF and the

capacity building of the MoE. Review of EEPCT‘s expenditures shows a significant amount of monitoring of

the LAB4LAB School but no equivalent monitoring of the rehabilitation of the CEO/DEO complexes, ALP, and

projects implemented under the EPF including the government school construction or textbook distribution.125

Sustainability and Exit Strategy

The country office argued that the sustainability of EEPCT programmes is due to their efforts and not due to

the EEPCT global design as COs were not involved in its inception.126

The planning for sustainability was

hindered by the fact that the EEPCT funding arrived late during the initial year and is only distributed annually.

Certain programmes supported under EECPT do not have clear exit or continuation strategies. This requires

immediate consideration, as programmes such as the LAB4LAB and the ALP are scheduled to be transferred

to the MoE in 2010. While the ALP was not intended to be sustainable, the system of programme delivery is a

national resource that could be used for other education programme purposes.

The Country Office emphasized that LAB4LAB was a multi-country programme and therefore linked with other

countries with regard to sustainability. Sustaining of the planned regional LAB4LAB network has not been

costed. At the national level, however, the office will provide support for the first year of LAB4LAB operation to

determine what is required technically and financially to maintain the completed school. This information can

be used for later governmental budgeting.

EEPCT has provided support to the MoE to enhance the EMIS and create the EPF. While there are no plans

to phase out this support, no benchmarks have been identified to mark the transition from direct technical

assistance to capacity building.

125 UNICEF, Utilization of Donor Funds – EEPCT Liberia, 2010. 126 UNICEF. Self-assessment. Monrovia. 2010.


3.9 Partnerships

The MoE is the currently the Country Office‘s sole implementing partner for the EEPCT programme. UNICEF

also coordinates with the MoE‘s other bilateral and multilateral education development partners as well as

national and international NGOs in the implementation of the LPERP, the development of national

frameworks, the development of the ESP, and support of the EPF. These partners include UNESCO, UNHCR,

UNDP, WFP, UNMIL, USAID, EC/EU, WB, OSI, IRC, LET, SC/UK, OXFAM, FAWE, and ActionAID.

The focus on working with and through the MoE potentially leaves a gap in building capacity from the

community level, specifically through Parent-Teacher Associations. As an example, the ALP programme is

overseen at the PTA level.

4.0 THE WAY FORWARD

4.1 Lessons learned and conclusions

EEPCT has significantly improved the Liberia Country Office‘s educational response and has already had a

significant impact on the education of Liberia‘s children. The LCO has implemented nine programmes as part

of the EEPCT programme: educational materials, instructional support, infrastructure support, and technical

support, Child-Friendly Schools, ALP, LAB4LAB/Talent Academies, EMIS and EPF.

EEPCT progress was enhanced through the UNICEF Country Office‘s adherence to existing programme

plans.

The strength of EEPCT in Liberia is its integration, where possible, into plans pre-negotiated with the MoE.

The Country Office specifically stated that it was ‗not implementing EEPCT as a country programme but as a

BEGE supported by other donors as well.‘127

The office‘s adherence to these plans is illustrated through its

integration of the LAB4LAB initiative into the pre-existing plans for CFS and its choice not to implement the

Talent Academy because it did not fit within the existing plan.

Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative.

Throughout the evaluation, the Country Office emphasized that ‗EEPCT was never meant to be a Liberia

Country Office Programme‘ as it was not part of the 2008-2012 Liberia Country Programme. EEPCT was

viewed as an HQ-driven initiative with limited involvement from the Country or Regional Offices.128

This was exemplified by the lack of clarity in the development and dissemination of EEPCT documentation

and understanding of how the programme would be integrated into UNICEF planning structures. As an

example, EEPCT funding came after the approval of the 2008-2012 UNICEF Liberia Country Programme for

Children (CPC), which had been aligned with the MoE‘s LPERP and Education Sector Plan (ESP). The initial

focus on the regional LAB4LAB and Talent Academies Initiatives was additionally confusing as it was not clear

whether EEPCT could support programmes outside these two initiatives. The promotion of EEPCT was also

cited by the country office as lacking awareness of the realities of field implementation. COs are judged

according to the implementation of the core programmes of Child Survival and Development (CSD), BEGE

and Child Protection, which are derived from government plans.129

The UNICEF Country Office was able to attract donor support by combining standard UNICEF programming

with LAB4LAB innovation.

127 Email, 16 August 2010. 128 UNICEF, EEPCT Comments on Draft Report, Liberia, 2010. 129 Email, 16 August 2010.


Overview - Lessons learned and conclusions:

(1) EEPCT progress was enhanced through the Country Office’s adherence to existing programme plans.

(2) Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative. (3) The Country Office was able to attract donor support by combining standard UNICEF

programming with LAB4LAB innovation. (4) UNICEF and the Liberian government did not have sufficient staff and technical expertise to

implement the breadth and complexity of EEPCT programmes. (5) Programme transitions should be conducted gradual, with benchmarks to ensure relevance and

sustainability. (6) Monitoring of programmes does not draw upon communities and is poorly supported through

programme transitions. (7) Programmes lack components such as assessments, baselines and plans to link improvements

to policy change through advocacy. (8) Decentralization supports the implementation of other interventions. (9) Significant participation of children and communities was lacking within EEPCT programming. (10) Emergency Education needs a structured approach.

The Country Office‘s combination of the innovative LAB4LAB programme with UNICEF‘s standard CFS model

helped leverage support from a significant donor. In 2010, the Japanese government contributed US $8.5

million to support the ‗construction of six LAB4LAB schools, 24 new MoE standard primary schools and the

renovation/expansion of 60 others to increase schooling for 22,500 children.‘130

The LCO stated that part of

the attraction for the Japanese was the application of CFS principles in this innovative new way.131

School

construction was seen as an entry point to incorporate CFS principles into standard practice.

UNICEF and the Liberian government did not have sufficient staff and technical expertise to implement the

breadth and complexity of EEPCT programmes.

Capacity assessments of COs and implementing organizations should be conducted before programmes are

initiated. As an example, the Country Office stated that in implementing the EPF it ‗lacks enough technical

staff to be able to effectively support the larger macro-processes.‘132

The lack of a monitoring and evaluation

officer in the office has had an equal effect on programme implementation. UNICEF implementing partners,

such as the MoE, should be subject to equal scrutiny in this respect. The constant reference to the limited

capacity of the government suggests that there should have been more modest interventions with greater

monitoring and technical support, as was seen by the additional support that was provided in the textbook

distribution and the finalization of school renovation. As an example, OSI assisted the textbook distribution

and was due to conduct an evaluation of the process during 2010.

Programme transitions should be gradual, with benchmarks to ensure relevance and sustainability

130 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of

Affairs, 2010. Pg. 9. 131Interview, New York, 17 November, 2010. 132 UNICEF,UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation. Monrovia. 2007.


If programme responsibilities are to be sustainable, there is a need to work within the timeframe and

capacities of beneficiary communities and organizations. For example, the revitalization of the CEO (County

Education Officers) Complexes should have included a longer transition period. As it stands, it was assumed

that the government would supply fuel to CEOs for school monitoring, which it could not do. Equally,

emergency programmes, such as ALP, should have definitive points for review and adjustment. The Country

Office is hoping to begin developing best practices for programme transitions by conducting an ALP

evaluation and an assessment of out-of-school children in 2011. Results from these studies will inform the

MoE‘s decision regarding the ALP. With regard to the LAB4LAB initiative, UNICEF will support the first year

of the school‘s operation to determine its operating costs.

Monitoring of programmes does not draw upon communities and is poorly supported through programme

transitions.

Programme monitoring is the responsibility of all parties involved in programme implementation, including

children. Programme planning should include sufficient long-term resources to develop a culture of monitoring

and evaluation internally but also within the MoE and communities which they serve. Children and

communities monitored textbook distribution and maintenance through the implementation of a textbook

policy; similarly, communities were mobilized to monitor ALP programmes in their communities. Building upon

these examples, there should be further community involvement in school construction and supply distribution.

Communities should be involved in the selection of contractors for school construction and should be informed

about upcoming supply distributions via radio transmissions, while materials received should be signed for by

a student, parent and the school principal.

Programmes lack components such as assessments, baselines and strategies to link improvements to policy

change through advocacy.

Few of the EEPCT-funded programmes were based on situational assessments and baselines; this will hinder

evaluation of these programmes and advocacy for change. For example, the lack of a situational assessment

and baseline for the LAB4LAB/CFS programme will hinder evaluation and thus the communication of the

programme‘s benefits to communities and government. Advocacy strategies also need to be included in the

development of targeted products. For example, while a documentation exercise for the EPF was undertaken,

the target audience for this product was unclear.

Decentralization supports the implementation of other interventions.

The EEPCT‘s capacity building of the CEO Centres was not envisioned as a programme and was described

by the Country Offices as part of their government capacity building initiatives. However, the evaluation team

would identify it as supporting the resilience of the education system. This support is in line with MoE‘s plans

for decentralisation, which it hopes will increase service efficiency, transparency, accountability and

responsiveness.133

EEPCT‘s support for the rehabilitation of CEO complexes and provision of motorbikes was

an initial step complemented by training, data collection and production of the National School Census. The

training of 92 DEOs and 14 CEOs to perform the census built MoE capacity and created the opportunity for

links between the community level (schools), the county level (CEOs and DEOs) and the National MoE.134

This capacity is important because it improves monitoring and evaluation at the community level, including

more efficient and timely data collection and improved programme management.

Significant participation of children and communities was lacking within EEPCT programming.

133 World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group. [ Accessed on September 18, 2010 at http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html] 134 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010.


A significant portion of EEPCT work was ‗upstream‘ in supporting the MoE. This strong upstream approach

did not bring communities and children into the decision-making and dialogue. By incorporating children‘s

voices into programme design and implementation, the efficiency and quality of EEPCT programming could

be enhanced. Children have valuable insights and energy to contribute in post-crisis situations. At both the

community and national level, children can help advocate change on issues such as gender-based violence in

schools. This should be an intentional area of focus in moving forward with CFS schools.

Emergency Education needs a structured approach that takes into account natural disaster and conflict.

With the on-going tensions in Guinea, UNICEF and the MoE would benefit from a more structured approach to

the implementation of emergency education responses. The Country Office chose not to implement any

Disaster Risk Reduction programmes as part of the EEPCT programme. Conflict in surrounding countries and

the potential resurgence of conflict overshadows the risk of natural disasters in the country. When asked

about disaster preparedness, one community-level key informant expressed that children should be taught to

flee if war came again.135

As mentioned above, Liberia also faces minor threats of natural disasters such as

floods and storms.136

4.2 Recommendations

Government/MoE:

RECOMMENDATION: Evaluate the relevance the ALP programme. This can be supported by UNICEF‘s

ALP evaluation and the UNESCO/UNICEF assessment of out-of-school children.

RECOMMENDATION: Establish an emergency education/preparedness section in collaboration with

the IASC Education Cluster Leads (UNICEF and Save the Children) based upon the FTI’s

recommendation.

RECOMMENDATION: Remain engaged with the implementation of CFS to identify best practices to

adopt as national policy.

RECOMMENDATION: Establish, and identify resources for, a monitoring plan for CEOs.

UNICEF Headquarters:

RECOMMENDATION: Lead the development of programme learning workshops around specific

emergency education programme components, such as ALP. The results of these workshops could then

be developed into a programme plan to make promising practices into best practices through thorough

assessment, baselines and evaluations.

RECOMMENDATION: Reinforce the application of the programme cycle at the global, regional and

country level through inclusion in workshops, review of proposals and newly developed programme

documents. This should emphasize the importance of tying evaluations to advocacy as well as training on

how to develop advocacy strategies.

RECOMMENDATION: UNICEF HQ should lead a lessons-learned review of the EPF to identify when

such a strategy should be applied, but also the implications for the CO regarding staffing and

technical support. If Pooled Funds become a common intervention, UNICEF should develop in-house

capacity to provide technical support.

135 Interview. Liberia. 1 August 2010. 136Preventionweb. Liberia profile (Accessed September 21, 2010 from (http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr) 2010.


RECOMMENDATION: Adapt the CFS model to be used in emergencies.

RECOMMENDATION: Identify resources to continue the LAB4LAB and Talent Academy programmes

to an established transition point. Both of these initiatives are high level, regional/global initiatives.

UNICEF is under some obligation to see them through to commonly identified transition points. This would

include updating the Global Framework of LAB4LAB and Talent Academies in collaboration with the Regional

Offices and COs. Such a process would include a sociological and political survey, identification of

interventions and indicators, and decisions regarding how LAB4LAB/Talent Academies will proceed as a

regional/global endeavour.

Regional Offices:

RECOMMENDATION: Identify and provide the support needed by EEPCT countries through existing

personnel, short-term contracts or consultancies. Participate actively in the development of long-term

LAB4LAB and Talent Academy planning.

RECOMMENDATION: Participate actively in the development of long-term LAB4LAB and Talent

Academy planning.

UNICEF Liberia Country Office:

RECOMMENDATION: Review opportunities for children’s participation in decision-making and

programme implementation. Specifically, children‘s involvement should be integrated into the expansion of

the CFS programme.

RECOMMENDATION: Initiate long-term implementation and transition plans for the Ganta LAB4LAB

school. The Country Office needs to develop a plan for covering continuing costs based upon the first-year

operational costs (including electricity, computer maintenance, radio licence, etc.). New partners, such as

Search for Common Ground/Talking Drum to lead radio programming, should be identified as part of this

planning.

RECOMMENDATION: Review the implementation of the CFS programme to include situational

assessments as well as baseline and evaluation strategies that will provide data which can be used

for advocacy.

RECOMMENDATION: Review the staffing that is needed to ensure sufficient monitoring and transition

of programmes. Possible areas of focus include the clustering of activities around (1) ALP – conducting the

ALP Evaluation, Out-of-School Children Assessment and coordination with the government – and (2)

CFS/LAB4LAB programme – implementation of the LAB4LAB School, coordination of additional schools with

Japanese funding and the roll-out of the CFS programme. These positions are important for the continuation

for the programme after EEPCT.

RECOMMENDATION: Identify and address missing technical support that has resulted from the

absence of a monitoring and evaluation officer. Short-term consultants could be hired to support

assessments, baseline and the development/application of indicators.

RECOMMENDATION: Review UNICEF’s strategy to support the Ministry of Education to monitor

schools, distribution and data collection.

RECOMMENDATION: Develop a multi-year CFS advocacy strategy to accompany the 2008-2012

Education sector plans. The CFS programme should be a product of a solid assessment, and include

development of indicators and a programme baseline. Moreover, this should be coordinated with the

development of messages and products for targeted audiences.


RECOMMENDATION: Document the ALP programme, including programme structure, guidelines,

monitoring structures, and government involvement.

RECOMMENDATION: Plan a dissemination and advocacy strategy for the ALP Evaluation and the

UNICEF/UNESCO assessment. Messages should be targeted on communities, the Liberian government,

UNICEF and the international community.


REFERENCES

Coyne, Geoff, et al., Accelerated Learning Programme for Positive Living and United Service: Mid-term

Evaluation, USAID, 2008.

Collaborativeye. Decentralization : Collaboration, Resilience, and NO PROFIT. 2010. [Accessed on

September 18 from http://www.collaborativeye.com/collaboration_journal/decentralization-collaboration-

resilience-and-no-profit.html]

Davies, Lynne. Understanding education‘s role in fragility: synthesis of four situational analyses of education

and fragility – Afghanistan, Bosnia-Herzegovina, Cambodia, Liberia, University of Birmingham, 2010.

Department of State, Background Note on Liberia. U.S. government. 2010. [Accessed by on September 18,

2010 at http://www.state.gov/r/pa/ei/bgn/6618.htm]

Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009).

Liberian Ministry of Education, 2010.

Ernst and Young and MGI MONBO & Company, Draft Audit report on the Project financial statements of the

Education Pooled fund of the Ministry of Education pooled Fund of the Ministry of Education Liberian Primary

Education Recovery Program (L-PERP), 2009.

Fast Track Initiative. Recovery of the Education Sector in Liberia. 2010. [Access on September 18, 2010 from

http://www.educationfasttrack.org/newsroom/focus-on/fti-annual-report/recovery-of-the-primary-education-

sector-in-liberia/]

Government of Liberia/UNICEF Liberia. Education Programme 2007 Annual Work Plan. 2007.

Nkutu, Anne; Bang, Trude and Dorothy Tooman. Protecting children‘s right to education: evaluation of NRC‘s

Accelerated Education Program in Liberia. Nordic Consulting Group. 2009.

Johannessen, Eva Marion, A study of former TEP students in Angola and Burundi, NRC, 2005.

Kaabwe, Stella. Note for the Record – Subject: Liberia Education Pooled Fund Replenishment.

UNICEF: Monrovia. 2009.

Liberia Institute of Statistics and Geo-information Services. 2008 Population and housing census: final results.

Monrovia. 2009.

Liberia Institute of Statistics and Geo-information Services. Liberia Demographic and Health Survey 2007.

Monrovia. 2008.

MOE. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. 2008.

MOE, County and District Education Officers, Republic of Liberia, n.d.

MOE. Concept for the improvement and upgrading of the Education Management System (EMIS).

Department of Planning, Research and Development. Government of the Republic of Liberia. 2008.

MOE, Education Supplies for Montserrado County, n.d.

MOE, Education Supplies for Schools in Grand Gedeh, n.d.

MOE, Education Supplies for Nimba, n.d.

http://www.collaborativeye.com/collaboration_journal/decentralization-collaboration-resilience-and-no-profit.html



http://www.state.gov/r/pa/ei/bgn/6618.htm




MOE. Liberian Primary Education Recovery Program prepared for Fast Track Initiative. Government of the

Republic of Liberia. 2007.

MOE, Kakata, Margibi County ALP Teachers Listing, n.d.

MOE, List of MOE/UNICEF supported schools in Montserroda County, n.d.

MOE, List of MOE/UNICEF supported schools in Nimba County, n.d.

MOE. List of MOE/UNICEF supported schools in Sinoe County, n.d.

MOE, Listing of ALP Schools and teachers in Bomi County, n.d.

MOE, Listing of ALP Schools and teachers-Grand Bassa County, n.d.

MOE, Listing of ALP Teachers from Bong County, n.d.

MOE. Maryland County ALP Schools and teachers per district, n.d.

MOE. National Policy on Girl‘s Education-simplified version. Government of the Republic of Liberia. 2008.

MOE. MOU for the Liberia Education Pooled Fund Documents. Government of the Republic of Liberia. 2008.

MOE. National Policy on Girl‘s Education-simplified version. Republic of Liberia. 2008.

MOE, [School rehabilitation lists for UNOPS and LACE], n.d.

MOE, A system in transition: the 2007/08 National School Census Report. Government of the Republic of

Liberia. 2008.

MOE, [List of MOE/UNICEF Supported Schools and ALP teacher in] Yarnee District, Rivercess County

MOE and UNICEF. Liberian Enrolment Initiative: enhancing enrolment and completion of education for girls,

[n.d.]

Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia Implemented by

UNICEF. 2007.

Osher, David, et al. UNICEF Child Friendly Schools Programming: Global Evaluation Final Report. American

Institutes for Research: Washington, D.C. 2009.

Preventionweb. Liberia profile. [Accessed September 21, 2010 from

http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr]. 2010.

Republic of Liberia, Liberia‘s progress towards the Millennium Development Goals 2008, Monrovia. 2008.

Rodriquez, Luis, et al. Mid-term assessment of Liberia Teacher Training Program.

USAID: Washington, D.C. 2009.

Save the Children. Rewrite the Future: Improving access and quality of education for children affected by

conflict in Grand Gedeh and Bong County. 2010.

Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one

process. UNICEF: Monrovia. 2009.

http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr


Schmidt, Caroline and Aleesha Taylor. Liberia‘s Education Pooled Fund: a case for private foundation

engagement in post-conflict education recovery. Open Society Institute. 2010.

Scott, Aldophus, ‗Never too late to learn, 2008. [Access on September 21, 2010 from

http://www.unicef.org/infobycountry/liberia_42354.html]

UN. MDGinfo. [Accessed on September 21 at http://www.devinfo.info/mdginfo2010/]. 2010.

UNEGI, Liberia: background, 2010. [Accessed on October 2, 2010 from

http://www.ungei.org/infobycountry/liberia.html]

UNDP, Human Development Report. 2009. [Accessed online September 18, 2010 at

http://hdrstats.undp.org/en/countries/data_sheets/cty_ds_LBR.html]

UNDP. Human Development Report 2009 – Liberia. [Accessed online September 18, 2010 at

http://hdrstats.undp.org/en/countries/country_fact_sheets/cty_fs_LBR.html]

UNESCO, The Impact of Women Teachers on Girls Education, Paris, 2006. [Accessed on November 18,

2010 from Link: http://unesdoc.unesco.org/images/0014/001459/145990e.pdf]

UNESCO, Basic Learning Initiative, Accessed on January 10,

http://www.unesco.org/education/blm/chap1_en.php.

UNICEF. Child Friendly Schools Manual. New York. 2009.

UNICEF. Education in Emergencies: a resource took kit. UNICEF Regional Office for South Asia. 2006.

UNICEF, Education in Emergencies and Post-Crisis Transition: consolidated 2009 progress report to the

Government of the Netherlands and the European Union, 2010.

UNICEF. Education in Emergencies and Post-Conflict Transition: a synthesis of main finds from evaluations

from 2004-2009. Evaluation Office Working Paper. UNICEF: New York. 2009.

UNICEF, [Global Framework of LAB4LAB and Talent Academies –Draft], New York, 2008.

UNICEF. Netherland‘s support to education in post conflict transition in Liberia. UNICEF: Liberia. 2010.

UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the

Government of Japan, Ministry of Affairs, 2010.

UNICEF, Proposal on Education in Emergencies and Post-Crisis Transitions (presented to the Ministry of

Foreign Affairs, Government of the Netherlands), 2006.

UNICEF. Report of the workshop of the Project ―Learning along borders for living across boundaries‘ and

‗Talent academies‘.2009.

UNICEF, [Results Framework Response], 2010.

UNICEF. Self-assessment. Monrovia. 2010.

UNICEF. Talent Academy Report: assessment of the talents, potentials, & preferences of out-of-school young

people in urban communities in Liberia. Monrovia. 2009.

UNICEF, UNICEF Liberia: G3001 Netherlands, UNICEF Liberia Country Office: Monrovia, 2007.

http://www.unicef.org/infobycountry/liberia_42354.html

http://www.devinfo.info/mdginfo2010/

http://www.ungei.org/infobycountry/liberia.html

http://hdrstats.undp.org/en/countries/data_sheets/cty_ds_LBR.html

http://hdrstats.undp.org/en/countries/country_fact_sheets/cty_fs_LBR.html

http://unesdoc.unesco.org/images/0014/001459/145990e.pdf

http://www.unesco.org/education/blm/chap1_en.php


UNICEF, UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation.

Monrovia. 2007.

UNICEF. UNICEF Liberia: Annual Report 2008. Government of Liberia –UNICEF Programme of Cooperation.

Monrovia. 2008.

UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation.

Monrovia. 2010.

UNICEF, Utilization of Donor Funds –EEPCT Liberia. 2010.

Wayne, Trokon and von Hahmann, Gail. Achieving quality in accelerated learning: Student assessment in

Liberia: paper presented at the annual meeting of the 53rd Annual Conference of the Comparative and

International Education Society, Francis Marion Hotel, Charleston, South Carolina. 2009 .

World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group.

[Accessed on September 18, 2010 at

http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html]

World Bank, Appraisal of the 2010-2020 Education Sector Plan, 2010. [Accessed on September 18, 2010 at

http://www.educationfasttrack.org/media/library/Country_Documents/Liberia/ESP_Appraisal_Endorsement_R

eport.Final-Fe-.27-2010.pdf]

World Bank. Data. 2010 [Accessed September 24, 2010 from http://data.worldbank.org/country/liberia]

World Bank and Government of Liberia, Public Expenditure Management and Financial Accountability

Review, 2009.

http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html

http://www.educationfasttrack.org/media/library/Country_Documents/Liberia/ESP_Appraisal_Endorsement_Report.Final-Fe-.27-2010.pdf

http://www.educationfasttrack.org/media/library/Country_Documents/Liberia/ESP_Appraisal_Endorsement_Report.Final-Fe-.27-2010.pdf

http://data.worldbank.org/country/liberia


ANNEXES

Annex 1: Methodology

Purpose

The purpose of the PREV is to determine the outcome and indicative impact of EEPCT and draw on lessons

learned and recommendations that will be useful for strengthening the programme‘s on-going practices and

policies in the years to come. Within this context, the PREV seeks to achieve three inter-related objectives:

Provide an outcome-indicative impact analysis of the EEPCT Programme (2006-2009)

Examine DAC evaluation criteria as applied to education in emergencies, transitions and fragility

Provide evidence-based conclusions, lessons learned and recommendations.

Approach

At the country level, the PREV‘s focus was on programme implementation, results and learning as they relate

to beneficiary and education systems outcomes, with attention to the incorporation of standards, and the level

of quality and innovation involved in implementation. It examined how the goals were translated and adapted

to country contexts and why. In addition, the PREV sought to determine the extent to which global and

regional strategies and outputs are supporting change at the country level by documenting global and regional

inputs in the six case study countries.

The country case studies were designed to both fit into the larger global assessment of the EEPCT

programme and to act as stand-alone studies of EEPCT implementation and outcomes in different national

contexts. This report represents a detailed description and analysis of EEPCT‘s operations, impact, lessons

learned and ways forward in Liberia, while the global report will synthesize findings on country, regional and

global levels.

Evaluation Methodology

The evaluation employed a sequential mixed-methods approach, drawing on both quantitative and qualitative

methods, combining comprehensive coverage with in-depth analysis. This approach strengthens validity

through triangulation, as well as by using the results of one method to support the informed development of

others. These methodologies allow for in-depth exploration of methods and innovation as well as the collection

of comparable data across countries.

Cross Cutting Issues

The PREV also examined five cross-cutting issues from perspectives that would affect EEPCT‘s ability to

meet its goals. These include:

Gender: Are equity, equality, and empowerment taken into account in EEPCT programmes? Do

gender elements of programming include the needs of girls and women and boys and men?

A rights-based approach to programming: Is a rights-based approach a cornerstone for all

programming? Does EEPCT help increase participation and equitable inclusion?

Disaster Risk Reduction: Does EEPCT help minimize vulnerabilities and disaster risks throughout a

society? Does it take measures to avoid (prevention) or to limit (mitigation and preparedness) the

adverse impacts of hazards, within the broad context of sustainable development?


Sensitivity to conflict and fragility: Have conflict and fragility-sensitive approaches been implemented

when possible and can these be institutionalized?

Accountability, monitoring, evaluation and learning: To what degree have best practices in monitoring,

evaluation and knowledge transfer been adopted in implementing the programme?

Data Collection and Analysis

Six countries were identified for the field research component and were used as case studies for the PREV.

The research plan allowed for six to eight weeks per country. A GCA-Inter-Agency Network for Emergency

Education (INEE) Memo of Understanding also enabled close collaboration with education agencies and their

staff in each country where a case study was conducted.

The evaluation employed a sequential mixed-methods approach - that is, it drew on both quantitative and

qualitative methods - to combine more comprehensive coverage with in-depth analysis. The approach aimed

to: strengthen validity through triangulation; enable the use of results of one method to help develop the

instrument of the other; extend the comprehensiveness of the findings and generate new insights. The

research team employed participatory methods in its qualitative data collection with beneficiaries—children,

parents, educators and local communities members—to ensure an upstream flow of findings and

recommendations. Careful attention was paid to structuring these participatory approaches to ensure they

were transparent, systematic and replicable. Field visits also documented the extent to which global and

regional contributions (INEE Minimum Standards for Education in Emergencies, Chronic Crisis and Early

Reconstruction, Education Cluster coordination and Regional Office technical support) contributed to results at

the country level.

Data Collection Tools

Data sources included document reviews, interviews with key informants, surveys, field visits and FGD that

were conducted with children, youth, parents, and educators. The above information, collected in each

instance according to the programming priorities of EEPCT, was used to identify and link results in terms of

approaches, processes and outputs.

Primary and Secondary Literature Review. This body of literature ranged from government documents to

publications produced by local and international NGOs to academic literature. The PREV Team also reviewed

financial allocations relative to the four programme goals and the cost-effectiveness of EEPCT.

EEPCT Logframe This instrument is used for tracking EECPT indicators over time per its four goals. This

instrument is also intended to provide data to the achievement of EEPCT comprehensive aim and goal.

UNICEF Self-Assessments. As part of the global analysis, COs assessed their implementation of EEPCT by

describing the local context, the specific aims and goals that were established for EEPCT. Cos also assessed

the extent to which these aims and goals are being achieved and, importantly, the reasons that they are not.

Adequacy Survey: An adequacy survey was used to examine whether programmes launched at global,

regional and country levels completed what they set out to do in terms of the activities, trainings, material

development and services planned in order to achieve specific objectives or results. The adequacy survey

focused on the key tasks, inputs and outputs, proposed in the EEPCT plans, and sought to verify the extent to

which UNICEF and its implementing partners met their ―input and output‖ commitments.

Individual Interviews: One-on-one interviews were conducted with relevant UNICEF staff, government officials,

NGO staff and other key stakeholders at the community, regional and global levels. Identification of ―key

informants‖ was done in consultation with UNICEF at all levels. Interview questions were guided by the

information garnered from the document reviews and the issues highlighted in the PRES.


Key Stakeholders Consultations: Stakeholders were identified at the global, regional and country levels. In

case study countries, the PREV evaluation team worked with the UNICEF CO to identify key government (e.g.

ministries of education, planning, child development and gender, etc.), civil society (e.g. international NGOs,

national NGOs, academic advisors, etc.), donors (e.g. EC, ECHO, DFID, etc.) and others who were most

knowledgeable of the education sector in general and the EEPCT programme in particular.

Such FGD were conducted using the following tools:

Emergency Education - DAC Score Card (EE-DAC): The tool was used at different levels (national, sub-

national and community) within Liberia to quantifiably capture perceptions of the obtainment of EEPCT Goals

based around the five OECD/DAC criteria (Relevance/Appropriateness, Effectiveness, Efficiency, Coherence

and Coordination and Impact and Sustainability).

It ensured consistent data collection at different levels (national, sub-national and community) within Côte

d‘Ivoire. The interviewees used the score ranging from 1 to 5 for the assessment based on the DAC criteria,

with 1 indicating ―much worse‖, 5 indicating ―much better‖, and 3 indicating ―the same/no change‖. Following

the rating, the interviewee provided reasons positive (+) or negative (-) scoring. Their perceptions on

recommendations for strengthening EEPCT were also enlisted. The Score Card approach allows quantitative

findings to emerge as well as qualitative explanations regarding these findings.

Participative Ranking Methodology (PRM): PRM is a participatory approach to data collection in which a

group of knowledgeable participants are guided in responding to a specific question or set of questions. This

methodology promotes an engaged and participatory process that rapidly highlights key findings while

providing the opportunity for deeper analysis as resources permit. The PRM is particularly effective at

engaging children, youth, parents and educators ensuring inclusivity (male and female). . Collected in a

structured manner, responses were consolidated and used to inform this formative evaluation.

Focus Group Discussion (FGD): The focus group discussions provided the opportunity to gather responses on

changes of specific areas of implementation in both qualitative and quantitative manners. These tools were

used specifically at the community level with children, youth, parents and, where available, educators.

Child Friendly Schools Criteria Checklists: Child Friendly School (CFS) checklists were used to evaluate

schools fulfilment of selected CFS criteria. 137

The tool was developed by the global PREV team for broad use

in case study countries, and not solely intended for programmes implementing CFS, to assess how well

EEPCT-assisted schools complied with CFS criteria. In Liberia, the tool was used to evaluate both the

LAB4LAB School as well as EPF supported schools to gauge consistency of education construction.

Field Site Selection

Field visit sites in Liberia were selected through both random and purposive sampling processes. Based on

consultations with LCO and the national research team, Monterrado and Nimba Counties were chosen for

field site visits due to the fact that the majority of the country‘s students are enrolled in these two counties.

The LCO was consulted for clarity on the EEPCT programme components. When these were identified, the

locations of programme sites were determined. Because the textbook programme that was administered as

part of the LPERP and supported by the EPF was nationalized, comparison groups of public schools not

benefitting from the programme were not possible. As the ALP was implemented in all counties, but not all

schools, a list of all UNICEF-supported ALP programmes was obtained from the MOE. With these lists,

schools were randomly selected using a random number generator. Sites were then selected based on

137 UNICEF, Child Friendly Schools Manual. New York. 2009. Criteria included the following: These criteria included the following: whether or not the school appeared child friendly; if classrooms were easily exited in emergency; all classrooms have windows; separate spaces are provided for teachers and administrators; teachers and administrator location allows for monitoring students; availability of water on school grounds; separate latrines for boys and girls; locks on latrine doors; appropriate ratio of latrines to boy and girl students; separate latrines for teachers; separate space with soap and water for hand washing; and presence of a disaster risk reduction plan.


accessibility and distance. Due to time and accessibility constraints, the evaluation sought to visit programme

rich sites.

Sample Selection

The categories of stakeholders for the purpose of sampling were selected at three different levels as follows:

The National Level (Government Officials, MOE, and international and national NGO Staff)

The Sub National Level: (Government Officials, CEOs and DEO, and international and national NGO staff)

The Community Level (Students – separated by gender and age, teachers – mixed gender groups of

teachers/educators, and parents –separated by gender when possible). FGD participation can be viewed in

Table 3:

Focus Group Discussion Participation

FGD

Participants

Total %

National Sub-National Community

Government 3 6 18 0 24 13.9

INGOs & NGOs 2 4 3 0 7 4

Teachers 8 0 0 43 43 24.9

Students 11 0 0 81 81 46.8

Parents 2 0 0 18 18 10.4

Total 26 10 21 142 173 100

Data Analysis

Quantitative data analysis was performed on the adequacy survey and EE-DAC Score Card. Data was

entered in Excel by national researchers on a daily basis and analysed using SAS. The participatory ranking

exercise and FGDs all yielded data that was primarily quantitative.

Analysis of qualitative data collected from key stakeholder interviews, FGDs and PRM exercises drew on

direct quotes to exemplify themes found during data collection and was summarized in a memo format, which

looked for emergent categories and how they related to the broader research. Themes and commonalities

within and across categories were identified. The literature review and interviews were used to add depth and

explanation to the themes that emerged, as well as to develop and identify emergent categories.

Evaluation Team

The overall management of the CGCA Evaluation Team consisted of two evaluation co-leaders, a

biostatistician, finance administrator, and programme officer. In addition to overseeing and supporting the

work of evaluation teams in the six case study countries, the co-leaders and support staff were responsible for

implementation of global evaluation activities, as well as final data analysis, synthesis and report writing.

Table 3


The Liberia case study team consisted of two co-team leaders, a Columbia University Research Associate

and three national researchers. Regular communication between the case study and global teams was

maintained and weekly updates were provided to the CGCA management team to ensure that the evaluation

process ran smoothly and to address any questions or difficulties as they arose.

Limitations

The evaluation team took multiple steps to ensure the methods used in Liberia were consistent with those

utilized by research teams in the other case study countries. To achieve this, several materials and plans were

made to mirror approaches and actions in each country, including:

The development of an introductory PowerPoint to be used at stakeholder consultations delineating the goals,

purpose, design and tools used in evaluation;

The creation of user guides for each tool and instructions for all data collection methods;

A central training was conducted, followed by parallel trainings in each country for the full field

research teams;

A consistent approach developed to data entry, including creation of a singular entry form, and

supervision of research teams during the field work stage;

Bi-weekly calls between research teams and the evaluation leaders;

The lack of global clarity and common understanding on multiple terms within the sector limits a theory based

approach to the EEPCT programme itself and, by extension, this evaluation. This limitation includes the

meaning of terms within the programme‘s name (emergency, post-crisis, transition) and within its goals

(quality education, resilient education system). For these reasons, the CGCA created blogs that seek to

provide a platform to clarify these terms. Despite the development of definitions, terminology and concepts

within the tools continued to be difficult for FGD participants at the community level to comprehend due to

differences in education level, insufficient English language abilities or lack of similar words within vernacular

language.

LCO did not establish a programme baseline, which made reporting on percent change in goal area outcomes

impossible. The evaluation team addressed some of these concerns through the use of retrospective

baselines to determine programme outcome related findings. Retrospective baselines helped to facilitate

plausible outcome determinations in the absence proper baseline and monitoring procedures.

In addition to the above overall framework limitations, the following had an impact in conducting the evaluation

at field level in Liberia:

1. Timing of the evaluation: The evaluation coincided with school holidays which created challenges for

gathering participants from all cohorts for FGDs at the community level. This was mitigated by

contacting regional officials and community members prior to the evaluation visit. However, far fewer

FGDs were run with parents, teachers and regional educators than with youth.

2. Knowledge of EEPCT: Few key informants or focus group discussion members knew of EEPCT.

This hindered the application of the PREV tools.

3. Decentralization of information: While the MOE is UNICEF‘s only implementation partner at present,

UNICEF did not hold all information regarding programme sites, locations and participants. Therefore,

it was necessary to obtain this information directly from different relevant parties at the MOE which

slowed the planning process.

4. Availability of CEOs and DEOs: In the case of Nimba County, the County Education Officer (CEO)

and District Education Officers (DEOs) were not present at the time of the PREV team‘s mission due

a workshop in another county. While a point person was appointed by the CEO to assist the team in


coordination, the decentralization of information at the county and district levels limited the PREV

team‘s ability to contact schools for site visits.


Annex 2: Daily Schedule

Date Time Description Status Responsibility

Ongoing Literature Review Completed

6 July 9:30am Introductory meeting with UNICEF CO and

Education Team

Completed Evaluation team

7 July Training on EE-DAC Scorecard

Training on local calendar

Completed Natasha

8 July

9:00am

Interview 1 with Former MOE

Interview 2 with MOE



9 July

10:30am-

12:30pm

1:00pm-

2:30pm

3:30pm

Training on PRM

Meeting with DEOs and assistants from

Montserrado, Grand Gedeh and Nimba

counties

PRM exercise with test group of

neighborhood children


12 July

10:00am

2:00pm

4:00pm

Preparation for launch

Launch of PREV at MOE


Completed Evelyn, Natasha

Evelyn, Natasha

Evaluation team

13 July

10.00a.m.

12:30pm

2:00pm

4:00 pm

EE-DAC Scorecard with national government

personnel

Data entry

EE-DAC Scorecard with national level NGOs

EEPCT Programme overview with UNICEF


Evaluation team

Natasha

14 July

10:00am

11:00 am

2:00pm

4:00pm

4:30pm

Data entry

EE- DAC Scorecard with UNICEF

Interview with MOE

Interview with former MOE

Retrieve additional documents from UNICEF

Incomplete

Complete

Evaluation team

Natasha

15 July 10:00 am

11:00am

Field Work Montserrado County

Interview 5 with CEO, Montserrado County



2:00pm

EE-DAC Scorecard with Montserrado County

CEO and District Education Officers

Field Site 1: Montserrado County

School 1, Todee District

Field Test tools

School Construction Checklist – School 1,

Montserrado County

16 July

10:00am

2:00pm

Evening


School 1, Todee District,

Field test tools


School 2, Careysburg District,

Data entry

Completed

Evaluation team

19 July

9:30am

Evening

Data entry De Giuli

Sylvia De Guili (Evaluation Specialist, NYC)

arrives in Monrovia


20 July

10:00am-

3:00pm

9:00pm

Data entry

Silvia meets with Natasha re scheduling


21 July

9:30 a.m. PREV Liberia team meeting with Silvia at

UNICEF


23 July

9:30am

11:00am

Evening

Interview 6 with UN

Interview 7 with World Bank

Carl Triplehorn arrives

Completed

Completed

Evelyn, Natasha

24 July 8:00am Interview 8 with UNICEF Regional Rep

Meeting with PREV Liberia Team

Review revised schedule

Completed Carl, Natasha

Evaluation team

25 July

3:00pm –

5:00pm

5:00pm

Team meeting

Complete team training

Evaluation team

Evaluation team


26 July

11:00am

Liberian Independence Day

Meeting with Silvia re schedule

Literature review

Completed Carl, Natasha

27 July

9:00am -

6:00pm

Evening

Field work in Montserrado and Bomi

Counties

School construction checklist:

School 2, Montserrado County

School 3, Bomi County

School 4, Bomi County

Interview 9 with UNICEF

Meeting with Isabel Crowley

Meeting with Silvia re inception report

Completed

Completed

Completed

Completed

Evaluation team

Carl

Carl

Natasha, Carl

28 July

9:00am -

6:00pm

Evening

Interview 10 with LACE

Interview 11 with Architecture for Humanity

Completed

Completed

Evelyn, Carl

Carl

29 July

9:00am -

6:00pm

Evening

Field site 2: Montserrado County

School 2, Careysburg District

Interview 12 with Principal

PRM/FGD

Data entry


9:00am Interview 13 with Save the Children Completed Carl

30 July

9:00am -

6:00pm

Evening


School 3, Left Bank District


PRM/FGD

Data entry


10:00am

11:00am

Interview 15 with former MOE

Interview 16 with PFMU

Completed Carl and Evelyn

31 July 2:00pm Interview 17 with MOE Completed Carl, Natasha


7:00pm

Interview 18 with DEO, Nimba County

Nimba County Planning


1 August

9:00am

3:00pm

Evening

Depart for Nimba County

Field Site 4: Nimba County

Programme Focus: CFS

FGD – Teachers

Key informant interview 20 with LAB4LAB

school

Data entry

Completed Evaluation Team

Evening Preparation for field work Evaluation Team

2 August

8:30a.m.

10:00am.

Visit LAB4LAB School


School 4, Sanniquellie District

Interview 21 with Principals

Conduct PRM/FGD with:

Children (9 – 13 years) boys

Children (9 – 13 years) girls

Youth (14 – 18 years) girls

Youth (14 – 18 years) boys

Total 4FGDs

Data entry


Montserrado County NGOs FGD –EE-DAC Incomplete Evelyn

3 August

09.00a.m.

– 06.00pm


School 5, Bain-Gar District



Children (9 – 13 years) girls

Teachers –mixed

Parents - mixed



Evening


School 6, Baing-Gar District



Teachers –mixed

Total 4 FGDs

Data entry

1:00pm

EE-DAC FGD with Nimba County CEO and

DEOS – Gbanga, Bong County

Completed Carl

4 August

9:00am -

6:00pm

Evening


School 7, Saclepea District


PRM/FGD with:

Children (9-13) boys

Youth (14-18) girls

Parents - mixed

Teachers

Total 3 FDG

Data entry


9:30am

2:00pm


Interview 26 with EU representative

Interview 27 with USAID


Completed Carl, Evelyn

5 August

9:00am -

6:00pm


School 8, Zoe-Geh District


PRM/FGD with:

Children (9-13) - girls



Evening

Youth (14-18) - boys

Teachers - men

Total 3 FDG

Data entry

8-12:00

12 –

Evening

Logistics

Data entry

Complete Carl, Evelyn

6 August

9:00am

12:00noon

2:00pm

Evening

EE-DAC FGD - Nimba county NGOs




PRM/FGD with:

Teachers - mixed




PRM/FGD with teachers

Data entry


11:00am Debriefing at UNICEF Completed Carl. Evelyn

7 August

8:00am

1:00pm

4:00pm

Return to Monrovia

Visit Kakata RTTI, Bong County

Team meeting and debriefing

Completed

Evaluation Team

8 August 6:00am Depart Monrovia to Côte d‘Ivoire Completed Carl, Natasha


Annex 3. Interviews Conducted: National, Sub National and Community Levels

Liberia Key Informant Interviews

Level Number of Interviews

Regional 1

National 20

Sub-national 2

Community 5

Total 28


Annex 4: Liberia EEPCT Evaluation Framework

Liberia EEPCT Programme

Year Activity Location of

Implementation

Goal 1: Improved quality education response in emergencies and post-conflict countries

Educational

materials and

instructional

support

2007 Trained 250 primary school teachers to receive C level

certification

Magribi, Grand

Bassa, Bong,

Nimba

2008 Dispersed support through pooled fund to purchase 1.2

million in English, Math, General Science and Social

Studies with respective teachers' guides, Textbook Policy

and Instructional Material Management Handbook and

teacher training

All government

schools

2009 Supported printing of 125,000 consolidated of

supplementary readers for grades 2 and 3.

All government

schools

Infrastructure

2007 Renovated Montserrado County Education Office Montserrado

2008 Renovated 2 DEO complexes Montserrado,

Bomi

2008 Constructed 1 DEO/CO office complex Gbarpolu

2008 Constructed 2 warehouses Bomi,

Montserrado

2008 Purchased and distributed 20,000 arm chairs 10 Counties

2008 Dispersed support through the Education Pooled Fund to

support to revitalization of three Rural Teacher Training

Institutes (RTTIs) including logistical support, building and

furnishing of teachers' housing, vehicles, cookers,

residential furniture.

2009 Constructed CEO office complex and warehouse Grand Bassa

2009 Constructed generator house, wells, and water towers for

CEO offices

Monteserrdo,

Bong, Bomi,

Gbarplu

2009 Repaired and installed two 25 KVA generators Bomi, Gbarpolu

2009 Furnished CEO office complex Gbarpolu

2009 Dispersed support through the Education Pooled Fund to All counties


support construction of 40 schools

Technical

Support

2007 Hired consultant to provide technical assistance for

development of sector wide approach

National

2008 Hired consultant to provide technical assistance for the

LPERP implementation plans

National

2009 Assisted MOE to develop 10-year Education Sector Plan

(ESP)

National

2009 Hired coordinator for Education Development Partners

Meetings

National

2009 Hired consultant to support government to write School

Abolish Policy

National

Child Friendly

Schools

2009 Trained 55 Trainer of Trainers for the Child Friendly

School Programme

Gbarpolu

2010 Trained an estimated 500 teachers and principals in Child

Friendly School approach

National

Goal 2: Increased resilience of education sector services delivery in chronic crises, arrested development

and deteriorating contexts

Goal 3: Increased education contribution to better prediction and preparedness for emergencies due to

natural disasters and conflict

ALP 2007 Procured 140,000 learners kits for ALP and Grade 1 National

2007 Procured and distributed recreational kits for ALP

2007 Hired 2 logisticians for end-user monitoring of ALP supplies

2007 Trained 994 teachers in ALP refresher course Montserrado,

Bomi, Bong, Lofa,

Maryland, Nimba,

Grand Gedeh

2008 Supported XXXXX student in ALP programmes including

payment of teachers, learning materials and monitoring

11 Counties

2008 Procured and distributed 66,125 learners kits to ALP and

Grade 1 students

11 Counties

2008 Hired logisticians to monitoring to monitor distribution of

learners kits

2008 Provided 948 teachers initial and refresher training 11 Counties


2008 Provided 276,000 copies of 3 types of Grade 1 and

Readers and ALP Level 1 in kits

11 Counties

2008 Developed, printed and disseminated ALP Policy

Guidelines

National-level

2009 Expanded ALP to four new counties Grand Bassa,

Rivercess,

Margibi, Sinoe

2009 Funded printing and distribution of 10,000 ALP Policy

Guidelines

National-level

2009 Supported 30,785 ALP students ( paying teacher, providing

materials, and monitoring)

11 Counties

2009 Printed and distributed 450 sets of ALP Manuals for a

Levels 1, 2, 3

11 Counties

LAB4LAB 2008 Paid for national focal point for LAB4LAB Ganta, Nimba

County

2008 Obtained title deed for LAB4LAB Ganta, Nimba

County

2008 Supplied transportation (2008-2010) Ganta, Nimba

County

2008 Supported staff DSA(2008-2010) Ganta, Nimba

County

2008 Awarded construction contract for LAB4LAB Ganta, Nimba

County

2009 Started construction of LAB4LAB Ganta, Nimba

County

2010 Opened LAB4LAB Ganta, Nimba

County

Talent

Academies

2009 Conducted an assessment for Talent Academies look a report

Goal 4: Evidence-based policies, efficient operational strategies and fit-for-purpose financing instruments

for education in emergencies and post-crisis situations.

EMIS

2007 Supported EMIS Technical Manager to develop tools for the

collection of educational data

Country-wide

2007 Trained enumerators in revised education tools Country-wide

2007 Conducted census

2008 Completed, published and distributed 15,000 2007-8

National School Census

Country-wide


2008 Supported EMIS Technical Manager to continue EMIS

assistance

Country-wide

2008 Trained enumerators on educational tools Country-wide

2009 Funded 2008-2009 census Country-wide

2009 Provided technical and financial support for first phase in 3

counties of national school mapping and micro-planning

Gbarpolu, Bomi,

Grand Cape

Mount

Education

Pooled

Fund

2007 Supported the Deptuy Director from PARMO to provide

technical support

Country-wide

2008 Support the Deputy Director from PARMO to provide

technical support

Country-wide

2008 Established Liberia Education Pooled Fund Country-wide


Annex 5: Evaluation Tools

EE-DAC Score Card

The EE-DAC Score Card included in the Inception Report, is one of the tools to be used in the country case studies undertaken in the EEPCT evaluation. It is based on EEPCT Goals consisting of (1) Improved quality of education response in emergencies and post crisis transition, (2) Increased resilience of education service delivery, (3) Increased education sector contribution to better prediction, prevention and preparedness for emergencies caused by natural disaster and conflict and (4) Evidenced-based policies, efficient operational strategies and fit-for-purpose financing instruments which are reflected in the form of evaluation questions. The OECD / Development Assistance Criteria (DAC) is applied for the assessment of the achievement of such Objectives as defined below: 1. Relevance / Appropriateness

Whether project is in line with local needs and priorities (as well as donor policies). Appropriateness is the tailoring of humanitarian activities to local needs, increasing ownership, accountability and cost-effectiveness accordingly

2. Effectiveness The extent to which an activity achieves its purpose or whether this can be expected to happen on the basis of outcomes. Implicit within the criterion is timeliness

3. Efficiency The outcomes- qualitative and quantitative – as a result of the inputs. This generally requires comparing alternative approaches to achieving an output, to see whether the most efficient approach has been used

4. Coherence and Coordination The need to assess security, developmental, trade and military policies as well as humanitarian policies, to ensure that there is consistency and, in particular, that all policies take into account humanitarian and human-rights considerations

5. Sustainability / Connectedness

Connectedness refers to the need to ensure that activities of a short term emergency nature are carried out in a context that takes longer term and interconnected problems into account.

The tool is to be used at Focus Group Discussions (FGDs) with the different stakeholder categories to assess EEPCT based on key selected elements stated as evaluation questions in the format provided. The different stakeholder groups will analyze in their own respective groups and come up with a collective ranking using the scores 1 to 5, with 1 being the lowest and 5 being the highest score for the achievement of the specific objectives. A score of 8 can be assigned if respondents collectively agree they do not know the answer, or a 9 if they choose not to respond at all. Following the scoring, the interviewees / focus group participants would be asked the reasons for their assessment with a (+) or (-) indicative of positive or negative reasoning. This would be followed up by qualitative recommendations or comments for improvement of programme success / delivery as perceived by the stakeholder groups. The format appearing in the following page should be used for the assessment and the process steps are outlined below:

Identify the different stakeholder categories at the different levels as follows and organize 3 different meetings at each level lasting 1 hour per group.

i. National level – Policy / decisions makers on education: Government, UN and NGOs

ii. Sub National / District level – Education Supervisors-Implementers: Government, UN and NGOs

iii. Community level – Teacher-Programme staff (only) The above meetings will bring together Government, UN, and INGO-National NGOs and educators-

programme staff who would for their own groups, discuss the questions and agree on a collective ranking (1-5, 8 or 9 for each) and indicate the score by a ―‖ on the respective box in the format.


Facilitate discussions in the groups to probe into the causal factors / reasons (+ positive or - negative) for the ranking and to come up with recommendations for improvement to be followed up. Ask groups to site 3 reasons for their rankings. Write down answers exactly as stated (do not interpret or summarize comments).

The role of the facilitators (2 facilitators per meeting—1 to facilitate and 1 to support and record) is

considered very significant in moderating / focusing the discussions and maintaining objectivity. They should be prepared in advance as follows:

- Make it clear to the participants that this evaluation is a learning process so that it lends itself for capacity building and self-assessment.

- Be familiar with the detailed programme activities related to EEPCT so that the group facilitation could be done effectively.

- It may be helpful to display on flip chart papers the four Goals and sub components so that when the assessment is done the participants could refer to them or else could be distributed as handouts.

- Be familiar with the definitions of DAC to be able to explain to the participants.

- It would be useful to display the definitions of DAC for reference by the participants or else could be distributed as handout.

- Have the formats for each group prepared on large sheets of paper as working document which could be self documented by the participants.

- If time permit group presentations for plenary consensus may be done.

It is envisioned that this tool will be used for field studies and key informant interviews at the global and regional levels. In each group scenario, a participatory approach will be adopted using the EE-DAC Score Card as a framework to collect the necessary qualitative data based on stakeholder perceptions. The scoring component of the score card will yield quantitative results as well. The EE-DAC Score Card forms filled by the different groups should be later consolidated into a single format with the summary of reasons and recommendations for each level. The consolidated formats for each level in the different countries can be taken up for comparative analysis for assessing the status in the overall analysis. An excel data entry programme will be used to enter data. Data should be entered daily. The initial pilot should be shared with NY team to verify data is being consistently collected and coded. Final results will be sent to NY as soon as completed. Data analysis results will be returned to the field team within 2 weeks to be incorporated into country case studies. Data will also be examined across countries for the synthesis report.


Questions

Score Reasons for the Score

( + Positive / - Negative)

Key Recommendations /

Strategic Focus (based on gaps

identified) 1

Much Worse

2 Worse

3 Same

4 Better

5 Much Better

8 Don’t Know

9 No

Resp

Relevance/Appropriateness

1. Has access to education in emergencies or post crisis transition changed since the programme began?

-

2. Has the quality of education in emergencies or post crisis transition changed since the Programme began?

-

3. Has girls‘ enrolment in schools changed since the Programme began?

-

Effectiveness

4. Have education system‘s ability to recover from emergencies changed since the Programme began?

-

5. Has government preparedness and early warning knowledge changed since the Programme began?

-


Efficiency

6. How well does the Programme perform activities compared to other similar programmes?

-

7. Has child safety in schools changed since the Programme began?

-

Coherence and Coordination

8. Has implementation of INEE minimum standards changed since the Programme began?

-

9. Has education cluster support of the Ministry of Education coordination role changed since the Programme began?

-

Sustainability/Connectedness

10. Has education sector‘s ability to prepare for and respond to emergencies changed since the Programme began?

-

11. Has government capacity to support education in emergencies and-or post crisis transitions changed since the Programme began?

-


12. Has monitoring and evaluation of the education system changed since the Programme began

-


FGD DATA COLLECTION FORM (CHILDREN, YOUTH)

Question: What are the strengths of the programme?

Date: Group: Children / Youth

Community: Gender: Girls / Boys

Moderator: Number of Children in Group:

Note taker: Age Range:

Key Strengths Identified:

Free list: Rank Order:

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:

(Write down what the children say using their exact words.)


FGD DATA COLLECTION FORM (CHILDREN, YOUTH)

Question: What are the weaknesses of the programme?

Date: Group: Children / Youth




Key Weaknesses Identified:


1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:



FGD DATA COLLECTION FORM (PARENTS, EDUCATORS)

Question: What are the strengths of the programme?

Date: Group: Parents / Educators

Community: Gender: Men / Women / Mixed

Moderator: Number of Participants:

Note taker:

Key Strengths Identified:


1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:

(Write down what the participants say using their exact words.)



Question: What are the weaknesses of the programme?




Note taker:

Key Weaknesses Identified:


1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:




Question: What makes an education system resilient?




Note taker:

Key Qualities Identified:


1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:



FGD DATA COLLECTION FORM (YOUTH)

Question: What makes an education system resilient?

Date: Group: Youth




Key Qualities Identified:


1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

COMMENTS:

(Write down what the youth say using their exact words.)


FGD DATA COLLECTION FORM (CHILDREN)

Date: Group: Children




Note establish a common time reference for both programme and control groups which could be “since the emergency” or another

commonly shared event.

Much

worse

Worse No

change

Better Much

Better

Don‘t

know

N/A-

NR

For Children 1 2 3 4 5 8 9

How much have your feelings about school changed since the

programme began, if at all?

How much has your attendance changed, if at all?

How much has girls‘ participation changed, if at all?

How much has boys‘ participation changed, if at all?

How much has your sense of safety and security changed, if at all?

How much has your ability to protect yourself if there is another

emergency changed, if at all?


How much has the amount students help each other changed, if at all?

How much has students‘ involvement in problem solving at school

changed, if at all?

How much has the use of fighting to resolve interpersonal differences

changed, if at all?

COMMENTS:



FGD DATA COLLECTION FORM (YOUTH)

Date: Group: Youth


Moderator: Number of Youth in Group:




Much

worse

Worse No

change

Better Much

Better

Don‘t

know

N/A-

NR

Youth 1 2 3 4 5 8 9

How much have your feelings about school changed since the

programme began, if at all?

How much has your attendance changed, if at all?

How much has girls‘ participation changed, if at all?

How much has boys‘ participation changed, if at all?

How much has your sense of safety and security changed, if at all?

How much has your ability to protect yourself if there is another

emergency changed, if at all?



How much has students‘ involvement in problem solving at school

changed, if at all?

How much has the use of fighting to resolve interpersonal differences

changed, if at all?

COMMENTS:

(Write down what the youth say using their exact words.)


FGD DATA COLLECTION FORM (EDUCATORS)

Date: Group: Educators



Note taker:



Much

worse

Worse No

change

Better Much

better

Don‘t

know

N/A-

NR

Educators 1 2 3 4 5 8 9

How much has your ability to teach changed, if at all?

How has the provision of teaching and learning materials changed, if at all?

How has the quality of the training changed, if at all?

How has the interaction between children and youth of different groups changed, if at all?

How have reporting and monitoring procedures changed, if at all?

How has the quality of education at the school changed, if at all?

How has the retention rate for girls changed, if at all?

How has the usefulness of the revised Teaching Kits changed, if at all?


How has the usefulness of the revised School in a Box changed, if at all?

How has the usefulness of the revised Early Childhood Kits changed, if at all?

How has the school‘s ability to respond to future emergencies changed, if at all?


How much has students‘ involvement in problem solving at school changed, if at all?

How much has the use of fighting to resolve interpersonal differences changed, if at all?

How much have your feelings about school changed since the programme began, if at all?

COMMENTS:

(Write down what the participants say using their exact words


FGD DATA COLLECTION FORM

(PARENTS/SCHOOL MANAGEMENT COMMITTEES/ PARENT TEACHERS ASSOCIATIONS)

Date: Group:

School Management Committees/Parent Teacher

Associations


Moderator: Number of Participants in Group:

Note taker:



Much

worse

Worse No

change

Better Much

Better

Don‘t

know

N/A-

NR

Parents/ School Management Committees/Parent Teachers Associations 1 2 3 4 5 8 9

How has community involvement in school emergency planning changed, if at all?

How has the community‘s ability to address safety and abuse in the school changed, if at

all?

How your confidence in the quality of education at the school changed, if at all?

How has community ownership over school construction changed, if at all?


How much has students‘ involvement in problem solving at school changed, if at all?


How much has the use of fighting to resolve interpersonal differences changed, if at all?

How much have your feelings about school changed since the programme began, if at

all?

COMMENTS:



Adequacy Survey Checklist

These questions should be completed in each country. The research team may use various sources to

answer them, should confirm answers with programme staff whenever possible, and should cite sources

of information when possible.

Programme Design

1. Before the programme began, was there a situational assessment?

2. Was there a baseline assessment?

3. Was gender considered when planning and implementing programmes? How so?

4. Are gender-related indicators included in the monitoring plan?

5. Were programme indicators and evaluations structured to look only at outputs or also impact?

6. Does the programme have a plan for discontinuation, phase-out or handover?

Programme Implementation

1. Did replenishment kits arrive within programme established time frames? 2. Did educators receive a minimum of quarterly training? 3. Can youth enrolled in ALP programmes pass a basic literacy and numeracy test? 4. Can children/schools/educational institutions demonstrate knowledge of an emergency response

plan for their school or educational institutions? 5. Does the country have radio programming and/or other remote educational programmes? 6. Does the country demonstrate a direct contribution to the EFA/MDG indicators? 7. Has technical support has been provided?

a. What?

b. By whom? - HQ:

- RO:

- Other:

Programme Learning

1. Has the government adopted the CFS model as a government programme?

2. Were good practice reports/trainings issued to partners at the country level?

3. For each programme implemented in this country, have evaluations been done? List dates of

evaluations for each programme.

4. Were the results of the evaluation shared? How? With whom?

5. Have the results and recommendations of the evaluations been integrated into programming?

How so?

6. How is programme progress and learning from the field level shared with regional and country

level? Is this sharing useful and productive?

7. Do donor or government initiatives reference EEPCT?


Financial

1. Is the EEPCT programme delineated in financial and/or programme documents?

2. Has there been an increase in government financial support for EEPCT programmes?

3. Are donor funds transferred to the field office as per project-established time frames?

4. Are CAF funds transferred to field offices per project-established time frames?

5. How much of the 2009 allocation of funds was spent (allocation v. expenditures)?

6. Has EEPCT support reached an appropriate number of beneficiaries, given programme costs

(needs vs. coverage)?

Best Practices/Standards

1. Do partner agreements reference INEE minimum standards?

2. Do peace education programmes use UNHCR/INEE/UNESCO programme materials?

3. Has the country achieved compliance with the Minimum Operating Security Standards?

4. Do government preparedness plans incorporate UNICEF methods and approaches?

5. Was a SWOT analysis done in country?

a. If so, did the programme address at least one SWOT recommendation?


