Formative Evaluation f UNICEF supported CBSR Programme Lao PDR · 4 Evaluation Findings 18 4.1 Evaluation Question 1 18 4.1.1 Relevance 18 4.1.2 Effectiveness 20 4.1.3 Efficiency

Formative Evaluationof UNICEF supportedCBSR ProgrammeLao PDR

Mott MacDonald tradingas Cambridge Education

Formative Evaluation ofUNICEF-supportedCommunity Based SchoolReadiness ProgrammeFinal Draft Report

December 2019

356392AT52 2 1

Cambridge Education

Cambridge Education22 Station RoadCambridge CB1 2JDUnited Kingdom

T +44 (0)1223 463500F +44 (0)1223 461007camb-ed.com

Formative Evaluation ofUNICEF-supportedCommunity Based SchoolReadiness ProgrammeLao PDR, 2013 - 2018Final Draft Report

December 2019

Mott MacDonald Limited trading asCambridge Education. Registered inEngland and Wales no. 1243967.Registered office: Mott MacDonald House,8-10 Sydenham Road, Croydon CR0 2EE,United Kingdom

Cambridge Education | Lao PDR, 2013 - 2018Final Draft Report

356392AT52 | 2 | 1 | December 2019

Issue and Revision Record

Revision Date Originator Checker Approver Description1 16 Aug

2019Kate Martin Ray Harris Rachel

PhillipsonDraft version 1

2 20 Sep2019

Kate Martin Ray Harris RachelPhillipson

Draft version 2 – responding tofeedback from validationworkshop in Vientiane (23-Aug)and comments from UNICEF LaoPDR office

3 12 Dec2019

Kate Martin Ray Harris RachelPhillipson

Final draft version – respondingto feedback from UNICEF andfollow up call with UNICEF team

Document reference: 356392AT52 | 2 | 1

Information class: Standard

This document is issued for the party which commissioned it and for specific purposes connected with the above-captioned project only. It should not be relied upon by any other party or used for any other purpose.

We accept no responsibility for the consequences of this document being relied upon by any other party, or beingused for any other purpose, or containing any error or omission which is due to an error or omission in data suppliedto us by other parties.

This document contains confidential information and proprietary intellectual property. It should not be shown to otherparties without consent from us and from the party which commissioned it.

This Re por t has be en p rep are d solely for use by t he p arty w hich c om mission ed it (the 'Client') i n co nnecti on wit h the cap tione d p roject . It s hould not be used for any oth er p urp ose. N o p erso n ot her tha n th e Client or any party who has expr essly a gre ed t er ms of relia nce wit h us (the 'Recipie nt(s )') m ay r ely on the cont ent, info rma tion or any view s exp ress ed in the R epo rt. This R epo rt is co nfide ntial and c ont ains p rop riet ary in tellect ual p rop erty and we ac cept no duty of ca re, resp onsibility or li ability t o any oth er recipi ent o f this R epo rt. N o re pre sent ation , wa rran ty o r un dert aking , exp ress or i mplie d, is made an d no res ponsi bility or liability is acce pted by us to any p arty oth er t han the Cli ent or a ny Reci pient (s), as t o the accu racy or c om plete ness of th e info rm ation cont aine d in t his Rep ort. Fo r t he av oida nce o f do ubt t his Re port do es no t in any way pu rpo rt to includ e a ny leg al, ins ura nce or fin ancial advic e or opini on.


356392AT52 | 2 | 1 | December 2019

Cambridge Education was contracted in May 2019 by UNICEF Lao PDR through the Long-TermAssistance (LTA) Framework to design and conduct a formative evaluation of the UNICEF-supported Community-Based School Readiness (CBSR) Programme, a joint initiative of the LaoPeople’s Democratic Republic (Lao PDR) Ministry of Education and Sports (MOES) and UNICEF.The CBSR programme, which is the object of this evaluation, aims to contribute to the joint targetfor achieving “improved capacity to implement and monitor inclusive, equitable and quality ECEservices” and 55% enrolment for 3- to 5-year olds children into early childhood education (ECE)by 2020.

The UNICEF-supported CBSR programme commenced in Lao PDR in 2013 with the objectivesto:

1. Promote readiness for pre-education for children 4 years and 8 months to 5 years and11 months

2. Encourage children to have basic care for him/herself independently and food behaviour3. Support communication skill, interaction with others and learning skill

To date, CBSR has operated across 11 districts in 5 provinces, with a total of 98 CBSR Centresthat are of varying statuses – either transitioned successfully to pre-primary classes, disconnectedor of unknown status, completing their course of external intervention in May 2019, or ongoingand still part of the intervention.

This evaluation was conducted between May and September 2019, with the report undergoinginitial validation at a stakeholder workshop hosted on 23 August 2019, and further writtenfeedback from UNICEF Lao PDR and from UNICEF’s Regional Office incorporated to enrich thefinal version. The evaluation is a formative one, as the programme is still ongoing and an impactevaluation was not planned. As agreed, and given the recognised constraints of the assignment,a case study approach was developed to build a ‘rich’ and in-depth understanding of the workingsof the programme, rather than focusing on wide, shallow, representation through sampling acrossall types and locations. The report, therefore, analyses mainly qualitative data to summariseconclusions and recommendations for the potential continuation, scale-up and mainstreaming ofthe CBSR modality within Lao PDR’s early childhood education sub-sector.


356392AT52 | 2 | 1 | December 2019

List of Acronyms

CBSR Community-Based School ReadinessCE Cambridge EducationDESB District Education and Sports BureauDFID Department for International DevelopmentDoECE Department of Early Childhood Education (MoES)DOF Department of Finance (MoES)DOP Department of Organisation and Personnel (MoES)DTE Department of Teacher Education (MoES)ECCE Early Childhood Care and EducationECD Early Childhood DevelopmentECE Early Childhood EducationEMIS Education Management Information SystemESDP Education and Sports Sector Development PlanESQA Education Standards and Quality Assurance CentreGoL Government of Lao PDRHRBA Human rights-based approachIEC Inclusive Education CentreLao PDR Lao People's Democratic RepublicLTA Long-Term AssistanceM&E Monitoring and EvaluationMML Mott MacDonald LimitedMoES Ministry of Education and SportsMoRES Monitoring of Results for Equity SystemsNGO Non-governmental organisationOECD-DAC The Organisation for Economic Co-operation and Development's Development

Assistance CommitteePESS Provincial Education and Sports ServiceRIES Research Institute for Educational ScienceToR Terms of ReferenceUK United KingdomUN United NationsUNEG United Nations Evaluation GroupUNICEF United Nations International Children's Emergency FundVEDC Village Education Development Committee


356392AT52 | 2 | 1 | December 2019

Contents

Executive Summary i

1 Background Context 11.1 The Social, Economic, Political and Demographic Context in Lao PDR 11.2 Responding to Early Childhood Needs 1

2 Purpose and Object of the Evaluation 42.1 Evaluation Purpose and Goal 42.2 CBSR Programme Expectations and Objectives 42.3 CBSR Programme Geographic Coverage 72.4 CBSR Programme Current Implementation Status 82.5 CBSR Programme Stakeholders 82.6 Developing a CBSR Programme Theory of Change 10

3 Evaluation Objectives, Scope and Methodology 113.1 Evaluation Objectives 113.2 Evaluation Criteria, Questions and Framework 113.3 Evaluation Scope 113.4 Rationale for Evaluation Design 113.5 Data Collection Methods 123.6 Sample Selection for Primary Data Collection in CBSR Centres 143.7 Data Sources and Methods of Analysis 143.8 Limitations of the Methodology and Data Collection 153.9 Ethical Considerations 17

4 Evaluation Findings 184.1 Evaluation Question 1 18

4.1.1 Relevance 184.1.2 Effectiveness 204.1.3 Efficiency 254.1.4 Sustainability 274.1.5 Equity 29

4.2 Evaluation Question 2 324.3 Evaluation Question 3 32

5 Conclusions 335.1 Conclusion Question 1 33

5.1.1 Highly relevant 335.1.2 Effective but with improvements needed 34


356392AT52 | 2 | 1 | December 2019

5.1.3 Inefficient given over-reliance on UNICEF funding and communitycontributions 34

5.1.4 Unsustainable unless changes are made to redress surroundingsystem-level blockages 35

5.1.5 Improving equity on language but weak in other aspects 355.2 Conclusion Question 2 365.3 Conclusion Question 3 37

6 Recommendations 386.1 Summary of recommendations and stakeholder responsibilities 396.2 Detailed Recommendations 406.3 In Summary 456.4 Proposed Adaptations to CBSR for UNICEF’s 2022 - 2026 Country

Programme 45

Appendices- attached separately 50

A. Terms of Reference

B. Criteria for selection of CBSR intervention

C. Agenda for the Validation Workshop

D. Theory of Change-Presentation

E. Draft Theory of Change

F. Theory of Change exercise

G. Evaluation Framework

H. Table of Sampling

I. List of CBSR Centres

J. Table of district/ community respondents (during case study research)

K. Table of central level respondents

L. Quotations against DAC criteria

M. Analysis of basic kit (provided by UNICEF to new CBSR Centres)


356392AT52 | 2 | 1 | December 2019

N. List of basic kit materials

O. CBSR Monitroing ReportsO.1 Monitoring Report CBSR MaiO.2 Monitoring Report SeponO.3 Monitoring Report Taoi

P. Overview of the Tanzania School Readiness Programme

Q. Feedback from the participants on the draft recommendations

TablesTable 1: Summary of CBSR Programme and Evaluation Characteristics 5Table 2: Data on Priority Provinces for UNICEF's New Country Programme - ranking ofdistricts based on selected programmatic indicators and N-MODA Indices 8Table 3: Description of Stakeholders with a Role in CBSR 9Table 4: Challenges and potential implications 15Table 5: Summary of Recommendations 39Table 6: Proposals for scaling up CBSR 45

FiguresFigure 1: Opportunities and concerns 2Figure 2: Lao Map: CBSR sites and data on UNICEF districts 7Figure 3: Full technical review 21Figure 4: CBSR –Community Education Centre 48

ChartsChart 1: Student Assessment Results by Grade Level: Overall Scaled Score 32

Cambridge Education | Lao PDR, 2013 - 2018 iFinal Draft Report

356392AT52 | 2 | 1 | December 2019

Executive Summary

Background Context and Object of the EvaluationThe UNICEF-supported Community-Based School Readiness (CBSR) Programme wasestablished as a joint initiative of the Lao People’s Democratic Republic (Lao PDR) Ministry ofEducation and Sports (MoES) and UNICEF. It uses a community-based approach to achieve“improved capacity to implement and monitor inclusive, equitable and quality early childhoodeducation (ECE) services.”

The Government of Lao PDR and UNICEF recognise that the global evidence is incontrovertible– weak services and toxic stressors damage children’s brains and their future potential. Insummary, the case for investment is:

1. Rights-based – every child has rights as enshrined in the UN Convention on the Rightsof the Child and reinforced in the Sustainable Development Goals (SDGs)

2. Physiological – the brain is almost fully developed by the age of six and thus thefoundation for the future potential of a child to learn and to be successful is establishedin this early period of their life

3. Economic – investment in the early years of a child’s education have the greatest ratesof return for governments – more than at any other educational stage

There has been remarkable progress in increasing access to ECE services in Lao PDR: over thecourse of 3 years from 2015 – 2018 the number of ECE centres increased by over one-third.Enrolment of 3 – 5 year olds increased by over a half achieving the MoES’s target for 2020.However, more than four out of five of 3 – 5 year old children are still not enrolled particularly inrural, isolated and ethnic minority communities where [there is] limited teacher and humanresource; [and there is] limited teaching-learning materials and other facilities.

Purpose, Goal and Objectives of the CBSR Programme

The purpose of the CBSR Programme is to help meet the national targets of more equitableaccess to quality ECE services through a community-based intervention in remote anddisadvantaged areas.

. The Programme has the following objectives:

● Promote readiness for pre-education for children 4 years 8 months to 5 years 11 months● Encourage children to have basic care for him/herself independently and food behaviour● Support communication skill, interaction with others and learning skill

The Evaluation Purpose and ObjectivesThe purpose of this evaluation was to assess how the CBSR initiative can and should continueto operate alongside Lao PDR’s existing institutional models (kindergartens and pre-primaryclasses) and to review the potential of the CBSR programme to be scaled-up nationally.

Key objectives of the evaluation, therefore, were:

● To assess the relevance, efficiency, effectiveness, sustainability and equity of the UNICEF-supported CBSR programme.

● To provide recommendations for further improvement of MoES’ relevant (sub)sector policies,plans, strategies and operations of the CBSR programme interventions for scaling up, includingthat in the next ESDP 2021-2025.

Cambridge Education | Lao PDR, 2013 - 2018 iiFinal Draft Report

356392AT52 | 2 | 1 | December 2019

● To provide strategic guidance to UNICEF in redefining its focus area of support to MoES formainstreaming of the CBSR Programme under the remaining half of the Country Programme(CP) and the next CP periods.

The Evaluation MethodologyAs described in Section 3 below, the Evaluation Team adopted a mixed methods approachfollowing United National Evaluation Group (UNEG) Norms and Standards. Findings are drawnfrom mainly qualitative data collected at 8 purposively selected CBSR Centres1 with additionalsecondary documentation and data and further statistical analysis.2

Data collection teams visited CBSR Centres and their communities in 2 provinces (one Northernand one Southern), 3 districts and 8 villages. Interviews and focus-group discussions were heldinvolving 53 community-level participants, also interviewing 11 provincial and district governmentrepresentatives and 19 key informants from 12 relevant central organisations and Departments.In addition, this data was bolstered by reviews of secondary reports and references, especiallythe Mid-Term Review Report (2018), the draft ECE Policy (2019), UNICEF’s Baseline Study (July2019), UNICEF’s Costed Plan for ECE (2017), and UNICEF’s Interim Strategy Paper (2018–2022).

Limitations of the MethodologyThe limitations in the data collection and resulting analysis (detailed in Section 3.8) included:

● Lack of baseline data● Unreliable data● Difficult timing● Geographical scope stretched limited resources● Perceptions of community respondents● Translation and interpretation● Potential Evaluation Team bias● Presence of other interventions at the Centres not mapped

These limitations were highlighted and discussed in advance, with UNICEF Lao PDR, who agreedthat they would not be a barrier to useful recommendations emerging.

Main Findings and ConclusionsThe CBSR programme has successfully met many of its objectives. There is potential for theprogramme to be further scaled up and institutionalised and become more cost-effective,However, the question of how CBSR will be financed and how it will then ultimately besustained on a national basis needs further exploration.

1 Between 2013 and 2018, the CBSR programme established 98 centres in 5 priority provinces – Saravan, Savannakhet, Phongsaly, Attapeu, Luangnamtha2 The priority provinces of UNICEF’s Country Programme are selected in coordination with the Government of Lao PDR on the basis of their rankings in terms of

overall poverty and performance. As per UNICEF’s selection documentation, factors such as the under 5 Child population, Poverty head count ratio, socialIndicators and geographical balance are taken into account. The provinces selected for the current country programme (2017-21), which include 4 out of 5of CBSR’s programme areas, were selected based on their high level of multiple overlapping deprivation among under 5 children in social dimensions ofHealth, Nutrition, Education, Water and Sanitation and Child Protection; their high disparities between economic growth and social indicators; thecommitment, capacity, coordination mechanism and budget allocation from the provincial administration; and the average normalized index applied inselection with the worst performing and most-in-need provinces and districts being selected for support. Luangnamtha was not a province in the period ofthe current Country Programme (2017–21) but was a priority in the CP under which the CBSR programme initially started.

Cambridge Education | Lao PDR, 2013 - 2018 iiiFinal Draft Report

356392AT52 | 2 | 1 | December 2019

The main findings and conclusions are as follows:

1. Is the CBSR programme relevant, effective, efficient, sustainable and equitable?

Relevance:

· There is an overwhelmingly positive consensus from 90% of government representativesthat the CBSR programme is relevant to Lao PDR’s needs and policy context. Communitydemand was also very high, with 96% of community-level respondents expressing support.

· However, in the five CBSR target provinces, enrolment of 3 to 5 year old children in ECEis very low, recorded at only 17% in Saravan Province (2014).3 This confirms the need forrapid, cost-effective solutions to a problem which is not being resolved currently by existinginstitutional models such as pre-primary classes and kindergartens.

· The inclusion and training of community members as the CBSR Facilitators increases therelevance of the programme in tackling the context of both shortage of Govenrment-trained-and-salaried personnel, but also tackling language barriers for non-Lao-speaking groups,which was a challenge specifically mentioned by 45% respondents.

Effectiveness:

· The CBSR programme is effective and could be made more so with minor adjustments(detailed in section 6.4).

· The CBSR curriculum and materials are in line with effective ECE programmes in othercountries.4

· Both the District Government ‘master’ trainers and CBSR Facilitators indicated almostunanimously (95% of respondents) their capacity to understand the training manuals and toteach effectively using the materials provided.

· 63% of respondents, commented on the particular effect of improved Lao languagecomprehension among CBSR-attending children.

· 100% of the 28 Centres involved in UNICEF’s Baseline Study had received the expectednumber of monitoring visits, which was contrary to pre-primary classes where less than45% of teachers reported having received monitoring support.5

Efficiency:

· 52% of the government respondents (National and local) were either explicit about lack ofavailability of government funding for expansion of CBSR or unable to comment.

· In light of the Government of Lao PDR’s reduction of planned expenditure6 on education andthe 8th ESDP’s recognition of the challenge for allocating more funding to ECE7, the cost ofproviding ECE/school readiness activities in communities which are rural, isolated and wherethere may be no roads must be rigorously controlled and supported to ensure maximum costeffectiveness.

· The costs of infrastructure development,8 provision of food during CBSR sessions, andongoing maintenance of learning spaces are inputs largely handled by communities,which lends far greater efficiency, flexibility and affordability to the model. 69% ofcommunity respondents indicated clear willingness to continue to bear such costs.

3 EMIS 2014 data collated by UNICEF for prioritisation of provinces under the 2017 – 2021 CP4 Examples include School Readiness programmes in Vietnam (facilitators receive a stipend) and Tanzania (no stipend, only in-kind community support).5 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019)6 ‘Already in February of 2017 the Government of Lao PDR was obliged to reduce planned government expenditures in education in anticipation of reducedgovernment revenues and increasing contributions to large infrastructure projects currently underway.’ - Dewees, A., Inthasone, S. and Loizillon, A., CostedAction Plan for ECE: Analytical Report (2017)7 Ibid.8 Refer to Annex B which presents the criteria for community selection to be part of CBSR, and highlights the need for communities to provide viable spaces

Cambridge Education | Lao PDR, 2013 - 2018 ivFinal Draft Report

356392AT52 | 2 | 1 | December 2019

· However, other costs that are not possible for communities to take on need to be borne byMoES (or external assistance – as UNICEF Lao PDR is currently fulfilling). Therefore, learningand teaching materials, training, a stipend for the CBSR Facilitator, and ongoing monitoringcosts, will require non-community sources of funding.

· Calculations provided in Section 4.1.39 show a cost per child of USD 196 in year one of theintervention, but with potential to decrease significantly in subsequent years, down to belowUSD 60 per child. This is still more costly than other international models, which haveachieved cost per child rates of less than USD 25 per child.10

Equity:

Language

· Despite the education policy and its specification for all instruction to take place in Laolanguage, many remote and ethnic minority communities are excluded and falling behind.Therefore, it is very positive that 100% of CBSR facilitators interviewed are bilingual in thelocal language as well as Lao. This finding was also corroborated by UNICEF’s BaselineStudy, which reported that across the 28 CBSR Centres sampled, 100% of the CBSRFacilitators were able to speak both local language and Lao.

Disability

· All community respondents stated that there were no children with disabilities withinthe community and therefore none attending CBSR centres. Although Head teachers andFacilitators pointed to two examples, there is currently a woefully limited pool of evidence todraw upon regarding the prevalence of disability among children. The baseline survey stated‘We do not know how many children with disabilities are in these villages and are not in school.The numbers suggest very few are in school, which potentially means that many are beingexcluded’.11 This evaluation supports the Baseline’ Survey’s statement that lack of dataprecludes a confident conclusion.

Gender

· Girls are at parity for access to the CBSR programme, although achievement is slightlybelow boys according to UNICEF’s baseline survey.

· More than half (65%) of CBSR Facilitators are women.· Systematic programme data is not currently being tracked on aspects such as gender, which,

as in the case of disability, precludes confident conclusions or recommendations.

Sustainability:

· Given the conclusion that the CBSR intervention should be scaled-up and mainstreamed,where relevant to do so, its integration into the education system will depend upon thesustainability of the model. However, the evaluation identified several key stressors onsustainability, namely:

a. Efficiencies in monitoring costs could be found, which would support sustainability, ifthere was reduction in the stipulated number of monitoring visits. At present the numberis higher than the norm for other primary and pre-primary institutions.

b. Despite the willingness of communities to provide in-kind support to CBSR, 90% ofcommunities indicated they are unable to support CBSR Facilitators with essential cashstipends and that external funding is required for this and other essential costs.

9 There are caveats in this analysis, which highlight the uncertainty of the Evaluation Team in reaching firm conclusions regarding cost efficiency. These caveatsinclude that information available was sometimes conflicting or missing or outdated; that there are costs included in the Costed Plan that we are notincluding in our analysis as their likelihood of being provided by government and/or overlap with existing ingredients is not clearly understood – thisincludes the School Block Grant for example; and finally that in performing this analysis key assumptions are made, most significantly that CBSR Centresprovide for an average of 20 children, and not the 10 children stipulated.

10 The School Readiness Programme in Tanzania has provided costing data that enables this comparison11 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019)

Cambridge Education | Lao PDR, 2013 - 2018 vFinal Draft Report

356392AT52 | 2 | 1 | December 2019

c. As reported by the Department of Planningand the Department of Personnel, as well asthe Provincial Governments, there are insufficient early childhood educators (pre-primaryteachers) deployed and salaried, which is exacerbated by provincial quotas that limit thenumber of pre-primary posts available. The GoL have a budget deficit for teacherdeployment and rely heavily on ‘volunteer’ teachers who are fully trained but not salarieddespite working in government schools. Therefore, although central EMIS data points torelatively high numbers of available (trained) pre-primary teachers, the data provided tothe evaluation team was conflicting and even if higher numbers are accurate this doesnot automatically equate to the appropriate deployment of such teachers where they areneeded.1213

d. The training and capacity of CBSR Facilitators will be ‘lost’ to the sector if traditionalmodels of career development are maintained as these are cumbersome andinaccessible to lowly-qualified individuals - 75%14 of the community Facilitators only havelower secondary education or below, which limits their career aspirations if not supported.

e. Low enrolment is most serious in Southern provinces.15 More targeted, province-specific,strategies for meeting ESDP targets are needed to address the underlying dynamics ofthis regional variation and adopt more flexible models of delivery than a centralisedapproach currently allows for.

f. The current CBSR ‘model’ is inherently unsustainable as it assumes the requirement to‘transition’ to pre-primary class. This assumes that young children can access a primaryschool in their close proximity, where a pre-primary class can be attached, and that theprimary school will have sufficient human resource to provide a dedicated pre-primaryteacher. Furthermore, even given these factors, the ‘transition’ and closure of CBSRcentres would ignore the benefits of the programme for 3 and 4-year old children, for thecommunity more widely, and in holistic areas of child development, such as water andsanitation, health, nutrition, etc.

Q1: Is the CBSR programme relevant, effective, efficient, sustainable and equitable?In conclusion, the CBSR programme is:

· Highly relevant· Effective but with the potential to be more effective with recommended changes· Reliant in crucial ways on UNICEF funding and, therefore, inefficient given lack of

MoES commitment to match this funding once UNICEF withdraw· Improving equity of minority groups, particularly in terms of language; but lack of

systematic data on disability and gender preclude robust conclusions· Unsustainable unless changes are made to redress surrounding system-level

blockages to the ongoing existence of the programme

2. Should the CBSR programme be scaled up and mainstreamed within the Lao PDR’sMoES plans and operations?

The CBSR programme should continue and should be rationalised and scaled up as part of the9th ESDP and next CP.

12 EMIS data provided by the GoL Department of Planning as part of the data collection for this evaluation (data for 2018 – 19) placed the total number of 5years olds in Lao PDR at 136,196, of which 114,635 are recorded as being in pre-primary (84%). Furthermore, this data records 14,709 pre-primaryteachers, meaning an average of 8 children per teacher. Obviously, the actual teacher:pupil ratio varies according to whether teachers are appropriatelydeployed and utilised at the pre-primary level.

13 In 2017, the MoES GoL released a national survey report on national teacher availability, deployment and salary coverage, but as this was only available inLao Language, the evaluation team had to rely on interview feedback regarding findings of this research. For example, senior staff of the Department ofPersonnel claimed that 12,000 more teachers are required to fulfil national needs; and staff at the EMIS Section of the Department of Planning said thatthe 2017 teacher survey showed that of 66,119 teachers, over 12,000 are voluntary. And of 4995 pre-primary teachers, 1965 are voluntary. These figures,however, contradict the 2018-19 data provided by the same EMIS department, which can not be fully explained by the difference in year of recording –almost 5000 pre-primary teachers versus almost 15,000 is not possible to explain by increased recruitment or changes in data gathering practices alone.

14 Ibid.15 Dewees, A., Inthasone, S. and Loizillon, A., Costed Action Plan for ECE: Analytical Report (2017)

Cambridge Education | Lao PDR, 2013 - 2018 viFinal Draft Report

356392AT52 | 2 | 1 | December 2019

Recommendations for how to achieve this are detailed in Section 6 and highlighted in the diagrambelow.

3. How should UNICEF intervene and support, through its current and upcomingCountry Programmes, in scaling up and institutionalising the CBSR model, and,furthermore, support the overall achievement of increased access to quality ECDservices for all of Lao PDR’s young children, but especially the mostdisadvantaged?

An adapted model, in which UNICEF redresses efficiency and sustainability issues in a targetedway, should be implemented under the existing and next CP, and factored in the 9th ESDP.Expansion of this adapted and integrated model to a wider variety of disadvantaged ‘clusters’ ofcommunities, could be achieved during the next CP, particularly through careful targeting andmore systematic monitoring of the intervention to enable ongoing scale up.

Cambridge Education | Lao PDR, 2013 - 2018 viiFinal Draft Report

356392AT52 | 2 | 1 | December 2019

Recommendations

The main resulting recommendations are captured in the graphic below and detailed in Section6. These recommendations are organised according to priority for action.

Recommendation Details Impact onDAC Criteria

Responsible Timing

1. Rationalisethe CBSRPackage

Rationalise the existing CBSRpackage, by:Reducing the number of monitoringvisits by District officialsEnhancing the role of local networksin supervisionReducing the toolkit provided toCBSR Facilitators to essentials only(proposed list provided)Increasing the facilitator-to-child ratioand having one facilitator per Centreunless ratios demand more

Rationalisationof the CBSRmodel willsubstantiallyimprove theeffectiveness,efficiency andsustainability ofthe programme

MOES/UNICEF

Immediatelyfor new CBSRCentres andwithin 1 yearfor existingCentres

2. Providecoordinatedand ongoing

dedicated technicalassistance support forenlarging andenhancing CBSR

Coordinate with all core fundingpartners to plan for immediate andongoing increases in access andquality of ECEPrepare Exit/Sustainability Plan tofacilitate handover to GoL in thelonger term

Thesustainability ofCBSR andother modelsfor youngchildren of LaoPDR will beenhanced

MOES/UNICEF/ otherDev. Partners

Immediately

3. Conduct afull impactevaluation of

the CBSR programme

Review baseline data evidence andassess feasibility of using existingbaseline for overall impact evaluationat endlineDepending on outcome of review,improve ability to determine impact ofCBSR at end line through additionalbaseline data collection and betterongoing data collection

Relevance ofthe programmewill improve

UNICEF Within 1 year

4.Mobiliseand supportlocalnetworks of

supervision anddecreased reliance onDESB for monitoring

To support changes to the CBSRpackage, reduce the reliance oncentrally-led monitoring by:Training VEDC leaders and HeadTeachers to provide supervision,mentoring and monitoring of CBSRcentresMinimising the role of DESB to qualityassuranceStreamline the reporting linesbetween VEDC/HTs and DESB tofacilitate the monitoring process

Localisedmonitoringnetworks wouldimprove theefficiency andsustainability ofthe programme

MOES/UNICEF

Within 18months

5. Supportand enhancedata,

mapping anddocumentation onCBSR and ECE

Review CBSR monitoring reports andfull data to produce a comprehensiverecord since 2013Plan for ongoing and more systematicdata collection, with a focus ontargeted indicators currently missingfrom any other source of dataImprove present pre-primary mappingto include CBSR centres and areas ofno coverage of pre-primary

Improved dataand informationwould assistboth MoES andUNICEF inongoingplanning andimprovementsto ECEnationally, thusenhancing therelevance of the

MOES/UNICEF

Within 18months

Cambridge Education | Lao PDR, 2013 - 2018 viiiFinal Draft Report

356392AT52 | 2 | 1 | December 2019


Responsible Timing

CBSRprogramme

6. Developa

communications andengagement plan

Enhance communications to ensurebetter targeting of messages on thecritical benefit to all children, theirfamilies and communities, and societyat large of quality ECE

Improved take-up and demandfor ECE willenhancesustainability

MOES/UNICEF

Within 2 years

7. Improveprocess of5-year-oldtransition

into Primary Grade 1

Enhance and formalise therelationship between CBSRFacilitators and Grade 1 Primaryteachers in the nearest primaryschoolPrepare Primary Grade 1 teachers forreceiving a diversity of children

Assistingchildren withthe transitionprocess willimproveeffectiveness,equity andsustainability ofthe programme

MOES Within 2 years

8. Pilot theexpansion ofCBSR

Centres to act as ECDHubs and review theoverall ‘model’ ofCBSR to redressinherent gaps

Review the model of institutionaltransition and rethink the assumptionthat CBSR Centres are only a‘bridging’ (temporary) mechanismExpand CBSR model to includeWASH, nutrition and health activities,parental education

Enhancing ahub approachwill improve allaspects of theprogramme:relevance,effectiveness,efficiency,equity andsustainability

UNICEF Within 2 years

9. Conductdeep-divestudies toenhance

understanding ofequity issues affectingECE access andquality

To build upon recommendation 5,also conduct deeper studies into thebarriers and exclusion factors that arecurrently blocking young children fromaccess to quality ECEProvide recommendations for theongoing rationalisation of the CBSRmodel, or other, different solutions toensure better access to quality ECEfor disadvantaged and marginalisedchildren

Outcomes ofsuch studieshave potentialto improveequity

MOES/UNICEF

Within 2 years

10. AddressECEFacilitator

and Teacher careeremployment blockages

Explore a range of options forincreasing the human resourceavailable for ECE services,particularly in harder to reachcommunitiesEnsure that those already providingECE services have access to careerpathways to enable them to remain‘useful’ members of the ECE sectorRecognise and celebrate the potentialfor the CBSR programme and similarinitiatives to enhance equity in harderto reach communities, due to thelivelihood options provided to under-qualified individuals and especially towomen

Improvedefficiency,equity andsustainability

MOES Within 3 years

Cambridge Education | Lao PDR, 2013 - 2018 1Final Draft Report

356392AT52 | 2 | 1 | December 2019

1 Background Context

1.1 The Social, Economic, Political and Demographic Context in Lao PDRThe Lao People’s Democratic Republic (Lao PDR) is a land-linked, mountainous country with ayoung and ethnically diverse population. Current estimates place Laos’ population at over 6.87million16, with declining fertility and birth rates, but improving mortality rates, now at 48.6 infantdeaths per 1000 live births. The population is dominantly located in rural areas (approximately70%). It is estimated that approximately 50% of the population are children and young peoplebelow the age of 25, and 32% are below the age of 14. The population is highly diverse, with 49official ethnic groups, comprised of 167 ethnic subgroups with different dialects, languages,cultures and traditions.17

Lao PDR has progressively integrated into the Association of South East Asia Nations (ASEAN)Economic Community (AEC). Its strong economic growth, driven primarily by natural resourcesand energy sectors, enabled Lao PDR to move from a low income to a lower middle-incomecountry in 2011. Lao PDR joined the World Trade Organisation (WTO) in 2013. However, growthof Gross Domestic Product (GDP) in Lao PDR has been declining in recent years, currently at6.7%, due primarily to natural disasters linked to declining agricultural and industrial production,as well as to an uncertain global environment and slow-down in China. Furthermore, in the lasttwo decades the Gini coefficient increased from 0.31 to 0.364, with significant disparities betweenurban and rural, wealth quintiles and the four main ethnic categories. One-third of the populationin upland areas are still below the poverty line compared to one-fifth of the population in lowlandareas.

To support the overall reduction of poverty and inequality, UNICEF Lao PDR has entered into afive-year Country Programme (CP) 2017-2021, agreed between the Government of Lao PDR(GoL) and UNICEF Lao PDR. The education outcome of the CP aims that “by 2021, children,especially most disadvantaged and in hard-to-reach areas, have access to quality early childhoodand basic education services, and complete the basic education cycle with improved learningoutcomes.”18 It is widely accepted internationally that investing in early childhood education19

(ECE) is crucial for success. Children who benefit from quality ECE programmes are betterprepared for primary school and will reach better education outcomes. Quality ECE also helpsreduce repetition and drop-out rates with even more pronounced positive outcomes amongchildren from vulnerable groups.20

1.2 Responding to Early Childhood NeedsThe GoL targets that 55% of children between ages 3 and 5 years old are in ECE by 2020, whichwould give 41,000 more children the opportunity for this level of education. While the EducationSector Development Plan (ESDP) places much emphasis on the role of the private sector inhelping to meet this need, the government sector intends to cover 18,000 additional children

16 International Monetary Fund: https://www.imf.org/en/Countries/LAO17 https://www.unicef.org/laos/18 The vision of the CP aligns with the United Nations Partnership Framework (UNPF) 2017-2021, as well as the GoL’s 8th National Socio-Economic

Development Plan (NSEDP) 2016-2020 and the Ministry of Education and Sports’ (MoES) 8th Education and Sports Sector Development Plan (ESDP)2016-2020

19 Although the terms ‘Early Childhood Development’, ‘Early Childhood Care and Education’, and ‘Early Childhood Education’ are sometimes usedinterchangeably, they do nonetheless refer to slightly different meanings in relation to early childhood. ECD is the most encompassing and takes accountof a child’s whole development from pre-conception up to the age of 8. A full understanding of ECD is available in UNICEF’s own guidance document:https://www.unicef.org/earlychildhood/files/FINAL_ECD_Programme_Guidance._September._2017.pdfIn contrast, the explicit mention of education in the terms ECCE and ECE tend to relate the development of the child more directly to early stimulus, earlylearning and preschool. The majority of programmes that focus on ECE are more concerned with the readiness of children to enter into formal schoolingand to progress through primary school successfully.

20 https://www.globalpartnership.org/focus-areas/early-childhood-care-and-education

https://www.imf.org/en/Countries/LAO

https://www.unicef.org/laos/

https://www.unicef.org/earlychildhood/files/FINAL_ECD_Programme_Guidance._September._2017.pdf

https://www.globalpartnership.org/focus-areas/early-childhood-care-and-education


356392AT52 | 2 | 1 | December 2019

between 2016 and 2022. To support this target, Output 1 of UNICEF’s CP Education Outcomeintends that: “By 2021, national and sub-national Government Ministries and other keystakeholders have improved capacity to implement and monitor inclusive, equitable and qualityECE services.” Collectively, these targets work to improve access, quality and equity of ECEservices, focusing on remote areas and under-served groups. There is also support for theESDP’s ECE sub-sector objective which is to provide the opportunity for children to learn Laolanguage and have readiness to study in primary education, focusing on reducing drop-out andrepetition in primary education.

Although there has been a remarkable progress in reaching the ECE targets, with MoES’Education Management Information System (EMIS) reporting 77.1%21 (age 5) and 56.2% (age 3-5) enrolment rates in 2018, long before the 2020 target year; nonetheless, clear and persistentdisparities in access continue. Children in rural areas, from poor families and non-Lao speakingethnic communities are lagging behind.

The ESDP reflects on progress against Goal 1 of Education for All (EFA) on Early Childhood Careand Education (ECCE), indicating that although this was broadly on track according to targets setin the EFA National Plan of Action (NPA) (2003 - 2015), issues of learning and quality in ECCEwould require more focus, as would issues around equity given the lack of access among smallgroups of mainly ethnic minority children from remote areas. Furthermore, there is no systematicmeasurement yet in place to assess the quality of existing ECE services and learning anddevelopment outcomes of young children.

Only 2.94% (2014) of GDP is allocated to education of which less than 6% is allocated to the ECEsub-sector. In some districts as many as 70% of 5-year-old children do not have ECEopportunities, and a variety of other concerning trends are also highlighted in Figure 1.22

Figure 1: Opportunities and concerns

21 Most recent data provided during this Evaluation implies a slightly higher rate of 77.6%22 Data from both the UNICEF Lao PDR website and Multiple Overlapping Deprivation Analysis on Stunting among children under 5 years, Lao PDR” UNICEF

2019


356392AT52 | 2 | 1 | December 2019

Therefore, in view of the upcoming development of the 9th NESDP and ESDP, and UNICEF’s newCP phase, there is an opportunity to assess the relevance and potential impacts of Community-Based School Readiness (CBSR) programme on the overall Lao PDR education sector. TheCBSR programme has supplemented the GoL’s expansion agenda in areas where Kindergartenor Pre-Primary Classes are as yet not feasible, not cost efficient and/or not established. Therefore,this evaluation (for which the full Terms of Refence are attached in Annex A) will assess how thissupplementary initiative can and should continue to operate as a legitimate part of the ECEsystem and in alignment with the Law of Education (2015), which recognises community-basedservices and the CBSR model as one model of ECE provision.


356392AT52 | 2 | 1 | December 2019

2 Purpose and Object of the Evaluation

2.1 Evaluation Purpose and GoalThe purpose of the evaluation is to draw lessons learned from the CBSR programme and to informthe forthcoming MoES ECE policies, strategies and budgets. The evaluation aims to furtherenhance and institutionalise the CBSR Programme in the education system.

Why was the evaluation commissioned at this point of time?This formative evaluation has been commissioned to provide evidence on the potential of CBSRto meet the needs of more children living in remote and disadvantaged communities who may nothave access to ECE. The CBSR Programme was initiated in 2013 in response to clear gaps inprovision of ECE in remote, disadvantaged and rural areas. UNICEF’s current CP (2017 – 2021)is continuing to provide support to 31 CBSR Centres until 2020. Therefore, the status ofimplementation of CBSR remains ongoing and at this time, it is still possible to adapt the presentUNICEF CP (2017-2021) and to further develop the next CP while continuing to support MoES todevelop the 9th ESDP (2021 – 2025). There is an opportunity in the upcoming plans to considerthe potential continuation and scale-up of the CBSR programme beyond its current end-point.

Who are the primary and secondary audiences/users of this evaluation?The primary targeted beneficiaries are pre-primary-aged children (4years and 8 months to 5 yearsand 11 months old), and the secondary beneficiaries include CBSR facilitators, the DistrictEducation and Sports Bureaus (DESB), Provincial Education and Sports Service (PESS) officials,and Village Education Development Committees (VEDC) members.

How the evaluation is intended to be used? What this use is expected to achieve?

The purpose of the evaluation is draw lessons learned from the CBSR programme and to informthe forthcoming MoES ECE policies, strategies and budgets. The evaluation aims to furtherenhance and institutionalise the CBSR Programme within the education system. The evaluationwill also “contribute to learning and knowledge sharing on equity-focused programming” (CPDCEP 2017-2021).

The evaluation results will be used to inform the relevant (sub)sector policies and strategies, suchas the next ESDP, the forthcoming ECE policy and MoES’s annual work plans, as well as theCBSR Programme’s operational, monitoring and evaluation framework and its curriculum andtraining contents. Where appropriate the evaluation may be used to promote CBSRinstitutionalisation and scale-up in the education sector plan and system, including in the nextESDP 2021- 2025.

The evaluation results will also provide strategic guidance to UNICEF in redefining its focus areasof support to MoES for improving and mainstreaming the CBSR Programme in the sector underthe remaining half of the current CP and the next CP periods.

In the following sections, the object of the evaluation – the CBSR programme – is described ingreater detail, to familiarise readers with its objectives, characteristics, scope, concernedstakeholders and beneficiaries, and underlying theory of change logic.

2.2 CBSR Programme Expectations and ObjectivesTo support reaching the national targets of more equitable access to quality ECE services, theCBSR Programme (referred to as ‘Play Groups’ or ‘Learning through play’ in Lao language) has


356392AT52 | 2 | 1 | December 2019

been an important part of the joint strategy of MoES and UNICEF. The programme commencedin Lao PDR in 2013 with the objectives of:

● Promoting readiness for pre-education for children 4 years and 8 months to 5 years and11 months

● Encouraging children to have basic care for him/herself independently and foodbehaviour

● Supporting communication skill, interaction with others and learning skill

Since the programme’s start in 2013, a total of 98 CBSR centres have been established in 5provinces – Phongsaly, Salavan, Savannakhet, Attapeu, and Luangnamtha. Apart fromLuangnamtha, the other four provinces all rank in the bottom half of provinces in terms ofcombined poverty, health, WASH and education indicators.23 With UNICEF support, the CBSRCentres have been largely managed by the DESBs under supervision of their respective PESSoffices and the overall guidance by the MoES, at central level. Under the guidance of the DESB,VEDCs in target villages also play a key role in supporting the implementation and monitoring ofCentres.

The table below summarises the characteristics of the CBSR programme and the specific scopeof the evaluation within the overall programme.

Table 1: Summary of CBSR Programme and Evaluation CharacteristicsOverall CBSR Programme Characteristics This Evaluation’s Scope

Geographiclocations

5 provinces, 11 districts and 98 communities covered overthe programme duration.According to UNICEF’s own tracking: 24 centrestransitioned to pre-primary classes in primary schools, 27centres will complete the CBSR programme intervention inMay 2019, 31 centres24 are still ongoing beyond May 2019with UNICEF support, and 16 centres are disconnected orof unknown status.

A representative sample across thecombination of geographic locations –covering at minimum both northern andsouthern provinces – and both ‘ongoing’(operational) and transitional Centres. Atotal of 8 Centres/Villages selected i.e.15% of the overall cohort – 4 inNorthern provinces and 4 in Southernprovinces.

Timeline andimplementationstatus

Initiation of the programme in 2013. Centres receive 2years of support from UNICEF and then ‘transition’ to publicPre-Primary classes at Government Primary Schools. In2019, 31 Centres are still ongoing and still due for at leastone more year of programme intervention. A further phaseof programming is now under review as part of thisEvaluation.

Review of the full period ofimplementation will be undertakenthrough a mixed methods approach,incorporating both primary datacollection at 8 case study sites whereCentres are ongoing or transitioned intopre-primary, as well as availablesecondary literature pertaining to ‘past’sites that are beyond the period ofUNICEF support and have mixedstatuses.There was no baseline assessmentundertaken at the start of the UNICEF’ssupport to the CBSR Programme in2013 and no consistent monitoringsystem put in place, however, therecent MoES-UNICEF Baseline Surveyof the current CP will supplement dataalready collected on each of the presentCBSR centres.

23 With reference to UNICEF Lao’s Selection of Priority Provinces for UNICEF’s Country Programme based on ranking of provinces based on selectedprogrammatic indicators and N’MODA indices

24 Note that only 28 CBSR Centres were included in the UNICEF Baseline Study


356392AT52 | 2 | 1 | December 2019

Overall CBSR Programme Characteristics This Evaluation’s ScopeProgrammecomponentsThe scale andcomplexity ofthe object ofthe evaluation,including thenumber ofcomponents, ifmore than one,and the size ofthe populationeachcomponent isintended toserve

The CBSR centres are established where the DistrictEducation Authority under MoES (called District Educationand Sports Bureau, DESB) are committed to providing aformal government pre-primary class in the samecommunity by taking over the UNICEF-supported CBSRafter two-years of direct support by UNICEF to CBSR. TheCBSR programme activities are undertaken in a flexibleenvironment, using available space and/or facility to bedetermined by the community, such as temples, communitycentres, spare primary classrooms, or even constructed bythe communities themselves. CBSR facilitators are chosenfrom the same target communities, led by the community(through Village Education Development Committee,VEDC) under the supervision of DESB.For each CBSR centre, UNICEF provides direct support fortwo years in the areas of:community mobilisation including orientation for VEDCs;training to CBSR facilitators and primary school principals,incentives (stipends) to CBSR facilitators;provision of teaching-learning materials;monitoring and coaching support to CBSR facilitators bytechnical officials for DESB and Provincial Education andSports Service (PESSs);learning exchange workshops.

ResourcesThe totalresources fromall sources,includinghumanresources andbudget(s) (e.g.concernedagency, partnergovernmentand otherdonorcontributions

UNICEF inputs have been minimised to enablesustainability of Government of Lao take-up. Inputs fromUNICEF are restricted to:training of District officials, Head Teachers and community-selected CBSR Facilitators;provision of Basic Kit of Learning Materials and Books;a small stipend provided to CBSR Facilitators;and support for PESS and DESB monitoring of CBSRCentres.A total of USD 927,300 of UNICEF funding has beencommitted between 2013 and 2019.Community inputs include (but not costed): infrastructure(as listed above), in kind support for community facilitators(such as food) and their time for monitoring attendance etc.

To assess, particularly through casestudy visits to Centres, the use andvalue of these resources, and thefeasibility for either Government of Laoor Communities to continue to providethem without UNICEF input.

Beneficiarypopulation size

193 community Facilitators, 2062 pupils (1049 girls and1014 boys), 5 province offices and 11 district offices.

Through primary data collection: 2provinces, 3 districts, 8 villages, 16community Facilitators and 149children; and 15 Central key informantsfrom 12 relevant organisations andDepartments.Through available secondary datareview: all targeted beneficiaries.

Intendedoutcomes

Promote readiness for pre-education for children 4 years 8months to 5 years 11 month.Train community-based facilitators to implement the CBSRcurriculum.Strengthen the capacity of VEDC and DESB to initiate andmaintain the CBSR centres.

All targeted beneficiaries.

BeneficiariesInstitutions ororganisations

Children ages 4 years and 8 months up to 5 years and 11months, as well as their parents and community members,CBSR Facilitators, VEDC members, and District, Provincialand Central Government representatives, particularly the

All targeted beneficiaries.


356392AT52 | 2 | 1 | December 2019

Overall CBSR Programme Characteristics This Evaluation’s ScopeCommunitiesSocial groups

Ministry of Education and Sports. Particularly focused onUNICEF-selected disadvantaged Provinces and Districts,and vulnerable groups including ethnic minorities.

Human rightsroles- Duty bearers- Rights holders

Duty bearers are the Ministry of Education and Sports, inparticular the Department of Early Childhood Education andher counterpart Departments; the Minister of Education andSport, Ministry of Finance and National Assembly; andUNICEF Lao and her associate offices.Right-holders including women, men, girls, boys, ethnicgroups, non-Lao speaking communities, remote anddisadvantaged communities needing access to earlychildhood education.

All duty bearers and rights holders arethe target of the data collection andreporting approaches.

Gender groups There is no specified gender target for the programme, butthrough provision of para-professional entry into teachingand raised status of the early childhood sector, theprogramme can impact upon both young women and youngmothers in particular.

Establish through questioning atcommunity levels and disaggregation ofdata where feasible, if the programme isimpacting gender roles, girls’ education,and access for women to jobs.

2.3 CBSR Programme Geographic Coverage

Figure 2 and Table 2 below provide an overview of the geographical coverage of the CBSRprogramme, which, to date, has operated across 11 districts in 5 provinces. The programme hasestablished a total of 98 CBSR Centres that are of varying statuses – either transitionedsuccessfully to pre-primary classes, disconnected or of unknown status, completing their courseof external intervention in May 2019, or ongoing and still part of the intervention.

Figure 2: Lao Map: CBSR sites and data on UNICEF districts

MoES/UNICEF CBSR Districts Visited duringEvaluation

Other MoES/ UNICEF CBSR districts


356392AT52 | 2 | 1 | December 2019

Table 2: Data on Priority Provinces for UNICEF's New Country Programme - ranking ofdistricts based on selected programmatic indicators and N-MODA Indices

2.4 CBSR Programme Current Implementation StatusAs outlined above, UNICEF is providing ongoing support to 31 CBSR Centres until 2020.Furthermore, in parallel to this Evaluation, and complementary to it, UNICEF Lao PDR havecommissioned baseline research into the status of ECE and basic education across the CPpriority provinces. The baseline data has been made available to this Evaluation, and there isopportunity to reflect upon and compare the findings of the baseline and, in turn, to influence theendline that will be done at close of the current CP in 2021.

During the lifespan of the programme, the most major change has been to the Law of Educationwhich was revised and published in 2015 to include the CBSR programme as an official model,along with pre-primary classes in primary schools and kindergartens. UNICEF and otherdevelopment partners played a crucial role in influencing the Law to incorporate a community-based model, by highlighting the feasibility and low-cost options through piloting of CBSR.

Following the ratification in Law, curricula and standards for community-based play groups havebeen developed, with the regulations awaiting approval (intended by end of 2019). TheDepartment for ECE, MoES, was itself formed as a dedicated Department in 201725 and it is nowleading on the development of a dedicated Early Childhood Education Policy, which stakeholdershope to see published in 2019. The political environment, therefore, has changed to bring greateremphasis and focus to the importance of early childhood development.

2.5 CBSR Programme StakeholdersGiven the high-level engagement of UNICEF with the GoL and the alignment of CBSR with theGovernment’s education targets, a large variety of stakeholders were necessarily involved in theprogramme, often in more than one capacity. This is illustrated in the table below.

25 The ECE Department was established in 2017, pursuant to Prime Minister Decree 67/PM 2017. The Agreement on the Organisation and Operations of theDepartment of ECE (508/MoES) dated February 2, 2018 was signed and issued in 2018 by the Minister of MoES.

Name ofProvince/District

U5 populationRanking (low

number -highest; High

number-lowest)

Enrolment inEarly Childhood

Education(5 years old)

%Source: EMIS 2015

[ECEenrolment]Ranking

(low number -worst; high

number-best)

All indicators(without Full

Immunisation)

Visited duringCBSR

evaluation?Ranking

Phongsaly 51.3 Y

May/Mai 2 44.3 3 6 YSamphanh 3 35.1 1 7 YSavannakhet 55.3

Phine 4 35.8 2 12

Sepone 5 31 1 15Phalanxay 12 44.3 3 10Saravane 56.3 YSaravane 1 59.3 5 6

Ta oi 4 66.2 6 5 Y Lao ngarm 2 38.1 2 8

Attapeu 67.8

Phouvong 5 62.1 3 3


356392AT52 | 2 | 1 | December 2019

Table 3: Description of Stakeholders with a Role in CBSRStakeholders Beneficiaries Role in implementation Role in EvaluationChildren aged between 4 yearsand 8 months and 5 years and11 months

Primary No, participants at Centres Observed during classroomand site visit

Underage and overage childrenof non-pre-primary age

Unintended No, participants at Centres Observed during classroomand site visit

Parents and communitymembers

Secondary Involved in the set-up, running and support ofthe Centre, the Facilitator and the children

Interviewed in focus groupdiscussion

CBSR volunteers (communityfacilitators)

Primary Day-to-day management and implementation ofthe CBSR programme

Interviewed one-to-one and asparticipants in wider focusgroup discussion

Primary School Head Teachers Secondary Supervising and supporting CBSR Facilitatorsand providing the basis for transition to pre-primary after 2 year of UNICEF intervention


Village Education DevelopmentCommittees (VEDC)

Secondary Helping to select the CBSR Facilitators, findingspace for the Centre to operate, mobilisingparents and communities around theprogramme, and maintaining ongoing support tothe Facilitator and Centre


District Education and SportsBureaux (DESB)

Secondary Helping to engage with, mobilise and selectbeneficiary communities, supporting therecruitment of CBSR volunteers, and conductingregular monitoring of the Centres

Interviewed one-to-one or assmall group of DESBrepresentatives

Provincial Education and SportsService (PESS)

Secondary Supporting participating Districts, particularlywith monitoring of the Centres, report-back tothe Central MoES, and mobilising additionalsupport where required (e.g. for any candidatesapplying for upskilling)

Interviewed one-to-one or assmall group of DESBrepresentatives

The Minister and Ministry ofEducation and SportsDepartments, includingDepartment for Early ChildhoodEducation, Department ofFinance, Department for TeacherEducation, Department ofPlanning, Department ofOrganisation and Personnel,Research Institute forEducational Science, InclusiveEducation Centre, EducationStandards and QualityAssurance Centre

Secondary Overall responsibility for the implementation ofEarly Childhood Education in Lao PDR and forachieving targets to increase access and qualityof ECE provision. This includes collaborationwith development partners to mobilise fundingand technical assistance to redress current gapsin these services. The Ministry creates the legaland political platform for uptake of CBSR andthe mandate throughout its Departments andgeographical Offices to support the programme.Furthermore, the Ministry is responsible formonitoring the initiative and its outcomes, inorder to effectively target its resources andplanning for the future.

Representatives of differentdepartments interviewed one-to-one, with support from theDepartment for EarlyChildhood Education andUNICEF in setting upinterviews

Other concerned Line Ministries,such as the Ministry of Health

Secondary No direct responsibility, except insofar as thesame children and communities impacted by theCBSR programme are also beneficiaries that aretargeted by these Ministries. Collaboration andintegration of ECD services is a collectiveagenda across Ministries.

Not involved directly

Lao’s National Assembly and theHead of State, the President

Secondary In service of Lao’s citizens, the NationalAssembly has the responsibility for assuringperformance of its Line Ministries in deliveringequitable access to quality education for all.

Not involved directly

Development Partners, includingPLAN International, Save theChildren, the World Bank

Secondary Fulfilling their organisational mandate and withaccountability to their constituents, thesepartners are engaged as peers in the delivery ofearly childhood education in Lao PDR.Specifically, there is collective bargaining powerin pursuit of such strategic activities as thedevelopment of the ECE Policy, as well as peer-to-peer mobilisation of support and funding forspecific initiatives such as CBSR. PLAN


356392AT52 | 2 | 1 | December 2019

Stakeholders Beneficiaries Role in implementation Role in EvaluationInternational, for example, is now implementingthe CBSR model directly in some areas.

UNICEF Funder As the funder and implementing agent of theCBSR Programme, UNICEF has overallresponsibility for mobilising partners,Government and communities, supporting theirtraining and implementation activities, andproviding resources for the ongoing delivery ofthe programme. UNICEF is also responsible formonitoring the programme and providingfeedback on its findings to influence the nationalGovernment of Lao ECE agenda.

2.6 Developing a CBSR Programme Theory of ChangeThe Theory of Change (ToC) is normally a tool used in the early design phase of a programmeto rationalise and justify the activities planned, and thereafter as an evaluation tool. However, inthe case of the CBSR programme, a dedicated ToC was not developed. That said, interviewswith UNICEF and central MoES actors during the evaluation indicate a clear recognition that theCBSR programme’s purpose is to achieve access for marginalised and disadvantaged groupsof children, and in doing so contribute to an outcome of increased rates of ECE access, and inturn achieve the impact of greater opportunities for children to learn and be ready for formal basiceducation.

Therefore, given the programme is still ongoing, with potential for scale-up in future, thisevaluation has served as a springboard for the design of a ToC going forward. The validationworkshop on 23 August (refer to the Agenda in Annex C) was used to gather the ‘owners’ of thefuture of CBSR, and some early proposals for a ToC were discussed via a participatory, group-work process. This process could viably be continued with UNICEF’s support and enhanced if thedecision is taken to scale up CBSR in future. The Evaluation Team’s initial work on a draft ToC,the presentation delivered at the validation workshop, and participant feedback during the groupexercise at the workshop, which would feed into further development of a ToC, are attached inAnnexes D to F. The Evaluation Framework (Annex G), which was developed for the purposes ofthis evaluation, would also be further developed to link to the ToC and to provide a tool for ongoingmonitoring and baseline/end-line impact evaluation.


356392AT52 | 2 | 1 | December 2019

3 Evaluation Objectives, Scope andMethodology

3.1 Evaluation ObjectivesThe purpose of the evaluation, as outlined in Section 2.1 above, is underpinned by the followingkey objectives:

1. To assess the relevance, efficiency, effectiveness, sustainability and equity of the UNICEF-supported CBSR programme.

2. To provide recommendations for further improvement of MoES’ relevant (sub)sector policies,plans, strategies and operations of the CBSR programme interventions for scaling up,including that in the next ESDP 2021-2025.

3. To provide strategic guidance to UNICEF in redefining its focus area of support to MoES formainstreaming of the CBSR Programme under the remaining half of the Country Programme(CP) and the next CP periods.

3.2 Evaluation Criteria, Questions and FrameworkThe Evaluation Framework is attached in Annex G. Against the DAC criteria, it provides:

● The questions raised by the assignment Terms of Reference (ToRs)● Tailored evaluation questions interpreted by the evaluation team● Primary data collection methods● Potential secondary data and literature sources● Indicators and critieria

3.3 Evaluation ScopeThe evaluation focused on:

1. The regulatory framework surrounding CBSR, including GoL-UNICEF Countryprogrammes 2012-2016 and 2017-2021; and relevant sector policies, plans, andstrategic and legal documents of the GoL and the MoES;

2. The implementation period of the CBSR Programme supported by UNICEF: 2013 to2018;

3. The sub-sector of Early Childhood Education (ECE) across a broad geographical areaand local to national stakeholders;

4. Selected districts that received direct support from UNICEF over the 2013-2019period; and

5. CBSR programme interventions supported by UNICEF including communitymobilisation, curriculum contents and materials, training for CBSR facilitators andPrincipals, incentives for CBSR facilitators, monitoring and coaching support, CBSRmanagement, and planning and supervision by DESB and PESS.

3.4 Rationale for Evaluation DesignThe Evaluation adopted a process evaluation design and used a mixed methods approach. AProcess evaluation was agreed to be most appropriate given the evaluation questions, the stage


356392AT52 | 2 | 1 | December 2019

of implementation the programme was at, and the evaluability of the programme, in particular thelack of baseline data. The Process design enabled qualitative explanatory data to be collected todetermine whether the CBSR programme is being implemented as originally intended, whichstakeholders are benefitting and what perceptions of CBSR stakeholders have. United NationalEvaluation Group (UNEG) Norms and Standards for Evaluations were followed. The rationale forthe design approach, agreed with UNICEF included the following:

· The evaluation was expected to be formative, informing ongoing implementation andfuture planning, and, as outlined in the Inception Report by the evaluation team:

o the timing was “too early to look at impacts”.o the evaluation was “not designed to provide an in-depth analysis of outcomes”,o the evaluation would adopt a “consultative approach”, ando CBSR centres would “be visited to explore the day-to-day implementation”.

· In the technical proposal we described how we would undertake a “case study approachto build a ‘rich’ and in-depth understanding of the workings of the programme, rather thanfocusing on wide, shallow, representation through sampling across all types.” This is inline with UNICEF sponsored guidance that “best use of human-centred data is achievedby employing qualitative methodologies to analyze and interpret qualitative data in waysthat contextualize, specify, complement … quantitative research strategies. It isimpossible to quantify everything that is important for human-centered research.”26

· The commencement of the Evaluation in late May 2019 placed pressure on the datacollection phase, since all primary schools close at the end of May and many of the CBSRtoo. Therefore, the sample was necessarily small, purposive and focused.

· Agreement, according to contractual limits, was for the two research teams to collect dataover a 2-week period, which necessarily limited the scope of the primary data collectionand encouraged an approach that limited movement between locations.

The specific questions the evaluation sought to answer are included in the evaluation frameworkin Annex G. To respond to these questions, a mixed method approach was selected:

● In response to the Evaluation ToRs that specified the importance of applying DACcriteria to this work;

● For the ability of mixed methods to enable triangulation of different types of datasources;

● To gather different perspectives and experiences of various stakeholders;● To enable the direct observation of the evaluators of the main ‘object’ and independent

verification of monitoring reports from the Government and UNICEF;● And, finally, to accommodate the limitations of the contractual package.

The approach incorporated:

● desk review of secondary literature and databases,● interviews with key informants,● focus groups at community level, and● quantitative data analysis – including of available costing data.

3.5 Data Collection MethodsDespite limitations (see section 3.8), a variety of secondary and primary data have been collected.The mixed methods approach enabled deep-dive qualitative data collected through this

26 Innocenti Working Paper: Literature Review on Qualitative Methods and Standards for Engaging and Studying Independent Children in the Developing World.Stuart C. Aitken and Thomas Herman. UNICEF/Innocenti Research Centre, May 2009. IWP-2009-05.


356392AT52 | 2 | 1 | December 2019

evaluation to be triangulated with both the primary data collated during UNICEF’s Baseline Surveydata (available in May 2019) as well as secondary literature covering key topics such as policy,costing, and other community-based models. Data collection methods included:

● Compiling and reviewing relevant secondary documentation/data: Of twelve maindocuments (or groups of documents e.g. selected monitoring reports) shared byUNICEF, and available in English, those that were particularly useful to this evaluationincluded:– The Interim Strategy and Costed Action Plan for ECE (2018-2022), October 2017– Costed Action Plan for ECE: Analytical Report, August 2017– The Concept Note on Community-Based Play– The Final Mid-Term Review Report, December 2018– Data and draft report of the Baseline Study commissioned separately by UNICEF

● Conducting visits to CBSR Centres: The evaluation adopted a purposive samplingapproach and selected CBSR Centres that were operational and, due to the contractualtiming, available to visit in June 2019 i.e. still open and running. This still enabledevaluation questions to tackle ‘without’ information (by asking the treated groups their‘before and after’ perceptions) that pertains to the context of communities where ECEservices are unavailable. The evaluation team conducted interviews and structuredobservations in 8 Centres27 involving a range of respondents. This included:– Key informant interviews with: Provincial Government representatives, District

Government representatives, female representative of the Village EducationDevelopment Committee, Head of the Village/Community where the CBSR Centreinitiative was active, Head Teacher of the overseeing Primary School, and the CBSRFacilitator/s

– Focus group discussions with additional parents and community members of thebenefitting village where feasible

– Observational data focusing on the infrastructure and site, as well as learningsessions in action

● Additional key informant interviews: In addition to the key informant interviewsconducted at provincial, district and community levels, a series of interview tools wereutilised to gain central Government and key development partner perspectives. Twelveorganisations/departments, including UNICEF itself, were interviewed, with a total of 19respondents.

● Drawing upon the UNICEF-commissioned Baseline Study: Reviewing the reportand relevant data to understand the ‘current’ picture of CBSR Centres – the Baselinecovered 28 Centres that are newly receiving programme support – and to compare withthe Case Study approach. Furthermore, reviewing any data from Pre-Primary classesthat were formerly CBSR Centre areas, although being mindful that such data haslimited value given the very small sample size and lack of ‘control’ parameters.

● Gaining wider stakeholder feedback: It is expected that a validation workshop will behosted by UNICEF Lao PDR on 23 August 2019 to provide feedback on the first draftof this Evaluation Report. The report will then be refined and improved, but, moreimportantly, the workshop itself is expected to lead to greater awareness of the potentialscale-up and mainstreaming of CBSR based on the evaluation’s recommendations.

27 The Evaluation, with UNICEF Lao PDR approval, aimed to achieve saturation with ‘rich’ (i.e. qualitatively deep) rather than ‘thick’ (quantitatively numerous)data.


356392AT52 | 2 | 1 | December 2019

3.6 Sample Selection for Primary Data Collection in CBSR CentresPurposive sampling aimed to encompass the following characteristics:

● a representative geography by selecting both northern and southern provinces anddistricts,

● inclusion of Centres falling into different ‘types’, namely those still operational as CBSRCentres versus those that had successfully transitioned to become public pre-primaryclasses at Primary schools,

● inclusion of Centres at different ‘stages’ of support, including either recently finishingtheir UNICEF support phase, versus those still at an earlier stage of receiving support.

In addition to exploring the quality and emerging outcomes of programmatic activities in Centresand communities, the Evaluation also sought to examine the strategic aim of transitioning CBSRCentres to government ownership as pre-primary classes. Since the original input from UNICEFwas formally agreed to be two years until the Government obligated itself to take over, one of theroles of the evaluation teams was to question respondents about any common circumstancesunder which Centres have successfully transitioned to become government pre-primary classes,and to discuss with relevant District Authorities where Centres did not continue. Therefore, whenselecting the evaluation sample, it was essential to choose areas where such transition hasoccurred. Data from UNICEF Lao PDR indicated that only in Laongam and Ta-oy (in Salavan28

Province) did some CBSR Centres successfully transition to become pre-primary classes.Therefore, when selecting the southern Province and District/s that were included in the fieldwork,Ta-oy was prioritised to enable discussions with stakeholders involved in the transition to beundertaken. In addition, in Mai (Phongsaly Province), certain CBSR Centres were scheduled totransition to pre-primary in May 2019, so, with the District officials the Evaluation Team tried todiscuss the transition status and plans.

The selection of the sample is illustrated in the table in Annex H and further detailed in a full listof the 98 programme Centres in Annex I.29

In agreement with UNICEF visits to CBSR Centres that had closed down were not arranged.However, one example of ‘unknown status’ was included, in Phia, where the overall case studyapproach and questions were applied, so as to avoid the possibility of the community perceivingthemselves as being examined for failure.

3.7 Data Sources and Methods of AnalysisAs outlined above, a wide variety of data sources were utilised to ensure accuracy, to collatedifferent perspectives on the object of the evaluation, and to enable overall triangulation toovercome certain data limitations. In addition to secondary sources, a comprehensive set ofprimary data was collected from stakeholder interviews and focus groups discussion (seeAnnexes J and K). The rationale for selecting these stakeholders included:

● Ensuring that all levels (central, provincial, district, community) of programmeintervention and benefit would be included;

● At each level triangulating across different experiences and viewpoints by includingseveral of the same ‘type’ of respondent;

28 Salavan –may also be spelt Saravan or Saravanh29 It is useful to note the background to the sample selection: the Evaluation Terms of Reference indicates that since 2013 the programme has established 98

CBSR centres. The Baseline Study commissioned by UNICEF Lao PDR visited and verified 28 Centres newly in operation. In contrast, the full list ofCentres shared by UNICEF Lao PDR indicates a total of 31 Centres newly in operation and continuing until 2020, and a further 54 Centres in operationeither as transitioned pre-primary classes or as Centres completing their UNICEF intervention in 2019.


356392AT52 | 2 | 1 | December 2019

● Attempting to surpass forms of reporting bias both by including respondents of differingaccountability to the programme results, as well as by including several questions in theinstruments that would cross-verify responses;

● Ensuring that aspects of gender and rights issues could be explored.

Based on district and community interviews content analysis was used to determine theoccurrence of themes and specific content.

We undertook a Teschian30 inductive approach to analysing our qualitative data: interviewtranscripts were deconstructed, segmented and divided into recurring headline themes thatemerged from the transcripts themselves. Weaker, ‘emergent’ sub-themes were also identified.Each was aligned to the most relevant evaluation question, with ‘connectors’ to other secondaryevaluation questions where these could be justified by repetition in the transcripts and resonancewith the known research evidence around ECE. Quantitative data and secondary sources werethen sought to triangulate conclusions.

3.8 Limitations of the Methodology and Data CollectionThere were a number of limitations, which were also highlighted as risks during the InceptionPhase of the Contract but were impossible to completely mitigate, and have thus affected thedata collection phase, as outlined in Table 4 below.

Table 4: Challenges and potential implicationsChallenge Description Implication & ResponseLack of baselinedata

There was no baseline assessmentundertaken at the start of UNICEF’ssupport to the CBSR Programme in 2013.A baseline across UNICEF’s 12 targetdistricts conducted in 2019 has onlycovered 28 ongoing CBSR Centres.

Lack of baseline (quantitative) data hasnecessitated a more qualitative approach toassessing evidence of change. It was notpossible to assess robustly the ‘impact’31 ofthe CBSR Programme. As indicated in theToRs, the evaluation looked only at assessingeffectiveness, and making recommendationsfor improvements to an ongoing programme.

Unreliable data Due to reliance on government monitoringand reporting, which does not yet officiallyresource for tracking ‘community-basedplay group’ data collection, and which,furthermore, relies on an EMIS systemthat is still paper-based and not yetdigitalised and streamlined (although thisprocess is now ongoing in 2019 withUNICEF support), the data on CBSRCentres is somewhat conflicting andunreliable.

The selection of the evaluation sample forprimary data collection relied upon dataprovided by UNICEF Lao PDR, but this data islimited in scope and thus weak as a basis forestablishing the parameters of a datacollection exercise.

Difficult timing There was a very tight timeframe forconducting the fieldwork. Within one weekof official commencement of the Contract,the inception report (with data collectioninstruments) was due, and then anadditional week later approval shouldhave been provided to enable datacollection planning to commence.Furthermore, the fieldwork itself could nottake place any later than the last two

Although coordination with UNICEF Lao PDRand the Ministry of Education and Sportsfocused on selecting from the cohort ‘known’to be functioning at the time of the datacollection, nonetheless the evaluation teamsencountered closure situations. The teamswould, in such cases, make strong efforts toensure interviews with the majority ofconcerned stakeholders were still captured,with the support of DESB Officers in managing

30 Tesch R.R. (1990) Qualitative Research: Analysis Types and Software Tools31 As recommended by E. Stern et al in Broadening the Range Of Designs And Methods For Impact Evaluations (April 2012), we are defining ‘impact’ here as

long term effects demonstrated to be uniquely attributable to the project. Effectiveness is as defined by the OECD DAC: the extent to which the project’sobjectives were achieved.


356392AT52 | 2 | 1 | December 2019

Challenge Description Implication & Responseweeks of June, to ensure that at leastsome of the sites would have operationalfacilities for the evaluation teams’observation.Therefore, within 5 weeks of commencingthe Contract, all primary data collectionneeded to be complete. The timingresulted in many Centres being closed atthe time of visits, and key stakeholdersmissing for interview.

this. The main gap in such instances,therefore, was in being able to observelearning sessions in action, which alsoinhibited observations of the relationshipbetween the CBSR Facilitators and thechildren, and the performance of the CBSRFacilitators in delivering the curriculum. In fiveout of eight cases, learning sessions couldnot be observed, and in two out of eightcases, the CBSR Centres had been closeddown and the former CBSR spacescommandeered for other uses.Therefore, combining limited time and poortiming with the limits on resources for theteam structures, the primary data collectionwas necessarily narrowed and has beentriangulated as much as possible with datacollection among central MoES Departments,as well as secondary sources, such as theBaseline Survey.

Geographicalscope stretchedlimited resources

The scope of geographical and case studycoverage needed to ensure, at minimum,the opportunity for very different contextsto be examined. Therefore, the teamswere split between a Northern and aSouthern province. However, this spreadthe already-limited resource even thinner.Budget limitations and programmedesign, therefore, restricted the feasiblesize of samples.

More time was consumed with the realities oftravel, as well as the difficulties therein –especially in Phongsaly where largedistances between sites and climaticconditions (rains) impacted access. Timeavailable for the actual primary data collectionwas thus reduced and some of the respondentgroups unavoidably small. Drawing robustconclusions from the quantitative data is notpossible, except for some recently availableMoES-UNICEF Baseline data. Therefore,both quantitative and qualitative data hasbeen triangulated.

Perceptions ofcommunityrespondents

Involvement of the MoES, Provincial andDistrict Government in the selection andvisiting of CBSR Centres was crucial froma protocol standpoint and to ease accessto the communities.

It is necessary to consider that this may havestrengthened the likelihood of reporting bias,due to respondents feeling ‘under watch’ andunable to report negatively.

Translation andinterpretation

Translation has been a major burden onthe small team resources available. Thecontractual allowance for translation wassmall and the majority of the translationneeds were fulfilled by one member of theteam given the need for capacity to handlecomplex interpretation and documenttranslation. The assumption that CentralGovernment staff and project staff wouldbe capable of speaking good English wasincorrect.

The Evaluation Team translated the questionsthemselves. This results in potentiallyinaccurate results due to variance inquestioning. At central level, the requirementfor interpretation also impacted on the flow ofquestioning and ability to delve deeper intospecific areas of interest or concern.

PotentialEvaluation Teambias

The primary data collection wasadministered by Lao team members.

There may have been differences in howrespondents answered questions that couldhave been incorrectly interpreted by the teammembers.

Presence ofotherinterventions atthe Centres notmapped

Programmes other than CBSR may behaving an effect which the Evaluation didnot have resources to delve into.

The evaluation questions aimed to delve intopotential other sources of support byquestioning about the facilities of the Centreand equity issues, but reliance is onrespondent recall.


356392AT52 | 2 | 1 | December 2019

3.9 Ethical Considerations32

UNICEF’s 2015 Procedure for Ethical Standards in Research, Evaluation, Data Collection andAnalysis33 was followed throughout:

1. all interviewees were volunteers and informed of their right to refuse to answer any questionor to withdraw from the interview at any stage;

2. personal names were not recorded;3. names of departments or villages were recorded for data analysis purposes and could be

specified in the final report.4. information on actions to assurance personal data privacy were given before the start of

each interview.

At village level only those with specific responsibility within the community were invited to attenda Focus Group Discussion (e.g. the chair of the VEDC). However, the Chair of the VEDC couldinvite parent representatives and other community representatives to attend. There was nocoercion involved in selection of participants. A consent form was read out and allowed to besigned, giving consent for questions to be asked about CBSR, but no names were recorded.

Children were not included as interviewees.

32 https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_Ethical_Standards.PDF33 https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_Ethical_Standards.PDF

Ethical Conduct Statement in District Interview Forms

UNICEF nominated Cambridge Education to conduct an evaluation of the Community-Based SchoolReadiness Programme. MoES has included community-based early childhood education programmesinto its ECE subsector of the ESDP (2016-2020) as a way of providing access to more children,particularly those in hard to reach communities. We are gathering information to support the Governmentof Lao in implementing its strategy for community-based ECE.

Consent

Please ensure that the participant has given verbal consent by having him/her read the followingstatements. If they do not respond to these statements, you may not proceed with the interview.

Your name will not be recorded as part of the research. Do you accept to be interviewed?

Yes / No

The name of the department will appear in our Evaluation Report. Do you confirm you understand andaccept this?

Yes / No

Signature:

https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_Ethical_Standards.PDF

https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_Ethical_Standards.PDF


356392AT52 | 2 | 1 | December 2019

4 Evaluation Findings

In this section, our findings are presented against the core evaluation objectives, namely:

1. To assess the relevance, efficiency, effectiveness, sustainability and equity of the UNICEF-supported CBSR programme.

2. To provide recommendations for further improvement of MoES’ relevant (sub)sector policies,plans, strategies and operations of the CBSR programme interventions for scaling up, includingthat in the next ESDP 2021-2025.

3. To provide strategic guidance to UNICEF in redefining its focus area of support to MoES formainstreaming of the CBSR Programme under the remaining half of the Country Programme (CP)and the next CP periods.

To highlight stakeholder perceptions, quotations from evaluation interviewees are included. Afuller selection of quotations grouped according to DAC criteria is provided in Annex L.

4.1 Evaluation Question 1

Is the CBSR programme relevant, effective, efficient, sustainable and equitable?

4.1.1 Relevance

Responses from over 80 different respondents at central,province, district and community levels resulted in 80% positiveconsensus that the CBSR programme is relevant to Lao PDR’sneeds and to both MoES policies and to the GoL/UNICEF CountryProgramme.

When CBSR started in 2013, it aligned with the Education SectorStrategy 2011-15 agenda to increase access to ECE in remoteareas. These areas do not have primaryschools nor the short-term prospect of anyformal centre-based ECE, due to operationaland/or financial blockages.

The early success with the CBSR pilotprompted the revision of Lao PDR’s Law ofEducation (2015) to include new and differenttypes of ECE provision (see Box 1), includingplay groups/ community-based ECE.UNICEF’s efforts to demonstrateinterventions that were affordable andflexible and had community leadership attheir core has clearly been relevant andhighly influential at central government level.

The relevance of the CBSR programme atthe community level was also confirmed, bythe use of the demand-led approach: onlythose communities perceiving the need forthe programme opt in. The DESB performs a

Evaluation Government Interviewee(2019): “CBSR is MoES mandatewhich is clearly stipulated inEducation Law amended in 2015with the aim to promote childrenliving in rural, remote andmountainous areas to get access toeducation with special priority forthose villages with no kindergartenand pre-primary class.”

Box 1: Different Types of ECE Provision in LaoPDRAlthough Kindergartens are primarily governmentrun, they are predominantly in urban areas andexpensive. They also comply with different normsand standards. For example, whereas a publicKindergarten should have at least 16 eligiblechildren, pre-primary classes should have at least20 children of appropriate age (5 years old) andshould be in areas where kindergartens are notavailable. Pre-primary classes are a middle ground,which are now the focus of the government, as theyallow the Government to utilise existinginfrastructure, normally on the same site as aPrimary School. Establishment of community-basedplay groups in contrast require at least 10 children,and these fall under the supervision, though notnecessarily the same physical location, of a PrimarySchool Head Teacher. This enables many of thepoorest, most remote communities to receivesupport from the public education system.


356392AT52 | 2 | 1 | December 2019

first ‘cut’ in terms of selection by analysing the demographics of their catchment and pre-selectingthe communities/villages that:

● do not have access to kindergartens or pre-primary classes,● are remote and containing ethnic minorities and/or high rates of poverty,● are ‘stable’, and● have an existing VEDC.

From there, VEDCs and communities are given an opportunity to opt in or out, opting in requiringcollective commitment to providing sufficient children, providing a space for the Centre to operate,and running a selection process within their village to identify an appropriate volunteer(s) tobecome the Facilitator(s).

Other key examples of the general consensus on the relevance of the CBSR programme include:

Government recognition· In Salavan province, the Government confirmed that an

expansion agenda is already factored into future plans andthat by 2020 they intend to phase out CBSR and ensure thatall children have access to ECE through pre-primaryclasses.

· In Phongsaly province, the Government highlighted that theprovincial education plan is targeted to increase access for5-year-old children in rural remote areas to at least 60%.

· District government officers expressed concern about theirtargets due to lack of resources, which cemented therelevance of the CBSR programme in plugging the gap forchildren 2 to 5 years old in some villages. There wasparticular emphasis on the ability of the CBSR programmeto introduce children to Lao language and thus improve theirtransition into primary school.Community value

· Community members indicated the value in having theirchildren attend CBSR, so as not to be “illiterate like me”.

· Parents noted the engagement of their children in reading,writing and singing, and saw the impact of this on theprogress of their children into primary school.

· Parents also saw the potential for the CBSR programme toenable them to work on their farms. All the communities inthe northern case studies made explicit mention of thecapacity of the CBSR to alleviate their childcareresponsibilities and enable them to progress their own work.

· The majority of communities also made explicit mention of the rationale for selection oftheir village for the programme i.e. because they had shown interest in engaging with theprogramme, because they as a community were providing in-kind support, and becausethey had enough children to attend the Centre.

Evaluation GovernmentInterviewee (2019): “Thedivision of ECE into its owndepartment shows theprioritisation going to thissub-sector. We know that ifchildren don’t passkindergarten then they arelikely to repeat Grade 1.”

Evaluation GovernmentInterviewee (2019): “ECE ismandated by Law and it isimportant for children toaccess it with their ownfriends and within theircommunities. This serviceneeds to be accessible,especially for workingparents.”

Evaluation ParentInterviewee (2019): “[As aresult of participation inCBSR]....she can play gamewith her brother at home.She sings and dance athome. She can read andwrite simple words in Lao.”


356392AT52 | 2 | 1 | December 2019

Community in-kind support· In GnangTai the villagers, including non-parents, are already

providing portions of rice to support the Centre’s facilitatorsand they express confidence that they will continue theinitiative. In fact, 55% of villages surveyed stated that theyalready provide rice to CBSR facilitators. However, twocommunities were so poor representatives stated that theycould not find these extra resources for facilitators.

Nonetheless, findings also showed:

· That the CBSR model currently emphasises transition to pre-primary, since this is the agenda of the GoL, but in doing sothere is no formal consideration for the 2, 3- and 4-year oldsalso accessing CBSR Centres and their route if pre-primaryclasses are successfully established;

· While the current criteria for selection ensure successful takeup and community support, they risk excluding communitiesthat may be most in need and not fulfilling the criteria forinvolvement as a result of their remoteness, poverty, lack ofawareness, etc.

Overall, the relevance of CBSR has been well evidenced. The CBSRprogramme is aligned with the policy context and meeting real needsby plugging current gaps in services. The shortcomings of the CBSRprogramme are not signs of low relevance but that the programme iscapable of becoming more relevant – see recommendations inSection 6.

4.1.2 Effectiveness

The overall CBSR ‘package’

Based on field observations by the Evaluation Team and accounts provided by interviewees, wecan conclude that the CBSR package is effective in delivering better ECE outcomes, developingschool readiness, encouraging basic skills and improving acquisition of Lao language among 4–6 year olds. A full technical review of the elements of the package is provided in Figure 3 belowand Annex M.

Evaluation Community-levelInterviewees (2019):

“The village was selected forCBSR because many childrenare not prepared for primaryeducation, our village is readyto support CBSR to thechildren to learn and preparethem for primary education.”

“We had consultation meetingwith parents who havedaughter/son completed uppersecondary school and do nothave a job. We explained tothem the objectives ofprogramme and need forfacilitators to teach children.After that parents asked theirdaughter/son if she/he wantsto be facilitator. Then villagesent the name of volunteers toDESB. The volunteer signedagreement with the village andwith DESB. Then sentvolunteers to attend trainingcourse on teachingmethodology organised byDESB, supported by UNICEF.There are two facilitators in thisCBSR centre, one girl and oneboy.”


356392AT52 | 2 | 1 | December 2019

Figure 3: Full technical review


356392AT52 | 2 | 1 | December 2019

Stakeholder responses on the technical elements of the CBSR package

According to key departments of the MoES at central level, thepartnership between UNICEF and the GoL to pilot the CBSRprogramme has directly resulted in development of an ECEcurriculum specifically tailored towards play groups and multi-ageteaching. The Research Institute for Educational Science (RIES)stated that “RIES is heavily involved in CBSR – we develop theECE curriculum for different groups of children”. Elaborating on this,one respondent confirmed that they had been involved indeveloping the guidelines for ‘Caregivers’34 which were adapted toensure ease of understanding and a more directive approach forindividuals of lower educational qualifications. Such materials didnot exist prior to the CBSR programme: the CBSR Centre pilot hasbeen instrumental in highlighting other effective content andpedagogical approaches that can be adopted.

Within the CBSR programme, the push during training towardsusing local materials to enhance and build upon the basic kits(provided by UNICEF – see Annex N) reduces reliance onexpensive and hard-to-get materials – although more rigorousmonitoring of this aspect of the programme would be needed toproperly assess whether CBSR Facilitators are indeed putting thisinto practice. Across the sample used in this evaluation it was onlypossible to observe 3 learning sessions, which does not allow for confident conclusions aboutuse of local materials.

60% of the communities visited by the evaluation team highlighted the importance of their CBSRFacilitator, indicating their growth in skill and capability in relation to delivering early learning. In40% of communities desire to see Facilitators given an opportunity to progress further in theirstudies and their teaching career was also emphasised.

Furthermore, the training and funding support provided by UNICEFfor DESB officers to monitor CBSR Centres four time per year, hasbeen effective from the perspective of community, VEDC andPrimary School recipients (as highlighted above), as well as fromthe perspective of key departments (Department of Panning, RIES,etc) of the MoES who require the information and data that isgenerated. In 100% of community visits, respondents recognisedthe existence and role of monitoring.

Building on these early successes in bringing community-basedearly learning formally on to the Government’s agenda, the MoESare now also working with UNICEF on the refinement of standardsfor play groups, which would take the government a step closertowards a formalised mandate for monitoring this service, as thisdoes not currently exist.

What has not been measured, but is a probable outcome, is that CBSR has opened updisadvantaged communities to the potential benefits of education and a demand for educationin the future. As children involved in the CBSR programme attend ECE instead of accompanying

34 Different terms are used for the volunteers working in play groups – during interviews, ‘caregiver’, ‘facilitator’, ‘volunteer’, and occasionally ‘teacher’ were allused interchangeably.

Evaluation Interviewee(2019): “The objective was tocreate an affordable, minimumpackage. We also provide aminimum incentive toFacilitators. The purpose is notto create a burden onsustainability. Through theteacher training given toFacilitators we emphasise theoptions for making use of localmaterials, and reports we havefrom DESBs indicate this to belargely successful. We alsosee many contributions fromVEDCs/communities.”

Evaluation Interviewees(2019):

“DESB monitored teachingtechniques of facilitatorsevery 3 months. DESBsupported facilitators toovercome problems thatfacilitators are faced.”

“VEDC come visit CBSR oncea month. VEDC providedadvices to facilitators and ifthere are some needs forimproving CBSR VEDCprovided support as they can.In addition, VEDC helpencourage and mobilise thevillagers/parents to send theirchildren to school.”

Evaluation Interviewee(2019): “We have seen that thechildren passing the CBSRhave good learning in pre-primary classroom. Theylearned from CBSR how tospeak Lao, write and read Laolanguage and when they are inpre-primary and grade 1 it iseasier for them to learn”


356392AT52 | 2 | 1 | December 2019

their parents to the fields, some child labour will have been reduced. However, as there is still arelatively high absenteeism rate (20% according to Baseline Study data) it has not solved theproblem completely.

According to various central interviews with key informants, a dedicated ECE Policy, which is asyet unpublished, as well as a parenting guideline that is also under development (unpublished),have the potential, once ratified and formally recognised, to enhance the resourcing ofcommunications and community engagement.

Learning outcomes

In terms of results of learning, and ‘readiness’ for primary school, the lack of baseline at thecommencement of the CBSR programme has prevented a full impact evaluation, but acombination of secondary sources nonetheless give an indication of the effectiveness of CBSR.The recent Baseline Study commissioned by UNICEF Lao PDR has produced results that providea useful proxy. This study has discovered that ‘student achievement levels are very low, which isgenerally consistent with previous student assessments in ECE as well as early primary gradesin Lao PDR. It should be restated that the 12 target districts are rural and remote, include somethe poorest areas in the country, and have many families that do not speak Lao at home. So lowresults should not be a surprise.’35

Further statistical analysis performed by this Evaluation Team36, led to the finding that expansionof all forms of ECE (kindergarten, pre-primary and CBSR) across the country has coincided withpositive change in key primary education indicators:

● A national increase in enrolment of 3-to-5-year olds up to 77.6% in 2017-18;● An associated incremental improvement in promotion rates from Grades 1 to 2, and 2

to 3, up to 85.5% and 92.6% respectively in 2017-18;37

● A national decrease of repetition and dropout rates in Grade 1 down to 8.4% and 6.1%in 2017-18 respectively.

Such findings are consistent with a wealth of international evidence that quality ECE can result inimproved primary school performance:

‘Students with preschool experience in most cases demonstrate higher scores on literacy,vocabulary, mathematics, or quantitative reasoning than non-attenders.’38

The linkages between low rates of pre-primary enrolment and poor learning outcomes are alsoclear – in OECD’s Program for International Student Assessment (PISA) results in 2012, evidenceshowed that:

‘15-year-old students who reported having attended pre-primary for more than one yearscored 53 points higher in mathematics (the equivalent of more than one year of schooling) thanstudents who had not attended pre-primary’.39

35 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 2936 Two members of the evaluation team gathered existing secondary data sources and met multiple times with the EMIS Department, to perform light-touchanalysis to determine enrolment ratios in ECE, new entrance into Grade 1 of primary school with preschool experience, net intake ratios in primary education,the cohort completion rate in primary education, and repetition rates. The team utilised data from: the UNICEF baseline survey, GoL EMIS, UNICEF costedstrategy document, UNICEF database, District Statistics Officers, and http://www.dataforall.org/profiles/laoeduinfo/37 Caution: new promotion guidance from MoES may affect data.38 Aglaia Zafeirakou, 201839 Report of OECD 2014, 12, quoted in Crouch, L. & Merseth, K.A. Stumbling at the First Step: Efficiency implications of poor performance in the foundationalfirst five years. Prospects (2017): 2

http://www.dataforall.org/profiles/laoeduinfo/


356392AT52 | 2 | 1 | December 2019

Ineffective elements of the programme

There are aspects of the CBSR programme where levels ofeffectiveness could be improved:

i. Although it is clear that overall access to early learning isincreased by the CBSR programme, there are still manyremote communities not receiving these services, and themodel of intervention is yet to be adapted to access the‘hardest to reach’ of these communities. This is succinctlyhighlighted in the Costed Action Plan:‘The distribution of the population imposes constraints onreaching ESDP targets through exclusive reliance on theCentre-based delivery model (Kindergarten and Pre-primary). Official norms forestablishment of new classes and local differences in access to existing school facilitiesresults in administrative, logistical and financial challenges40 to providing the currentmodel to children in more remote and less densely populated communities. ReachingESDP participation targets requires reaching many more of these children.’41

ii. Both the ongoing support to and development of Facilitators, as well as the ongoingmonitoring support provided by Districts, may not be sustainable under currentplanning (see Section 4.1.3). This will impact on the long-term effectiveness of theprogramme.

iii. There are also some results that are more difficult to measure, such as the influence ofthe programme on attitudes. The Baseline Study found that 40 -50% of parents see‘support’ as providing some form of learning support for their children, whereas the otherhalf perceive ‘support’ to be purely financial and practical (refer to the chart below).42 Inaddition, the Baseline Survey highlighted that although parental literacy had a big impacton the treatment of parents towards their child’s learning – being almost twice as likelyto support their child – there was still only a 44% engagement rate43 according toBaseline results.44 Therefore, although parents and community representatives aremore aware of the importance of education and the part they have to play in ensuringthat their children have access to quality pre-primary education, there is much more todo to prompt parents to turn awareness into action..

iv. Staffing of ECE is problematic45, with a number of trained teachers unable to berecruited to staff ECE classes, due to limited teacher quotas that are set at centrallevel. As highlighted by the Department of Personnel, in 2019 the quota was 936, butover 12,000 more teachers are needed overall. If the GoL were able to immediatelyloosen these quotas in targeted districts (such as those most in need on UNICEF’s CPranking – Sekong and Salavan at the bottom right now), then teachers would be able tostaff new pre-primary classes when CBSRs are closed to transition to village pre-primary classes. It is worth noting that even if quotas were ‘relaxed’ some teacherswould not want to be re-located to more remote communities, particularly if they did notunderstand the local language.

40 Administrative challenges applying the norms for establishment and for age requirements (3-5 for Kindergarten and 5 years of age for Pre-primary) may notpermit establishment of accessible services in some communities. Areas where existing school infrastructure is not accessible to ECE age children willcontinue without service. Even in cases where establishment norms can be met, the current model will often require both construction of a new classroomand funding of a new teaching post for small numbers of children-significantly increasing the cost per student.

41 Dewees, A., Inthasone, S. and Loizillon, A., Costed Action Plan for ECE: Analytical Report (2017): 1642 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 4143 The engagement rate refers to the percentage of parents who help with homework by gender, language group, SES quintile and district.44 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 4045 In 2017 the MoES released a report on results of a national survey into teacher availability, deployment and salary status. This report is only available in Lao

language and thus not reviewed under the scope of this evaluation. However, it merits deeper analysis in the context of wider rationalisation of theeducation sector workforce and consideration of how to manage shortages of capacity, especially within the ECE sub-sector.

Evaluation Interviewee(2019): “There were alsocases where the communitywas too small to justify ourintervention. It is true that astrategy is needed for thesemost-scatteredcommunities. Perhaps it issafe to say that we arereaching the harder toreach but not the hardest toreach i.e. the last mile.”


356392AT52 | 2 | 1 | December 2019

Overall, however, the CBSR programme is proving effective in terms of getting children living inrural and disadvantaged communities into some form of ECE. Overall enrolment rates of childreninto ECE, not disaggregated across types of ECE, have increased across the country withapproximately 225,000 children aged 3-to-5 accessing some form of ECE (constituting 58% ofthe overall 3-to-5 year old population) (EMIS 2018-19). This access allows 5-year-old children,who may not have Lao language as their mother tongue, to be better prepared for entry into Grade1 of Primary School as they understand and can express themselves in Lao language with theirteachers.

4.1.3 Efficiency

Assessing the efficiency of the CBSR model has been hampered by the lack of reliable andaccessible data on ECE and on CBSR specifically. The GoL captures almost no data on thelatter, with the exception of ad hoc monitoring reports, and when asked for data about the CBSRCentres, UNICEF Lao PDR produced some Government monitoring reports (see Annex O).UNICEF also provided a spreadsheet listing the CBSR centres involved in the programme but thelist contains various conflicting or confusing information, which had to be further discussed andelaborated to enable the evaluation team to utilise it (see outcome of these discussions in AnnexI).

In the case of wider sectoral data, information about children with disabilities is not captured,nor the numbers of children out of school, nor is data disaggregated between kindergartensand pre-primary classes, while CBSR is generally excluded altogether. Overall, this has madeanalysis, particularly of costs, very difficult. Insofar as data has been sourced, the CBSR modelis deemed very broadly efficient, but with sustainability concerns that link to the affordability ofsome key areas of the programme, if external assistance is removed.

Assessing individual cost elements of CBSR

The costs of infrastructure development,46 provision of food, and ongoing maintenance of learningspaces are inputs largely handled by communities, which lends far greater efficiency, flexibilityand affordability to the CBSR model, but obviously also reduces the reliability of the fundingsource. Nevertheless, respondents during data collection indicated appetite and willingness tobear some of these costs and provided examples of doing so already – see quotations fromcommunities in Annex L.

The main additional costs, which must ultimately be borne by GoL, and which are currentlyprovided by UNICEF Lao PDR, are:

● Developing and distributing teacher and learning books and guides (which are includedin the basic kits that UNICEF provides)

● Provision of basic materials kits – perhaps on a three-year rolling basis (in the currentCBSR programme only one kit is provided because the UNICEF support only runs for2 years)

● Monitoring CBSR Centres – four times a year for DESB, twice a year for PESS, andonce a year for MoES (according to support provided by UNICEF under CBSR)

● Provision of a stipend to Facilitators on a monthly basis

The estimated costs of underpinning inputs are:

46 Refer to Annex B which presents the criteria for community selection to be part of CBSR, and which highlights the need for communities to provide viablespaces


356392AT52 | 2 | 1 | December 2019

Description Unit CostBasic kit of materials (including story books andteacher guides)

LAK 2,050,000 per kit

Training of volunteer Facilitator LAK 4,000,000 per FacilitatorTraining of Government Technical Officers/Officials LAK 4,620,000 per representative (covering more than

one Centre)Stipend for volunteer Facilitator LAK 200,000 per month (LAK 1,800,000 for 9 months

of the programme)

The data collected for the case studies of the evaluation highlights much optimism amongrespondents from both government and the community about the capacity to take on the runningof CBSR Centres, with the exception of covering the ongoing cost of paying Facilitators. Over80% of communities expressed concern about their ability to levy the funds from its members forFacilitator stipends. However, this is a signal of the wider context of financing of ECE in Lao PDR,particularly in remote and disadvantaged communities, which is inadequate for the need.Communities should not bear the full brunt of costs for ECE for their children – this being an issueof equity and sustainability, rather than an issue of efficiency.

The average national annual wage in Lao PDR is USD 1060, whereas Facilitators are only gettingUSD 195 (9 months). While this reflects the circumstances of their context and community life, itis nonetheless well below average. UNICEF Lao PDR deliberately kept the Facilitator stipendvery low during design, so as to keep the costs of the CBSR Programme affordable and minimiseperverse incentives. However, 14% of CBSR Centres have closed down since the original startof the programme in 2013, which in turn constitutes over a third of the Centres that have not beenreceiving UNICEF support for more than 2 years (i.e. the original cohorts), which indicatesconcerns about the affordability of CBSR Centres (particularly stipends) for communities wherethere is no external assistance given by either Government or another party.

In addition, since efficiency will determine sustainability, it is worth exploring whether similarresults could be achieved by adapting the model further to reduce costs. Therefore, rationalisationof key aspects of the model could be trialled e.g.:

· Although not directly comparable, volunteers on a school readiness programme inTanzania47 have been able to produce impressive results with only 10 days of training(5+5 days), a small set of 16 story books and a very minimal toolkit to produce teachingaids. What this evidence suggests is that a reduction in the starter training package couldbring a layer of efficiency to the programme.

· Furthermore, currently the costs of District monitoring are relatively high, so a reductionin the number of DESB visits while increasing more localised supervision and networking,using the VEDC and nearby primary school head teacher, could produce another level ofefficiency.

Overall, the costs of the CBSR programme are difficult to compare directly with Kindergarten andpre-primary models, and a judgement on overall efficiency is not confidently made by thisevaluation.48 Furthermore, regardless of the exact costs entailed, comparisons also need to factorthat ‘kindergarten or pre-primary expansion to more remote areas may be infeasible or may haveso few students as to raise the cost per student considerably – to a level higher than the estimate

47 School Readiness Programme in 9 regions of Tanzania (see Annex P for more details)48 The Costed Action Plan for ECE: Analytical Report (2017) commissioned by UNICEF attempts to make such comparisons, but the factors that push the model

beyond comparability lead to lack of confidence by the evaluation team in the resulting conclusions. For example, whereas it is relatively easy under theCBSR programme to assess the cost of training community Facilitators, and the associated orientation and training of government and communityrepresentatives, these costs are less visible in the case of pre-service training for Kindergarten and Pre-primary teachers and ongoing monitoring activitiesof relevant Government departments.C


356392AT52 | 2 | 1 | December 2019

for the community-based model.’ 49 So, not only should CBSR be assessed in terms of itsefficiency as a standalone initiative, but it must be factored that ‘incorporating the community-based model as a regular government funded and managed service…provides more flexibility inaddressing the operational constraints concerning how to reach children in areas wherekindergarten and pre-primary are impractical and/or infeasible.’50

The CBSR programme (and other community-based ECE programmes) are an essentialcomponent of education provision, and not a luxury if GoL is aiming for equity across the country.Without significant investment in ECE to implement the ‘centre’ based ECE model of pre-primaryclasses and kindergartens for all, then the community-based model should be put on an equalfooting in terms of Government funds with the other ECE models.

4.1.4 Sustainability

The broader context: is the environment enabling?

An enabling environment for public service delivery can be summarised into the followingessential factors:

· Political and legal frameworks· Institutional platforms· Financial backing· Human resource capacity· Accountability mechanisms· User demand and perceptions of value

In the case of the CBSR programme, many of these essentialelements are missing or weak. Therefore, although there ispolitical will at certain levels and demand within communities, LaoPDR’s education system is not yet fully enabling, even though theCBSR model itself has the potential to be sustained with relativelylight-touch support.

The International Monetary Fund (IMF) recently released theirstatement resulting from consultations with Lao PDR, whichprovide a useful context for projections about the early childhoodsector. [IMF] Directors highlighted that spreading the benefits ofeconomic growth would require consistent investment in humancapital. Improving education attainment, skills training, reducingthe regulatory burden on SMEs, increasing access to finance,easing trade regulations, and introducing policies that supportgreater participation of women in the formal economy would makegrowth more inclusive. The IMF emphasised the need for ongoingtechnical assistance from the Fund and development partners.51

They went on to indicate the priority for investing in people, withparticular mention of the need for increased education spendingand the need to support women to enter the workforce by

49 Dewees, A., Inthasone, S. and Loizillon, A., Costed Action Plan for ECE: Analytical Report (2017): 2550 Ibid,51 International Monetary Fund Executive Board Completes the 2019 Article IV Consultation with Lao P.D.R (August 2019):

https://www.imf.org/en/News/Articles/2019/08/08/pr19315-lao-imf-executive-board-completes-the-2019-article-iv-consultation-with-lao-pdr

Evaluation Interviewee(2019): “PESS do not havecapacity to allocate budget forthis issue. PESS has plan toincrease community-basedECE, but it is difficult to provideteachers because of limitedquota of teachers.”

Evaluation Interviewee(2019): “Yes, our communitycan continue CBSR afterUNICEF stop support. But wedo not have stipend/salary forthe CCG (facilitator). We havelabour, local materials, in-kindsupport from community. Weneed technical support fromDESB.”

Evaluation Interviewee(2019): “We want to continue.But we know that it will be verydifficult for us to afford theCBSR because our people arevery poor and they do not havework to do to earn the money.”

Evaluation Interviewee(2019): “If community peoplelack understanding andawareness of the preparation ofchildren for their readiness toenter primary school theprogramme will not besuccessful.”

https://www.imf.org/en/News/Articles/2019/08/08/pr19315-lao-imf-executive-board-completes-the-2019-article-iv-consultation-with-lao-pdr


356392AT52 | 2 | 1 | December 2019

“providing affordable childcare and promoting women’s access to finance.”52

Combining this context, which is being seen across the ASEAN region, with Lao PDR’s nationaldecline in GDP and economic growth, and it is evident that the opportunity for CBSR centres tobe maintained is imperative but on fragile ground. Investment into the ECE sub-sector andspecifically into maintaining ‘play groups’ would rely upon either:

● UNICEF Lao PDR (or another donor) continuing to support CBSRs until 2021 andbeyond (included in a new CP);and/or

● GoL providing funding and support similar to that which pre-primary classes andkindergartens enjoy (i.e. through School Block Grant funding).

It is not equitable to expect that communities alone can sustainlocal interventions into early childhood, but, with sufficient support,parents and communities do play a vital part. However, it will bethrough the ongoing support of UNICEF Lao PDR in planning withthe GoL for phased handover and long-term ECE planning, thatCBSR might become a more mainstreamed part of the earlychildhood sector.

Is the CBSR model itself sustainable?

A closer examination of the elements of the CBSR model itself reveals different aspects that arehighly sustainable, versus some that are less sustainable in current form. These are as follows:

· Materials provided to CBSR centres (story books and teacher guides) are of good qualityand appropriate for the age group. These are accepted and owned by the MoES and thusalready institutionalised.

· The pre-primary curriculum that is used as the basis for running CBSRs has beendeveloped by a government agency (RIES) and thus again is institutionalised already.

· Training is implemented by MoES (DESB officers) and again is owned by the MoES.Given the short duration of training and the ability to ‘outsource’ its delivery within thewider Government network of DESB offices, there is immense opportunity to sustain theinitial development of CBSR Facilitators by the current model. Since CBSR Facilitatorsare not monitored beyond the stipulated 2 years of the CBSR programme, nor yet knownto be offered ongoing career opportunities, however, it is difficult to make a judgement asto the longer-term sustainability of the training approach. Furthermore, as highlighted inthe efficiency findings, the costs of delivery of training could be reduced, which is pertinentin the resource-stretched setting currently surrounding the ECE sub-sector.

· The demand and attitudes of parents and communities that has been highlighted infindings above, are a highly positive influence on the sustainability of CBSR. The BaselineSurvey indicated a 44% engagement rate53, which is low but already significant giventhe remoteness of many of the affected communities. Furthermore, over 50% of villagesprovide direct in-kind support to CBSR Facilitators, even members without children, whichagain indicates implicitly the support of the community for the programme. Theprogramme is increasing awareness of the importance of education and plays a part inthe overall increase in enrolment rates across Lao PDR. In disadvantaged andmarginalised communities, CBSR is able to highlight to community members that theycan play an active role in ensuring that their children have access to quality pre-primary

52 ASEAN and the IMF: Working Together to Foster Inclusive Growth, Christine Lagarde, IMF Managing Director:https://www.imf.org/en/News/Articles/2018/02/25/sp022718-jakarta-MD-asean-and-the-imf-working-together-to-foster-inclusive-growth53 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 40

Evaluation Interviewee(2019): “There is ongoing needfor support from UNICEF forbudget and training. It isimportant to take this initiativeto scale across the wholecountry, because otherwise therisk is that Centres will closedown.”

https://www.imf.org/en/News/Articles/2018/02/25/sp022718-jakarta-MD-asean-and-the-imf-working-together-to-foster-inclusive-growth


356392AT52 | 2 | 1 | December 2019

education. The attendance of children at CBSR Centres also has the positive effect ofenabling parents to spend longer at their work/farming, knowing children are looked after,with a suggestion that this can make communities more productive.

· Monitoring is an essential element of all ECE services,made even more crucial in the case of CBSR Centres due tothe light-touch training and materials interventions. Lone orduo CBSR Facilitators need to be part of a wider network ofaccountability to ensure that young children are safeguarded. However, although moremonitoring is usually helpful, in the context of Lao PDR, where resources are alreadystretched, there must be careful consideration of the appropriate, minimal levels ofmonitoring required by the Government. DESB monitoring of pre-primary classes is 50%less than that adopted by the CBSR programme. The Baseline Study highlights that 50%of pre-primary and primary teachers in UNICEF-MoES supported districts reported theirclassrooms had not (yet) been visited by a Government official during the 2018 to 2019school year, whereas CBSRs indicated levels of monitoring close to programme targets(of 4 visits per year).54 Furthermore, despite the low levels of monitoring of pre-primaryand primary schools surveyed, these levels were still marginally better than for non-UNICEF-supported districts. Therefore, on this continuum, the implication is that lessexternal assistance results in lower levels of government monitoring, which would implythat once UNICEF support for CBSR stops, communities and primary schools under theprogramme may suffer from reduced monitoring support.

· The minimal stipends for community facilitators are a core issue of sustainability. Ashighlighted above, the stipend is very low in comparison to pre-primary and primaryschool teacher salaries, and other minimum wage levels. However, 90% of communitiesindicated they would not be able to support the facilitator with a cash stipend, despitewillingness to support with other in-kind contributions. The technical capacity and financialresource for providing initial training of CBSR Facilitators, as well as some basicmaterials, would also be impossible for communities to provide.

· The future of community facilitators and of the ECE teaching sector as a whole isunder stress. Currently, CBSR Facilitators are managed as a short-term resource, eventhough there are overall shortages of qualified ECE teachers and huge issues withregards to mobilising available ECE teachers to remote areas. The system is not well-equipped currently to meet the demands of the MoES goals for ECE, but simultaneouslythere is no accessible pathway for utilising the valuable capacity that trained CBSRFacilitators are.

· Underpinning the CBSR model is the logic that after 2 years the Centres will transitionfrom being community-owned-and-run to becoming government-owned pre-primaryclasses. However, this transition has not proved highly effective in the currentimplementation of the CBSR programme and there is insufficient phase-out planning inplace. Furthermore, even if the transition process is more effectively managed, theremoval of the CBSR mechanism from within a community creates a void for 3 and 4-year-old children who will not be catered for by the existence of a pre-primary class.Therefore, the concept of transition as it is currently managed within CBSR is potentiallyundermining the opportunities for CBSR to expand and grow to meet service needsthat were not originally intended, but which are clearly present in remote communities.

4.1.5 Equity

It can be seen the results of the CBSR are considerable in terms of individual children, who wouldnot have access to education otherwise. The important outcome of increased proficiency in Lao

54 MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 12

Evaluation Interviewee(2019): “One districtmember of staff cannotcover 100 villages”


356392AT52 | 2 | 1 | December 2019

language, using the CBSR process, not only increases the proportion of children accessing ECEbut importantly, has an effect of reducing repetition and drop out at primary level, which is a keyMoES goal.

Equity is explored in the CBSR programme under the 3 broad categories that were both (i) mostamenable to rapid field observation and (ii) known to have greatest impact on early years learningoutcomes:55

● Language● Disability● Gender

Language

UNICEF and MoES worked collectively to recognise that, despite thePolicy and its specification for all instruction to take place in Laolanguage, many remote and ethnic minority communities areexcluded and falling behind in their education due to a lack of accessto Lao language learning.56

The disadvantaged become even more disadvantaged once theyenter Grade 1, as children from such backgrounds continue to fallfurther and further behind as language blocks them from maintainingpace with children already fluent and teachers who are not trained todeal with mixed language contexts.57 This, therefore, presents anopportunity to use the early learning years as a platform for languageacquisition.

The CBSR programme has the potential more than any otherinstitutional model to transmit effective language developmentstrategies among young children. This is because the Facilitatorscome from local communities and they are bilingual, which enablesthem to switch easily between local language and Lao language, tofacilitate the children’s understanding. This is not so for manyqualified ECE teachers, who lack the relevant language skills for thecommunities most in need of services. Although the EducationQuality Assurance Centre advise “teachers to know both the ethniclanguage and Lao and be able to support children to move into Lao”there remains a shortage of ECE teachers and issues with quotas, which means that remoteareas are even less likely than normal to attract appropriately equipped teachers.

The Baseline Study too supported that for language ‘the benefits of local recruitment are clear,as all of the CBSR facilitators reported being able to speak the local language that is spoken bytheir students. None of the CBSR Facilitators self-identified as Lao-Thai ethnic group.’58

55 Other categories of equity such as ethnicity, statelessness, poverty, and refugee status were not explored due to lack of current data in any systematic form.However,, there are proven correlations between these omitted variables and those that we did explore.56 Language issues hamper learning where children do not have Lao as their mother tongue. ESDP 2016- 2020: Mid-Term Review Report (MoES 2018 p.4 )57 Build capacity and improve skills of teachers to work with children whose first language is not Lao. Recommendation in ESDP 2016- 2020: Mid-Term ReviewReport (MoES 2018 p.5)


Evaluation Interviewee(2019): “Since their childrenattend CBSR they canspeaking Lao language,singing, dancing, readingand writing some words,count, washing hand beforeand after eating. In addition,their children speak withpeople politely, especiallythey respect adults.”

Evaluation Interviewee(2019): “Some communitieslack teachers because theydo not want to go to suchremote areas”

Evaluation Interviewee(2019): “Languagedisconnects thesecommunities further, notleast because it meansthose populations cannotaccess media or print….We try to help teachers… touse both Lao and their locallanguage, and to see thatthis is ok.”


356392AT52 | 2 | 1 | December 2019

Disability

When examining the issue of disability, the findings were less optimistic, despite that the GoLhas put in place an Action Plan for Inclusive Education. Only 25% of the communities surveyedreported any kind of physical issue – one a child with a harelip and one with sight problems. Thesephysical issues were also only pointed out by figures of education authority i.e. the CBSRFacilitator or the Primary School Head Teacher. Community and VEDC members unanimouslyclaimed there were no disabled children in their village, and Province and District educationofficers were unable to provide any data or examples to illustrate levels of disability within theircatchment.

UNICEF’s Baseline Study too found very little reporting of disability,with only 0.4% of children enrolled being (reported as) disabled.Although it is plausible that some of these small communities indeedhave no children with disabilities or special learning needs, there isgreater likelihood that a combination of poor understanding, lack ofscreening and cultural shame are leading to non-reporting. Childrenwith severe disabilities are usually kept at home, and often ‘hidden’to the outside world, due to stigma and discrimination. Socio-economic factors such as gender,ethnicity, poverty and distance to schools play a large part in determining the length of time achild with a disability will stay in school.

Furthermore, respondents from MoES confirmed that there is no dedicated budget to addressissues of disability, no dedicated monitoring of disability, and no dedicated policy or strategypaper. The Lao Disabled People’s Association (LDPA) corroborates this in its 2016 Policy Briefwhich pointed to a lack of statistics and an estimate that around 8% of the population could suffersome form of disability if global projections are an indication. UNICEF’s 2017 LSIS II dataindicates that 2% of children aged 2 to 4 have functional difficulty in at least one domain (seeing,hearing, walking, fine motor, communicating, learning, playing, controlling behaviour). In ruralareas without road, 3.8% of children have functional difficulty in at least one domain. There aredisparities between the North (1.7%) and the South (4.3%). In the Sekong province, thepercentage increases up to 22%.

Although some respondents stated that ‘all’ children are ‘welcome’ at CBSR Centres, there wasrecognition that there are limited training and guidelines available to properly manage specialneeds of children. Despite increased Government efforts to improve the situation of children withdisabilities, including ratification of the UN Convention on the Rights of Persons with Disabilitiesand the promulgation of national laws and legislations in this regard, huge challenges remain intranslating policies and laws for children with disabilities into practice at the community level.

Gender

Finally, with regards to gender, the CBSR programme has several opportunities to improve equity– by enhancing opportunities for young women to access a livelihood; by improving the potentialfor mothers and parents to network; by allowing parents to attend to their daily work; and byenhancing the potential for girls to attend school due to the decreased costs of having a Centrenearby and flexible to the village context. Girls and boys are at parity for enrolment in CBSRCentres with 1049 girls and 1014 boys aged 3 to 5 years old enrolled. Furthermore, more womenare CBSR Facilitators than men – 127 women and 66 men.

Interestingly, very few respondents provided any specific comments on the issue of genderdisparity. Even a gender focal representative from the MoES focused mainly on the disparity inthe gender of senior leadership figures in schools, rather than specifically dealing with girl childrenor community members. Although this does not mean that there is no issue, it does limit the

Evaluation Interviewee(2019): “‘Light’ cases ofspecial needs are factored,but for more extreme casesthere is no specificationexcept that teachers mustbe inclusive.”


356392AT52 | 2 | 1 | December 2019

conclusions that can be reached. There is some evidence, from the recent Baseline Study, ofsome underachievement by girls (see the chart below).

As reported by the Asian Development Bank: ‘there is a need to focus on reducing genderdisparities and vulnerability in remote rural areas that are home to smaller ethnic groups. Thesegroups are at particular risk of being left behind during this period of rapid economicdevelopment’.59

Chart 1: Student Assessment Results by Grade Level: Overall Scaled Score60

Source: MoES-UNICEF Baseline Survey, 2019

4.2 Evaluation Question 2

Should the CBSR programme be scaled up and mainstreamed within the LaoPDR’s MoES plans and operations?

In view of the evaluation’s broadly positive findings, and the ongoing gaps in access to ECE,especially for the most disadvantaged children, we recommend the CBSR programme shouldcontinue, should be rationalised, and should be scaled up as part of the 9th ESDP and next CP.Recommendations for how to achieve this are detailed in Section 6.

4.3 Evaluation Question 3How should UNICEF intervene and support, through its current and upcomingCountry Programmes, in scaling up and institutionalising the CBSR model, and,furthermore, support the overall achievement of increased access to quality ECDservices for all of Lao PDR’s young children, but especially the mostdisadvantaged?

The findings presented above indicate clear requirement for a community-based programme ofECE intervention to continue. The form that this takes is still open to adaptation, but the CBSRprogramme has evidenced a broadly feasible, affordable and effective model. Therefore, anadapted model, in which UNICEF redresses efficiency and sustainability issues in a targeted way,should be implemented under the existing and next CP, and factored in the 9th ESDP. Someproposed adaptations are outlined in Section 6.4 below. Expansion of this adapted and integratedmodel to a wider variety of disadvantaged ‘clusters’ of communities, could be achieved during thenext CP, particularly through more consistent and systematic targeting and monitoring of theintervention to enable ongoing scale up.

59 https://www.adb.org/documents/country-gender-assessment-lao-peoples-democratic-republic


404.6 454.4 482.5542.1

424.5 449.5 485.4550.1

0

200

400

600

CBSR Pre-Primary Grade 1 Grade 2

Female Male

https://www.adb.org/documents/country-gender-assessment-lao-peoples-democratic-republic


356392AT52 | 2 | 1 | December 2019

5 Conclusions

Following the same format as presented in the Findings section above, we now presentconclusions in response to the three main questions of this evaluation, as well as an overallanalysis of the long-list of questions raised in the terms of reference.

5.1 Conclusion Question 1Is the CBSR programme relevant, effective, efficient, sustainable and equitable?

In conclusion, the CBSR programme is:

5.1.1 Highly relevantAreas of Strong Relevance

There is significant evidence for the relevance of the CBSR programme, and for its ongoingrelevance. Primary sources unanimously support the need for the programme, and its alignmentwith GoL priorities and policy. The CBSR programme specifically targets areas where disparitiesexist, which is supported by the MoES Education Law (2015) and its recognition of the need forcommunity-based initiatives. The CBSR programme answers to both the aims of the UNICEF CP:‘The programme will ensure that children, especially the most disadvantaged and in hard-to-reachareas, have access to quality early childhood and basic education services’, as well as to theobjectives of the MoES ESDP 2016-20 to ‘support expansion of education access to rural andremote areas and create favourable conditions for poor and disadvantaged children, girls, ethnicgroups and disabled children to receive good care and be well prepared for their schooling.’

Areas that Undermine Relevance

There are some inherent flaws within the CBSR design that could be addressed to improve theprogramme’s relevance. Firstly, the GoL targets 3-to-5-year olds, not only 5 year olds. However,the CBSR programme has currently been established as an interim ‘bridging’ programme fortransition into pre-primary classes, which would then only serve 5-year olds in any ongoing ECEservices (pre-primary classes). 3 and 4 year old children accessing CBSR (since it is wellevidenced by the Baseline Study that many are61) will cease to have any Centre that they can goto. Furthermore, the assumption that transition from community-based to pre-primary is the‘natural’ and desirable transition to make also fails to consider the unique characteristics of manyof the remote and minority communities currently being catered for by CBSR. As also stated inUNICEF’s Costing Strategy ‘Pre-primary classes can only be established in existing Primaryschools…This may leave children unserved where a Primary school is not accessible and inremote communities may result in small (and expensive) Pre-primary classes with significantnumbers of children age 3 and 4 in the same community remaining without access.’62 Finally, theCBSR programme has not strongly addressed the more holistic elements of early childhooddevelopment needs. The relevance of the programme is reduced by its failure to use the ‘platform’of the Centres as a base for much wider potential opportunities to influence health, WASH,nutrition and parenting agendas.

61 Almost half of the children who entered CBSR centres in 2018-19 were below the official ECE entrance age of 5 years, and most of these children wereyounger than 56 months (4.5 years). In pre-primary classrooms the proportion of under-age children is smaller, but still significant (about 20 percent). - MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report (July 2019): 1062 Dewees, A., Inthasone, S. and Loizillon, A., Costed Action Plan for ECE: Analytical Report (2017): 16


356392AT52 | 2 | 1 | December 2019

5.1.2 Effective but with improvements neededEffectives elements of CBSR

The CBSR ‘package’ has been expertly reviewed by a member of the Evaluation Team anddeemed to be well designed and fit for purpose. The ability of the government and Facilitators touse the CBSR guides and curriculum speaks to a well-tailored, appropriate level of instruction fora range of recipients, many of whom are only qualified to lower secondary levels (in the case ofFacilitators). Furthermore, primary data sources evidenced both government and communityownership of the CBSR programme: “We have observed personally that compared to first-timevisits, ethnic group children are gaining in confidence and their integration is improving, andCaregivers (Facilitators) are doing a “good job” – children like being with their Caregivers.” Levelsof monitoring support provided by VEDCs, Primary School Head teachers, and DESB wereevidently at good levels according both to secondary and primary data sources.

Ineffective elements of CBSR

Currently, the main safeguarding fail-safe for children accessing CBSR is that their communitiesare deeply engaged in the selection of the CBSR Facilitator who will care for the children and inongoing engagement with the running of the Centre. This is a highly effective route towardssafeguarding of children, but it is in not sufficiently capitalised due to the strong reliance on DESBmonitoring and support. Furthermore, and as outlined above, the CBSR programme is notproviding sufficient linkage with support for nutrition, health and screening, which is beingprovided to recognised government schools. Finally, the CBSR programme does not appear tohave been highly successful in training CBSR Facilitators to generate their own classroom andlearning resources using local materials, as there was limited evidence of any such resourcesbeing generated beyond the use of the Basic Kits provided by UNICEF.

5.1.3 Inefficient given over-reliance on UNICEF funding and communitycontributions

Aspects of efficiency

UNICEF Lao PDR has deliberately and consciously created an affordable, low-cost package fordelivery of the CBSR programme. There has been particular attention to the types of basicmaterials provided upfront; the way in which the government have been engaged to deliver oncurriculum and guidance, training of CBSR Facilitators, and monitoring support; emphasis inCBSR Training on the usefulness of local materials for generating ongoing classroom resources;and maintaining very low rates for CBSR Facilitator stipends. Furthermore, responses fromcommunity level have indicated a broad willingness to provide support in-kind for the running ofCBSR centres i.e. in terms of infrastructure, food, and time/support to learning sessions.

Inefficient elements of CBSR

Data on costs and financing of CBSR, and in the ECE sector more broadly, is weak and notsystematically collected or analysed. Furthermore, certain aspects of the CBSR model could berationalised to reduce cost, given the resource-stretched context of ECE in Lao PDR – namely interms of monitoring and training costs.

Confident conclusions about the overall efficiency of the CBSR model, particularly in comparisonto other early learning provision, are not possible to make. Although evidence collected duringthis evaluation hint at the CBSR programme being done at low cost, particularly given the relativeimpact that appears to be achieved, can only indicate a positive hypothesis of efficiency.


356392AT52 | 2 | 1 | December 2019

Nonetheless, the programme’s reliance on external funding and community contributions cannotbe deemed efficient unless GoL factor the CBSR programme in their ongoing ECE-sector-wideplanning.

5.1.4 Unsustainable unless changes are made to redress surrounding system-levelblockagesAspects of sustainability

There are several ingredients of the CBSR programme that enhance its sustainability. Firstly, it isdemand-led – the process for selecting participating provinces, districts and communities is notonly driven by UNICEF Lao PDR priorities and GoL agendas, although they are a key factor, butalso by the process of opting in and committing time/resource to the programme. In the case ofPESS and DESB, this equates to signing an MoU with UNICEF Lao PDR that commits the localgovernment to participate in training, conduct monitoring, and manage a transition of the CBSRinto a pre-primary class after 2 years of intervention. Whereas, in the case of communities, itmeans providing appropriate space for the learning to take place and ensuring that two volunteersare selected to be trained and to run the CBSR Centre.Secondly, current material costs are low and could probably be reduced further. This has theeffect of further reducing the requirement for government investment and allowing CBSR Centresto subsist well with low levels of investment from external sources, including GoL.Finally, the willingness of most communities to provide in-kind support to CBSR Facilitators andCentres indicates both a clear demand, as well as a clear ability to continue to provide some levelof these contributions.

Elements that undermine sustainability

The Teacher Survey (2018) commissioned by the MoES has highlighted huge issues in terms ofteacher shortages in schools, volunteer teaching to ‘cover’ for unallocated but needed positions,and low levels of qualified teacher employment due to lack of quota allowances. Given that thereare no accessible career routes for trained and experienced CBSR Facilitators, there is acontradictory gap in services emerging in needy communities beyond the 2 years of UNICEFCBSR support.Furthermore, the provision of a stipend to CBSR Facilitators, albeit kept at very low levels, isclearly not possible for communities to maintain. Nor can communities be expected to providetraining, teacher books and some basic materials. These resources must be provided by GoL oranother external source, but there are no clear funding streams yet devoted to this in GoL plans.Currently, the CBSR programme is legitimised by the Law of Education and pending ECE Policy,but it is not otherwise a core element of the ECE system and thus fragile in terms of itssustainability.

5.1.5 Improving equity on language but weak in other aspectsElements of equitable provision

Underpinning the objectives of CBSR, for improved readiness and communication skills ofchildren, is a focus on familiarity with Lao language. This has been one of the major successesof the programme. On the issue of gender, evidence emerging fails to present a clear picture, butit is positive that an almost equal numbers of boys and girls have been enrolled at the Centres(1049 and 1014 respectively), and out of the CBSR Facilitators two-thirds are female.


356392AT52 | 2 | 1 | December 2019

Equity concerns

There is a major issue in terms of disability and the very opaque picture right now as to the statusof disability among children, including as part of the CBSR programme. Disability is not trackedby GoL and since UNICEF Lao PDR do not undertake any independent and thorough tracking ofits own, there are no external sources right now for probable levels of disability. This severelyweakens the opportunity to respond to needs and to ensure appropriate levels of focus oninclusion, training and communications around the issue.

5.2 Conclusion Question 2

Should the CBSR programme be scaled up and mainstreamed within the LaoPDR’s MoES plans and operations?

As reported in the 2018 Mid-Term Review report, enrolment rates of children aged 3 – 5 yearsold have increased to 56% (exceeding the 55% target for 2020) and enrolment rates of childrenaged 5 have increased to 77.1%. Additional data from the EMIS Section of the Department forPlanning, MoES, for 2018 – 2019 now places enrolment of 5 year olds at closer to 84% and overallaverage enrolment for 3 – 5 years old at 58%. Nonetheless, there are major disparities betweendistricts due to factors of geography/remoteness, poverty, minorities, language etc. Therefore,across UNICEF CP target provinces, in some Districts, the enrolment rate is less than half of thenational average. This compounds the need for rapid and flexible solutions to meet with theaccess gap.

Furthermore, in a context where Lao PDR’s GDP has been declining, only 2.94% (2014) of GDPis allocated to education, and less than 6% of the education state budget is allocated to the ECEsub-sector, there is also a funding gap to redress. The CBSR programme provides oneopportunity for this given its low-cost and affordable model. Although there are efficiency andequity concerns to redress, these would be tackled through the mainstreaming of the programmewithin MoES plans as they are concerns that arise largely due to lack of system-level take-up ofCBSR and over-reliance on community and UNICEF support.

Finally, there are issues surrounding the number of government pre-primary teachers availablein classrooms. In 2017, the MoES GoL released a report on a national teacher survey into theavailability, deployment and salary status of teachers. Since the report is only available in LaoLanguage, the evaluation team had to rely on interview feedback regarding the findings of thesurvey. For example, senior staff of the Department of Personnel indicated that 12,000 moreteachers are required to fulfil national needs; and staff at the EMIS Section of the Department ofPlanning indicated that out of 66,119 teachers, over 12,000 are ‘voluntary’, meaning they aregovernment trained and working in government schools, but not being paid government salaries.Therefore, even where teachers are available, provincial quotas limit the numbers who can bedeployed and salaried. Furthermore, since 90% of the education budget is already utilised to payteacher salaries, the additional funding required to compensate existing ‘volunteers’, let alone topay for more teachers, is untenable in the current climate. Therefore, CBSR Facilitators are asource of capacity that could assist in plugging the capacity gap within the ECE sector, not onlyas a short-term measure, but also as a means for creating a longer-term widening of careerpathway options within the sector.

Therefore, according to the findings presented above against the DAC criteria, and given theongoing gaps, especially for the most disadvantaged children, the CBSR programme shouldcontinue, should be rationalised, and should be scaled up as part of the 9th ESDP and next CP.Recommendations for how to achieve this are detailed in Section 6.


356392AT52 | 2 | 1 | December 2019

5.3 Conclusion Question 3

How should UNICEF intervene and support, through its current and upcomingCountry Programmes, in scaling up and institutionalising the CBSR model, and,furthermore, support the overall achievement of increased access to quality ECDservices for all of Lao PDR’s young children, but especially the mostdisadvantaged?

As already highlighted, the proposed adaptations to CBSR and recommendations for UNICEF’sintervention are outlined in Section 6 below. In conclusion, UNICEF should continue to interveneand support CBSR through its current and upcoming CP. This support should be targeted, trackedand adapted continuously. Furthermore, the CBSR model should be reviewed in terms of itsinherent design flaws, particularly with regards to the linkage between CBSR Centres and Pre-Primary classes, and with regards to CBSR Centres as holistic hubs for wider ECD services.


356392AT52 | 2 | 1 | December 2019

6 Recommendations

The starting point for the recommendations in this Section lies firmly within the conversations atnational, district and village levels, interviews with UNICEF personnel and through discussion atthe validation workshop held in Vientiane in August 2019. As per the agenda in Annex C, theobjectives of the validation workshop were:

● To present the CBSR formative evaluation findings in relation to relevance, efficiency,effectiveness, sustainability and equity of the MoES/UNICEF CBSR programme; and

● To share and draw comments and feedback from key stakeholders for furtherimprovement of the programme among MoES counterparts and other key concernedagencies.

The 15 government workshop participants were invited, through group work, to review, judge andcomment on the draft recommendations. Their responses are presented in Annex Q and it is onthe basis of this feedback that the recommendations below have been refined.

The overarching recommendation of this evaluation is to:

Continue, with improvement, to implement, scale-up and mainstream the CBSRprogramme, which can continue to meet the needs of disadvantaged, remote and non-Lao-speaking communities. In doing this, the overall aim should be for the CBSRprogramme to become a part of the ECE system, to complement existing ECE services,and to be a part of the MoES’s budgeting and planning cycles. Furthermore, the mandateof the CBSR centres as educational centres could be expanded to encompass widerECD and community needs, or similarly complementary initiatives could be attached toachieve this.The recommendations below elaborate this overarching recommendation to contribute to learningand knowledge sharing on equity-focused programming within the present UNICEF CountryProgramme (2017-2021) and the next. The aim is to ensure not only that the CBSR programmeis continued and improved, but that it is more feasible for the GoL to take it up without the externalassistance of UNICEF Lao PDR and to enhance the overall sustainability of the approach.63 Therecommendations below, therefore, provide a prioritised, actionable list for elements of the CBSRprogramme that could either continue in current form, continue but be improved, or, should bechanged. In addition, the suggested responsibility of various stakeholders in practically takingforward these recommendations are presented.

63 The recommendations will be refined following the initial feedback of stakeholders (government and development partners) who will gather at a workshop on23 August. The aim will be to guide participants through different group activities to ensure that a wide variety of views and ideas are captured and that theGoL examines more closely what it could commit to as regards taking the recommendations forward.


356392AT52 | 2 | 1 | December 2019

6.1 Summary of recommendations and stakeholder responsibilities

Table 5: Summary of RecommendationsRecommendation Details Impact on

DAC CriteriaResponsible Timing

1. Rationalisethe CBSRPackage

Rationalise the existing CBSRpackage, by:Reducing the number of monitoringvisits by District officialsEnhancing the role of local networksin supervisionReducing the toolkit provided toCBSR Facilitators to essentials only(proposed list provided)Increasing the facilitator-to-child ratioand having one facilitator per Centreunless ratios demand more

Rationalisationof the CBSRmodel willsubstantiallyimprove theeffectiveness,efficiency andsustainability ofthe programme

MOES/UNICEF

Immediatelyfor new CBSRCentres andwithin 1 yearfor existingCentres

2. Providecoordinatedand ongoing

dedicated technicalassistance support forenlarging andenhancing CBSR

Coordinate with all core fundingpartners to plan for immediate andongoing increases in access andquality of ECEPrepare Exit/Sustainability Plan tofacilitate handover to GoL in thelonger term

Thesustainability ofCBSR andother modelsfor youngchildren of LaoPDR will beenhanced

MOES/UNICEF/ otherDev. Partners

Immediately

3. Conduct afull impactevaluation of

the CBSR programme

Review baseline data evidence andassess feasibility of using existingbaseline for overall impact evaluationat endlineDepending on outcome of review,improve ability to determine impact ofCBSR at end line through additionalbaseline data collection and betterongoing data collection

Relevance ofthe programmewill improve

UNICEF Within 1 year

4.Mobiliseand supportlocalnetworks of

supervision anddecreased reliance onDESB for monitoring

To support changes to the CBSRpackage, reduce the reliance oncentrally-led monitoring by:Training VEDC leaders and HeadTeachers to provide supervision,mentoring and monitoring of CBSRcentresMinimising the role of DESB to qualityassuranceStreamline the reporting linesbetween VEDC/HTs and DESB tofacilitate the monitoring process

Localisedmonitoringnetworks wouldimprove theefficiency andsustainability ofthe programme

MOES/UNICEF

Within 18months

5. Supportand enhancedata,

mapping anddocumentation onCBSR and ECE

Review CBSR monitoring reports andfull data to produce a comprehensiverecord since 2013Plan for ongoing and more systematicdata collection, with a focus ontargeted indicators currently missingfrom any other source of dataImprove present pre-primary mappingto include CBSR centres and areas ofno coverage of pre-primary

Improved dataand informationwould assistboth MoES andUNICEF inongoingplanning andimprovementsto ECEnationally, thusenhancing therelevance of theCBSRprogramme

MOES/UNICEF

Within 18months


356392AT52 | 2 | 1 | December 2019


Responsible Timing

6. Developa

communications andengagement plan

Enhance communications to ensurebetter targeting of messages on thecritical benefit to all children, theirfamilies and communities, and societyat large of quality ECE

Improved take-up and demandfor ECE willenhancesustainability

MOES/UNICEF

Within 2 years

7. Improveprocess of5-year-oldtransition

into Primary Grade 1

Enhance and formalise therelationship between CBSRFacilitators and Grade 1 Primaryteachers in the nearest primaryschoolPrepare Primary Grade 1 teachers forreceiving a diversity of children

Assistingchildren withthe transitionprocess willimproveeffectiveness,equity andsustainability ofthe programme

MOES Within 2 years

8. Pilot theexpansion ofCBSR

Centres to act as ECDHubs and review theoverall ‘model’ ofCBSR to redressinherent gaps

Review the model of institutionaltransition and rethink the assumptionthat CBSR Centres are only a‘bridging’ (temporary) mechanismExpand CBSR model to includeWASH, nutrition and health activities,parental education

Enhancing ahub approachwill improve allaspects of theprogramme:relevance,effectiveness,efficiency,equity andsustainability

UNICEF Within 2 years

9. Conductdeep-divestudies toenhance

understanding ofequity issues affectingECE access andquality

To build upon recommendation 5,also conduct deeper studies into thebarriers and exclusion factors that arecurrently blocking young children fromaccess to quality ECEProvide recommendations for theongoing rationalisation of the CBSRmodel, or other, different solutions toensure better access to quality ECEfor disadvantaged and marginalisedchildren

Outcomes ofsuch studieshave potentialto improveequity

MOES/UNICEF

Within 2 years

10. AddressECEFacilitator

and Teacher careeremployment blockages

Explore a range of options forincreasing the human resourceavailable for ECE services,particularly in harder to reachcommunitiesEnsure that those already providingECE services have access to careerpathways to enable them to remain‘useful’ members of the ECE sectorRecognise and celebrate the potentialfor the CBSR programme and similarinitiatives to enhance equity in harderto reach communities, due to thelivelihood options provided to under-qualified individuals and especially towomen

Improvedefficiency,equity andsustainability

MOES Within 3 years

6.2 Detailed Recommendations

1. Rationalise the CBSR PackageAs outlined, the CBSR ‘package’ is already producing effective results, in terms of training take-up and implementation, basic materials use, and perceived progress in children’s learningoutcomes. Therefore, it would be feasible for the existing package to be used as the basis forscaling up of CBSR. However, the findings also evidence the opportunity to improve the packagethrough small changes. Proposed changes (detailed in Section 6.4 below) include:


356392AT52 | 2 | 1 | December 2019

● Reduce the number of monitoring visits by District officials to be aligned with currentGovernment norms for primary and pre-primary schools (i.e. 2 visits per year)

● Enhance the role of local networks in supervision through training of Head Teachersand VEDC members, rather than relying on District officials as the main monitoring body

● Reduce the basic toolkit provided to CBSR Facilitators to essentials only (remove itemsthat could be generated by the CBSR Facilitator using local materials (e.g. puppets) oranyway found in the local community (e.g. musical instruments); also capitalise on othersources of funding for the kits e.g. items such as soap and a towel should ideally formpart of linked WASH initiatives for ECD

● Review the norms and standards for teacher/facilitator-to-child ratio and consider theproposal to have 15-to-17 children per facilitator/teacher, which would generally reducethe requirement to one facilitator per Centre (instead of two)

2. Provide coordinated and ongoing dedicated technical assistancesupport for enlarging and enhancing CBSR

In the period while the GoL is fundraising and planning for expansion of ECE and mainstreamingof CBSR, children of 3, 4 and 5 years old cannot wait. The CBSR programme is covering only afraction of the real need.

Development partners need to coordinate with each other and with the GoL to massively scale-up and systematise the response to ECE access and quality, with particular emphasis on remoteand disadvantaged communities.

This could be achieved through a dedicated ECE programme of support, with longer-term,sustainable handover at a reasonable point in the future that enables the GoL to plan for full take-up of services. Such assistance could also focus on how non-State actors can play their part in afuture where all children are accessing good ECD.

3. Conduct a full impact evaluation of the CBSR programmeOne of the major limitations in conducting this present evaluation was the lack of baseline data toenable an impact evaluation. Now that UNICEF have commissioned a Baseline survey coveringall of the CP 2017-21 districts64, this creates a springboard for monitoring the outcomes of theprogramme in terms of learning, but UNICEF Lao PDR will need to cross-check whether theBaseline has covered all the necessary indicators to enable a comparison of:

a. Relative improvement of learning performance of children, disaggregated by gender andethnicity, over the course of their attendance at the CBSR centre;

b. Rates of transition and retention once/if those children transition to Primary Grade 1;c. Performance of the children relative to children who either don’t attend any form of ECE,

or who access either pre-primary classes or kindergarten;d. Other indicators of ECD i.e. access to WASH, nutrition and health support, and any

noticeable improvements in these aspects that could then be linked with improvements(or not) in learning outcomes.

The lack of such data currently hampers UNICEF Lao PDR and GoL from providing evidence forthe recommendations for scale-up and expansion of ECE, and in particular the expansion of

64 The UNICEF Country Programme 2017-2021 has seven priority provinces where UNICEF’s support and interventions across sectors, including education, willconverge. Within the seven provinces, for the first two years the programme (2017-18 and 2018-19 school year) education intervention activities arefocusing on three provinces (Phongsaly, Saravane and Savannakhet). Attapeu was added in the 2018-19 school year as part of the flood emergencyresponse and recovery support. The remaining three priority provinces (Bokeo, Huapanh, and Xiengkhuang), which include six districts, will be integratedinto the programme during the second phase of interventions. Within each province the two districts were selected on the basis of key education indicators,with a focus on early childhood education (ECE) and basic education levels, to identify the most disadvantaged districts. – in Cambridge Education |MoES-UNICEF Lao PDR Education Programme Baseline Survey 2019: Final Report: 5


356392AT52 | 2 | 1 | December 2019

CBSR. Ultimately, to convince decision makers, the benefits of quality ECE need to be clearlyspelt out and costed, including:

● Proficiency in Lao by those children who do not speak Lao at home and the impact thisis likely to have on achievement by grade 2 and 3.

● Reduction in repetition at grade 1 – a key aim for MoES.● Reduction in drop-out rate – another key aim for MoES.● Parents spending longer at their farms, knowing their children are looked after, making

communities more productive, while reducing child labour.● Children arriving at grade 1 at the correct age and ready for formal schooling.● Grade 1 teachers being more positive and competent when teaching with a language

diverse class.● Disparities between urban and rural communities reduced, leading to improved social

cohesion.

4. Mobilise and support local networks of supervision and decreased relianceon DESB for monitoring

One of the aspects of rationalisation of the CBSR package (recommendation 1) is to reduce thereliance on centrally-led monitoring by the DESB. The current CBSR model emphasises theformal, centralised route of government(district)-led monitoring and data collection. However, bothprimary and secondary data, points to the current stretch that Provinces and Districts areundergoing in managing their monitoring and evaluation responsibilities. Except where UNICEFsupport is enabling higher levels of input, the Baseline provides evidence of much lower levels ofmonitoring input. As ECE expands, the pressure will also expand, but not necessarily the humanand financial resource to meet it. Furthermore, where more accessible areas are not receivingmandated levels of monitoring support, the story will be grim for institutions based more remotely,such as will be the case for the communities and areas that the expansion needs to reach.

Therefore, it is recommended that although government monitoring should continue and increasewhen resource allows, this could be bolstered by building networks of localised accountabilitymechanisms i.e. involving Primary School Head teachers, VEDCs, and parents themselves. Thiscould be achieved by:

● Training VEDC leaders and Head Teachers to provide supervision, mentoring andmonitoring of CBSR centres;

● Minimising the role of DESB to a quality assurance function; and● Streamlining the reporting lines between VEDC/HTs and DESB to facilitate the

monitoring process e.g. through the use of technology or specific communicationplatforms.

5. Support and enhance data, mapping and documentation on CBSR and ECE

The current levels of data collection and associated record-keeping surrounding the CBSRprogramme specifically, but also to a large extent in the wider ECE and education sector,demands improvement.

It was difficult to obtain clear data on the status of all the CBSR communities involved in theprogramme, particularly those that had received support more than 2 years ago. Furthermore, itwas not possible to obtain clear costing information to enable comparison of CBSR, pre-primaryand kindergarten models. Finally, the lack of disaggregated data for different ECE models furtherhampered comparisons.


356392AT52 | 2 | 1 | December 2019

To redress the information and data gaps, some proposed steps include:

● Perform a dedicated archiving exercise that reviews CBSR monitoring reports and fulldata to produce a comprehensive record since 2013

● Introduce regular record-keeping tools for more systematic data collection of simplifiedbut useful indicators, with a focus on targeted indicators currently missing from any othersource of data

● Improve present pre-primary mapping to include CBSR centres and areas of nocoverage of pre-primary – although at present mapping does exist showing main ECEsites across the country; CBSR centres are not featured in this, and, furthermore, theclusters of children who are not served by any ECE provision are also not captured onthe map. Therefore, using data collated in the Baseline Survey and widening to any newdistricts/communities that CBSR reaches, the existing mapping could be enhanced togeolocate all CBSR Centres on the Lao PDR country map. A fuller picture of where alltypes of ECE provision currently are, which could then be cross-analysed with data onchild populations and likely non-served children in this context, would provide a usefultool for planners and can generally be achieved at relatively low cost, and thenmaintained with appropriate training of Government personnel.

6. Develop a communications and engagement plan

During the evaluation, the team were given access to a number of highly useful documents thatare as yet unpublished, or in certain cases documentation that would be expected that did notexist (or were not shared) e.g. costed action plans for the expansion of ECE, standards forcommunity-based school readiness, and a communications plan. In the case of communicationsand community engagement, a significant push should be applied, not only for CBSR but for ECEmore widely. The CBSR programme has the potential to support the use of GoL’s strongcommunications networks and options to greatly enhance overall awareness of the importance ofECE but also the opportunity for communities to come together to improve the options for theirchildren. Basic standards, stimulation techniques, nutrition and health messages, could all formpart of national communications, to simplify messaging around ECD and empower parents andcommunities to act. This should form an integral and much greater of the CBSR programme goingforward.

7. Improve process of 5-year-old transition into Primary Grade 1

Although the current CBSR model already creates an accountability relationship between thenearest Primary School and the CBSR Centre, a direct peer-to-peer relationship between theCBSR Facilitator and the Primary Grade 1 teacher is not formally built in to the training and themodel of delivery. Furthermore, there is no evidence currently to track children transitioning fromCBSR centres into Grade 1, to assess the numbers successfully transitioning, and then to tracktheir ongoing performance. Therefore, this recommendation is provided on the basis of globalevidence and practice.

It is suggested that a formal relationship between the CBSR Facilitator and Grade 1 teacher ofthe affiliated Primary School is established, with the following potential benefits:

· To improve the transition experience of CBSR children into Grade 1;· To enhance the opportunities for both CBSR Facilitators and Grade 1 teachers to develop

professionally and to modify their respective classroom practices;· To ensure greater levels of mentorship support for the CBSR Facilitators.


356392AT52 | 2 | 1 | December 2019

It could also be an option for training to VEDCs and Primary School Head Teachers to beexpanded to include the Grade 1 Teacher, and/or for Grade 1 Teachers to attend part of theCBSR Facilitator training and for the relationship between the two parties to feature in the trainingdelivery, with a focus on the transition of children from ECE to Primary Grade 1.

8. Pilot the expansion of CBSR Centres to act as ECD Hubs and review theoverall ‘model’ of CBSR to redress inherent gaps

Currently, the CBSR programme is designed so that GoL agrees to ‘transition’ CBSR centres topre-primary classes after 2 years of UNICEF support. However, this could lead to closure of theCBSR, which leave communities without any service. This indicates the need for MoES andUNICEF Lao PDR to improve the planning and support to ensure that CBSR Centres continueuntil pre-primary classes are definitely established.

In addition, MoES and UNICEF Lao PDR should review the model of institutional transition andrethink the assumption that CBSR Centres are only a ‘bridging’ (temporary) mechanism. Although5 year olds might be served better by pre-primary classes, this does not cater for 3 and 4 yearolds, who could continue to be served by CBSR centres, since many children of these ages arealready attending the centres.

Furthermore, the overall benefits of having an ECD ‘hub’ within every village, which the CBSRcentres could create a platform for, is also lost if they are only seen as a temporary mechanism.Bringing 3-to-5 year olds together, just at a critical time in their cognitive, socio-emotional andphysical development would pay dividends in terms of the country’s human capital. This alsogives the GoL the opportunity to treat CBSR Centres as wider ‘hubs’ for the holistic needs ofchildren’s development and the community at large – including, for example, nutritional support,vaccinations, screening for disability, parental awareness and education, and early stimulation ofyoung children.

Therefore, it is recommended that CBSR Centres are treated as a long-term solution, as opposedto a short-term solution that bridges until alternatives become available. Due to geography, lackof roads, ethnic status, language, lack of primary school and poverty, many communities cannot(and may continue not to) be served by the ‘centralised’ ECE model of kindergarten and pre-primary classes.

9. Conduct deep-dive studies to enhance understanding of equity issuesaffecting ECE access and quality

More data is required to create a picture of the needs and problem spots in terms ofinclusion/exclusion across Lao PDR. This will need review, with MoES, on indicators toincorporate into national data collection and EMIS. However, in addition, a number of studiescould be commissioned by UNICEF or other partners to create a deeper understanding of thecomplexities and specific barriers affecting excluded children in the context of Lao PDR. Waitingfor data is not advisable – inclusion must be heightened on the agenda and efforts made to beginto tackle it within CBSR. Rapid, deep studies into key inclusion concerns, such as disability, couldbe undertaken to identify specific recommendations for ECE programming in the context ofdisadvantaged, remote communities and low resource settings.

10. Address ECE Facilitator and Teacher career employment blockages

Currently, there are:

1. Insufficient numbers of qualified early childhood education teachers;2. Insufficient quotas for placement of existing ECE teachers;


356392AT52 | 2 | 1 | December 2019

3. Lack of accessible pathways for paraprofessionals, such as CBSR Facilitators, tobecome upskilled and qualified to teach ECE.

Therefore, there is a need to explore a wider, flexible range of options for increasing the capacityavailable to deliver ECE services, particularly in harder to reach and more disadvantagedcommunities. This could commence with a deep-dive review of the current qualifications pathwaysinto ECE and developing proposals for potential alternative (experiential) routes and trainingoptions to open up the sector to achieve the growth of the overall cadre of ECE professionalsavailable.

Qualifications options and career pathways should be tackled in parallel to working with theDepartments of Teacher Education and Organisation & Personnel to address the issues of quotashortages and teacher deployment targets. In the short-term, there is a need to urgently addressthe needs of disadvantaged, under-performing and minority districts.

Furthermore, in the next round of recruitment of community CBSR facilitators, female applicantswith at least lower secondary education should be encouraged to apply, to maintain the existingprevalence of young women in the CBSR programme.

6.3 In SummaryAlthough the CBSR initiative fulfils a clear need with effective interventions that are appropriatelytailored to the context of communities and ECE in Lao PDR, the direct influence over system-level change rests with the Government. To scale up and sustain both CBSR and the widerexpansion of ECD services, there must be a shift in the approach to teacher quotas andmobilisation, a change of career pathway opportunities in ECE, and greater overall financing andresource to the sector to enable effective regulation, monitoring and support (at minimum). Thepart that development partners such as UNICEF Lao PDR can play in this is not only as funderand implementer, but to support overall governance and systems level strengthening, widespreadand systematic expansion, enabling targeting where the need is greatest, and enhancing ofcapacity across all aspects of holistic ECD provision. A whole system approach is needed.

6.4 Proposed Adaptations to CBSR for UNICEF’s 2022 - 2026 CountryProgrammeAs an addendum to the above recommendations, UNICEF Lao PDR requested the EvaluationTeam to produce proposals for adapting the present CBSR model, in light of modifying the currentCP (2017-2021) and the next CP. This request also builds upon the response to Questions 2 and3 of the ToR in terms of whether and how the CBSR should be mainstreamed and scaled up inLao PDR, as well as developing on Recommendations 1, 4 and 7.

Table 6: Proposals for scaling up CBSR

To scale up CBSR within a sustainable and comprehensive costing package, the followingproposals are made:

Action Responsibleorganisation

Training materials

Proceed with existing plans but with a supplement to the Training Curriculum focusedon making Teaching and Learning materials using local resources. Horizontal learningexchanges between community facilitators can quickly promote the use of local resourcesfor making teaching and learning aids; and some adaptation/supplements to present trainingmaterials can help to ensure inclusion of 3 and 4 year old children. Examples of possibleguidance can be found with the Tanzanian School Readiness Programme (Annex P).

UNICEF


356392AT52 | 2 | 1 | December 2019

Training

Keep the number of training days currently provided the same (16 days) but adapt theplan to include at least one cluster meeting for ‘horizontal’ learning amongst communityfacilitators (see below).

It is suggested that two cluster meetings be held, one midway through the training packageand one towards the end, with the presence of Grade 1 teachers for a smooth transition intoPrimary School. Some additional costs may be required to mobilise such meetings. Clustermeetings can also provide the model for enhancing the CBSR model for the next UNICEFCP by selecting communities that can operate as a cluster for improved professionaldevelopment of facilitators and decentralised monitoring.

MoES/ UNICEF

CBSR Toolkit

Reduce the number of items in the kit. Items such as puppets and musical instrumentsare not necessary as these can be made by the community facilitators using local materialsand community input. This would reduce the overall cost of the kit.

UNICEF

Training

More comprehensive training for VEDC and/or Head Teacher instead of just ‘orientation’for 3 days. Training should focus on close-to-centre and eventually cluster-based supervisionand monitoring.

Training for village ‘mothers’ – 2 days training to support community CBSR facilitators withthe village 3 and 4 year old children. Apart from supporting 3 and 4 year old children in theclassroom, these ‘mothers’ could also be learning about early stimulation activities, nutritionand health to be shared amongst other parents to reduce the incidence of stunting. The useof the World Bank multi-grade materials (being trialled in 2019) could be explored, as wellas PLAN’s work with mother groups, and the utilisation of the present UNICEF parentingpackage would bolster this.

UNICEF

Monitoring

Reduce days for DESB to equivalent levels of monitoring of government pre-primaryclasses (i.e. 2 visits per year instead of 4 days per year). Baseline data shows averagenumber of visits to pre-primary classes was 2, and some not even that, so mainstreaming ofthe CBSR into Government systems requires consideration of this. It is assumed that onceUNICEF withdraws its funding, GoL will continue with present support to monitoring, ratherthan expanding to match present UNICEF funding. With a proposed increase in the numberof CBSR sites, monitoring 4 times per year is not sustainable at present budget levels.

MoES/ UNICEF

Community Facilitators

At present there are two community facilitators per 10 children. It is proposed to reduce thisto 1 facilitator per 15-to-17 5-year-olds. In addition, Centres could be formally opened upto 3 and 4 year old children when VEDC also identifies 4/5 volunteer ‘village mothers’ ableto provide rolling support to the Facilitators.

A professional diary should be kept by the community facilitator and could be used asevidence of proof of learning for further study/college entry.

MoES/ UNICEF

Clustering and Horizontal Learning

The 30 planned sites for CBSRs in the present UNICEF Country Programme (CP) shouldbe clustered as 3 or 4 sites per cluster. Clusters would then operate as support, learningand monitoring networks. Experienced community facilitators would be invited for the lasttwo days of initial training of new Facilitators to provide ‘buddies’ for new recruits. Thesebuddies can provide very practical and contextualised ‘tips’ as the new facilitators startimplementing. Much of the learning about making teaching aids would come from cluster-based sessions. It would also be feasible to use DESB officers trained through CBSR to trainnew teams in neighbouring districts, if extending to new districts. UNICEF’s new CP should

MoES/ UNICEF


356392AT52 | 2 | 1 | December 2019

continue to scale up with adapted and extended CBSR or community-based ECEmodel (as trialled during present CP). Clustering would facilitate more horizontal sharing ofideas and materials.

Change of Routine

Reduce the planned face-to-face contact between Facilitators and children down to 4days but extend the morning session to 4 hours per day. On the 5th day (Friday), twiceper month the facilitator could then visit their local primary school, to work with the grade 1teacher and be mentored by the primary head/experienced teacher. Then, on the other twoFridays, when not visiting the Primary School, the Facilitator could prepare activities andlocally made teaching aids, write student records, and record data on absentees.Guidebooks would need to be adapted by RIES accordingly to reflect such a change. Thelast time the guidebooks were revised was in 2015 (so they are due a revision in any case).

MoES/ UNICEF

VEDC

VEDC could be mandated to oversee the identification of 4/5 volunteer village ‘mothers’who, with the support of the community, would care for the 3 and 4 year old children inpreparation of attending the school readiness class.

VEDC could prepare a village map of all 3/4/5 year olds and children with disabilities(and other inclusion indicators). This could also improve the registration of children underfive in the village, family profiles (employment status of parents, number of siblings etc).Regular update of such maps to be mandated e.g. on an annual basis.

Prepare (for wider external awareness/planning) agricultural calendar to map whenchildren are needed by their families for harvest etc.

Develop a health calendar to map when children are more likely to be sick – periods ofmalaria, chest infections/colds etc, which may support better health and nutritional supportto CBSR Facilitators and Centres and have the knock-on effect of reducing occurrences.

MoES/ UNICEF

Primary School Head Teachers

Prepare termly schedules for community CBSR facilitator visits to the primary school andfor school representative to visit CBSR Centre.

Link with the local secondary school to seek out those interested in teaching as a career,to volunteer with CBSR, as work experience, to learn about child development.

MoES/ UNICEF

DESB

Map areas of disadvantage which are without ECE services. Prioritise funds for thesevillages with a link to the District Socio-economic Development Plan.

Monitor CBSR Centres with a quality assurance agenda not just monitoring, as regularmonitoring would be taken over by VEDC and primary school. Close-to-centre monitoringmay also be an equity issue as often harder-to-reach communities lack attention due totransport costs and poor road access.

Analyse reports from Head Teachers and provide feedback.

Identify facilitators for scholarship.

Continue to develop their own facilitation skills for CBSR and VEDC training.

MoES/ UNICEF

UNICEF

Engage an intern/PhD student to work with Education section on an accurate record ofall aspects of CBSR programme from 2013-2019.

Collate all cost records and monitoring records (HT-DESB; DESB-PESS, PESS-UNICEF) and lessons learned at 2015 etc.

UNICEF


356392AT52 | 2 | 1 | December 2019

Ensure better coordination between UNICEF sections – e.g. place emphasis on WASHinterventions when CBSR centres are formed (see Figure 4) below.

Lobby for (and devise mechanisms for) per capita funding (via school block grants) for all5 year old children, including those attending CBSR Centres or other models of ECE.

Create a library of infographics and communication material based on achievements ofCBSR.

Produce a newsletter to exchange good practice and raise status of Community Facilitators(including their success stories).

Support MoES with the provision of Certificates for community facilitators after two years.

Support MoES to review training and progression of community facilitators in light ofbroader workforce requirements.

Increase budget for next CP to allow for integration of ECD/ECE services. Lobby GoLto increase funding for EC services – integrated provision across ministries through atechnical working group. 8th ESDP promised an increase in ECE budget, but this has notbeen achieved. Perhaps a stronger lobby (all Development Partners) for the 9th ESDP mightensure that the message that any investment will be more than justified by a reduction inrepetition and later benefits, such as improved maternal education. Furthermore, thislobbying could be linked to analysis of wider spending and where ECE/ECD budgets arecurrently targeted.

9th ESDP

Explore options for a sustainable financial mechanism to pay facilitator stipendsthrough the primary school. This responds to an issue whereby even if the DESB could fundfacilitator’s stipends there was no mechanism to do so.

Provide per capita SBG to primary school for all 5 year old children in catchment area–not just for those in school.

Allow 3 and 4 year old children to join CBSR, cared for by village mothers.

MoES

Figure 4: CBSR –Community Education Centre

The concept behind the community education centre in Figure 4 above is to provide an integratedprogramme for ECD/ECE as well as a sustainable location for a number of community activities,such as parental education and adult literacy. The village technology centre could be the site for

CBSRCommunityeducation

centre

ParentalEducation

Health

Nutrition

Play basedcurric. WASH

Adult literacy

Climatechange and

riskreduction

Villagetechnology

centre


356392AT52 | 2 | 1 | December 2019

solar power, toilet hardware, water systems, etc. Not only would these activities enliven thecommunity but would ensure sustainability of the CBSR centre, even if/when a pre-primary classis established.

Infrastructure may have to be improved, particularly if it is to be used as a health facility.BEQUALS may have an input here. Other Development Partners could site their interventions inthese centres knowing that an integrated approach will be family and child friendly.


356392AT52 | 2 | 1 | December 2019

Appendices- attached separately

A. Terms of Reference 51B. Criteria for selection of CBSR intervention 52C. Agenda for the Validation Workshop 53D. Theory of Change-Presentation 54E. Draft Theory of Change 55F. Theory of Change exercise 56G. Evaluation Framework 57H. Table of Sampling 58I. List of CBSR Centres 59J. Table of district/ community respondents (during case study research) 60K. Table of central level respondents 61L. Quotations against DAC criteria 62M. Analysis of basic kit (provided by UNICEF to new CBSR Centres) 63N. List of basic kit materials 64O. CBSR Monitroing Reports 65P. Overview of the Tanzania School Readiness Programme 66Q. Feedback from the participants on the draft recommendations 67


356392AT52 | 2 | 1 | December 2019

camb-ed.com

Documents

Formative Evaluation f UNICEF supported CBSR Programme Lao PDR · 4 Evaluation Findings 18 4.1 Evaluation Question 1 18 4.1.1 Relevance 18 4.1.2 Effectiveness 20 4.1.3 Efficiency