i

Final evaluation of Plan Laos Maternal, Newborn & Child Health & Nutrition Program

And

Mid-Term Evaluation of Water, Sanitation and Hygiene Program

In Meung, Paktha and Pha Oudom Districts

Bokeo Province, Lao PDR

Evaluation team:

Dr Thongdeuane Nanthanavone, Team Leader and Primary Author

Dr Khampheng Phongluxa, Ms Amphone Chanthamith

July-August 2015

ii

Acknowledgement

The consultants would like to acknowledge the support they received from the staff of Plan International Lao PDR, the Bokeo Team and District Team Leader of all three target district. The team expresses special appreciation to Hilda Winartasaputra, Dr Kalana Peiris, and Yuko Yoneda for their guidance throughout the evaluation process. Our special thanks also go to the National Managers of WASH and MNCH-N and their provincial counterparts without their supports the evaluation would not have completed successfully. The team also greatly appreciates the contributions from representatives of Plan implementation partners and other international organizations who generously gave their time to talk with the evaluation team.

iii

Abbreviations

ANC Antenatal care

BEP Basic Education Program

BCC Behavioral change communication

CLTS Community Led Total Sanitation

CWS Community Water System

DEB District Education Board

DHO District Health Office

DHO District health office

DRDO District Rural Development Office

DRDPRO District Rural Development & Poverty Reduction Office

DWTs District WASH teams

ECED Early Childhood Education and Development

EU European Union

EPI Expended project immunization

FGD Focus Group Discussion

GFWS Gravity Fed Water System

HC Health Centre

HH Household/s

HP Hygiene Promotion

HW Hand Washing

IEC Informational and Educational Communications

INGOs International Non-Government Organizations

KII Key informant interview

LRC Lao Red Cross

LWU Lao Women's Union

LYU Lao Youth Union

LSIS Lao Social Indicators Survey

MCH Mother and child health

M&E Monitoring and Evaluation

MDG Millennium Development Goal

MHV Model Health Village

MNCH-N Maternal neonatal and child health and nutrition

MoE Ministry of Education

MoH Ministry of Health

MoU Memorandum of Understanding

MTR Mid Term Review

NCA Norwegian Church Aid

iv

NGO Non-Government Organization

O&M Operations and Maintenance

OD Open Defecation

ODF Open Defecation Free

PADETC Participatory Development Training Centre

PCA Participatory Community Assessment

PDR People's Democratic Republic

PED Provincial Education Department

PES Provincial Education Service

PHD Provincial Health Department

PNC Postnatal care

PRDPRO Provincial Rural Development & Poverty Reduction Office

PW Pregnant women

RWSS Rural Water and Sanitation Strategy

SBA Skill birth attendant

S&H Sanitation and Hygiene

SIDA Swedish International Development Agency

SLTS School Led Total Sanitation

SO Specific objective

SoQ School of Quality

SSHT School Sanitation and Hygiene Triggering

ToR Terms of Reference

ToT Trainers of Trainor

TT Tetanus toxoid

TWG Technical Working Group

UNICEF United Nations Children's Fund

U5Y Under five years

VEDC Village Education Development Committee

VHC Village health committee

VHV Village health volunteer

WASH Water, Sanitation and Hygiene

WHO World Health Organization

WRA Women in reproductive age

WS Water System

WSP Water and Sanitation Project (World Bank)

v

Contents

Acknowledgement ........................................................................................................... ii

Executive Summary .......................................................................................................... x

1. Introduction ................................................................................................................. 1

1.1 Country Context .......................................................................................................... 1

1.2 Plan International in Lao PDR ..................................................................................... 3

1.3 The WASH and MNCH-N Programs ............................................................................ 4

1.4. The evaluation ......................................................................................................... 10

Purpose of the evaluation ................................................................................................ 10

Desk review ...................................................................................................................... 11

Primary Data Collection ................................................................................................... 11

Data management and analysis ....................................................................................... 14

Constrains and Limitations .............................................................................................. 14

2. Results, achievement and progress ............................................................................. 16

2.1 Achievements of the WASH program ....................................................................... 16

Achievements and progress toward specific objective 1 ................................................ 16

Achievements and progress toward specific objective 2 ................................................ 19

Achievements and progress toward specific objective 3 ................................................ 21

Achievements and progress toward specific objective 4 ................................................ 23

2.2 Achievements of the MNCH-N program ................................................................... 25

Achievements and outcome for specific objective 1 (SO1): ............................................ 25

Achievement and progress for specific objective 2: ........................................................ 30

Achievements and progress toward specific objective 3 ................................................ 34

3. Relevance ................................................................................................................... 36

3.1 Relevance of the WASH Program ............................................................................. 36

Relevance of the project design ...................................................................................... 36

Water supply system rehabilitation and construction .................................................... 37

Sanitation component ..................................................................................................... 38

Hygiene promotion component ...................................................................................... 38

3.2 Relevance of the MNCH-N Program ......................................................................... 39

Delivery mechanism ......................................................................................................... 40

Monitoring and reporting ................................................................................................ 40

Focus on nutrition ............................................................................................................ 40

vi

4 Effectiveness ............................................................................................................... 42

4.1 Effectiveness of the WASH program ........................................................................ 42

Hygiene promotion component ...................................................................................... 42

CLTS .................................................................................................................................. 45

Construction/rehabilitation of WASH facilities ............................................................... 48

Participatory governance in WASH sector ....................................................................... 50

Working relationships with government partners .......................................................... 50

4.2 Effectiveness of the MNCH-N Program .................................................................... 52

4.3 Monitoring and Evaluation ....................................................................................... 59

5 Impacts and synergy .................................................................................................... 61

Impacts at the household level ....................................................................................... 61

Impacts at the community level ..................................................................................... 63

Impacts at the local government levels .......................................................................... 65

Synergy .......................................................................................................................... 66

6 Sustainability ............................................................................................................... 68

6.1 Sustainability of WASH ............................................................................................. 68

6.2 Sustainability of MNCH-N ......................................................................................... 74

7 Scalability .................................................................................................................... 77

7.1 WASH ........................................................................................................................ 77

7.2 MNCH-N .................................................................................................................... 78

8 Gender, Accountability and Inclusion ........................................................................... 79

8.1 Gender, accountability and inclusion in the WASH program ................................... 79

8.2 Gender, accountability and inclusion in the MNCH-N program ............................... 81

9. Conclusion .................................................................................................................. 83

9.1 WASH ........................................................................................................................ 83

9.2 MNCH-N .................................................................................................................... 84

10 Recommendations ..................................................................................................... 86

ANNEXES ........................................................................................................................ 88

Annex I: ToR ..................................................................................................................... 88

ANNEX III: DOCUMENTS GATHERED AND REVIEWED ................................................... 104

ANNEX IV: DOCUMENTS GATHERED AND REVIEWED ................................................... 106

Annex V-A: HH Survey .................................................................................................... 109

Annex V-B: Interview Guides ......................................................................................... 125

1) KII guide for central level stakeholders ..................................................................... 125

vii

2) KII guide for local level stakeholders (PHO, DHO, DEO, HC) ..................................... 127

3) Interview: Local Authorities/CLTS/ VHC – Village level ............................................. 129

4) FGD guides for Women, Men, and Children .............................................................. 132

5) Observation checklist ................................................................................................. 136

viii

List of Figures

Figure 1: Number of children accessed health services during health events ..................................... 32

Figure 2: Number of women of reproductive age (WRA) and pregnant women (PW) accessed health services during health events ............................................................................................................... 32

Figure 3: A place to wash hands and hand washing with soap ............................................................ 43

Figure 4: Knowledge about critical time to wash hands ....................................................................... 44

Figure 5: Knowledge about causes of diarrhea ..................................................................................... 44

Figure 6: CLTS participation and access to sanitation by ethnic groups surveyed ............................... 47

Figure 7: Sanitation coverage in target district against baseline .......................................................... 49

Figure 8: ODF achievement in villages and schools .............................................................................. 49

Figure 9: the role of men and women in decision making about toilet construction .......................... 50

Figure 10: The access to mother and child health services at baseline survey compared to the end line survey in 2015 ................................................................................................................................ 53

Figure 11: The access to new born and child care at baseline survey compared to the end line survey in 2015 .................................................................................................................................................. 54

Figure 12: Wasting (Weight for Height/length) by district ................................................................... 56

Figure 14: Stunting in three target districts .......................................................................................... 58

Figure 15: Stunting by age in month and target district ....................................................................... 58

Figure 17: Persons responsible for fetching water ............................................................................... 61

Figure 18: Fee payment by Plan and none-Plan supported CWSs ........................................................ 70

Figure 19: Fee payment by Plan and none-Plan supported CWSs ........................................................ 73

ix

List of Tables

Table 1: Timeline of key events for both programs ................................................................... 9

Table 2: Sampling frame for the household survey and FGDs ................................................ 12

Table 3: The targets and the reported achievements of the objective 1 ................................ 18

Table 4: The targets and the reported achievements for the CLTS intervention component (source: annual narrative reports 2013-2015) ........................................................................ 19

Table 5: The targets and the reported achievements for the WASH infrastructure construction ............................................................................................................................. 22

Table 6: The targets and the reported achievements for the WASH governance .................. 24

Table 7: Achievements of establishing of MHV in three target districts ................................. 26

Table 8 Number of annual and quarterly meetings and number of staff attended ............... 27

Table 9 Training courses organized between 2012 and 2014 and number of participants attended the training course ................................................................................................... 28

Table 10: Summary outreach activities and health events held in communities between 2012 and 2014 and number of beneficiaries attended .................................................................... 31

Table 11: Summary of direct beneficiaries accessed to each type of health services in communities between 2012 and 2014 .................................................................................... 33

Table 12: Summary of estimated water and sanitation conditions in the 4 target districts ... 36

Table 13: Number of people with increase knowledge of hygiene practices ......................... 45

Table 14: Number of people participated in the CLTS sessions .............................................. 46

Table 15: CLTS participation and construction of the toilets .................................................. 46

Table 16: Status of delivery at baseline and end-line survey .................................................. 55

x

Executive Summary

Background

Lao PDR has experienced an average economic growth of 8% and has moved up the ladder to a low-middle income country status in 2012. However, as experienced in many emerging economies, inequality has been going up and the distribution of the benefits from the growth has not been even particularly between urban, rural and remote sectors. Data from the WHO/UNICEF Joint Monitoring Program1 showed that at the coverage of 76 % (water) and 74% (sanitation), Lao PDR had met the MDG targets for both. However, that still left almost a quarter of the population (24 %) without access to improved sources of drinking water and almost one third (29 %) without access to improved sanitation.

Links between WASH and under-nutrition are well-established. The World Health Organization (WHO) estimates that 50% of malnutrition is associated with repeated diarrhoea or intestinal worm infections as a result of unsafe water, inadequate sanitation or insufficient hygiene2. Commonly caused by consumption of water or food contaminated by human and animal faeces, diarrhoea is directly related to poor sanitation and hygiene conditions. Diarrhoea is a leading cause of death in children under-five globally and its constant presence in low-income settings contributes significantly to under-nutrition. A lack of access to clean safe water within close proximity to home has many indirect effects on nutrition. People are often left with no choice but to drink unsafe water from unprotected sources. The time spent collecting water, bringing it home or suffering from water-related illnesses prevents young people from getting an education, which has a significant impact on their health, wellbeing and economic status.

In Bokeo, sanitation reflects national trends of low coverage. It is particularly so for the three target districts of Plan. Sanitation coverage rates are below national target rates, especially when considering actual usage. Open defecation rates are still high, despite the provision of toilet materials to village households in the past. Maternal Mortality Rate, Perinatal Mortality Rate (IMR & <5 MR) and stunting levels are high. From a health sector point of view, service coverage is low; service quality is poor and cultural and physical barriers are many and consequently the service usage is low.

Water Sanitation and Hygiene (WASH) and Maternal, Neonatal, Child Health and Nutrition (MNCH-N) Programs

The first phase of the WASH program in Bokeo began in 2009. The MNCH-N started 2011 when the second phase of WASH started. Both have the Memoranda of Understanding signed with Provincial Health Department (PHD). The largest grant for the WASH program runs until June 2016, so this evaluation serves as a mid-term evaluation for WASH. The project has initially focused on 3 districts in Bokeo and was expanded into Oudomxay and Saravane provinces in 2014. The main grant for the MNCH-N program has been implemented only in Bokeo, since the inception of the MNCHN programme. This project has come to an end in June 2015. Following the inception of the MNCHN programme in 2011/12, a program review was undertaken in 2013 to ensure Plan’s MNCH work is aligned with MOH priorities and global

1 Progress on Sanitation and Drinking Water, UNICEF & WHO (Joint Monitoring Programme), 2015 Updates 2 World Health Organisation (2008c) Safer water, better health: Costs, benefits and sustainability of interventions to protect and promote health.

xi

best practices. This resulted in a revised approach for MNCHN. This evaluation will serve as the end of project evaluation for this project.

The overall objective of the WASH project is stated as “Improved health conditions resulting

from changed and sustained WASH behaviours among children, women, and ethnic and other

vulnerable groups in 170 selected villages in Pha Oudom, Mueng, and Paktha districts of

Bokeo and in Houn district in Oudomxay province by 2016, thereby contributing to Lao PDR’s

MDGs and development goals for 2020”. It has four inter-related outcome objectives:

1: Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming “safe” drinking water among community members, particularly among women and children in selected target communities and schools;

2: Achieved and maintained ODF status in 80% of the selected target villages and schools;

3: Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages; and

4: Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level.

At the district level, Plan established a technical working team called District WASH Teams

(DWT). The DWTs are multi-sectoral teams comprised of technical staff from District

Education and Sports Bureau (DESB) and District Health Office (DHO) as well as members of

Lao Women Union (LWU) and Lao Youth Union (LYU).

Four closely linked set of activities were implemented to achieve the above objectives. These are Community-led Total Sanitation (CLTS) in the target villages which include pre-triggering meeting with community, triggering sessions with communities and post-triggering follow up session by the District Wash Teams (DWT). A similar set of CLTS activities were conducted in schools called SLTS. WASH facilities (toilets, hand washing stations and water supply) were provided in schools and Water Supply to some communities to further promote improved sanitation and hygiene practices. These activities were re-enforced by Behaviour Change and Communication (BCC) including WASH related awareness raising events.

The Overall Objective of the MNCHN project is “to improve the health and nutrition status of the population of three Bokeo (Pha Oudom, Mueng, and Paktha) Districts in Lao PDR, through an integrated primary health care approach that emphasizes the health of women and children”. The program has three strategic objectives (SOs):

1. To contribute to the development of a sustainable Primary Health Care system in Bokeo Province;

2. To promote Universal Access to Quality Health Care for the families of three Bokeo Districts; and

3. To engage Bokeo Communities for health by seeking their advice, and encouraging local reflection, prevention and promotion activities

A number of activities were implemented to achieve these objectives. The first set of activities designed to develop a sustainable primary health care system included strengthening the health management information system, strengthening district, health centre and outreach

xii

management and improving the quantity and quality of human resource for health and nutrition. The programme also promoted universal access to quality health care through health day, mobile clinics, nutrition education and food demonstration, and children day. The programme also promoted health care seeking behaviour by encouraging care at the household level, improving awareness of community about the rights, needs and problems related to pregnant women, new born and children, and improving health service providers to make them user-friendly.

The evaluation

The evaluation employed a mixed method combining quantitative and qualitative methods. The evaluation followed the assessment criteria set in the ToR including: relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan’s Child Centred Community Development standards). Sampling was purposive to capture the changes in the most remote villages with higher percentage of poor Households and people representing ethnic (minority) groups. The method employed included: Focus Group Discussions (FGD), Key Informant Interviews (KII), Household Survey (HHS) and site observation.

Findings

1) Relevance

The WASH and MNCH programs both were relevant in terms of responding to the needs of the communities. In the baseline, the target communities had recorded a low rate of sanitation coverage, limited access to safe drinking water and poor hygiene practices.

On one hand, the WASH program employed the CLTS approach to promote behavioural change in sanitation and hygiene practices at school and community levels. Adoption of the CLTS approach has been supported by the government through its recognition of the approach in National Plan of Action for Rural Water Supply and Sanitation 2012 and endorsement through the WASH Technical Working Group. The school hygiene and sanitation component is well aligned with the education sector strategy regarding improving hygiene practices amongst children.

On the other hand, the MNCH-N Program in Bokeo is aligned with government policy and relevant to the current needs in the province. The project design was consistent with the need of the Lao government in providing the universal access to health care services particularly for remote rural population with the aim to reduce the malnutrition and child and maternal mortality rates. It is evident that the MNCH-N Program in Bokeo had been designed to implement through the existing health care system of the province including supporting to strengthen the capacity of PHD, District Health Office (DHO) and Health Centre (HC) staff. The program had made considerable effort to make the program remain relevant and consistent with the government policy. For example, government institutions like Nutrition Centre, MCH Centre, and EPI were invited by Plan to be core lecturers so that it reflected the common understanding concerning to MNCH-N lessons and ensured the implementation goes in the same direction. It had achieved the contextual harmonization of working groups between project staffs and government partners. This continued engagement with the government counterpart also ensured the Program remained relevant and coherent with the government plan and approach.

xiii

2) Effectiveness

Overall, the evaluation found that the project’s hygiene promotion activities were effective in raising awareness and increasing knowledge about improved hygiene practices. The results varied in terms of behavioral change promotion. Students both boy’s and girl’s groups reported that they drink water treated and stored properly. However, we also observed that some children drank water directly from the taps (public water points) suggesting that this knowledge has not necessarily been turned into practice. Improved hygiene practices at HH level is reported to have been adopted by villagers in target communities. FGD respondents reported that they did not have a concept of washing hands with soap before the project started. Now they learned that they should wash hands with soap at critical times (such as after defecation, after cleaning children’s bottoms, before eating, before feeding children and before preparing food). In addition, the HHS shows that the hygiene and sanitation in villages had been improved: so far achieved 78% of the project target in terms of number of Open Defaecation Free (ODF) villages, 34.5% of mothers had better knowledge about how to dispose children’s faeces properly. More people knew about three critical times to wash their hands (Meung 23%, Paktha 12% and Pha Oudom 35%)

This evaluation also confirmed that the CLTS approach was effective in promoting private investment in household (HH) sanitation. The evaluation proved that community members who participated in the triggering sessions tended to construct new toilets than those who did not. The evaluation also found that there were some other factors influencing household’s investment in the toilet construction. These include subsidies from the government or a history of subsidies by other projects, availability of employment which increases the financial capacity of the villagers and access to the sanitation markets.

Construction/rehabilitation of water supply systems has contributed to the effectiveness of Hygiene Promotion (HP) in schools as the facilities (where functional) provided beneficiaries in schools with the means to adopt and practice the recommended sanitation and hygiene practices. Gender equality was also promoted at HH level. However, the role of women making decision about toilet construction remained minimal.

Overall, the MNCH-N program had successfully achieved its objectives. As the HHS indicates, access to Ante Natal Care (ANC) visit ≥ 4 times, Post Natal Care (PNC) visit within 7 days of delivery, PNC visit within 6 weeks and PNC visit ≥3 times had increased over time and was particularly so comparing to the baseline survey. In addition, mothers had better knowledge on four danger signs to take a child to health facilities (7.7% in this survey compared to 3.4% baseline). This is due to the parenting orientation as well as the MNCH-N that synergistically delivered mother and child health messages. Knowledge of mothers on danger signs of pregnancy and delivery had been improved. The survey shows that more mothers were aware of at least two or ≥ 3 critical signs at postnatal period that need care at health facilities (39.6% and 18.9%, respectively).

Concerning the nutritional status of children < 2 years, this evaluation observes that the program has improved the nutritional status of the target children particularly for wasting (weight for height/length) among children < 2 years. As shown in this HHS, the percentage of severe and moderate wasting has decreased from 6.5% and 13.7% in baseline survey to 4.3% and 6.5% in this HHS, respectively. In addition, the MNCH-N program activities succeeded to encourage mothers to give babies the initial feed of breast milk within one hour of birth

xiv

(74.8%), and to give complementary food beside the breastfeeding to children 6-9 months (92.0%).

However, there are some areas where the program results were less successful. Regarding delivery, the program had not achieved the set target to encourage mothers to give birth at health facilities with Skilled Birth Attendant’s (SBA) assistance. The KII respondents reported that it is hard to change this behaviour within a short time frame because this culture is deeply rooted in most Lao mothers. The HHS indicates that the program had not achieved the target of reducing stunting (height for age) among children < 2 years. The percentage of severe and moderate stunting in this HHS was higher than the baseline (22.6% and 41.6% vs 17.1% and 38.2%, respectively).

3) Impacts and synergy

Plan-supported Community Water Systems (CWS), and those supported by other agencies, had reduced women’s workload. As women are responsible for fetching water, CWS reduced time and energy women have to spend on collecting water. FGD participants (women groups) reported that they were happy that they did not have to carry two buckets of water home sometimes as far as 10 km for household use and drinking. Now they could save time and energy from this reduced workload. They reported to be freer now to watch TVs, learn some cooking lessons from there, learn how to properly care their children etc.

Furthermore, more villagers were able to connect water to their dwelling (PVC pipe or plastic pipe) which freed up women from water collection duty. All FGD participants reported that positive behavioral changes concerning using toilet, washing hands with soap, drinking safe water and keeping environment clean had been taking place after the Plan’s interventions. Almost all of the FGD participants were aware of the importance of using toilets, washing hands with soap at critical times, drinking safe water and keeping surrounding environment clean.

The positive and negative changes were found at the household level after the introduction of MNCH-N programs’ activities. For example, percentage of mothers who had ANC visit ≥ 4 times had increased by about 20%. Women of Reproductive Age (WRA) who received vaccination against Tetanus increased by about 25%. However, there remain some challenges regarding mother and child health. Pregnant Women (PW) still worked hard in the field as elder people taught them that working hard would help make delivery easier. In addition, PW did not have access to diverse and quality food. There was no special attention being given to diets of PW. Results of HHS indicates that the knowledge on danger signs of PW and PNC, although still low, has improved (knowing 2 critical signs of PNC 39.6%, knowing ≥3 critical signs of PNC 18.0% and knowing critical sign of labor 18.0%).

Program has shown some positive changes in terms of gender equality. It was reported that now more women become committee members and even sitting at leadership positions. Children’s rights and participation were recognized at HH and community levels i.e. their participation in the school activities and planning process in the communities (initiated by Plan). These changes had never taken place before the program started. The program has not yet involved people with disability (PWD) and this survey found some family members had taken care of the people with hearing impairment.

The community members reported that before, children were frequently sick of diarrhea and some children died. The village authorities interviewed reported that over the last three years

xv

there was no case of under-five mortality in the villages surveyed. The integration of WASH and MNCH-N programs has given a positive impact not only on the acceleration of the development of Model Healthy Villages but also addressing malnutrition among children < 2 years particularly wasting. From the field observation, the team found that very few sanitation facilities were constructed in villages where there was no WASH program present. At the same time, more women and men reported seasonal diarrhoea outbreaks in villages where there was only MNCH-N program presented.

4) Sustainability

Sustainability of hygiene promotion in schools rested on the integration of hygiene promotion activities into school curriculum and the hygiene promotion activities led by teachers and model/volunteer students. Sustainability of the WASH facilities depended largely on the ownership of the committees. It was reported that the school administration fund was the only source of money to be spent on maintenance of school WASH facilities. The committee reported that minor maintenance could be done by the CLTS committee members. According to education officials interviewed, the schools that meet the government SoQ criteria would receive encouragement fund (1000kip per student per year). However, none of the schools that had met the criteria had received the money yet.

The Institutional structure developed under the project related to school WASH is the establishment of DWTs, Village Education Development Committees (VEDC), CLTS committees and involvement of village chiefs and village cluster heads. Over the project period, these entities were actively involved in the school WASH activities. The key to sustainability of CLTS are continuous follow-up among households to ensure maintenance of toilets and sustained use.

Plan is striving to ensure local ownership and community-led processes in relation to WASH. Local governance structures such as VEDCs and CLTS committees were established to empower community participation. However, across the three districts visited, officials both Plan and government counterparts, reported that community ownership remained an issue of concern. At community level, even village authorities, still believed that WASH facilities belong to Plan. Villagers were under the impression the project is ‘owned’ by the district or provincial departments.

Fee collection by the water committees is a crucial source of funding for maintenance. The results from HHS show a low rate of fee payment contributing to water fund at the community level with only 37% paying the fee regularly. This result is vastly different from the rate at which it was reported in the 2015 annual report (80%). Sampled villages for the HHS being the remotest with higher percentages of poor HHs could explain this difference, which also shows that alternatives should be sought for poorer and more remote communities when it comes to sustainability. When the Plan’s CWS are compared with other villages supported by other organisations, the result is not significantly different.

The success of fee collection or maintenance fund management is also dependent on the sustainability of the CLTS committees themselves. Sustainability of CLTS committees at village level remains a great challenge. One of the reasons is that at the moment they are not functioning as they supposed to be. From interviews with CLTS committee members, the team found that the CLTS committees stand-alone without any links to the other village development committees such as the Village Health Committee (VHC) mandated by the government to carry-out health activities, members have limited skills in encouraging other

xvi

villagers to improve sanitation and in many villages, the committee members themselves had not built toilet yet. Another reason is the lack of incentive/reward for the effort for collective gain. This together with the fact that they are not fully volunteered in becoming the committee members can lead to unsustainability of the initiative itself. It is unlikely that DWTs will continue to function after the financial support from Plan has ceased. Therefore, CLTS activities must remain and be carried out by VHC members (in this case CLTS at community level need to be integrated into VHC). It is likely that working with existing committees will ensure sustainability.

Sustainability of MNCH-N activities in community depends on the ownership of health care providers including village authorities and actors like Village Health Volunteer (VHV), VHC and village facilitators in organizing an outreach activities and health events in communities. It is likely that the activities initiated by the program will be sustainable as they have been aligned with the government policy and constantly responding to the needs of the community. This is well evident throughout the program implementation. Stakeholders at the central level acknowledged that Plan has contributed significantly to improvement of the mother, newborn and child health and nutrition in Bokeo. Plan was well regarded as contributing to supporting these MNCH-N projects in remote provinces of Lao PDR which have been facing funding shortages.

Sustainability of outreach activities and health events in communities, local stakeholders (KII) shared his view that if the program is ended the outreach activities would be continued however the coverage and completeness would be less, and vaccination and health promotion is unlikely due to the government has no budget. One KII respondent stated that integrated mobile clinics might not be possible.

Now health staffs were able to use computer for routine work and conduct health events in village and knew how to do microplanning. Program had supported the renovation of health centres including supporting medical equipment and water supply and sanitation facilities in health facilities, logistics support to 3 target districts and official equipment and materials were supported by the program.

Annual and quarterly meetings with the PHD and key government counterparts ensured opportunities for reflections, planning, and adjusting the project plan if needed. In terms of accountability upward, reporting to the donor and government counterparts, the project has done considerably well. All the necessary information was reported back to the donors.

Plan promoted ownership amongst both provincial and district government counterparts over the implementation of programs’ activities in communities. PHD and DHO staff took lead in monitoring and supporting HCs. The implementation of some activities, however, were delayed and not implemented as according to plan due to staff turnover within Plan and the counterparts alike. As a result, some key information such as population figures for the target group was not collected.

5) Scalability

CLTS as an approach should be scalable. The team found that the approach not only increase toilet coverage but also made progress on stopping Open Defaecation. A working approach of DWTs should be rearranged. For sustainability purpose, the team should only work together at the triggering stage only; an individual team member can take on a follow up role

xvii

over a certain number of target villages. We found that the current form of DWTs would not be sustainable. Construction and rehabilitation of CWSs should be scaled up. It is strongly evident that water supply and availability of water all year round is the critical factor for HHs to get access to improved sanitation practices including building and using toilets and washing hands with soap.

Another key intervention that should be scaled up is the application of Gender and WASH Monitoring Tools (GWMT) beyond mere monitoring to supporting communities to tackle the identified gender inequities. The team found that WASH improvements reduce women workload and, to address gender inequality it needs to be designed and monitored well.

In addition, the KII respondents emphasized that a proper monitoring system has to be established to follow up and support communities because communities are very poor and people have low, self-awareness.

Regarding the scaling up of activities some challenges need to be carefully considered: lack of staff particularly at health centre and DHO, limited capacity to perform required tasks and staff who are used to low usage rates may consider an increasing usage rate over whelming. The establishment of many new committees in villages should be avoided.

6) Gender and social inclusion

It was found that CLTS sessions at the village level remained dominated by the heads of the households and they are usually men. The government officials’ attitudes toward different roles and responsibilities of men and women have changed since they had participated in the training about gender and social inclusion organized by Plan.

Although no quotas have been set for women for trainings, DWTs, CLTS members, etc. women are generally promoted to take part into the committees and trainings and given equal opportunity to participate in Program activities. For example, 20 provincial and district health officials (including 10 female) had been trained on CLTS in Bokeo. Program staff keeps record on training, workshop as well as seminar participants disaggregated by gender. Other disaggregation (by ethnic background etc.) is not performed.

The effort to reach more marginalized and people living in the most remote areas intensified during FY15. The project’s target villages included more remote communities with more difficult access. With these challenges Plan introduced an interns system where it drew young people from target communities to get involved.

There is little evident to show that disability has been integrated in the project designs. An interview with Plan staff confirmed that they had participated in the disability classification workshop which helped them to recognize more comprehensively about types of disabilities.

In terms of physical facilities of WASH and health facility WASH, we observed that technical designs of water points have not been disability sensitive.

7) Conclusions and Recommendations The recommendations for the future direction of the program are provided on short-term and long-term basis.

xviii

Short-term recommendations

1. Despite impressive achievements in many respects of the program, the nutrition status of the target children under 2 years old has yet to achieve the target due to poor feeding practices and poor sanitation and hygiene conditions. As the national trend toward refocusing on nutrition is rising, it is important that Plan joins the government and Development Partners (DPs) by using the existing platforms such as SUN to scale up WASH in Nutrition approach.

2. WASH and MNCH-N programs should be integrated as the outcome from WASH program can complement the outcome of other program. Synergized outcomes could provide positive impact to the nutrition status of children under 2 years and mother neonatal and child health and nutrition in community.

3. At the moment, there is no link between WASH promotion at village level and the Health Center staff. With a smaller DWT some WASH responsibilities should be delegated to HCs because they are close to communities and have the overall responsibility of the health status of people in the catchment area.

4. WASH and MNCH-N can jointly operate (convergent implementation). At the community level with CLTS committees, water committees, VHC integrated into one committee but keeping members with mechanical skills for toilet construction and water supply system maintenance.

5. WASH, particularly water supply should provide water beyond drinking and household use to cover animals raising and gardening. By doing this WASH can contribute more directly to improvement of nutrition outcome in the communities because we know that part of the root causes of malnutrition are the lack of access to food. Animal raising and gardening can increase food availability thus access to food.

6. Food hygiene can be integrated into intervention. Hygienic food preparation and storage should be integrated into the intervention.

7. Some WASH facilities in schools and communities started to break. It is important to urgently address this problem. To do this, the project has to change its monitoring and follow up arrangement. Follow up support beyond one year is needed to help the party responsible in the village level build their competence and experience.

8. Rather than working from Plan office, provincial managers should take the office space provided by PHD. This will help foster better coordination and communication.

9. The DWTs are unlikely to be sustainable and remain effective beyond the project life. Number of DWT members should be reduced to two or three as the bigger the team the more unlikely it will be sustainable and effective. This is to redirect more resources into community level. Plan should rely on the village level. It is important that Plan start to build a strong and competent WASH team at the community level, ensuring the each community has a fee collection system that can cover the minor maintenance cost. Sustainability of the results of the interventions relies on the organisation at the community level with only periodic support from the district level.

10. It is important to make sure a strong governance system at the community level is in place. This includes transparency of the water fund management, strong and competence technical person who can deal with practical repair and maintenance of the facilities.

Long-term recommendations

There was consensus among key stakeholders that CLTS works and could lead to effective use and behavioral changes amongst target population. It is recommended that Plan:

11. Promotes cohesion within the WASH sector concerning adoption of CLTS that is still lacking. Plan should work with MOH and through WASH TWG to advocate for wider adoption of the approach. At the moment, there is an ineffective coordination or communication between central and local government counterparts, such as variation of their view on CLTS approach.

xix

12. Plan should also continue to increase efforts to get other stakeholders on board in terms of CLTS adoption. This will include clarifying with key stakeholders that CLTS works but takes time to get the results.

13. School WASH is clearly inextricably linked with the rights of the child and enhanced child survival and must therefore remain a key priority for Plan

14. Convergent planning and implementation for a more integrated, comprehensive WASH and MNCH-N must be strengthened to build the potential synergies.

15. Maintenance of WASH facility in schools should be the responsibility of the VEDC because in the foreseeable future there will not be any improvement in the budget situation of the school.

16. Engage more actively with the sanitation material suppliers in terms of outreach activities and advertising of their products to the local communities.

17. Apart from addressing the communication (intern)/engagement strategies (language) adopted by Plan, an approach where specific interventions are amenable to change at community level by different ethnic groups may need to be considered. For example, the “prescription of solutions by District Authorities” model should be changed to “solution exploration by community” model. Furthermore, slowest groups might need more engagement and more encouragement or different approach/strategies altogether.

18. Report on program of Plan should be distributed during the meeting of technical working groups or health research forums. To make sure that the concerned institutions at the central do miss out on the progress of the program.

19. It would be effective in implementing if DHO and HC meet each other and discuss about the role and function of organization to lead the health event in villages. DHO should work closely with health providers and encourage the assigned staffs to perform their tasks as expected and with high responsibility.

20. PHD should report about the Plan program implementation to MOH and national health forums.

21. The representative from the Central Ministry who attended the annual meeting should report to its Director in a written manner.

22. Generally speaking, malnutrition has not seen significant improvement, PHD and DHO together with Plan should coordinate with local agriculture and forestry for the promotion of home gardening, protecting and nurturing natural sources as good food source for community. At the same time some knowledge on the important of eating variety of food and more food groups should be explained carefully and in ethnic language.

23. The delivery at health facilities rate remains low, due to cultural barriers among ethnic women, such as they were too shameful to let other people see their PHD and DHO have to be sure about gender of health provider at health centre, at least one women should be presented at HC for delivery of health service particularly MNCH-N services. PHD should discuss with Plan or other international organization for more SBA development to ensure that delivery attended by SBA and PHD should consider about recruiting the SBA produced by Plan into HC level.

1

1. Introduction

1.1 Country Context

Lao PDR is a land-locked country in South-East Asia. It is a very diverse country with 49 officially recognised ethnic groups. Ethnic groups, which make up almost half of the population, mainly live in highland areas. The government of Laos (GoL) with technical and financial support from Development Partners (DPs) has been addressing the infrastructure and societal gaps between the lowlands and highlands. However, the differences are still significant, especially in terms of access to basic public services. Regardless of the progress made by the economic sector, social sectors’ development has been lagging behind. In the last decade, the country has enjoyed an average Gross Domestic Products (GDP) growth of 8%, and in 2012, became a low-middle income country. Yet, inequality is increasing and rural and remote ethnic communities are not benefitting from the economic growth. In other words, the significant progress in the social development has not been fully going hand-in-hand with the reported economic growth. At the score of 0.569, the UN 2013 Human Development Report positions Lao PDR as the 139th out of 187 countries and territories in the World3.

Significant progress has been made in the provision of water supply and sanitation services across the country. Millennium Development Goal (MDG) No. 7 commits countries to halve by 2015 the proportion of people without sustainable access to safe drinking water and basic

sanitation. Data from the WHO/UNICEF Joint Monitoring Program4 showed that at the coverage of 76% (water) and 74% (sanitation), Lao PDR had met the MDG targets for both. However, that still left almost a quarter of the population (24%) without access to improved sources of drinking water and almost one third (29%) without access to improved sanitation. Moreover, disparities between urban and rural areas have also persisted. In rural areas, where the majority of the poor live, access remained well below that of the urban areas: 69% and 58% for water and sanitation respectively. Only 38.9% of all schools currently afforded access to WASH services including for students and teachers5. Given the recent global evidence links open defecation and inadequate disposal of human faeces with stunting, it is highly concerned that in Lao PDR, open defecation remains widespread and a challenge for mother and child health.

Furthermore, the Lao Social Indicator Survey (LSIS) reported a National Maternal Mortality Ratio (MMR) is 357 per 100,000 live births; an Under-five Mortality (U5M) at 73 per 1,000 live births; Infant Mortality Rate (IMR) at 68 per 1,000 live births with Perinatal Deaths (PNMR) accounting from 32 of these6. The MDG target for maternal mortality is 260 per 100,000; for U5M is 70 per 1,000 live births; for IMR is 45 per 1,000 live births and for stunting is 34% in U5.

Lao PDR continues to have very high chronic malnutrition rates: the stunting rate among children U5 is 44% nationally or nearly every second child under the age of five in Lao PDR is stunted. These rates are even worse among ethnic groups living in remote areas. Every fifth rural child is severely stunted. A nutrition survey in a remote, mountainous community in

3 United Nation Human Development Report 2014: sustaining human progress 4 Progress on Sanitation and Drinking Water, UNICEF & WHO (Joint Monitoring Programme), 2015 Updates 5 Lao Ministry of Education’s Education Management Information System (EMIS) 2010). 6 Lao Social Indicator Survey 2012.

2

Sekong Province found a stunting rate of 67 %.7 Micronutrient deficiencies affect large parts of the population, with over 40 % of children under five and 63 % of children under two suffering from anaemia, and almost 45 % of children under five and 23 % of women between 12 and 49 years of age affected by vitamin deficiency. On the current trends Lao PDR is unlikely to reach the mother and child health targets even though the under-five mortality rate is very close to the MDG target (from 73 to 70). In sum, urban-rural disparities, gender inequality, limited coverage of services/supply, unhealthy behaviours have been the root causes of poor health indicators.

In recognition of these challenges, in 2013, the GoL and the Ministry of Health (MoH) launched the National Action Plan for Rural Water Supply and Sanitation and Operational Guidelines for Scaling up Rural Sanitation in Lao PDR. These guidelines were developed to provide a systematic framework for the planning, implementation, monitoring and evaluation of rural sanitation Programs under the umbrella of the National Plan of Action. This includes the roles and responsibilities of relevant state agencies at all levels. The intention is to ensure a coordinated approach nationwide, based on common national goals and priorities. The guidelines are founded on five guiding principles: 1) equity to ensure that all people living in remote areas have access to improved WASH services; 2) sustainability of services and behaviour changes; 3) sanitation demand creation and strengthening supply of sanitation goods and services; 4) shared responsibilities between stakeholders (households, government, public sector); and 5) learning and innovation.

In the area of maternal, new born and child health and nutrition, the MoH has spent a great effort to reduce maternal and child death and free from malnutrition, food insecurity and poverty. The strategy and planning framework for the integrated package of maternal neonatal and child health services 2009 – 20158 and national nutrition strategy and plan of action 2010 – 20159 were developed as the best platform for MNCH-N implementation at all levels within the country. Currently the free delivery policy has been issued and implemented in all districts of the country particularly in remote districts10.

Furthermore, there has been an increased focus on improved nutrition in women and children, from the highest level of the government. A Prime Ministerial Decree was enacted in mid-2013 establishing for the first time an inter-ministerial Nutrition Committee, and Secretariat. In late 2013, a UN Task Force (IFAD, UNICEF and WFP) developed a new Food and Nutrition Security Action Plan (FSNAP- December 2013), based on the convergence model of multi-sectoral planning and monitoring, and sectoral implementation, across four sectors (Education, Health & WASH and Agriculture). Phase one of this plan prioritises Luang Namtha, Oudomxay and Saravane, with phase two adding Phongsaly, Xieng Khouang, Houaphan and Sekong. It is important to note that Bokeo is outside these priority provinces though the planned expansion to Oudomxay and Saravane are aligned with the multi-sectoral priority

7 CARE Nutrition Survey in Dak Cheung 8 Strategy and Planning Framework for the integrated Package of maternal Neonatal and Child Health Services 2009-2015 9 National Nutrition Strategy and Plan of Action 2010 - 2015 10 Decree on subsidy delivery and treatment children under five years of age, No 273 and dated 18.09.2014. The policy is a form of results-based financing (RBF) that will remove user fees and charges for medicines, provide beneficiaries with small incentive payments, and reimburse health facilities for provision of MCH-related care. The free MCH policy builds on the relatively positive experience of several smaller-scale donor-financed pilots that have implemented similar interventions in selected regions of Lao PDR over the past few years.

3

targeted provinces. At the time of this evaluation the inter-ministerial National Nutrition Committee, supported by the MOH and Food and Agriculture Organisation (FAO) are revising the government’s National Nutrition Policy and Plan of Action, and adopting the convergence model and 22 priority nutrition actions identified in the FNSAP. Approval of the new government policy is expected in the latter half of 2015.

In light of the challenges and policy context outlined above Plan International Lao PDR (Plan) has focused on the WASH and MNCH-N interventions. In carrying out its interventions, Plan also take into account gender inequality including deeply embedded cultural norms and practices, such as early marriage, women’s workload, lack of their voice in household decision making as well as lack of access to information, resources and services on health, nutrition and WASH.

1.2 Plan International in Lao PDR

In response to the emerging policy context as well as the community needs, Plan collaborated closely with the GoL, primarily Nam Saat Centre under the Department of Hygiene and Health Promotion in the MoH. It also collaborates with other international development agencies working in sanitation and hygiene promotion - particularly the World Bank-WSP, SNV and UNICEF. This would help to foster an adoption of a common approach i.e. Community led total sanitation (CLTS) to be scaled up within the framework of the National Plan of Action. It is also important to note that Plan intends to address gender inequality, limited coverage of WASH in remote areas and unhealthy/unhygienic behaviours that underlies poor health/nutritional outcome for women and children. It is evident to some extent that some key principles of gender equality were integrated into both WASH and MNCH-N programs. The evaluation assessed gender inequality including deeply embedded cultural norms and practices, such as early marriage, women’s workload, lack of their voice in household decision making as well as lack of access to information, resources and services on health, nutrition and WASH. These are very important preconditions that will be incorporated in the analysis for this evaluation.

Plan works with the Government of Lao PDR and the Development Partners (DP) to address the issues faced by marginalized children, especially girls, through participation, partnerships, advocacy and high quality Programming. The 2nd (current) Country Strategic Plan (CSP) 2014-19 has five key Country Programs (CPs)

This evaluation focuses on two of the CPs. Some background to the two programs is presented in the next section.

These two Countries Programmes are expected to directly contribute to the following key results of the CSP

1. 78,000 children11 and their families demonstrate improved sanitation and hygiene practices in 400 open defecation free villages.

2. Less than 20% of boys and girls under 5 years of age are stunted in Plan target districts12

11 Approximately 50% of the total population of children U18 in 9 Plan target districts of Pha Oudom, Paktha and Meung (Bokeo), Pak Beng, Houn and Nga (Oudomxay) and Saravane, Ta-Oi and Samoui (Saravane) 12 Stunting rates for children U2 in 2013 in Bokeo are: Pha Oudom: 48.8%, Meung 37.7% , Paktha 24.8%. Stunting rates per district are not available for Oudomxay and Saravane, However, the stunting rates per Province for children U5 are: Oudomxay 54.9%, Saravane 54.4%

5 Country Programmes of Plan International in Lao PDR

1.Early Childhood Care and Development CP(ECCD)

2.Basic Education CP (BEP) & Child Protection CP (CRCP)

4.Water Sanitation and Hygiene CP (WASH) 5.Maternal, Neonatal and Child Health and

Nutrition CP (MNCH-N)

Source: 2nd Country Strategic Plan 2014-19, Plan International in Lao PDR

4

1.3 The WASH and MNCH-N Programs

Both Programs targeted the mountainous districts of Meung, Paktha and Pha Oudom of Bokeo which are amongst the poorest districts in the country. The total population is 73,500 and characterized by high ethnic diversity. Major ethnic groups include the Khmu, Lamed, Hmong, Lue, Akha, Muser, Tai Dam and the majority Lao. These make 80 % of Bokeo’s population with the Khmu being the largest13. There are 146 villages within the target districts of which 49% are classified by GoL as being ‘poor’. Concerns over cost of transport and treatment in addition to service quality were widely reported as factors limiting household access to formal health care services and in particular SBAs.

In 2010, the WASH Program began its piloting of CLTS in three communities before expanding its activities to 35 villages and schools in Pha Oudom, Paktha and Mueng in 2011-12. Initially, WASH activities were meant to provide better sanitation, hygiene conditions and knowledge for children in target schools. However, to ensure greater impact, expansion to school-hosted communities was encouraged and implemented under the MoU between Plan, Provincial Education Department and Ministry of Education and Sports (MoES). For WASH, the core funding of the program comes from Australian Aid’s ANCP and Plan Australia.

In 2012, MNCH-N Program began to implement its activities across the three districts. Since then, the main MNCH-N grant under the MNCH-N Program has been implemented only in Bokeo. This project came to an end in June 2015. Following its inception in 2012, the MNCH-N Program review was undertaken in 2013 to ensure its alignment with MOH priorities and global best practices. The current evaluation will serve as an end of project evaluation for this project. However, the Memorandum of Understanding (MoU) which Plan has with the PHD for the Program outlasts this project and runs until December 2016. Plan intends to continue to engage with MNCH-N with a renewed focus on nutrition, in a way that is integrated with WASH.

Program design and components

The WASH project design was developed in a workshop in 2012 involving government stakeholders. It was agreed that the multi-sectoral approach to implementation was effective and would be adopted. Accordingly, the new MoU between Plan and Provincial Health Department (PHD) makes specific reference to the multi-sectoral partnerships and broadly defines the roles and responsibilities of each of the four sectors involved, i.e. DHO, DESB, Lao Women Union (LWU) and Lao Youth Union (LYU), as well as leadership arrangements14. Improving health facility WASH has become part of cross-sectoral activities for MNCH-N, however, collaboration at the community level has not been concretely implemented. As the sectoral divide at the community level was not present.

The WASH project of Bokeo lays out its goal, key objectives and implementation strategy in the Project Design Summary 2012-2016. The overall goal states as “Improved health conditions resulting from changed and sustained WASH behaviours among children, women,

13 Maternal, Neonatal and Child Health and Nutrition: July 2014 to June 2019

14 A case study of Plan’s ECCD and WASH programme in Bokeo Province, funded by Austrlian AID, May 2014.

Figure 1: Map of Bokeo Province with relative positions of the three Districts where Plan works

5

and ethnic and other vulnerable groups in 170 selected villages in Pha Oudom, Mueng, and Paktha districts of Bokeo and in Houn district in Oudomxay province by 2016, thereby contributing to Lao PDR’s MDGs and development goals for 2020”. There are four objectives:

1. Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming “safe” drinking water among community members, particularly among women and children in selected target communities and schools. This is based on the fact that hygiene practices and sanitation standards are low in target villages with high reported incidences of diarrhoea. The rationale is that improving and sustaining good hygiene behaviours will improve health conditions. Therefore, both schools and villages should meet basic WASH standards for measurable health impacts.

2. Achieved and maintained ODF status in 80% of the selected target villages and schools. This objective aims to address low toilet coverage and ineffective usage of toilets is common in many villages. Less than half of the primary schools have adequate school WASH facilities. Therefore achieving and sustaining community ODF status (100 % toilet access) is a critical milestone for realizing the health benefits from having toilet coverage in villages.

3. Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages. Water supply maintenance is often poor and systems do not meet village service delivery requirements. Furthermore, lack of water availability can be an impediment to improved hygiene and sanitation practices. Therefore, improved access to safe water supplies will improve the health and quality of life of targeted communities.

4. Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level. The rationale is that there have been few women in WASH leadership positions and lack of government policies recognizing ODF as a nationally-recognized sanitation outcome.

To achieve these objectives the Program focuses on five key areas of intervention including: Community-led total sanitation (CLTS) in villages and schools, Hygiene Promotion in Schools and associated communities , School WASH facilities and associated community water supply improvements, and Behaviour Change and Communication. The above objectives are to be met through these key interventions.

These field level interventions were complemented by a series of research and advocacy activities that Plan and its partners have conducted collaboratively. These include the following:

Co-hosting with WSP, SNV, UNICEF, and National NamSaat launching the Lao PDR National Rural Water Supply, Sanitation, and Hygiene National Plan of Action (RWSSH NPA) in September 2013.

Co-hosting with WSP, SNV, UNICEF, and National NamSaat arranging a workshop in Vientiane on developing operational guidelines for the sanitation and hygiene promotion components of the RWSSH NPA in October 2013.

Supporting National Supply Chain Survey and Consumer Research initiated by WSP, UNICEF, SNV, Plan and National NamSaat in June & July 2014.

6

Co-hosting with WSP, SNV, UNICEF, and national NamSaat arranging the first National Scaling up Sanitation workshop (April 2014) for national, provincial and district government counterparts working in WASH sector and NGO representatives.

Contributing by Plan WASH to development of Lao PDR WASH in Schools publication printed in 2014.

This has been the area of focus that Plan has adopted with its strong trust on the CLTS. Plan is reliant on the community participation and initiatives trying to promote ownership of the development activities by communities and local governments.

As stated in the WASH Program document, government agencies and donors began to realise the ineffectiveness of the direct subsidies approach to household sanitation in Lao PDR. Newer approaches for scaling up sanitation coverage in rural areas include Community Led Total Sanitation (CLTS) and active engagement of the private sector. CLTS is currently being implemented in Lao PDR by Water and Sanitation Project of the World Bank (WSP-WB), Netherland International development (SNV), Plan, and World Vision among others15. Plan has adopted the CLTS approach as opposed to direct subsidies to increase sanitation coverage and increase hygienic practices.

The main focus of national level work is advocacy for the provisions under the RWSS Strategy. To this end, the Program increased advocacy work with the MoH and with key partners such as UNICEF, WSP-WB, and SNV and others. The review of national CLTS guidelines and trainers’ guides as well as a national level Training for CLTS Trainers was conducted in cooperation with the National Centre of Environmental Health and Water Supply (Namsaat), SNV, WSP-WB and Plan Laos. Development of village level water safety planning guidelines and manuals for use on a national scale has been developed in a similar way with the addition of the World Health Organisation (WHO) and CAWST.

Implementation mechanism

The focus of the intervention was placed at the local levels from provincial to community with most responsibilities rested on the district level. At the district level, Plan established a technical working team called District WASH Teams (DWT). A DWT was established in each district with approval from each District Governor which means these officials were assigned to work with Plan. The DWTs are multi-sectoral teams comprised of technical staff from DESB and DHO as well as members from LWU and LYU, who were responsible for providing guidance and monitoring for construction of school WASH infrastructure, community water supply systems and facilitate CLTS steps and hygiene promotion activities. Although previously implemented as part of a wider Basic Education Program under a MoU with the Ministry of Education and Sports (MoES), learning from this pilot phase, the WASH Program came under a new MoU with the Provincial Department of Health (MoH) in 2014.

The District WASH Teams (DWTs) are comprised of up to eight members, with an equal number from the District Education and Sport bureau (DESB) and the District Health Office (DHO), with additional members coming from the Lao Women's Union (LWU) and Lao Youth

15 WASH 2012 – 2016, Project Design Summary, Pha Oudom, Paktha, and Mueng Districts in

Bokeo Province and Houn District in Oudomxay Province, Lao PDR.

7

Union (LYU). Up until recently (with the signing of the MoU which broadly defines the role of each sector), the roles and responsibilities of each sector/team member had not been documented, and the teams had been first led by a member of the DESB supported by a Plan staff member and this changed under the new MoU arrangements to a member of Nam Saat section of the DHO. The Mobile Clinics team draws mainly from the health offices at the district level though sometimes LWU staff joint as back up support.

At the district level, MNCH-N relies on Mother and Child Health Section of the DHO to take lead in the planning, implementation and reporting. For example, Integrated Primary Health Care (PHC) Mobile Clinics intervention (i.e. Integrated Outreach) started with the district undertaking a process of micro-planning where the preparation for the outreach takes place. The Integrated Mobile Clinics activities involved provision of ANC, Vaccinations, Nutrition education and growth monitoring. The implementation involved at least one or two staff of HCs and VHWs in addition to District level government staff and Plan’s technical staff

A District level steering committee, chaired by the District Governor's Office, provides over sight, meeting six monthly to review the implementation of all Plan supported Programs in the District, including WASH and MNCH-N. The multi-sectoral review of WASH and ECCD found that while Program documents specify a provincial level Implementation Management Committee (IMC), it appears that this has not functioned as intended, perhaps due to sectoral differences being more firmly entrenched at higher levels as well as the fact provincial level departments are relatively distant from implementation in the field.

Community/school engagement

The village cluster heads (Kumbans), VEDC, village chiefs, and village health volunteers were actively engaged in the hygiene promotion activities at schools and CLTS triggering processes. They also supported school and community organizations for infrastructure development such as construction of toilet and hand washing facilities at schools, community household toilets, and improvements to the existing school/community water supply as may be necessary.

Plan staffing

The Plan WASH staff, with support from their colleagues working in other Plan Programs in the same communities, ensures that all activities were implemented in the highest quality possible and that all of the support requirements for the WASH and MNCH-N initiatives were drawn from the government counterparts. Both Programmes, at their initial stages had International Programme Managers. Currently, an International Public Health Advisor provides technical support and overall management support to both WASH and MNCH-N programmes while two National managers each supervising the respective Provincial level managers were posted to manage the projects and the coordinators for each program at the district level.

9

Table 1: Timeline of key events for both programs

Description 2009 2010 2011 2012 2013 2014 2015 2016

Pilot Project

Evaluation of the Pilot Project

Commencement of the current project

MOU process with PHD

Baseline Survey

Program review

Project Expansion to Oudomxay, Houn district Jan

Project expansion to Saravane, Saravane and Ta Oi districts March

An internal review with PHD Jun

Evaluation* Jul/Aug

Project continuation in 2 districts at different levels of engagement in Bokeo and Oudomxay Jun

MOU with PHD expires

December

Jun

MNCH-N WASH Combined

10

The overlap of goals and potential of synergy between WASH and MNCH-N had always been recognized by Plan since the beginning of both Programs. However, apart from the baseline survey and the present evaluation, the two Programs evolved independently from each other. Acknowledgement of the fact that integration of WASH and MNCH-N strategies would yield better outcomes was growing with the global evidence and experience that Plan gained through implementing these two sectoral projects. Plan intends to continue to engage with MNCH-N with a focus on nutrition, in a way that is integrated with WASH.

As indicated above, an evaluation of the MNCH Program is to be conducted jointly with the MTR for the WASH Program. This Evaluation is expected to provide empirical evidence and recommendations for Plan to be incorporated into the process of integration of the two Programs with a renewed focus on nutrition.

1.4. The evaluation

Purpose of the evaluation

As indicated in the ToR, Plan International Lao PDR (hereafter referred to as Plan) wishes to have an evaluation of the Maternal, Newborn and Child Health and Nutrition (MNCH-N) Program conducted jointly with a review of the WASH Program. This evaluation is funded by Plan International Australia (PIA).

The specific objectives of the evaluation are to:

1. Assess impact and outcomes achieved through the respective projects against the approved project designs, and identify areas of synergy between the two Program.

2. Assess the likelihood of sustainability of the impact and outcomes at the grassroots, district and provincial levels, with particular attention to the capacity of the Provincial and District health care delivery system and linkage to the private sector in sustained access to WASH services.

3. Draw lessons learnt on Plan’s role, approaches and its financial investments vis-à-vis the public and private sector actors on Health and Water, Sanitation and Hygiene ( WASH) in Bokeo Province.

4. Assess the alignment of the MNCH-N and WASH Program designs with the Food and Nutrition Security Action Plan (FNSAP16), and contribution of the Programs to addressing stunting. Highlight opportunities and gaps in the current Program designs that are of relevance to future Program development on a WASH-MNCH-N integrated design with a stronger focus on nutrition in Oudomxay and Saravane provinces. This is especially relevant as Oudomxay and Saravane are prioritized provinces in the geographical focus of FNSAP.

5. Provide evidence on effectiveness of interventions and models used in the WASH and MNCH-N Program, and identify elements of replicability and scalability, while recognizing factors to be considered in adaptation to specific contexts.

16 During the evaluation, the government with FAO support commenced a review of the National Nutrition Policy and Plan of Action, drawing from the Food and Nutrition Security Action Plan referred to here. At the time of this evaluation report finalization, the policy review process was not yet complete.

11

6. Gather voices and feedback on the two Program from diverse population groups in the target communities. Facilitate participatory, yet critical reflection and learning among Plan and the government counterparts on how changes happen at the community level.

The evaluation primarily employed quantitative and qualitative methods. The evaluation followed the assessment criteria set in the ToR including: relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan’s child centered Community Development standards).

Specific interview guides have been developed by the team with support from Plan Regional Specialists. These will be used with partners, implementers and participants in order to obtain responses to the questions in the evaluation matrix.

Desk review

During the inception phase we also conducted a review of key Program documents and information received from Plan. Secondary sources of information were also reviewed including key national documents and sectoral documents.

Primary Data Collection

The Evaluation Team has identified a mix of quantitative and qualitative tools to collect the data needed for the evaluation. Given the large number of target villages, type of activities and complexity of Program designs and monitoring framework, the adopted mixed method would allow the evaluation to capture depths and breaths of the data. This also was adopted in order to resonate with the methodology adopted in the baseline survey. Since Plan has just finished an internal review on MNCH-N raw data from this review was utilized and re-analyzed, where they were relevant.

These comprise the following:

Quantitative Data

A HH survey using a highly-structured quantitative questionnaire was conducted in July 2015. Using a multi-level cluster sampling methodology, the survey sampled 378 households living in 20 target villages across three target districts of Bokeo. The sample villages were representatives of diverse ethnic groups, a mixture of good and poor performers, easy and difficult access, full and partial coverage by MNCH-N and WASH programs. The inclusion criteria are as follows:

At the villa level:

a representative of at least one major ethnic group

either with difficult or easy accessibility to a district center or health centre

either good or bad performers according to the reported indicators

either covered by both or one program (to compared impacts and synergy)

At the household level:

Households with children under the age of five years old were prioritized. The evaluators learned from Plan’s staff that if households with children were not prioritized, many of the questions concerning MHCH-N activities would be left blanked. In these households women were asked to take part in the survey. In the case where households with children could not

12

be found, men were asked to take part in the survey. This is to strike for a gender balance of the respondents.

The selection of sampled villages was conducted in a participatory workshop with Plan National and provincial managers of WASH and MNCH-N in Plan CO in Vientiane. It was decided to take 20 % of target villages with overlapping coverage (both MNCH and WASH) as key determinant. However, to compare the results, two villages with presence of only MNCH-N were included.

The length of the Program presence was also considered when selecting sample villages in order to assess the likelihood of sustainability and functioning of the local government institutions. (For example, target villages where Plan was present for the last 3 years). Within a village, 20 % of total HHs were randomly selected to participate in the HH survey. Table 2 below shows the sample frame for the HH survey and FGDs.

Table 2: Sampling frame for the household survey and FGDs

Name Ethnicity No. HH

No. HH survey sampled

No. HH FGDs sampled

Meung (MM)

1 Phonsavang Museu 117 23 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities

2 Monelaem Akar 64 15 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities

3 saychaleun Khmu 76 15

4 Longphabath Museu 169 33

5 Namkhaleu Leu 74 15

6 Donemoun Leu 48 15

Paktha (PKT)

1 HouayNorkhom Hmong 68 15 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities

2 Houaythong Khmu 116 23 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities

3 Houaymong Khmu 99 19

4 HouayKhot Hmong 42 15

5 Houaysaed Khmu 155 31

6 Donemixay Khmu 63 15

Pha Oudom (POD)

1 Phonkeo Khmu 108 21 6-8 men and 6-8 women 6-8boys and 6-8 girls

13

Name Ethnicity No. HH

No. HH survey sampled

No. HH FGDs sampled

Village authorities

2 Thamphakae Hmong 89 17 6-8 men and 6-8 women 6-8boys and 6-8 girls Village authorities

3 Tinpha Hmong 80 16

4 Mokkhatheung Khmu 52 15

5 Namyaokao Leu 85 17

6 Houaykoun Khmu 105 21

7 Parkhard Lao 123 24

8 Phienghad Khmu 59 15

Total 6 Ethnicities 1792 380

The questionnaire collected demographic information, opinions, and factual data. The development of the tool has been based on previous experience in similar surveys by the Team, and the support from Plan Regional Office.

It is important to note that the sampling approach adopted in the joint WASH and MNCH-N baseline survey 2013 was not followed for the following reasons. The baseline methodology was based on the Lot Quality Assurance Sampling (LQAS) methodology which used the supervision areas (SAs) or catchment of a HC/district hospital as a key determinant of a subgroup of survey population. The target populations for the baseline were women 15-49 with a child <1 month, children 6 – 23 months, and children 12 - 23 months. A questionnaire was developed for each specific population. Adopting such an approach would not allow the evaluation to ensure conditions such as remoteness, reported performances, ethnic group specialty as described above. Using a proportion to size, villages were selected based on a district as a key determinant of a subgroup of evaluation population. By using either SA or district catchment the sample size of the baseline survey and the evaluation was not much different -392 and 378 respectively.

Qualitative Data

Key informant interviews were conducted at central, provincial, district and communities.

Representative of key stakeholders were selected for interview. They have been identified through the project stakeholder mapping

Focus group discussions were conducted aimed at collecting opinions from a variety of stakeholders at local level, comprising men and women, boys and girls (separate group) with an attention to ethnic origin and disability inclusiveness and elimination of all form of discrimination. On village levels this included interviews with 24 focus groups from 6 sample villages of three target districts.

In each village, the team was introduced to participants to provide them with general information in an introductory meeting chaired by Head of Village in presence of 20 to 36 villagers before they were asked to form groups for FGDs.

14

Technical observation: evaluators conducted site visits to representative water and sanitation facilities to examine the work quality, and functionality of operation and maintenance. Observation was made with three HCs using prepared checklists.

Data management and analysis

Stata software, version 10.1 (Stata Corp., College Station, TX, USA) was used for quantitative data management and analysis.

There were 378 respondents participating in the survey. However, after screening 13 cases were eliminated because of conflicts of information in their responds. Of 365 women interviewed, 80.6% belonged to the age group of 18 – 34 years old; 48.8 % were from Khmu ethnic group. The average age was 26.8 years (ranged 15 – 49 years); a high proportion of women did not attend school (42.7 %); followed by attending a certain grade of primary school (31.2 %). 95.1 % of women interviewed got married and the age when they married were mainly from 15 – 17 years old and 18 – 20 years old (41.4 % and 32.3 % respectively).

The nutrition status of children aged less than 2 years was evaluated. 279 children were measured 47.3 % of which was female. The average age in month (m) was 12.2 m (SD = 7.0 m, min= 0 m and max 23 m).

Data analysis was structured into two components: performance evaluation and qualitative assessment. The performance evaluation was based on measurement of the achievement of the main results compared with the project’s initial purposes in term of outputs, outcome and goal. The qualitative assessment assessed the Program designs, approaches adopted, implementation process and monitoring and reporting system.

Project achievements were evaluated based on the degree of the realisation of the objectives in terms of relevance, effectiveness, impact and synergy, sustainability, scalability, gender, accountability and inclusion (in relation to Plan’s child centered Community Development standards) as set in the TOR.

Constrains and Limitations

The evaluation was taking place during the two months’ school break period. The team could therefore neither meet children at school nor observe the school WASH facilities being used. Instead children were met in the village premise and efforts were made to request teachers or school principals, if they lived in the village, to enable the team to visit the schools and observed the conditions of the school WASH facilities.

As mentioned earlier, the Plan began in 2012. With staff turnover in Plan and partner organizations, it was difficult to obtain some important Program background information that was not documented. For example, the information on the changing proposition about disability inclusiveness in WASH facilities was difficult to obtain.

In some cases, it is also possible some information might be government oriented bias. During the field work, government officials, i.e. the district coordinators, accompanied the evaluation team to some of the meetings at village level. While these officials to a great extent facilitated a successful schedule and did not in any way directly interfere with the team’s role as an independent entity, possibility of their mere presence having an impact on the outcome of the interviews cannot be completely ruled out.

15

It is of note that the difference in the sampling technique adopted in the current evaluation and the baseline survey. The difference made it difficult to directly compare the results with the baseline. This was more so for MNCH-N Program where more comparison was made.

Gender and social inclusion was integrated into a gender analysis based on the gender equality principles in the Gender WASH Monitoring Tools (GWMT). However, we could not arrange FGDs by age groups such as young people, middle age and elders as the tools indicated due to difficulty in field work arrangement. We were able to conduct FGDs with groups of women, men, boys and girls separately. In terms of gender, the analysis focuses on four key indicators including gender and WASH roles at household level, gender and decision making at household level, gender and community WASH participation and gender and decision-making at community level.

This analysis assumes that the common social norm in all ethnic groups is that women are the bearer of the home chores including taking care of children, cooking, fetching water and cleaning of general environment of the house.

16

2. Results, achievement and progress

2.1 Achievements of the WASH program

This section presents the achievement and progress of the interventions towards the set objectives. This report presents only the results of the Midterm Evaluation of the WASH Program in Meung, Parktha and Pha Oudom Districts of Bokeo Province. It has to be pointed out that the WASH project suffered from the late start-up due to delay in the signing of the MoU. Therefore, to some extent this delay in MoU singing could have an impact on rolling out of the project activities.

Achievements and progress toward specific objective 1

“Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming “safe” drinking water among community members, particularly among women and children in selected target communities and schools”

As of June 2015, the project has achieved 3 of the 10 annual targets by 100% making those three highly likely to meet the targets set for 2016 . These include indicator 1.8 “% of ODF villages that maintain Ban Sam Saat, Model Healthy Village, SoQ, ODF standards 1, 2, and 3 years after certification”, indicator 1.9 “% of target villages that have village sanitation and hygiene regulations” and indicator 1.10 “% of trained District WASH team members undertaking project activities on regular basis in schools and villages”.

The progress reports did not monitor achievement under indicators 1.3-1.7. The results indicators 1.4 “% increase among women and children who know least three critical times to wash their hands (target 80%)” and 1.6 “% increase among women and children who know at least two reasons that cause diarrhoea (target 80%)” were discussed under the section Effectiveness. The indicators 1.3 “Number and % of episodes of diarrhoea reported in sample target villages”, 1.5 “% of reduced costs for medical expenses related to poor sanitation” and 1.7 “% reduction of number of sick days from work or school resulting from being sick (with diarrhea)” were not assessed in the evaluation as the MER review recommended that these indicators were too difficult to record the progress and should be dropped.

o Indicator 1.1 “total number of target villages meeting Project ODF and HP conditions (officially declared) that correspond to Model Healthy Village indicators related to WASH (No. 1, 2, 7, noting that Ban Sam Saat indicators are included in MHV indicator no. 2” - 78% (91 villages) of the target (117 villages) has been achieved by June 2015. Indicator 1.1, therefore, is well on track and likely to achieve the end of the project’s target of 91% of the target villages (128 villages).

52

70

91

23

40

53

0

20

40

60

80

100

120

2 0 1 3 2 0 1 4 J U N - 1 5 J U N E 2 0 1 6 ( T A R G E T )

I N D I C A T O R 1 . 1 A N D 1 . 2 P R O G R ES S

Villages

Schools

Linear(Villages)Linear(Schools)

17

o Indicator 1.2 “total number of target schools meeting Project and HP (star topics) conditions (Officially declared) that correspond to Lao PDR School of Quality indicators related to WASH criteria and inclusion” - 45% of the target achieved

DWT members and village authorities interviewed reported that there were three main aspects to difficulty achieving the targets. According to the district staffs interviewed, one reason is that there was generally a low level of awareness of good hygiene practices among a large section of community members therefore they did not build toilets even after attending CLTS process. The view was well supported by the VHCs and VWCs. The second aspect is that many community members were so poor that make them worried about having something to eat than thinking about “where to defecate”. Some KII respondents reported that the implementation of CLTS activities such as triggering and post-triggering following ups at the community level contributed to the low performance for indicators “1.1” and particularly for “1.2”. They were of the opinion that many DWT members lack the facilitation skills that are participatory, the skills that would encourage villagers to build toilets and improved other good hygiene practices. They admitted that the CLTS process relies on the participatory and active engagement methods which many facilitators have limited skills on. It is true that there had been refresher trainings provided to DWT members annually, but many reported that some members were still not comfortable applying the participatory methods. The KII respondents reported that the training they received and the field work where they could put new knowledge and skills in to practice were not frequent enough. It has to be cautious for the analysis of performance for indicators 1.2 though that the testimonies from school officials were not obtained and incorporated into the analysis. In conclusion, the low performance on some parts of the indicators resulted from ineffectiveness of the CLTS implementation by DWTs as well as the low level of general awareness about health issues from the communities.

Indicator 1.1 and indicator 1.2 involved behavioral changes to meet the target. The KII respondents reported that the poor success rate is attributed to general poverty among villagers of the target villages that make them unable to afford toilet construction cost. They also claimed that behavioral change takes time. One respondent said “many of the villagers only learned to use salt a few years ago, now we want them to use toilet, it is not going to be an easy objective to achieve”. The participants referred to some of the target communities that had just resettled from their traditional temporary settlement that moved from place to place. Before, they had little exposure to the outside world and even salt was difficult for them to have access to. They also claimed the issues concerned behavioral change such as values, attitudes, self-efficacy and habits were deeply rooted within the ethnic minority groups who until recently live their life close to the nature. The FGD results also shows that villagers spend a lot of time in the field, they were shy to interact with officials including the evaluators and their cultural beliefs such as their use of traditional healers do not support behavioral change.

18

Table 3: The targets and the reported achievements of the objective 1

Indicators Year 1

(Jul 12-Jun13)

Year 2

(Jul 13-Jun14)

Year 3

(Jul 14-Jun15)

Jun ‘16 targets/# Achieved (% of

target achieved)

Objective 1: Personal and household hygiene in selected target communities and schools, improved and maintained, particularly by women and children (All indicators cumulative targets)

1.1) total number of target villages meeting Project ODF and HP conditions (officially declared)

52* 18 21 117/91 (78%)

1.2) Total number of target schools meeting Project and HP (star topics) conditions (Officially declared)

23 17 13 117/53 (45%)

1.3) - 1.7) some of these indicators were assessed in this evaluation and presented in the effectiveness section.

1.8) % of ODF villages that maintain Ban Saat, Model Healthy Village, SoQ, ODF standards 1, 2, and 3 years after certification

96% 96% 95% 100%/95%

1.9) % of target villages that have village sanitation and hygiene regulations

100% 100% 100% 100%/100%

1.10) % of trained District WASH team members undertaking project activities on regular basis in schools and villages

100% 100% 20 (10 f),

Active 18 (9 f) 100%/100%

*included 2011-12 target villages.

Under indicator 1.1, one of the key important conditions is the consumption of HH treated/boiled water. To this end, women are still facing disadvantages. For FGD respondents, safe drinking water almost equaled boiled water. They also admitted that boiled water is only available at home (villagers reported that they did not bring with them boiled water to the fields). When considering the fact that women spend more time than men in the field: collecting firewood, weeding in the upland rice field (hai), or searching for food in the mountain/forests, they are likely to miss out on boiled water.

As shown by indicator 1.2, the number of schools meeting the “star topics” conditions had been under-achieved. This meant that children are not able to have access to improved sanitation and hygiene while they are at schools. Given the challenges, some of which are beyond this project’s sphere of influence, the targets set by the project for indicator 1.2 can be too ambitious. It was reported by some children in the FGDs that in some schools teachers locked up the toilets keeping it for their own use, claiming children did not keep the toilets cleaned. This was the case in two out of six target villages visited by the evaluators. This has negative impact on children’s access to sanitation facilities. VEDCs reported that even though teachers were oriented about toilet uses, some still keep the toilet use to themselves. The key reason given was “children make the toilet dirty”.

19

It was reported that schools across the 3 project districts have functioning gender-segregated toilets that are being used by students. However, due to the school vacations, we could not observe how children use the WASH facilities in schools (evaluators managed to visit three schools). From three schools what was observed was that the toilets have separate rooms for boys and girls. They all looked clean with water available. The latrines were not modified for disability inclusiveness but looked child friendly: the water storage and taps located at the level easy to reach by a child. There are hand-washing stations built with three levels of heights suitable for different age groups to use. The school premise was kept clean particularly around the building and toilets. Even though there were places made particularly for putting soap, there was no soap there, because it was the vacation period. It was evident though that soap had been in use in the stations which was confirmed by the accompanying teachers and village authorities. Since hand washing with soap (HWWS) is a new concept in the area, the evidence of HWWS in this hand washing station shows the presence of “enabling conditions for behaviour change” that was taking place as a result of the project intervention.

Achievements and progress toward specific objective 2

“Achieved and maintained ODF status in 80 % of the selected target villages”

Interviews with the project staff revealed lack of clarity about indicator 2.1 and 2.2, particularly around their difference from 1.1 and 1.2. However, document review clarified 2.1 as the number of places where both school and village was ODF. As can be seen from Table 3 below, by June 2015, the project has achieved ODF status in both school and village in a total number of 46 villages in Bokeo, 95 % of the June 2016 target according to the monitoring reports, but this could be inaccurate due to the fact that there was no clarity among project staff about the indicator. Similarly, under 2.2, what has been reported is the number of schools achieving ODF (a total number of 53, which is the same as 1.2) rather than the number of WASH facilities meeting the Ministry of Education standard which is found to be the meaning of the indicator 2.2.

Indicator 2.4 Number of additional people with increased access to basic sanitation (no. of HH toilets x 6 persons/HHs avg.) has increased steadily moving toward the 2016 target. In year 1, the project has enabled 5,694 people to have access to improved sanitation, this increased to 9,912 by June 2014 and 15,804 by June 2015. The project has achieved 88 % of its 2016 target of 18,000 people. By this rate of performance the project is likely to over-achieve its target by the end of the project. For the rest of the achievement under this objective see Table 3 below.

In summary, the project had made significant progress toward objective 2, only about 20-25 % remains to be done to achieve the 2016 targets. However, the target regarding ODF village remain off-track in terms of meeting the targets. But there are many factors contributing to this under-performed indicator. One of the reasons is the over-ambitious target as mentioned earlier.

The project progress reports show that by June 2015, the ODF retention rate has been 95% of total ODF villages. During the FY15, the slippage rate is 4 to 5 %. By June 2013, out of 32 ODF villages, 7 villages had slipped, losing the ODF status. By June 2015, 3 of these villages remained non-ODF. This means that some villages that lost ODF status had been reinstated the ODF status as a result of those slipped HHs rebuilding their toilets. The KII respondents

20

reported that the slippage in their villages mainly occurs with the dry-pit type of toilets structures of which are simple and prone to extreme weather events such as heavy rain. Based on the narrative reports of the project slippage have also occurred due to termite damage of local woods being used for toilets.

Table 4: The targets and the reported achievements for the CLTS intervention component (source: annual narrative reports 2013-2015)

Indicators Year 1

(Jul 12-Jun13)

Year 2

(Jul 13-Jun14)

Year 3

(Jul 14-Jun15)

Jun ‘16 targets/ % Achieved

Objective 2: Achieved and maintained ODF status in 80 % of the selected target villages. (All indicators cumulative targets)

2.1) Number of officially declared ODF communities (both school & households ODF)

18 26 46 94/95%

2.1a) Number of village toilets constructed

979 1,648 2,414 3000

2.2) Number of (Plan) school WASH facilities in target villages that meet MoE standards (toilets and hand washing stations) repaired or constructed (AND FULLY OPERATIONAL)

25 40 53 53/100%

2.3) Number of buildings and facilities made accessible to people with disabilities (mainly applied to School WASH facilities)

DESIGN

modification 3 redesigned 25/12%

2.4) Number of additional people with increased access to basic sanitation (no. of HH toilets x 6 persons/HHs avg.)

5,694

9,912 15,804 18,000/88%

2.5) % toilet coverage in all target villages to date (including those triggered in 2011-12)

76% 81% 81% 90%/84%

2.5) % toilet coverage in all target villages to date (including those triggered in 2012-13) (CUMMULATIVE TARGETS)

67% 69% 69% 90%/72%

2.5) % toilet coverage in all target villages to date (including those triggered in 2013-14) (CUMMULATIVE TARGETS)

47% 62%

76%

90%/84%

2.5) % toilet coverage in all target villages to date (including those triggered in 2014-15) (CUMMULATIVE TARGETS)

72% 90%/

21

2.6) % of “poorest” (in triggered villages) having access to toilet facilities

17%- 42% 53-59% 90%/80%

2.7) % Increase of women making decisions in household about toilet provisions

72% joint baseline

2%

90%/

During interviews with the FGDs and KIIs respondents, the team found that most villagers prefer the pour-flush type of toilets because the limited space at their home premise means that they have to build toilets close to their house. In cases where proper dry pit toilets were not built and maintained well/hygienically, people reported that it cannot contain the unpleasant smells. So the pour flush were preferred over the dry pit. Facilitators reported that it is quite a dilemma for them to promote low cost dry pit toilets among those low/no income communities while at the same time have to promote a well/hygienically built toilets. This also indicates that facilitators often tried to influence the toilet type.

More discussion on effectiveness of CLTS and progress toward targets will be presented in the EFFECTIVENESS section.

Indicator 2.7 shows huge difference between baseline and evaluation findings. In the baseline, 72% of the women reported they made decisions concerning sanitation provision in the household. In this evaluation only about 2% of the respondents reported wife made the decision concerning toilet construction in the household. This difference may come from the fact that the evaluation asked the respondents to choose between husband, wife or together. This may lead to the difference in the results.

Achievements and progress toward specific objective 3

“Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages”

Under this objective, key activities include construction of Community Water Systems CWSs, training on water safety plan, implementation of the village regulations and establishment and training of the village water committees. There were 879 additional persons (537 female) who have been provided with access to safe water. This result has been achieved by the construction of 2 additional CWSs in Year 3 supported by ANO in Houaysoua and Tinpha villages in Bokeo. By June 2015, a total number of beneficiaries that Plan WASH Program provided access to safe water reached 2,754 (including 1,460 female). The achievement of 48 % of the 2016 targets as shown in Table 4 below reported only ANO funded CWS (2 CWS FY15). However, the achievement of the 2016 target will be met by CWS supported by other projects. For example, there are 2 other projects on going with 10 village water system. Training on water safety planning was conducted for a total of 26 participants (including 2 female) supported by UKNO. The participants came from DWTs, water supply officials from all 5 districts of Bokeo province including the two non-target districts and the environmental health and water supply centre (Central Nam Saat). The KII respondents reported that the

22

knowledge and skilled gained is being used by the participants in the villages where the new water systems are being built. See Table 4 below for more details.

By June 2015, 8 villages implemented village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines. The village authorities interviewed reported that the training and implementation of the village regulations and the water safety plan contributed to the development of health conditions that contributed to meeting the conditions of MHV, the initiative under MNCH-N Program. All eight water committees in Bokeo were established and regularly collected fees for maintenance of CWS according to an agreement in each village. The arrangement in terms of maintenance fund for CWS varies from village to village. The most common practice found from interviews with village authorities is that they designated water use groups organised by public water points. The HH in the catchment of a particular water point was held responsible for the maintenance of that water point.

Table 5: The targets and the reported achievements for the WASH infrastructure construction

Indicators Year 1

(Jul 12-Jun13)

Year 2

(Jul 13-Jun14)

Year 3

(Jul 14-Jun15)

Jun16 targets/

% achieved

Objective 3: Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages. (All indicators cumulative targets)

3.1) Number of people provided with increased access to safe water

250 / 118 f 1,875 / 923 f

2,754/1,460 f 10,000/48%

3.2) District and provincial authorities water safety plan monitoring system in place and updated regularly

Not started Not started 26 from other 2 non-target districts

21/26 (124%)

3.3) % Village regulations include water catchment protection in line with O&M and WSP guidelines

5 11 13 21/24%

3.4) % Water committees collect dues from user households regularly (depending on village decision)

11 13 21/62%

3.5) Number of safe, sufficient and protected water supply points constructed or rehabilitated

1 CWS 2 CW 8 CWS 17/47%

3.6) Number of village committees that have training on water safety planning (including 3 2011-12 villages + 10 2012-13 villages)

5 village water committees

7 village water committees

13 village committees

21/21%

3.7) % of “poorest” participating in water supply decisions in their villages

Not reported this quarter.

Not reported this quarter.

Not reported

This quarter

23

Achievements and progress toward specific objective 4

”Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level”

This objective focuses on key good governance principles such as participation in the decision making concerning WASH in schools and communities, satisfaction of end users with the management and governance of WASH system or activities established or initiated by the WASH Program and women and marginalized population contribution to such management and governance. The first indicator of this objective is concerned with participation in decision making in the WASH in schools and communities. However, monitoring on this indicator in school is difficult and has not been done recently. The evaluation surveyed participation in the WASH governance and management at community level with 87% of the respondents reported they were actively involved in the decision making concerning WASH facilities in the community.

By June 2015, Behavioral change communication (BCC) activities had not been fully implemented by the district WASH teams. On average, 50 % of planned activities had been implemented. For example, 33 % of BCC activities were implemented during Year 3 or FY 2015 under ANO grants support. It is important to note that the percenrage had dropped from 75 % in year 2 to 33 % in Year 3. This is a significant decline within one year timeframe. It was reported in the progress reports that the reason for under-achievement of the target is time constraints faced by the PHD and DHO staff. The time constraints faced by staff should not be that much different over one year given there was no reduction in the number of government staff during this period. According to the KII results (presented in the Effectiveness section), the reasons for many activities being delayed are many including high staff turnover rate for both Plan and the implementing partners, ineffective communication and coordination between project staff and the government staff, delay in the process of activities planning, implementation and reporting.

In terms of women participating in village meetings related to WASH, according to GWMT findings, it was shown that even though all women had been invited, many women in rural areas were not able to participate fully in the meetings because of language barrier (most meetings had been conducted in Lao Lum and there was no particular method to facilitate the meetings with participants from a diverse language background) and they are shy to share their opinions. For women taking on leadership (decision making) roles, 50% of District WASH team members are women in Bokeo. 14 % of those assuming leadership (decision making) roles at the local level (village committees) are currently women although even they may not be confident and have equal status to the male counterparts.

The first training about disability, inclusion, and equity for Plan staff at Bokeo PU was conducted under a partnership with the Lao Disabled People Association (LDPA). The training was held for 4 days by 3 (2 female) trainers from LDPA (one trainer is a person with disability). The total number of participants was 21 (9 female), this included 5 staffs from WASH team (1 female). KII respondents reported that they understood more about comprehensive categories of disabilities and were able to recognise different types of disabilities exist in the project areas. They claimed to have had encouraged participation of PWDs in the project activities. However, they admitted they did not have specific technics/skills for working with PWDs. See more details achievement in Table 5 below.

24

From the KII interviews, it is clear that the project implementation partners including provincial and district officials were actively engaged with the project activities. Many activities of the projects including those under MNCH-N program were in their plan but lack the budget to implement them. The government at all levels said that they were cooperative with development partners because they had access to financial and technical assistance by doing so. Moreover, the government staff at district and HC levels reported that they gained hands-on experiences from many technical/practical trainings. The government staff admitted that while they were able to put what they have learned from the project into practice by the project funding but this could finish once the financial support ceased with the project’s exit.

Table 6: The targets and the reported achievements for the WASH governance

Indicators Year 1

(Jul 12-Jun13)

Year 2

(Jul 13-Jun14)

Year 3

(Jul 14-Jun15)

Your 4

(Jul 15-Jun16)

Objective 4: Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level. (All indicators cumulative targets)

4.1) End-users expressing their opinions and are active in decision making in schools and communities

88% 87% 80%/110%

4.2) % of activities in District BCC plans achieved by DWTs

50% 75% 33% 100%/33%

4.3) Level of satisfaction among end users with WASH governance in their villages, toilet and water supply provisions

n/a

n/a

n/a

80%/

4.4) Number of women (% of total participants) expressing they have been invited and participate in village meetings related to WASH

87% 33% 90%/37%

4.5a) % of women on District WASH teams

60% 60%. 50% 60%/100%

4.5b) % of women assuming leadership (decision making) roles at local (village committees) village CLTS committees

22%

14%

50% /40%

4.5c) % of women assuming leadership (decision making) roles at local (village committees) Village water committees

31% 50%/62%

4.6) Total number of women, especially caretakers, provided with gender awareness raising/training that enables them to better participate in community processes

11 women No training 36 2000/0%

(target for both UDX and Bokeo)

25

4.7) Number of men provided with awareness raising/training on gender issues and women’s equal rights

n/a 10 18 2000/0%

2000/0%

(Target for Bokeo?)

4.8) Number of professionals trained in “disability inclusion”

n/a n/a 21 (9 female) 25/0%

2.2 Achievements of the MNCH-N program

This section evaluates and discusses the achievements of the specific objectives and outcome of MNCH-N Program in Bokeo.

Achievements and outcome for specific objective 1 (SO1):

“Contribution to the development of a sustainable primary health care system in Bokeo province”

Strengthening the Health management Information System

The MNCH-N Program successfully contributed to improving the Health Management Information System (HMIS) through strengthening the capacity of PHD and DHO staffs. These included provision trainings in computer skills. In this regard, two training courses were provided. The first was organised in Pha Oudom in 2012; 25 DHO staff (5 women) attended. The second was held in Paktha in 2014; 17 DHO staff (8 women) attended. Plan also supported the DHOs and PHD in compilation of integrating village and health centre level family/patient files including disease specific statistics, in three target districts to facilitate vital information gathering and storage. Capacity of the PHD and DHO staff regarding this work had been significantly improved.

The new obtained knowledge and skills were usefully applied to the staff routine job. The site observation conducted during the field visit found that DHO staff were not comfortable using the computer; however, they were able to use it for their daily work and stored key health information properly as a result of support by Plan. At health centres, the data files were well-arranged and stored and important information for target communities were available. For example, information about a number of target children for vaccination, monitoring growth, and children U5Ys were recorded and updated regularly. DHO and HC staff interviewed commented that the trainings were very useful and applicable to their routine tasks. The increased knowledge and skills facilitated their job immediately such as micro planning process which took place every month.

At the village level, village health committees and village heads were provided with trainings on vital statistics and other health related data recording and reporting which fed into the national system. The data concerning the conditions of MHV was prioritized because MHV was known as a part of village data statistics that facilitate village committees as well as the local government institutions in improving the health and hygienic practices in the communities. MHV is the government initiative that Plan has been supporting where Plan also presents, at the moment in Paktha and Meung districts.

26

Table 7 below shows that initially 10 villages were targeted and 04 out of 10 (40%) achieved MHV status and 06 villages were under monitoring. In 2014, 24 villages were targeted, 19 out of 24 (79.2%) achieved the status and 05 villages were in the process. The rate of achievement increased significantly from 40% in 2012 to 79% in 2014, an increase of almost two folds in two years. The majority of the KII respondents believed that the integration of WASH and MNCH-N program activities in the same village gave a synergistic effect which accelerated the target villages to achieve the MHV status. This was due to the two programs delivered the same or similar health and hygiene messages. The integration of the programs will be discussed in greater depth in the section on “Impacts and Synergy”.

Table 7: Achievement of establishing of MHV in three target districts

FY Target village/s Achieved MHV MHV In process

2012/13 10 04 (40%) 06

2013/14 24 19 (79.2%) 05

However, assessment of the MHV’s conditions remains a challenge. It was reported that the assessment happened ineffectively. The KII respondents admitted that they (leadership within the provincial government) were preoccupied with the mandate of meeting the quantitative target. This often led to compromise of the conditions during the assessment of the MHV indicators17. The indicators in the government’s MHV evaluation framework that involve some subjective evaluation were the most problematic. The respondents said that assessment of indicators such as indicator 1 “Eating, Drinking, and Living Clean” was difficult and the assessors often gave a “pass” assessment assuming their management wanted it that way.

Strengthening district, health centre and outreach management

To ensure monitoring and supervision in communities, Plan purchased 02 four wheel vehicles, 01 motor boat and 19 motorbikes. This allowed the government counterparts and Plan staff to have all year round access to the communities. A quarterly meeting and annual meeting was help and attended by PHD, DHO and Plan staff. The meetings were meant for implementing staff and the government counterparts to reflect on the project progress, identified issues/problems and collectively addressed those concerns/issues as they arisen. A study visit to Xayaboury province was arranged, supported by an Australia Children’s Aid Project.

Plan had supported a series of reviews and planning meetings as part of the project implementation. So far, 17 meetings were held (3 annual meetings, 10 quarterly meetings and 4 orientation meetings). Total participants attended were 712 (256 women). Table 8 below shows a high number of staffs attending the meetings. But these meetings often took

17 MHV Indicators: 1) There is Eating, Drinking, and Living Clean; 2) There is a clean Latrine; 3) Environment is clean; 4) There is

vaccination according to MoH standards; 5) Pregnant women have access to MCH services in the local Health center; 6) Elimination of

mosquito breeding areas and all persons sleep with mosquito nets; 7) There is year-long access to safe water; 8) Animals are kept in pens

away from the house; 9) There is knowledge on TB prevention; 10) There is knowledge on HIV prevention.

27

place behind the schedule. KII respondents reported that health staffs had many responsibilities and it was difficult for them to be available at the same time for the meeting. In other words, there are too many meetings that demanded too many people to attend. In fact a monthly DHO meeting was organized at DHO and chaired by DHO director/deputy director, with a representative of each health centre within its catchment areas attending the meeting. In order to address the issue of too many meetings the PHD and DHO should incorporate a quarterly meeting in every third monthly meeting of DHO. During this meeting, the program works can be discussed.

The MNCH-N Program Plan staff regularly reported progress to headquarter. The government counterparts at the district level reported quarterly and annually to PHD. In turn, PHD reported to the MoH on six monthly and annual bases. However, KII respondents at the central level reported that the core concerned institutions at the central level such as the Nutrition Centre and Ministry of Education and Sports claimed did not receive any reports concerning the MNCH-N Programs’ activities in Bokeo. In principle, the PHD must report monthly, six monthly and annually to its vertical and horizontal lines. Reporting to its vertical lines means reporting from VHVs to HCs. Then the HCs report to DHO to the inter-HC meetings held at DHO that chaired by DHO director, DHO to PHD through inter-DHO meeting held at PHD that chaired by PHD director. PHDs report to the MOH (concerned departments and specialist centres: nutrition centre, MCH centre, etc.). Reporting to its horizontal line means reporting to the district/provincial inter-sectoral meetings (Education, Health, Agro-Forestry, Social Welfare, Information and Culture, Transportation and Communication, etc.) chaired by district/provincial governor.

Table 8 Number of annual and quarterly meeting and number of staffs attended

Number of meeting and its participants Total

Item

No

me

eti

ng 2012 2013 2014

No

staff F No

staff F No

staff F No

staff F

Orientation meeting at PHD (1) and DHO (1 each district) 4 167 54 167 54

Quarterly meeting in 3 districts 3 86 28 86 28

3 Quarterly meetings held in 3 districts 3 111 52 111 52

six monthly meeting held at POD 1 65 34 65 34

Annual meeting held at POD 1 37 3 37 3

Annual meeting at PHD 1 40 15 40 15

Annual review meeting at PHD 1 49 15 49 15

Quarterly meeting held in 3 districts 3 157 55 157 55

Total 17 253 82 213 89 246 85 712 256

28

The findings above show that there is some conflict between the principles and what really occurred in practices in terms of reporting and communication. If the PHD reported to its vertical line the core concerned institutions would have been informed about the progress of the program. Yet they were unaware of the progress. This could mean that PHD did not perform its functions properly. If they did report to its vertical line, then the reports might not have been used as should have been the case.

Improving the quantity and quality of human resource for health and nutrition

The target of improving the quantity of human resource for health and nutrition had been achieved. Table 9 below shows that several training courses on MNCH-N were organized for PHD, DHO, HC and village facilitators/VHV between 2012 and 2014, with a total number of 898 (387 women) attended. In addition, the program had successfully trained 25 women Skill Birth Attendants (SBA) drawing from ethnic and indigenous high school graduates by using another funding source to complement the activities in this project. They were now working as volunteers in the villages and they would be recruited by the government as health staff next year. Plan continued to support 13 students and they were currently undergoing the course in nursing schools and 11 of them would become SBAs. Building a cadre of SBAs, especially from the ethnic minorities, is responding to the need of the Bokeo province.

Even though there was some improvement in terms of quantity, development of the quality of health services remained a challenge. The evidence from the household survey indicates that the ability of DHOs and HCs to take ownership of the program’s activities is limited. Some health centres lacked basic medical equipment and medicine (sometime has no medicine). They have not enough scales and measurement equipment for growth monitoring.

Table 9 Training course organized between 2012 and 2014 and number of participants attended the training course

No Training course 2012 2014

No staff

F No staff

F

1 Training for PHD and DHO FGD facilitators for conduction of PCA in community, trained facilitators conducted FGD with target community members and organized meeting with government counterpart of three districts

402 264

Sub-Total 402 264

MNCH and Nutrition activities

1 Long term training course for midwife (SBA) in Vientiane Technical Health College for 2 year

25 25

2 Long term training course for midwife (SBA) in provinces 13 10

Training for PHD, DHO and HC staffs:

3 4-days training of trainer (TOT) on Parenting Orientation concerning MNCH-N for PHD and DHO staffs

39 30

4 Training on nutrition for PHD and DHO in 5 districts to be trainer for HC and VHV

yes

29

5 TOT on basic nutrition and nutrition treatment from all 5 DHO, expected to be trainer for HC and VHV

33 20

5 TOT for Micro planning training by 3 main trainers from EPI and MCH center

18 10

6 Training on micro planning for HC staff from 19 HC 64 26

7 Training for DHO on how to prepare micro planning in terms of integrated MCH package services

3 0

8 Training on cool chain system management for DHO 9 5

9 Training on cool chain system management for HC staff 6 2

Training for village facilitators and VHV:

10 Trainer on Parenting Orientation continued to train in 57 villages of 3 districts

246 83

11 3-days Training on Parenting Orientation Modules for village facilitators

117 53

12 3-days training on Parenting Orientation modules for VHV 208 97

13 Training for VHV on Positive Parenting Program yes

14 Training on positive parenting module for VHV 117 26

Sub-Total 310 138 588 249

Health Management Information System

14 Training on computer and Microsoft Words for DHO staff of Pha Oudom district by trained DHO and Plan staff

25 5

knowledge increased from 49.9% - 92.2%

15 Training on computer skill for 2 days for DHO and HC staff in Paktha district

17 8

16 Training on MHV components for 6 VHC 38 7

Sub-Total 25 5 55 15

Total 737 407 643 264

During 2013 and 2014, Plan organized three study visits to similar health programs in order to exchange experiences and learn from one another. These are:

Health unit of Namtheun 2 electronic company in Khammouane province, there were 10 district health staffs, 2 PHD staff and 3 Program staff and 4 from other districts like

30

Tonpheung and Houaxai districts joined this visit. The visit focused on integrated health centre, integrated MNCH-N package outreach, and quality of services;

Xaiyaboury district, 20 participants from model health centre, district health staff, PHD, MNCH-N coordinators attended in this visit; and

2014 the study visit was held in Louangnamtha province the UNICEF project area. MNCH team did observe the micro planning training and learnt a lot about well preparation of planning.

Overall, Plan made a significant contribution to the development of the PHC system in Bokeo province through the improvement of capacity of health and none health staff therefore it improves the PHC system in this province.

Achievement and progress for specific objective 2:

“Promotion of universal access to quality health care for the families of three Bokeo districts”

The desk review’s findings show that the Plan’s MNCH-N Program had supported the routine outreach activities plus growth monitoring, mobile clinics integrated with EPI, quarterly health days, health promotion days, Measles-Rubella campaign, children days, nutrition education and food demonstrations, and implementation of parenting orientation in communities. As shown in Table 10 below, a total number of health events had been increasing each year such as 01 event in 2012, 03 and 08 in 2013 and 2014 respectively. A number of villages covered by health events had also been increasing each year such as in 2012, 2013 and 2014 the health events covered 16, 58, and 329 villages, respectively. A total number of beneficiaries reached 69,334 persons. It is important to note that the number of beneficiaries had increased from 3,900 in 2012 to 11,459 in 2013 and 53,975 in 2014. In three districts, the target number of children under five years (child U5Y) was 11,129 and women of reproductive age (WRA) were 17,112. It was reported by the Program that the total number of direct beneficiaries such as children U5Y and WRA who benefited from health events held in communities was 15,052 persons and 24,214 persons (noted: two health events did not disaggregate child U5Y and WRA).

Number of direct beneficiaries accessed to each type of delivery health services during the health events in villages was summarized in the Table 9 and Table 10 above. A number of children U5Y vaccinated, received deworming and vitamin A had been increasing gradually throughout the period from 2012 to 2014 (see Figure 13). Children U5Y had been monitored concerning their growth. A number of women of reproductive age received tetanus toxoid (TT) vaccine increased from 285 in 2012 to 498 in 2013 but it was not observed in 2014 due to the information was not made available for some health events reports. Pregnant women received ANC services had increased over time such as 60, 273 and 296 in 2012, 2013, and 2014, respectively as well as for PNC service a high number of mothers received the PNC visit (Figure 14, Table 10). These achievements are cornerstone of improvement of maternal mortality rate and infant mortality rate in this province and also in Laos therefore it has contributed to achieve the MDG1, 4 and 5 as planned.

31

Table 10: Summary outreach activities and health events held in communities between 2012 and 2014 and number of beneficiaries attended

Activities

Total Village (152)

Total Beneficiaries

attended

Direct beneficiaries

Children U5Y (in 3 districts

11,129)

WRA (in 3 districts 17,112)

In 2012

Quarterly Health day in 2012 16 3,900 3,117 783

Sub-total 16 3,900 3,117 783

In 2013

Quarterly Health day in 2013 30 10,000 3357 6,643

Outreach activities in 2013 28 1,459 568 598

Sub-total 58 11,459 3,925 7,241

In 2014:

Mobile clinics integrated with EPI 47 3,544 2,873 671

Mobile clinics integrated with EPI 134 10,807 3414 7,393

Outreach activity plus growth monitoring 139 2,482 1,510 972

Six times health promotion day 4,130 na na

Two times Health promotion day 222 130 92

Nutrition education and food demonstration 9 553 83 470

Measles-Rubella campaign for children 9m to 10 years

18,597 na na

Children day 13,640 6592

Sub-total 329 53,975 8,010 16,190

Total 403 69,334 15,052 24,214

32

Figure 2: Number of children accessed to health services during health events

Figure 3: Number of woman of reproductive age (WRA) and pregnant woman (PW) accessed to health services during health events

The results from the household survey show, that most children accessed vaccination in village such as 95.5% accessed to measles vaccine whereas the baseline data was 69.2% and 97.3% accessed to DPT3-Hib-HepB3 whereas the baseline data was 65.2%. The access to ANC and PNC care had been considerably improved such as ANC visit 4 times was 47.2% (baseline data was 26.6%), PNC visit within 7 days 58.3% (baseline data was 8.7%), PNC visit within 6 weeks was 35.1% (baseline data was 10.5%).

Plan also supported the improvement of cold chain system management and immunization training. Plan together with government counterpart organized the concerned training to 15 (7 women) health staffs in 2014 (9 DHO staff (5 women) and 6 HC staff (2 women)). It is very important activity to support the MOH in terms of improvement vaccination coverage among children and women in remote areas where it is hard to access.

Overall, the evidence above indicates that the MNCH-N Program has achieved the target in terms of the number of direct beneficiaries to access to health care services in communities: vaccination, deworming, receiving Vitamin A, monitoring child growth, nutrition education, family planning, PW received tetanus toxoid vaccine, ANC, PNC. This achievement was not identified in previous years before implementing MNCH-N Program in Bokeo (report in 2010). The Program significantly contributed to improving the access to basic health need among

33

rural remote population particularly WRA, mothers and children not only in Bokeo but also contributed to the overall achievement of Lao PDR.

The KII results show a mixed picture of results in terms of quality of services provided. KII respondents both Plan and government staff reported that they were very satisfied with the mobile clinics and outreach activities. The strengths of this initiative included the following. It was built on the current government program thus there was high commitment and encouragement from the government side. Involvement of HC staff and VHVs made the implementation smoother: they were familiar with localities and the people, their culture, rituals and general life conditions of the target communities. Nutrition education and food demonstration was very effective. The district staff believed it could be used us awareness raising strategy because it allowed communities not only to see but to taste the food as well. In addition, it did not need a lot of talks (in Lao language) which was difficult to understand for some ethnic groups particularly women. Almost all of the respondents shared the view that the program could have allocated higher budget for this activity. However, they were not sure how effective the health messages were absorbed by the participants of the events particularly during the health promotion day. The staff claimed that for the events such as the health day, the evaluators would not be able to properly assess unless it was held more frequently than how they were doing now.

Table 11: Summary of direct beneficiaries accessed to each type of health services in communities between 2012 and 2014

Type of health services Direct beneficiaries

2012 2013 2014 Total

Children under five years

Child vaccination 205 903 5,391 6,499

Child received deworming 380 1,288 - 1,668

Child received vitamin A 197 1,372 1,092 2,661

child monitoring growth - 3,357 3,544 6,901

Sub-total 782 6,920 10,027 17,729

Women of reproductive age -

WRA received TT 285 498 783

WRA received TT1 - 188 188

WRA received TT2 - 103 103

PW received TT 28 64 92

PW received TT2 - 792 792

PW received ANC 60 273 296 629

ANC visit >= 1 time - 266 916 1,182

ANC visit >= 4 time - - -

34

PNC visit 31 73 84 188

Family Planning 7 414 - 421

Health check and basic treatment 372 953 - 1,325

Nutrition education and food demonstration - - 553 553

Sub-total 783 2,832 2,641 6,256

Total 1,565 9,752 12,668 23,985

Note: Number of beneficiaries accessing to each health services in some health events was not available

Achievements and progress toward specific objective 3

“Engagement of Bokeo communities for health by seeking their advice and encouraging local reflection, prevention and promotion activities”

Plan had successfully completed a Participatory Community Assessment Analysis (PCA) in three target districts after the MOU was signed in 2012. The PCA aimed to assess the health problems and needs in communities and to identify the potential actions and local resources to address those problems and needs. This approach was used specifically with individual, families and communities to initiate a process of empowerment among women, their partners, families and communities. The government partners, PHD and DHO staffs, also benefited from this exercise as they had been trained and had a chance to learn by doing particularly on how to facilitate the FGD and capacity to conduct the FGD with community members (WRA, mother, mother in law, husband, village leader and VHV). Community members in each cluster had taken the opportunity to share their views, worries and wishes concerning the MNCH-N problems and needs.

The staff interviewed believed that PCA was an effective approach because it gave an opportunity for both health care providers and users to meet and discuss the common health problems and needs in communities. In addition, this participatory approach gave the opportunity to remote rural people to express their opinion so that they would not feel neglected by society. They could initiate some preventive health activities based on their lay knowledge.

Based on the PCA assessment, the program was designed with four appropriate interventions including:

1) Providing capacity of care at the household level for pregnant women, newborn and children. This intervention was designed to address the low rate of PW attending the ANC and delivery at health facilities. It was meant to address the root cause of the low rate problem such as shyness, traditional beliefs and financial issues. In addition, they were not aware of the important of health services and the consequences of not using the services. It was reported that PW and postnatal women work hard and the family members thought that it is a normal way of life. Food taboos are still widely practiced in the community;

2) Improving awareness of community of the right, needs and problems related to pregnant women, newborn and children. This intervention addressed the issue of premature marriage. Many young girls got married and pregnant before the age of 18 years old. FGDs participants said that families needed more labor so they encouraged their daughters to get married early.

35

This has also resulted in school drop outs. Women did not seek care in health facilities. Some sought treatment from a shaman in traditional ways. Women groups reported that husbands did not care for their pregnant wife. For example, they said that when pregnant women wanted to go to the health facility to seek ANC, husbands went to rice fields or forests. This is particularly important for women in rural areas as they could not travel to health facilities alone due the road conditions and long distance of travel;

3) A linkage for social support between health services and communities. this intervention responded to the findings of PCA which indicated that villagers could not access to health services due to the long distance of traveling and communities could not afford for the fee of transportation; and

4) Improving health services. The evidence of PCA indicated that the services were available for mothers only on Wednesday. In addition the health facility lacked of human resources and medical equipment that are challenging for mother health child services.

In summary, the program has achieved the specific objective 3 through the application of PCA assessment. Although health issues for villagers remain less important, health care and treatment seeking are becoming better (field KII respondents). In regards to this change the evidence from desk review revealed that the number of children who vaccinated, received deworming and vitamin A during the health event held in communities has increased over time (Table 9 and 10), the HHS indicated that PW had attended ANC ≥ 4 times and postnatal women sought PNC within 7 days and within 6 weeks more and more (Figure 13). In addition, interviewed KII respondents stated that some health staffs were enthusiastic to perform their tasks in community such as health centre staffs asked to be leaders the health event by themselves since they felt more confident after attended training. Interviewed PHD staff really supported this initiative from Plan.

36

3. Relevance

3.1 Relevance of the WASH Program

The WASH project in Bokeo was designed based on the premise and best practice that the availability and use of improved water and access to sanitation in schools and communities will contribute to improvement in health and nutrition conditions among women and children. SLTS was drawn on the “WASH in schools” model of UNICEF that aims to bring safe drinking water, improved access to clean sanitation facilities and improved health conditions to schools.18

Relevance of the project design

The Evaluation concludes that the project is relevant as indicated below:

- The CLTS and SLTS approach have been well tested in Laos where the alternatives such as direct subsidies from external organisations have been proved failure because it has not led to effective use of sanitation facilities and sustainable behaviour change of key hygiene behaviours.

- Adoption of this approach has been endorsed by the government as stated in the National Plan of Action for RWSS and this has been confirmed by KII respondents.

- The targeting and prioritization of the interventions are appropriate as the project target rural areas where the toilet coverage and access to improved hygiene practices were low.

The project was relevant in terms of responding to the need of the community. The sanitation situation in the three target districts for 2012-16, Pha Oudom, Mueng and Paktha in Bokeo reflects national trends of low coverage in rural and poor districts. Among the three districts, sanitation coverage rates are below national target rates, especially when consider actual usage. Open defecation rates are still high, despite the provision of toilet materials to village households.19

Table 12: Summary of estimated water and sanitation conditions in the 4 target districts

Coverage estimates 2012 Pha Oudom Mueng Paktha

School WASH facilities 38% 67% 40%

School HP minimal minimal minimal

Water supply coverage (reported) 85% 100% 94%

Existing water supplies with low or no functionality (reported)

29% 10% 42%

Toilet coverage (estimated) 39% 59% 60%

Effective use (estimated) 31% 47% 48%

Source: WASH program document, 2012

18 Practice Brief, SNV Laos, Water, Sanitation & Hygiene sector, August 2013 19 Plan Laos, “Sanitation and hygiene practices observed in 9 villages in Bokeo Province.” November 2011

37

The selection criteria of the target villages, however, are quite ambitious with a number of selection criteria can be very difficult to meet. Furthermore, some criteria are subjective and difficult to assess. Criteria such as villages with strong leadership and fully committed to the process can be very difficult to assess as the strong leadership and good commitment are subjective and cannot be assessed easily at the outset. The same hold true for an active and enthusiastic school director with good coordination with community. Most importantly, the villages and schools to be selected should have functioning and adequate water supplies20. The condition 5 of new target villages to consider stated in the WASH program document. Plan staff interviewed admitted they might not get any target villages if they follow all 10 criteria strictly. To be discussed in the effectiveness section, however, KII respondents reported that they could not manage to follow up and support post-triggering process at the community level because they worked in too many villages.

Water supply system rehabilitation and construction

The initial design of the project focused on the improved sanitation and hygiene behaviours in schools and host communities including use of toilets and hand-washing with soap. However, KII results show that many host communities had inadequate water supply. To promote sustained behaviour of hand-washing with soap and effective use of toilets, availability of water at both communities and schools is necessary.

FGD results and site observations in six communities visited show that existing water supply systems both in communities and schools were inadequate as shown by broken facilities or not enough water in the dry season. Similarly the HHS results show that across three districts surveyed (details presented in the effectiveness section).

An increase in number of water supply systems construction from 3 to 8 by June 2015 means Plan has incorporated the recommendation from the WASH evaluation 2012. By increasing the number of CWS, it has addressed the need for water supply among the target communities as it emerged. As staff has pointed out the assessment of the water facilities at the beginning were not accurate. The assessment did not assess whether the facilities were functioning well or not. Therefore when the implementation stated to progress it had been found that many water supplies were not functioning. Thus Plan has to increase construction of rehabilitation of CWS.

It can be concluded that construction of water supply systems is very relevant to the community’s needs. Furthermore, the majority of the officials and plan staff reported that they had not done enough to provide water supply to communities. They acknowledged that this comes down to the limited funding available for infrastructure construction. The KIIs revealed that initial assessment of the water systems was inaccurate saying broken facilities were not reported properly.

20 Water supplies (ideally) in villages and schools are functioning adequately to meet village needs. If not, adequate resources either from Plan, government or other organizations can be mobilized during program implementation process;

38

Sanitation component

WASH Program adopted CLTS to pursue its objective of ODF in the target communities. Adoption of this approach has been endorsed by the government through its recognition of the approach in National Plan of Action for RWSS.

KII results confirmed that direct sanitation subsidies had not been working meaning low rate of effective use and poor maintenance. Key informants reported to the evaluators that there were multiple reasons for villagers not using toilet and not maintaining it well. One of the reasons is that there was inadequate water, broken facilities or lack of water during the dry seasons. In terms of toilet use, some said that people don’t have habits of using it or there are still bushes around so that it is more convenient for them to go defecate in the open.

The evaluation team found that Plan’s staff has internalised the CLTS approach well. The government counterparts also believed the approach has been successful in encouraging households to take actions towards sanitation improvement. However, government counterparts also admitted that CLTS results occur very slowly which sometimes do not match the urgent need to meet government (as well as Plan) targets in time. The team also found that there were many CLTS target villages that had been introduced with the government funded subsidies (i.e. Mokso village in Pha Oudom district). The PHD has been inconsistent in their view on CLTS. In 2013, when a “pro-poor” sanitation strategy using “smart subsidies” was considered they opposed it. Now, they introduced a none-discriminatory subsidies to a group of

chosen villages (pro-poor not being considered). It is not a planned activity to introduce subsidy in the target villages or at least not in the Plan’s plan. However, the WASH pilot project evaluation in 2012 suggested that Plan study the potential use of “smart subsidies”. A smart subsides review was undertaken in 2013. Given the complexities of smart subsides, Plan Laos decided not to pursue smart subsides within the project until sanitation marketing options had been tested first within the target areas.

A constraint on the activities of the TWG is that it lacks a staffed secretariat or at least a fulltime officer to spearhead its work beyond the monthly meetings, for example working on advocacy and resource mobilization for sanitation. Currently there is only a designated coordinator for whom this is not a full-time job.

Hygiene promotion component

This component aims at behavioral changes in hygiene practices at school and community level. This component is well aligned with the education sector strategy regarding improving hygiene practices amongst children. It is clear that some indicators within this component are aligned with the SoQ indicators. Also some indicators are aligned with MOH indicators concerning MHV. Therefore, this component is aligned with the sectoral strategy.

The FGDs results indicate that hygiene awareness-raising is very relevant particularly among children. Children participating in the FGDS reported that washing hand with soap has become a new practice. A place to wash hands, however, seems to be a new concept and difficult to buy in. The evaluators explored the concept through FGDs and site observation and came to conclusion that a place to wash hands for the villagers means a bucket with water and soap (dish washing liquid included). The common practice for hand washing takes place in the morning when they wake up, in the evening when they take shower (or wash their body). This is the time when they wash hands with soap. Before and after meals is also common time for hand washing without soap. For those who have toilets, after using toilet is

39

also a time for hand washing. Awareness raising in relation to washing hand with soap is very relevant to improve hygiene practice and need more efforts (more analyse will be provided in the effectiveness section).

The majority of FGDs participants including children groups were well aware of the importance of drinking water treated and stored properly. The participants acknowledged that drinking un-boiled water is a major cause of diarrhoea and other diseases. When probed whether everyone drinks boiled water all the time, the participants admitted most villages drink boiled water when they are at home which means they drink un-boiled water when they are out working in the field. The evaluators did not have a site inspection at schools, but through interviews with VEDCs we learned that most school students drink treated water by using the project distributed water filters and storing it in a closed container.

3.2 Relevance of the MNCH-N Program

The MNCH-N Program in Bokeo is well aligned with the government policy and relevant to the current needs of the province. The project design was consistent with the need of the GoL to provide a universal access to health care services particularly for remote rural population with an aim to reduce malnutrition and child and maternal mortality rates. The program activities complemented the MNCH-N strategies of the government. The program also met the need of the community. This is because the Program design was based on the participatory stakeholder consultation and need assessment.

The program remained relevant throughout the program period. The current HHS shows that the number of women delivered at health facility remains low (27.0%). The majority of women surveyed delivered at home (66.7%) and received assistance from relatives/friends (62.2%). The ANC visit ≥ 4 times is 47.2%. However, the newborn and child health care show high indication with 70.8% of newborn were protected against tetanus and 74.8% of new born was breastfeed within one hour.

It is evident that the MNCH-N Program in Bokeo had been designed to implement through the existing health care system which makes it coherent with the government structure.

The target areas were selected appropriately. As the 2011 LSIS indicated, in Bokeo 46% of children under five years were stunted which is above the national figure (44%) and also infant and under five mortality rates are above the national figure (92/1000 LB and 110/1000LB vs 68/1000LB and 73/1000LB, respectively). The three target districts were the poorest of the province. Therefore, targeting from selection of the province to district prioritisation was very appropriate. The program responds to the community and local government institution needs.

During the implementation of the Program, government institutions like Nutrition Centre, MCH Centre, and EPI were invited by Plan to be core lecturers so that it reflected the common understanding concerning to MNCH-N lessons and ensured the implementation goes in the same direction. It had achieved the contextual harmonization of working groups between project staffs and government partners. This continued engagement with the government counterpart also ensured the Program remained relevant and coherent with the government plan and approach.

40

Delivery mechanism

A steering committee for the project was set up at the provincial level. Following the evaluation in 2012, which reported “the less than satisfactory experience of Plan and other organisations with provincial level committees and implementation” the function of the provincial committee, was shifted to the district level. This means a reduced level of provincial involvement. The KII results show that the distribution of roles and responsibilities between the district and provincial authorities has caused a certain level of difficulty. Cooperation at a higher level is ineffective resulting in a lack of cohesion of the WASH and MNCH-N intervention. For example, the KII respondents reported that evaluation of MHV conditions was undertaken without an involvement of WASH staff (the issue was internal though). The issue of ineffective communication between district and provincial authorities was also raised. For example, the districts authorities reported to their provincial counterparts in a written form, but the provincial authorities demanded an oral reporting as well. The higher level was not updated on the progress of the project by the district as it was supposed to be the case.

DWTS and CLTS Committee were established as delivery mechanism for the WASH activities. The KII results show also that there was lack compliance with their roles by DWTS and CLTS Committee members. Members of the DWTs reported during the interviews that some of the members did go to the field but were not very active particularly those who are not from health background.

Monitoring and reporting

The WASH Program has adopted a comprehensive monitoring and reporting system. The monitoring system disaggregated data by indicators, genders, poverty and social inclusion indicators such as disability inclusiveness. As recommended by the 2012 evaluation, some indicators were harmonised with those of the MNCH-N Program. It was suggested that Plan Harmonize M&E indicators with the School of Quality and Model Healthy Village indicators of government while internal indicators relevant to Plan long-term visions such as ODF to be maintained. To ensure coverage of both government indicators, it was also suggested that the WASH, Education and Health Programs of Plan have a common/ integrated M&E system.

Focus on nutrition

Nutrition is now a priority on the Ministry of Health agenda and with WASH, Mother and Child Health and Nutrition co-located under the same Directorate there is real potential to better plan, coordinate and develop mutually supportive interventions, including both nutrition specific and nutrition-sensitive interventions such as WASH. Members of Nam Saat are now regular attendees of meetings of the Nutrition Technical Working Group. National learning events hosted by Nam Saat, and supported by [Plan], the World Bank, have on several occasions invited representatives from the national Nutrition Centre to discuss the interrelationships and operational linkages between nutrition and WASH21.

The FNSAP was developed in the second half of 2013 in response to a growing awareness that the country was off track to meet its nutrition related targets under MDG 1. This was confirmed by the 2013 MDG Report which highlighted an increase in food poverty as well as

21 Lao People's Democratic Republic Strengthening Water Supply, Sanitation and Hygiene Sector Coordination in Lao PDR: Supporting Sector Reform for Scaling Up Rural Sanitation –Synthesis Report

41

high rates of malnutrition and stunted growth among children12. In mid-2013, the Prime Minister signed a decree establishing the National Nutrition Committee and Secretariat (now based within the Ministry of Health), and the Multi-sectoral Food and Nutrition Security Action Plan (FNSAP) was finalised with responsibility for coordination placed with the National Nutrition Committee.

Since late 2013, there has been a commitment to improving coordination and alignment by both Government as well as Development Partners, in addressing chronic malnutrition. A 'road map' for implementation was developed and agreement reached on starting implementation in six districts in three provinces - Luang Namtha, Saravane and Oudomxay. At the time of writing this report, micro-planning had commenced in Oudomxay Province, including the education, health and agricultural sectors.

The renewed focus on nutrition is also in line with the MOH’s effort to improve the nutrition status of children under-five years across the country with the approach called ‘Convergence Plan’. This approached is well adopted by DPs such as UNICEF, WHO, Save the Children and WFP. The plan involves relevant sectors including health, education, WASH and agriculture and forestry.

42

4 Effectiveness

4.1 Effectiveness of the WASH program

This section presents findings and analysis of four major components of the WASH project including hygiene promotion, SLTS/CLTS, construction/rehabilitation of WASH facilities and WASH sector governance.

Hygiene promotion component

The interventions under this component mainly included hygiene promotion activities undertaken by DWTs on a regular basis in villages and schools, hygiene promotion events to reinforce hygiene promotion messages undertaken by DWTs at the target communities and schools and support monitoring of the exiting government program such as MHV and SoQ. Overall, the evaluation found that the project’s hygiene promotion activities were effective in raising awareness and increasing knowledge about improved hygiene practices. The project had variable results in turning this hygiene knowledge to practice.

The evaluation team managed to conduct FGDs with students in the communities outside schools setting. The students knew least three critical times to wash their hands. They reported that they applied hand washing steps every day. When asked if it has a knock on effect on their parents, the responses were negative. They said their parents were not interested in washing hand in many steps. This suggested that that the project’s strategy of school children taking messages home to influence the rest of their family’s hygiene behaviors was variably successful.

Students both boy and girls groups reported that they drink water treated and stored properly. However we also observed that some children drank water directly from the taps (public water points) suggesting that this knowledge has not necessarily been turned into practice. In some villages, students also reported that they have water filter devices. Some said their teachers do not know how to use the filter properly. We crossed checked with HC staff if it would be possible if a teacher would not know how to use the filters. HC staff confirmed that that could be the case. Even though the teachers or at least the school principals were oriented about how to the device, in some schools those oriented left the school without proper transition of tasks and routines.

Improved HP at HH level is reported to have been adopted by villagers in target communities. FGD respondents reported that they did not have a concept of washing hands with soap before the project started. Now they learned that they should wash hands with soap at critical times (such as after defecation, after cleaning children’s bottoms, before eating, before feeding children and before preparing food).. However, they also admitted that washing hands with soap is not a common practice particularly before eating. For them, washing hands with soap is done in the morning when they wash their face or take a shower and evening when they return from the field. They also reported that hand washing (without soaps) takes place before and after meals (usually three meals a day).

It was observed that a place to wash hands was available in the form of a bucket of water which is usually for general purpose and not for handwashing only. A sink, tippy tap or other type of handwashing place was not present. The project did not promote that a flowing water source when washing hands is more hygienic than within a closed container (e.g. bucket) which can be a source of recontamination. As seen in Figure 4 below the rate of respondents

43

who have a place to wash hands varies significantly from 94% in Meung to just 37 % in Pha oudom. After cross-checking with FGDs and interviews, this could mean the respondents interpreted the concept of a place to wash hands differently. FGD and interview respondents reported that a place to wash hands as a sink does not exist but a bucket of water is available at almost all households. If the respondents replied that they did not have a place to wash hands they might not have a sink. Even though the survey team agreed that a place to wash hands is a designated place for washing hands where water and soap are present all the time. This can be a weakness in administration of the survey, but also this tells us that the concept is open for interpretation.

Figure 4: A place to wash hand and hand washing with soap

Source: this evaluation HHS, 2015 Another important finding is the rate of respondents who know at least three critical times to wash hands (target 80%). As Figure 4 below shows, only about 19% of the respondents know at least three critical times to wash their hand which is significantly below the target of 80%. In all districts, the results of 10%, 17% and 29% were lower than the 2013 baseline (Meung 23%, Paktha 12% Pha Oudom 35%). The different sampling technique adopted may explain the results. In the baseline survey, more women and easier accessible communities were sampled. Thus, the results in relation to knowledge about the critical time to wash hands can be higher this evaluation.

FGD results show variation opinions amongst men and women with women seemed more knowledgeable about critical time to wash hand. A number of reasons could be attributed to this including the effectiveness of the hygiene promotion sessions. Another reason could be that some community members also received hygiene knowledge from health education delivered through mobile health outreaches. The HC staff interviewed reported that more women were engaged during the outreaches. This suggests that reinforcing key hygiene messages from different sources is a useful strategy, but may also mean that CLTS committees at the community level were less effective in promoting hand washing with soap and motivating community members to change their hygiene behaviour habitually. KII respondents confirmed that CLTS committee members were less effective in encouraging

44

other villagers to improve health and hygiene conditions because they are also villagers. Some of them are shy and not skilful in explaining things.

Figure 5: Knowledge about critical time to wash hand

Source: this evaluation HHS, 2015 Knowledge about the causes of diarrhea can be associated with the motivation for villagers to decide to build a toilet. All FGDs respondents including boy and girl groups reported that diarrhea is caused mainly by open defecation and drinking of untreated water (un-boiled water). They also said that un-boiled water is contaminated because they practice open defecation. HHS results show that eating and drinking cause diarrhea. With the majority reported eating food without properly cooked cause diarrhea. See Figure 6 below.

Figure 6: Knowledge about causes of diarrhea

Source: this evaluation HHS, 2015

45

This result could have been attained through many interventions by Plan such as PPGs, MNCH-N and WASH which covered improved sanitation and hygiene practices. It is not reasonable to disaggregate HHS data by program-based interventions. From FGDs, we found no major difference in the results related behavioral changes between men and women. i.e. improved hygiene practices. The table below show the number of people who had participated in the SLTS/CLTS sessions. By June 2015, the total number of people who had participated in the hygiene promotion activities reached 15,611 including men, women, boys and girls. It can be seen clearly that the project had promoted participation of women and children resulting in 576 more women than men participated in the sessions.

Table 13: Number of people with increase knowledge of hygiene practices

Men Women Boys Girls TOTAL

5,835 6,411 2,011 1,354 15,611

Source: WASH annual report FY15

CLTS

Key interventions under this component included establishment of DWTs in each target district, community and school consultation/orientation including establishment of CLTS committees and water committees, construction of WASH facilities in schools and communities, CLTS triggering and follow ups, and technical support of operation and maintenance of WASH facilities.

This evaluation confirmed the effectiveness of the CLTS approach. Figure 9 below shows a number of respondents participated in the triggering sessions against the number of new toilet constructed in the year. The table shows the effectiveness of CLTS in Meung district where in 2012 a higher number of respondents reported to have participated in a triggering session corresponds with a higher number of toilets being constructed. The table below shows clearly that the triggering and follow up sessions were effective in encouraging people to build toilets. The follow-up sessions/hygiene promotion was primarily focused on stopping open defection, and hand washing with soap was seen as a secondary action. Given that other conditions are more or less the same because they were randomly sampled for the survey, higher proportion of people participated in the CLTS sessions build toilets.

46

Table 14: Number of people participated in the CLTS sessions

Meung Paktha Pha-oudom

No % No % No %

Participated* Have toilet now 35 71% 71 73% 85 73%

Not participated* Have toilet now 33 50% 9 50% 11 34%

*including participation in the follow up sessions Source: this evaluation HHS, 2015 It is important to note that even though the CLTS approach is effective at ceasing open defection in villages, the toilet coverage rate is still low because implementation of the approach was not done properly. These include the fact that the teams could ensure triggiering and follow-up sessions were undertaken at a time where maximum participation by community member (or a representative of a HH) were able to attend. Further, the project didn’t make concerted efforts to undertake additional triggering sessions for villages that were not able to attend the original session. The table below shows survey respondents’ participation in the CLTS sessions. In Meung in particular, there were fewer respondents participated in the sessions than those who were not. Across three districts, a high proportion of respondents participated in the CLTS sessions eventually built their toilets (71 % in Meng, 73 % in Paktha and 73 % in Pha Oudom. See table below for more details).

Table 15: CLTS participation and construction of the toilets

Meung Paktha Pha-oudom

No % No % No %

Participation in the CLTS sessions 49 43% 97 84% 116 78%

No Participation in the CLTS sessions 66 57% 18 16% 32 22%

Source: this evaluation HHS, 2015 There are challenges contributing to effectiveness of CLTS. Firstly, there was difficulty operatinalising DWTs. Planning for visits to target communities involves a lot of team building and logistic preparation. This was reported to be quite a challenging exercise. As mentioned earlier, the team is multi-sectoral including health, education, LWU and LYU. In the interviews, DWT members reported that it was quite a challenge to form a full team for CLTS community visits and more so lately when there were members from District Rural Development Office and Planning Office who joined the team. They claimed some DWT members particularly those from District Education Offices were in charge of too many responsibilities and the project’s responsibilities were seen outside their core mandate. The reason was the effective communication and lack of a team long-term plan.

The difference between ethnic groups is explored in the table below. On average, about 50 % of the villagers surveyed participated in CLTS process (triggering session which takes place once in a village). The lower the number of villagers who showed up in these sessions, the more difficult it would be for villagers to decide to build a toilet. In other words, if they are

47

not triggered there would be no desire to change Figure 11 below also shows that the rate of participation varies from ethnic to ethnic with Kouy and Aka among the lower rate of participation and Muser is the lowest. In consequence, the rate of respondents having a toilet also varies with Muser having the least number of toilets.

Figure 7: CLTS participation and access to sanitation by ethnic groups surveyed

Source: this evaluation HHS, 2015

The DWT members interviewed revealed that with additional team members it did not mean that the team would become more effective. Some members were just hanging around and did not perform any important team tasks. The reasons included those hanging around lack technical background in primary health care; the trainings were not frequent enough, there was no clear leadership structure within the team to facilitate team work. The evaluation team found that there were other conditions contributing to household investment in the toilet construction or effectiveness of CLTS. These include subsidies from the government or a history of subsidies by other projects, availability of employment which increases the financial capacity of the villagers. Many key informant interview participants expressed their view against CLTS approach claiming it was not effective in the Lao context for reasons such as there were still a lot of forest nearby the villages, and water supply was not reliable –these views demonstrate that behaviour change interventions had not been fully successful. While the CLTS approach might not lead to ODF in a short timeframe it can lead to increase in toilet coverage and effective use. The project had 3 years in a pilot phase which was followed by 4 years in full implementation so sanitation progress was a lot lower than anticipated. However, this is within a context where the project has not been operating in a strong enabling environment for CLTS and hence the project had to make concerted efforts to improve this simultaneously. It is important to highlight that the HH survey findings show a high proportion of respondents did not participate in the CLTS triggering session. This perhaps is part of the reason why people believed CLTS was not effective. Table 8 shows numbers and proportion of respondents’ participation in the CLTS triggering session. At the community level, the CLTS process faced some advantages and challenges. DWT members reported that interns that Plan has supported helped a lot in terms building trust

48

between ‘outsiders’ and the community members. The trust between them was important because it made people come to the meetings, without it people would not come to the sessions thus would not receive messages. The CLTS process were also interactive and engaged. People were having fun while absorbing the messages. However, there were challenged DWTs across three districts shared. They all had a few ‘difficult villages’. This means that people were not cooperative, they did not come to the meetings, they usually were the ethnics that speak limited Lao, and often spend most of the time out in the fields. Thus they faced the challenges in terms of language barriers, deeply rooted habits of poor hygiene practices amongst the ethnic minority groups and limited number of people joining the sessions. The project did not have specific and nuanced strategies to work with different ethnic groups which were not only a missed opportunity but a necessity when working with ethnic diverse communities.

It is less evident in the assumption that hhouseholds and communities upgrade to more durable toilets types over the long-term. KII respondents reported that poverty is the key reason. They said CLTS and hygiene promotion activities were effective among better off households, but less effective for the poor. Even these people were aware of the important of the toilet and wanted to upgrade it they still could not due to their financial barrier.

There is also limited implication from the sanitation marketing exercise. This is limited demand for sanitation facilities that encourage private sector to expand goods and services to remote areas. The Evaluation Team did not meet with any sanitation suppliers. So we were not able to interview any suppliers. Monitoring on WASH activities and practices in schools and villages being adopted and sustained by relevant government provincial and district agencies is unlikely if there is no project support. At the time of the evaluation, the provincial and district authorities reported they won’t have budget specifically allocated for WASH activities in general.

Maintenance of water facilities in schools has not been integrated to standard activities of relevant government agencies. It has to be noted that Plan’s supported follow ups of WASH facility only last for one year at a particular community. The facilities were mainly built one or one and half years ago when the team visited. The VECs were not very clear about how they would find the money for WASH in schools maintenance.

Construction/rehabilitation of WASH facilities

Construction/rehabilitation of WASH facilities has contributed to the effectiveness of HP in schools as the facilities (where functional) provide beneficiaries in schools with the means to adopt and practice the recommended sanitation and hygiene practices. Tools and approaches used in promoting hygiene and sanitation such as the Bluebox, the SSHT toolkit, and the CLTS toolkit – all recognized and accepted tools by the Government – and the puppet shows from the project appear to be effective means to convey messages to beneficiaries, particularly children. The challenge remains at the level of behavioral change. KII respondents strongly claimed that some ethnics groups did join the puppet shows, events and health education sessions but they did not put into actions. As stated above, this could be because specific and nuanced strategies to work with different ethnic groups was not applied within the project.

In terms of toilet coverage, the project has made significant progress toward to the 2016 target. As in Figure 12 below, there is only 3 % below the target of 80 % for toilet overage across the target communities. Pha Oudom has seen lowest increase in the coverage as the

49

WASH did not expanded to many of the villages for the imminent resettlement because of the dam construction along the Tha river.

Figure 8: Sanitation coverage in target district against baseline

Source: WASH annual report FY15, Plan Laos.

Achievement of ODF status remains a challenge. Overall, just above 50 % of the 2016 target had been achieved (see Figure 5 below). In particular, Meung had made the least performance compared to other two districts (48% for village and 50% for school). KII respondents reported that achieving ODF status was ambitious for the following reasons. First of all, the timeframe of the project in their opinion was short and the impact of CLTS was slow. They argued that ensuring elimination of OD would not be easy: deeply rooted habits of OD, financial constraints and availability of water supply. Another reason was associated with expansion of the community. This means that it was common to see young families, newly married couple, to separate from their parents’ house and build new one. During the first few years they were unlikely to be able to afford the toilet. When asked if they could share with their parents, the respondents said that could not happen all the time since their new house was not necessary close to their parents’. Other reasons included resettlements and disaster. In terms of resettlement, it was reported that new comers often settled adjacent to the existing communities and they could not benefit from the existing water supply system, without water it was very difficult for them to build and use toilets.

Figure 9: ODF achievement in villages and schools

Source: WASH annual report FY15, Plan Laos.

50

Participatory governance in WASH sector

In general, the WASH sector governance in Bokeo has undergone major transformations since 2011. Plan’s support has definitely contributed to these positive changes for the governance of the sector at provincial, district and community levels. Establishment of DWTs was one of its kind even exist in the province. The same for CLTS committees and having in place WSPs. The governance structure at the same time promoted gender equality and social inclusion. It promoted gender equality by encouraging more women to assume positions in the committees including in the DWTs. As presented in the previous section, 50-60% of the DWT members were women. There were women in the CLTS committees and 14% assumed leadership positions.

Gender quality was also promoted at HH level. However, the role of women in decision making about toilet construction remained minimal. As shown in Figure 7 below, there were very few women or wife made the decision but the majority of respondents reported they made the decision together. There is no baseline information to compare the progress on this topic, but in relation to governance it has been the intervention that promotes gender equality.

Figure 10: the role of men and women in decision making about toilet construction

Decision making concerning investment in toilet construction also varies from one ethnic to another. Joint decision making means husband and wife make decision together. The Figure shows that Laoloum demonstrate higher percentage of household jointly made decision-more gender equality regarding toilet construction. Higher inequality can be found in Aka and Hmong. This will be discussed more in the gender, accountability and social inclusion section. For all ethnic groups, the graph below shows that hardly any women made the final decision with regards to building a toilet at their household. However, they were able to influence the decision whether the HH will build a toilet or not in the joint decision mode. A joint decision making means either husband or wife can veto the decision if one disagrees.

Working relationships with government partners

The Project established and maintains working relationships with the NamSaat department at the national level, with the PHD and PED at the provincial level, and with DWTs and the District Governor at the district level. With non-government organizations and donor

51

partners, the Project actively participates in the TWG, providing technical inputs to proposed policy or approaches in the WASH sector, and also supports the regular meetings and reporting of the group.

At central level, the KII respondents from the government counterparts shared the view that Plan should engaged with line Ministries from central down to local levels. This may not reflect the government point of view as it has been agreed that such development project as Plan’s ones can coordinate with the Provincial level. It is clear in the MoUs that many of the reporting line i.e. from district

Good working relationships with the government also depend on the relationships between project staff and that of government formal and informal. It was reported through KIIs that at national level, the Program Manager has worked well with the NamSaat 22 while the Provincial WASH Coordinator has good working relationships with the provincial agency and district agency staffs. Project district WASH staff also has to have good working relationships with their counterparts in the different agencies at district level as they work closely and often. A formal agreement between the Project and government would greatly clarify the roles and responsibilities of both parties, including those of the WASH officers and of the DWTs.

During the interviews, Plan staff described their working relationship with the government partners as good working relations. They were quick to raised issues of the official availability and commitments to other responsibility that affect the implementation of the Program activities. Plan staff shared the views that assigned government staff take the responsibilities to Plan supported development activities secondarily compared to other priorities. Some claimed that the assigned officials had not been acknowledged for their performance working with INGO supported initiatives by their office recognition system. Therefore, they have not been too active working with Plan. Some district staff still don’t understanding the ownership of the development activities. It was supposed that plan and reports be shared at the district level. It is the district responsibility to share the report and plans to their higher levels. The same should apply for the PHD with the central level. Ownership of the district staff has not been informed by their leadership management. Coordination between plan and government authority need better understanding. On the other hand, the government counterparts had also something to say about Plan staff. They claimed Plan staff didn’t coordinate well in rolling out activities particularly during the planning process. They said lack of effective communication make it difficult for them to integrate Plan activities into their schedule. At the provincial level, PHO said in worse case, short notices often came from Plan which forced them to allocate staff who might not be those trained and cooperated with Plan before. This affects the effectiveness of what they are doing.

22 Consultant’s perception and observations from several discussions with the Program Manager. Unfortunately, the Consultant had no

opportunity to meet with other WASH NGOs or other organizations working with NamSaat at the national level due to time constraints. The Consultant requested an interview with the NamSaat representative who participated in the Design Workshop but was reluctant to discuss these issues without explicit permission from the NamSaat Director. .

52

These accounts are not novel. They have been documents in other reviews. However, it is worth repeating these issues again for the following reasons. First is that they implies dissatisfaction from both sides over one another. This can negatively affect the implementation of the project activities in the future and it can get worse if left unaddressed. The second reason is that the issues raised by the two sides are interrelated: officials’ availability and lack of proper coordination. This means that to some extent they were telling the true.

4.2 Effectiveness of the MNCH-N Program

The evaluation found that there are seven underlying factors that could influence the achievements in the MNCH-N Program in Bokeo.

For health providers:

1) All year round access to villages is important. Difficult road condition in the rainy season, mountainous areas, scattering of settlement and resettlement made the implementation of some health events delayed; and

2) There were unclear roles and responsibilities amongst district health staff and health centre staff for the conduction of health events in communities particularly as for who would take lead in conducting health events.

For communities:

3) Percentage of women who preferred delivery at home was high. 66.7% of women surveyed reported to delivery at home and most delivery was assisted by relatives/friends (62.2%). Cultural beliefs and distance were the key factors;

4) Limited knowledge on child feeding practices and shortage of food for households because communities mainly rely on food available in the forest including vegetables, mushrooms, bamboo shoots, fish and etc. The HHS shows that the percentage of children 6-23 months received exclusive breastfeeding for the first six months is low and even lower than the baseline and national figures (33.8% vs 52.7% vs 40.4%, respectively). The percentage of children 6-23 months received four or more food groups is low in both end line and baseline surveys and lower than the national figure (20.9% vs 20.6% vs 35.0%, respectively). In addition, FGD participants expressed that if the child does not eat properly it is normal for them;

5) Distance from home to health facility plays a role. The survey shows that some women reported to live far from health facilities, over 5 km or more than one hour walk (39.2%). Even though the internal review found that 60% of the HHs own a motorbike and it could be a good mode of transport for ANC and other health problems, it is not suitable for delivery or final stage of pregnant check-ups;

6) Low education attainment, 42.7% of respondents are without schooling and 31.2% attended only primary school. Low education causes poor health seeking behaviours as FGD participants accepted; and

7) Language barrier – most women cannot speak Lao language so it would be difficult when seeking health care and communicating for behavioral changes (cited by field KII respondents), this survey is also evident that 90.4% of respondents are ethnic people from

53

Muser, Hmong, Koui, Khmu, Lamade, and AkHHa. Therefore for the next MNCH-N intervention the seven core important factors should be considered carefully.

Access to MNCH-N services

The program had successfully achieved the objective “to improve the access to health care services in the target communities” thus contributing to the Bokeo province to achieve the MDG 4 and 5. As the HHS indicates, access to ANC visit ≥ 4 times, PNC visit within 7 days of delivery, PNC visit within 6 weeks and PNC visit ≥3 times had increased over time and was particularly so comparing to the baseline survey. In addition, more WRA had an increased access to family planning services, tetanus toxoid vaccination and iron supplementation. Mothers have had better knowledge on diseases that drive children to health facility immediately (Figure 10). This achievement is consistent with the results from the desk review concerning the number beneficiaries learned about health education from health events conducted during the project life.

Moreover, access to new born and child health care had improved over the time. After the program intervention more and more children had access to child health care services except for children 6-23 months access to food ≥ 4 food groups which remain unchanged (Figure 11). It is true that access to more food groups remains the same compared to the baseline. The FGD results revealed that villagers mainly rely on food available in the nature such as forest vegetables, mushroom, bamboo, game and fish in rivers. While these are nutritious food but the quantity and diversity and quality are very low thus this dietary style minimally contributed to improvement of the nutrition status.

In addition, the findings show that mothers have better knowledge on four reasons that needs to take a child to health facility (7.7% in this survey whereas 3.4% was in the baseline survey. This is because of the program provides the parenting orientation regarding MNCH-N in the communities by trained DHO /VHV and village facilitators and this implementation has been monthly monitored.

Figure 11: The access to mother and child health services at baseline survey compared to the end line survey in 2015

54

Regarding delivery, the program had not achieved the target to encourage mothers to give birth at health facilities and assisted by SBA (Table 16). The key informants explained that it was hard to change this behaviour within a short time frame because this culture was deeply rooted in most Lao mothers. They were “programed” to give birth at home particularly among the ethnic women. FGD results also revealed that they were shame to give birth with other people presence rather than family members or relatives. The KII respondents particularly the HC staff reported that ethnic women in the target communities particularly Hmong were the least group using the health facility for child delivery. They often sought help when they had complication i.e. prolong laboring, excessive bleeding and the like. Even so they would make a phone call to check if the female health staff were available at the facilities. This means that they are too shy to allow male staff to provide service even when their life was at great risks.

Figure 12: The access to new born and child care at baseline survey compared to the end line survey in 2015

The continuation of health education conducted during the outreach activities and implemented as part of parenting orientation in communities had contributed to behavioural changes amongst the beneficiaries. However, birth delivery had not significantly changed for the following reasons. FGD participants with men and women said that health education usually participated by women, but the decision whether they would use health facilities for delivery was made mainly by men. Women also did not want other people to see their secret part of the body. Men also did not want other to see that part of their wife. Many were still confused about free delivery policy including HC staff. Some claimed they were charged when using health facilities for child delivery. Transportation and distance to the facilities were also mentioned as part of the reason.

Table 16 below should be viewed in comparison to the baseline as followings. The current Lao Social Indicator Survey (LSIS), completed in 2011, reported that PW had attended ANC visit ≥4 times was 36.9%, delivered with skill birth attendants was 41.5%, and delivered at health facility was 37.5%. After delivery 39.5% of PNC women visited PNC services and 40.6%

55

brought newborn for health checked. In addition, 18.2% of women got pregnant at early age and 49.8% of women in reproductive aged had utilized the contraceptive23.

Table 16: Status of delivery at baseline and end-line survey

Item Baseline survey

End line survey National figure

Delivery at health facility 29.6% 27.0% 37.5%

Delivery at home na 66.7% Na

Delivery assisted by SBA 31.5% 29.7% 41.5%

Delivery assisted by relative and friends na 62.2% na24

Effectiveness of knowledge of mothers on the danger signs of pregnancy and delivery

The knowledge of mothers on danger signs of pregnancy and delivery had been improved. The survey results show that more mothers were aware of at least two and ≥ 3 critical signs of PNC that they need to seek care at health facilities (39.6% and 18.9%, respectively). This improvement was corresponding to an increase of access to PNC service in this HHS such as the access to PNC within 7 days and within 6 weeks and the number of PNC visit ≥ 3 times has been increased over time during the project life. Similar observation is noted for the mothers’ knowledge on critical signs of labor course that need to seek care at health facilities (18%). This positive change of mothers’ knowledge is a consequence from the parenting orientation activities implemented in communities that included various topics related to MNCH-N. The parenting orientation activities are effective in increasing the knowledge of mothers and influenced health seeking behaviours regarding delivery and PNC service with SBA or at health facilities.

These achievements can attributed to the program design that addresses the local needs. The program mainly focuses on the provision of quality health services in the target communities and to reach all people in remote areas. The SBAs education significantly complemented the MNCH-N activities while addressing the cultural and language obstacles. For instance, the ethnic women were too shy to utilize health care services with male health staff delivering the ANC visit. Strengthening of the capacity of the health staff does lead to better management of outreach activities and launching of health events in the communities. The outreach activities are very appropriate for the rural remote setting regarding access to health care services. The KII respondents in Bokeo stated that poverty and low education are the reasons to blame for villagers not seeking health care services. The limited exposure to modern health treatment is also another reason. Most mother perceived that when children are sick or crying and do not eat properly, they thought it is common for their children. Sometime they thought that these are part of the growing up of the child.

23 Lao Social Indicator Survey (LSIS) 2011-2012. December 2012. MOH, NSC, UNFPA and UNICEF. 24 Please note that the sampling technique adopted in baseline survey and in this evaluation is different therefore we are not making a direct comparison. It should also be note that the coverage rate of the intervention is almost 100% of the population so the sampling technique would not make much difference in terms of the probability to be sampled of the population.

56

Effectiveness on nutrition status of children

Concerning to the nutrition status of children < 2 years, the evaluation observes that the program had contributed to improving the nutrition status of the target children particularly wasting (weight for height/length) among children < 2 years of age. Figure 12 below shows that percentage of wasting had decreased, severe and moderate wasting rates had decreased from 6.5% and 13.7% as reported in the baseline to 4.3% and 6.5% respectively. This improvement is a synergy effect of the integration between WASH and MNCH-N programs. The HHS shows that hygiene and sanitation in target villages has been improved, 78% of target villages achieved ODF, 34.5% of mothers had better knowledge on how to dispose of child feces properly, at least people knew about three critical times to wash their hand (Meung 23%, Paktha 12% and Pha Oudom 35%). FGD participants show that they knew the cause of diarrhoea correctly. This improvement could contribute to reducing the diarrhoea episode among children U5Y, but in this survey we did not measure the diarrhoea episode, and consequently improved nutrition status of children.

Figure 13: Wasting (Weight for Height/length) by district

In addition, the MNCH-N program activities was successful in encouraging mothers to feed baby with colostrum within one hour of birth (74.8%), and to give complementary food beside the breastfeeding to children 6-9 months (92.0%). The percentage of wasting in this HHS is still higher than the national figure (severe wasting is 1.4% and moderate wasting is 5.9%, LSIS). In order to improve the wasting in the target communities in the future Plan needs to focus on the nutrition status of children aged between 0-5 months because they are the most suffered from both severe and moderate wasting (Figure 13). The promotion of exclusive breastfeeding for the first six months of age should be emphasized intensively in order to ensuring the child growth. In this HHS, the exclusive breastfeeding is lower than the baseline (this HHS 33.8% vs baseline survey 52.7%).

It is important to note that children aged 0 – 5 months had moderate wasting whereas children aged 6 – 23 months were moderately stunted when compared with other age groups (Figure 13). This finding is consistent with the results from the interviews with mothers who accepted that they were not able to provide their children with a variety quality food groups. Few children received a minimum acceptable diet apart from milk, breastfeeding (Figure 11). In Lao PDR, most mothers introduced grilled masticated sticky rice wrapped by banana leave to child at early age and some mothers introduced it even few days after birth.25 In-depth interviews with local authorities and observation in villages also indicate that mothers have

25 NIOPH and UNICEF, 2013. Formative research on young child feeding practices in southern Lao PDR.

57

low education, ethnic people and cannot speak Lao language so that the access to media for health education on nutrition is very limited. Household food insecurity is common in villages because it mainly rely on the natural sources or forest surrounding villages such as vegetables, fish, games, bamboo, mushroom etc. but forest food is scare in dry season (the dry season takes about 6 months of the year). Moreover, mothers have poor knowledge on young child feeding practices. Therefore, the factors contributing to wasting and stunting among children are multi-facets and need multi-sectoral solutions.

To improve food availability and accessibility, an integration of MNCH-N, WASH and agriculture programming could be part of the solutions. While MNCH-N and WASH improved hygiene practice and increase knowledge on proper dietary, agriculture will promote availability of food. For example, agriculture can introduce household gardening, food demonstration that uses locally available or home grown food. The KII results and field observation all came to a conclusion that food availability and accessibility remain a major challenge for the target communities. They claimed that introducing agriculture component into the program would greatly contribute to increase access to food.

During the field work in villages the evaluators noted that mothers did not pay attention to child’s health even the child had mild fever and were crying and mother still said that it is normal for the child to have such conditions. The MNCH-N and WASH programs started around the same time and there were some important evidence to show that the two programs were complementing each other. For example, health facilities including 11 health centers and 02 DHO had access to safe and clean water and clean toilets which belong to activities under WASH. In addition, many villages declared ODF and mothers dispose of child feces in a proper way as a result of the two programs promoting convergent health messages. Therefore, for instance, diarrhea cases among children had been decreased (one of the main cause for malnutrition among children. moreover, many villages achieved MHV; a high number of target beneficiaries accessed MNCH-N services: ANC and PNC visits in communities, etc. this evidence points to the conclusion that multi-sectoral program should be paid more attention to.

Figure 14: Wasting by age in months and district in target districts

58

The HHS indicates that the program has not achieved to reducing the stunting (height for age) among children < 2 years. The percentage of severe and moderate stunting in this HHS was higher than the finding of baseline survey (22.6% and 41.6% vs 17.1% and 38.2%, respectively (Figure 14)). However the percentage of stunting in this survey is lower than the national figure (44.2% was in LSIS and 41.6% is in our HHS). Although stunting in this survey is lower than in the national figure, the attention to improve stunting among children mainly children aged between 12-23 months is needed because they are the most affected from stunting (Figure 15). Stunting in this age group could be a result of poor nutrient intake, poor knowledge on child feeding practices, poor care from the time of birth, chronic diseases like diarrhoea and pneumonia, burden of parasitic infection and poor personal hygiene and sanitation. The KII at community level viewed that the program had limited impact on nutrition due to the program focused more on monitoring on nutrition but not so much attention on treatment and demonstration. Therefore the promotion children 6-23 months access to variety of food groups and minimum acceptable diet through the promotion of home gardening, improving the mother’s knowledge on young child feeding practices, promotion of good personal hygiene and sanitation and regular distribution of deworming to children under five years are essential to continue in Bokeo. Protection and management natural food resource such as nurturing and protection NTFPs, domestication of wild varieties of NTFPs, or fish conservation could be other alternatives to increase variety of and access to food.

Figure 15: Stunting in three target districts

Figure 16: Stunting by age in month and target district

59

Effectiveness of program’s approach employed

The designed approach employed for the program activities in order to improve the access to health service in communities: outreach activities and health events and building SBA to work in communities fits with the situation of target districts and provided a significant impacts on the access to health services in community. As we know target districts locate in mountainous areas and have poor public transportation so the access to health services is difficult. In addition the practices of ethnic community members deter the access to health services at health facility, for instance ethnic women dislike to consult with male health providers and ANC visit, deliver and PNC at health facility due to they are shame to allow others to see the parts of their bodies, also husband played an important role in decision making for household members to seeking health care at health facility: husband do not like to allow women to give birth at health facility and bring children to vaccinate. So that the outreach activities, health events and SBA working in community as volunteer held in villages are appropriate approach to employ (FGD and KII respondents).

The local government partners totally supported the health day events organized in villages and they stated that if government had some financial support they would replicate this in communities by themselves (KII respondents). In this program government counterpart had joined meeting and provided some supports and advices and solved any problems when the program faced with difficulties.

Communities also supported the health events in village and acknowledged the plan activities (KII respondents and FGD). They wished Plan to continue program activities in communities and encourage communities to use health services.

However there were some constraints concerning the implementation of program observed such as the implementation of activities in communities was sometime delay due to the late communication between government counterpart and Plan staff based in province. Sometime outreach activities and health events did not reach to the resettlement and scatter villages. At commune level, there was a VHC that composed of 4-5 people and village facilitators to assist the program activities and collect some vital information in communities however the health staffs admitted that the VHC did not perform their task as planned only head of village and VHV work.

4.3 Monitoring and Evaluation

The Program documents stated that annual and quarterly meetings with the PHD and key government counterparts will ensure opportunities for reflections, planning, and adjusting the project plan as may be necessary. Every year, during these meetings, target villages and budget planning will be reviewed for following year’s initiatives. The monitoring framework employed by the project will conform to government policies and adapt with future modifications of the policy regarding sanitation in future.

For the foreseeable future, the Program monitoring framework will comply to Model Healthy Village, Development Village, and School of Quality standards with the addition of ODF as a condition. Specific project indicators, especially for measuring project initiatives to promote greater leadership on the part of women and socially excluded groups.

The WASH Program had established a monitoring system to keep track on all indicators. The star topics indicators monitoring system were particularly developed to a better stage compared to other indicators. The Project uses both quantitative and qualitative information

60

to monitor progress. However, the qualitative data was difficult to collect and had not been recorded and reported properly. The WASH reports mixed up data on indicators under objective 1 and objective 2 regarding the conditions of ODF and a number of officially declared ODF.

As it had been found in the evaluation of the pilot project evaluation, there is complexity in using the system as the indicators (star topics) have both physical and behaviour indicators of achievement measured by both qualitative and quantitative means of verification. We believe that there are many indicators that would take a lot of time to collect data in order to assess its progress. For example, indicators numbers: 1.3) Number and % of episodes of diarrhoea reported in sample target villages; 1.4) % increase among women and children who know least three critical times to wash their hands (target 80%), 1.5) % of reduced costs for medical expenses related to poor sanitation (MNCH-N baseline WASH), 1.6) % increase among women and children who know at least two reasons that cause diarrhoea (target 80%); 1.7) % reduction of number of sick days from work or school resulting from being sick (with diarrhoea). To collect this information would take time and demanding for both adult, children and data collectors.

Monitoring and review processes of the Project do monitor participation of women and men within Project activities. This is reflected in monitoring tools used which segregate gender information and monitors the roles of women and children in WASH. At present, Project monitoring does not yet monitor PWD.

61

5 Impacts and synergy

Impacts at the household level

In almost all FGDs, both women and men shared the view that Plan supported CWSs (including those supported by other agencies) did reduce women’s burden. Women are responsible for fetching water, CWS reduced time and energy women have to spend on collecting water. Furthermore, whether this is intentional or un-intentional, villagers were able to connect water to their dwelling (PVC pipe or plastic pipe) which freed up women from water collection duty. However, this practice is not formally recognized and officially accepted. The practice can have negative impacts on the poor who could not effort the pipe or HH far away from the public water points or those located at a higher location within the village.

The pipe distribution water system is being experimented in five communities in Attapeu province. The clear benefits include reducing women burden in collection water and convenient for supplying water to toilets. FGDs results show that convenience of water supply to the toilet is reported to be one of the key factors for the toilet use and cleanliness. FGDs respondents also reported that lack of water means toilets are left dirty and smelly-the un-pleasant environment that no one want to come near it.

In terms of gender equality, it is true that CWSs does alleviating women’s workload, but it does not change the power relations between men and women per se. It can be argued that as a result of the reduced workload women are free to engage in other activities that are empowering them. These activities can include joining in income generating groups, engage in learning activities.

Figure 16 below shows percentage of persons responsible for collecting water for HH use. It is clear that women are the bearer of this duty. We don’t have baseline data to compare change in the sharing of this duty but it shows women are still taking on the HH chores as the norm would suggest them to be.

Figure 17: Persons responsible for fetching water

The MNCH-N program has brought about positive changes in knowledge and behavior concerning mother and child health among the beneficiaries though with limitation in term

62

of behavioural changes. The FGD participants acknowledged that they learned about the benefit of vaccination form health day events and outreach activities. Although secondary data and HHS indicate that the access to health services among women and children had increased over time during the project life, FGD participants revealed that many villagers did not take their children to get vaccination; they believed the children will get sick after being vaccinated. Some households intentionally left home when the health events took place in villages. Women preferred to bring children for vaccination but husband disagreed for that performance. This scenario implies that more health education concerning the adverse event of vaccination related to health should be emphasized to warrant all target children get vaccinated and head of household is the target for this health education.

Regarding the ANC and PNC services, FGD participants reported that many PWs sought ANC visit at HCs, some PWs wanted to go for ANC visit at HCs but they could not go alone due to the distance and bad road conditions. They needed their husband who speaks better Lao to accompany them but their husband was not always cooperative as they preferred to go to the field over accompanying wife. Most PWs still delivered at home particularly ethnic women because they were shame to allow others to know about their reproductive health conditions and shy to let other see their secret part of the body. Sometimes they asked health center staffs to come assist at home. Some husband did not allow their wife to get deliver at health facility. PW would delivery at health facility just only when they felt that the delivery faced difficulties (FGD). Even with free delivery many ethnic women still did not use services at health facilities.

Plan’s support to build SBAs from local high school graduates and work in the community responded the cultural and social needs of the community. The FGD results are consistent with the results of HHS as the high percentage of mothers had ANC visit ≥ 4 times (47.2%) and most of them delivered at home (66.7%) and it was assisted by relative and friend (62.2%). Moreover pregnant women still worked hard in the field as normal people because elderly people perceived that the more working hard the more easily delivery will be done. In addition PW did not have a special food for themselves they will eat as normal people and good food would be shared with all family members. SBA and health center staffs should continue to provide health education to PW and husband concerning to the danger signs of pregnancy and delivery. Results of HHs indicates that the knowledge on danger signs of PW and PNC was still low (knowing 2 critical signs of PNC 39.6%, knowing ≥3 critical signs of PNC 18.0% and knowing critical sign of labor 18.0%).

Many KII respondents said that awareness of general health care amongst villagers remains low. Health care and health treatment seeking behaviors had changed for the better. However, a large number of people still did not see the importance of health care services (cited by FGD participants). The KII respondents noted that mothers did not take a good care of their child health. They said that even when baby had a fever, cry and did not eat properly mothers still thought that was normal for babies and did not need special attention.

It is important to note that during the cultivation period villagers left home for work in the field and missed out on health events in the village. For the next intervention this issue should be taken into account. Informed communities in advance needed to be ensured (KII field respondent).

Generally speaking, the MNCH-N program has shown the positive impact on the gender equality and child right at the household level. Now women can represent the HH in the

63

village meetings. In many villages, women become members of committees while children participated in the activities in school. It was reported that at home women and men have equal right.

Impacts at the community level

The projects have generally brought about positive impacts on the lives of target communities. Hygiene promotion activities in schools and communities were reported to have resulted in positive changes at the community and school levels. These changes included cleaner surrounding environment around people house, school premise as well as common places like village meeting hall. Almost all FGD participants reported that positive behavioral changes concerning using toilet, washing hands with soap, drinking safe water and keeping environment clean were evident in their village. Almost all of the participants were aware of the importance of using toilets, washing hands with soap at critical times, drinking safe water and keeping surrounding environment clean. However, the results were varied in terms of converting this knowledge to behavior change (as outlined in the previous section).

The results of these positive behavioral changes were also reported by FGD participants. They said in general health conditions among community members have been improved such as a reduction in a number of episodes of diarrhea in their communities. As we know that diarrhea is one of the main causes of malnutrition. So that in this HHs the achievement of MNCH-N program in addressing the wasting among children < 2 years was observed such as the severe and moderate wasting had decreased from 6.5% and 13.7% in baseline to 4.3% and 9.5% in this end-line survey, respectively. Thus to ensuring the achievement of MHV and addressing the malnutrition: wasting the integration of WASH and MNCH-N should be prioritized. This integration of WASH and MNCH-N program can be scale up to other provinces of Lao PDR. The FGD participants’ claim is consistent with the PHD authorities report which indicated that incidence of diarrhea has reduced in the target communities and believed to have occurred as a result of Plan intervention. They were not sick as often as before (i.e five years ago when there was no intervention) due to improved hygiene practices mentioned above. As a result they said, they could save a lot of money from spending on health treatment. They were not absent from work as often as before so they were able to work and fetch for food in the forest. They were able to better provide food for the family.

These changes were tangible in the sense that villagers were able to realize themselves and thus were likely to improve over time. The intervention of the WASH project can be considered as a catalyst which triggers awareness and actions. Village health committees (including CLTS committees) will continue to function even not in the current form. Thus the positive changes will be maintained. Additionally, as the communities make progress economically they would afford TVs, travels to city, visits to relative in the cities. These will continue their exposure to improved health conditions which will also contribute to their desire to change.

At the community level, the CLTS committees and village authorities interviewed reported that village regulations were in place and were used to re-enforce their work in promotion of hygiene practices. They also reported that sometimes they needed back up by district authorities in re-enforcing the regulations as the district authorities had more power and from experiences villagers seemed to listen to.

Water safety plans (WSPs) and O&M guidelines were also in place in villages where Plan support water supply, even though some villages were not able to show us a copy. Protection

64

of water source catchment areas was well implemented and villagers were well aware of the importance of such protection. Only one out of six villages visited reported they had problem applying WSP (Houaynorkhom, Paktha District). Generally speaking, it was reported that the village authority there was very weak and was not cooperative. This was evident when the team visited the village. Villagers including the head of the village were walking around the village but refused to come to meet with us.

Construction or rehabilitation of school WASH facilities and CWS were very relevant and remained relevant to the community needs. The intervention had brought about significant impacts on people’s life at the community level particularly children and women. As will be discussed in greater details in the Gender and Social Inclusion section, women and children are the group responsible for fetching water. CWS bring water closer to their house thus reduced the workload of them significantly. FGD participants (women groups) reported that they were happy that they did not have to carry two budgets of water sometimes as far as 10 km for household use and drinking. Now they could save time and energy from this reduced workload. They were freer now to watch TVs, learning some cooking lessons from there, learning how to take care of kids from there, etc. in some villages where there are PPG program going on, women reported that they took part in the group activities learning and discussing about child care, health care, hygiene and related topics which improved their general knowledge about good life styles.

The MNCH-N Program has also brought positive impacts at the community level and these impacts were well acknowledged by village authorities. The KII respondents stated that before children were frequently sick of diarrhea and some children died but now no children died of such disease. Authorities at villages level acknowledged that Plan activities helped materialized their mandates in encouraging health care seek at health facilities by villagers. They reported that as a famer themselves it had been difficult to encourage other to change behaviors; outreached activities delivered by DHO and PHD staff under Plan’s support were very powerful in encouraging positive behavioural changes. The village authorities proposed a continuation of supported activities in their villages. In addition, children were very active in participating in the parenting education session organized in village.

After the program intervention, the coverage rates of vaccination, contraceptive use, ANV and PNC visits within 7 days and 6 weeks had increased, particularly among WRA and children under-five years of age. The program had created substantial impacts on primary health care services among target beneficiaries through outreach activities, mobile clinics integrated with EPI, quarterly health days and children days. The majority of beneficiaries in target districts accessed to vaccination, deworming, vitamin A, growth monitoring, iron supplementation, family planning, ANC, and PNC. More women had better knowledge on critical signs of child illness that need to seek care immediately and the danger signs of PNC and labor course of delivery that require care at health facilities after participated in the parenting orientation activities led by trained health staffs and village facilitators.

Plan had contributed to improved quality of MCH services in communities. Plan has supported 25 ethnic women with high school graduation to study SBA. Now they completed and voluntarily worked as SBA in communities and HCs. These women had obtained knowledge and competency to provide health care services in their own community. However, not all had been recruited as government staff yet. This intervention was well regarded by FGD participants as “useful and user-friendly”. The SBAs were easy to talk to, no language barrier;

65

and cheap sometime free. Therefore, this addressed the issues of shyness and long distance travel to health facilities.

Moreover, training on Parenting Orientation Modules concerning MNCH-H was implemented in community by trained health and none-health staffs (education, Social and Welfare, Cultural and Information, LWU and Youth Union of provincial and district levels). This activity has made VHVs, VHCs, and village facilitators confident in providing services concerning the MNCH-N in their community.

Impacts on the institutional level

The program has improved the quality of health care services. Through the several short training courses on MNCH-N activities and long training course on SBAs the PHD, DHO and HC staffs and trained SBA had become more confident and competent to provide quality health services. Health facilities including 11 health centers and 02 district hospitals were supported with WASH facilities in order to provide healthy environment and cleanliness of the facilities. In addition, Plan provided enabling environment for delivery of MNCH-N services at HCs by expansion of two HCs such as the room for MCH services, labour room, and waiting room (cited by Plan staff) and equipped with necessary medical equipment for delivery.

Through the capacity building training and study visit, health staffs at all levels gained more experiences and now feel more confident in performing their tasks. PHD and DHO staffs expressed that the computer skill is new for them but now they can work properly for the daily work and other purposes. PHD and DHO staffs also have capacity to supervise the program activities in communities such as data collection, parenting orientation on MNCH-N, outreach activities and other health events. Beside that health center staffs discussed that they now can develop a micro-planning for the program activities without the assistance of DHO so that they said that they even have confidence in leading the health event in community by themselves and if it is possible they wish to do so.

The evaluators think that the application of PCA in communities from the start of project for identifying health problem has brought an important positive impact on the solidarity and unanimity between health staffs and communities towards the mutual addressing of health problems in communities although Plan had played an important role in designing the appropriate program activities in corresponding with the identified problems.

Impacts at the local government levels

The project has made the most important contribution at the district level. This included knowledge and competency strengthening for district officials. District officials interviewed named trainings they participated in as CLTS TOT training, basic gender principles training, gender WASH monitoring toolkit training and disability right, inclusion and equity. The officials shared the view that the trainings not only exposed them to the content i.e. gender equality principles or gender wash monitoring indicators, but also the facilitation and promotion skills. Some even proposed the ideas of more support in the form of on the job coaching for example when conducting CLTS sessions. Most importantly, the officials said they were more competent speaking at the public settings such as community meetings or at the workshops or formal meetings. They said in the formal meetings before even if they had some ideas but were too shy to share them. Now they have been more active and participatory in such settings.

66

In terms of organizational structure such as DWTs, experiences on multi-sectoral coordination and planning had had been improved or initiated over the project period. This coordination and cooperation particularly between education and health sectors has contributed to effectiveness of CLTS such as improved hygiene practices at the community level or increase in a number of ODF villages. Since improved health conditions among target communities depend on this multi-sector cooperation, DWTs have made important improvement both in terms of teamwork and achievement of the team mandate.

The introduced organization such as DWTs were unlikely to be sustainable and the one of the key reasons is the lack of financial capacity from the government system to continue to make it functioned. Interviews with PHD and DHO all pointed to the same conclusion. They admitted that the government at least at the provincial level would not have financial capacity to fund DTWs and continue Plan initiated activities related to CLTS. Therefore it is unlikely that DWTs would continue to exist and function as it has been until now.

Through a quarterly meeting the PHO, DHO and HC staffs had a good culture for exchanging experience amongst them and mutual reviewing all important issues. Also through the monitoring and supervision of the program heath staffs had better communication and coordination with local committees at village level including SBA working as volunteers, VHC and village facilitators. These performances promoted the implementation of program activities at village level going smoothly and ensured the implementation going to the right way and sustainability.

Synergy

Synergy between the WASH and MNCH-N programs occurred when both programs presented. The two programs were complemented and closely linked. The WASH program provides access to WASH facilities in schools and communities at the same time promotes proper hygiene behaviours. The MNCH-N program, on the other hand, provides access the mother and child health care services. The two are complementarily contributed to disease prevention and treatment. Integration of MNCH-N and WASH programs is well acknowledged as an important approach to contribute to MCH and nutrition outcome and impacts. At the national level the integration has already been performed and set it as ‘convergence plan’. It involves 4 sectors: education, health, WASH and agriculture.

The community members reported that before children were frequently sick of diarrhea and some children died. The villager authorities reported that over the last three years there was no case of under-five mortality. The integration of WASH and MNCH-N programs has given a positive impact not only on the acceleration of the development of MHV but also addressing malnutrition among children < 2 years particularly wasting. From the field observation, the team found that very few sanitation facilities were constructed in villages where there was no WASH program present. At the same time, more women and men reported seasonal diarrhoea outbreaks in villages where there was only MNCH-N program presented.

The above impacts are a synergy effect of integration between WASH and MNCH-N programs. As indicated in this HHS the hygiene and sanitation in villages had been improved: 78% of target villages achieved ODF, 34.5% of mothers had better knowledge on how to dispose of child feces properly, at least people knew about three critical times to wash their hand (Meung 23%, Paktha 12% and Pha Oudom 35%) and FGD participants knew the cause of diarrhoea correctly. The National Nutrition Strategy 2009 emphasises the causes of under-

67

nutrition including adequate amount of diverse and quality food intake and proper childcare and feeding practice. However, there was a limited impact on nutrition element even though integrated into the MNCH-N program.

The integration of WASH and MNCH-N programs had created positive impacts not only on the acceleration of the development of MHV26 but also addressing malnutrition among children < 2 years particularly wasting. Criteria for the MHV status included both WASH and MNCH-N program.27Implementation of two program activities in the same village also promoted the development of MHV faster. In Bokeo province the two programs activities had been implemented in target districts so the evidence showed that the program greatly achieved the MHV (67.6%-23 out of 34 target villages) and 11 villages are in process.

26 1) implementing comprehensive health education – availability of information board or loud speaker, 2) provision of clean water and environment in collaboration with concerned unit, 3) implementing a process of three hygiene, 4) having trained VHV on health promotion, prevention and able to solve basic health problem, 5) contribution to promote the vaccination among target children and women and reaching to 90%, 6) implementing a process that supports safe pregnancy, delivery – ANC visit ≥ 4times, delivery and PNC at health facility and attended by SBA, 7) implementation of integrated package of mother neonatal and child health services, 8) having functioning village drug kit in case the village is not under catchment areas of health center, 9) transferring patient to health facility in time, and 10) having village health committee to promote health and prevent diseases, and recording and reporting all birth and death occurred in a village. 27 H-SDP - ADB III, Ministry of Health, 2012. Manual on establishing the model health village.

68

6 Sustainability

6.1 Sustainability of WASH

In this evaluation, ‘Sustainability is the ability of prevailing local structures, processes and people to continue their role and functions after the withdrawal of all forms of support from Plan. As a result, the WASH program activities, outputs, outcomes and impacts would persist during an extensive time period. The evaluation documented the presence of the conditions and factors associated with WASH sustainability and the likelihood or prospects for sustainability.

School WASH Infrastructure

As a result of the school break, we could only manage to make site observation at three schools. We observed functionality and cleanliness of the WASH facilities. We found low quality construction work. (i.e. leaking water storage). Sustainability in its simplest sense is the durability of the structure. Plan staff admitted that supervision role for construction work in schools had been transferred to the government counterpart. They now acknowledged that the transition means an increased risk of poor quality work. Plan and the counterparts had addressed the issue internally.

The maintenance of the completed facilities is the responsibility of the village education development committees (VEDCs) and the school principles. The project established the committees to oversee the functioning and maintenance of the infrastructure. The committees reported that the school administration fund was the only source of money to be spent on maintenance of school wash facilities. The committee reported that minor maintenance could be done by the CLTS committee members, the technical persons. Technical capacity for maintenance of these facilities has been developed by provision of O&M training. Unless the technical person moved out of the village immediately and someone else does not take up the role, the maintenance mechanism will remain there.

Hygiene in schools

Sustainability of hygiene promotion in schools rested on the integration of hygiene promotion activities into school curriculum and the hygiene promotion activities led by teachers and model/volunteer students. These activities were based on the UNICEF hygiene promotion toolkit called the Bluebox. FGDs with children revealed that they practiced hand washing with soap once a day, three times per week when the school semester was on. There is no standard in terms of how many time per week is good, but students seemed to know well for good hygiene practice such as 7 steps of hand-washing. According to the education officials interviewed hygiene practice will continue to be implemented even without financial support from outside. Therefore, this is likely to be sustainable.

Funding for hygiene activities such as provision of soap is the key to its sustainability. According to education officials interviewed, the schools that meet the government SoQ criteria would receive encouragement fund (1000kip per student per year). However, none of the schools that had met the criteria had received the money yet. The possibility that the money would be made available is very limited as confirmed by the MoES. Other schools would fund the hygiene practice activities by the school administration cost either allocated from district education office or collected from student parents.

69

Institutional sustainability

The Institutional structure developed under the project related to school WASH is the establishment of DWTs, VEDCs, CLTS committees and involvement of village chiefs and village cluster heads. Over the project period, these entities were actively involved in the school wash activities. As discussed earlier, DWTs may not be sustainable in its current form therefore; maintenance of the school facilities may need to be re-arranged after project had end.

Sanitation infrastructure

Construction of the latrine by self-investment was well monitored and reported. According to KII participants, pour flush type of toilets were the most popular to villagers. From the project’s experiences, dry pit toilets were prone to extreme weather events. This becomes the main reason for some villages to lose their ODF status. Therefore, mmaintaining and keeping these facilities is critical to sustaining changed behaviours among children in schools and villagers in the communities. The key to sustainability of CLTS are continuous follow-up among households to ensure maintenance of toilets and sustained use.

However, for some villagers dry pit toilets are the first stage toward permanent toilets. A critical factor therefore is the maintenance of the dry pit toilets which are structurally unsustainable especially in challenging geographical and weather conditions. A distinction has to

be made between the maintenance of just the toilet-seat and building, and the system to evacuate and treat excreta safely.

Behavioral changes

FGD participants reported that the deeply rooted habits of OD started to change now partly due to understanding of the negative health impacts of the OD and partly due to the convenience of having a toilet at home. Children groups reported that they were taught at school to use the toilet and wash hands with soap after using the toilet. Some were able to apply the practice at home, some even persuaded their parents to build and use toilets and practice hand washing with soap. The majority of men and women of FGD participants reported that they used toilet and wash hands with soap all the time. However, there were some who admitted washing hands without soap or not washing hand at all.

Institutional

Plan is striving to ensure local ownership and community-led processes in relation to WASH. Local governance structures such as VEDCs and CLTS committees were established to empower community participation. However, across the three districts visited, officials both Plan and government counterparts, reported that community ownership remained an issue concern. At community level, even village authorities, still believed that WASH facilities belong to Plan. Villagers were under the impression the project is ‘owned’ by the district or provincial departments.

Support from outside for CLTS committees is unlikely to be sustainable. At the moment, there still strong commitment in following up from DWTs. They had achieved a satisfactory result as discussed earlier. However, the current support will cease when the project had ended. Sustainability of the CLTS committees is unlikely unless the governance structure is supportive of it. This can include integration of the initiative into the district WASH plans and budget. However, interviews with PHD officials and DWT members reveal that there is no particular budget line for the CLTS initiative. Adoption of the CLTS approach by local government

70

institutions can lead to sustainability of the initiative and demand for sanitation at the community level.

The project’s exit strategies

The project’s strategy to phasing out has integrated in the design by setting a WASH team at the district level. At the national level, Plan had done quite extensive work. Figure 2 below shows some of Plan’s engagement with other development partners in attempts to promote adoption and expansion the CLTS approach. For example, the applied approach and developed training manuals are shared at national level with the NamSaat, the WSP, SNV, and with Plan’s country Program. As well, representatives from PHD Bokeo were supported to experience first-hand the project experiences of SNV in Savannakhet – where such experience and knowledge is expected to be shared at all Health ministry levels.

Water supply system management

Water use systems vary from village to village. One of the alarm signs in terms of sustainable use of water is water distribution. CWS distributed water by a water point usual shared by 5 to 10 HHs. Accordingly, many village authorities interviewed allocated a water point and the HH using it as a water group. If minor maintenance was needed i.e. water tap replacement, the group members would have to share the cost. This mechanism was widely adopted and seemed to work well. In other villages, a pool fund for the whole community was established and water fee was collected in advance. However, village authorities interviewed admitted that were not able collect the fee from all the HHs. This was well confirmed by the HHS conducted for this evaluation. Figure 1 below shows percentages of people who pay the fee by three categories: regular, sometimes and never between target communities with Plan’s support CWS and none-Plan support system. The proportion of regular fee payers is just 37 % which is quite low (see Figure 1).

Figure 18: Fee payment by Plan and none-Plan supported CWSs

This result is vastly different from the rate at which it was report in the 2015 annual report (80%). When compare the Plan’s CWS with other villages supported by other organisations the result is not significantly different. There were 6 out of 8 target villages where Plan supported CWS sampled for the survey. The results from the HHS show a limited difference in terms of fee payment between Plan’s CWSs and the rest of the target villages where water systems also exist. From interviews with the project staff, we learned that non-Plan supported

71

water supply systems had been established before Plan started the intervention in the area. Some sort of water committees also exist.

All 8 villages have started implementing village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines. The training was conducted on 26-28 January 2015. This means that the committee members have just been trained and likely that they would not enough time to put all the knowledge recently gained into effective application.

Institutional sustainability

A large part of sustainability of water facilities depend on community ownership and participation. At the community level, Plan International Laos and DWTs form and train CLTS Committees two weeks after triggering. Committees replace natural leaders and comprise five to seven community leaders who collect data, promote behavior change, and report progress to DWTs. In the interviews, the committee members reported they did not have a written plan or reports. In some villages, the team observed that when asked how many toilets were built this year, the member counted fingers to come up with the number of toilet built. This implies that they did not have any written form of paper (it is a norm that they bring notebook or any formal records to meetings such as meeting with the team).

When asked if incentive is important for water committees to function well and be sustainable, the members indirectly replied ‘yes it is important’. At the moment, water committee as well as all other committees established at the community level rested on the premise of volunteerism. In fact, this is a great myth. Some would volunteer to the collective good of their communities. But it is unnatural to fund enough volunteers in every community. Some water committee members admitted that they were selected by the community and had to fulfil the obligation rather than by a genuine volunteer per se.

However there are some constraints might be contributed to Program sustainability. Ownership of government counterpart, high rotation of staff will need more training, workload for HC staffs and community leaders, lack of budget for organizing health events in villages so it will lead to few health events held in villages or may be none and villages are scattered. These constraints can be threat to the sustainability of the MNCH-N Program.

As a result of the school break, the team could only manage to make site observation at three schools. The team observed functionality and cleanliness of the WASH facilities. The team found constructed facilities to be low quality (i.e. leaking water storage). Sustainability in its simplest sense is the durability of the structure. Plan staff admitted that supervision role transferred to the government counterpart mean increased risk of poor quality construction work.

While DWTs and CLTS committees promote good inter-departmental cooperation at the district level and good team work at the village level, the sustainability of these structures depend on the formalization of the structures and the institutional change within the local government structure. For example, there must be budget arrangement and commitment by the government to carry on with the initiatives. The problem at the moment is that there is no downward accountability from the government side. In general, the government system does not respond to community needs. The system is either financial or technical incapability to do so.

72

At village level, the situation might be different as the CLTS committees have the formalized and recognized (by district authorities) village regulations. As long as the communities recognize and accept the authority/ validity of the regulation, the CLTS committees even if not formalized can continue with encouragements to households as regards maintenance and use of toilets, keeping the environment clean, and other guidelines contained in their village regulations. At this level that Plan should refocus their effort on. It is unlikely that DWTs will continue to function after the financial support from Plan has ceased. Therefore, CLTS activities must remain and be carried out by VHC members (in case CTLS at community level is integrated into VHC). It is likely that working on fewer committees will ensure sustainability.

Review of the international experiences, it is widely acknowledge that that sustainability of school WASH facilities remains a great challenge. The limited budgetary resources allocated to school operation and maintenance, particularly in rural schools, means that few schools have any funds available to pay a caretaker or cleaner for regular cleaning and maintenance of school WASH facilities – the standard management model for school WASH facilities in most developed countries.

This experience was found to be the case with the UNICEF WASH in School Program28. For Plan’s intervention, school WASH leans its sustainability on VEDCs. From interviews with VEDC members the team found some inconsistency is the management and maintenance system. In some villages, a separate fund was kept for maintenance of the facilities. While other villages rely on the school administration budget allocated to school by district education office. Other source of the maintenance budget is said to come from “the prize money” given to school that meet SoQ standards. This has not been confirmed by all village authorities interviewed to be the case for all schools.

Sustainability of CLTS committees at village level remains a great challenge. One of the reasons is that at the moment they are not functioning as they supposed to be. From interviews with CLTS committee members, the team found that they have limited skills in encouraging other villagers to improve sanitation. In many villages, the committee members themselves had not built toilet yet. Another reason is the lack of incentive/reward for the effort for collective gain. This together with the fact that they are not fully volunteered in becoming the committee members can lead to unsustainability of the initiative itself.

Support from outside for CLTS committees is unlikely to be sustainable. At the moment, there still strong commitment in following up from DWTs. They had achieved a satisfactory result as discussed earlier. Sustainability of the initiative is unlikely unless the governance structure is supportive of it. This can include integration of the initiative into the district WASH plans and budget. However, interviews with DH officials and DWT members reveal that there is no particular budget line for the CLTS initiative. Adoption of the CLTS approach by local government institutions can lead to sustainability of the initiative and demand for sanitation at the community level.

A critical factor that must be considered in sustainability of CLTS is the nature and type of toilets used. Experience from other countries, (notably in Cambodia where conditions are similar (lack of water, high rainfall during the wet season, high poverty rates)) show that dry-pit toilets are not sustainable from a durability perspective especially in the wet season. In project target areas, some villages are located in hills and toilets are built on slopes at the

28 UNICEF WASH programme evaluation 2015.

73

fringes of the village. Where toilets are dry-pits, there is a constant and high risk of toilet structure collapse and pit-overflow from rainfall. In flat-land villages, the risk is pit flooding and over-flow. Both consequences present an unpleasant dilemma for HHs and may result to OD during the wet season – repairing their dry-pit toilets in the dry season. Improved latrine designs addressing the existing risks in target villages would address these issues in the short –term. However, for the long-term and for villages that are not at risk of relocation, encouraging conversion to durable pour-flushed toilets is the ideal response. This however, requires that HH are economically- capable to convert to such type of toilets and/ or that low-cost durable toilets are accessible to these HHs.

A large part of sustainability of facilities and personal and household hygiene practices depend on community ownership and participation. At the community level, Plan International Laos and DWTs formed and trained CLTS Committees two weeks after triggering. Committees replaced natural leaders and comprise five to seven community leaders who collect data, promoted behavior change, and reported progress to DWTs. In the interviews, the committee members reported they did not have a written plan or reports. The team observed that when asked how many toilets built this year, the member counted fingers to come up with the number of toilet built. This implies that they did not have any written form of paper (it is a norm that they bring notebook or any formal records to meetings such as meeting with the team).

Figure 19: Fee payment by Plan and none-Plan supported CWSs

Community member’s participation is also key to sustainability. In this respect, the team look into active contribution of villagers to maintenance of the facilities-one way is by paying fee to the water fund. The results from HHS show a low rate of fee payment contributing to water fund at the community level, 37% pay the fee regularly. This result is vastly different from the rate at which it was report in the 2015 annual report (80%). When compare the Plan’s CWS with other villages supported by other organisations the result is not significantly different. There were 6 out of 8 target villages where Plan supported CWS sampled for the survey. The results from the HHS show a limited difference in terms of fee payment between Plan’s CWSs and the rest of the target villages where water systems also exist. From interviews with the project staff, non-Plan supported water supply systems had been established before Plan started the intervention in the area. Some sort of water committees also existed.

74

All 8 villages have started implementing village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines. The training was conducted on 26-28 January 2015. This means that the committee members have just been trained and likely that they would not put all the knowledge recently gained into effective application. However, FGDs with in non-Plan supported water supply system villages

In CLTS, the applied approach and developed training manuals are shared at national level with the NamSaat, the WSP, SNV, and with Plan’s country Program. As well, representatives from PHD Bokeo were supported to experience first-hand the project experiences of SNV in Savannakhet – where such experience and knowledge is expected to be shared at all Health ministry levels.

8 villages implemented village regulations that include catchment protection in line with Operations & Management (O&M) and Water Safety Plan (WSP) guidelines

Feedback from District Officials29 indicated that to government CLTS is working in terms of increasing people’s knowledge about the about dangers of OD and motivates them to construct toilets on their own. Officials also observe that the delegation of authority to village leaders can facilitate reaching and maintaining ODF.

In villages where CLTS has not been more effective, the cause has been attributed to lack of leadership/ influence by the village head and the CLTS committees and in a few villages (such as Houay Aen and Houay Norkom), the reliance of people without toilets to potential external support or subsidy, and general lack of concern for village level improvements.

6.2 Sustainability of MNCH-N

Access to MNCH-N in community

Sustainability of access to MNCH-N activities in community depends on the ownership of health care providers including village authorities like VHV, VHC and village facilitators in organizing an outreach activities and health events in communities and the perceived needs and benefits of communities towards the program intervention activities. In this regards the access to health care services in community is likely to sustain once the program ended. When we looked at the provider site since the program implemented the capacity of health staffs at all level including village authorities had been improved through attending several training courses, they now had confidence to conduct the outreach activities and health events in community, managed the outreach activities, they had capacity in making micro-planning for organizing the health events in community, and health staffs also had proper knowledge on MNCH-N services, integration. When we looked at the utilizer site, it is community, more community were aware of the important of vaccination for children and women as a result of health events, culture barriers for the access to health care services at health facilities (FGD, KII respondents), though health issue remains less important, health care and treatment seeking is becoming better, and desk review we found more and more communities participated the health events during the project life. So that the evaluators would conclude that the program activities would be sustainable in communities technically however financially would be limited.

29 Discussions with District Government and District Agencies in Paktha and Phaoudom

75

It is likely that the sustainability of the program activities rested upon the alignment and continued alignment with the government policy and constantly responding to the needs of the community. This is well evident throughout the program implementation. .

View of stakeholders concerning to the sustainability of Plan program

Stakeholders shared different points of views concerning the sustainability of program activities in community once the program ended. Stakeholders at central level acknowledged that Plan has contributed a significant level to improve the mother, newborn and child health and nutrition in provinces and what Plan designed in implementing is in line with the need of Lao government, it is good that Plan succeeded to look for fund to supporting these MNCH-N project in remote provinces of Lao PDR and so far there are many provinces still need the similar activities but lacking of budget.

In addition, Local stakeholders (KII) impressed that generally it is good with the external assistance because our government lacked of funding to finance the project in community and they supported the implementation of program in province by providing general direction and coordination and solving all obstacles and problems day by day and providing the working space for Plan to make an office within PHD but in fact Plan did rent office outside. Some program activities had been delay to implement not going according to plan due to health staffs was assigned for other activities unless they were informed in advance by Plan staff based in province, in addition the ownership of DHO and HC staffs was still an issue (KII respondents).

Concerning to the sustainability of outreach activities and health events in communities, local stakeholders (KII) shared his view that if the program is ended the outreach activities would be continued however the coverage and completeness would be less, and vaccination and health promotion is unlikely due to the government has no budget. One KII respondent stated that integrated mobile clinics might not be possible.

In summary, based on the point of view of stakeholders the evaluators concludes that though local stakeholders supported the program activities in province and acknowledged the importance of the program that it gives positive impact on population health, program activities are unlikely to sustain due to limited ownership of government counterpart in conducting the program unless the government counterpart re-stimulate the concerned health staff since the stakeholders have already seen the important of this program interventions and improved coordination between local partners and Plan staff.

Institutional sustainability

The short and long training courses had contributed to the main part of institutional sustainability. These trainings had made the staff competent to perform their job effectively. More importantly, on-the-job trainings and follow up supports had effectively increased the staff ownership of staffs in conducting the program activities and their routine assignments. Since the MNCH-N program implemented the program had strengthened the capacity of institution by improving the capacity, experience and skills of health staffs and providing enabling environment for working in target health centres. Several training courses had been provided to government counterparts and community leaders including VHC, VHV and village facilitators. Now health staffs were able to use computer for routine work and conduct health events in village, knew on how to develop microplanning, Program had supported the renovation of health centres including supporting medical equipment and water supply and

76

sanitation, logistics supports to 3 target districts and official equipment and materials were supported by the program focusing to improve the MHIS, manage the outreach activities, improve the capacity of human resources, improve the access to health care services particularly MCH and nutrition services and strengthen the partnership networks. In addition to that the institution had a quarterly and annually between government counterpart and Plan staff to review the progress and discuss on the weak points and plan for the next step. In addition, to ensure that the implementation aligned with the government policy, the MNCH-N program communicated with the core concerned institutions of the MOH to be core trainers for the training courses.

The project partners (PHD and DHO staffs) have confidence and commitment to continue monitoring and supporting health centres and communities to implement the outreach activities/mobile clinics/other health events and to manage the HIS. A large proportion of village leaders and VHV have been trained on parenting orientation related to MNCH-N activities by DHO and PHD trainers so that they have confidence and committed to assist at commune level to promote community to utilize health services particular the MCH and EPI services. Between 2012 and 2014 the utilization of MCH services in community is improved and vaccination coverage rate among children and WRA is high.

Overall MNCH-N program contributes significantly to strengthen the health system at local level, it makes the big changes in Bokeo PHD, with this improvement it will contribute to reduce maternal and child mortality rate and improve nutrition status among children under five years in remote and rural areas of the province as well as of Lao PDR. So the sustainability would be highly realized.

However there are some constraints might be contributed to program sustainability. Ownership of government counterpart, high rotation of staff will need more training, workload for HC staffs and community leaders, lack of budget for organizing health events in villages so it will lead to few health events organized in villages. These constraints can be threat to the sustainability of the MNCH-N program. PHD should look for some supports to maintain the implementation of outreach activities in community and encourage the health staffs for more.6.2 Access to MNCH-N in community.

77

7 Scalability

7.1 WASH

Overall, it is possible to scale up WASH intervention by taking the CLTS approach. The CLTS approach has been proved effective and this evaluation findings show the enabling and disabling factors. The CLTS as an approach should be scaled up. The approach not only increase toilet coverage but also made progress on stopping OD. If the context was similar to three target district of Bokeo, operationalization of the approach has to take into account the followings: consider if the number of target villages should be reduced if there is not a strong enabling environment particularly for non-subsided sanitation. We found that the set target is too ambitious given the current context. This together with a limited number of personnel within WASH section at the district level means the many activities were not implemented as planned. The population was also cultural diverse. This is concerned with communication and facilitation by DWTs and communities. When this happen more frequent visits/follow up is needed. Given a large number of target villages this is not possible.

A working approach of DWTs should be rearranged. For sustainability purpose, the team should only work together at the triggering stage only; an individual team member can take on a follow up role over a certain number of target villages. We found that the current form of DWTs would not be sustainable. Thus Plan should consider village as the core unit for all the CLTS work. A Stronger WASH team at village level could be built with performance incentives for government staff whilst there isn’t a strong enabling environment. Over the course of the project such as 4 years, the village team should have achieved key WASH target already. When the project ended the number of WASH team members should be reduced so that incentive created at the village level can sustain it.

Construction and rehabilitation of CWSs should be scaled up. It is strongly evident that water supply and availability of water all year round is the critical factor for HHs to get access to improved sanitation practices including building and using toilets and washing hands with soap. Keeping the environment clean including the toilet depends on the availability of water. Unpleasant experiences resulted from the lack of water for toilet cleaning drew people off from building a habit of using toilets. This intervention was well recognised and highly regarded by all key stakeholders. However, scaling up this intervention should accompany by a well related arrangement i.e. the village level water use management. In terms of maintenance and sustainability, it is important technical maintenance capacity is built and financial arrangement is in place. The water point user group can be the best practice model to be considered.

Another key intervention should be scaled up is the application of Gender and Wash Monitoring Tools (GWMT). The team found that WASH addresses gender inequality and reduces women workload if designed and monitored well. The findings show that women were most responsible for fetching water, treating water and other household chores. Adopting GWMT approach can effectively address gender inequality. However, at the moment the project did have clear intervention activities that encourage or empower women. At the household level, there is no particular intervention that would increase women role in WASH related decision making even though the GWMT framework suggested women’s role in decision making is one of the key indicators.

78

7.2 MNCH-N

Scaling up the MNCH-N program is highly likely to be successful because it is aligned with the government policy. Mother, neonatal and child health and nutrition are already in the list of priority programs of the MOH. Plan focuses on the poor and marginalised people in rural areas where access to quality health services and water and sanitation facilities remain limited fill the gap left by the government.

In-depth interviews with stakeholders at the central level indicated that stakeholders were aware of the contribution of INGO including Plan that would give a significant impact on mother and child health and nutrition status and therefore ensuring the reduction of maternal and child death and malnutrition in Lao PDR. They acknowledged that Plan MNCH-N program in Bokeo is an important part in contributing to improve the MNCH and nutrients among marginalized people of Lao PDR.

There is high potential to expand the MNCH-N program to Saravane province. The KII respondents stated that the lesson learned from Bokeo MNCH-N program can be replicable however the situation analysis is needed to conduct to identify the local need and to see whether the similar project had been implemented by other international organization because in Saravane there were a lot international organization implementing projects there.

The KII respondents viewed that the situation analysis is needed as the first priority in order to avoid the overlap as now there are many development partners and INGOs interested and invested in nutrition and MCH activities. For instance, in Saravane province the UNICEF provided big support for nutrition and MCH intervention and they work closely with government counterparts and reports regularly on the progress of intervention to concerned government institutions. In addition, the KII respondents emphasized that once the program started a proper monitoring system has to be established to follow up and support communities because communities are very poor and people have low, self-awareness could not be expected to happen.

Narrative information from a senior staff of MOH said "it is very important that the Plan International in Lao PDR help us to improve the nutrition status in our country because our government cannot provide sufficient everything to everyone at the same time due to limited financial support and now most activities still rely on international supports but I just would like to recommend to NGOs that we have to work together and focus in the most remote area with difficulties to access and poor communities to help vulnerable people access to quality health services, it is also important to keep in mind that the regular report on the progress of program must be submitted to concerned government institutions"

Regarding the scaling up of activities some challenges need to be carefully considered: lacking of staff particularly at health centre and DHO, limited capacity to perform tasks, already workload to HC and village authorities such as VHV and village head, support materials and some budget allocations. The establishment of many new committees in villages should be limited but promote to use the existing VHC (this committee composes of a representative of village organization in a village)

79

8 Gender, Accountability and Inclusion

8.1 Gender, accountability and inclusion in the WASH program

The key strategies employed by the WASH project including participation, women’s representation, government attitudes, recognition, upfront agenda and monitoring.

Women participation

FGDs results show that participation in the project activities such as CLTS sessions at the village level remained dominated by the head of the household and the head of the household are usually men. This is particularly true for ethnic groups in remote areas where many women are still illiterate. It is not clear how Plan promoted women participation in such events as village meetings. The Plan staff and government officials involved in implementing the project activities reported that they encouraged women participation but was not able to explain how they did that. The advanced notice calling for people to join in the meetings did not emphasize encouragement of equal participation of men and women.

The project incorporated gender sensitivity in the design of latrine by separating the girl and boy rooms. Ministry of Education and Sports does not have a standardized model for latrines in schools. UNICEF chooses to build a climate change adapted, gender sensitive and disabled inclusiveness model for its WASH in Schools Program. However, other development organization i.e. CRS does not adopt the model. The results of FGDs with boys and girls show a positive result for equal participation between boys and girls. Almost all FGD participants, boys and girls, shared the view that there was no inequality in their participation in the hygiene activities. They said the teachers encouraged participation by boys and girls equally.

As per the participation in the project activities, we observed equal enthusiasm by women and men and girls and boys to participate in meetings with the evaluators. During the early part of our field work, the area suffered an extended period of drought, therefore, many villagers stayed in the villages rather than going out to fields (as we expected to be the case during the rainy season). This situation does us favour by providing large population to choose from for FGDS. Besides, we observed that they want to take part in discussions with us equally. For latrine in schools, separate rooms for boys and girls were ensured. The design of hand washing station was inclusive of different aged groups. This was not found at the health facilities WASH and WASH in communities.

Women representation

Interviews with village authorities as well as discussions with men and women groups revealed that more men present at CLTS committees. The same were claimed to be true for the water committees. The CLTS village regulations have a condition that at least two members should be women. All six villages we visited achieved these targets. However, only one of these villages has a woman deputy head of CLTS (Houaythong village).

Although no quotas have been set for women for trainings, DWTs, CLTS members, etc. women are generally promoted to take part into the committees and trainings and given equal opportunity to participate in Program activities. KII interviews results show that opportunities for women to take on leadership roles are encouraged such as making presentations etc. within the Plan arranged events, i.e. quarterly meetings and internal reviews.

As found in the pilot project evaluation children participation was not well articulated in the project design as well as implementation. Interviews with village authorities revealed that

80

general trend is that HH including women and children participation in the project planning and activities had been very limited. The village heads said that women did not come to village meetings, the norm is head of the HH come to meetings and the head usually was men.

Government attitudes

According to interviews with government officials, the attitudes toward different roles and responsibilities men and women can play has changed since they had participated in the training about gender and social inclusion organized by Plan. For example, they reported that before male officials always thought that female officials should serve beers when male officials play sports like “Petong’. Now they shared the serving role by choosing to a rotating system. However, when asked if they what concretely is the have they done in relation to gender equality both the DHO and PHD levels.

The project provided the workshops/trainings aim to develop the awareness of participants about the meaning and significance of gender issue on WASH as well as other project supported activities. However, there remain challenges in terms of gender inequality in the project areas.

Upfront agenda

Promotion of women and members of socially excluded groups into leadership positions within village i.e. water and CLTS committees was clearly stated in the project MOU. However, there is little evident to show that there is a clear definition of “socially excluded groups”. Thus Plan staff and implementing partners were struggled to explain what they mean by socially excluded groups and what have been done to promote them into leadership roles.

Monitoring

The Program keeps record on training, workshop as well as seminar participants disaggregated by gender. Other disaggregation (by ethnic background etc.) is not performed. For example, 20 provincial and district health officials (including 10 female) had been trained on CLTS in Bokeo. This shows effort by Plan to promote gender equality in its interventions.

Ethnicity

Plan has provided scholarships to female and male students of ethnic minority groups within the target communities to further SBA study who will then return to serve their community. This activity is recognized as an attempt to overcome the gender barriers to using of health facilities and services by ethnic groups.

The effort to reach more marginalized and people living in the most remote areas intensified during FY15. The project’s target villages included more remote communities with more difficult access. With these challenges Plan introduced an interns system where it drew young people from target communities to get involved. These interns helped with community engagement including helping with the languages and other cultural issues.

Disability

In terms of physical facilities of WASH and health facilities WASH, we observed that technical designs of water points have not been disability sensitive. The Plan staff admitted that at the beginning the designs did not take into account the details about disability inclusiveness of water points. The interest in the concept of disability inclusiveness has just emerged as the implementation has progressed. It is not easy to change the design of the project once the

81

MOU was singed. According the interviews with key stakeholders at the central level, change to the MoU could be made but it would take long times.

There is little evident to show that disability has been integrated in the project designs. An interview with Plan staff confirmed that they had participated in the disability classification workshop which helped them to recognize more comprehensively about type of disabilities. However, there had not been interventions that integrated on disability into their design in the beginning stage. In the expansion phase i.e. to Oudomxay and Saravane disability sensitive facilities and awareness raising about disabilities will be taken into account

Accountability

The project accountability mechanism was ensured by a comprehensive reporting system. A monthly report was produced regularly by the provincial mangers to report to their National counterparts. The National managers then used these reports to compile the Quarterly reports to be submitted to the donor. In addition, annual and quarterly meetings with the PHD and key government counterparts ensured opportunities for reflections, planning, and adjusting the project plan if needed. In terms of accountability upward, reporting to the donor and government counterparts, the project has done considerably well. All the necessary information was reported back to the donors. The other important aspect of accountability was missing that is accountability downward, reporting back to the communities. In this regard, the project has now clear strategy and has not done as much as the upward reporting.

Furthermore, it was reported by the National managers that the reports were “focused on completion of activities than the changes in the villages”30. The reports were lacking the information on what has been achieved in relation what was planned. In 2014, however, there was some effort to improve the quality of the report. Some features were added to the monthly reports to ensure progress was tracked against the plan and indicators relevant to the monitoring framework were also added. The Plan staff admitted that the indicators were quite complicated and confused to the government officials.

Accountability in terms of securing sustainability of the initiatives beyond the project life is very unlikely. KII respondents expressed the concern that the government at the moment only worried about budgeting and reporting back to the donor. This means that they did not see the importance of be accountable to the community needs, thus mobilizing the funding for continuation of the activities beyond the project.

8.2 Gender, accountability and inclusion in the MNCH-N program

The MNCH-N program promoted the gender equality in all activities and all level of health system in Bokeo. Its interventions were integrated with gender equality empowerment. For example, gender equality was promoted through ensuring equal number of men and women taking part in the training course supported by Plan. The program had supported 25 women high school graduates to study midwife in Vientiane Nursing College and now became SBA. In 2013, Plan supported 13 people to study midwife in Saravane and Attapeu Nursing Colleges, 10 of them are women. In addition, all WRA in villages are targeted for parenting orientation program related to MNCH-N programs’ activities.

Both provincial and district government counterparts and Plan staffs had ownership to implement programs’ activities in communities and PHD and DHO staffs had responsibility to

30 WASH programme annual report FY15

82

monitor and support HC for the implementation. However some activities were still delayed and not implemented as expected and some HC staffs cannot run activities by themselves and they need support from DHO as a result some key information such as population figure for the target group were not collected. The strength points for the implementation is Plan together with PHD and DHO have met quarterly and annually in each district for mutual review the achievements and plan for the next steps of the implementation and address the weak points. Some HC staffs complained concerning to the leading institution when health events take place in villages, HC wishes to take a lead the health by themselves since they are able to develop a micro planning.

The program involved not only government partners from health sector but also involved staffs from other sectors such as education, culture and information, social welfare, LUW, Youth Union, village leaders and VHV to implement this program. Beside that the program involved vulnerable and marginalized people: women, children, ethnic people and remote and rural people. Women and children are direct beneficiaries of the program and remote rural and ethnic people are prioritized to benefit from this program intervention. From the desk review, it showed that the number of direct beneficiaries in all target districts utilized the health services is higher than the number of direct beneficiaries in the target districts. The number of children accessed to vaccination particularly for DTP3-Hib-BepB3 and measles-rubella vaccines, vitamin A and deworming was increased by years; the access to MCH services among women was improved particularly for 4 ANC visits, PNC visit with 7 day or 6 weeks. Results from desk review are consistent with the results of household survey concerning the access to MCH and nutrition services. However the evaluation does not see that disable people benefit from this program.

There is good evidence that lack of access to water continues to impact women significantly through the burden of water collection. Women carry two-thirds of the burden for water collection. Significant time savings are associated with WASH - time that can be used to work in the fields to improve household food security.

Women spend more time than men in the field: collecting firewood, weeding in the upland rice field (hai), or searching for food in the mountain/forests. Women participated in the FGDs accepted they worked more than men and that they were not forced to do so. It was a norm to work more they said. For example, even if they went to field together with their husband women walked home carrying a bundle of firewood while men carried nothing. Reducing women's time and energy in domestic chores in order to further free up time for adequate childcare and nutrition, therefore, should involve interventions that address both practical gender needs, but also those that address norms and traditional beliefs in order for the impacts to be sustainable.

The health risks of water related deceases affects women and girls more than men and boys as it is in most societies their task to care for sick family members.

83

9. Conclusion

This report has presented results from a mixed method evaluation comprising of key informant interviews, FGDs, a household survey on WASH and MNCH-N in three district of Bokeo and the site observation. The evaluation concludes that the two programs are relevant in terms of responding to the needs of the community and aligning with the government policy and priorities. The two programs complemented the sectoral programs and consistent with the adopted approaches within the sector. Given the difficult social and geographical context the two programs had produced some great achievements with integration of the Plan’s core values such as gender equality, disability sensitivity and social inclusion. The evaluation also show survey results that underscore the fact that WASH and MNCH-N related challenges continue to plague the rural population of Bokeo.

9.1 WASH

The WASH interventions are by definition cross-sectoral. They both are inter-related at the very least, the Ministries of Health, Ministries of Education and Sports, who are responsible for ensuring hand washing facilities and toilets, are available in schools. Ministry of Health is responsible for ensuring that there is access to water in health facilities. Plan’s efforts in Bokeo have involved working with the Department of Education and Sports and the Department of Health. At the district level, they worked in the more multi-sectoral setting as demonstrated by the composition of the DWTs and to a lesser extent the mobile clinic teams.

The results throughout the body of the report confirmed that adoption of the CLTS approach is appropriate in addressing the community needs and government policy contexts. The approach has been proved effective in increasing coverage thus lead to ODF achievement in the community. However, there were some contextual factors that remained a challenge for the WASH to make progress. The evaluation shed some light on why such a slow progress has occurred. These included a lack of a coherent and effective approach adopted by the sectors between CLTS and subsidies. The community with history of subsidies waits for such a hand out and did not invest on their sanitation facilities. Whilst the project made some concerted efforts, at the point of writing the project was yet to find the key strategies/triggers that have accelerated progress for WASH improvements at the community level with relation to changing behaviours habitually. Indeed, encouraging demand for WASH services – whether this is investing in and using a latrine, or washing hands after defecation and before eating or safe storage of water in the household – remains a major challenge.

In terms of coordination, the project had established effective mechanisms such as steering committees and a number of meetings. However, in practice, due to the delay in coordination at some point rolling out of the project activities delayed considerably. According to the staff interviews this is partly due to lack of the effective cooperation and ownership of the government counterpart at provincial and district levels.

Sustainability of the WASH facilities was also critical and challenging. Some facilities were built with poor quality as a result of poor supervision. Operation and Maintenance of the facilities remain of a concern. The water committees are functioning well. However, there were signs of weakness. In the villages where the facilities have been built for more than one year malfunctioning started to be seen.

Financial support for sustainability of the activities initiated by Plan was not foreseeable. The government at all level could not ensure the availability of the funding to continue the

84

activities. Sanitation had a low profile on the government agenda. There is no financial allocating from the government toward this sector. This will have two key negative implications. One is that the facility built would not get adequate maintenance thus would not be useable. The other implication is that the new skills acquired such as community triggering would soon be forgotten as the staff had no chance to put it into practice.

The project had some specific gender strategies that mainly related to women’s participation and progressing practical gender changes, which has had moderate progress. Through the introduction of the GWMT, the project had commenced monitoring strategic gender indicators within its fourth year of implementation.

9.2 MNCH-N

Overall, Plan’s MNCH-N program in Bokeo had significantly contributed to the improvement of the access to integrated MNCH-N packages in three target districts. The outcomes have been achieved through Plan’s supports such as mobile clinics, outreach activities, health promotion day, quarterly health day, children day, measles campaign and functioning SBAs. For example, the direct beneficiaries accessed to the DTP3-Hib3-HepB3, measles, and anti-tetanus vaccines, vitamin A, deworming, ≥ 4 ANC visit, PNC visit within 7 days and 6 weeks, ≥ 3 PNC visit, iron tablet, and contraceptive had increased. Mothers had better knowledge on the 4 important signs to bring children to health facilities immediately. Beside, some dissatisfactory outcome was observed. The nutrition status among children aged less than 2 years: wasting and stunting remained very high due to poor diet and food unsecured in the households. HH food mostly comes from natural sources surrounding their villages like forest, mountain, rivers, and ponds.

However, there are some areas where the results are limited. For example, mothers did not take a great care for their children. Even when children were crying and having fever mothers still thought that was normal for the children. Delivery at home remained highly practiced and most delivery attended by relative/friends because women were shy to use the health services at the facilities.

Through several training courses and study visits to successful programs in other provinces, the staffs become more competent in many areas. They were able to conduct community consultation exercise,in the villages i.e. facilitating FGDs with mothers, fathers, and grandmothers. They become core trainers on PCA and Parenting Orientation related to MNCH-N for village facilitators and VHV, monitor and support to health centres on data management and recording. Strengthening capacity of health staffs at district level had been undertaken through training on MNCH-N and Parenting Orientation related to MNCH-N activities. DHO staffs formed as MNCH-N team in collaboration with health centre staff to conduct health events in communities. However, ownership among some health staffs were limited because some of them believed that their work with the Plan supported activities were not recognised appropriately by their performance assessment system.

At the health centre level, beside the strengthening capacity of health staffs, the program supported the extension of health centre building for MCH services such as room for delivery, MCH services and waiting including some essential obstetric equipment and water supply and sanitation. However it is highly concerned to the maintenance of properties in health centre and cleanliness of working places.

85

Moreover, the health management information system was improved by supporting skills on computer and Microsoft Words to PHD and DHO staffs, supporting quarterly and annual meetings among PHD, DHO, health committee and Plan. Plan supported some official equipment and materials to facilitate the collecting, keeping and storing data. In addition Plan paid attention to promote the MHV as it is served as the main statistic for village and now 9 villages were declared to be MHV.

In terms of coordination for the MNCH-N program, the feedback was all positive. Plan collaborated with PHD very well, likewise other sectors like education, cultural and information, social welfare, LWU, Youth Union. The feedback on its relationship with key institutions at the central level as mixed. Some said MCH centre including EPI program and Nutrition centre and supported to NIOPH did not receive update on the progress made by the program. This may have happened due to some disfunctioning of some procedure with the government as discussed earlier. Officially, Plan is able to report six monthly to concerned institutions at central level.

86

10 Recommendations

The recommendations for the future direction of the program are provided on short-term and long-term basis.

Short-term recommendations

1. Despite impressive achievements in many respects of the program, the nutrition status of the target children under 2 years old has yet to achieve the target due to poor feeding practices and poor sanitation and hygiene conditions. As the national trend toward refocusing on nutrition is rising, it is important that Plan joins the government and Development Partners (DPs) by using the existing platforms such as SUN to scale up WASH in Nutrition approach.

2. WASH and MNCH-N programs should be integrated as the outcome from WASH program can complement the outcome of other program. Synergized outcomes could provide positive impact to the nutrition status of children under 2 years and mother neonatal and child health and nutrition in community.

3. At the moment, there is no link between WASH promotion at village level and the Health Center staff. With a smaller DWT some WASH responsibilities should be delegated to HCs because they are close to communities and have the overall responsibility of the health status of people in the catchment area.

4. WASH and MNCH-N can jointly operate (convergent implementation). At the community level with CLTS committees, water committees, VHC integrated into one committee but keeping members with mechanical skills for toilet construction and water supply system maintenance.

5. WASH, particularly water supply should provide water beyond drinking and household use to cover animals raising and gardening. By doing this WASH can contribute more directly to improvement of nutrition outcome in the communities because we know that part of the root causes of malnutrition are the lack of access to food. Animal raising and gardening can increase food availability thus access to food.

6. Food hygiene can be integrated into intervention. Hygienic food preparation and storage should be integrated into the intervention.

7. Some WASH facilities in schools and communities started to break. It is important to urgently address this problem. To do this, the project has to change its monitoring and follow up arrangement. Follow up support beyond one year is needed to help the party responsible in the village level build their competence and experience.

8. Rather than working from Plan office, provincial managers should take the office space provided by PHD. This will help foster better coordination and communication.

9. The DWTs are unlikely to be sustainable and remain effective beyond the project life. Number of DWT members should be reduced to two or three as the bigger the team the more unlikely it will be sustainable and effective. This is to redirect more resources into community level. Plan should rely on the village level. It is important that Plan start to build a strong and competent WASH team at the community level, ensuring the each community has a fee collection system that can cover the minor maintenance cost. Sustainability of the results of the interventions relies on the organisation at the community level with only periodic support from the district level.

10. It is important to make sure a strong governance system at the community level is in place. This includes transparency of the water fund management, strong and competence technical person who can deal with practical repair and maintenance of the facilities.

87

Long-term recommendations

There was consensus among key stakeholders that CLTS works and could lead to effective use and behavioral changes amongst target population. It is recommended that Plan:

11. Promotes cohesion within the WASH sector concerning adoption of CLTS that is still lacking. Plan should work with MOH and through WASH TWG to advocate for wider adoption of the approach. At the moment, there is an ineffective coordination or communication between central and local government counterparts, such as variation of their view on CLTS approach.

12. Plan should also continue to increase efforts to get other stakeholders on board in terms of CLTS adoption. This will include clarifying with key stakeholders that CLTS works but takes time to get the results.

13. School WASH is clearly inextricably linked with the rights of the child and enhanced child survival and must therefore remain a key priority for Plan

14. Convergent planning and implementation for a more integrated, comprehensive WASH and MNCH-N must be strengthened to build the potential synergies.

15. Maintenance of WASH facility in schools should be the responsibility of the VEDC because in the foreseeable future there will not be any improvement in the budget situation of the school.

16. Engage more actively with the sanitation material suppliers in terms of outreach activities and advertising of their products to the local communities.

17. Apart from addressing the communication (intern)/engagement strategies (language) adopted by Plan, an approach where specific interventions are amenable to change at community level by different ethnic groups may need to be considered. For example, the “prescription of solutions by District Authorities” model should be changed to “solution exploration by community” model. Furthermore, slowest groups might need more engagement and more encouragement or different approach/strategies altogether.

18. Report on program of Plan should be distributed during the meeting of technical working groups or health research forums. To make sure that the concerned institutions at the central do miss out on the progress of the program.

19. It would be effective in implementing if DHO and HC meet each other and discuss about the role and function of organization to lead the health event in villages. DHO should work closely with health providers and encourage the assigned staffs to perform their tasks as expected and with high responsibility.

20. PHD should report about the Plan program implementation to MOH and national health forums.

21. The representative from the Central Ministry who attended the annual meeting should report to its Director in a written manner.

22. Generally speaking, malnutrition has not seen significant improvement, PHD and DHO together with Plan should coordinate with local agriculture and forestry for the promotion of home gardening, protecting and nurturing natural sources as good food source for community. At the same time some knowledge on the important of eating variety of food and more food groups should be explained carefully and in ethnic language.

23. The delivery at health facilities rate remains low, due to cultural barriers among ethnic women, such as they were too shameful to let other people see their PHD and DHO have to be sure about gender of health provider at health centre, at least one women should be presented at HC for delivery of health service particularly MNCH-N services. PHD should discuss with Plan or other international organization for more SBA development to ensure that delivery attended by SBA and PHD should consider about recruiting the SBA produced by Plan into HC level.

88

ANNEXES

Annex I: ToR

Plan Laos Maternal & Child Health and WASH Programs

Bokeo Province

Consultant’s Terms of Reference (TOR) for

Evaluation, Facilitation, Design and Documentation

May - June 2015

1.0 Background

Plan is an international child-centred community development organisation without political, religious or governmental affiliations. Plan is committed to quality programming through our Child-Centred Community Development (CCCD) approach. CCCD is a rights-based approach to development where communities are supported to develop the structures and skills they need to provide a safe and healthy environment in which children are able to achieve their full potential.

Our rights-based approach is guided by clear standards and obligations provided by international human rights instruments, primarily the United Nations Convention on the Rights of the Child. Child rights are protected and realized by raising awareness amongst children, youth, families, communities, the government and others about the rights of every child, and by working to build the capacity and commitment of those responsible for ensuring these rights are recognised and supported. This means that the protection and access to rights of all community members is essential for the creation of a positive development environment.

Plan operates in 5-year cycle of Country Strategic Plans in which the issues each Plan office envisions to address, the strategies and other programmatic definitions are given by way of “Country Programmes”. Plan operationalises it’s Country Programmes through projects implemented at multiple levels. Plan Laos’s current Country Strategic Plan 2014-19 has five key Country Programmes (CPs):

Early Childhood Care and Development CP(ECCD)

Basic Education CP (BEP)

Child Protection CP (CRCP)

Water Sanitation and Hygiene CP (WASH)

Maternal, Neonatal and Child Health and Nutrition CP (MNCHN)

While Lao PDR has made significant progress in its human development indicators over the years, it is still struggling to achieve optimal standards (including the achievement of MDGs) in many sectors, especially in the areas of WASH and MNCHN. Plan International in Lao PDR therefore envisions working with the Government of Lao PDR and the Development Partners to address the issues faced by marginalized children, especially girls, in Laos through participation, partnerships, advocacy and high quality programming. Plan has programme or

89

project-based Memorandums of Understanding (MoUs) with the relevant provincial authorities that it works in.

Since designing of the current country programmes was undertaken, there has been an increased focus on improved nutrition in women and children, from the highest level of government. A Prime Ministerial Decree was enacted in mid 2013 establishing for the first time an inter-ministerial Nutrition Committee, and Secretariat. In late 2013 a UN Task Force (IFAD, UNICEF and WFP) developed a new Food and Nutrition Security Action Plan (FSNAP- December 2013), based on the convergence model of multi-sectoral planning and monitoring, and sectoral implementation, across four sectors (Education, Health & WASH and Agriculture). Phase one of this plan prioritises Luang Namtha, Oudomxay and Saravane, with phase two adding Phongsaly, Xieng Khouang, Houaphan and Sekong.

2.0 WASH and MNCHN Program Overview

The Plan Laos WASH program in Bokeo province, which was considered in its pilot phase, started in 2009/10 and ran until 2011/12. Plan first commenced an MNCH Program in Bokeo province in 2011/12. Both programs are implemented in collaboration with the Provincial Health Department (PHD), each under a separate MoU.

Currently Plan’s largest grant in the WASH programme runs for a four-year period from July 2012 to June 2016 (attachment 1- project design). The project has initially focused on 3 districts in Bokeo and was expanded into Oudomxay in January 2014. The project will be phased out from 2 of the 3 districts in Bokeo in July 2015. Activitieis will continue to be implemented in the remaining one district in Bokeo and Oudomxay at different levels of engagement. This evaluation will be undertaken only in Bokeo, and serve as the Mid Term Review of this grant, to inform a sustainability strategy in Bokeo and facilitate replication of lessons learnt and good practices in Oudomxay, and Saravane where Plan has started WASH projects recently.

The main MNCHN grant under the MNCH programme has been implemented only in Bokeo, since the inception of the MNCHN programme. This project will come to an end in June 2015 (attachment 2 – project design). Following the inception of the MNCHN programme in 2011/12, a program review was undertaken in 2013 to ensure Plan’s MNCH work is aligned with MOH priorities and global best practices. This resulted in a revised approach (attachment 3) for MNCHN. This evaluation will serve as an end of project evaluation for this project. However, the Memorandum of Understanding which Plan has with the PHD for MNCHN outlasts this project and runs until December 2016, and Plan intends to continue to engage with MNCHN with a focus on nutrition, in a way that is integrated with WASH, with its own funding or new funding to be raised. The evaluation will help to refocus the MNCH programme in Bokeo, and identify lessons and good practices on nutrition programming that may be replicated in other provinces, when funding opportunities arise.

At the start of the implementation of this revised approach in the MNCH programme, a combined WASH and MNCH baseline was done in September 2013 in Bokeo.

Plan International Australia (PIA) has been supporting Plan Laos build its WASH programming since its inception while Plan International Germany (GNO) supported the MNCHN programme from the beginning. The program funded by PIA is primarily through Department

90

of Foreign Affairs and Trade of the Australian Government. Plan Germany funded the MNCHN programme through “GNO Girl’s Fund”.

The overlap of goals and potential of synergy between WASH and MNCHN had always been recognized by Plan since the beginning of both programmes. However, apart from the baseline and this evaluation the two programmes evolved independently from each other. Acknowledgement of the fact that integration of WASH and MNCHN strategies would yield better outcomes was growing with the global evidence and experience that Plan gained through implementing these two sectoral projects. In the beginning of 2014, this thinking lead to the creation of a role within Plan at national level that envisioned integrated programming in WASH and MNCHN for better outcomes. This evaluation is expected to provide empirical evidence and recommendations for Plan to initiate the process of integration.

91

2.1 WASH Project

The goal of the Plan Laos WASH project in Bokeo and Oudomxay is to “improve the health conditions resulting from changed and sustained WASH behaviours among children, women and ethnic and other vulnerable groups in 170 selected villages in Pha Oudom, Mueng, and Paktha districts of Bokeo and in Houn district in Oudomxay province by 2016, thereby contributing to Lao PDR’s MDGs and development goals for 2020.” Further, the project has four intermediate outcomes/objectives that seek to contribute to the project’s goal:

1. Intermediate outcome/objective 1: Improved and maintained good personal and household hygiene, hand washing with soap at critical times, and consuming “safe” drinking water among community members, particularly among women and children in selected target communities and schools

2. Intermediate outcome/objective 2: Achieved and maintained ODF status in 80% of the selected target villages and schools

3. Intermediate Outcome/Objective 3: Active water management committees capable of maintaining their water systems and water source catchment areas for serving the needs of villages and schools in selected target villages

4. Intermediate Outcome/Objective 4: Effective and participatory governance in the WASH sector by Government authorities from the provincial to the village level

Please read the full project design document which outlines the project’s strategies and Approaches (Attachment 1). To note:

The project design had a number of key strategies for promoting gender equality

The project works through District WASH teams (a number of government staff and minimal Plan staff)

The project originally planned to end in Bokeo by July 2016.

The project started working in Oudomxay in January 2014.

The project has also incorporated Water Safety Planning (Attachment 4), a study into Smart Subsidies (Attachment 5) and a draft Child Protection Risk Assessment (Attachment 6) –these activities were not part of the original project design and hence are not reflected in the project design document (but rather were based on opportunities, learnings and compliance requirements).

Plan has supported the production of two key comprehensive studies under this project. They are

1. Consumer Research to Inform Rural Sanitation Behavior Change and Marketing Communications for Lao PDR’ (Attachment 5)

2. Supply Chain Analysis for Rural Sanitation Products and Services in Lao PDR (Attachment 7).

92

Review of both these reports and analysis of the extent to which the recommendations should be implemented in a revised project design needs to be considered as part of this consultancy. Plan has also completed a number of studies on specific topics in WASH to inform the sector as well as our own work (see WASH bibliography Plan Laos) which will provide the context, the operating environment and the specific challenges in Lao PDR for WASH programming. These reports can be made available to the consultant to be referred to as needed.

93

2.2 Maternal, newborn, child health and nutrition project

The Overall Objective of the MNCHN project is “to improve the health and nutrition status of the population of three Bokeo (Pha Oudom, Mueng, and Paktha) Districts in Lao PDR, through an integrated primary health care approach that emphasizes the health of women and children”.

Its Specific Objectives are:

SO1: To contribute to the development of a sustainable Primary Health Care system in

Bokeo Province

SO2: To promote Universal Access to Quality Health Care for the families of three Bokeo

Districts

SO3: To engage Bokeo Communities for health by seeking their advice, and encouraging local reflection, prevention and promotion activities

As outlined above, the Project Design document (attachment 2), revised project design (attachment 3) and the baseline report (attachment 8) are available for reference and analysis by the consultants. There is also a basic MNCHN situation analysis which was done in 2011 before the start of the project for reference. In addition to these documents, routine monthly monitoring data is available for selected MNCHN indicators collected during the project period.

3.0 Purpose of the combined WASH Mid Term and MNCH end of project evaluation

This evaluation will be undertaken in Bokeo, as an MTR for WASH project and an end line evaluation for the MNCHN project. The evaluation will not be undertaken in Oudomxay province.

While the WASH and MNCHN programs were designed and delivered independently from each other, Plan intends to integrate the two programs with a renewed focus on nutrition, keeping in line with the country strategic plan 2015-2019 and the government and donor priority to reduce stunting through multi-sectorial programming31. The current funding environment in the health sector also compels Plan to rationalize and refocus investment in health programming more strategically and selectively. The joint evaluation will not only allow us to holistically assess impact and synergy of the two programmes on health and nutrition, but also to identify lessons and opportunities for intersectoral programming.

Specifically, this evaluation aims to:

1. Assess impact and outcomes achieved through the respective projects against the approved project designs, and identify areas of synergy between the two programmes.

31 The Government of Laos’s Food & Nutrition Security Action Plan (FNSAP) integrates Health and WASH interventions to address nutritional outcomes.

94

2. Assess the likelihood of sustainability of the impact and outcomes at the grassroots, district and provincial levels, with particular attention to the capacity of the Provincial and District health care delivery system and linkage to the private sector in sustained access to WASH services. Draw lessons learnt on Plan’s role, approaches and its financial investments vis-à-vis the public and private sector actors on Health and WASH in Bokeo Province.

3. Assess alignment of the MNCH and WASH programme designs with the FNSAP, and contribution of the programmes to addressing stunting. Highlight opportunities and gaps in the current programme designs that are of relevance to future programme development on a WASH-MNCH integrated design with a stronger focus on nutrition in Oudomxay and Saravane provinces. This is especially relevant as Oudomxay and Saravane are prioritized provinces in the geographical focus of FNSAP.

4. Provide evidence on effectiveness of interventions and models used in the WASH and MNCHN programmes, and identify elements of replicability and scalability, while recognizing factors to be considered in adaptation to specific contexts.

5. Gather voices and feedback on the two programmes from diverse population groups in the target communities. Facilitate participatory, yet critical reflection and learning among Plan and the government counterparts on how changes happen at the community level. .

The evaluation will feed into a revised project/programme design for an integrated WASH and MNCHN programme in Oudomxay and Saravane Provinces. This evaluation will also inform the sustainability plan for the current interventions in Bokeo beyond 2015.

4. Scope of the consultancy

Specifically, the evaluation should address the following questions as part of the evaluation and, where relevant, highlight differences between the two provinces:

(The key questions have to be read from the perspectives of both MNCH and WASH programmes unless specified otherwise)

Relevance

1. The extent to which the project designs, implementation and monitoring processes is being/has been executed

2. The extent to which the project design is relevant - from a technical point of view taking into account the prevailing and emerging issues, new opportunities in the respective local contexts and from the perspectives of local government and communities, particularly women and children

3. The extent to which activities and outputs are in line with and has been able to inform the local government priorities, capacities, and approaches and coordinated with other local stakeholders (eg. Civil society, private sector);

95

4. Consistency with national sector policies and the extent to which the project experiences have informed/can influence the ongoing policy dialogue in the MNCH/WASH/nutrition/health sectors;

5. Identify and assess any underlying external factors (social, cultural, political or economic) which influence the achievement of project objectives and analyse the extent to which the project design takes these into account. Ie. Whether there are locally-based mechanisms designed to address those, overcome those or avoid those from becoming obstacles.

Effectiveness

1. Assess the effectiveness of the strategies and interventions and their progress towards achievement of project results and objectives against the key result indicators.

2. Assess the effectiveness of the approaches employed, particularly the approaches used to promote health seeking, food and nutrition, sanitation and hygiene behaviour change (this includes ODF slippage).

3. Assess the effectiveness of project monitoring, including the measures of progress and reporting approaches.

Impact and synergy

1. Assess the degree to which the project objectives and expected results have been achieved as intended in the MNCH project and the likelihood of achieving those in the WASH in the remainder of the project timeframe.

2. Assess the degree to which MNCHN and WASH programmes have complemented each other, particularly in achieving health and nutritional outcomes and impact

3. Draw lessons on inter-sectoral programming to achieve greater synergy

Sustainability

Referring to project objectives, results, outputs and indicators, the Consultant will assess the sustainability of project benefits at community and local government institution (LGI) level, specifically:

1. Appropriateness of Plan’s role and contributions, as perceived by local stakeholders.

2. The extent to which project training, empowering approaches and capacity-building at community level and in Local Government Institutes (LGIs) have led to the presence of sustained demand/incentives for change, and the necessary skills for the effective planning, management, implementation, maintenance and independently undertaking Primary health/WASH roles and responsibilities

3. The extent to which project approaches contributed to LGI and beneficiary ownership of the project, specifically ownership and responsibility for maintenance and management of continued health care provision, water systems and latrines;

4. Establish to what extent beneficiaries contribute to decision-making throughout the project cycle, including the extent to which beneficiary communities (including children and women)

96

were aware of the basis for decisions and/or involved in decisions about the selection of sites, technology and service level options.

5. The extent to which the promotion of improved hygiene practices has been integrated into the water, sanitation, education and health services as a result of the project interventions;

6. Assess local operation and maintenance arrangements for water and sanitation facilities: who is responsible for maintenance, while making a distinction between the need for corrective inputs and preventive maintenance;

7. The extent to which the supply chain has been developed to deliver WASH products and services regularly at an affordable rate within the context outlined in the supply chain study;

Scalability

1. In comparison to the documented good practices by Plan and other stakeholders on primary health care (eg. Save the Children), nutrition (LANN approach), CLTS, behavior change communication, social marketing etc, what are the specific interventions and programme delivery models that can be recommended for scaling up and replication with respect to the projects evaluated

Gender, Accountability and Inclusion (in relation to Plan’s Child Centered Community Development standards – attachment 10)

1. The extent to which the project has methods to challenge the gender norms and barriers related to MNCHN/WASH such as the women’s workload, and promoted equality and inclusion, especially with regards to the role of women and girls in decision making at the community and household levels.

2. Extent to which and ways that the project has ensured participation of children and youth in program planning and activities.

3. Extent to which and ways that the project has ensured participation of people with disabilities and ethnic minorities in program planning and activities.

4. Extent to which and ways that the project promotes accountability towards beneficiaries and key stakeholders.

Under each of the headings above, the evaluation should provide a number of specific, practical and realistic recommendations that Plan is able to take onboard in a revised project design document.

5.0 Methodology, deliverables and timing

The evaluation will be conducted in all project districts within Bokeo, with a combination of villages targeted by both programmes. Review of the financial elements of the action is excluded from this consultancy.

97

5.1 Desk research and review

A review and analysis of documentation relating to the program, within Laos MNCHN and WASH sector and the development sector is an integral aspect of this evaluation. The following reports and documentation are available for reference during the review (including but not limited to):

1. Attachments 1 to 10 as listed above

2. Baseline data in project villages of Bokeo province

3. MNCH and WASH MoUs

4. Quarterly and annual narrative progress reports

5. National MNCH strategy 2009-15

6. National MNCH strategy evaluation report (draft)

7. National Plan of Action for Rural Water Supply, Sanitation and Hygiene

8. Food and Nutrition Security Action Plan Lao PDR

9. EASAN CLTS progress report

5.2 Field work

In investigating the key review questions a team of evaluators is expected to facilitate the active and meaningful involvement of project beneficiaries, including boys and girls, women and men, relevant village structures, local government, community organizations and other interested parties (including Plan Laos staff and Government partners) in the review process. It is expected that the methodology will include both qualitative and quantitative data collection methods and will consider both primary and secondary data sources.

5.3 Personnel and reporting lines

The evaluation team will lead the review and be responsible for the whole review process, including detailed review planning, field work (data collection, interpretation, discussion and analysis), drafting the initial draft report, feedback sessions and production of the Final Evaluation Report in three components as described below.

The evaluation team will report to the Plan Laos Public Health Advisor (Dr. Kalana Peiris) and work collaboratively and communicate regularly with Plan Laos WASH Manager (Dr. Ketsadasak Kiettisak), Plan Laos MNCHN Manager (Dr. Boualaphanh Inthaxay)

Plan Laos will provide necessary logistical and administrative support, including transport, organisation of meetings, and collection of documents/information required.

5.4 Deliverables

The evaluation team will produce the following deliverables:

1. Inception Report - this report should detail the review plan, the key review questions and for each the methodologies to be used (and where relevant), sample sizes and criteria for sampling. It will include copies of the data collection tools to be used and outline the expected timing of the review and the roles/requirements for involvement of Plan Laos staff. A draft

98

evaluation plan will be submitted to the WASH/MNCHN programme team in Plan Laos and PIA. PIA and Plan Laos will review and provide feedback to the Consultant and the evaluation plan will be finalized by the Consultant before the commencement of field work upon receiving approval by Plan Laos Country Director or the Deputy Country Director - Programmes

2. Draft evaluation report – a first draft of the review report in three components:

1. Summary report highlighting WASH/MNCHN integration, strengths, collective impact, opportunities and recommendations for going forward

2. Detailed Report on WASH

3. Detailed Report on MNCHN project

The two detailed components each should include the context of the review, its purpose, methodology and results, and preliminary analysis of the evaluation data disaggregated by gender and disability, (plus collated raw data as appendices if relevant). This should be first presented to the Plan Laos programme management team by way of a Power-point presentation at a Country Office level validation workshop. Then the draft report should be circulated to Plan Laos, PIA and GNO for comments and be taken into account for the Final Evaluation Report.

3. Final Evaluation Report - the Final Evaluation Report in three components in the same outline described above. Dissemination of the final report is the sole responsibility of Plan.

The design and implementation of the evaluation must take into account and abide by Plan International’s policies and Standards. This means, for example, ensuring children are at the centre of assessment, that principles of gender equality, inclusion and non-discrimination are considered and acted upon throughout, and that the meaningful participation of children and other key stakeholders is promoted in the design and implementation of the assessment as per Child Centered Community Development standards. Guidance on Plan principles on the conduct of direct research with children and young people will be provided to the recruited consultant/s. Furthermore, the assessment is required to be conducted in-line with Plan International’s Child Protection Policy and internal guidelines on Child Protection and ethical standards in Monitoring, Evaluation and Research.

5.0 Tasks, expected outputs and suggested timeframe

This consultancy is 40 person-days duration to be carried out within the period between May to July 2015, and field work to cover, tentatively, the dates of 04th June 2015- 30th June 2015 (exact dates to be confirmed by Plan Laos). The final report should to be submitted before 15th July 2015.

99

No Activities Outputs Responsible Duration

1 Briefing discussion with Plan Lao and PIA (following background reading)

Consultants

Plan Laos

PIA

1 day

2 Production of Draft Inception Report including detailed draft methodology, work plan, staffing, and data collection tools.

Draft Inception report submitted for approval to Plan Laos and PIA

Consultants 2 days

3 Consultant to revise and finalise report on the basis of comments received by Plan

Final inception report agreed between consultant and project staff.

Consultants 2 days

4 Desk-based and field work:

Work with project staff to prepare for field work as needed

Undertake field work (as per approved methodology). This should include discussions with key stakeholders (Government and WASH actors) at various levels.

Conduct data entry and analysis (as per approved methodology e.g. could be done as a participatory exercise with project staff).

Presentation of the key findings in-country

Qualitative and quantitative data collection completed.

Raw quantitative and qualitative data files

consolidated; first draft report submitted for review to Plan

Consultants 19 days

[+2 travel days]

The number of days in-country to be

negotiated within the 19 days (+2 travel days)

5 Draft the MTR report including the validation workshop

Draft MTR report produced

Consultants 5 days

6 Finalisation of the evaluation report on the basis of feedback from Project Staff and PIA.

Final report produced Consultants 4 days

Total Consultant Work/Days 35 days

Budget

The MTR expenses will be covered by Plan Laos WASH and MNCH program. Accommodation, transport and visa will be organised and covered by Plan in line with Plan’s policies and procedures.

Other Requirements

Travel requirements to comply with contract requirements, including reimbursement for in-country expenses and accommodation.

100

The consultant must meet Plan Laos professional indemnity and public liability insurance requirements.

The consultant to invoice Plan Laos one lump sum payment upon completion of the work

Approval of the final report by the Country Director or Deputy Country Director Programs Plan Laos.

Plan is open to negotiate date changes with reasonable advanced notice from the Consultant due to the implications/flow-on affects changes can incur unless they affect the deadline for final report.

6.0 Expected Profile of consultants/evaluation team

The consultants/evaluation team is expected to demonstrate the following competencies:

Experience/expertise in relevant areas namely MNCHN, WASH, hygiene education and behaviour change programs/evaluations.

Experience in conducting evaluations and expertise in design, field experience in data collection, data collation and analysis and good report writing. This includes being able to demonstrate strategic thinking, and practical and realistic recommendations.

Good understanding and evaluation of social inclusion, gender and disability in programs –particularly beyond practical aspects to strategic changes.

Experience in participatory evaluative approaches, including strength-based ways of working with NGOs, government and community members.

As the expertise required ranges across two wide sectors, applications from teams with members who have relevant experience and expertise are preferred. However, consultants who have expertise in one of the two areas are also encouraged to apply as individual consultants. In the event that two or more individual consultants are selected, Plan will facilitate team formation.

7.0 How to apply

If you’re interested to apply, please email your interest to the contacts listed below no later than 18th

May 2015, with the following information:

A brief summary with Curriculum Vitae, of all key members (if applied as a team) of the team, demonstrating you meet the criteria above – please highlight the area of expertise (WASH or MNCHN) if applied as an individual or highlight the member who will be the WASH lead, and the member who will be the MNCHN lead if applied as a team.

Confirmation of your consultancy rate

Confirmation of your ability to work in the dates provided and meet the required deadlines

One recent example of similar evaluation report you have written.

At the next stage, preferred applicants will be contacted for an online discussion.

For further information please contact:

Dr. Ketsadasak Kiettisak - Ketsadasak.Kiettisak@plan-international.org

Dr. Kalana Peiris –kalana.peiris@plan-international.org

OR for queries specific to WASH project: Ms. Lee Leong –lee.leong@plan.org.au

101

Annex II: Results from end line survey in 2015 compared to baseline survey - indicators for MNCH-N

No Indicator

District Prov coverage

1

National

Coverage1

MM PKT POD Total

Access to health care (n=365)

1 % of women living within 5kms or 1 hours walk of a health facility

48.7 46.9 24.4 39.2

Baseline data 60.7 58.7 42.1 51.5

Family Planning

2 Contraceptive Prevalence Rate (n = 353) 66.1 53.6 51.9 56.9

Baseline data 48.9 50.9 48.4 49.2

ECCD

3 % women who know 4 reasons to take a child to health facility urgently

6.4 0 4.5 7.7

Baseline data 0 3.7 4.9 3.4

Pregnancy, Delivery and Postnatal Care

ANC visit (n=72) (n=17) (n=25) (n-29)

4 % women ≥ 1 ANC visit in last pregnancy 23.5 61.5 44.8 45.8 37.9 54.2

Baseline data 57.9 62.8 47.3 54.2

5 % women ≥ 4 ANC visit in last pregnancy 52.9 38.5 51.7 47.2 34.3 49.5

Baseline data 22.2 38.5 20.8 26.6

6 % of WRA who delivered in last year whose infants were protected against tetanus at birth

70.6 84.6 58.6 70.8 72.4 65.8

Baseline data 27.8 42.5 59.3 46.2

7 % of women who received iron supplementation during the last pregnancy

100 96.2 96.5 97.2 34.3 49.5

Baseline data 56.9 62.6 37.6 50.0

Delivery (n = 111)

8 % births taking place in a health facility 19.3 28.6 31.1 27.0 30.7 37.5

102

Baseline data 28.4 35.1 26.7 29.6

9 % of delivery at home 70.9 65.7 64.4 66.7

Baseline data na na na na

10 % births attended by a skilled birth attendant 19.3 37.1 31.1 29.7 32.1 41.5

Baseline data 29.1 38.4 28.5 31.5

11 % of birth attended by relative/friends 70.9 62.9 55.6 62.2

Baseline data na na na na

Post Natal Care (PNC) n=36

12 % women receiving PNC visit within 7 days of delivery 58.8 57.1 58.3 58.3 1.4 4.9

Baseline data 7.9 16.8 3.9 8.7

13 % women receiving PNC visit within 6 weeks of delivery

58.1 20 31.1 35.1 3.0 7.1

Baseline data 13.6 15.9 5.6 10.5

14 % women ≥ 3 PNC visits 23.5 0 8.3 13.4

Baseline data 4.5 10.6 1.3 5.2

Satisfaction with health care services (n=111)

15 Number (%) of women reporting satisfaction with health care received during last pregnancy

80.6 85.7 77.8 81.1

Baseline data 72.8 71.8 61.9 67.3

Newborn and child health care

16 % Children born in last 12 months who were breastfed within 1 hour of birth (n=111)

87.1 57.1 80.0 74.8 41.0 39.1

Baseline data 69.8 62.5 74.3 69.8

17 % infants 6-23 months old who were exclusively breastfed for first 6 months (n = 201)

25.6 50.8 26.3 33.8 41.8 40.4

Baseline data 58.7 65.2 42.1 52.7

18 Proportion of infant of 6-9 months receiving breast-milk and complementary food (n=201)

95.3 88.9 92.6 92.0 81.9 ±90

Baseline data 90.9 85.7 84.4 85.9

19 % children 6–23 months of age who receive foods from 4 or more food groups (n=201)

11.6 26.9 21.1 20.9 43.8 35

103

Baseline data 36.5 10.8 18.6 20.6

20 % breastfed and non-breastfed children 6–23 months old who receive complimentary foods the minimum number of times (n=201)

6.9 3.2 7.4 5.9 40.4 43

Baseline data 78.3 73.3 82.9 78.9

21 Proportion of children 6–23 months of age who receive a minimum acceptable diet (apart from breast milk) (n=201).

20.9 36.5 20 25.4

Baseline data 24.2 9.9 12.9 14.8

22 % children whose feces is disposed of appropriately (n=201)

34.9 44.4 26.3 34.5 15.6 18.6

Baseline data 31.2 26.8 26.9 27.9

23 % children 12-23 months old immunized against measles (at 1 year old) (n=111)

100 94.6 93.8 95.5 53.2 63.7

Baseline data 73.0 67.9 67.9 69.2

24 % children 12-23 months old who have received DTP3-Hib-HepB3 (by 1 year old)

100 97.3 95.8 97.3 37.6 55.5

Baseline data 82.4 70.9 52.6 65.2

25 % children who received a dose of Vitamin A in the previous 6 months (n=111)

96.1 54.1 58.3 65.8 73.3 59.1

Baseline data 60.9 85.8 66.3 70.7

Knowledge on MCH (n = 111)

26 % of women who know two critical signs of PNC for seeking care at health facility

51.6 37.1 33.3 39.6

Baseline data na na na na

27 % of women who know ≥ 3 critical signs of PNC for seeking care at health facility

6.5 0 40.0 18.0

Baseline data na na na na

28 % of women who know two critical signs of labor course for seeking care at health facility

9.7 20.0 22.2 18.0

Baseline data na na na na

104

ANNEX III: DOCUMENTS GATHERED AND REVIEWED

Plan’s Key Documents

Plan Laos Country Strategic Plan (CSP) 2014-2019

What is Child Centred Community Development?

Child Centred Community Development: Operational Standards

Child Protection Risk Assessment Form

MNCH and WASH MoUs : Provincial Health Department, Bokeo Province and Plan International ( two separate MOUs)

WASH Project Design Summary 2012 – 2016

Plan Laos WASH Project Final Evaluation – Revised Draft Report August 2012

Supporting the Poor to Access Sanitation in Bokeo Province, LAOS

Supply Chain Analysis for Rural Sanitation Products and Services in Lao PDR

Maternal, Newborn and Child Health (MNCH) plus Nutrition Program 2012

Plan Laos Maternal Newborn & Child Health Project Concept Note 2011

Plan International Laos : Maternal Newborn and Child Health and Nutrition Project - Bokeo

Baseline data in project villages of Bokeo province

Quarterly and annual narrative progress reports of WASH and MNCH-N FY12-FY15

Sector Documents

Success Factors for Women’s and Children’s Health Lao PDR 2014National MNCH

Food and Nutrition Security Action Plan Lao PDR strategy 2009-15

National MNCH strategy evaluation report (draft) 2014

National Strategy For Rural Water Supply Sanitation and Hygiene, Draft v.3- March 2011

National Plan of Action for Rural Water Supply, Sanitation and Hygiene, 2012

Regional CLTS Review 2012 – UNICEF, Plan and Water Aid: Laos Country Overview

WSP : Lao People's Democratic Republic Strengthening Water Supply, Sanitation and Hygiene Sector Coordination in Lao PDR Supporting Sector Reform for Scaling Up Rural Sanitation – Synthesis Report 22 May 2015

WSP : Progress on Investing in the Next Generation : Children grow taller, and smarter, in rural villages of Lao PDR where all community members use improved sanitation

Progress on Sanitation and Drinking Water, UNICEF & WHO (Joint Monitoring Program), 2015 Updates

Testing CLTS for scalability: CLTS learning series, Lao PDR, Plan USA and NUC, Sepember 2014

Ministry of Health (2012), National Health Sector Reform Strategy: 2013-2025, Vientiane: Ministry of Health.

Ministry of Planning and Investment (2011),

105

The Seventh Five-year National Socio-Economic Development Plan (2011-2015),

EAP CLTS review (UNICEF, Plan and Water AIDs) September, 2012.

National Health Sector Reform Strategy: 2013-2025, Vientiane: Ministry of Health (2012),

Lindelow et al, World Bank (2011), “Government Spending on Health in Lao PDR: Evidence and Issues,” World Bank, Washington, DC.

Lao Social Indicator Survey 2012.

CARE International, Dak Cheung Food Security Project Sekong Province. Participatory Diagnostic on nutritional understandings and practices associated with food, infant weaning and feeding and associated health issues, August 19, 2011

UNFPA Assessment 2014

Evaluation of Lao pdr’s strategy and planning framework for integrated package of maternal neonatal and child health services 2009-2015

106

ANNEX IV: DOCUMENTS GATHERED AND REVIEWED

Name Title Organisation Village/Place

Hilda Winartasaputra Regional WASH Specialist Plan International Asia Regional Office

Hanoi, Vietnam

Dr. Kalana Peiris Public Health adviser Plan Vientiane

Hilda Winartasaputra Regional WASH Specialist Plan Vientiane

Dr. Ketsadasak Kiettisak WASH National Program manager Plan Vientiane

Dr Boualaphanh Inthaxay MNCH-N National Program manager Plan Vientiane

Mr Thipphavanh Malaithong act.WASH National Program manager Plan Vientiane

Viengkhone Volavong CR, CP and Gender officer Plan Vientiane

Mr Somexay Inthasone BEP and ECCD Program Manager Plan Vientiane

Dr Amphone Keooudom MNCH-N provincial Program manager Plan Bokeo

Ms Nali Sengthavong WASH provincial Program manager Plan Bokeo

Ms Sounisa Ounalom CR, CP and Gender Coordinator Plan Bokeo

Dr. Bounthom Phengdy Director of Nutrition Center Nutrition Center, MOH

Vientiane

Mr. Sengsay Improvement of Health System UNFPA Vientiane

Ms Oulayvanh Improvement of Health System UNFPA Vientiane

Mr. Bandit Leuanvilay Project officer water, sanitation and hygiene section

UNICEF Vientiane

Technical staff Department for Women Development LWU Vientiane

Ms Souvannaly a technical staff for environmental health unit WHO Vientiane

Dr Keokham Silikham DDG

PHO

Bokeo

Dr Veokham Head of Mother and Child Bokeo

Dr Somsak Head of Water Supply and Environmenal Health

Bokeo

Mr Joker Primary education Division PES

Mr Bounmy Manilay Head of HC, Kengphak HC Paktha

Mr Ken Head of HC, Hardsa HC

107

Dr Sivisay Head of DHO

DHO Dr Manith Saline Deputy Head /Coordinaor

Ms Chansamon MNCH

Mr Khanthong Phonechaleun DWT/Education coordinator DEO

Mr Sasakchit Khounphong DWT, education

Mr Houmpheng Head of village

Houaythong village

Mr Amnaouy Deputy Head of village

Ms Chansouk VHC

Mr Xiengphanh VHC

Paktha

Mr Xiengloth CLTS

Mr Bounlord CLTS, Head

Mrs Khamman CLTS, washing hand with soap

Mr Boualanh Environmental cleanliness

Mr Nath Sanitation

Ms Peun LWU, Intern of Plan

Mr Xiengthongphanh Deputy Head of village

Houaynorkhom Mr Sik Teacher

Mr Chanthaly Head of village cluser

Dr Khamchanh Deputy Head of DHO/Coordinaor

Pha Oudom

Mr Khamphan Head of HC, Hardnam

MsThongkhan MNCH

Ms Bounam Nurse, Hardnam HC

Mr.Bounkong DWT

Mr Sensouksavath DWT/Health Coordinator

Mr Khamseuk DWT/Deputy Head of DEO

Mr Phetphouthian Head of HC, Houanamkha

Mr.Bounkong DWT

Mr Thongkhan MNCH

108

23 Participants (16 women) VHC, PPG members, villagers Chomxay Village

50 participants (18 women) VHC, PPG members, villagers Houaysang

10 Participants (16 women) VHC, PPG members, villagers Houaykoun

Mr Alor Head of Village

Monelaem Village

Meung District

Mr Bounkeo Deputy Head of Village

Mr Khamphet CLTS

Mr Asang CLTS

Mr Chantah Head of Village Phousavang Village

8 Women Beneficiaries Phousavang Village

5 men Beneficiaries Phousavang Village

109

ANNEX V-A: HH SURVEY

WASH and MNCH-N Evaluation Questionnaires July-August 2015

Interviewer Name: Date:

Supervisor:

District: Khum Ban:

Village: Health Facility:

Interviewed HH No: (1 - 20)

Sections Completed:

Water supply, Sanitation and Hygiene

110

Woman 15-49

Child <1

Child 6 - 23 months

Child 12 - 23 months

Water Supply

No Question Response Code

1 What is the source of your drinking water? (select one)

Piped into dwelling 1

Public taps 2

Boreholes 3

Protective well 4

Un-protected well 5

Rain water 6

River 7

spring 8

1.1 others........................................................

2 Do you treat your drink water? Yes 1

No, continue Q4 2

3 How do you treat water? (select one answer)

Boiling 1

Chlorine 2

Strain it through a cloth 3

Use a water filter 4

Treatment with sun light 5

Leave it for sometime 6

4

Do you wash hand with soap? Yes 1

Sometimes 2

111

No 3

5 Do you have soap at your house now? Yes 1

No 2

6 what are the key causes of diarrhea

poor hygiene practices 1

washing hand without soap 2

eating food without properly cooked 3

drink unboiled water 4

OD practice 5

7

Did any of your household member experience diarrhea last week?

Yes 1

no 2

8

How do you rank the quality of water that you are using?

Very good 1

Good 2

Poor 3

Very poor 4

9

How do you rank the quality of water from the system?

Very good 1

Good 2

Poor 3

Very poor 4

10

How many times has the water system broken down since it was constructed? ............times........litres

11

What do you use the water collected from the

drinking 1

cooking 2

Bathing 3

using in the toilet 4

washing hand 5

gardening 6

for animals 7

12

Who is responsible for fetching water in the household? (If connected to the house skip here)

Men 1

Women 2

Boys 3

Girls 4

13

Is the water enough to meet your need? Yes 1

No 2

14

Is the water collected from the water system enough to meet your need?

Yes 1

112

No 2

15

Are there any problems with water supply from the system?

No 1

yes, broken .................time

16

Do you know who the water committee manner in your village are?

Yes 1

NO 2

17

If yes, mention one or two name:

1 ………………………………………..

2 ………………………………………..

18

Have you paid regular fee to the water committee?

Regularly 1

Sometimes 2

Never 3

19

Have you been invited to any meetings concerning water by water committee?

Never 1

Yes 2

19.1 If yes, what is the purpose of the meeting? ...................................................................................

19.2

If yes, who participate? Husband 1

Wife 2

20

Are there any women in the water committee?

yes 1

No 2

20.1

Yes, if yes what are their role? ...................................................................................

21

Are there any problems with water supply from the system?

Yes 1

No 2

21.1

What can be done to improve? ...................................................................................

22

Is there enough water in the school in your village?

yes 1

no 2

22.1

What is the water used for?

drinking, washing hand and using in the toilet

1

washing hand and using in the toilet 2

using in the toilet only 3

113

Water Supply

No Question Response Code

1 What is the source of your drinking water? (select one)

Piped into dwelling 1

Public taps 2

Boreholes 3

Protective well 4

Un-protected well 5

Rain water 6

River 7

spring 8

1.1 others........................................................

2 Do you treat your drink water? Yes 1

No, continue Q4 2

3 How do you treat water? (select one answer)

Boiling 1

Chlorine 2

Strain it through a cloth 3

Use a water filter 4

Treatment with sun light 5

Leave it for sometime 6

4

Do you wash hand with soap? Yes 1

Sometimes 2

No 3

5 Do you have soap at your house now? Yes 1

No 2

6 what are the key causes of diarrhea

poor hygiene practices 1

washing hand without soap 2

eating food without properly cooked

3

drink unboiled water 4

OD practice 5

7

Did any of your household member experience diarrhea last week?

Yes 1

no 2

8

How do you rank the quality of water that you are using?

Very good 1

Good 2

114

Poor 3

Very poor 4

9

How do you rank the quality of water from the system?

Very good 1

Good 2

Poor 3

Very poor 4

10

How many times has the water system broken down since it was constructed? ............times........litres

11

What do you use the water collected from the

drinking 1

cooking 2

Bathing 3

using in the toilet 4

washing hand 5

gardening 6

for animals 7

12

Who is responsible for fetching water in the household? (If connected to the house skip here)

Men 1

Women 2

Boys 3

Girls 4

13

Is the water enough to meet your need? Yes 1

No 2

14

Is the water collected from the water system enough to meet your need?

Yes 1

No 2

15

Are there any problems with water supply from the system?

No 1

yes, broken .................time

16

Do you know who the water committee manner in your village are?

Yes 1

NO 2

17

If yes, mention one or two name:

1 ………………………………………..

2 ………………………………………..

18

Have you paid regular fee to the water committee?

Regularly 1

Sometimes 2

Never 3

19

Have you been invited to any meetings concerning water by water committee?

Never 1

115

Yes 2

19.1 If yes, what is the purpose of the meeting? ....................................................................................

.............

19.2

If yes, who participate? Husband 1

Wife 2

20

Are there any women in the water committee?

yes 1

No 2

20.1 Yes, if yes what are their role?

.................................................................................................

21

Are there any problems with water supply from the system?

Yes 1

No 2

21.1

What can be done to improve? ....................................................................................

.............

22

Is there enough water in the school in your village?

yes 1

no 2

22.1

What is the water used for?

drinking, washing hand and using in the toilet

1

washing hand and using in the toilet

2

using in the toilet only 3

Hygiene Promotion

No Question Response Code

1

Have you ever participated in or seen or heard of importance of hygiene practices (using toilets/stop OD, hand washing with soap and drinking safe water?

Yes 1

No, Q4 2

2

If Yes are they relevant to the following (choose all relevant)

Poster 1

Listening to education from health workers/committee,

2

Triggering session, 3

Visit by CLTS committees, 4

Campaign event, 5

Your children 6

3 Yes 1

116

Do you find them easy to understand/attractive and interesting

No 2

4 Do you wash hand routinely?

Yes 1

No 2

5

19. Do you a place for family member to wash hand

Yes 1

No 2

6

Are there the followings in the place?

Only water 1

Water and soap 2

Towel 3

Nothing 4

7

is there a handwashing station in the school in your village?

Yes 1

No 2

8

Are there the followings in the handwashing station?

Only water 1

Water and soap 2

Nothing 3

Towel 4

9 When you do think you should wash hand with soap (check all relevant)

After using toilet with soap without soap

before eating with soap without soap

before cooking with soap without soap

after cleaning child feiece

with soap without soap

before feeding a child

with soap without soap

none of the above

Women 15-49 Years

No Question Response Code

1 What is your date of birth

2 So how old are you now?

3 Which ethnic group do you belong to? (Circle only one)

Lao Loum 1

Lao Lue 2

Lao Pouthai 3

Muser 4

Hmong 5

Kuy 6

Khmu 7

Lamed 8

Akha 9

3.1 Other (Specify)

.......................................................

117

4 What is the highest level of education you have attained? (Circle only one)

None 1

Some primary 2

Completed primary 3

Some secondary 4

Completed secondary 5

Higher 6

5 What is your marital status?

Not married -> Q7 1

Married 2

Divorced /Widowed 3

Other 4

6 What age were you when you got married? (Circle only one)

<15 1

15 - 17 2

18 - 20 3

21 - 25 4

> 25 5

7 Do you have any children? Yes 1

No -> 10 2

8 What age were you when your first child was born?

..................

9 When your last child was born did you plan or want to have a child - at that time?

Yes 1

No 2

10 Are you pregnant now?

Yes 1

No --> Q 12 2

Don't know --> Q 12 3

11 Did you plan / want to get pregnant?

Yes --> Q 12 1

No --> Q 15 2

Don't know --> Q 15 3

12 Do you want to have a / another child?

Yes 1

No --> Q 13 2

Don't know --> Q13 3

13 When do you want to have this child?

Now 1

Anytime 2

In next 12 months 3

In next 2 years 4

More than 2 years 5

Don't know 6

118

14 Are you or your partner currently using any form of contraception?

Yes 1

No 2

15 How far away from your house is the nearest health clinic / hospital? in Km ..............

16 How do you get there usually? (Circle only ONE answer)

Walk 1

Bicycle 2

Tractor 3

Motorcycle 4

Car / Bus 5

Boat 6

Other 7

17 How long does it take to get there? ..............min..........hr

18 Sometimes children have severe illnesses and should be taken to a health clinic immediately. What symptoms would make you take a child to the clinic urgently? (Circle all that apply)

Not able to drink or breastfeed

1

Child getting sicker each day 2

Child develops a high fever 3

Child has fast breathing 4

Child has difficult breathing 5

Child has blood in stool 6

Child is drinking poorly 7

Child has a cough 8

18.1 others....................................................

19 Do you have any children under 2 years old? Yes 1

No --> FINISH 2

20 What age is/are your child(ren) last three

Yrs Months

Yrs Months

Yrs Months

119

Based on the answer to Q 20, note which of the following questionnaires should be used:

Woman with a child <1 year (0-11 months)

Woman with a child 6-23 months

Woman with child 12 - 23 months

Child <1 Yr (0-11 months)

No Question Response Code

1 How old is your child? Month

2 During your pregnancy with this child, did you see anyone for antenatal care?

Yes 1

No, Q 6 2

3 How many times did you see someone?

4 During these visits was your blood pressure taken? Yes 1

No 2

5 During these visits were you weighed? Yes 1

No 2

6 During your last pregnancy did you receive any injections in the arm to prevent the baby getting tetanus (fits after birth)?

Yes 1

No > Q 8 2

7 How many did you receive? (max 5) ..............

8 Did you receive any injections before this last pregnancy or during previous pregnancies?(max 5)

Yes 1

No, > Q12 2

9 How many have you received in total?

10 In what year did you receive the first one?

11 In what year did you receive the last one?

12 During your last pregnancy, did you receive /take any iron tablets/syrup? (Show a sample of tablet / syrup)

Yes 1

No 2

13 How many did you take?

14 Do you know any signs or symptoms during pregnancy that should make a woman go immediately to the clinic / hospital? (Circle all that apply)

Convulsions (fits) 1

Pale eyelids 2

Vaginal bleeding 3

Difficulty breathing 4

Severe headache 5

120

Swollen hands/legs/ face 6

High fever 7

Severe abdominal pain 8

Other 9

Don't know 10

15 Where was your last child delivered? (circle only one)

At home 1

At relative / friend’s home 2

In forest 3

At TBA home 4

Health clinic 5

Hospital 6

Maternity clinic 7

Other 8

16

What are the danger signs during labor / delivery that would make you go for health care immediately? (Circle all that apply)

Convulsions (fits) 1

High fever 2

Heavy vaginal bleeding 3

Offensive discharge 4

Probe: Anything else?

Labor longer than 12 hours 5

Malpresentation (feet or arm first)

6

Placenta not delivered within 30 minutes

7

Other 8

Don't know 9

17 Who assisted at the delivery of your last child? (Circle only one)

Doctor 1

Nurse/Midwife 2

Village health volunteer 3

TBA 4

Relative /friend 5

No-one 6

18 Was your child weight at birth? Yes 1

No 2

19 Was he/she weighed soon after birth? (in case at home)?

Yes 1

No 2

20 How long after birth was the child weighed?

1 day 1

2-4 days 2

4-7 days 3

More than 7 days 4

Don't remember 5

121

21 When the child was born, did you breastfeed him/her? Yes 1

No 2

22 How long after the baby was born did you first put him/her to the breast?

Immediately 1

Within 1 hour 2

Between 1 - 4 hours 3

After 4 hours 4

23 After delivery of your last child, did you see or were you visited by anyone to check on your health? (<42 days)

Yes 1

No, > 26 2

24 How long after delivery did this visit take place? Days Weeks

25 How many times did you see someone after the birth?

26

What are the danger signs after delivery that would make a woman go for health care immediately?(Circle all that apply)

Convulsions (fits) 1

High fever 2

Heavy vaginal bleeding 3

Offensive discharge 4

Probe: Anything else?

Severe headache 5

Severe abdominal pain 6

Other 7

Don't know 8

27 How would you rate the quality of care you received – both ANC and PNC? (Circle only one)

Very good 1

Good 2

Neither good nor bad 3

Bad 4

Very bad 5

Didn’t get any 6

NOW MEASURE THE CHILD

Child's Weight

gms

Child's Height

cms

Name (measure once only)

Child's Age

Date………./………./20…….

Sex

122

Child 6-23 months

No Question Response Code

1 How old is your child? months

2 At what age did you give your last child anything to eat or drink, other than breast milk? (This includes water or other fluids)

<1 month 1

2-6 months 2

After 6 months 3

3 Are you still breastfeeding your child? Yes 1

No 2

4 Are you giving foods other than breastmilk? Yes 1

No 2

5 What type of foods do you give your child? (Circle all that apply)

Grains, roots and tubers 1

Legumes and nuts 2

Dairy products (milk, yogurt, cheese) 3

Flesh foods (meat, fish, poultry liver/offal) 4

Eggs 5

Vitamin-A rich fruits and vegetables 6

Other fruits and vegetables 7

5.1 others ...............................................

6 How often do you give your child some food during the day?

2 times 1

3 times 2

4 times or more 3

7 What happens with the stools of young children when they do not use the latrine or toilet facility?

Children always use toilet or latrine 1

Thrown into toilet or latrine 2

Thrown outside the yard 3

Buried in the yard 4

Not disposed of or left on the ground 5

Don't know 6

123

NOW MEASURE THE CHILD

Child's Weight gms

Child's Height cms

Name (measure once only)

Child's Age Date………./………./20…….

Sex

124

Child 12-23 Months

No Question Response Code

1 How old is your child? Months .............

2 Do you have an immunisation record card for your last child? (If yes, ask to see the card)

Yes 1

No 2

3 Card seen? (If card available, check Q 6 from the card)

Yes 1

No 2

4 Has your last child been immunised against measles?

Yes 1

No 2

Don't know 3

5 Has your child received vaccinations to protect against DPT/HepB/HiB?

Yes 1

No 2

Don't know 3

6 How many times has he/she been vaccinated? ...................

7 In the last 6 months, has your child received a dose of Vitamin A? (Show a sample of capsule)

Yes 1

No 2

Don't Know 3

NOW MEASURE THE CHILD

Child's eight

gms

Child's Height

cms

Name measure once only)

Child's Age

Date………./………./20…….

Sex

125

ANNEX V-B: INTERVIEW GUIDES

1) KII guide for central level stakeholders

Introduction:

Good morning or good afternoon!

Thank you very much for meeting with us today. I am Khampheng Phonluxa, a independent consultant for Plan international, we conduct an "Evaluation of the MNCH-N and WASH implementation in Bokeo province" This Program is implementing by Plan international in close collaboration with PHD and DHO in Bokeo province. We are here to learn from you so that we can do our work better for Bokeo province and for other provinces in future. We would like to understand more about the MNCH-N and WASH implementation. We appreciate your time in helping us so we can work better together. We have a series of questions to ask you. Some of the questions may be sensitive but we would like you to answer as honestly and openly as you can. I think we can learn a lot from each other!” All information given will be confidential. The interview will take place maximum 60 minutes.

Notes: this guide is flexible depends on the role and function of organization

A. General information concerning interview and interviewee

Date of interview:

Interviewee’s name:

Organization to be interviewed:

Sex (M/F):

Time to start the interview:

Current Position:

Time to finish the interview:

Year of current position:

B. MNCH-N and WASH Programs implementation

1. Could you please tell us about the role and function of your institution in relation to MNCH-N and WASH activities?

a. What is the key activity of your organization? b. What is your responsibility in relation to MNCH-N and WASH?

After the interviewer explains about what is Plan doing to those who do not know about Plan in Lao PDR, please continue to question about:

126

2. What do you think about the Plans’ Programs implementing in Bokeo? 3. Are the activities of Plan [MNCH-N and WASH] meeting the need of the health sector

of Lao PDR? a. If yes why? b. If no, why? Which points we need to focus on according to your opinion?

4. Do you think that the Program has contributed to significant impacts to mother and child health and nutrition and WASH in Lao PDR? What impacts are there?

a. As now water and Sanitation MDG target Lao PDR has already achieved, do you think that Plan support has contributed to this?

5. How can we keep the Program activities implemented in Bokeo to be sustainable once the Program is ended?

a. MNCH b. WASH

6. For example Plan has supported SBA but now they are not yet recruited into the government health work force, what do you think can be done about this?

7. In sanitation Plan has supported CLTS what do you think the government can do to support the sustainability of ODF and to support the scalability?

8. Concerning to the small-scale private provider and entrepreneur what has the government done or will do to engage them in WASH improvement?

9. What are your recommendations to be the capacity of village committee like water committee, education committee, CLTS committee?

10. Plan also support the school WASH what do you think the government should do to sustain and scale up the results?

11. What are the potential challenges to sustain and scale up the results of Program? a. How to solve them?

12. What do you think if we replicate the lessons learnt from Bokeo in other provinces like in Oudomxay and Saravane province?

a. What additional activities need to be added? b. What activities need to be removed? c. What suggestion would you made for this scaling up?

13. What do you think if we integrate MNCH-N and WASH activities together? a) Is it reasonable to do that? And How to integrate them? b) Is there any potential synergetic outcomes expected? What are there? c) What potential challenges and barriers would be faced? d) Any comments and suggestion in relation to the integration of MNCH and

WASH? 14. Plan promote inclusion in the project activities by targeting people with disability,

ethnic minority group, women and children as beneficiaries, do you think that this is aligned with government policy? How has your organization translated this policy into implementation?

15. How can Plan strengthen the focus on inclusion in the future through the WASH and MNCH-N Programs especially through the government mechanism in order to sustain and scale up?

THANK YOU FOR YOUR VALUABLE TIME

127

2) KII guide for local level stakeholders (PHO, DHO, DEO, HC)

Introduction:

Good morning or good afternoon!

Thank you very much for meeting with us today. I am Khampheng Phonluxa, a independent consultant for Plan international, we conduct an "Evaluation of the MNCH-N and WASH implementation in Bokeo province" This Program is implementing by Plan international in close collaboration with PHD and DHO in Bokeo province. We are here to learn from you so that we can do our work better for Bokeo province and for other provinces in future. We would like to understand more about the MNCH-N and WASH implementation. We appreciate your time in helping us so we can work better together. We have a series of questions to ask you. Some of the questions may be sensitive but we would like you to answer as honestly and openly as you can. I think we can learn a lot from each other!” All information given will be confidential. The interview will take place maximum 60 minutes.

Notes: this guide is flexible depends on the role and function of organization

C. General information concerning interview and interviewee

Date of interview:

Interviewee’s name:

Organization to be interviewed:

Sex (M/F):

Time to start the interview:

Current Position:

Time to finish the interview:

Year of current position:

D. MNCH-N and WASH Programs implementation

16. Could you please tell us about the role and function of your office/institution in relation to MNCH-N and WASH activities?

a. What is the key activity of your organization? b. What are the key activities cooperated with Plan?

17. What do you think about the Plans’ Programs implementing in your district?

128

18. Are the activities of Plan [MNCH-N and WASH] meeting the need of the health sector of your district?

a. If yes why? b. If no, why? Which points we need to focus on according to your opinion?

19. Do you think that the Program has contributed to significant impacts to mother and child health and nutrition?

20. 21. What is your responsibility in relation to MNCH-N (focus on achievement and

challenges since 2012) a) Tell us about MB clinics? b) Tell us about Micro-planning for MCH integrated service c) How about MHV d) MHV links with behavior change communication on nutrition

and health outcomes e) SBA scholarships f) Information and data management

22. How can we keep the Program activities implemented in the district to be sustainable once the Program is ended?

a. MNCH b. WASH

23. For example Plan has supported SBA but now they are not yet recruited into the government health work force, what do you think can be done about this?

24. In sanitation Plan has supported CLTS what do you think the government can do to support the sustainability of ODF and to support the scalability?

25. Concerning to the small-scale private provider and entrepreneur what has the government done or will do to engage them in WASH improvement?

26. What are your recommendations to be the capacity of village committee like water committee, education committee, CLTS committee?

27. Plan also support the school WASH what do you think the government should do to sustain and scale up the results?

28. What are the potential challenges to sustain and scale up the results of Program? a. How to solve them?

29. What do you think if we replicate the lessons learnt from Bokeo in other provinces like in Oudomxay and Saravane province?

a. What additional activities need to be added? b. What activities need to be removed? c. What suggestion would you made for this scaling up?

30. What do you think if we integrate MNCH-N and WASH activities together? a) Is it reasonable to do that? And How to integrate them? b) Is there any potential synergetic outcomes expected? What are there? c) What potential challenges and barriers would be faced? d) Any comments and suggestion in relation to the integration of MNCH and

WASH? 31. Plan promote inclusion in the project activities by targeting people with disability,

ethnic minority group, women and children as beneficiaries, do you think that this is

129

aligned with government policy? How has your organization translated this policy into implementation?

32. How can Plan strengthen the focus on inclusion in the future through the WASH and MNCH-N Programs especially through the government mechanism in order to sustain and scale up?

THANK YOU FOR YOUR VALUABLE TIME

3) Interview: Local Authorities/CLTS/ VHC – Village level

Efforts will be made to invite members of VEDC, CLTS Committees, SBAs and Water Users Committees to one location.

Depending on the situation, one group can be requested to focus two topics first and then each will take turn to add to the feedback on each topic.

Name of interviewer:

Date:

Village, Name, Position:

Introduction: We are here to review the WASH and MNCH projects supported by PLAN LAOS from 2012. We want to use the information to improve PLAN projects in the future, so your comments and opinions are very important to us. Please feel free to talk freely. Your comments will be recorded, but we will not quote you in our report. We will share a summary of our final report with you later.

1. General information

1.1. What are the most common health problems in the area?

1.2. Are there any water and sanitation problems?

1.3 Are there many diarrhea cases among children under-five ?

1.4 are there many other diseases than diarrhea ?

1.4 Are there many malnourished children ?

2. General information

2.1. Please tell us about the project activities. What was your role?

2.2. Have any other activities concerning MNCH-vaccination, vit A supplement, de-worming, ANC, delivery care, PNC, Nutrition, sanitation, hygiene, or drinking water been implemented in the village?

2.3. Were community members including children, women and PWD able to participate in project activities? if yes could you give examples, if no why ?

130

2.4 What results do you think the project intervention has brought to your village?

2.5. Have you seen any change in the health conditions of children since 2011/2012? What are they = how about nutrition?

2.6 What have gone well in the Project Implementation – and why

2.7 What have not gone well and why?

2.8. Do you have any recommendations to improve Plan support?

3. Drinking Water Supply

3.1 What is the situation of drinking water supply now? Have you seen any improvement since 2011/2012? If yes what are these improvements? If no why

3.2 How active is the Water Committee? Have they been trained? Have they applied what they have been trained for? Can you explain z?

3.3. Do you think women have played more important role in the Committee now? If yes can you give an example; if no why?

3.4 If the system is functioning, do you think the water supply system now will continue to function? If yes for how long; if no why and what should be done to make it continue to function

3.5 Do you think HHs have enough water for drinking, cooking, washing and using toilet?

If no, what should be done to enable this?

3.6 Do you think the poorest, the people with disability and the ethnic minority group have benefit from the water supply improvement? If yes can you explain how this has happened; if no why and what should be done?

4. CLTS

4.1 What changes have you seen in the sanitation situation in this village since 2011? OD practice, using toilet, using toilet for small children, keeping yard clean, keeping animals at pens?

4.2 If this village is ODF – how long has it been ODF? What have people done to keep the ODF status?

4.3 If this village is not ODF – when was it triggered? When do you think it will reach ODF? What are the constraints to date? What have been done to address the constraints?

4.4 are there local masons and local materials suppliers who are active in marketing their services and products – making it easy for people who want to build or upgrade toilets? If yes how and if no why

4.5 How often do you have contact with the DWT? When was their last visit? With Plan staff? When was their last visit?

5. SLTS

5.1 What changes have you seen in the school environment particularly availability of water, toilets and hand washing facilities? Have you seen improvement? If yes what kind of improvements and if no, why and how to attain improvement?

131

5.2 can you tell what went well during the project implementation? And why?

5.3 can you also tell what did not go well during the project implementation? And why?

5.4 can you share recommendations to make SLTS more effective (schools have water, toilet and hand washing facilities that are used by all children including children with disability? children participate in student clubs that are able to conduct hygiene activities?

6. Hygiene promotion

6.1. Do you think people (men, women, boys and girls) practice HWWS? If yes, how this has happened? If no why

6.2 How do children learn about hygiene? (Family, peers, school, health workers, other)

6.3. How are children educated about hygiene in schools? (In what classes, how often,)

6.4. What role do you play in hygiene promotion?

6.5. Do you have any recommendations to make hygiene promotion more effective?

7. MNCH

7.1. Do you think with Plan’s support the needs for vaccination of children U5, reproductive age women (TT vaccine), pregnant women (anti-tetanus) have been met?

7.2 do you think with Plan’s support women in this village have been able to visit health centres for ANC, delivery and PNC? If yes can you give examples? How many times women go for ANC visit? If no, why

7.3. Do many women in this village deliver at home? If yes, why they deliver at home? Who help them?

7.4 In the last three years, were there any children under five who died in this village? If yes, what were the reasons?

7.5 in the last three years, were there any mothers who died during pregnancy or after delivery in this village? If yes, what were the reasons?

7.6. Do you think the SBA has sufficient capacity to perform her tasks? If yes can you give examples; if no why?

7.7. Do you think the SBA has sufficient medical equipment to perform their tasks? If yes, can you examples and if no why

7.8 is there a parenting education session in this village about mother and child health and nutrition? If yes how useful the sessions are? If not why and how to make them useful

7.8 If there is no more Plan support do you think the services for women and children will continue? If yes, how. If no, why and what should be done

7. Wrap up

7.1. What other recommendations do you have for improving the water, sanitation, and hygiene situation in your village?

7.2. How could you get people interested in improving their sanitation and hygiene?

132

7.3 How could you get local private entrepreneurs to be interested in providing services for water and sanitation?

4) FGD guides for Women, Men, and Children

FGD for Women and Men

Village Name:

Note to facilitators: Focus group discussions should have 6-10 women.

When asking about perceptions on over-all changes, provide a time reference (before the project, before activities in the villages, etc.)

Introduction:

“Thank you very much for meeting with us today. I am ______and this is _________and we are here to learn from you so that we can do our work better for you and your family. We would like to understand more about how you use water and toilet facilities and keep the environment around you clean. We appreciate your time in helping us so we can work better together. We have a series of questions to ask you. Some of the questions may be sensitive but we would like you to answer as honestly and openly as you can. I think we can learn a lot from each other!”

A. Water use and sanitation (drawing map and discussion)

Water source

Storage

Treatment

Type of Use

Discussion about the map in relation to knowledge practice and facilities

If water system exist, also discuss:

Availability

Management/use/maintenance

Waste water management

B. Toilets and toilet use (discussion and bullet pointed on flipchart by one member)

133

1. Have a toilet at home:

a. since when

b. How it came about

c. What benefits

d. What cost

2. No toilet, plan to have? Why and why not?

3. Who take care of the toilet? Clean, maintain water? Any issue on keeping it in use?

4. How do they clean their bottom with or without toilet?

5. Overall, what has changed in relation to toilets and toilet use since 2012 (use relative time reference)

a. Personal hygiene

b. Health benefits

c. General perception on toilet use

C. Hygiene Promotion (group discussion and bullet pointed on flipchart)

1. How hygiene promotion information reach you

2. What materials/methods

3. Who provided

4. When how often

5. Understanding of the content: language, picture, visuals

6. Useful? What ways? What you want to see improved?

7. Overall, what has changed since 2012 or a few year back why? Or why not? In relation to

a. household hygiene knowledge and awareness (elders, adults, children)

b. your own hygiene and practice

c. perceived health benefits

D. Hand washing

1. Do you have a place to wash your hands at home?

a. If yes,

i. Do you use soap?

ii. What kind of soap do you use?

134

iii. Is soap available in your house at all times?

If no, where do you get soap?

iv. If you don’t use soap, why not?

b. If no,

i. Where and how do you wash your hands?

ii. Do you use soap?

iii. What kind of soap do you use?

iv. Is soap available in your house at all times?

If no, where do you get soap?

v. If you don’t use soap, why not?

2. During what times of the day do you wash your hands?

Check here if with soap

Where hand-washing is performed

a.

b.

c.

d.

e.

3. Over-all, what do you think has changed in your and your family’s hand-washing practices?

a. On frequency of hand-washing

b. On hand-washing during critical times

c. On perceived health benefits

E. On participation in project community activities

Have you participated in community activities conducted by the project? (examples: community planning meetings; community awareness raising or learning sessions; community monitoring; other project activities at communities)

1. If yes,

a. Can you recall the type or nature of these activities and describe them?

135

b. Do you think enough (at least half) of the participants in the activities were women?

i. If yes, did women attend voluntarily (without being strongly encouraged to attend by village leaders)?

ii. If no, why did no or few women attend the activities?

c. Can you describe how you participated in these community activities?

d. During the activities, were you given the opportunity to voice out your views/ concerns? How?

2. If no,

a. Why did you not attend/ participate in these activities?

b. Did you want to attend but could not?

c. If yes, what were the reasons that prevented you from participating?

3. What was your role and responsibilities in the project?

a. What support or training provided by Plan/partners?

b. How did this support/training help you in the implementation of project?

4. What were the vulnerable groups in your area? (women, ethnic groups, children, very poor, trafficked women and children, child labor, etc. based on local nuances) before the project started?

a. Were they specifically targeted as beneficiary of the project?

b. How?

c. What was their roles and responsibility in the different components of the project? (probe on a per component basis)

MNCH-N

1. Do you think the vaccination services provided by the project met your needs? 2. Do you use health facilities for ANC, delivery, PNC? Why? 3. Where do women in your village give birth? Why do they give birth there? Has there

been any changes to choice they choose to give birth? 4. When women give birth at home who help them? Are there any SBAs to help? Do they

have adequate skills? 5. Over the past three years, has there been any maternal death during their pregnancy,

child delivery, or post-delivery? 6. Did you use health care services in health facilities near your villages? Are you satisfied

with the services you received? Why? 7. Over the past three years, has there been any child under five years of age die? 8. Have you participated in the outreach activities, health day events? If so do you think

these events are useful? What do you want to see improved?

136

F. Factors influencing the results/changes

5. Who are the actors in the WASH enabling environment? Whose performance has improved (individuals, groups, organizations, institutions), and what is the nature of the improvement? Why has their performance improved?

6. Is the water system functioning well at project evaluation? 7. What motivated individuals to improve their hygiene behaviour? 8. What motivated individuals to improve their hygiene behaviour? 9. Number of additional students participating in school hygiene behaviour change

Programs? 10. Is there villages which have slipped after one year of becoming ODF and WHY? 11. Are the WASH committees fully functioning? 12. How satisfy are you with WASH governance in their villages, toilet and water supply

provisions? 13. Are there mason that have been trained by the project? 14. What changes have occurred in the availability of affordable sanitation products? 15. How has access to water improved for users in terms of: reliability of supply;

accessibility; equity of access; and water quantity and quality? 16. What changes have occurred in attitude and practice relating to safe water use in

targeted areas? 17. What has enabled women to participate and influence decisions on village

committees/District WASH teams? What remains the barriers for more equal decision making?

THANK YOU VERY MUCH!!!

5) Observation checklist

HEALTH CENTRE OBSERVATION CHECKLIST

Description Yes No remarks

General cleanliness

Does the health centre surrounding look clean

Rubbish lying around either in or around the health centre

Animal waste lying around around the health centre

Animals roaming around the health centre

137

Description Yes No remarks

There are flies around the health centre

If yes, state whether many or a few

There are flies inside the health centre

If yes, state whether many or a few

The consultation room looks clean

The waiting area looks clean

The delivery room looks clean

Latrine/toilet

There is a toilet for patients / visitors

The toilet for visitors consists of one room for men and one for women

The floor and slab looks clean

There is bad smell

There are flies

There are feces around the hole

There is water for cleaning if it is a pour flush toilet

There is a place to wash hands near the toilet with water and soap

Drinking water

The drinking water is clean

The drinking water is stored in a covered container

The drinking water container looks clean

Handwashing facility

There is place to wash hands and there is water and soap present

Health Promotion

138

Description Yes No remarks

There are information or education materials ( posters, messages etc) on hygiene ( washing hands with soap, using toilet, drinking safe water

There are information or education materials on children’s health : vaccination, using ORS, vit A supplementation, growth monitoring etc

There are information or education materials on women’s health : ANC, delivery, PNC, family planning