SNV WASH Project in Ethiopia - Aidenvironment WASH Project in Ethiopia Final impact evaluation Commissioned by SNV Netherlands October 2015 Project number 2529 Aidenvironment Barentszplein

SNV WASH Project in Ethiopia Final impact evaluation

Commissioned by

SNV Netherlands

October 2015

Project number 2529

Aidenvironment

Barentszplein 7

1013 NJ Amsterdam

The Netherlands

+ 31 (0)20 686 81 11

[email protected]

www.aidenvironment.org

Project number 2529 ii

Contents

Abbreviations and definitions iv

Executive Summary v

1. Introduction 1

2. Methodology 3 2.1 Baseline study 3 2.2 Overview methodology 4 2.3 Survey methods, sampling and analyes 5 2.3.1 User and facility level: households and schools 5 2.3.2 Water schemes 7 2.3.3 Capacity building efforts 8 2.4 Analysis of result pathways 10 2.5 Limitations of the methodology 10

3. Results 12 3.1 Households 12 3.1.1 Access to safe water 12 3.1.2 Access to improved sanitation 13 3.1.3 Incidence of diarrhea and analysis 15 3.1.4 Access to information and training on WASH 16 3.1.5 Additional analysis on explaining factors 17 3.2 Schools 18 3.2.1 Access to safe water in schools 18 3.2.2 Access to improved sanitation in schools 19 3.2.3 Functionality of WASH club and analysis 19 3.2.4 WASH related school management and analysis 20 3.2.5 School attendance and analysis 21 3.3 Water schemes 21 3.3.1 Functionality of water schemes 21 3.3.2 Performance of the WASH committees 23 3.3.3 Preventive and Operational capacity training by SNV 23 3.3.4 Comparison with baseline situation 24 3.4 Capacity building 25 3.4.1 Capacity building interviews and focus group discussions per woreda 25 3.4.2 Interviews at regional level 31 3.4.3 Interviews at the Hawassa TVETC 33 3.4.4 Interviews at UNICEF – Hawassah branch 34 3.4.5 Conclusions from the regional level interviews 35

4. Analysis and conclusions 36 4.1 Summary of main findings 36 4.1.1 Comparison of woredas 36 4.1.2 School level findings 37 4.2 Analysis of impact pathways 39 4.2.1 Impact pathway 1. Household access to improved sanitation 39 4.2.2 Impact pathway 2. Schools access to water and improved sanitation 40

Project number 2529 iii

4.2.3 Impact pathway 3. Improved management of water schemes 42 4.2.4 Impact pathway 4. Capacity building of institutions responsible for WASH 44 4.3 Evaluation questions 45

Annex 1: Theory of change (reconstructed by the evaluation team) 51

Annex 2: Evaluation Questions (extract from ToR) 53

Project number 2529 iv

Abbreviations and definitions

AKVO Akvo (organization in the Netherlands)

CLTSH Community Led Total Sanitation and Hygiene

FGD Focus Group discussion

GDL guided distance learning

GLOWS Guided Learning on water and Sanitation

MDG Millennium Development Goals

M&E Monitoring & Evaluation

MHM Menstrual Hygiene Management

MIS Management Information Systems

NGO Non Governmental Organisation

O&M Operations and Maintenance

POM Preventive Operations and Maintenance

SNNPRS Southern Nations, Nationalities and People Regional State

SNV Netherlands Development Organisation

SPSS Statistical Packages for Social Scientists

TTLM Teaching Training learning Materials preparation

TVETC Technical vocational education training center

UAP Universal Action plan

UNICEF United Nations Children’s Fund

VIP ventilated improved pit

WASH water sanitation and hygiene

Definitions

Improved sanitation facility: one that hygienically separates human excreta from human contact.

(WHO definition). The improved sanitation facilities include flush/pour flush (to piped sewer

system, septic tank, pit latrine), ventilated improved pit (VIP) latrine, pit latrine with slab, and

composting toilet.

Unimproved sanitation facilities include: public or shared latrine, open pit latrine or bucket latrine

Secure sanitation facility: sanitation facility with secured privacy (in schools)

Traditional sanitation facility: open field defecation or traditional latrine facilities that are pit

latrines without a slab.

Project number 2529 v

Executive Summary

Background

In the period of 2007 to 2012, SNV implemented a WASH project in 6 woredas (districts) of the

SNNPRS (Southern Nations, Nationalities and People Regional State) in Ethiopia. The project aimed

at increasing access to water supply, sanitation services, and improved sanitary practice in the

communities and at schools. The project introduced the CLTSH (Community led Total Sanitation and

Hygiene) approach in the 6 woredas. The project chose a model of close collaboration with, and

capacity building of, the local government offices, focused at strategic planning, improved service

delivery, community involvement, monitoring and evaluation. Aidenvironment carried out an impact

evaluation of this project in 2015, with the purpose to assess the continuing impacts of the SNV

WASH project, especially the improved access to improved water and sanitation for the target groups

as well as the effects on the capacities of the government functionaries in the WASH sector.

Methodology

The impact evaluation focused at household level, school level, water schemes and capacities of

institutions within targeted woredas and regional level. Mixed methods were used including

structured and semi-structured surveys, in-depth interviews and focus group discussions. The

household and schools surveys focused on access to safe water, access to improved sanitation and

information and training on WASH. The surveys were translated in local language and use was made

of digital data collection. In total 1450 household interviews were conducted, selected from 58

kebeles (rural municipalities) in the 6 woredas. In total 50 schools were selected and 100 school

interviews (50 pupils and 50 teachers) were conducted, equally distributed over the 6 woredas.

Survey data were analysed using SPSS, descriptive statistics and inferential analysis. In total 50 water

schemes were visited, with surveys to determine functionality. To evaluate capacity building effects of

relevant staff (within the WASH, education and health sectors) at woreda level, the 5C (5 capabilities)

approach of evaluating capacity building was used as a basis for interviews, using a balance score

card system specifically developed for this study. The study did not include control groups for

households, schools or water schemes. The study did include two non-targeted woredas, as control

groups, to make a comparison in capacities between staff within targeted and non-targeted woredas.

To establish the main changes in time, the study compared the current values with the baseline

values (from 2007). This posed some challenges because the baseline data and the approach used

were not well defined. Also, a reliable midterm measurement survey was missing. To draw

conclusions on the contribution by the SNV project, the quantitative and qualitative results were used

to assess whether defined impact pathways (based on the project’s theory of change) could explain

the changes in time, including the relative contribution by the SNV project and other (external)

factors.

Findings

In the analysis of access to water, in line with the SNV baseline survey we used three criteria: a) the

type of main water source and its safety (by interview and observation), b) the travel distance or

travel time to the safe main water source being less than 1.5 km or travel time not being more than 30

minutes, and c) the access of clean water storage. When taking into account these three criteria,

access to water for households in the project area has declined from 53% in 2008 to 46.4% in 2015.

Currently travel time was more than 30 minutes for 40.7% of the respondents, which is still better

than the average of 62% for rural areas in Ethiopia, according to the Ethiopia Demographic and

Health survey of 2011. In the analysis of access to improved sanitation, we adopted four criteria, in

line with the baseline survey: a) the presence of an improved latrine facility; b) cleanliness of the

latrine, c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.

Project number 2529 vi

When taking into account these criteria, we found that an increase of access to improved sanitation

within households from 27% to 41.4%. Looking at the various factors that influence access to

sanitation (according to the impact pathway), we conclude there is a significant relation and plausible

contribution by the SNV project to the improvement in access to improved sanitation. There is also a

significant correlation between access to safe water and access to improved sanitation for households,

suggesting that access to safe water influences improved sanitation. The decline in the incidence of

diarrhoea is highly significant (from 52% to 17%), and can be partly explained by the improved access

to improved sanitation.

Concerning the schools, access to water has significantly improved (from 7% to 22%, or 52% when

water sources beyond the school premises are also considered). In the schools 94% have a latrine

facility, but access to improved sanitation declines to 28% if we also take into account aspects of

cleanliness and safety. This can be explained by limited access to water within the school compounds

or limited management by teachers of WASH within schools (only 26% scores very good). Most

student-based WASH clubs (72%) are functional. We cannot conclude there is a significant relation

by the SNV project to access to improved sanitation in schools but a contribution by the SNV project

is plausible based on evidence from interviews and focus group discussions.

To determine functionality of water schemes, three criteria were used: water discharge, water quality

and waiting time. It was found that for the majority (72%) of water schemes in the project area

functionality has improved. The SNV project has contributed to the improved functionality of water

schemes in the project area, by training on operations & maintenance and supporting local WASH

committees. For instance, the time it takes before a reported failure is repaired has considerably

reduced. However, for 70.6% of the water scheme users, waiting time at the water scheme was more

than 30 minutes (with a maximum of 5 hours), reducing their functionality. We conclude there is a

significant contribution by the SNV project to the improved functionality of water schemes, based on

evidence from interviews and focus group discussions This also means that access to water would

probably have been lower (than 46.4%, see above) if SNV had not contributed to improved

functionality of water schemes. The remaining low functionality of water schemes is mainly due to

low volumes of water from existing water schemes in relation to high and increasing water demand,

with underlying causes of relatively few new water schemes being constructed or low water volume

per water scheme.

Opinions of woreda staff are positive with respect to the services being provided by the SNV project.

At woreda level, strategic planning, service delivery, community involvement and monitoring have

much improved. The targeted woreda’s have better capacities for WASH strategic planning and

WASH service delivery than control woreda’s. The model of training-on-the–job (GLOWS) has been

very effective in terms of improving service delivery. The response rate to repairs has increased. Use

of the Management Information System (MIS) has failed due to lack of support and continuity at

regional level. We conclude there are positive changes in capacities of woreda level staff to manage

WASH related aspects and plausible evidence for a significant contribution by the SNV project.

With respect to sustainability of the results, most capacity building results will sustain, such as the

integrated strategic planning approach (education, health and WASH) and the improved service

delivery. Also, most local water schemes are well maintained with WASH committees (88%),

including fees being charged. Among the targeted woreda’s, there is some evidence that the targeted

woreda’s are better able to access funds from donors for investment objectives, presumably because

of their improved planning and funding requests. Nevertheless, funds for investments in new water

schemes and large-scale maintenance of existing water schemes remain insufficient. The GLOWS

approach is reported to be costly and still not formally recognised, and therefore uncertain to be

continued. Altogether, there are insufficient recurrent financial inputs to the WASH sector in the

Project number 2529 vii

region, e.g. through fees or taxes. From an environmental angle, it is worrisome that water boreholes

have to go deeper and yet cannot supply sufficient water for the whole population in the region. This

may lead to depletion of underground water resources. More attention could be given to rainwater

capture and/or underground storage. There may also be need for collaboration and integration with

natural resources and watershed management for improving vegetation coverage around water

sources so as to improve water recharge.

Thus, there are remaining challenges, which are mainly related to external factors, including the need

for institutional support to use the MIS and continue the use of GLOWS, additional hardware

especially on new water schemes, operations and maintenance of large water schemes to increase

water discharge, and improved motivation of staff and improved skills on specific tasks.

Conclusions and recommendations

The approach taken by SNV in the WASH sector is innovative by focusing on capacity building of

local public institutions, on awareness raising and change of practices, involving training institutes

and networking. This focus by SNV on the ‘software’ was part of a programme partnership with

UNICEF that focused on the ‘hardware’. Overall, this evaluation shows that the SNV approach has

generated significant results, but the ‘hardware’ component now seems to constitute the main

constraint. The contribution by SNV is positive in relation to change of behaviour on sanitation and

hygiene at household level, improved maintenance of water schemes, build up of capacities at woreda

level, among the communities, WASH committees and WASH clubs at schools. Also, there is

increasing private sector involvement in the WASH sector. Although the funding from national

sources to the WASH sector has improved for the 6 targeted woreda’s, funds for investments in new

water schemes and large-scale maintenance of existing water schemes remain insufficient.

Considering the remaining challenges, there is a potential role for SNV to play as follows:

Scaling the successes achieved in the sanitation sector, by capitalizing on the significant correlation

between access to improved sanitation for households and the support and training provided;

Strengthen the WASH monitoring and evaluation system within SNV and with local partners,

including proper establishment of a baseline measurement, and including a technology to monitor

water discharge of water systems and waiting time;

Strengthen a joint learning and evaluation system and assure continuity in follow-up or refresher

training, to assure further improvement and/or to avoid fall-back;

Take the lead in proactively engaging in advocacy and lobby to narrow the existing gap on demand

for safe water, by lobbying for budget increases to develop additional water facilities and

developing more sustainable solutions ;

Further strengthen private sector engagement in the WASH sector;

Pay more attention to rainwater capture, underground water recharge and storage and watershed

management, to avoid water depletion and to increase access to water from sustainable sources.

Project number 2529 1

1. Introduction

The SNV WASH project

In the period of 2007 to 2012, SNV implemented a WASH project in 6 woredas (districts) of the

SNNPRS (Southern Nations, Nationalities and People Regional State) in Ethiopia. The project aimed

at increasing access to water supply, sanitation services, and improved hygiene practice in the

communities and at schools. The project introduced the CLTSH (Community led Total Sanitation and

Hygiene) approach in the selected woreda’s. To assure sustainability of its efforts, the project chose a

model of close collaboration with, and capacity building of, the local government offices. Capacity

development focused at community needs assessment, planning and introducing a Management

Information Systems (MIS). It supported the woreda’s to develop their WASH strategic plan aligned

with the UAP (Universal Access Plan) at national level. Another element of the project was to

improve Operations and Maintenance (O&M) of selected water schemes. To do so, within each

woreda the technicians of the WASH team were mobilised and trained on O&M aspects. In each

woreda, water schemes were supported on O&M through the local WASH committees (committee

responsible for O&M of the water scheme) to properly manage user fees by users, keep the water

scheme clean, install a fence and establish effective drainage.

At schools the approach of multi-stakeholder inquiry was applied. This a form of action research to

raise awareness and then make a plan to improve WASH performance. To do so, SNV selected 10-15

schools in each woreda. SNV also introduced the MHM (Menstrual Hygiene Management) for girls at

schools (introduction of low-cost pads). Within the selected schools, SNV also introduced the

establishment of WASH clubs and stimulated these clubs to come together and develop an action

plan.

SNV also worked with the local vocational college (TVETC) to improve their curriculum and develop

practical skills of their graduates in operating and maintaining water and sanitation facilities. This

was later on expanded with an approach of guided distance learning (GDL), which was formulated as

GLOWS (Guided Learning on water and Sanitation).

Based on the above project headlines, the evaluation team reconstructed the theory of change (see

Annex 1) and a number of impact pathways (see chapter 4). The theory of change distinguishes

between a range of ‘software’ or capacity building activities, and assumes that either the ‘hardware’ is

already available or that resources for ‘hardware’ are made available by other partners. The

partnership with UNICEF would assure that hardware (water schemes) could be provided where this

was deemed necessary and useful. The functionality of water schemes is an important element of the

theory of change. The assumption was also that upscaling would take place as a result of increased

awareness, increased finance allocation for WASH hardware provision by the district, and improved

capacities within woredas to manage the existing water schemes.

This impact evaluation study

Aidenvironment has been commissioned the assignment to carry out an impact evaluation of this

project. The purpose of the evaluation documented in this report is to assess the continuing impacts

of the above SNV WASH project. It will include assessing the effects on the capacity and ways of

working of the government functionaries in the WASH sector as well as any lasting effects on

improved access to improved water and sanitation for the target groups in these woredas. The

findings will be used both for accountability to the donors and other stakeholders, and for

organisational learning to improve SNV’s practice in WASH and of working with local governments

in particular.


The scope of the evaluation can be characterized as follows:

focus on the WASH project in 6 woreda’s of the SNNRPS region in Ethiopia.

focus on the OECD/DAC criteria impact, effectiveness and sustainability.

cover the period from the start of the project until now. The effort will be to assess the changes that

the project brought and whether those changes have been sustained until now or not.

focus on the 6 project woreda’s in the SNNP region and comparison areas/ woredas selected for the

comparison of assessment in the SNNP region.

A set of research questions was formulated that form the basis for this evaluation (see Annex 2).

This report

An inception report has been delivered in February 2015. To do so, Aidenvironment studied the

available documentation and had a 3-day workshop in Ethiopia (28-30 January), together with the

local consultants and relevant SNV staff, to prepare the impact study. The fieldwork was carried out

in March-April. Data were received in May and this draft report was prepared in June 2015.

Following feedback by the Steering Committee a final version of this report was prepared in August

2015.

This report is organized by the following chapters:

Chapter 2: methodology followed for the field data collection and analysis of the data

Chapter 3: results of the data collection and analyses, for household survey, school survey, water

scheme survey and capacity building analysis separately

Chapter 4: conclusions, validation of impact pathways and responses to the main evaluation

questions.

There are a series of appendices in a separate document that provide all the basic data.


2. Methodology

2.1 Baseline study

According to the ToR the SNV WASH project conducted in 2007 a baseline study as well as in 2011 a

midterm evaluation study. During the inception phase we reviewed the quality of these studies.

Baseline study

In 2007 an extensive baseline study was executed. The main characteristics of the baseline study

executed in 2007 are the following:

Baseline survey undertaken in the selected 6 woredas, the Kebele and major urban centres are the

basic unit for the survey in each Woreda

The basic units for data collection and observations are the households, the institutions, Kebele

WaSH/Water scheme committees, Woreda WaSH team and Town WaSH Teams.

The data were collected by local capacity builders, Woreda WASH teams, Kebele baseline survey

teams and interviewers at the Woreda, Kebele, institutions and water source levels.

Both quantitative and qualitative data were collected by interviewing of household members and

informants from institutions based on a structured questionnaire and also by observations.

50 households were sampled randomly as respondents per Kebele.

For all schools, Health centers, Markets and drinking Water sources surveys were done.

A team of 3 interviewers each interviewing 7-8 households in a day.

A Kebele baseline survey team consisting of 1 man and 1 woman was trained at each Kebele

In each Woreda and Kebele, a focus group discussion was held with the Woreda WaSH team and

Kebele WASH/water scheme committees to collect relevant WaSH information.

During verification of the baseline data collected in 2007 several woredas found the data did not

correspond to their perception of the reality of the field. Especially, access to sanitation data were

found to be too positive. This was assumed to be mainly the result of the fact that many households

construct latrines but do not use them, or have constructed very simple latrines because this was a

requirement to be able to receive food aid. These latrines were not used and therefore cannot be

qualified as ‘access to sanitation’. Apparently enumerators too easily provided a positive qualification

of access to sanitation if only a simple latrine had been produced.

Therefore, in 2008 a verification survey was carried out, with the following approach:

Adopt both observation and interviewing of households and institutions in the Kebeles. In total 58

kebeles were selected from the 6 woredas (see in below table)

Out of 50 households interviewed in the previous survey, 25 were selected randomly and re-

interviewed. Each data collector interviewed 5 units (households and or institutions) per day.

Female teachers collected the data and fewer questions were asked. Female teachers were preferred

because they were presumed to be more credible and therefore unlikely to lie about the WaSH

status and they could easily win the confidence of the household members and therefore be allowed

to observe cleanliness in the households and water storage containers. The teachers signed an

agreement with SNV to provide accurate data.

A Woreda employee not associated with the WaSH activities supervised the data validation process.

The verification process generated new and more reliable data, leading to corrections on the previous

set of baseline data. The impact evaluation team does not fully understand how the validated data

were combined with the initial survey data to result in a final value, which is the reason why we took

as the baseline the data from the verification survey (see relevant tables in chapter 3).


As verified during the inception phase, according to SNV the verified access to water baseline data

have taken into account:

distance to the water points (at most 1.5 km)

cleanliness of the water storage containers

As verified during the inception phase, according to SNV the verified access to sanitation data have

taken into account:

use of the latrines (if a latrine is not used, it falls in category of open defecation)

cleanliness of the latrines (if latrines are unclean it would assume open defecation).

absence of faeces in the compound.

Midterm evaluation 2011

During the inception phase it was found that SNV did not carry out a midterm evaluation survey.

However, surveys have also been carried out by the Water bureau of the SNNPRS and consultants

have made use of these data in case study and other reports for SNV. For instance, a report for SNV

entitled ‘report on six SNV supported woreda’s in SNNPRS’ makes use of the 2008/09 and 2010/11

inventory of water access by the SNNPRS (mentioning that these are unpublished data). However,

there are different reasons why these data cannot be used:

we do not have access to the survey sheets and do not know which questions were asked and how

the data had been interpreted and analysed;

the data for 2011 vary in almost every available document; moreover, the 2011 data in the formal

report of this survey (which only appeared in 2014) are all different from the data mentioned in the

case study reports and show outliers that cannot be correct (this is even admitted in the report).

There is also available a Demographic and Health Survey (DHS) from 2011. However, this survey

does not provide data at woreda level. We did make use of the DHS survey questionnaire to compile

our own questionnaire.

2.2 Overview methodology

The evaluation focused at four different levels of evaluation: household user level; facilities (school)

level (schools), water schemes level and local government capacities level. In terms of facilities, only

schools were assessed, because at the level of markets and health facilities no significant effects are

expected. A variety of methods was used to collect data to assess the impact of SNV’s WASH-project

in Ethiopia: structured and semi-structured surveys, in-depth interviews and focus group discussions

(FGD). Table 1 provides an overview of the methods used at the three evaluation levels.

It should be emphasized that the study did not include control groups for households, schools or

water schemes. On the contrary, the study did include control groups at woreda level. This was done

according to the ToR and is understandable for two main reasons. First, as direct effects of the SNV

WASH project we expect results at the level of local woreda agencies, reason why controls are

included. According to the theory of change capacity building will result in effects at household,

school and water scheme levels. However, most likely these effects will expand beyond the focus

woredas, also in view of the fact that other NGOs may have interventions that directly target

households, schools and water schemes. Thus, controls at woreda level are in line with the fact that

we want to test the specific SNV WASH theory of change. The second reason is pragmatic, being the

high costs that would be related to control groups at household, school and water scheme levels.


Table 1. Overview of methodology

User level (households)

and facilities (schools)

Water schemes Institutional capacity

building efforts

Type of

method

Before-after comparison:

comparing baseline study

data with impact data

Mixed methods:

quantitative survey and

qualitative FGD

Surveys based on baseline

survey, for comparison

Contribution analysis

based on impact

pathways

In-depth interviews and

FGDs

Interview scheme based

on questionnaire baseline

study


based on impact

pathways

Control group approach:

selection of 2 Woredas for

comparison

In-depth interviews and

FGDs

Interview scheme based

on questionnaire baseline

study


based on impact

pathways

For each of the 4 levels mentioned in table 1, we will describe briefly the methodology implemented.

2.3 Survey methods, sampling and analyes

2.3.1 User and facility level: households and schools

Survey construction

For household and school level, separate surveys were constructed: a structured survey for household

level interviews and a semi-structured survey for school level interviews. Both surveys have been

developed in collaboration with SNV Ethiopia and local consultants.

The household surveys focused on 3 topics: access to safe water, access to sanitation and awareness,

received support and information. The school level surveys has separate sections for students and

teachers, covering as topics: access to safe water, access to improved sanitation, membership of

WASH-club, management of WASH-facility (teachers only) and school attendance (teachers only).

Note that these surveys were designed as based upon the baseline survey formats in order to allow for

before (baseline)-after comparisons. Both surveys can be found in appendix 1. Both surveys have

been translated into local language and have been uploaded in the AKVO tool to allow for digital data

collection.

Data collection

The surveys for households and schools were conducted in the period between March 15th and April

7th, 2015. The household and school surveys were conducted by a team of 10 local enumerators, in

collaboration with and supervised by the 2 local consultants. All local enumerators and consultants

were trained extensively beforehand on the understanding of the questionnaire, the use of the AKVO

tool, and the ethical criteria to be respected. The enumerators and consultants also received training

in using the AKVO tool, a tool for data collection and analysis. A survey coordinator was present

during data-collection to support the team, to ensure sampling procedures were followed and to

perform quality checks on data collected.

As four different local languages are spoken in the 6 surveyed woredas, local translators were

deployed to translate the local language to Amharic. These local translators were selected from each

surveyed village, taking into account a) their ability to speak the local language, b) their ability to

speak the Amharic language and c) neutrality and prevention of bias by making sure the translators

did not work for the kebele administration and/ or WASH-sector.


Households: sampling

In total 1450 Household interviews were conducted using the AKVO -tool and included in the

analysis.1 Due to technical reasons the data of 24 interviews were not saved into the tool and could

therefore not be included in the analysis. These 24 interviews are from different villages, so did not

create a bias. In total, data of 1426 interviews are available for analysis.

The 1426 households were randomly selected from 58 kebeles (rural municipalities), situated in 6

woreda’s (districts). To ensure comparability, the woreda’s and kebeles that were selected for the

survey are identical to those selected in the baseline study. Selection of villages within the kebeles was

made at random: for each of the 58 kebeles a list of villages was made and, using the random

numbers-method, 1 village per kebele was selected. Likewise, the selection of households within the

selected villages was made at random: per village a list of households was prepared and the random

numbers method was used to select 30 households per village. Note that at average there are about 10

villages per kebele. The number of households per kebele generally varies between 40 and 70.

Schools: sampling

In total 50 schools were selected and 100 school interviews (50 pupil interviews and 50 teacher

interviews) were conducted, using the AKVO tool. For 2 woreda’s a list of SNV-targeted schools was

available. In these woreda’s we selected all targeted schools for the survey (8 schools in Misrak

Badewacho and 5 schools in Kedida Gamella). The selection of the remaining 37 schools in the other

4 woreda’s took place by randomly selecting schools from the villages where the household surveys

were carried out. School characteristics (such as first cycle primary, second cycle primary, secondary,

preparatory) were included in the sampling procedure, to ensure different types of schools were

included in the sample.

Data Cleaning and Analysis

The data submitted to the server for storage were accessed on AKVO dashboard and inspected by the

survey coordinator for quality check on daily basis throughout the data collection period.

At the end of the data collection for the quantitative household and school surveys, the 1426 data

forms for the household survey and 100 data forms for school survey submitted to the dashboard

were exported into EXCEL for making ready to import into SPSS (Statistical Packages for Social

Scientists). The three EXCEL data sets (one for household survey, and two for teachers and students

surveys in the school) were converted into SPSS files for cleaning, processing and analysis. Then the

household, student and teacher surveys were used for generating quantitative insights which were

interpreted and analysed on apparent correlations between parameters.

In analyzing the quantitative data in SPSS, descriptive statistics and inferential analysis were carried

out. Descriptive statistics, like percentages and ratios, were calculated for describing the key

statistical results. Chi-square and correlation tests were done for assessing association between

selected dependent variables (which are incidence of diarrheal diseases) and the

independent/explanatory variables like access to safe drinking water and improved sanitation.

Note that in the analysis of criteria for access to water, in line with the SNV baseline survey we

introduced three criteria: a) an observation is made of the safety of the main water source, b) travel

distance to the safe main water source is less than 1.5 km, and c) people have the possibility of clean

water storage. We also looked at waiting time because this could be a more reliable indicator than

distance to the nearest water source.

1 To calculate the number of household interviews, the following formula for determining sample size was used: n = ((z2 x p (1-p)) /d2,

where z2 stands for confidence level of 90%, p stands for prevalence of WASH indicators, d stands for acceptable difference/ margin of

error of 5%.


In the analysis of access to improved sanitation, we also adopted four criteria, in line with the

baseline survey, as follows: a) observation that there is an improved latrine facility; b) cleanliness of

the latrine, c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.

Note that an improved latrine facility for MDG monitoring is defined as one that hygienically

separates human excreta from human contact. The improved sanitation facilities include flush/pour

flush (to piped sewer system, septic tank, pit latrine), ventilated improved pit (VIP) latrine, pit latrine

with slab, and composting toilet. This compares with open field defecation or traditional latrine

facilities that are pit latrines without a slab.

2.3.2 Water schemes

For assessing impact on the water schemes, focus group discussions (FGD) were held. The checklist

of questions for the FGDs is based on the baseline survey and has been constructed in collaboration

with SNV Ethiopia and the local consultants. The FGD checklist can be found in appendix 1.4.

In total 50 Water schemes have been visited, consisting of boreholes, shallow wells, hand dug wells

and springs. Of the 50 water schemes that were surveyed, 25 were constructed before 2007 (when the

project started) and 25 were constructed after 2007. Also, SNV support that was provided to these

water schemes was a selection criterion, so that half of the old and new water schemes received

maintenance support from SNV and the other half did not. Thus we have 4 categories:

12 constructed before 2007 and no SNV operation and maintenance support received

13 constructed before 2007 and SNV operation and maintenance support received

12 constructed in or after 2007 and no SNV operation and maintenance support received

12 constructed in or after 2007 and SNV operation and maintenance support received

The focus group discussions were carried out with users and WASH committees (WASHCO’s). Based

on the questionnaire results, the water schemes and WASH committees were assessed and classified

in terms of their current functionality (of water schemes) and operations and maintenance

performance of the WASH committees.

Functionality of water schemes

To determine functionality, we developed 3 criteria to assess water schemes: water discharge, water

quality and waiting time. We defined three levels of water scheme functionality, as follows.

Functionality level of

water schemes

Criteria

Good Discharge volume = good or low / less during dry season, and

Water quality= good or moderate / lower during rainy season, and

Waiting time = <30 min

Partly Discharge volume = low / less during dry season, and

Water quality = moderate or lower during rainy season, or

Waiting time = >30 min

Poor Discharge volume = not functional, or

Water quality = low

Please note that in the above classification of functionality we may deal with cumulative (and) criteria

or with facultative (or) criteria. The criterion of waiting time more than 30 minutes was introduced


because of the very high proportion of users that have to wait long to be able to collect water. In the

literature, a waiting time of more than 30 minutes is commonly considered as being too long.2

Performance of the WASH committees

To determine performance, WASH committees were assessed on 7 criteria: presence of WASH

committee, satisfaction of users about WASH committee, charge of fees, fees used for Operation &

Monitoring (O&M) and satisfaction of fee use, incidence of conflict and resolution of conflicts. Using

these criteria, we defined 2 levels of performance, as follows.

Performance of

WASH committee

Criteria

Good Presence of WASH committee

Satisfied with WASH committee, or partly satisfied

Fees charged

Fees used for O&M

Satisfied with use of fee

No incidence of conflict or resolution of conflicts

Poor Absence of WASH committee

Users not satisfied with WASH committee

Conflicts existent and/or not solved

2.3.3 Capacity building efforts

Survey construction

The checklist of questions for evaluating capacity building has been developed in collaboration with

SNV Ethiopia and the local consultants. As a basis we took the capacity building interventions that

took place during the SNV project period, and the 5C approach of evaluating capacity building (as

also being adopted by SNV). The survey checklists can be found in appendix 1.4.

Separate surveys were designed for the 4 different levels where interviews were held. To assess the

impact of capacity building efforts, focus group discussions (FGD) and in-depth interviews were

performed at 4 levels.

Level Method Number

Woredas (8 woredas, of which 6

targeted and 2 control woredas)

- Individual in-depth interview

- FGD WASH team

- 3–4 per district

- 1 per district

Regional authorities Individual in-depth interviews 3–4

TVETC Individual in-depth interviews 1 -2

UNICEF Individual in-depth interviews 1 -2

Sampling method

In total 8 woredas were visited, including the 6 SNV-targeted woredas and 2 control woredas. The 2

control woredas (Baloso Bombe and Angecha) were selected on the basis of the following criteria:

presence of water schemes a bit comparable to the target woredas;

near / not too far from the target woredas (for reasons of logistics and comparison);

no past interventions by SNV, no major interventions by other NGOs.

2 Queuing time at a water source is no more than 30 minutes. See :

http://www.spherehandbook.org/en/water-supply-standard-1-access-and-water-quantity/

http://www.spherehandbook.org/en/water-supply-standard-1-access-and-water-quantity/


Appendix 2.2 provides names of consulted respondents, through individual interviews or FGDs.

Scoring capacity building

In order to classify the wealth of information generated on capacity building by using the surveys, we

developed a balance score system to assess capacity building efforts on 7 aspects, in line with the 5C

approach of evaluating capacity building efforts as being used and promoted by SNV:

Strategic planning of WASH sector

Finance for WASH sector

WASH service delivery

Gender aspects in WASH

Communication and private sector participation in WASH setor

WASH database and monitoring and evaluation system

Partnerships for support to the WASH sector.

The balance score card makes an assessment of the information for each of the above 7 aspects, as

indicated in the below assessment matrix.

Current situation score

(1= lowest score)

‘What has changed’ score

(a=lowest)

Strategic

planning

There is no strategic plan (1)

Strategic plan follows the Growth and

Transformation Plan (GTP), but little or no

special attention for WASH (2)

There is a strategic plan with special

attention for WASH (3)

No improved focus on WASH in strategic

plan (a)

Improved design of strategic plan but no

improved result (yet) or lessons learned

(b)

Improved design and implementation of

strategic plan, possibly leading to improved

WASH services (c)

Financing Low priority (awareness) and no adequate

budget for WASH and high dependency on

donors (1)

There is high priority and budget for

WASH but not yet sufficient, dependency

on donors (2)

There is high priority and adequate budget

for WASH ( 3)

No improvement of budget and / or

priority (a)

Improved priority (awareness) but no

increased budget (b)

Increased budget and improved priority

for WASH (c)

WASH

delivery

No or limited WASH services are provided,

or are not of good quality (1)

WASH services are provided but do not

fully meet demand client (2)

WASH services are provided in terms of

coverage and utilization, are adequate in

terms of coverage and quality needs of

client (3)

No change or no improvement in service

delivery capacities (a)

Improved capacity of staff but does not

translate into improved services (b)

Improved capacity of staff to deliver

WASH services and improved services

delivered (in terms coverage and quality)

(c)

Gender and

youth aspects

Limited or no involvement in WASH and

no specific service delivery for women,

youth and vulnerable groups(1)

Good involvement of women ,youth and

vulnerable groups, OR specific service

delivery is provided ( 2)

No change in gender, youth and vulnerable

social groups inclusion, or situation has

become worse (a)

Improved gender, youth and vulnerable

group inclusion OR improved gender ,

youth , vulnerable group focus / priority (b)


2.4 Analysis of result pathways

The results of the three levels of evaluation were analysed and interpreted to assess whether the

identified impact pathways could be validated or not. To do so we combined the quantitative data (of

the household and school surveys and the water schemes) and compared these with the qualitative

data and scores on capacity building based on the FGDs and individual interviews, to draw

conclusions whether impact pathways have been realized or not.

Combining quantitative and qualitative scores is a challenge when using a mixed approach. In our

study, we have given scores to the capacity building results according to the 5C approach (see

previous section) in order to be able to match the scores with quantitative results per woreda.

However, we found the variation between woredas to be minimal. Also, we looked at specific

qualitative information to support or disqualify quantitative data.

2.5 Limitations of the methodology

The above methodology has been worked out as part of the inception report and submitted to the

Steering Committee that was assigned for this impact evaluation (with representatives of the donor,

SNV and an impact evaluation research institute). It has been approved with some minor

adjustments that were incorporated in the final version.

In spite of the rigorous method, the impact evaluation met the following limitations:

Good involvement of women, youth

vulnerable groups AND specific WASH

service delivery is provided for women and

youth (3)

Improved inclusion of gender aspects in

WASH activities and service delivery ( c)

Community

and private

sector

participation

No or little participation of community in

WASH (1)

Communities participate in WASH but

there is not sufficient capacity (fees, skills,

human) (2)

Community (and private sector if

applicable) participates in WASH,

expresses their own demands for

support(3)

No improvement in community

participation, in terms of water payment

and maintenance (a)

Improved community participation, but sill

insufficient to meet maintenance needs (b)

Improved participation community,

enough to meet maintenance needs (c)

Data base,

monitoring

and evaluation

There is no management information

system (MIS) or data base on WASH and

no resources for WASH monitoring (1)

There is a MIS or data base on WASH but it

is not used or not up-to-date (2)

There is a MIS and up to date data base on

WASH (3)

No improvement or change in WASH data

base and monitoring (a)

Introduction or involvement of staff in

M&E activities or MIS, but no follow-up (b)

Improved involvement of staff in WASH

M&E activities or MIS, adequate resources

(c)

Partnerships No partners active in the field that

contribute to WASH strategy (1)

Some partners active in the field that


Many partners active in the field that


Partners have left the area (a)

The level of support has not changed (b)

Level of support for WASH strategy has

increased (c)


One is the fact that the impact evaluation results were supposed to be compared with baseline data.

The baseline data appeared to be difficult to interpret, in spite of a verification survey of which the

final results are more reliable. Also, the criteria for assessment of access to water and access to

improved sanitation are not always clear. Also, a midterm assessment of the project results was

considered too unreliable. As a result, uncertainties remain with respect to the comparison between

baseline and impact data, and reliable trends were not always easy to establish.

In the design of the impact evaluation method we used as much as possible the same criteria as had

been used during the baseline survey. However, as noted above, in some cases the exact criteria

were not celar, e.g. to determine access to improved sanitation. Also, some adjustments had to be

made in order to stay in line with national (Ethiopia) and international standards. Therefore, the

methodology was not exactly the same as had been applied during the baseline survey.

There are no controls at household or school level. However, we believe that having controls at this

level would have been costly and there were no baseline data for control woredas (reason why it

was not part of the ToR).

The matching of quantitative and qualitative data remains a challenge but we believe to have

developed an approach to generate conclusions that make best use of both sources of information.

Ideally, one would either carry out a qualitative survey and then test resulting insights through

quantitative surveys, or the other way round carry out a quantitative survey and validate the

findings by a qualitative survey. We have done neither because the quantitative and qualitative

surveys were done in parallel (at the same time). Thus, for instance, we did not validate through

focus group discussions the quantitative finding that WASH clubs or WASH management capacity

at school did not make a difference in access to sanitation at schools.

Lastly, it remains difficult to fully attribute observed changes or differences to the SNV WASH

project, due to the fact that there are many intervening NGOs as well as government services. Over

the last few years, both have been also been active in the WASH sector. However, it has been

possible to draw firm conclusions on the contribution by SNV to the observed changes, which in

most cases are very positive.


3. Results

3.1 Households

3.1.1 Access to safe water

Access to safe water is first of all based on the statement by the respondent if there is access to a

water source. We then included in the analysis of access to safe water the following three criteria, in

line with the SNV baseline survey:

a) an observation is made to judge the safety of the main water source,

b) travel distance to the safe main water source is less than 1.5 km, and

c) an observation is made if people have the possibility of clean water storage.

When just considering the presence of a water source (looking neither at distance nor at the

possibility of safe storage) 88% of the respondents have access to safe water. When using the above

criteria only 46% of the respondents have access to safe water (Table 2). These respondents have

access to water that is safe, relatively nearby (under 1.5 km distance) and that can be stored safely.

Almost 54% of the respondents do not have access to safe water. When looking at access to safe water

without taking into account the possibility of safe storage, 53% of the respondents report having

access to safe water. Travel distance to collect water from safe main source is the variable which has

the highest influence on household’s access to safe drinking water. Among households who collected

water from safe (protected) sources, close to one-half (47%) travelled over 1.5 km for getting water

from these safe main sources, which cannot be considered as ‘access to safe water’. We found that for

40.7% of the respondents travel time to a water scheme was more than 30 minutes, while for 70.6%

of the respondents waiting time at the water scheme was more than 30 minutes.

Table 2: Access to safe water in the 6 woredas together, according to three criteria.

Access to

safe

water

Access to safe

main water source

(no criteria)

Access to safe source

(with first two criteria)

Access to safe water

(with all three criteria)

Count (%) Count (%) Count (%)

Yes 1255 88.1 667 53.2 659 46.4

No 169 11.9 586 46.8 760 53.6

Total 1424 100.0 1253 100.0 1419 100.0

It is interesting to observe that the proportion of households with a source of water at more than 1.5

km distance (47%) is comparable to the proportion of households that have to travel more than 30

minutes (40.7%).

When looking at woreda-level, results show that access to clean water, varies among woredas.

Highest is 70% in Misrak Badewacho having access to safe water, compared to the lowest score of

17% in Demboya. Results for these and other woredas are summarized in Table 3, with indication of

the baseline values and the conclusions with respect to trends.


Table 3: Baseline (2008) and current access to safe water in the 6 woredas, with trend

Woreda

Baseline

2008

Access to safe main

water source (2015)


(all 3 criteria) (2015)

Trend 2008-

2015

(none/3 criteria) (%) Count (%) Count (%)

Alaba 50 325 89.0 74 20.4 Up / down

Boloso Sore 20 170 85.0 104 52.3 Up / up

Demboya 60 86 57.3 26 17.3 down / down

Kedida Gamella 30 182 90.5 139 69.5 Up / up

Misrak B. 80 237 94.8 175 70.3 Up / down

Shashego 80 255 98.8 141 54.7 Up / down

All 53 1255 88.1 659 46.4 Up / down

Given the uncertainties in the baseline survey, there are now two options for comparison between

baseline and impact survey data. The first option is that of comparing the baseline data with the

simple ‘access to safe main water source’ scoring (assuming that the baseline survey did not take into

account distance to water and clean storage). In this case, 5 of the 6 woredas show an improvement,

and only in Demboya there is a slight decline. The second option is that of comparing the baseline

with the more realistic data on access to water (assuming that the baseline data took into account

distance to water and clean storage). In this case, there is a decline in 4 woredas and an improvement

in 2, and an average decline in the totality of 6 woredas. Given our understanding of the baseline

survey, we assume that the second option is most likely the correct one, thus there is a deterioration

in most of the woredas.

3.1.2 Access to improved sanitation

Access to improved sanitation is not only based on the statement by the respondent that there is a

sanitation facility. We also included in the analysis four additional criteria, as adopted by SNV in the

baseline survey: a) observation that there is an improved latrine facility; b) cleanliness of the latrine,

c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.

The results show that, first of all, while 72% of the respondents perceive their latrine facility as

improved,observations showed that many of these cannot be considered as improved (e.g. pit latrine

without slab), so that based on this first criterion only 49% has access to a improved latrine facility

(Table 4).

Table 4: Perception and access to improved sanitation in the 6 woredas together.

Access to

improved

sanitation

Perception and awareness to

improved sanitation

Actual access to improved

latrine facility (observation)

Count Percent (%) Count Percent (%)

Yes 1031 72.4 703 49.4

No 393 27.6 721 50.6

Total 1424 100.0 1424 100.0

When applying the other 3 criteria, 41% of the respondents should be considered as having access to

improved sanitation (table 5). These respondents have access to a latrine facility that is improved,


clean, regularly used and used by everyone so that the compound is clean. Thus, the compound would

also be open defecation free (ODF). Based on the four criteria it must be concluded that almost 59%

of the respondents do not have access to improved sanitation.

Table 5: Access to improved sanitation in the 6 woredas together, according to three criteria.

Access to

improved

sanitation

Observation of

safe latrine

Including

cleanliness

Including regular use and

cleanliness compound


Yes 703 49.4 629 44.2 589 41.4

No 721 50.6 795 55.8 835 58.6

Total 1424 100.0 1424 100.0 1424 100.0

When looking at woreda-level, results show that access to improved sanitation varies among

woredas. Highest is 57% in Kedida having access to improved sanitation, compared to the lowest

score of 30% in Demboya. Results for these and other woredas are summarized in Table 6, with

indication of the baseline values and the conclusions with respect to trends.

Table 6: Baseline (2008) and current access to improved sanitation in the 6 woredas, with trend

Woreda Baseline

(a)

(2008)

(%)

Improved

sanitation (1

criteria) (2015)

Baseline

(b)

(2008)

Improved

sanitation (4

criteria) (2015)

Trend

(safe / safe &

clean)

Count (%) (%) Count (%)

Alaba 50 138 37.8 20 117 32.1 down / up

Boloso Sore 50 105 52.5 10 79 39.5 up / up

Demboya 60 60 40.0 30 45 30.0 down / stable

Kedida 90 125 62.2 40 114 56.7 down / up

Misrak B. 60 126 50.4 20 110 44.0 down / up

Shashego 70 149 57.8 40 124 48.1 down / up

All 63 703 49.4 27 589 41.4 Down / up

Given the uncertainties in the baseline survey, there are now two options for comparison between

baseline and impact survey data. The first option is that of comparing the baseline data (baseline (a),

column 2) and current data (columns 3 and 4) for the improved sanitation indicator that only took

into account the first criterion (assuming that the baseline survey did not take into account

cleanliness aspects and regular use). In this case, 5 of the 6 woreda’s show a decline, and only in

Boloso Sore there is a slight increase. The second option is that of comparing the improved sanitation

baseline value (baseline (b), column 5) with the improved sanitation indicator that took into account

all four improvement criteria (columns 6 and 7) (assuming that the baseline survey took into account

aspects of cleanliness and usage). In this case, there is an improvement in 5 of the 6 woredas and one

woreda where there is no change. Given our understanding of the baseline survey, we assume that

the second option is most likely the correct one, thus we observe a significant improvement in most

cases.


3.1.3 Incidence of diarrhea and analysis

The survey also looked at the incidence of diarrhea. It appears that 17% has suffered from diarrhea

during the last year, and 83% has not. Looking at the incidence of diarrhea during the last 2 weeks, of

those who suffered from diarrhea in the last year, 32% suffered from diarrhea during the last 2

weeks. The baseline survey in 2008 gives an average value of 52% that suffered from diarrhea, it is

not clear during which time range. In any case, whether within one year or whether within the last 2

weeks, the incidence of diarrhea has strongly declined.

We now explore the relation between the incidence of diarrhea and access to safe water (overview

tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5% level of significance)

confirm that there is a significant relationship between access to safe drinking water and incidence of

diarrhea over the last one year (Table 7). The result corroborates that the households who did not

have access to safe drinking water were more likely to be affected by diarrheal diseases in the last one

year than households who had access to safe and clean water. Furthermore, the result enables to

draw a conclusion that over the last one year, households who accessed safe and clean water were less

exposed to contracting diarrheal diseases as compared to those who had no access to safe and clean

water. However, the statistical tests show no significant relation with the incidence of diarrhea over

the last 2 weeks.

Table 7. Relationship between access to safe water and incidence of diarrhea (absolute frequencies)


Yes No

Incidence of

diarrhea over last

year*

Yes 91 154

No 568 606

Total 659 760

Incidence of

diarrhea in last 2

weeks**

Yes 30 48

No 61 106

total 91 154

* Relationship between access to safe water and incidence of diarrhea over last year is significant:

Chi-square (df) = 1, p <.05.

** Relationship between access to safe water and incidence of diarrhea over last 2 weeks is not

significant: Chi-square (df) = 1, not significant

Table 8. Relationship between access to improved sanitation and incidence of diarrhea (absolute

frequencies)

Access to improved sanitation

Yes No

Incidence of

diarrhea over last

year*

Yes 83 162

No 506 673

Total 589 835

Incidence of

diarrhea in last 2

weeks**

Yes 21 57

No 62 105

Total 83 162

* Relationship between access to improved sanitation and incidence of diarrhea over last year is

significant: Chi-square (df) = 1 , p <.05.

** Relationship between access to improved sanitation and incidence of diarrhea over last 2 weeks is

significant: Chi-square (df) = 1, p<.05.


We now explore the relation between the incidence of diarrhea and access to improved sanitation

(overview tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5% level of

significance) show that there is a statistically significant relation between incidence of diarrhea in the

last one year and access to improved sanitation (Table 8). There is a similar relation between

incidence of diarrhea in the 2 weeks and access to improved sanitation.

We can conclude that households with access to improved sanitation were less likely to be exposed to

diarrheal disease over the last one year and during the last two weeks, in contrast to those households

without access to improved sanitation. The fact that both relations were found with the incidence of

diarrhea in the last one year and the last two weeks suggest a more fine-grained and significant

relation with access to improved sanitation than with access to water.

3.1.4 Access to information and training on WASH

The survey also looked at whether households had received access to information and training

support on WASH facilities. We found that 74% of the respondents received both information and

training support, 9% received either of the two and 8% none at all. This information or training

support indicator was not captured during the baseline survey.

We now explore the relation between the scores on information and training support on WASH and

access to safe water (overview tables see appendix 3.1). The statistical tests (Chi-square and

correlation at 5% level of significance) indicate that there is no significant relationship between

access to safe drinking water and the receipt of information or support on WASH.

We now explore the relation between the scores on information and support on WASH and improved

sanitation (overview tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5%

level of significance) confirm that there is statistically significant relation between household’s access

to improved sanitation and the information and training support they obtained on WASH (Table 9).

It means that households who have got information and training support on WASH are more likely to

access improved sanitation than their counterparts who have not got any information and support.

Table 9. The relationship between scores on information and training support on WASH and access

to improved sanitation (absolute frequencies).

Access to information and training support on WASH

no Either Both

Access to safe

drinking water*

Yes 53 127 479

No 67 116 577

Total 120 243 1056

Access to

improved

sanitation**

Yes 23 89 477

No 97 156 582

Total 120 245 1059

* Relationship between access to safe drinking water and access to information / training support on

WASH is not significant: Chi-square (df) = 2 , ns.

** Relationship between access to improved sanitation water and access to information/ support on

WASH is significant: Chi-square (df) = 2, p <.05.

We conclude that provision of information and support on WASH to households is related to access

to improved sanitation. On the contrary, information and support on WASH is not related to access

to safe water. This may be explained by the fact that access to safe drinking water is not so much a


matter of behavioral change rather than the hardware (wells, boreholes, …) being available close to

communities and households and also being maintained and being operational.

3.1.5 Additional analysis on explaining factors

Additional analyses were carried out to identify factors explaining some of the patterns that we

found. This was particularly useful to do for the household survey results, given the large data set and

availability of information on household characteristics. For the school survey, this was not

considered very relevant, as the data set is much smaller and there is also considerable variability

between schools (e.g. school size, location, history, type of school) while many of these factors were

not included in the survey.

The following statistical analyses were carried out for the household survey data, with corresponding

results.

Statistical analysis of relation between access to safe drinking water and sex of head of household:

no significant relation was found

Statistical analysis of relation between access to safe drinking water and age of head of household:


Statistical analysis of relation access to safe drinking water and family size: no significant relation

was found

Statistical analysis of relation between access to improved sanitation and sex of head of household:


Statistical analysis of relation between access to improved sanitation and age of head of household:

here we find a statistical relation (see table 10). The result shows that the households headed by

young score better than households headed by aged ones for changing their attitude and behavior

for accessing improved sanitation. In other words, the aged heads of households may lag behind to

change their attitude and behavior for accessing improved sanitation due to long-standing

traditional beliefs on support of defecating openly or continuing o use the traditional unimproved

latrine facilities.

Table 10. The relation between access to improved sanitation and age of head of household

Access to improved

sanitation

Age of head of household, in years

18 – 30 30 – 64 65 and above Total

Have access * 208 / 44.7% 363 / 40.6% 18 / 27.7% 589 / 41.4%

No access * 257 / 55.3% 531 / 59.4% 47 / 72.3% 835 / 58.6%

Total 465 / 100% 894 / 100% 65 / 100% 1424 / 100%

* Relationship between access to improved sanitation and age of head of household is significant:

Chi-square (df) = 2 , p<0.5.

Statistical analysis of relation between access to improved sanitation and family size: here we find a

statistical relation (see table 11). The result shows that the households with large family size are

better than those households with small ones for changing their attitude and behavior for accessing

improved sanitation. In other words, the households with large family may have better opportunity

for getting a wide range of WASH information through different channels on benefits of having

improved sanitation for changing the attitude and behavior of the household for switching from

traditional beliefs and thinking to access improved sanitation. Large families may have better

resources especially labor for constructing / building improved latrine facility than their

counterparts with small family size.


Table 11. The relation between access to improved sanitation and size of family

Access to

improved

sanitation

Family size, number of members

Total 2 or less 3–5 6-7 8-10 > than 10

Have access * 20 / 37.0% 181 / 36.9% 188 / 37.3% 175 / 51.8% 25 / 58.1% 589 / 41.4%

No access * 34 / 63.0% 309 / 63.1% 311 / 62.3% 163 / 48.2% 18 / 41.9% 835 / 58.6%

Total 54 / 100.0% 490 / 100% 499 / 100% 338 / 100% 43 / 100% 1424 / 100%

* Relationship between access to improved sanitation and family size is significant: Chi-square (df) =

4, p<0.5.

Overall, the additional analyses show that access to water cannot be explained by factors related

household characteristics, but access to improvd sanitation does show relations with head of

household and family size, but not for sex of head of household. This is due to the fact that sanitation

is a facility/service at household level that can be created directly by households, while the water

supply is a collective facility/service, its development depends often on external funding.

3.2 Schools

3.2.1 Access to safe water in schools

According to the results of the surveys, out of 50 schools there are 11 with access to safe water. Table

12 shows the results with respect to access to safe water in the school compounds, as distributed per

per woreda.

Table 12: Baseline (2007) and current access to safe water in schools in the 6 woredas, with trend

Woreda

Baseline (%)

(2007)

Access to safe water in schools (2015) Trend

Positive / total per Woreda (%)

Alaba 2 1 / 13 7.7 Up

Boloso Sore 6 1 / 8 12.5 Up

Demboya 4 2 / 6 33.3 Up

Kedida Gamella 15 2 / 5 40.0 Up

Misrak B. 14 2 / 8 25.0 Up

Shashego 0 3 / 10 30.0 Up

All 7 11 / 50 22.0 Up

We observe that 22% of the schools have access to safe water within the school compound in 2015,

which is considerably higher than in 2007 (7% only).

Our survey also found that of those that do not have access to safe water within the school compound

(39 out of 50), 15 have access to safe water outside the school, meaning that water is being fetched

outside the school. The remaining 24 schools that do not have access to safe water within the school

compound, also do not have access to safe water outside the school compound. This means that 48%

of the schools surveyed (24 schools) do not have access to water within or outside the school.


3.2.2 Access to improved sanitation in schools

With respect to access to improved sanitation, we look at safe latrine facilities and whether these are

well maintained and clean, using the following three criteria: a) observation that there is an improved

latrine facility; b) cleanliness of the latrine, c) security / privacy of the latrine. Table 13 gives an

overview. Table 14 compares the baseline valued of 2007 with the current values, per woreda.

Table 13: Access to improved sanitation in schools, in different categories according to criteria

Access to

improved

sanitation

Access to improved

sanitation (1 criteria)

Access to improved


Access to improved



Yes 47 94.0 14 28.0 13 26.0

No 3 6.0 36 72.0 37 74.0

Total 50 100.0 50 100.0 50 100.0

Table 14: Baseline (2007) and current access to improved sanitation in schools in 6 woredas

Woreda

Baseline

value (%)

(2007)

Access to improved


Access to improved


Trend

Safe/ safe&clean

Count (%) Count (%)

Alaba 54 10 76.9 3 23.1 Up / down

Boloso Sore 35 8 100.0 2 25.0 Up / down

Demboya 41 6 100.0 1 16.7 Up / down

Kedida Gamella 65 5 100.0 1 20.0 Up /down

Misrak B. 56 8 100.0 3 37.5 Up / down

Shashego 40 10 100.0 3 30.0 Up / down

All 49 47 94.0 13 26.0 Up / down

Due to uncertainties with respect to the baseline values, there are now two different options for

comparison. The first option is when we compare data on improved sanitation with the baseline value

(assuming that the baseline survey did not take into account aspects of cleanliness and security). In

this case, we observe a considerable improvement (positive trend). This may be explained by the fact

that almost all schools nowadays have established improved latrine facilities, which was not the case

in 2007. The second option is when comparing the baseline value with the indicator on improved

sanitation (assuming that the baseline survey took into account aspects of cleanliness and usage). In

this case, we observe a downward trends in all cases. This may be due to the fact that many schools

do not have access to safe water within the school compound so that the sanitation facilities are not

well maintained. In addition, as almost all schools don’t have hired cleaners for cleaning toilets and

they use students to do so, and in some schools students are supposed to clean toilets as punishment

when they violate school regulations, it can hardly be expected that toilets are kept clean at all times.

3.2.3 Functionality of WASH club and analysis

Of the 50 surveyed school, 46 (92%) has a WASH club available. However, only 36 (72%) of the

schools have a functional WASH club. Functionality was defined on the basis of the fact that the

WASH club members come together at least once a month. We analyzed whether there is any relation

between functionality of the WASH club and access to safe water or access to improved sanitation


within schools. The statistical analysis (Chi-square and correlation at 5% level of significance) did not

show any significant relation (Table 15).

Table 15. The relationship between functionality of WASH-clubs and access to safe drinking water

and improved sanitation (absolute frequencies).

Functionality of WASH-club

Not functioning functioning

Access to safe

drinking water*

Yes 4 7

No 10 29

Total 14 36

Access to

improved

sanitation**

Yes 3 10

No 11 26

Total 14 36

* Relationship between access to safe drinking water and functioning of WASH-clubs is not

significant: Chi-square (df) = 1 , ns.

** Relationship between access to improved sanitation water and functioning of WASH-clubs is not


3.2.4 WASH related school management and analysis

Among the 50 surveyed schools, we also asked 8 questions to school management level to assess their

capacity of managing the WASH facilities. The eight questions have been grouped for rating the level

of management capacity as: poor (if the answers for two of the eight questions are positive); good (if

answers to three to five questions are positive); and very good (if answers for at least six of eight

questions are positive). Note that in 4 cases teachers responded ‘don’t know’ for some of the

management related questions, these were removed from the analysis.

Out of the 46 responses, 4 (9%) were considered to have poor WASH management capacity, 29 (63%)

fairly good, and 13 (26%) had very good capacity.

We analysed whether there is any relation between school WASH management capacity and access to

safe water or access to improved sanitation within schools. The statistical analysis did not show a

significant relation (Table 16).

Table 16. The relationship between WASH-management (by school management) and access to safe

drinking water and improved sanitation (absolute frequencies).

WASH-management by school management

poor good Very good

Access to safe

drinking water*

Yes 0 8 2

No 4 21 11

Total 4 29 13

Access to

improved

sanitation**

Yes 0 5 3

No 4 24 10

Total 4 29 13

* Relationship between access to safe drinking water and WASH-management by school

management is not significant: Chi-square (df) = 2 , ns.

** Relationship between access to improved sanitation water and functioning of WASH-clubs is not



3.2.5 School attendance and analysis

Among the 50 surveyed schools, we also asked in the survey if school attendance had improved in the

last 6 years. Out of the 50 schools, 47 (94%) gave a positive response, 3 (6%) gave a negative

response.

We analysed whether there is any relation between improved school attendance during the last 6

years and access to safe water or access to improved sanitation within schools. The statistical analysis

(Chi-square and correlation at 5% level of significance) did not show a significant relation (Table 17).

Table 17. The relation between improved school attendance during the last 6 years and access to

safe water or access to improved sanitation within schools

Improved school attendance

No yes

Access to safe

drinking water*

Yes 1 10

No 2 37

Total 3 47

Access to

improved

sanitation**

Yes 0 8

No 3 39)

Total 3 47

* Relationship between access to safe drinking water and improvement of school attendance is not


** Relationship between access to improved sanitation water and improved school attendance is not


3.3 Water schemes

3.3.1 Functionality of water schemes

The functionality of water schemes is based in 3 criteria: water discharge volume, water quality and

waiting time. We first present the findings on these 3 criteria.

Water discharge

The classification of water discharge volume per water scheme and per woreda is presented in Table

18. It shows that:

8% of the water schemes no water was coming out of the water point

12% of the water scheme had low water discharge or less water discharge during the dry seasons

80% of the water schemes had good water discharge.

Table 18: Functioning of water discharge per woreda

Water

discharge

Total Kedida

gamela

Alaba Misrak

Badawacho

Demboya Shashego Boloso

Sore

no 8% 57% 0% 29% 0% 10% 9%

moderate 12% 0% 0% 0% 20% 0% 0%

yes 80% 43% 100% 71% 80% 90% 91%

Total 100% 14% 10% 14% 20% 20% 22%


Water quality

The classification of water quality according to the respondents to the survey gives the following

results:

14% stated that water quality is poor (of which 4 cases – 8% - are obvious because of no discharge)

8% stated that water quality is less during the rainy season

78% stated that the water quality is good

Waiting time

The classification of waiting time per water scheme and per woreda is presented in Table 19. It shows

that:

Of all water schemes surveyed, for 38% the waiting time is 30 minutes or less, which is within our

criteria of a functional water source. Even though a large part of the water schemes were reported

to have good water discharge (80%), 63% of these schemes have a waiting time of > 30 minutes

which we considered as ‘partly functional’.

At the far end of the waiting time, 18% of the water schemes have a waiting time of more than 5

hours.

Looking at the waiting time for water schemes within woredas, waiting time is by far lowest in

Boloso Sore (all water schemes 30 minutes or less), and by far longest in Alaba (60% with more

than 5 hours waiting).

Table 19: Waiting time per water scheme, per woreda

Woreda % of

surveyed

schemes

Waiting time

30 min or

less

30 min-

2hr

2-5 hr > 5 hrs

Kedida Gamela 8% 33% 67% 0% 0%

Alaba 13% 0% 40% 0% 60%

Misrak Badawacho 13% 40% 0% 20% 40%

Demboya 20% 0% 50% 38% 13%

Shashego 23% 22% 22% 44% 11%

Boloso Sore 25% 100% 0% 0% 0%

Total 100% 38% 25% 20% 18%

Functionality

Based on the classification of the water schemes to assess functionality (see section 2.2.2) we found

the following results (Table 20). For easy comparison of relative scores per woreda, we attributed

values to each of the 3 categories: functional receives a score of 2, partly functional a score of 1, not

functional a score of 0. Table 17 shows that:

Few water schemes (24%) are fully functional, most (62%) are partly functional;

The highest scores on functionality of water schemes is found in Boloso Sore (1.37), followed by

Shashego (1.10); the lowest score is found in Kedida (0.88). All the others are in between with

almost the same score (1.0).

The most discriminating factor for water schemes to be partly functional is waiting time. There are

also 3 cases with a low discharge volume or no water during the dry season as well as having poor

water quality and a long waiting time. There are 3 cases with moderate or lower water quality

during the rainy season.

Among the water schemes that are not functional (7 cases or 14%), there are 4 cases with no

discharge volume, and there are 3 cases with poor water quality.

Note that several woreda’s report that water schemes are functional, but then observe that water

discharge is very low – we have integrated this finding in the functionality scores.


Table 20: Functionality of water schemes

Functionality

water schemes

Total Kedida

gamela

Alaba Misrak

Badawacho


Sore

Functional (2) 24% 29% 0% 29% 0% 20% 55%

Partly functional (1) 62% 43% 100% 43% 100% 70% 27%

Not functional (0) 14% 29% 0% 29% 0% 10% 18%

(score) 0.883

1.00 1.01 1.00 1.10 1.37

3.3.2 Performance of the WASH committees

Based on the classification of the WASH committees to assess performance (see section 2.2.2) we

have found the following results (Table 21).

Table 21: Performance of WASH Committees

Performance

WASH

Committees

Total Kedida

gamela

Alaba Misrak

Badawacho


Sore

Good 88% 88% 100% 100% 82% 100% 90%

Poor 12% 14% 0% 0% 30% 10% 9%

We can observe the following:

By far most WASH committees (88%) are performing well, as based on our set criteria. This means

the members are satisfied with the WASH committee, fees are charged, fees are used for O&M,

members are satisfied with the fees, and if there are conflicts, these are resolved;

Among the not performing WASH committees (12%), in 2 cases there was no WASH committee, in

2 cases the users were not satisfied with the WASH committee and in 2 cases no fees were charged

and conflicts did occur.

In terms of conflicts, in total 19 water schemes did not have any conflicts, in all other cases there

have been minor or major conflicts but these have been resolved (25) or not (4).

3.3.3 Preventive and Operational capacity training by SNV

During the survey we also asked for the perception of the users and WASH committee on the

progress on operation and maintenance of the water scheme during the last few years (Table 22). In

general almost 70% of the water schemes have made progress on Operation and Maintenance. More

than half of the progress has been made since 2008 and 16% continuously make progress.

As indicated in section 2.2.2, the 50 surveyed water schemes were evenly distributed over 4 distinct

categories: those constructed before and after 2007, and those with and without SNV training on

operations and maintenance received. Table 20 shows the relation between the perceived progress

and training on preventive operations and maintenance (POM) received with these 4 categories. We

did not carry out statistical analyses but compared to what extent the water schemes within a certain

category of progress in operations and maintenance are distributed over these 4 categories of water

schemes. It seems that the frequencies over the 4 different categories of water schemes is generally

3

For example the score for Kedida is composed as follows: (29*2) + (43*1) + (29*0) / 100 = 0.88


evenly distributed. Of the WASH committees that observed progress in POM, 56% did receive

training by the SNV project. Looking at the WASH committees that do not observe any progress, a

slight minority (43%) received SNV training, while the majority are older schemes.

Table 22: Progress in terms of Operations and Maintenance

Progress on

Operation and

Maintenance

Total Distribution of water schemes with different levels of operations

and maintenance over the 4 categories

Before 2007

No SNV POM

training

Before 2007

+ SNV POM

training

After 2007

No SNV POM

training

After 2007

+ SNV POM

training

Progress since

2008-2015

54% 19% 26% 26% 30%

Progress before

2008

2% 0% 0% 0% 100%

Continuous

progress

16% 25% 13% 25% 38%

No progress 28% 29% 36% 29% 7%

It can be concluded that if progress is made on POM there is no clear relation with ‘age’ of the water

scheme and whether SNV training has been provided on operations and maintenance issues. It must

be observed that many other agencies also provide training on operations and maintenance.

Table 23 shows the relation between the performance of the WASH Committee and the 4 categories

of water schemes (POM training and ‘age’). The distribution of scores is equally distributed over the

four categories of water schemes, but the number of not performing WASH committees is rather

small. It suggests that there is no clear relation between the performance of the WASH Committees

and preventive operational and maintenance training having been provided by SNV or by others.

Table 23: Relation between performance WASH Commitees and POM training

Performance

WASH

committee

Total Distribution of water schemes with different levels of

performance of WASH committee over the 4 categories

Before 2007

No SNV POM

training

Before 2007

+ SNV POM

training

After 2007

No SNV POM

training

After 2007

+ SNV POM

training

Good 88% 27% 27% 20% 25%

Poor 12% 17% 17% 35% 35%

3.3.4 Comparison with baseline situation

We will now compare the current situation with the baseline measurement of water schemes. For the

baseline situation, an assessment was made of three parameters:

The functionality of water schemes – probably implying whether there is water discharge

The management of water schemes – probably implying whether there is a WASH committee

managing the water scheme

The recovery of O&M costs by user fees – probably implying whether user fees are being paid.

Table 24 shows a comparison between the first two parameters and our findings.


Table 24: A comparison of baseline (2007) and current (2015) information on water schemes and

WASH committees

Functionality

water schemes

and WASH Co

Kedida

gamela

Alaba Misrak

Badawacho


Sore

2015 Functional 29% 0% 29% 0% 20% 55%

Partly functional 43% 100% 43% 100% 70% 27%

Not functional 29% 0% 29% 0% 10% 18%

Poor performing

WASH Co

14% 0% 0% 30% 10% 9%

2007 Not functional 14% 39% 33% 21% 53% 25%

No / poor

management

49% 50% 73% 52% 47% 40%

No cost recovery 35% 32% 97% 34% 57% 100%

From the comparison we can draw the following conclusions:

The proportion of non functional water schemes has declined between 2007 and 2015 for all

woredas, except for Kedida. However, it should be noted that functionality in the 2015 survey was

defined as no discharge (4 cases) or poor water quality (3 cases), whereas in the baseline survey

this probably only refers to no discharge. What has caused the improvement for Kedida is unclear.

The proportion of no or poor management or no cost recovery in 2007 is always (much) higher

than the proportion of poorly performing WASH committees in 2015. This means there is a

considerable improvement in terms of the performance of WASH committees.

3.4 Capacity building

As explained in section 2.2.3, the impacts on capacity building were evaluated at four different levels:

Woredas (8 woredas, of which 6 targeted and 2 control woredas), regional authorities

TVETC and UNICEF. These will now be treated subsequently.

3.4.1 Capacity building interviews and focus group discussions per woreda

As indicated in section 2.2.3, the capacity building assessments were analysed using a balance score

card approach, using a 3-point scoring system for current status (A, B, C), and trend over the last few

years (1, 2, 3) – see appendix 6.4 for detailed narratives of each woreda. The list of persons who were

interviewed or who took part in the focus group discussions is listed in appendix 2.2.

Part of the interview and focus group discussion was to ask for the contribution by SNV to any of the

observed changes. In some cases the answers were not affirmatively positive or negative, in which

case we noted unclear. We also noted some relevant quotes that characterize the situation. Following

are the results for each of the 8 woreda’s separately, Tables 25-30 including the 6 targeted woreda’s

and then Tables 31-32 the 2 control woreda’s.


Table 25: Overview of capacity building results Woreda Boloso Sore

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes Important improvements but now tendency to

retreat back and concerns about future

Financing 2 C yes Budget increased from 25,000 birr to 500,000 birr.

Awareness and priority has increased, but budget is

not enough.

Service delivery 2 C yes Response rate much improved, but too few staff to

manage 300 water schemes, currently only 2 staff on

maintenance, suggest a new structure

Incidence of diarrhea declined

Increasing population so demands

Gender aspects 3 C Not clear Gender participation has much improved last 7 years.

Percentage of women in kebele WASH committee at

least 50%. Much improved

Community

participation +

private sector

2 C yes Water schemes managed by communities

Private sector involvement of artisans

Improved functionality of water schemes

Data base +

monitoring

2 C yes Staff used MIS initially only

Needs specific budget for M&E WASH

Partnerships 2 B? Not clear Several NGOs have shifted their priority from WASH

to agriculture and food security

Table 26: Overview of capacity building results Woreda Misrak Badewacho

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes We used to have plans that were abstract, now much

better, with budget and responsibilities

Learned to carry out a good problem analysis

Financing 3 C yes Due to SNV training on fundraising we are able to

prepare our own projects

WASH budget has become top priority

UNICEF selected this woreda as a model

WASH budget still not enough

Service delivery 2 C yes Non functionality of water schemes has declined

from 32% to 12%

There is mainly transport shortage: 1-2 motorbikes

for all WASH staff

GLOWS has been very useful

Gender aspects 3 C partly Specially improved women and girls participation in

schools

Community

participation +

private sector

3 C partly There is a cooperative for spare parts supported by

another NGO, this has greatly contributed to

functionality of water schemes

Data base +

monitoring

2 C yes Staff uses MIS occasionally

The budget for M&E is very limited

Partnerships 3 C Not clear Support by NGOs is increasing in providing ‘full

packages’ e.g. schools with toilets and water supply


Table 27: Overview of capacity building results Woreda Kedida Gamella

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes Efforts to scale up the good practices to more

schools and households

Financing 2 C yes We have an operational budget only and it has

increased, but not for construction, here we depend

on donors

Several new schemes constructed recent years

Service delivery 2 C partly We lack electric and mechanical engineers, this

should be supplied by the region

The staff positions are not attractive

Gender aspects 3 C yes Gender participation has focused at schools

Collaboration with departments of women &

children affairs

Community

participation +

private sector

2 C Not clear There are traders who supply spare parts

There are currently 4 cooperatives producing slabs

and one enterprise supporting sanitation in schools

Data base +

monitoring

2 B yes Staff only used MIS once

Not enough staff and budget

Partnerships 2 ? Not clear

Table 28: Overview of capacity building results Woreda Shashego

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes Important improvement in collaboration within

woreda between health, education and water

Financing 2 C yes Budget and priority has increased but is not enough

for construction

The contribution by donors remains very important


Service delivery 2 C yes The spare-parts shop has been very useful, provided

by another NGO

Budget and staff for small mechanics is OK, but for

large mechanical issues it is missing

Gender aspects 3 C yes We follow the SNV WASH manual for women

participation

Women now feel ashamed if there is no latrine in

their household

Fetching water has changed from an open storage to

a closed jerry-can

Community

participation +

private sector

2 B unclear There are traders who supply spare parts

Two active artisans had to stop their business

because the woreda water office did not

acknowledge

Data base +

monitoring

2 C yes Staff used MIS only once

M&E is not a priority and there are no funds, but

some updates are made

Partnerships 3 B? Not clear Several NGOs are busy in WASH


Table 29: Overview of capacity building results Woreda Alaba

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes Important improvements in planning, especially by

collaboration between 3 core sectors

However, water coverage remains far too low

Financing 2 C yes Budget from the government has increased to 3

million birr, but inadequate for new constructions

mainly.


Service delivery 2 C yes Response rate much improved, but few staff

Functionality of water schemes much improved

Gender aspects 3 C Not clear Gender awareness and participation has much

improved.

However, there remains a serious water shortage,

reason why girls cannot go to school in some cases.

Community

participation +

private sector

2 C yes Much improved but some communities require

more support

Nor private sector providing spare parts

Data base +

monitoring

2 C yes Staff used MIS initially only

Data collection on WASH is continuing, we are up-

to-date with the data base

We receive lump sum for WASH, not specific M&E

Partnerships 2 B? Not clear

Table 30: Overview of capacity building results Woreda Damboya

Aspect

Scores Contribution

SNV

Quotes Status Trend

Strategic plan 3 C yes Important improvements but coverage is still

insufficient in large part of the woreda and for

several schools

Financing 2 C yes Donors provide software but the hardware is still

missing

Especially a problem for kebeles in mountain areas

Service delivery 2 B yes Response rate for maintenance improved

Plumbing and electronic experts are missing

On-the-job training has been good

There has been good training but recently declined

again

Gender aspects 3 C yes Gender participation in WASH committees has

much improved

Community

participation +

private sector

2 C yes Much more ownership developed among

communities, now reached 80-90%

Yet, some CBOs still do not have enough capacity

Data base +

monitoring

2 C yes Staff trained on MIS but did not use it

Data base on water schemes is up-to-date

No specific budget on M&E

Partnerships 2 B Not clear Several NGOs are not anymore active in the WASH

sector


Table 31: Overview of capacity building results Woreda Boloso Bombe (control woreda)

Aspect

Scores

Quotes Status Trend

Strategic plan 2 B No integrated woreda WASH strategic plan, but separate plans for

the 3 different units, no WASH sector coordination

This year the integration will take place

Access to water has improved in recent years

Financing 2 B Awareness on WASH is low, e.g. compared to malaria and

HIV/AIDS

Budget for WASH is only 90,000 birr, it slightly increased this year

No specific government budget for sanitation, this is all covered by

a donor.

Service delivery 2 B No on-the-job training on WASH

Quite a lot of community awareness raising activities took place on

hygiene and sanitation

CLTS approach has been adopted

Most households and schools now do have latrines, but their use

and maintenance is low and awareness is low

Lack of spare parts

Gender aspects 3 C Gender participation has improved last years.

Percentage of women in kebele WASH committee at least 50%.

Women and youth have priority in service delivery

Community

participation +

private sector

2 B Community participation in WASH has increased considerably,

especially since 2012

No private sector involvement

Data base +

monitoring

2 C No training on MIS

Training received on monthly data collection water scheme

functionality, which is currently being done

Very limited budget, staff and logistics for M&E

Partnerships 2 B Apart from government, 3 NGOs active in WASH sector


Table 32: Overview of capacity building results Woreda Angecha (control woreda)

Aspect

Scores

Quotes Status Trend

Strategic plan 2 B No integrated woreda WASH strategic plan, but separate plans for

the 3 different units, no WASH sector coordination

This is the situation in spite of support by AfD

Initiatives were taken to make kebeles ODF, but this initiative has

been retreated

Financing 2 B Priority and awareness on WASH is low, thus budget is also low

Budget is just enough for one hand-dug well per year

There is a high dependency on donors, both for maintenance and

for constructing new water schemes

Service delivery 2 B Since 2011 there has not been any additional water scheme

constructed, the communities are forced to do this by themselves

Received on-the-job training on WASH

Community awareness raising activities took place on hygiene and

sanitation

CLTS approach has been adopted

Most households and schools have latrines, their use and

maintenance is low and awareness is low; there is no proper system

for maintenance of water schemes

Lack of spare parts

Gender aspects 2 B Training on gender and development, but not on gender and WASH

Community

participation +

private sector

2 B Ownership and participation by community in WASH has increased

No private sector involvement

Data base +

monitoring

2 B Training on MIS was received but it was not used

M&E is only done for newly constructed water schemes

The data base for WASH sector is not updated

Limited budget, staff and logistics for M&E

Partnerships 2 B Since 2011 very little support by NGOs

Conclusions from comparing targeted and control woreda’s

Both the scores for current situation and the score for improvement trends during the last few

years are more positive for the 6 targeted woreda’s than for the 2 control woreda’s.

Most significant are the following conclusions and differences per aspect:

Strategic planning: the targeted woreda’s much appreciate integrated strategic planning

(involving health, education and water sectors). The SNV WASH project has contributed

significantly. The control woreda’s are aware of the risk of fragmented planning and also want to

move to a more integrated approach.

Finance: the targeted woreda’s have been able to attract more government and donor funding, as

a result of better planning and support in fundraising. The SNV WASH project has contributed

significantly. However, funds are still insufficient especially for large scale constructions and

maintenance of water schemes. Control woreda’s seem to remain more dependent on donor

funding.

Service delivery: the targeted woreda’s have been able to give more training on WASH awareness

raising as well as POM aspects, reason why water schemes are stated to be well maintained and

more functional than in the past. Also, at household level communities better maintain their

latrines. The response rate to repairs has increased. The SNV WASH project has contributed

significantly. However, service delivery on maintenance of large water schemes and large-scale


mechanical issues remains problematic even in targeted woreda’s because of limited staff, budget

and logistics. In the control woredas less new water schemes seem to have been constructed, and

there are more concerns about their maintenance and sustainability.

Gender awareness: there does not appear to be much difference between the targeted and

control woreda’s, even though some targeted woreda’s specifically mention training provided by

the SNV project and the use of their manual. Important has been the new regional regulation for

women to be nominated for leadership positions and at least 50% women in WASH COs and

other WASH activities.

Community participation and private sector involvement: in the targeted woreda’s communities

show sense of ownership and participation. Also, there are initiatives of private sector, e.g. in

spare part provision. The SNV WASH project has contributed significantly, although other NGOs

also play an important role. In the control woredas there are more concerns about community

ownership and less private sector involvement.

Monitoring and evaluation. Training on the MIS has not been followed up by further support,

and is therefore not anymore used. While some woredas state that the data base on WASH is up-

to-date, others say they have insufficient staff to maintain the data base.

Partnerships: most targeted woreda’s have many partners willing to invest in the WASH sector,

although some seem to have shifted their attention to agriculture and food security. This might

be explained by the fact these woreda’s have better and more integrated WASH strategic plans

and funding strategy. The control woredas have less donor and NGO support.

3.4.2 Interviews at regional level

From the interviews at regional level, the following main points can be extracted (the full interview

narratives are found in appendix 6.3).

1. Regional WASH Strategic Planning

In the past, efforts were scattered but there is now a joint approach by the ministries of water,

health and education. Under the integrated (‘one WASH’) approach there is more efficiency and

effectiveness in services. In the coming five years schools through the woreda education office

schools are entitled to construct their own latrines and water points.

For the education sector school improvement project, water, sanitation and hygiene is one of the

components and WASH is one of the quality indicators for schools.

The majority of schools have latrines constructed from locally available materials. Access to water

at schools remains challenging; sanitation and hygiene is difficult where there is limited access to

water. Schools without access to water are encouraged to look for alternative sources of water - like

collecting water from nearby springs, store using water tanks, use roof catchments for water

collection etc. There will be more roof catchment techniques to collect and store water.

A major change has been that from only construction to work in changing the behavior of the

community, in order to avoid that new water schemes are misused or not maintained.

During 2010-2014 the regional water coverage showed some improvement and reached 67%. But

the sector still faces a number of challenges:

limited number and low capacity contractors for water resource development,

limited number of private sector involved,

low government encouragement and motivation of staff, e.g. the WASH sector needs hydrologist,

geologist, electro mechanics.

There is need to change the structure of the water sector at region, zonal and woreda levels, using a

cascade structure, with three core processes (teams):

Water resources study and design


Water schemes construction supervision

Drinking water schemes equipments maintenance and administration

2. WASH sector Financing

Both Governments annual budget allocation and donors support are annually increasing

No budget used to be allocated before for sanitation and hygiene, this has much improved.

There remains limited capacity both at Zonal and Woreda levels. Human resources management is

a critical factor.

Water schemes construction are also planned for institutions-schools and health posts/centers.

3. WASH service delivery

About 95% constructed their own latrines though it might not be according to the expected

standard; there is a move from open field, to traditional pit latrine, to improved latrine.

There is need for continuous training at woreda level WASH support unit so as to enhance WASH

delivery services.

The majority of woreda staff are graduates from TVETC- water supply and sanitation, electro

mechanical, and water quality departments.

Before GLOWS intervention, the graduates were theory dominated and the majority of teachers

also lack practical aspects. With GLOWS and RiPPLE this has much improved, the regional water

bureau actively participated in design, training and joint assessment of the impact of the GLOWS

and reported to SNV. There were graduates who fear to go to rural water schemes, but after the

GLOWS intervention, the graduates were more motivated.

4. Community and private sector participation

Community participation: An important change has been made to encourage active

participation of the community. However, sometimes this does not work, possible reasons are:

No proper community mobilization and consultation is done,

Low level of expertise on community mobilization.

No proper local ownership is created.

In schemes where capacity building was done, O&M can be handled by the community but not for

major maintenance.

In schemes where construction was done without ownership development, communities claim

support even for simple issues.

Private sector participation: Private sector participation has much increased. Currently there

are groups who produce latrines from stone and supply slabs, and trained local artisans, but there

remains the need to organize them. There is work on spare parts supply; however it was found that

spare-parts sale only does not generate much benefit. Alternatives are being developed, such as:

cooperative supply of spare parts by allocating one shelf in their sales shop;

developing a town water service;

established outlet shop with some seed money supply;

working with traders for fast moving goods.

Recently the initiative has been taken to work on Guidelines for Private sector involvement in

Maintenance and spare parts supply.

5. Gender aspects:

Now it is recognized that women play a great role in water handling and utilization.

There is a new regional regulation for women to be nominated for leadership positions and at least

50% women in WASCOs and other WASH activities design and implementation.

The education sector gives attention to the school girls’ hygiene and sanitation.

There is a Gender Focal Unit under the Regional Water Bureau.


6. Data base, M&E

On MIS: there was a gap from regional side in continuity. After the responsible person in the region

for MIS WASH left, no new person was assigned. The use of the software faced some challenges.

There is a gap regarding joint evaluation and lessons learned with SNV, except for GLOWS. It

would be good to have an opportunity for joint review on progress, challenges and sharing of

lessons to scale up the good practices of the SNV WASH interventions.

At regional level there was responsible staff for WASH inventory. But after the person left, no

responsible staff or unit was assigned to take over regional WASH inventory.

Recently, in collaboration with RiPPLE a regional WASH resource centre was established.

There are staff, financial and logistics constraints to conduct regular M&E. There is a need to build

more capacity at woreda levels.

7. Partnership

Main partners and active currently: UNICEF, World Bank, AfD bank, JICA, SNV and RiPPLE.

Trends in support: The donors support is increasing from year to year.

More support is required on the hardware parts for WASH in schools; even though for hygiene and

sanitation more software is needed, it is better if donors also give attention to the hardware.

In the region, currently there are more than 15 partners working on hygiene and sanitation, but still

there are 53 woredas without any donor. In the future if all donors come in one system of support

there is possibility to utilize more efficiently the available support.

3.4.3 Interviews at the Hawassa TVETC

From the interviews at the TVETC, the following main points can be extracted (the full interview


Vocational training:

WASH training was introduced in 2010 to the water technology department through SNV.

About 21 graduates annually and in total 150 graduates benefited since the vocational training

project was introduced (for five years) for 7 woredas.

At initial stage it was as GDL one curriculum for testing. Later 7 curriculums were developed.

The curriculum is still in use, some additional improvement has been made:

Water treatment was added on the module

POM was also added to the existing WASH training curriculum, for one year but now discontinued.

Strengths of the SNV project:

SNV is seen as one of the strategic partners

SNV contributed to Teaching Training learning Materials preparation

Within a short period the water related curriculum was considerably improved

This contributed in making our college to reach higher level

This was a good input for improvement of the water department.

It gave direction for the water technology department

It helped to change the teaching from theory dominated to practical

Weaknesses:

At initial stage of implementation there was coordination problem but later it was solved in close

consultation with SNV and college administration.

Different types of materials including training inputs are required when implementing the

curriculum.


At woreda level there is low level of awareness and attention.

In some woredas there is an absence of graduates in the water sector.

Distance learning (GLOWS): was introduced in 2011. It is estimated over three years about 70

graduates benefited from the GLOWS project. The trainees were nominated by the regional water

bureau not by the college. Graduate employment places: woreda water offices, irrigation offices,

different industries including sugar factories etc. Its strengths are:

The approach is an improvement as compared to GDL

It helped to know the graduates performance as a feedback

The practical orientation in teaching is of learning significance

It filled the skill gaps of TVETC graduate experts on jobs in water sector

It helped to motivate the experts on the jobs

It includes to trace the graduates and identify their gaps

It created linkage with the community to work with and set the direction of the college

However, the application of GLOWS is costly. Vehicles, DSA, professional fee is required for the team

field supervision, e.g. per annum 4 supervisions are supposed to be conducted. Skilled experts are

required for training and field supervision. Without donors support it is difficult to implement.

GLOWS is still being applied, but the application depends on the available budget. It can’t be applied

by itself. No information about the government plan to finance the project.

Another challenge the project faced was accreditation. The graduates want the certificate to get

recognition by their employers in promotion to higher level. This requires government commitment.

There has been discussion on this issue but it is still not resolved.

The finance is managed by the college and the college provides training as well as pays the college

staff for the training they provided for the trainees coming from different organizations. However,

the service user organizations should pay for the services provided. It is better if SNV gives the

trainees finance for the college water sector utilization rather than paying for trainees.

Improved graduates performance: GLOWS served the college to know its graduates situation in

the field. During implementing GLOWS the college teaching staff got an opportunity and was able to

observe the gaps and performance of its graduates. After WASH and GLOWS the graduates’

performance has highly improved both in technical capacity and motivation for more works in the

community.

Other TVETCs that adopted GLOWS are: Soda and Dila TVTECs from SNNPR, TVETCs in Oromia

and Amhara regions.

3.4.4 Interviews at UNICEF – Hawassah branch

From the interviews at UNICEF, the following main points can be extracted (the full interview


Strategic partnership with SNV. SNV and UNICEF collaboration started some years back, with a

new agreement made in 2011. Advantages of the collaboration are:

Mutual complementarity; SNV introduce has an added value in WASH interventions.

SNV working on the software component has advantage with regard to sustainability of the

hardware.

Disadvantage: There is a joint relation to finance and capacity, a small number of woredas was

reached, there is need in the future to identify ways to replicate.


Improvement in collaboration. A national level study shows that 25-40% water schemes are

non-functional. It has been shown that this can be improved if the water schemes are managed and

owned by the community. The collaboration with SNV has generated as results:

Schemes become stronger in financial management

POM training brought change in functionality of schemes.

Effective elements: Software and hardware components feed each other. Missing one of the

components doesn’t bring change. SNV working on software component is a great change.

WASH service delivery. There is some improvement in both the water coverage and utilization,

according to the national inventory that was done in 2012.

Up scaling effectiveness: There are a number of factors determining the potential for scaling up

government and partners’ capacity. A major problem is high staff turnover.

Community and private sector participation

Water: maintenance of water schemes is still minor in majority of the target woredas , but there is

an increase in contribution of finance which is an indicator of increasing ownership.

Sanitation: nowadays in most places a household latrine exists; it may need to be improved.

Schools: there is still a problem in terms of access to water supply hardware.

Private sector participation: The private sector plays an important role in WASH. It is possible to

work both software and hardware components, e.g. in small schemes construction, sanitation

facilities, spare-parts supply. Hardware parts can be supplied by local private sectors like artesian

and cooperatives in the community.

3.4.5 Conclusions from the regional level interviews

From the above interviews and focus group discussions, the following main conclusions emerge:

A change during the last years has been one from the focus at constructions towards behavioural

change on sanitation mainly. The woredas have more budget to spend on water schemes and

sanitation and hygiene. As a result the population is more aware, have latrines, and these are better

maintained. Most households now have latrines, these are not always the improved ones.

The integrated approach (education, health and sanitation) to WASH has approved. It has led to a

focus at water & sanitation within schools.

Access to water within schools remains a challenge and schools are encouraged to use alternative

sources, such as collecting from nearby springs, storage tanks and rainwater harvesting.

Service delivery by public agencies on WASH has much improved and the practical training

(GLOWS) has contributed to expertise and staff motivation. However, there are still complaints

about high levels of staff turn-over, staff motivation and lack of qualified hydrologists, geologists

and electro mechanics. These are external factyprs beyond the reach of donors such as SNV.

The collaboration with SNV has shown that operations and maintenance of water schemes can be

much improved (at national level 25-40% of water schemes are not functional). However,

maintenance of deep bore hole water systems is still a challenge.

The private sector plays an increasing role in the WASH sector. It constructs latrines and provides

spare parts for water schemes, but also does not address major maintenance issues.

The management information system (MIS) is not being used due to poor continuity by regional

level and problems with software.

At regional level it is felt that continuity and follow-up evaluation and learning by SNV is missing.

More (donor) support is required on the hardware aspects of WASH, especially for schools.

UNICEF mentions that too few woredas were reached in terms of hardware.


4. Analysis and conclusions

4.1 Summary of main findings

4.1.1 Comparison of woredas

We now present an overview of the main findings per woreda, with the aim to recognize relations and

patterns. Table 33 shows subsequently:

The current survey (2015) findings on access to water per woreda and the most likely trend

emerging from the comparison with the 2008 baseline values (see for uncertainties with respect to

the baseline survey values previous chapter table 3);

The current survey (2015) findings on access to improved sanitation per woreda and the most trend

emerging from the comparison with the 2008 baseline values (see for uncertainties with respect to

the baseline values previous chapter table 6);

The functionality of water schemes, using an index score

The functionality of WASH Co performance, using an index score.

Table 33: Data and trends on main findings at household level and with respect to water schemes.

Woreda


Access to improved

sanitation

Functionality

water scheme

(index score)

Performance

WASH Co’s

(index score) % 2015* Trend 2008* % 2015 Trend 2008

Alaba 20.4 Down 32.1 up 1.00 1.00

Boloso Sore 52.3 up 39.5 up 1.37 0.91

Demboya 17.3 Down 30.0 stable 1.00 0.70

Kedida 69.5 Up 56.7 up 1.01 0.86

Misrak B. 70.3 Down 44.0 up 1.01 1.00

Shashego 54.7 Down 48.1 up 1.10 0.90

All 46.4 down 41.4 up 1.10 0.88

* Values in 2015 take into account three criteria including distance to the nearest water scheme and

clean water storage, most likely similar to the 2008 baseline survey

Following are conclusions and explanations when comparing the different values per woreda:

Access to water has declined in most woredas, except in Boloso Sore and Kedida,

from an average of 53% in 2008 to 46.4% in 2015. We assume that the baseline survey also

took into account distance (not more than 1.5 km) or travel time to the water scheme (not more

than 30 minutes). When doing so in this survey, average access to water is 46.4%, which is the most

correct and realistic value. Access to water has improved in Boloso Sore and in Kedida. Access to

water when taking into account distance to water is especially low in Alaba and Demboya. Due to

great distance to water, currently travel time was more than 30 minutes for 40.7% of the

respondents, significantly affecting access to water.

According to the Ethiopia Demographic and Health survey of 2011, 13% of households reported

having water on their premises, 36% in rural areas have water within 30 minutes walking, and 62%

in rural areas have to spend more than 30 minutes walking. This means that, when looking at

distance to nearest water scheme, the data in the survey area of our study are somewhat better.

Access to improved sanitation has improved in most woredas, from an average of

27% to 41.4%. In our survey 72.4% of the respondents state that they have access to improved


sanitation, but when observing the latrine facility it turns out that for only 49.4% of the

respondents the latrine is of an improved type, and when taking into account cleanliness the value

comes to 41.4%. When comparing this with the 2008 baseline on improved sanitation, there is an

improvement in 5 of the 6 woredas and only in Demboya there is no change.

There is a correlation between access to safe water and access to improved

sanitation. Figure 1 shows the significant relation between access to safe water and access to

improved sanitation. This relation is in conformity with the results of focus group discussions

which indicate that access to water influences whether latrine facilities can be properly used.

Figure 1: relation between access to safe water and access to improved sanitation in 6 woredas.

There is a relation between access to water (and access to improved sanitation) and functionality of

water schemes. Functionality of water schemes shows variation between woredas, with Boloso Sore

scoring highest and Alaba and Demboya scoring lowest (table 25). The performance of WASH Co’s

also shows variation, with Alaba and Misrak Badewacho scoring highest, and Demboya scoring

lowest. We observe low scores for all indicators at Demboya. Alaba also has low scores on all

indicators except for WASH Co performance. This suggests that poor functionality of water

schemes leads to low scores on access to water (and thus also to access to improved sanitation).

4.1.2 School level findings

Table 34 gives an overview of the findings on access to water and access to improved sanitation for

schools, per woreda, with trends on the comparison between the 2008 baseline and current survey.

Table 34: Data and trends on main findings in schools.

Woreda


(2015)

Access to improved

sanitation (2015)

(%) Trend (%) Trend*

Alaba 7.7 up 23.1 Up / down

Boloso Sore 12.5 up 25.0 Up / down

Demboya 33.3 up 16.7 Up / down

Kedida Gamella 40.0 up 20.0 Up / down

Misrak B. 25.0 up 37.5 Up / down

Shashego 30.0 up 30.0 Up / down

All 22.0 up 26.0 Up / down

y = 0,3785x + 23,786

0

10

20

30

40

50

60

0 20 40 60 80

Access to safe and clean sanitation

Access to water


* Before and after the slash: without / with criteria of maintenance, regular use and security,

respectively

Following are conclusions and explanations when comparing the different values per woreda:

Access to water has considerably improved for schools in all woredas, although the average is still

rather low (increase from 7% to 22.0%). Access to water implies that there is a water tap or source

within the school compound. Several schools also have access to water outside the school

compound (30%). This means that in total 48% of the schools surveyed do not have access to water

within or outside the school.

Almost all schools have a latrine facility (94%) but the proportion of latrine facilities that is also

clean is much lower (26.0%). When we compare data on safe latrine facility with the baseline value,

we can observe a considerable improvement (positive trend). This may be explained by the fact that

almost all schools nowadays have improved latrine facilities. However, when we look at the

indicator on improved sanitation (taking into account cleanliness and usage) the situation is less

positive and comparing the baseline value (49%) with current value (26%) shows a decline. This

may be a consequence of the fact that many schools do not have access to safe water within the

school compound so that maintenance of the sanitation facilities may be difficult, but it may also be

due to lack of supervision by school teachers. We suspect that the 2008 baseline data did not

adequately take into account cleanliness of the latrines, so that the decline may be less than

suggested.

It is uncertain whether there is a relation between access to water and access to

improved sanitation within schools. The survey data do not show a significant relation

between access to water and improved sanitation within schools. This finding seems to be in

contrast with the results from focus group discussions, which suggest that many schools have

latrine facilities but cannot use them properly because they do not have access to water. However,

the absence of this relation may be due to the fact that schools that do not have access to water

within the school premises, are able to acquire access to water from outside the school or acquire

water through trucks and storage in water tanks. Also, the numbers are small so that a significant

relation will be difficult to acquire.


4.2 Analysis of impact pathways

In the inception phase, impact pathways were identified, with indicators and assumptions to be

validated. We now analyse to what extent these impact pathways can be validated by our quantitative

data and qualitative insights from focus group discussions.

4.2.1 Impact pathway 1. Household access to improved sanitation

Input Output Outcomes Impact

Ob

jectives / iIn

dica

tors

1. Households trained on CLTS, by woreda staff

2. HH received support and information on water & sanitation 3. Awareness raised among HH water users on improved sanitation and safe hygiene practices

4. HH construct sanitation facilities 5. HH maintain sanitation facilities and use these in a clean and safe way 6. HH access to improved sanitation

7. Reduced incidence of diarrhea and other water-borne diseases

Assu

mp

tion

s

a. Woreda staff capacitated on sanitation by the SNV project

b. HH are accessible and open to receive information and support in WASH

c. Access to water for households d. Households have means to construct sanitation facilities

Figure 2: Results on impacts pathway for household access to improved sanitation

Changes between baseline (2008) and current (2015) [numbers and letters refer to Figure 2]

1. Woreda water and sanitation (WASH) and health officers raised awareness on the need for

WASH using the Community Led Total Sanitation (CLTS) approach. All households have been

reached in 6 woredas.

2. Over the last 6 years 74.1% received both information and support on WASH, 9% received either

of the two and 8% none at all.

3. Currently, 72.4% are aware of the need of improved sanitation.

4. Currently, 49.4% have improved latrine facilities.

5. Currently, 44.2% have improved latrine facilities and also keep them clean.

6. Currently, 41.4% have improved latrine facilities, keep them clean and use them regularly so that

their compound and surroundings are clean (‘open defecation free’). This is a considerable

improvement as compared to the baseline in 2008 (27%).

7. Currently, the incidence of diarrhea over the last year is 17%. This is a considerable

improvement from 52% in 2007.

Assumptions

a. Woreda staff was capacitated by SNV, as well as other NGOs (but SNV was most important)

b. All households can be reached by woreda extension staff.

c. Of surveyed households, 46.4% have access to safe water. This is most likely a decline as

compared to the baseline in 2008 (53%), which is mainly the result of long distances to, and

waiting times at water schemes.

d. Construction of improved latrines is possible even for low-income households.

We conclude that the situation has improved in many respects. But what can we conclude about the

contribution by the project and the causality of these changes?

A+

C+

B+ D+


Causality and contribution analysis [letters refer to items in Figure 2]

A. There is a positive relation between woreda staff capacitated to provide CLTS extension (a) and

their performance in providing services on WASH to households (1). This was found when

comparing the results of capacity building among woredas targeted by the project with the

control / non-targeted woredas. The comparison shows that the targeted woredas score

consistently better on WASH related capacities, as a result of better (on-the-job) training mainly.

B. There is a significant positive relation between access to improved sanitation (2) and information

and training support received on WASH (6). This causality is supported by the results of FGDs,

supporting the usefulness of a shift to awareness raising and behavioural changes.

C. There is a significant positive relation between access to improved sanitation (6) and access to

safe water (c). This causality is supported by the results of FGDs, and will result mainly from the

fact that water schemes have been better maintained.

D. There is a significant positive relation between access to improved sanitation (6) and incidence of

diarrhea (7). This causality is supported by the results of FGDs. The relation cannot be fully

understood by the actual presence of latrines but also by the improved behavior of users.

The causality B is further supported by the analysis of underlying factors, showing that there is a

significant positive relation between access to improved sanitation and young age of household head

(younger age scoring better). This suggests that young family heads more easily change their

behavior. There is also a positive relation with family size, suggesting that larger families change their

behavior more easily. There are no significant relations between access to water and household

characteristics, suggesting that the main causal factors are beyond the household.

We conclude there are significant correlations and plausible underlying causal explanations (based

on interviews and/or focus group discussions) for linkages between elements of this impact chain.

This implies that this impact pathway is validated: the contribution by the SNV project is plausible.

.

4.2.2 Impact pathway 2. Schools access to water and improved sanitation

Input Output Outcomes Impact T

arg

ets / ind

icato

rs

1. School teachers and pupils trained on CLTS 2.WASH clubs are formed and supported

3. Among school teachers and pupils increased awareness on clean and safe water use and sanitation facilities 4. School management capacity to support WASH in schools 5. WASH clubs are functional

6. Schools construct latrine facilities 7. Schools assure sanitation facilities are managed in a clean and safe way

8. Improved school attendance

Assu

mp

tion

s a. Woreda staff capacitated on sanitation training schools

b. Schools can be reached to raise awareness on WASH

c. Access to water at school compounds

Figure 3: Results on impact pathway for schools access to water and improved sanitation

Changes between baseline (2008) and current (2015) (numbers and letters refer to Figure 3]

A+

D?

B?

C?


1. Woreda water and sanitation (WASH) and health officers raised awareness among school

students and teachers on the need for WASH. Many schools have been reached in 6 woredas.

2. In addition, at all schools WASH clubs have been formed.

3. Over the last 6 years 74.1% received both information and training support on WASH, 9%

received either of the two and 8% none at all.

4. Of the surveyed schools, 63% have fairly good and 26% has very good WASH management

capacity among the teachers.

5. Currently, 72% of schools have WASH clubs that are functional.

6. Currently, 94% of the schools have a latrine facility, which is higher than the baseline value of

49%. This can be explained by the fact that many latrine facilities have been constructed

following integration of the education sector in WASH.

7. Currently, 28% of the schools have latrine facilities that are also clean and safe, which is lower

than the baseline value of 49%. This can be explained by the fact that very few schools have

access to water within their compounds.

8. Currently, in 94% of the schools, attendance has improved over the last 6 years.

Assumptions


b. All schools can be reached by woreda extension staff.

c. Access to water within the school compounds is 22%. This is a considerable improvement

compared to the baseline situation (7%). Several schools also have access to water outside the

school compound (30%). Woreda’s claim higher levels of access to water by schools than 22%,

which is probably due to the fact that external sources of water (beyond the school) are

included.

We conclude that the situation has improved in many respects. But what can we conclude about the

contribution by the project and the causality of these changes?


A. There is a positive relation between woreda staff capacitated to provide WASH extension (a) and

the capacities at schools on WASH (1). This was found when comparing the results of capacity

building among woredas targeted by the project with the control / non-targeted woredas. The

comparison shows that the targeted woredas score consistently better on WASH related

capacities, and was supported by the results of interviews at regional level.

B. There is not a significant relation between WASH management capacity in schools (4) and access

to water or access to improved sanitation within schools (7). This may be explained by results

from the FGDs at regional level, revealing that access to water within schools remains the

dominant problem, even if schools use alternatives to access water.

C. There is not a significant relation between the functionality of the WASH clubs (5) and access to

water or access to improved sanitation within schools (7). The explanation is similar as above.

D. There is not a significant relation between school attendance (8) and access to water or access to

improved sanitation within schools (7). However, the results of FGDs provide plausible evidence

that there is such a relation in several woredas.

We conclude there are no significant relations and underlying causal explanations (based on

interviews and/or focus group discussions) for linkages between elements of the impact chain. This

implies that this impact pathway is not validated. However, a contribution by the SNV project is

plausible, given the available evidence from interviews and focus group discussions.


4.2.3 Impact pathway 3. Improved management of water schemes


Ta

rgets / in

dica

tors

1.Local WASH committees are supported and trained

2. Awareness raised among water users and WASH co on operation and maintenance 3. WASH Co’s are performing well and water users pay fees

4. Water schemes are well maintained. 5. Water schemes remain functional

6. Improved access to water for water users

Assu

mp

tion

s

a. Woreda staff capacitated on O&M b. Water schemes are constructed

c. WASH Co’s receive support by woreda officers d. Water users have capacity to pay

e. Water scheme can be maintained in time f. Water reserves, rainwater and climate change

Figure 4: Results on impact pathway for improved management of water schemes

Changes between baseline (2008) and current (2015) (numbers and letters refer to Figure 4]:

1. SNV has directly supported the training of WASH committees, and woredas have trained more.

All water schemes that were surveyed have WASH committees.

2. The survey showed that WASH committees are well aware of the need for operations and

maintenance.

3. By far most WASH committees (88%) are performing well, as based on our set criteria. This

means the members are satisfied with the WASH committee, fees are charged, fees are used for

O&M, members are satisfied with the fees, and if there are conflicts, these are resolved.

4. Of the WASH Co’s, 72% stated that operations and maintenance of the water schemes had

improved over the last years.

5. Based on the classification of the water schemes to assess functionality, it appears that few water

schemes (24%) are fully functional, most (62%) are partly functional. The most discriminating

factor for being partly functional is waiting time. Of all water schemes, 63% have a waiting time

of > 30 minutes which is considered as ‘non- functional’, of which 18% have a waiting time of

more than 5 hours. This means that even if the water scheme provides water, and the water is of

good quality, the volume of water discharge is often very low. This finding is confirmed by results

of the FGDs, especially those at regional level. Note that several woreda’s report that water

schemes are functional, but then observe that water discharge is very low – we have integrated

this finding in the functionality scores.

6. Of surveyed households, 46.4% have access to safe water, and thus have water available to use

the latrines. This is most likely a decline as compared to the baseline in 2008 (53%), which is

mainly the result of long distances to, and waiting times at water schemes.

Assumptions


b. Few new water schemes have been constructed (information from woredas and UNICEF)

c. Woredas provide services to WASH Co’s, but expertise is missing on hydrology, geology and

electro mechanics

d. Users have little capacity to pay for water use

e. The maintenance of water schemes has much improved during the last time, the time it takes

before a reported failure is repaired has considerably reduced (results of FGDs)

f. We do not have information on water reserves and rainfall data, but given the low water

discharge of many water schemes it appears that reserves are low.

A+

B? C? D?


g. All schools can be reached by woreda extension staff.

We conclude that the situation has improved in many respects, but many water schemes remain

partly functional because of the long waiting times due to poor relation between water discharge and

water demand. Poor water discharge may be due to limited water reserves and recharge from rainfall.

The number of water schemes remains too low. But what can we conclude about the contribution by

the project and the causality of these changes?


A. There is a positive relation between woreda staff capacitated to provide WASH extension (a) and

local WASH committees (1).

B. We did not find a relation between performance of the WASH Co’s (3) and whether POM training

had been provided by the SNV project (1), or with the ‘age’ of the water scheme.

C. We did not find a relation between perceived progress in maintenance (4) and performance of

the WASH committees (3).

D. We did find plausible evidence that higher functionality of water schemes (5) leads to more

access to water (6). The main determining factor is probably the waiting time at the water

scheme.

We conclude there are no significant relations for linkages between elements of the impact chain.

The interviews and results of FGDs show that this is mainly due to the following main factors. First,

many water schemes provide too low volumes of water in relation to water demand, which affects

their functionality. Second, too few new water schemes have been constructed. Third, there is the fact

that the lifespan of facilities is 15 -20 years and old facilities cannot serve current populations growth.

In absolute numbers the coverage increases but not in terms of % of demand being served.


4.2.4 Impact pathway 4. Capacity building of institutions responsible for WASH


Ta

rgets / in

dica

tors

1. Training of woreda and regional WASH officers 2. On-the-job training of WASH graduates

3. Awareness raised among Woreda and regional officers 4. Improved skills of WASH graduates

5. Improved WASH planning 6. Improved financial resources for WASH in woredas 7. Improved service delivery on WASH 8. Improved community participation

9. Improved access to water

Assu

mp

tion

s

a. Woreda staff motivated to improve WASH capacities b. Regional staff motivated to improve WASH capacities

c. Woreda and Region have capacity & budget for WASH d. Woreda and Region have staff available for WASH

Figure 5: Results on impact pathway for capacity building of institutions responsible for WASH

Changes between baseline (2008) and current (2015) (numbers and letters refer to above figure 5]:

1. SNV has directly supported the training of Woreda and regional level WASH officers.

2. SNV has supported Awassa TVETC and GLOWS on-the-job training of WASH graduates, this has

reportedly been much appreciated.

3. The results of FGDs show increased awareness at woreda and regional level on WASH related

aspects.

4. The results of interviews show evidence of improved skills of WASH graduates.

5. The results of FGDs show improved WASH planning skills within woredas targeted by the

project, as compared to control woredas. The SNV WASH project has contributed significantly.

The control woreda’s are aware of the risk of fragmented planning and also want to move to an

integrated approach.

6. The results of FGDs show that targeted woreda’s have been able to attract more government and

donor funding, as a result of better planning and support in fundraising. The SNV WASH project

has contributed significantly. However, funds are still insufficient especially for large scale

constructions and maintenance. Control woreda’s seem to remain more dependent on donor

funding.

7. The results of FGDs show that targeted woreda’s have been able to give more training on

awareness raising as well as POM aspects, reason why water schemes are stated to be well

maintained and more functional than in the past, and at household level communities better

maintain their latrines. The response rate to repairs has increased. The SNV WASH project has

contributed significantly. However, service delivery on maintenance of large water schemes and

large-scale mechanical issues remains problematic even in targeted woreda’s because of limited

staff, budget and logistics. In the control woredas less new water schemes seem to have been

constructed, and there are concerns about their maintenance and sustainability.

8. The results of FGDs show that in targeted woreda’s communities show sense of ownership and

participation. Also, there are initiatives of private sector, e.g. in spare part provision. The SNV

WASH project contributed significantly. In the control woredas there are more concerns about

community ownership and less private sector involvement.


9. We do not have data on access to water in control woredas so this comparison cannot be made.

Assumptions

a. Woreda staff was found to be motivated to be involved in the SNV training, but there are also

indications that currently staff is not sufficiently motivated and turn-over is high. This may be

due to limited funds, means of transport, and low investments in new water schemes.

b. At regional level staff motivation was not evaluated. However, regional staff on MIS has left

which is the main reason for lack of continuity on monitoring using MIS.

c. The budget for WASH investments is reportedly too low.

d. There is shortage of staff on specific subjects related to WASH, being geologist, hydrologist and

electro-mechanics. These issues mainly relate to the construction of new water schemes and

maintenance of large water schemes.

We conclude that the situation has improved in many respects, but capacities remain low as related

to the construction of new water schemes and the maintenance of large water schemes. This may

largely explain why access to water remains too low.

We do not have additional information on causal relations in the above scheme.

We conclude there is plausible evidence for positive changes and relations in the above impact chain.

However, there are remaining challenges which are mainly related to the following factors:

Need for additional hardware, especially new water schemes, operations and maintenance of large

water schemes, especially to increase water discharge;

Need for improved motivation of staff, probably mainly due to available funding, skills and

mechanics;

Continuity in follow-up, joint learning and evaluation of progress by SNV.

4.3 Evaluation questions

We now answer the evaluation questions based on the above analyses. To do so, a summary and

selection has been taken from the set of evaluation questions (see Annex 2).

Impact

When taking into account distance to the water scheme (not more than 1.5 km) or travel time to the

nearest water scheme (30 minutes), as well as cleanliness of water storage, access to water for

households in the project area has declined from 53% in 2008 to 46.4% in 2015. Currently travel

time was more than 30 minutes for 40.7% of the respondents, which is still better than the average

of 62% for rural areas in Ethiopia, according to the Ethiopia Demographic and Health survey of

2011. Most likely, without the project access to water would have been lower because we found that

of the majority (72%) of water schemes in the project area functionality has improved.

The project has made a significant contribution to access to improved sanitation within

households, which has increased from 27% to 41.4%. We conclude there is a significant relation and

plausible contribution by the SNV project to the improvement in access to improved sanitation (see

impact pathway figure 2). For instance, there is a positive relation with access to information and

training support on WASH, and also with younger age of the household head. This shows that

behavioural change on sanitation and hygiene has taken place, especially among younger people.

There is also a significant correlation between access to safe water and access to improved

sanitation for households, suggesting that access to safe water influences improved sanitation.


The decline in the incidence of diarrhoea is highly significant (from 52% to 17%), and is most

significantly related to the improved access to improved sanitation within households. It cannot be

excluded that other factors also play a role, but improved sanitation has certainly contributed to

less incidence of diarrhoea.

Concerning the schools, access to water in their own premises has significantly improved (from 7%

to 22%). The majority of schools do not have access to water in their own premises, but many have

access to water beyond the school (30% of the schools). Schools are encouraged to use alternative

sources of water, such as collecting from nearby springs, storage tanks and rainwater harvesting.

Within the schools 94% have a latrine facility, but the score of access to improved sanitation

declines to 28% if we also taken into account aspects of cleanliness and safety, which seems lower

than the baseline value of 49% (but it is uncertain to what extent these criteria were used during

the baseline survey). Low access to improved sanitation (including its use, cleanliness and safety)

could be explained by low access to water within the school compounds as well as limited

management by teachers of WASH within schools (only 26% scores very good). Most student-based

WASH clubs (72%) are functional.

We have not been able to conclude that there is a significant relation by the SNV project to access to

improved sanitation in schools (see impact pathway figure 3), but a contribution by the SNV

project is plausible, given the available evidence from interviews and focus group discussions.

For the majority (72%) of water schemes in the project functionality has improved. The SNV

project has contributed to this improvement, by training on operations & maintenance and

supporting local WASHCO’s. However, for 70.6% of the respondents waiting time at the water

scheme was more than 30 minutes. Based on the long waiting time at water schemes, we classify

the majority of schemes (62%) as partly functional. We have not been able to conclude that there is

a significant relation by the SNV project to improved functionality of water schemes (see impact

pathway figure 4), but a contribution by the SNV project is plausible, given the available evidence

from interviews and focus group discussions.

The interviews and results of FGDs show that low functionality of water schemes is mainly due to

the following main factors. First, many water schemes provide too low volumes of water in relation

to water demand, which affects their functionality. Second, too few new water schemes have been

constructed. Third, there is the fact that the lifespan of facilities is 15 -20 years and old facilities

cannot serve current populations growth. In absolute numbers the coverage increases but not in

terms of % of demand being served.

With respect to the contribution by SNV, this is positive in relation to change of behaviour on

sanitation and hygiene at household level, improved maintenance of water schemes, build up of

capacities at woreda level, among the communities, WASH committees and WASH clubs at

schools.

It seems that the main remaining problem is the fact that in the 6 woredas and in the region as a

whole, available water is too low in relation to the needs of the local populations. The number of

new water schemes constructed has been too limited in recent years, because funds and capacities

for constructing new water schemes are limited, and also the water reserves may be limited. Wells

are already very deep.

Effectiveness

Opinions of woreda staff are positive with respect to the services being provided by the project.

At woreda level, strategic planning, service delivery, community involvement and monitoring have

much improved. Gender involvement has improved due to implementation of the national policy

mainly. The targeted woreda’s have better capacities for WASH strategic planning and WASH

service delivery than control woreda’s. These are systemic changes that will remain after the

project. Other (non targeted) woreda’s claim that they would like to receive similar support as they

see the advantages of these changes.


The model of training-on-the–job (GLOWS) has been very effective in terms of improving service

delivery. The response rate to repairs has increased. The GLOWS training and the SNV WASH

project contributed significantly.

Use of the MIS has failed due to lack of support and continuity at regional level. In a general sense,

monitoring remains a weak point. The recent regional WASH survey is of mediocre value and the

data could not be used for comparison purposes.

There are a number of external factors that influence staff performance within woredas and

regional WASH agencies, being high levels of staff turn-over, staff motivation and lack of qualified

hydrologists, geologists and electro mechanics. Although financial resources available for WASH

have increased, and are higher in targeted woreda’s than control woreda’s, there is still insufficient

funding available.

We conclude there is plausible evidence for positive changes and a contribution by the SNV project

to improved capacities on WASH in woreda’s (see impact pathway figure 5). However, there are

remaining challenges which are mainly related to the following factors:

Need for additional hardware, especially new water schemes, operations and maintenance of

large water schemes, especially to increase water discharge;

Need for improved motivation of staff, related to available funding, skills and mechanics;

Continuity in follow-up, joint learning and evaluation of progress by SNV.

Sustainability

From an institutional angle, several of the capacity building changes at woreda level are structural

(systemic) changes that will last. For instance, there is wide acknowledgment of the usefulness of

integrated strategic planning (education, health and WASH). Also, there is firmly established

insight that for improvement of sanitation and hygiene the change of behaviour is at least as

important as the hardware. Service delivery has improved and private sector initiatives are

emerging, which seem to be processes that will continue.

From a local financial and maintenance angle, most local water schemes are well maintained with

WASH committees (88%) performing well, as based on our set criteria: the members are satisfied

with the WASH committee, fees are charged, fees are used for O&M, members are satisfied with the

fees, and if there are conflicts, these are resolved. Also, the time it takes before a reported failure is

repaired has considerably reduced.

From a regional financial angle, the funding from national sources to the WASH sector has

improved for the 6 selected woreda’s. Also, there is some evidence that the targeted woreda’s are

better able to access funds from donors for investment objectives, presumably because of their

improved planning and funding requests. The control woreda’s have less donor and NGO support.

Nevertheless, funds for investments in new water schemes and large-scale maintenance of existing

water schemes remain insufficient. Lastly, although WASH training of graduates will most likely

remain, the GLOWS approach is reported to be costly and still not formally recognised, and

therefore uncertain to be continued. Altogether, there are insufficient recurrent financial inputs to

the WASH sector in the region, e.g. through fees or taxes.

From an ownership point of view, in the targeted woreda’s there is more community ownership

than in the control woreda’s (e.g. through the WASHCo’s).

From a private sector point of view, there is more private sector involvement than in the control

woreda’s, which will also contribute to sustainability as it shows that a business case may be there

(e.g. for spare parts of water schemes maintenance).

From an environmental angle, it is worrisome that water boreholes have to go deeper and yet

cannot supply sufficient water for the whole population in the region. This may lead to depletion of

underground water resources. More attention could be given to rainwater capture and/or

underground storage. There may also be need for collaboration and integration with natural

resources and watershed management for improving vegetation coverage around water sources so

as to improve water recharge.


Lessons learnt and recommendations

The approach taken by SNV in the WASH sector is innovative by focusing on capacity building of

local public institutions, on awareness raising and change of practices, involving training institutes

and networking. This focus by SNV on the ‘software’ was part of a programme partnership with

UNICEF that focused on the ‘hardware’. Overall, this evaluation shows that the SNV approach has

generated significant results, but the ‘hardware’ component now seems to constitute the main

constraint.

The original project document did not reflect a clear theory of change that shows how the different

components of the project, together with expected partnerships with UNICEF and regional

institutes, were expected to improved WASH performance in the region. This theory of change was

reconstructed as part of this impacts evaluation.

Capacity building by the SNV project at woreda level has been effective, but exchange events,

evaluation and learning for continuous improvement has been felt missing.

A significant proportion of households need to travel long distances for accessing safe drinking

water coupled with waiting long for fetching water. These are indications for the changes on

attitude and behaviour of households for using safe water over times. It is important that all WASH

and other development actors in the region strengthen their concerted efforts to ensure that safe

water sources are adequately and close to the communities and households as per the national

standards. The reduction in travel and waiting times for accessing safe water may enable water

collectors for having more time for development or productive activities.

As still a significant proportion of households incorrectly perceives their available latrine facilities

as improved ones, there is a need to enhance the awareness and education campaigns through

CLTS for enabling that households and communities have the correct perception. To this effect, it is

recommended to continue the awareness raising and follow up by health extension workers as they

are entry points to reach the communities and households.

Although there are positive effects at school level on improving skills and knowledge on WASH,

there is still insufficient management of WASH aspects by school teachers and insufficient access to

water for schools. Therefore, the governmental and non-governmental WASH actors in the region

may need to strengthen their partnership and collaboration for improving access to safe water and

improved sanitation in schools.

There are remaining challenges with a potential role for SNV to play as follows:

Strengthen the WASH monitoring and evaluation system within SNV and with local partners,

including clear and undisputable ways of defining and monitoring access to water and access to

improved sanitation. Reliable surveys must be carried out once in a few years to assess whether

progress is being made. SNV could support these surveys and provide quality assurance.

Underlying a good monitoring system is the need to define an integrated theory of change or

strategy for the WASH sector, with included software and hardware components, and linkages to

the education and health sectors. The theory of change should include insight in root causes of the

currently remaining problems in the sector.

SNV could introduce methods and technologies to monitor the discharge of water schemes (e.g. by

sensors) and not only whether they are functional (i.e. providing water or not).

It is recommended to share the insights of this evaluation with the responsible WASH staff at

woreda and regional levels, as part of a learning and evaluation exercise, and to assure continuous

improvement. There is also need for continuity in follow-up or refresher training, to assure further

improvement and avoid fall-back.

SNV could take the lead in proactively engaging in advocacy and lobby with the governmental and

non-governmental WASH actors who engage in hardware development, to narrow the existing gap

on demand for safe water and its supply as per the national standards of accessing within 1.5 km

and waiting time of less than 30 minutes, by addressing defined root causes.

SNV could further strengthen private sector engagement in the WASH sector;


SNV could consider stimulating young family heads who have adopted improved sanitation

facilities and sanitation and hygiene behaviour to convince or support other households.

More attention could be given to rainwater capture and/or creating underground water storage, as

well as collaboration and integration with natural resources and watershed management for

improving vegetation coverage around water sources so as to improve water recharge.

Page/of 51/1

Annex 1: Theory of change (reconstructed by the evaluation team)

Outcomes Outputs

Awareness raised at household and school levels, organization of WASH Committees

At Woreda / Kebele level: WASH plans are made Plans are implemented Monitoring of WASH

occurs

Impact

Improved access to

safe and clean water

Installed new water schemes

Maintenance of existing water

schemes

Upscaling of water schemes

Graduates with experience

find jobs in WASH sector

Improved WASH skills of graduates

from TVETCs

Enabling national policies

SNNPR have financial means

Assumptions

Capacities built of SNNPR water resources bureau

Hardware provided (e.g. piped water systems, sanitation facilities) (UNICEF?)

Capacities built of graduates from TVETC: guided learning of graduates

Capacities built at Woreda and Kebele level, strategic planning, MIS, CLTS

SNNPR support Woreda / Kebele to

develop WASH plans, aligned to UAP

Institutions supported at national

level – UAP with WASH priorities?

National level UAP integrates WASH

priorities

New water schemes installed,

protected and supported

Water schemes maintained

Improved management of water

schemes, use of safe water,

management of sanitation facilities

Reduced incidence of water

related diseases / diarrhoea

Donors finance hardware

Coordination between NGOs

Installed sanitation facilities,

households and schools

Maintenance of sanitation

facilities Upscaling of

sanitation facilities

Improved access to

safe & clean sanitation

Software provision:

Woreda and Kebele have

financial and human

resources available

Page/of 53/2

Annex 2: Evaluation Questions (extract from ToR)

The evaluation questions are largely based on the DAC/OECD criteria for evaluating development

assistance

Impact

Conduct a household survey and a facility survey in the project area. Compare the progress on key

project impact indicators – compare the baseline conducted in 2007 to the survey conducted in

2011, and compare them to the survey to be conducted in 2014. Comment on factors that

influenced the achievement of impacts. In particular, assess the impact on the following key

indicators:

Proportion of households with access to water supply within 1.5km distance

Proportion of households with access to safe latrine

Number of schools with access to improved water supply

Number of schools with access to safe latrine in schools

Presence and status of water and sanitation facilities in the household

Hand washing practice and Hygiene behaviour

Incidence of diarrhoea

Assess attribution of impacts to the efforts made under this project. What other factors apart from

the project’s interventions influenced the achievement of these impacts?

Assess the change in hygiene behavior among the community compared to the past.

Effectiveness

Assess the model of capacity building that the project followed:

Assess the long term effects of capacity building on improvements in functional drinking water

and sanitation facilities.

Assess the long term effects of capacity building on key operational aspects of the woredas such

as planning, provision and maintenance of water and sanitation facilities. Are the practices in the

project area different than in non-project areas? Are the needs assessments strategies and the

planning tools that were introduced by the project still operational? Are the partners still using

the key approaches that the project introduced?

Assess the capacity building model that this project used and comment on effectiveness of the

strategies employed in this project.

What is the view of the local functionaries of the support provided by the project? Did they find it

useful and if yes, how so? What are the main reasons for the continuance or discontinuance of

the practices that project introduced?

Assess whether the changes introduced by the project were institutionalized (systemic changes)

in the working of the local government? If so how? If not, why not?

What is the view of the local functionaries of the support provided by the project? Did they find it

useful and if yes, how so? What are the main reasons for the continuance or discontinuance of the

practices that project introduced?

Assess the prevalence of non-functional water schemes, and the number of water schemes not

recovering O&M costs. Comment on the reasons for non-operation.

Assess if the vocational colleges are still using the curriculum changes that SNV helped bring? Are

those changes institutionalized including the distance education for the technicians in the field?

How did the setting up of MIS system helped with the achievement of the project objectives? Is the

MIS system implemented by the project still operational? And is it being used for planning and

monitoring purposes?


Sustainability

Assess the lasting impact of the project as evidenced by the continuation of project benefits brought

about by the project, continuation of productive WASH strategies, and institutionalization of

changes/innovations/strategies introduced by the project.

Lessons learnt

What lessons can be drawn regarding sustainability based on project elements that were still

functioning and those not functioning anymore?

What elements of SNVs approaches were more (or less) effective in contributing to envisaged

impacts?

What could have worked better?

Documents

SNV WASH Project in Ethiopia - Aidenvironment WASH Project in Ethiopia Final impact evaluation Commissioned by SNV Netherlands October 2015 Project number 2529 Aidenvironment Barentszplein