- 0 -

Jerusalem Children and Community Development Organization

(JeCCDO)

End-Term Evaluation Report on HIV and Livelihoods Project in Kebele

02 of Debre Berhan Town, Ethiopia

March, 2016

Addis Ababa

- 1 -

Table of Contents

Pages

List of acronyms 3

Acknowledgements 4

List of Tables and Figures 5

Executive Summary 6

Chapter 1: Introduction 9

1.1. Background of the Organization 9

1.2. Background of the Project 9

1.3. Objectives and Scope of the Evaluation 9

1.3.1. Objectives 9

1.3.2. Scope 10

Chapter 2: Evaluation Methodology 11

2.1. Evaluation design 11

2.2. Study Respondents and Participants 11

2.3. Sampling Techniques 12

2.4. Data Collection Tools and Procedures 12

2.4.1. Data collection techniques 12

2.4.2. Procedures 12

2.5. Data Quality Assurance 13

2.6. Ethical Considerations 13

2.7. Methods of Data Analysis and Interpretation 13

2.7.1. Data analysis 13

2.7.2. Data Interpretation 14

Chapter 3: Evaluation Findings 15

3.1. Description of Study Participants 15

3.1.1. Survey Respondents 15

3.1.2. Participants of Qualitative Interviews 16

- 2 -

3.1.3. Observations 17

3.2. Level of Accomplishment of Project Outcomes and Outputs 17

3.2.1. Children and Community Accessed Quality and Adequate Services 17

3.2.2. Improved Livelihoods of Children, Guardians, Youth and PLHIV 22

3.2.3. Community Cared for OVC and Needy people 27

3.2.4. Improved Partnership Through Joint Learning 29

3.3. Project relevance, effectiveness and efficiency 30

3.3.1. Project relevance 29

3.3.2. Effectiveness 31

3.3.3. Efficiency 32

3.4. Success indicators (sustainability, Impact and Replication) 33

Chapter 4: Innovativeness, Challenges Experienced and Lessons Learned 36

4.1. Was it an innovative project? 36

4.2. Challenges Experienced 36

4.3. Lessons Learnt from the project 39

Chapter 5: Conclusions and Recommendations 40

5.1. Conclusions 40

5.2. Recommendations 41

- 3 -

List of acronyms

BCC Behavior Change Communication

BST Business Skill Training

CBOs Community Based Organizations

CCC Community Care Coalition

CDA Community Development Association

CLA Cluster Level Association

FGD Focus Group Discussion

GO Governmental Organization

HIV Human Immunodeficiency Virus

IDI In-depth Interview

JeCCDO Jerusalem Children and Community Development Organization

KII Key-informant Interview

MGO Non-Governmental organization

MSB Micro-and-Small Business

OVC Orphan and Vulnerable Children

PLHIV Persons Living with HIV

RH Reproductive Health

SHG Self-Help Group

TSW Transactional Sex Workers

SWMPHC Sanitation, Waste Management and Preventive Health Care

ToR Terms of Reference

- 4 -

Acknowledgements

On behalf of the Consultancy Firm, the evaluation team is indebted to express its gratitude to different

parties participated in the evaluation process. First of all, the team thanks JeCCDO which developed the

trust and provided us the opportunity to conduct the evaluation. We would like to extend our deepest

acknowledgement to the different target groups who participated in the interview processes. Special

thanks goes to PLHIV, OVC and their guardians, TSW, community members, youth members, street

children, CBO leaders, youth association leaders and CDA leadership from Debre Berhan Kebele 02

(villages 7/1, 2/1 and 2/2) who unreservedly participated in various interviews we conducted.

We would also like to thank members of the steering team, JeCCDO staff and directors of the two

primary schools indicated in the report who have been in the team’s side whenever needed including

participation in direct interviews. The team has special thanks to project partners including ACSI and

other local government stakeholders who have contributed much in the process of this evaluation.

THE EVALUATION TEAM

- 5 -

List of Tables and Figures

Page

Tables

Table 2.1: Sample of respondents drawn from each group 11

Table 2.2: Number of qualitative interview participants 11

Table 3.1: Project commonest and target groups addressed in the evaluation 15

Table 3.2: Measures of food availability and in-take by PLHIV 24

Table 3.3: Status of Project Financial Utilization 33

Figures

Fig. 3.1: Inside view of St. Michael Meeting Hall and year of establishment 28

Fig. 3.3: Internal facilities arrangements of the two schools received capacity building support 29

- 6 -

Executive Summary

Jerusalem Children and Community Development Organization (JeCCDO) is a local non-

governmental organization established in 1985 to respond to challenges of children affected by the then

drought and civil war. The project under the end-term evaluation is located in Debre Berhan town of

North Shewa Zone Administration, Amhara National Regional State. The project has been financially

supported by STOP AIDS NOW! (SAN!), based in The Netherlands. The target beneficiaries were

Orphan and Vulnerable Children (OVC), their guardians, street children, unemployed youth, People

Living with HIV and AIDS (PLHIV), transactional sex workers (TSWs), and community groups at large.

The overall goal of the project was improved well-being of HIV affected children through economic and

social empowerment of their guardians and communities. The project was implemented in three villages

(7/1, 2/2 and 2/1) of Kebele 02 for a period of 2012 to 2015 with a total budget of Birr 8,049,999. The

evaluation aimed to gauge whether or not the project has brought meaningful and positive changes in the

economic and social lives as well as empowerment of the target community.

The quantitative research design has chosen a survey method using questionnaires to generate

data from five specific target groups. The qualitative research design, on the other hand, used different

techniques, including case study, storytelling and phenomenological explorations through which

participants told their personal/family stories, expressed the phenomena within the context of the project

interventions and their experiences. In the survey design, the plan was to reach a total of 266 respondents

from five specific groups and we succeeded to reach 245 respondents. On the other hand, we reached 86

individuals for the qualitative interviews (44 males and 42 females).

The community survey showed that 92% of the respondents participated in sanitation, waste

management and preventive health care education programs. Among the community members who

participated in this evaluation, 94% of them believed that the health education campaign brought a

difference in terms of their action on personal as well as community level sanitation. In addition, the

community survey revealed that all respondents claimed they knew how HIV could be transmitted from

person to person. The knowledge of the community using condoms as a means of prevention from HIV

infection increased from 91.0% (as reported in the baseline survey) to 96.6% after the project

intervention. All youth participated in this study were confident that they had the knowledge on the

causes of HIV transmission and prevention mechanisms as well. Responses on stigma and discrimination

have shown a mix of results. Community members indicated that stigma and discrimination are non-

existent whilst some the PLHIV believed that stigma and discrimination are still manifested in different

forms, despite significant improvements over time.

Children have confirmed that their guardians pay treatment fee whenever they get sick.

Moreover, they added that they have been obtaining good care and support by their guardians. The

findings reveal that the CD4 count of 18 of 24 (75%) PLHIV who were benefited from the project has

progressively increased. Furthermore, the nutrition status of these groups is improved compared to the

situation prior to the project. Housing renovation enabled poor elders and those affected by HIV to live in

decent houses. Some of these houses also used to generate additional income to the family by renting a

portion of the rooms by some and running small business activities by others. The project provided

important support to strengthen community structures, including Iddirs and Community Development

Association (CDA).

The project has benefited OVC guardians to participate in business skills training. Thirty-five of

38 guardians interviewed in this evaluation were involved in Income Generating Activities (IGAs). All of

those guardians who took loan claimed without access to the loan, it would be difficult to start their IGAs.

Youth participated in the IGAs have satisfaction in the content and comprehensiveness of the business

skill training (BST). The evaluation team observed diversified IGAs where youth have been involved.

Despite the existing dissatisfaction on the amount of current business capital or existing high pressure on

- 7 -

loan payback, the youth reported that it had not been possible to start the business without the support

they got from the project. Fourteen of the 25 youth participated in the IGAs were satisfied by their current

earned incomes. PLHIV had also reported that the contents of BST were comprehensive. Those PLHIV

who took credit were satisfied with the amount of startup capital they received and were confident they

would pay-back the loan. However, only 9 of the 24 PLHIV reported they were not satisfied by their

current monthly income. One of the two street children participated in the interview reported that he

received Birr 2000 financial support from the project and started shoe shining activity. At the time of the

evaluation, the boy was able to pay for house rent of Birr 1000 monthly. The other street boy expressed

he was very happy when he received the financial support of Birr 2000. TSW who participated in IGA

expressed “From the income I generate by selling local drinks, I am able to pay Birr 1050 for house rent

per month and also I feed the children and my grandmother. After I have engaged in the IGA, life is

improved, thanks to JeCCDO.”

According to the leaders of Community Based Organizations (CBOs), the activities of the project

have addressed the most pressing needs of the target groups. For example, provision of direct nutritional

support enabled PLHIV to leave their beds, become healthy and start to participate in IGAs. The pre-

school facilities encouraged many parents to send their kids to pre-primary school education. The

community-based project activities solved many sanitation problems. IGAs brought significant changes in

the livelihoods of OVC, youth, PLHIV and TSWs. The community capacity building activities were in

line with the government’s focus that aimed to strengthen capacities of the community at the grassroots

level. The links made between HIV and livelihoods improvement made the project more relevant given

that improvement in livelihoods has proved to be an effective strategy to reduce HIV infection and

associated challenges of stigma and discrimination. The CBO leaders believed that project beneficiaries

.became more assertive to claim their rights and discharge their responsibilities compared to the prior

project implementation period. Our respondents stated, “...women who used to involve transactional sex

have become women’s peace team/group members and started to engage in productive activities other

than transactional sex.” The team has the impression that the project is found to be effective in reaching

the target groups and achieving the set outcomes stated in the project document.

The community leaders commented that the procedures adopted to facilitate loan services brought

unintended consequences. One of the complaints from the leaders regarding the loan was poor

communications created between the Project Coordination Office and the beneficiaries regarding the new

modality adopted to take loans. Data indicated that majority of the loan beneficiaries took loans towards

the end of the project period since the process took more than it was expected. The project interventions

have been efficient in particular in the areas of renovation of houses, school capacity building,

construction of communal latrines and availing of waste disposal bins. The use of local resources,

including existing community structure and local government facilities has been exemplary.

The school principals commented that if such a support was expanded to other public/government

schools in Debre Berhan and elsewhere, this would make many poor families to send their kids to school.

Many of the groups participated in the evaluation agreed it is early to report on the impact of the IGAs.

However, some of the changes in attitude regarding accepting the new loan modality can be sited as the

impact registered from the implementation of IGAs.

The evaluation team has observed some indicators that confirm innovativeness of the project.

Among others are: Active participation of the community through its local structure, enabled to wisely

utilize the project resources, targeting appropriate beneficiaries, has reduced dependency syndrome (if not

completely addressed) and motivated beneficiaries to have active role in project implementation. The

savings encouraged some families to start small IGAs using their savings. Further, JeCCDO’s courageous

move to create a model to work with micro-finance institution, private sector, academic institutions and

through a multi-stakeholder approach is found to be an innovative practice.

- 8 -

According to the assessment of the evaluation team, it is possible to conclude that JeCCDO is

successful in achieving the overall goal and specific outcomes of the project. However, there was one

project component which had a challenge. The team learned that JeCCDO had insufficiently prepared its

staff to introduce and coordinate the new loan modality, which resulted in resistance from some of the

potential beneficiaries to accept the new loan modality. Generally speaking, the project has been relevant,

effective and efficient as it utilized the meager resources to achieve the overall stated goal and specific

project outcomes.

Key lessons drawn from this project are:

• Strengthening transparency, reducing distrust and continuity of awareness raising activities are

core components of innovativeness.

• Adoption of innovation takes time; therefore, the project period for such innovative project would

have been different from the period given for projects entrusted with conventional service

delivery.

The team has the observation that JeCCDO had strong passion to adopt innovative livelihood

project and tried to implement it which was beyond its available capacity (technical and human resources)

at the project level. Competent human resources and technical capacities at various levels would have

been seriously thought and planned to think, process, design, implement and to monitor the innovative

project. On the basis of the summary presented above, the team would like to mention the following

recommendations among the list in the report.

• For future similar innovative projects, narrowing the scope of the target group will contribute to

successful implementation of the project.

• JeCCDO needs first to learn and internalize the basic components of asset based approach to

consider the existing assets and prepare itself towards implementation of the approach while

trying to adopt an innovative project.

• For future similar initiatives, negotiations on the time factor should be made with fund providers,

and government as well as other partners in order to internalize and implement innovative

projects.

• Internal capacities of JeCCDO should be revisited in the future to implement similar innovative

projects.

• The use of micro-finance institutions to facilitate loans should be encouraged but proper planning

is needed to make sure potential loan recipients and other stakeholders are aware of the policies

and guidelines of micro-finance institutions.

- 9 -

Chapter 1: Introduction

1.1. Background of the Organization

Jerusalem Children and Community Development Organization (JeCCDO) is a local non-

governmental organization established in 1985 to respond to challenges of children affected by the then

drought and civil war. JeCCDO has passed a number of developmental phases. It initially paid attention

to institutional child care services, and moved into integrated child focused community development

approach, implementing community led development programs. Long-term capacity building of CBOs’ is

at the heart of JeCCDO’s development approach. JeCCDO has four development objectives namely;

improving access and quality of social services to children and vulnerable community members,

strengthen the livelihood of disadvantaged families so that they can be able to support their children,

developing the organizational and operational capacity of CBOs and strengthening social, entrepreneurial

and organizational capacity of JeCCDO. The vision of JeCCDO is “to see Ethiopia where no child lives in

poverty” and its mission is “to advance the care and protection of children within the family and

community for their all rounded development.”

1.2. Background of the Project

The Project under the end-term evaluation had been implemented in Debre Berhan town of North

Shewa Zone Administration, Amhara National Regional State with the financial support of STOP AIDS

NOW in the Netherlands. The Project consisted of two major components of HIV and Livelihoods

improvement, targeted to address the needs of multiple segments of the community in Villages 7/1, 2/1

and 2/2 of Kebele 02 in the town. The project was initiated after it had passed through a number of

processes which involved the interest of different actors, including the local government, the donor and

the implementing Organization (JeCCDO). The Project identified three pillars at its initial stage, namely,

(1) development and implementation of innovative program, (2) linking learning and research and (3)

evidence-based international policy influencing. Based on the baseline assessment; problem analysis and

target identification were conducted. Therefore, OVC and their guardians of OVC, street children,

unemployed youth, PLHIV, transactional sex workers, and community groups at large were identified as

primary target groups. In addition, CBOs, schools, youth associations and association of PLHIV were

identified for collaboration in the direct interventions of the Project. In addition GOs’, NGOs’, academic

institutions and the private sector were also stakeholders of the project. The Project was implemented by

setting four major outcome indicators which included (1) children and communities accessed quality and

adequate services, (2) improved livelihoods of children, guardians, youth and PLHIV, (3) communities

effectively cared for OVC and needy people and (4) improved partnerships through joint learning, which

together were assumed to contribute towards the goal of improving the well-being of HIV affected

children through economic and social empowerment of their guardians and communities. The project was

implemented from July 2012 to December 2015 with the total budget of birr 8,049,999.00 secured from

the STOP AIDS NOW! in the Netherlands.

1.3. Objectives and Scope of the Evaluation

1.3.1. Objectives

The evaluation aimed to gauge whether or not the Project has brought meaningful and positive

changes in the economic and social lives as well as empowerment of the target community. The specific

objectives include (1) to examine the extent to which the Project has contributed to the reduction in the

vulnerability of the target community to HIV/AIDS and its impact; (2) to assess the level of

accomplishment of the set objectives of the Project; (3) to draw and document lessons learnt from both

- 10 -

best practices and gaps of the Project; (4) to assess the level of relevance of the Project Components and

effectiveness of sustainability strategies, and (5) to provide recommendations to undertake similar

projects in the future.

1.3.2. Scope

There are two dimensions of the scope of the evaluation study, which include the geographic

coverage and the groups who participated in the evaluation process. Geographically, the evaluation was

conducted on the Project implemented in three specific Villages [7/1, 2/2 and 2/1] of Kebele 02 in Debre

Berhan town. The evaluation addressed groups including OVC and their guardians, youth, PLHIV, TSW

and the community at large. In addition, different stakeholders including CBOs, schools, GOs, PLHIV

association, and micro-finance institution who had participated during the Project implementation were

involved in the data collection process.

- 11 -

Chapter 2: Evaluation Methodology

2.1. Evaluation design

The evaluation applied a mixed methods design (MMD) whereby the process applied

combination of quantitative and qualitative methods of data collection, analysis, interpretation and

presentation of findings. The quantitative method has chosen a survey design. Survey instruments were

developed to generate data from five specific target groups who had been direct beneficiaries/participants

of the Project. The qualitative method applied case study design using specific data collection instruments

including FGD, KII, IDI, observation and photography. Through these tools stories and lived experiences

of the participants were documented. Cases were identified from each group to participate in the selected

data collection techniques to explain their own experiences or other cases as the result of their

participation or witnessing what happened in the individual or collective life of the community members

where they belong.

2.2. Study Respondents and Participants

The Team uses the term respondents to describe individuals involved in the survey. On the other

hand, the term participant is used to express individuals took part in the qualitative interviews. The

survey was designed to collect data from five specific groups who were the beneficiaries of tailored

Project activities devoted to benefit each group. The groups from which quantitative data were generated

include OVC, OVC guardians, youth, PLHIV and community members. The following table presents the

sample drawn from each group.

Table 2.1: Sample of respondents drawn from each group

Group population Planned sample Actual

OVC 150 45 39

OVC guardians 150 45 38

Youth 70 28 27

PLHIV 70 28 24

Community members 2000 120 117

Total 2440 266 245

The qualitative interviews, on the other hand, involved many other groups in addition to those

listed in Table 2.1 above. The specific individuals and groups participated in the qualitative data

collection include: OVC, OVC guardians, youth members, street children, PLHIV, transactional sex

workers, community members; youth association/club members, CBO leaders, SHG members, school

principals, steering committee members, experts from government offices and taskforce members. The

team also made interviews with JeCCDO staff members from the Project site and the Central Office. The

number of participants drawn from each group against the type of interviews each person/group has

participated is summarized in Table 2.2 below.

Table 2.2: Number of qualitative interview participants

Data collection tools Targets interviewed Number of participants % achieved

against plan Male Female total

IDI – interview

guide/protocol

OVC 2 -- 2 100

OVC guardians -- 2 2 100

Transactional sex workers -- 2 2 100

PLHIV 2 2 4 100

Street children 2 -- 2 100

- 12 -

JeCCDO staff 3 -- 3 100

FGD checklists OVC 9 4 13 100

OVC guardians -- 6 6 100

Community members 4 11 15 80

Communal toilet users 4 4 8 100

Steering committee 2 3 5 100

Youth 6 5 11 100

Youth association 3 3 6 100

KII – interview

guide/protocol

Iddir leaders 3 -- 3 100

Stakeholders 2 -- 2 80

School principals 2 -- 2 100

Total 44 42 86

2.3. Sampling Techniques

A descriptive sample survey was applied to draw survey respondents. Depending on the size of

the population, the evaluation team considered different percentages as a sample to each group. For the

community survey, the team considered 10% for the OVC and OVC guardians a 45% sample was

considered. On the other hand, for youth and PLHIV a 40% sample was considered. Each sample unit was

drawn using a systematic simple random sampling technique where the first element was chosen

randomly from the list of the target population followed by systematic picking of the next elements until

the last element is reached based on the pre-determined sample size. For the qualitative interviews, each

participant was drawn purposefully who were supposed to have sufficient information about the project.

2.4. Data Collection Tools and Procedures

In this section, the Team presents a summary of the specific tools used and the procedures

followed to solicit the quantitative and the qualitative data.

2.4.1. Data collection techniques

As described above the Team used two major data collection approaches-quantitative and

qualitative and appropriate tools were developed, accordingly. Survey was the tool used to collect

quantitative data. For this purpose, five specific survey tools were designed to conduct the surveys with

five specific groups. The Team further developed tools to generate primary qualitative data from the

target groups described above. The specific qualitative tools used for this particular evaluation include:

FGD checklists, IDI guide/protocol, KII guide/protocol, and observation checklists. In addition, the Team

used photography as means of gathering visual data.

2.4.2. Procedures

The first step taken following the agreement signed between JeCCDO and our consulting firm

was submitting the draft tools along with the inception report. This step was followed by incorporating

the comments given on the data collection tools and resubmitted the tools to get the final approval by

JeCCDO. The next step was the recruitment and training of data collectors and supervisors. Along with

other logistical arrangements, JeCCDO has provided the Team with letter of support before it left for field

data collection. Upon arrival to the project site, the Team made brief discussion with JeCCDO staff and

immediately deployed the data collectors for interview which took six days. After the field interview was

completed, the Team worked on data entry, data transcription, analysis and interpretation which led to the

task of writing the draft report. The draft report was submitted to JeCCDO as scheduled and the Team

received valuable comments that enriched the report. The final step was incorporating the comments from

JeCCDO and finalizes the report as it appears in this version.

- 13 -

2.5. Data Quality Assurance

Assurance to the quality of data generated from both the primary (field) and the secondary

(document) sources have been obtained by applying different techniques to maintain the quality. Quality

of the primary data was maintained by taking at most care in the process of recruiting persons for the

interviews. Before deploying the data collectors to conduct interviews, intensive training was given on the

survey tools as well as interview protocols. The field supervisors paid maximum attention and checked

the filled questionnaires for their correctness and consistency. In order to avoid personal bias by data

collectors while taking notes of the qualitative interviews, each interview session was recorded using

digital/audio-tape recorders. This helped the transcription process to be bias-free since transcribers listen

to the audio-records and wrote the transcripts word for word. Trained statistician managed the survey

including coding, data entry, data cleaning and analysis. An experienced qualitative researcher led the

process of data transcription, coding and theme development of the qualitative data. The above-mentioned

actions together maximized the data quality and gave the highest assurance that any technical or human

made errors were very low in this evaluation.

2.6. Ethical Considerations

The consents the Team secured from each participant were the primary indicator for the ethical

soundness of the evaluation process. The Project was implemented among vulnerable groups that needed

to consider ethical issues at times of interviews and in presenting the data. First of all, the interview

process was taking place professionally where the feelings of each interviewee were assessed before the

start of the interview and at the end as well. Furthermore, in this report, data are presented in an

aggregated manner (for the survey) and pseudo-names were used when necessary to report individual

cases (for the qualitative data). All audio-records and transcripts, as well as survey questionnaires are kept

in a safe way within the auspices of the Consulting Firm and such raw data will be transferred to JeCCDO

in an official manner.

2.7. Methods of Data Analysis and Interpretation

2.7.1. Data analysis

Our sources of data, as described above, were both primary (raw data from field) and compiled

and processed information (from available documents). The first step we took to analyze the quantitative

data was to code, enter into computer using Census and Survey Program (CSPro) and then transferred the

data to SPSS program. This was followed by data cleaning and verification. The statistician computed

basic descriptive statistics (such as frequencies and percentages) by considering the socio-demographic

characteristics as independent variables that influenced the outcomes of other variables (such as use of

condoms, adopting personal and community hygiene, practicing OVC care, involving in IGA, and other

issues). The qualitative data analysis has passed through a number of steps. The initial step in this regard

was doing a verbatim transcription of each interview. The transcripts were repeatedly reviewed by the

Team to identify major issues/themes emerged out of the many interviews conducted across each groups.

On the basis of the emerged issues, the team made a qualitative data summary which latter was

triangulated with the quantitative data at the stage of data interpretation and compiling the report. The

Team extracted useful information from available documents, some of which used as

benchmarking/baseline to measure the changes brought in the lives of the target groups.

2.7.2. Data Interpretation

- 14 -

Interpretation is the way the Team gives meanings to data generated from both the quantitative

and the qualitative parts of this evaluation. The interpretation of the quantitative data is presented in a

more aggregated way, but translated into meaning(s) by taking the statements made about the Project

outcomes into account. On the other hand, interpretation of the qualitative data is more individualized and

subjective to the meaning of individual services as understood by each person/group who was either

service recipient or who coordinated the services and participated in this evaluation. The Team made data

triangulation of the survey results and information from various groups participated in the qualitative

interviews where we found similar meanings and expressions. In order to give strong meanings to the

representation of data from the survey and the deeper-meanings from the qualitative information, we

conducted data corroboration by making correspondence and matching between the numeric data and that

of the meaningful verbal expressions made by our data sources.

- 15 -

Chapter 3: Evaluation Findings

This Chapter presents the results of the evaluation. The first section of the Chapter presents basic

profiles of the evaluation respondents/participants. The second section is devoted to present on access to

quality and adequate services. The third section dwells on the results documented on social care,

community capacity building and empowerment. Section four describes livelihoods improvement of

target groups. The fifth sections on the other hand presents project relevance, effectiveness and efficiency.

The last section is devoted to present results on success indicators (output, outcome, sustainability,

replication and impact). Before presenting the findings it is necessary to describe the project components

and groups reached through the implementation of each component.

Table 3.1: Project commonest and target groups addressed in the evaluation

Project component Target groups addressed

OVC support OVCs

Guardians of OVCs

Livelihood improvement Guardians of OVCs

PLHIV

Youth

Street children

Transactional sex workers

Sanitation, waste management and preventive health care Households and community members

Housing renovation Selected families

Pre-school children support Children between the age of 4-6 years

Parents/guardians

Schools

Community capacity building CBOs and youth clubs

3.1. Description of Study Participants1

There were two sets of targets: (1) respondents of the quantitative survey and (2) participants of

the qualitative in-depth interviews, FGDs and key-informant interviews.

3.1.1. Survey Respondents

There were five specific groups who participated in the survey as described in Table 2.1 in

Chapter 2. Profiles of these target groups are summarized as follows.

Socio-demographic description of the respondents:

The plan was to reach a total of 266 respondents from five specific groups and the Team

succeeded to reach 245 of them. In this regard, about 92% of the planned targets were reached. The socio-

demographic, employment and income profiles of the respondents are briefly summarized as follows.

Gender: The majority of the respondents were females that accounted for 75.9% of the total respondents.

The proportion of female respondents was higher among the OVC guardians (94.7%), PLHIV (91.7%)

and the community members (79.5%). From the OVC participated in the survey 48.7% were females.

1 The term participant also serves for survey respondents and physical settings included in the

observations

- 16 -

Age: The age of the respondents goes as low as 10 years among children, and as high as 50 and above

years among OVC guardians, PLHIV and the community members. Nearly half (46.9%) of the survey

respondents were drawn from the most active ages from 25 to 49 years.

Education: The Team made a crude classification of educational levels of respondents - illiterate, primary

education (including those able to read and write), and junior-secondary education and above. Some

22.5% of respondents from all groups were illiterate; many of them came from the community members.

About half (50.6%) of the respondents attained primary level of education; almost half of which belonged

to the community members and the majority of the respondents from children also belonged to this

category.

Marital status: We regrouped marital status into single, married and others2 category. There is a fair

distribution of respondents in the three categories of marital status with a little high percentage of those

with single marital status (35.9%), followed by respondents with married status (33.0%) and those in

‘other’ category (31.1%), respectively.

Monthly income: The Team succeeded to secure 194 valid responses. A little more than half (52.0%) of

the respondents earned a monthly income of 500 birr or less. About twenty-eight percent (27.6%) of the

respondents earned an income between 501and 1000 birr per month. The remaining, 20.4% earned

above1000 birr per month. The group which was found to be in the low earning category was PLHIV

where 78.9% earned birr 500 or low monthly. The highest earners were from the community members in

which 27.3% of the valid respondents found to earn more than 1000 birr per month. On the other, only

10.5% of OVC guardians and 22% of the youth who participated in the survey earn monthly income of

1000 Birr or above.

Employment/source of income: Thirty-seven of the thirty-eight OVC guardians who participated in the

survey have been involved in personal/family income generating activities. On the other hand, among 24

PLHIV who participated in the survey, 21 were self-employed, and the other three lived on aid from

relatives and NGOs. Similarly, the majority of the youth (77.8%) were self -employed, 14.8% were

employed by private business owners and other two lived on aid from relatives/NGOs. The respondents

from the community members (the largest group in this survey) reported that 88(75.2%) were self-

employed, 9.4% were government employees, still other, 6% were employed by private business owners

and the remaining, 9.4% lived on aid from relatives/NGOs.

3.1.2. Participants of Qualitative Interviews

Profiles of participants in the qualitative interviews (FGD, IDI and KII) are summarized as

follows.

The qualitative interviews were conducted with each target group as planned. The overall gender

proportion was more or less equal (44 males versus 42 females) as described in Table 2.2 in the

methodology chapter. However, there were minor variations in some of the groups between the planned

targets versus the actual interviews. For example, from the planned three FGDs with the community

members, only two of them were conducted. The Team was forced to change planned FGDs with

government experts into individual interviews due to difficulties to pool them together for the FGDs.

Some of these experts were also the members of the Steering Committee of the Project and they

participated in the FGD organized for the Committee.

2 Others include those who are widowed/r, divorced, separated or engaged.

- 17 -

3.1.3. Observations

Observations were made to a number of facilities using the prepared checklists and writing

additional field notes. Specifically the team conducted systematic observations to the renovated Iddir hall,

communal toilets, youth IGAs, OVC guardians’ houses, pre-primary education facilities and renovated

houses.

3.2. Level of Accomplishment of Project Outcomes and Outputs

3.2.1. Children and Community Accessed Quality and Adequate Services

One of the primary expected outcomes of the Project was to create access for children and

communities for quality and adequate services. To realize this outcome, a number of activities were

carried out some of which include; support to OVC, educate families on childcare, education on condom

use, provide support to pre-school education, etc. These and other activities were provided to generate

some corresponding outputs, some of which include; direct support to 150 OVC during year one of the

project to ensure they remain in school in year 2 and 3, enabling 150 guardians, in educating their

children, guardians become aware of child care and development, conducting public education on child

care, waste management, stigma & discrimination, etc. In order to verify the extent to which the activities

were implemented and the intended outputs and outcomes have been achieved, the Team analyzed data

produced from different sources and presents the results corresponding to each performance indicator.

Improved Access to Social Care: Social care provision includes a number of activities that were carried

out by the project to ensure access to quality and adequate services to the target groups focusing on

children at the center of these services.

a) Improved Education: All OVC participated in the evaluation remembered they received educational

support from the Project. On the basis of their memory regarding the educational support provided two

years ago we asked children to rate their school achievement when they received support. Thirty-eight

(97.5%) of them reported their achievement was satisfactory which was attributed to the support.

Students’ self-report indicated that by the time of data collection for this evaluation, 97.5% of them

performed to the level expected by the school standards. Given that direct support given by the Project

was two years ago, what contribute for the current success of children is an important issue to analyze. On

the basis of 24 valid responses in this regard, JeCCDO’s support to guardians through IGAs, relatives’

assistance, and support from neighbors contributed for their educational success. A group of children who

participated in the FGD also witnessed that JeCCDO’s assistance with school materials and clothing

helped them to study better and improve academically.

b) Improved Health and Nutrition of OVC: One of the prominent problems identified prior the project

implementation was high prevalence of infection and illness among OVC. A number of action steps,

including medical support, improving waste disposal, construction of communal latrines, and other health

care related services were provided to improve the health condition of OVC and the target community at

large. In order to evaluate the health outcomes of such efforts made by the Project data were generated

from OVC themselves, guardians and other community members participated in the evaluation.

Children’s survey revealed that three months prior to the time of field work for the current

evaluation, 26(66.7%) of them were sick by infectious illnesses, which was much more less than the

62.0% reported in the baseline survey3. From those who were sick, 20(76.9%) reported the illness was

related to respiratory infections. Children were asked about their health service seeking behavior and the

majority, 35(89.7%) visited modern health institutions and 32(82.1%) reported their guardians covered

3 Note: In the baseline survey the timeframe was not indicated clearly whether it was prior three, six, or 12 months. Further the

sample size was higher compared to what was covered in the evaluation.

- 18 -

the treatment expenses. This was against a 47.0% of children in the baseline survey who visited modern

health services in times of sickness.

As a measure of availability of adequate and balanced diet, children were asked to report on how

they feel in relation to their physical strengths. To the question presented about their physical strength,

30(76.9%) reported they felt strong all the time. Other seven children reported they had some sickness,

but they felt strong mostly. Thirty-seven (94.9%) reported their health was in a good condition all the

time, which could be attributed to good and healthy food and nutrition. For example, 38(97.4% of them

reported they got adequate food at least two times a day. Although children feel that they have good

physical strength and their personal health is in good condition, they are still worried about the

environmental hygiene. Some 22(56.4%) of the respondents believed that they were not living in a safe

environment, which was free of illness.

c) Improved Care of OVC: One aspect of protecting OVC from any form of abuse is to ensure they get

access to quality and adequate services at home, at school and in the community. Concerning the

treatment they receive at home, 36(92.3%) of them reported there was positive care and treatment. In this

regard,, 38(97.4%) of children reported their parents/guardians gave them at most the highest level of care

they could. While they were at home 36(92.3% of the OVC felt that there was an adult around them who

could take care of them in case they were hurt or feel sad. School level care provided to OVC is improved

from time to time. For example, children participated in the FGD reported that in their schools, it is safe

to play and interact with other children unlike in the community setting. However, unlike their home and

the school where most of the OVC participated in this evaluation reported they receive maximum

protection and social care, only 20(51.4%) of the group trusted their community members who could

provide them social care, despite the fact that 38(97.4%) of them believed the community members had

adequate resources.

d) Improved housing and community latrines: Improving the housing conditions of needy households

and construction of communal latrines were part of the activities carried out to improve access to quality

and adequate services to children and communities. The findings on both subjects are described as

follows.

As a social care component, housing renovation enabled poor and destitute elders and those

affected by HIV to live in decent houses which have improved the health conditions of both children and

seniors. The team made visits to some of the renovated houses and were also told by the owners that the

houses were almost about to collapse before the renovation was made. One of the house owners stated

that, “thanks to Jerusalem [JeCCDO], I and my family members start living in a real home in which we

feel safe and get peace.” The team randomly picked three renovated houses and checked for availability

of basic facilities. During the visit we interviewed the owners whether the renovations were real needs.

Three of them expressed, the roofs and walls were almost about to fall. They also expressed that they

were unable to make any maintenance. The maintenances made to the houses were within JeCCDO’s

objective of supporting the needy where children will grow safely and with the necessary social care. All

the houses we made observation accommodated either children, elders, PLHIV or are female headed apart

from other vulnerable adults living in them. After the maintenance was made, at least one of the three

households the Team paid visits rented part of the house to generate additional income to the family.

The community members who participated in the latrine users FGD underlined that the severity

of the problem of sanitation facilities before the construction of the communal latrines. They stated that

before the latrines were built, villagers (children, youth, male and female) used the open field or plastic

bag fliers to defecate and dispose their personal wastes/excretions. The participants unanimously

applauded the construction of communal latrines and put their level of satisfaction in the following words:

- 19 -

The construction of the communal latrines in order to assist the local community to solve the

prevailing sanitarian problem of the villagers in Kebele 02 is very good not only for the individual

users but also for the community at large. By so doing, Jerusalem [JeCCDO] has done big job and

made very good contributions to the target community. The use of the latrines by the community

members contributes to maintain the environment clean and decrease the incidence of

communicable diseases.

The evaluation Team also observed two of the three communal latrines constructed by the

project. Both of the visited communal latrines were functional during the visits although some problems

were observed in relation to availability of water, , in accessibility to persons with disabilities and

management of facilities. However, the latrines were free of unfriendly smells, which showed that users

cared for the latrines to remain clean, in general terms. Beyond sanitary purposes, the communal latrines

were found to be used as social forums through which villagers who were common users of the latrines

come together to discuss other matters.

Increased Knowledge of Health and Sanitation: Participation in the community campaign to promote

community health and sanitation was one of the major activities carried out by the Project. The

community survey showed that 108(92.0%) of the respondents participated in sanitation, waste

management and preventive health care education programs. From the respondents who reported their

participation in the community education program, 87(74%) attended every single session organized

throughout the Project life. This is a primary indicator for knowledge transfer regarding preventive health

care and sanitation. The result from the community survey is supported by the views of OVC participated

in one of the FGDs. Children witnessed by saying that:

There were trainings and campaigns which helped the community to be aware about sanitation

and community health. There is also a change in behavior within the community after the

training. …before the training the parents did not care much about children’s hygiene but now

whenever they see something parents are careful to the child’s hygiene. In the community, the

knowledge about hygiene is good. But there are still trashes around the field where we are

playing.

The community survey revealed that all respondents claimed they knew how HIV could be

transmitted from person to person. All of the respondents also attributed HIV transmission to unsafe

sexual practice. The majorities were found to give less emphasis to blood transfusion as a means of HIV

transmission as only 37(31.8%) of them agreed with the statement “blood transfusion can transmit HIV.”

On contrary, 72(61.5%) agreed with the statement about not taking blood transfusion prevents infection

by HIV. This contradiction may be a sign of lacking adequate awareness about the transmission as well as

prevention methods of the HIV. The knowledge of the community using condoms as a means of

prevention from HIV infection increased from 91.0% in the baseline survey report to 96.6% in this report.

This is a great improvement in knowledge building as a result of Project implementation apart from other

sources of information that could impact positively on community knowledge. In another vein, all youth

participated in this study were confident that they had the knowledge on means of HIV transmission.

The Project designed a training package for youth on HIV/AIDS prevention, skills of negotiation to

practice safe sex, and other important topics as reported by 25(92.6%) of the respondents.

One cannot separate knowledge from attitude as the former is the basis for any change that occurs

in the later. Some statements made by participants in this evaluation witness that the community groups

have brought attitude change on health and sanitation beyond knowledge creation as the result of the

project intervention. Among the community survey participants, 110(94.0%) believed that the community

education campaign brought a difference in terms of their action in terms of keeping their personal

hygiene and environmental sanitation. The FGD conducted with community groups supported the survey

- 20 -

result from the same group. FGD discussants witnessed the differences in attitude towards preventive

health and sanitation before and after the implementation of the Project. They stated:

The residents in most blocks of the Kebele started [after participating in the health education

campaign] disposing dry wastes not in the sewerages, but in the dry waste bins which are put in

place nearby their houses. And even, there are individuals who have organized themselves in

associations and are collecting garbage from households on scheduled days. Like other places, the

householders pay certain amount of money with their monthly payment for water consumption

bills. Generally, community members are practicing proper handling of garbage from their

households because they usually put it into a sack and wait for the collectors.

Although the above quotation describes the existence of changes in attitude and practices, the

discussion made with other groups and observations made in the sites, confirmed that the dry waste bins

were not properly utilized due to some reasons, one of which was mentioned as lack of coordination with

the local municipality to dispose the collected wastes. As the result, the community members decided not

to use the bins since they became causes for stinky (bad) smells. Such a decision is, of course, a sign of

attitudinal change and preventive health care action. Otherwise, the community members might not care

whatever happened to their village, if it was not the changes that happened as they were aware about the

possible risks related to poor sanitation.

The Team put few questions for community members to reflect on their attitude towards the

labeling scenarios with regard to who is supposed to be primary victims of HIV. The response tells that

the respondents have the correct attitude not identifying certain groups as primary carriers of HIV. This

implies, in the Team’s interpretation, that the HIV education campaign organized by JeCCDO works very

well helping community members to develop the right and positive attitude.

Reducing the social and economic impacts of HIV is one of the objectives of the Project. As the

result a number of activities were planned and implemented including health education, training, inclusive

IGAs [which also paid attention to enable those PLHIV to join the IGA activities]), direct support for

PLHIV, OVC and OVC guardians, etc. Measuring the level of stigma and discrimination being observed

towards those infected and affected by HIV is a direct indicator of observable changes in attitude. In this

regard, the Team was able to generate data to describe the situation with regard to stigma and

discrimination against those affected by HIV and AIDS. The baseline survey indicated that 40% of

PLHIV participated in the study were worried that they encountered various forms of discrimination.

There is an impressive result to see the rate of responses documented from the community members

regarding the extent to which the level of stigma and discrimination are reduced almost to the zero level.

Two areas of success reported by PLHIV: first the efforts made by JeCCDO to change the attitudes of the

community members towards PLHIV and, second their families as well as the state of the art that PLHIV

also started to feel confident and hopeful because of stability in their internal feelings positive changes

they saw from the community. These are some signs of positive impact brought by Project interventions.

On the contrary, the in-depth interviews held with PLHIV supports the community’s views. Four

PLHIV who participated in the in-depth interviews agreed that some years back (especially at times they

knew their HIV status) was the difficult times within themselves and from the community members.

However, through time, especially as the result of the Project interventions made by JeCCDO and the

government, have brought many positive changes. Here is what the participants reported.

Yes, long time ago, I faced a problem in stigma and discrimination. Currently, I am happy as any

man I can work and cover my expenses. First, what I did is I told the truth to my neighbors about

my HIV positive status. They were happy and, afterwards, they supported me and I have a good

communication. The past 12 months, I did not face any stigma and discrimination. I was living in

a Kebele owned house, but because of rehabilitation it was destroyed and as a replacement to my

- 21 -

previous house, I got land on which I have constructed a house. Now, I have my own house and

live freely.

The above quotation tells us a transformation in internal stability and external support, which

make the participant to feel strong and become productive. In a similar way here is what another in-depth

interview discussant shared with us the changes she observed in community members attitude towards

PLHIV. She expressed:

The most difficult time was when I knew my HIV status. It was difficult for me to convince my

mother. But now, after lobbying her several times, she is ok with me. No other person has

discriminated me because of my HIV status. Currently, I am working in a restaurant as a cashier.

Now, there are changes in the community towards PLHIV. When I see myself and my friends, we

are confident; we play, eat and communicate well within the society. There is discrimination

sometimes, but there is a change through times and sigma is becoming less.

It is clear that from the last sentence in the above quotation we can conclude that there still exists

stigma and discrimination but changes are observed positively and such stigma and discrimination have

decreased from time to time due to the concerted efforts made by the community through JeCCDO’s

social care support provided in terms of direct support, education and capacity building activities.

Parallel with the increase in knowledge and changes in attitude, is the actual practice, which

supposedly ensures success in obtaining the outcomes of access to quality and adequate services from the

health perspective. The Team generated data from different sources to evaluate the extent of putting in

action the knowledge on health, sanitation and HIV by the community members.

There are also evidences that show there are certain levels of discriminatory attitudes towards

PLHIV. For example, it was only 54% of the PLHIV respondents who believed that stigma and

discrimination against people affected by HIV/AIDs has been decreasing over the last three years. Other

37.5% of respondents from PHIV group again felt that they are labeled differently in the community or

work place because they are HIV positive.

With regard to changing knowledge into practice, between 88%-93% of the community members

participated in the survey reported that they put in action the knowledge they get from the community

health education campaign at various levels (personal level 93%, at family level 93%, at community level

93%, for better waste disposal, and to use communal latrine

Apart from the practical changes observed on personal hygiene, family and community

sanitations, the extent that members of the community have demonstrated some practical skills to prevent

and protect HIV/AIDS was another component the evaluation has paid attention. To measure this, the

Team included some questions using which data on practice of HIV prevention was generated. As a

practice question, the Team put the following statement to community members: “Have you ever tested

for HIV?” We documented 112 valid responses; out of which 104(88.9%) of them were found to be in the

“yes, I am tested for HIV” category. This is a fascinating result if such a high percentage of the

community members have already known their HIV status given that HIV is claimed as a socially

unaccepted disease in many communities. Though we have this interesting response on HIV test status,

other follow-up questions we made about practices used to protect oneself from HIV infection showed

very fragmented responses and in some cases are contradictory result that brings suspicion concerning the

response on the status of HIV test was exaggerated. For example, there were some respondents who

believed that HIV can be transmitted through shaking hands or greetings.

- 22 -

3.2.2. Improved Livelihoods of Children, Guardians, Youth and PLHIV

Under the outcome indicators of “improved livelihoods” the project set out different objectives to

achieve. These objectives include; increased income and savings, improved access to basic services as

the result of improved household income, increased consumption and nutrition among PLHIV and their

families, increased employment rate, develop marketable skills and decreased youth delinquency. The

evaluation has documented plausible results to verify the successful achievements of each of the above

objectives.

Increased Income and savings: Quantitative data were generated from the guardians, youth and PLHIVG

and qualitative interviews were conducted with TSW and street children to generate data on livelihood

related activities carried out by JeCCDO. The team asked the different groups regarding their current

income, expenditure, and saving statuses. The guardian’s survey documented that 34% of them have

monthly income of less than 300 Birr and other 32% reported monthly income of between 300-600 Birr.

Majority of the guardians who participated in the baseline survey reported that their average income was

found to be 374 Birr. The evaluation report documented that majority (65.8%) earn more than 300 Birr

out of them 24% earn more than 1,100 Birr per month.

It is fair to say that saving is a good indicator of livelihoods improvement among the guardians.

Out of the total 38 guardians who participated in the survey, 32(84.2%) reported they had savings of up to

birr 500/month. The place where guardians save their money indicated that 14(36.8%) of them saved in

ACSI, whereas, four (10.5%) of them saved in one of the formal banks. The rest of them used to save in a

form of revolving fund within their SHG

The income level of the youth who participated in the evaluation is an indicator of changes not

only in income level among the youth but also occupational status. The youth survey result indicated that

80% of them earn more than 500 Birr per month and . 30% of them earn more than 1000 Birr per month.

From the total youth participated in the baseline survey, 24% of them reported they did not have any jobs

and nearly 36% of these youth needed business skill training. Saving is another important indicator of

livelihoods improvement. From the 25 youth who reportedly involved in IGAs, 16(64.0%) of them had

the practices of saving certain amounts from their monthly incomes. For example , eight (32.0%) of them

were found to save less than 75 birr per month, two (8.0%) of them saved an amount which ranged from

75 birr to -150 birr, and six (24.0%) of them saved more than 150 birr monthly. Here, the most important

thing not the amount of saving but the youth’s knowledge about the concept of savings and the practice

and habit of saving developed among the youth.

The PLHIV survey documented that 17 (85.0%) of them reported they live on their own business.

The maximum amount of monthly income earned by these respondents was Birr 500.00 and majority

(75%) of them were not satisfied with their monthly income. The baseline survey reported that the

monthly average income of the PLHIV was 215 Birr. This implies the project brings some change in the

monthly income of PLHIV as compared to the situation at baseline. Although the majority of PLHIV

reportedly generate less income compared to other target groups involved in income generating activities ,

they have experience of savings. Nineteen (86.2%) of them reported they save some amount of money out

of their income. The amounts of monthly savings range from Birr 75 to 150.

Given that only small number of street children was addressed through the livelihoods project,

only two children who had IGAs have participated in IDI from them the Team gathered data on their

stories of success/challenge of participation in IGAs. One of the two street children who participated in

the interview reported that he received 2000 birr financial support from the Project and started shoe

shining activity. Currently, the boy was able to pay for house rent of 1000 birr monthly. The same boy

reported that as the result of continuous advice made by the Project about the advantage of savings, he

was able to make a daily savings of 10 birr. The boy attributed his business success to the support made

- 23 -

by the Project. He stated “the quality of services I received from Jerusalem [JeCCDO] was very good.

The services transformed my livelihood to a better position. I received training on basic business skills,

financial management, identification of different types of IGAs and other potential engagements to make

a living.”

Similarly, other street boy who participated that he was happy when he received the startup

capital of 2000 birr free of interest from the CBO [in a different modality than the money disbursed by

ACSI in a form of loan]. The boy received the loan after participated in a number of training sessions

that helped him to learn about how to respect work, how to identify the appropriate IGA, and how to save

from one’s daily income. This young boy started his business in August 2015 and did not speak much

about profit and saving.

The Team conducted IDI with two TSWs to learn about their business story. A transactional sex worker at

the age of 25 and who participated in IGA with the support of the Project reported that she has got the

opportunity to participate in business skill training, health education and access to loan. She said:

What I think now is I should change myself from the present life style [as transactional sex

worker]. If I continue in this life, I know that I will get worse in my health status. I used to

live as a transactional sex worker because I did not have any other means of income.

Thanks to Jerusalem [JeCCDO], my life is changed. Along with my friends, I was selected

to participate in IGAs. First, I took training on business identification, marketing, and

savings. Then, I got loan using which I started to produce tela and areke [local alcoholic

beverages or drinks]. ACSI also participated in the loan process. There was no problem to

get the loan. I am paying 275 birr per month to ACSI and I am also making some savings

through ACSI and Equb [traditional saving]. Through my income from selling of local

alcohols, I pay 1050 birr for house rent per month and also I feed my children and my

grandmother. After I have engaged in the IGA, life is changed to the better, thanks to

Jerusalem [JeCCDO].

The other TSW reported that although she took training to start business, she did not successfully

start the business since she disliked taking loans in group. This woman wanted to get money not in a form

of loan and thus she did not take the loan. However, since she took good training on personal health

protection, income management, savings and positively living with HIV, she felt that she was one of the

major beneficiaries of the Project and her life changed after she took the training.

The Team’s interpretation with regard to improvement of income and savings is constructive.

Despite the fact that all the groups participated in the project were from the disadvantaged segments of

the society who lack the means to satisfy their basic needs, in this short project period, it become evident

that, many of them have become successful to increase their income and developed the culture of savings.

What is more important is not the amount of money reproved as savings. But the most important thing is

that the participants developed the practice and culture of savings, which can bring much difference in

the lives of the respondents and their household members.

Improved Health and Nutrition of PLHIV: PLHIV participated in the evaluation were asked how they

evaluate their health status since the time they became beneficiaries of the Project . All the respondents

reported they knew their updated CD4 status, which is an indication that PLHIV have developed a

positive living with the virus.. About 18(75.0%) of the respondents reported their CD4 counts improved

from time to time which, in turn, showed improvement in immunity. All of which can be accounted to the

support they have received from the Project under evaluation. All the respondents took ART on regular

basis. This is another indicator for increased adherence to ART and understanding of the nature of the

virus and experiencing positive living.

- 24 -

The other problem of PLHIV as indicated in the baseline report was lack/limited nutrition. In response to

this, the project has taken steps including direct nutritional support, and involving PLHIV in IGAs. To

measure how far the interventions increased food availability and improve nutrition, PLHIV were asked

questions that generate data on the status of food supply and daily intake. The levels of improvements

regarding food supply and nutrition status of PLHIV are presented in Table 3.2 below.

Table 3.2: Measures of food availability and in-take by PLHIV

Measures of food availability and in-take Count (Yes) %

Food availability in the household increases from time to time 24 100.0

There is some food surplus left from daily consumption 8 33.3

The household has access to fresh and nutrient food such as vegetables, animal products,

cereals and other processed food items

17 70.8

The household is stable in food supply and every member in the household eats at least

three times a day

20 83.3

Members of the household can make preferences of the type of food they can take since

there is a regular supply of variety of food types

22 91.7

The nutrition support from JeCCDO helped me improve my health 24 100.0

It is a great achievement compared to the baseline situation during which 71% of them had the

life experience where they run out of adequate and nutrient food. The following story is a testimony made

by one of the ladies participated in the IDI conducted with the group which strengthens the survey

findings as summarized in the above table.

A Story of Women Living with HIV

As a women working as transactional sex worker, life was challenging for me. I was ill for several weeks during

which I passed hard times. But I did not know what caused my illness. It was during my stay at the hospital; I came

to know about my HIV positive status. Following the intensive counseling I received from the professionals, I made

the decision to struggle to overcome all the challenges. The stigma and discrimination related to HIV was highest in

the village where I used to live. When I shared the news of my HIV positive status, my Mom immediately

abandoned me. The only support I received was from my elder sister.

I heard that Jerusalem [JeCCDO] is supporting HIV positive people through the association of PLHIV and I was

registered in the Association so as to receive the support. Soon after I started getting the nutrition support, advice,

and proper use of ART, I got fully recovered. The support I got from Jerusalem was not only in kind. I got training

in life skills and business skill which opened my dream to start business. Currently I am working and make some

savings.

Develop Marketable Business Skills (MBS): Marketable business skill training was assumed by many of

the evaluation participants as the most sustainable and important part of the livelihood activities. The first

question we presented to all those who participated in IGAs was “have you ever participated in MBS

training?” Amongst the total of 38 OVC guardians who participated in this evaluation, 31(81.0%) of them

have benefited from the marketable business skills training organized by JeCCDO. Regarding the

contents of the training, the respondents answered that it was more comprehensive and need-based. On a

similar vein from the 27 youth who participated in the survey, 25(92.6%) of them attended marketable

business skills trainings (MBST) organized by the Project. Out of those who participated in the training,

24(88.9%) confirmed that the trainings were jointly organized by JeCCDO and ACSI. The majority of the

respondents agreed about the comprehensiveness of the training contents. It was reported in the baseline

survey that from the youth who participated in the survey, 35.7% of them were in needs of business skill

- 25 -

training. Therefore, the livelihood component of the project has played important role in development of

business management skills among young people of the target community.

Among the 24 PLHIV who participated in the survey, 18(75.0%) reported that they participated

in MBST. The majority (88.9%) of them reported that JeCCDO organized their training. The contents of

the MBST were also identified by PLHIV as comprehensive as it was reported by the OVC guardians and

the youth group. Apart from its contents, the PLHIV further reported that the training delivery system and

other modalities considered the special capacity of PLHIV as confirmed by 17(94.4%).

Participation in IGAs as a Sign of Access and Control over Resources: Among the groups engaged in

livelihood activities all were dominated by female targets. Generally, from 77 respondents drawn from

three groups (PLHIV, OVC guardians and youth, 64(83%) were females. In specific terms, among 37

OVC guardians who involved in private business (IGAs), 35 were females. Similarly, from 17 PLHIV

who reported engaged in IGAs, 15 were females. On the other hand among the 23 youth who reported

involved in IGAs, 14 were females. Gender specific analysis on IGAs and related resource control tells

that in this particular project women and girls dominate in proportion.

The IGAs become a means to many OVC to participate in resource control and decision making.

Among those 35 guardians who reported they engaged in the business, for the majority (48.6%) the

source of start-up capital was loan from JeCCDO facilitated through ACSI. The credit makes many of the

OVC guardians to develop confidence to run independent business with full responsibility and

management of the resources. For example, 16 of the total 17 OVC guardians who took loan from ACSI

and participated in the survey have the confidence that they have the capacity to pay back the loan. Group

level decision making was strengthened among the OVC guardians as the result of their participation in

the IGAs. OVC guardians established their own SHGs which facilitated to share information and solve

common problems that might affect their individual business and other personal and family lives. The

data on participation in self-help groups (SHG) indicated that 15(42.9%) of those 35 guardians who got

involved in IGAs were also members of SHGs. All those who are members of SHG believed that their

SHGs could function independently in the absence of JeCCDO’s support. Apart from IGAs support

among group members, 13(86.7%) agreed their group provided social support to its members.

Like that of the OVC, from those 23 youth who reported their participation in IGAs, 11(47.8%)

of stated the source of business start-up capital was loan from ACSI. For the 15(57.7%) of the youth

participated in the survey and users of loan from ACSI, it would not be possible to start the business

without the support they got [the training and loan from JeCCDO and ACSI], and also their family would

have been without adequate food. This is one of the strong remarks made by loan users about the supports

they benefited from the project.

The under review/evaluation also created access to resources and able to control same. Nearly

half (47.3%) of PLHIV who involved in IGAs, were beneficiaries of loan from ACSI. From those who

took loan from ACSI, seven (36.8%) were satisfied with the amount of money they received and 10

(52.6%) of them were confident they would pay-back the loan according to the schedule. Furthermore,

11(57.9%) of them informed the evaluation Team that had it not been the support from JeCCDO who

facilitated MBST and accessed to loan, they would not have been able to feed themselves and their family

members. Significant proportion (64.3 %) of the respondents had the plan to take another loan in order

to expand their business.

Participation in IGAs becomes a good indicator of the disadvantaged groups (OVC guardians,

PLHIV, youth, TSW) to get access to resources. Since many of them are also women apart from being

under the disadvantaged groups of the society, the project in general and the IGA in particular,

demonstrated that JeCCDOs objective to access and control over resources by gender category is

- 26 -

achieved that benefited many women in the target groups to receive loans, start their business and

exercise both independent as well as joint decisions over the resources generated from their business.

Decreased Youth Delinquency: One of the objectives of the project is to see youth delinquency

behavior is decreased at the end of the project period. The plan was to engage youth in IGAs and other

social functioning activities so that they can be protected any anti-social behavior and practices. We

coined a number of survey questions that could measure the degree of positive behavior change among

youth towards their own life and participation to change the life of other vulnerable groups (such as

children, elders and PLHIV). All youth participated in the survey agreed that young people’s behavior

and practices towards caring for others were positively changed in their locality. According to the youth

members participated in the survey, the anti-social practice within their neighborhoods decreased from

time to time. At this point in time, the neighborhood is not any more a risky place to walk around dark

corners in evening. It was impressive also to document that 88.9% the respondents agreed that young

people identified themselves as resources for their community and started providing free labor and other

services while there were community-based activities such as cleaning and disposing wastes.

A similar view was shared among the key informant from the youth association as well as from

the FGDs conducted with the youth and community groups. During our data collection, we observed that

the youth association members were organizing and conducting music shows in order to engage in

edutainment for effective implementation of its objectives in relation to HIV awareness creation and

IGAs. The members of Temsalet Youth Association with whom we conducted interviews explained that:

We are participating in different types of community development activities. We will also

continue to bring timely and tested experiences on HIV/AIDS to the attention of the local

community. We also organize conversations about interesting and inspiring issues on

social development, presenting very interesting history of successful personalities from all

corners of the world, sharing information on entrepreneurial facts which have not been

undiscovered and required due attention by the community and the local administrators.

Female youth FGD participants witnessed that the youth’s behavior had been changed and

become job creators more than job seekers these days, as they stated:

There have been positive changes towards job creation. The youth groups who participate in

IGAs facilitated by JeCCDO in partnership with ACSI, and others in their business activities who

produce and sell various types of commodities, equipment and materials which are different by

their models, styles and qualities, are becoming successful. By so doing, some of the young

people managed to make a huge amount of money out of their business.

Female youth members who participated in the FGD also described the level of youth

participation in the community’s social care and development when they stated:

We, the youth, are actively participating in the development agenda of our local community. We

engage in mobilizing the local community members, creating awareness about HIV/AIDS,

reproductive health maintenance of very old housing units in which poor and elderly people are

living, supporting and caring needy and vulnerable people, creating awareness about disability

and disabled persons, participating in behavior change communication) and asking people and

issues in polite manner.

Data from the youth survey confirmed that youth members become active participants of local

activities which demonstrated the development of positive youth behavior. For instance, from those who

participated in the survey, 51.9% reported they were members of local anti-HIV/AIDs club. In addition a

- 27 -

similar proportion of respondents participated in local resource mobilization during the last 12 months

prior to the commencement of the evaluation. During the project period, about 96% of the youth survey

respondents reported they have participated in various community service such as waste disposal,

cleaning pavements of roads, housing renovation, etc.

From the above paragraphs, the evaluation team can make the following remarks regarding the

reduction of youth delinquency. The active participation of youth members in IGAs, as well as other

community development activities put the group to bring behavior change concerning their personal life

as well as in the care and support of others. The active participation of the youth in local clubs help

community members to see the youth not as dependents but as functional unit of the community. In

addition, the youth themselves realize that their involvement in the production of goods (IGAs) and

services (participation in local resource mobilization and care of the needy) are vital to ensure community

well-being.

3.2.3. Community Cared for OVC and Needy people

The third expected outcome as indicated in the project document is that at the end of the project

period communities effectively cared for OVC and needy people. To achieve this outcome, JeCCDO put

specific outputs to which include, change in behavior and attitude among the communities on care and

support, specific number of vulnerable people cared by communities, children’s rights being protected by

the community, participation of community members in committees, building community capacities and

functional community systems and procedures put in place within CBOs. The evaluation has assessed the

achievements of these objectives as summarized in the following pages.

The Project document identified that the community had limited capacity to support OVC, the

elderly and PLHIV who needed support. Although communities have their own institutions such as

Iddirs, youth and women’s associations, they were with insufficient capacities to deal with the

community problems. Another gap indicated in the baseline report was about the CBOs’ capacity which

lacked well-established systems that could guide the institutions. Cognizant of the limited capacities of

the community institutions to care for children and their care providers, as well as PLHIV; JeCCDO has

identified a number of capacity building activities by setting targeted outcome as described above In

order to measure the extent to which JeCCDO has implemented its planned activities and achieved the

targeted outcome, we have made an extensive discussion with community institutions (CBO/Iddir

leaders), school directors, steering committee and stakeholders. We also conducted observations of the

capacity building activities on schools, renovation of Iddir meeting hall, training of different community

groups, and the mobilization of community resources, including labor contribution to Project activities.

In order to gauge views about changes in terms of capacity building of the community and its

institutions, we raised relevant questions to the community survey respondents using the following

indicators.

a. Strengthening community systems, awareness & care;

b. Provide care & support, awareness and promotion on community health & hygiene;

c. Establish waste management & disposal systems & level of volunteerism; and

d. Strengthening of CBOs and CDA.

According to the survey data, more than 90.0% of the respondents believed that JeCCDO’s

contributions to building the community capacities were successful. Apart from the above facts presented

from the community survey responses, FGD participants witnessed that the Project provided important

support to strengthen the community structures, including Iddir facilities. Some of Iddir facilities such as

St. Michael Iddir Meeting Hall were renovated and providing multi-purpose services to the community so

as to generate some income which the community has already start to spend part of the income to help

- 28 -

OVC, PLHIV and other needy people in their neighborhood. The Team had also observed the renovated

Meeting Hall and its internal facilities as presented in the figures below.

Fig. 3.1: Inside view of St. Michael Meeting Hall and year of establishment

Apart from the Iddirs, the Project has strengthened a Community Development Association

(CDA), which is serving the entire community of Kebele 02. This association had its IGA activities

through which it generated income to support the needy. The Kebele administration also provided the

association with an office. The association closely worked with the Community Care Coalition (CCC),

which was a newly established structure by the government at the community level. The CCC and CDA

closely worked together to recruit service recipients such as those who participated in IGAs, direct

services and educational supports.

Other capacity building activities took place in two primary schools (Bruh Tesfa and Model

number II schools). In these schools, JeCCDO has invested to strengthen early childhood education. The

Directors at Biruh Tesfa Primery school reported that the support from the Project helped the Schools to

strengthen early childhood education which resulted to attract many poor families to send their kids to the

pre-primary education being provided in these schools. As a result of the support from the project , those

abovementioned two schools became primary choices to those families who used to send their kids to

private kindergartens. For example, as the result of school capacity intervention made by the project, the

number of pre-school children enrolled to Bruhtesfa primary School increased three-fold from 73 prior to

the intervention to 254 at the time of field work conducted for this particular evaluation. The Team’s

observations also witnessed that the two schools tried their best to make the environment more conducive

encouraged parents to send their children who sought for pre-formal school services. However, we also

observed that Bruhtesfa Primary School had well-organized facilities compared to the Model number II

Primary School. The following pictures display the differences clearly.

29

Fig. 3.3: Internal facilities arrangements of the two schools received capacity building support

Bruhtesfa first cycle primary school Model No. II first cycle primary school

The above findings on community efforts to serve the needy particularly OVC, PLHIV and other

groups through local institutions including CBOs and schools demonstrate the objectives described at the

beginning of this section are successfully achieved. The strengthening of CBOs and the efforts made to

generate income through renting of Halls shows behavioral changes among community members for the

care of OVC and other targets. Commitment and capacity of Iddir leaders to liaison with local

government structure and able to use resources such as office spaces indicate that functional community

systems and procedures are being in place.

3.2.4. Improved Partnership Through Joint Learning

Strong partnership is a sign of project sustainability since partners shall be aware of their

responsibilities and participate accordingly across all stages of the project’s life. Cognizant of the

importance of partnership for project sustainability, JeCCDO has described in the project document,

Improved Partnership through Joint Learning as one of the four project outcomes. To ensure the

realization of this outcome, specific objectives were defined, which include; enhanced ownership of

project activities, strong team spirit among stakeholders, and willingness of stakeholders to participate in

scaling up of program activities.

JeCCDO had initiated partnership with various stakeholders by bringing together different

institutions to form a consortium through which joint learning can be achieved. As the team reviewed the

baseline survey document, project reports, minutes of different meetings there have been various efforts

made to bring together government institutions, NGOs, the private sector, higher learning institution and

CBOS, so that the formation of sustainable and improved partnership could be possible. Some indicators

of the efforts made to make stakeholders actively participate in the process of improving partnership

include the following.

• Debre Berhan University was given to conduct the baseline survey and market assessment on

opportunities and training needs of IGAs

• A taskforce that brought together GOs, NGOs, private hospital, PLHIV association, CBOs,

micro-finance institution, etc, was formed and become functional

• Amhara Saving and Credit Institution (ACSI) received the responsibility to administer the loan

disbursement and payback process

• Continuous activity performance review meetings held by the taskforce

• The CBO committee and the CCC jointly recruited project beneficiaries including loan

beneficiaries

30

The above activities demonstrate the steps taken by the project to realize the joint learning

through which improved partnership shall be created that become responsible to takeover caring for

vulnerable groups after the project period. However, data from different sources (taskforce FGD,

interview with government experts and CBO committee) indicated that the strength of the partnership

did not move as expected. Therefore, the evaluation Team has the view that the fourth outcome

(Improved partnership through joint learning) is not achieved to a satisfactory level compared to the

remaining three outcomes. Some of the challenges that hampered to see a strong partnership is

described in chapter four under the “challenges” section.

3.3. Project relevance, effectiveness and efficiency

In this section we will present three important components of the project evaluation-relevance,

effectiveness and efficiency-which provide important information to conclude whether the project has

achieved its long term goal and short term outcomes as stipulated in the project document. Theoretically

project relevance is defined as “statement of benefit to be gained and problems to be resolved with a help

of the project.” On the other hand “effectiveness measures the appropriateness of the goals that an

organization is pursuing and the degree of pursuing these goals.” Effectiveness is all about applying

knowledge, tools, techniques and resources. Finally, “efficiency measures how well and productively the

project use its resources (time, money, staff and material) to achieve goals.” With this general

understanding we will present summary results on project relevance, effectiveness and efficiency as we

gathered data from the qualitative interviews, observation and review of relevant documents such as

financial and physical reports.

3.3.1. Project relevance

Concerning project relevance, data were gathered from project beneficiaries, JeCCDO’s staff

members, Steering Committee members, government stakeholders and other participants of the study. As

stated above, project relevance is seen from the project benefits that the target groups have received. In

this connection, the Team had conversations with selected community leaders to tell about the relevance

of the Project components to the direct beneficiaries and community development needs. According to the

opinions of the community leaders, each Project component brought meaningful changes to individuals,

families and groups who participated in the Project. The activities of the Project have addressed the most

pressing needs of the target groups. They further mentioned that the nutrition support enabled PLHIV

who were bedridden due to sickness and lack of adequate food [in quantity and quality] and who later get

better and started working. The pre-school support enabled poor families to send their children to a place

where the kids could be protected safer and attended pre-school classes, while parents/guardians were

performing their work. Other activities such as health education, construction of communal latrines and

waste disposal systems solved many health problems. Most importantly, the support for OVC guardians

and youth in terms of IGAs brought significant livelihoods changes among OVC, youths, PLHIV and

TSW. Therefore, it was their comment that JeCCDO has brought unfamiliar approach mainly in working

with microfinance institutions through which loan is provided to target groups instead of direct

assistances . According to the views of some participants, the new approach is less relevant at this time in

point since the target groups would prefer to receive continuous direct support than the loan.

JeCCDO’s Project staff members fully agreed that the project was relevant since each component

was identified by joint actors (community members, local administration and other stakeholders). In

addition, the Project had focused on integrated and holistic approach through which it identified different

activities that brought clear change among the lives of vulnerable groups. For example, through direct

nutrition support, those bedridden HIV patients recovered from their sickness and joined the IGAs

through which they started to assist their OVC. Such integrative and holistic approach made the Project to

be considered as relevant.

31

Members of the stakeholders to the Project who participated in the qualitative interviews agreed

that the Project components were relevant to the local community since the objectives were to address the

pressing needs of the target groups. Specifically, the Project components such as support to OVC, PLHIV

and the youth IGAs were found to be more relevant. Furthermore, all interventions of the project

including the community capacity building activities were in line with poverty reduction and

development priorities and strategies of the government. Our evaluation on the Project relevance was

based on document review and consultation of the Project beneficiaries and stakeholders enabled us to

appreciate the integrated and well thought approach. First of all, the Project Components emerged out of

problem analysis and needs assessment as well as baseline information that facilitated target identification

and the design of the Project activities. The link made between HIV and livelihoods improvement make

the Project more relevant given that improvement in livelihoods has proved to be an effective strategy to

reduce vulnerability to HIV infection and associated challenges such as stigma and discrimination.

Although child support is central in JeCCDO’s development approach, the efforts made to link child

support with family empowerment and community capacity building strategies make the Project more

relevant apart from its innovativeness and effectiveness as we will discuss later.

Regarding the relevance of the project to the national and regional policy levels particularly

adopting a new partnership with micro-finance institutions to ensure sustainability of loan disbursement

and payback system, which had not been used by NGOs in most of the cases, the Team acknowledges the

efforts made by JeCCDO and it is highly relevant approach. However, this approach was not appertained

by some of the beneficiaries as well as members of the taskforce.

3.3.2. Effectiveness

Effectiveness as we described earlier is a measure of a project against the appropriateness of the

goal. The goal of this Project as presented in the Project proposal/document was to “improve wellbeing of

HIV affected children through economic and social empowerment of their guardians and communities”

with four specific outcomes that are supposed to lead to achieve the broader goal. From the views of

community leaders, the Project components were found to be effective while gauged from different

perspectives. The leaders ascertain that project beneficiaries became more assertive to claim about their

rights and to pursue their responsibilities compared to the prior Project implementation period.

Furthermore, some of the innovative components of the Project, including participatory capacity building

activities made for the CBOs and the Schools attributed to the development of positive attitude towards

development. The CBO leaders also brought another example to witness the effectiveness of the Project

and then stated that “...women who used to practice transactional sex work have changed to become

women’s peace team/group members and started to engage in production of goods and services by

leaving their transactional sex work.”

However, the community members questioned the effectiveness of some Component of the

Project, particularly when they viewed the recruitment and selection of “real vulnerable members” of the

community. Some individuals participated in the community FGD as well as Steering Committee

members complained that some most destitute groups had been left out in the process of beneficiary

selection. According to this informants, , such “unfair” process could have reduced the effectiveness of

the Project. However, the evaluation Team could not find any evidence from the community survey as

well as other specific groups to confirm the complaint .

Given that there were a number of challenges encountered in the due process of the Project

implementation, the overall conclusion of the evaluation Team is that, the Project is found to be effective

in reaching the target groups and achieving the set outcomes stated in the Project document. The

effectiveness of the Project is more visible in addressing the pressing needs of children via integrated

approach, including direct support to OVC, reducing vulnerability to HIV and other communicable

diseases, community capacity building, school capacity building assisting different groups to participate

32

in IGAs and strengthening local networking by bringing different stakeholders together to act in a form of

consortium.

Review of the three year (until June 2015) Project activity reports indicate that the output/result

level measure of the Project can be generally characterized as effectivel. Few examples of output level

success are presented as follows. Capacity building of two pre-schools was achieved as planned during

the project period. Regarding educating the community on waste management and disposal, 1764 of 2000

targeted community members (88.2%) were reached. From 300 individuals targeted to take loans 261

(87.0%) took the loans. From the total of 1,500,000.00 birr allocated for loan, 821,000.00 (54.7%) were

disbursed, On the other hand, the report indicated that a number of capacity development activities were

made to the community development association (CDA).

3.3.3. Efficiency

Efficiency measures how well and productively the project use its resources (time, money, staff

and material) to achieve goals in a cost conscious manner. The community leaders articulated that the

approaches chosen to meet the needs of the target groups were innovative and appropriate in terms of

period of implementation and financial allocation. The budget allocated for the Project was used in

accountable and efficient manner. However, the community leaders commented that the procedures

adopted to facilitate loan services brought unintended consequences. One of the strong complaints from

the leaders regarding the loan was inadequate communication between the Project Coordination Office

and the beneficiaries regarding the loan modalities which reduced the efficiency of the project

management. The interest rate was found to be higher than expected and did not communicate to the

borrowers properly.

Youth participated in the data collection process for this evaluation reported about inefficient use

of time, especially in relation to the IGAs. The loan releases process had some challenges which occurred

as the result of misunderstanding between the beneficiaries and the Institution that provided the loan. As a

result, the actual loan disbursement was delayed almost by two years until the misunderstanding was

resolved. Data indicated that majority of the loan beneficiaries took loans towards the end of the project

period.

Many of the groups who participated in the interviews [community members, CBO leaders,

school principals, and Steering Committee members] witnessed that in other areas of the Project

Components such as renovation of houses, school capacity building, construction of communal latrines

and availing of waste disposal bins; the Project was efficient since it used local resources, including

existing community structure and local government facilities.

We found that the loan disbarment schedule was not as efficient as planned due to a number of

factors, including miscommunications created between the beneficiaries and the Project holders. Specific

to the IGAs, as one of the components of the HIV and Livelihoods Project, not all the planned targets

were reached and according to the information from the project staff, there was still more than 500,000.00

birr in ACSI’s account which was supposed to be distributed among the community members who were

interested to take loans. ACSI’s representative who participated in the FGD facilitated for the Steering

Committee members confirmed that loan distribution would continue as long as the applicants fulfill the

required criteria set by the Project. Concerning the money, it will continue to be used for the same

purpose as planned during the project life period; this will still be well thought-out as an indicator for

efficient utilization of the financial resources although time is a factor of efficiency.

The loan distribution process was very painful as described by other JeCCDO staff members both

from the Project and Central Office. According to them, there difficulties to convince the beneficiaries. At

the stage of beneficiary recruitment, there could be information gap within JeCCDO. During the initial

33

stage of the Project, the staff members of JeCCDO at the project site might not have the adequate

knowledge that loan will be distributed according to ACSI’s Loan Policy. As a result, they could have

been informed potential users about availability of money for loan which would follow similar procedures

as it happened in other JeCCDO’s Projects such as the one in Kebele 06 of the same town town. The new

loan disbursement strategy considered by JeCCDO as innovative approach was not communicated to the

users as expected. The information gap about the nature of loan disbursement was also confirmed by

stakeholders who participated in individual interviews. All these identified gaps might have undesired

effects on the overall efficiency in the implementation of the project components and achievements of the

intended outcomes.

As one aspect of looking at the efficiency of the Project, we reviewed the financial utilization

status. The financial utilization was evaluated based on the data we received from the Central Office of

JeCCDO. The summary is presented in Table 3.3 below.

Table 3.3: Status of Project Financial Utilization

Budget description Planned

budget (in

Birr)

Utilized

budget (in

Birr)

Variance between

planned budget &

actual expenditure

% of budget

utilization

Program cost 6,405,999.00 6,328,238.26 77,760.74 98.8%

Program Coordination & Admin. costs 1,644,000.00 1,649,287.20 -5,287.20 100.3%

Total budget 8,049,999.00 7,977,525.46 72,473.54 99.1%

Table 3.3 shows that the financial utilization report looks perfect and indicates efficient use of

money allocated to the overall project, although disparities are observed when we look at the planned

versus expenditures against major project outcomes as budget items. For example, rate of budget

utilization for outcome No.3 stated as “communities effectively cared for OVC and needy people” is as

high as 107.8%, the same report for outcome No.4, which states as “enhanced partnership through joint

learning” stands at 65.%. Therefore, disparities observed in financial uitlization are also reflected in

physical activity outcomes. In terms of physical outputs and outcomes, enhanced partnership through

joint learning is the area where least outputs are registered as reported by various data sources

(beneficiaries, partners, JeCCDO staff members and members of the steering committee).

3.4. Success indicators (sustainability, Impact and Replication)

Indicators of success of the Project can be seen from various angles. Achievements of direct

outputs/outcomes could be one category of success indicators. Changes in perception and attitude of the

target groups and other actors in appreciating or looking the innovativeness of the Project can be

considered as another success indicator. Measuring the confidence of community based institutions/CBOs

and CDA to continue the existing services to the needy is still another indicator of project success.

Witnesses from the government authorities [at various levels] by appreciating the efforts made to bring

positive change among the life of the vulnerable groups in particular and the community in general could

be the indicator to talk about replication of the existing Project components and the impact of the program

beyond the target groups directly benefitted from the project. Using the above scenarios as checkers of

success indicators, the Project has documented a number of successes as we have generated data from

various sources about the subject matter.

Data from the field interviews indicated that the Project documented attitudinal changes on loan

modalities. The practice of financial support by other NGOs or by other projects of JeCCDO was either

free donation or with some proportion of loan with little or without interest. After a painstaking effort

made by the Project, the Steering Committee and some government stakeholders and members of the

34

community were convinced to accept the loan modality which we believed the project changed the

mentality of target groups from complete dependency into self-reliance. This is a good indicator of

attitude change among the members of the community. However, as to issues of sustainability of project

components, the youth groups have argued that it is early to think about it. They stated “all those efforts

which have been made by JeCCDO to support OVC and their guardians, PLHIV, young people, street

children and the community at large would not have been continued sustainably in the absence of

JeCCDO’s supports in the future.” The concern from the youth was that they are not fully ready as a

group to independently function without established institution like JeCCDO at this moment in time.

They feel, two or three more years are needed for each group to independently function. In the views of

the evaluation Team, due to the delayed take-off time for the IGAs the comments given by the youth

group seem to be valied. Therefore, some more time might have been needed to continue facilitating the

business engagement of the group before a complete project termination takes place

Community capacity is another area about which different community groups participated in this

evaluation gave their comments leaders of the CBOs in the community had firm confidence on their Iddir

capacity to manage available resources and to mobilize additional resources (e. g. finance, property and

human) effectively as it had been expected even in the absence of the Organization to provide support for

OVC, elders, PLHIV, etc. in the future. On the other hand he young people were reluctant to agree

whether or not community capacities to support OVC, PLHIV and other needy people would continue in

the absence of JeCCDO. They believed, the humanitarian feelings among members of the community

were more dominant than the actual and established capacity to continue helping the groups. The Team’s

observation in this regard supports what CBO leaders argue about community capacity. The community

through its CBOs has already established strong system and functional structure which was visible during

the field work. Community members have already started supporting vulnerable groups through its CBOs,

CBOs also established good relationship with the local government structure. Furthermore, CBOs have

sustainable income generating activities. All these together witness existence of community capacity,

although capacity building is a continuous process and the community needs to strengthen its capacities to

support more of its vulnerable members.

The issues of community capacity and sustainability are viewed differently by different group in

the community. Some of the community members strongly accepted the positions of the CBO leaders and

believed that many of the Projects owned by CBOs and CDA would continue sustainably, although they

are not certain about individual and group level activities, including the IGAs. However, JeCCDO’s

Project staff confirmed that community capacities to care for vulnerable groups were built through

improving their structure and internal sources of incme, continuous training and community level

awareness. The CBOs have experts to facilitate the community capacity building process, which is an

indicator of efficient use of human resources.

The Steering Committee members, CBO leaders, and participants of IGAs believed that the

integrated approach JeCCDO initiated in addressing the issues of HIV by improving the capacities of

individual families, groups and CBOs through livelihood improvement was an exemplary and innovative

Project. This approach can be replicated to other communities in Debre Berhan or elsewhere. Many of the

participants in this evaluation drawn from the direct beneficiaries and the stakeholders as well as the

Steering Committee, commented that the loan modality should not be replicated as it is. They appreciated

the steps taken to equip potential loan receivers with basic business skills through training and continuous

advice and support, but the loan was set with the high interest rate. Participants believed that if JeCCDO

will be still interested to assist other community groups to improve their livelihoods through IGAs, it has

to keep on providing the technical and training assistances, but it had to revisit the loan modality,

especially the interest rate and the grace period to start paying back the loan.

Various groups also shared their experiences during the interview that enabling PLHIV to be

productive who were bedridden due to their illness is a rare phenomenon in many other projects. This

35

innovative approach had to be expanded within the present Project site as well as to other communities.

Further, the school principals appreciated the school capacity building activities done to strengthen the

Pre-School Program. If such a support was expanded to other public/government schools in Debre Berhan

and elsewhere, this would make many poor families to send their kids to school. The final outcome is to

protect children of pre-school age from various abuses and helping them to get the necessary preparation

and competence to enroll in the formal school phase.

The capacity building activities done to CBO facilities is confirmed by the community target

groups, Steering Committee and government stakeholders as one of the best efforts made by JeCCDO and

the activities that CBOs have been started are sustainable. By implication, if CBOs activities are

sustainable, other services such as OVC and PLHIV supports will be sustainable via the CBOs assistance.

However, the concern about sustainability of the specific supports is that, CBOs need to develop strong

structure which focuses on community support rather than simply focusing on strengthening financial

capacities of these CBOs. Many of the groups participated in the evaluation agreed that, it was not the

time to comment on impact factors as some of the Project components such as IGAs were just begun.

However, the evaluation Team has the view that there are a number of changes observed at

individual, family and even community levels which can be cited as the beginning of the impact of the

project on the lives of different groups. For example, some of the individual level impact include, the

TSWs’ decision to transform themselves from staying in the sex business to participate in other forms of

IGAs, street children’s commitment to withdraw from street life and start business, the PLHIVs’ decision

to become productive rather than staying as aid recipients, etc. The family level impact of the project can

be seen from different dimensions. For example, parents/guardians change of behavior to give attention

for the care and right of the child over all other family issues, their commitment to send children to pre-

school education, and commitment to dispose their dry waste in a proper manner, etc. Community level

impacts of the project is very visible. The establishment of sustainable IGA source is one of the visible

impacts of the project. The schools’ capacity to accommodate more children in their pre-formal school

education program is the impact of the project. Another important statement about the project impact is

the link created between the direct supports with that of capacity building at individual, family or

community level which reduce dependency syndrome and opens the eye for project beneficiaries to

participate in development activities, whatever a small scale level it may be.

36

Chapter 4: Innovativeness, Challenges Experienced and Lessons Learned

4.1. Was it an innovative project?

In the opinion of the evaluators, the project proposal was well written, had coherence and defined

every section in communicative and transparent manner. The section that described about innovativeness

had provided sufficient explanation on the concept and specific definitions tailored to the project

objectives. Accordingly, JeCCDO has defined the term innovation or program innovation as “the use of

better or more effective process, ideas, services, etc. that are accepted by communities, government

authorities, NGOs and business groups.” Innovation is seen in detail as: (a) move from giving out

something to beneficiaries to working with them in all phases of program cycle management, (b) looking

community in totality to address their development constraints instead of targeting a specific problem, (c)

focusing on brining change (result) at the level of individual, community and stakeholders, (d) sharing

knowledge and expertise among stakeholders, (e) build confidence and trust, and (f) active use of

documents, information, knowledge transfer and networking, .

JeCCDO’s staff at the project level believed that implementing multi-stakeholder project

planning and management could be seen as one indicator of innovativeness in this project. This project

from the beginning made the Kebele administration, the local community organization called Yelmat Frie,

Association of PLHIV, Advisory Board/Steering Committee, other stakeholders including government

experts, Debre Berhan University and the private sector to participate in the process. Participation enabled

the project to create synergy and enhance partnership as well as ownership among stakeholders.

Brining ACSI as a business institution to play important role in the process to MSB training and

loan disbursement as well as loan collection has never been tried by any other NGO or by JeCCDO in

other projects as far as the knowledge of the evaluation team is concerned. This makes the project to be

innovative. The association made between subsidy/direct support with the idea of saving among the OVC

and their guardians taught them to practice the concept of “start small and grow big.” The savings

encouraged some families to start small IGAs using their savings. As part of encouraging individuals to

think about independent life through self-employment is one innovative idea of the project.

The links made between the CBOs, volunteer associations and the community care coalition as a

single entity to recruit project beneficiaries, assist the Kebele to address the neediest individuals and

families encouraged collaboration and sense of ownership. These demonstrate sustainability of some

working modalities after the project period and we can see this as innovative idea to ensure sustainability.

Although JeCCDO has these and many more innovative ideas being practiced as project

components, the gap we identified was the strategies applied to diffuse the innovation. There are many

golden innovations that come to the scene by many, but usually failure occurs due to inadequate diffusion

processes. We summarized the gaps in relation to diffusion of the innovations and other related challenges

in the following section.

4.2. Challenges Experienced

This evaluation has identified some areas which have gone in a different dimension more than

what the project was expected to achieve. One of the important focuses of the project was to build the

community’s capacities not only in material terms but also morally, and attitudinally. Community

resistance particularly in the loan modality was identified as one of the challenges the project has

37

experienced. Apart from the challenge in relation to the loan, community members have also expressed

their fear about the existence of community capacity to the level that it can sustainably support the

vulnerable members, although Iddir leaders are confident their institutions will manage to support

vulnerable groups. Here are the statements made by one of the participants in the community FGD and

which were authenticated by all others attended the discussion. He stated:

Truly speaking, at present our capacity has not yet reached to the level of sustainably

supporting the needy people. Resource limitation is among the factors. We suggest

JeCCDO to continue the interventions in the Kebele and further developing the capacity of

community. A good system to be worked out how the community structures able to support

the needy people.

Barriers of communication between the Project Coordination Office and IGA beneficiaries found

to be another area that couldn't be properly addressed. It is the conviction of youth who repeatedly

complained that they had been misinformed about the loan modality. According to their explanation,

initially they were told that they will pay back 40% of the loan and will own 60%. With this

understanding, they participated in BST and latter on they were told the loan would be paid back 100%

with interest. According to female FGD participants from the youth group, such communication barrier

developed distrust. They added, “although the modality was very innovative to connect the loan with

experienced micro finance institution (ACSI), the organization was stronger to chase defaulters than

providing the necessary technical support while they are on business activities. ACSI’s staff came to us

by motor bicycle to undertake a house to house visit to remind us to pay back the loan. They didn't

regularly visit us to encourage and advice us on our business.” The evaluation team came to the

understanding that JeCCDO had lacked sufficient experience in loan disbursement through micro-finance

institution. We believe that ACSI might not develop sufficient skill to deal with the social components of

changing community attitude from dependency to assuming responsibility for self development. That has

brought a serious gap to communicate the innovativeness of the loan modality to the beneficiaries and

make the necessary follow-up to document the process that will create the forum to learn for

improvement. In short JeCCDO was not sufficiently prepared itself and develop the required capacity

how to diffuse innovations, how to monitor the diffusion process and how to learn out of the process of

such diffusion of innovations.

On the other hand, participants of the evaluation from JeCCDO’s Project Office confirmed that

the loan system strictly followed the government development strategy and it benefitted many of the

community members to start business, without such support they could not have been able to start their

IGAs. However, the staff believed that previous experiences by JeCCDO itself and other NGOs might

have brought dependency syndrome which was reflected on this project. JeCCDO and ACSI repeatedly

provided trainings to address dependency syndrome. According to the staff member, ACSI was not only

facilitating loans, it also provided training and timely monitoring and follow up. However, the staff

members did not deny the hard time they passed through until loan beneficiaries clearly understood the

objectives why JeCCDO needed to change its loan modalities from the previous experiences.

Another member of JeCCDO staff from the project site stated that the Organization (JeCCDO)

was shortsighted about the complex nature of the project. With the assumption that it would not be a

difficult one to manage by simply considering its small geographic scope, JeCCDO did not take the

necessary preparation to staff the project with adequate and experienced personnel. Such limited

preparedness was coupled with staff turnover as explained by a staff from the Central office of JeCCDO.

This was the internal challenge the project had experienced.

One of the staff members from project site also stated that the project was not as efficient as it

was expected to be since it was implemented in the midst of various challenges surrounding it. For

38

example, in the presence of many actors providing services that aggravate community dependency,

members of the community targeted for this particular project might not appreciate our innovative

approach and did not actively participate in some of the project activities.

The external challenge was the difficulties to break the existing practice of “giving out something

to beneficiaries rather than working with them in all phases of program cycle management.” IGA

beneficiaries were aware of the experience providing business startup capital on non-returnable basis or a

loan with no interest. This becomes one of the challenges for JeCCDO in relation to smoothly

implementing the loan management. For example, JeCCDO itself used to give business start up capital

with a 40% loan and 60% grant modality in other Kebeles in Debre Berhan town. Therefore, one of the

complaints coming from IGA beneficiaries has been, “why we are different from other communities and

put pressure on us to pay a interest and 100% loan.”

Another external challenge was limited participation of stakeholders to own the innovativeness of

the project. The staff of JeCCDO from the project site reported that some government office experts

prefer to invest more time on projects where they could get some personal remuneration than supporting

the project which focuses on participation and community ownership. JeCCDO staff members underlined

that some of these experts became rather, anti-advocates of the innovative approach. The evaluation team

also sensed presence of dissatisfaction from some of the government experts who participated in the

interviews regarding JeCCDO’s position in some of the project components mainly its loan approach and

loan management through ACSI. But the team is also confident to declare that these issues are more

related to personal attitudes of the experts rather than government policies and regulations. The

government policies and regulations are found to be fully supportive of JeCCDO’s project

implementation modality. A challenge from the community perspective was that the project site was the

most fragile one where many social ills are taking place which is difficult to address all problems in this

small project.

Experts from government stakeholders (Health Office and Office of Women and Children

Affairs) claimed that there was no strong coordination between JeCCDO and their offices that make them

less aware of the details of the activities of the project under evaluation4. Although the government

experts expressed there existed less joint planning between the local government and JeCCDO, members

of the stakeholders (drawn from different institutions including government offices, apart from the health,

women and children offices) asserted that staring from the initiation of the project to implementation,

partners had been working together and joint planning was part of the regular exercises.

The final point as a challenge was mentioned from the community members. The community

members claimed that JeCCDO has identified itself different from other local NGOs by adopting the

strategy of assisting the needy not in a form of direct support but through involvement in IGAs (through

100% loan modality), community capacity building, etc. But as members of the community, they said

“we were not adequately informed (lack or limited orientation) the reasons why JeCCDO wanted to

change its previous community support system into a new experience for which we are alien.”

Although a number of activity specific challenges were described by the community members we

involved during the evaluation process, the above-stated are the major ones which draw our attention and

we feel it can help to think differently in future endeavors of similar projects. With the above issues

described as challenges related to process and activities of the project, we summarized the major lessons

learned during the implementation of the project in the following section.

4 However, the evaluation team came to learn that the experts with whom interviews were conducted were new to their respective

offices. Therefore, the team feels that the issue is not poor coordination but less exposure of the experts to learn more about the

project.

39

4.3. Lessons Learnt from the project

The project was innovative, dynamic, and entertained many internal and external challenges out

of which a number of lessons can be drawn. Out of the many lessons one can identify from the huge data

gathered, the following are the major ones.

(1) Integrating direct support with capacity building activities at individual, family and community

levels is the first lesson to be learned as good practices.

(2) The innovative approach JeCCDO has followed ensured the ownership of the community based

organizations to lead projects in the future with minimum or no support since they have already

owned their own sustainable income generating activities, example Kdus Michael Iddir’s IGA

project.

(3) Capacity building and strengthening local networks are more sustainable than direct project

outputs as demonstrated by the capacity building activities made to CBOs, schools and the

strengthening of local institutions such as Yelmat Frie PLHIV Association.

(4) Addressing dependency syndrome is not a simple assignment and JeCCDO did not fully realize

this challenge and did not work hard to raise awareness among the community and different

stakeholders including the local government. Strengthening of inclusive approach and all-

encompassed consensus on the innovativeness of the project components would have better

facilitated the process and reduced the role of "evil-advocates” against the innovation diffusion.

(5) Design for attitude change not only among the direct project beneficiaries but for others who are

outside of the project but who may influence the project implementation and act accordingly is as

important as implementing the project.

(6) JeCCDO integrated direct support with saving practice. We learned that OVC who used to

receive some financial support were also made to deposit a small token of many as a saving,

using which their guardians started petty level IGAs. This is an important lesson to learn to erode

the feeling of dependency through time.

(7) It was stated in the project document that business startup capital would be given as a loan

through ACSI. Especially when we plan to introduce innovative approaches for project

implementation, it is advisable to work harder that every stakeholder understands such new

approach and if possible agrees with the implementation modalities. Otherwise, the resistance to

implement the innovative project implementation modality will encounter resistant groups as it

was observed in the project under evaluation.

(8) We also draw a lesson that the exercise JeCCDO has made on the innovative project demands

innovation diffusion strategies from which important experiences can be shared with project

stakeholders and beyond.

(9) Adoption of innovation takes time; therefore, the project period for such innovative project would

have been different from the conventional project period (three years). Innovativeness should also

include thoughts about project period.

(10) JeCCDO put outcome as success indicator in this innovative project. But project of this type need

to give equal attention to (1) knowledge building about the new ideas, (2) persuasion of potential

users of the innovation, (3) ensuring decision of the innovation users, and (4) implementation of

the innovative project. We learned JeCCDO gave more emphasis to the implementation phase

and little or no emphasis to the remaining phases of those listed above.

40

Chapter 5: Conclusions and Recommendations

This chapter presents two important sections on concluding remarks about the process, outcome

as well as our observations of the project; and about the recommendations on pertinent issues that will

help for the project holder for future planning and replication. The concluding remarks are purely the

views of the evaluation team based on the analysis of data generated through primary sources and review

of documents related to this project.

5.1. Conclusions

The evaluation results summarized in line with each activity component are diverse and in certain

aspects very impressive and existing, while in other aspects it documented variability of feelings of the

target groups. To cite some of the impressive and astonishing results of the project are those in connection

to the impacts made on PLHIV, community capacity and school capacity, about which across all target

groups the results are applauded. Some of the ambiguities about the project are registered in relation to

IGA component where we found various opinions, feelings and positions. The direct beneficiaries of the

project (OVC guardians, street children, PLHIV and TSWs) have appreciated the innovative approach

and their assessment about the benefits gained from the livelihoods activities is positive and promising.

The youth have had mixed feelings of this component of the project. They appreciate the capacity

building components of the project (trainings on skills) but they have not yet bought the modality of

100% loan with interest implemented by JeCCDO through ACSI.

Our conclusion why such a state of variability appears could be attributed to JeCCDO’s internal

factors and the external environment. In our view, JeCCDO has strong passion for adopting innovative

program implementation compared to its technical and human resources capacities at various levels to

think, process, design, implement and to monitor the innovative program approach. Further, JeCCDO

might overlook that in the process of innovation diffusion, individual actors could play significant roles

either to facilitate or to deter the new paradigm. Some agents/stakeholders of the project were not ready to

accommodate the innovative idea that JeCCDO has tried to adopt. There seem to precipitate high

resistance from the external environment which has put high pressure on JeCCDO in a situation that the

organization did not have the appropriate tool to deal with such challenges.

In regards to achieving results the evaluation findings reveal the following:

Outcome 1: “Children and communities accessed quality and adequate services”, the outcome is

achieved at output and outcome levels.

Outcome 2: “Improved livelihoods of children, guardians, youth and PLHIV”, the outcome is

successfully achieved at output level and premature to evaluate at outcome level.

Outcome 3: “Communities effectively cared for OVC and needy people”, the outcome is successful at

output level and partially successful at the outcome level.

Outcome 4: “Improved partnerships through joint learning” on the basis of our data, this is found to be

the area where the project achieved less, except the success documented both at output and outcome level

for such partnership established with CBOs.

Though an arithmetic addition may mislead to our conclusion, the cumulative outcome of Outcome1 +

Outcome2 + Outcome3 + Outcome4 will lead us to measure the project goal. Three of the four project

outcomes are remarkably achieved. This make us to be safe to conclude that as an innovative project,

which was a learning ground for JeCCDO, communities, CBOs and others, the success rate is very high,

41

especially in terms of implanting the innovation to the mindset of the community groups, CBOs and

JeCCDO itself.

5.2. Recommendations

The recommendations are not different from those points appearing in lessons learned section.

However, for JeCCDO to take some practical actions for similar innovative projects, we recommend the

following:

1. The project was implemented to address the complex needs of diverse groups that made the

innovation diffusion more challenging. Therefore, for future similar innovative projects,

narrowing the target groups will contribute to successful implementation of the project.

2. Participatory and development oriented project requires strong pre-project preparation where the

community’s readiness to move from the mentality need-based to asset/strength-based approach.

The later is the sign of strong community. JeCCDO is recommended to establish the forum to

more internalize the basic components of asset/strength based approach and prepare itself towards

implementation of the approach. For example, the baseline survey solely focused on documenting

needs for skill training, lack of start-up capital, less condom usage, etc. JeCCDO would have paid

attention to available skills, existing community network to support vulnerable groups, etc. As the

result of the focus of the baseline survey, the project was planned to address the needs as a

priority rather than exploiting the potentials within the community.

3. Time is a factor to make community initiated project to be feasible and sustainable. Given that

JeCCDO’s innovative project needed more time for preparation, understanding of the modalities

and implementation, compared to those conventional projects being carried out so far, the most

ideal timeframe for similar projects is five to seven years. Therefore, for future similar initiatives,

negotiations on the time factor should be made with project stakeholders including donors,

government and other partners.

4. Internal human and technical capacities are important within JeCCDO to implement similar

projects in the future. One of the challenges the evaluation process has identified was low human

capital as well as less-preparedness within the existing capacity from JeCCDO’s side to put

experienced staff with strong commitment to implement the innovative project. Therefore,

appropriate capacity including experiences should be in place for implementing similar projects

in the future.

5. Resource ownership is an issue in projects like the one JeCCDO has initiated since its final goal

is to build community capacity to care for citizens. The decision made to put the loan fund in a

financial institution which has experience in management compared to community development

orientation, brings element of distrust from partners of the innovative project. The evaluation

team has the view that a 100% perfection of effectiveness cannot be expected from such a project

which tries to test a new innovative approach. Therefore, in order to maximize the effectiveness

of a project of this type, a hybrid modality as a test/transition period where both the financial

institution as well as community-based organizations can work hand-in-hand to manage the loan

process can be adopted.