Upload
james-treasure-evans
View
35
Download
2
Tags:
Embed Size (px)
Citation preview
1 for people, for good
Concern Universal Impact Report Impact assessment of environmental conservation and livelihoods Ghana April 2011
2
for people, for good
Contents
Acronyms ...................................................................................................................................... 3
Executive Summary ...................................................................................................................... 4
Purpose of Study .......................................................................................................................... 5
Methodology ................................................................................................................................. 6
Findings ........................................................................................................................................ 7
Conclusions and recommendations ............................................................................................ 12
Annex 1: villages covered by the study ....................................................................................... 15
Annex 2: Research Team Members ........................................................................................... 16
Annex 3: Key Informant Interviews ............................................................................................. 17
Annex 4: Assessment Expenditure ............................................................................................ 18
Annex 5: Stories of Change – Contribution Analysis ................................................................... 19
Annex 6: Questionnaire Analysis – main findings ....................................................................... 20
Annex 7: Limitations/ Lessons ................................................................................................... 26
Annex 8: Timetable of study ........................................................................................................ 27
Annex 9: Terms of Reference ..................................................................................................... 28
3
for people, for good
Acronyms
ADRA Adventist Development and Relief Agency
CU Concern Universal
DA District Assembly
DFID Department for International Development (UK)
DLO District Liaison Officer
EC European Commission
EoP End of Project
FASCU Farmers Supportive Services and Community Utilities
GOG Government of Ghana
GTZ German Technical Cooperation
HH Household
LAST Livelihood Asset Status Tracking
MOFA Ministry of Food and Agriculture
MTE Mid Term Evaluation
NCRC Nature Conservation Research Centre
NGO Non-Governmental Organization
OIC Opportunities Industrialization Centres
PRA Participatory Rural Appraisal
REEP Rural Environmental Empowerment Project
SDDO Sunyani Diocesan Development Office
USAID United States Agency of International Development
4 for people, for good
Executive Summary In April 2011 Concern Universal Ghana and local partner organisation Nature Conservation Research Centre (NCRC) conducted a field based impact study of an environmental conservation and livelihoods programme (Rural Environmental Empowerment Project – REEP) in the Brong Ahafo region of western Ghana. The field work was conducted over five days in 14 out of the 51 project villages. The overall objective of the REEP project was to reduce poverty by empowering communities toward environmentally sustainable development and resource management. This was to be achieved by increasing environmental conservation and awareness among rural communities through participatory analysis of environmental problems and potential solutions, and the promotion of sustainable livelihood options. The project aimed to promote local actions which would reverse the trend of diminishing natural resources and bio-diversity in Brong-Ahafo region. The main findings of the assessment are: The project has reduced poverty by empowering people in the target communities to take better care of their environment and manage resources (EC overall objective) but more importantly has reduced poverty by increasing incomes. There are now less bushfires in the target area and this has led to improvements in
agricultural productivity and environmental conservation (EC specific objective 1) and more specifically:
Fire volunteers continue to be active, even though they no longer receive formal support from the government or non-governmental organisations.
Greater use of tree agriculture (cashew, plantain, cocoa) has led to an increase in forest cover and vegetation, and there are visible changes to the landscape since 2005.
Although the project promoted effective livelihood options, many of these have not proved to be sustainable and they have not had an impact on environmental practises (EC specific objective 2).
Reading stories of change in Botokrum
5
for people, for good
Purpose of Study The Rural Environment Empowerment Project (REEP) impact assessment was carried out to answer the question “what long-term difference has the REEP made to the project participants?” The final external evaluation demonstrated that the activities were implemented on time and to budget. Three years on, we were interested in finding out whether the project has had a lasting impact and what difference has been made to people‟s lives. The impact study has given us some understanding of the sustained changes in people‟s lives brought about by REEP and the „significant‟ changes (positive/ negative/ intended/ unintended) brought about by the project. Environmental Conservation and Climate Change Adaptation has been a priority for Concern Universal in Ghana over the last six years and a majority of funding has been spent in this area of work. Concern Universal Ghana recently reviewed its country strategy and has selected agricultural livelihoods and food security as a key programme focus. This, therefore, was a good time to reflect on, and capture learning from, past experience. The new strategy aims to address agricultural livelihoods, food security, climate change and environment issues among women and vulnerable groups in deprived districts in Eastern, Volta and BrongAhafo Regions and the northern savannah zone (Northern and Upper West Regions). The new strategy acknowledges that the fight against climate change cannot be won without
considering the rising energy needs of women and vulnerable groups in Ghana, nor can poverty be effectively addressed without accounting for the impacts of climate change on agriculture, especially among women and vulnerable groups. BrongAhafo Region was selected as the focus area for this study because the region is the food basket for Ghana and Concern Universal has implemented environment and climate change adaptation projects in the region between 1999 and 2008. The largest project implemented in the region by Concern Universal was the Rural Environmental Empowerment Project (2004-08, funded by various donors). Thus the project was selected because it is the largest „old‟ project CU Ghana has implemented in the BrongAhafo Region and it offers an opportunity to look at long-term impact over different timescales (2004-08 and 2008-11). Concern Universal is currently implementing one project, the BrongAhafo Environment and Livelihoods project (funded by the Irish Embassy in Abuja) in the Forikrom area of Techiman District. This study did not conduct research in Forikrom, as the focus was on assessing long-term impact of past interventions.
Growing Cocoa in Botokrum
6
for people, for good
Methodology The impact assessment randomly chose a sampling size of 15 villages from the 50 villages that originally took part in the project (confidence level of 90% and a margin of error of 5%). The villages were first stratified into three categories to ensure a more representative sample from the final random selection. The percentage of villages chosen from each strata equaled the percentage that the group represented amongst the three groups. The subgroups were developed to make sure that we covered a representative sample of the villages, in the event that we were unable to visit more than 15 villages. The three sub-groups were: (1) Communities involved in Component 1 (Environmental Education and Awareness only); (2) Communities involved in Component 2 (Sustainable Livelihoods) only and (3) Communities involved in both components. These were further clustered into (1) high performing villages, medium performing villages and low performing villages; and (2) villages within 10km of Sunyani CU office, villages 10-50km of Sunyani CU office and villages more than 50km from the Sunyani CU office. Household surveys were done within each of the 15 villages selected. The family units to be surveyed were selected semi-randomly in each village. The number of household surveys completed in each village depended on the village population based on a total survey target of 300 necessary to reach a 90% confidence level.
Within each village, CU staff held focus group discussions (FGD) with groups of up to 20 people comprising of men and women. The group participants included people who participated in the CU project and those who did not. It was not possible to separate the groups into male and female groups. In each community the impact assessment teams employed the following methodologies: Direct observation: With a village representative, or representatives, the assessment team conducted village walks/ transects to observe natural resources, present land use, vegetation, physical features and cropping systems. Semi-structured interviews (direct and indirect beneficiaries): Using a number of open-ended questions around the areas of enquiry interviews were conducted with a representative sample of: Beneficiaries (people in representative sample of communities) – farmers Partner (Nature Conservation Resource Centre) Local Government (Fire Service) Stories of change: through the FGD and semi-structured interviews the research team identified individuals that were able to tell more detailed stories of change about the project. Participants were offered the opportunity to use a camera / video camera to record the changes they described, or simply wrote them down (with accompanying photo). A story of change was to…
… demonstrate evidence of change in a community.
7
for people, for good
… show the power of involving beneficiaries in planning change.
… show how a community has transformed itself.
… show how an individual or community can do things they never thought possible. Questionnaires (household survey): A short questionnaire was designed with closed (Yes/No) questions on the main areas of enquiry. Desk based research (baseline/ reports/ end of project findings/ evaluation): The programme team reviewed the project documents to draw out information on impact. An assessment of project documents was especially useful in developing impact indicators and to track what has changed since project inception. Impact indicators developed were more generic than monitoring indicators, and as they needed to represent community priorities were developed in consultation with beneficiaries during the study. The impact assessment framework also took into account other projects/ programmes that have been carried out in our project area that had the same objectives (may also be private sector/ community initiatives). The study also looked at data on climatic conditions and demographics, with a view to assess what impact these had on the changes observed and the impact of CU‟s interventions. The study attempted to assess whether CU‟s interventions (or other interventions) have resulted in stronger community capacity to adapt to climate change / increased climatic variability, although it was
likely to be difficult to draw any firm conclusions on this.
Findings The final evaluation for the REEP project was conducted in August 2008, two months after the end of the project. One of the key objectives of this assessment was to verify the sustainability of the achievements that were recorded in 2008. 2008 evaluation achievement: New techniques in fire management have given communities the ability to control and prevent bushfires giving the communities renewed confidence to plant high value tree crops.
2011 update: With a new confidence in their ability to protect their land from fires, many REEP communities have begun to expand plantations of high value tree crops like cocoa, cashew, exotic mango, and palm nut, as well as teak.
Discussing stories of change in Tainso
8
for people, for good
The re-emergence of cocoa as a choice tree crop for farmers in some parts of the Brong Ahafo Region is an important indicator of the renewed confidence of farmers in their ability to prevent and control bushfires. Cocoa was an important tree crop largely abandoned after the devastating 1983 bushfires. Farmers in most districts in Brong Ahafo have been reluctant to plant cocoa until recently because they have been afraid to risk losing the trees to bush fires. This evaluation finds that in the southwestern districts cocoa is thriving again. Communities reported that soil fertility has improved with the reduction of fires. In all districts planting of tree crops has intensified in the past two to five years. 2008 evaluation achievement: The project has strengthened relationships of the REEP communities with district fire service offices. 2011 update: Fire volunteers continue to plan ways to support their activities through a variety of community-based enterprises, including forming growers‟ associations, tree planting, animal rearing, and communal labour. The
district fire service officers are eager to help them make linkages to get technical advice and seedlings and continue to support the associations formed. 2008 evaluation achievement: Grasscutter rearing has given poor rural women a new source of income and pride. 2011 update: Under component two, the project trained 40 women in five communities in the rearing of grasscutters and 36 women in mushroom production. Unfortunately very few of the women originally supported by the project have been able to rear grasscutters. The research found that although the women were able to gain a short-term capital benefit through the sales of the grasscutters very few were able to rear them and give them a long term sustainable source of off-farm income. The mushroom production has continued, but in one community visited in the assessment the mushroom house was no longer in use and the women originally involved in the production of
mushrooms have since stopped.
Cocoa beans drying in Nkyekemam
Grasscutter rearing project
9
for people, for good
2008 evaluation achievement: The REEP provided 90,000 seedlings to communities with high survival rates. 2011 update: Tree seedlings were planted in all five districts. The tree planting was part of an effort to establish agroforestry demonstration plots in 20 of the target communities. Three types of seedlings were provided by the Project: Lucaena, Acacia, and Mango. In all of the communities where the tree seedlings have been distributed the tress have been planted and continue to thrive and are harvested sustainably for firewood/ charcoal. 2008 evaluation achievement: Study on charcoal trade led to the negotiation of an agreement with the Traditional Council of Nkoranza to require all charcoal in the district to be produced from sustainable community woodlots. 2011 update: The 2008 study surveyed charcoal production and trade in three districts: Kintampo North, Kintampo South, Wenchi, Tain, and Nkoranza (which has the highest levels of charcoal trade in the country). Through engagement with the Traditional Council of Nkoranza, the project brokered an agreement to stop all charcoal production that does not come from community woodlots, which can be monitored for replanting. The Nkoranza Traditional Council MoU is a critically important outcome of the project‟s efforts to control the expansion of charcoal production through community-based charcoal associations and continues to be enforced.
Impact relative to overall project objective: Objective: Reduce poverty by empowering rural communities towards participatory and environmentally sustainable resource development and management. Impact: The principle beneficiaries of the project, the community-based fire volunteers and their families, repeatedly stated throughout the research that they have been able to protect their farms and adjacent forest land from forest fires. The increases in the level of protection have led to visible increases in environmentally sustainable resource management.
Impact relative to specific project objectives: Intended impact 1: Increased environmental conservation and awareness among rural communities through participatory analysis of environmental problems and solutions
Interviewing cassava farmers in Nchiraa
10
for people, for good
The project was successful in raising awareness and understanding of the impact of bush fires on the local environment. Bush fires were used as an entry point to wider environmental conservation issues, although the field research did not examine whether the project participants gained awareness of other conservation issues. The interviews and group discussions showed that people are aware of the need to protect the environment and are willing to take actions to do this; 61% of people interviewed have changed their farming practices as a result of the project and 55% of those people that said they had changed their farming practices are now involved with community fire management activities, for example getting help from fire volunteers when they need to burn land. The focus on bush fires was a very good entry point to conservation awareness because it meant that education on environmental issues focused on the impact that environmental destruction is having on agriculture. Because of the clear link formed between fire management and agricultural production project participants actively formed volunteer fire committees to deal with this issue and these committees continue to be active three years after the end of the project.
The volunteer fire committees have had a long term impact and have been successful in fighting bush fires since the end of the project; 92% of people interviewed said there were less bushfires now than there were in 2007. Of these respondents 32% said that fire management had led to improvements in the environment and 67% said this had led to an improvement in yields (agricultural output). As well as the community participants, District fire service officers also believe that the project has had a significant impact on bushfires, however no official fire service documents could be found during the assessment to verify this. All districts reported that there were no fires in REEP communities in 2007-2008 and that there were a low level of fires in target communities from 2008-11. Although it was possible to find records from Nkoranza and Kintampo District Fire Offices showing that the total number of fires recorded in the districts have dropped between 2005 and 2008 (see REEP final evaluation for details) it was not possible to get any further official data on this. While the fire service did not officially record the difference between REEP and non-REEP communities, both Nkoranza and Kintampo South Districts Fire Service said that there were no bushfires in REEP communities in those districts from 2008-11.
Fire volunteers in Kokuma
11
for people, for good
Intended impact 2: Promotion of sustainable livelihood options through rural communities adopting sound environmental practices
Although the project promoted effective livelihood options, some of these proved not to be sustainable after the end of the project and there was no evidence to support any link with the livelihood activities to improving environmental practises. Out of the interview cohort 32% stated that they had received support from the project. Of these, 20 people had received grasscutter (agouti or cane rat – a prized meat in West Africa) rearing support. Although other livelihood support was provided by the project, for example mushroom farming, the assessment did not interview anyone who had received this support and in one community an unused mushroom „house‟ showed that not all the livelihoods activities continued after the end of the project.
In Kokuma (population 1200) the assessment team met with five women who had benefitted from the grasscutter distribution and training programme. All the women that received the grasscutters were either widows or single mothers. After attending a training course in Sunyani for 11 days the women were each given six grasscutters to rear. The clear message from the women was that the grasscutters were a very welcome asset and they had all benefited from being given the animals. Ms Adoa Saa, who currently has eight grasscutters, was able to reinforce the walls of her house with money that she made selling grasscutters. Ms Hannah Osei told us that she was able to sell three grasscutters for about £10 each and as a result was able to pay for someone to dig a pit latrine with cement walls for her family. The assessment team also heard some very powerful individual success stories, such as one woman saying that the money from the grasscutter rearing saved her daughter‟s life as she was able to afford an emergency caesarean with the money she had from selling the animals. However, despite these positive short-term outcomes over the longer term very few of the women have been able to continue rearing the animals after the end of the project and only one person that we interviewed, Ms Adoa Saa, still had living grasscutters. None of the women we spoke to could give a very clear explanation of why the animals died, which is partly a result of the lack of husbandry and veterinary support given to the women by project staff during the project. The animals have proved hard to rear and a number of the women interviewed stated
Members of the Grasscutters project in Kokuma
12
for people, for good
that they would prefer to receive sheep or chickens next time as there are people in their villages who know how to tend to these animals when they are sick.
Conclusions and recommendations This section builds on the findings of the impact assessment to draw some conclusions about the relevance, effectiveness, efficiency, equity, impact and sustainability of CU‟s environmental conservation and livelihoods work in Brong Ahafo Region, and to make recommendations both for future programming, and for future impact assessments. Relevance The agricultural sector contributes significantly to Ghana‟s economy. It accounted for as much as 35.8% to the country‟s GDP in 2006 and employs 55% of its labour force. The sector is dominated by smallholder farmers who produce about 90% of Ghana‟s food output. It is estimated that about 55% of farmers have farm holdings less than 1ha. CU‟s agricultural livelihoods and food security programming is highly relevant to the Ghanaian context, since it directly addresses the main challenges faced in project areas. Agriculture in Ghana remains under developed; characterized by low productivity as a result of low adoption of improved production practices, inefficient input and output markets and limited access to technical advice and bushfires. CU works closely alongside partner organisations and local structures (District Assemblies), and this helps ensure the continuing relevance of CU‟s work. Relatively long (5-year) projects are
important to allow time for activities to produce intended results. Recommendations:
Continue to design and deliver multi-year community-led food security projects in partnership with existing local structures.
Effectiveness As outlined in this report, CU‟s livelihoods work in Ghana – represented by the REEP project – has been highly effective in meeting a number of its objectives, especially:
Increasing crop diversification: with a new confidence in the ability to protect land from fires many of the farmers that benefited from this project have begun to expand plantations of high value tree crops like cocoa, cashew as well as teak.
Strengthened partnerships with the district fire service offices that are eager to help the farmers make linkages to get technical advice.
Efua cooking in Agubie
13
for people, for good
Overall, CU‟s work has improved environmental conservation and food security in the REEP impact area. Areas where REEP appears to have been somewhat less effective include promotion of small scale enterprises as demonstrated by the failure of the mushroom farming and in some cases grasscutter farming. Recommendations:
Continue to promote modern farming techniques and high value tree crop diversification.
Small scale enterprises should be assisted with proper business planning and linked with the value chain from production to marketing.
Livestock rearing projects should be planned with government extension workers to ensure that after the end of the project a support network for veterinary and husbandry care will be in place.
Efficiency The final external evaluation report (August 2008) was very positive and demonstrated that the activities were implemented on time and to budget. The total cost of the REEP was roughly €400,000 (~€8,000 per targeted village over 5 years). Given the lasting impacts on conservation the project methodology and modus operandi has been highly efficient. Recommendations:
However, from the research carried out for the impact assessment it was clear that for a project of this size (staff/ budget) 50 villages was too many. Concern Universal should, in future, aim to increase staff time per target village.
Equity
It is very difficult to ensure that all project benefits are equally distributed. The study highlighted various examples where this had not been possible, for example where certain people were selected for specialist training and support as tree seedling producers and people with land and skills received mango, acacia seedlings, whereas those who lacked these resources did not. Thus, overall, there might have been a risk that the project might have contributed to a gradual widening of the gap among target beneficiaries. However, evidence during the impact assessment showed that improvement in environment (vegetation cover) and reduction in bushfires has given all people to grow tree cash crops e.g. cocoa and cashew among others. Recommendations:
Assess the equity impact of all project elements, and carefully consider alternatives.
Impact The survey data and the group discussion responses seem to indicate clearly that crop diversification and improved farming techniques were the most significant and enduring impacts of the project. Comparison of the survey data with REEP mid-term and end-of-project results shows that many of the impacts of the project have endured – for example, expansion of plantations of high value tree crops like cocoa and cashew. However, other impacts that were seen at the end of the project in 2008 have eroded over the last three years – for example, mushroom and grasscutter farming.
14
for people, for good
Recommendations:
In order to ensure that livelihoods projects have a greater impact, Concern Universal should consider better to use low value animals for rearing e.g. chickens instead of grasscutters which require high technical skills to avoid high mortality of the animals which are also capital intensive.
There is need for extension support throughout and after project (e.g. better links with Ministry of Agriculture and local veterinary support)
Cash transfers should be considered instead of introducing new products. Of which, there is need for more monitoring support (i.e. have to work in fewer villages) if new animal is going to be introduced
Group formation process and ownership should be built right at the beginning including entrepreneurial capacity of the women in mushrooms production coupled with better monitoring, encouragement of diversification, support in marketing and access to markets
There is need to work with local partners that are based on the ground to ensure sustainability assuming they will continue working local communities. Thus, there is a high potential for follow up than an organisation that would not continue working in the area.
CU should ensure that it has unrestricted funding to enable staff to visit former projects, to ensure that lessons are learned, and to develop understanding of factors influencing long term project impact and use this in programming and/or advocacy.
Future impact assessments: The impact assessment exercise reported here has
been a valuable learning process. The principle of revisiting old projects to assess impacts – both before/after, and with/without – should be a set practice in Concern Universal with clear budgets available.
Sustainability This study has demonstrated that a number of key outcomes of REEP have indeed been sustained over the three years since the end of the project (for example, strengthened relationships between fire volunteers and district fire service offices and increased high value tree crop diversification). However, mushroom and grasscutter farming have largely failed despite being a success somewhat during project implementation and right at the end of the project (evidenced by the final external evaluation report). In discussion with communities, it was clearly shown that building group cohesion with proper accountability mechanisms of funds and sharing of dividends especially when money is kept in a group account is critical for the success of small scale enterprise projects among others. Recommendations:
Ownership of livelihoods projects and mechanisms on sharing profits need to be built right at the start of the project not necessarily promotion of women‟s groups.
Access to markets and marketing are key for the success of livelihoods enterprises as was built in mushroom enterprise in addition to building ownership.
15 for people, for good
Annex 1: villages covered by the study
Village No. of questionnaires % of questionnaires
Agubie 1 0% Basakrom 2 1% Botokrum 24 9% Buokum 1 0% Fiema 20 8% Kojobadukrom 25 10% Kokuma 16 6% Krabonso 46 18% Kwamepim 23 9%
Kwamesiekrom 24 9% Nchiraa 17 7% Nkyekeman 24 9% Odumase 15 6%
Tainso 18 7%
District No. of questionnaires % of questionnaires
Berekum 48 19% Jaman 74 29% Kintampo 62 24% Nkoranza 35 14%
Wenchi 37 14%
16
for people, for good
Annex 2: Research Team Members
Mr MacDuff Phiri, Concern Universal Ghana (Lead)
Ms Juliette Lampoh, Concern Universal Ghana (research team member)
Mr James Agyei-Ohemeng, NCRC (local partner) Ghana (research team member)
Mr James Treasure-Evans, Concern Universal UKO (evaluation adviser)
Ms Vida Adjepong, Concern Universal Ghana (research team member)
Ms Annette Taylor, Concern Universal Ghana (research team member)
Mr Dan Akuoko, Concern Universal Ghana Driver (research team member)
17
for people, for good
Annex 3: Key Informant Interviews Village Chiefs in 14 villages (see village list)
Chairmen of village Fire Volunteers in 14 villages (see village list)
District Fire Staff in 5 districts
Mr James Agyei-Ohemeng, Programme Co-ordinator of NCRC (Project Partner)
The key informant interviews largely confirmed the impact assessment findings i.e. increased agriculture as a result of reduction in bush fires. The key informants mentioned that there were a lot of bush fires in 1980s and 1990s but this has reduced significantly in recent years. For instance the Wenchi District Fire Officer mentioned that only two bush fires were dealt with in the district between January and February 2011 (peak of the dry season). It should be mentioned though that it was very difficult to get records of bushfires from all the district fire offices visited for the period between 2008 and 2011. An interview with the Project Coordinator confirmed the following challenges:
The distances between villages and the number of villages that the project worked in (52)
Also, institutional challenges – transfers from EC to CU to NCRC took a long time
As a result of large number of villages it would have been helpful to have more staff
Nevertheless, despite these challenges the key informants confirmed that the training of fire volunteers was a great success and most communities still have fire volunteer groups working and this has led to positive environmental changes. The livelihoods activities were not hugely successful. There was a need to improve. The success of the REEP, however, has also been possible because of other projects that are being implemented in the region either by government or by other NGOs. These include Forestry Commission – programme to minimize bushfires, wildfire project and Fire Service – education on bushfires through radio programmes. In terms of lessons learnt about the impact of the project after having visited the communities, the Project Coordinator mentioned that a litte sustenance can drive communities forward, especially the fire activities and the tree planting. These activities were very successful; the tree planting was especially successful in communities we didn‟t visit. However, in future, greater concentration on the livelihoods work, especially thinking about the access to markets, consider using bee keeping as a principal conservation and income generating activity, introduce exchange visits between communities to see what success looks like in other places and possibly setting up a reward for the best community would have served as a good incentive, would have been good to have an award ceremony with District officials.
18 for people, for good
Annex 4: Assessment Expenditure In order to enable judgements to be made about the cost-benefit of this impact assessment exercise, a summary of the costs involved is provided below.
Item # Unit cost Total Notes
UKO staff member 1*8 days (salary, not extra incurred cost) 8 £200 £1600 (Estimate.) CU Ghana staff 4*7 days (salary, not extra incurred cost) 28 £96 £2688
Research Assistant 1*5 days 5 £77 £385 Partner Staff 1*5 days 5 £42 £210
Travel in Ghana (driver and fuel for 2 cars, 5 days) 2 £208
£416
Accommodation and meals in Sunyani
£892 UKO Travel UK to Ghana
£800
Preparation and writing up 10 £100 £1000 (Estimate.)
TOTAL £7991
19
for people, for good
Annex 5: Stories of Change – Contribution Analysis A total of 135 people gave stories of change comprising of 78 males, 56 females and one group as per the distribution below with 18% and 16% of the stories coming from Krabonso and Kojobadukrom respectively.
36% of the respondents indicated that their 'Story of Change' was of high significance and high contribution to project and positive and this mostly had to do increase in crop production indicated by increased food productivity and high yields due to increased soil fertility, reintroduction of cash crops like ginger, pepper, cocoa, cashew and plantain. Most of the community members mentioned that as a result of increased income they have been able to send their children to school.
20
for people, for good
Annex 6: Questionnaire Analysis – main findings 256 people consisting of 118 females (46%) and 138 males (54%) answered the questionnaires in the 14 target villages in 5 districts. The highest number of respondents was from Krabonso (18%) while in terms of districts, most questionnaires were answered in Jaman District (29%). Most of the respondents were farmers (190) representing 74%. The rest were farmers and traders (4%), traders (6%) and other (15%).
21
for people, for good
In measuring poverty rate in the communities, the respondents‟ ability to have income to meet their expenditure throughout the year and access to farming land were used as proxy indicators. 57% of the respondents reported have enough income all year round as compared to 43% who did not while 79% of the respondents indicated having access to farming land as compared to 21%. Those adjudged to live in extreme poverty stood at 11% i.e. had Double negatives (assessed as extremely poor - very little money or land).
83 respondents (32%) reported that they had participated in the REEP while 173 (68%) did not participate in the project. Of the 83 who participated in the project the following support was received.
22
for people, for good
The following activities are still ongoing. Activities no longer ongoing include advocacy training, charcoal production support and land use training.
23
for people, for good
45% of the respondents indicated that the Project has been most important in changing their lives. The biggest change being economic improvement (41%).
24
for people, for good
With regard to changes in farming practices 156 respondents (61%) reported that they have changed their farming practices as a result of the project while 100 did not. Out of 156 respondents (61% of total) the following number highlighted the following responses…..
25
for people, for good
With regard to bushfires, 235 respondents (92%) indicated that there are now less bushfires in the communities than there were before 2007.
Of those that thought there were less bushfires now (235 - 92%), they thought it was making a difference in the following ways:
26 for people, for good
Annex 7: Limitations/ Lessons
Sampling of villages The original project worked in 50 villages (this may have been too many, but that would be a project lesson, rather than an evaluation lesson and won‟t be discussed here), we had hoped to pick a representative random sample (12-15 villages) and stratify using a number of key variables (distance from Sunyani/ village performance – subjective view of partner and Concern Universal staff/ population/ involvement in component 1 and component 2 activities). This proved to be very difficult for a number of reasons: There was no verifiable population data for the target villages (partner and Concern Universal staff had not visited area since the end of the project and the district does not keep accurate census data) The random sample that was selected was not acceptable to the partner for purely practical (logistical) reasons, i.e. it would have been impossible to get to all the randomly selected villages in one week and the cost would have been too high (fuel for cars) The variables we stratified against were subjective (high/ low performance for example), not a problem in itself but calls into question the exercise of doing it. How to overcome this? This will always be difficult in the context of budget constraints (i.e. can‟t fund this work through project budget) Sampling of respondents – We had also hoped to sample randomly in each of the villages we
visited but this turned out to be impractical. The best we could hope for was to sample semi-randomly (every fifth household) but even this proved to be very difficult given the limited resources of the study (low number of experienced researchers and very little time – around 2 or 3 hours per village). This was particularly difficult on the last day of the evaluation when we tried to collect data on a Saturday. Saturday is traditionally used for ceremonies (funerals). If a funeral is taking place in a village it is very likely that almost the whole village will have been invited and there is almost no chance of finding anyone available. Researcher/ respondent bias – This was not a major issue, although it is worth being aware that the research team was made up mostly of Concern Universal and partner staff who implemented the project and would want to prove that the project had had a lasting impact. However, having said this the current Programme Director and Country Director were only employed in 2007 i.e. during the last year of the REEP project so there was no a huge vested interest. The same can not be said for the partner; the staff member who participated in the assessment was the project co-ordinator for the REEP project and felt a great deal of ownership over the project.
27
for people, for good
Annex 8: Timetable of study
Date Team A Team B
Monday, 11 April 2011
Training – Accra (whole day) Training – Accra (whole day)
Tuesday, 12 April 2011
Depart for Sunyani Spend night in Sunyani
Depart for Sunyani Spend night in Sunyani
Wednesday, 13 April, 2011
Impact assessment at Botokrom - Berekum District
Impact assessment at Nkyekemam – Berekum District
Thursday, 14 April 2011
Basakrom - Jaman District
Kojobadukrom – Jaman District
Spend night in Sunyani
Kwamepim – Jaman District
Kwamesiekrom – Jaman District
Spend night in Sunyani
Friday, 15 April 2011
Fiema – Nkoranza District
Odumase – Nkoranza District
Spend night in Sunyani
Krabonso - Kintampo District
Kokuma – Kintampo District Spend night in Sunyani
Saturday, 16 April 2011
Buokum – Wenchi District
Agubie – Wenchi District Spend night in Sunyani
Tainso – Wenchi District
Nchiraa – Wenchi District Spend night in Sunyani
Sunday, 17 April 2011
Return to Accra Return to Accra
28 for people, for good
Annex 9: Terms of Reference Study title
Assessing the long term impact of Concern
Universal‟s support to environmental
conservation and climate change adaptation in
Ghana through an ex-post evaluation of the
Rural Environment Empowerment Project in
BrongAhafo Region, Ghana.
Rationale
Concern Universal wants to deepen its work on
impact assessment and during 2011 will be
carrying out four separate impact assessments
in Ghana, Colombia, Kenya, and Gambia. The
Ghana impact assessment is being carried out
to assist CU Ghana learn from its previous
environmental conservation and livelihoods
work in order to a) demonstrate effectiveness of
our approach and b) further improve current
programming.
The Impact Assessment will be conducted to
answer the question „what long-term difference
did we make in the project areas?‟ The final
external evaluation demonstrated that the
activities were implemented on time and to
budget, but so what? In the long term, what
difference have these activities made to
people‟s lives? The impact study will attempt to
answer the following key questions:
What are the sustained changes in people‟s
lives brought about by REEP/ CU environmental
projects?
What are the „significant‟ changes (positive/
negative/ intended/ unintended) brought about
by the REEP/ CU environmental projects?
Focus
Environmental conservation and Climate
Change Adaptation has been a priority for the
CU Ghana over the last six years and a majority
of funding has been spent in this area of work.
Since Concern Universal Ghana has just
reviewed its country strategy and has selected
agriculture livelihoods and food security (ALFS)
as a key programme focus, this is a good time
to reflect on, and capture learning from, past
experience. The new strategy (ALFS
programme) aims to address agriculture
livelihoods, food security, climate change and
environment issues among women and
vulnerable groups in deprived districts in
Eastern, Volta and BrongAhafo Regions and the
northern savannah zone (Northern and Upper
West Regions). The ALFSP acknowledges that
we cannot fight climate change without
considering the rising energy needs of women
and vulnerable groups in Ghana, nor can we
effectively address poverty without accounting
for the impacts of climate change on agriculture
especially among women and vulnerable
groups
BrongAhafo Region has been selected as the
focus area for this study because the region is
the food basket for Ghana and CU has
implemented environment and climate change
adaptation projects in the region between 1999
and 2008. The largest project implemented in
the region by Concern Universal was the Rural
Environmental Empowerment Project (2004-08,
various donors). Concern Universal is currently
29
for people, for good
implementing one project, the BrongAhafo
Environment and Livelihoods project (funded by
the Irish Embassy in Abuja) in the Forikrom
area of Techiman District. This study will not
conduct research in Forikrom, as the focus is on
assessing long-term impact of past
interventions.
This project has been selected because it is the
largest „old‟ project CU Ghana has implemented
in BrongAhafo Region and it offers an
opportunity to look at long-term impact over
different timescales (2004-08 and 2008-11).
Reports from other short-term projects in the
region will also be used in the desk based
research to give Concern Universal an idea of
the impact of all interventions in the region.
Sampling frame
The impact assessment will aim to choose a
sampling size that will give us a confidence
level of 90-95% and a margin or error of 5-10%.
We know that the REEP project worked in 50
communities but before choosing the final
sample size we will need to verify the population
of these communities. We will decide on the
final sampling size by using the Raosoft sample
size calculator:
http://raosoft.com/samplesize.html
REEP
Number of communities
that project worked in
50
Number of households TBC
Number of individuals TBC
Proposed sample villages
Proposed sample
households
Selection of sample villages and households
Once we have decided on the sampling frame
and number, CU Ghana staff will suggest two
villages that they think provide particularly clear
examples of „good‟ long-term impact and two
villages that highlight „poor‟ long-term impact.
The remainder will be randomly sampled from
the database of project villages.
Within each village, CU staff or the village head
will identify households that they think provide
particularly clear examples of „good‟ long-term
impact. These households will be interviewed
as case stories.
Within each village, the village head will be
asked to bring together up to 20 people (10 men
and 10 women) who participated in the CU
project, to participate in a focus group
discussions. If more appropriate, the focus
groups will be separated in to male and female
groups.
Within each village, the researchers will conduct
a random household (LAST) survey. The
participants will be selected from the database
of project participants.
Proposed methodology
The methodology proposed is simple and user
friendly. It will build on existing monitoring and
30
for people, for good
evaluation systems, or if this is not possible
then will put in place clear guidelines on
carrying out further impact studies. The
methodologies have been chosen to be
participatory and help in the promotion of further
downward accountability, as well as support
programmatic learning.
The research team will make every effort to
avoid bias by emphasising that we want to hear
about all changes, positive or negative, and
whether or not associated with a particular
organisation. We will clearly explain, before
each data collection session that this research
is going to help Concern Universal develop
better projects so honesty is the best policy.
Whether answers are positive or negative will
not influence the chances of more project
funding for the community.
The study will follow up specifically on the Rural
Environment Empowerment Project (2004-
2008) funded by the EC and other donors. The
objectives of the project were to 1. Increase
environmental conservation and restoration
among rural communities, and 2. Promote
sustainable livelihood options through rural
communities adopting sound environmental
practices. The external evaluation says that the
objectives were achieved but the impact
assessment will try and delve deeper in to; did
poverty reduce as a consequence of the project,
has there been a lasting change on the
environment?; do communities feel that they
now have more control over natural resources?;
and have these changes been significant?
The impact assessment will focus on a number
of „areas of enquiry‟. Areas of enquiry will allow
us to look at change within the community and
people‟s perceptions of change. When looking
in to the areas of enquiry we will attempt to
answer some specific questions: what has
changed; what was the effect of the change;
and was the change significant? Although this
may change during the development of the
study the proposed areas of enquiry are:
Poverty – income/ access to land/ assets/ social
networks
Land – conservation/ access/ community
woodlots/ participatory resource management
Water – conservation/ availability
Empowerment – (proxy indicators to be
developed by community)
Using the areas of enquiry as the basis of our
assessment the study will use a number of
quantitative and qualitative tools to collect data
and triangulate results. The most likely research
methods to be used are:
Direct observation: With a village
representative, or representatives, the
assessment team will conduct village walks/
transects to observe natural resources, present
land use, vegetation, physical features and
cropping systems
Semi-structured interviews (direct and indirect
beneficiaries): Using a number of open-ended
questions around the areas of enquiry
31
for people, for good
interviews will be conducted with a
representative sample of:
Beneficiaries (people in representative sample
of communities) – farmers
Partner (NCRC)
Local government (Forestry Department, District
Assembly, Fire Service)
National government (Natural Resource
Commission, Ministry of Food and Agriculture)
Stories of change: through the FGD and semi-
structured interviews the research team will
identify individuals that are able to tell more
detailed stories of change about the project.
Participants will be offered the opportunity to
use a camera / video camera to record the
changes they describe, or simply to write them
down (with accompanying photo). A story of
change will…
… demonstrate evidence of change in an
organisation.
… show an individual‟s personal journey.
… show the power of involving beneficiaries in
planning change.
… show how a community has transformed
itself.
… show how an individual or community can
do things they never thought possible.
Questionnaires (household survey): A short
questionnaire will be designed with closed
(Yes/No) questions on the main areas of
enquiry. The household survey design will be
based on a design previously used for the
project.
Desk based research (baseline/ reports/ end of
project findings/ evaluation): The programme
team will review the project documents to draw
out information on impact. An assessment of
project documents will be especially useful in
developing impact indicators and to track what
has changed since project inception. Impact
indicators will need to be more generic than
monitoring indicators, and as they need to
represent community priorities will be developed
in consultation with beneficiaries during the
study. In the preparation phase of the impact
assessment we will find whatever baseline
figures we have in the 50 target communities in
the following areas:
Income figures (or well-being – health/
education)
Levels of environmental degradation (forest
cover?)
„livelihood options‟ (referring to fuel/ energy
production?), less environmentally exploitative
Conservation farming methods
Crop diversification
Number of natural disasters (bush fires)
Partner capacity (proxy indicators)
The IA framework will also take in to account
other projects/ programmes that have been
carried out in our project area that had the same
objectives (may also be private sector/
community initiatives). The study will also look
at data on climatic conditions and
demographics, with a view to assessing what
impact these had on the changes observed and
the impact of CU‟s interventions. The study will
attempt to assess whether CU‟s interventions
32
for people, for good
(or other interventions) have resulted in stronger
community capacity to adapt to climate change /
increased climatic variability, although it is likely
to be difficult to draw any firm conclusions on
this.
Expected Outputs
Ghana Impact Assessment (max 24 pages for publication, plus appendices), including
summary on a page and one page introduction
Framework for Impact Assessments that could be used by Ghana team
Training on impact assessments