View
805
Download
0
Category
Preview:
Citation preview
Searching for outcomes in rural Tanzania: Harvesting directly from those influenced
Implications for evaluating behavioural change initiatives with communities
John Mauremootoo, Richard Smith, Dunstan KishekyaPresentation given at the American Evaluation Association on 12th November 2015 in the session entitled: The strengths and challenges of Outcome Harvesting for Evaluating in Complex Situations. Experiences from around the World.
The Project
A vehicle for tackling polarizing
social issues
The ProjectThe Team Tanzania (The Team) one of 15 completed or ongoing projects implemented in Africa and Asia by Search for Common Ground (SFCG) using The Team concept – The Team is a vehicle for tackling polarizing societal issues by stimulating learning in a persuasive, but non-confrontational manner.
For Tanzania the issue chosen was gender equality
Using an ‘edutainment’ approach (TV and radio soap operas) the project planned to reach and influence a wide audience.
Goal To contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania by influencing changes in gender attitudes and behaviour of the general public
Priority gender equality themes targeted in the 18 month project
1. Inheritance & women’s consideration in inheritance issues
2. Women’s leadership
3. Gender-based violence
4. Retention of girls in secondary school
5. Rape
The Project Theory of Change
Reach
Response
Resonance
Numbers who watch and listen to the show
The attitude and knowledge changes at the individual and institutional levels
The actions that have been triggered by the programmes
Evaluation Method
Identification, description and interpretation of outcomes through Outcome Harvesting (using an OM-inspired definition of Outcome as changes in actions, relationships, policies or practices of one or more social actors influenced by the intervention).
With especial emphasis on harvesting outcomes of social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions.
Evaluation challengesThe “Blank Slate of:• Poor understanding of outcome concept among project implementation team• Little or no knowledge of outcomes among the project implementation team• No documentation of outcomes in reports
Short duration of assignment given the fact that all outcome data had to be collected from scratch
Data collection via focus group discussion
Focus Group Discussion Guiding questions explained issues such as:• The definition of
outcomes• The definition of
contribution• How to determine how
importance of The Team’s contribution• Sources of
substantiation• Similar outcomes• Negative outcomes
Data collection via focus group discussion
“Treatment” “Control”
Harvesting outcomes through focus group discussions with social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions
Harvesting outcomes from ‘control groups’ – target group representatives who had not participated in the mobile screenings and discussions
Follow up
Follow-up work to finalise and substantiate results descriptions
Results: The Team Contribution54 outcomes and 10 “proto-outcomes”
Series1
0 5 10 15 20 25 30 35 40 45
Useful (25)
Very important (25)
Important (14)
percentage
Strengthened and / or brought forward changes that may have happened to some extent anyway
Either essential to the change or greatly accelerated it
Strengthened and / or brought forward changes that may have happened to some extent anyway
Helped realise a change that may not otherwise have happened or would have happened very much more slowly
Results: Outcomes directly related to priority issues
Inheritance & women’s consideration in inheritance issues
Results relating to shared ownership of goods and property
Women’s leadership
Women taking more responsibility in work, home and in political life
Gender-based violence
Men stopping beating their wives and others encouraging men to stop beating their wives
Retention of girls in secondary school
Parents prioritising girls education and others encouraging parents to prioritise girls education
Rape
No results provided
Results: Other Outcomes
Financial implications of gender equality Participatory decision making
Implications for evaluating behavioural change initiatives with communities
1. Outcome Mapping Concepts
OM concepts of outcomes as behavioural change and contribution and the differences between knowledge, attitude and behaviour were easy and intuitive to grasp for the participating communities.
2. Knowledge & Attitude Changes
Claimed attitude changes even without behavioural change are important and can be readily assessed using OH
These “proto-outcomes” helped:• To understand the claimed process of change even where there have been no
tangible changes in behaviour• To compare the quality and process of change between “treatment” and “control”• Inform the change agent
3. Utility of Control Groups
In situations where there is a well-defined theory of change the control group provides a counter-factual
4. Focus Group Format
The focus group format is effective for discussing less sensitive topics but may not be the best way of evaluating results concerning sensitive issues and totally inappropriate for the most sensitive such as rape
5. The value of Follow up
The importance of an iterative follow-up process to enrich the result description and enhance its credibility.
6. The Timing of the Evaluation
a) It was surprising there were outcomes from a very ambitious project when its 18 months had not yet finished,
b) It was a snap shot & provided credible evidence of changec) Follow up was needed, e.g. after say 2 yearsd) It ideally should really have been a formative evaluation.
Now What?
If we consider the evaluation as an intervention then both its findings and the process need to be built on to help maximise positive change.
This was not formally done for a variety of reasons which in our opinion was a lost opportunity as is often the case for evaluations.
• Ansila Marandu (evaluation assistant)• Paul Glick (Search for Common Ground)• Cornelia Wamba (Search for Common Ground)• Stella Msami (Search for Common Ground)• VanessaCorlazzoli (Search for Common Ground)• The Mvomero Organizations Coalition• Women and Girls Fight illiteracy and Poverty
Organisation Save the Children Tanzania• Kate Dyer (AcT and KPMG)• Layla Ghaid (AcT and KPMG)
Acknowledgements
ImagesScreenshots from The Team Tanzania on YouTube
ImagesPublic domain
All Other ImagesJohn Mauremootoo (Creative Commons Attribution: CC-BY)
Recommended