Monitoring, Evaluation & Learning (MEL) · 2019-03-29 · Monitoring, Evaluation & Learning (MEL) Our credentials WYG has an in-house MEL team of 25, a core pool of trusted associates

www.wyg.com

creative minds safe hands

Monitoring, Evaluation & Learning (MEL)

WYG’s dedicated Monitoring, Evaluation and Learning (MEL) division supports our clients and their implementing partners across the globe to undertake independent investigation and evaluation, to design and operationalise MEL systems, and to improve MEL capability. We believe that appropriate and robust MEL inputs improve the impact and sustainability of development interventions, and increase value for money (VfM).

We help funders and implementers ensure MEL addresses accountability, learning, and evaluation needs – while being able to share knowledge and communicate results effectively. In response to evolving client needs, we also undertake non-fi duciary compliance and audit work, ensuring programmes operate in accordance with gender, confl ict sensitivity, safeguarding and other sector specifi c guidelines and obligations – while our work on result harvesting supports programmes to adequately capture output and outcome results, and consider the strength of evidence underpinning result claims.

Monitoring, Evaluation & Learning (MEL)

www.wyg.comMonitoring, Evaluation & Learning (MEL)

We combine sector knowledge with in-depth MEL experience to advise on and undertake MEL activities at each stage of the programme lifecycle. We:

• undertake investigation and draw on evidence to inform the design of new programmes

• develop and periodically update Theories and Change (ToCs), result frameworks (logframes), VfM metrics and gender strategies

• create, improve and help operationalise MEL strategies, systems and tools

• put in place knowledge management and knowledge sharing systems

• help develop a culture of learning and reflection, and improve operational processes to enable ongoing adaptation

• design and undertake independent verification, monitoring and evaluation, including undertaking qualitative fieldwork and developing quantitative tools and interpreting data

• provide training and build capacity of partners.

WYG operates at the cutting edge of MEL, and continues to expand its use of technology solutions. We also develop new approaches to:

• ensure appropriate MEL and reporting systems are in place in complex, multifaceted and system approach interventions

• undertake thematic and across portfolio (not only project) investigation and evaluation

• undertake developmental evaluation and ongoing learning, feeding in lessons ‘in real time’ to improve impact and sustainability

• evaluate programmes which claim to operate in a problem driven and adaptive or politically smart manner

• evaluate ‘hard to measure’ outcomes and impacts, such as soft power, influence, research uptake and secondary benefits

• support donors to operationalise payment by results (PBR).

We pride ourselves on undertaking contextually appropriate and robust investigation and evaluation. We deliver MEL support services which are tailored to programme needs, and provide clear and practical recommendations that can be operationalised to improve performance and sustainability, while capturing learning to be shared more widely.

Where we workWe work in fragile and conflict affected environments (FCAS) as well as emerging economies middle- and income countries, and our services can be tailored for the UK and US domestic markets. Our current flagship programmes include the largest MEL contract awarded by HMG to date - the Evaluation and Learning of the £1.2B Cross Government Prosperity Fund - and we support the Conflict, Stabilisation and Security Fund in West Africa to improve MEL across its portfolio of programmes in Mali, Nigeria, and Sierra Leone. We hold a portfolio of DFID evaluations, and continue to grow our client base, working with other donors and supporting UK and US private sector partners, non governmental organisations (NGOs) and funders to improve MEL.

Our MEL team have wide sector experience, and partner with sector specific experts where required to ensure MEL solutions address the unique needs of each focus area – while aligning with international best practice in monitoring, evaluation and learning. Our current projects include work in the areas of:

• governance and accountability

• peacebuilding, security and justice

• private sector and economic development

• livelihoods and rural development

• climate, green cities and energy

• education .


Our credentialsWYG has an in-house MEL team of 25, a core pool of trusted associates and a wider network of sector and geographic experts. This resourcing approach allows us to offer value for money to our clients, while ensuring we can provide the best sector specific MEL solutions. Our staff are experienced development practitioners drawn from development advisory, capacity building, NGO and academic backgrounds, and all our technical staff hold Master’s degrees in a related field. They are supported by experienced in-house project management and financial experts specialising in development. Our team holds 15 different nationalities and have worked across the globe.

WYG MEL operates in accordance with OECD DAC Quality Standards for Development Evaluation and hold membership of the UK Evaluation Society and European Evaluation Society. We are committed to balancing rigour and explanatory power with qualitative nuance and practical recommendations. In addition to operating in accordance with DFID quality assurance (EQUALS) guidance, WYG is supporting DFID in the operationalising of adaptive management approaches, testing development evaluation approaches, and informing thinking on the review of politically smart and adaptive programmes through our programmes.

We emphasise continuous professional development, and embrace a culture of learning and reflection. WYG MEL aims to contribute to the broader debate on development policy and to share our experiences with others in the industry through publications and attendance at events.

Delivering complex and portfolio level MELTo deliver development at scale, aid interventions are increasingly becoming larger and lengthier in scope and scale. Donors are recognising the timeframes required to achieve impact when dealing with problems which, at its core, requires a change in behaviour, and understand the need to address challenges using a systems approach which considers the complex linkages and interconnections at play. Together with an emphasis on achieving value for money and reducing overhead and management costs, this means that development programmes are becoming bigger and more complex.

This has resulted in an increased demand for MEL services at portfolio level, and the need to improve MEL, knowledge management and communication across workstreams, implementing partners, programmes and even government departments. WYG has specific experience in undertaking MEL of long term, sizable and multifaceted programmes, and continue to develop new solutions to support portfolio level MEL.

Nigeria, Sierra Leone, Mali and Sahel region

Provision of MEL Services for CSSF West Africa

2018 – 2021, DFID

Conflict and Security Niger, South Sudan, Madagascar, Yemen, Pakistan, Nepal, Bangladesh, Cambodia, Myanmar

Independent Process Monitoring and Evaluation of the DFID contribution to the UNICEF Programme – Accelerating Sanitation and Water for All (ASWA) in Neglected, Off-Track Countries

2014 – 2018, DFID / UNICEF

Water and Sanitation

Southern Africa

M&E of Climate Resilience Infrastructure Development Facility (CRIDF)

2017 – 2020, DFID

Climate and Private Sector Development

China, India, Brazil, Mexico, Colombia, South East Asia, Nigeria, South Africa, Kenya, Horn of Africa, Turkey

Prosperity Fund Evaluation and Learning

2017 – 2021, FCO and cross-departmental HMG

Infrastructure, Business Environment, Anti-corruption, Digital, Trade, Health, Future Cities

Democratic Republic of Congo

M&E Support Unit to the International Security and Stabilisation Support Strategy (ISSSS)

2015 – 2018, DFID

Conflict, security and peacebuilding


Independent Process Monitoring and Evaluation of the DFID contribution to the UNICEF Programme – Accelerating Sanitation and Water for All (ASWA) in Neglected, Off-Track Countries, 2014 – 2018, DFID / UNICEF

Water and SanitationNiger, South Sudan, Madagascar, Yemen, Pakistan, Nepal, Bangladesh, Cambodia, Myanmar

DFID was supporting UNICEF’s implementation of the Accelerating Sanitation and Water for All in Neglected, Off-Track Countries programme, which involved delivery of water, sanitation and hygiene (WASH) both through direct implementation and strengthening of the enabling environment to improve and sustain levels of access to WASH services. The aims and objectives of the IPME, being delivered by WYG, were to assess programme results and performance, and formulate recommendations to allow programme improvements to be made in real time. Given that women and girls were disproportionately impacted by lack of access to WASH facilities, the evaluation paid particular attention to gender issues.

As part of the IPME, WYG provided customised capacity building support to UNICEF’s WASH country programmes to strengthen their monitoring and evaluation systems and ensure their relevance for measuring changes over time. This involved harmonising DFID/UNICEF WASH indicators and milestones to ensure reporting was consistent and meaningful; strengthening quality of data collection, and analysing processes to measure value for money and sustainability. WYG also conducted monitoring and verification of ASWA results and evaluated ASWA’s processes to understand reasons behind the achievement/non-achievement of results

M&E of Climate Resilience Infrastructure Development Facility (CRIDF); 2017 – 2020, DFID

Climate and Private Sector DevelopmentSouthern Africa

Climate Resilience Infrastructure Development Facility (CRIDF) is a DFID supported programme working to provide long-term solutions to water issues that affect the lives of the poor in Southern Africa. The focus is not on building short-term water infrastructure, but on working with organisations to show them how they can better build, manage and finance their own water infrastructure. CRIDF is working with 12 different countries in Southern Africa that share water resources. In so doing, CRIDF aims to improve the lives of over 200 million people.

WYG is designing and delivering the internal M&E of CRIDF. The M&E Framework is underpinned by an adaptive management strategy. The M&E Framework itself will not change during the programme lifespan, but the tools and templates that support the framework will be adapted based on the learning from the process. The M&E system designed by WYG is based on measurement of deliverables at the different stages of the results chain, as described in the CRIDF logframe; and at the activity, project and programme level. WYG is collecting a mixture of quantitative data (through tools like satisfaction surveys/scorecards and the ICF transformational change tool) and qualitative data (through case studies designed to capture policy influencing which has led to change, test ToC assumptions, identify barriers to behavioural change and understand beneficiary perceptions).

M&E Support Unit to the International Security and Stabilisation Support Strategy (ISSSS); 2015 – 2018, DFID

Conflict, security and peacebuilding Democratic Republic of Congo

Over a three year period, WYG served as the M&E Unit located within the UN Stabilization Mission (MONUSCO) in the DRC. From our office in Goma, we helped implement the International Security and Stabilization Support Strategy (ISSSS), the agreed framework to support the transition from peacekeeping to peacebuilding in Eastern DRC. ISSSS is an umbrella of projects, funded and implemented by different donors, UN agencies, local and international NGOs.

WYG designed and implement an M&E Strategy for ISSSS, which tracked progress of all ISSSS projects and the contribution of the ISSSS programme as a whole to stabilisation outcomes, in turn informing future programming. Support included advising ISSS implementing partners on data collection and data quality, and interpreting portfolio level data to provide a picture of progress and inform recommendations for improvement. To ensure sustainable uptake of MEL, WYG also helped build stakeholders’ capacity in M&E and providing ad hoc and ongoing MEL technical assistance.

Provision of MEL Services for CSSF West Africa; 2018-2021, DFID

Conflict and Security

Nigeria, Sierra Leone, Mali and Sahel region

WYG and our partners IMC Worldwide and First Call have been appointed as MEL advisors to the UK Conflict Stability and Security Fund (CSSF) in West Africa. This £135M fund consists of a large portfolio of security & justice, serious organised crime, stabilisation, security sector and defence reform programmes in Nigeria, Sierra Leone, Mali and the Sahel region, implemented by the FCO, DFID, MOD, Home Office and other HMG entities.

WYG is supporting HMG to identify MEL weaknesses in, and to make improvements to, both MEL systems and capacity. This includes ensuring appropriate Theories of Change (ToCs), result frameworks (logframes) and VfM metrics are in place, and that robust MEL systems suited to the local context and programme needs are developed and operationalised. WYG is also advising the fund on portfolio level MEL to measure effects and impact of the fund as a whole, and is supporting independent verification and evaluation. Other services include undertaking reviews of programmes’ ability to meet HMG and international gender, conflict and security, and other relevant standards and obligations, and measuring ‘hard to measure’ outcomes such as influencing and soft power. Advise and support on generating a culture of learning and reflection and implementing politically smart and adaptive approaches are also provided, as is support to inform the evidence base on what works and why in a number of thematic areas.

Prosperity Fund Evaluation and Learning; 2017 – 2021, FCO and cross-departmental HMG

Infrastructure, Business Environment, Anti-corruption, Digital, Trade, Health, Future Cities

China, India, Brazil, Mexico, Colombia, South East Asia, Nigeria, South Africa, Kenya, Horn of Africa, Turkey

The UK cross-government Prosperity Fund aims to remove barriers to economic growth and promote the economic reform and development needed to reduce poverty in partner countries. The Fund also looks to create opportunities for international business including UK companies as a result of this economic growth, as a secondary benefit. The Fund’s budget is £1.2 billion over 6 years across a portfolio of projects. The Prosperity Fund forms a strategic proportion of the UK’s total Official Development Assistance (ODA) commitment of 0.7% of Gross National Income (GNI). It represents a shift in the ODA resources spent on assisting countries to grow and develop by focusing on multi-year programmes in middle-income countries where 70% of the world’s poor live and where 60% of global growth will come from by 2030 (OECD). WYG leads a three-company consortium offering evaluation and learning services to the Prosperity Fund across Asia, Latin America and Africa.


Facilitating adaptive management and real time learningIn response to the growing acknowledgement that traditional approaches to institutional reform have often had limited impact, a movement to ‘do development differently’ arose. Three key approaches in particular have gained traction within the development community in an attempt to increase impact through provision of more appropriate support: the need for programmes to (1) work in a politically smart manner, (2) be problem driven, iterative and adaptive, and (3) be demand led.

A central requirement of these three approaches is the need for ongoing data to inform programme decisions at both operational and strategic level – it is necessary to understand what is working and what not, the changes in partner needs, and developments in the local political contexts that can help or hinder reform. As a result, the ability to generate information on an ongoing basis (as opposed to periodically reporting on programme process and performance) is paramount.

Following its acquisition of LDP (responsible for the design and delivery of the highly successful LASER programme) WYG has generated proven track of supporting and evaluating adaptive and politically smart programmes, and of supporting projects to document and share learning.

Zambia

Independent Evaluation of the Zambia Accountability Programme (ZAP)

2015 – 2019, DFID

Governance

Afghanistan

Strategic Support to the Ministry of Interior Affairs Phase Two (SSMI-2)

2016 – 2019, DFID

Governance, Conflict and Security

Nepal

Mid-Term Review of Nepal’s Accelerating Investment and Infrastructure (MTR AiiN)

2018 – 2019, Landell Mills/DFID

Investment and Infrastructure

Somaliland

Energy Security Resource Efficiency in Somaliland (ESRES) Real Time Learning (RTL) Process

2015 – 2018, DFID

Energy


Strategic Support to the Ministry of Interior Affairs Phase Two (SSMI-2); 2016 – 2019, DFID

Governance, Conflict and SecurityAfghanistan

WYG is leading the evaluation of Strategic Support to the Ministry of Interior Affairs Phase Two (SSMI-2) – the first problem driven iterative and adaptive (PDIA) programme operating in an FCAS environment – on behalf of DFID.

SSMI-2 is a technical assistance project working within specific areas of the Ministry of Internal Affairs (MoIA) in Afghanistan, aiming to achieve improvements in the Ministry’s capacity to manage its resources, with particular focus on the Afghan National Police (ANP). Our evaluation examines what works, in what circumstances, why and the extent to which the results claimed by SSMI-2 can be evidenced and sustained.

WYG has been tasked with monitoring the performance and progress of the project through its lifetime, examine the effectiveness of the project in supporting the Ministry of Interior Affairs (MoIA) in improving service delivery of the Afghan National Police (ANP) through improved allocation and use of resources, understanding how the combination of interventions has led to those changes and what the critical elements for effective implementation are; and collecting data, information and lessons to build the evidence base. This has been done through extensive in country qualitative data collection throughout the life of the programme.

Highlights of the SSMI evaluation has included an appraisal of the programme’s Political Economy Analysis (PEA) and logframe, and an assessment of the problem driven iterative adaptation (PDIA) approach

.

Energy Security Resource Efficiency in Somaliland (ESRES) Real Time Learning (RTL) Process; 2015 – 2018, DFID

EnergySomaliland

Energy Security and Resource Efficiency in Somaliland (ESRES) is a programme funded though the International climate Fund (ICF) to improve access, affordability and sustainability of electricity in Somaliland. ESRES is intended to support innovative renewable energy and energy efficient technologies, governance approaches and business models to improve access to electricity for essential institutions (especially hospitals and health centres/clinics) and local communities.

Using a locally-developed evidence base, WYG’s real time learning worked with the ESRES programme during its Phase 1, focusing on updating the programme ToC and improving implementation during its 30-month pilot. The ESRES implementation programme itself was adaptive and the RTL methodology mirrored this by adopting a flexible research and engagement programme running alongside the activities of the implementing agent and the wider stakeholder group (community institutions, community members, businesses, IPPs and service providers such as hospitals and schools). RTL engagement activity included field-based hybrid mini-grid assessments, qualitative focus group discussions with community representatives, key informant interviews among all stakeholder groups and case studies. The RTL assignment aimed to generate robust evidence on the achievements and limitations of ESRES Phase 1. This guided programme implementation and contributed to the design of its expansion phase.

Independent Evaluation of the Zambia Accountability Programme (ZAP); 2015 – 2019, DFID

GovernanceZambia

Since 2015 WYG has been serving as the Independent Evaluation Partner (IEP) on the DFID funded Zambia Accountability Programme (ZAP), a five year, £26.5 million programme, aimed at ‘improving accountability and responsiveness – up and down – in the delivery of public goods and services’ in Zambia.

ZAP is as a politically smart and adaptive programme and draws on evolving knowledge of the local context to respond to and exploit changes in the political environment. ZAP focusses on three thematic areas (political processes, education and inclusive growth), with a cross-cutting knowledge sharing element, targeted simultaneously at the local, provincial and national level. Interventions in the education and political processes workstreams are spread geographically across the country.

As IEP, WYG undertook a theory-based evaluation of ZAP to help DFID consider if a multi-pronged, bottom up and top down approach to improve accountability as delivered by ZAP was suitable, and to understand what was working or not, why, how and in what contexts. Using a Development Evaluation (DE) approach, we generated ongoing learning through cyclic investigations to help DFID-Z and its implementing partners to adapt and refine ZAP by testing the intervention hypotheses; to support interrogation and sense-making of evaluation findings; to monitor the impact of the changes made to the programme as a result; and to document lessons.

The IEP support itself was adaptive and evolving in nature, and as ZAP moved through its life cycle greater emphasis was placed on result identification, in response to DFID-Z’s changing needs, with result harvesting helping to ensure a detailed understanding of programme achievements. In the last year of ZAP, we are focussed on exploring the outcomes of interventions, documenting reflections and learning, and providing recommendations to improve the design and delivery of future large scale, multifaceted and politically smart programmes

Mid-Term Review of Nepal’s Accelerating Investment and Infrastructure (MTR AiiN) 2018 – 2019, Landell Mills/DFID

Investment and Infrastructure Nepal

In partnership with Landell Mills, WYG undertook a mid-term review of a DFID flagship adaptive programme – the Accelerating Investment and Infrastructure in Nepal (AiiN) programme. Following on from Nepal Centre for Inclusive Growth (CIG), one of the first projects to test an adaptive approach, AiiN operates in a politically smart and demand driven way, addressing investment climate reform and infrastructure development through three pillars of work.

WYG provided the strategic and technical leadership through the team leader, and led on aspects related to adaptive programming and politically smart working. WYG was tasked with reviewing the Theories of Change at portfolio and programme level, undertaking ToC refresher workshops and making recommendations for improvement. WYG also developed an approach to review the quality and appropriateness of adaptation, and considered both the appropriateness of the project management and MEL approach and systems. Other inputs included evaluating the gender, institutional development and stakeholder engagement inputs, and verifying outcomes to inform payment by result decisions.

A year on from the MTR, we are undertaking a further review of the ToC, proposing evaluation questions, recommending the revised MEL strategy for the programme extension and advising on programme lesson learning and knowledge sharing.


Conducting independent and theory-based evaluationsIn addition to verifying programme performance and results, WYG is an industry leader in the evaluation of ‘difficult to measure’ outcomes and complex, multifaceted programmes. We often support DFID to verify not only what programmes have achieved – but what worked (or not), when, why and how. Through theory-based evaluations, we help understand the requirements for success, and are able to support the design of future programmes more likely to succeed.

Evaluation of the Comprehensive Agriculture and Rural Development Facility (CARD-F), 2015 – 2019, DFID

Agricultural and Rural Market DevelopmentAfghanistan

The evaluation of the Comprehensive Agriculture and Rural Development Facility (CARD-F) seeks to strengthen agricultural value chains, aiming to increase rural incomes, create jobs and improve the trade balance mainly through import substitution in Afghanistan. The purpose of the evaluation is to examine whether CARD-F Phase II achieved its goal to provide jobs and increase the incomes of beneficiaries, and have a wider positive effect on the rural economy. The independent evaluation is focusing on two value chains: Poultry and Protected Horticulture (‘greenhouses’), which together constitute 74% of planned expenditure under CARD-F Phase II. The evaluation adopts a mixed methods theory-based approach employing Contribution Analysis, examining the underlying theories of change for both value chains with both quantitative and qualitative data.

WYG has overall responsibility for delivery of this evaluation, working with our on the ground partner, Altai, and collecting data in four stages: a 2016 baseline, two midlines (2017 and 2018), and an endline in 2019. Data is being collected through enumerated surveys with beneficiaries and key informant interviews with a wide range of stakeholders in the two value chains. The evaluation is also draws on secondary national-level data to understand agriculture sector and economy-wide changes (e.g. on market price monitoring, trade figures). WYG designed the evaluation methodology and field instruments, and have provided the Project Director, Quantitative Lead, and Conflict and Gender Specialists, Contribution Analysis Expert, and Data Analysts to the CARD-F evaluation team. Altai, manages the field data collection while WYG leads on data analysis and production of key reports.

Afghanistan

Evaluation of the Comprehensive Agriculture and Rural Development Facility (CARD-F)

2015 – 2019, DFID

Agriculture and Rural Market Development

Ghana

Market Development in the North of Ghana Independent Evaluation

2014 – 2017, DFID

Agricultural and Rural Market Development / Private Sector Development

Nigeria

Independent Evaluation for Promoting Pro-poor Opportunities in Commodities and Services Markets (PrOpCom) Mai-karfi Programme in Nigeria

2013 – 2018, DFID

Agricultural and Market Rural Development, Private Sector Development

Nigeria

Independent Evaluation of the Stability and Reconciliation Programme in Nigeria (NSRP)

2013 – 2018, DFID

Conflict and Security


Market Development in the North of Ghana Independent Evaluation; 2014 -2017, DFID

Agricultural and Rural Market Development / Private Sector DevelopmentGhana

Ghana MADE is a Making Markets Work for the Poor (M4P) programme, intended to stimulate economic growth and reduce poverty in the Northern Savannah, through improved incomes and resilience of poor farmers and small-scale rural entrepreneurs in northern Ghana. WYG used qualitative and quantitative data collection and analysis to conduct a theory-based impact and process evaluation. The objective of the MADE evaluation were to: i) assess whether M4P delivered within the context of “thin markets”, such as the northern savannah can deliver benefits for poor producers and entrepreneurs; ii) contribute to the evidence base on what works in Northern Ghana; iii) inform the international debate on the value of the M4P approach, and thereby feed into future policy and funding decisions relating to tackling poverty reduction and achieving the MDGs; iv) inform DFID and Government of Ghana of sustainable approaches to developing market systems and which interventions have the greatest impact.

The WYG evaluation of Ghana MADE drew on data collected by the MADE programme, as well as conducting fieldwork to fill gaps or supplement existing data. WYG gathered qualitative data via key informant interviews and focus group discussions to build up a picture of the programme in action, and to set out, revise, and substantiate or invalidate the programme ToC. Contribution Analysis was used as an analytical framework to come to robust conclusions about the programme’s contribution to outcomes. The evaluation framework focused on a series of ‘keystone nodes’ within market specific theories of change identified as central to the success of the M4P designed interventions. This innovative keystone node approach provided both opportunities to learn about the MADE programme as well as about the evaluation process itself.

Independent Evaluation of the Stability and Reconciliation Programme in Nigeria (NSRP); 2013-2018, DFID

Conflict and SecurityNigeria

WYG undertook the evaluation of the Stability and Reconciliation Programme in Nigeria (NSRP), a five year, £39m programme (2012-2017) aiming to help reduce violence and improve stability in selected regions of Nigeria (the northern states of Kano, Borno, and Yobo; the middle belt states of Kaduna and Jos; and the southern states of Delta, Bayelsa, and Rivers). This would be done by 1) improving conflict management mechanisms, 2) mitigating against drivers of conflict, 3) increasing participation of, and reducing violence against, women and girls and 4) improving conflict prevention policy and practice. NSRP supported existing or created new opportunities for state and non-state actors to engage through multi-stakeholder’s platforms (MSPs). WYG used a hybrid theory-based/case-based design, triangulating both primary data collected by WYG (often in conflict affected areas) and internal NSRP monitoring data.

WYG developed a series of nine case studies looking at specific NSRP interventions using RAPID Outcome Assessment (ROA) methods, with key informant interviews and a ROA workshop used to collect data for the case studies. WYG also gathered a mixture of qualitative and quantitative data to assess participation and voice of multi-stakeholder platform members, as well as the outputs and outcomes of platform activity. This data was collected via a survey of 271 NSRP multi-stakeholder platform participants. Social Network Analysis was subsequently carried out on the survey data using the web-based application ‘Kumu’ to assess the network properties of the platforms and their member organisations. Finally, WYG employed Qualitative Comparative Analysis to explore the conditions associated with NSRP platform success and to provide a programme level overview which reinforced the insights from the case studies and platform survey.

Independent Evaluation for Promoting Pro-poor Opportunities in Commodities and Services Markets (PrOpCom) Mai-karfi Programme in Nigeria; 2013 -2018, DFID

Agricultural and Market Rural Development, Private Sector Development Nigeria

Over the five year period (2013-18) of the PrOpCom (Promoting Pro-poor Opportunities in Commodities and Services Markets) Mai-karfi programme, we are delivering an independent process and impact evaluation of its achievements in relation to key OECD Development Assistance Committee evaluation criteria.

PrOpCom Mai-karfi operates under the M4P (Making Markets Work for the Poor) approach of changing market systems in order to address inequitable growth. It builds on the positive experience of its predecessor programme in Nigeria and aims to increase the incomes of 650,000 poor men and women in northern Nigeria by stimulating sustainable growth in selected rural markets to make them more inclusive for poor people.

WYG’s evaluation combines quantitative and qualitative survey methods to assess the impact of the PropCom Mai-karfi programme. In relation to M4P, our work is helping build an evidence base on the relevance, efficiency, impact and sustainability of M4P programmes, providing guidance on what measures are effective and thereby informing future developments.

WYG drew on quantitative beneficiary level data, collected from 3,800 farmers and small rural enterprises, as well as qualitative key informant interviews (KIIs), focus group discussions (FGDs) and case studies. This evidence is being used to build up a picture of the programme in action, and to set out, revise, and substantiate or invalidate the programme ToC. The evaluation used contribution analysis to construct a ‘contribution story’ and to come to robust conclusions about how and to what extent PropCom did (or did not) contribute to observed outcomes. The evaluation has been designed with flexibility built in since PropCom itself is a flexible programme by design.

The evaluation provides in-depth analysis of the programme’s contribution towards raising household incomes, and seeks to understand the relationship between the M4P approach, poverty reduction and systemic change within markets. Related knowledge and research has been published, including a paper on Warehouse Receipt Systems for food grains.


Generating knowledge and researchHMG invests more than £1B in research every year, but the outcome and uptake of research can remain elusive. In response to client needs, WYG is expanding its offering to support the uptake of research and the influence of policy, helping to position the UK as an exporter of expertise. In addition, we help our clients to consider if and how research related work have led to change, and how and why this has happened. We are also increasingly supporting programme to reflect on the knowledge contained in interventions, and how this can be shared more widely.

Sub-Sahara Africa (Ethiopia, Tanzania, Ghana, Malawi, Mali and Zambia)

Sustainable Agricultural Intensification Research and Learning in Africa (SAIRLA)

2015 – 2020, DFID

Agriculture and Rural Development

Rwanda

Evaluation of Results Based Aid (RBA) in Rwandan Education

2013– 2015, DFID

Education

Western Balkans: Albania, Bosnia and Herzegovina, Kosovo, Macedonia, Montenegro, Serbia

CSSF Target audience Analysis in Western Balkans (TAA WB)

2018 – 2019, FCO

Media audience Analysis


P-06

83/0

3/19

Sustainable Agricultural Intensification Research and Learning in Africa (SAIRLA); 2015 – 2020, DFID

Agriculture and Rural DevelopmentSub-Sahara Africa (Ethiopia, Tanzania, Ghana, Malawi, Mali and Zambia)

WYG is leading the implementation of the Sustainable Agricultural Intensification Research and Learning in Africa (SAIRLA) programme, which seeks to generate new evidence and design tools to enable governments, investors and other key actors to deliver more effective policies and investments in sustainable agricultural intensification (SAI) that strengthen the capacity of poorer farmers’, especially women and youth, to access and benefit from SAI. As grant manager, SAIRLA oversees the use of £5M invested by DFID in eight research programmes operating across six countries on SAI research, while our partner National Research Institute (NRI) supports technical quality assurance of research.

WYG is also responsible for delivering technical support to five in-country learning alliances, which aims to bring together local stakeholders to support the uptake of research and the influencing of policy based on robust evidence. The cross-cutting nature of the research and social learning allows SAIRLA to compare and contrast evidence, highlight potential synergies and identify SAI design tools that can be shared across and adapted from country to country.

WYG leads on the effective communication, dissemination and sharing of lessons. In addition, WYG is also responsible for the design of and implementation of the programme monitoring and evaluation.

Evaluation of Results Based Aid (RBA) in Rwandan Education; 2013– 2015, DFID

EducationRwanda

The objective of the Results Based Aid (RBA) programme was to pilot the provision of additional results-based aid on improvements in the number of students completing primary (P6), lower secondary (S3) and upper secondary (S6) education; and the competency of teachers in Rwanda to use English as the means of instruction. RBA was composed of three key elements: payments based on results; recipient discretion on how results are achieved; and verification of results acts as the trigger for disbursement. Adopting a realist evaluation approach, WYG undertook process and impact evaluations of the programme between 2013 and 2015.

The purpose of the WYG led RBA evaluation was to determine whether the additional incentive of RBA impacted on completion at key stages of education and additional teachers becoming competent in the use of English as the medium of instruction. The evaluation also considered the response of the Government of Rwanda and other key actors to RBA and sought to establish the various processes that led to observed results. The evaluation focused on impact; via an econometric model developed by WYG, which explored trends in and factors affecting completion. The evaluation also examined process; through annual literature reviews and qualitative primary research conducted at national, district and school-based levels to explore the response to RBA and the drivers of change in relation to completion and English language proficiency. Lastly, the evaluation also included in-depth modelling of the potential value for money (VfM) of RBA.

CSSF Target audience Analysis in Western Balkans (TAA WB); 2018 – 2019, FCO

Media audience Analysis Western Balkans: Albania, Bosnia and Herzegovina, Kosovo, Macedonia, Montenegro, Serbia

WYG, in partnership with Ipsos, was commissioned to provide a robust target audience analysis (TAA) in the Western Balkans that articulates the perceptions and attitudes of individuals towards:

• political, social and economic circumstances• political actors (both domestic and foreign)• the various outside parties actively engaged in the region

(specifically towards the UK, EU, NATO, US, Russia, Turkey, the Gulf countries and China)

• media consumption.

This research explored why and how these perceptions came about and identify key influencers in terms of media sources in each of the six Western Balkans (WB6) countries: Albania, Bosnia and Herzegovina (BiH), Kosovo, Macedonia, Montenegro and Serbia.

The TAA provided an evidence base for future activity by HMG, thereby informing and improving HMG strategic communications and messaging in the region.

The Target Audience Analysis drew on IPSOS’ expertise to collect both qualitative and quantitative data to assess which audience the UK would be better placed to target for future programming, conducting quantitative surveys, focus group discussion (FGD) and key informant interviews (KII). In parallel, WYG conducted media monitoring between September and November 2018 to assess the kinds of content produced by different international actors and media channels. This also identified the main media channels and their respective audience numbers and demographics.

WYG then analysed and triangulated all data and produced a final report, including 6 annexes specific for each country analysed.

Documents

Monitoring, Evaluation & Learning (MEL) · 2019-03-29 · Monitoring, Evaluation & Learning (MEL) Our credentials WYG has an in-house MEL team of 25, a core pool of trusted associates